Open-source AI models for enterprise products

Terminology

Downloadable is not a complete definition of open source

The Open Source Initiative’s Open Source AI Definition describes freedoms to use, study, modify and share and identifies data information, code and parameters as preferred materials for modification. An enterprise does not need to settle every industry debate before acting, but it must avoid collapsing distinct artifacts into one label. “Open weights” may accurately describe a model whose parameters are downloadable while training code or sufficient data information is unavailable.

Build a component and rights table for the exact release. Record the license for weights, code, tokenizer, evaluation assets, serving stack and any adapter. Note acceptable-use terms and separate service terms. A model can be technically runnable and legally unsuitable for the intended product. Another can be permissible to use internally but create obligations when redistributed. Legal counsel determines the conclusion; engineering supplies a complete and versioned inventory.

Model selection

Benchmark the deployed system on the enterprise job

Leaderboards help form a shortlist, not make a production decision. Build cases from the actual workflow, including common inputs, difficult edge cases, multilingual terminology, long context, adversarial content and required abstention. Score the properties that matter: factual support, classification cost, schema validity, tool choice or user task success. A small model can outperform a larger one on a narrow task after good context design, while a strong general score can conceal a fatal domain error.

Benchmark the exact serving configuration. Precision, quantization, context length, decoding, runtime and hardware affect quality and speed. Include retrieval and guardrails when those are part of the application, and compare like with like against hosted models. Measure tail latency and concurrency rather than one local request. Repeat after upgrades, adapters and optimizations. The production artifact is the system that runs, not the unmodified checkpoint described in a model card.

• Use a protected holdout that model selection cannot overfit.
• Report critical errors separately from average quality.
• Test every material operating language.
• Benchmark the intended quantization and hardware.
• Store every model, runtime and evaluation version.

Ownership

Self-hosting exchanges supplier dependence for operating responsibility

Running inference inside chosen infrastructure can improve control over data paths and release timing. It also makes the enterprise responsible for capacity, availability, patching, scaling, observability, abuse controls and incident recovery. Accelerators are not the entire cost. Include idle capacity, redundancy, engineering, storage, networking, energy, orchestration and support. Compare cost per completed business task at realistic utilization rather than price per token or hour.

Design stable interfaces around the model so product state does not become hostage to one serving stack. Keep prompts, schemas, retrieval configuration and evaluation assets versioned outside the weights. Prove that a new model can be canaried and an old one restored. For low or volatile demand, a managed endpoint may remain economically and operationally stronger. A hybrid architecture can route sensitive or predictable work locally and use an approved hosted service for other cases.

• Name an owner for uptime, patching, capacity and quality.
• Model total cost at normal, peak and idle utilization.
• Keep application contracts independent of model APIs.
• Test upgrade, canary, rollback and disaster recovery.
• Compare hybrid options instead of forcing one deployment ideology.

Security

Treat models and adapters as executable supply-chain artifacts

OWASP’s LLM supply-chain guidance highlights risks across models, data, libraries, repositories, adapters and deployment platforms. A popular download or public benchmark is not an integrity guarantee. Use approved repositories, pin immutable identifiers, verify available checksums or signatures and scan accompanying code and containers. Avoid loading arbitrary serialization formats or remote code in production. Promote artifacts through a controlled registry after evaluation.

Track provenance for fine-tunes, quantizations and adapters as carefully as the base model. An adapter changes behavior and can introduce targeted failure even when the base checkpoint is trusted. Maintain a bill of materials covering serving libraries and infrastructure. Monitor security notices and upstream maintenance, but decide patch urgency based on the deployed exposure. Retirement includes deleting obsolete artifacts, credentials and caches and retaining the evidence needed to explain historical outputs.

• Pin exact model, code, container and adapter versions.
• Verify source and integrity before artifact promotion.
• Disallow unreviewed remote code in production loading.
• Evaluate each derived model as a new release.
• Keep a patch and retirement owner for the complete stack.