A production-ready AI product has a validated user outcome, explicit behavioral and data boundaries, tested failure handling, observable operations and named owners who can release, support, change and stop it.
A prototype demonstrates possibility under selected conditions. Production exposes ambiguous requests, missing context, permission edges, hostile content, provider failures, model changes, long inputs, concurrency, cost spikes and users who develop unexpected habits. A team can have an impressive model score while the complete product remains unsafe, uneconomic or impossible to operate.
Production readiness is not a label awarded to a model or a final security review. It is an evidence-backed release decision about the complete socio-technical system. The team should be able to explain who benefits, which behavior is acceptable, what happens when components fail, how impact is observed and who has authority to intervene.
Product contract
Start with a user outcome and a bounded behavioral promise
“Answers questions with AI” is not a production requirement. Name the user, the task they are trying to finish, the inputs available, the decision they retain and the system effect. A contract-review assistant might identify clauses and propose issues, while a qualified person decides the legal position. This boundary defines what to test and what the interface must communicate.
Write unacceptable outcomes explicitly. These may include exposing another tenant’s data, fabricating a source, taking an irreversible action without approval or silently returning an incomplete analysis as complete. Add supported languages, document types, task volume and accessibility needs. Marketing, help content and in-product cues should match the tested scope so users are not invited into behavior the operating team cannot defend.
| Area | Evidence before release | Named owner |
|---|---|---|
| User value | Observed completion of a valuable job by target users | Product |
| Behavior | Versioned evaluations and critical-failure results | Product and engineering |
| Data and security | Reviewed data flows, permissions and threat controls | Security and privacy |
| Reliability | Load, dependency failure, fallback and recovery tests | Engineering and operations |
| Release | Rollout limits, support model, rollback and incident plan | Service owner |
Evaluation
Evaluate the complete workflow and every material change
Model benchmarks answer useful but narrow questions. Product evaluation must include retrieval, prompt construction, structured output, tools, permissions, user review and downstream state. Use deterministic checks where rules are exact, expert rubrics where judgment is real and outcome measures where user behavior matters. Record case provenance, allowed data use, expected behavior and severity.
The evaluation set is part of the product, not a one-time gate. Version it and keep protected cases away from routine tuning. Create a change matrix that says when to re-run which suites. A new source parser may need extraction and permission tests. A model update needs behavior, cost and latency regression. A tool schema change needs side-effect and idempotency tests. Inspect failures individually before accepting an improved aggregate.
- Include refusal and escalation as expected successful outcomes.
- Test missing, contradictory, stale and malicious context.
- Separate critical failure rates from average quality scores.
- Store enough configuration to reproduce a tested release.
- Require acceptance criteria before examining the new results.
Engineering
Treat model output as untrusted input inside a reliable application
A model should not decide its own effective permissions. Resolve user identity and resource scope in application code, expose only the necessary tool operation and validate every argument. Keep retrieved content in an untrusted data boundary so instructions inside a document cannot redefine system authority. Encode output for its destination and never pass generated code, queries or markup directly into privileged execution.
Reliability engineering must surround the probabilistic component. Set deadlines and cancellation, bound retries and make side effects idempotent. Use circuit breakers or reduced-function modes when providers fail. Distinguish a confident answer from a confirmed system action. The user should see whether an action is proposed, awaiting approval, accepted, failed or recovered. OWASP guidance on improper output handling and excessive agency is directly relevant when generated content reaches tools.
- Minimize tool capability and data returned to the model.
- Confirm high-impact actions with the person who bears the consequence.
- Validate types, ranges, resource ownership and policy before execution.
- Make repeated requests safe or detect duplicate operation keys.
- Provide a non-AI recovery path for time-sensitive critical work.
Operations
A production team needs visibility, authority and an exit path
Monitor service health and product behavior separately. Availability and latency can be healthy while answer usefulness declines. Combine infrastructure events with sampled quality review, user correction, escalation, refusal, cost and completed-job measures. Segment by model, prompt, language, task and customer context without turning observability into unrestricted data collection.
Define support and incident playbooks before broad access. Triage should distinguish model behavior, bad or missing data, permissions, integration, interface and user misunderstanding because each needs a different owner. Release progressively and document the maximum authority at every stage. The team should be able to hold a model version, disable a tool, reduce autonomy, route all cases to review or remove the feature without trapping the underlying business process.
- Alert on critical behavior and sudden distribution shifts, not every imperfect sentence.
- Attribute inference, retrieval, tool and review costs to completed jobs.
- Give support staff reproducible context with sensitive values minimized.
- Exercise rollback and reduced-authority modes before an incident.
- Define retirement, data deletion and replacement responsibilities.
What good looks like
Useful outcomes from production AI product checklist
- The release decision is tied to a measurable user job and explicit unacceptable outcomes.
- Representative evaluations cover normal, difficult, absent, adversarial and changing conditions.
- Identity, data, tool and approval boundaries are enforced outside model instructions.
- Failures produce understandable refusal, fallback, recovery or escalation rather than silent corruption.
- Quality, latency, cost, safety and user impact are observable under a controlled rollout.
- A named operating team owns incidents, model or data changes, support and retirement.
Operating model
How to run the work
- 01
Define the production contract
Write the target user, supported job, intended decision or action, operating environment and excluded uses. Define successful completion, acceptable variation, critical failure and required human authority. Treat the document as a product contract that connects business value, evaluation, interface behavior and release scope.
- 02
Prove behavior on representative cases
Create versioned cases from real task variation, including incomplete context, ambiguous instructions, conflicting sources, hostile content and downstream failures. Define expected outcome or rubric before tuning. Measure the end-to-end system with retrieval, tools, permissions and interface, then protect a held-out regression set.
- 03
Engineer boundaries and recovery
Enforce identity, authorization, tenant scope, data classification, retention and tool permissions in deterministic application controls. Validate structured output before use. Add timeouts, rate limits, idempotency, bounded retries, confirmation for material actions and a safe state when a dependency is unavailable.
- 04
Prepare production operations
Version models, prompts, retrieval configuration, tools, policies and evaluations. Define observable events, quality samples, cost attribution, support triage, incident severity, rollback and provider-change procedures. Give operators enough trace context to diagnose a failure without unnecessarily exposing sensitive payloads.
- 05
Release authority gradually
Begin with bounded users, data and actions. Compare observed outcomes with the production contract and examine failures by severity, not only averages. Expand scope only after evidence supports it. Preserve a manual or deterministic path for critical work and maintain an emergency means to reduce or remove AI authority.
Evaluation
Questions that change the decision
- What user outcome proves the feature is valuable beyond novelty or output preference?
- Which error classes are tolerable, recoverable, reviewable or release-blocking?
- What data can each user and model path access, store, log and transmit?
- Which external actions require confirmation and how are duplicate effects prevented?
- What change in model, prompt, source, tool or policy requires regression testing?
- Who can pause the feature, roll back a release and communicate an incident?
Failure modes
Where teams lose control
Optimizing a curated demo set can hide failure on ordinary messy inputs.
A permission described in the system prompt may be ignored or manipulated if code does not enforce it.
Automatic retries around a non-idempotent tool can duplicate payments, messages or records.
Logging full prompts for debugging can create a second uncontrolled store of sensitive data.
Provider or model updates can change behavior without an application code release.
Users may extend a useful assistant into unsupported consequential decisions when boundaries are vague.
Average quality can improve while a rare critical failure class gets worse.
Measurement
Measure the finished job
Measure the completed workflow, including review effort and exceptions. Output volume on its own is not evidence of a better process.
- end-to-end task completion and accepted outcome by representative case class
- critical failure, refusal, escalation, human correction and recovery rates
- p50 and p95 latency for complete user jobs under expected concurrency
- total variable cost per accepted outcome, including retries and human review
- permission denials, adversarial-input catches and blocked unsafe actions
- production drift between released and current evaluation results
- support cases and incidents by model, data, integration and product cause
Questions
Common questions
When is an AI prototype ready for production?
It is ready for a bounded release when target users achieve the intended outcome, critical behavior and data boundaries are tested, failures recover safely, operations are observable and named owners can support, change and stop the system. A good demo alone is insufficient.
What should a production AI evaluation include?
It should test the complete workflow on representative normal, difficult, missing and adversarial cases. Measure task outcomes, evidence or tool correctness, critical failures, human corrections, latency and cost. Include expected refusals and escalations rather than scoring every non-answer as failure.
Should AI product permissions be controlled by prompts?
No. Prompts may describe intended behavior, but identity, authorization, resource scope, tool capability and approval must be enforced by deterministic application controls. Treat model output and retrieved content as untrusted at privileged boundaries.
Does production readiness require a specific AI model?
No. It requires evidence that the chosen complete system meets the product contract. The architecture should version and, where practical, isolate model dependencies so the team can compare, hold, replace or roll back a model without rebuilding the business workflow.
Sources
Primary references
- Artificial Intelligence Risk Management Framework National Institute of Standards and Technology
- LLM05:2025 Improper Output Handling OWASP Gen AI Security Project
- LLM06:2025 Excessive Agency OWASP Gen AI Security Project
About the author
Tony Kim
Tony writes about applied AI, dependable product engineering and the systems that turn complex response work into controlled delivery.
Zeke
AI product engineering for moving a software brief into a reliable production product.
Product leaders, founders and engineering teams. Start with the workflow, constraints and evidence you already have.
See Zeke→