A large language model, or LLM, is a machine-learning model trained on extensive language data to predict tokens and produce or transform text. Modern LLMs can also process code, structured data and, in multimodal variants, images or audio.
Fluent output is easy to mistake for verified knowledge or deterministic software. The model does not automatically know current private facts, preserve business rules, enforce permissions or produce the same result every time. A demo can therefore look complete while the production workflow remains unsafe or economically unstable.
An LLM should be treated as a probabilistic component inside a controlled product. Start with the user decision and acceptable error, then design context, tools, validation, human review and fallbacks around the model.
System design
The model is not the product
A production application adds the user experience, data access, retrieval, workflow state, authorization, evaluation, monitoring and recovery that a model endpoint does not provide. For a proposal assistant, the model may draft language, but the product must still select approved sources, preserve citations, route review and export the accepted answer.
This distinction prevents architecture by benchmark. A more capable model may not improve the workflow if retrieval is weak, the prompt lacks the right facts or the review screen hides uncertainty. Evaluate the complete path from user input to accepted outcome.
| Need | Possible model role | Application control |
|---|---|---|
| Current facts | Synthesize provided context | Retrieve and authorize sources |
| Structured answer | Generate candidate fields | Schema validation and retry |
| Business action | Recommend or select a tool | Policy, confirmation and execution |
| Quality | Produce a candidate result | Evaluation, review and monitoring |
Selection
Model choice is a portfolio decision
Different tasks reward different models. A small model may classify or extract quickly and cheaply; a larger model may handle ambiguous reasoning better. Open-weight deployment can offer control but adds hosting and operations. A managed model can accelerate delivery but introduces provider, location and lifecycle considerations.
Keep the application boundary replaceable. Use task-specific evaluation rather than brand preference, and route only where measured value justifies complexity. The best model is the one that meets the acceptance threshold within the full production constraint, not the one with the largest general benchmark.
- Benchmark with representative inputs and languages.
- Measure complete workflow quality rather than eloquence.
- Include context, output and tool cost.
- Test failure and unavailability behavior.
- Preserve versioning and rollback.
What good looks like
Useful outcomes from large language model
- The team chooses tasks based on error tolerance and value rather than novelty.
- Model output is grounded in permitted context where current facts matter.
- Deterministic rules and permissions remain outside the model.
- Quality, latency, cost and safety are measured on representative work.
- Model or provider changes can be evaluated and rolled back.
Operating model
How to run the work
- 01
Define the task and failure budget
Specify the input, desired output, user decision and consequence of error. Separate drafting, extraction, classification, search and action-taking because they need different controls. Define when the system should abstain, ask for more information or require a human.
- 02
Design the application boundary
Decide what context the model may receive, which tools it may call and which rules remain deterministic. Apply identity and authorization before retrieval or action. Validate structured outputs and keep transaction, policy and calculation logic in ordinary code where possible.
- 03
Evaluate on real cases
Create a versioned test set representing common, difficult and adversarial inputs. Measure task correctness, unsupported claims, refusal behavior, latency and cost. Review slices by document type, language, customer context and risk rather than trusting one average score.
- 04
Operate model change
Log model, prompt, context and tool versions with appropriate privacy. Monitor quality and spend. Test new models against the same acceptance criteria, use staged rollout and retain a rollback path. Treat provider updates as dependency changes, not invisible improvements.
Evaluation
Questions that change the decision
- Does the task benefit from flexible language understanding or generation?
- What error is acceptable and which failure requires human approval?
- Which facts must come from retrieval or tools rather than model memory?
- Which model meets quality, latency, privacy and cost requirements on real cases?
- How will the system behave when context is missing or the model is unavailable?
Failure modes
Where teams lose control
Confident language can contain unsupported or outdated statements.
Prompt and context changes can alter behavior without a code change.
Private input can cross an unintended processing or retention boundary.
Model output used as executable instruction can create security exposure.
Token cost and latency can become uneconomic at production volume.
Measurement
Measure the finished job
Measure the completed workflow, including review effort and exceptions. Output volume on its own is not evidence of a better process.
- task success against a versioned evaluation set
- unsupported claim and abstention rate
- human correction by failure category
- latency and cost per completed user outcome
- tool-call and structured-output validity
- quality change by model and prompt version
Questions
Common questions
What does LLM stand for?
LLM stands for large language model, a model trained on extensive language data to predict and generate token sequences for tasks such as drafting, extraction, classification and question answering.
Is an LLM the same as generative AI?
An LLM is one type of generative AI focused primarily on language-like data. Generative AI also includes models for images, audio, video and other synthetic content.
Does an LLM know company information?
Not automatically. Private or current company facts must be supplied through permitted context, retrieval or tools. The application should enforce access and show which sources support important statements.
How do you choose an enterprise LLM?
Define the task and acceptance criteria, then test candidate models for quality, safety, language, latency, cost, deployment and data-handling requirements on representative cases.
Sources
Primary references
- LLM glossary entry NIST
About the author
Tony Kim
Tony writes about applied AI, dependable product engineering and the systems that turn complex response work into controlled delivery.
Zeke
AI product engineering for moving a software brief into a reliable production product.
Product leaders, founders and engineering teams. Start with the workflow, constraints and evidence you already have.
See Zeke→