An AI agent is a software system that interprets a goal, chooses intermediate steps and uses available tools or information to make progress. Many current agents use a language model for planning or tool selection, but the surrounding software controls identity, state, permissions and execution.
Agent is often used for any chatbot or automated flow. Real agency increases the number of paths the system can take and the impact of untrusted input, tool errors and accumulated state. Giving a model broad credentials and a vague goal turns flexibility into an uncontrolled execution surface.
Use the least agency required for the outcome. Bound the goal, expose narrow tools, authorize every action at execution time, checkpoint irreversible steps and measure the full trajectory rather than only the final answer.
Architecture
Agents choose paths; workflows encode paths
A workflow has known transitions: extract a field, validate it, request approval and write the result. A model may assist one transition without controlling the path. An agent receives a broader goal and decides which tool or intermediate step to use based on changing context.
That flexibility is valuable for investigation, open-ended support or multi-source work, but it increases evaluation complexity. If the business process is stable, explicit orchestration is easier to test, audit and recover. Agentic behavior should solve real variability, not decorate a simple automation.
| Pattern | Best fit | Primary control |
|---|---|---|
| Deterministic workflow | Stable rules and sequence | Tests and state machine |
| Model-assisted step | Flexible language inside a fixed path | Input, output and review |
| AI agent | Goal needs adaptive multi-step work | Tools, identity and trajectory |
| Human-led process | High ambiguity or consequence | Expert judgement and evidence |
What good looks like
Useful outcomes from AI agent
- The team distinguishes a fixed workflow from a model-directed agent.
- Every tool has a narrow schema, permission boundary and observable result.
- The agent cannot inherit broader authority than the initiating user or service.
- High-impact actions require deterministic policy and appropriate approval.
- Failed trajectories can be reproduced, interrupted and recovered.
Operating model
How to run the work
- 01
Prove that agency is needed
Describe the goal, environment variability and decisions that cannot be expressed reliably as a fixed workflow. Compare an agent with deterministic orchestration plus one or two model steps. Choose the simpler design when the path is stable.
- 02
Constrain identity and tools
Give the agent a dedicated identity and minimum permissions. Design tools around specific business actions instead of raw database, shell or browser access. Validate inputs and outputs, enforce tenancy and policy in the tool, and set budgets for calls, time and spend.
- 03
Control state and action
Separate trusted instructions, retrieved content, user data, working memory and durable memory. Treat external content as untrusted. Require confirmation or human approval for sending, purchasing, deleting, publishing or changing material records. Use idempotency and compensating actions.
- 04
Evaluate complete trajectories
Test goals with normal, ambiguous, adversarial and failure cases. Inspect tool choices, permission checks, loops, intermediate state, final result and side effects. Monitor live runs, provide a stop mechanism and retain enough trace for incident analysis without over-collecting sensitive data.
Evaluation
Questions that change the decision
- Does the task require dynamic planning or would a fixed workflow be safer?
- Which tools and data are strictly necessary for the goal?
- Which actions are reversible, and which require approval before execution?
- What state may persist between steps, sessions or users?
- When must the agent stop, abstain or escalate?
Failure modes
Where teams lose control
Untrusted content can redirect the agent toward an attacker’s goal.
Overprivileged tools turn a reasoning error into a material system change.
Poisoned or cross-user memory can influence later decisions.
Loops consume money and time while compounding incorrect state.
Several individually reasonable actions can create an unsafe combined outcome.
Measurement
Measure the finished job
Measure the completed workflow, including review effort and exceptions. Output volume on its own is not evidence of a better process.
- goal completion under defined acceptance criteria
- tool selection and argument validity
- policy denials and approval outcomes
- unnecessary steps, loops and cost per trajectory
- side-effect and recovery success rate
- human escalation by reason and risk
Questions
Common questions
What is an AI agent?
An AI agent is software that interprets a goal, chooses intermediate steps and uses tools or information to make progress, usually with a model guiding some decisions.
How is an AI agent different from a chatbot?
A chatbot primarily exchanges messages. An agent may plan across steps, call tools, maintain state and cause actions in other systems. A chatbot can expose an agent, but conversation alone does not create agency.
Should AI agents have production credentials?
Only narrowly scoped, policy-enforced credentials required for the task. The tool layer must authorize each action, and high-impact or irreversible changes should require additional confirmation or human approval.
When should we not use an AI agent?
Do not use one when a deterministic workflow handles the known path more reliably, or when failures cannot be bounded, observed, approved and recovered within the business risk.
Sources
Primary references
- Securing Agentic Applications Guide OWASP Gen AI Security Project
About the author
Tony Kim
Tony writes about applied AI, dependable product engineering and the systems that turn complex response work into controlled delivery.
Zeke
AI product engineering for moving a software brief into a reliable production product.
Product leaders, founders and engineering teams. Start with the workflow, constraints and evidence you already have.
See Zeke→