Agentic software development with bounded authority

Scope

Add agency only where variation creates real value

Agency is not a product feature by itself. It is a choice to delegate parts of control flow to a probabilistic component. Start by drawing the task as if no agent existed: trigger, inputs, decisions, systems, outputs and exceptions. Stable routing, authorization, calculations and state transitions belong in ordinary code. Model-directed choice becomes valuable where the next useful step depends on variable evidence that cannot be enumerated economically.

Define the goal so the application can verify it. “Resolve the request” invites the model to decide what resolution means. “Prepare a proposed classification with cited evidence and route unresolved conflicts to the named owner” has observable completion. List non-goals and prohibited actions. Set the population and time boundary. An agent designed for internal research should not silently become a customer-communication agent because it discovers a send tool.

• Model the deterministic workflow before adding an agent.
• Delegate the smallest variable decision.
• Define completion in externally observable terms.
• Name non-goals and prohibited outcomes.
• Constrain eligible cases and operating duration.

Tools

Treat every tool as a security and transaction boundary

OWASP describes excessive agency as harmful action enabled by excessive functionality, permissions or autonomy. Avoid general tools when the job needs a narrow one. A mailbox summary agent needs a read operation over an authorized scope, not an email client with send and delete functions. Validate every argument and output. The tool implementation rechecks actor, resource and operation under server-side policy even if the model has already reasoned about permission.

Classify tools by effect. Read operations still need data-scope control and can expose untrusted instructions. Proposal operations create a draft without external consequence. Execution changes a system outside the agent and needs stronger policy. Make side effects idempotent, attach a unique operation key and record the target system’s durable result. For material actions, present the accountable person with the intended effect, parameters, evidence and reversibility, not a vague “approve agent plan” button.

Runtime

Own state, stopping and recovery outside the model conversation

The message history is context, not the canonical task record. Store goal, actor, current phase, gathered evidence, pending approvals, tool outcomes and completion state in typed application storage. Distinguish a proposed fact from a confirmed observation and a model inference from an external result. A worker restart should reconstruct the task without asking the model to remember. Sensitive context receives deliberate access, retention and deletion rather than indefinite accumulation as memory.

Every loop needs limits and progress tests. Cap steps, elapsed time, tokens, money, retries, fan-out and side-effect count. Define what progress means for the task and stop repeated cycles that do not reduce an open condition. Dependencies require timeout and circuit-breaker behavior. When a tool partially succeeds, query its durable status before retrying. Recovery may resume from a checkpoint, compensate a confirmed action, route to a person or close as failed. It must not be narrated into existence.

• Keep typed canonical state outside conversation history.
• Separate observation, inference, proposal and confirmation.
• Budget steps, time, cost, retries and effects.
• Detect no-progress loops and dependency degradation.
• Design resume, compensation and manual takeover.

Evidence

Test the route taken, not only the final answer

An agent can reach the correct answer through an unacceptable trajectory, such as reading unauthorized data and later omitting it. Evaluation therefore records tool selection, arguments, observations, intermediate decisions, policy checks, state transitions and external effects. Test normal cases alongside missing data, ambiguity, conflicting instructions, indirect prompt injection, slow dependencies and partial success. Use deterministic assertions for permissions and state, rubric review for judgment and simulations for tool behavior.

Move through exposure levels. Begin in a simulated environment, then shadow real work without acting, then propose changes for explicit approval. Expand autonomy only for case families and tools whose evidence supports it. NIST’s secure development guidance and AI risk work both reinforce lifecycle controls and documented responsibilities. Keep a versioned release record, observe confirmed outcomes and maintain an independently reachable kill switch. Production incidents become minimized trajectory regressions before autonomy expands again.

• Score outcome, trajectory, policy and efficiency separately.
• Test malicious observations and partial tool success.
• Progress from simulation to bounded real exposure.
• Expand by case family and tool, not by broad label.
• Preserve shutdown, replay and regression evidence.