The most important question about an AI agent is not what it can do. It is what it is authorised to cause. A system that summarises a negotiation may be useful. A system that sends the summary may create a confidentiality event. A system that drafts a payment schedule may save time. A system that changes a beneficiary can move an asset. Capability crosses into authority when an output changes the world. That boundary should be designed before the agent receives tools.
Separate four modes of work
- Prepare: gather records, normalise data, identify deadlines and draft material.
- Recommend: compare options, state assumptions and propose an action.
- Approve: make the binding decision on behalf of an authorised person or body.
- Execute: transmit, publish, transfer, sign, delete, purchase, grant or revoke.
Many deployments blur these stages because the same interface performs all of them. Keep the distinction explicit even when one workflow contains several stages. The system may prepare and recommend broadly while approval and execution remain constrained.
Classify actions by consequence
Use more than financial value. An action can be low-cost and irreversible: publishing a name, disclosing a diagnosis, deleting evidence, revoking a credential during an incident or sending a message that changes a negotiation. Assess at least five dimensions:
- Reversibility: can the action be fully undone?
- Externality: does it affect a third party or leave the private boundary?
- Authority: does it bind a person, company, trust, account or public position?
- Sensitivity: does it reveal or transform protected information?
- Uncertainty: can the system reliably understand the instruction, context and target?
The higher any dimension, the stronger the human and technical gate.
A practical authority matrix
- Auto-execute: narrow, reversible internal housekeeping with clear limits and reliable validation.
- Execute with notice: low-consequence actions where a human can promptly inspect and reverse the result.
- Draft for approval: external communications, configuration changes, new recipients and actions touching sensitive information.
- Dual approval: payments, asset movement, access changes, legal commitments and material public statements.
- Prohibited: actions the system cannot perform under any prompt, including disabling its own oversight, expanding its own authority or bypassing an active hold.
Do not make the model responsible for obeying this matrix through instruction alone. Enforce it in the tool layer. If an agent may not send, the send capability should not be present. If it may spend only within a limit, the transaction service should enforce the limit independently.
Approval must be meaningful
A human click is not a control when the reviewer receives an opaque bundle, has no time and assumes the machine is usually right. For consequential approval, show:
- The precise action that will occur and the target.
- The source instruction and identity that initiated it.
- The records and assumptions used.
- Material uncertainty, conflicts and policy exceptions.
- The change from the current state.
- A clear ability to reject, modify or escalate.
Avoid approval fatigue by keeping routine reversible actions out of the queue. Human attention should be concentrated where judgment and authority actually matter.
Constrain sequences, not only single actions
Individually harmless actions can form a dangerous chain. An agent may create a user, assign a role, export a file and send a link—each apparently permitted—while the sequence exfiltrates private data. Define sequence controls and transaction envelopes. Limit the number, rate and combination of actions. Require a new approval when the goal changes, the recipient changes or the agent encounters an exception. Short-lived credentials and narrowly scoped tools reduce what a compromised or confused agent can assemble.
Build a hard stop
Every autonomous workflow needs a reliable stop controlled outside the agent. It should disable execution, preserve relevant state and alert a named human. The agent must not be able to remove the stop, alter the logs that justify it or grant itself a new route around it. Test the stop during ordinary operations. A kill switch nobody has exercised is a diagram, not a control.
Measure avoided consequence
Do not judge the deployment only by tasks completed. Track attempted policy violations, rejected actions, escalations, reversals, unexplained tool calls and time to human intervention. The goal is not maximal autonomy. It is the largest safe envelope in which the system can produce reliable value. Outside that envelope, it must stop cleanly and hand authority back to a person who can be held accountable.