The preceding articles describe governance as something performed around AI systems: registering them, tiering them, reviewing them, monitoring them. That model works when systems produce outputs that humans then use. It strains badly when systems take actions on their own initiative, at machine speed, across many transactions.
An agent that executes hundreds of actions an hour cannot be governed by a committee that meets monthly.The oversight must move inside the system. This article describes a design pattern I call Governed Autonomy:embedding the three lines of defense into the architecture of the agent itself.
Why Perimeter Governance Fails for Agents
Governance layered around a system rests on three assumptions that agentic AI invalidates.
That review can precede consequence. With a recommendation engine, a human reads the output before anything happens. With an agent, the action may complete before any human is aware of it.
That the reviewed artifact is the operating artifact. An agent that adapts its approach, selecting different tools, decomposing tasks differently, or adjusting its own instructions, is not necessarily the system that was validated.
That volume is human-scale.A control requiring human attention per transaction fails at the volumes agents are deployed to handle. The organization then faces a choice between abandoning the control and abandoning the automation, and in practice the control is what gets abandoned.
The response is not to prohibit autonomy, which forfeits the value, nor to accept it ungoverned, which forfeits the control.It is to make the controls part of the system's own operation so that they run at the same speed as the thing they govern.
The Governed Autonomy Design Pattern
Governed Autonomy maps the familiar three lines onto four architectural layers. The correspondence is deliberate: the same separation of duties that governs an organization can govern an agent.
Layer one: the bounded agent, first line inside the system
The agent itself is the first line, and its own design is its first control. Four properties matter:
Bounded goals. The objective is specified narrowly enough that the space of acceptable actions is knowable. An open-ended objective pursued by means of the agent's own selection is where goal drift originates, and goal boundedness belongs in the risk tiering rubric for exactly this reason.
Least-privilege tools. The agent holds only the permissions its purpose requires. This is the highest-leverage control available, because an agent's risk is very largely a function of what it can reach. Most agents are granted far broader access than their task requires, for convenience.
Declared intent before action. The agent states what it is about to do and why, in a form the policy layer can evaluate and a human can read. This makes the reasoning inspectable and traceable (explainability) rather than implicit.
Self-reported confidence, so that low-confidence actions can be routed differently from high-confidence ones.
Layer two: the policy and guardrail layer, second line inside the system
Between intent and execution sits a deterministic control layer that evaluates every proposed action against policy before it executes. It is deliberately not itself a language model: it is rule-based, auditable, and predictable, because a probabilistic control governing a probabilistic system compounds uncertainty rather than reducing it.
This layer enforces the action tiering ladder in code. It classifies each proposed action, read-only, retrieve and summarize, recommend, act with approval, fully autonomous, and applies the corresponding requirement. It blocks actions outside the permitted set, enforces limits on value, volume, and rate, applies data handling rules, and routes anything above the approved tier to a human.It is, in the most literal sense, independent challenge implemented as software.
Layer three: the human checkpoint at the point of irreversibility
Human review is a scarce resource and must be spent where it changes outcomes. The design principle is to place checkpoints at points of irreversibility rather than uniformly through the workflow: where an action cannot be undone, where an external party will see or act on it, where a book of record or regulatory reporting system is written to, or where value or a client relationship is committed.
The quality of the checkpoint matters as much as its placement.A reviewer presented with a bare proposal will approve reflexively. A reviewer presented with the proposed action, the agent's stated reasoning, the evidence retrieved, the confidence level, and a clear accept or reject decision is performing a real control. This is why override rate is such a diagnostic indicator: a checkpoint with a near-zero override rate is usually ceremonial.
Layer four: independent monitoring and the audit record, assurance inside the system
Every proposed action, policy decision, human approval or rejection, and executed effect is logged immutably, in sufficient detail to reconstruct any episode afterwards. Monitoring runs independently of the agent's own execution path, watching for drift, anomalous patterns, permission use outside the normal envelope, and degradation against the approved evaluation baseline.
A kill switch belongs here, and it must be tested. An emergency stop that has never been exercised is a design intention rather than a control.
A Worked Example: Treasury Funding
Consider an agent supporting daily liquidity management. It monitors cash positions across entities and currencies, forecasts intraday requirements, and proposes funding actions.
Under Governed Autonomy, the agent operates with a bounded goal, maintain positions within defined buffers using an approved instrument set, rather than an open instruction to optimize funding. Its tools are scoped to the specific accounts and instruments its mandate requires. It declares each proposed action with its reasoning and its confidence.
The policy layer classifies each proposal. Retrieving positions and generating forecasts is read-only and proceeds automatically. Internal transfers below a defined threshold between pre-approved accounts execute autonomously within rate and value limits. Anything above the threshold, anything involving an external counterparty, and anything outside the approved instrument set is staged for human approval. Actions outside the permitted set are blocked outright rather than escalated.
The treasury professional reviewing a staged action sees the proposal, the positions and forecast that drove it, the agent's reasoning, its confidence, and the limit it exceeded. They approve or reject. Monitoring watches for forecast accuracy degradation, unusual clustering of proposals, and any drift from the approved baseline.
The result is an agent that handles routine liquidity management at machine speed while every consequential and irreversible action passes a human. The autonomy is real, and it is bounded by architecture rather than by policy documents.
A Second Example: The Line at the Book of Record
The same pattern resolves a case I encountered directly. An automated reconciliation capability compared accounting postings between ledger systems and systems of record, then explained each break in plain language and drafted the adjustment to close it. The operations team proposed allowing it to post those adjustments to the general ledger itself.
Two objections applied. On principle, it crossed a line written into the operating model: no AI output posts to the general ledger, or to any system supporting regulatory reporting, without a human checkpoint. On evidence, the model's error rate was concentrated in the most complex and ambiguous breaks, exactly the entries that would have posted unreviewed.
The redesign is Governed Autonomy applied. The agent retained full authority up to the point of irreversibility: detection, explanation, and drafting all remained automated. Its output was routed not to the ledger but to the application already used for manual adjustments, where an operations reviewer validated the entry, read the generated commentary, and approved or rejected it, and only on approval did the adjustment post. All of the automation's value was preserved; the single irreversible act stayed under human control; and because errors clustered in the hardest breaks, human attention landed precisely where the risk was.
Why Architecture Outperforms Policy
A policy stating that agents must not take irreversible actions without approval depends on every builder reading it, understanding it, agreeing with it, and implementing it correctly, and on nobody quietly making an exception under deadline pressure.
A policy layer that will not execute such an action does not depend on any of that.
It scales. Controls execute per transaction at machine speed rather than per review cycle.
It is consistent. The same rule applies identically every time, without fatigue or exception.
It is self-evidencing. The control produces its own audit record as a byproduct of operating, rather than requiring a separate effort to demonstrate compliance.
It fails closed.A well-designed policy layer blocks what it cannot evaluate, rather than permitting it.
None of this removes the need for the second line as an organizational function. Someone independent must define the tiering, approve the permission sets, set the thresholds, review the monitoring, and challenge the design. Governed Autonomy does not replace the second line; it gives the second line an instrument that operates at the speed of the systems it governs.
Design Principles
Bound the goal before bounding the tools. An unbounded objective will find a way to use whatever permissions exist.
Grant least privilege, and review permissions as a first-class governance artifact. Record what the agent can do, not only what it is for.
Make the control layer deterministic.Do not govern a probabilistic system with another probabilistic system.
Place human checkpoints at irreversibility, not uniformly. Scarce attention spent everywhere is attention spent nowhere.
Give the reviewer enough context to disagree.A checkpoint without evidence is a rubber stamp with extra steps.
Log everything, immutably. Any action must be reconstructable after the fact.
Prohibit runtime self-modification. Changes to rules, prompts, or objectives (emergent behavior) pass through a governed release, so that the system running is the system validated.
Test the kill switch. An untested emergency stop is not a control.
With the first line building and owning, and the second line setting standards, challenging, measuring, and now designing controls into the systems themselves, one question remains: who assures that any of this is real? That is the third line, and it is the subject of the next article.
Explore more: Start Here · Topics · Archive · About