Every serious conversation about AI governance eventually arrives at the three lines of defense. It is a familiar model, borrowed from decades of banking practice, and it is tempting to assume it transfers to artificial intelligence unchanged. It does not. The model still holds, but each line must be rebuilt for a technology that behaves unlike anything the framework was originally designed to govern.
This series works through all three lines, with the bulk of its attention on the second: the independent oversight function that most organizations are still inventing. But we begin where risk is actually created: the first line.
Where AI Risk Is Actually Created
The first line of defense is not a control function. It is the business: the people who build, buy, deploy, and use AI systems. Product owners, engineers, data scientists, and increasingly ordinary business users who have discovered that a well-crafted prompt can accomplish what once required a development team. The first line owns the risk it creates, and it owns the controls that manage that risk day to day.
What makes AI different is not that the first line owns risk, that has always been true, but how much wider the first line has become. In a conventional technology estate, the population capable of creating material risk is bounded by who can write and deploy code. Generative AI dissolves that boundary. Anyone who can write an instruction in plain language can now build something that makes decisions, drafts client-facing content, or triggers a downstream action. The first line has quietly expanded to include most of the organization.
Anyone who has managed an end-user computing problem will recognize the shape of this immediately. In my own experience leading a finance transformation, we found thousands of undocumented spreadsheet models running critical processes with no inventory, no named owner, and no review. Those models were built by capable, well-intentioned people solving real problems with the tools available to them. The risk was not malice or incompetence; it was invisibility. Generative AI reproduces that pattern at greater speed, with more capable artifacts, and with an output that is far harder to eyeball for correctness than a formula in a cell.
What the First Line Owns
Effective first-line ownership in an AI context comes down to six responsibilities. None of them belong to the oversight function; all of them are performed by the people closest to the work.
1. Registration
Every AI system, agent, or material prompt-driven workflow is declared into a central inventory before it reaches production. Nothing runs unregistered. This single discipline determines whether governance is possible at all.
2. Documentation
Purpose, intended use, data sources, model and version, known limitations, and out-of-scope uses. A model card is not bureaucracy; it is the artifact that makes independent review feasible and that answers the question an auditor or a client will eventually ask.
3. Risk self-assessment
The builder scores the system against the enterprise tiering rubric:autonomy level, data sensitivity, client exposure, reversibility. The second line validates the score, but the first line produces it, because the builder knows what the system actually does.
4. Control design and operation
Guardrails, input validation, output filtering, human checkpoints, logging, access limits. The first line designs these into the system and operates them daily.
5. Testing and evaluation before release
Not a demo. A structured evaluation against a representative test set, with a documented accuracy or groundedness threshold appropriate to the system's risk tier.
6. Monitoring and incident reporting
Watching for drift, degradation, and misuse in production, and escalating promptly when something goes wrong rather than quietly patching it.
The Shift from Deterministic to Probabilistic Ownership
Traditional first-line control ownership rests on an assumption that no longer holds: that a system, given the same input, produces the same output, and that testing therefore establishes correctness.Generative and Agentic systems are probabilistic.The same question can yield different answers. Correctness is not binary but distributional: a system is right most of the time, and the interesting question is where and how it is wrong.
This has a direct consequence for what first-line testing must become. It is not enough to confirm that a system works; the builder must characterize how it fails.Which inputs produce errors? Are failures concentrated in a particular category: the complex, the ambiguous, the rare? Do failures announce themselves, or does the system fail confidently, producing fluent and plausible output that happens to be wrong?
The Fluent and Silent Failure
That last question matters more than any other. A system that fails loudly is manageable. A system that fails fluently is dangerous, because the human reviewer downstream has no signal that anything is amiss. In one retrieval-augmented generation implementation I worked on, the system began producing well-written answers that were not reliably grounded in the source corpus. Nothing about the output looked wrong. The fix was not a better model but better retrieval, a hard constraint that the system answers only from retrieved context and cite its source, an explicit fallback when evidence was insufficient, and an evaluation harness that scored groundedness and citation accuracy so that quality was measured rather than assumed.
The Citizen Developer Question
Organizations broadly split into two postures on business users building with AI. One forbids it, which produces shadow usage that is entirely invisible to governance. The other permits it without structure, which produces a large population of ungoverned systems in production. Both fail, for opposite reasons.
The workable answer is a governed citizen-developer program, and the design principle behind it is simple: make the governed path the path of least resistance. When I faced the end-user computing problem, mandating migration would have failed: the owners of those spreadsheets had no reporting line to me and every incentive to resist. Instead, we taught them to build properly and made registration in the inventory the gateway to support, better tooling, and a supported platform. Compliance became self-interested rather than enforced. Thousands of models were registered and tiered, and more than four hundred were remediated into governed enterprise platforms.
The same logic applies directly to AI.If registering an AI use case is how a team gets access to the approved models, the enterprise API, the evaluation tooling, and engineering support, registration stops being a tax and starts being a benefit.If registration is purely a compliance burden with nothing on the other side, teams will build outside the perimeter, and the inventory will describe a fiction.
Where the First Line Predictably Fails
Four failure modes recur often enough to be treated as design assumptions rather than surprises.
1. Optimism in self-assessment
Builders systematically under-tier their own systems. This is not dishonesty; it is proximity. The person who built the system knows how it is meant to be used and unconsciously scores it against intended use rather than possible use. This is precisely why the second line validates tier assignments rather than accepting them.
2. Demo-driven confidence
A system that performs impressively in a demonstration has been tested against the cases the builder thought of. Production supplies the cases nobody thought of.
3. Scope creep after approval
A system approved as an internal drafting aid gradually becomes a client-facing capability without ever returning through the gate. The approved risk profile and the operating risk profile silently diverge.
4. Marking their own homework
The most consequential failure. When the team that builds a system is also the team that certifies it as safe, the certification carries no independent weight. This is not a criticism of the first line: it is a structural limitation, and it is the entire reason the second line exists.
What Good First-Line Practice Looks Like
In a mature organization, the first line does not experience governance as an external imposition.
Intake is a normal step in the delivery lifecycle rather than a gate encountered late.
Risk tiering happens at design time, when it can still influence architecture, rather than at release, when changing anything is expensive.
Evaluation harnesses are built alongside the system, not retrofitted under pressure.
Human checkpoints are placed deliberately at the points of irreversibility rather than sprinkled uniformly.
Escalation is understood as a normal professional act rather than an admission of failure.
The tell for maturity is where the conversation with the oversight function happens. If the first line is talking to the second line during design, the program is working. If those conversations happen at release, the program is a bottleneck. If they happen after an incident, there is no program.
The Handoff
A well-run first line is necessary and insufficient. It can build carefully, document honestly, test rigorously, and still fall short:not from weakness but from structure. Builders cannot provide independent assurance about their own work, cannot see across the portfolio to spot concentration and correlation, and cannot easily hold a line against their own delivery pressure.
That is the gap the second line fills: independent challenge, portfolio-level visibility, and the authority to say not yet. The next five articles examine what that function must actually do,beginning with the harder question of what its mandate should be when the existing regulatory rulebook was not written for the technology in front of us.
Explore more: Start Here · Topics · Archive · About