What happens when a control turns out to be missing, weak, or broken, and how an organization decides in advance how much of that it is willing to live with.
These are old disciplines and they are unglamorous. They are also the mechanics that distinguish a functioning risk practice from a review board with a rubric.
Getting the Vocabulary Right
Risk management has a precise vocabulary that is routinely used loosely, and the looseness matters because these terms carry different consequences.
Inherent risk is the exposure before any controls are considered.
Residual risk is what remains after controls that both exist and operate. Describing residual risk while calling it inherent is the most common confusion in the field.
Risk appetite is the aggregate level of risk an organization will accept in pursuit of its objectives, set at the top.
Risk tolerance is the acceptable variation around appetite for a specific risk or metric: more granular, usually numeric. They are not interchangeable.
Control design effectiveness asks whether the control, if it operated, would mitigate the risk.
Control operating effectiveness asks whether it actually operated, consistently, over a period, with evidence. A control can be beautifully designed and never once run.
Preventive controls stop the event: a pre-deployment evaluation gate, an input filter, a permission boundary.
Detective controls identify that it occurred: groundedness monitoring, output sampling, drift detection.
Corrective controls limit damage afterwards: rollback, kill switch, model reversion, client notification.
An issue is a known control weakness, formally recorded, rated, owned, and tracked.
A finding is an issue raised by an independent party, internal audit or a regulator.
Self-identified issues are viewed far more favorably than audit-identified ones, which is an argument for a culture that rewards raising them.
KRI (Key Risk Indicator): A metric that provides early warning of increasing risk exposure.
KPI (Key Performance Indicator): A metric that measures progress toward a business or operational objective.
KCI (Key Control Indicator): A metric that measures whether a control is operating effectively and consistently.
Attestation is a named individual formally confirming a state of affairs. It is the cheapest control that reliably changes behavior, because it converts a diffuse expectation into personal accountability.
The Risk Assessment Mechanic
Every second-line risk assessment follows the same seven-step shape. The shape is domain-independent; only the taxonomy changes.
Define the risk taxonomy: the named categories in scope. For AI: hallucination and factual error, bias and unfair outcomes, lack of explainability, robustness and adversarial manipulation, model and data drift, data leakage and confidentiality, privacy, intellectual property, security, third-party dependency, misuse, and insufficient human oversight.
Score inherent risk: likelihood against impact, before controls. Keep the scale course; a five-by-five matrix invites false precision that the underlying judgment cannot support.
Map controls to risks: which specific control addresses which specific risk. The gaps in this mapping are the finding. A list of controls that is not mapped to a list of risks tells you nothing about coverage.
Assess control design and control operation effectiveness:two separate judgments, in that order.
Derive residual risk: what remains after controls that both exist and operate.
Compare residual to appetite, and choose a treatment: treat by adding controls, tolerate by formally accepting, transfer by insuring or contracting away, or terminate by not doing the thing.
Report and monitor: the residual position, its movement since last assessment, and what drove the movement.
The discipline that makes this useful rather than ceremonial is step three.Most weak assessments enumerate risks in one column and controls in another without ever asserting which control addresses which risk, which conceals exactly the gaps the exercise exists to find.
Issue Management
An issue is what a failed control becomes once it is formally recognized. The lifecycle has seven stages, and each has a characteristic way of going wrong.
Identification: Issues arise from first-line self-identification, second-line challenge or testing, or third-line audit. The failure mode is cultural: organizations that punish self-identification receive fewer issues and more incidents.
Rating: Severity assigned against defined criteria, typically impact combined with how widespread the weakness is. The failure mode is ratings negotiated downward by the business until the rating carries no information.
Ownership: A single named accountable owner, senior enough to direct the fix. Committees do not own issues; people do.
Root cause analysis: Why the control failed, not what went wrong. Skipping to a remediation plan produces recurrence, because the plan addresses the symptom.
Remediation plan: Dated milestones with interim compensating controls where the exposure persists in the meantime. NOT a plan with a single milestone eighteen months out.
Tracking and reporting: Aging, past-due, and severity reported to governance forums. Reporting counts rather than ageing hides the problem, ageing is the real signal.
Closure validation.The second line validates, with evidence, that the control now operates effectively. Closing an issue when the plan is delivered rather than when the control is proven to work is the single most common weakness in the whole process.
That last point deserves emphasis because it is where good intentions quietly fail. A remediation plan delivered on time tells you that work was done. Only evidence of the control operating over a period tells you the risk was actually reduced.
AI-Specific Issues
Several issue types recur in AI programs and are worth naming in advance so they are recognized rather than rediscovered.
Ceremonial human checkpoints: a required review with a near-zero override rate and a median review time of seconds. The control exists on paper and does nothing.
Evaluation evidenceinvalidated by a model version change that nobody detected.
Scope creep past the approved use: a system approved as an internal aid now reaching clients, without returning through the gate.
Conditions of approval that decayed after go-live, particularly logging and monitoring that was implemented and then disabled.
Inventory drift: production systems whose recorded model, data sources, or tool permissions no longer match reality.
Permission accumulation:an agent granted additional tool access incrementally, each grant reasonable, the aggregate well beyond its purpose.
Risk Acceptance and Its Rules
Sometimes a control gap cannot be closed on the timeline the business needs, and the organization chooses to operate outside its own standard. That is legitimate, provided it is a decision rather than a drift.
Six rules make it a decision.
A named accountable executive.Not a team, not a function. A person senior enough that the acceptance is a real act.
A defined and narrow scope.Which system, which use case, which period. Open-ended acceptances become permanent policy by accident.
A mandatory expiry date, with re-approval requiredrather than automatic renewal. This is the single most important discipline in the mechanism, because it forces the exposure back into daylight.
Compensating controls documented. Accepting a risk is not the same as doing nothing about it.
Regulatory breaches cannot be accepted.You may accept operating outside your own standard. You may not accept operating outside the law. Knowing exactly where that line sits is what separates a practitioner from someone reciting process.
Aggregate reporting. Individually reasonable acceptances can sum to an unreasonable position. Report the total and the expiry pipeline, not only the individual decisions.
Setting Risk Appetite for AI
Appetite statements fail in a predictable way: they are written at a level of generality that makes them unusable. "We have moderate appetite for AI risk" cannot be applied to any actual decision.
Four principles make appetite operational.
Express appetite by tier and by outcome rather than globally."We have no appetite for AI-generated content reaching a client as fact without a traceable source" is a statement someone can act on. It resolves a real decision.
Anchor on a small number of zero-tolerance statements. The mandatory top-tier triggers are the natural candidates: no autonomous write to a book of record or regulatory reporting system, no runtime self-modification in production, no irreversible external action without human approval. Everything else becomes a tolerance range.
Attach a measurable metric to each statement, or it is decoration. An appetite statement that cannot be breached observably cannot be governed.
Have the business own the statement, with the second line drafting it.Appetite imposed by the risk function is appetite ignored by the business.
The test of an appetite framework is whether it has ever caused someone to decide differently. If no proposal has been shaped or stopped by it, the statements are describing what the organization was going to do anyway.
Control Testing and the Audit Relationship
Two forms of testing operate in parallel and neither substitutes for the other. The first line tests its own controls continuously. The second line tests independently on a risk-based cycle, sampling from the population over a defined period and inspecting evidence, with sample size reflecting frequency and criticality. The governing principle is that absence of evidence means the control is ineffective, not that it is probably fine.
When internal audit raises a finding, the productive posture is narrow:do not argue the finding. Argue the rating if, and only if, you hold evidence that supports a different one. Otherwise accept the finding, agree the plan, and deliver it. A written management response should state agreement or disagreement, the plan, the owner, and the date. Disagreement is permitted and occasionally correct, but it must be evidenced and it escalates. Reputation with the third line is a currency best spent rarely.
Conclusion
If there is one formulation that captures the distinction running through this entire article, it is this: "the control was well designed, but we had no evidence it had operated during the period, so we treated it as ineffective."
That sentence contains the design-versus-operating distinction, the evidence standard, and the conservative default when evidence is absent. It is also, in practice, what most AI governance programs will discover about themselves the first time they are tested seriously.
The next article in this series covers the surrounding terrain a second-line AI function must be conversant in even where it does not own it: privacy, security, and the regulatory map.
Explore more: Start Here · Topics · Archive · About