The Question Nobody Has Answered Yet
Every model risk framework in banking rests on a quiet assumption: a model is a single, identifiable thing. It has an owner. It has inputs, outputs, and a documented methodology. It sits in an inventory with a tier, a validation date, and a performance monitoring plan. When it drifts, you recalibrate it. When it fails, you know where to look.
Agentic AI breaks that assumption in a specific and underappreciated way. The problem is not that agents are powered by large language models; institutions are already adapting their frameworks for LLMs. The problem is the orchestrator: the supervising agent that decomposes a task, routes work to specialist agents, evaluates their outputs, retries or reroutes when results are unsatisfactory, and assembles the final answer. In a multi-agent architecture, the orchestrator is not just another model in the inventory. It is a manager. And our validation frameworks were never designed to validate a manager.
So the question every model risk officer should be asking before the first multi-agent system touches a production process is simple to state and hard to answer: who validates the orchestrator, and what does validating it even mean?
Why This Is Not Just Another Model Validation
Consider what a traditional validation, in the spirit of SR 11-7, actually examines: conceptual soundness of the methodology, quality of the data, developmental evidence, outcomes analysis, and ongoing monitoring. All of it presumes a stable mapping from inputs to outputs that can be tested, benchmarked, and challenged.
Now look at what an orchestrator does. Its output is not a number or a classification. Its output is a plan: a dynamically generated sequence of decisions about which agents to invoke, in what order, with what instructions, and with what tolerance for their answers. Three properties make this qualitatively different from anything in the existing inventory.
1. The unit of risk is the interaction, not the component
You can validate each specialist agent in isolation and still have no idea how the system behaves as a whole. A retrieval agent that is individually sound, a calculation agent that is individually sound, and an orchestrator that is individually sound can jointly produce failures that none of them would produce alone: circular delegation loops, error compounding across handoffs, or an orchestrator confidently accepting a wrong answer because it arrived well formatted. Emergent behavior is not a defect in any component; it is a property of the composition. Component-level validation is necessary but nowhere near sufficient.
2. The decision path is non-deterministic and non-reproducible
Run the same request through a multi-agent system twice and you may get two different execution paths that both arrive at acceptable answers. Which one do you validate? Traditional backtesting assumes you can replay history against the model and compare outcomes. When the object under test is a probabilistic planner choosing among probabilistic workers, replay does not reproduce the decision path, and outcomes analysis has to be redefined around distributions of behavior rather than point comparisons.
3. The orchestrator changes the effective risk tier of everything beneath it
A low-risk summarization agent becomes something else entirely when an orchestrator can chain it into a workflow that feeds a funding decision. Tiering in most inventories is assigned per model, based on materiality and reliance. In an agentic architecture, materiality is a function of what the orchestrator is permitted to connect. The tier of the system is set by the orchestrator’s reach, not by the modesty of any individual agent.
What SR 11-7 and SR 26-2 Give Us, and Where the Gap Is
SR 11-7 remains remarkably durable. Its core demands, effective challenge, independent validation, documented conceptual soundness, and ongoing monitoring, apply to agentic systems without modification. The gap is not in the principles; it is in the object those principles are pointed at.
Supervisory expectations for AI have been moving in the right direction by pressing on explainability, dynamic model behavior, and the governance of systems that adapt after deployment. But even the most current guidance largely contemplates a model, singular.
Multi-agent systems force us to extend the discipline along three axes.
1. From model validation to system validation
The inventory needs a new kind of entry: the composition itself, registered, tiered, and validated as a first-class object, with the orchestrator’s routing logic, delegation rules, and acceptance criteria treated as methodology subject to effective challenge.
2. From outcomes analysis to behavioral analysis
Validators need to test the distribution of execution paths, not just the distribution of answers: how often does the orchestrator retry, escalate, or override a specialist? Under what perturbations does the plan destabilize? What happens when a downstream agent is degraded, slow, or adversarially manipulated? This looks less like back-testing and more like chaos engineering with a validation report attached.
3. Periodic revalidation to continuous supervision
An orchestrator that can revise its own plans in response to intermediate results is, in a meaningful sense, never the same system twice. Annual validation cycles were built for static models. Agentic systems need runtime evidence: full decision-path logging, replayable traces, and monitored behavioral envelopes, so that validation becomes a continuous property of the system rather than a point-in-time certificate.
Controls Validate Differently Than Guardrails
A distinction I have written about throughout this series matters enormously here. Controls are deterministic and structural: the orchestrator cannot invoke an agent outside its approved registry, cannot exceed a spending or action budget, cannot touch a production system without a human approval gate. Guardrails are probabilistic and behavioral: output evaluation, tone and policy classifiers, anomaly detection on decision paths.
The validation implications are different in kind. Controls can be validated the way we validate access management: prove the boundary holds under adversarial testing and prove it cannot be reasoned around by the orchestrator itself. Guardrails can only be validated statistically: measure their catch rates, false positive rates, and degradation over time, and accept that they will sometimes fail. A validation report that does not distinguish between the two is making an implicit and dangerous claim: that probabilistic protections are load-bearing walls. They are not. In a multi-agent system, controls define the blast radius; guardrails only reduce the frequency of incidents inside it. The orchestrator’s permission boundary is therefore the single most important object in the entire validation.
A Practical Frame: Five Pillars Applied to the Orchestrator
Mapping this to the five-pillar governance model I use for agentic AI:
1. Accountability structures
Every orchestrator needs a named human owner accountable for the system’s aggregate behavior, not just for the component models. If an agent managed by the orchestrator causes a loss, accountability cannot dissolve into the architecture. The management chain of agents must terminate in a management chain of people.
2. Inventory and classification
Register the composition, not just the components. Tier the system by the orchestrator’s maximum reach: the most consequential action it could take through any permitted chain of delegation, not the average case.
3. Independent validation
Validators need new instruments: decision-path replay, multi-agent stress scenarios, degraded-component testing, and adversarial prompts aimed at the orchestrator’s planning layer rather than at any single agent. Effective challenge must include the question: can the orchestrator be induced to assemble a harmful plan out of individually harmless steps?
4. Use controls and guardrails
Enforce the distinction above. Deterministic permission boundaries, action budgets, and human approval gates on irreversible actions are the validated perimeter. Behavioral guardrails are monitored, measured, and never treated as the last line of defense.
5. Change management and issue remediation
Define what a change even is. A new specialist agent added to the registry, a revised system prompt on the orchestrator, an upgraded foundation model underneath any component: each of these changes the system and should trigger proportionate revalidation. And every orchestrator needs a tested kill switch with a defined safe state, because remediation in an agentic system is measured in minutes, not review cycles.
The Uncomfortable Conclusion
Here is the honest answer to the title question. Today, in most institutions, nobody validates the orchestrator, because the orchestrator does not exist as an object in the model risk framework. It hides between the entries in the inventory. The individual agents get validated; the manager that directs them does not.
That is exactly the kind of gap that supervisory findings are made of. The institutions that get ahead of this will do three things: register multi-agent compositions as first-class inventory objects, build validation practices around behavior and decision paths rather than point outputs, and hold the deterministic control boundary to a far higher evidentiary standard than the probabilistic guardrails inside it.
The second line of defense has spent a decade learning to challenge models. The next challenge is bigger: learning to challenge management, when the manager is a model.
Explore more: Start Here · Topics · Archive · About