Aggregate Interaction Risk: A New Governance Dimension for Multi-Agent AI

“A system is never the sum of its parts; it's the product of their interaction.” - Russell Ackoff, pioneering systems thinker and former Wharton professor

Russell Ackoff made this observation decades before the rise of agentic AI. Yet it is becoming increasingly relevant as banks move from deploying individual AI agents to building multi-agent architectures.

A Systems Thinking Lens for Multi-Agent AI

One of the less discussed elements of the revised U.S. interagency model risk management guidance, SR 26-2, is its treatment of model risk at both the individual and aggregate levels. The guidance recognizes that interactions and dependencies among models, reliance on common assumptions, data or methodologies, and other factors affecting several models simultaneously can create aggregate risk even when individual models appear sound.

Generative AI and agentic AI are outside the scope of SR 26-2 guidance. The guidance nevertheless notes that banks’ broader risk-management and governance practices should inform appropriate controls for tools and systems outside its scope. Also, SR 26-2's treatment of aggregate model risk offers a useful systems-thinking lens as banks develop governance approaches for multi-agent AI.

Understanding the design, data, permissions, tools, performance, and risk profile of each participating agent remains essential. Banks also need to consider those risks collectively across the multi-agent system. But even that aggregate view of individual-agent risk may not reveal the full risk of the system.

Is the aggregate risk of a multi-agent system simply the sum of the risks associated with its component agents?

No. Additional risk can emerge from how agents exchange information, depend on shared resources, hand off tasks, delegate authority and collectively influence decisions or actions.

Conceptually, the aggregate risk of a multi-agent system can be viewed as:

Aggregate Risk = Σ (Individual Agent Risks) + Aggregate Interaction Risk

where:

• Σ (Individual Agent Risks) represents the combined risks associated with each participating agent when considered individually.

• Aggregate Interaction Risk represents the additional risk created through interactions, dependencies, handoffs, delegated authority, shared resources, and correlated behavior across the multi-agent system.

This is a conceptual governance framework, not a quantitative formula. Interactions may amplify, mitigate, or reshape the risks associated with individual agents. The key point is that evaluating each agent separately, even when those assessments are viewed in aggregate, may not fully capture the risk of the overall system.

Both dimensions matter. This article focuses on the second: Aggregate Interaction Risk, a dimension that may be overlooked when governance remains centered on individual agents.

From Individual Agents to Agentic AI Ecosystems

Most agentic AI governance today appropriately begins with the individual agent-related questions: Does it perform as intended? Does it use approved data and tools? Does it remain within its permissions? Are its outputs reliable? Does it escalate appropriately?

Those controls remain essential. But once multiple specialized agents begin collaborating within the same business process, governance must also ask a second set of questions:

• How does information move from one agent to another?

• Can errors, unsupported assumptions, or stale data propagate across the workflow?

• Are downstream agents independently challenging an output, or merely reinforcing the same upstream conclusion?

• Do shared models, tools, data sources or memory create concentration risk?

• Can permissions or authority accumulate as tasks pass from one agent to another?

• Is there clear end-to-end ownership when several agents collectively influence a decision or action?

  
  

These interaction-level controls become increasingly important because a multi-agent system can produce risk that is not visible when each agent is assessed in isolation.

The need to govern these interactions is becoming more than a theoretical concern as banks increasingly deploy or consider multi-agent systems.

• BNY reports 134 "digital employees" developed through its Eliza platform and describes them as multi-agentic AI solutions that operate autonomously alongside human colleagues.

  
  

• Citi has introduced Arc, an enterprise platform for building and scaling AI agents across the firm within its existing technology and risk governance framework. The bank describes future workflows in which teams of AI agents collaborate on tasks such as research, analysis, preparation and execution, with every agent monitored, auditable and governed.

  
  

• DBS Bank has disclosed a commercial credit workflow in which as many as 70 specialized AI agents analyze financial statements, annual reports, industry information and customer interactions before producing a draft credit memorandum for human review. The bank has also described an enterprise control plane including an agent registry, guardrails, observability, traceability and agent evaluations.

  
  

• UBS has described a reference architecture in which an orchestration engine discovers relevant agents from an agent library, formulates execution plans, and coordinates multiple agents, tools, memory, guardrails, and human checkpoints. While this represents an architectural vision rather than a disclosed production deployment, it illustrates the multi-agent environments banks are preparing to govern.

These developments suggest that the unit of governance may need to expand from the individual agent to the end-to-end agentic workflow. Banks will continue governing individual agents, but they may also need to govern the broader system that emerges when those agents begin working together.

When Good Agents Go Wrong Together

Consider a commercial underwriting workflow.

One agent extracts borrower information. Another analyzes cash flow and leverage. A third evaluates collateral. A policy agent reviews underwriting standards and exceptions. Another prepares the credit recommendation for the credit officer.

Each agent may perform exactly as designed. Yet an incorrect borrower classification, stale financial information, or unsupported assumption introduced upstream can silently propagate throughout the workflow.

Downstream agents may appear to independently validate the recommendation when they are actually relying on the same upstream information. Their agreement creates a consensus illusion. Confidence increases because several agents reach similar conclusions, even though they are all reinforcing the same original error.

The aggregate risk of the workflow is therefore not simply the combined risks of its participating agents. Additional risk emerges from the way information, assumptions, confidence and decisions propagate through the system.

Context may degrade during handoffs. Confidence may be amplified through repeated reuse of outputs. Permissions may accumulate through delegation.

Shared memory, data sources, tools, foundation models or orchestration layers may become concentration points.

What begins as a localized error may cascade into pricing, risk ratings, approval decisions, customer treatment, portfolio reporting and, at sufficient scale, strategic risk decisions.

The participating agents may all remain within their individual operating parameters while the overall workflow produces an outcome outside the bank's risk appetite.

The Missing Dimension

The aggregate risk of a multi-agent system cannot be understood solely by evaluating each participating agent and combining those assessments.

Banks still need to understand the design, data, models, tools, permissions, performance, and risk profile of each agent. Viewed collectively, these form the aggregate individual-agent risk represented by the first term in the framework, as defined earlier:

Aggregate Risk = Σ (Individual-Agent Risks) + Aggregate Interaction Risk

The second term captures a different source of risk: what emerges when agents begin operating as an interconnected system.

An interaction risk may arise during a single exchange. One agent may pass incomplete context to another. A downstream agent may interpret uncertainty as fact. Several agents may rely on the same data source, model or upstream conclusion without providing truly independent challenge.

Aggregate Interaction Risk emerges when these interactions and dependencies combine across a broader workflow. Errors can propagate, assumptions can become reinforced, permissions can accumulate and shared components can become concentration points. The resulting system behavior may be difficult to infer from the performance of any individual agent.

A bank may have well-controlled individual agents and still face material risk at the system level. Hence, these two dimensions require different forms of governance.

• Individual-agent governance asks whether each agent is properly designed, controlled, and operating within its defined parameters.

• Interaction governance asks what happens when those agents exchange information, share resources, delegate authority, and collectively influence an end-to-end decision or action.

Implications for Banks

As banks adopt multi-agent architectures, governance may need to assess both dimensions of aggregate risk.

• Agent inventories: For material multi-agent systems, agent inventories may need to be supplemented by interaction maps documenting information flows, shared dependencies, delegated authority, common tools, shared memory and orchestration pathways.

  
  

• Validation: Testing and independent review may need to assess error propagation, cascading failure scenarios and correlated failures across agents and shared components. Banks may need to assess whether erroneous assumptions introduced by one agent are detected, challenged, contained or amplified by downstream agents.

  
  

• Monitoring needs to increasingly focus on interaction pathways and end-to-end outcomes, not only individual-agent performance. Changes to one agent, shared model, data source, or tool could alter the behavior of multiple downstream agents even when none independently breaches its performance thresholds.

  
  

• Accountability also needs to extend beyond ownership of individual agents. A clearly designated business or risk owner may need to understand and remain accountable for the end-to-end effects of agent interactions across the business process.

  
  

• Third-party: The agentic AI system's interaction map may also need to incorporate third-party dependencies where internal agents rely on vendor models, external tools, cloud services or third-party data.

  
  

• Risk appetite statements may eventually establish limits for interaction complexity, shared dependencies, delegated decision authority and the potential severity of cascading failures.

  
  

Banks are not starting from scratch. Existing operational resilience, technology-risk and third-party-risk frameworks already provide mechanisms for identifying critical dependencies and concentration risk.

In the U.S, the Interagency Guidance on Third-Party Relationships: Risk

Management supports maintaining a complete inventory of third-party relationships and assessing concentration risk. In the EU, DORA requires financial entities to maintain a register of ICT-service arrangements and manage ICT third-party dependencies and concentration risk.

These existing dependency, inventory, and concentration-risk practices provide a useful foundation for governing multi-agent AI. These frameworks provide an important foundation for mapping the structural dependencies surrounding a multi-agent system. The additional challenge is to understand the risks associated with its behavioral interactions: how information, assumptions, confidence, permissions, and decision authority propagate among agents and influence the end-to-end outcome.

The Next Frontier

As multi-agent architectures develop, the central governance question cannot remain limited to:

"Does each agent operate as designed?"

It should also consider the additional dimension:

"Can every agent operate as designed while the overall multi-agent system operates inside the bank's risk appetite?"

Answering that question may require banks to broaden their governance outlook as they move from individual-agent applications to multi-agent architectures.

This could include mapping interaction pathways and shared dependencies, assigning end-to-end ownership of agentic workflows, testing how errors and assumptions propagate across agents, and defining limits for delegated authority, interaction complexity and cascade severity.

Existing model, technology, operational risk, compliance, third-party risk and AI governance practices remain essential foundations. The opportunity is to connect these disciplines across the entire multi-agent workflow so that governance captures both dimensions of aggregate risk: the combined risks of individual agents and the additional risks created through their interactions.

As Ackoff’s systems-thinking principle suggests, the behavior of the whole depends on the interactions among its parts.

For banks, the next frontier of multi-agent AI governance is therefore not simply better oversight of individual agents. It is the ability to see, test, and control the risks that emerge only when those agents begin operating as a system.

That is why Aggregate Interaction Risk may become an essential governance dimension for banks adopting multi-agent AI.

About the Author

Naresh Raheja

is a risk management specialist focused on AI governance, model risk management, and extreme event-related financial risks. He has worked across banking, insurance, and regulatory environments, including roles at the Office of the Comptroller of the Currency and Moody's.