Multi-agent architectures attract attention because they sound sophisticated. In production they are frequently a complexity tax paid for no return.

That said, there are real cases where splitting helps. Here is how to tell them apart.

Start with one

The default should always be a single agent with a well-defined scope and the tools it needs.

One agent is easier to evaluate, easier to debug, cheaper to run, and has one place where context lives. Most tasks people split across three agents would be better served by one agent with three tools.

Split only when you hit a specific limitation you can name.

Good reasons to split

Genuinely different tool sets

An agent handling customer enquiries and an agent processing invoices need entirely different tools, different prompts and different evaluation sets. Combining them produces an agent that is mediocre at both and a prompt nobody can reason about.

Different permissions

This is the strongest reason. An agent that reads customer data should not have the ability to issue refunds. Splitting by permission boundary means the capability does not exist in the component handling untrusted input — which is a far stronger guarantee than instructing one agent not to use a tool it has.

Different models

A fast cheap model classifying incoming requests, routing to a more capable model only for the cases that need it. This is really routing rather than multi-agent, and it is one of the most practically valuable patterns.

Genuinely parallel work

Researching five companies simultaneously is faster with five parallel workers than one agent doing them in sequence. The work must be genuinely independent for this to help.

Different failure domains

If one part of a pipeline is unreliable — a flaky third-party service — isolating it means its failures do not take down the rest.

Bad reasons to split

Because the prompt got long. Shorten the prompt, or narrow the scope. Splitting does not reduce total complexity; it redistributes it and adds coordination.

Because it sounds more sophisticated. It is not an architecture decision if the driver is how it looks in a diagram.

To mimic an organisational chart. A "manager agent" directing "specialist agents" maps nicely onto a slide and badly onto a system. Software does not need the coordination structures humans need.

Because one agent occasionally gets confused. Fix the scope, the tools or the evaluation set. Adding agents rarely fixes a reasoning problem and reliably adds new ones.

Orchestration patterns

Deterministic pipeline

A fixed sequence: agent A always runs, then B, then C. Code controls the flow; agents do the work at each step.

Best for well-understood processes with a known shape. Predictable, debuggable, cheap. This covers the large majority of real cases and should be your default when you do split.

Router

A fast classifier decides which specialist handles the request, then hands off once.

Best for workloads with distinct categories — support versus sales versus technical. Simple and effective.

Supervisor

A coordinating agent decides which specialist to call and when, potentially several times.

Best for genuinely open-ended tasks where the sequence cannot be known in advance.

Costs: unpredictable number of calls, unpredictable cost, harder to evaluate, and the supervisor becomes a single point of confusion. Use only when a deterministic pipeline genuinely cannot express the task.

Parallel workers

The same agent runs across many inputs simultaneously, results collected at the end.

Best for batch work over independent items. Straightforward and effective.

The costs of splitting

Worth being explicit, because they are consistently underestimated.

Context loss at every handoff. Agent A knows something agent B needs and did not pass on. This is the most common failure mode in multi-agent systems and it is frustrating to debug.

Multiplied cost. Each agent carries its own system prompt and context. Three agents on a task is often three times the tokens.

Latency accumulation. Sequential agents add their latencies. For anything interactive this adds up quickly.

Harder evaluation. You now need to evaluate each agent and the system as a whole. A failure could be in any component or in the handoff between two.

Harder debugging. Tracing a wrong outcome across four agents and six handoffs is substantially harder than tracing it through one.

More failure modes. Each agent and each handoff can fail independently.

Keeping it manageable

If you do split:

Prefer deterministic orchestration. Let code decide the flow and agents do the work. Agents deciding among themselves is where unpredictability enters.

Pass structured context explicitly. Not a conversation transcript — a defined object with the fields the next agent needs. This prevents the context-loss failure mode almost entirely.

Give each agent a narrow, testable job, with its own evaluation set.

Log the whole trace, with a shared identifier across agents, so you can follow one request end to end.

Set a global budget on total calls and total cost per request, so a loop between agents cannot run away.

Limit depth. Agents calling agents calling agents becomes impossible to reason about. Two levels is usually enough.

A practical rule

Before adding a second agent, write down the specific limitation forcing it. "The prompt is getting complicated" is not one. "The agent handling untrusted customer input must not have refund permissions" is.

If you cannot name the limitation precisely, you probably need better scoping on your single agent rather than a second one.

What tends to work

In production, the architectures that hold up are unglamorous:

  • One agent with well-chosen tools, for most tasks
  • A router plus two or three specialists, where categories are genuinely distinct
  • A deterministic pipeline of focused steps, for known processes
  • Parallel workers for batch work

The open-ended supervisor pattern, where agents negotiate among themselves, remains difficult to make reliable and expensive to run. It is where most of the published excitement is and where least of the production value currently sits.

If you are designing an agent system and want a view on whether to split it, describe the task.