$cat why-multi-agent-pipelines-fail.md

Why Multi-Agent Pipelines Fail: Context Lost in Handoffs

July 27, 202614 min readby MDflowview as .md

A chain of emerald wireframe cubes passing a stream of glowing particles that leaks away in the gaps between them and falls into one larger solid glowing node below, on a dark terminal grid

Every team that builds an agentic system reaches the same fork. You have a workflow with four clear steps, so you build four agents, one per step, and an orchestrator to connect them. It maps cleanly onto how the work is done today. It looks right on a whiteboard.

Then you look closely at the output and it is subtly, expensively wrong. Every step is individually defensible and the answer as a whole does not hold together.

Subbiah Sethuraman and Abhilash Asokan of ZS Associates gave a talk at AI Engineer about doing exactly this, catching it, and rebuilding. It is one of the more useful fifteen minutes in the whole conference, because they name the failure precisely rather than blaming the model.

TL;DR — Multi-agent pipelines fail because context leaks at every handoff, judgment gets dispersed across agents that never see each other's reasoning, and no agent shares an understanding of the domain. The fix is not a better handoff schema: pull the deterministic work out of the agent entirely, let one agent own the reasoning end to end, keep parallelism only for investigation, and give it a durable shared layer of domain knowledge to navigate. That knowledge has a structured half — a graph — and a prose half, which belongs in plain markdown every agent can read. MDflow is a home for the prose half. Start free.

What went wrong: one agent per step

The pipeline mirrored the human workflow, and that turned out to be the design flaw. ZS works in pharma commercial analytics, where an analyst investigating a brand follows four steps: detect a signal (prescriptions dropped), find the source (where is the drop concentrated), find the driver (why), then synthesise an action and an outlook.

So they built an agent per step — a signal detection agent, a source localisation agent, a driver attribution agent, a synthesis agent — with an orchestrator wiring them together. The system produced a tidy information packet: brand prescriptions down 18% in a territory over four weeks, cause identified as a payer moving the drug to a lower coverage tier, recommended action, projected outlook.

The cause was right. The action was not. Because patients now faced a higher out-of-pocket cost, the recommendation should have engaged the payer side. Instead it said what a generic recommendation always says: send more reps to talk to doctors. And because the action was wrong, the outlook attached to it was wrong too.

Their diagnosis is the sentence worth pinning to the wall:

At each level, it derived the right fact, but there is no single agent which understands the end-to-end picture.

This is the characteristic failure of pipeline-shaped agent systems. It does not look like a crash or a hallucination. It looks like a plausible report that quietly contradicts itself.

The three root causes

None of them is model quality. All three are architecture, and all three generalise well past pharma.

1. A language model was doing a deterministic job

Signal detection — "did this metric drop enough to be worth investigating?" — is statistics. It has thresholds, seasonality and significance tests. Handing it to an LLM means sometimes it applies a statistical method, sometimes it eyeballs the numbers, and sometimes it promotes noise to a signal. You have taken the most verifiable step in your pipeline and made it non-deterministic for no gain.

2. Context leaked at every handoff

The driver attribution agent worked out that a payer coverage change had made the drug expensive for patients. The synthesis agent downstream never understood the weight of that — what an insurance tier change implies for prescription behaviour. It received a conclusion without the reasoning that produced it.

This is the core mechanic. A handoff is a lossy compression step. What crosses the boundary is a summary; what stays behind is the chain of evidence, the hypotheses that were rejected, and the confidence attached to each. Add a boundary and you add a loss.

Cognition made the same argument from the coding-agent side in Don't Build Multi-Agents (Walden Yan, June 2025), with two principles that have aged well: share context — pass full traces, not just messages — and actions carry implicit decisions, because a subagent that cannot see its sibling's work will make a conflicting assumption and nobody will notice until the pieces are assembled. Their illustration is a game clone where one subagent builds a Mario-style background while another builds an incompatible bird, and the final agent inherits both mistakes with no way to reconcile them.

3. No shared understanding of the domain

None of the agents knew what the business actually meant. What TRX is, how it relates to NRX, which KPI drives which, what a payer tier change implies downstream. Each agent inferred the relationships from tables at run time, which does not scale and, worse, invents relationships that are not in the data.

This is the failure that persists even if you fix the first two. You can consolidate to one perfectly context-preserving agent and it will still reason badly if the meaning of your metrics exists only in the heads of your analysts.

The fix: watch a coding agent, then copy it

Their first instinct was to redesign the topology — better handoff schemas, better tool boundaries — and they deliberately did not do that. Instead they opened an empty directory, gave Claude Code nothing but bash and the database, handed it a real signal, and watched how it worked.

That produced three design decisions.

Separate the deterministic part completely. Signal detection moved out of the agentic system into a plain pipeline: statistical methods, guardrails, thresholds, prioritisation. It scans the data, identifies signals per KPI, and puts them on a queue. A signal landing on the queue wakes the agent. The agent's job is now to investigate a signal, never to decide what counts as one.

One agent owns the reasoning. They consolidated the four agents into one — and this is the part people misread, so it is worth being exact. They did not remove parallelism. They removed distributed reasoning. Sub-agents are still spawned dynamically for focused investigations ("what did rep activity do in this region?"), exactly as a coding agent spawns a search. What comes back is a result, not a judgment. The judgment stays with the main agent, which has seen everything.

Give it a domain model to navigate. They sat with pharma domain experts and built a knowledge graph: geographies, payers, accounts, brands, KPIs, and the relationships between them — including how one KPI drives another. Then the key move:

The knowledge graph is not just something the agent looks up for data. It is a control plane for the agent.

Every edge in the graph is a hypothesis the agent may evaluate, and the graph bounds which ones exist. The loop is: start at an entity, look at its neighbourhood, derive hypotheses from the edges, go back to the real data and test each one, keep traversing where the evidence supports it, stop when the root cause is found or the hypotheses run out. Roughly fifty turns and a lot of tokens later, it produces analysis that used to take an analyst three to four weeks — in twenty to thirty minutes.

The graph is doing something specific here: it turns an unbounded permutation space (which region × which payer × which account × which KPI) into a walkable set of testable claims.

But Anthropic's multi-agent system works — what gives?

Both things are true, because they parallelise different things. Anthropic's research system reported that a lead agent with parallel subagents outperformed a single-agent baseline by 90.2% on their internal research eval — while burning roughly 15× the tokens of a chat interaction.

Look at what those subagents do: they search, in parallel, over a read-only world, and hand findings back to a lead agent that owns the synthesis. Nobody downstream acts on a sibling's conclusion without the lead seeing it. The judgment is still single-threaded. That is the same shape ZS arrived at, and the same shape Cognition recommends.

So the useful rule is not "single agent good, multi-agent bad". It is:

Parallelise breadth — retrieval, search, independent investigations, anything read-only. Cheap, and it compounds.
Serialise judgment — one agent holds the end-to-end picture and makes the calls. Every split here costs you coherence.
Delete the deterministic parts — if a threshold or a query can decide it, do not ask a model to.

Which applications benefit most

Analytics and root-cause investigation — anywhere the workflow is detect → localise → explain → recommend, which is most of BI, observability and revenue analysis.
Support triage and incident response — where a wrong action attached to a correct diagnosis is the expensive failure mode.
Agentic coding — the original home of this argument; plan, implement and review lose fidelity across every context boundary.
Regulated workflows — claims, compliance, underwriting, clinical operations, where "the recommendation did not follow from the finding" is an audit event.
Anything with heavy domain jargon — pharma, finance, logistics, energy, where an agent that does not know what your metrics mean will confidently misread them.
Multi-team agent estates — where five teams each ship an agent and each one re-derives the same domain knowledge from scratch.

How MDflow fits

MDflow is a shared context substrate — the place where the domain knowledge lives so agents read it rather than re-infer it, and so it survives the handoffs. We should be precise about the seam, because it is the same one we drew in A Semantic Layer for AI Agents: a graph owns structure and scale — entities, edges, high-volume traversal. Prose owns meaning: what this metric means, why the 2024 method differs, which source is authoritative, what convention the answer has to follow. ZS's team spent weeks extracting that second half out of domain experts' heads. Almost none of it was written down anywhere first.

What already lines up today

Folder descriptions are written domain knowledge, ranked as retrieval signal. Every folder carries a description saying what its documents are for, and mdflow_get_context ranks those descriptions above folder names and document titles — retrieval without a vector database. A folder described as "definitions and reporting conventions for commercial KPIs — TRX, NRX, payer tiers and how they relate" is exactly the shared understanding the ZS agents lacked, and it is one sentence you can edit.

One substrate, many agents. A hosted remote MCP server — Claude and the ChatGPT app connect with an OAuth sign-in and no pasted token, while Cursor, Codex and Claude Code use a Personal Access Token — plus an HTTP API with an OpenAPI spec, a VS Code extension, an n8n node and a web clipper. When five agents read from the same workspace, the domain knowledge is not something each of them carries a private, drifting copy of.

Sub-agent findings can land in a document instead of a summary string. An agent that finishes an investigation can write the result back to the workspace — the evidence, not just the verdict — so the next run, the next agent, or the human reviewing it reads the reasoning rather than a compressed conclusion. Version history keeps a line-by-line diff and non-destructive restore on every write path, and the Document Log names the actor (automated · <token name>), so an agent-written finding is attributable rather than anonymous.

Markdown, so it is inspectable and portable. Documents are plain markdown with YAML frontmatter, and every shared document has a raw .md twin any tool or model can fetch — this post included, at /blog/why-multi-agent-pipelines-fail.md. A domain model you cannot read and correct is not shared context; it is a second black box next to the first one.

Governance on the knowledge itself. Workspaces scope a domain per client or product line, email shares carry viewer or commenter roles, and per-document client-side encryption covers material that should never be readable by anything but you.

Where we are headed

Direction, not a dated commitment: folder descriptions that express relationships between folders, not only what each contains — the prose analogue of an edge; serving a whole collection to an agent as one cross-linked bundle so a definition, its exceptions and its decision log arrive together instead of as three separate retrievals; richer typed frontmatter so a document can carry owner, status and freshness as signals an agent weighs; and scoped tokens so an agent's reach into the substrate matches its job.

The bottom line

The instinct to build one agent per workflow step is a human org chart projected onto software, and ZS's first takeaway is to resist it: derive the architecture from the work, not from how the team is organised.

What survives that derivation is a smaller, stranger system than the whiteboard version. Deterministic work runs deterministically, before any model is involved. One agent owns the reasoning end to end and spawns help for investigation, not for judgment. And underneath it sits a durable layer of domain knowledge that every agent reads — the structured half in a graph, the prose half in markdown.

Context does not need to be handed off if it was never split up.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

Why do multi-agent pipelines fail?

They usually fail for three reasons, none of which is model quality. Context leaks at every handoff, because a downstream agent receives a summary rather than the reasoning that produced it. Judgment is dispersed, so no single agent owns the end-to-end picture and each step can be individually correct while the whole answer is incoherent. And no agent shares an understanding of the business domain, so each one re-infers what the metrics mean from raw data. ZS Associates described exactly this failure in their pharma analytics pipeline at AI Engineer in July 2026.

Is a single agent better than a multi-agent system?

For reasoning, usually yes. The pattern most teams converged on by 2026 is a single agent that owns judgment end to end, with parallelism kept for retrieval and investigation rather than for decisions. Cognition argued this in "Don't Build Multi-Agents" in June 2025, and ZS Associates reached the same conclusion by watching how a coding agent solved the problem in one context. Anthropic's research system does run subagents in parallel, but they are read-only searchers whose findings a lead agent synthesises — the judgment is still single-threaded.

What is the difference between parallelism and distributed reasoning?

Parallelism means running several units of work at the same time — fetching data, searching sources, testing hypotheses. Distributed reasoning means splitting the judgment itself across agents, so one decides the cause and another independently decides what to do about it. Parallelism is nearly free and worth keeping. Distributed reasoning is what breaks, because every split is a place where the reasoning behind a conclusion is dropped and only the conclusion survives.

Which parts of an agentic workflow should not be agentic?

Anything deterministic. If a step can be done with a query, a statistical test or a threshold, a language model will do it more slowly, more expensively and less reliably. ZS Associates moved signal detection out of their agent entirely into a deterministic pipeline with statistical methods, guardrails, thresholds and prioritisation, and put the resulting signals on a queue. The agent wakes up to investigate a signal it is handed, not to decide what counts as one.

How do you give every agent the same domain knowledge?

Write it down once in a durable place both people and agents read, and have agents load it at run time instead of inferring it from raw data. That means the definitions of your metrics, how they relate, which source is authoritative and the conventions for how a question has always been answered. Structure and scale belong in a graph or a warehouse; the meaning belongs in plain prose. In MDflow that prose lives as markdown documents with folder descriptions that act as retrieval context over MCP, so every agent reads the same substrate rather than carrying its own copy.