7 Production Multi-Agent Design Patterns

A central orchestrator receives the top-level task, decomposes it into subtasks, dispatches each subtask to a specialized worker agent, and aggregates the results. The orchestrator is the single accountability point — it knows the full goal; workers know only their slice.

How it works in practice

The orchestrator holds the goal and a reasoning loop. It inspects each worker’s output before deciding the next dispatch. This is not a simple scatter-gather — the orchestrator can adapt its plan mid-flight based on what workers return. If the Research Agent surfaces an unexpected complication, the orchestrator can spawn an additional specialist to handle it before proceeding.

Real-world application: Customer support routing

A canonical example is tiered customer support. A first-line subagent extracts ticket type and sentiment, then attempts knowledge-base resolution. An escalation subagent evaluates the outcome and routes to the right specialist (billing, tech, complaints). A handoff subagent packages context for any human takeover. The orchestrator holds state across the full customer session and decides at each stage whether resolution has been achieved or escalation is required.

Drammeh’s 2026 incident-response study — 348 controlled trials — found that this orchestrator pattern produced a 100% actionable recommendation rate compared to 1.7% for single-agent systems, with 80× greater action specificity and 140× improvement in solution correctness.

Each agent in the chain receives the full output of the previous agent, transforms it according to its specialty, and passes the result forward. The order is fixed at design time. No branching, no parallel lanes — pure linear transformation.

The document processing template

Google’s ADK team describes the canonical pipeline example: a Parser Agent turns a raw PDF into clean text; an Extractor Agent pulls structured data from that text; a Summarizer Agent generates the final synopsis. Each step is focused, testable, and replaceable independently of the others. When the Extractor changes (because a new document type arrives), the Parser and Summarizer need not change at all.

The legal tech application is nearly identical: a Template Selection Agent identifies the appropriate base contract from client specifications; a Clause Customisation Agent modifies standard terms; a Regulatory Compliance Agent reviews against applicable law; a Risk Assessment Agent produces a liability rating and protective language recommendations.

Latency math you need to do before committing to pipeline

A five-stage pipeline where each stage takes 2 seconds adds a minimum of 10 seconds of end-to-end latency. For batch processing jobs that run overnight, this is fine. For a customer-facing feature that has to feel real-time, this is a UX catastrophe. If your latency budget is under 3 seconds, pipelines with more than two stages are almost certainly the wrong pattern. Consider Fan-Out (Pattern 3) for parallelisable components or a single carefully-scoped agent.

A dispatcher fans the task out to multiple agents working in parallel on independent subtasks. When all complete, an aggregator fans the results back in and synthesises a unified output. Wall-clock time collapses from N×T to approximately T (the time of the slowest agent) plus aggregation overhead.

The market intelligence pipeline

Consider competitive intelligence: four parallel collection agents — a news scraper, a financial data agent, a social sentiment analyser, and a competitor website monitor — all run simultaneously. After a timeout window, an Analysis Agent receives all four summaries and identifies cross-source trends. A Report Agent drafts the executive summary. What would take 8–10 minutes sequentially takes under 90 seconds in a well-tuned fan-out. Anthropic’s own research multi-agent system uses exactly this topology for parallel research subqueries.

Designing for partial failure

Never design a fan-in that blocks indefinitely. In production, one of your parallel agents will occasionally time out, hit a rate limit, or return an error. Your aggregator must handle partial results gracefully: define a minimum threshold of successful agent completions required before proceeding, and have a defined behaviour for missing inputs (use last known value, use empty slot, abort with informative error).

import asyncioasync def fan_out_with_timeout(agents, task, timeout=30):    tasks = [asyncio.create_task(agent.run(task)) for agent in agents]    results = await asyncio.gather(        *[asyncio.wait_for(t, timeout=timeout) for t in tasks],        return_exceptions=True  # don't let one failure kill the rest    )    # Separate successes from failures    valid = [r for r in results if not isinstance(r, Exception)]    if len(valid) < MIN_REQUIRED:        raise InsufficientResultsError(f"{len(valid)}/{len(agents)} agents succeeded")    return valid

A Generator Agent produces an initial output. An Evaluator Agent — typically using a separate model or a defined rubric — scores that output and generates structured feedback. The feedback is passed back to the Generator, which produces a revised version. The loop continues until the output clears a quality threshold or an iteration cap is hit.

Where this pattern earns its cost

Literary translation is the textbook example: the first attempt captures the plot but misses emotional register. The evaluator flags specific passages with structured feedback — “the original irony is lost in line 3, idiomatic expression in line 7 is too literal.” The generator incorporates that feedback in its next pass. Two cycles of this produces a translation that would have required three human review rounds to achieve otherwise.

DoorDash’s production grocery assistant (MAGIC) uses a trajectory-aware evaluator system called MAMUT-GEPA that optimises prompts across agents using multi-turn simulation and trajectory-level scoring. Their evaluator-optimizer pattern demonstrated measurable quality improvements over local sub-agent optimisation alone, with the gains compounding across the full conversation arc rather than plateauing at individual turn level.

A router agent classifies incoming requests and dispatches them to the most appropriate specialist — without the caller needing to know which specialist exists or what they handle. Routing can be rule-based, intent-classification-based, or handled by an LLM reasoning over available agents and the current context.

The multi-domain support case

A user contacts support without knowing or caring whether their issue is a billing question, a technical fault, or a compliance matter. A router agent classifies the intent from the first message and dispatches to the appropriate domain specialist. The billing agent has access to payment systems; the technical agent has access to infrastructure logs; the compliance agent has access to regulatory documents. No single specialist needs knowledge of the others’ domains.

The OpenAI Agents SDK implements this natively through its Handoffs primitive. The router makes a handoff decision, and the receiving agent gets the full conversation context plus any metadata the router attaches. Microsoft’s AI agent design patterns guide specifically recommends this pattern when the domain is unpredictable — and recommends starting centralised (with one router making all decisions) and decentralising only when concrete bottlenecks emerge.

When dynamic handoff beats orchestrator-worker

The key distinction is predictability of subtask structure. If you know at design time which specialists will be needed and in what order, use an Orchestrator-Worker. If the required specialist only becomes clear from the content of the request, use Dynamic Handoff. In real systems, these often compose: an orchestrator dispatches to a router for the domain-classification step, then coordinates the specialist output with other agents.

A top-level orchestrator manages sub-orchestrators, each of which manages its own team of specialist workers. Responsibility and context propagate downward; results aggregate upward. The pattern mirrors the organisational structure of effective engineering teams — a principal engineer delegates to team leads who delegate to individual contributors.

The complexity ceiling

This pattern exists because some problems genuinely cannot be solved by a single orchestration layer. An enterprise knowledge management system might have a top-level orchestrator that splits work between an Ingestion Sub-Orchestrator (which manages chunking, embedding, and indexing agents) and a Retrieval Sub-Orchestrator (which manages query expansion, ranking, and reranking agents), each with distinct tool sets and failure modes.

Context degradation across hops

The deepest practical problem with hierarchical architecture is context fidelity. A goal defined at the top level must be communicated accurately through two or more orchestration layers before it reaches the workers who actually act on it. Every hop is an opportunity for context to be summarised, rephrased, or subtly misrepresented. In production, pass structured context objects — not natural language summaries — between orchestration levels. Define explicit schemas for what each level passes down and receives back.

AdaptOrch’s 2026 benchmark (arXiv:2602.16873) reported 12–23% performance improvements over static single-topology baselines using identical underlying models — but these gains came entirely from intelligent topology routing at each level, not from the hierarchy itself. A flat system with smart routing matched hierarchical performance in most test cases. The message: when topology selection is the goal, routing beats hierarchy.

At one or more defined checkpoints, the system pauses automated execution and routes a decision, a draft, or an action proposal to a human for review and approval. The human’s input — approval, rejection, or modification — is injected back into the agent workflow before execution continues. The system is not fully autonomous; it is augmented autonomy with structured human oversight.

Where automation ends and judgement begins

Ally Tech Labs, deploying LangGraph for customer-facing AI, built exactly this: AI agents draft responses, which are routed to human reviewers before going live. The system improves over time through continuous evaluation of human edits — the reviewer’s corrections become training signal. What begins as a fully-supervised workflow gradually requires fewer corrections as the agents calibrate to human preferences, but the gate remains. The human is not a bottleneck; the human is the quality control layer.

Calibrating gate placement

Too many gates create review fatigue — humans begin rubber-stamping rather than genuinely evaluating. Too few gates miss critical intervention points. The discipline is identifying which decisions are actually high-stakes. A simple decision heuristic: if the agent’s action would be irreversible within 24 hours, it requires a gate. If it can be corrected or reverted easily, it may not.

• Always gate — actions that modify production data, trigger financial transactions, send external communications, or create legal commitments.
• Often gate — content that will be published publicly, plans with significant resource implications, escalations that affect customer relationships.
• Rarely gate — internal analysis, draft generation, data retrieval, classification tasks.