# How do you design a graph of AI agents?

Date: 2026-09-10 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/design-graph-of-agents/

**Answer:** Start from the units of work and the checks each unit must pass, then add the nodes that connect them, which are a splitter that breaks the job into units, workers that run in parallel where units are independent, plain code steps, and gates that can fail a unit. Send a failed unit back alone with the evidence of what failed, turn repeated failures into lessons for the brief, and keep a person at the step where a mistake would do real damage.

A single AI loop makes one piece of work correct: produce it, check it, fix what failed, repeat. Real jobs usually contain many pieces. Thirty help articles to update after a price change, forty sources to read for a report, a dozen files to change in a codebase. A **graph** is the plan that organizes those pieces: which units of work exist, which step handles each one, what can run at the same time, where the checks sit, and where failed work goes next.

The word comes from mathematics, where a graph is a set of **nodes** (points) joined by **edges** (connections). In an agent system a node is a step that does something, either a model call or ordinary code, and an edge says which step runs next. Frameworks such as LangGraph build workflows this way explicitly, and Anthropic's guide to building agents describes the common shapes. You do not need a framework to design one, though. The design questions come first.

## The four kinds of node

Most agent graphs are built from four roles.

| Role | What it does | Usually | Example |
|---|---|---|---|
| Splitter | Breaks the job into units and writes a brief for each | A model, checked by code | Lists the 30 articles that mention the old price |
| Worker | Does one unit | A model | Rewrites one article |
| Code step | Does something exact and repeatable | Plain code | Fetches a page, counts words, merges results |
| Gate | Decides whether a unit passes | Code first, a model judge only where code cannot decide | Fails any article that still shows the old price |

Anthropic calls the first two together the **orchestrator-workers** pattern: a central model breaks the task down, hands pieces to workers and combines what comes back. Its multi-agent research system works that way, with a lead agent planning the search and starting several subagents that search at the same time, each with its own context window.

Two points about the splitter. Its output is a list, so check it with code: does it cover every item it should, with no duplicates? And each unit's brief must stand on its own. Anthropic reports that vague instructions to its research subagents led them to duplicate each other's work or leave gaps; each one needed an objective, an output format, guidance on tools and sources, and clear limits.

Gates deserve the most care, because a graph is only as reliable as its checks. A gate is real only if it can fail a unit on its own with evidence the worker cannot argue with, such as a test that fails, a quote that is not in the source, or a file changed outside the plan. The companion article on [what makes a check real](/en/learn/what-makes-ai-check-real/) covers this in detail.

## Decide what runs in parallel

Units that do not depend on each other can run at the same time. Anthropic calls this **sectioning**, one of its two forms of parallel work (the other, **voting**, runs the same task several times and compares the results). In LangGraph, a node can fan out to the same worker once per item with `Send`, each copy getting its own input, and a **reducer** (a small function that says how results are combined) collects the outputs, for example by adding each one to a list. The OpenAI Agents SDK documentation shows the plain-Python version: run independent agents together with `asyncio.gather`, and keep the flow in your own code when you want speed, cost and behavior to be predictable.

Parallel work has two real costs, and the design must account for both.

**Tokens.** Anthropic reports that its multi-agent research setup used about fifteen times as many tokens as an ordinary chat. That pays off for broad research questions, much less for small jobs.

**Consistency.** Workers that cannot see each other's work make decisions independently, and those decisions can clash. Cognition, the company behind the Devin coding agent, argued in 2025 that this makes parallel subagents fragile for tasks like writing code, where every part depends on choices made in the others. Anthropic made a related point: tasks in which all agents need the same context, including most coding, are a poor fit for splitting across agents today.

The practical rule: split only along lines where units really are independent, and put any shared decision (the new price, the tone, the naming convention) in every unit's brief before the workers start. Then add a code step after the join that checks the units against each other, not only one by one.

## Two ways back: correction and learning

A graph has two kinds of return path, and mixing them up is a common design error.

| | Correction edge | Learning edge |
|---|---|---|
| Purpose | Fix this unit now | Stop the same failure in future units |
| Carries | The unit, the evidence of what failed, the scope of the fix | A short lesson |
| Goes to | The worker, for one more attempt | The brief or instructions every worker reads |
| Example | "Paragraph 3 still says $10/month; change only that paragraph" | "Write prices as $12/month, never as 12 dollars" |
| Limit | A fixed number of attempts, then a person | Reviewed before it becomes permanent |

The **correction edge** is the short loop. It should be specific: what failed (the gate's output, copied in full) and how much of the unit may change. "Fix the price in paragraph 3" invites a small, checkable change; "try again" invites a rewrite that can break what already passed.

The **learning edge** is the longer loop. When several units fail the same way, the cause is usually the brief, not the workers. The Reflexion paper (Shinn and colleagues, 2023) showed one form of this: an agent turns feedback from a failed attempt into a short written reflection, keeps it in memory, and reads it before the next try. Anthropic's long-running coding agents use a plainer form, a progress file and a git history each new session reads before starting work. In a graph, the lesson goes into the shared brief, so units that have not started yet benefit too. Keep lessons short, and have a person review them before they become permanent instructions, because a wrong lesson spreads to every unit.

## Return one failed unit, not the whole batch

When three units out of thirty fail a gate, send back those three. Rerunning all thirty wastes tokens, and worse, gives twenty-seven units that already passed a chance to come back different and fail. To make this possible, keep each unit's status, output, attempt count and last failure separately, rather than one shared blob of results.

Set a limit on attempts per unit (two or three is common) and route anything that hits it to a person with the full failure history. Add a limit for the whole run as well. LangGraph, for example, stops a run that exceeds its recursion limit, a cap on the number of steps, and raises an error. Anthropic notes that maximum-iteration limits are a common way to keep agents under control.

The control flow fits in a page of plain Python. In this sketch, `split`, `work`, the gates and `lesson_from` are functions you supply; each gate returns a list of failure messages, empty when the unit passes.

```python
from concurrent.futures import ThreadPoolExecutor

MAX_ATTEMPTS = 3

def run_graph(job, split, work, gates, lesson_from):
    units = split(job)                      # splitter: list of {"id", "task"}
    lessons = []                            # learning edge: shared by future units
    results, attempts, evidence = {}, {}, {}
    pending = [u["id"] for u in units]
    by_id = {u["id"]: u for u in units}

    while pending:
        with ThreadPoolExecutor() as pool:  # workers run in parallel
            outs = pool.map(lambda uid: work(by_id[uid], lessons, evidence.get(uid)), pending)
            outputs = dict(zip(pending, outs))
        retry = []
        for uid, out in outputs.items():
            attempts[uid] = attempts.get(uid, 0) + 1
            failed = [msg for gate in gates for msg in gate(by_id[uid], out)]
            if not failed:
                results[uid] = out
            elif attempts[uid] < MAX_ATTEMPTS:
                evidence[uid] = failed      # correction edge: this unit only
                retry.append(uid)
            else:
                results[uid] = {"escalate": failed}
        new = lesson_from([evidence[u] for u in retry])
        if new and new not in lessons:
            lessons.append(new)
        pending = retry
    return results                          # next: one human review of the joined result
```

Only the units in `retry` go around again, each with its own evidence. Units that hit `MAX_ATTEMPTS` come out marked for escalation instead of looping forever.

## A worked example: updating thirty help articles

A software company changes its basic plan from $10 to $12 a month. Its help center has hundreds of articles; some mention the price.

1. **Splitter (code, then model).** A search finds every article containing "$10" or "10 dollars". A model reads each hit and keeps only those about the basic plan's price, writing a one-line brief per article. A code check confirms that every search hit was either kept or rejected with a reason. Result: 30 units.
2. **Workers (model, in parallel).** Each worker gets one article, its brief, and the shared decisions: the new price, the date it applies, the wording for existing customers.
3. **Code gates.** Each rewritten article must no longer contain the old price, must contain "$12/month" where the old price was, must differ from the original only in paragraphs that mentioned the price, and must keep every link working.
4. **Judge gate (model).** A separate model checks one question per article, pass or fail: does the article still say anything that contradicts the new price, such as an annual total worked out from the old one?
5. **Correction edge.** Say four articles fail: three still have an annual total of $120, one changed a paragraph it should not have touched. Each goes back alone, with the gate's message and the paragraph it may change.
6. **Learning edge.** Three failures share a cause, the annual total. The lesson "also update annual totals: $144/year" is added to the brief, after a person glances at it, so any articles still in progress get it.
7. **Join and human gate.** A code step assembles all thirty changes into one review page, with a diff per article, the gate results, and the four that needed correction. A person reviews that page once and approves the publish. Publishing is the only step that reaches customers, so it is the only step a person must approve.

The brief each worker receives can follow a fixed template:

```prompt
You are updating one help-center article.
Unit: {article_id} — {title}
Goal: {one-line brief from the splitter}
Shared decisions (apply exactly): {new price, date, wording}
Lessons from earlier units: {lessons, or "none yet"}
Scope: change only paragraphs that mention the price. Leave every other sentence as it is.
Checks your output will face: old price absent, "$12/month" present, no changes outside price paragraphs, links unchanged.
{If this is a retry: "Your last attempt failed these checks:" followed by the gate output, word for word.}
Return the full article text and nothing else.
```

> **Try it:** Take a job you would hand to several agents and fill in four columns before writing any code: the units, the check each unit must pass, the evidence a failed check returns, and the one step a person must approve. If a column is hard to fill in, the graph is not ready to build.

## When is a single agent loop better than a graph?

Often. Anthropic's advice is to start with the simplest design that works and add structure only when it clearly improves results, and the graph above is more structure than many jobs need. If the work is small, or if every part depends on decisions made in the other parts, one agent working through the task in order, with real checks along the way, is usually cheaper and more consistent. Cognition's argument against parallel subagents is strongest in exactly these cases: when context is split, decisions drift apart.

A graph earns its cost when the units are genuinely independent, when there are enough of them that parallel work saves real time, and when each unit has a check that can fail it on its own. Research across many sources fits that description; many coding tasks do not.

There is also a middle path. A single main agent can hand narrow, read-only questions to helpers (find every file that uses this function, summarize this document) while keeping all the decisions and edits in its own thread. That keeps one consistent line of decisions and still gets help with the work that splits cleanly. The [multi-agent workflow guide](/en/learn/build-multi-agent-workflow/) on this site walks through the building blocks if you decide a graph is the right fit.

## Sources

- Building effective agents: https://www.anthropic.com/engineering/building-effective-agents
- How we built our multi-agent research system: https://www.anthropic.com/engineering/multi-agent-research-system
- Graph API overview (LangGraph docs source): https://github.com/langchain-ai/docs/blob/7aee47ad2fe0f18f297fa8f3d96598dc5d27dfa3/src/oss/langgraph/graph-api.mdx
- Agent orchestration (OpenAI Agents SDK documentation): https://github.com/openai/openai-agents-python/blob/fc6afebc48d5526bdfa0aea9315cb039c61d58b5/docs/multi_agent.md
- Reflexion: Language Agents with Verbal Reinforcement Learning: https://arxiv.org/abs/2303.11366v4
- Don't Build Multi-Agents: https://cognition.ai/blog/dont-build-multi-agents
- Effective harnesses for long-running agents: https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents

---

From the Guide to Artificial Intelligence, by Robert Scales, published by Holistic Hybrid Inc. Share and adapt it under CC BY-NC-SA 4.0 (https://creativecommons.org/licenses/by-nc-sa/4.0/): credit the guide, link to https://guidetoartificialintelligence.com/en/learn/design-graph-of-agents/, no commercial use without permission, and keep the same licence.