Part 4 of 6 · 6 min read
Interview questions and limits
Covers: Interview questions, with the follow-ups; Common mistakes; What the example does not do.
Interview questions, with the follow-ups
What is the difference between an orchestrator-workers design and a handoff design?
"In orchestrator-workers a central agent decomposes the task, dispatches parts to workers and combines the results, so there is one coordinator that sees everything and is a single point of failure. In a handoff design, control moves peer to peer: the agent that has the task decides who gets it next and passes the context along, so there is no central bottleneck but control can bounce, which is why I set a hop limit. On my test task, handoff cost fewer calls (2.3 against 3.2) and recovered misroutes, while orchestrator-workers handled tickets needing two domains more reliably." Follow-up: "Which would you pick?" Orchestrator-workers when the task must be decomposed at run time; handoff when the specialists can tell whether a request is theirs.
When is a router enough?
"When each request has one domain and the classifier is accurate. Routing is a conditional workflow: cheap, predictable and easy to test. It fails silently on requests that need two domains, because it hands the request to one specialist. On my task it beat the single agent with only single-domain tickets (0.76 against 0.70) and scored zero on two-domain ones." Follow-up: "How would you know?" I would log the share of requests that needed more than one specialist and watch it before choosing the shape.
When does a critic agent help, and when does it hurt?
"It helps when it catches errors the worker would not, at a cost of calls and latency. It hurts when it rejects correct work, because the revision is a fresh attempt that can be wrong. In my simulation, a critic that detected half the errors and rejected one in ten correct answers raised accuracy from 0.71 to 0.77 for about 1.7 more calls; a critic that detected nothing and rejected 30 percent lowered it to 0.65. I would measure the critic's detection and false-rejection rates on labelled examples before shipping it." Follow-up: "Should the critic be a different model?" Often yes, since a model reviewing its own output shares its blind spots; I would test that rather than assume it.
How do you prevent infinite loops between agents?
"A hop limit and a message limit on every conversation, a rule that an agent cannot hand back to the one that just sent it without new information, and an explicit failure path (escalate to a human or return a partial answer) when the limit is reached. The limit has a cost: in my test a limit of two ended some legitimate two-step handoffs, so I would set it from the longest valid path." Follow-up: "How do you find loops?" Traces: count visits per agent per task and alert on repeated visits.
Common mistakes
- Choosing the shape from a framework's name rather than from the task's structure.
- Using a router for tasks that need several domains.
- No hop or round limit on handoffs, critics and group chats.
- A critic with no measured detection rate.
- Letting the orchestrator be a weak model, since it caps the whole system.
- Counting only the number of agents, not the calls on the critical path.
- Giving every agent the full history on every handoff, which grows the context at each hop.
What the example does not do
- No models. Accuracies are assumptions; I measure how shapes transform them.
- No shared state, no tools, no failures of tools or networks. Those come in the next modules.
- Independent critic draws. A real critic reviewing the same model's output has correlated errors, which makes it less useful than this model suggests.
- No group chat. Conversation-driven shapes need a model of turn selection, which this module does not build.
- Optional real-model experiment (I did not run this): label 100 real tickets by domain count, run the single agent, a router, an orchestrator and a handoff design on them, and record accuracy, calls and tokens.
Check yourself
A routing agent sends every ticket to one specialist. A ticket needs both billing and legal. What happens? (It cannot be completed correctly: a route picks one specialist. With every ticket needing two domains, routing scored 0.00 against 0.49 for a single agent.)
For the curious A critic is only as good as its detection
A critic that never rejects adds calls and changes nothing (accuracy stays 0.71). A critic that rejects 30 percent of correct answers and catches none of the wrong ones made accuracy fall to 0.65. Measure how often the critic catches real errors and how often it rejects good work before you pay for it.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| A hop limit on handoffs | Control can bounce between agents; a cap guarantees it stops. |
| Typed results from workers | The orchestrator context grows with every brief and result. |
| Pick by task mix | One domain per request suits a route; mixed requests need decomposition. |
Draft: confirm which orchestration features your framework provides and how it limits loops.
What you must be able to say (one page)
- An orchestration shape is the rule that decides which agent acts next. Shapes run from explicit control (sequential, conditional and parallel workflows, where the developer fixes the path) to autonomous (plan-based orchestration, handoff and group chat, where agents decide at run time). Control and autonomy trade off.
- Five shapes, and the one failure each is known for: 1. Single agent: no coordination, but one prompt must cover everything. 2. Route (a conditional workflow): a classifier sends the task to one specialist. Cheap and predictable; cannot complete a task that needs two domains. 3. Orchestrator-workers (plan-based): the orchestrator decomposes at run time and workers execute in parallel. The orchestrator is a single point of failure and its context accumulates every brief and result. 4. Handoff: the current agent passes control and context to a peer, without a central coordinator. Control can bounce, so it needs a hop limit. 5. Critic loop: a second agent reviews each result and can reject it. It costs calls and latency, and a poor critic makes results worse.
- Shapes compose: any node can itself be a multi-agent system behind one interface, so a workflow can sit inside an orchestrator, or a critic inside a worker.
- Quote these numbers, with the caveat that agents are probabilistic stand-ins with assumed accuracies (customer tickets needing one or two of three domains, 30 percent two-domain, 20,000 tickets per shape):
| Shape | Accuracy | Calls per ticket | Tokens per ticket |
|---|---|---|---|
| single agent | 0.63 | 1.0 | 1,300 |
| route | 0.53 | 2.0 | 2,270 |
| orchestrator-workers | 0.71 | 3.2 | 3,459 |
| handoff | 0.74 | 2.3 | 2,405 |
| orchestrator-workers plus critic | 0.80 | 5.0 | 5,671 |
- Route beats the single agent when every ticket has one domain (0.76 against 0.70, at the lowest added cost) and is the worst when every ticket has two (0.00 against 0.49). The right shape depends on the task mix, not on the framework.
- A critic is only worth its cost if it detects errors: with a critic that never rejects, accuracy is unchanged (0.71) and cost rises by 1.2 calls; with a critic that rejects 30 percent of correct answers and detects none of the wrong ones, accuracy falls to 0.65.
- A hop limit is a guarantee, not a frequent save: in this model loops were rare (0.6 percent at a deliberately high confusion rate), and a limit of two cut off legitimate handoffs and cost about 2.5 points of accuracy.