Parallel work and orchestrators

Parallelisation

Two flavours:

• Sectioning: independent sub-tasks run at the same time (a tone check, a policy check and a length check on one draft).
• Voting: the same task several times, then the most common answer (module 2).

r = parallel(llm, {"tone": ..., "policy": ..., "length": ...})   # r.results, r.errors, r.seconds

In the demo each call sleeps 0.2 seconds, and three in parallel take about 0.2 seconds, not 0.6. A failing worker lands in r.errors and the others still return, which is how you keep a partial answer.

• Use when sub-tasks do not depend on each other, or when you want several attempts.
• Cost: you pay for every call, so parallel saves time, not money. Watch rate limits (Level 2, module 5).
• Fails by hitting limits, or by merging results badly.

Orchestrator-workers

An orchestrator call decides at run time what subtasks are needed. Workers do them, and a final call combines the results.

o = orchestrate(llm, ticket, "PLAN\n{input}", "WORK\n{task}\n{input}", "COMBINE\n{parts}", max_subtasks=5)

The difference from parallel sectioning is that the number of subtasks is not known in advance. The example ticket has three problems (billing, bug, login), so it plans three subtasks and uses 1 + 3 + 1 = 5 calls. A one-problem ticket uses 3 calls. That variable cost is why max_subtasks exists. A test checks that a ticket with twelve complaints is capped at four workers.

• Use when the subtasks depend on the input in ways you cannot list in advance.
• Fails by a bad plan (too many, duplicate or missing subtasks) and an unbounded fan-out. Cap it, and handle an empty plan (a test covers that).