Part 7 of 8 · 3 min read
Exercises
Try each exercise before opening its hint.
Easy
Easy 1: See the speed-up (20 minutes)
Run python fanout.py in code/async-fanout. Edit the __main__ block to use 50 prompts and a limit of 10. Predict the time before running, then compare.
Done when: your prediction was within 20% (calls / limit × delay).
Easy 2: Break it on purpose (20 minutes)
Replace await asyncio.sleep(delay) in fake_model with time.sleep(delay). Run the fan-out again. Explain why the time is back to the sequential figure even with limit=20.
Done when: you can describe in two sentences what the event loop was doing.
Hint
with blocking sleep, tasks never overlap because each holds the loop for its full duration.
Easy 3: Sequential, concurrent or parallel? (15 minutes)
For each situation say whether you would use plain sequential code, asyncio, or separate processes: (a) 200 independent model calls, (b) resizing 10,000 images, (c) a chat that must wait for one answer before the next question, (d) reading 50 small files from disk. Give one reason each.
Done when: you chose asyncio for (a), processes for (b), and can justify all four.
Medium
Medium 1: Report failures properly (30 minutes)
Write summarise_results(prompts, results) that takes the output of fan_out and returns (successes: dict, failures: dict) keyed by prompt. Add a test using the "bad" prompt trick from fake_model.
Done when: failed prompts appear with their error message and successful ones with their answer.
Hint
isinstance(result, BaseException) also catches CancelledError; decide whether you want that.
Medium 2: Retry inside the limit (40 minutes)
Combine module 3 and 4. Write call_with_retry(prompt) that wraps fake_model in up to 3 attempts with backoff (use await asyncio.sleep, not time.sleep). Make fake_model fail the first two times for one prompt. Use it inside fan_out.
Done when: all prompts succeed and the peak concurrency is still at most the limit.
Hint
put the semaphore outside the retry loop so a retrying task keeps its slot, or inside so it does not. Think about which is better and say why.
Medium 3: A per-call timeout (30 minutes)
Wrap each call in asyncio.timeout(0.5) and make one prompt sleep for two seconds. The slow prompt should end up in the failures with a timeout message while the others succeed. Test it.
Done when: the test passes and runs in about half a second, not two.
Hint
asyncio.timeout raises TimeoutError; use asyncio.gather(..., return_exceptions=True) to collect it.
Hard
Hard 1: A stoppable agent step (60 minutes)
Write run_step(prompt, stop: asyncio.Event) that starts three fake tool calls at once and returns when either all finish or stop is set, whichever comes first. If stopped, every unfinished tool call must be cancelled and must log "cleaned up". Test both paths.
Done when: after a stop, no task from your function is still running (check with asyncio.all_tasks()).
Hint
asyncio.wait({tasks, stop_waiter}, return_when=asyncio.FIRST_COMPLETED) is the tool. Cancel the pending ones and await them to let finally run.
Hard 2: A rate limiter, not just a concurrency limit (60 minutes)
A semaphore limits how many calls run at once, but providers also limit calls per second. Write an async RateLimiter(per_second) using an injected clock and sleep, and use it in the fan-out so that no more than per_second calls start in any one-second window. Test it by recording start times.
Done when: the recorded start times never put more than per_second calls in a one-second window.
Hint
keep a list of recent start times and sleep until the oldest is more than one second old.
Hard 3: First result wins (50 minutes)
Write fastest(prompts_to_providers) that sends the same prompt to three fake providers with different delays and returns the first successful answer, cancelling the others. If all three fail, raise an error that lists each failure. Test the winner, the cancellation and the all-fail case.
Done when: the slow providers are cancelled (they never log "finished") and the all-fail error lists three reasons.
Hint
asyncio.wait(..., return_when=FIRST_COMPLETED) in a loop; a task that failed is not a winner, so keep waiting for the rest.