Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-tracing.
Done when: you report the error-trace retention for head 10 percent and the tail rule.
Easy 2: Read a trace (30 minutes)
Print by_name for three runs.
Done when: you say which span type dominates time and cost.
Easy 3: Cost by hand (30 minutes)
A run has three model calls of 1,500, 1,900 and 2,300 input tokens and 200 output tokens each.
Done when: you compute the cost and match cost().
Hint
price constants are in tracing.py.
Medium
Medium 1: Parallel tool calls (3 hours)
Make two tool spans overlap.
Done when: self_time is wrong for the parent and you fix it with a critical-path calculation.
Hint
the critical path of parallel children is the longest one.
Medium 2: A tail rule on judge scores (3 hours)
Add a low-quality flag to some runs and keep flagged runs.
Done when: you report storage and retention of flagged runs.
Hint
tail decisions run after the root span ends.
Medium 3: Cost per user (2 hours)
Add a user id attribute and aggregate cost per user.
Done when: you identify the top 1 percent of users by cost.
Hint
rollup works for any attribute.
Hard
Hard 1: Real OpenTelemetry (8 hours)
Instrument a real agent with the OpenTelemetry SDK. (I did not run this, so your numbers are the ground truth.)
Done when: you report the attribute names used, the overhead and a screenshot or dump of the trace tree.
Hint
check the current attribute names in the specification repository first.
Hard 2: Trace-linked evals (5 hours)
Store a trace id with each eval result. (I did not run this.)
Done when: you can open the trace of any failing case from the CI report.
Hard 3: Redaction (6 hours)
Add a redaction step before export. (I did not run this.)
Done when: you show that test personal data never reaches the exporter, with a test.
Hint
redact in a span processor, before the exporter.