CSE Roadmap

Core tier · about 5 hours · 6 parts

Part 4 of 6 · 5 min read

Moving into a framework

And the end of Level 5.

Moving an agent into a framework

When you do move, do it in this order so that nothing is lost on the way:

  1. Freeze the behaviour. The ten tasks, with their expected statuses, are the specification. Add the model id and prompt version to the header so you can compare like with like.
  2. Port the tools first, as they are: same names, same descriptions, same argument checks. The tool layer is the part most likely to move over unchanged.
  3. Put the guards back. Recreate the step, word and repeat limits and the partial answer in the framework's own terms, then test them on too-long and stubborn before anything else.
  4. Turn off or inspect every default from the five questions.
  5. Check the trace against your own: one line per model call and one per tool call, with results, errors and retries.
  6. Run the ten tasks and compare status, answer and tool calls, as same_behaviour() does for the three shapes. Differences are bugs in the port or choices the framework made for you. You now know which.

Tip: watch the dependency, not only the code

A framework is a dependency that changes. Pin its version, keep the ten tasks in your continuous integration, and read its release notes for the five areas above. A change in a default is a change in your agent's behaviour even if you did not touch your code.

Putting it together: mini-agent

mini-agent is a framework-free research agent: an agent with search and file tools, step and cost limits, and a readable trace for every run. It brings the earlier modules of this level together:

Part File Module
tool schema, argument checks, workspace tools.py 2
the loop, limits and partial answer kernel.py 1 and 6
the model stand-in model.py (scripted)
events, header, timeline kernel.py 7
the same agent as SDK loop and graph sdk_loop.py, graph_agent.py this module
the ten tasks and the scorer corpus.py, project.py this module

Three things it should do, and whether it does:

  • It completes ten defined tasks. Yes: 10 of 10, including a write to a file, a two-step lookup, a question with no answer and a flaky tool. With a scripted model, so this only shows that the loop and tools are sound. With a real model the score will be lower; measuring it is Level 9.
  • Every run leaves a trace a stranger could follow. Yes, as a timeline of the kind shown below; the full recorder with replay is module 7.
  • It stops gracefully when a budget runs out. Yes: the step limit, the word budget and the repeat guard each end with a partial answer that says what was gathered and why it stopped.
TASK  In which city is the office of the person who leads the billing team?   [toy-research-1, prompt v1, kernel-1]
  0  I need the billing file.  ->  read_file(path=teams/billing.md)
       'Team: Billing\nLead: Priya Nair\nBudget: 420000\nEngineers: 7\nProject: Ledger\nLaunch: 2026-03-14'
  1  Now find the leader's office.  ->  search_docs(query=Priya Nair office)
       'people/priya_nair.md: Name: Priya Nair | Office: Lisbon | Role: team lead\nteams/billing.md: Team: Bi'
  2  I have what I need.  ->  finish
END   done after 2 steps, 127 words: The office is in Lisbon.

What this agent cannot do yet, and where it goes next: use tools you did not write and cannot trust (Level 6: tools, MCP, least privilege); remember across runs and manage a long context (Level 7); work with other agents (Level 8); be measured on real tasks (Level 9); and be made safe against a hostile input (Level 10).

A checklist

  1. Write the loop once yourself, against the raw tool-calling shape, with limits and a trace.
  2. Keep a regression suite of tasks, including the bad cases, and run it on every change of model, prompt or framework.
  3. Keep the guards in code, outside the model call.
  4. Never let a framework retry a side effect unless the tool is idempotent.
  5. Audit the defaults with the five questions before trusting a framework.
  6. Make sure you can print exactly what the model was sent and what each tool returned.
  7. Adopt a framework for a named reason: durable state, approval, handoffs, tracing.
  8. Pin the version and re-run the suite on upgrade.

Interview angle

"Would you use a framework or build on the raw SDK?" Say: it depends on what the framework would give me, and give the pieces. Describe the loop in a few sentences (send messages and tool schemas, read the stop reason, run the tool myself, send back a tool result, stop on a limit), then say that a framework is the same loop plus state, retries, limits and tracing. Start raw for a small, single-agent system where I want strict control; choose a framework for durable, pausable workflows or many agents. Then show you have been bitten: name the defaults you would check (hard stop, retries on side effects, truncation, hidden prompts, swallowed errors) and say you would keep a regression suite to find out. Weak answers recite a feature list or say "frameworks are bloated" or "frameworks are the standard", with no reasons.

Follow-ups to expect: "How do you stop an agent from looping forever?" (limits in code outside the model: steps, budget, repeat detection, with a partial answer; module 6) and "What are the common failure modes of ReAct?" (loops, wrong tool choice, ignoring errors, premature finish, cost growth; modules 1 and 6).

Check yourself

A framework upgrade changes a default retry count from 0 to 2, and one of your tools sends an email. What goes wrong and how would you have noticed? (The email can be sent twice. Pin versions, keep your task suite in continuous integration, and make side-effect tools idempotent.)

For the curious Small fixes in the combined agent

The limits object accepted repeat_n=1, which would stop every run after one step, and negative limits. It now refuses both. The argument check also rejected a whole number for a number argument (5 for a float) while accepting True for an integer; it now allows the first and refuses the second.

What you will use daily at work

Practice What it looks like
A raw loop first, then a framework You know what the framework decides for you before you adopt it.
A task suite in CI Ten or more tasks, including bad cases, run on every model, prompt or framework change.
Pinned versions Framework and SDK versions are fixed and upgraded on purpose.

Draft: confirm the SDK and framework your team has chosen, and its current release notes.