Questions one search cannot answer, and retrieval in a loop.
The multi-hop problem
Multi-hop question
A question whose answer needs several facts joined in sequence, where later facts cannot be found without the earlier ones.
Take "Who manages the team that owns the billing service?". Fact 1 ("The billing service is owned by the Payments team") shares the words billing service owned. Fact 2 ("The Payments team is managed by Priya Nair") shares only team managed, which the question does contain, but so do many other passages. The word "Payments" is the key, and it is not in the question: it is in fact 1. A single search cannot use information it has not retrieved yet.
In the example, single-pass retrieval with three passages answers every one-hop question and none of the two-, three- and four-hop ones. Retrieving more passages helps only a little:
Passages retrieved
1 hop
2 hops
3 hops
4 hops
All 24
3
100%
0%
0%
0%
42%
5
100%
17%
0%
0%
50%
8
100%
50%
0%
0%
58%
More passages mean more tokens, more distractors (module 6 showed the cost), and still no guarantee for deeper chains.
Agentic RAG: retrieval in a loop
Agentic RAG
A RAG system in which an agent controls retrieval: it decides what to search for, evaluates what came back (relevant? complete? current? contradictory?), and chooses the next action (search again, rewrite the query, switch source, use a tool, or stop and answer).
The example's loop:
Plan the hops from the question (service, then team, then manager, then office).
Search for one hop, using the entity found by the previous hop: "Payments team managed by".
Read the passages and ask: did we get this hop? If not, widen the search once, then give up.
Repeat for the next hop, up to a step limit.
search('billing service owned by team', k=3) -> ['Payments']
search('Payments team managed by', k=3) -> ['Priya Nair']
search('Priya Nair based in office', k=3) -> ['Pune']
Each query was written after the previous result, which is the whole trick.
Single pass, an agentic loop, and a pre-built graph
A fuller agentic RAG agent also does more than chain: it validates sources (prefers the 2025 policy to a 2020 blog post), reconciles conflicts (a draft budget against the final report), decomposes comparisons into sub-searches, and uses tools when the knowledge base has a gap (a live web search for yesterday's launch).
What it costs and what can go wrong
single pass
agentic loop
searches per question (example)
1.0
2.1
model calls per question
2.0
3.1
accuracy on 24 questions
42%
100%
Three failure modes to design for:
Error compounds. If each hop succeeds 90% of the time, a four-hop chain succeeds 0.9⁴ = 66% of the time. Checks at each hop (module 5) matter more than before.
Loops and drift. Without a step limit the agent can search forever. The example stops after 8 searches.
Conflicts. One outdated passage that contradicts a current one drops the agentic score from 100% to 92%. Letting the agent prefer the newest passage restores it to 100%, which is the "reconcile conflicts" behaviour. Without a rule for conflicts, the agent returns both answers and an answer set of two is wrong.