What the model sees: Tool design

What the model sees

refund_order(order_id: id, amount: number, reason: enum)
  - Refund part or all of one order. Use for refund, money back, arrived damaged
    or charged twice. Moves money, so confirm with the user first.
    The amount may not exceed the order total.

That is the entire interface. Everything the model will ever know about this tool is in those lines, plus what comes back after a call. So a tool has four parts to design:

Name: short, specific, verb first (get_order, not order or tool_7).
Description: what it does, when to use it, when not to, and any side effect, in the words users actually use.
Parameters: types, enums for closed sets, required or optional, an example for anything with a format.
Result and errors: short text that says what happened and, on failure, what to do next.

Descriptions decide the choice

The example gives a word-matching "model" the same 12 tools described four ways, and 24 requests such as "Refund 20 dollars on ORD-1042 because it arrived damaged":

Choosing the right tool out of 12, by how the tools are described

Descriptions	The 24 requests	12 held-out requests	Nearly tied
`tool_a`, `tool_b`, no text	8%	8%	100%
clear names, no text	58%	17%	50%
names plus one sentence	83%	42%	33%
what, when to use it, when not to	96%	58%	4%

Three lessons, one of which is a warning about my own experiment:

Words are the interface. With cryptic names every choice is a tie. Clear names get you most of the way on easy requests. A sentence on when to use it and when not to (the "Not for searching and not for delivery questions" in get_order) pushes it further.
Write in the user's words. The good descriptions include phrases people say ("money back", "arrived damaged", "where is my parcel"). A tool described only in your internal vocabulary is invisible to someone using theirs.
Test on requests you did not write the descriptions for. I wrote the "good" descriptions while looking at the 24 requests, then wrote 12 new ones. Accuracy dropped from 96% to 58%: the descriptions had partly fitted the test. A real description review needs a held-out list of real user phrasings, ideally from logs.

Overlapping tools

If two tools say the same thing, the model has no reason to prefer either, and which it picks can change between runs. In the example, adding tools with a different name and identical description lowers the expected accuracy of a model that must guess among the tied tools:

Duplicate tools added	Expected accuracy
0	94%
2	85%
4	76%
6	70%

Fixes: merge them; or make the difference explicit in each description ("get_product: details and size. Not for stock levels." against "check_stock: units left"); or split by argument instead of by tool.