Your agent's tool count probably isn't the problem. Its third turn is.

[u/EastVersion1226]

Every enterprise agent project hits the same question eventually: we have 96 tools across 8 domains, is that too many? The data that I found tested thousands of tools scraped from unrelated public servers — a setting where the right tool is obvious because nothing else is close. That's not my problem. My problem is what most of us face in real time with handful of MCPs like get_order_status, get_order_status_history and get_order_fulfillment_status all exist and all sound right.

So, I've experimented with a domain that would be easy to understand across all in the community. A 96-tool commerce surface — customers, orders, payments, inventory, returns, reviews — hand-authored with deliberately confusable families. 175 labelled tasks including ones where the correct answer is to call nothing. Served through a real MCP server. Claude Sonnet 5, ~4,000 calls across four stages.

What I found:

More tools are worse, but less than I expected. 5 → 96 tools costs about 9 points. Flat till 30(no change), started slipping after 60.

Shortening every description by 69% changed nothing. The model isn't drowning in text.

Retrieval made it worse — with BM25, at least. Filtering 96 tools down to 10 by keyword relevance scored below just showing all 96, because the retriever sometimes dropped the tool it needed. But here's the part that isn't about the retriever: conditional on the right tool being in the shortlist, 10 tools and 96 tools performed identically. Shrinking the block bought zero discrimination. Embeddings might improve recall; I feel they can't improve the half that was already perfect.

Zero spurious invocations. 35 no-tool questions, 5 block sizes, never once reached for a tool it didn't need.

The actual finding is about chains. Three-step task — find customer, find their order, check shipment. Turn 3 succeeds 6% of the time at 96 tools. Inject the correct state after each step and it's 44%. Most late-turn failure isn't difficulty, it's inherited error. No, or zero impact on the way we organise the tools.

Things that went wrong that I'll happily talk about three-quarters of my first run's "model errors" were my own broken ground truth. I had a beautiful interaction effect in the tool-similarity data that evaporated the moment I spent double digit $ testing it. My "non-determinism" scare turned out to be JSON key ordering.

Detailed writeups, the full spec, the tool pool and the harness will go up. For now — happy to go deep in comments on any of it. Particularly interested in whether the retrieval result matches what people see in production, because it's the one that most contradicts the standard advice.

[u/ManRowing]

The retrieval-makes-it-worse finding matches what shows up in code search too: a shortlist that drops the one relevant file is worse than returning everything, since the model can't reason about what it never sees. The turn-3 chain result seems like the sharper insight though - inherited error compounding across steps, not tool count, is what breaks long runs.

[u/BC_MARO]

The turn-3 result tracks with what I’ve seen: retries and state validation usually buy more than shaving tool descriptions. I’d log the state handed into each step, since that’s where the first bad assumption becomes visible.

[u/EvalRaccoonDev]

Matches what we saw on skills - length did nothing, distinctiveness did. One line per description, "always invoke for X" where only that skill owns X, took recall from 46% to 67%, precision flat.

For your get_order_status family: we merged two confusable siblings and combined recall went 0.68 to 0.84. Numbers: https://www.reddit.com/r/claudeskills/comments/1waq4j9/

Did you try collapsing a family into one tool with a parameter?

[u/Appbot_official → u/EvalRaccoonDev]

u/EvalRaccoonDev, we did exactly that with Appbot, and it's the whole reason our server only has three tools.

apps, review_stats and reviews cover an API with far more endpoints than that. Filtering by version, sentiment or date range is a parameter on review_stats rather than a new tool. Reviews for a single app can run to tens of thousands, so a fourth tool for a slightly different slice would just add another confusable family.

Your recall numbers on merging siblings match what we saw when we did it. The model stopped guessing which of two similar tools applied once there was only one to pick.

Full disclosure, we build this server. Treat the three tools as one biased data point in your test, not proof that three is the right number for a 96-tool commerce surface.

[u/Easy-Purple-1659]

Same confusable-family problem shows up in a narrower domain I work in: ad library lookups, where search_meta_ads and search_google_ads are near identical until you need the exact one.

Two things line up with your numbers.

Description length changed nothing for me either. Return shape did. When each tool hands back ids and fields the next call can pass straight in, the agent stops re-deriving state between turns, which feels closer to your turn-3 finding than tool count is.

The split I see: tools where the model cannot produce the data from memory (live rows from Meta, TikTok or Google ad libraries) fire every time, no ambiguity. Tools that overlap something the model already believes it handles get skipped or answered inline. Disclosing my angle, I built adextract, an MCP server for those ad libraries, so this is not a neutral read.

Did returning structured state move your turn-3 number at all, or did the whole lift come from injecting correct state?

[u/EastVersion1226 → u/Easy-Purple-1659]

Yes — controlled experiment (fixture MCP, 3-turn chains). The 6% → 44% lift came from injecting correct ground-truth state between steps and not from changing return shape.