Given deliberately out-of-scope tasks, a model correctly declined 100% of the time against a small, well-documented tool catalogue. Against a large, fuzzy one: 50%. Half the time it found a plausible-looking tool and called it. The figures come from Teng Li's scan of 36 popular MCP servers, published in July, using his own evaluation harness.
That failure mode is not the one most teams are monitoring for. The agent did not error. It did not time out. It did not retry. It picked something that looked right and used it, and your server returned 200.
The failure that looks like usage
Every dashboard in this category is built around errors. Success and failure rates, error counts, latency percentiles, uptime. That instrumentation is blind to misrouting by construction: a wrong tool called correctly is a successful call.
Worse, it looks like health. A tool getting unexpected traffic reads as adoption. It goes in the deck. If anything you conclude that tool is your most valuable one and invest in it, when what is actually happening is that agents keep landing there because your catalogue is ambiguous and it is the nearest plausible target.
LiveMCPBench points the same way from a different angle. Across 95 tasks on 70 servers and 527 tools, the authors found retrieval errors — the agent failing to match a task to the semantically right tool — dominating the error distribution. Not reasoning failures. Not execution failures. The agent choosing wrong from a catalogue it could not disambiguate.
The best model on that benchmark completed 78.95% of tasks. Most models managed 30–50%. A large share of that missing half is agents doing something confidently incorrect.
Why catalogues go fuzzy
Nobody sets out to build an ambiguous tool catalogue. It happens through two ordinary decisions.
The first is mapping tools one-to-one onto an existing REST API. Endpoints were designed for a developer who reads the docs once; the resulting tools are fragments of jobs with names that only make sense if you already know the resource model. patch_issue_field and update_issue and save_issue are perfectly clear to the team that built them and near-indistinguishable to a model that has never seen your product.
The second is growth without curation. Every tool added is loaded into the agent's context on every request, and each one is another candidate for the router to confuse. Catalogues get bigger monotonically because removing a tool feels like a breaking change, and nothing in the tooling tells you a tool is doing harm.
Then there are descriptions. Teng Li's finding was blunt: almost every server he graded D or F had a descriptions score of zero while other metrics looked fine. One popular server produced 132 of its 134 errors from undocumented parameters alone. Separately, a 2026 Queen's University study of 856 tools across 103 MCP servers found 97% carried at least one quality issue and 56% did not state their purpose clearly.
The description is not documentation. It is the routing logic. It is the only thing standing between a request and the wrong tool.
The tells
Misrouting is invisible in error rates but it is not invisible. Three signals in ordinary session traffic:
Diversion after failure. An agent errors on one tool and its next call is to a different, adjacent tool. That sequence is the agent judging your contract unusable and routing around it. It is the single clearest evidence that two tools overlap in the model's reading — and it points at which two.
Volume without outcomes. A tool with high call counts that rarely appears in sessions reaching a terminal action. Traffic that does not convert into finished work is not adoption; it is a wrong turn that lots of agents are taking.
Argument thrash. Repeated calls to one tool with mutated arguments. The agent is guessing at a contract you did not document. Sometimes it guesses right, which is the worst case, because the task completes and the defect stays hidden.
All three are sequence properties. They only exist if something is recording sessions — the ordered set of calls an agent made pursuing one job — rather than individual requests. Per-request instrumentation cannot see any of them, which is a large part of why this failure mode has stayed unexamined.
What to do about it
Fix the pairs, not the catalogue. Find the two tools agents actually confuse — diversion tells you which — and make each description say when to prefer it over its neighbour. Explicitly. By name.
Then cut. If a tool is a fragment of a job rather than a job, fold it into the tool that completes the job. Fewer, task-shaped tools reduce both the routing surface and the token cost of every request, and the evidence says small well-documented catalogues route almost perfectly.
And document every parameter that validation enforces. An undocumented required field is not an edge case; on the servers that were measured, it was the single largest source of errors.
The uncomfortable part is that none of this is discoverable from the metrics most teams have. You cannot fix a misrouting problem you are currently reading as growth.
That gap is what Vesta exists to close: read the sessions, find the pairs agents confuse and the tools that take traffic without producing outcomes, name the change, and measure whether it worked. Request access.
Sources
- I lint-scanned 36 popular MCP servers. A third of them are failing your agent. — Teng Li, July 2026 (100% vs 50% correct refusal on small well-documented vs large fuzzy catalogues; 100% vs 84% tool-selection accuracy; a third of servers graded D/F; description scores of zero; 132 of 134 errors from undocumented parameters)
- LiveMCPBench: Can Agents Navigate an Ocean of MCP Tools? — Mo Guozhao et al. (95 tasks, 70 servers, 527 tools; retrieve errors dominating the error distribution; Claude Sonnet 4 at 78.95%, most models 30–50%)
- Queen's University, study of MCP tool descriptions, 2026 (856 tools across 103 servers; 97% with at least one quality issue, 56% not stating purpose clearly)
- Writing effective tools for AI agents — Anthropic
If you build a surface that agents use, and you want to know how they actually use it, Vesta is in early access.