Optimising for AI agents

An agent never sees your product. It sees a list of tool definitions, and whatever your server returns when it calls one. Everything a human gets for free — a label next to a field, a tooltip, an example, the ability to try again and look at what happened — has to be carried by that contract or it does not exist.

That makes the unit of work the tool, not the page and not the feature. A single badly-typed parameter on one tool can stall every task that touches it, while the rest of the server looks perfectly healthy.

In practice it is a loop with four steps. Observe: capture every agent session, tool call, error and latency, because none of it shows up in web or product analytics. Interpret: read behaviour across many sessions rather than alerting per request — the pattern is what tells you something is wrong, not the individual failure. Recommend: name a specific change to a specific tool, with the evidence behind it. Measure: compare behaviour before and after to confirm it moved the number you cared about.

The changes themselves are usually small and unglamorous: tighten a free-text parameter into an enum, disambiguate two tools whose descriptions overlap, document a required field the description never mentioned, cut a response that was blowing up the agent's context window, or rewrite an error so it says how to fix the call rather than only what was wrong.

They act on different artifacts. AEO is content work: prose an answer engine can extract, schema.org markup, an llms.txt, crawlable server-rendered pages. Product-layer work is the tool definitions and JSON schemas an agent loads before it does anything, and the responses it gets back.

They are measured differently too. AEO's outcome is share of voice in generated answers. The product layer's outcome is task completion — and the gap between them is where the money leaks. Being recommended by an assistant is worthless if the agent acting on that recommendation gets a schema error on its first call and quietly moves on to a competitor.

Generative engine optimisation (GEO) is another name for the content side. Neither AEO nor GEO covers the tool layer, which is why it needs naming separately.

Most published writing on AXO so far describes the content layer, because it grew out of SEO. The definition generalises cleanly, though: swap "website" for "product" and every principle still holds. Readable structure, unambiguous statements, reliable availability and machine-legible meaning matter just as much in a JSON schema as in a web page — arguably more, since an agent calling a tool has no ability to skim, guess or ask.

The two layers are won by different teams and measured differently. The content layer is marketing work, measured in citations and share of voice. The product layer is engineering work, measured in task completion. A company can be excellent at one and hopeless at the other, and most are — which is why it is worth naming which layer you are talking about.

Vesta works on the product layer. See what agent experience is and how the two layers differ in practice.

What differs between them is distribution, not protocol: where the server is listed, how it is reviewed, and how authentication is brokered on the way in. The tool definitions an agent reads, and the calls it makes, are the same underneath.

Two things worth knowing. There is no accepted collective noun for these surfaces, so it is clearer to name the three than to invent an umbrella term. And "ChatGPT Plugins" is a different, retired thing — the 2023 surface, since replaced — so a guide written for plugins does not describe how any of this works today.

This has a consequence most teams underestimate: your tool description is a production API surface. It is loaded on every session, it decides routing, and changing its wording changes behaviour as surely as changing the code behind it. A study from Queen's University in 2026 covering 856 tools across 103 MCP servers found 97% of tool descriptions carried at least one quality issue, and 56% did not state their purpose clearly.

Ambiguity is the expensive failure. When two tools have overlapping descriptions, the agent does not error — it picks one, and it may well pick the wrong one. Worse, when a call fails, agents frequently divert to an adjacent tool rather than retrying correctly, so a bad contract on one tool shows up as unexplained traffic on a different one.

The patterns that recur:

• Type and format mismatches. The description says priority; the schema wants an integer 1–4. Agents send "High" and the call errors.
• Undocumented requirements. A field is required by validation but absent from the description, so the agent cannot know to send it.
• Ambiguous tool pairs. Two tools that overlap in wording, so the agent routes to the wrong one and never sees an error at all.
• Oversized responses. A tool returns an unbounded list that consumes the agent's context window, and the task dies several turns later for reasons that look unrelated.
• Uninstructive errors. A 400 that states the violation but not the correction, which turns one recoverable mistake into an abandoned task.

None of this is visible in uptime monitoring. These are healthy responses and valid protocol errors — the server is behaving exactly as written. What you see instead is agents starting tasks and not finishing them.

The common mistake is a one-to-one mapping from an existing CRUD API. It produces a long list of tools that each do a fraction of a job, which forces the agent to chain several calls correctly to accomplish anything — and every link in that chain is a place to fail. Consolidating those into a single task-shaped tool usually improves completion and cuts tokens at the same time.

A useful test: can you name the tool after something a user would actually ask for? create_and_assign_issue passes. patch_issue_field does not.

Concretely:

• Disambiguate against siblings. If two tools could plausibly serve the same request, each description should say which one to prefer and why.
• Name formats, not concepts. "An integer 1–4, where 1 is urgent" beats "the priority". Put the constraint in the JSON schema as well as the prose — the schema is enforced, the prose is only read.
• Say what is required. Anything validation rejects must be documented, or agents will keep omitting it.
• Make errors instructive. The error string is part of the contract. It should name the correction, not just the violation.
• Bound your responses. State the shape and the limits, and paginate rather than returning everything.

Treat descriptions as shipped surface area: version them, and change them on evidence rather than intuition. The 2026 Queen's University study of 856 tools across 103 MCP servers found 97% carried at least one quality issue — this is the single least-maintained part of most servers.

• Task completion — of the jobs agents start, how many finish. The headline number.
• Task types supported — how much of what agents try to do with you actually works. Rising completion on a narrowing set of tasks is not progress.
• Turns and latency — how much back-and-forth it takes to reach the result.
• Tokens per task — what it costs an agent to use you, including the context your tool definitions and responses consume.

Read them per tool, not per server. Users fix tools, so a server-level average mostly hides the one contract that is doing the damage. First-try success — the share of calls to a tool that succeed without a retry — is the best leading indicator, because it moves before completion does.

Be careful with call volume specifically. It looks like engagement and often is not: a tool with unusually high traffic may be called repeatedly because it keeps failing and the agent keeps trying.

What breaks, specifically: there are no page views or clickstream to model a funnel from; session boundaries are set by the agent's client rather than by you; identity is indirect, because the caller is an assistant acting for an end user; and an MCP server running over stdio never touches your web stack at all, so nothing reaches your tag manager in the first place.

The replacements are structural rather than cosmetic. The event is the tool call, not the page view. The session is the task. The conversion is task completion. And the funnel is the sequence of tool calls an agent made on its way to finishing — or the point at which it gave up and switched to something else.

They compose rather than compete. Observability is the substrate, usually OpenTelemetry, and it is the right tool for debugging a specific incident: one trace, one broken call, one deploy. Analytics sits on top of the accumulated traffic and answers a different question — not "what went wrong at 14:03" but "which contract is costing us the most completed tasks, and what should we change about it".

The practical test is what happens after the alert. An observability tool tells you save_issue errors 41% of the time. You still have to work out that agents are sending priority as free text, that the schema wants an integer, that the affected agents divert to update_issue rather than retrying, and that the fix is an enum plus a rewritten description. That gap is the analytics job.

Two things make that comparison trustworthy. Version the tool contract, so "before" and "after" are defined by which version of the description and schema an agent actually saw, rather than by a timestamp. And hold the population steady, because agent traffic is not stationary: a new model release from a major vendor can change how agents use your server more than your own change did, which is why you measure against a baseline rather than reading a raw time series and taking credit for the slope.

The failure mode to avoid is shipping a batch of contract changes at once. If four tools change in the same week you can see that completion rose, but not which change earned it — and next quarter you will not know what to do again.

The ranking signals on these marketplaces are usage-driven, and the review and rating machinery that eventually governs app stores barely exists yet. That makes this an unusually open window: the thing that compounds is real usage, and real usage is downstream of task completion. A server agents finish tasks with gets used more, ranks higher, and gets used more again.

It is worth separating this from being recommended. An assistant suggesting your product in prose is answer engine optimisation and is won with content. Being kept in an agent's rotation is won with the contract.

We wrote this up in more detail in The MCP marketplace is the new App Store.

That difference is why a mechanical one-to-one mapping from endpoints to tools tends to disappoint. It produces a long list of fragment-sized operations, each needing to be chained correctly, at a token cost paid on every request. Tools that are shaped like whole tasks — the thing a user would actually ask for — consistently do better than tools shaped like your resource model.

So: not a separate product, but a genuinely separate interface layer on top of the same system, designed against a different reader and measured on task completion rather than endpoint uptime.

The practical difference is what lands on your desk. A dashboard gives you a chart and leaves the diagnosis, the fix and the verification to you — which is fine if you have someone whose job is to sit with agent traces every week, and useless if you don't. Vesta gives you a ranked list of changes, each with the evidence that ranked it: the tool, the failure mode, how many sessions it affected, what agents did instead when it failed, and the tokens it wasted.

It also closes the loop. Because tool contracts are versioned, the before-and-after comparison is well defined, so you find out whether a change actually moved completion rather than assuming it did.

Vesta covers Python (the official mcp SDK and FastMCP) and TypeScript, over stdio, streamable HTTP and SSE — which means it covers your server whether it is listed as a Claude Connector or a ChatGPT App. It is one instrument() call, it runs in-process, it exports asynchronously over OpenTelemetry, and it is fail-open. Early access is invite-only: request access.

Be careful searching for this. "MCP analytics" more often returns MCP servers built for analytics products — the official Amplitude, PostHog, Mixpanel and GA4 servers that let an agent query your product data. That is the opposite artifact: those give agents access to your analytics, rather than giving you analytics about your agents.

The question that separates the three: after the tool tells you something is wrong, how much work is left? If the answer is "a week of reading traces to work out why", you bought observability. Vesta is built so the answer is "review the recommendation and ship the change".

The same instrument() call covers your server whether it is listed as a Claude Connector or a ChatGPT App, because all three are the same artifact underneath. See the install guide for configuration.