A PayPal-affiliated preprint describes SCOUT, a system for finding and invoking tools in a large catalog. In its main retrieval test, the right tool appeared first in 38 of 45 evaluable queries, a Hit@1 score of 84.8%, and appeared in the top five in 43 of 45, or 95.6%. The report gives a mean reciprocal rank, or MRR, of 0.821; this score rewards results that appear near the top. The full benchmark contained 49 queries, but four were catalog gaps and were not evaluable.
An important comparison concerned the amount of tool-schema text placed in the model's context. Tool schemas are structured descriptions of what tools can do. The paper compares 140,200 tool-schema tokens before SCOUT with 1,300 tokens in production, and describes that difference as a 99% reduction. Against a 200,000-token context window, the figures were 70.1% and 0.8%, respectively. The comparison concerns context load; it does not report whether the change improved end-to-end task completion or cut inference costs.
A gateway for a crowded catalog
SCOUT exposes two functions to a client. A natural-language request goes to tool_search, which retrieves relevant tool schemas; execute_tool then routes the invocation to the appropriate backend server. The client therefore receives selected descriptions through the search step, rather than needing to carry every schema at once.
Behind the interface, SCOUT combines dense search, which looks for semantic similarity, with BM25, which matches keywords. Each branch retrieves three times the requested top-k number of candidates, and Reciprocal Rank Fusion combines the rankings with k=60. Server-scope authorization is applied inside Milvus at query time rather than as a filter after retrieval.
The index is status-driven and contains only tools from ACTIVE servers; a change in server state triggers the corresponding index action. Ingestion uses text-embedding-3-large to produce 3,072-dimensional vectors. During updates, insert-before-delete upserts are used to preserve zero search downtime. Production and development connectors can be registered together, with natural-language intent routing discovery and execution to the corresponding environment.
The benchmark had a narrow reach
SCOUT's main benchmark covered nine domains and a live catalog of 2,000 indexed tools. Four of its 49 queries were catalog gaps, leaving 45 evaluable cases. Hit@1 asks whether the right result is first; Hit@5 asks whether it appears among the first five. On those cases, SCOUT scored 84.8% on Hit@1, 95.6% on Hit@5 and 0.821 on MRR. No confidence intervals were reported.
A separate 35-query expansion set produced Hit@1 of 75.6% and Hit@5 of 93.9% among evaluable cases. The report does not include a controlled head-to-head test against full catalog injection, BM25-only retrieval or dense-only retrieval. Its figures therefore describe the reported SCOUT implementation and query sets, rather than establishing superiority over those alternatives.
Production numbers come from short windows
In 1,921 production retrieval requests observed over 24 hours, server-side processing averaged 572 milliseconds. The median was 440 ms, and P95 - the point below which 95% of observations fell - was 936 ms. These are descriptive figures for that window; no uncertainty interval was reported.
During a reported 24-hour operational window, logs recorded 31,701 MCP sessions, 55,314 total MCP tool calls and 1,926 tool_search invocations. The fallback rate was 0.0% in that window. The paper also reports validation across six MCP clients spanning three major providers without client-side modifications. The zero-fallback figure applies only to the reported window.
The open questions are practical
The report flags several practical constraints. Tool-description quality varies, top-k is manually tuned, and SCOUT has no feedback-loop learning from successful invocations. Schema changes can take up to 15 minutes to propagate.
Taken together, the evidence supports feasibility for this PayPal implementation, but it does not settle whether the approach is better than competing discovery designs. The report does not show that retrieved tools execute successfully, improve end-to-end task completion, lower inference costs or generalize beyond the PayPal catalog and query sets. Longer testing across catalogs, traffic levels, clients and tool-description quality, along with direct comparisons, would be needed to answer those questions.
Paper data and sources
Original title: Hybrid Semantic Tool Discovery for Enterprise MCP Gateway: Architecture and Implementation
Authors: Olympia Saha, Amy Wang, Srinivasan Manoharan
Journal/Repository: arXiv
Status: Preprint, not yet peer-reviewed
First online: 2026-08-25
DOI: Not available
Original paper · Full text