# Cutting MCP Round-Trips with Jev (Part 3) ## The ping-pong problem MCP pays every day MarketDX lets AI assistants (like Claude) pull market data through **MCP** — a set of "tools" the LLM can call itself: find stocks, pull financials, run screens, and so on. Sounds simple, but one step is quietly expensive: **turning a human phrase into the values the system actually understands.** Take a real question: > *"Retail stocks whose current assets are accelerating sharply"* Before it can screen anything, the system has to convert three pieces into tokens the database knows: - "current assets" → `CurrentAssets` - "accelerating sharply" → `accelerating` - "retail" → `2550` (a GICS code) We do this with a family of `find_*` tools (`find_fsconcept`, `find_fspattern`, `find_gics`, …). The trouble is how they work: 1. The LLM calls `find_fsconcept("current assets")` → the system returns a **list of candidates (top-k)**, maybe 10 of them 2. The LLM **reads them and picks** the right one 3. Repeat with `find_fspattern` → top-k → pick 4. Repeat with `find_gics` → top-k → pick 5. **Only then** call the real screening tool with the chosen values Count the round-trips: **four back-and-forths** before the real work even starts. Each one has network latency, and — worse — **the LLM has to burn reasoning tokens reading the top-k and deciding every single time.** That's the "ping-pong" MCP hits on nearly every query. The more complex the question (several concepts), the more rounds, and the longer the user waits. ### How `find_*` works inside (vector search 101) First, how does `find_*` turn "current assets" into `CurrentAssets`? This is the heart of the story. The key technique is a **vector embedding** — it turns the *meaning* of text into a long list of numbers (a vector with thousands of dimensions). Text with similar meaning ends up with vectors that sit close together in that space. We do two things: 1. **Prepare once, ahead of time:** every concept in the catalog ("Total Revenue", "Current Assets", "Gross Margin", …) is turned into a vector by an embedding model (we use Gemini's) and stored. 2. **At query time:** take the user's phrase ("current assets"), embed it the same way, and find its **nearest neighbours** (by cosine similarity) — closer vector = closer meaning. Sort by closeness, take the top k = the **top-k**. (There's also a first pass that matches exact words/aliases — *lexical match* — before the semantic step, but the idea is the same: "return the ~10 most similar candidates.") **Here's the weakness:** "close in vector space" doesn't always mean "correct." Near-meaning terms — "return on equity" vs "dividend per share", "allowance for doubtful accounts" vs "provision for doubtful accounts" — have vectors so close they get mixed together in the top-k, and sometimes the *wrong* one even ranks higher than the right one. That's exactly why the system *doesn't dare* commit to a single answer. ### Why return top-k at all? Good question — why doesn't `find_*` just return one answer and be done? Because **rank 1 isn't always right.** We measured it on `find_fspattern` (which matches financial-statement "patterns"): blindly trusting the vector's #1 is correct only **71%** of the time — but the right answer is somewhere in the **top-10 100%** of the time. ``` find_fspattern (51 patterns): trust rank 1 blindly → 71% correct answer is in top-10 → 100% ``` See that 71% → 100% gap? **That's why top-k exists** — the system isn't confident enough to commit, so it hands the choices back to the LLM. And that handoff *is* the ping-pong. --- ## The idea: move "the choosing" to the server If the problem is "the system won't choose, so it punts to the LLM" — what if we had something that **dares to choose**, on the server side? That's where Jev comes in. From Parts 1–2 we know it's a fast, cheap "decision model": give it options and a question, and it returns **one answer plus a confidence score** — no long text to generate. So the new shape is: ``` Before: user → LLM picks route → find_* returns top-k → LLM picks → call real tool (ping-pong) After: user → LLM sends raw phrase into the tool → tool lets Jev choose → carry on (one round) ``` The LLM sends "current assets" straight into the screening tool; the tool calls Jev internally to turn it into `CurrentAssets`, then screens — **cutting out the whole "return top-k, let the LLM pick" detour.** The question is whether Jev chooses accurately enough to do this. We tried two scenarios. --- ## Small catalog: let Jev pick from the whole set at once `find_fspattern` has 51 patterns total — small enough to hand the entire set to Jev as options and let it choose in one shot. **Results:** ``` find_fspattern (51 patterns, Thai & English queries): Jev picks once → 92.9% correct vector rank 1 (old way) → 71.4% correct ``` Jev leaves the old rank-1 in the dust and gets close to the 100% ceiling that normally requires the LLM to re-pick — **but in a single shot, no ping-pong.** ### Surprise: most of the "misses" were our fault At first we had a case that looked like a Jev miss: the query *"the stock's value doubled"* — Jev answered "higher than the rest" instead of "compound growth." We nearly blamed Jev — until we looked at **what we'd fed it.** We were only sending each pattern's *short definition*, and the compound-growth definition talked only about "annual rate (CAGR)" — the word "doubled" never appeared. Yet the real database had another description field spelling out *"the value doubles or triples."* **The fix:** feed the full description that the search itself actually uses — and Jev got it right immediately. > This echoes what we learned in Part 2: **before blaming the model, look at what you fed it.** Almost every "miss" turned out to be context we didn't fully supply, or a reference answer of ours that was too narrow. --- ## The main case: too many candidates to fit — vector + Jev This is the common, important one: **most catalogs are too big to hand to Jev whole.** The financial-concept finder (`find_fsconcept`) has **428 entries** (statement lines + ratios + valuation multiples); the stock finder has tens of thousands; the economic-data finder tens of thousands more. Dumping all 428 into Jev at once won't work (expensive and over the limit). But we don't have to throw away what we have — **let the existing vector search pre-filter first:** ``` raw phrase → [vector search narrows to top-15] → [Jev picks 1 of 15] → token ``` Vector search does what it's good at (narrow thousands to ten); Jev does what it's good at (pick the right one out of ten) — **and the choosing still happens on the server, no ping-pong back to the LLM.** **Results:** ``` find_fsconcept (428 entries, English queries): vector rank 1 (old way) → 68% correct vector top-15 → Jev picks → 100% correct ``` Jev fixed **every** case the vector mis-ranked — genuinely hard ones, like: - "ผลตอบแทนผู้ถือหุ้น" or "return on equity (ROE)" in English — vector guessed "dividend per share" → Jev picked `roe` - "หนี้ที่มีภาระดอกเบี้ยรวม" or "total interest-bearing debt" in English — vector guessed "accrued interest" → Jev picked `total_debt` - "งานระหว่างก่อสร้าง", "เงินลงทุนที่ถือจนครบกำหนด", "กำไรขาดทุนเบ็ดเสร็จอื่นสะสม" or "construction in progress", "held-to-maturity investments", "accumulated other comprehensive income" — deep accounting terms the vector drifted on, but Jev picked correctly Even more impressive, it got the *type* right: *"gross profit"* → the statement line (`gross_profit`), while *"gross margin"* → the ratio (`gross_margin`). Different things — Jev doesn't confuse them. --- ## And the cost? — vector + Jev is still cheaper You might assume adding Jev makes it more expensive. The opposite is true. | Approach | Options Jev reads | Cost / 1,000 calls | |---|---:|---:| | Whole catalog (51) | 51 | $0.35 | | **vector filter → Jev (10–15)** | ~10 | **$0.08** | Pre-filtering with vectors is **~4× cheaper**, because Jev's cost is driven by the length of the options you feed it — reading 10 options is far cheaper than reading 400. On top of that, the vector's embedding cost is nearly zero, because we store the vectors of previously-seen queries permanently (repeat query = no recompute). And Jev itself bills for input only, output is free, and it's several times cheaper than a general LLM. **In round-trips:** from **4 rounds → 1.** The user gets an answer noticeably faster, and the client-side LLM doesn't have to break focus to sift top-k on every question — it can spend those tokens on what matters. **That's a genuinely better MCP experience.** --- ## Lessons worth keeping (the real ones) Shipping this for real means being honest about the limits. We found four. **1. The context you feed matters more than you think.** Almost every Jev "miss" was us feeding an incomplete description, or a reference answer that was too narrow. Fixing the descriptions in the database improves the *whole* system — vector and Jev both consume the same descriptions. **2. "Asking to confirm" has to be smart, or it destroys the UX you just fixed.** Sometimes Jev genuinely isn't sure — the question is ambiguous, or two options are neck-and-neck. We designed the tool to look at the *shape* of the confidence, not just one number: - clearly confident → use it - borderline but usable → answer anyway, with a note on how it interpreted the phrase (don't block) - two options genuinely close *and* genuinely different → *then* ask the user to confirm (rare) The principle: only ask to confirm when "the right answer exists but Jev can't pick it cleanly" — not every time it's unsure. Otherwise you're back to ping-pong. **3. Multiple languages → translate to English first (and this is a vector limitation, not Jev's).** Testing deep accounting terms across languages, we found the Thai word for *goodwill* (ค่าความนิยม) fell to rank 45 in the vector search — out of the top-15, so Jev never saw it as an option. On the surface it looks like Jev failed. But we proved it's a **cross-language weakness of the embedding, not Jev** — ask the same term in English ("goodwill") and it ranks #1 instantly; Jev even knows that のれん (Japanese) means goodwill. Luckily our MCP tools already require the client to translate to English before sending — so this doesn't bite in practice. **4. Razor-thin accounting distinctions — Jev sails through.** We built the meanest test we could: pairs of terms that differ by a hair but mean different things. | Distinction to make | Jev got it right? | |---|---| | treasury stock ↔ buybacks | ✅ | | provision (P&L) ↔ allowance (balance sheet) | ✅ | | deferred tax asset ↔ deferred tax liability | ✅ | | current ↔ non-current | ✅ | | interest-bearing debt ↔ total liabilities ↔ borrowings (excl. leases) | ✅ | **25 brutal pairs — Jev scored 100% at an average confidence of 0.998**, versus the vector's rank-1 at 48%. This is the scariest failure mode (being confidently wrong), and Jev passed it comfortably in English. --- ## Conclusion: when to collapse a resolver with Jev From what we've tried, the playbook is clear: - **Small catalog (a few dozen):** let Jev pick from the whole set in one shot. - **Large catalog (hundreds / thousands / tens of thousands):** use vector search to pre-filter to ~10–15, then let Jev pick — **you still collapse the round-trips *and* pay less.** - **Add a cache shortcut:** phrases you've resolved before (e.g., "current assets" → `CurrentAssets`) get stored, so next time you return instantly without calling Jev again. All of this turns MCP from "four rounds of ping-pong per question" into "one round" — faster, cheaper, and lighter on the client LLM. **A caveat:** don't flip everything in one day. The safe path is to run Jev **in parallel** with the old resolver on real traffic for a while, compare the results, watch the confidence distribution, then enable it piece by piece — and keep a fallback in case Jev is slow or down (we measure ~1 second normally, but with a tail that can spike into the tens of seconds). Still, the direction is clear: **a resolver is the "round-trip tax" MCP pays on every query — Jev pays that tax for you.** And no matter how many candidates there are, the vector + Jev pairing collapses it, both faster and cheaper.