VerityData MCP vs. Web Search and the SEC API: What the Numbers Show

We gave an AI research agent the same SEC filings tasks three ways and graded every answer against EDGAR. Two of three methods always got there eventually. Here’s what it cost them to get there — and what happened when one didn’t.

—

We asked an AI agent to do something simple: pull one paragraph out of Apple’s most recent 10-K. Item 9B, “Other Information”. It’s 85 words, buried inside a filing that runs several hundred thousand characters.

Given only web search, the agent never found it. Four of five attempts burned through 3 to 3.9 million tokens searching, opening pages, and re-searching before hitting the session’s turn limit. The fifth gave up early and returned a one-word answer.

When we tried it again ourselves — this time in the Claude web app on “High” effort — the agent ran a dozen searches, visited as many pages, still couldn’t locate the text, and instead tried to reconstruct the paragraph from news coverage. The reconstruction was wrong.

That’s not a speed problem but one of AI hallucination and output quality.

Why We Ran This Experiment

We’ve heard from clients and prospects that token costs are becoming a real constraint on how they deploy AI in research workflows. Every search, page load or re-read of a filing draws down usage limits or racks up API spend. It’s the same shift we’ve written about before. As firms move AI from ad hoc pilots into always-on production systems, the parts of the stack that used to be a minor inconvenience, such as how an agent actually gets its data, become the thing that determines whether the workflow holds up at all.

So rather than make the case in the abstract, we ran the numbers. We gave a Claude agent (Sonnet 5) three real SEC research tasks, three different ways: once connected only to the VerityData MCP server, once with only built-in web search, and once calling the SEC’s own API directly. Same prompts, same model, same settings, every configuration runs five times. Every answer was graded mechanically against the official EDGAR record — no LLM judged its own homework. We tracked total tokens processed (the figure that draws down usage limits on Claude subscriptions) and wall-clock time.

A caveat worth stating up front: we didn’t set out to catalogue every obstacle in web search or the SEC API, but we hit some anyway. Web search was inconsistent run to run, landing on different sources each time. The SEC API’s own rate limit (10 requests per second) slowed it down more than once.

Here’s what we found.

Three Tasks Done Three Ways

1. Every Insider Filing at Warner Bros. Discovery

Getting the complete list matters less than what it costs to confirm it’s complete.

The task: List every SEC Form 4 for Warner Bros. Discovery over a six-week window — accession number, filing date, insider name — graded on exact match against EDGAR.

All three eventually got there; however, web search needed 20 to 39 tool calls to page through EDGAR’s Form 4 index. The SEC API’s submissions endpoint listed every filing in one shot, but insider names live inside the individual Form 4 documents, so the agent still had to open each one. It finished at 1.4x Verity’s token cost.

Insider transactions are Verity’s core domain. One structured query returned the complete list using 2.4× fewer tokens and 7.6x faster than web search.

2. One Paragraph in Apple’s 10-K

A general-purpose search tool doesn’t know a filing has sections; it only knows the filing is long.

This is the task from the opening, the one web search never completed.

The SEC API did complete the task, but no endpoint serves a section of a filing on its own, so the agent paged through the entire document to reach 85 words, landing at 8.6x Verity’s token cost. Verity’s filing tools address documents section by section: ask for Item 9B, get exactly Item 9B, five out of five times, in under 20 seconds.

Each failed web attempt processed roughly 40x the tokens of a complete Verity MCP run, and still came back empty.

3. A Year of Apple Cover Pages

When every method gets the right answer, the only thing left to compare is what it took to get there.

The task: For every 10-K and 10-Q Apple filed over a twelve-month window (four filings), report the cover-page facts, form type, accession number, fiscal period end, exact shares outstanding.

Every approach landed on the right answer here; the difference is entirely in what it took to get there. Cover-page facts are structured data, and Verity MCP returns them as structured data, one call per filing. Web search needed a median of 15 searches and up to 41 tool calls per run to find each filing, open it, and read the cover page by brute force.

The SEC API found the same facts in the SEC’s XBRL data and landed in between, at 2.4x Verity’s token cost.

The Real Difference

Look across all three tasks and a pattern holds: when the underlying data is genuinely unstructured relative to the request, whether a single section inside a sprawling filing, a name buried inside an individual document, the gap between purpose-built access and general-purpose search doesn’t just widen, it becomes a completeness problem.

Two of three tasks, every method eventually landed on the right answer; it was a question of cost. The third task, the one that most resembles what an analyst actually asks for in practice — “get me this specific piece of a filing” — is where web search didn’t just get slower. It failed outright, and tried to paper over the failure with a plausible-sounding guess.

That’s the distinction that matters as research workflows move from a human occasionally pulling a filing to an agent doing it continuously, unsupervised, as part of a larger pipeline. A person who can’t find something in a 10-K knows they’ve failed and says so. An agent that can’t find something may hallucinate a confident answer instead. And everything built downstream of that answer inherits the error.

Bottom Line

Token efficiency is the easy part of this story to sell: 2.4x to 8.6x fewer tokens across three representative research tasks, in some cases 7x or more faster. But the more consequential finding is the one that doesn’t fit neatly into a multiplier. An agent given only web search couldn’t complete a routine filings task at all, and instead of failing visibly, it fabricated an answer. For research teams putting AI agents into production, that’s the real argument for purpose-built data access: not just that it’s cheaper, but that it’s the difference between an agent that tells you it couldn’t find something and one that quietly makes something up.

How to read these numbers: Tokens processed includes fresh input, cache writes, cache reads, and output. For Clause subscriptions, this is the figure that draws down usage limits. All figures are medians of five runs; token comparisons use successful runs only, since a cheap wrong answer isn’t a win. Every answer was checked mechanically against SEC EDGAR (data.sec.gov), including accession numbers, dates, share counts, and verbatim text, with no LLM grading its own output.

See VerityData MCP in Action

The VerityData MCP Server gives AI agents structured, section-level access to SEC filings, insider transactions, and more. It’s the same access we used to run this benchmark.

Request Access