skip to content
$cat stop-ai-hallucinating-numbers.md

How to Stop AI Hallucinating Numbers: Pointers, Not Guesses

15 min readby MDflowview as .md
A grid of glowing emerald value chips on a dark terminal grid, each tethered by a luminous arrow to a cell inside a stacked translucent document, with one chip in front unlit and its tether severed into empty space

A fund manager will not sign off on a number they cannot trace. That is not technophobia — it is the job. And it is why, in the words of one founder building for that audience, the majority of finance AI tooling in production today is being used to format slide decks rather than to produce the analysis inside them.

The usual diagnosis is that models hallucinate and will stop once they get good enough. That framing has a problem, and naming it is what makes the fix obvious.

TL;DR — To stop AI hallucinating numbers, stop letting the model write them. Store each fact deterministically under a stable reference during pre-processing, let the model plan and select references rather than values, and resolve those references in code. A model that can only dereference cannot invent a figure that does not exist — an unresolved reference fails the lookup instead of becoming a guess. MDflow applies the same split to documents: mdflow_get_document is a lookup that cannot drift, mdflow_get_context is a search that can.

This post is built on Vinoo Ganesh's talk "AI Can't Make Up a Number It Can't Look Up" (CEO and co-founder of Kepler, published 7 November 2026), the architecture it describes, and the Anthropic case study behind it. Kepler's figures are the company's own; they are flagged as such below.

Why "the models will get better" is the wrong fix

Hallucination is not a defect that scales away, because the mechanism that produces it is the mechanism that makes the model useful.

Ganesh puts it bluntly: another word for hallucination is alpha. A model that recombines, extrapolates and proposes is doing the thing you hired it for. Strip that out and you have a search box. The creativity is not a bug to be trained away — if it were, there would be only one fund, running one model, arriving at one answer.

So the problem is not that the model is imaginative. The problem is that the system lets an imaginative component emit a figure that a human will later be asked to defend. Those are different failures, and only the second one is architectural.

How to stop AI hallucinating numbers: three failure points

A wrong number arrives by one of three routes, and they need three different fixes.

  1. Wrong document retrieved. You pulled something plausible from the wrong place. This is the failure everyone optimises for, and the only one better retrieval actually solves. It is also harder than it looks: is the right source the regulator's filing, your own firm's internal model, or the one analyst your PM trusts more than the filing?
  2. Right document, wrong value. You found the 10-K and lifted the wrong line out of it. Ask for Palantir's 2023 revenue, open the 2025 10-K, and there are several revenue figures on the page — only one of them is the 2023 one. A per-citation filter catches this, but only after the value has been produced, which means the check is a cleanup, not a guarantee.
  3. Mental math. The model did arithmetic by predicting tokens. We have had a CPU instruction that adds two numbers correctly since the 1940s; there is no reason to spend tokens approximating it.

Notice what this decomposition does to the usual roadmap. "Improve the retriever" addresses one of three. "Add citations" addresses the second one late. Neither touches the third.

A lookup cannot get it wrong. Search can.

This is the line the whole architecture turns on.

Run the same semantic search ten times with the same query and you may get ten slightly different candidate sets, because the components are probabilistic and there is entropy in the ranking. That variance is a feature when you are exploring and a defect when you are reporting a number.

A lookup has no such property. A key either resolves or it does not. Ask a hash map for a key that was never inserted and you get nothing back — not an approximation, not the nearest neighbour, nothing. That failure mode is the one you want sitting underneath a number that someone will put their name on.

So the design question stops being how do we make retrieval more accurate and becomes which parts of this answer are allowed to come from search at all.

The architecture: code writes, the model dereferences

Here is the inversion, and it is genuinely an inversion rather than an extra layer.

During pre-processing, code — not the model — extracts facts from unstructured sources and persists each one under a stable identifier with its provenance attached: which filing, which page, which line item. This is the expensive step, and it happens once, offline, rather than at query time. (Pulling a figure out of a 10-K during inference is also a large token bill; doing it in a batch pipeline is not.)

At query time the model plans. It decomposes the question, resolves ambiguity, decides which facts the answer needs and which operations combine them, and emits a structured plan. What it emits in place of each figure is a reference — Ganesh's framing is that the model writes a reference to a number, not the number itself.

Code resolves the references and does the math. Retrieval is a job for a query engine; computation is a job for a formula engine. The model's output is a plan that still has holes in it, and the holes are filled by a system that cannot improvise.

A sketch of the shape, in pseudocode:

# The model emits this — references, never values
plan = {
    "op": "divide",
    "numerator":   "filing:ACME:FY2023:net_income",
    "denominator": "filing:ACME:FY2023:revenue",
}

# Code resolves. An unknown key fails the lookup; it does not become a guess.
num = store.get(plan["numerator"])     # raises on miss
den = store.get(plan["denominator"])   # raises on miss
answer = num.value / den.value         # a CPU does the arithmetic

# The value carries where it came from, not a post-hoc annotation
answer.provenance = [num.source, den.source]

The consequence is the part worth internalising: if the model cannot dereference a reference, that number cannot appear in the output. Not "appears with a warning". Cannot appear. The system refuses rather than inventing, because inventing is no longer one of the operations available to it.

This is also what separates verification from citation. A citation is an after-the-fact audit — the answer exists and a source is stapled to it. Verification is a check that happens before the value is permitted to exist. Ganesh's own anecdote makes the distinction sharp: a competitor, having seen the pitch, copied the verification layer and left the structural work behind it alone. The post-processing check is the visible part. It is not the part that does the work.

Searching wide is exactly what an LLM avoids

The dereference-only design still needs good candidates underneath it, and the retrieval stack Ganesh describes is deliberately shaped against the model's instincts.

The pipeline: approximate nearest neighbour plus BM25 to retrieve on the order of 200 candidate documents, reciprocal rank fusion to collapse those into roughly 60, then a cross-encoder that scores query and document chunk together to produce the final ranking. Hybrid storage underneath — Kepler names turbopuffer as one component.

The reason to build this explicitly rather than hand the task to an agent is a behavioural one. Agents do not like searching wide, because breadth is expensive and the reward arrives on the first plausible hit. Ask a model to research something and it will tend to plan a route, take the first good result, and proceed — depth-first dressed up as a plan. Casting a wide net and then narrowing is the opposite instinct, and it is the one that finds the right source rather than an acceptable one. Build it in code and the agent cannot shortcut it.

Five ways multi-tenant retrieval leaks

The least glamorous section of the talk is the one most likely to matter to anyone shipping this, because it is the part nobody blogs about. If your retrieval layer serves more than one customer, these are the failure modes:

  1. The tenant filter is missing. One firm's query reaches another firm's documents. In a finance context this is not a bug report, it is an incident.
  2. A caller claims to be someone else. Nothing binds the identity in the request to the documents the request is allowed to reach.
  3. Results are stale. The index answers from a version of the world that no longer exists — and because an answer comes back, nothing looks wrong.
  4. A direct read bypasses the search rules. The ACL is enforced in the search path, and then a fetch-by-id endpoint skips it entirely.
  5. The index is not populated. An empty result is indistinguishable from "no such fact", and an agent downstream may fill the silence.

The common fix is to scope the search itself with security built in, rather than filtering afterwards — which is the same principle as the rest of the architecture. Enforce the constraint where it cannot be routed around, not at the end where it can.

What this does not fix

Three honest limits, two of them stated by Ganesh himself.

Refusals drop sharply but not to zero. Kepler's production data shows the rate of "I could not dereference this, so I will not produce it" falling steeply as the deterministic store's coverage improves. It does not reach zero, because inference remains probabilistic and a model can still produce a stray figure after the fact. Hence a post-processing verification pass on top of the structural one — belt and braces, deliberately.

The accuracy figures are the vendor's own. Kepler reports 94% accuracy against 38–46% for other models, and both Anthropic's case study and the AI Engineer talk page scope that figure to specialised recall tasks such as mapping financial-statement labels to standardised taxonomy codes — not end-to-end financial accuracy. No public benchmark or denominator accompanies it. The architecture is the transferable idea; the number is a company metric.

It costs you a pre-processing pipeline. Everything above assumes someone has already done the work of extracting facts into a structured store with stable identifiers. That is the expensive part, it is domain-specific, and no model does it for you. The pay-off is that it is amortised across every query instead of repeated in every context window.

Which applications benefit most

  1. Financial research and analysis — where the figure is the deliverable and an analyst signs their name under it.
  2. Legal drafting and citation — the same structure fixes hallucinated case law. A precedent that is stored deterministically either exists in the store or does not; one that cannot be dereferenced cannot be cited.
  3. Clinical and pharmaceutical reference — dosages, interactions and thresholds, where a plausible wrong specific is far worse than an admitted gap.
  4. Compliance and audit reporting — any output a regulator may later ask you to reconstruct, line by line.
  5. Engineering and scientific specifications — tolerances, material properties, constants.
  6. Internal metrics and BI agents — the moment an agent reports a company number into a deck, the same question applies: can it be traced, or was it generated?

The shared test is simple. Is a wrong specific worse than a missing one? If yes, the model should not be the thing that writes it.

How MDflow fits

Let us be precise about this, because the honest answer is more useful than an overclaim. MDflow is a markdown workspace, not a numeric fact store. It does not extract line items from filings. What it does provide is the shape the architecture above depends on, one layer up — at the level of documents rather than figures.

What already lines up today:

  • A lookup that cannot drift, alongside a search that can. This is the central distinction, and MDflow exposes both sides of it explicitly. mdflow_get_document takes a UUID and returns that document or fails — a dereference. mdflow_get_context ranks folder descriptions, folder names and titles and returns the best matches — a search. An agent that needs the spec should be handed the id; an agent that is exploring should search. Mixing the two up is the document-level version of letting the model write the number. See the MCP docs.
  • Stable pointers between documents. Internal links use [Title](/doc/{id}), which binds to the document's identity rather than to its title or its position in the tree. Rename it, move it to another folder — the pointer still resolves. That is what makes a reference usable as a reference.
  • A dereference target an agent can actually read. Every shared document has a raw markdown twin with its frontmatter intact, so "go fetch the source" returns the source rather than a rendering of it. The same pattern backs /llms.txt, the agent card and the OpenAPI spec — discovery that does not depend on scraping.
  • Scope, declared rather than inferred. A folder description states what the documents inside it are for, and that description is the primary retrieval signal. It is how you constrain a search up front instead of filtering results afterwards.
  • A tenant boundary in one place. Every REST and MCP operation is scoped to the authenticated owner server-side. One place to be right is a design choice, and it is the one the five failure modes above argue for.
  • Replayability. Version history keeps prior revisions of a document, so "what did this say when the agent read it" has an answer. Combined with comments and sharing, the audit trail is on the document, not reconstructed from logs.

Where we are headed — direction, not a dated commitment. The gap between a document store and the architecture in this post is fact-level addressing: today a pointer resolves to a document, and the useful next step is resolving to a named block inside one, so an agent can cite the row rather than the file. Richer structured frontmatter and collection-level context for agents point the same way. The principle stays fixed: anything an agent is going to assert should be something it fetched, not something it composed.

The bottom line

The fix for hallucinated numbers is not a better model and not a louder citation. It is removing the model's permission to write the value at all: code extracts and stores each fact under a stable reference, the model plans and selects references, and code resolves them. A reference that does not resolve fails loudly instead of quietly becoming a guess.

The questions worth asking any vendor — Ganesh's own closing list — are concrete. Does the model write references, or values? Is every citation source-checked, or annotated after the fact? Is the arithmetic done by a CPU or by a language model? And the sharpest one: what happens to a number that came from nowhere?

If the honest answer is "it gets printed", you do not have a verification problem. You have an architecture problem.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

Why does AI hallucinate numbers even when the right document is in context?

There are three separate failure points, and only one of them is retrieval. The system can fetch the wrong document; it can fetch the right document and lift the wrong figure out of it, such as reading a 2025 revenue line when the question asked about 2023; or it can do arithmetic token by token instead of computing it. A better retriever fixes the first failure and leaves the other two untouched.

Can a newer model stop AI from hallucinating numbers?

Not structurally. Generation is probabilistic, so any model that is allowed to emit a figure directly can emit a wrong one, and the same creative recombination that makes a model useful is what produces an invented number. A larger model lowers the rate. It does not change what the system is permitted to do.

What is the difference between citation and verification?

A citation is an after-the-fact annotation: the answer already exists and a source is attached to it. Verification is a check performed before the value is allowed to appear, against a store that either holds the fact or does not. Citation tells you where an answer claims to come from. Verification decides whether it may be printed at all.

What does a model do if it cannot dereference a reference?

It refuses that value. In a dereference-only design, an unresolved reference fails the lookup instead of silently becoming a model-supplied number, so a missing fact surfaces as a gap rather than as a confident wrong answer. Refusal rates drop as coverage improves, but they do not reach zero, which is why a post-processing check still runs.

Does this only apply to finance?

No. Any domain where a wrong specific is worse than a missing one has the same shape: legal citations, medical dosages, engineering tolerances, inventory counts, compliance thresholds. Finance surfaces it first because the figures are audited and somebody has to be accountable when one is wrong.

Further reading