---
title: "Provenance for AI Agent Memory: Where Did That Fact Come From?"
description: "Provenance for AI agent memory means tracing every derived fact back to its source. Why source IDs break under LLM synthesis — and what markdown can and cannot do."
author: "MDflow"
date: 2026-07-20
reading_time: "15 min"
canonical_url: https://mdflow.cz/blog/provenance-for-ai-agent-memory
md_url: https://mdflow.cz/blog/provenance-for-ai-agent-memory.md
---

# Provenance for AI Agent Memory: Where Did That Fact Come From?

*Published July 20, 2026 · 15 min read*


An agent looks up a patient and gets back a clean, confident sentence: *patient has a penicillin allergy*. Three sources went into it — a long electronic health record, a PDF lab report, and something the patient typed into an intake chatbot. The sentence does not say which. Hand it to a doctor without that distinction and you have built a machine for laundering an unverified claim into a clinical fact.

That example opens Daniel Chalef's AI Engineer talk [*Citation Needed: Provenance for LLM-Built Knowledge Graphs*](https://www.youtube.com/watch?v=H7puB0RwJMM), and it is the question every memory layer eventually has to answer: **when an agent retrieves a fact, can you point at the exact source it came from?**

> **TL;DR** — LLMs synthesise. The fact your agent stored often appears nowhere verbatim in its inputs, so the paper trail is destroyed at write time unless you engineered it in. A single source ID does not survive multi-source synthesis, entity merging, or invalidation by newer data — lineage has to be an evolving *set*. Graph memory like [Graphiti](https://github.com/getzep/graphiti) solves this by keeping sources verbatim and linking every derived fact back to them. File-based memory is genuinely weaker at *fact-level* lineage — but stronger somewhere the graph is not: the artifact is a document, so the citation can live next to the claim. [MDflow](https://mdflow.cz) adds the parts a file gives you for free only if someone records them: version history on every write path, and an actor on every change.

## What is provenance in AI agent memory?

**Provenance is the record of how an artifact was built and why.** In the [W3C PROV](https://www.w3.org/TR/prov-overview/) vocabulary it is a graph of three things: *entities* (the data), *activities* (the processes that produced it), and *agents* (who or what was responsible). For an agent memory store it collapses into three practical questions:

1. **Where did this come from?** Which specific sources produced this stored fact.
2. **Should I trust it?** What those sources are — a verified clinical record, or a chatbot transcript.
3. **What happens if I delete a source?** Which derived facts survive, and which must go with it.

Chalef's framing is that provenance is usually treated as a compliance chore and is in fact three other things as well: a debugging tool (*why does my agent believe this?*), a trust filter (*which sources do I let into this answer?*), and a deletion policy (*what has to be erased with the source?*).

The reason it is hard is stated in the first thirty seconds of the talk: LLMs pull together data from many sources and do so **non-deterministically**. They interpret and synthesise, and the resulting summary, extracted fact, or structured record **may not appear verbatim in the source inputs**. Synthesis destroys the paper trail. If you did not build lineage into the write, you cannot recover it afterwards by grepping.

## Why a source ID on each fact does not work

**Because a source ID assumes one input and one deterministic transformation.** In a data warehouse that holds: a pipeline copies or mutates a value, the source is known, you stamp it. Context pipelines run by LLMs break it four ways:

| What happens | Why the single ID fails |
| --- | --- |
| One prompt, several sources | A fact is synthesised from *one or more* of them — the relationship is many-to-many from the start |
| Entity resolution | "J. Smith" and "John Smith" merge into one identity whose facts now come from many places |
| New data invalidates old | The store keeps changing underneath your pointer; the fact is superseded, not replaced |
| Append-only logging | Reconstructing lineage from a change log gets impractical once changes are constant |

The conclusion Chalef draws is the load-bearing one: **lineage needs to be an evolving set of links that survives mutation** — not a foreign key, and not a log you replay afterwards.

## How graph memory answers it

**A set of links between facts and their sources is, structurally, a graph.** That is the argument for building provenance into the data structure rather than beside it, and it is what [Graphiti](https://github.com/getzep/graphiti) — the open-source temporal graph framework underneath Zep's agent memory — does. Five mechanisms are worth stealing regardless of what you store your memory in:

**1. Sources are kept verbatim.** Ingested data becomes an *episode* node, byte-for-byte as it arrived, and stays retrievable. Everything derived points back at it. Tracing a fact to its source is then just a graph walk.

**2. Facts are bi-temporal.** Each fact carries when it was true in the world *and* when it was recorded or invalidated by the system. Superseded facts are marked invalid, not deleted — and the episode that did the invalidating is recorded against them, so *why it changed* survives too.

**3. Merges union their sources.** When two entities resolve into one, the merged node keeps the episode links from **both** sides. Keeping only one side would silently drop a source, which is the failure mode you would never notice.

**4. Tags at ingestion project onto everything derived.** Tag an episode `EHR` on the way in and every entity and fact extracted from it inherits the tag. An agent that wants only verified clinical sources filters on it while walking the graph — one action at write time buys veracity filtering at read time.

**5. Deletion cascades by support, not by reference.** When a source has to go — retention policy, right-to-be-forgotten request — the rule is *a fact is deleted only if no remaining episode supports it*. In the talk's example, deleting the intake-chat source leaves the allergy fact alive (two other sources still support it) but removes the contact-preference fact, which had no other parent.

One nuance from the talk deserves repeating, because it is where most designs go wrong. Two facts can have the identical shape — three parent sources, one unverified — and demand **opposite** policies. For a drug allergy, *any* unverified parent should block a prescription. For consent on file, *every* parent must be verified. The store's job is to expose which sources carry which tags. Applying the business rule is your agent's job. Provenance is infrastructure; trust policy is situational.

## The markdown objection, taken seriously

The most interesting moment in the talk is an audience question about the industry's swing toward file-based memory — wikis, markdown, `AGENTS.md`-style knowledge bases. Chalef's answer is blunt, and worth quoting rather than paraphrasing away:

> Markdown suffers from provenance. […] It's very difficult when you mutate lines in a file to understand the lineage or the provenance of why those changes occurred.

He adds that files work well for desktop and single-user, single-agent cases, and that multi-agent, multi-user, multi-source scenarios at enterprise scale are where they break down.

**Take the fair half first.** A markdown store does not do entity resolution, so it has no merge-union problem to solve — and no merge-union solution either. It does not compute fact-level lineage, so it cannot tell you that sentence 4 of a document came from source B while sentence 5 came from sources A and C. It has no automatic cascade delete: remove a source document and nothing recalculates which derived claims lost their support. If your compliance requirement is *prove which of these three ingested records produced this specific clause*, a file will not do it by itself.

**Now the half the critique skips.** The reason a graph has to reverse-engineer lineage is that its derived artifact is a *synthesised triple* — a thing with no room inside it to hold a citation. A markdown document is different in kind: it is prose, and prose has always carried its own references.

- **The citation goes inside the artifact.** `Patient reports a penicillin allergy ([intake chat, 2026-07-14](/doc/…); unconfirmed by [lab report](/doc/…))` is a fact and its provenance in one object that survives copying, exporting, and being pasted into a context window. No walk required, and nothing to keep in sync — which is precisely the class of bug the talk describes when the store mutates underneath a pointer.
- **Sources can stay verbatim, as documents.** Graphiti's first principle — keep the raw source and link to it — is not a graph feature. It is a discipline. A folder of captured sources plus documents that link to them is the same shape with worse joins and better readability.
- **Line-level lineage over time is what diffs are.** "Why did this line change?" is exactly the question a version diff answers, and it answers it with the *actual* before-and-after rather than an inference about it. What a file lacks is not history — it is the automatic *derivation* links between a change and the inputs that caused it.

So the honest scoreboard: graphs win on **derivation** lineage (this fact ← these sources, unioned and mutated over time). Files win on **portability and legibility** of the citation, and tie on keeping sources verbatim. Which one you need depends on whether your agent's memory is a set of extracted facts or a set of documents.

## Which applications need provenance most

1. **Regulated domains** — healthcare, finance, legal. The veracity distinction (verified record vs. patient-typed claim) is the difference between a decision support tool and a liability.
2. **Right-to-be-forgotten and retention.** Deleting a source is only half the job; the derived context is the other half, and you cannot do it without knowing what derived from what.
3. **Agent-written knowledge bases.** Once agents write into a shared store, "which agent wrote this, when, and from what?" becomes the first question anyone asks about a wrong entry.
4. **Research and analysis agents.** Findings are only as good as their weakest source, and a synthesis that flattens a preprint and a peer-reviewed paper into one confident sentence is worse than no synthesis.
5. **Debugging retrieval.** Half of "the agent hallucinated" turns out to be "the agent faithfully repeated something bad in the store." You cannot tell those apart without lineage.

## How MDflow fits

**MDflow does not build a fact graph — it makes the file half of the answer complete.** The three things a markdown store needs before provenance is real are: sources kept verbatim, an inline citation convention, and a record of every change with an actor attached. The first two are conventions any agent can follow. The third is infrastructure, and it is the part MDflow supplies.

### What already lines up today

**Every write is versioned — including the ones agents make.** MDflow captures the previous version of a document on every saved change, and capture covers *every* write path: the in-app editor, the [HTTP API](/docs/api), and the [MCP server](/docs/mcp). You get a line-by-line markdown diff between any previous version and the current text, and a non-destructive restore. That is the direct answer to "why did this line change" — retention is 30 days or the last 300 versions per document, and it is a Pro feature. See [version control for documents](/blog/version-control-for-documents).

**Every change has an actor.** The Document Log at `/log` is a cross-document feed of created, edited, shared, and deleted events, and each row names who did it: `you` for browser actions, and `automated · <token name>` for anything that arrived over the API or MCP — Claude, the [Web Clipper](/clipper), [n8n](/docs/n8n), the VS Code extension, the iOS app. Click an edited row for the diff of that specific change. This is W3C PROV's *agent* class, in the place you actually need it: when a document says something wrong and you need to know which integration put it there. Deleted documents keep their who-what-when even though the document is gone.

**Sources can be captured verbatim, with their metadata.** The [Web Clipper](/clipper) saves a page as clean markdown and writes typed frontmatter properties at the top — pre-populated from page variables including URL, author, published date, and domain. That is Graphiti's episode-with-tags pattern expressed in markdown: the raw source stored as its own document, tagged at ingestion, ready to be linked from anything derived from it.

**Citations resolve, and stay resolvable.** Internal `@`-mention links between documents give a derived note a real pointer to its sources inside the workspace. Outside it, every shared document has a raw `.md` twin whose YAML frontmatter carries the title, canonical URL, markdown URL, and visibility — so an agent can cite and fetch a source in one request. This post has one, at [`/blog/provenance-for-ai-agent-memory.md`](https://mdflow.cz/blog/provenance-for-ai-agent-memory.md).

**Trust scope lives in folder descriptions.** Every folder carries a description of what its documents are *for*, and `mdflow_get_context` ranks those descriptions above folder names and titles when an agent asks about a topic — [retrieval without a vector database](/blog/folder-descriptions-agent-context). A description is a reasonable place to state a source policy ("clipped vendor documentation, unverified") that everything retrieved from the folder inherits by convention. [Workspaces](/faq) bucket it further, per client or product line.

**What MDflow does not do**, stated plainly: it does not compute fact-level lineage, it does not merge entities, and it does not cascade deletions across derived documents. Provenance in MDflow is per-document and per-version, plus whatever citations the agent writes down. If you need *this clause ← these three ingested records*, you want a graph, and [Graphiti](https://github.com/getzep/graphiti) is open source.

### Where we are headed

Direction, not a dated commitment: richer typed frontmatter so a document can carry source, confidence, and freshness as structured fields an agent can filter on rather than prose it has to interpret; serving a whole collection as one cross-linked bundle so a derived note and its sources arrive together instead of as separate retrievals; and tighter review round-trips on agent-proposed edits, so a change with a questionable source can be caught before it becomes something a future session reads as fact.

## The bottom line

Deriving context is lossy and generative. That is the whole problem: the artifact your agent stored is not a copy of anything, so the link back to its sources exists only if something recorded it at write time. Logging it afterwards does not work, and a single source ID does not survive synthesis, merging, or invalidation.

If your memory is a graph of extracted facts, build lineage into the graph — that is what Graphiti does, and the deletion rule alone (*a fact dies only when no source supports it*) is worth the architecture. If your memory is a folder of documents, the answer is different but not absent: keep the sources, cite them inline, and make sure every change is recorded with an actor and a diff. Then "why does my agent believe this?" has an answer you can read.

[Start free](/login) · [Connect an AI agent](/docs/mcp) · [Read the API docs](/docs/api)

## Frequently asked questions

### What is provenance in AI agent memory?

Provenance is the record of how a derived artifact was built and why — which sources produced a stored fact, which later source changed or invalidated it, and what survives if a source is deleted. It matters because an LLM synthesises memory non-deterministically, so the stored fact often appears nowhere verbatim in its inputs and the paper trail is destroyed unless it was engineered in at write time.

### Why can't I just store a source ID on each fact?

Because a single source ID assumes one deterministic input. LLM-derived memory breaks that in four ways: one fact is synthesised from several sources at once, entities get merged so their sources must union rather than replace, new data invalidates old facts so the pointer target keeps mutating, and an append-only log of all those changes gets impractical to reconcile at scale. Lineage has to be an evolving set that survives mutation, not a single foreign key.

### How does Graphiti track provenance?

Graphiti, the open-source temporal graph framework behind Zep, keeps source data verbatim as episode nodes and links every derived entity and fact edge back to the episodes that produced it. Facts are bi-temporal, carrying both when they were true in the world and when they were recorded or invalidated. Merged entities keep the episode links from both sides, tags applied at ingestion are inherited by everything derived from that episode, and deleting a source removes a fact only if no remaining episode supports it.

### Is markdown bad at provenance?

Zep's Daniel Chalef argues file-based memory breaks down on provenance, because it is hard to reconstruct why a line in a file changed and harder still to manage at scale across many agents, users, and sources. That critique is fair for fact-level lineage. What markdown does have is that the artifact is a document rather than a synthesised triple, so the citation can live inline next to the claim, sources can be stored verbatim as their own documents, and version history gives line-level lineage for every change over time.

### How do I keep an audit trail of what an AI agent wrote to my notes?

In MDflow every write path is recorded. [Version history](/blog/version-control-for-documents) captures the previous version of a document on every saved change, including writes arriving through the [MCP server](/docs/mcp) and the [HTTP API](/docs/api), and shows a line-by-line diff you can restore from. The Document Log at `/log` is a cross-document feed of created, edited, shared, and deleted events with the actor on every row — `you` for browser actions and `automated · <token name>` for anything an agent or integration did. Version history is a Pro feature; the log is recorded for every account.

## Further reading

- [Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI](https://www.youtube.com/watch?v=H7puB0RwJMM) (AI Engineer, July 2026) — the talk this post is built on.
- [How Zep tracks provenance in agent memory](https://blog.getzep.com/how-zep-tracks-provenance-in-agent-memory/) — episodes, metadata projection, and the deletion cascade in detail.
- [Graphiti on GitHub](https://github.com/getzep/graphiti) — the open-source temporal graph framework.
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://arxiv.org/abs/2501.13956) — the paper behind the architecture.
- [PROV-Overview — W3C](https://www.w3.org/TR/prov-overview/) — the standard vocabulary for entities, activities, and agents.
- [PROV-AGENT: Unified Provenance for Tracking AI Agent Interactions in Agentic Workflows](https://arxiv.org/abs/2508.02866) — extending PROV to agentic systems.
- MDflow: [agent memory consolidation](/blog/agent-memory-consolidation), [AI agent memory vs RAG vs a plain markdown file](/blog/ai-agent-memory-vs-rag-vs-markdown), [version control for documents](/blog/version-control-for-documents), [letting agents write to your knowledge base](/blog/ai-agents-write-to-knowledge-base), and the [MCP setup guide](/docs/mcp).