$cat ai-agent-memory-vs-rag-vs-markdown.md

AI Agent Memory vs RAG vs a Plain Markdown File (2026)

July 9, 202612 min readby MDflowview as .md

Three glowing emerald paths — a tangled memory graph, a vector point cloud, and a clean stack of Markdown sheets — converging into a single geometric AI agent core on a dark terminal-grid background

If you are building AI agents in 2026, you have already hit the wall that every team hits: the model is smart, but it forgets. Close the chat, start a new one, and everything it "knew" is gone. Give an agent a job that spans your whole knowledge base and it cannot fit it in the prompt. The models are not the bottleneck anymore — their memory is.

Three approaches now compete to solve this, and developers argue about them constantly: dedicated agent memory systems, retrieval-augmented generation (RAG), and — the answer more people are quietly landing on — a plain Markdown file. This post compares all three honestly, with the benchmarks that matter, and shows where each one actually wins.

TL;DR — Agent memory systems (mem0, Zep, Letta) auto-extract facts into their own database. RAG embeds your docs and retrieves chunks at query time. A plain Markdown knowledge base is authored, curated context an agent reads directly — and for personal or small-team knowledge under ~100k tokens it often beats both on accuracy and simplicity. The catch with Markdown is scale and retrieval. MDflow is the middle path: hosted Markdown you own, with a lightweight keyword-ranked retrieval layer instead of a vector database, plus agent write-back.

The three approaches, defined

All three exist to answer one question — how does an agent get the right context it did not have in the prompt? — but they answer it at very different levels.

1. Dedicated agent memory systems. These are frameworks that sit between your agent and its history. They watch conversations, extract salient facts ("the user prefers TypeScript", "the deploy target is Vercel"), store them in their own store, and replay the relevant ones into later prompts. The leaders in 2026 are mem0, Zep, and Letta (formerly MemGPT). Memory is derived automatically from interaction.

2. Retrieval-augmented generation (RAG). RAG takes a corpus of documents, splits them into chunks, embeds each chunk into a vector, and stores those vectors in a database. At query time it embeds the question, finds the nearest chunks, and pastes them into the prompt. Knowledge is re-derived at query time from raw text via similarity search.

3. A plain Markdown file (or folder of them). Here the knowledge is human- or agent-authored, curated, and read directly. The agent either loads the whole file, or reads a small index first and pulls only the relevant documents. No embedding model, no vector store, no chunking. This is the pattern Andrej Karpathy popularized with his April 2026 "LLM wiki" gist — three folders (raw/, wiki/, index.md) where the model reads the index, decides what is relevant, and loads only those articles. Knowledge is authored and authoritative.

The crucial difference: memory and RAG re-derive knowledge (from conversations, or from similarity search), while Markdown keeps it curated. That single distinction drives almost everything below.

Why "just use a bigger context window" is not the answer

Before comparing the three, it is worth killing the tempting fourth option: skip retrieval entirely and paste everything into a million-token context window.

This does not work, and there is data. Chroma's 2025 "context rot" study tested 18 state-of-the-art models — including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3 — and found that model reliability degrades as input length grows, even on simple retrieval and text-copying tasks. It is an architectural property of transformer attention, not a bug that training fixes.

The mechanism was named years earlier. The "lost in the middle" paper (Liu et al., 2023) showed that models use information at the start and end of a long context well, but information buried in the middle gets systematically underweighted — accuracy can drop 30% or more depending on where the answer sits.

The takeaway for all three approaches: curated, just-in-time context beats a giant prompt. Whatever you use to give an agent memory, its job is to put a small, relevant set of tokens in front of the model — not everything you have.

What the benchmarks actually say

Agent memory has become a benchmark battleground in 2026, and the numbers are worth reading with a skeptical eye.

mem0 leads on community adoption (~48k GitHub stars) and, in a May 2026 paper, reported 92.5% on LoCoMo and 94.4% on LongMemEval at under ~7,000 tokens per retrieval call — pitched as a token-efficient memory layer you bolt onto any agent framework.
Zep, built on its Graphiti temporal knowledge graph, reported 63.8% on LongMemEval versus mem0's 49.0% in a head-to-head, with up to 90% lower latency, and is the option with SOC 2 Type 2, HIPAA, and GDPR certification for regulated industries.
Letta (MemGPT) takes a different shape entirely: an operating-system-inspired runtime where the agent manages its own memory like RAM — a "main context," a "recall store" of recent history, and an "archival store" it pages in and out via function calls.

But notice the tension: mem0 and Zep publish conflicting numbers, and Zep has published a rebuttal arguing mem0's benchmark misconfigured its system (claiming a corrected 75.14% on LoCoMo). When vendors benchmark each other, the scores tell you less than the architecture and the governance model do. Choose on portability, ownership, and compliance — not on a leaderboard that shifts with the next paper.

And the plain-Markdown camp? It rarely appears on these leaderboards, because there is nothing to benchmark: the agent reads a curated document. For knowledge that fits — Karpathy's guidance is under roughly 100,000 tokens for a personal or small-team base — direct reading has no retrieval step to get wrong, so it often is the accuracy ceiling the other systems are chasing.

Which approach fits which application

None of the three is universally best. Here is the honest split.

Large, changing, multi-user corpora → RAG. Millions of documents, updated constantly, queried by thousands of users? You cannot fit that in a context window and you should not hand-curate it. RAG's similarity search over embeddings is still the right tool. It is not dead — its scope just narrowed.
Long-running conversational agents that must remember users → a memory system. A support bot or companion app that needs to recall what each user said across months of sessions benefits from automatic fact extraction. This is mem0/Zep/Letta's home turf.
Personal and small-team knowledge bases → a Markdown knowledge base. Notes, specs, runbooks, project context — knowledge you author and want to stay authoritative. Under ~100k tokens, a curated Markdown base an agent reads directly beats RAG on accuracy and beats a memory system on transparency: you can see, edit, and trust every fact.
Agentic coding and "living docs" → Markdown, decisively. Coding agents already thrive on repo-local Markdown (CLAUDE.md, AGENTS.md). A structured, navigable Markdown knowledge base is the natural upgrade — and, unlike an opaque memory store, it is diffable in a pull request.

The common thread: the more you want to own, read, and trust your agent's knowledge, the further you move from a black-box store toward plain Markdown. The more it is machine-scale and machine-derived, the more RAG or a memory system earns its keep.

How MDflow fits — the practical middle path

The plain-Markdown approach has one real weakness, and it is not accuracy — it is retrieval and scale. A single index.md works until your knowledge outgrows the context window, and a folder of files on your laptop is not something ChatGPT, Claude, or Cursor can reach. MDflow exists to keep the Markdown approach's ownership and curation while fixing exactly those two gaps.

What lines up today

Markdown you own, not a proprietary memory blob. Every MDflow document is plain-text Markdown — the same thing an agent reads and a human edits. There is no opaque extracted-facts database you cannot inspect. Your knowledge stays authored and authoritative, which is the whole point of the Markdown camp.

A retrieval layer without a vector database. MDflow's answer to "how does this scale past one context window?" is not embeddings — it is folder descriptions as a curated ranking signal. Every folder carries a human-written description of what belongs in it, and those descriptions cascade from the top of your workspace down to each document. mdflow_get_context scores folder descriptions first, then names and titles, and returns only the relevant Markdown — readable context plus structured JSON. It is keyword-ranked retrieval you can read and tune, not a black-box similarity search. (We wrote up the mechanics in Folder Descriptions as Agent Context and why curation beats a bigger prompt in Context Engineering for AI Agents.)

Agents read and write — so memory stays current. The weakness of a hand-kept Markdown base is that humans forget to update it. MDflow's MCP server and HTTP API, authenticated with a revocable Personal Access Token, let Claude, ChatGPT, Cursor, and Codex create, update, move, and organize documents — the same agent-maintained-wiki pattern we cover in The Karpathy-Style Wiki. The bookkeeping LLMs are good at finally has somewhere durable to live.

Reachable from any client, with a raw .md twin. Append .md to any shared MDflow link and you get the document as plain Markdown with YAML frontmatter over open CORS — an agent can fetch and cite it in one request. That is the portability of a local file with the reach of a hosted service.

Governed and reversible. Because every write path — editor, API, and AI agent — captures automatic version history with line diffs and one-click restore, agent write-back is safe: nothing an agent does is unrecoverable. Add public and private sharing, collections, anchored comments, and optional client-side AES-256 encryption, and you have knowledge you can actually trust an agent to touch.

Where we are headed

This is direction, not a dated commitment, but the shape is clear:

Serving a whole collection to an agent as one cross-linked bundle over HTTP, so an agent can pull an entire curated knowledge set — closer to Karpathy's index.md + articles model, hosted and multi-client.
Richer typed metadata on documents (aligned with emerging standards like Google's Open Knowledge Format), so a folder of notes becomes a set of typed, queryable concepts without leaving the editor.
Agent-assisted enrichment — letting an agent propose folder descriptions and cross-links for knowledge you already have, so the retrieval signal improves itself.

The bottom line

The "memory vs RAG vs Markdown" debate has a boring, correct answer: it depends on scale and ownership. RAG for machine-scale, changing, multi-user corpora. A memory system for agents that must remember individual users over time. And for the knowledge you author and want to keep authoritative — personal notes, team specs, project context, coding-agent memory — a curated Markdown knowledge base wins on accuracy, transparency, and trust.

MDflow's bet is that most people and teams live in that third category, and that the only things stopping plain Markdown from being the default were retrieval and reach. So we built a keyword-ranked context layer instead of a vector database, made every document reachable by every major agent, and made agent write-back safe with automatic version history. Curated Markdown you own, with memory that scales — that is the middle path.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

What is the difference between AI agent memory, RAG, and a Markdown file?

They solve the same problem — giving an agent context it did not have in the prompt — at different levels. A dedicated memory system (mem0, Zep, Letta) extracts facts from conversations and stores them in its own database to replay later. RAG embeds your documents into vectors and retrieves the closest chunks at query time. A plain Markdown file is human-written, curated knowledge an agent reads directly, either whole or through a lightweight index. Memory and RAG re-derive knowledge automatically; Markdown keeps it authored and authoritative.

Is RAG dead in 2026?

No, but its scope narrowed. RAG is still the right architecture for large, frequently changing, multi-user corpora that cannot fit in a context window. For personal and small-team knowledge bases under roughly 100,000 tokens, a curated Markdown knowledge base an agent reads directly (the pattern Andrej Karpathy popularized in April 2026) often beats RAG on accuracy and simplicity because there is no chunking, embedding, or retrieval step to get wrong.

Why not just put everything in the context window?

Because bigger context makes agents worse, not better. Chroma's 2025 "context rot" study found that reliability degrades as input length grows across 18 leading models, even on simple retrieval. The older "lost in the middle" finding (Liu et al., 2023) showed models attend well to the start and end of a long context but poorly to the middle. Curated, just-in-time retrieval beats stuffing everything into the prompt.

Which AI agent memory system is best in 2026?

It depends on the use case. Mem0 leads on community adoption and consumer apps; Zep (with its Graphiti graph) leads on benchmark accuracy and ships SOC 2, HIPAA, and GDPR compliance for regulated industries; Letta (formerly MemGPT) offers an operating-system-inspired runtime for autonomous agents. Benchmark numbers are contested between vendors, so weigh ownership, portability, and governance alongside the scores.

How does MDflow fit into the memory vs RAG debate?

MDflow is the middle path: hosted Markdown you own, with a lightweight retrieval layer instead of a vector database. Folder descriptions become a keyword-ranked context signal that mdflow_get_context uses to return only the relevant documents, agents read and write back over MCP or the HTTP API, and every document is portable Markdown with a raw .md twin. You get the curation and portability of the plain-Markdown approach with retrieval that scales past a single context window.