$cat folder-descriptions-agent-context.md

Folder Descriptions as Agent Context: Retrieval Without a Vector Database

Q: How is folder-description retrieval different from RAG?

RAG (retrieval-augmented generation) chops documents into chunks, embeds them as vectors, and retrieves the nearest chunks to a query at run time. Folder-description retrieval ranks curated, human-written descriptions of whole folders and returns the documents inside the top matches. RAG re-derives relevance from raw text every query; folder descriptions encode relevance once, by a person, and reuse it. They are complementary — you can index the same markdown for RAG later — but for durable, governed knowledge the curated signal is more precise and far cheaper to run.

July 6, 202612 min readby MDflowview as .md

Glowing emerald folders emitting labeled ribbons of light that cascade into a geometric AI agent core, with a faded unused embedding cloud to the side, on a dark terminal-grid background

Retrieval-augmented generation taught a whole industry the same reflex: if an AI agent needs to know something, embed everything, store the vectors in a database, and search by similarity at query time. It works. It is also, for a large class of knowledge, more machinery than the job requires — an index to build, a pipeline to keep in sync, and a similarity score that re-guesses at run time what a person could have just written down.

There is a quieter approach that predates embeddings and still holds up: describe your folders. If the knowledge is already organized, and each folder carries a short human-written description of what lives inside it, an agent can rank those descriptions and read the right documents — no vectors, no index, no reindexing. This post is about that pattern, why it is a real alternative to RAG for durable knowledge, and how MDflow makes it a first-class primitive.

TL;DR — Folder descriptions are agent context. A folder's description says what belongs inside it; MDflow uses that description as the primary ranking signal when an agent asks for context on a topic — scoring descriptions first, then folder names and document titles, and returning the most relevant markdown. It is retrieval without a vector database: no embeddings, no index to rebuild, and relevance encoded once by a human instead of re-derived every query. MDflow ships this today as the mdflow_get_context tool.

What are folder descriptions as agent context?

Folder descriptions as agent context means using a short, human-written summary of each folder as the signal an AI agent ranks during retrieval — instead of embedding document text into vectors and searching by similarity. In MDflow, every folder can carry an optional description. People write it to remember what belongs in a folder; agents read it to decide which folder is relevant to the task in front of them.

Concretely, a workspace might look like this:

Acme (workspace)
└── Billing
    ├── description: "Stripe integration, invoices, dunning, tax. Source of
    │   truth for how we charge customers and handle failed payments."
    ├── Stripe-webhooks.md
    ├── Dunning-policy.md
    └── Tax-rules-by-region.md

When an agent asks for context on "why did the customer's card get retried three times", it does not need a vector search over every document body. The folder description already says this folder is the source of truth for dunning and failed payments. That one sentence, written once by a person who knew what they meant, is a sharper relevance signal than the nearest-neighbor cosine distance of a chunk that happens to contain the word "retry."

This is retrieval by curation rather than derivation. RAG derives relevance from raw text at query time. A folder description states relevance up front, in plain language, and reuses it on every query. The description came from someone — a teammate, or an agent itself — who understood the material. That is exactly the understanding embeddings try to reconstruct statistically.

Why this is useful — for developers and AI agents

For developers, the appeal is everything you don't have to build.

No embedding pipeline. There is no model to call on every write, no vector store to provision, no dimension mismatch to debug, and no cost that scales with document count. You write a folder description; that is the whole ingestion step.
No reindexing, ever. Retrieval reads the live folder descriptions and titles. Edit a document, move it to a different folder, or rewrite a description, and the next query reflects it instantly. There is no index drifting out of sync with the source — the source is the index.
It is inspectable and debuggable. When RAG returns the wrong chunk, you are reverse-engineering a similarity score. When folder-description retrieval returns the wrong folder, you read the description and fix the sentence. Relevance is a string you can edit, not a tensor you can only nudge.
It rides on organization you already do. Filing documents into named folders is work teams do anyway. This approach turns that existing structure into a retrieval layer for free, instead of asking you to stand up separate infrastructure beside it.

For AI agents, it is the difference between reading a map and wandering the territory.

Curated, not re-derived. The agent gets relevance a human already got right, with provenance attached. Less of the plausible-but-wrong retrieval that raw-chunk similarity produces.
Progressive disclosure. The agent can scan folder descriptions — a cheap, high-signal table of contents — and only then open the documents that matter, loading full bodies just-in-time instead of flooding its context window.
Agents can maintain the signal. A folder description is itself markdown an agent can write. When an agent creates or reorganizes folders, it can keep the descriptions accurate — the exact bookkeeping humans abandon and LLMs are good at. The retrieval layer improves as the agent works, rather than rotting.
Structured plus readable. The agent gets both a ranked, machine-readable list and the human-readable markdown bodies, so it can reason over structure and content together.

To be clear, this is not "RAG is wrong." Embeddings are the right tool when you have a large, unstructured, fast-changing corpus with no meaningful hierarchy — a pile of support tickets, a scrape of the web, a chat history. The point is that not all knowledge is that shape. A curated knowledge base with folders and descriptions is structured on purpose, and structure is a retrieval signal you are otherwise paying an embedding model to reconstruct.

Which applications benefit most

Folder-description retrieval shines wherever knowledge is curated and hierarchical rather than a raw firehose.

Internal knowledge bases and runbooks. Engineering docs, on-call runbooks, architecture decisions, and service catalogs already live in named folders. A description per folder turns them into an agent-navigable graph with zero extra infrastructure.
Agent memory and "second brain" workspaces. A personal or team knowledge base an agent reads and writes benefits from a signal the agent can maintain itself. The folder description is the schema layer; the documents are the content.
Coding assistants over project docs. Tools like Claude Code, Cursor, and Codex thrive on repo-local markdown. Folders of specs, plans, and design notes — each described — give an agent a precise way to pull the right context without stuffing the whole repo into the prompt.
Support and product copilots over governed docs. When answers must come from an authoritative, reviewable source rather than a fuzzy nearest match, curated folders give citable, provenance-bearing context.
Small-to-medium knowledge bases generally. Below the scale where you truly need approximate nearest-neighbor search, an embedding stack is pure overhead. Keyword-scored descriptions are simpler, cheaper, and often more precise because a human wrote the relevance in.

The common thread: anywhere someone can say, in a sentence, what a folder is for, that sentence is a better first-pass retrieval signal than a vector — and it costs nothing to run.

How MDflow fits

MDflow was built on this bet before "context engineering" had a name: knowledge should be portable markdown that people and agents read the same way, organized by folders whose descriptions carry meaning. That is not a roadmap item; it is how retrieval works in the product today.

What already lines up today

Folder descriptions are the primary retrieval signal. Every MDflow folder can carry a description (up to 5,000 characters). It is not decoration: when an agent calls mdflow_get_context, the folder description is weighted highest of any field. Under the hood, MDflow tokenizes the topic into keywords and scores every folder and document by keyword overlap — a folder's description weighs most, its name less, and a document's title carries its own weight while inheriting its folder's score. Zero-scoring items are dropped, the rest are ranked, and the top documents come back as trimmed markdown plus structured JSON. This is deliberately lexical — keyword matching over curated text — which is exactly why it needs no embeddings and no vector database.

No index to maintain. Because scoring runs over live folder descriptions, names, and titles at call time, there is nothing to reindex. Rewrite a description or move a document and the very next retrieval reflects it. The knowledge base and the retrieval layer can never fall out of sync, because they are the same thing.

It never leaves the agent empty-handed. If a topic matches a folder's description but none of that folder's document titles, MDflow still returns the documents inside the matched folder. The description alone is enough to surface the right content — the whole idea of the pattern.

Nested folders with a cascading, compounded description. Folders nest to any depth, and MDflow assembles a compounded description — the labeled chain of context from the workspace down through every parent folder to where the agent is looking. Ask mdflow_get_folder or fetch a document and you get its full path plus that cascade, so the agent knows what region of the knowledge graph it is reading before it opens a single body. That is progressive disclosure, built in.

Curated context, not derived chunks. mdflow_get_context hands the agent readable markdown the way a person wrote it, ranked by a signal a person authored — with the guidance that folder descriptions are the primary context signal. No chunk boundaries splitting a sentence, no embedding drift, no similarity score to second-guess.

Agents produce the signal, not just consume it. MDflow's HTTP API and MCP server, authenticated with a Personal Access Token, let Claude, ChatGPT, Cursor, and Codex create folders, move documents, and update folder descriptions — so the retrieval layer stays current as an agent works. Relevance improves with use instead of rotting.

Raw markdown, discovery, and governance around it. Append .md to any shared link for plain markdown with YAML frontmatter over open CORS; discover the workspace through llms.txt, an agent card, and an OpenAPI spec; and govern it with sharing, collections, version history, and optional client-side encryption. The curated context sits inside a full agent-ready surface, not off to the side.

Where we are headed — direction, not a dated commitment

Folder descriptions are a strong first-pass signal, and there is room to make the retrieval richer. The following is the shape of our thinking, not a promise:

Hybrid retrieval when it earns its keep. Keeping curated folder descriptions as the primary, always-on signal, and optionally layering semantic search over document bodies for the large, unstructured corners where nearest-neighbor genuinely helps — so you get precision by default and recall when you need it.
Agent-assisted descriptions. Letting an agent propose or refine a folder's description from the documents inside it, so the primary signal writes itself as knowledge accumulates.
Typed folders and richer ranking. Adopting lightweight types and tags as additional retrieval signals alongside the description, so an agent can filter by kind before it reads.
Collection-level context. Serving a whole collection to an agent as a cross-linked, described bundle, so retrieval can span a curated set, not one document at a time.

The principle underneath all of it stays the same: a sentence a human wrote about what a folder is for is a retrieval signal worth ranking first — and often, it is all you need.

The bottom line

A vector database is one way to do retrieval, not the definition of it. For curated, hierarchical knowledge — the kind that lives in named folders because someone organized it that way — a plain-language folder description is a sharper, cheaper, and more durable signal than an embedding you have to compute, store, and keep in sync. Relevance written once by a person beats relevance re-derived on every query.

MDflow makes that the default: describe your folders, connect your agent, and retrieval just works — no index to build, nothing to reindex, and a signal your agent can keep current itself.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

Can an AI agent do retrieval without a vector database?

Yes. A vector database is one way to do retrieval, not the only way. If your knowledge is already organized into folders with human-written descriptions, an agent can rank those descriptions with plain keyword scoring and read the documents inside the best-matching folders — no embeddings, no index to rebuild, no similarity search. MDflow's mdflow_get_context tool works exactly this way.

What is a folder description in MDflow?

A folder description is an optional piece of text you attach to any folder that says what belongs inside it and why. People use it to keep a workspace organized. Agents use it as the primary ranking signal during retrieval: when an agent asks for context on a topic, MDflow scores folder descriptions first, then folder names and document titles, and returns the most relevant markdown.

How is folder-description retrieval different from RAG?

RAG chops documents into chunks, embeds them as vectors, and retrieves the nearest chunks to a query at run time. Folder-description retrieval ranks curated, human-written descriptions of whole folders and returns the documents inside the top matches. RAG re-derives relevance from raw text every query; folder descriptions encode relevance once, by a person, and reuse it. They are complementary — you can index the same markdown for RAG later — but for durable, governed knowledge the curated signal is more precise and far cheaper to run.

Do I have to maintain embeddings or reindex when documents change?

No. Because retrieval reads live folder descriptions and document titles, there is no embedding index to keep in sync. Edit a document, move it to another folder, or rewrite a folder description, and the next retrieval reflects the change immediately. The only thing worth maintaining is the folder description itself — and an AI agent can keep that current for you over the API or MCP server.

How does MDflow's mdflow_get_context tool rank results?

It tokenizes the topic into keywords, then scores every folder and document by keyword overlap. Folder descriptions carry the heaviest weight, folder names less, and document titles their own weight, with a document inheriting its folder's score. Folders and documents that score zero are dropped; the rest are sorted, and the top documents are returned as trimmed markdown plus structured JSON. If only a folder matches, its documents come back anyway so the agent is never left empty-handed.