---
title: "Agent Context Comes in Shapes, Not Queries"
description: "Agent context comes in shapes, not queries. Why text-to-SQL and vector search cannot answer 'what are we missing?' — and the three structures that can."
author: "MDflow"
date: 2026-10-22
reading_time: "16 min"
canonical_url: https://mdflow.cz/blog/agent-context-shapes-not-queries
md_url: https://mdflow.cz/blog/agent-context-shapes-not-queries.md
---

# Agent Context Comes in Shapes, Not Queries

*Published October 22, 2026 · 16 min read*


Two tools solved data access for AI agents. Text-to-SQL opened the warehouse; vector search opened the document store. Between them, an agent can now reach almost anything an organisation owns, and reaching the data is no longer the hard part.

It still returns confidently wrong answers. Not because something was missing from the index, but because the only way it was ever allowed to look at the material was one slice at a time, with each slice selected by its resemblance to the question already being asked.

> **TL;DR** — Giving an agent query access to your data is not the same as giving it context. A query returns the slice that resembles the question, which leaves three ordinary questions structurally unanswerable: *what are we missing*, *what patterns run across everything*, and *how do these records relate*. The fix is to build persistent **shapes** over the corpus — a semantic layer over structured data, a table-of-contents tree over documents, theme clusters over the whole set — that an agent can traverse rather than sample. The document-shaped one is by far the cheapest to get, because a nested tree of markdown with stated purposes and internal links already is one, which is what [MDflow](https://mdflow.cz) stores.

## What "context comes in shapes, not queries" means

A shape is a persistent structure over your corpus that an agent can traverse. A query is a one-off slice, selected for its resemblance to the question. The claim is that the first is context and the second only looks like it.

The framing comes from Zach Blumenfeld, an AI research engineer at Neo4j, in his AI Engineer workshop [AI on Your Lakehouse: Context Comes in Shapes, Not Queries](https://www.youtube.com/watch?v=kRkcNOsRyYg) (July 2026). His worked scenario is a fictional national auto-repair chain: technicians working on cars in bays, a library of vehicle manuals plus safety bulletins and recall notices sitting in cloud storage, and a warehouse of every repair the chain has ever logged. Leadership wants a copilot for the floor.

The naive build is the one most teams ship. Vector search over the manuals, text-to-SQL over the repair tables, a model in the middle. It pulls data correctly. It is also, in Blumenfeld's words, sometimes confidently wrong — and it gets worse with scale, because a few tables become hundreds of look-alike tables and a few hundred documents become millions.

The distinction that explains why is between a **single-record question** and an **estate-level question**. "How do I replace this part on this vehicle?" is a single-record question, and query access handles it. "What are we not covering?" is an estate-level question, and query access cannot handle it at any scale.

## The three questions a query interface cannot answer

### "What are we missing?"

This is a request to prove a negative, and similarity search cannot do it by construction. Semantic search retrieves things that resemble the query; an absence resembles nothing. There is no embedding in your index sitting close to a manual that was never written.

Ask which vehicle models rolling into the bays have no documentation, or which safety bulletins have never once been referenced in a repair, and a vector index will cheerfully return the nearest documents that *do* exist. The gap never surfaces. You get an answer about coverage you have, phrased as though it were an answer about coverage you lack.

### "What patterns run across everything?"

Top-k is a sample, not a census. "What are we repeatedly failing to fix?" or "which recalls cluster together?" require touching the whole corpus, and a retrieval step that returns the ten best matches has already thrown away the thing being asked about.

### "How do these records relate?"

In a warehouse with one big look-alike schema — hundreds of tables with similar names and overlapping columns — the model has to guess which joins are correct. Guessing joins is exactly where confidently wrong lives, because a plausible-looking join produces a well-formed result set with wrong numbers in it.

All three failures are silent. A query interface always returns something, which is why these get caught in production rather than in testing.

## Shape 1: connections, or the semantic layer

The first shape is a metadata graph over the warehouse that tells the agent which tables exist, what the business terms mean, and how things legally join.

Blumenfeld draws a distinction here that is the cleanest version of it I have come across, in a summer where both words have been used interchangeably by nearly everyone:

- An **ontology** helps something interpret and reason about the data.
- A **semantic layer** helps something understand what the consistent, agreed-upon terms are, so it can query the data accurately.

The important architectural move is that the data does not move. Neo4j calls this a *virtual graph*: you keep the rows in BigQuery, Databricks or Snowflake, you keep using SQL, and you load only the metadata into a graph that guides the agent. [Neocarta](https://neo4j.com/blog/genai/build-a-semantic-layer-from-gcp-with-neocarta/), the Neo4j Labs library used in the workshop, reads a warehouse's schema catalog — databases, schemas, tables, columns, foreign keys as join edges, embeddings over the object descriptions — and exposes the result to an agent over MCP. On BigQuery, a GCP project maps to a database node and a dataset to a schema node, with business terms from the GCP Knowledge Catalog attached to tables and columns.

Neo4j's own benchmark on this is worth quoting with its caveat attached: on 278 BigQuery Census public tables, they report [token reductions of up to 81%](https://neo4j.com/blog/genai/cut-text2sql-token-costs-up-to-81-and-stop-paying-for-wrong-answers-a-neo4j-semantic-layer/) and a cheaper model going from two of four questions correct to four of four, where without the layer two of those questions looped for twenty tool calls and gave up. That is a vendor-published result from one small test, so treat the numbers as directional. The failure mode behind them is not in doubt, though — an agent burning tool calls flailing at join ambiguity is something anyone who has pointed a model at a wide schema will recognise.

## Shape 2: table of contents, or the containment tree

The second shape is the hierarchy your documents already have, made explicit and addressable, so an agent can walk it instead of only searching it.

The structure is a containment tree: library, then folders, then documents, then sections nested several levels deep. Across the tree run **link edges**, where a section of one manual references a section of another — a cross-reference that the tree alone cannot express.

The implementation detail that makes it work is that every node's identifier is a **hierarchical URI built from its path**: the library root, then `/manuals`, then the file name, then the section, then the subsection. Two useful properties fall out of that choice:

1. The agent can traverse. Walk up for enclosing context, down for detail, sideways along a link — rather than issuing another similarity search and hoping.
2. You can scope retrieval by **prefix filter with no graph traversal at all**. Combining a hierarchical-ID prefix with ordinary full-text search gets you a cheap first-stage filter down to one branch of the corpus, which is how the workshop's agent narrows a search before it starts reading.

Here is the part most worth stealing. This is a **lexical, document-structure graph, not an entity-extraction pipeline.** Nothing in it requires an LLM to read your prose and infer entities and relationships. If your documents already carry inherent structure and interlinking, you capture most of the value by simply *recording the structure that is already there* — which is a week of work rather than a quarter of it, and it does not drift every time the extraction prompt changes.

Blumenfeld is honest about the limit. For some domains the entities genuinely have to be in the graph — his example was life sciences, where specific ontologies for molecules and medications carry meaning the text does not — and there the inference falls apart without them. But for a corpus of manuals, bulletins, procedures and specs, letting the agent infer entities from the text while it traverses the structure is enough.

One small naming lesson from the same session: keep relationship names consistent. `HAS_FOLDER`, `HAS_DOCUMENT` and `HAS_SECTION` force every query to branch; a single `CONTAINS` keeps traversal simple, which matters more than descriptive precision once an agent is writing the queries.

## Shape 3: themes, or the patterns nobody specified

The third shape is clusters discovered by community detection over the document graph, surfacing groupings that nobody knew to look for.

The algorithm is **Leiden**, from Traag, Waltman and van Eck's [From Louvain to Leiden: guaranteeing well-connected communities](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6435756/) (*Scientific Reports* 9:5233, 2019). It replaced Louvain for a concrete reason: in the authors' experiments Louvain produced communities that were badly connected — up to 25% poorly connected and up to 16% outright disconnected — while Leiden guarantees connectedness and runs faster.

Two practical notes from the workshop. First, **collapse the links upward before clustering**: aggregate section-to-section cross-references to the document level using the hierarchical IDs, so the graph you cluster is documents linking to documents. The communities come out cleaner and the interpretation is easier. Second, Leiden is hierarchical, so you choose the granularity you want rather than accepting whatever the algorithm's first pass produces.

Layering an LLM summary over each community is optional and is exactly what Microsoft's GraphRAG does — same algorithm, followed by hierarchical community summaries ([From Local to Global](https://arxiv.org/abs/2404.16130), 2024).

The operating cost deserves stating plainly, because it is the shape with the worst one. Leiden runs over a projection of the entire graph. It is a batch job, not an index you append to. If your corpus changes constantly you re-run it, regenerate the theme IDs, and — a pattern that long predates AI, borrowed from fraud and anti-money-laundering teams — keep the snapshots so you can watch themes evolve over time.

And one caveat that applies to all three shapes: a structure graph inherits the defects of the corpus underneath it. A malformed section or an erroneous cross-link is faithfully encoded, and nothing in the pipeline will flag it.

## Why this matters for developers, and for agents

**For developers**, the shape is where institutional knowledge actually lives. Which tables may be joined, what a folder of documents is for, which procedures supersede which — none of that is in the rows, and most of it is currently in the heads of three people and one Slack thread. Building a shape is mostly the act of writing it down somewhere both a person and a model will read it. The corollary is uncomfortable: if nobody can state what a branch of your corpus is for, no retrieval system can infer it for you.

**For AI agents**, the shape is the difference between sampling and knowing. An agent handed only a query interface has no vocabulary for doubt about coverage — it cannot tell the difference between "there are no documents about this" and "my query did not match any documents about this", so it answers both the same way. Traversable structure lets it establish where it is in the corpus, what encloses the thing it is reading, and what the thing it is reading points at. It also lets it compose: Blumenfeld's agent glues shapes together for questions nobody anticipated, going from a theme to the documents inside it to the warehouse records they reference.

## Which applications benefit most

1. **Technical documentation and field support** — manuals, bulletins, recalls and procedures, where coverage gaps are the expensive failure and cross-references are dense.
2. **Regulated and compliance-heavy corpora** — where "which policy applies here, and what superseded it?" is a traversal question and a wrong answer is a reportable event.
3. **Wide look-alike warehouses** — hundreds of similarly named tables, where join ambiguity burns tool calls and silently corrupts results.
4. **Engineering knowledge bases** — specs, ADRs and runbooks that already link to each other heavily, which is the best possible input to a document-structure graph.
5. **Research and literature corpora** — where theme discovery over a large set is the actual product, not a nice-to-have.
6. **Customer support knowledge bases** — where the estate-level question ("what are people asking that we have never documented?") is worth more than any single retrieval.

## How MDflow fits

MDflow is not a graph database. It does not run Leiden, it has no query language, and shape 3 is not something you will build in it.

What it is worth noticing is that shape 2 — the one with the best cost-to-value ratio, and the one Blumenfeld explicitly frames as recording existing structure rather than extracting new structure — is not a thing you build in MDflow either. It is what MDflow already is.

What lines up today:

- **Documents are plain markdown in a nested folder tree**, which is a containment tree in the literal sense: workspace, folders inside folders, documents inside them. No ingestion step turns your corpus into the shape; the corpus is stored in the shape.
- **Folder descriptions supply the one thing a bare file tree lacks** — a statement of what a branch is *for*. This is the part a graph build usually has to invent, and `mdflow_get_context` ranks folder descriptions as its primary retrieval signal, above folder names and document titles. See [folder descriptions as agent context](/blog/folder-descriptions-agent-context).
- **Every document and folder carries its full path and a compounded description** — the workspace-to-folder cascade of descriptions, returned alongside the body. That is the hierarchical-URI benefit from the workshop, delivered with the document rather than requiring a traversal to reconstruct.
- **Internal links are first-class.** Typing `@` inserts a live link to another document in the workspace, which is the cross-tree link edge: the part of the shape containment alone cannot express.
- **Raw `.md` twins with YAML frontmatter** give every document a machine-readable address, so the thing an agent cites and the thing it reads are the same bytes.
- **MCP and the HTTP API expose the hierarchy, not just the text** — 25 tools across documents, folders and workspaces, every call scoped server-side to the token's owner, plus a personal access token for pipelines that are not MCP clients. [MCP docs](/docs/mcp) · [API docs](/docs/api).
- **Version history, comments and Team workspaces** cover the maintenance half: a shape that nobody argues about or updates stops describing the corpus within a quarter.

Where MDflow stops, stated plainly: there is no community detection, no automatic theme discovery, and no Cypher. If you need shape 3, you need a graph platform. If your problem is a thousand-table warehouse, shape 1 is Neocarta's job or its equivalent, not a document workspace's.

Where we are headed, as direction rather than a dated commitment: richer structured frontmatter for documents, and better programmatic access to the hierarchy itself, so an agent can ask for the shape of a branch as directly as it asks for a document's text. The principle we will not trade away is that the file stays plain markdown you can walk away with.

## The bottom line

Access was the problem of 2024. Text-to-SQL and vector search solved it, and in solving it they made a subtler problem visible: an agent that can query everything and traverse nothing cannot tell you what it does not have.

The three shapes are the fix, and they are not equally expensive. A semantic layer over a warehouse is real engineering. Theme detection is a recurring batch job with real operational weight. The containment tree over your documents is neither — it is mostly the discipline of writing down what each part of the corpus is for and letting documents point at each other, which is work your team should be doing anyway and which happens to be exactly what a graph build would charge you to reconstruct later.

Start there. The absence of that structure is why your agent sounds confident about a corpus it has only ever sampled.

[Start free](/login) · [Connect an AI agent](/docs/mcp) · [Read the API docs](/docs/api)

## Frequently asked questions

### What does "context comes in shapes, not queries" mean?

It means the useful unit of context for an agent is a persistent structure over the whole corpus, not a slice retrieved per question. A query returns whatever resembles the question that was asked. A shape is a standing representation of how the material is organised and how its parts relate, which an agent can traverse: up for enclosing context, down for detail, sideways along a cross-reference. The phrase comes from Zach Blumenfeld's AI Engineer workshop on building agent context over a lakehouse.

### Why can't vector search answer "what documentation are we missing?"

Because similarity search retrieves things that resemble the query, and an absence resembles nothing. Asking which vehicle models have no manual, or which safety bulletins have never been used, is a request to prove a negative. There is no document in the index whose embedding is close to a document that does not exist, so the search returns the nearest present match instead and the gap never surfaces. Answering it requires traversing the full set and comparing it against an expected set.

### What is the difference between an ontology and a semantic layer?

An ontology helps a system interpret and reason about data — what the entities are, which classes are disjoint, what values are legal. A semantic layer helps a system understand what the consistent, agreed-upon terms are so it can query the data accurately — which table holds revenue, how two tables legally join, what the business calls a given column. The two overlap, but the distinction is between reasoning about meaning and getting the query right.

### Do I need a graph database to give my agent a shape?

Not for the document-structure shape. The table-of-contents shape is a lexical graph that records hierarchy and cross-references that your documents already have, rather than an entity-extraction pipeline, so a nested folder tree of markdown files with stated purposes, stable addresses and internal links gives you the same traversable structure without a graph platform. Theme discovery by community detection does need one, because it runs a clustering algorithm over a projection of the whole graph.

### What is the "table of contents" shape for agent context?

It is a containment tree over a document corpus — library, folders, documents, then nested sections — where each node has a hierarchical address built from its path, plus link edges that cross the tree where one document references another. It lets an agent walk the structure instead of only searching it, and because the addresses are hierarchical, a prefix filter scopes retrieval to a branch with no graph traversal at all.

## Further reading

- [AI on Your Lakehouse: Context Comes in Shapes, Not Queries](https://www.youtube.com/watch?v=kRkcNOsRyYg) — Zach Blumenfeld, Neo4j, AI Engineer workshop (July 2026)
- [From Louvain to Leiden: guaranteeing well-connected communities](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6435756/) — Traag, Waltman & van Eck, *Scientific Reports* 9:5233 (2019)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://arxiv.org/abs/2404.16130) — Edge et al., Microsoft Research (2024)
- [Build a Semantic Layer from GCP with Neocarta](https://neo4j.com/blog/genai/build-a-semantic-layer-from-gcp-with-neocarta/) — Neo4j Labs
- [Search is not memory](/blog/search-is-not-memory) — the same gap, diagnosed from video
- [Why agentic systems need ontologies](/blog/why-agentic-systems-need-ontologies) — the reasoning half of shape 1
- [Context engineering for AI agents](/blog/context-engineering-for-ai-agents) — what to put in the window once you have the structure
- [MDflow MCP docs](/docs/mcp) · [API docs](/docs/api) · [FAQ](/faq)