skip to content
$cat prose-engineering-for-ai-agents.md

Prose Engineering: Your Agent Is Mostly Not Code

19 min readby MDflowview as .md
Emerald wireframe document pages whose paragraph lines turn into circuit traces feeding a glowing verification lattice, with some traces fracturing, on a dark terminal-grid background

Coding agents made code cheap. They did not make understanding it cheap, and they did not make the other half of the system cheap either — because in a real production agent system, the larger half is not code at all. Prose engineering is the missing discipline: treating the natural-language part of an agent system — its instructions, policies, decision tables and procedures — the way we have treated source code for forty years, with review, linting, coverage, mutation testing and deliberate deprecation.

The sharpest field report on this came from Dmitry Buykin of Maersk, in an AI Engineer talk published on 11 November 2026. The system in question runs in about 110 countries across roughly 10,000 scenarios of prose instructions; the worked example throughout the talk is someone asking to change the amount on an order that has already been approved. By Buykin's own estimate, about three-quarters of the system is prose. And prose, as the talk puts it, has no compiler.

TL;DR — In production agent systems, most of the behaviour lives in natural language, not code, and nothing fails to build when that language stops making sense. Prose engineering closes the gap: lint every changed instruction file like a source file, mutation-test instructions to find which ones are load-bearing, promote a path to code only once it repeats and agrees, and retire prose with the tests that kept it alive. All of it assumes one thing — a single, durable, expert-readable copy of the prose. That is what a markdown workspace like MDflow is for. Start free.

What prose engineering is

Prose engineering is the practice of applying code-grade discipline to the natural-language artifacts that determine an agent's behaviour. The term is ours; the idea is Buykin's, stated in the talk as a flat necessity — prose needs engineering too, because most of the system is prose.

That phrasing is worth sitting with, because it inverts how most teams think about agent work. The mental model is usually "we write code, and there is a prompt attached to it." In a mature system it is closer to the reverse: there is a large, structured body of written procedure, and some code that fetches facts for it and executes what it decides. Two kinds of agents meet in such a system — one that wrote the instructions, and one that follows them in production. The instructions are the product.

The moment you accept that, every question you already ask about code becomes a question about prose. Who reviews it? What breaks if we change it? Which lines are dead? How do we know a change landed? None of those have answers by default, because the tooling that answers them for code does not exist for paragraphs.

Prose has no compiler

Nothing in a body of agent instructions fails to build. A contradictory rule, a dead clause, a sentence that disagrees with the code next to it — all of them parse perfectly, because there is no parse.

The talk gives the clean example. The prose says the approval limit is 1,500. The code enforces a different number. The same order is fine in one place and too high in the other, and nothing throws an error. There is no type system to catch it, no test that fails, no build that goes red. It simply produces a wrong answer somewhere in one of 110 countries, at a rate nobody is measuring.

Buykin's team borrowed the obvious instrument from software testing: mutation testing. Break one sentence on purpose, and watch for a red test. The result is the finding that should make every team reading this uncomfortable — almost none turned red. The prose was carrying the behaviour, and the test suite was not watching the prose.

That is not an isolated observation. A NeurIPS 2026 workshop paper by Dipankar Sarkar, Which Prompt Mutations Can an Agent Actually See?, set out to measure coverage over what it calls the agent policy layer. Hand-annotating 12 of 40 real-world agent skill prompts, byte-exact to the originals, it found them rule-dense but almost branch-free: 455 rules against 9 template-expressible conditionals and no loops.

That ratio explains why ordinary tooling does nothing here. Code coverage works by walking branches. Agent prose is hundreds of rules with almost no branches, so there is nothing for a branch-coverage tool to walk. The paper's response is to define the metrics that do fit — rule, condition/branch and rule-pair coverage — along with six mutation operators and a static check for rules that a higher-priority sibling silently shadows.

There is a second-order problem too, which the talk names and which has no analogue in code at all: a negative instruction is not free. Even a line like do not call X teaches the model that X exists. Prose you add to prevent a behaviour can introduce it.

What a prose compiler actually looks like

You cannot buy one. You assemble it out of four moves, three of which Buykin's team built and one of which the research has formalised.

1. Lint every changed prose file into a ledger, exactly like code. Not a style check — a consistency check against the rest of the system, run on every change, with the result recorded. This is the cheapest of the four and the one most teams skip entirely.

2. Mutate instructions one at a time to find out which ones are load-bearing. The practitioner write-up Prompt Mutation Testing by Tian Pan (May 2026) gives the clearest operator set: for each instruction, generate four mutants — deletion, negation, dilution (changing must to should), and displacement (moving the instruction to the opposite end of the prompt) — then score compliance against adversarial inputs written to probe that specific rule. The vocabulary it lands on is the useful export: instructions that survive perturbation without affecting behaviour are decorative; instructions that break things when touched are load-bearing. Most teams have never found out which of theirs are which. (The post is practitioner writing and most of its numbers are unsourced, so take the method and leave the percentages.)

3. Mutate the prose to grade the tests, not only the prose. Test-Driven AI Agent Definition (TDAD) by Tzafrir Rehan (arXiv, 9 March 2026) compiles agents from behavioural specifications and guards against the obvious failure — a test suite that passes because it is weak. Among its three mechanisms is semantic mutation testing via a post-compilation agent that generates plausible faulty prompt variants, with the harness measuring whether the test suite detects them. That inverts what is under test: the mutants are prose, but the thing being scored is the suite. On evolved specifications it reports mutation scores of 86–100%.

4. Make every new check fail once on purpose before you trust it. This is Buykin's rule and it is the one that costs nothing. A check that has never been seen to go red is not a check; it is a decoration with a green light attached.

Two warnings come with the set. Lint cannot stop two copies of a procedure drifting apart — only having one copy can. And the generator should not grade itself: in the Maersk system a second agent reviews the first's work from a checklist, without the author's reasoning, precisely so it cannot be talked into agreeing. Over six months that review grew into most of the checking. (We have written about why separately, in the generator cannot be the validator.)

The prose/code boundary moves in both directions

The useful rule from the talk: judgment stays wherever the answer varies, in a place an expert can read. Everything else is a candidate for code.

Buykin's team operationalised this with what they call a walk — the path one agent takes through the prose on a given task. When agents keep walking the same path, that path becomes a workflow tool, and the prose that described it is deleted. The qualifier matters: a walk becomes code only when it repeats and agrees across countries. What still varies stays in prose.

Their first attempt at this failed in an instructive way. They built the workflow tools, but left the prose naming the steps inside them — so the agents ran those steps again. The fix was to strip the prose back to decisions only, and let the tools own the mechanics.

The direction most teams never consider is the other one. That same team moved hundreds of lines of expressions out of code and into prose, because the domain experts could not read them. A regex is fine for spotting one critical signal; a wall of them is a rule set that has been hidden from the only people who can tell you whether it is right.

What settles out is a clean division: code prepares the facts and carries out the decision, experts own the decision table, the model picks a row, and if no row fits, a person takes over. That is a far more disciplined structure than "the agent decides," and it puts the varying part where it can be audited. It is the same instinct behind behavior specs and spec-driven development: write the opinion down somewhere a human can argue with it.

One document, two readers

Agent prose has to satisfy two audiences at once, and they do not want the same thing. The agent reads the blocks, so the blocks have to be exact. The expert reads the text around them, and signs it.

That is the design constraint behind the single-copy rule. Buykin's team keeps one copy and describes a second copy as a source of drift. The everyday version of that failure needs no agents at all: a procedure that exists as a wiki page, a repo file and a pasted prompt is three procedures, and at least two of them are wrong.

It is also why the guidance to domain experts in that system is as short as it is: business tells us what, we work out how. Experts cannot specify mechanisms they cannot see, and developers cannot invent policy. Each expert reviews a task exactly the way the agent saw it — same inputs, same context — which is the only review that means anything. On the agent's side, context discipline is the mirror image: when the model was allowed to choose what to load, it loaded everything, and then invented a pricing check because prices happened to be visible. Whatever sits in the context becomes a reason to act, a point we have made at length in context engineering for AI agents.

The memory split that falls out of this is worth stealing on its own. Agents forget everything when a session ends, so the memory has to live in files — and the team keeps two kinds: what happened, and what is true now. The next session starts from what is true now. That is exactly the distinction between process data and state data, and conflating the two is how agents end up re-litigating decisions that were settled months ago.

Retiring prose is the part everyone skips

Left alone, an agent keeps adding conditions until today's tasks pass. This is the entropy problem, and it is the reason the talk exists: the team's first response was a full rewrite, and the rewrite did not hold. The agents kept writing and the mess grew back.

The replacement is a weekly sweep with three pieces worth copying:

  • A ratchet. Gauges whose count can only go down. Not a target — a constraint.
  • Evidence before removal. Anything no real task reaches gets a name and its evidence written down before it goes. "No evidence" is a stated verdict, not a shrug.
  • A retired variant takes its test with it. This is the subtle one. Tests were keeping dead variants alive: the agents kept every option that a test still mentioned, because the test was proof the option mattered. Deleting the variant and leaving the test is how dead prose regenerates.

The honesty gauges around it are just as transferable. Before building one new check, the team replayed its rule over several thousand past sessions and found it would have fired on less than 1% of them. By Buykin's account the agents spend something like nine-tenths of their time on a change just checking it, and the ledger exposed the failure mode that makes all of this necessary — coding agents finished most of a batch and quietly skipped the rest, especially when context filled up. Most items looked covered; roughly a quarter could actually be closed.

And the first gauge, in the talk's telling, was a person: when the lead was too tired to explain what had changed, that was the signal to stop and fix the process, even with a green dashboard. Which is a good description of why prose engineering is worth building at all — not to go faster, but so the system does not run through one head.

Which teams need prose engineering most

  1. Regulated operations at country scale. Approvals, limits, eligibility, KYC — anywhere the same procedure varies by market and local law, and the variation is the point.
  2. Agentic customer operations. Refunds, claims, exceptions and escalations, where the policy is written by a non-engineer and executed thousands of times a day.
  3. Coding-agent platforms. Teams whose instruction files, skills and review checklists have quietly become the largest unreviewed artifact in the repository.
  4. Anything with a decision table. Pricing, routing, risk scoring, entitlements. If a model is picking a row, the table is production code written in prose.
  5. Multi-team agent estates. The moment two teams maintain overlapping procedures, drift is the default and the single-copy rule becomes load-bearing.

How MDflow fits

Every practice above assumes something that is easy to say and hard to arrange: one durable copy of the prose, readable by agents, editable by the people who know whether it is right. That is the layer MDflow occupies.

What lines up today

MDflow stores everything as plain markdown — the format agent instructions, runbooks and decision tables are already written in. There is no export step between a document in a workspace and something an agent reads.

Folder descriptions are the piece that maps most directly onto the boundary question. A folder's description states what belongs inside it, and MDflow's retrieval ranks those descriptions as the primary signal, above folder names and document titles. That is a natural place to scope a body of procedure: this folder holds the approval limits and escalation rules for European road freight. Topic retrieval is an MCP tool — mdflow_get_context — while the HTTP API gives an agent the explicit route instead: list folders, list documents, fetch a body, with a Personal Access Token.

Version history (Pro) is what makes retirement reversible. If a weekly sweep deletes a clause and a real task turns up three weeks later that needed it, you want the line-by-line diff and a one-click restore rather than an archaeology session. Worth stating plainly: version history is private to the document owner and is deliberately not exposed through the API or MCP server, so an agent cannot read or rewrite a document's past.

Comments are the quiet fit for the two-readers problem. Share a document — publicly by link, or privately to specific people by email — with commenting turned on, and a reviewer can attach a comment to a selected passage in the read-only markdown source. The owner reads and moderates those comments from the normal editor. A domain expert who thinks a limit is wrong says so against the exact paragraph without editing it, which is the review model the talk describes: experts read the text and sign it rather than rewriting the blocks the agent depends on.

The Document Log answers the other half: every create, edit, share and delete is recorded with its actor, and anything arriving through the API or MCP server is marked as automated with the token name. When the question is which agent changed this procedure, and what exactly did it change, that is the feed.

What MDflow does not do, and we will not imply otherwise: it does not lint your prose, does not run mutation operators, does not compute rule coverage and does not detect drift. There is no prose compiler in it. It is where the single copy lives — not the instrument that tells you the copy has gone wrong. Build that in your eval harness; keep the prose somewhere a human can edit it and an agent can read it.

Where we are headed

Direction rather than a dated commitment, and deliberately vague about timing because none of it is scheduled. The folder description is the primitive we keep coming back to, and the gap between what a folder is for and what an agent retrieves from it is where we think the next useful work is. The other thread is the round trip: MDflow already edits markdown in place in GitHub, Google Drive and OneDrive, and agent instructions increasingly live in repositories, so narrowing the distance between where prose is version-controlled and where it is edited is a gap worth closing. Treat both as interests, not commitments.

The bottom line

The uncomfortable implication of the Maersk talk is not that agents write bad code. It is that the thing agents are actually executing — the policy, the procedure, the written judgment of your domain experts — has been shipping for years with none of the discipline we would insist on for a hundred lines of Python.

Prose engineering is the correction. Lint it. Mutate it to find out which lines are load-bearing. Promote a path to code only when it repeats and agrees, and move rules back into prose when the experts can no longer read them. Make every check fail once on purpose. Retire a variant with its test. And keep one copy, because two copies is a drift generator with extra steps.

None of that works if the prose is scattered across a wiki, a repo and a pasted prompt. Put it somewhere single, versioned and agent-readable first; the discipline is much easier to add afterwards.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

What is prose engineering?

Prose engineering is the practice of applying code-grade discipline to the natural-language parts of an agent system — the instructions, policies, decision tables and procedures the model actually follows. That means review, linting, coverage, mutation testing and deliberate deprecation, applied to prose rather than to source files. It matters because in production agent systems the prose is often the larger half of the system, and unlike code it has no compiler to tell you when it has stopped making sense.

Why can't you just test agent instructions the way you test code?

Because there is almost no structure in them to cover. A NeurIPS 2026 workshop paper that hand-annotated 12 of 40 real-world agent skill prompts found them rule-dense but almost branch-free — 455 rules against 9 template-expressible conditionals and no loops. Code coverage works by walking branches; prose is hundreds of rules with barely any branches, so conventional coverage tooling has nothing to latch onto and you need rule-level metrics instead.

What is prompt mutation testing?

Prompt mutation testing perturbs one instruction at a time and checks whether behaviour changes. Common operators are deletion, negation, dilution — turning must into should — and displacement, moving the instruction to the other end of the prompt. Instructions that survive perturbation without changing behaviour are decorative; instructions that break things when touched are load-bearing. The point is to find out which of your instructions are actually doing work.

When should a piece of agent prose become code?

When the path through it repeats and agrees across cases. In the Maersk system described at AI Engineer, a repeated path through the prose gets turned into a workflow tool and the prose that described it is deleted — but only when it repeats and agrees across countries. Anything whose answer still varies stays in prose, in a place a domain expert can read. The boundary moves both ways: that team also moved hundreds of lines of expressions out of code and into prose because the experts could not read them.

Does MDflow test or lint agent prose?

No. MDflow is a markdown workspace, not a linter or an evaluation harness — it has no mutation operators, no coverage metrics and no drift detection. What it provides is the single durable copy that prose engineering assumes: plain markdown that agents read over MCP or the HTTP API, folder descriptions that scope what a set of documents is for, version history on Pro so you can see what a procedure said before it was edited, and passage-anchored comments on shared documents so a domain expert can question a line without rewriting it.

Further reading