$cat generator-cannot-be-the-validator.md

The Generator Cannot Be the Validator

November 2, 202615 min readby MDflowview as .md

An emerald wireframe robotic arm on the left extruding a stream of identical lattice cubes, and a separate caliper-and-gauge mechanism on the right measuring one of them while rejected cubes fall away, on a dark terminal grid background

A thousand enterprises have each spent over a million dollars rolling out agentic coding this year — a figure Snyk's CTO put on stage at AI Engineer in July, from his own customer base. The tooling works, the velocity is real, and the budget is already committed. What almost nobody has an answer for is the question that comes immediately after: can you trust what your agents just shipped, and can you reconstruct how they did it?

The reflexive answer is to point another model at the output. That instinct is the thing worth examining, because it quietly collapses two roles that every mature engineering discipline keeps apart.

TL;DR — Generator validator separation is the architectural decision agentic systems turn on: whatever produced an artifact must not be the only thing that judges it. Asking the same model to re-check its own work is not a second opinion — Snyk's VulnBench JS 1.0 found that across five identical repetitions, 80 of 161 model-only findings showed up in just one run. The findings that were stable were the ones a deterministic tool already had. Independence has to be structural: a different process, a different provenance, or a different objective — and all three need a written standard to check against, which is what a markdown workspace like MDflow holds.

What generator/validator separation means

It means the system that produces an artifact is not the system with final authority over whether that artifact is acceptable.

The framing comes from Manoj Nair, Chief Innovation Officer and CTO at Snyk, opening the first security track at AI Engineer in July 2026 with a talk titled Through the AI Fog: The Architectural Decision Agentic Security Depends On. His one-line version: can the generator and the validator be the same? His answer, drawn from 18 months of watching roughly 5,000 enterprise customers adopt agents, is no.

Separation of duties is not a new control. Accounting has had it for centuries, and application security has had reviewer-is-not-author rules for as long as there have been code reviews. What is new is that the rule is now contested by default, because a single model is fluent enough at both jobs that collapsing them no longer looks like a shortcut. It looks like efficiency.

Independence is the whole point, and it comes in three usable flavours:

Different process — a deterministic check against a probabilistic generator. Linters, type systems, static analysers, schema validation, tests. These fail in boring, repeatable ways, which is exactly their value.
Different provenance — a different model, a different vendor, a different prompt lineage. Weaker than the first, because two frontier models trained on overlapping corpora share more blind spots than anyone would like.
Different objective — a check that optimises for something the generator was not asked about at all: supply-chain health, data egress, policy conformance, cost.

The failure mode this guards against is not the model being bad. It is the model being consistently bad in a way it cannot see, and then being asked to look.

The repeatability evidence

The sharpest argument against self-validation is not philosophical, it is measured: ask the same model the same question five times and you get five different answers about what is wrong with your code.

Snyk published the study behind that claim as Snyk VulnBench JS 1.0: Can LLMs Find the Same Bugs Twice? (Tal, Kloos, Rudich, Thoemmes & Nair, June 2026). The design is deliberately boring: 300 scans in total — 10 small JavaScript fixture projects, six configurations including Snyk Code SAST and several Claude setups, five repetitions each. Same code, same prompt, same harness, every time.

The headline result is that repeatability was uneven, and the unevenness lands in the worst possible place:

Across 250 model runs, 80 of 161 unique unmatched findings appeared in only one of the five identical repetitions. Only 22 appeared in all five.
By contrast, when a model matched a finding the deterministic scanner already had, it was stable: 134 of 158 reference-matched findings (84.8%) appeared in all five runs.
Snyk Code SAST reproduced its reference set deterministically, with zero variation across runs.

Read those two bullets together, because the combination is the interesting part. The model is most reliable exactly where you need it least — re-confirming what a deterministic tool already flagged — and least reliable exactly where you need it most, on the novel findings nothing else caught. The scenario everyone imagines when they say "we'll have an LLM review it" is the second category. That is the category where a single run is close to a coin flip on whether you ever hear about the issue.

Two caveats, because the paper states them and a post that omitted them would be doing the same thing it is warning about. First, the F1 figures in the study — the best LLM configuration reached 75.4% against the Snyk Code reference — are not an accuracy measure. The reference set is Snyk Code's own output, so the deterministic tool scores 100% against itself by construction, and Snyk says plainly that this should not be read as true vulnerability-detection accuracy. Second, it is a vendor's benchmark on ten small fixtures, with Nair as a co-author of both the paper and the talk.

What survives those caveats is the part that does not depend on the reference at all: run-to-run variance on identical inputs. You do not need a ground truth to observe that a system gave you a different answer five times in a row.

And the paper's own conclusion is not "determinism wins". It is complementarity. Models consistently found familiar, high-signal exploit shapes, and in one case surfaced a likely gap in Snyk Code itself; the deterministic scanner was better at systematically enumerating repeated data-flow sinks. The recommendation is to combine the two, which is just generator/validator separation restated: keep both, and do not let either one be the sole authority.

Why a newer model is not the fix

Model risk is not a single ranking, so "use the better model" is not an answer to a question about safety.

Nair showed red-team results from Snyk's own attack tooling, run against newly released models as they ship. Two frontier models resisted PII-extraction attacks completely — zero successful extractions with Snyk's attacks, an improvement on their own earlier behaviour. A widely discussed open model leaked under the same attacks 100% of the time. Straightforward enough, until the next test: on decision override, the ordering inverted. The frontier models did worse; the open model held at zero.

Treat the numbers as vendor red-team data on a specific attack suite, not as a leaderboard. The structural point stands regardless of the exact figures: if which model is safest depends on which attack you are facing, then no single model can be the thing you trust, and "we upgraded to the newest model" is not a control. It is a change that needs re-validating, by something that is not the model.

The VulnBench paper makes the same point about stability rather than safety, and it is the cleaner evidence because it does not depend on a red team. In its words, the model configurations "do not line up by cost or recency": Opus 4.6 at medium and high reasoning sat close together with low variance and the strongest model F1, while the newer Opus 4.7 Max and Sonnet 4.6 High moved more across their repeated runs. Recency bought variance, not reliability.

This is also why the validator should be chosen per threat class rather than per benchmark. A type checker does not have opinions that drift when a new checkpoint ships. See model-agnostic AI architecture for the same argument applied to the rest of the stack.

You cannot govern what you do not know exists

Separation only helps if you know what is in the system to begin with, and most teams do not.

Two numbers from the talk, both from Snyk's customer telemetry and both worth taking as directional rather than precise. Across 3,000-plus customers, for every model found in a repository there were roughly three times as many agentic components — agents, tools, and the things they call. And across 4,800-plus customers, the security backlog grew 108% quarter over quarter: not a flat vulnerability rate multiplied by more code, but a backlog roughly doubling, in the same period everyone adopted the agents that were supposed to help.

The inventory problem has a specific shape in agentic systems, because the dependencies are no longer just packages:

Skills and MCP servers are dependencies now, with far less scrutiny than npm packages get. Snyk's own scan of 3,984 published skills found 13.4% carried a critical issue and 76 contained live malware — we covered that research in the agent supply chain.
The dangerous ones do not look broken. In Snyk's live demo, a plausible competitive-analysis skill echoed an authorization header and pulled its own classification logic from a YAML file hosted on a third-party site — so the skill file could stay byte-identical while its behaviour changed underneath you.
The protocol-level flaws are architectural, not bugs. The GitHub MCP prompt-injection chain disclosed by Invariant Labs in May 2025 — a poisoned public issue steering an agent into leaking private repository contents — was explicitly not a defect in the server's code. It is what Simon Willison calls the lethal trifecta: private data, untrusted instructions, and a way to exfiltrate.
Agents invent surface area on their own. Nair described production agents at large customers making convenience copies of PII into databases nobody had inventoried, for the entirely reasonable-sounding purpose of not having to fetch it again.

The timing pressure is not hypothetical either. In late June 2026 the Five Eyes intelligence partners issued a joint warning that AI is on pace to bypass prevailing cybersecurity practice in months, not years — the sort of statement that is easy to discount right up until the assumption it invalidates is one of yours.

Which applications benefit most

AI-assisted coding at any real volume — where the generator is producing faster than any human reviewer can read, and the temptation to let it self-certify is strongest.
Regulated or audited software — where "a model said it was fine" is not an answer anyone can file, and the trail of who checked what has to survive an auditor.
Agents with write access to production — infrastructure, migrations, deploys, customer data. The blast radius makes independence non-negotiable.
Teams installing third-party skills and MCP servers — an inventory-and-vetting problem that the generating model has no visibility into at all.
Multi-agent pipelines — especially ones where a reviewer agent is a sibling of the author agent, sharing a model and often a system prompt. That is self-validation wearing a costume.
Anything where the model's output is the product — generated reports, summaries, recommendations shipped to customers under your name.

How MDflow fits

Here is the honest version of the connection, because the relevant part is narrow.

A validator needs something to validate against. A deterministic scanner carries its own rules, but everything else — what your organisation considers an acceptable dependency, which data may leave which boundary, what a reviewer must confirm before an agent's change merges, which skills are approved — is written policy. Nair's line about this was the most quotable thing in the talk: AI governance cannot live in a Confluence page or a PDF. Not because wikis are bad, but because a policy an agent cannot load at the moment of decision is not operative, and a policy nobody can diff cannot be audited afterwards.

That written standard is what MDflow is for. What lines up today:

Plain markdown in a nested folder tree, so the standard is diffable text rather than a rendered document. A policy change shows up as a diff a reviewer can argue with.
Folder descriptions state what a branch is for, and mdflow_get_context ranks those descriptions above folder names and document titles — so an agent asking "what are the rules here?" lands in the right place. See folder descriptions as agent context.
MCP, the HTTP API and personal access tokens put the same documents in front of the coding agent, the review agent and the CI job, with every call scoped server-side to the token's owner. 25 MCP tools across documents, folders and workspaces. MCP docs · API docs
Raw .md twins with YAML frontmatter, so the bytes an agent cites are the bytes a human reads — no rendering layer between the policy and the thing enforcing it.
Version history and comments cover the part that matters after an incident: what the standard said on the day the change shipped, and who argued about it. That is also the cleanest separation available between what an agent wrote and what a human actually reviewed.
Team workspaces so the standard is shared infrastructure rather than one engineer's folder.

Where MDflow stops, stated plainly: it does not scan packages, it does not vet skills or MCP servers, it runs no static analysis, and it enforces nothing. It is not a gate and it will not block a merge. It is where the gate's criteria live so that both the agent and the reviewer are working from the same text — which is the half of this problem that most teams have not solved, while buying tools for the half they have.

Where we are headed, as direction rather than a dated commitment: richer structured frontmatter so a document can carry machine-readable fields a pipeline can act on, and better programmatic access to the hierarchy itself. The principle we will not trade away is that the file stays plain markdown you can walk away with.

The bottom line

The agentic stack has made generation nearly free and left validation exactly as expensive as it ever was. The cheap way to close that gap is to ask the generator to grade itself, and it does not work — not because models are bad at review, but because run-to-run variance on identical inputs means a single pass is a sample rather than a verdict, and the sampling is worst precisely on the novel findings you were hoping to catch.

So decide the architecture deliberately. Keep at least one deterministic check in the loop. When a second model is the only option, make it independent in some way that actually costs you something. Know what agentic components you have before you reason about them. And write the standard down somewhere both the human and the agent can read, because an unstated policy cannot be violated — it can only be ignored.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

What does "the generator cannot be the validator" mean?

It means the system that produces an artifact must not be the only system that judges whether the artifact is acceptable. In an agentic pipeline, the model that wrote the code, the plan or the config is structurally the wrong thing to ask whether that output is safe, because it will apply the same assumptions, the same blind spots and the same training to both jobs. Separation of duties is an old control in security and accounting; what is new is that a single model is now plausible enough at both roles that teams collapse them without noticing.

Why can't I just ask the model to review its own output a second time?

Because a second run is not a second opinion. Snyk's VulnBench JS 1.0 study ran the same code, the same prompt and the same harness five times per configuration across 300 scans, and found that model-only findings were unevenly repeatable: 80 of 161 unique unmatched findings appeared in only one of the five identical repetitions, while only 22 appeared in all five. Running the same model again samples the same distribution rather than applying a different standard.

Does this mean LLMs are bad at finding vulnerabilities?

No, and the research is explicit about it. In the same Snyk study, findings that matched the deterministic reference were stable — 134 of 158 appeared in all five runs — models consistently found familiar high-signal exploit shapes, and in one case a model surfaced a likely gap in the deterministic tool itself. The paper's own conclusion is to combine agentic LLM review with deterministic static analysis rather than treat either as a replacement for the other.

Is a newer or larger model the fix?

Not on its own, because model risk is not a single ranking. In red-team data Snyk presented at AI Engineer, frontier models resisted PII-extraction attacks that an open model fell to every time, while on a decision-override test the ordering reversed and the open model held where the frontier ones did not. If the safest model depends on which attack you are facing, no leaderboard position substitutes for an independent check against your own policy.

Where should the rules a validator checks against live?

Somewhere versioned, diffable and readable by both the human reviewer and the agent — which in practice means plain text in your workflow, not a slide deck or a wiki page nobody opens. A policy that only exists as prose in a PDF cannot be loaded into an agent's context at the moment it matters, and a policy nobody can diff cannot be audited after the fact.