AI Code Verification: The New Bottleneck for Agents
Everyone is optimizing the wrong half of the loop. The models write code faster every quarter; the humans and tools that decide whether that code is any good have barely moved. That gap is where the value leaks out.
Tariq Shaukat, CEO of Sonar, made the case bluntly in his AI Engineer talk In the Land of AI Agents, the Verifiers Are King. His company sells code verification, so read the pitch with that in mind — but the evidence he stacks up comes from outside his company, and it is uncomfortable.
TL;DR — AI code verification, not generation, is now the bottleneck. Frontier models can attempt long tasks but only at roughly coin-flip reliability, and the code they emit still carries bugs, security issues, and excess complexity. A Carnegie Mellon study found the velocity boost from AI coding tools disappears in about two months while a ~30% rise in static analysis warnings and a ~41% rise in complexity persists. The fix is to make verification part of the loop rather than a gate at the end — and half of that is preemptive: giving agents written constraints, not just context. MDflow is a home for those written constraints, reachable by every agent over MCP.
What is AI code verification?
AI code verification is independently checking that agent-generated code is correct, secure, and maintainable — not merely that it runs. It is the discipline that answers the question "the tests pass, but should this ship?"
The distinction that makes it necessary is functional correctness versus everything else. Sonar's LLM code-quality leaderboard runs thousands of identical Java assignments through dozens of models and puts every line through SonarQube. On functional correctness, the models do well and keep improving. On the rest, the picture is flatter and occasionally inverted: when Claude Sonnet 4 improved its pass rate by 6.3% over Claude 3.7 Sonnet, the share of high-severity bugs in its output rose by 93%. Sonar's summary of the whole cohort is that every model shares "a fundamental lack of security awareness and a bias for messy code."
A better benchmark score means the code more often does the thing. It does not mean the code is safe to keep.
Why the 50% number matters more than the hours
Because the headline capability numbers are measured at coin-flip reliability, and reliability is what production needs. METR's time-horizon work measures the length of task — in human expert hours — that an agent completes with a given success rate. The famous doubling-every-seven-months curve is the 50% curve.
Raise the bar to 80% and the same models collapse to a much shorter horizon. METR found 80% time horizons run 4–6× shorter than 50% horizons, and Shaukat's version of the arithmetic is the memorable one: a frontier model quoted at 16–18 hours at 50% success is somewhere near three and a half hours at 80%. As one of his customers' CTOs put it, an employee who was right 80% of the time would still be having a conversation about it.
So the capability is real and the reliability gap is also real. Every hour of unreliable output is an hour of verification work that somebody has to absorb.
The evidence: velocity up, quality down, then velocity back down
The productivity gain from AI coding tools is real, large, and temporary — and what replaces it is technical debt. This is the most important result in the talk, and it is not Sonar's own data.
Hao He, Courtney Miller, Shyam Agarwal, Christian Kästner, and Bogdan Vasilescu at Carnegie Mellon ran a difference-in-differences study, Does AI-Assisted Coding Deliver?, on 807 open-source repositories that adopted Cursor against 1,380 matched controls, scoring code with SonarQube each month. The shape:
| Metric | Effect |
|---|---|
| Lines added, month 1 | +281.3% |
| Commits, month 1 | +55.4% |
| How long the velocity gain lasted | ~2 months, then gone |
| Static analysis warnings | +29.7%, persistent |
| Code complexity | +40.7%, persistent |
| Doubling code complexity → future lines added | −64.5% |
| Doubling static warnings → future velocity | −50.3% |
Read the last two rows again. The quality damage is not a separate cost you pay later in some abstract "maintainability" currency — it is a direct, measured brake on the very velocity the tools were bought for. The paper's own framing is a feedback loop: warnings and complexity are major drivers of the long-term slowdown.
The industry-scale data agrees. The 2025 DORA report found AI adoption finally correlating positively with throughput — a reversal from 2024 — while still correlating negatively with delivery stability. More change, less stable delivery, unless the control systems underneath are strong. DORA's own summary is that AI is a mirror and a multiplier: it amplifies whatever discipline you already had.
Guide, verify, solve
Shaukat's framework wraps the generation step in three disciplines. Strip the branding and it is a sensible decomposition.
1. Guide — which is preemptive verification
The cheapest bug to verify is the one the agent never writes. Guidance splits into two things that are usually conflated:
- Context — what your codebase is. Architectural awareness, module boundaries, semantic navigation, how a million (or a hundred million) lines fit together.
- Constraints — what you will and will not accept. Coding standards. Dependencies that are approved and dependencies that are banned. Security guardrails. Review criteria. And, critically, the architecture you are trying to move toward, not just the one you have.
Shaukat's observation is that the industry talks about context constantly and about constraints almost never. In Sonar's testing, supplying both cut token consumption for a given problem by over 30% — the agent stops exploring branches you were never going to accept.
That asymmetry is worth sitting with. Codebase context is derivable — tools can index a repo, build a call graph, generate a map. Constraints are not derivable. "We are migrating off this ORM." "No new dependencies without a security review." "This module is frozen; route changes through the adapter." None of that is in the code. It exists in people's heads, in Slack threads, and in the reviewer who catches it on the fourth attempt.
An agent cannot infer a rule that was never written down. It will produce code that is plausible, tested, and quietly against policy — and the cost lands in review, where the bottleneck already is.
2. Verify — zero-trust and multi-layered
Do not let the model that wrote the code be the only judge of it. Two principles:
Zero trust. Every model has biases and, as Sonar's leaderboard shows, a distinct coding personality. Verifying Claude's output with Claude checks for the mistakes Claude is inclined to notice. Use different models and different techniques.
Multi-layered. Combine algorithmic verification — data flow, control flow, taint analysis, secret scanning, known vulnerability patterns, deterministic and cheap — with agentic verification, which is the only thing that can look at intent and business logic and the unknown unknowns. Neither covers the other's blind spot. Sonar reports that customers running a multi-layered approach see 44% fewer AI-derived production outages; that is vendor-reported and unaudited, so treat the direction as more credible than the decimal.
3. Solve — verified code maintenance
Technical debt is now generated at the same rate as code, so paying it down has to be a continuous process too. The interesting argument here is the self-interested one: clean code is not just a human preference. Agents have to read the codebase before they can operate on it, and Sonar's measurements show meaningfully fewer tokens and less reasoning needed to complete identical tasks on a cleaned codebase than on a typical one.
That closes the loop with the Carnegie Mellon result from the other direction. Complexity slows agents down the same way it slows people down — it just shows up on the invoice instead of the calendar.
The three loops
The framework nests: an inner agentic loop where the agent gets context, constraints, and in-loop verification while it works; a CI verification loop where pull requests get algorithmic plus agentic review against quality gates and evals; and a code maintenance loop that keeps the substrate clean. Shaukat's claim is that these compound — in one test with a large bank, a guide-verify-solve setup produced a 92% reduction in issues over a session, not because any single pass is 92% better but because each loop feeds the next. Compounding runs the other way too: neglect verification and the same structure produces the downward spiral the Carnegie Mellon data describes.
Which teams benefit most
- Large existing codebases. Greenfield projects have no architecture to violate. A 10-year-old repo has an intended architecture, a deprecated half, and a dozen rules that only exist in the head of whoever has been there longest.
- Regulated and security-sensitive work. Finance, health, infrastructure — where "the tests passed" is not an answer to an auditor, and banned-dependency policy is a compliance artifact.
- Teams that just turned on agentic coding. They are inside the first two months of the Carnegie Mellon curve right now, and the numbers look fantastic.
- Platform and DevEx teams. They own the loops. Whether verification is designed in or bolted on is their call, and it is being made by default in most organizations.
- Anyone running more than one coding agent. Every agent that does not read your constraints is a separate source of the same violation, discovered separately in review.
How MDflow fits
MDflow does not verify code. It runs no static analysis, no security scanning, no evals. That belongs to SonarQube, Semgrep, CodeQL, your test suite, and your review agents.
What MDflow is good for is the guide half — and specifically the half of that which nothing can derive for you: the constraints, written down, in a place every agent can read.
What already lines up today
Constraints as plain markdown, not tool configuration. Coding standards, architecture decision records, dependency policy, migration plans, "definition of done" checklists, and review criteria are prose. They are written by people, argued over by people, and read by both people and models. MDflow stores them as plain markdown files with no proprietary format between the author and the agent.
Folder descriptions tell the agent what a rule set is for. Every folder carries a description stating what belongs inside it, and mdflow_get_context ranks those descriptions ahead of folder names and document titles before returning matching markdown bodies. A folder described as "Architecture decisions for the payments service — every ADR here is binding for new code in that module" is a retrieval signal you authored, not one a model guessed. That is why folder descriptions matter more than file names.
Workspaces scope the rules to the codebase. One workspace per product line or client keeps the payment service's constraints from leaking into the marketing site's agent.
Every agent reads the same rules. The same store is reachable from Claude, ChatGPT, Cursor, Codex, and Claude Code over the remote MCP server with OAuth or a Personal Access Token, from CI and scripts over the REST API, and from the editor through the VS Code extension. One canonical set of constraints beats five drifting copies of AGENTS.md.
Version history makes a rule change auditable. Line-by-line diffs and non-destructive restore, plus a Document Log that names the actor on every write — including automated · <token name> for API and MCP writes. When a constraint changes, you can see when and by whom. (Version history is a Pro feature and is private to the document owner; it is not exposed over the API or MCP.)
Where the source already lives, stay there. The GitHub integration lets you edit markdown that lives in your repositories directly from a workspace and commit back, so specs and ADRs stay next to the code they constrain rather than drifting in a wiki.
What MDflow does not do. No static analysis, no security scanning, no evals, no quality gates, no CI integration that blocks a merge. It holds the written half of the loop; the verification half is other people's tools, and should be.
Where we are headed
Direction, not a dated commitment. The frontier we find interesting for a markdown-native store is making constraints legible as constraints rather than as prose that happens to contain rules — descriptions that can carry authority and freshness, a way to see which documents an agent actually read before it wrote something, and scoped tokens so an agent's reach narrows to the slice its job needs. Knowing what an agent read is the missing input to knowing why it wrote what it wrote.
The bottom line
The generation half of the loop is improving on its own; you do not need to do anything for it. The verification half only improves if somebody designs it — and the Carnegie Mellon numbers are what happens when nobody does: two great months, then a permanent 30% more warnings and 40% more complexity, which then quietly eat the velocity that started the whole thing.
Verification splits into a part that is genuinely hard and a part that is merely neglected. The hard part is multi-layered, zero-trust checking, and it needs real tools. The neglected part is preemptive: writing down the rules the agent is going to break, in a place it can read them, before it writes a line.
That second part is not a tooling problem. It is a documentation problem that has been waiting for a reader patient enough to care.
Start free · Connect an AI agent · Read the API docs
Frequently asked questions
What is AI code verification?
AI code verification is the practice of independently checking that code produced by an AI coding agent is not just functionally correct but also secure, maintainable, and consistent with your architecture and standards. It combines algorithmic analysis — data-flow and control-flow checks, secret scanning, known vulnerability patterns — with agentic review that looks at intent and business logic. The point is that passing tests proves the code does something, not that it does the right thing safely.
Why is verification the bottleneck for AI coding agents?
Because generation got dramatically faster and review did not. Models can now attempt tasks that would take a human many hours, but their reliability drops sharply as the accuracy bar rises — METR measures 80% time horizons at 4–6× shorter than 50% ones — and the code they emit still carries bugs, security issues, and excess complexity. When output volume rises and the review capacity behind it stays flat, the queue moves to review, and unverified code accumulates as technical debt rather than shipped value.
Do AI coding agents actually create technical debt?
Measurably, yes. A Carnegie Mellon study of 807 repositories that adopted Cursor, matched against 1,380 control repositories, found a 281.3% jump in lines added and a 55.4% jump in commits in the first month, with the velocity gain gone after roughly two months. What persisted was a 29.7% increase in static analysis warnings and a 40.7% increase in code complexity — and the paper links that complexity back to slower future development, with a doubling of complexity associated with 64.5% fewer lines added later.
What are context and constraints for AI coding agents?
Context is what your codebase is: architecture, module boundaries, how the pieces relate. Constraints are what you will and will not accept: coding standards, approved and banned dependencies, security guardrails, and the architecture you intend to move toward. Most teams document context and leave constraints in people's heads, so agents write plausible code that quietly violates rules nobody ever wrote down. Context is derivable from the repository; constraints are not.
How does MDflow help with AI code verification?
MDflow does not verify code and does not run static analysis. It is a place to keep the guide half of the loop — the written constraints an agent needs before it generates anything. Coding standards, ADRs, dependency policy, and review checklists live as plain markdown, each folder carries a description that says what it is for, and mdflow_get_context ranks those descriptions first when an agent asks. The same store is reachable from Claude, ChatGPT, Cursor, and Codex over MCP or the REST API, so every agent reads the same rules.
Further reading
- In the Land of AI Agents, the Verifiers Are King — Tariq Shaukat, Sonar (AI Engineer, July 2026) — the talk this post is built on.
- Does AI-Assisted Coding Deliver? A Difference-in-Differences Study of Cursor's Impact on Software Projects — He, Miller, Agarwal, Kästner & Vasilescu (Carnegie Mellon), MSR '26. The transient-velocity, persistent-debt result.
- Measuring AI Ability to Complete Long Software Tasks — METR. Where the 50% vs 80% time-horizon gap comes from.
- The Coding Personalities of Leading LLMs — Sonar. Functional correctness rising while security and complexity do not.
- Balancing AI tensions — the 2025 DORA report — throughput up, delivery stability down.
- MDflow: folder descriptions as agent context, AGENTS.md, CLAUDE.md, SKILL.md and llms.txt, spec-driven development with markdown specs, the agentic coding loop, and connecting an agent over MCP.