---
title: "Why AI Software Factories Fail: The Maintainability Gap"
description: "The lights-off AI software factory fails for a reason no harness can fix: models are trained to pass tests, not to keep a codebase maintainable. What works instead."
author: "MDflow"
date: 2026-07-31
reading_time: "16 min"
canonical_url: https://mdflow.cz/blog/why-ai-software-factories-fail
md_url: https://mdflow.cz/blog/why-ai-software-factories-fail.md
---

# Why AI Software Factories Fail: The Maintainability Gap

*Published July 31, 2026 · 16 min read*


Every company and its mother now claims to have built a coding agent factory that ships most of its code. The narrative is confident: models are good enough, code is free, you are the bottleneck, spend more tokens. And the loudest version of it says the last human step — reading the code — can go too.

At AI Engineer, Dex Horthy of HumanLayer stood up and argued the opposite, from experience. His talk, [*Harness Engineering is not Enough: Why Software Factories Fail*](https://www.youtube.com/watch?v=Ib5GBkD555M), makes a claim that is more uncomfortable than the usual "you're holding it wrong": the failure is not in your scaffolding. It is in how the models were trained, and no amount of loops can prompt it away.

> **TL;DR** — **The lights-off AI software factory fails because coding models are trained to make tests pass, and no benchmark can cheaply punish them for eroding a codebase.** Bad architecture costs weeks or months, which is far too slow a signal to reinforce, so agents optimise for the green check and quality quietly decays. More harness engineering raises the floor but cannot close that gap. What works is moving the human earlier: a product review, a system architecture, a program design and a vertical-slice plan, agreed before the agent starts, so review stays fast and you keep reading the code. Those four artifacts are markdown documents — and [MDflow](https://mdflow.cz) is a place to keep them where every agent can read them.

## What is an AI software factory?

**An AI software factory is a delivery pipeline in which agents, not people, do the building.** Horthy traces the term *software factory* back to a 1968 NATO conference, and the 2022 shape of it is familiar to everybody: work lands in a tracker, someone builds the thing, automated checks run, a human reviews the code, a human tests the change, it ships, users complain, monitoring pages someone at 3am, and around it goes.

The agentic version swaps exactly one box. *Someone builds the thing* becomes *an agent builds the thing*, wrapped in an orchestrator, a harness, a sandbox, a model. Building drops from days to minutes. Everything downstream of it does not.

So you push on the downstream. Agentic code review. Agentic regression testing. Route incidents straight into the factory so nobody gets woken up — they wake up to a pull request instead. Pipe user feedback in as well. Now your only job is how much you can stuff into the queue, and how fast you can review what comes out.

The **lights-off software factory** is the last step of that logic: stop reviewing. Nobody reads the code. Invest instead in testing, monitoring, and rollout, and let volume do the rest.

## Why lights-off software factories fail

**Because models cannot maintain and improve codebase quality over time without human steering, and that limitation is baked in at training time.**

HumanLayer tried it. In July 2025 they went full lights-off on their own product. The result, in Horthy's telling, was months of accumulated slop, at least one issue no amount of advanced prompting could get an agent to solve, and eventually having to dig back into a codebase the team had stopped reading three months earlier — while the site was down and users were angry.

That is one company's anecdote. The industry-scale telemetry says the same thing. Faros AI's [*AI Engineering Report 2026: The Acceleration Whiplash*](https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways), published April 2026 from two years of data across roughly 22,000 developers and 4,000 teams, found throughput and damage rising together:

| Metric | Change |
| --- | --- |
| Epics completed per developer | **+66%** |
| Task throughput per developer | **+33.7%** |
| Bugs per developer | **+54%** |
| Incidents per pull request | **+242.7%** |
| Pull requests merged with **no review** | **+31.3%** |
| Median time in review | **+441.5%** |

Read the last two rows together. Review is slower *and* more code is bypassing it entirely — which is exactly what happens when generation speeds up and the human capacity behind it does not. We covered that squeeze from the verification side in [*AI Code Verification: The New Bottleneck for Agents*](/blog/ai-code-verification-loop). Horthy's talk explains why the squeeze cannot simply be automated away.

### The training argument

Here is the mechanism. To make a coding model better at a task, you generate many attempts at a problem, score them, and reinforce the behaviour that scored well. On a benchmark like SWE-bench Multilingual, the score is binary: did the hidden test pass, and did you avoid breaking the other tests? Test-file edits are reverted first, because models are perfectly happy to comment out a failing test.

Nothing in that reward function can see program design. There is no channel through which "this works, but it added a fourth way of doing the same thing" becomes a penalty. So you get the artifacts everyone recognises: a try-catch wrapped around something that should never have thrown, a cast bolted on to silence a type error, a special case bolted on beside three existing special cases. The tests are green. The model was rewarded.

Horthy's framing is the sharp one: **tests give feedback in seconds, but the cost function of bad architecture is measured in weeks, months, maybe years.** Reinforcement learning cannot propagate a reward signal across that gap. It is not that labs do not care about maintainability — it is that maintainability has no cheap verifier.

### The benchmarks are starting to notice

The frontier is moving, slowly, and the new benchmarks are the interesting evidence — because they show how far from solved this is once you stop scoring "the test passed."

- **[FrontierCode](https://cognition.com/blog/frontier-code)** (Cognition, June 2026) asks whether a maintainer would actually merge the change. It grades regression safety, test quality, scope discipline, style, and adherence to a codebase's own conventions, with tasks authored by 20+ maintainers of the repositories they come from at over 40 hours of work per task. On the hardest Diamond subset, the best model scored **13.4%**.
- **[SWE-Marathon](https://arxiv.org/abs/2606.07682)** (Abundant AI) runs ultra-long-horizon, project-scale work — clone a library, build a full-stack product, write a C compiler in Rust. The best configuration resolved **26%** of tasks, with the longest single rollout burning hundreds of millions of tokens over nine hours.
- **DeepSWE** (Datacurve) takes the contamination angle: large tasks on repositories that were never built in the real world, so the answer cannot be in the training set.

Compare those numbers with the 50%-and-climbing scores that SWE-bench-style evals produce and the gap tells you something precise. Models are genuinely good at *closing a ticket*. They are not yet good at *leaving the codebase in a state a maintainer would sign off on*.

## Why more harness engineering cannot close the gap

**Because the reviewer has the same blind spot as the author.** This is the part of the argument that stings, and it is one sentence: if the model knew what good code looked like, it would probably have written it that way in the first place.

Adversarial review bots, quality-gate prompts, an extra pass with "be critical" in the system prompt — all of it raises the floor. None of it adds knowledge the training run did not put there. You can catch a null check the model forgot. You cannot reliably catch "this is the fourth abstraction doing the same job and the third one should have been deleted," because that judgement is exactly the capability that was never rewarded.

Harness engineering is real engineering and it is worth doing. It is just not a substitute for a capability the weights do not have. **For now, you are stuck reading the code.**

## What replaces it: four artifacts before the agent starts

**If review cannot be removed, make review cheap.** That is the practical core of the talk, and it inverts the usual advice. Instead of spending more tokens downstream trying to repair a pull request nobody agreed on, spend thirty minutes upstream making sure the pull request is one you already expect.

The observation underneath it: *you do not have too many pull requests, you have too many bad ones.* A good pull request is a pleasure to review — you are reading confirmation of a decision you already made. A pull request needing even 20% rework is an intellectual and emotional tax on both the reviewer and whoever submitted it.

HumanLayer's answer is four written artifacts, produced with AI assistance but agreed by humans, before implementation begins. Small changes still go straight to the agent; this is for work that would be expensive to get wrong.

1. **Product review.** What problem are we solving, for whom, what is the desired behaviour, what does success look like. Mockups where the surface is visual.
2. **System architecture.** How the pieces fit: services, endpoints, schemas, component contracts, data models, and the constraints the design has to respect.
3. **Program design.** The layer Horthy says is most underrated in agentic coding — types, method signatures, file layout, call stacks. Not "the model can figure it out once the architecture is right," but the actual shape of the code. Dylan Mullroy at Cloudflare uses call graphs as a planning artifact for the same reason.
4. **Vertical slices.** The order of implementation, cut end-to-end rather than layer-by-layer, including cross-repository coordination and what gets tested between phases.

Note what these are. They are not tool configuration, not YAML, not prompt files. They are **documents** — the same documents good teams wrote before AI, now doing double duty as the context an agent reads and the yardstick a reviewer reads against.

The economics are simple: alignment is shorter because AI gathers the information fast, coding is faster because the agent does it, and review is faster because you already agreed on the answer. You move quicker *and* you keep reading every line.

## Which teams benefit most

1. **Teams on codebases older than six months.** Horthy's observation is that agents start to struggle after roughly three to six months of AI-paced shipping — brownfield arrives much sooner than it used to.
2. **Teams already running a factory.** If you have agentic build, review and regression testing wired up and quality is still sliding, the missing piece is upstream, not another loop.
3. **Multi-repository organisations.** Vertical slices exist precisely because a feature crosses services, and no single agent context holds all of them.
4. **Regulated and high-blast-radius work.** Where "the tests passed" is not an answer to an incident review, and the architecture decision has to be written down anyway.
5. **Anyone who tried lights-off and quietly turned the lights back on.** You already have the evidence. What you probably do not have is the planning stack that makes the human step affordable.

## How MDflow fits

**MDflow is not a harness, an orchestrator, or a code review tool.** It does not build, test, or verify anything. What it is: a markdown workspace that both people and agents read and write — which is exactly the shape of those four planning artifacts.

### What already lines up today

**The artifacts are plain markdown.** A product review, an architecture document, a program design, a slice plan. These are prose, tables, diagrams-as-text and code fences, written by people and read by both people and models. [MDflow](https://mdflow.cz) stores them as plain `.md` with no proprietary layer between the author and the agent, and every shared document has a [raw `.md` twin](/markdown-ai) with YAML frontmatter for agents that prefer to fetch rather than call a tool.

**Folder descriptions tell an agent what a set of documents is *for*.** Each folder carries a description of what belongs inside it, and [`mdflow_get_context`](/docs/mcp) ranks those descriptions *above* folder names and document titles before returning matching bodies. A folder described as *"Program design documents for the billing service — method signatures and call stacks agreed before implementation"* is a retrieval signal you wrote, not one a model inferred. That is [why folder descriptions matter more than file names](/blog/folder-descriptions-agent-context).

**Every agent reads the same plan.** The same workspace is reachable from Claude, ChatGPT, Cursor, Codex and Claude Code over the [remote MCP server](/docs/mcp) with OAuth or a Personal Access Token, from CI and scripts over the [REST API](/docs/api), from the editor via the [VS Code extension](/docs/vscode), and from automations through the [n8n node](/docs/n8n). A plan that only one tool can see is not a plan, it is a preference.

**Workspaces scope plans to the system they describe.** One workspace per product line or client keeps the billing service's program design from surfacing when an agent is working on the marketing site.

**Version history makes the plan auditable.** Line-by-line diffs, non-destructive restore, and a Document Log that names the actor on every write — including `automated · <token name>` when an agent writes. When an architecture decision changes, you can see when and by whom. (Version history is a Pro feature, private to the document owner, and not exposed over the API or MCP.)

**Where the source belongs next to the code, keep it there.** The GitHub integration lets you edit markdown living in your repositories directly from a workspace and commit back, so a spec that should version with its code does, while the cross-cutting documents that outlive any branch stay in the workspace.

**Agents can write plans, not just read them.** An agent can draft the architecture document into the right folder over MCP, and a human can review and edit it in the browser. That is the intended division of labour in Horthy's workflow: AI accelerates the alignment, humans own the decision.

**What MDflow does not do.** No code generation, no review, no static analysis, no CI gates, no evals. It holds the written half. The building and checking are other people's tools, and should be.

### Where we are headed

Direction, not a dated commitment. What interests us for a markdown-native store is making planning artifacts legible *as* artifacts rather than as prose that happens to contain decisions — descriptions that can carry authority and freshness, a way to see which documents an agent actually read before it wrote something, and scoped tokens so an agent's reach narrows to the slice its task needs. If you are going to hold humans responsible for the code, knowing what the agent was told is the first thing you need.

## The bottom line

The seductive version of agentic engineering says the human step is the bug. The evidence says the human step is the only thing currently pricing in a cost that models cannot see, because nobody has built a cheap verifier for "will this codebase still be workable next quarter."

That will change. FrontierCode, SWE-Marathon and DeepSWE are early attempts at exactly that verifier, and when it exists the training loop can chase it. Until then the winning configuration is not fewer humans, it is *earlier* humans: thirty minutes of product review, architecture, program design and slice planning that turns a six-hour review into a twenty-minute one.

Which makes those four documents the highest-leverage artifacts in the whole factory. They deserve a better home than a chat log, a stale wiki, or a `plan.md` that lived and died on one branch.

[Start free](/login) · [Connect an AI agent](/docs/mcp) · [Read the API docs](/docs/api)

## Frequently asked questions

### What is an AI software factory?

An AI software factory is an automated software delivery pipeline in which coding agents, not people, do the building. Work enters a tracker, an agent implements it inside a harness, automated and agentic review run over the result, and the change ships to production — with incidents and user feedback routed straight back into the queue. The term *software factory* predates AI; the agentic version simply replaces the *someone builds the thing* step with *an agent builds the thing*.

### What is a lights-off software factory?

A lights-off software factory is the fully autonomous version, where nobody reads the generated code at all. Human review is removed on the theory that testing, monitoring and rollout systems can catch whatever slips through. HumanLayer ran this configuration on its own codebase in July 2025 and abandoned it after months of production incidents and code the team no longer understood.

### Why do AI software factories fail?

Because coding models are trained against benchmarks that reward making tests pass and cannot penalise eroding a codebase. Reinforcement learning needs a reward signal, and the cost of bad architecture shows up in weeks or months — far too slowly to propagate back to the coding episode that caused it. So agents optimise for the green check: defensive `try`/`catch` blocks, unnecessary casts, duplicated logic. Quality degrades in a direction the training loop is blind to.

### Can better harness engineering fix code maintainability?

It can raise the floor but not close the gap. Prompts, tool loops, adversarial review bots and orchestration all help, but a review agent judging code quality is limited by the same training limitation as the agent that wrote it. As Horthy puts it, if the model knew what good code looked like, it would probably have written it that way in the first place. Maintainability is a model training problem, not a scaffolding problem.

### What should teams do instead of a lights-off factory?

Keep humans reading the code, and move the human effort earlier so review stays cheap. Front-load four written artifacts before any agent starts: a **product review**, a **system architecture**, a **program design** with types and call stacks, and a **vertical-slice** implementation plan. Roughly 30 minutes of upfront alignment can remove hours of review, because a pull request that matches an agreed plan is fast to read and rarely needs rework.

### Where should AI planning documents live?

Somewhere durable, versioned, and readable by every agent you use. Product reviews, architecture documents, program designs and slice plans are markdown artifacts that outlive the branch that consumed them, and they are useless if only one tool can see them. MDflow stores them as plain markdown in folders whose descriptions say what the folder is for, reachable from Claude, ChatGPT, Cursor and Codex over [MCP](/docs/mcp) or a [REST API](/docs/api).

## Further reading

- [Harness Engineering is not Enough: Why Software Factories Fail — Dex Horthy, HumanLayer](https://www.youtube.com/watch?v=Ib5GBkD555M) (AI Engineer, July 2026) — the talk this post is built on. HumanLayer also publish [a written version](https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/wsff.md).
- [The AI Engineering Report 2026: The Acceleration Whiplash](https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways) — Faros AI, April 2026. Throughput, bugs, incidents and review times across ~22,000 developers.
- [Introducing FrontierCode](https://cognition.com/blog/frontier-code) — Cognition, June 2026. The benchmark that asks whether a maintainer would merge the change.
- [SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?](https://arxiv.org/abs/2606.07682) — Abundant AI. Project-scale tasks at billion-token scale.
- [Software Factories, Light and Dark](https://addyosmani.com/blog/software-factories/) — Addy Osmani, on why advice for vibe coding a side project and advice for a ten-year-old enterprise system are not the same advice.
- MDflow: [AI code verification, the new bottleneck](/blog/ai-code-verification-loop), [spec-driven development with markdown specs](/blog/spec-driven-development-markdown-specs), [the agentic coding loop](/blog/agentic-coding-loop), [folder descriptions as agent context](/blog/folder-descriptions-agent-context), and [connecting an agent over MCP](/docs/mcp).