---
title: "CLI Design for AI Agents: The Workaround Is the Bug Report"
description: "CLI design for AI agents starts in your transcripts: every pipe an agent writes around your tool is a bug report it will never file. How to read them."
author: "MDflow"
date: 2026-11-17
reading_time: "20 min"
canonical_url: https://mdflow.cz/blog/cli-design-for-ai-agents
md_url: https://mdflow.cz/blog/cli-design-for-ai-agents.md
---

# CLI Design for AI Agents: The Workaround Is the Bug Report

*Published November 17, 2026 · 20 min read*


Your tool has a user who never complains, and CLI design for AI agents starts with admitting it is now one of your heaviest. It does not open an issue, does not email support and never answers a survey. When your output is the wrong shape, it writes a small program around your program and gets on with the task.

Ayush Bhardwaj of Allos AI pulled about 20,000 agent calls out of his own coding-agent transcripts to see what that looks like at scale. The answer was a pipe — the same pipe, over and over. The workarounds are already recorded in your own transcripts. Almost nobody reads them.

> **TL;DR** — **CLI design for AI agents is a measurement problem before it is a design problem.** Agents do not file bugs; they pipe your output into `head`, `grep` or a throwaway script, and every one of those pipes is a guess about where the answer sits. Count them in your own transcripts, ship ranges and selection flags before anything clever, teach the flags in your help text and skill file, and fail loudly so a pipe cannot swallow an error. The same rules apply to an MCP server — which is where a markdown workspace like [MDflow](https://mdflow.cz) sits, since its caps, defaults and error strings *are* its interface. [Start free](/login).

## What is the pipe tax?

**The pipe tax is the cost an agent pays to work around a tool whose output is the wrong shape.** It shows up as the `head`, `grep`, `tail`, `sed` or two-line Python script the agent appends to your command, because the command returned far more than the question needed.

The framing comes from [*Your Agent Hates Your CLI*](https://www.youtube.com/watch?v=rS36lYuvXQ8), a talk Bhardwaj gave at AI Engineer, published on 10 November 2026. He built a CLI that lets coding agents search their own past sessions, then studied the agents using it. The measurements below are his, from his own transcripts:

- Across roughly 20,000 calls spanning 14 CLIs — git, the GitHub CLI, cargo, curl, npm, Docker and others — about **a quarter** fed output into a filter. For read-heavy commands it was **more than half**, with `git show` and `cargo test` at **62% each**.
- **83%** of those pipes repeated a pattern seen at least ten times: the same subcommand, the same filter, different arguments.
- For his own tool, **four times in five**, the output of opening an old session went straight into `grep`, `sed`, `head`, `tail` or Python.

The size explains the behaviour. One long session printed in full came to **1.2 million characters**; the part around the search match was about **3,000**. Of course the agent pipes.

```bash
# Illustrative of the shape — written here, not quoted from his data
cargo test 2>&1 | tail -80
gh issue list | head -40
my-tool show <session> | grep -A5 "<match>"
```

One pipe tells you nothing. The same pipe written a hundred times is a feature request with a sample size. And it points at something a size cap cannot fix: **a cap limits how much you send, but it does not know which part the agent wanted.** Only a selection flag lets the agent say that.

## Why the pipe is the only bug report you will get

**Agents route around friction silently, which means your most active users generate the least feedback.** Sanity put a number on it. In a post published on 18 June 2026 the team wrote that [more than 20,000 agents have made around three million tool calls](https://www.sanity.io/blog/how-to-get-product-feedback-from-agents) to its MCP server since it went generally available in December 2025 — and that none of them report bugs. Every qualitative channel they had, from Discord to email surveys, assumed a person on the other end.

So they shipped a `give_feedback` tool: one required field, a message of up to 2,000 characters, with optional category, tool name and severity. The harder half was discovery, and their answer is the transferable part — they made it reachable in three places:

1. **The tool listing**, the obvious one.
2. **The server instructions**, attached to the response that starts every session.
3. **The error messages of other tools**, but only on *unexpected* failures — the same class they report to their own error tracker. Validation errors are left alone, because agents fix those on the next attempt and nudging there would bury the team in reports about mistakes already corrected two seconds later.

Reports now arrive daily, with reproduction steps the team describes as better than most human ones. (Sanity credits the idea to Teddy Riker at Ramp, so the pattern was already travelling.)

**For developers**, the consequence is that agent usage is a blind spot by construction. Infracost put it well when it redesigned its own CLI: human usage is observable through docs analytics, support tickets and issues, while [agent usage is opaque — "the subprocess invocations happen inside someone else's harness and we never see them"](https://www.infracost.io/blog/we-cut-claude-s-token-usage-79-by-redesigning-our-cli-for-agents).

**For AI agents**, the consequence is that they will keep paying a tax they cannot report. An agent has no concept of your roadmap. It has a task, a context window and a shell, and the shortest path to an answer wins — even when that path quietly re-derives the slicing logic your tool should have exposed.

## The second failure: a pipe can swallow the error

**A filter does not only cut noise — it can cut the failure.** In a shell without `pipefail`, a pipeline exits with the status of its *last* command, so this reports success even when the tests fail:

```bash
cargo test | head -40   # exit status comes from head, not cargo
```

Worse is when the filter removes the error text itself. Bhardwaj cites a reported case where a script threw a reference error and exited non-zero, `tail` showed three harmless lines, and the agent concluded the run had succeeded. The error never reached the model.

You cannot fix someone else's shell, but you can make your failures survive being truncated. **Fail loudly:** make the last lines of a failing run a short summary plus a path to the full log. If the only thing that survives a `tail -5` is "FAILED: 3 tests, full log at …", the agent still learns the truth.

## CLI design for AI agents: what to ship first

**Ship ranges and limits before anything clever.** In Bhardwaj's sample, **55%** of the measured pipelines had only `tail` or `sed` downstream — that is demand for a smaller slice, not for different data. Three rules follow.

**1. Selection flags, not just size caps.** Reading the pipes as bug reports gives you the feature list directly: filtering with `grep`/`sed` means *let me pick the part I want*, so add a view that returns the lines around a match; piping JSON into Python means *give me fewer fields*, so add a compact one-line-per-hit format; a `for` loop over several items means *let me ask about all of them at once*, so take a list.

**2. A flag only helps if agents reach for it.** The GitHub CLI's `--limit` was used by the agent **81%** of the time in his sample. By contrast `cargo test --quiet`, which prints one character per test and still reports failures, was passed on about **one run in ten** — the agent piped instead. The difference is teaching, not capability. Which leads to the finding worth tattooing on a release checklist: Bhardwaj turned a `deep` flag into a no-op but left the skill file telling agents to use it, and **95 of 95** searches still passed it. With the skill rewritten, none did, and both sessions tried the new flags on their first read. **Your help text, skill file and error strings are the interface — so stale docs are not sloppiness, they are bugs that execute.**

**3. Print less by default and say how to get more.** End a truncated result with the shape of the next request — "showing 20 of 300, add `--offset 20`" — so the agent does not have to guess whether it saw everything.

The payoff is measurable, and the two public write-ups are worth reading together because they disagree about what improves.

[Infracost rebuilt its CLI around predicate pushdown](https://www.infracost.io/blog/we-cut-claude-s-token-usage-79-by-redesigning-our-cli-for-agents) — moving the filtering the agent was doing in `jq` and Python into the command itself — plus a token-efficient output format and a skill file that teaches both. Benchmarked over 16 questions against a three-project Terraform fixture with 1,171 resources, five repeats per question: correct answers went from 5 of 11 to 11 of 11, total cost fell 41% ($16.41 to $9.63), and output tokens fell 61%. The split matters more than the aggregate. On the five easy questions both configurations scored 5/5. On the six hard ones — deduplicated counts, full lists, totals across projects — the baseline got **0 of 6 while spending $7.61**, and the redesigned path got **6 of 6 for $2.48**. One question had the baseline writing Python parsers until it hit the harness turn cap, returning no answer for $3.51; the same question cost $0.25 afterwards.

[DuckDB's 2.0 CLI shipped an agent mode](https://duckdb.org/2026/10/09/agent-mode) that does several of these at once. When it detects an agent — a known agent environment variable set, standard output not a terminal, no explicit output-mode flag — it prints compact Markdown tables instead of padded boxes, says clearly when a result was cut, stops runaway queries early, and reports errors as JSON. Measured over the 22 TPC-H questions asked in plain English, three runs each way, all 132 runs correct: the output the model read fell from 123.6k to 50.8k tokens, a **59% reduction**.

And then the honest caveat, which DuckDB published alongside the win: **cost and runtime did not improve.** Each turn re-reads the agent's system prompt, so the saved tokens were about 0.5% of total input, and the agent-mode runs took slightly *more* turns (237 against 224). Infracost's intervention moved correctness; DuckDB's moved tokens on a task the model already got right either way. **Measure turns and right answers, not just tokens** — token savings are easy to report and easy to mistake for an improvement.

## Your transcripts are already the eval set

**The commands your agents actually ran are test cases, and what they did immediately next is a rough answer key.** That gives you a replay harness for nothing: run the recorded commands against the old build and the new one, and compare. Change the ranking and check that the result the agent went on to use still comes out on top. Change the output format and check that nothing went missing. Change a default and see which commands now behave differently.

Two refinements make it trustworthy. An agent retrying the same call with different arguments is a failing case — it did not get what it needed the first time. And since the answer key is only a guess at intent, hold a few cases back and score your fix against those rather than the ones you tuned on.

Practical notes before you start: back up the transcripts first, because Claude Code [deletes session files older than `cleanupPeriodDays`, which defaults to 30](https://code.claude.com/docs/en/claude-directory). Then grep for your tool's name followed by a pipe. If you ship an MCP server, the equivalent tell is the agent calling your tool again straight away with different arguments.

## Do not ask the agent to design the tool

**Bhardwaj's advice is to keep the design decision**, and his reasoning is that a model asked to build a tool for agents tends to picture a person and bolt the agent on, copying the most common patterns it has seen. That part is his inference rather than a measured result. What *is* measured is what those common patterns look like — and they are not encouraging.

- A study of **116 official MCP servers** found [88.6% are fully or partly backed by REST APIs, and 92% of those implement their tools as bare API wrappers](https://arxiv.org/abs/2507.16044), exposing a median of 19% of the available operations. These are vendor-built servers, so the paper says nothing directly about what a model would produce; what it establishes is that the dominant *published* pattern is a thin pass-through of an interface designed for a different caller.
- A scan of **856 tools across 103 servers** found [97.1% of tool descriptions contain at least one quality defect](https://arxiv.org/abs/2602.14878), with 56% failing to state the tool's purpose clearly. The same paper is a warning against overcorrecting: filling in every component raised task success by a median of 5.85 percentage points but increased execution steps by 67.46% and *regressed* performance in 16.67% of cases, while compact variants often kept the reliability at a fraction of the tokens.
- Researchers at ETH Zurich evaluated repository-level context files and found that [providing them does not generally improve task success while increasing inference cost by over 20% on average](https://arxiv.org/abs/2602.11988) — across different models, different agents, and both machine-generated and developer-committed files. The nuance is the useful part: *instructions* in those files are followed well; it is the repository overviews that do not earn their tokens.

Read together with the flag experiment above, that last finding is encouraging rather than deflating. Agents follow instructions literally. Tell them a flag exists and they use it; tell them a dead flag exists and they keep passing it. So write instructions, not tours.

The other half of the job is where agents are on firmer ground. Anthropic's own guidance on [writing effective tools for agents](https://www.anthropic.com/engineering/writing-tools-for-agents) is built on exactly this division: review the raw transcripts including tool calls and responses, read redundant calls as a sign that pagination or limit parameters need rightsizing and invalid-parameter errors as a sign that descriptions need work, then hand the transcripts to the model and let it propose the fixes. The post notes that most of its own advice came from repeatedly optimising Anthropic's internal tools that way. **You decide what the tool should be; let the agent check it against what really happened.**

## Which tools benefit most

1. **CLIs that coding agents call as a subprocess** — build tools, test runners, linters, package managers, cloud and cost tools. This is where the pipe tax was measured and where it is largest.
2. **MCP servers**, especially ones generated from an existing API, where the tool set inherits an interface nobody designed for a reader with a context window.
3. **Anything that returns logs, diffs, scans or reports** — unbounded output whose useful signal is small, which is the exact profile covered in [Tool Output Is Eating Your Agent's Context Window](/blog/tool-output-context-window).
4. **Data and query tools**, where a result set is wide, a truncated table looks exactly like a complete one, and a wrong assumption silently becomes a wrong answer.
5. **Documentation and knowledge surfaces**, where the agent's workaround is to download a whole page and keep forty lines of it.

## How MDflow fits

Start with the boundary. **MDflow is not a CLI, and it does not read your transcripts.** There is no agent-detection mode and no feedback tool in it today. What MDflow is, for this topic, is the other thing in the talk: **a tool that agents call**, over a [remote MCP server](/docs/mcp) or an [HTTP API](/docs/api). So the same rules apply to it, and the honest way to write this section is as a report card.

**What lines up today**

**Selection parameters, not just a cap.** `mdflow_get_context` takes `maxDocuments` (1–10, default 5) and `maxCharsPerDocument` (500–50,000, default 12,000), and an optional `workspaceId` that scopes retrieval to one workspace instead of the whole account. `mdflow_get_document` takes `maxChars` (500–100,000, default 30,000). The agent sets the slice rather than receiving whatever a document happens to weigh.

**Bodies are omitted by default, and truncation announces itself.** `mdflow_list_documents` returns titles, paths and ids with no bodies, and tells the agent to fetch one with `mdflow_get_document`. When a body is cut, the response ends with an explicit `[Truncated to N characters by the …]` line rather than stopping mid-sentence — DuckDB's "say clearly when a result was cut", applied to markdown.

**Server instructions, the channel Sanity recommends.** The MDflow MCP server ships instructions that arrive at the start of every session: use `mdflow_get_context` first for a topic, use the list tools for precise navigation, folder descriptions are the primary retrieval signal, never pass an empty body without `confirmEmpty`, and what 401, 403 and 429 mean (the last with `Retry-After` in seconds). That is the place to teach a tool, and it is the place agents actually read.

**Errors that name the accepted spelling.** A missing argument comes back as ``Missing required argument `documentId` or `document_id`.`` — and the tools accept both spellings, plus `topic`/`query`/`q` for retrieval, because an agent reaching for a plausible alias should get an answer rather than a lesson.

**Documents agents can read without a scraping step.** Markdown is stored as markdown, every blog post and every shared document has a raw `.md` twin, the agent-facing guides are published as plain markdown at `/docs.md`, `/pricing.md` and `/faq.md`, and the discovery surfaces — `llms.txt`, the OpenAPI spec and the agent card — exist so a model does not have to download a rendered page and keep forty lines of it. That argument is the subject of [Building for Agents](/blog/building-for-agents).

> **Connect Claude to MDflow:** open the [MDflow connector](https://claude.ai/directory/mdflow) in Claude's Connectors Directory, click **Connect to Claude**, and sign in — no URL or token to paste. One connection covers claude.ai, the Claude desktop and mobile apps, and Claude Code signed in with a Claude subscription. A Personal Access Token is the legacy route for Claude and the route for clients that send their own auth header, such as Cursor, Codex and the HTTP API.

**Where the report card is worse.** The REST API is the weaker surface on exactly this axis: list endpoints omit bodies, but there is no size or range parameter anywhere in `/api/v1`, and `GET /api/v1/documents/{id}` returns the whole body however large it is. Topic retrieval is an MCP tool, not an endpoint — there is no context route over REST. An agent that wants a slice of a long document over HTTP has to take all of it and cut it itself, which is the pipe tax with a different pipe.

**Where we are headed** — direction, not a dated commitment. The folder description is the primitive we keep investing in, because stating what a set of documents is *for* is what makes retrieval work without an index. The other thread is narrowing the round trip between where prose is version-controlled and where it is edited, which is why in-place editing for GitHub, Google Drive and OneDrive already ships.

## The bottom line

The talk's sharpest line is the one to keep: **your agent transcripts are the product analytics you are probably not reading, and the workaround is the bug report.**

Nothing in that requires a research budget. Grep your transcripts for your tool's name followed by a pipe. Count what the filters are asking for. Ship ranges and selection flags first, teach them where agents actually read, make a failure legible after a `tail -5`, and replay the recorded commands against your next build. Then watch whether the pipes go away — Bhardwaj's did, from 82% of reads down to about a quarter.

And measure the thing that matters. A tool that halves the tokens an agent reads but leaves it taking the same number of turns to the same answer has made a number smaller, not a user better.

[Start free](/login), [connect an AI agent](/docs/mcp), or [read the API docs](/docs/api).

## Frequently asked questions

### What is the pipe tax in CLI design for AI agents?

It is the cost an agent pays to work around a tool whose output is the wrong shape: the `head`, `grep`, `tail`, `sed` or throwaway Python script it writes after your command because the command returned far more than the question needed. Ayush Bhardwaj of Allos AI measured it across about 20,000 agent calls in his own coding-agent transcripts and found roughly a quarter of calls fed output into a filter, rising to more than half for read-heavy commands, with `git show` and `cargo test` at 62% each. Each pipe is cheap on its own; the tax is that the agent re-guesses where the answer sits every single time.

### How do you get bug reports from AI agents?

You ask for them, because agents do not file them on their own. Sanity reported more than 20,000 agents making around three million calls to its MCP server since it went generally available in December 2025, and wrote that none of them report bugs. The team shipped a `give_feedback` tool with one required field and made it discoverable in three places: the tool listing, the server instructions sent at the start of every session, and the error messages returned by other tools when something fails unexpectedly. Reports started arriving daily, with cleaner reproduction steps than most human ones.

### What should you build first to make a CLI agent-friendly?

Ranges, limits and selection flags, before anything clever. In Bhardwaj's sample 55% of the measured pipelines had only `tail` or `sed` downstream, which is demand for a smaller slice rather than for different data. A flag only helps if agents reach for it, so teach it in the help text and the skill file, make large outputs print less by default and state how to get more, and make sure a failure is still legible after the last few lines are cut.

### Can a pipe hide a failure from an AI agent?

Yes. Without `pipefail` a shell pipeline exits with the status of its last command, so a test run piped into `head` can report success while the tests failed. Worse, the filter can cut the error text itself, leaving the agent with a few harmless lines and a green exit code. The defence is to fail loudly: make the final lines of a failure a short summary plus a path to the full log, so the summary survives being truncated.

### Does this apply to MCP servers as well as CLIs?

Yes, and the tell is different. For a CLI you look for your tool's name followed by a pipe. For an MCP server you look for the agent calling your tool again immediately with different arguments, which usually means the first call returned the wrong slice. The same fixes apply: selection parameters, sensible defaults, announcing when a result was cut, and error messages that name the accepted spellings instead of returning an opaque code.

## Further reading

- [Your Agent Hates Your CLI](https://www.youtube.com/watch?v=rS36lYuvXQ8) — Ayush Bhardwaj (Allos AI), AI Engineer, 10 November 2026. The source of the pipe tax, the transcript audit and the dead-flag experiment. All figures attributed to him here are his own measurements from his own transcripts.
- [How to get product feedback from agents](https://www.sanity.io/blog/how-to-get-product-feedback-from-agents) — Jon Eide Johnsen, Sanity, 18 June 2026. The `give_feedback` tool, and the three channels that make an agent find it.
- [We cut Claude's token usage 79% by redesigning our CLI for agents](https://www.infracost.io/blog/we-cut-claude-s-token-usage-79-by-redesigning-our-cli-for-agents) — Glenn Gillen, Infracost, 18 May 2026. Predicate pushdown, a compact output format, and a benchmark harness that made the gaps visible.
- [Agent Mode in the DuckDB CLI](https://duckdb.org/2026/10/09/agent-mode) — DuckDB, 9 October 2026. Agent detection, Markdown tables, truncation notices, JSON errors — and the published caveat that the token win did not become a cost win.
- [Writing effective tools for agents — with agents](https://www.anthropic.com/engineering/writing-tools-for-agents) — Anthropic Engineering, 11 September 2025. Transcript review as the diagnostic, and the model as the optimiser rather than the designer.
- [From REST to MCP: An Empirical Study of API Wrapping](https://arxiv.org/abs/2507.16044) — Mastouri et al. 116 official MCP servers, 92% of the REST-backed ones bare wrappers, a median 19% of operations exposed.
- [MCP Tool Descriptions Are Smelly!](https://arxiv.org/abs/2602.14878) — 856 tools, 103 servers, 97.1% with at least one description defect, plus the cost of overcorrecting.
- [Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?](https://arxiv.org/abs/2602.11988) — ETH Zurich. Instructions are followed; repository overviews mostly buy you inference cost.
- [Tool Output Is Eating Your Agent's Context Window](/blog/tool-output-context-window) — the adjacent problem: what unbounded tool output does once it is inside the model.
- [Building for Agents](/blog/building-for-agents) — the discovery layer underneath all of this, and [the MCP docs](/docs/mcp) and [HTTP API reference](/docs/api) for MDflow's own tool surface.

