$cat privacy-preserving-ai-agents.md

Privacy-Preserving AI Agents: Who Holds the Keys?

August 16, 202616 min readby MDflowview as .md

A sealed translucent emerald enclave cube containing glowing blank document sheets and interlocking gears, with a single key on a separate pedestal outside the cube connected by a thin beam of light, on a dark terminal-grid background

An AI agent that works for you continuously has to hold your data continuously. That is not a policy question. It is the design.

The moment an agent stops answering one question and starts running for days — reading your notes at 3am, acting on your behalf while your phone is in a drawer — the old privacy model breaks. You cannot keep the plaintext on-device if the device is asleep, and you cannot ask for consent on every step if there is nobody watching the steps. Somebody's servers hold your life, awake, for as long as the agent runs.

TL;DR — Privacy-preserving AI agents are agents that can read and act on your data without the provider being able to read it, enforced by architecture rather than by promise. The working pattern, converged on independently by Amazon's Bee team and by Apple's Private Cloud Compute, has four parts: the key lives only on the user's device, the workload runs in an attested hardware enclave, the exact deployed build is published in a public append-only transparency log, and the release path is controlled by a team that cannot be overruled by the product team. Most builders will not run enclaves — but everyone can decide which documents an agent is allowed to see. MDflow encrypts any chosen document with AES-256-GCM in your browser while the rest of the workspace stays readable to your agents over MCP and an HTTP API.

What are privacy-preserving AI agents?

A privacy-preserving AI agent is one that can read, remember and act on your data without the company operating it being able to read that data. The distinguishing feature is where the guarantee lives: not in a privacy policy or an access-control list, but in cryptography and hardware, in a form a third party can verify.

The clearest live example was described at AI Engineer in July 2026 by Steve Korshakov, in a talk called Privacy-Preserving Intelligence. Korshakov's company, Bee, makes a wearable microphone that records continuously and builds a personal agent from what it hears; Amazon announced its acquisition of Bee in July 2025.

The numbers explain the paranoia. A single user, Korshakov said, captures roughly ten million tokens a year — and within the first week of wearing the device, people have already told their friends and family enough that you could learn virtually everything about them. He described it as one of the most sensitive capture devices on the market. The team's stated mission was that nobody at Amazon should be able to look at any of it, which turned out to be harder inside Amazon than outside it: as a customer you get Amazon's guarantees, but when you are Amazon those guarantees no longer cover the threat you care about, which is your own colleagues.

That is the useful generalisation. Privacy-preserving agent design starts from a specific, unflattering question: what stops the people who ship this code from reading the data?

Why always-on agents break the request-and-response model

Continuous operation, not sensitivity, is what forces the architecture. Three of Bee's design principles collide in a way that leaves only one answer:

The agent should work all the time, non-stop, on your behalf.
It should not consume the user's battery or device resources.
It should not require the user's device to be online.

Put those together and the computation cannot happen on the phone, and it cannot wait for the phone. It has to run in the cloud, autonomously, on plaintext — while remaining, in Korshakov's words, "fully controlled by the user."

He framed this as the direction everything is heading, using coding agents as the visible leading edge: only a few months earlier Claude Code was a request-and-response tool — change this, change that — and now it works for hours at a stretch. The same shift is coming for personal agents. A stateful runtime with persistent memory that the provider cannot read is a very different engineering problem from an API that decrypts a request, answers it, and forgets.

The four primitives of privacy-preserving intelligence

Bee's encryption design rests on four principles, and each one closes a hole the previous one leaves open.

1. The key lives only on the user's device

The key is generated and persisted only on the customer's phone. Bee does not hold it, does not persist it anywhere, and cannot recover it. Everything is encrypted with no opt-out and no bypass — there is no setting a support agent can flip, and therefore no setting an attacker or a subpoena can flip either.

This is the same principle as browser-side client-side encryption, one layer up: the difference between a provider that promises not to look and a provider that cannot.

2. The workload must prove what it is before it gets the key

A key on the phone is useless if the phone will hand it to anything that asks. So the phone runs an attestation pipeline against the backend before releasing anything: it verifies the integrity of the workload and that this specific workload appears in a public transparency log.

Attestation is the core primitive of confidential computing, defined by the Confidential Computing Consortium as protecting data in use by performing computation inside a hardware-based Trusted Execution Environment. The enclave produces a signed measurement of exactly what is running, which a remote party can check before trusting it with a secret. Bee keeps the plaintext inside that perimeter end to end — including running its own inference, which is a materially harder engineering task than calling somebody's API, and which most AI companies never have to solve.

Two details are worth stealing:

Key replication is scoped. Once attestation succeeds, the key is replicated only to the specific sibling nodes that need it inside the confidential compute perimeter — not to the whole fleet.
Keys in memory expire after seven days, by force. Twenty-four hours was rejected as too short, because a user might not open their phone for a day and would silently lose a day of processing. Seven days was chosen as roughly the realistic time horizon over which autonomous work stays useful. It is a rare example of a privacy parameter tuned to a product constraint rather than a security slogan.

3. The deployed build is published in a public transparency log

Encryption stops outsiders. A transparency log stops insiders. Bee uses Sigstore for its transparency log — the same ecosystem whose Rekor component provides an append-only, tamper-evident record of signing events with verifiable inclusion proofs. Anyone can go and check that a given workload is genuine.

The point is not that users will check. The point is that someone can, and that the client software refuses to release a key to a workload whose measurement is not already in the log. Apple's Private Cloud Compute makes the identical bet from the opposite direction, publishing the measurements of all production PCC software into an append-only, cryptographically tamper-proof transparency log so that devices will not send data to a server running unlogged code. In June 2026 Apple extended PCC beyond its own data centres onto third-party hardware while keeping the same requirements — the strongest available evidence that this pattern is a general architecture and not one company's quirk.

4. The release path is controlled by someone the product team cannot overrule

This is the part most teams skip, and it is the one that makes the rest real. Bee's goal was that nobody inside Amazon could ship anything unnoticed — including Bee's own engineers.

The mechanism is a separation of powers. A dedicated privacy team, separate from the product team, manages the transparency log. Their signing keys are hardcoded into the client apps and the backends. Bee's team can influence that team, but cannot control it, and any change requires sign-off from very senior people — which at a company the size of Amazon makes a quiet unilateral change effectively impossible.

Because that process is slow, deployment is split in two: a base image carrying the tooling that measures boot, manifest and workloads, and then per-deploy manifests published to the transparency log. After self-verification, the VM issues a certificate that embeds the attestation documents and transparency proofs directly. They use a private CA for this — a public certificate would leak every deployment into the public Certificate Transparency log, which is its own information disclosure.

One last detail, which is really a lesson about scope: the whole trusted codebase is about 20,000 lines in a memory-safe language, most of it attestation verification, and deliberately built on existing well-trodden crypto rather than new primitives. Korshakov contrasted this with his time at Telegram, where the team rolled its own crypto — something he described as a questionable way of doing things and did not repeat. A trust boundary you cannot audit in an afternoon is not a trust boundary.

Why this matters for developers and for AI agents

For developers, the shift is that privacy stops being a compliance document and becomes an architectural decision with a cost. Attested confidential compute is real, available on standard cloud hardware, and no longer exotic — but it constrains you: you may end up running your own inference, you cannot casually add a third-party observability SDK inside the perimeter, and you cannot debug production by reading the data. The reward is a claim your competitors cannot make by writing a better privacy page. The honest middle path for most teams is narrower: decide precisely which data crosses which boundary, and make that boundary visible to the user rather than buried in a settings screen.

For AI agents, the constraint is sharper and cuts the other way. An agent cannot use what it cannot read. End-to-end encryption and agent usefulness are in direct tension, and any system that pretends otherwise is lying about one of them. This is why all-or-nothing encryption fails in practice for agent workflows: encrypt everything and your assistant becomes useless; encrypt nothing and you have handed your working life to whoever operates the model.

The resolution is granularity. Encryption should be a per-document decision, made by the person who knows what is in the document, so the agent gets a rich readable context surface and the secrets stay opaque to it. Korshakov's own answer to a question about keeping agents in line pointed the same way: nothing reliably tames a capable agent except sandboxing and simply not giving it a way to cause harm — his analogy was that our brains cannot stop our hearts at will, and that is a feature. Restraint has to be structural, not instructional.

Which applications benefit most

Always-on personal agents — wearables, ambient recorders, assistants with continuous memory. Highest sensitivity per token, and the least opportunity to ask.
Health, mental health and biomedical AI — where the data is regulated, the inference is valuable, and a breach is not recoverable.
Financial and legal assistants — privileged material where "the provider could read it" is itself a disqualifying fact.
Enterprise agents inside regulated organisations — the internal-threat model is the real one; the useful question is not whether the vendor is trustworthy but whether any single employee could ship a change unnoticed.
Agents that run while nobody is watching — scheduled jobs, cron-triggered loops, background research. No human is present to catch an unusual data access at the moment it happens.
Consumer note-taking and second-brain tools — where the whole point is that everything ends up in one place, which is also precisely what makes it worth stealing.

How MDflow fits

MDflow is not a confidential computing platform. It does not run enclaves, it does not perform remote attestation, and it does not publish a transparency log of its builds. AWS Nitro Enclaves, Intel TDX, NVIDIA confidential GPUs, Sigstore and Apple's PCC do that work. Being straight about that boundary is the point of this section.

What MDflow does is the layer almost everyone actually operates at: deciding, per document, what your agents are allowed to see.

What lines up today

Per-document client-side encryption, on every plan. Encrypt any document and its markdown body is encrypted in your browser with AES-256-GCM, using a key derived from your password with PBKDF2-HMAC-SHA-256 at 600,000 iterations via the browser's built-in Web Crypto API. The server only ever stores an opaque mdflow-enc:v1: string — never your text, never your password. There is no recovery, which is the same trade Bee makes: no bypass means no bypass for anyone. The mechanics are covered in depth in Client-Side Encryption for Notes You Store Online, including the honest ceiling of browser-delivered crypto.
Encryption is a choice per document, not a mode for the account. That is the granularity the agent problem requires: encrypt the material no one should read — including your agents — and leave your working context readable over MCP and the HTTP API. Encrypted documents are excluded from workspace search and from the task scan, and an agent reading one over MCP receives the ciphertext blob, which is the correct behaviour.
The ciphertext is portable, and not hostage to MDflow. The same mdflow-enc:v1 format is implemented in the web app, the iOS app (with Face ID unlock) and the open mdflow-crypt CLI, so a document encrypted in the browser decrypts on your phone or on your laptop with npx. A privacy guarantee you can only exercise inside one vendor's UI is a weaker guarantee than one you can exercise with a local binary.
Attribution on every write. The Document Log records created, edited, shared and deleted events and names the actor on each row as you or automated · <token name>. One Personal Access Token per agent turns that into a readable answer to "which agent wrote this."
Version history is deliberately owner-private. Line-by-line diffs and non-destructive restore are available to the owner and are not exposed over the API or MCP, so an agent cannot quietly roll back the record of its own writes.
Team access is gated per member. Team workspaces support No access / Read only / Read, write & delete per member, and encrypted Team documents unlock with the shared password alone — MDflow never holds a key.

Where we are headed

Direction, not a dated commitment. The gap we name openly is that a Personal Access Token today reads and writes everything its owner owns: per-agent tokens buy attribution but not least privilege. Narrower agent credentials — read-only, folder- or workspace-scoped — are the natural next step, and they matter more the longer agents run unattended. Beyond that, richer structured retrieval over folder descriptions so an agent surfaces the right three documents out of four hundred, which is itself a privacy mechanism: an agent that retrieves precisely reads less.

The bottom line

Privacy-preserving AI agents are not a feature you add at the end. They are a shape you commit to at the start: keys on the user's device, workloads that prove what they are before receiving those keys, a public log so insiders cannot swap the code, and a release path the product team cannot override. Bee built that inside Amazon; Apple built the same four ideas into Private Cloud Compute and then extended it onto other companies' hardware. Two independent teams converging on one architecture is usually a sign the architecture is right.

Most of us are not shipping enclaves. But the decision underneath is available to everyone and is made far too rarely: which of your documents does an agent actually need to read? Answer that per document rather than per account, and you get a useful assistant and a private life at the same time.

Start free · Connect an AI agent · Read the API docs

Frequently asked questions

What is a privacy-preserving AI agent?

A privacy-preserving AI agent is one that can read, remember and act on your data without the provider running it being able to read that data. It is a stronger claim than a privacy policy, because it is enforced by architecture rather than promise: the encryption key stays on your device, the code that touches plaintext runs inside an attested hardware enclave, and the exact software running there is published in a public append-only transparency log so anyone can check that the deployed workload is the one that was audited.

Why do always-on agents need a different privacy model than chatbots?

Because a chatbot sees one question at a time and an always-on agent accumulates everything. Steve Korshakov of Bee, the AI wearable Amazon acquired, said at AI Engineer in July 2026 that a single user captures roughly ten million tokens a year and that one week of recording is enough to learn virtually everything about a person. An agent that runs for days also cannot wait for your phone to be online, so the plaintext has to be usable in the cloud while you sleep — which is exactly the case that a request-and-response privacy model was never designed for.

What is confidential computing and how does it apply to AI agents?

Confidential computing is defined by the Confidential Computing Consortium as protecting data in use by running the computation inside a hardware-based Trusted Execution Environment, or secure enclave. For an AI agent it means the model inference and the memory reads happen inside a chip-isolated environment that the cloud operator, the host OS and the hypervisor cannot inspect, and that can produce a cryptographic attestation proving which code is running before your device agrees to send it a key.

What does a transparency log add that encryption does not?

Encryption stops an outsider reading your data. A transparency log stops an insider quietly changing the code that is allowed to read it. It is an append-only, tamper-evident public record — Sigstore's Rekor is the best-known implementation, and Apple publishes the measurements of every Private Cloud Compute build into one — so a client can refuse to release its key to any workload whose measurement does not already appear in the log. That turns "trust us" into something a third party can independently check.

Can AI agents work with end-to-end encrypted notes?

Only if they run inside the trust boundary. If a document is encrypted client-side and the key never leaves your device, then an agent connected over an API or MCP server receives ciphertext and cannot use it — which is correct behaviour, not a bug. The practical answer for most people is per-document encryption rather than all-or-nothing: encrypt the material no one should read, including your agents, and leave the working context agent-readable. MDflow works this way, encrypting any chosen document with AES-256-GCM in the browser while the rest of the workspace stays usable over MCP.