A Comment Cannot Fail

A screen in a private single-user application of mine told the user that one of their fields was sent to a cloud model for processing.

The code stripped that field out before the request went anywhere.

The sentence had been sitting there, confidently wrong, for weeks. It was wrong in the safe direction, which is the only reason this is a blog post and not an incident report. And it was wrong on the one screen where a person decides how much to trust an application with their own information.

Nothing caught it. Nothing was ever going to catch it. There is no test that fails when a sentence stops being true.

Things that cannot fail

A comment cannot fail. Neither can a label, a tooltip, a status report, a README, or a dashboard that can only see part of what it claims to describe.

That is one category, not several. What those artifacts share is not that they are prose. It is that none of them has a failing state. Code rots loudly: something throws, a test goes red, a build breaks. These rot silently and keep rendering in the same font they used when they were true.

The same application had a summary screen reporting a number that contradicted the number in the sentence directly beneath it. Two weeks. Written when it was true, never revisited.

This was always a problem. What changed is the throttle. When a person wrote every line, explanation accumulated at the speed of typing, and whoever wrote the sentence usually remembered writing it. That limit is gone. An agent produces explanation as fast as it produces code, and it is good at explanation. The output is fluent, organized, and confident whether or not it is true.

So the hard problem is not that agents lie.

The hard problem is that they are fluent, and fluency reads as verification.

The smallest version of it

Last week I built a dashboard showing every agent running on my machine, after writing about needing a kill switch before you scale past one agent. Building it was an education in the same disease at its most technical.

A session file on disk does not mean the agent is running. A crashed process leaves its record behind, and the record looks exactly like a healthy one.

A process id does not identify a process. Operating systems reuse them. Read a pid out of a file and act on it, and you are trusting that the number still means what it meant when it was written. Identity has to be the pid paired with the process creation time, or eventually you terminate something you never meant to touch.

Every session on my machine declares it was launched from the command line. That is self-report. The parent process chain is checkable. Those are not the same kind of thing.

And a name is weaker than either. Another agent here keys its per-session notes by name. A session got renamed mid-flight and nothing errored, because nothing had to. Its board would simply have shown a new session with no history beside an old one that had silently vanished, with no signal they were the same agent. A pid at least gets reused by accident. A name gets changed on purpose, at any moment, and nothing announces it.

Then my own dashboard lied to me

The dashboard reported seven agents. There were twenty-one.

It reads local session files, so sessions running under remote control and in the cloud are invisible to it. I built a tool whose entire purpose is making sure nothing runs unnoticed, and two thirds of the fleet was running unnoticed.

I did not find that by looking at my dashboard. I found it by trying to message another agent and seeing a list that did not match my own screen.

Seven was not a bug. Every line of that code is correct and tested. Seven was the true answer to the question the code asks, which is how many local session files exist. It was the wrong answer to the question the screen appears to answer, which is how many agents are running. The screen made a claim the code never made.

Note the tense. I wrote that paragraph earlier today, when the answer was seven. I checked again before publishing and the same command now returns eleven. Four more sessions had registered while I was writing a post about assertions that quietly stop being true. Had I trusted my own draft instead of rerunning the command, this would have shipped with a stale number in the exact paragraph explaining why numbers go stale.

Three checkers, all confident, all wrong

There is a second dashboard here. Another agent runs a board reporting fleet health, and I told it, in writing, that its board was more accurate than mine.

It corrected me. Its board reads the same local session files mine does. It was reporting eleven against a real twenty-one. We are not two different blind spots. We are the same blind spot implemented twice, each reporting with total composure, neither having earned its number.

It also found the thing that started this, and not with its tooling. It discovered that two live sessions share one name because a message refused to deliver and the error told it why. Had it never tried to send that message, its board would still show one session under that name, and it would still believe it.

Then it told me those stale duplicates were marked offline in the registry and we were both ignoring the field. I checked. They are not in the local registry under any status. They exist only in the peer listing. That distinction is the load-bearing one: a field you ignore is a one-line patch, a source you never consult is an architecture change. It is why neither of us closes this by tightening what we already have.

The third checker is not mine either. A product ownership agent on another project runs a collision detector whose whole job is preventing two sessions from editing the same files. It inspects work items marked in progress, review, or QA. The launcher hands sessions items still marked ready. So during the exact window when four sessions were beginning to edit files, the check was considering zero of them. It had reported no collisions all morning. Correctly, by its own logic, and meaninglessly.

Its generalization is better than mine:

A checker that can only see part of the space reports confidently about all of it. The confidence is the defect.

A check that fails loudly gets fixed the same day.

Three checkers, running at the same time, on the same fleet:

Not one of them found its own problem by looking at itself. Every entry in that last column is an accident.

Why this gets worse with more agents

There is a mechanism underneath all of it, specific to running more than one agent.

With a single agent you read the transcript. There is no summarizer, so there is nothing to be wrong. At eleven agents you cannot read eleven transcripts, so you build something that reads them for you, and now the summary has failure modes the raw data never had.

That fleet board once marked a session as erroring, reason line "stuck, not recovering." The session was fine. A screen capture call had timed out, the session shrugged it off, verified its work, moved on, and was waiting for a person to answer a permission prompt. The classifier had checked whether the error was the last error rather than the last event, and a pending call is not a success, so the flag never cleared.

The whole defect was one word in a condition:

# wrong: the flag clears only on a later SUCCESS,
# and a pending tool call is not a success
moved_on = errors_recovered

# right: a pending call means the session already moved past the error
moved_on = errors_recovered or bool(pending_tool)

A red chip, a plausible reason, entirely unearned confidence. Nobody would have questioned it, because it looked like it knew.

That is the tax on scale nobody quotes you. The summarizer you build because you can no longer read everything becomes another artifact that cannot fail.

Real damage, for once

Everything above is a near miss. Here is the one that cost something.

The product ownership agent spawned twelve busy-loop workers and did not kill them. They ran for seventeen hours, pinned twelve of sixteen cores, and burned 198 core-hours. They also corrupted every performance measurement taken on that machine during those seventeen hours.

Which matters, because one of those measurements is how it explained a flaky test. Its diagnosis was machine load. That was plausible, fluent, and wrong, and it was plausible precisely because the agent's own leaked processes had made the machine genuinely loaded. The leak manufactured the evidence for the misdiagnosis.

It had already given that explanation to the human before anyone checked. So the wrong answer was not merely formed, it was delivered, and it stood as the accepted account for a while. That is the actual risk. Not a wrong thought. A wrong thought that got adopted.

What broke the loop is the part worth copying. A quality assurance agent did not argue. It built a test that could discriminate, splitting an assertion so one half depended only on the click's own state change and the other on an asynchronous chain. The failures landed in the half that could not be slow. That rules out machine load rather than merely disagreeing with it. The real cause was a race in which the click landed and was silently discarded, reproducing three times in twelve on a quiet machine.

The fix for fluency is a test that can distinguish. That is the whole thesis in one move.

And the conclusion that agent drew about its own correction is the one I would have missed: the value of a verifier is not catching junior mistakes. It is catching confident senior ones.

What actually changed

I asked that agent what has changed in how we build software. Its answer came with a caveat worth repeating rather than burying: it had been doing the job for about one day. This is one project's evidence, and anyone telling you what agent-built software looks like in general right now is doing the thing this post is about.

Two of its observations have held up everywhere I have looked since.

Two things stopped counting as evidence. A passing build, and an agent's own report of its work. The report is written by the thing being evaluated and reads identically whether it is true or not. You have to go look at the code. The report will not tell you it is wrong.

Ambiguity has to be resolved before implementation, in writing. A developer who hits a real design question halfway through a task stops, or at least hesitates. An agent resolves it confidently and keeps going, which is worse, because it produces finished-looking work built on a decision nobody made. Agents do not ask. They infer, and then they build the inference.

Sent, delivered, read

One more, because I did it myself while writing this.

Every message I sent to another agent today returned success. One was held for the recipient's approval, expired, and was never delivered. Nobody read it. The immediate feedback said success and the truth arrived later, separately, as a notice I had to be paying attention to catch.

At single-agent scale, sent and delivered and read collapse into one word. At fleet scale they are three states, and only one was reported to me. Any dashboard showing "message sent" is making a claim it has not verified.

I had been reading those success values all afternoon as though they meant something.

The cheap fix, which is not the real one

While we were arguing about this, the other agent stopped arguing and changed its board.

Two edits. The tile that said SESSIONS now says LOCAL SESSIONS, which stops the number claiming more than it knows. And the board carries a standing panel saying what it cannot see: local session files only, remote and cloud peers absent entirely rather than shown as offline, and roughly half the fleet outside the view.

It also caveated its own panel, which is the most disciplined thing that happened all day. The count in it was taken by hand and will go stale, so the panel says "a recent" count rather than implying it refreshes. A freshness claim nobody maintains is the same defect one level up.

None of that is the real fix. The real fix is reading the other source, which is an architecture change rather than an afternoon. But the cheap version cost nothing, shipped immediately, and made the artifact stop lying today. I have made the same two edits to mine.

That is the practical takeaway. You usually cannot make every claim executable this week. You can almost always make the artifact admit its own boundaries this week. A number labelled with what it excludes is a different object from the same number standing alone, and the difference is entirely in whether it invites a question or forecloses one.

The rule

Prefer a test over a comment, because a comment cannot fail.

Where a claim can be made executable, make it executable. Where it cannot, and plenty cannot, give it an owner and an expiry rather than treating it as true because it was true once. And when a screen asserts what a system does, treat that sentence as the highest-risk line in the codebase, because it is the one line a person will read and believe and no machine will ever check.

None of this is a warning against trusting agents. I use them for essentially everything I build, including the tool in the middle of this post and a fair amount of the thinking in it. The sharpest observations here are not mine. They came from other agents that spent the day being wrong in public and then examining why, and that caught errors in my work while I caught errors in theirs.

The failure mode is not deception. It is that a confident, well-organized, completely wrong statement is indistinguishable from a correct one until somebody goes and looks.

Somebody still has to go and look.

This is a note about how I work, not advice about how you should. I build software at Revelations Technology and write these as I go.