Pimp My IDE / Garage dispatch
Back to garage
September 27, 2026 | agent verification / failure ownership

No more shrugware.

An agent can write the patch. Your shop still needs a way to explain why it failed.

The take. Faster generation makes unexplained failure more expensive. Give every important change an expected contract, a reproducible break, an independent witness, and a named next check.
Build a failure handoffRead the harness report

The bottleneck moved.

Datadog's harness-first report argues that agents can produce software faster than teams can verify it. Its two systems projects use several different witnesses: shadow state, deterministic simulation, formal specifications, compatibility suites, and staging telemetry.[1]

The useful lesson is not that every codebase needs TLA+. It is that each claim needs a mechanism that can prove it wrong. A compiler result answers one question. A unit test answers another. Neither tells you how the change behaves under production traffic.

A green check is useful only when you can name the claim it checked.

Do not normalize mystery.

Patrick Xia calls out a worse failure mode: systems produce opaque answers, users hit opaque failures, and the investigation ends with a shrug.[2] The piece is an opinion, not an incident report. Its warning still lands. A confidence score or an agent label does not replace a failure model.

When the software breaks, the handoff needs four facts. What did we expect? What happened? How can another person reproduce it? Who owns the next test? Without those facts, speed at the front of the pipeline becomes waiting at the back.

The environment is not the witness.

VS Code 1.139 can run agent sessions inside Dev Containers on remote SSH, Tunnel, and WSL projects. The release notes say the remote folder needs a supported Dev Container configuration and Docker on the host.[3]

That is a useful execution boundary. It can make the toolchain repeatable. It does not prove the patch is correct. Keep environment placement and verification evidence on separate lines in the receipt.

Build the cheapest falsifier first.

Start with the smallest check that can reject the claim. Add stronger witnesses when the failure cost, timing, concurrency, or state space demands them. Datadog describes this as using the lightest mechanism that can falsify a hypothesis first.[1]

  • For a parser, pair examples with malformed input and round-trip properties.
  • For a service, compare a shadow implementation against known behavior.
  • For concurrent state, control time and inject faults.
  • For a deployment, watch the same invariant in staging or production telemetry.

The rack below does not verify software. It fixes the handoff that comes before verification by refusing to generate a clean investigation card until all four facts are present.

Interactive makeover / bug handoff

Failure witness rack

Traditional purpose replaced: paste an error into chat and ask someone to investigate. Better version: connect contract, observation, reproducer, and next owner into one copyable handoff.

Connect the witness chain

Each station lights when its field has useful text. The route reaches the handoff only when every station is connected.

0 of 4 witness stations connected.

Investigation cardDRAFT

Copy the next move

This card records the investigation request. It does not claim the cause is known or the fix has passed.

What completion means. Four fields are present. No test ran here. Attach logs, revisions, commands, and results in the real investigation.

Sources and limits

Open the source log
  1. Datadog, "Closing the verification loop: Observability-driven harnesses for building with agents", published March 9 and read September 27, 2026. This is a first-party engineering report about redis-rust and Helix. Its performance figures and claims about production-like staging come from Datadog.
  2. Patrick Xia, "The normalization of inexplicable failures", published and read September 27, 2026. This is an opinion piece about evals, confidence scores, ownership, and accepting opaque failure.
  3. Visual Studio Code 1.139 release notes, released September 23 and read September 27, 2026. The notes document remote Dev Container support for agent sessions and its prerequisites.
  4. Hacker News discussion 49868243, verified through the Hacker News API and page on September 27, 2026. It is the discovery route for the Datadog report, not evidence for its engineering claims.

Source boundary. The page separates a vendor engineering report, an opinion essay, product release notes, and a discussion link. The interactive rack creates a handoff template. It does not execute tests, inspect telemetry, or identify a root cause.