Essay

The Velocity Paradox: Faster Code, More Work to Verify

A large test conversion shows why faster implementation shifts attention to the evidence needed before adoption.

· Updated 5 September 2026

Four days of background AI work changed about 600,000 lines in a PHPUnit-to-Pest conversion experiment. The full test suite passes. As of 5 September 2026, further validation and an adoption decision are still pending.1

That is a useful result. It makes a large conversion concrete enough to assess. It also leaves a different job: deciding whether the converted suite still gives us the confidence we need.

This is the velocity paradox I care about. As implementation gets faster, definition and verification can become a larger share of the work. A large diff makes progress visible before it makes the adoption decision easy.

What a passing suite establishes

In a test conversion, passing tests are necessary evidence. They are also the thing being changed.

A converted test might still pass after losing an assertion. A fixture change could remove the condition the test was meant to exercise. A changed setup hook could affect isolation between tests. These are failure cases to check, not defects I have established in this experiment.

For a conversion like this, I would want to compare:

  • which cases run before and after, including skipped tests and data providers
  • whether assertions and expected exceptions retain their meaning
  • whether setup and teardown preserve the same state between tests
  • whether representative tests still fail when the behaviour they protect is deliberately broken

Those checks answer a narrower question than “is the new suite green?” They ask whether the conversion preserves useful tests.

The recorded result is still a feasibility result. It does not tell us how much human time the conversion saved, how much review remains, or whether adopting it is worthwhile. I would keep those questions separate when deciding the next step.

Give the agent a decision it can work within

A request such as “convert these tests” leaves the agent to fill in several choices. Should it preserve each test’s structure? May it combine repeated cases? Should it also improve fixtures?

For an initial sample, I would use a brief like this:

Convert this representative test file to Pest.
Preserve the cases, assertions, fixtures, and expected exceptions.
Keep cleanup and unrelated refactoring out of this change.
Flag behaviour you cannot translate directly.
Run the focused tests and show how the old cases map to the new ones.

This is a proposed brief, not the recorded prompt from the experiment. Its purpose is to make one conversion easy to assess before repeating the approach.

The right sample should exercise the difficult patterns, rather than only the easiest file. If the sample exposes an unclear setup rule, resolve it before asking an agent to repeat that assumption across the suite.

That is where extra thought can save work: at a decision that would otherwise be copied many times.

What the research adds

The research is useful here because it separates the feeling of speed from measured results.

In METR’s July 2025 study, 16 experienced open-source developers completed 246 tasks in repositories they knew well. Allowing early-2025 AI tools made tasks take 19% longer on average, even though participants believed the tools had helped. This was a result for that setting and those tools, not a general verdict on AI-assisted development.2

METR’s February 2026 follow-up complicates that picture. The researchers believed newer tools likely provided more benefit, but warned that participant and task selection made their new estimate unreliable. Developers who did not want to work without AI were less likely to participate or submit some tasks.3

DORA’s 2025 research looked at a different level: software delivery across teams. It reported positive relationships between AI adoption and delivery throughput, alongside a negative relationship with delivery stability. Those associations do not establish what caused the outcome of a particular migration.4

My conclusion from these sources is practical: measure the work through review and acceptance. Neither a quick first draft nor a result from another team’s workflow can settle whether a change helped yours.

Match verification to the change

A copy edit and a test-framework conversion need different levels of scrutiny. The consequence of a mistake, the amount changed, and the strength of the available checks should determine the review effort.

For a repeated conversion, that may mean examining representative transformations before scaling up, checking exceptions separately, and splitting the result so reviewers can follow it. For a small local edit, reading the diff and running a focused check may be enough.

An AI tool can help with this work too: map old cases to new ones, identify unusual patterns, or propose a check. Its explanation still needs to agree with the files and observed behaviour. Two agents agreeing is not independent evidence that a conversion preserved an assertion.

A useful record would include elapsed time, human review time, corrections, and what remains unresolved. Four days of background work is elapsed time; it is not four days of engineer effort. Keeping those measures separate makes the next comparison more useful.

Start learning early

There is no need to finish a complete specification before trying anything. A small conversion can reveal the questions the specification needs to answer. The mistake is scaling an unresolved assumption before that feedback arrives.

That connects this workflow to building agency: take an early step that produces useful evidence, then use it to decide how far to go.

For this conversion, a passing suite gives us a concrete next question: does the changed suite still detect the failures we rely on it to catch?

Revised 5 September 2026: added the test-conversion example, reduced the research to three directly relevant sources, and replaced broad claims about slowing down with specific verification questions.

Footnotes

  1. Experiment record, “Testing a PHPUnit-to-Pest conversion”, observed 5 September 2026. About 600,000 lines changed over four days of background AI work; the full suite passes, with further validation and adoption still pending. ↩

  2. METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, 10 July 2025. ↩

  3. METR, We are Changing our Developer Productivity Experiment Design, 24 February 2026. ↩

  4. Google Cloud / DORA, Announcing the 2025 DORA Report, 23 September 2025. ↩