✳Experiments
What happens if I try this?
Updated 5 September 2026. Tests of AI code review and test conversion, alongside earlier workflow notes. These are observations from my own workflow, not controlled benchmarks.
Further validation pendingObserved 5 September 2026
Testing a PHPUnit-to-Pest conversion
- Question
- Could background AI work carry out a large test conversion that is worth taking through further review?
- Setup
- PHPUnit, Pest, and background AI work
- Constraint
- A passing suite is one check. It does not establish that the conversion is ready for adoption.
- Observed result
- About 600,000 lines changed over four days of background AI work, and the full suite passes. This is a feasibility result, not a shipped migration.
Next step: Further validation and an adoption decision are still pending.
Related: Engineering practices at Tillo
Observed trade-offObserved 5 September 2026
How much context does an AI reviewer need?
- Question
- How does limiting Harbourmaster to the diff affect review cost and useful findings?
- Setup
- TypeScript, OpenCode, a review panel, verification agent, and coordinator
- Constraint
- Cost and runtime need to be considered alongside accuracy and useful findings.
- Observed result
- The initial version used too many tokens exploring the codebase. Moving to diff-only review reduced cost and runtime, but narrower context also reduced accuracy and findings.
Next step: Keep checking findings when choosing the context and models for a review.
Related: The AI review tool
Snapshot: 30 August 2026April–August 2026
Agent-assisted delivery loops
- Question
- Testing where multi-agent workflows genuinely improve engineering output, and where orchestration overhead outweighs the gains.
- Setup
- Codex, Cursor, repo checklists, and parallel agent handoffs
- Constraint
- Useful speed only counts if review quality, integration discipline, and ownership stay intact.
- Observed result
- Parallel agents help most with repo orientation, first-draft implementation, and comparison work. The failure mode is orchestration theatre: too many agents, vague briefs, and more review debt than progress.
Next step: Keep the loop small: one clear owner, explicit checks, and tight handoffs instead of open-ended autonomy.
Related: The Velocity Paradox
ConcludedMarch 2026 to April 2026
Specification-first building
- Question
- Tested whether clearer intent, quality gates, and better prompts could improve architecture and communication when using AI tools.
- Setup
- Prompt scaffolds, acceptance criteria, and lightweight decision logs
- Constraint
- AI punishes vague intent immediately. Bad briefs turn into plausible but generic output.
- Observed result
- The biggest gain is not raw speed. It is forcing clearer thinking earlier, which improves architecture notes, pull requests, and conversations with humans as well as tools.
Next step: Carry the useful parts into future work: acceptance criteria, small decision records, and explicit checks.
Related: What I am focused on now
ConcludedMarch 2026
Product-system sketches
- Question
- Used small tools and interface sketches to expose where product behaviour and backend constraints did not line up.
- Setup
- Small Astro pages, interface notes, and API-shape experiments
- Constraint
- Interesting product ideas usually fail at the joins between interface, workflow, and backend reality.
- Observed result
- The strongest sketches are the ones that expose permissions, trust, handoffs, and maintenance load early. If those stay vague, the concept is not ready.
Next step: Use permissions, trust, handoffs, and maintenance load as an early checklist for future sketches.
Related: Selected work
How to read these notes
Each entry records a narrow question and the constraint that kept the test honest. A concluded entry stays here so its date and result do not drift into a claim about current work.
A useful result can be negative. If a workflow adds more review than it saves, that is evidence to make the loop smaller or stop it.