Building Systems That Stay Legible

A slow API request tells you that somebody is waiting. It does not tell you which part of the system needs to change.

In Q2 2024, I worked with the Partners and API team at Tillo on slow endpoints serving more than 500,000 requests a day. We used Datadog APM to find bottlenecks. The work included resolving N+1 queries and moving slow external calls into background jobs. Average response time on the affected endpoints fell by 20%, measured with Datadog APM during that quarter.¹

That scope matters: the result describes average response time on those endpoints. It does not establish that every endpoint improved, or that the slowest requests improved by the same amount.

The part of this work I want to carry into other projects is the connection between a symptom, an explanation, and a change. A system becomes easier to maintain when another engineer can follow that connection.

Four questions for a legible system

I call a system legible when an engineer can answer these questions without reconstructing its history from fragments:

1. What is this part responsible for?
2. Which assumptions does it rely on?
3. How does it fail?
4. Where should the next change go?

A complicated system can still answer them clearly. A small system can leave all four answers in the original developer’s head.

Make the delay explainable

The API investigation involved two different kinds of delay. They called for different changes.

An N+1 query pattern repeats database work for each item in a result. For example, a request might fetch a list of records and then issue a separate query for a related record inside a loop. Looking at the endpoint alone hides that repetition; a trace can show how database calls accumulate during the request.

A slow external call raises a different question: does the caller need its result before receiving a response? If so, moving it to a background job would change the contract. If not, the request may be waiting for work that can finish separately.

These are the design questions behind the two changes in our investigation. The following table explains the patterns; it is not a reconstruction of a particular production trace.

Delay	Question to answer	What the change must preserve
A database query repeated for each item	Can the request fetch the required data together?	The same records, permissions, and filtering
An external call inside the request	Must this work finish before the response?	A clear meaning for the response, with failures and retries handled separately if work moves to a job

A faster response is only one part of the result. Moving work to a job also creates a period in which the request has returned but the operation is unfinished. Someone operating that system needs to distinguish pending work from completed or failed work.

This is why I want a performance change to leave an explanation behind: what waited, why it waited, what changed, and how we measured the result. That gives the next engineer a starting point when the behaviour changes again.

Give expected variation a home

In the same Partners and API role, I refactored a client webhook service to use strategies so we could expand the offering more efficiently. My April 2025 résumé records that work, but does not preserve the original implementation or a measured reduction in development time.

The useful design question is where supported differences belong. In a webhook service, authentication, payloads, and failure handling are possible sources of variation. Adding a strategy is worthwhile when it gives those differences a clear place without forcing every caller to understand them.

For a service like this, I would review the boundary with three questions:

• What must every implementation do?
• Which behaviour may each implementation vary?
• Which failures must the common delivery process handle?

The last question keeps the interface honest. If a partner-specific implementation can silently change what counts as successful delivery, the caller still needs to know its internals.

An extension point also needs a limit. A new interface is not useful merely because another partner might need something different one day. Start with the differences the system actually supports, and add another responsibility when a real integration requires it.

Leave a useful account of the change

For the next performance fix or integration change, I would put five things in the pull request:

• Symptom: the behaviour that prompted the work.
• Evidence: a trace, failing example, or contract that explains it.
• Decision: the responsibility being changed and why it belongs there.
• Failure: what happens when the new path cannot complete.
• Verification: what was checked, including the scope of any measured result.

This does not need a long design document. A trace and a few precise sentences can be enough. The test is whether another engineer can use them to explain the change without asking its author.

For how this applies when an AI tool writes the first implementation, read The Velocity Paradox.

Revised 5 September 2026: expanded the API example, clarified the performance measure, and replaced broad project summaries with specific design questions.

Footnotes

1. API performance work at Tillo, Q2 2024. The project record gives the request volume, changes, measurement tool, and result; its facts were confirmed for the site’s 5 September 2026 update. ↩