Turning a Measurement Into a Change

The failure is rarely analytical. It is producing a correct finding that nobody acts on, until the reporting is ignored.

Why findings stall

They point at architecture. "The god module invalidates everything" is correct and is a quarter of work.

They belong to another team.

No owner, so the finding sits in a report.

Too many at once, so none is pursued.

The measurement is disputed, usually on the grounds that the average hides something — which is frequently a fair objection.

No mechanism to turn a finding into work, because the programme was scoped as reporting.

The sequence that works

One finding at a time, the largest.

Verified with the people affected, which settles the data dispute before it starts and usually improves the finding.

Sized: how much time, across how many people, how often.

Owned by someone who can change it, named.

With a date.

Re-measured afterwards, the same way, which is the step that decides whether the programme continues.

Reported, including when the change did not work.

Choosing the first one

Cheap, so it needs no capital case.

Visible, so people notice the programme did something.

Owned by you, not by another team.

Uncontroversial, which means not about how anyone works.

Common candidates: an editor exclusion list, a cache key fix, cancelling superseded pipeline runs, splitting a slow test file, raising a memory limit. All are hours of work and all are noticeable.

Verifying with developers

The step that is skipped and that determines whether the finding is real.

Take the number to the people it describes.

Ask whether it matches their experience.

Ask what the measurement does not know, which is where the instrumentation gaps surface.

A finding that surprises everyone who does the work is usually a data problem; one that confirms what they have been saying is usually correct and has been ignored.

Re-measuring honestly

Same measure, same definition, same period length.

After a settling period, not immediately.

Report the distribution, not just the median, because a change that improves the median and worsens the tail has made the experience worse.

Report null results. A programme that only reports successful interventions is selecting, and everyone reading it works that out.

The reporting habit

A standing slot in an existing forum, not a separate meeting.

A short list of open findings with owners and dates.

Closure reported, including what the re-measurement showed.

Without this the programme becomes a dashboard, and a dashboard nobody acts on is deleted within a year — usually correctly.

The open findings list

The artefact that keeps a measurement programme from becoming a dashboard.

One row per finding: what, the sizing, the owner, the date, the status.

Short. More than about eight open items means none is being pursued.

Reviewed in an existing forum, not a dedicated one.

Closed only after re-measurement, not after the change was made.

Deferred items kept with their sizing, which is the evidence for the next resourcing conversation and prevents rediscovering the same finding every six months.

Reporting a null result

The discipline that makes the positive results believable.

We changed X, we re-measured, nothing moved.

Say why you think that is, usually that it was not the constraint.

Say what you will try next.

Do not quietly drop it, which is what usually happens and which is noticed.

A programme that has never reported a failed intervention is selecting its results, and the people reading it work that out within a year.