Running Only What Is Affected

Most changes affect a small fraction of the codebase. Running everything is insurance against not knowing which fraction.

How selection works

From the dependency graph. A change to module A affects A and everything that depends on it. Tests in unaffected modules are skipped.

From coverage data. Record which tests execute which code; on a change, run the tests that touched the changed lines.

From history. Tests that have failed with changes to these files before.

The first is the most reliable and requires the module structure to be real. The second is more precise and more fragile. The third is a heuristic and should not be used alone.

The risk

Selection can be wrong. A test genuinely affected by a change might not be selected, the change merges, and the failure surfaces later somewhere confusing.

Which is why selection needs a safety net, always.

The safety net

Run everything on the main branch, after merge, on a schedule or on every merge. Selection speeds up the pull request loop; the full suite still runs somewhere.

Run everything before release.

Measure escapes: failures caught by the full run that selection skipped. This is the number that tells you whether selection is trustworthy, and almost nobody tracks it.

Widen the selection when escapes appear. Selection is a tuning exercise, not a switch.

Where selection is unreliable

Worth knowing before trusting it.

Reflection and dynamic loading, which the static graph cannot see.

Configuration and data files, where a change affects behaviour without touching code.

Generated code, where the generator is the real dependency.

Integration and end-to-end tests, which touch everything and are therefore always selected — meaning selection saves nothing on the slowest part of the suite.

Shared fixtures, where a change to test infrastructure affects tests that no code-level analysis links to it.

Handle these by declaring them: files that always trigger the full suite, listed explicitly, reviewed periodically.

What it saves

Proportional to how modular the codebase is. A well-structured codebase might run a small fraction of tests on a typical change. A monolith with one module runs everything, always, and no selection tool changes that.

Which returns to the dependency graph: selection is one more thing that structure determines and tooling cannot fix.

Cheaper things to do first

If selection looks like a large project, these deliver sooner.

Split fast from slow. Run unit tests on every change, integration tests less often. Crude and effective.

Split by directory, manually, using ownership boundaries that already exist.

Run the affected module's own tests first, so the most likely failure surfaces early even if everything eventually runs.

Fail fast, so a broken change stops rather than completing a twenty-minute suite.

What to measure

Proportion of tests selected on a typical change.

Time saved against the full run.

Escape rate, which is the correctness measure and the one that matters.

Proportion of changes that trigger the full suite anyway, which tells you whether the always-run list has grown until selection does nothing.

The always-run list

Selection needs an explicit set of triggers that bypass it, and that set needs maintaining.

Files that always trigger the full suite: build configuration, shared test infrastructure, dependency lock files, CI definitions.

Declared explicitly, in one place, with a comment saying why each is there.

Reviewed quarterly, because the list grows.

Measured: what proportion of changes hit it. If most changes trigger the full suite anyway, selection is doing nothing and the list is the reason.

Measuring escapes

The correctness measure for selection, and the reason a safety net is not optional.

Compare the full run on the main branch against what selection chose for each merged change.

A failure the full run caught and selection skipped is an escape.

Track the rate. Zero over a long period means selection can be trusted and possibly widened; a rising rate means it cannot.

Investigate each one, because escapes cluster around specific mechanisms — reflection, configuration, generated code — and each investigation improves the always-run list.