The Dependency Graph

You cannot make a build fast if everything depends on everything. The graph is the constraint, and it is a code organisation problem wearing a build tooling costume.

What the graph decides

How much can be skipped. A change to a leaf module invalidates only its dependents. A change to a module everything depends on invalidates everything.

How much can run in parallel. Independent subtrees can build simultaneously; a chain cannot.

How much can be cached. Fine-grained modules produce fine-grained cache entries with better hit rates.

How well test selection works, since knowing which tests are affected requires knowing what depends on what.

The shapes that hurt

The god module. One module — usually called common, core or utils — that everything depends on and that changes weekly. Every change to it rebuilds the world.

The cycle. Modules that depend on each other, forcing the tool to treat them as one unit.

The wide fan-in. A module with a hundred dependents. Not necessarily wrong, and it makes that module's stability critical.

The deep chain. A long dependency path that cannot be parallelised, so the build's minimum duration is the sum of the chain regardless of how many cores you have.

The monolith with no modules at all, which can only ever rebuild everything.

Measuring the shape

Longest path through the graph, which is the theoretical floor on build duration.

Fan-in per module, which identifies the god modules.

Rebuild breadth: for a typical change, how many modules are invalidated. This is the number that predicts everyday experience.

Cycles, which should be zero.

Most build tools can emit the graph. Producing these four numbers is an afternoon and the result usually surprises everyone.

Breaking up a god module

The most common necessary intervention and the least popular.

Find out what actually uses what. Frequently the hundred dependents use different, non-overlapping slices of it.

Split by consumer, not by theme. A module split into strings, dates and collections is tidier; a module split by which teams depend on which parts is faster.

Extract the stable parts first. The pieces that never change can move into a module that never invalidates.

Move interfaces away from implementations, so consumers depend on something that changes rarely.

Do it incrementally, and measure rebuild breadth before and after, or you cannot tell whether it helped.

Enforcing the shape

Graphs decay. Without enforcement, someone adds the convenient dependency and the boundary is gone.

Declare allowed dependencies explicitly where the tool supports it.

Fail the build on a cycle.

Alert on fan-in crossing a threshold, which catches a new god module while it is still small.

Track the longest path over time, which is the single number that says whether the structure is getting better or worse.

The uncomfortable conclusion

Build performance work eventually becomes architecture work.

Caching, parallelism and better hardware all have ceilings set by the graph. Past a point, the only remaining move is to change how the code is organised, and that is a much larger conversation than the one the build performance project was scoped for.

Say this early, rather than after six months of tuning has exhausted the easy gains.

The four numbers

An afternoon of work that describes the ceiling on every other improvement.

Longest path, which is the floor on build duration.

Fan-in per module, which identifies god modules.

Rebuild breadth for a typical change, which predicts everyday experience.

Cycle count, which should be zero.

Most build tools can emit the graph and these four numbers follow from it.

Report them quarterly. They move slowly, they move in one direction without attention, and they bound everything else in this collection.

Making the case for structural work

The hardest sell in this field, and it needs the earlier wins to be credible.

Show the longest path as the floor on build duration.

Show rebuild breadth for a typical change.

Show that the cheap work is exhausted, with the record of what was already tried.

Size it honestly, in quarters rather than weeks.

Offer a first increment: extract one stable module from the god module, measure the change in rebuild breadth, and decide whether to continue.