Parallelism and Its Ceiling

Parallelism is the first thing people reach for and the one with the hardest ceiling. Knowing where the ceiling is prevents spending money on machines that change nothing.

The ceiling

The longest path through the dependency graph.

No amount of parallelism makes a chain of ten sequential tasks faster than the sum of those ten. If your longest path is eight minutes, eight minutes is your floor with infinite cores.

Compute it. Most build tools can report it, and the number is frequently most of the total duration — which means the parallelism conversation is over before it starts and the real work is the graph.

Where parallelism goes wrong

Contention on a shared resource. Twelve parallel tasks writing to one database, one port, one temporary directory. Adding workers increases contention and can reduce throughput.

Memory pressure. Each worker needs memory; when the sum exceeds what the machine has, swapping makes everything slower. This is the most common cause of a build that gets slower with more workers.

Disk saturation, which parallel builds hit before CPU saturation on many machines.

Cold caches on new workers, where scaling out adds machines that have to fetch everything.

Scheduling overhead on very fine tasks, where coordination costs more than the work.

Finding the right level

Measure, do not assume. Run the same build at several worker counts and plot duration.

The curve flattens, and then frequently rises. The flattening point is the useful setting; past it you are paying for nothing.

It differs by machine. A developer laptop and a CI runner have different optima, and one setting for both is wrong for at least one.

Re-measure after major changes, because the optimum moves with the graph.

Test parallelism specifically

Splitting across machines is bounded by the slowest shard, not by the number of shards.

Split by measured duration, not by file count. Equal counts produce wildly unequal shards.

Rebalance periodically, because durations drift.

Watch for tests that cannot run in parallel — shared fixtures, fixed ports, database state. These force serialisation and frequently determine the shard boundaries.

Diminishing returns arrive quickly. Going from four to eight shards helps; from sixteen to thirty-two usually adds startup overhead and cost for seconds of wall time.

The cost side

Parallelism trades money for wall time, and the exchange rate worsens as you add workers.

Compute the cost per minute saved at each level. It rises steeply past the flattening point.

Compare against the alternative: caching and selection reduce both cost and duration, and should be exhausted first.

What to do instead when the ceiling is reached

Shorten the critical path, which means the graph.

Move work off the blocking path into post-merge or scheduled runs.

Cache more, so the path is not executed at all.

Accept the number and say so, which is a legitimate outcome: "eight minutes is the floor given the current structure, and reducing it is an architecture project" is a clearer position than continuing to add runners.

Plotting the curve

One experiment that replaces an argument about worker counts.

Run the same build at several worker counts — two, four, eight, sixteen.

Plot duration against workers.

The curve flattens, then frequently rises.

The flattening point is the setting; past it you are paying for nothing, and past the inflection you are paying to be slower.

Repeat per machine class, because a laptop and a CI runner have different optima and one setting for both is wrong for at least one.

Naming the floor

A legitimate outcome that is frequently avoided because it sounds like failure.

"Eight minutes is the floor given the current structure."

Say it, with the longest path as the evidence.

Stop adding runners, which is money for nothing past that point.

Present the structural work as the only remaining move, sized honestly.

A clearly stated ceiling is a better position than continued spending that produces no measurable change and eventually exhausts the programme's credibility.