Making Tests Faster

The reflex response to a slow suite is more parallelism. That reduces wall time and leaves the underlying cost, which then reappears as a compute bill.

Find where the time is

Get per-test timings. Every test framework can emit them; most teams have never looked.

Sort descending. The distribution is almost always extreme: a small number of tests account for a large share of the total.

That list is the work queue, and it is usually short enough to fix in a week.

What makes tests slow

In rough order of frequency.

Real input and output. Database, filesystem, network. Frequently used where a fake would be equivalent.

Sleeps. A test that waits a fixed two seconds for something asynchronous. Replace with polling on a condition, with a timeout.

Setup repeated per test that could be done once per class or per suite.

Container or process startup per test rather than shared.

Full application boot for a test exercising one function.

Oversized fixtures. Loading a hundred thousand rows to test a filter.

Cryptography with real parameters in test contexts, where weakened parameters are safe and much faster.

The framework-level wins

Parallel execution within the suite, which needs tests to be isolated — and finding out that they are not is itself valuable.

Shared expensive resources. One container for the suite rather than one per test.

Class-level rather than method-level setup, where the state permits.

Lazy initialisation of anything not every test needs.

Turn off what tests do not need: logging at debug level, coverage instrumentation on runs where it is not collected, retries on local runs.

Where isolation and speed conflict

The genuine trade-off, and it should be decided rather than drifted into.

Fully isolated tests — fresh state per test — are slow and reliable.

Shared state is fast and produces order-dependent failures that are painful to debug.

A reasonable middle: share expensive immutable resources, isolate mutable state.

Test the isolation deliberately: run the suite in random order periodically. Failures indicate hidden dependencies, and those will eventually cause a flaky failure at the worst time.

Parallelism, used correctly

After the above, not instead of it.

Within the suite first, which needs no infrastructure.

Then across machines, by splitting the suite.

Split by measured duration, not by file count. Equal file counts produce wildly unequal shards, and the slowest shard sets the wall time.

Watch the tail. Wall time is the slowest shard; a well-balanced split has shards within a few percent of each other.

What to measure

Total suite duration and the ninetieth percentile, separately.

Slowest twenty tests, as a standing report.

Suite duration per thousand tests, which shows whether tests are getting slower as they get more numerous.

Setup time as a proportion of total, which is frequently the majority and is invisible without asking.

The slowest twenty

A standing report that keeps the suite from drifting.

Collect per-test durations on every run.

Publish the twenty slowest, weekly, with their trend.

Assign the top few as work, not as a suggestion.

Watch for new entries, which are tests that got slower rather than tests that were always slow.

The list is short and the distribution is extreme, which means a week of attention on the top of it frequently halves the suite.

Random order runs

A scheduled check that finds hidden coupling before it becomes a flake.

Run the suite in random order, on a schedule rather than on every change.

Failures indicate order dependence, which is state leaking between tests.

Fix the leak, not the order.

Record the seed so a failure is reproducible.

These failures are the ones that later appear as unexplained flakes under parallelism, which makes finding them deliberately much cheaper than finding them by accident.