Making Tests Faster
The suite is usually slow for a small number of identifiable reasons. Finding them before parallelising, which mostly hides the problem.
Procedure
Testing
A suite that fails randomly is worse than one that is slow, because it destroys trust in the signal. Reliability is worth more than speed at the margin and is measured far less often.
6 notes
Flaky Tests
A test that fails intermittently costs more than a slow one, because it destroys trust in the whole signal. Detecting, quarantining and fixing them.
Procedure
Running Only What Is Affected
Running the whole suite on every change is simple and wasteful. Selection is the largest available saving and the easiest to get subtly wrong.
Analysis
The Shape of the Suite
The pyramid is advice about cost and feedback speed, not a rule about counts. What it is actually claiming, and where it is wrong.
Analysis
Coverage, Used and Misused
A useful diagnostic and a destructive target. What the number actually tells you, and what happens when it becomes a threshold.
Analysis
Test Data and Fixtures
Setup frequently costs more than the assertions. Where fixture time goes, and the trade-off between realism and speed.
Procedure