Caching in Continuous Integration

A fresh runner has nothing: no dependencies, no build outputs, no container layers. Restoring that state is frequently longer than the work itself.

What is worth caching

In rough order of return.

Dependency downloads. Package manager caches, module directories. Large, slow to fetch, and identical across most runs.

Build outputs, through the build tool's own remote cache, which is a different and better mechanism than a generic file cache.

Container layers, where images are built in the pipeline.

Compiler and tool caches, where the toolchain has one.

Test fixtures that are expensive to generate.

Not: source code, which is cheap to fetch, or anything that changes every run.

Keying it correctly

The part that determines whether it works.

Key on the thing that determines the content. Dependency caches key on the lock file hash, not on the branch name.

Include the runner environment in the key: operating system, architecture, language version. A cache restored onto a different platform is worse than useless.

Use restore fallbacks. An exact-match key with a prefix fallback means a changed lock file still restores a mostly-correct cache and updates it, rather than starting from nothing.

Never key on the commit, which guarantees a miss on every run.

Never key on the branch alone, which shares between unrelated changes and produces stale content.

When caching makes things slower

Real and common.

Restoring more than you save. A large cache over a slow link can exceed the time to just do the work. Measure both.

Caching something cheap to recompute.

Cache upload on every run, including runs where nothing changed. Upload conditionally.

Cache size growth, where an unbounded cache accumulates every version of everything and the restore gets slower each week.

Compression cost exceeding transfer savings on fast networks.

Measure restore time, save time and hit rate. A cache with a low hit rate and a long restore is a net loss and should be removed.

Correctness

A stale cache produces wrong results, and diagnosing that is expensive because the build is green on a machine and red on another.

Key precisely enough that a restored cache is genuinely valid.

Version the key so you can invalidate everything by bumping it when something changes that the key does not capture.

Do not let untrusted contexts write. Forked pull requests should read only, or a malicious change can poison what everyone else restores.

Warm versus cold

Autoscaled runners start cold, which is the trade-off in the queueing note: adding capacity can raise execution time.

Persistent runners keep warm caches and accumulate state, which causes its own class of bug — a build that works only on a machine that has built it before.

A reasonable arrangement: ephemeral runners with a good remote cache, so state comes from a controlled source rather than from whatever was left on the machine.

What to report

Hit rate per cache, separately. One overall figure hides which cache is broken.

Restore and save duration.

Cache size over time, which catches unbounded growth before it becomes the bottleneck.

Execution time with and without a warm cache, which is the number that justifies the whole arrangement.

The restore-versus-recompute test

Some caches cost more than they save, and the check takes one experiment.

Time the pipeline with the cache enabled.

Time it with that cache disabled, everything else unchanged.

Compare. If disabling is faster, remove the cache.

Common causes of a net loss: a large cache over a slow link, caching something cheap to recompute, uploading on every run.

Repeat per cache, not for all of them together, because one bad cache can hide the benefit of three good ones.

Bounding cache growth

An unbounded cache gets slower every week until someone deletes it urgently.

Set a size limit and an eviction policy.

Key precisely enough that stale entries fall out naturally rather than accumulating.

Version the key prefix, so everything can be invalidated deliberately.

Monitor size over time, which catches the growth before restore duration becomes the bottleneck.

A cache that takes ninety seconds to restore is not a cache; it is a slow download with optimistic branding.