Merge Queues

A pull request that passed CI can still break the main branch, because it was tested against the state of the branch when it was created rather than the state it will merge into.

The problem

Two changes pass independently and conflict semantically. One renames a function, the other adds a call to it. Both are green; the merge is broken.

Nothing in a standard pull request workflow catches this.

It gets worse as merge frequency rises, which means it appears as an organisation grows and is often misread as a general decline in quality.

What a merge queue does

Changes queue rather than merging directly.

Each is tested against the result of everything ahead of it in the queue.

If it passes, it merges. If it fails, it is removed and the queue continues.

The main branch is therefore only ever updated by a change that was verified against exactly the state it landed on.

The costs

Latency. A change waits for everything ahead of it. In a busy repository this can be substantial.

Compute. Every change is tested at least twice: once on the pull request, once in the queue.

Complexity. Another system to understand, configure and debug.

Failure handling. When a queued change fails, someone must work out whether it was the change or an interaction, and flaky tests make that ambiguous.

Batching, and its trade-off

Testing changes one at a time is slow. Most queues batch several and test them together.

If the batch passes, all merge — one test run for several changes.

If it fails, you know one of them broke it and not which. The queue then bisects, which costs more runs.

Batch size is a tuning parameter: larger batches are cheaper when things pass and much more expensive when they fail.

Which means batch size should follow the failure rate. A repository with frequent failures should batch less, and a queue that batches aggressively over a flaky suite spends all its time bisecting.

The prerequisite

A merge queue over a flaky test suite does not work.

Every flaky failure removes an innocent change, triggers a bisect, and burns capacity. The queue amplifies flakiness rather than tolerating it.

Fix the flake rate first. This is the most common reason merge queue adoptions are abandoned, and it is predictable in advance.

When it is worth it

High merge frequency into one branch, where semantic conflicts actually occur.

A reliable suite, as above.

A cost of breakage that exceeds the latency cost, which is the case when a broken main branch blocks many people.

Not worth it for a small team with infrequent merges, where the occasional broken build is cheaper than the infrastructure.

What to measure

Time from approval to merge, which is what the queue adds.

Queue throughput against merge demand — a queue that cannot keep up is a growing backlog.

Rejection rate, and how much of it is flakiness rather than genuine conflict. This is the health measure.

Main branch break rate before and after, which is whether it worked.

Tuning batch size to the failure rate

The parameter that decides whether a queue helps or thrashes.

Measure the proportion of queue entries that fail.

Low failure rate: batch aggressively. Most batches pass and one run covers several changes.

High failure rate: batch smaller or not at all, because every failure triggers a bisect that costs more than the batching saved.

Recompute monthly, since the failure rate moves with the flake rate.

If the failure rate is dominated by flakes, fix those first. No batch size makes a queue work over an unreliable suite.

Reading the rejection rate

The health measure for a queue, and it needs splitting.

Genuine conflicts are the queue working: two changes that were individually fine and interact badly.

Flake rejections are the queue amplifying an existing problem, and they remove innocent changes.

Split the rate, which requires the flake detection described elsewhere.

A queue whose rejections are mostly flakes should be paused, not tuned. Fixing the suite is the prerequisite and the queue makes the flakiness more expensive, not less.