Compaction, and when to force it

Latency has been drifting up for two weeks with no deploy and no unusual growth, and the one metric that moved in the same shape is segment count. Background compaction is running, it just isn’t keeping up with ingest, and every segment it hasn’t merged yet is another structure every query has to search.

Compaction is the maintenance task most likely to be quietly broken, because when it falls behind nothing fails — queries just get slower in proportion to how far behind it is.

Why segment count multiplies work

Most engines that accept continuous writes do not modify one big index in place. They accumulate new records into a fresh segment, index that segment, and merge segments together in the background. The reason is that inserting into an existing graph or cluster structure is expensive and contends with reads, while building a small new structure is cheap and can happen off to the side.

The consequence is that a query has to search every segment and merge the results. Ten segments means ten traversals, ten result heaps, and one merge. The per-segment cost is not proportional to its size either — a traversal has a fixed setup cost and a search-breadth floor, so twenty small segments cost considerably more than one segment holding the same records.

Two secondary effects make it worse than the multiplier suggests:

• Recall degrades. Each segment is searched with its own budget and returns its own top candidates. Splitting the same data across many segments means each traversal sees a thinner slice of the neighbourhood, and the merged result can miss neighbours that a single index would have found. So a compaction backlog costs quality, not just time.
• Deleted records survive longer. In most designs, space from deletes is reclaimed during merge. A compaction backlog is therefore also a tombstone backlog — see why deleting vectors doesn’t free memory.

Is it keeping up?

Three signals, in the order they are useful.

Segment count over time. Flat or sawtooth is healthy: it climbs as writes arrive and drops when merges land. A staircase that only goes up means compaction is losing. Chart it next to your query latency on the same axis window — the correlation is usually unmistakable and it saves an argument.

Time since last successful merge. More diagnostic than the count. A high count that is actively being worked through is a temporary state after a bulk load. A high count with nothing merging in hours is a broken configuration.

The proportion of segments that are small. A healthy tiered merge policy leaves you with a few large segments and a handful of recent small ones. A long tail of small segments that never grow up means the merge policy’s thresholds are not being met — often because a size or count trigger is set for a much lower ingest rate than you now have.

Why it falls behind

Work through these before forcing anything, because forcing compaction on a system that is structurally unable to keep up buys you a few hours.

1. It is throttled. Most engines rate-limit background maintenance so it doesn’t starve queries. That limit was set for a default workload. If your ingest rate went up by an order of magnitude and the limit didn’t, this is your answer.
2. It is competing for the same resource as ingest. Compaction is IO-heavy and CPU-heavy in different phases. If ingest is saturating the same device, merges will make slow progress indefinitely.
3. There isn’t enough free space. Merging writes a new segment before dropping the inputs, so it needs headroom roughly proportional to the size of the segments being merged. On a volume that is nearly full, merges fail or are skipped — sometimes silently, sometimes in a log nobody reads.
4. Long-running readers are holding old segments open. Snapshot isolation means a segment cannot be dropped while something is reading it. A stuck client or a long analytical scan can pin generations of segments.
5. The merge policy is doing what it was told. Some configurations deliberately avoid merging segments above a maximum size. Once your segments hit that ceiling you accumulate large segments forever, which is fine, and then people misread the flat-ish count as a problem.

Forcing it

Only after the above, and only with the following in place.

1. Confirm free space. You need room for the merge output alongside its inputs, plus whatever your snapshot and write-ahead log need at the same time. This is the most common way a forced compaction turns into an outage.
2. Pick the window. Off-peak, and long enough that you are not going to cancel it halfway. Cancelling a merge usually discards its progress, so a forced compaction interrupted at 90% has cost you the IO and given you nothing.
3. Reduce or pause ingest if you can. Not always possible; when it is, the merge finishes far sooner and the arithmetic stops being a race.
4. Trigger it, then watch query latency while it runs, not afterwards. Expect degradation during the merge — you are adding a large IO and CPU consumer. What you are watching for is whether the degradation is tolerable, and whether it stays flat or escalates.
5. Check segment count actually dropped, and that latency came down with it. If the count fell and latency didn’t, segments were not your problem and you should go back to the diagnosis order.
6. Fix the underlying cause in the same week. A forced compaction is a symptom treatment. If you forced it once you will force it again, and the second time will be during an incident.

Rollback: there isn’t one, and this is the important thing to understand before starting. Compaction is not reversible — the merged segment replaces its inputs and the old layout is gone. What you have instead is abort: stop the operation and accept that the work is wasted and the segments remain unmerged. Take a snapshot first, so that a merge that corrupts something (rare, but this is the operation that rewrites your data) has a recovery path. Verify the snapshot restores before you rely on it — the procedure is in backing up a vector database.

The steady state you want

Compaction should be boring, which means the configuration has to match the ingest pattern rather than the default.

• Give maintenance a resource budget you chose. If your engine throttles background work, set the throttle deliberately against your actual ingest rate. A limit nobody has revisited since installation is the single commonest cause of this whole problem.
• Separate bulk work from continuous work. Large backfills should not go through the same path as trickle writes; see bulk loading a collection that’s already serving.
• Alert on segment count and merge age, not just latency. Both are leading indicators. Latency is the lagging one, and by the time it pages you the backlog is large enough that the fix is disruptive.
• Keep free-space headroom as a policy, not a reaction. Compaction, snapshots and rebuilds can all want space simultaneously, and the moment they do is the moment you least want to be resizing a volume.

If you only take one habit from this: put segment count on the same chart as p99 latency, on the default dashboard, permanently. It converts the most common slow-drift incident on a vector database from a multi-hour investigation into a glance.