Enabling vector compression on a live collection

You need the index to fit in less memory, the obvious lever is storing each vector component in fewer bytes, and somebody has found the configuration flag. Turning it on requires rebuilding the index, changes result quality by an amount nobody in the room can state, and on several engines cannot be reversed without another rebuild.

This post is about rolling that change out so you learn the recall cost before you are committed to it. The mechanics of what different compression schemes discard is a separate subject and belongs elsewhere; here it only matters that the cost is a recall cost, it is workload-specific, and it is measurable in an afternoon.

Why you cannot look up the answer

The recall cost of compressing a vector depends on the distribution of the vectors themselves — how much of their variance sits in the components you are coarsening, how close together your true nearest neighbours are, and how many of them you need. Two collections with identical dimensionality and identical record counts, embedded with different models over different corpora, will lose different amounts.

Which means any figure you find is a measurement of somebody else’s data. It tells you the direction of the effect and nothing about the magnitude on yours. The magnitude is the only part you need, and the only way to get it is to build both and compare.

The good news is that this is a cheap experiment relative to what it decides, and it is the same instrument you already use for restores and rebuilds: the fixed query set from backing up a vector database.

The properties that determine whether this is worth trying

Before spending a build on it, check four things about your situation.

Whether reranking is available. Several engines can search the compressed representation to gather candidates and then rescore a shortlist using full-precision vectors held elsewhere. Where that is available it changes the calculus completely, because most of the recall lost to compression is recovered at the rescoring step and the memory saving is retained. Find out whether your engine does this before measuring anything, because the answer determines what you are measuring.

Where the full-precision vectors live if you rerank. On disk, usually. That makes this partly a disk-residency change, with the failure modes in running a vector index from disk — but the good version of them, because the reads are few and happen once per query rather than at every traversal hop.

Whether your recall requirement has any slack. If your current recall is already at the edge of acceptable, compression is the wrong lever and you should be looking at sharding or archiving instead. Establish the requirement as a number before you start, so that the go/no-go decision is arithmetic rather than a judgement made while looking at a chart.

Whether the parameter is fixed at collection creation. On many engines it is. That means there is no in-place enable and no in-place disable, and your rollout and your rollback are both new collections. Plan the footprint accordingly.

The rollout

Record the baseline. Recall against your fixed query set on the current collection, and p50/p95/p99 for your real query mix. Both, from the same window. Without the recall baseline you will have nothing to compare against and will end up arguing about whether results “seem worse.”
State the acceptance threshold now. How much recall you are willing to spend, written down before you see the result. This is the single step that stops the decision from being made by whoever wants the memory back most.
Build a compressed copy alongside. New collection, same source data, same dimensionality, same metric, same connectivity, compression enabled. Load from your source corpus. Everything else held constant, so that the difference you measure is the one you changed.
Measure the actual footprint. Resident memory for the compressed collection versus the original, measured rather than derived from the compression ratio. The vector payload shrinks; the structural and metadata terms do not, so the total saving is always less than the ratio suggests and sometimes much less — the term breakdown is in sizing a vector index in memory. If the measured saving does not solve your problem, stop here; you have spent one build and learned the answer.
Measure recall on the compressed copy. Same query set, same top-k. Compare against step 1 against the threshold from step 2.
Look at where it lost, not just how much. Aggregate recall hides the shape. Sort your queries by how much recall each lost and read the worst ones. Compression tends to hurt hardest where true neighbours are close together and closely spaced — if those queries are your important ones, an acceptable average is not acceptable.
Check latency in both directions. Compressed distance computations are often faster, and a smaller footprint improves cache behaviour, so latency may improve. If reranking is in play, compare the tail specifically, because the rescoring step adds reads.
Tune before deciding, if the first result fails. Raising the search-breadth parameter recovers recall at a latency cost, and on a compressed index you have latency budget to spend. Re-measure both. A compressed index at higher breadth can land at the same recall and the same latency as the original with a smaller footprint, and that is the outcome worth working for.
Shadow, then shift a fraction of reads. Send sampled live queries to both, log both result sets, and read the disagreements by hand before moving real traffic.
Keep the uncompressed collection for a full traffic cycle. Then drop it.

Rollback: point reads back at the uncompressed collection. That is instant and it is the only cheap rollback available, because reversing compression in place is another rebuild. It requires the old collection to still be receiving writes or a replayable queue of them, and it requires you not to have dropped it early — which is tempting, because the whole reason you are doing this is that you are short of memory. Budget for the overlap before you start. A compression rollout undertaken with no room for the double footprint has no rollback, and should be treated as a one-way change with a snapshot as the only way back.

The trap in the arithmetic

The reason this rollout is memory-hostile is worth stating plainly: the mitigation for being short of memory temporarily requires more memory than you have. Both collections are resident during the overlap.

Ways out, in order of preference:

Build the compressed copy on a temporarily larger machine, then shrink after the old collection is dropped. Costs a resize, keeps the rollback.
Do it one shard or one partition at a time, so the overlap is one unit’s worth rather than the whole collection’s. Requires that queries do not span units mid-migration, for the same reason shard-by-shard reindexing does — see reindexing without downtime.
Accept a snapshot as the rollback and do it in place. Fastest, and the rollback is now hours instead of seconds. Only reasonable if you have restored that snapshot recently enough to believe in it.

Fold whichever you choose into the projection in capacity headroom for a growing index, because the mitigation’s own footprint is part of the capacity problem it is solving.

Afterwards

Keep the recall probe running hourly. Compression means your quality now has a configuration dependency it did not have before, and a future parameter change can move it. The probe is described in what to monitor.
Record the measured recall cost and the date in your own notes, next to the configuration. In a year, somebody will ask what compression cost you, and the answer should not be a guess.
Re-measure after an embedding model change. The recall cost is a property of the vector distribution, so a new model invalidates the measurement even at identical dimensionality.
Re-check the footprint saving as metadata grows. Compression shrank one term. If payload fields keep being added, the fraction of your footprint that compression can help with keeps falling, and the lever you already pulled is not available again.