Capacity headroom for a growing index

The collection grew 4% last month, as it has every month, and this month latency went up by a factor of five. Nothing else changed. The working set stopped fitting, and the growth rate that was comfortable for a year became an incident in a week.

Vector databases fail this way because their performance depends on a structure being resident, and residency is binary. The job of a capacity policy here is not to track utilisation — it is to predict the date you cross the line, far enough ahead that the fix is a planned change.

The three limits, and which one you hit first

Utilisation dashboards usually track one resource. You have at least three ceilings and they arrive in a workload-dependent order.

Limit	Symptom on arrival	Lead time you get
Memory residency	Latency cliff, or the process is killed	Days, if you’re watching the right thing
Disk capacity	Compaction fails, snapshots fail, writes may fail	Hours, and it can be sudden
Index build capacity	A rebuild no longer fits or no longer finishes in the window	None — you find out when you need it

The third is the one nobody plans for and it is the most dangerous, because it is latent. Your serving index is fine; you simply have lost the ability to rebuild it, which means you have lost your recovery procedure for a whole class of incident. That capability disappears silently, at whatever size the peak build footprint exceeds the machine — see sizing a vector index in memory for why peak is well above steady state.

So headroom is not one number. It’s a headroom figure for each ceiling, and the binding one changes over time.

The projection

You need a chart that extrapolates, not a gauge that shows now.

The terms, all measured rather than assumed:

bytes_per_record   = measured, from a scratch collection with real data
growth_rate        = records added per week, from your own write metrics
peak_multiplier    = peak build footprint ÷ steady-state footprint, measured once
usable_capacity    = machine limit × (1 − reserve fraction)

And the number you actually want:

weeks_of_headroom = (usable_capacity ÷ peak_multiplier − current_footprint)
                    ÷ (bytes_per_record × growth_rate)

Dividing usable capacity by the peak multiplier is the part that gets left out. It means you run at a fraction of the machine so that a rebuild still fits. That feels wasteful right up until the first time you need to rebuild under pressure.

Two notes on the inputs. bytes_per_record must be measured, not derived — the metadata and overhead terms are too variable to estimate, and measuring is a ten-minute job. growth_rate should be the trailing rate, plus any known step changes: a new tenant onboarding, a new document source, a re-chunk that multiplies record count without adding documents. Step changes are what break projections built purely from history.

Setting the policy

Three thresholds, each attached to an action rather than a feeling.

Watch. The projection shows the memory ceiling inside a quarter. Action: review the growth assumptions, confirm the step changes in the pipeline, and put the mitigation on a roadmap. No urgency, no change.

Act. The projection shows the ceiling inside a month. Action: execute a mitigation now. This is the threshold that matters — it must be far enough out that the mitigation is a normal change with a normal review, not an emergency.

Stop. Headroom is inside the peak build multiplier, meaning you can no longer rebuild. Action: treat as an incident even though nothing is wrong yet, because your recovery capability is gone. Reduce or expand until a rebuild fits again, and do not accept “but it’s serving fine” as a resolution.

Alert on all three from the projection, not from current utilisation. A utilisation alert at 85% gives you no information about whether you have three weeks or three months.

The mitigations, and their lead times

Choosing under pressure means choosing the fast one, which is rarely the right one. Know these in advance:

Reduce bytes per record. Compression cuts the largest term and keeps everything resident. Requires a rebuild and a recall check, so the lead time is days — enabling vector compression on a live collection.

Reduce record count. Archive or drop what is never queried. Fastest if your partitioning allows a drop, and it is the only mitigation that improves every ceiling at once. See archiving vectors nobody searches.

Move to disk. Keeps recall, spends tail latency, needs storage you have benchmarked. Lead time is days including the validation, and it should not be attempted for the first time during an incident — running a vector index from disk.

Scale up. Immediate, if the next instance size exists and a restart is acceptable. Buys one doubling and no more, and it is worth being honest that this is a deferral rather than a fix.

Shard. The answer past a certain size regardless, because it also caps the blast radius of a rebuild. The longest lead time by far, which is exactly why it should be started at the Watch threshold rather than the Act one — splitting a collection across shards.

Rollback for each: scale-up reverts by resizing back, provided you haven’t grown into the larger machine. Compression, disk residency and sharding all revert by keeping the previous collection until a full traffic cycle has passed — which means each one temporarily increases your footprint, and the projection has to account for the mitigation itself. Archiving reverts only if the archive is restorable; verify that before dropping anything.

What to do this quarter

Measure bytes_per_record against real data in a scratch collection. Record the date and the value; re-measure when the embedding model or payload schema changes.
Measure your peak build multiplier once, by building a full-scale index in a scratch environment and watching resident memory. Record it.
Build the projection as a chart with the ceiling drawn on it. A line and a horizontal rule beats any gauge.
Set the three alerts against projected dates.
Add the projection to whatever review you already do monthly, alongside the signals in what to monitor, and re-derive the growth rate each time rather than trusting the original.

The habit that pays: whenever someone proposes a change to chunking, the embedding model, or the document sources, ask what it does to record count and to bytes per record. Most capacity surprises on a vector database are not organic growth. They are a pipeline change whose storage consequence nobody computed.