Reindexing without downtime

Someone upgraded the embedding model. The new vectors have different dimensionality — or the same dimensionality but a completely different geometry, which is worse, because nothing errors. Every vector in the collection is now wrong and the index has to be rebuilt from scratch.

This will happen several times in the life of a system. Embedding models improve, chunking strategies get revised, a metadata field becomes filterable. How gracefully you handle it is mostly decided before the first rebuild, by which strategy you set up.

Why you can’t just update in place

Two reasons, and the second is the one that catches people.

Mixed geometries don’t compare. Vectors from two different embedding models occupy different spaces. Cosine similarity between them is a number, and it is meaningless. A partially-migrated collection returns confident nonsense — no errors, no warnings, just quietly wrong results ranked against each other.

Graph indexes are built, not assembled. In the HNSW family, each vector’s neighbour lists were chosen relative to the vectors present at insertion time. Replacing vectors underneath an existing graph leaves the connectivity describing relationships that no longer hold. Some engines handle incremental updates well; none of them handle “every vector changed” as an update.

So the rebuild is a rebuild. The only question is where the new index lives while it’s being built.

Strategy 1: in-place rebuild

Drop the collection, rebuild it, resume serving.

Cost: full downtime for the duration of the build, which at meaningful scale is hours.

When it’s right: genuinely more often than people admit. Internal tools, batch pipelines, anything with a maintenance window and no overnight users. If you can take the outage, this is the simplest and least expensive option and there is no shame in it.

Rollback: none, unless you took a snapshot first. Take a snapshot first. See backing up a vector database — and note that a snapshot you’ve never restored is not a rollback plan.

Strategy 2: blue-green collections

Build the new index alongside the old one under a different name, then switch reads over atomically.

The procedure:

Snapshot the current collection. Non-negotiable, and verify it’s readable.
Create the new collection with the new dimensionality/settings, under a versioned name — docs_v4 next to docs_v3.
Backfill it from your source of truth. Not from the old collection: you need to re-embed, and re-embedding from stored vectors is impossible anyway. This is why the source corpus must live somewhere that isn’t the vector store.
Let it catch up on writes. During the backfill, new documents are still arriving. Either dual-write to both collections from the start, or track a watermark and replay the tail.
Verify before switching. Run a fixed query set against both. You are not looking for identical results — the model changed, so results should differ. You are looking for the new collection returning sensible results at the expected count and latency. A collection that silently backfilled 60% of the corpus looks fine until you count.
Switch reads by flipping an alias if your engine supports one, or a config value if not.
Keep the old collection for a defined period — one deploy cycle at minimum.
Drop the old collection once you’re confident.

Cost: roughly double the steady-state memory for the overlap period, plus the operational complexity of dual-writes. Both collections are fully resident simultaneously.

Rollback: flip the alias back. This is the entire reason blue-green is worth its cost — rollback is one operation and takes effect immediately. Do not skip step 7; an old collection dropped too eagerly turns a thirty-second rollback into a fresh multi-hour rebuild.

When it’s right: the default for anything user-facing, if you can afford the memory.

Strategy 3: shadow reads before the switch

Blue-green with a verification phase in front of the cutover.

After step 5, send a sampled fraction of live production queries to both collections, serve the old results, and log both. Now you’re comparing on real traffic rather than a fixed query set.

What to look at:

Latency distribution on the new collection — p50, p95, p99, not the mean.
Result overlap at your top-k. Low overlap isn’t automatically bad, but it should be explicable by the model change.
Empty or short result sets. A sharp rise means a filter or a backfill gap.
The queries where they disagree most. Read a dozen by hand. This finds problems no aggregate metric surfaces.

Cost: the extra query load on the new collection, plus somewhere to log and diff. Worth it when a bad switch is expensive.

Rollback: you haven’t switched yet, so there’s nothing to roll back. That’s the value.

Strategy 4: shard-by-shard

If the collection is sharded, rebuild one shard at a time.

Cost: memory overhead is only one shard’s worth rather than the whole collection’s — the reason to choose this. But during the migration your shards are genuinely inconsistent with each other, which is only acceptable if queries are naturally partitioned (per-tenant, per-region) so no single query spans both old and new geometry.

If queries fan out across all shards, this strategy is unsafe. You’d be merging scores from two embedding spaces, which is the mixed-geometry problem with extra steps.

Rollback: per shard, which makes it granular but slow — you may need to roll back several.

When it’s right: large multi-tenant deployments where the memory doubling of blue-green is genuinely unaffordable and tenants map cleanly onto shards.

Choosing

	Downtime	Peak memory	Complexity	Rollback
In-place	Hours	1× + scratch	Lowest	Restore from snapshot
Blue-green	None	~2×	Moderate	Flip alias
Shadow reads	None	~2×	Higher	Nothing to undo
Shard-by-shard	None	1× + one shard	Highest	Per shard

Pick on the memory row first, since it’s the hard constraint, then on how expensive a bad switch would be.

The prerequisite nobody sets up in advance

Your source corpus must be re-embeddable from something other than the vector database.

You cannot recover source text from vectors. If the only copy of your chunked, cleaned, ready-to-embed corpus is in the vector store’s payload field, every rebuild starts with extracting it — assuming you stored it at all. Plenty of teams discover at rebuild time that they didn’t.

Keep the chunked corpus in object storage or a relational table, keyed by the same IDs, with the chunking parameters and model name recorded alongside. This costs almost nothing and converts a rebuild from an archaeology project into a batch job.

Before the first one

Rehearse it. Build a scratch collection at a tenth of production scale, run your chosen strategy end to end, and time it. You will find at least one thing — usually the backfill throughput, sometimes a dimensionality setting that can’t be changed after creation.

Then watch memory closely during the real run; what to monitor covers which signal moves first, and sizing covers why the peak is where it is.