Partitioning by the filter you always use
Every query your service sends carries the same metadata filter — a tenant ID, a date range, a document source — and the engine is evaluating it against the whole collection on every request. Latency correlates with how selective the filter is rather than with how much data matches it, bulk loads have nowhere to go but the live write path, and a retention job that deletes one month has to delete it record by record.
Those are three separate complaints with one cause: a boundary that exists in your query pattern does not exist in your storage layout. Partitioning moves it into the layout.
What partitioning buys, and it is three things at once
Most discussions of this treat it as a query-performance change. It is, but that is the smallest of the three benefits.
A filter becomes a routing decision. When the partition key matches the filter, the engine does not evaluate a predicate at all — it selects which structures to search and ignores the rest. The work saved is not the predicate evaluation, which is cheap. It is the traversal of neighbourhoods that could never have returned a matching result, which is the expensive part of a selective filtered search.
Bulk loads get somewhere to go. New data that belongs to a partition nobody is querying yet can be loaded at full speed without contending with serving traffic, then made visible. This is the cheapest of the strategies in bulk loading a collection that’s already serving, and it only exists if the partition boundary happens to match the shape of the incoming data.
Deletion becomes a drop. Dropping a partition reclaims its space immediately and completely. No tombstones, no merge to wait for, no vacuum. This is the single most effective answer to the problem in why deleting vectors doesn’t free memory, and like the previous benefit it is available only if the boundary matches how you delete.
The three want different keys, which is the actual decision.
Choosing the key
Score each candidate key against four questions. The answers rarely all point the same way.
| Question | Why it decides things |
|---|---|
| Does it appear in nearly every query? | If some queries omit it, those queries fan out across every partition and get slower, not faster |
| Do you delete along it? | Determines whether reclamation is a drop or a compaction problem |
| Does new data arrive along it? | Determines whether bulk loads can bypass the live path |
| Is the resulting size distribution even? | Skew concentrates load and memory on one partition |
Tenant is the usual winner in multi-tenant systems because it satisfies the first three at once — queries always carry it, offboarding a customer is a delete along it, and onboarding one is a load along it. Its weakness is the fourth: tenant sizes are almost never even, and a handful of large tenants will dominate. See isolating tenants in one vector store for handling that skew once you have chosen this key.
Date is the usual winner in archival or log-shaped corpora, because deletion is retention and retention is a date. Its weakness is the first question: if most queries do not restrict by date, partitioning by it makes every query fan out and buys you nothing on the query path — you keep only the deletion benefit, which may still be worth it.
Where the answers conflict, resolve toward deletion and growth rather than toward query speed. Query speed has other remedies — search parameters, more memory, compression. Reclamation and bulk loading have almost none, and a collection you cannot cheaply shrink is a capacity problem with no exit. This is the opposite of how the decision is usually framed, and it is the one worth arguing for.
What partitioning is not
It is not sharding. Partitions divide data within a deployment to change what a query has to search; shards divide data across machines to change how much any one machine holds. They are frequently configured together and they solve different problems — see splitting a collection across shards. You can partition on one machine and gain everything above.
It does not reduce total memory. The same vectors are resident, in more structures. In fact the total usually rises slightly, because each partition carries its own index overhead and per-structure fixed costs are paid more times. Work that through sizing a vector index in memory before committing, because a partitioning change made to relieve memory pressure will make it marginally worse.
It does not help unfiltered queries. They now search every partition and merge. If a meaningful fraction of your traffic is unfiltered, measure that fraction before you start.
Migrating onto a partition key
You cannot usually add a partition key to a collection that has data. Treat this as a rebuild with a different target shape.
- Measure the fraction of queries carrying the candidate key. From your query logs, not from memory. This number decides whether to proceed and nobody ever has it to hand.
- Check the size distribution. Group your existing records by the candidate key and look at the largest and smallest groups. If one group holds most of the data, you have solved less than you think and should read the skew section of the tenancy post first.
- Decide the partition count and whether it is fixed. Date keys grow partitions forever, so decide the retention boundary now and the archival path with it — archiving vectors nobody searches. Tenant keys grow with customers, so confirm your engine tolerates the partition count you will have in two years rather than the one you have today.
- Build the partitioned collection alongside the current one. Same dimensionality, same metric, same index parameters. Load from your source corpus, not from the old collection. The procedure is reindexing without downtime; the only difference is the target layout.
- Verify per partition, not just in aggregate. Record counts per partition against the source. A migration that routed a whole key value to the wrong partition passes a total count check perfectly.
- Run the recall probe with filters attached. Your fixed query set should include filtered queries. Run them against both collections and compare. This is the check that catches a routing bug, because a query filtered to a partition that does not hold its data returns an empty result rather than an error.
- Shift a fraction of reads, then the rest. Watch client-observed latency split by whether the query carried the partition key. The unfiltered slice is the one that may have got worse.
- Keep the unpartitioned collection for a full traffic cycle, then drop it.
Rollback: point reads back at the unpartitioned collection, which is why step 4 builds alongside rather than converting. Two conditions make that real: the old collection must still be receiving writes or a replayable queue of them, and it must not have been dropped early to reclaim the double footprint. Write the drop date down at step 8 rather than deciding it in the moment.
Afterwards
Partitioning adds a failure mode you did not have: imbalance. Add these signals to the dashboard described in what to monitor:
- Record count per partition, as a distribution rather than a total. The shape drifting is your warning that the key stopped being even.
- Partitions touched per query. If this climbs, queries are arriving without the key and you have lost the benefit silently.
- Query latency split by filtered and unfiltered. These are now genuinely different workloads and averaging them hides the regression.
- Empty-result rate per partition. A routing bug or an empty partition presents this way and nothing else catches it.
The habit worth adopting: whenever someone proposes a new query pattern that does not carry the partition key, treat it as a capacity change rather than a feature. It is one.