Isolating tenants in one vector store
One customer onboarded a corpus several times larger than anyone else’s, and started sending queries with a filter that matches a tiny fraction of it. Your p99 is now their p99. Separately, your capacity projection is no longer a projection of your product’s growth — it is a projection of that one account’s growth, and you find out about their plans after they execute them.
Multi-tenancy on a vector store creates both problems at once: a shared performance envelope and a shared capacity ceiling. They have different fixes and the same starting point, which is being able to attribute anything to a tenant at all.
Attribution comes first
You cannot isolate what you cannot measure. Before choosing an isolation model, get these per tenant:
- Record count and footprint. Footprint from record count times your measured bytes-per-record from sizing a vector index in memory is close enough, and it is available immediately.
- Query rate, split by filtered and unfiltered.
- Query latency distribution. Per tenant, at percentiles. The aggregate p99 tells you that something is slow; the per-tenant breakdown tells you which account and therefore which conversation to have.
- Write and delete rate. A tenant re-syncing their whole corpus nightly is an indexing load problem, not a query problem — when ingest outruns indexing.
- Growth rate. Per tenant, so the capacity projection can be decomposed.
Most engines will not give you these directly, so they usually come from your service layer rather than from the database. Emit them there. This is the cheapest work in this post and it changes every subsequent decision.
The four isolation models
| Model | Blast radius of one noisy tenant | Cost |
|---|---|---|
| Shared collection, filter by tenant ID | Everyone | Lowest |
| Shared collection, partitioned by tenant | Query cost isolated, resources shared | Low |
| Collection per tenant | Query and structure isolated, machine shared | Moderate |
| Deployment per tenant | Everything isolated | Highest |
Shared collection with a tenant filter is where everyone starts and it is fine for a long time. Its failure mode is the selective-filter problem: a filter matching a small fraction of a large collection can force repeated traversals, so a small tenant in a big collection is the expensive case rather than the cheap one. That is counterintuitive and it is why small tenants sometimes have the worst latency — the diagnosis is step 2 of when your vector database gets slow.
Partitioning by tenant removes the filter cost by making it a routing decision, and makes offboarding a drop rather than a scattered delete. It is the highest-value change here relative to its cost, and the full argument and migration are in partitioning by the filter you always use. It does not isolate memory or CPU: all tenants still share the machine.
Collection per tenant adds real structural isolation — separate index parameters per tenant if you want them, independent rebuilds, independent compression decisions. The cost is a collection count that grows with your customer base and per-collection fixed overhead paid once per tenant, which for a long tail of small tenants can exceed the data itself. Find your engine’s practical limit on collection count before choosing this, and check it against your sales forecast rather than today’s number.
Deployment per tenant is the only model that isolates resources, and it is the only one where a tenant’s rebuild cannot affect anyone else. It multiplies everything you operate by your customer count. Reasonable for a small number of large accounts; unmanageable as a general policy.
The arrangement that usually wins is mixed: partitioned shared collection for the long tail, dedicated collections or deployments for the handful of accounts that are large or noisy enough to matter. That mapping also makes the largest tenants the ones you can shard independently — see splitting a collection across shards.
Limits, which do the actual protecting
Isolation of storage does not stop one tenant consuming all the query capacity. That requires limits, and they belong in your service layer because the engine generally will not enforce them per tenant.
Rate limit per tenant, not just globally. A global limit protects the database and lets one tenant consume the whole allowance. Per-tenant limits are what keep a shared envelope fair.
Cap the expensive parameters per tenant. Search breadth, top-k, and the number of results a query may request are all knobs that let a caller spend more of your latency budget. If tenants can set them, they can set them badly. Cap them.
Cap record count per tenant, and enforce it at ingest. This is the one that protects your capacity plan. A tenant with no cap is an unbounded commitment, and the moment you discover it is the moment their backfill is already running. The cap does not have to be low — it has to exist, and it has to fail the write rather than accept it.
Set a query timeout per tenant. A slow query from one tenant occupying a worker is a slow query for everyone behind it. Timeouts and their interaction with retries are covered in query timeouts and retry storms.
Isolate the bulk path. A tenant onboarding should not push a large load through the live write path. Route it as in bulk loading a collection that’s already serving, and make that the standard onboarding procedure rather than something you do when the load is big enough to worry about.
Moving a tenant out of the shared collection
The operation you will do repeatedly, so it is worth having as a procedure.
- Confirm the tenant’s footprint and query rate from your per-tenant metrics, so the destination is sized rather than guessed.
- Create the destination — collection or deployment — with the same dimensionality, metric, and index parameters unless you are deliberately changing them. Change one thing at a time.
- Load the tenant’s records from your source corpus, filtered to that tenant, on the bulk path. Not by copying out of the shared collection, which does not give you re-embeddable source data anyway.
- Dual-write or queue the tenant’s writes for the duration, and decide which explicitly.
- Verify count and recall for that tenant specifically. Their slice of the fixed query set, run against both. If your query set has no queries for this tenant, add some before you start.
- Route the tenant’s reads to the destination. In your service layer, per tenant, which means you need tenant-aware routing before the first migration — build it once.
- Watch the shared collection’s latency afterwards. Removing a tenant should improve it. If it does not, that tenant was not your problem and you should re-read your attribution metrics.
- Then remove the tenant’s records from the shared collection, along the partition boundary if you have one, and reclaim.
Rollback: route the tenant’s reads back to the shared collection. That works only while their records are still there and still current, which is why step 8 is last and separate — the temptation is to reclaim immediately, and doing so converts a routing rollback into a reload. Keep both for a full traffic cycle, and treat step 8 as its own scheduled change with its own date.
What goes on the dashboard
- Footprint and record count per tenant, ranked. The top few entries are your capacity plan.
- Query latency per tenant at p99. Ranked the same way. This is how you find the noisy neighbour before they find you.
- Limit rejections per tenant. A tenant hitting their cap is a conversation to have proactively; discovering it from a support ticket is worse.
- Growth rate per tenant, feeding the decomposed projection in capacity headroom for a growing index. A single aggregate growth line hides the step change of one account onboarding, which is the most common way the projection turns out to be wrong.
The general principle: in a multi-tenant vector store, your capacity plan and your latency budget are both being written by your customers. Per-tenant attribution and per-tenant limits are how you take the pen back.