You're building RAG for 500+ enterprise customers. A bug causes one tenant's chatbot to retrieve another's private data. How do you architect retrieval so this can never happen — even with a buggy query?

What you need to know

The strongest control is the one that does not depend on application code being correct. That is why this answer leans on the database.

Row-level security with pgvector

1ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;2ALTER TABLE chunks FORCE ROW LEVEL SECURITY;          -- applies to the table owner too34CREATE POLICY tenant_isolation ON chunks5  USING (tenant_id = current_setting('app.tenant_id')::uuid);67-- per request, inside a transaction, from the verified token:8SELECT set_config('app.tenant_id', $1, true);         -- true = local to this transaction9SELECT id, text FROM chunks ORDER BY embedding <=> $2 LIMIT 10;

Two details interviewers like. Superusers and roles with BYPASSRLS ignore policies, so the application must connect as an ordinary role. And FORCE ROW LEVEL SECURITY makes the table owner obey the policy too. Setting the tenant with set_config(..., true), which behaves like SET LOCAL, limits it to the current transaction, so a pooled connection does not carry one tenant's id into the next request.

Choosing the isolation model

Remove the ability to get it wrong

1. Resolve scope at the edge — the gateway verifies the token and puts the tenant in the request context.
2. No tenant parameters — service methods read scope from the context only; code review rejects any new retrieval path that bypasses the scoped data layer.
3. Verify results — every chunk carries its tenant id; the retriever asserts on it and fails closed.
4. Test continuously — contract tests per retrieval path, plus a canary document per tenant probed on a schedule.

Also scope anything derived from retrieval: semantic caches, conversation memory and logs must all be keyed by tenant.

Metrics that prove it

Zero cross-tenant hits on canary probes, and 100% of retrieval calls carrying a resolved tenant scope in traces. The second catches a new code path before it becomes an incident.

A real-life example

Scenario (illustrative numbers). A legal-document startup runs RAG for 520 law firms on Postgres with pgvector, filtering by tenant_id in application code. A new "similar clauses" endpoint ships without the filter, and a firm's associate sees a clause from another firm's merger agreement. The firm reports it, and the startup has a breach notice to write.

The fix: RLS with a forced policy, an application role without BYPASSRLS, and a transaction-local tenant setting per request from the verified token. The team replays the buggy endpoint in staging: it now returns only the caller's own clauses, because the database filters rows before the query code sees them. They add contract tests for all 14 retrieval endpoints and a canary probe every 10 minutes; tracing shows 100% of retrieval spans with a tenant scope.

Follow-up questions to expect

• "Does RLS slow vector search?" — The policy adds a filter; index tenant_id, and with HNSW check filtered recall, since selective filters reduce results from approximate search.
• "What about a dedicated vector database?" — Use its namespace or tenant-partition feature, plus the same edge-derived scope and post-retrieval check.
• "How do admins do cross-tenant support?" — A separate, audited admin path with its own role, never by switching off the tenant scope in the normal path.