How do access controls work in enterprise RAG systems?

What you need to know

1. Ingest — copy the source ACL onto every chunk: tenant_id, allowed_groups, maybe classification.
2. Authenticate — get the user id from the verified session token, never from the request body or question.
3. Resolve — look up the user's groups in the identity provider, with a short cache.
4. Filter inside the search — the store only considers chunks the user may read.
5. Audit — log user, tenant, query and returned chunk ids.

The filter in code

Here is a working version with Chroma through LangChain. allowed_groups is stored as a list on each chunk.

1def acl_filter(tenant_id: str, groups: list[str]) -> dict:2    if not tenant_id or not groups:3        raise PermissionError("no tenant or groups: deny by default")4    clauses = [{"allowed_groups": {"$contains": g}} for g in groups]5    group_rule = clauses[0] if len(clauses) == 1 else {"$or": clauses}6    return {"$and": [{"tenant_id": {"$eq": tenant_id}}, group_rule]}78def retriever_for(user):9    return vs.as_retriever(search_kwargs={10        "k": 20, "filter": acl_filter(user.tenant_id, user.groups)})

I tested it with three HR chunks: an engineer in all_staff got only the leave policy; an HR user also got the pay bands; a user with no groups was refused. Two details I found by running it:

• In Chroma, {"allowed_groups": {"$in": [...]}} on a list field returned nothing at all, with no error. $in checks whether a single value is in your list; for list fields you need $contains. Filter syntax differs between stores, so test every filter against real data.
• Chroma rejects an $or with only one clause, so the helper handles the single-group case.

Filter before, not after

• Pre-filtering applies the filter while searching, so you still get k permitted results.
• Post-filtering searches first, then removes forbidden results. You may get 0 of your 20, and forbidden text has already been loaded into your app's memory. Avoid it for security.

Isolation between tenants

For B2B products with many customer companies, a separate collection or namespace per tenant is safer than one shared index with a tenant_id filter. With a shared index, one forgotten filter leaks data. With separate collections, the wrong collection name returns nothing useful.

Keeping permissions fresh

When someone leaves a group, access should end quickly. Options: re-sync ACLs from the source on change events; or keep the chunk filter coarse and, for the final top results, check each document's current permission against the source system before showing it. Complex sharing rules ("anyone the owner shared with") often need a relationship-based permission service, such as OpenFGA or SpiceDB, queried at retrieval time.

A real-life example

A hospital's guideline search also indexes department protocols. Oncology research protocols are limited to the oncology group; general guidelines go to all clinical staff. Each chunk stores department and allowed_groups. A cardiology nurse asking about chemotherapy side effects gets the general guideline, while an oncology nurse also gets the research protocol.

A security review finds one gap: when a doctor moves from oncology to cardiology, the HR system updates the directory, but the search tool caches group membership for 24 hours. The team cuts the cache to 15 minutes and subscribes to directory change events to clear it immediately. They also add a nightly test that logs in as a synthetic cardiology user and asserts that zero oncology-only chunks come back for 40 oncology questions.

Follow-up questions to expect

• "Why not let the model decide what a user may see?" — Because once text is in the prompt, the model can reveal it. Access control must happen before the text reaches the model.
• "What about documents shared with individual users?" — Store allowed_users as well as groups, or check a permission service for the final candidates.
• "How do you handle permission changes at scale?" — Event-driven ACL updates on chunk metadata, plus a nightly reconciliation against the source system.