Your RAG system retrieves support documents that sometimes contain Aadhaar numbers and account numbers from past tickets. A user query accidentally surfaces another customer's PII. How do you prevent PII leakage at the retrieval and generation layers?

What you need to know

Why this leak happens

Past tickets were indexed as they were written, including customers' Aadhaar and account numbers. Retrieval finds chunks by similarity, not by who owns them, so a question like "why was my account frozen?" can pull another customer's similar ticket. Once PII is in the index, it is one similar-sounding query away from anyone.

The four layers

1. Redact before embedding — detect PII and replace it with typed placeholders like <AADHAAR_1>, so the text still reads naturally.
2. Scope retrieval — filter by tenant, owner and sensitivity, with the filter built on the server from the authenticated session and applied inside the search.
3. Scan the output — run the same detector on the generated answer; block or redact.
4. Detect and audit — a hit at ingestion means redaction is failing upstream and should alert someone; a hit at output is a near-miss incident.

Detecting Indian identifiers properly

A plain "12 digits" regex flags order numbers and misses formatted Aadhaar numbers. Validate with the checksum, which removes most false alarms.

1import re23D = [[0,1,2,3,4,5,6,7,8,9],[1,2,3,4,0,6,7,8,9,5],[2,3,4,0,1,7,8,9,5,6],[3,4,0,1,2,8,9,5,6,7],4     [4,0,1,2,3,9,5,6,7,8],[5,9,8,7,6,0,4,3,2,1],[6,5,9,8,7,1,0,4,3,2],[7,6,5,9,8,2,1,0,4,3],5     [8,7,6,5,9,3,2,1,0,4],[9,8,7,6,5,4,3,2,1,0]]6P = [[0,1,2,3,4,5,6,7,8,9],[1,5,7,6,2,8,3,0,9,4],[5,8,0,3,7,9,6,1,4,2],[8,9,1,6,0,4,3,5,2,7],7     [9,4,5,3,1,2,6,8,7,0],[4,2,8,6,5,7,3,9,0,1],[2,7,9,3,8,0,6,4,1,5],[7,0,4,6,9,1,3,2,5,8]]89def verhoeff_ok(num: str) -> bool:10    c = 011    for i, d in enumerate(reversed(num)):12        c = D[c][P[i % 8][int(d)]]13    return c == 01415AADHAAR = re.compile(r"\b[2-9]\d{3}[ -]?\d{4}[ -]?\d{4}\b")16PAN = re.compile(r"\b[A-Z]{5}[0-9]{4}[A-Z]\b")1718def find_aadhaar(text):19    return [m for m in AADHAAR.finditer(text) if verhoeff_ok(re.sub(r"\D", "", m.group()))]

Presidio and similar tools include some Indian recognisers; combine them with checks like this and an ML-based detector for names and addresses, then test on your own tickets.

Question the corpus

A resolved ticket's resolution is knowledge ("frozen accounts are released after re-KYC"). The customer's identity is not. Often the best fix is to index a cleaned summary of the resolution, not the transcript.

Measure both directions

A real-life example

Scenario, numbers made up. A bank's service assistant indexes 2 million past tickets. A customer asking about a failed NEFT transfer is shown part of another customer's ticket, including an account number.

The team takes the index offline for the affected product, re-ingests with checksum-validated Aadhaar and PAN detection, account-number patterns tied to the bank's formats and an ML name detector, and indexes resolution summaries instead of transcripts. Every chunk gets a customer_id or public tag, and retrieval filters on the session's customer ID. On a labelled sample of 1,000 tickets, detector recall is 98% for Aadhaar and 95% for account numbers; the output scan catches the few that remain. The incident is reported under the bank's breach process.

Follow-up questions to expect

• "Why not just scan the output?" — It is a backstop; the PII is still in the index and in prompts sent to the model, and formats the scanner misses leak straight through.
• "Why must the filter be built on the server?" — A filter supplied by the client or chosen by the model can be manipulated; identity must come from the authenticated session.
• "Do placeholders hurt retrieval?" — Very little; the meaning of the text stays, and nobody should be searching past tickets by someone's Aadhaar number anyway.