SecurityIntermediate
PII Redaction & Data Masking: Cleaning Sensitive Customer Data Before LLM Ingestion
Direct Answer & Overview
Personally Identifiable Information (PII) redaction strips or masks credit card numbers, Social Security Numbers, phone numbers, and names from text prompts before they reach external LLM APIs, ensuring airtight compliance.
1.Client-Side & Gateway-Level PII Masking
Even with Zero Data Retention policies, transmitting raw SSNs, medical record numbers, or banking credentials over the wire increases breach exposure. Masking data before sending it to third-party APIs eliminates liability at the source.
2.Rule-Based + NER Masking (Microsoft Presidio)
High-assurance pipelines combine regular expressions (for structured patterns like credit cards, emails, and phone numbers) with Named Entity Recognition (NER) models (such as spaCy or HuggingFace transformers) to identify names, physical addresses, and organization titles.
3.Reversible Tokenization (Pseudonymization)
To allow the AI to answer contextually without seeing actual identities, pipelines substitute real entities with synthetic tokens: 'John Doe' becomes `<PERSON_1>`, and 'Acme Corp' becomes `<ORG_1>`. After receiving the AI's response, a local lookup vault reverses the synthetic tokens back to the original values before displaying the result to the end user.
Reversible PII Masking Wrapperpython
import re
class ReversiblePIIMasker:
def __init__(self):
self.vault = {}
self.counter = 0
def mask(self, text: str) -> str:
# Regex for US Social Security Numbers
ssn_pattern = r"\b\d{3}-\d{2}-\d{4}\b"
def replace_ssn(match):
self.counter += 1
token = f"<SSN_TOKEN_{self.counter}>"
self.vault[token] = match.group(0)
return token
return re.sub(ssn_pattern, replace_ssn, text)
def unmask(self, text: str) -> str:
for token, original in self.vault.items():
text = text.replace(token, original)
return text
masker = ReversiblePIIMasker()
clean_prompt = masker.mask("Customer SSN is 123-45-6789. Check tax liability.")
print("Sent to LLM:", clean_prompt)
# Output: Customer SSN is <SSN_TOKEN_1>. Check tax liability.Frequently Asked Questions
Does masking PII reduce model reasoning quality?
No. Since the model reasons about relationships rather than specific identities, synthetic tokens like <PERSON_1> preserve 100% of semantic context.
Can open-source libraries like Presidio run on edge gateways?
Yes, Microsoft Presidio runs locally as a lightweight Python service with sub-10ms latency per paragraph.
What is the difference between anonymization and pseudonymization?
Anonymization permanently destroys the mapping back to the individual; pseudonymization uses a reversible key/vault to restore identity in authorized contexts.
A100
API100 Engineering Team
Infrastructure & Latency Research