Detecting PII exposure without another model call

ReclaimLLM now flags personally identifiable information — emails, phone numbers, SSNs, and credit card numbers — that shows up inside a captured session or passes through the LLM gateway. This is a new anomaly detector, AD-016, and it runs alongside the 14 detectors that already cover secrets, external network egress, and usage anomalies. Findings show up in the same place those do: the Signals dashboard, with the same severity tiers and alerting rules.

What's new on the session detail page itself: if a session has a flagged finding, you'll see a "Detected PII in this session" panel right above the transcript. Click an entry and it jumps straight to the tool call or message that matched, auto-expands it if it was collapsed, and highlights the exact substring inline.

How it works

This detector does not call a model to decide whether something is PII. That would undercut the point of the product — if detecting leaked data required sending that data to another LLM first, we'd be reintroducing the exact risk we're trying to catch. Instead, detection is pure regex matching followed by checksum validation:

• Credit card candidates are confirmed with a Luhn checksum.
• SSN candidates are checked against the Social Security Administration's own invalid-range rules.
• Phone candidates exclude reserved fictional test ranges.

This keeps false positives low without any machine learning, and detection cost is negligible — arithmetic on strings, not an API call. It runs as part of the existing hourly/daily anomaly sweep, scanning tool-call previews for agent/hook sessions and request/response previews for proxy traffic. No new background job, no new storage, no new external dependency.

Severity is tiered differently depending on where the match was found:

• Proxy traffic: PII in a request/response that went through the gateway has already reached the model provider's API — starts at a higher severity floor, escalates further on repeat occurrences.
• Agent/hook sessions: a local file read or tool-call result hasn't left the machine yet — starts lower, escalates to critical only if the same session also shows outbound network activity.

On the session detail page, a second, independent implementation of the same matching logic runs entirely in your browser, against content you're already viewing — nothing is sent anywhere. It exists purely to make an already-confirmed finding navigable.

A bug we caught while building this

We found that the transcript's "OnlyMessages" filter (on by default) collapses tool-call and thinking blocks out of view, while our PII badge was computed from a message's full content. That meant a "PII: phone" tag could appear with nothing visible underneath, because the real content was sitting inside a collapsed block — which looked exactly like redaction even though nothing was redacted. Fixed: a flagged block is never collapsed by that filter now.

Limitations

• Only structured PII with a fixed format — email, phone, SSN, credit card. Names/addresses aren't in scope.
• Obfuscated or reformatted data (split across lines, base64-encoded) won't be caught — it's a regex, not a semantic classifier.
• Previews are bounded-length; a flagged value past the preview window in a very large payload may be missed.
• Luhn validity rules out most noise but doesn't guarantee a sequence is a real card number.

If you hit a false positive, a false negative, or navigation that doesn't land where you expect, tell us — this is exactly the kind of detector where real-world session data surfaces edge cases we haven't thought of.