Redact sensitive data

The redactors are in tnsai-quality, package com.tnsai.quality.redaction. For path and credential masking without that module, see Mask paths and credentials. To mask PII in action input, see PII input guardrails.

Quick start

static AgentBuilder redacted(AgentBuilder builder, LLMClient base) {
    Redactor redactor = PatternRedactor.withDefaults();

    // Conversation memory: messages are scrubbed before they are stored.
    MemoryStore memory = new RedactingMemoryStore(new InMemoryStore(), redactor);

    // Agent events: every field that can carry user input is scrubbed.
    AgentEvents.setPublisher(
        new RedactingAgentEventPublisher(new Slf4jAgentEventPublisher(), redactor));

    // LLM call logs: prompts, responses, tool arguments and errors are scrubbed.
    LLMClient observed = new CapturingLLMClient(base, JsonLLMPricingRegistry.defaultRegistry(),
        new RedactingLLMCallPublisher(new Slf4jLLMCallPublisher(), redactor));

    return builder.llm(observed).memoryStore(memory);
}

AgentEvents.setPublisher replaces the publisher for the whole JVM. To stack redaction with sampling, see Sampling.

Default patterns

PatternRedactor.withDefaults() uses RedactionPatterns.defaults(). Each pattern is also a constant on RedactionPatterns, such as RedactionPatterns.EMAIL.

A match becomes [REDACTED:<name>], for example [REDACTED:email]. No default pattern matches a placeholder, so redacting text twice gives the same result.

Custom patterns

static Redactor withCustomPatterns() {
    RedactionPattern medicalRecord = RedactionPattern.of(
        "medical_record_id", "MRN-\\d{8}", Severity.CRITICAL);
    RedactionPattern plate = RedactionPattern.of(
        "license_plate_tr", "\\b\\d{2}\\s?[A-Z]{1,3}\\s?\\d{2,4}\\b", Severity.MEDIUM,
        match -> match.length() >= 7);         // runs after the regex matches

    List<RedactionPattern> patterns = new ArrayList<>(RedactionPatterns.defaults());
    patterns.add(medicalRecord);
    patterns.add(plate);
    return new PatternRedactor(patterns);
}

The optional validator checks each regex match, for rules a regex cannot express, such as a checksum. A pattern without a validator accepts every match.

Combine redactors

Every type below implements Redactor, so they nest freely.

static Redactor perTenant() {
    Redactor everything = PatternRedactor.withDefaults();
    Redactor cardsOnly = new PatternRedactor(List.of(RedactionPatterns.CREDIT_CARD, RedactionPatterns.US_SSN));

    Redactor dispatcher = TenantPolicyRedactor.builder()
        .tenant("acme", cardsOnly)
        .defaultRedactor(everything)
        .build();
    return new AuditingRedactor(dispatcher, LoggingRedactionAuditListener.atInfo());
}

TenantPolicyRedactor looks up the tenant on every call: first RedactionContext.tenantPolicyId(), then the tenant id of the call's EventContext. With no tenant, or one that is not registered, it uses the default redactor.

A RedactionAuditEvent carries the scope, the event context, a count per pattern, the highest severity and the total number of findings. It never carries the redacted text. LoggingRedactionAuditListener.atInfo() and atDebug() log at that level, and always log CRITICAL findings at WARN. An exception thrown by a listener is logged and does not stop redaction.

Mask paths and credentials

com.tnsai.redaction in tnsai-core holds maskers that need no tnsai-quality dependency.

static void mask(Path runFolder, Path homeFolder) {
    Masker outside = Masker.hidingOutside(runFolder);
    outside.apply("read /home/ann/.ssh/id_rsa");     // "read [REDACTED:path]"

    Masker names = new PathRedactor(Map.of(runFolder, "<run>", homeFolder, "~"));
    names.apply(runFolder + "/notes.log");           // "<run>/notes.log"
}

hidingOutside does not recognize relative paths, ~/..., Windows drive paths, percent-encoded paths, the path part of a URL, or the part of a path after a space. PathRedactor matches a folder only in the form the platform writes it.

Write your own redactor

Redactor has three methods:

• scrubString(input, context) returns a RedactionResult with the new text and one RedactionFinding per match (pattern, offsets, placeholder, severity).
• scrubValue(value, context) walks maps and lists and scrubs every string in them. It has a default implementation.
• maybeContainsSensitive(input) is a cheap pre-check.

An implementation must be thread-safe, because one instance serves every sink. It must also be idempotent: scrubbing the output again gives the same text. RedactionContext names the sink through RedactionScope, for example MEMORY_WRITE, LLM_PROMPT or LLM_RESPONSE.

Trade-offs

• Patterns can match harmless text, such as an order number shaped like a phone number. Add a validator to the pattern to tighten it.
• Formats outside the catalog, such as names, addresses or driver's licences, are not caught. Add custom patterns or your own Redactor.
• Redacting very large payloads, such as long LLM responses, adds latency. Put sampling before redaction so only the events that are kept get scrubbed.

Related

• PII input guardrails
• Guardrails
• Observe LLM calls
• Sampling