refactor: redact PII with core's guard engine, and restore tool arguments too - #28

The server edition had its own copy of everything the desktop gateway does for redaction: matching, one placeholder per value, holding back a split placeholder in a stream, and restoring frame by frame. The two copies had drifted, each missing something the other had. tw-guard (core v0.37.0, ThinkWatchProject/ThinkWatch-Core#149) is now the one engine; the patterns and the {{EMAIL_1}} scheme stay ours.

What changes

• think_watch_common::pii is the single home of the pattern config and the at-rest redactor. PiiPatternConfig existed twice (common and gateway), and so did redact_blob. Both second copies are deleted.

• PiiRedactor keeps only what an in-flight redactor needs:

  • which parts of a decoded request to look at;
  • apply_to, which carries the values onto the raw request;
  • restore_body for a whole response.

  Matching is core's scan_text: patterns run on the decoded text as written, and restoring into JSON escapes what it puts back.

• Tool-call arguments are restored now. The stream shaper uses core's FrameRestorer, with one lane per content block or tool call. Before, a model asked to email [email protected] called the tool with {{EMAIL_1}} as the address. New unit test: a_tool_calls_arguments_get_the_callers_pii_back.

• Saving a pattern compiles it exactly as the redactor will, with NFA and DFA capped the same way. The placeholder prefix must be 1–32 letters, digits or underscores; a brace in it would make a placeholder indistinguishable from text.

• The admin "try patterns" endpoint reads the label up to the last underscore, so CUSTOM_EMAIL is no longer reported as CUSTOM.

• PiiRedactor::new() hard-coded the six seed patterns a second time, only for tests. Tests now build from the list db/seeds.sql ships.

• GatewayError::PolicyBlocked (new in core) maps to 403 policy_blocked.

Verification

• cargo fmt --check and cargo clippy --workspace --all-targets -D warnings are clean.
• cargo nextest run --workspace --lib --bins --tests: 630 passed. 9 stream-restorer unit tests moved to core with the code, and 1 is new.
• Integration suite, run with core as a local path patch before tagging (same code as v0.37.0):
  • 224 passed, 23 failed.
  • 22 of the failures are identical to dev's baseline.
  • The 23rd, successful_login_decays_subnet_failure_counter, is a proof-of-work grinder hitting its 10M-iteration cap at difficulty 21 (~1% per run). It passed when rerun.
  • pii_streaming and content_filter_pii pass.

No release.

🤖 Generated with Claude Code