Files
optimclaw/src/workspace/privacy.rs
T
212d661e20 feat(workspace): layered memory with sensitivity-based privacy redirect (#1112)
* feat(workspace): layered memory with sensitivity-based privacy redirect

Introduce MemoryLayer type for named memory layers with sensitivity
levels and write permissions. Layers map to synthetic user_id values
in workspace tables, enabling shared/private memory isolation.

- Add MemoryLayer, LayerSensitivity types with default_for_user()
- Add layer-aware write methods (write_to_layer, append_to_layer)
- Add PatternPrivacyClassifier to guard shared layer writes
- Add optional 'layer' parameter to memory_write tool and HTTP API
- Add 'redirected' and 'actual_layer' fields to write response
- Add MEMORY_LAYERS env var (JSON) for layer configuration
- Workspace user_id now derived from GATEWAY_USER_ID (was hardcoded "default")
- 10 integration tests for layered memory operations

Addresses prerequisite for Issue #59 (multi-tenancy).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add explicit default to memory_write layer schema

Add "default": "private" to the layer parameter's JSON schema so
LLM tool consumers can see the default without reading code.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract resolve_layer_target to deduplicate layer writes

Consolidate shared layer-lookup, writable check, and privacy
classification logic from write_to_layer and append_to_layer into a
single resolve_layer_target helper.

Flagged on #349 review — the duplication originates in this PR.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback on layered memory PR

- Fix email regex pipe bug in TLD character class (privacy.rs)
- Add append support to web memory_write handler via `append` field
- Validate MemoryLayer name/scope: reject empty, check duplicates
- Remove hardcoded 'private' default from tool schema; omit layer
  fields from output when no layer specified
- Document scope isolation risk for multi-tenant (Issue #59)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address adversarial review findings

- CRITICAL: fix identity file protection bypass via trailing slash
  (normalize target path before protection checks)
- HIGH: check private layer is writable before privacy redirect
- HIGH: map LayerNotFound/ReadOnly to proper 4xx HTTP status codes
- HIGH: honor `append` field in non-layer HTTP write path
- MEDIUM: remove redundant DB fetch in append_to_layer (narrower
  TOCTOU window)
- MEDIUM: remove dead memory_write_handler from handlers/memory.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: opt-in privacy classifier, force override, confidence scoring

Address review feedback from @zmanian:

- Privacy classifier is now opt-in via with_privacy_classifier() instead
  of always-on. Default hardcoded patterns (doctor, therapy, email, phone)
  had unacceptable false positive rates in household contexts. LLM chooses
  the correct layer via system prompt; regex can't improve on that.
- Add ConfigurablePrivacyClassifier for operator-supplied patterns.
- PatternPrivacyClassifier defaults narrowed to hard PII only (SSN,
  credit card, credentials).
- Add force param to write_to_layer/append_to_layer to skip classifier.
- PrivacyClassifier trait returns SensitivityResult { is_sensitive,
  confidence } instead of bool, ready for probabilistic classifiers.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove redundant heartbeat match arm in memory_write

The heartbeat arm was identical to the catch-all — resolved_path
already points to paths::HEARTBEAT when target is "heartbeat".

Addresses review feedback from gemini-code-assist on #1112.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: return Result from PatternPrivacyClassifier::new()

Replace .expect() with proper error propagation per project
no-panics policy. Remove Default impl (unused in production).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: move memory_layers from GatewayConfig to WorkspaceConfig

Resolve merge conflicts between HEAD (transcription, search, env helpers)
and the workspace config branch. GatewayConfig no longer owns memory_layers;
WorkspaceConfig::resolve() handles parsing, validation (name length >64,
character set, empty scope, duplicates), and fallback defaults.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: strengthen privacy classifier and layer isolation coverage

Add 8 privacy classifier edge case tests (format variants, keywords,
longer documents, empty/partial inputs) and 5 layer write isolation
integration tests (cross-scope invisibility, overwrite, empty path,
sensitive-to-private no-redirect).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: tautological test assertion and add WorkspaceConfig validation tests

Replace always-true `is_ok() || is_err()` in write_empty_path_to_layer
with actual behavior assertion (write succeeds with normalized empty path).

Add 8 unit tests for WorkspaceConfig::resolve() covering valid JSON parsing,
invalid JSON, empty/long/invalid-char layer names, empty scopes, duplicates,
and default fallback behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: cargo fmt after staging merge

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: [email protected] <[email protected]>
2026-03-20 22:15:29 -07:00

277 lines
8.1 KiB
Rust

use regex::Regex;
/// Result of privacy classification, including confidence level.
///
/// Confidence enables downstream callers to apply thresholds (e.g., only
/// redirect above 0.8) and supports future upgrade to LLM-based classifiers
/// that produce probabilistic scores.
#[derive(Debug, Clone)]
pub struct SensitivityResult {
pub is_sensitive: bool,
pub confidence: f32,
}
/// Classifies content as potentially sensitive for privacy purposes.
///
/// Used to guard writes to shared memory layers -- if content is flagged
/// as sensitive, it can be redirected to the private layer instead.
pub trait PrivacyClassifier: Send + Sync {
/// Classify content and return sensitivity with confidence score.
fn classify(&self, content: &str) -> SensitivityResult;
}
/// Pattern-based privacy classifier using regex matching.
///
/// Default patterns target hard PII (SSN, credit card numbers) where silent
/// redirect is clearly correct. Ambiguous terms (health vocabulary, contact
/// info) are intentionally excluded — they cause false positives in household
/// contexts and silently redirect content the user intended to share.
///
/// Operators who need broader coverage should use `ConfigurablePrivacyClassifier`
/// with domain-specific patterns.
pub struct PatternPrivacyClassifier {
patterns: Vec<Regex>,
}
impl PatternPrivacyClassifier {
pub fn new() -> Result<Self, regex::Error> {
let pattern_strs = [
// SSN — always PII
r"\b\d{3}-\d{2}-\d{4}\b",
// Credit card (basic) — always PII
r"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b",
// Credentials and auth tokens — high-confidence PII
r"(?i)\b(password|passwd|api[_-]?key|auth[_-]?token|secret[_-]?key)\b",
];
let patterns = pattern_strs
.iter()
.map(|p| Regex::new(p))
.collect::<Result<Vec<_>, _>>()?;
Ok(Self { patterns })
}
}
impl PrivacyClassifier for PatternPrivacyClassifier {
fn classify(&self, content: &str) -> SensitivityResult {
let is_sensitive = self.patterns.iter().any(|p| p.is_match(content));
SensitivityResult {
is_sensitive,
// Regex is binary — matched or not. Always full confidence.
confidence: if is_sensitive { 1.0 } else { 0.0 },
}
}
}
/// User-configurable privacy classifier.
///
/// Accepts custom regex patterns at construction time, allowing operators
/// to tune sensitivity for their use case (e.g., drop health terms that
/// cause false positives, add domain-specific patterns).
///
/// ```
/// use ironclaw::workspace::privacy::ConfigurablePrivacyClassifier;
/// use ironclaw::workspace::privacy::PrivacyClassifier;
///
/// let classifier = ConfigurablePrivacyClassifier::new(vec![
/// r"\b\d{3}-\d{2}-\d{4}\b".into(), // SSN only
/// ]).unwrap();
/// assert!(classifier.classify("SSN: 123-45-6789").is_sensitive);
/// assert!(!classifier.classify("saw the doctor today").is_sensitive);
/// ```
pub struct ConfigurablePrivacyClassifier {
patterns: Vec<Regex>,
}
impl ConfigurablePrivacyClassifier {
/// Create a classifier from user-supplied regex strings.
///
/// Returns an error if any pattern fails to compile.
pub fn new(pattern_strs: Vec<String>) -> Result<Self, regex::Error> {
let patterns = pattern_strs
.iter()
.map(|p| Regex::new(p))
.collect::<Result<Vec<_>, _>>()?;
Ok(Self { patterns })
}
}
impl PrivacyClassifier for ConfigurablePrivacyClassifier {
fn classify(&self, content: &str) -> SensitivityResult {
let is_sensitive = self.patterns.iter().any(|p| p.is_match(content));
SensitivityResult {
is_sensitive,
confidence: if is_sensitive { 1.0 } else { 0.0 },
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn classifier() -> PatternPrivacyClassifier {
PatternPrivacyClassifier::new().unwrap()
}
// Hard PII — must always trigger
#[test]
fn detects_ssn() {
let result = classifier().classify("My SSN is 123-45-6789");
assert!(result.is_sensitive);
assert_eq!(result.confidence, 1.0);
}
#[test]
fn detects_credit_card() {
let result = classifier().classify("Card: 4111 1111 1111 1111");
assert!(result.is_sensitive);
assert_eq!(result.confidence, 1.0);
}
#[test]
fn detects_password() {
assert!(classifier().classify("my password is hunter2").is_sensitive);
}
#[test]
fn detects_api_key() {
assert!(
classifier()
.classify("set the api_key to sk-1234")
.is_sensitive
);
}
// Household content — must NOT trigger (previous false positives)
#[test]
fn allows_normal_household_content() {
let result = classifier().classify("We need to buy groceries for dinner Saturday");
assert!(!result.is_sensitive);
assert_eq!(result.confidence, 0.0);
}
#[test]
fn allows_doctor_mention() {
assert!(
!classifier()
.classify("the doctor's office called about Saturday")
.is_sensitive
);
}
#[test]
fn allows_email_address() {
assert!(
!classifier()
.classify("email [email protected] about the leak")
.is_sensitive
);
}
#[test]
fn allows_phone_number() {
assert!(
!classifier()
.classify("call the restaurant at 555-123-4567")
.is_sensitive
);
}
#[test]
fn allows_medical_terms_in_context() {
assert!(
!classifier()
.classify("Started new medication for anxiety")
.is_sensitive
);
}
#[test]
fn configurable_with_custom_patterns() {
let c = ConfigurablePrivacyClassifier::new(vec![
r"\b\d{3}-\d{2}-\d{4}\b".into(), // SSN only
])
.unwrap();
assert!(c.classify("SSN: 123-45-6789").is_sensitive);
// Health terms no longer trigger with SSN-only config
assert!(!c.classify("saw the doctor today").is_sensitive);
}
#[test]
fn configurable_rejects_bad_regex() {
let result = ConfigurablePrivacyClassifier::new(vec!["[invalid".into()]);
assert!(result.is_err());
}
#[test]
fn configurable_empty_patterns_allows_everything() {
let c = ConfigurablePrivacyClassifier::new(vec![]).unwrap();
assert!(!c.classify("My SSN is 123-45-6789").is_sensitive);
}
// Format variants
#[test]
fn detects_credit_card_no_separators() {
assert!(
classifier()
.classify("card 4111111111111111 on file")
.is_sensitive
);
}
#[test]
fn detects_credit_card_with_dashes() {
assert!(
classifier()
.classify("Card: 4111-1111-1111-1111")
.is_sensitive
);
}
#[test]
fn detects_ssn_bare() {
assert!(classifier().classify("123-45-6789").is_sensitive);
}
#[test]
fn detects_auth_token_keyword() {
assert!(
classifier()
.classify("set auth_token to abc123")
.is_sensitive
);
}
#[test]
fn detects_secret_key_keyword() {
assert!(
classifier()
.classify("the secret_key is sk-prod-xyz")
.is_sensitive
);
}
#[test]
fn detects_pii_in_longer_document() {
let content = "Meeting notes from Thursday.\n\
Discussed budget and timeline.\n\
SSN is 999-88-7777 for the insurance form.\n\
Action items: follow up with vendor.";
assert!(classifier().classify(content).is_sensitive);
}
#[test]
fn empty_string_is_not_sensitive() {
assert!(!classifier().classify("").is_sensitive);
}
#[test]
fn partial_ssn_not_sensitive() {
assert!(
!classifier()
.classify("code 123-45 in the system")
.is_sensitive
);
}
}