mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-27 08:00:17 +00:00
feat(workspace): layered memory with sensitivity-based privacy redirect (#1112)
* feat(workspace): layered memory with sensitivity-based privacy redirect Introduce MemoryLayer type for named memory layers with sensitivity levels and write permissions. Layers map to synthetic user_id values in workspace tables, enabling shared/private memory isolation. - Add MemoryLayer, LayerSensitivity types with default_for_user() - Add layer-aware write methods (write_to_layer, append_to_layer) - Add PatternPrivacyClassifier to guard shared layer writes - Add optional 'layer' parameter to memory_write tool and HTTP API - Add 'redirected' and 'actual_layer' fields to write response - Add MEMORY_LAYERS env var (JSON) for layer configuration - Workspace user_id now derived from GATEWAY_USER_ID (was hardcoded "default") - 10 integration tests for layered memory operations Addresses prerequisite for Issue #59 (multi-tenancy). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add explicit default to memory_write layer schema Add "default": "private" to the layer parameter's JSON schema so LLM tool consumers can see the default without reading code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: extract resolve_layer_target to deduplicate layer writes Consolidate shared layer-lookup, writable check, and privacy classification logic from write_to_layer and append_to_layer into a single resolve_layer_target helper. Flagged on #349 review — the duplication originates in this PR. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review feedback on layered memory PR - Fix email regex pipe bug in TLD character class (privacy.rs) - Add append support to web memory_write handler via `append` field - Validate MemoryLayer name/scope: reject empty, check duplicates - Remove hardcoded 'private' default from tool schema; omit layer fields from output when no layer specified - Document scope isolation risk for multi-tenant (Issue #59) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address adversarial review findings - CRITICAL: fix identity file protection bypass via trailing slash (normalize target path before protection checks) - HIGH: check private layer is writable before privacy redirect - HIGH: map LayerNotFound/ReadOnly to proper 4xx HTTP status codes - HIGH: honor `append` field in non-layer HTTP write path - MEDIUM: remove redundant DB fetch in append_to_layer (narrower TOCTOU window) - MEDIUM: remove dead memory_write_handler from handlers/memory.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: opt-in privacy classifier, force override, confidence scoring Address review feedback from @zmanian: - Privacy classifier is now opt-in via with_privacy_classifier() instead of always-on. Default hardcoded patterns (doctor, therapy, email, phone) had unacceptable false positive rates in household contexts. LLM chooses the correct layer via system prompt; regex can't improve on that. - Add ConfigurablePrivacyClassifier for operator-supplied patterns. - PatternPrivacyClassifier defaults narrowed to hard PII only (SSN, credit card, credentials). - Add force param to write_to_layer/append_to_layer to skip classifier. - PrivacyClassifier trait returns SensitivityResult { is_sensitive, confidence } instead of bool, ready for probabilistic classifiers. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: remove redundant heartbeat match arm in memory_write The heartbeat arm was identical to the catch-all — resolved_path already points to paths::HEARTBEAT when target is "heartbeat". Addresses review feedback from gemini-code-assist on #1112. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: return Result from PatternPrivacyClassifier::new() Replace .expect() with proper error propagation per project no-panics policy. Remove Default impl (unused in production). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: move memory_layers from GatewayConfig to WorkspaceConfig Resolve merge conflicts between HEAD (transcription, search, env helpers) and the workspace config branch. GatewayConfig no longer owns memory_layers; WorkspaceConfig::resolve() handles parsing, validation (name length >64, character set, empty scope, duplicates), and fallback defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test: strengthen privacy classifier and layer isolation coverage Add 8 privacy classifier edge case tests (format variants, keywords, longer documents, empty/partial inputs) and 5 layer write isolation integration tests (cross-scope invisibility, overwrite, empty path, sensitive-to-private no-redirect). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: tautological test assertion and add WorkspaceConfig validation tests Replace always-true `is_ok() || is_err()` in write_empty_path_to_layer with actual behavior assertion (write succeeds with normalized empty path). Add 8 unit tests for WorkspaceConfig::resolve() covering valid JSON parsing, invalid JSON, empty/long/invalid-char layer names, empty scopes, duplicates, and default fallback behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: cargo fmt after staging merge Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: [email protected] <[email protected]>
This commit is contained in:
@@ -0,0 +1,158 @@
|
||||
use serde::Deserialize;
|
||||
|
||||
/// Sensitivity level for a memory layer.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum LayerSensitivity {
|
||||
#[default]
|
||||
Private,
|
||||
Shared,
|
||||
}
|
||||
|
||||
/// A named memory layer with read/write permissions and a scope.
|
||||
///
|
||||
/// Layers map to synthetic `user_id` values in the workspace tables.
|
||||
/// The `scope` field is the user_id used for DB queries on this layer.
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
pub struct MemoryLayer {
|
||||
pub name: String,
|
||||
pub scope: String,
|
||||
#[serde(default = "default_true")]
|
||||
pub writable: bool,
|
||||
#[serde(default)]
|
||||
pub sensitivity: LayerSensitivity,
|
||||
}
|
||||
|
||||
fn default_true() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
impl MemoryLayer {
|
||||
/// Build the default layer set: a single private layer for the given user_id.
|
||||
pub fn default_for_user(user_id: &str) -> Vec<MemoryLayer> {
|
||||
vec![MemoryLayer {
|
||||
name: "private".to_string(),
|
||||
scope: user_id.to_string(),
|
||||
writable: true,
|
||||
sensitivity: LayerSensitivity::Private,
|
||||
}]
|
||||
}
|
||||
|
||||
/// Extract read scopes (all layer scope values).
|
||||
pub fn read_scopes(layers: &[MemoryLayer]) -> Vec<String> {
|
||||
layers.iter().map(|l| l.scope.clone()).collect()
|
||||
}
|
||||
|
||||
/// Extract writable scopes only.
|
||||
pub fn writable_scopes(layers: &[MemoryLayer]) -> Vec<String> {
|
||||
layers
|
||||
.iter()
|
||||
.filter(|l| l.writable)
|
||||
.map(|l| l.scope.clone())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Find a layer by name. Returns None if not found.
|
||||
pub fn find<'a>(layers: &'a [MemoryLayer], name: &str) -> Option<&'a MemoryLayer> {
|
||||
layers.iter().find(|l| l.name == name)
|
||||
}
|
||||
|
||||
/// Find the private layer (first layer with Private sensitivity).
|
||||
pub fn private_layer(layers: &[MemoryLayer]) -> Option<&MemoryLayer> {
|
||||
layers
|
||||
.iter()
|
||||
.find(|l| l.sensitivity == LayerSensitivity::Private)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn default_for_user_creates_single_private_layer() {
|
||||
let layers = MemoryLayer::default_for_user("alice");
|
||||
assert_eq!(layers.len(), 1);
|
||||
assert_eq!(layers[0].name, "private");
|
||||
assert_eq!(layers[0].scope, "alice");
|
||||
assert!(layers[0].writable);
|
||||
assert_eq!(layers[0].sensitivity, LayerSensitivity::Private);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_scopes_collects_all() {
|
||||
let layers = vec![
|
||||
MemoryLayer {
|
||||
name: "private".into(),
|
||||
scope: "alice".into(),
|
||||
writable: true,
|
||||
sensitivity: LayerSensitivity::Private,
|
||||
},
|
||||
MemoryLayer {
|
||||
name: "shared".into(),
|
||||
scope: "shared".into(),
|
||||
writable: true,
|
||||
sensitivity: LayerSensitivity::Shared,
|
||||
},
|
||||
MemoryLayer {
|
||||
name: "reports".into(),
|
||||
scope: "reports".into(),
|
||||
writable: false,
|
||||
sensitivity: LayerSensitivity::Shared,
|
||||
},
|
||||
];
|
||||
let scopes = MemoryLayer::read_scopes(&layers);
|
||||
assert_eq!(scopes, vec!["alice", "shared", "reports"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn writable_scopes_filters_read_only() {
|
||||
let layers = vec![
|
||||
MemoryLayer {
|
||||
name: "private".into(),
|
||||
scope: "alice".into(),
|
||||
writable: true,
|
||||
sensitivity: LayerSensitivity::Private,
|
||||
},
|
||||
MemoryLayer {
|
||||
name: "reports".into(),
|
||||
scope: "reports".into(),
|
||||
writable: false,
|
||||
sensitivity: LayerSensitivity::Shared,
|
||||
},
|
||||
];
|
||||
let scopes = MemoryLayer::writable_scopes(&layers);
|
||||
assert_eq!(scopes, vec!["alice"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn find_returns_matching_layer() {
|
||||
let layers = MemoryLayer::default_for_user("alice");
|
||||
assert!(MemoryLayer::find(&layers, "private").is_some());
|
||||
assert!(MemoryLayer::find(&layers, "shared").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserialize_from_json() {
|
||||
let json = serde_json::json!({
|
||||
"name": "shared",
|
||||
"scope": "shared",
|
||||
"writable": true,
|
||||
"sensitivity": "shared"
|
||||
});
|
||||
let layer: MemoryLayer = serde_json::from_value(json).unwrap();
|
||||
assert_eq!(layer.name, "shared");
|
||||
assert_eq!(layer.sensitivity, LayerSensitivity::Shared);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deserialize_defaults() {
|
||||
let json = serde_json::json!({
|
||||
"name": "private",
|
||||
"scope": "alice"
|
||||
});
|
||||
let layer: MemoryLayer = serde_json::from_value(json).unwrap();
|
||||
assert!(layer.writable); // default true
|
||||
assert_eq!(layer.sensitivity, LayerSensitivity::Private); // default
|
||||
}
|
||||
}
|
||||
+178
-3
@@ -45,6 +45,8 @@ mod document;
|
||||
mod embedding_cache;
|
||||
mod embeddings;
|
||||
pub mod hygiene;
|
||||
pub mod layer;
|
||||
pub mod privacy;
|
||||
#[cfg(feature = "postgres")]
|
||||
mod repository;
|
||||
mod search;
|
||||
@@ -61,6 +63,17 @@ pub use search::{
|
||||
FusionStrategy, RankedResult, SearchConfig, SearchResult, fuse_results, reciprocal_rank_fusion,
|
||||
};
|
||||
|
||||
/// Result of a layer-aware write operation.
|
||||
///
|
||||
/// Contains the written document plus metadata about whether the write
|
||||
/// was redirected to a different layer (e.g., sensitive content redirected
|
||||
/// from shared to private).
|
||||
pub struct WriteResult {
|
||||
pub document: MemoryDocument,
|
||||
pub redirected: bool,
|
||||
pub actual_layer: String,
|
||||
}
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use chrono::{NaiveDate, Utc};
|
||||
@@ -344,20 +357,29 @@ pub struct Workspace {
|
||||
bootstrap_completed: std::sync::atomic::AtomicBool,
|
||||
/// Default search configuration applied to all queries.
|
||||
search_defaults: SearchConfig,
|
||||
/// Memory layers this workspace has access to.
|
||||
memory_layers: Vec<crate::workspace::layer::MemoryLayer>,
|
||||
/// Optional privacy classifier for shared layer writes.
|
||||
/// When None, writes go exactly where requested — no silent redirect.
|
||||
privacy_classifier: Option<Arc<dyn crate::workspace::privacy::PrivacyClassifier>>,
|
||||
}
|
||||
|
||||
impl Workspace {
|
||||
/// Create a new workspace backed by a PostgreSQL connection pool.
|
||||
#[cfg(feature = "postgres")]
|
||||
pub fn new(user_id: impl Into<String>, pool: Pool) -> Self {
|
||||
let user_id_str = user_id.into();
|
||||
let memory_layers = crate::workspace::layer::MemoryLayer::default_for_user(&user_id_str);
|
||||
Self {
|
||||
user_id: user_id.into(),
|
||||
user_id: user_id_str,
|
||||
agent_id: None,
|
||||
storage: WorkspaceStorage::Repo(Repository::new(pool)),
|
||||
embeddings: None,
|
||||
bootstrap_pending: std::sync::atomic::AtomicBool::new(false),
|
||||
bootstrap_completed: std::sync::atomic::AtomicBool::new(false),
|
||||
search_defaults: SearchConfig::default(),
|
||||
memory_layers,
|
||||
privacy_classifier: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -365,14 +387,18 @@ impl Workspace {
|
||||
///
|
||||
/// Use this for libSQL or any other backend that implements the Database trait.
|
||||
pub fn new_with_db(user_id: impl Into<String>, db: Arc<dyn crate::db::Database>) -> Self {
|
||||
let user_id_str = user_id.into();
|
||||
let memory_layers = crate::workspace::layer::MemoryLayer::default_for_user(&user_id_str);
|
||||
Self {
|
||||
user_id: user_id.into(),
|
||||
user_id: user_id_str,
|
||||
agent_id: None,
|
||||
storage: WorkspaceStorage::Db(db),
|
||||
embeddings: None,
|
||||
bootstrap_pending: std::sync::atomic::AtomicBool::new(false),
|
||||
bootstrap_completed: std::sync::atomic::AtomicBool::new(false),
|
||||
search_defaults: SearchConfig::default(),
|
||||
memory_layers,
|
||||
privacy_classifier: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -444,6 +470,32 @@ impl Workspace {
|
||||
self
|
||||
}
|
||||
|
||||
/// Configure memory layers for this workspace.
|
||||
///
|
||||
/// Also updates read_user_ids to include all layer scopes.
|
||||
pub fn with_memory_layers(mut self, layers: Vec<crate::workspace::layer::MemoryLayer>) -> Self {
|
||||
self.memory_layers = layers;
|
||||
self
|
||||
}
|
||||
|
||||
/// Set a privacy classifier for shared layer writes.
|
||||
///
|
||||
/// When set, writes to shared layers are checked against the classifier
|
||||
/// and redirected to the private layer if sensitive content is detected.
|
||||
/// When unset (the default), writes go exactly where requested.
|
||||
pub fn with_privacy_classifier(
|
||||
mut self,
|
||||
classifier: Arc<dyn crate::workspace::privacy::PrivacyClassifier>,
|
||||
) -> Self {
|
||||
self.privacy_classifier = Some(classifier);
|
||||
self
|
||||
}
|
||||
|
||||
/// Get the configured memory layers.
|
||||
pub fn memory_layers(&self) -> &[crate::workspace::layer::MemoryLayer] {
|
||||
&self.memory_layers
|
||||
}
|
||||
|
||||
/// Get the user ID.
|
||||
pub fn user_id(&self) -> &str {
|
||||
&self.user_id
|
||||
@@ -501,7 +553,9 @@ impl Workspace {
|
||||
/// Append content to a file.
|
||||
///
|
||||
/// Creates the file if it doesn't exist.
|
||||
/// Adds a newline separator between existing and new content.
|
||||
/// Uses a single `\n` separator (suitable for log-style entries).
|
||||
/// For semantic separation (e.g., memory entries), use `append_memory()`
|
||||
/// which uses `\n\n`.
|
||||
pub async fn append(&self, path: &str, content: &str) -> Result<(), WorkspaceError> {
|
||||
let path = normalize_path(path);
|
||||
let doc = self
|
||||
@@ -526,6 +580,127 @@ impl Workspace {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Resolve the target scope for a layer write, optionally applying privacy guards.
|
||||
///
|
||||
/// Validates that the layer exists and is writable. When a privacy classifier
|
||||
/// is configured on the workspace AND `force` is false, checks shared-layer
|
||||
/// writes for sensitive content and redirects to the private layer.
|
||||
///
|
||||
/// By default no classifier is set — writes go exactly where requested.
|
||||
/// This is intentional: the LLM chooses the correct layer via system prompt
|
||||
/// guidance, and a regex classifier can't improve on that decision without
|
||||
/// unacceptable false positive rates in household contexts (e.g., "doctor",
|
||||
/// "therapy", phone numbers). Operators who want a safety net can configure
|
||||
/// one via `with_privacy_classifier()`.
|
||||
///
|
||||
/// # Multi-tenant safety (Issue #59)
|
||||
///
|
||||
/// Layer scopes are currently used directly as `user_id` for DB operations.
|
||||
/// In a multi-tenant deployment, an operator could configure a scope that
|
||||
/// collides with another user's ID, granting write access to their data.
|
||||
/// Future work should namespace or validate scopes to prevent this.
|
||||
///
|
||||
/// Returns `(scope, actual_layer_name, redirected)`.
|
||||
fn resolve_layer_target(
|
||||
&self,
|
||||
layer_name: &str,
|
||||
content: &str,
|
||||
force: bool,
|
||||
) -> Result<(String, String, bool), WorkspaceError> {
|
||||
use crate::workspace::layer::{LayerSensitivity, MemoryLayer};
|
||||
|
||||
let layer = MemoryLayer::find(&self.memory_layers, layer_name).ok_or_else(|| {
|
||||
WorkspaceError::LayerNotFound {
|
||||
name: layer_name.to_string(),
|
||||
}
|
||||
})?;
|
||||
|
||||
if !layer.writable {
|
||||
return Err(WorkspaceError::LayerReadOnly {
|
||||
name: layer_name.to_string(),
|
||||
});
|
||||
}
|
||||
|
||||
if !force
|
||||
&& layer.sensitivity == LayerSensitivity::Shared
|
||||
&& let Some(ref classifier) = self.privacy_classifier
|
||||
&& classifier.classify(content).is_sensitive
|
||||
{
|
||||
tracing::warn!(
|
||||
layer = layer_name,
|
||||
"Redirected sensitive content to private layer"
|
||||
);
|
||||
let private = MemoryLayer::private_layer(&self.memory_layers)
|
||||
.ok_or(WorkspaceError::PrivacyRedirectFailed)?;
|
||||
if !private.writable {
|
||||
return Err(WorkspaceError::PrivacyRedirectFailed);
|
||||
}
|
||||
return Ok((private.scope.clone(), private.name.clone(), true));
|
||||
}
|
||||
|
||||
Ok((layer.scope.clone(), layer_name.to_string(), false))
|
||||
}
|
||||
|
||||
/// Write to a specific memory layer.
|
||||
///
|
||||
/// Checks that the layer exists and is writable. Uses the layer's scope
|
||||
/// as the user_id for the database write. For shared layers, sensitive
|
||||
/// content is automatically redirected to the private layer unless
|
||||
/// `force` is set.
|
||||
pub async fn write_to_layer(
|
||||
&self,
|
||||
layer_name: &str,
|
||||
path: &str,
|
||||
content: &str,
|
||||
force: bool,
|
||||
) -> Result<WriteResult, WorkspaceError> {
|
||||
let (scope, actual_layer, redirected) =
|
||||
self.resolve_layer_target(layer_name, content, force)?;
|
||||
let path = normalize_path(path);
|
||||
let doc = self
|
||||
.storage
|
||||
.get_or_create_document_by_path(&scope, self.agent_id, &path)
|
||||
.await?;
|
||||
self.storage.update_document(doc.id, content).await?;
|
||||
self.reindex_document(doc.id).await?;
|
||||
let document = self.storage.get_document_by_id(doc.id).await?;
|
||||
Ok(WriteResult {
|
||||
document,
|
||||
redirected,
|
||||
actual_layer,
|
||||
})
|
||||
}
|
||||
|
||||
/// Write to a layer, with append semantics.
|
||||
pub async fn append_to_layer(
|
||||
&self,
|
||||
layer_name: &str,
|
||||
path: &str,
|
||||
content: &str,
|
||||
force: bool,
|
||||
) -> Result<WriteResult, WorkspaceError> {
|
||||
let (scope, actual_layer, redirected) =
|
||||
self.resolve_layer_target(layer_name, content, force)?;
|
||||
let path = normalize_path(path);
|
||||
let doc = self
|
||||
.storage
|
||||
.get_or_create_document_by_path(&scope, self.agent_id, &path)
|
||||
.await?;
|
||||
let new_content = if doc.content.is_empty() {
|
||||
content.to_string()
|
||||
} else {
|
||||
format!("{}\n\n{}", doc.content, content)
|
||||
};
|
||||
self.storage.update_document(doc.id, &new_content).await?;
|
||||
self.reindex_document(doc.id).await?;
|
||||
let document = self.storage.get_document_by_id(doc.id).await?;
|
||||
Ok(WriteResult {
|
||||
document,
|
||||
redirected,
|
||||
actual_layer,
|
||||
})
|
||||
}
|
||||
|
||||
/// Check if a file exists.
|
||||
pub async fn exists(&self, path: &str) -> Result<bool, WorkspaceError> {
|
||||
let path = normalize_path(path);
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
use regex::Regex;
|
||||
|
||||
/// Result of privacy classification, including confidence level.
|
||||
///
|
||||
/// Confidence enables downstream callers to apply thresholds (e.g., only
|
||||
/// redirect above 0.8) and supports future upgrade to LLM-based classifiers
|
||||
/// that produce probabilistic scores.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SensitivityResult {
|
||||
pub is_sensitive: bool,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
/// Classifies content as potentially sensitive for privacy purposes.
|
||||
///
|
||||
/// Used to guard writes to shared memory layers -- if content is flagged
|
||||
/// as sensitive, it can be redirected to the private layer instead.
|
||||
pub trait PrivacyClassifier: Send + Sync {
|
||||
/// Classify content and return sensitivity with confidence score.
|
||||
fn classify(&self, content: &str) -> SensitivityResult;
|
||||
}
|
||||
|
||||
/// Pattern-based privacy classifier using regex matching.
|
||||
///
|
||||
/// Default patterns target hard PII (SSN, credit card numbers) where silent
|
||||
/// redirect is clearly correct. Ambiguous terms (health vocabulary, contact
|
||||
/// info) are intentionally excluded — they cause false positives in household
|
||||
/// contexts and silently redirect content the user intended to share.
|
||||
///
|
||||
/// Operators who need broader coverage should use `ConfigurablePrivacyClassifier`
|
||||
/// with domain-specific patterns.
|
||||
pub struct PatternPrivacyClassifier {
|
||||
patterns: Vec<Regex>,
|
||||
}
|
||||
|
||||
impl PatternPrivacyClassifier {
|
||||
pub fn new() -> Result<Self, regex::Error> {
|
||||
let pattern_strs = [
|
||||
// SSN — always PII
|
||||
r"\b\d{3}-\d{2}-\d{4}\b",
|
||||
// Credit card (basic) — always PII
|
||||
r"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b",
|
||||
// Credentials and auth tokens — high-confidence PII
|
||||
r"(?i)\b(password|passwd|api[_-]?key|auth[_-]?token|secret[_-]?key)\b",
|
||||
];
|
||||
let patterns = pattern_strs
|
||||
.iter()
|
||||
.map(|p| Regex::new(p))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(Self { patterns })
|
||||
}
|
||||
}
|
||||
|
||||
impl PrivacyClassifier for PatternPrivacyClassifier {
|
||||
fn classify(&self, content: &str) -> SensitivityResult {
|
||||
let is_sensitive = self.patterns.iter().any(|p| p.is_match(content));
|
||||
SensitivityResult {
|
||||
is_sensitive,
|
||||
// Regex is binary — matched or not. Always full confidence.
|
||||
confidence: if is_sensitive { 1.0 } else { 0.0 },
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// User-configurable privacy classifier.
|
||||
///
|
||||
/// Accepts custom regex patterns at construction time, allowing operators
|
||||
/// to tune sensitivity for their use case (e.g., drop health terms that
|
||||
/// cause false positives, add domain-specific patterns).
|
||||
///
|
||||
/// ```
|
||||
/// use ironclaw::workspace::privacy::ConfigurablePrivacyClassifier;
|
||||
/// use ironclaw::workspace::privacy::PrivacyClassifier;
|
||||
///
|
||||
/// let classifier = ConfigurablePrivacyClassifier::new(vec![
|
||||
/// r"\b\d{3}-\d{2}-\d{4}\b".into(), // SSN only
|
||||
/// ]).unwrap();
|
||||
/// assert!(classifier.classify("SSN: 123-45-6789").is_sensitive);
|
||||
/// assert!(!classifier.classify("saw the doctor today").is_sensitive);
|
||||
/// ```
|
||||
pub struct ConfigurablePrivacyClassifier {
|
||||
patterns: Vec<Regex>,
|
||||
}
|
||||
|
||||
impl ConfigurablePrivacyClassifier {
|
||||
/// Create a classifier from user-supplied regex strings.
|
||||
///
|
||||
/// Returns an error if any pattern fails to compile.
|
||||
pub fn new(pattern_strs: Vec<String>) -> Result<Self, regex::Error> {
|
||||
let patterns = pattern_strs
|
||||
.iter()
|
||||
.map(|p| Regex::new(p))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(Self { patterns })
|
||||
}
|
||||
}
|
||||
|
||||
impl PrivacyClassifier for ConfigurablePrivacyClassifier {
|
||||
fn classify(&self, content: &str) -> SensitivityResult {
|
||||
let is_sensitive = self.patterns.iter().any(|p| p.is_match(content));
|
||||
SensitivityResult {
|
||||
is_sensitive,
|
||||
confidence: if is_sensitive { 1.0 } else { 0.0 },
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn classifier() -> PatternPrivacyClassifier {
|
||||
PatternPrivacyClassifier::new().unwrap()
|
||||
}
|
||||
|
||||
// Hard PII — must always trigger
|
||||
#[test]
|
||||
fn detects_ssn() {
|
||||
let result = classifier().classify("My SSN is 123-45-6789");
|
||||
assert!(result.is_sensitive);
|
||||
assert_eq!(result.confidence, 1.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_credit_card() {
|
||||
let result = classifier().classify("Card: 4111 1111 1111 1111");
|
||||
assert!(result.is_sensitive);
|
||||
assert_eq!(result.confidence, 1.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_password() {
|
||||
assert!(classifier().classify("my password is hunter2").is_sensitive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_api_key() {
|
||||
assert!(
|
||||
classifier()
|
||||
.classify("set the api_key to sk-1234")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
// Household content — must NOT trigger (previous false positives)
|
||||
#[test]
|
||||
fn allows_normal_household_content() {
|
||||
let result = classifier().classify("We need to buy groceries for dinner Saturday");
|
||||
assert!(!result.is_sensitive);
|
||||
assert_eq!(result.confidence, 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_doctor_mention() {
|
||||
assert!(
|
||||
!classifier()
|
||||
.classify("the doctor's office called about Saturday")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_email_address() {
|
||||
assert!(
|
||||
!classifier()
|
||||
.classify("email [email protected] about the leak")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_phone_number() {
|
||||
assert!(
|
||||
!classifier()
|
||||
.classify("call the restaurant at 555-123-4567")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn allows_medical_terms_in_context() {
|
||||
assert!(
|
||||
!classifier()
|
||||
.classify("Started new medication for anxiety")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn configurable_with_custom_patterns() {
|
||||
let c = ConfigurablePrivacyClassifier::new(vec![
|
||||
r"\b\d{3}-\d{2}-\d{4}\b".into(), // SSN only
|
||||
])
|
||||
.unwrap();
|
||||
assert!(c.classify("SSN: 123-45-6789").is_sensitive);
|
||||
// Health terms no longer trigger with SSN-only config
|
||||
assert!(!c.classify("saw the doctor today").is_sensitive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn configurable_rejects_bad_regex() {
|
||||
let result = ConfigurablePrivacyClassifier::new(vec!["[invalid".into()]);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn configurable_empty_patterns_allows_everything() {
|
||||
let c = ConfigurablePrivacyClassifier::new(vec![]).unwrap();
|
||||
assert!(!c.classify("My SSN is 123-45-6789").is_sensitive);
|
||||
}
|
||||
|
||||
// Format variants
|
||||
#[test]
|
||||
fn detects_credit_card_no_separators() {
|
||||
assert!(
|
||||
classifier()
|
||||
.classify("card 4111111111111111 on file")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_credit_card_with_dashes() {
|
||||
assert!(
|
||||
classifier()
|
||||
.classify("Card: 4111-1111-1111-1111")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_ssn_bare() {
|
||||
assert!(classifier().classify("123-45-6789").is_sensitive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_auth_token_keyword() {
|
||||
assert!(
|
||||
classifier()
|
||||
.classify("set auth_token to abc123")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_secret_key_keyword() {
|
||||
assert!(
|
||||
classifier()
|
||||
.classify("the secret_key is sk-prod-xyz")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn detects_pii_in_longer_document() {
|
||||
let content = "Meeting notes from Thursday.\n\
|
||||
Discussed budget and timeline.\n\
|
||||
SSN is 999-88-7777 for the insurance form.\n\
|
||||
Action items: follow up with vendor.";
|
||||
assert!(classifier().classify(content).is_sensitive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_string_is_not_sensitive() {
|
||||
assert!(!classifier().classify("").is_sensitive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ssn_not_sensitive() {
|
||||
assert!(
|
||||
!classifier()
|
||||
.classify("code 123-45 in the system")
|
||||
.is_sensitive
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user