mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
* fix: comprehensive security hardening across all layers Critical: - Replace --dangerously-skip-permissions with explicit tool allowlist via settings.json (Claude Code bridge) - Constant-time token comparison (subtle crate) in web auth and orchestrator auth to prevent timing attacks High: - Revoke tokens and clean up handles on container creation failure - Drop SETUID/SETGID capabilities from containers (keep only CHOWN) - Disable redirect following in HTTP tool and WASM wrapper (SSRF) - Reject URL userinfo (@) in WASM allowlist parser (host confusion) - Fix binary body bypassing leak detection (from_utf8 -> from_utf8_lossy) - Protect identity files from LLM overwrites (prompt injection defense) - Prevent tool shadowing: built-in tools cannot be replaced dynamically - User-scoped job APIs: list/detail/cancel/restart/prompt/events/files - CORS restricted to localhost origins, WebSocket origin validation - Sandbox shell fail-closed: no silent fallback to unsandboxed execution - Scrub secrets from log broadcaster before SSE broadcast - XSS sanitization on rendered markdown in web UI - WASM epoch ticker thread so timeout deadlines actually fire Medium: - Cap state transition history at 200 entries - SSE/WebSocket connection limit (100 max) - Request body size limit (1MB) - Response body size limit enforcement in WASM HTTP - UTF-8 safe string truncation (routine engine, shell tool) - Fix PolicyAction::Sanitize to actually run the sanitizer - TOCTOU fix in scheduler and context manager (hold write lock) - Project file serving moved behind auth - Path traversal guard on project_id - Session file permissions set to 0600 on unix - AtomicUsize for routine running_count (panic-safe) - Completion detection hardened against false positives and tool injection - Tool output no longer drives job completion (only LLM response) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address security review findings across all layers - Fix path traversal sandbox bypass via lexical normalization (file.rs) - Fix SSRF via DNS rebinding with pre-request hostname resolution (http.rs) - Add token budget enforcement on LLM calls (reasoning.rs, state.rs) - Fix cross-user chat history leak with ownership verification (store.rs, server.rs) - Add sliding-window rate limiter on gateway chat endpoint (server.rs) - Harden extension install: HTTPS-only, 50MB cap, WASM magic validation (manager.rs) - Add destructive command blocklist that overrides shell auto-approval (shell.rs) - Add 5MB response body size cap to HTTP tool (http.rs) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: deduplicate shared helpers and remove dead code Extract floor_char_boundary and llm_signals_completion into src/util.rs, unifying diverging phrase lists from agent/worker.rs and worker/runtime.rs. Remove dead RespondResult::usage(), duplicate PROTECTED_IDENTITY_FILES constant, double LeakDetector scanning in WebLogLayer, and invalid 0.0.0.0 origin from WebSocket allow list. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review findings and CI test failures - Fix record_failed_approve: .truncate(true) wiped the attempts file before reading, so failed pairing attempts never accumulated and rate limiting never triggered. - Guard wizard WASM test: skip gracefully when channel build artifacts are absent (CI doesn't compile wasm32-wasip2 targets). - Fix DNS rebinding check: use port 0 instead of hardcoded 443, since the port is irrelevant for hostname resolution. - Remove hardcoded CORS port 3001: the dynamic addr.port() entries already cover the actual server port. - Require WebSocket Origin header: reject connections that omit it entirely, since browsers always send Origin for WS upgrades and a missing header indicates a non-browser client bypassing the check. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR review findings - store.rs: reintroduce file locking around read-modify-write in record_failed_approve (concurrent callers could clobber each other). - sse.rs: replace load+check+fetch_add with atomic fetch_update in both subscribe_raw() and subscribe() to prevent overshooting max_connections. - ws.rs: decrement WS tracker before early return when subscribe_raw() returns None (connection limit reached), fixing a counter leak. - server.rs: parse WS Origin host exactly instead of prefix matching, preventing bypass via crafted origins like http://localhost.evil.com. - workspace_integration.rs: skip tests gracefully when Postgres is unreachable instead of panicking (fixes 10 CI failures). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add Origin header to WS integration tests The Origin header requirement added in a3b0190 broke the WS gateway integration tests. Test clients now send Origin: http://127.0.0.1:{port} to match the server's localhost validation. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
359 lines
11 KiB
Rust
359 lines
11 KiB
Rust
//! Tracing layer that broadcasts log events to the web gateway via SSE.
|
|
//!
|
|
//! ```text
|
|
//! tracing::info!("...")
|
|
//! │
|
|
//! ▼
|
|
//! WebLogLayer::on_event()
|
|
//! │
|
|
//! ▼
|
|
//! LogBroadcaster::send()
|
|
//! │
|
|
//! ├──► broadcast::Sender<LogEntry> (live subscribers)
|
|
//! └──► ring buffer (recent history for late joiners)
|
|
//! │
|
|
//! ▼
|
|
//! SSE /api/logs/events
|
|
//! ```
|
|
|
|
use std::collections::VecDeque;
|
|
use std::sync::{Arc, Mutex};
|
|
|
|
use serde::Serialize;
|
|
use tokio::sync::broadcast;
|
|
use tracing::field::{Field, Visit};
|
|
use tracing_subscriber::Layer;
|
|
|
|
use crate::safety::LeakDetector;
|
|
|
|
/// Maximum number of recent log entries kept for late-joining SSE subscribers.
|
|
const HISTORY_CAP: usize = 500;
|
|
|
|
/// A single log entry broadcast to connected clients.
|
|
#[derive(Debug, Clone, Serialize)]
|
|
pub struct LogEntry {
|
|
pub level: String,
|
|
pub target: String,
|
|
pub message: String,
|
|
pub timestamp: String,
|
|
}
|
|
|
|
/// Broadcasts log entries to SSE subscribers.
|
|
///
|
|
/// Created early in main.rs (before tracing init), shared with both
|
|
/// the tracing layer and the gateway's SSE endpoint.
|
|
///
|
|
/// Keeps a ring buffer of recent entries so browsers that connect
|
|
/// after startup still see the boot log.
|
|
pub struct LogBroadcaster {
|
|
tx: broadcast::Sender<LogEntry>,
|
|
recent: Mutex<VecDeque<LogEntry>>,
|
|
/// Scrubs secrets from log messages before broadcasting to SSE clients.
|
|
leak_detector: LeakDetector,
|
|
}
|
|
|
|
impl LogBroadcaster {
|
|
pub fn new() -> Self {
|
|
let (tx, _) = broadcast::channel(512);
|
|
Self {
|
|
tx,
|
|
recent: Mutex::new(VecDeque::with_capacity(HISTORY_CAP)),
|
|
leak_detector: LeakDetector::new(),
|
|
}
|
|
}
|
|
|
|
pub fn send(&self, mut entry: LogEntry) {
|
|
// Scrub secrets from the message before it reaches any subscriber.
|
|
// This is defense-in-depth: even if code elsewhere accidentally logs
|
|
// a secret, it won't be broadcast to SSE clients.
|
|
entry.message = self
|
|
.leak_detector
|
|
.scan_and_clean(&entry.message)
|
|
.unwrap_or_else(|_| "[log message redacted: contained blocked secret]".to_string());
|
|
|
|
// Stash in ring buffer (for late joiners)
|
|
if let Ok(mut buf) = self.recent.lock() {
|
|
if buf.len() >= HISTORY_CAP {
|
|
buf.pop_front();
|
|
}
|
|
buf.push_back(entry.clone());
|
|
}
|
|
// Broadcast to live subscribers (ok to drop if nobody listening)
|
|
let _ = self.tx.send(entry);
|
|
}
|
|
|
|
/// Subscribe to the live event stream.
|
|
pub fn subscribe(&self) -> broadcast::Receiver<LogEntry> {
|
|
self.tx.subscribe()
|
|
}
|
|
|
|
/// Snapshot of recent entries for replaying to a new subscriber.
|
|
pub fn recent_entries(&self) -> Vec<LogEntry> {
|
|
self.recent
|
|
.lock()
|
|
.map(|buf| buf.iter().cloned().collect())
|
|
.unwrap_or_default()
|
|
}
|
|
}
|
|
|
|
impl Default for LogBroadcaster {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
/// Visitor that extracts the `message` field and all extra key-value
|
|
/// fields from a tracing event.
|
|
///
|
|
/// The terminal formatter shows something like:
|
|
/// INFO ironclaw::agent: Request completed url="http://..." status=200
|
|
///
|
|
/// We replicate that by capturing both the message and the extra fields.
|
|
struct MessageVisitor {
|
|
message: String,
|
|
fields: Vec<String>,
|
|
}
|
|
|
|
impl MessageVisitor {
|
|
fn new() -> Self {
|
|
Self {
|
|
message: String::new(),
|
|
fields: Vec::new(),
|
|
}
|
|
}
|
|
|
|
/// Build the final message string: "message key=val key=val ..."
|
|
fn finish(self) -> String {
|
|
if self.fields.is_empty() {
|
|
self.message
|
|
} else {
|
|
format!("{} {}", self.message, self.fields.join(" "))
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Visit for MessageVisitor {
|
|
fn record_debug(&mut self, field: &Field, value: &dyn std::fmt::Debug) {
|
|
if field.name() == "message" {
|
|
self.message = format!("{:?}", value);
|
|
// Strip surrounding quotes from Debug output
|
|
if self.message.starts_with('"') && self.message.ends_with('"') {
|
|
self.message = self.message[1..self.message.len() - 1].to_string();
|
|
}
|
|
} else {
|
|
self.fields.push(format!("{}={:?}", field.name(), value));
|
|
}
|
|
}
|
|
|
|
fn record_str(&mut self, field: &Field, value: &str) {
|
|
if field.name() == "message" {
|
|
self.message = value.to_string();
|
|
} else {
|
|
self.fields.push(format!("{}={}", field.name(), value));
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Tracing layer that forwards events to a [`LogBroadcaster`].
|
|
///
|
|
/// Only forwards DEBUG and above. Attach to the tracing subscriber
|
|
/// alongside the existing fmt layer.
|
|
///
|
|
/// Log messages are scrubbed through `LeakDetector` in `LogBroadcaster::send()`
|
|
/// (the single funnel point for all log output, including late-joiner history).
|
|
pub struct WebLogLayer {
|
|
broadcaster: Arc<LogBroadcaster>,
|
|
}
|
|
|
|
impl WebLogLayer {
|
|
pub fn new(broadcaster: Arc<LogBroadcaster>) -> Self {
|
|
Self { broadcaster }
|
|
}
|
|
}
|
|
|
|
impl<S: tracing::Subscriber> Layer<S> for WebLogLayer {
|
|
fn on_event(
|
|
&self,
|
|
event: &tracing::Event<'_>,
|
|
_ctx: tracing_subscriber::layer::Context<'_, S>,
|
|
) {
|
|
let metadata = event.metadata();
|
|
|
|
// Only forward DEBUG+
|
|
if *metadata.level() > tracing::Level::DEBUG {
|
|
return;
|
|
}
|
|
|
|
let mut visitor = MessageVisitor::new();
|
|
event.record(&mut visitor);
|
|
|
|
let entry = LogEntry {
|
|
level: metadata.level().to_string().to_uppercase(),
|
|
target: metadata.target().to_string(),
|
|
message: visitor.finish(),
|
|
timestamp: chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Millis, true),
|
|
};
|
|
|
|
// LeakDetector scrubbing happens inside broadcaster.send()
|
|
self.broadcaster.send(entry);
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_log_broadcaster_creation() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
// Should not panic with no receivers
|
|
broadcaster.send(LogEntry {
|
|
level: "INFO".to_string(),
|
|
target: "test".to_string(),
|
|
message: "hello".to_string(),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
});
|
|
}
|
|
|
|
#[test]
|
|
fn test_log_broadcaster_subscribe() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
let mut rx = broadcaster.subscribe();
|
|
|
|
broadcaster.send(LogEntry {
|
|
level: "WARN".to_string(),
|
|
target: "ironclaw::test".to_string(),
|
|
message: "test warning".to_string(),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
});
|
|
|
|
let entry = rx.try_recv().expect("should receive entry");
|
|
assert_eq!(entry.level, "WARN");
|
|
assert_eq!(entry.message, "test warning");
|
|
}
|
|
|
|
#[test]
|
|
fn test_log_entry_serialization() {
|
|
let entry = LogEntry {
|
|
level: "ERROR".to_string(),
|
|
target: "ironclaw::agent".to_string(),
|
|
message: "something broke".to_string(),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
};
|
|
let json = serde_json::to_string(&entry).expect("should serialize");
|
|
assert!(json.contains("\"level\":\"ERROR\""));
|
|
assert!(json.contains("something broke"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_recent_entries_buffer() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
|
|
for i in 0..5 {
|
|
broadcaster.send(LogEntry {
|
|
level: "INFO".to_string(),
|
|
target: "test".to_string(),
|
|
message: format!("msg {}", i),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
});
|
|
}
|
|
|
|
let recent = broadcaster.recent_entries();
|
|
assert_eq!(recent.len(), 5);
|
|
assert_eq!(recent[0].message, "msg 0");
|
|
assert_eq!(recent[4].message, "msg 4");
|
|
}
|
|
|
|
#[test]
|
|
fn test_recent_entries_cap() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
|
|
// Overflow the buffer
|
|
for i in 0..(HISTORY_CAP + 50) {
|
|
broadcaster.send(LogEntry {
|
|
level: "INFO".to_string(),
|
|
target: "test".to_string(),
|
|
message: format!("msg {}", i),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
});
|
|
}
|
|
|
|
let recent = broadcaster.recent_entries();
|
|
assert_eq!(recent.len(), HISTORY_CAP);
|
|
// Oldest should be msg 50 (first 50 evicted)
|
|
assert_eq!(recent[0].message, "msg 50");
|
|
}
|
|
|
|
#[test]
|
|
fn test_recent_entries_available_without_subscribers() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
// No subscribe() call, just send
|
|
broadcaster.send(LogEntry {
|
|
level: "INFO".to_string(),
|
|
target: "test".to_string(),
|
|
message: "before anyone listened".to_string(),
|
|
timestamp: "2024-01-01T00:00:00.000Z".to_string(),
|
|
});
|
|
|
|
let recent = broadcaster.recent_entries();
|
|
assert_eq!(recent.len(), 1);
|
|
assert_eq!(recent[0].message, "before anyone listened");
|
|
}
|
|
|
|
#[test]
|
|
fn test_message_visitor_finish_message_only() {
|
|
let v = MessageVisitor {
|
|
message: "hello world".to_string(),
|
|
fields: vec![],
|
|
};
|
|
assert_eq!(v.finish(), "hello world");
|
|
}
|
|
|
|
#[test]
|
|
fn test_message_visitor_finish_with_fields() {
|
|
let v = MessageVisitor {
|
|
message: "Request completed".to_string(),
|
|
fields: vec![
|
|
"url=http://localhost:8080".to_string(),
|
|
"status=200".to_string(),
|
|
],
|
|
};
|
|
let result = v.finish();
|
|
assert_eq!(
|
|
result,
|
|
"Request completed url=http://localhost:8080 status=200"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn test_message_visitor_finish_empty() {
|
|
let v = MessageVisitor::new();
|
|
assert_eq!(v.finish(), "");
|
|
}
|
|
|
|
#[test]
|
|
fn test_broadcaster_has_leak_detector() {
|
|
let broadcaster = LogBroadcaster::new();
|
|
// Verify the leak detector is initialized with default patterns
|
|
assert!(broadcaster.leak_detector.pattern_count() > 0);
|
|
}
|
|
|
|
#[test]
|
|
fn test_leak_detector_scrubs_api_key_in_log() {
|
|
let detector = crate::safety::LeakDetector::new();
|
|
let msg = "Connecting with token sk-proj-test1234567890abcdefghij";
|
|
let result = detector.scan_and_clean(msg);
|
|
// Should be blocked (OpenAI key pattern)
|
|
assert!(result.is_err());
|
|
}
|
|
|
|
#[test]
|
|
fn test_leak_detector_passes_clean_log() {
|
|
let detector = crate::safety::LeakDetector::new();
|
|
let msg = "Request completed status=200 url=https://api.example.com/data";
|
|
let result = detector.scan_and_clean(msg);
|
|
assert!(result.is_ok());
|
|
assert_eq!(result.unwrap(), msg);
|
|
}
|
|
}
|