mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 15:40:18 +00:00
Improve test infrastructure: StubChannel, gateway helpers, security tests, search edge cases (#623)
* feat(testing): add StubChannel test double for Channel trait Adds StubChannel to src/testing.rs alongside StubLlm. Supports message injection via mpsc sender, response/status capture, and configurable health check toggling. Includes handle methods for use after ownership transfer to ChannelManager. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(testing): wire StubChannel into TestHarnessBuilder Add with_stub_channel() builder method that creates a StubChannel pre-registered in a ChannelManager. Tests can inject messages via the sender and verify routing through the manager. The channel field on TestHarness is Optional, defaulting to None for backward compat. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: gate external-service tests behind integration feature flag Replace silent try_connect() skip pattern with explicit feature gating. cargo test now runs only self-contained tests. cargo test --features integration runs tests requiring PostgreSQL. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(channels): add ChannelManager unit tests using StubChannel Cover add/start_all stream merging, respond routing, unknown channel errors, health_check_all with mixed health, empty-channels error path, and injection channel merging -- all via StubChannel test double. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: document test tier separation (unit/integration/live) Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add architecture boundary check script Grep-based checks for three architecture boundaries: - Direct database driver usage (tokio_postgres/libsql) outside src/db/ - .unwrap()/.expect() in production code (warning only) - Direct std::env::var reads outside config layer (warning only) The DB driver check is a hard violation; the other two are warnings for gradual cleanup. Run with: bash scripts/check-boundaries.sh Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(search): add RRF edge case tests for empty inputs, limits, and config modes Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(security): add regression tests for skill installer ZIP and SSRF protections Add 11 regression tests covering the security controls in skill_tools: ZIP extraction safety: - Valid SKILL.md extraction works correctly - Non-SKILL.md entries are ignored (returns error) - Path traversal entries (../../SKILL.md) do not match - Nested path entries (subdir/SKILL.md) do not match - Oversized entries (>1MB uncompressed) are rejected SSRF prevention: - Loopback addresses (127.0.0.1) are blocked - Private ranges (10.x, 172.16.x, 192.168.x) are blocked - Link-local addresses (169.254.x) are blocked - Public IPs (8.8.8.8, 1.1.1.1) are allowed - IPv4-mapped IPv6 unwrapping logic works correctly - Metadata endpoints and .internal/.local hostnames are blocked - Normal hostnames (github.com, clawhub.dev) are allowed Also documents a known gap: url::Url::host_str() returns bracketed IPv6 addresses that std::net::IpAddr cannot parse, so IPv4-mapped IPv6 URLs currently bypass IP-based checks in validate_fetch_url. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(testing): extract TestGatewayBuilder to eliminate gateway test duplication Both ws_gateway_integration.rs and openai_compat_integration.rs manually constructed GatewayState with 19+ fields. Extracted to a shared builder in src/channels/web/test_helpers.rs that provides sensible defaults and lets tests override only what they need. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: add implementation plans for testing batches 1 and 2 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): close IPv6 SSRF bypass in validate_fetch_url validate_fetch_url used host_str() which returns bracketed IPv6 (e.g. "[::ffff:7f00:1]") that IpAddr::parse() cannot handle, silently skipping IP-based SSRF checks for all IPv6 URLs. Switch to url::Host enum matching to extract proper IpAddr values without string parsing. IPv4-mapped IPv6 addresses like ::ffff:127.0.0.1 are now correctly unwrapped and blocked. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(skills): add activation criteria limits enforcement tests Adds test_activation_criteria_enforce_limits to verify that enforce_limits() correctly trims excess patterns (>5), keywords (>20), and tags (>10), and filters out short keywords/tags (<3 chars). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(wasm): add security regression tests for WASM tool loader Add 6 tests covering: tool name path separator rejection, empty name rejection, nonexistent file handling, invalid WASM bytes rejection, dotfile discovery behavior, and subdirectory non-recursion. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: address PR review feedback - Remove plan files from repo (ilblackdragon review) - Replace CLAUDE.md test tier rules with pointer to check-boundaries.sh - Add Check 4 to check-boundaries.sh: enforces integration tests are gated behind the 'integration' feature flag Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add try_connect silent-skip pattern check to check-boundaries.sh Check 5 catches try_connect() and similar silent-skip patterns in integration tests. Tests should use feature gates to fail loudly when prerequisites are missing, not silently return. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): harden skill fetch SSRF checks * fix(scripts): use bash arrays in check-boundaries.sh tier violation check Refactor Check 4 in check-boundaries.sh to use bash arrays and printf instead of string concatenation with echo -e. This is more robust with special characters in filenames and avoids portability concerns with echo -e. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.6
parent
cf96a3253c
commit
45ec691f4c
+214
-1
@@ -2,6 +2,7 @@
|
||||
//!
|
||||
//! Provides:
|
||||
//! - [`StubLlm`]: A configurable LLM provider that returns a fixed response
|
||||
//! - [`StubChannel`]: A configurable channel stub with message injection and response capture
|
||||
//! - [`TestHarnessBuilder`]: Builder for wiring `AgentDeps` with defaults
|
||||
//! - [`TestHarness`]: The assembled components ready for use in tests
|
||||
//!
|
||||
@@ -18,14 +19,19 @@
|
||||
//! ```
|
||||
|
||||
use std::sync::Arc;
|
||||
use std::sync::Mutex;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
||||
|
||||
use async_trait::async_trait;
|
||||
use rust_decimal::Decimal;
|
||||
use tokio::sync::mpsc;
|
||||
|
||||
use crate::agent::AgentDeps;
|
||||
use crate::channels::{
|
||||
Channel, ChannelManager, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate,
|
||||
};
|
||||
use crate::db::Database;
|
||||
use crate::error::LlmError;
|
||||
use crate::error::{ChannelError, LlmError};
|
||||
use crate::llm::{
|
||||
CompletionRequest, CompletionResponse, FinishReason, LlmProvider, ToolCompletionRequest,
|
||||
ToolCompletionResponse,
|
||||
@@ -189,12 +195,138 @@ impl LlmProvider for StubLlm {
|
||||
}
|
||||
}
|
||||
|
||||
/// A configurable channel stub for tests.
|
||||
///
|
||||
/// Supports:
|
||||
/// - Message injection via the returned `mpsc::Sender`
|
||||
/// - Response capture for assertion
|
||||
/// - Status update capture
|
||||
/// - Configurable health check failure
|
||||
///
|
||||
/// # Usage
|
||||
///
|
||||
/// ```rust,no_run
|
||||
/// let (channel, sender) = StubChannel::new("test");
|
||||
/// sender.send(IncomingMessage::new("test", "user1", "hello")).await.unwrap();
|
||||
/// // ... run agent logic that calls channel.respond() ...
|
||||
/// let responses = channel.captured_responses();
|
||||
/// ```
|
||||
pub struct StubChannel {
|
||||
name: String,
|
||||
rx: tokio::sync::Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
|
||||
responses: Arc<Mutex<Vec<(IncomingMessage, OutgoingResponse)>>>,
|
||||
statuses: Arc<Mutex<Vec<StatusUpdate>>>,
|
||||
healthy: AtomicBool,
|
||||
}
|
||||
|
||||
impl StubChannel {
|
||||
/// Create a new stub channel and its message sender.
|
||||
///
|
||||
/// The sender is used by tests to inject messages into the channel's stream.
|
||||
/// The channel captures all responses and status updates for later assertion.
|
||||
pub fn new(name: impl Into<String>) -> (Self, mpsc::Sender<IncomingMessage>) {
|
||||
let (tx, rx) = mpsc::channel(64);
|
||||
let channel = Self {
|
||||
name: name.into(),
|
||||
rx: tokio::sync::Mutex::new(Some(rx)),
|
||||
responses: Arc::new(Mutex::new(Vec::new())),
|
||||
statuses: Arc::new(Mutex::new(Vec::new())),
|
||||
healthy: AtomicBool::new(true),
|
||||
};
|
||||
(channel, tx)
|
||||
}
|
||||
|
||||
/// Get all captured (message, response) pairs.
|
||||
pub fn captured_responses(&self) -> Vec<(IncomingMessage, OutgoingResponse)> {
|
||||
self.responses.lock().expect("poisoned").clone()
|
||||
}
|
||||
|
||||
/// Get a shared handle to the response capture list.
|
||||
///
|
||||
/// Call this *before* moving the channel into a `ChannelManager`,
|
||||
/// since `add()` takes ownership.
|
||||
pub fn captured_responses_handle(
|
||||
&self,
|
||||
) -> Arc<Mutex<Vec<(IncomingMessage, OutgoingResponse)>>> {
|
||||
Arc::clone(&self.responses)
|
||||
}
|
||||
|
||||
/// Get all captured status updates.
|
||||
pub fn captured_statuses(&self) -> Vec<StatusUpdate> {
|
||||
self.statuses.lock().expect("poisoned").clone()
|
||||
}
|
||||
|
||||
/// Get a shared handle to the status capture list.
|
||||
pub fn captured_statuses_handle(&self) -> Arc<Mutex<Vec<StatusUpdate>>> {
|
||||
Arc::clone(&self.statuses)
|
||||
}
|
||||
|
||||
/// Set whether `health_check()` succeeds or fails.
|
||||
pub fn set_healthy(&self, healthy: bool) {
|
||||
self.healthy.store(healthy, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Channel for StubChannel {
|
||||
fn name(&self) -> &str {
|
||||
&self.name
|
||||
}
|
||||
|
||||
async fn start(&self) -> Result<MessageStream, ChannelError> {
|
||||
let rx = self
|
||||
.rx
|
||||
.lock()
|
||||
.await
|
||||
.take()
|
||||
.ok_or_else(|| ChannelError::StartupFailed {
|
||||
name: self.name.clone(),
|
||||
reason: "start() already called".to_string(),
|
||||
})?;
|
||||
let stream = tokio_stream::wrappers::ReceiverStream::new(rx);
|
||||
Ok(Box::pin(stream))
|
||||
}
|
||||
|
||||
async fn respond(
|
||||
&self,
|
||||
msg: &IncomingMessage,
|
||||
response: OutgoingResponse,
|
||||
) -> Result<(), ChannelError> {
|
||||
self.responses
|
||||
.lock()
|
||||
.expect("poisoned")
|
||||
.push((msg.clone(), response));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn send_status(
|
||||
&self,
|
||||
status: StatusUpdate,
|
||||
_metadata: &serde_json::Value,
|
||||
) -> Result<(), ChannelError> {
|
||||
self.statuses.lock().expect("poisoned").push(status);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn health_check(&self) -> Result<(), ChannelError> {
|
||||
if self.healthy.load(Ordering::Relaxed) {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(ChannelError::HealthCheckFailed {
|
||||
name: self.name.clone(),
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Assembled test components.
|
||||
pub struct TestHarness {
|
||||
/// The agent dependencies, ready for use.
|
||||
pub deps: AgentDeps,
|
||||
/// Direct reference to the database (as `Arc<dyn Database>`).
|
||||
pub db: Arc<dyn Database>,
|
||||
/// Stub channel sender + manager, present if `with_stub_channel()` was called.
|
||||
pub channel: Option<(mpsc::Sender<IncomingMessage>, ChannelManager)>,
|
||||
/// Temp directory guard — keeps the test database alive. Dropped
|
||||
/// automatically when the harness goes out of scope.
|
||||
#[cfg(feature = "libsql")]
|
||||
@@ -214,6 +346,7 @@ pub struct TestHarnessBuilder {
|
||||
db: Option<Arc<dyn Database>>,
|
||||
llm: Option<Arc<dyn LlmProvider>>,
|
||||
tools: Option<Arc<ToolRegistry>>,
|
||||
stub_channel: bool,
|
||||
}
|
||||
|
||||
impl TestHarnessBuilder {
|
||||
@@ -223,6 +356,7 @@ impl TestHarnessBuilder {
|
||||
db: None,
|
||||
llm: None,
|
||||
tools: None,
|
||||
stub_channel: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -244,6 +378,15 @@ impl TestHarnessBuilder {
|
||||
self
|
||||
}
|
||||
|
||||
/// Include a `StubChannel` wired into a `ChannelManager`.
|
||||
///
|
||||
/// The harness will expose the sender (for injecting messages) and
|
||||
/// the manager (for routing responses) via [`TestHarness::channel`].
|
||||
pub fn with_stub_channel(mut self) -> Self {
|
||||
self.stub_channel = true;
|
||||
self
|
||||
}
|
||||
|
||||
/// Build the harness with defaults applied.
|
||||
#[cfg(feature = "libsql")]
|
||||
pub async fn build(self) -> TestHarness {
|
||||
@@ -280,6 +423,15 @@ impl TestHarnessBuilder {
|
||||
max_actions_per_hour: None,
|
||||
}));
|
||||
|
||||
let channel = if self.stub_channel {
|
||||
let (stub, sender) = StubChannel::new("stub");
|
||||
let manager = ChannelManager::new();
|
||||
manager.add(Box::new(stub)).await;
|
||||
Some((sender, manager))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let deps = AgentDeps {
|
||||
store: Some(Arc::clone(&db)),
|
||||
llm,
|
||||
@@ -300,6 +452,7 @@ impl TestHarnessBuilder {
|
||||
TestHarness {
|
||||
deps,
|
||||
db,
|
||||
channel,
|
||||
_temp_dir: temp_dir,
|
||||
}
|
||||
}
|
||||
@@ -653,6 +806,48 @@ mod tests {
|
||||
assert_eq!(response.finish_reason, FinishReason::Stop);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_stub_channel_inject_and_capture() {
|
||||
use futures::StreamExt;
|
||||
|
||||
let (channel, sender) = StubChannel::new("test-channel");
|
||||
|
||||
// Start the channel to get the message stream
|
||||
let mut stream = channel.start().await.expect("start failed");
|
||||
|
||||
// Inject a message
|
||||
sender
|
||||
.send(IncomingMessage::new("test-channel", "user1", "hello"))
|
||||
.await
|
||||
.expect("send failed");
|
||||
|
||||
// Read it from the stream
|
||||
let msg = stream.next().await.expect("stream ended");
|
||||
assert_eq!(msg.content, "hello");
|
||||
assert_eq!(msg.user_id, "user1");
|
||||
assert_eq!(msg.channel, "test-channel");
|
||||
|
||||
// Send a response and verify it was captured
|
||||
let response = OutgoingResponse::text("world");
|
||||
channel
|
||||
.respond(&msg, response)
|
||||
.await
|
||||
.expect("respond failed");
|
||||
|
||||
let captured = channel.captured_responses();
|
||||
assert_eq!(captured.len(), 1);
|
||||
assert_eq!(captured[0].1.content, "world");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_stub_channel_health_check() {
|
||||
let (channel, _sender) = StubChannel::new("healthy");
|
||||
channel.health_check().await.expect("health check failed");
|
||||
|
||||
channel.set_healthy(false);
|
||||
assert!(channel.health_check().await.is_err());
|
||||
}
|
||||
|
||||
// === Database CRUD coverage for untested trait methods ===
|
||||
|
||||
#[cfg(feature = "libsql")]
|
||||
@@ -705,6 +900,24 @@ mod tests {
|
||||
assert!(!deleted);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_harness_with_channel() {
|
||||
let harness = TestHarnessBuilder::new().with_stub_channel().build().await;
|
||||
|
||||
let (sender, channel_manager) =
|
||||
harness.channel.as_ref().expect("channel should be present");
|
||||
|
||||
// Inject a message via sender
|
||||
sender
|
||||
.send(IncomingMessage::new("stub", "user1", "test message"))
|
||||
.await
|
||||
.expect("send failed");
|
||||
|
||||
// Verify channel is registered in the manager
|
||||
let names = channel_manager.channel_names().await;
|
||||
assert!(names.contains(&"stub".to_string()));
|
||||
}
|
||||
|
||||
#[cfg(feature = "libsql")]
|
||||
#[tokio::test]
|
||||
async fn test_settings_bulk_operations() {
|
||||
|
||||
Reference in New Issue
Block a user