mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build P0 items from the automated QA plan (#352): - Add validate_tool_schema() that checks OpenAI strict-mode rules (type: object, required keys in properties, nested object/array recursion) with 10 unit tests and 6 integration tests covering all core built-in tools - CI test matrix now runs with --all-features, default features, and --no-default-features --features libsql to catch dead code behind wrong cfg gates - CI clippy now runs the same 3-feature matrix with --all flags - Docker build job added to catch missing files in Dockerfile Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P1 automated QA tests and fix LeakDetector prefix shadowing bug P1 test coverage: config round-trip (settings + bootstrap), shell tool arg handling, safety adversarial tests (sanitizer, leak detector, allowlist), turn persistence (conversations, metadata, pagination, jobs), and a clippy fix for libsql-only builds. Fixed a real bug where AhoCorasick non-overlapping prefix iteration caused shorter prefixes (e.g. "sk-") to shadow longer ones (e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key detection. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P2 automated QA tests: chaos, lifecycle, collision, and recovery Cover all P2 items from the automated QA plan: - Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors) - Failover chaos tests (hanging failover, all-fail, tools path, single provider) - Value estimator boundary tests (negative cost, zero price, zero earnings) - Context length recovery test (ContextLengthExceeded -> compact -> retry) - WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation) - Extension registry collision tests (same-name different-kind coexistence) - Extension filesystem collision tests (separate dirs, detect_kind priority) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P3 concurrent stress tests for ContextManager and SessionManager Tests verify thread safety of double-checked locking, TOCTOU prevention, and RwLock-based concurrent access patterns under load. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add dispatcher loop guard and self-repair stuck job tests Dispatcher: test force_text mechanism prevents infinite tool call loops, verify iteration bound arithmetic guarantees termination for all configs. Self-repair: test stuck job detection, recovery within attempt limits, manual escalation when limit exceeded, graceful degradation without store/builder dependencies. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure design doc Python + Playwright framework with mock LLM server for deterministic browser-level testing of the web gateway. Covers connection/auth, chat round-trip with SSE streaming, and skills lifecycle scenarios. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure implementation plan 10-task plan covering: scaffolding, mock LLM server, helpers, conftest fixtures, connection/chat/skills test scenarios, CI workflow, README, and integration run. Co-Authored-By: Claude Opus 4.6 <[email protected]> * scaffold: E2E test project with pyproject.toml Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E helpers with DOM selectors and port discovery Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: mock OpenAI-compat LLM server for E2E tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E conftest with session fixtures for mock LLM and ironclaw Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 1 -- connection and tab navigation tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 2 -- chat message round-trip tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 3 -- skills search, install, remove tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add weekly E2E test workflow with Playwright Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: E2E test README with setup and usage instructions Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test integration fixes from first run - Use temp file DB instead of :memory: (libSQL :memory: doesn't persist tables across execute_batch) - Fix installed skills selector: #skills-list not #installed-skills - Add pytest-timeout to dependencies - Improve skills install/remove test with wait_for instead of fixed sleeps 8 passed, 1 skipped (skills install depends on ClawHub availability) Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1) Add src/tools/schema_validator.rs with validate_strict_schema() that checks tool parameter schemas against OpenAI function calling strict-mode rules: type object at top level, required keys in properties, enum type consistency, array items definitions, nested object recursion, and additionalProperties. 17 tests validate all 34+ built-in tool schemas across 5 test groups: - 9 simple tools (echo, time, json, http, shell, file read/write/list/patch) - 4 job tools (create, list, status, cancel) - 4 skill tools (list, search, install, remove) - 13 inline schemas for extension, routine, and complex job tools - 4 memory tool schemas Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test reliability for HTML injection and SSE reconnect - HTML injection: test sanitization directly via JS injection instead of depending on full LLM round-trip (avoids intermittent 404 from mock) - SSE reconnect: increase wait times for DB persistence and relax assertion to check total message count after history reload Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt formatting Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add WASM and MCP tool schema validation tests (QA 1.1) Extends the schema validator with representative WASM tool schemas (weather, HTTP client, batch processor, status), MCP tool schemas (default, file read, SQL query, strict mode), and defect detection tests for common external schema issues (missing type, typo in required, array without items, enum type mismatch). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add auth middleware and compaction module tests Auth middleware (8 new tests): valid/invalid bearer tokens, query param fallback, case sensitivity, empty tokens, whitespace handling. Compaction module (16 new tests): truncation strategy, summarize strategy with mock LLM, workspace fallback, format_turns helper, sequential compactions, coherence after compaction, token decrease verification. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add config round-trip integration tests (QA 1.2) Test the full bootstrap .env lifecycle: write via the same format as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy, and assert values match. Covers LLM backend selection, embedding disable flag, onboard completion flag, session token keys, multi-key preservation across upsert, and special characters (spaces, equals, quotes, backslashes, hashes). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4) Value estimator (14 new tests): zero/negative prices, large values, negative cost, exact margin boundaries, custom margin configuration. Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates when all tool calls fail (regression guard for PR #252 infinite loop) and when max iterations are reached. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add failover edge cases and provider chaos tests (QA 2.6/4.1) Failover edge cases (4 new tests): cooldown at zero nanos, half-open failure reopens circuit, all providers fail gracefully (no panic), single failing provider with cooldown. Provider chaos tests (15 new tests): flakey provider with retries, hanging provider with timeout, garbage provider, circuit breaker trip/recover, failover chain cascading, non-transient error stops chain, full stack integration (retry + failover + circuit breaker). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback on QA tests - Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs) - Refactor bootstrap.rs to expose path-parameterized variants so config_round_trip tests call real code instead of reimplementations - Remove deprecated event_loop fixture, use dynamic ports, minimal env, session-scoped browser, and wire HEADED=1 in E2E conftest - Add cross-referencing doc comments between schema validators - Simplify array validation logic in tool.rs - Bump e2e.yml checkout@v4 to @v6 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt and fix clippy warning in signal.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: improve E2E fixture error reporting and prevent stdin blocking - Add --no-onboard flag to prevent wizard from blocking in CI - Pipe /dev/null to stdin to prevent any stdin reads from hanging - Add RUST_BACKTRACE=1 for crash diagnostics - On server startup timeout, dump stderr to pytest output so CI logs show why the server failed to start Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set session-scoped event loop for E2E async fixtures pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to None (function scope), causing session-scoped async fixtures to be re-evaluated per test function with independent event loops. Each test then independently attempts to start the ironclaw server, times out at 120s, and wastes ~24 minutes of CI before the job is cancelled. Setting asyncio_default_fixture_loop_scope = "session" ensures all session-scoped async fixtures share a single event loop, so the server starts once and is reused across all tests. Also adds -x flag to pytest in CI to stop on first failure instead of running all 19 tests when the fixture is broken. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set test loop scope to session to match fixture loop scope With asyncio_default_fixture_loop_scope=session but asyncio_default_test_loop_scope=function (the default), tests run on a per-function event loop while fixtures produce objects (Playwright pages, browser contexts) on the session event loop. This event loop mismatch causes the test to hang indefinitely awaiting Playwright operations that are bound to the wrong loop. Setting both scopes to "session" ensures a single event loop is shared across all fixtures and tests, eliminating the deadlock. Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add roll-up jobs to match branch protection required checks Branch protection expects "Code Style (fmt + clippy)" and "Run Tests" status checks, but only individual job names were reported. Add roll-up jobs that aggregate results and report the expected names. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
662 lines
21 KiB
Rust
662 lines
21 KiB
Rust
//! Tool trait and types.
|
|
|
|
use std::time::Duration;
|
|
|
|
use async_trait::async_trait;
|
|
use rust_decimal::Decimal;
|
|
use serde::{Deserialize, Serialize};
|
|
use thiserror::Error;
|
|
|
|
use crate::context::JobContext;
|
|
|
|
/// How much approval a specific tool invocation requires.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum ApprovalRequirement {
|
|
/// No approval needed.
|
|
Never,
|
|
/// Needs approval, but session auto-approve can bypass.
|
|
UnlessAutoApproved,
|
|
/// Always needs explicit approval (even if auto-approved).
|
|
Always,
|
|
}
|
|
|
|
impl ApprovalRequirement {
|
|
/// Whether this invocation requires approval in contexts where
|
|
/// auto-approve is irrelevant (e.g. autonomous worker/scheduler).
|
|
pub fn is_required(&self) -> bool {
|
|
!matches!(self, Self::Never)
|
|
}
|
|
}
|
|
|
|
/// Per-tool rate limit configuration for built-in tool invocations.
|
|
///
|
|
/// Controls how many times a tool can be invoked per user, per time window.
|
|
/// Read-only tools (echo, time, json, file_read, etc.) should NOT be rate limited.
|
|
/// Write/external tools (shell, http, file_write, memory_write, create_job) should be.
|
|
#[derive(Debug, Clone)]
|
|
pub struct ToolRateLimitConfig {
|
|
/// Maximum invocations per minute.
|
|
pub requests_per_minute: u32,
|
|
/// Maximum invocations per hour.
|
|
pub requests_per_hour: u32,
|
|
}
|
|
|
|
impl ToolRateLimitConfig {
|
|
/// Create a config with explicit limits.
|
|
pub fn new(requests_per_minute: u32, requests_per_hour: u32) -> Self {
|
|
Self {
|
|
requests_per_minute,
|
|
requests_per_hour,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Default for ToolRateLimitConfig {
|
|
/// Default: 60 requests/minute, 1000 requests/hour (generous for WASM HTTP).
|
|
fn default() -> Self {
|
|
Self {
|
|
requests_per_minute: 60,
|
|
requests_per_hour: 1000,
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Where a tool should execute: orchestrator process or inside a container.
|
|
///
|
|
/// Orchestrator tools run in the main agent process (memory access, job mgmt, etc).
|
|
/// Container tools run inside Docker containers (shell, file ops, code mods).
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
|
pub enum ToolDomain {
|
|
/// Safe to run in the orchestrator (pure functions, memory, job management).
|
|
Orchestrator,
|
|
/// Must run inside a sandboxed container (filesystem, shell, code).
|
|
Container,
|
|
}
|
|
|
|
/// Error type for tool execution.
|
|
#[derive(Debug, Error)]
|
|
pub enum ToolError {
|
|
#[error("Invalid parameters: {0}")]
|
|
InvalidParameters(String),
|
|
|
|
#[error("Execution failed: {0}")]
|
|
ExecutionFailed(String),
|
|
|
|
#[error("Timeout after {0:?}")]
|
|
Timeout(Duration),
|
|
|
|
#[error("Not authorized: {0}")]
|
|
NotAuthorized(String),
|
|
|
|
#[error("Rate limited, retry after {0:?}")]
|
|
RateLimited(Option<Duration>),
|
|
|
|
#[error("External service error: {0}")]
|
|
ExternalService(String),
|
|
|
|
#[error("Sandbox error: {0}")]
|
|
Sandbox(String),
|
|
}
|
|
|
|
/// Output from a tool execution.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct ToolOutput {
|
|
/// The result data.
|
|
pub result: serde_json::Value,
|
|
/// Cost incurred (if any).
|
|
pub cost: Option<Decimal>,
|
|
/// Time taken.
|
|
pub duration: Duration,
|
|
/// Raw output before sanitization (for debugging).
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub raw: Option<String>,
|
|
}
|
|
|
|
impl ToolOutput {
|
|
/// Create a successful output with a JSON result.
|
|
pub fn success(result: serde_json::Value, duration: Duration) -> Self {
|
|
Self {
|
|
result,
|
|
cost: None,
|
|
duration,
|
|
raw: None,
|
|
}
|
|
}
|
|
|
|
/// Create a text output.
|
|
pub fn text(text: impl Into<String>, duration: Duration) -> Self {
|
|
Self {
|
|
result: serde_json::Value::String(text.into()),
|
|
cost: None,
|
|
duration,
|
|
raw: None,
|
|
}
|
|
}
|
|
|
|
/// Set the cost.
|
|
pub fn with_cost(mut self, cost: Decimal) -> Self {
|
|
self.cost = Some(cost);
|
|
self
|
|
}
|
|
|
|
/// Set the raw output.
|
|
pub fn with_raw(mut self, raw: impl Into<String>) -> Self {
|
|
self.raw = Some(raw.into());
|
|
self
|
|
}
|
|
}
|
|
|
|
/// Definition of a tool's parameters using JSON Schema.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct ToolSchema {
|
|
pub name: String,
|
|
pub description: String,
|
|
pub parameters: serde_json::Value,
|
|
}
|
|
|
|
impl ToolSchema {
|
|
/// Create a new tool schema.
|
|
pub fn new(name: impl Into<String>, description: impl Into<String>) -> Self {
|
|
Self {
|
|
name: name.into(),
|
|
description: description.into(),
|
|
parameters: serde_json::json!({
|
|
"type": "object",
|
|
"properties": {},
|
|
"required": []
|
|
}),
|
|
}
|
|
}
|
|
|
|
/// Set the parameters schema.
|
|
pub fn with_parameters(mut self, parameters: serde_json::Value) -> Self {
|
|
self.parameters = parameters;
|
|
self
|
|
}
|
|
}
|
|
|
|
/// Trait for tools that the agent can use.
|
|
#[async_trait]
|
|
pub trait Tool: Send + Sync {
|
|
/// Get the tool name.
|
|
fn name(&self) -> &str;
|
|
|
|
/// Get a description of what the tool does.
|
|
fn description(&self) -> &str;
|
|
|
|
/// Get the JSON Schema for the tool's parameters.
|
|
fn parameters_schema(&self) -> serde_json::Value;
|
|
|
|
/// Execute the tool with the given parameters.
|
|
async fn execute(
|
|
&self,
|
|
params: serde_json::Value,
|
|
ctx: &JobContext,
|
|
) -> Result<ToolOutput, ToolError>;
|
|
|
|
/// Estimate the cost of running this tool with the given parameters.
|
|
fn estimated_cost(&self, _params: &serde_json::Value) -> Option<Decimal> {
|
|
None
|
|
}
|
|
|
|
/// Estimate how long this tool will take with the given parameters.
|
|
fn estimated_duration(&self, _params: &serde_json::Value) -> Option<Duration> {
|
|
None
|
|
}
|
|
|
|
/// Whether this tool's output needs sanitization.
|
|
///
|
|
/// Returns true for tools that interact with external services,
|
|
/// where the output might contain malicious content.
|
|
fn requires_sanitization(&self) -> bool {
|
|
true
|
|
}
|
|
|
|
/// Whether this tool invocation requires user approval.
|
|
///
|
|
/// Returns `Never` by default (most tools run in a sandboxed environment).
|
|
/// Override to return `UnlessAutoApproved` for tools that need approval
|
|
/// but can be session-auto-approved, or `Always` for invocations that
|
|
/// must always prompt (e.g. destructive shell commands, HTTP with auth).
|
|
fn requires_approval(&self, _params: &serde_json::Value) -> ApprovalRequirement {
|
|
ApprovalRequirement::Never
|
|
}
|
|
|
|
/// Maximum time this tool is allowed to run before the caller kills it.
|
|
/// Override for long-running tools like sandbox execution.
|
|
/// Default: 60 seconds.
|
|
fn execution_timeout(&self) -> Duration {
|
|
Duration::from_secs(60)
|
|
}
|
|
|
|
/// Where this tool should execute.
|
|
///
|
|
/// `Orchestrator` tools run in the main agent process (safe, no FS access).
|
|
/// `Container` tools run inside Docker containers (shell, file ops).
|
|
///
|
|
/// Default: `Orchestrator` (safe for the main process).
|
|
fn domain(&self) -> ToolDomain {
|
|
ToolDomain::Orchestrator
|
|
}
|
|
|
|
/// Per-invocation rate limit for this tool.
|
|
///
|
|
/// Return `Some(config)` to throttle how often this tool can be called per user.
|
|
/// Read-only tools (echo, time, json, file_read, memory_search, etc.) should
|
|
/// return `None`. Write/external tools (shell, http, file_write, memory_write,
|
|
/// create_job) should return sensible limits to prevent runaway agents.
|
|
///
|
|
/// Rate limits are per-user, per-tool, and in-memory (reset on restart).
|
|
/// This is orthogonal to `requires_approval()` — a tool can be both
|
|
/// approval-gated and rate limited. Rate limit is checked first (cheaper).
|
|
///
|
|
/// Default: `None` (no rate limiting).
|
|
fn rate_limit_config(&self) -> Option<ToolRateLimitConfig> {
|
|
None
|
|
}
|
|
|
|
/// Get the tool schema for LLM function calling.
|
|
fn schema(&self) -> ToolSchema {
|
|
ToolSchema {
|
|
name: self.name().to_string(),
|
|
description: self.description().to_string(),
|
|
parameters: self.parameters_schema(),
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Extract a required string parameter from a JSON object.
|
|
///
|
|
/// Returns `ToolError::InvalidParameters` if the key is missing or not a string.
|
|
pub fn require_str<'a>(params: &'a serde_json::Value, name: &str) -> Result<&'a str, ToolError> {
|
|
params
|
|
.get(name)
|
|
.and_then(|v| v.as_str())
|
|
.ok_or_else(|| ToolError::InvalidParameters(format!("missing '{}' parameter", name)))
|
|
}
|
|
|
|
/// Extract a required parameter of any type from a JSON object.
|
|
///
|
|
/// Returns `ToolError::InvalidParameters` if the key is missing.
|
|
pub fn require_param<'a>(
|
|
params: &'a serde_json::Value,
|
|
name: &str,
|
|
) -> Result<&'a serde_json::Value, ToolError> {
|
|
params
|
|
.get(name)
|
|
.ok_or_else(|| ToolError::InvalidParameters(format!("missing '{}' parameter", name)))
|
|
}
|
|
|
|
/// Lenient runtime validation of a tool's `parameters_schema()`.
|
|
///
|
|
/// Use this function at tool-registration time to catch structural mistakes
|
|
/// (missing `"type": "object"`, orphan `"required"` keys, arrays without
|
|
/// `"items"`) without rejecting intentional freeform properties.
|
|
///
|
|
/// For the stricter variant that also enforces `additionalProperties: false`,
|
|
/// enum-type consistency, and per-property `"type"` fields, see
|
|
/// [`validate_strict_schema`](crate::tools::schema_validator::validate_strict_schema)
|
|
/// in `schema_validator.rs` (used in CI tests).
|
|
///
|
|
/// Returns a list of validation errors. An empty list means the schema is valid.
|
|
///
|
|
/// # Rules enforced
|
|
///
|
|
/// 1. Top-level must have `"type": "object"`
|
|
/// 2. Top-level must have `"properties"` as an object
|
|
/// 3. Every key in `"required"` must exist in `"properties"`
|
|
/// 4. Nested objects follow the same rules recursively
|
|
/// 5. Array properties should have `"items"` defined
|
|
///
|
|
/// Properties without a `"type"` field are allowed (freeform/any-type).
|
|
/// This is an intentional pattern used by tools like `json` and `http` for
|
|
/// OpenAI compatibility, since union types with arrays require `items`.
|
|
pub fn validate_tool_schema(schema: &serde_json::Value, path: &str) -> Vec<String> {
|
|
let mut errors = Vec::new();
|
|
|
|
// Rule 1: must have "type": "object" at this level
|
|
match schema.get("type").and_then(|t| t.as_str()) {
|
|
Some("object") => {}
|
|
Some(other) => {
|
|
errors.push(format!("{path}: expected type \"object\", got \"{other}\""));
|
|
return errors; // Can't check further
|
|
}
|
|
None => {
|
|
errors.push(format!("{path}: missing \"type\": \"object\""));
|
|
return errors;
|
|
}
|
|
}
|
|
|
|
// Rule 2: must have "properties" as an object
|
|
let properties = match schema.get("properties").and_then(|p| p.as_object()) {
|
|
Some(p) => p,
|
|
None => {
|
|
errors.push(format!("{path}: missing or non-object \"properties\""));
|
|
return errors;
|
|
}
|
|
};
|
|
|
|
// Rule 3: every key in "required" must exist in "properties"
|
|
if let Some(required) = schema.get("required").and_then(|r| r.as_array()) {
|
|
for req in required {
|
|
if let Some(key) = req.as_str()
|
|
&& !properties.contains_key(key)
|
|
{
|
|
errors.push(format!(
|
|
"{path}: required key \"{key}\" not found in properties"
|
|
));
|
|
}
|
|
}
|
|
}
|
|
|
|
// Rule 4 & 5: recurse into nested objects and check arrays
|
|
for (key, prop) in properties {
|
|
let prop_path = format!("{path}.{key}");
|
|
if let Some(prop_type) = prop.get("type").and_then(|t| t.as_str()) {
|
|
match prop_type {
|
|
"object" => {
|
|
errors.extend(validate_tool_schema(prop, &prop_path));
|
|
}
|
|
"array" => {
|
|
if let Some(items) = prop.get("items") {
|
|
// If items is an object type, recurse
|
|
if items.get("type").and_then(|t| t.as_str()) == Some("object") {
|
|
errors
|
|
.extend(validate_tool_schema(items, &format!("{prop_path}.items")));
|
|
}
|
|
} else {
|
|
errors.push(format!("{prop_path}: array property missing \"items\""));
|
|
}
|
|
}
|
|
_ => {}
|
|
}
|
|
}
|
|
// No "type" field is intentionally allowed (freeform properties)
|
|
}
|
|
|
|
errors
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// A simple no-op tool for testing.
|
|
#[derive(Debug)]
|
|
pub struct EchoTool;
|
|
|
|
#[async_trait]
|
|
impl Tool for EchoTool {
|
|
fn name(&self) -> &str {
|
|
"echo"
|
|
}
|
|
|
|
fn description(&self) -> &str {
|
|
"Echoes back the input message. Useful for testing."
|
|
}
|
|
|
|
fn parameters_schema(&self) -> serde_json::Value {
|
|
serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"message": {
|
|
"type": "string",
|
|
"description": "The message to echo back"
|
|
}
|
|
},
|
|
"required": ["message"]
|
|
})
|
|
}
|
|
|
|
async fn execute(
|
|
&self,
|
|
params: serde_json::Value,
|
|
_ctx: &JobContext,
|
|
) -> Result<ToolOutput, ToolError> {
|
|
let message = require_str(¶ms, "message")?;
|
|
|
|
Ok(ToolOutput::text(message, Duration::from_millis(1)))
|
|
}
|
|
|
|
fn requires_sanitization(&self) -> bool {
|
|
false // Echo is a trusted internal tool
|
|
}
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_echo_tool() {
|
|
let tool = EchoTool;
|
|
let ctx = JobContext::default();
|
|
|
|
let result = tool
|
|
.execute(serde_json::json!({"message": "hello"}), &ctx)
|
|
.await
|
|
.unwrap();
|
|
|
|
assert_eq!(result.result, serde_json::json!("hello"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_tool_schema() {
|
|
let tool = EchoTool;
|
|
let schema = tool.schema();
|
|
|
|
assert_eq!(schema.name, "echo");
|
|
assert!(!schema.description.is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn test_execution_timeout_default() {
|
|
let tool = EchoTool;
|
|
assert_eq!(tool.execution_timeout(), Duration::from_secs(60));
|
|
}
|
|
|
|
#[test]
|
|
fn test_require_str_present() {
|
|
let params = serde_json::json!({"name": "alice"});
|
|
assert_eq!(require_str(¶ms, "name").unwrap(), "alice");
|
|
}
|
|
|
|
#[test]
|
|
fn test_require_str_missing() {
|
|
let params = serde_json::json!({});
|
|
let err = require_str(¶ms, "name").unwrap_err();
|
|
assert!(err.to_string().contains("missing 'name'"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_require_str_wrong_type() {
|
|
let params = serde_json::json!({"name": 42});
|
|
let err = require_str(¶ms, "name").unwrap_err();
|
|
assert!(err.to_string().contains("missing 'name'"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_require_param_present() {
|
|
let params = serde_json::json!({"data": [1, 2, 3]});
|
|
assert_eq!(
|
|
require_param(¶ms, "data").unwrap(),
|
|
&serde_json::json!([1, 2, 3])
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn test_require_param_missing() {
|
|
let params = serde_json::json!({});
|
|
let err = require_param(¶ms, "data").unwrap_err();
|
|
assert!(err.to_string().contains("missing 'data'"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_requires_approval_default() {
|
|
let tool = EchoTool;
|
|
// Default requires_approval() returns Never.
|
|
assert_eq!(
|
|
tool.requires_approval(&serde_json::json!({"message": "hi"})),
|
|
ApprovalRequirement::Never
|
|
);
|
|
assert!(!ApprovalRequirement::Never.is_required());
|
|
assert!(ApprovalRequirement::UnlessAutoApproved.is_required());
|
|
assert!(ApprovalRequirement::Always.is_required());
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_valid() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"name": { "type": "string", "description": "A name" }
|
|
},
|
|
"required": ["name"]
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert!(errors.is_empty(), "unexpected errors: {errors:?}");
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_missing_type() {
|
|
let schema = serde_json::json!({
|
|
"properties": {
|
|
"name": { "type": "string" }
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("missing \"type\": \"object\""));
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_wrong_type() {
|
|
let schema = serde_json::json!({
|
|
"type": "string"
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("expected type \"object\""));
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_required_not_in_properties() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"name": { "type": "string" }
|
|
},
|
|
"required": ["name", "age"]
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("\"age\" not found in properties"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_nested_object() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"config": {
|
|
"type": "object",
|
|
"properties": {
|
|
"key": { "type": "string" }
|
|
},
|
|
"required": ["key", "missing"]
|
|
}
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("test.config"));
|
|
assert!(errors[0].contains("\"missing\" not found"));
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_array_missing_items() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"tags": { "type": "array", "description": "Tags" }
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("array property missing \"items\""));
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_array_with_items_ok() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"tags": {
|
|
"type": "array",
|
|
"items": { "type": "string" }
|
|
}
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert!(errors.is_empty(), "unexpected errors: {errors:?}");
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_freeform_property_allowed() {
|
|
// Properties without "type" are intentionally allowed (json/http tools)
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"data": { "description": "Any JSON value" }
|
|
},
|
|
"required": ["data"]
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert!(
|
|
errors.is_empty(),
|
|
"freeform property should be allowed: {errors:?}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_nested_array_items_object() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"headers": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"name": { "type": "string" },
|
|
"value": { "type": "string" }
|
|
},
|
|
"required": ["name", "value"]
|
|
}
|
|
}
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert!(errors.is_empty(), "unexpected errors: {errors:?}");
|
|
}
|
|
|
|
#[test]
|
|
fn test_validate_schema_nested_array_items_object_bad() {
|
|
let schema = serde_json::json!({
|
|
"type": "object",
|
|
"properties": {
|
|
"headers": {
|
|
"type": "array",
|
|
"items": {
|
|
"type": "object",
|
|
"properties": {
|
|
"name": { "type": "string" }
|
|
},
|
|
"required": ["name", "missing_field"]
|
|
}
|
|
}
|
|
}
|
|
});
|
|
let errors = validate_tool_schema(&schema, "test");
|
|
assert_eq!(errors.len(), 1);
|
|
assert!(errors[0].contains("headers.items"));
|
|
assert!(errors[0].contains("\"missing_field\""));
|
|
}
|
|
}
|