4 Commits

Include renames

Author	SHA1	Message	Date
Henry ParkandGitHub	7d745d5479	tools: improve routine schema guidance (#1089)	2026-03-13 11:24:45 -07:00
4 people	6e1ed939cc	Add event-triggered routines and workflow skill templates (#756) ... * Add event-triggered routines and workflow skill templates * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 <[email protected]> * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback for event_emit security and quality Security fixes: - Require approval (UnlessAutoApproved) for event_emit, matching routine_fire - Enable sanitization on event_emit payload (external JSON reaches LLM) - Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id Correctness fixes: - Rename source → event_source in event_emit for consistency with routine_create - Use json_value_as_filter_string for filter parsing (handles numbers/booleans) - Case-insensitive matching for event source and event_type - Add debug logging for missing filter keys in payload - Fix skill_install_routine_webhook_sim test missing .with_skills() - Fix schema_validator test for event_emit payload properties Code quality: - Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout) - Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs - Add test section headers in e2e_routine_heartbeat.rs - Clarify event_emit description to specify system_event routines only Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 <[email protected]> * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> * fix: make routine_system_event_emit test create routine before emitting - Add routine_create step to trace fixture so event_emit has a matching routine to fire - Assert fired_routines > 0, not just key presence (Copilot review) - Add .with_auto_approve_tools(true) since event_emit now requires approval Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: renumber test headers after system_event test insertion Test 4 was duplicated (routine_cooldown and heartbeat_findings). Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: merge staging and add missing RoutineEngine args in test RoutineEngine::new on staging requires `tools` and `safety` params. Update system_event_trigger_matches_and_filters test to pass them. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address new Copilot review comments - Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim test so event_emit doesn't block on approval - Fix module-level doc comment for event_emit to specify system_event trigger [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: deduplicate json_value_as_string helper Remove private `json_value_as_string` from routine_engine.rs and use the identical public `json_value_as_filter_string` from routine.rs, eliminating divergence risk. (Copilot review) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Henry Park <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>	2026-03-10 11:08:04 -07:00
3 people	ae89a52ac2	feat(routines): approval context for autonomous job execution (#577) ... * feat(routines): add approval context for autonomous job execution Routines and background jobs were unable to use any tools that required approval (file ops, shell, message, http), making them effectively useless. This adds an ApprovalContext system that lets autonomous jobs pre-authorize tools at dispatch time. - Add ApprovalContext enum with Autonomous variant that auto-approves UnlessAutoApproved tools and optionally pre-authorizes Always tools - Add tool_permissions field to RoutineAction::FullJob for pre-authorizing Always-gated tools (e.g. destructive shell, cross-channel messaging) - Add Scheduler::dispatch_job_with_context() to thread approval context through to workers - Set message tool default channel/target from routine NotifyConfig so routines can send results without cross-channel approval - Fix Completed→Completed state transition error in worker (plan marks job completed, then direct loop or outer run() tries again) Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(routines): add E2E trace for routine news digest workflow Add a 3-turn trace fixture and test that exercises: - Turn 1: routine_create with full_job mode and tool_permissions - Turn 2: Simulated digest workflow with echo + memory_write - Turn 3: Verification via memory_search [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(test): wire RoutineEngine into test rig for routine_create E2E - Add `with_routines()` to TestRigBuilder that passes a RoutineConfig to Agent::new, enabling routine tool registration during agent startup - Add Turn 2 (routine_list) to the trace to verify routine persistence in the database after routine_create - Fix formatting issues flagged by CI (cargo fmt) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(scheduler): deduplicate dispatch_job and dispatch_job_with_context Extract shared logic into private `dispatch_job_inner` to prevent divergence when dispatch behavior changes in the future. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(routines): add routine_fire tool and real E2E routine execution test - Add `routine_fire` tool that calls `RoutineEngine::fire_manual` to trigger a routine on demand. Registered alongside the other 5 routine tools (now 6 total). - Rewrite the routine_news_digest E2E trace to exercise the full execution stack end-to-end: 1. routine_create (manual trigger, full_job, tool_permissions: [message]) 2. routine_fire → RoutineEngine → Scheduler::dispatch_job_with_context → autonomous Worker consuming TraceLlm steps 3. Worker calls echo → memory_write → message (broadcast to test channel) 4. Test verifies the message broadcast arrived, proving ApprovalContext correctly allowed the Always-approval message tool - Register message tools in TestRig so routines can send messages to the test channel via channel_manager.broadcast(). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(routines): wire HttpInterceptor through scheduler for routine worker http calls Propagate http_interceptor from AgentDeps → Scheduler → WorkerDeps → JobContext so that routine workers (and any scheduler-dispatched workers) can use the ReplayingHttpInterceptor for mock HTTP responses during tests. Changes: - Add http_interceptor field to Scheduler and WorkerDeps - Set job_ctx.http_interceptor in Worker before tool execution - Add with_http_exchanges() builder method to TestRigBuilder - Replace echo tool with http tool in routine_news_digest trace - Test now exercises real http tool with mock response → memory_write → message Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review comments from Copilot on PR #577 - Extract `ApprovalContext::is_blocked_or_default()` helper to deduplicate approval check logic in worker.rs and scheduler.rs - Extract `parse_tool_permissions()` helper to deduplicate JSON array parsing in routine.rs and builtin/routine.rs - Fix test name: `test_mark_completed_twice_does_not_error` → `test_mark_completed_twice_returns_error` (matches actual behavior) - Fix ApprovalContext doc comment to clarify it only models autonomous mode - Fix flaky index-based assertion in routine_news_digest test — now uses content-based search instead of fixed position - Fix stale comment: echo → http in routine test header - Add TODO for subtask approval context propagation (latent, not in active code paths) - Add TODO for global message tool context race in routine_engine Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in is_blocked_or_default test Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(test_rig): destructure self in build() to avoid partial-move fragility Destructure TestRigBuilder at the top of build() instead of accessing self.* fields after moving self.http_exchanges. While the prior code compiled (remaining fields are Copy), it was fragile and would break if any non-Copy field were added. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: clarify that routine_fire bypasses cooldown Manual fires are explicitly user-initiated and intentionally bypass cooldown checks (which only apply to automated cron/event triggers). Updated tool description and fire_manual docstring to make this clear. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(routines): fix message tool approval in routine context Two fixes for message tool failures in autonomous routine jobs: 1. MessageTool::requires_approval() now returns UnlessAutoApproved when the explicit channel param matches the default channel (was Always, causing "requires authentication" errors for routine workers). 2. routine_create tool now accepts notify_channel and notify_user params, wired into NotifyConfig. Without these, routines had channel: None, so set_message_tool_context was never called, causing "No channel specified" errors. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(message): remove approval requirement from message tool The message tool only sends to user-owned channels via ChannelManager::broadcast (TUI, Telegram, Slack, web gateway, etc.). It cannot reach arbitrary external services, so approval adds friction with no security benefit. This also eliminates the routine context errors entirely since approval is never checked. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review comments — routine_fire approval + test rename - routine_fire now returns UnlessAutoApproved since firing a routine can dispatch a full_job with pre-authorized Always-gated tools - Rename test_approval_context_never_always_passes to test_approval_context_never_is_not_blocked for clarity Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review nits — update stale docs and comments - Remove 'message' from tool_permissions example (no longer Always) - Reword message tool approval comment for accuracy - Clarify with_routines() docstring re: tool registration vs engine wiring Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-03-07 05:21:58 +00:00
3 people	a24fd3e8a3	Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353) ... * Add automated QA: tool schema validator, feature-flag CI matrix, Docker build P0 items from the automated QA plan (#352): - Add validate_tool_schema() that checks OpenAI strict-mode rules (type: object, required keys in properties, nested object/array recursion) with 10 unit tests and 6 integration tests covering all core built-in tools - CI test matrix now runs with --all-features, default features, and --no-default-features --features libsql to catch dead code behind wrong cfg gates - CI clippy now runs the same 3-feature matrix with --all flags - Docker build job added to catch missing files in Dockerfile Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P1 automated QA tests and fix LeakDetector prefix shadowing bug P1 test coverage: config round-trip (settings + bootstrap), shell tool arg handling, safety adversarial tests (sanitizer, leak detector, allowlist), turn persistence (conversations, metadata, pagination, jobs), and a clippy fix for libsql-only builds. Fixed a real bug where AhoCorasick non-overlapping prefix iteration caused shorter prefixes (e.g. "sk-") to shadow longer ones (e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key detection. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P2 automated QA tests: chaos, lifecycle, collision, and recovery Cover all P2 items from the automated QA plan: - Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors) - Failover chaos tests (hanging failover, all-fail, tools path, single provider) - Value estimator boundary tests (negative cost, zero price, zero earnings) - Context length recovery test (ContextLengthExceeded -> compact -> retry) - WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation) - Extension registry collision tests (same-name different-kind coexistence) - Extension filesystem collision tests (separate dirs, detect_kind priority) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P3 concurrent stress tests for ContextManager and SessionManager Tests verify thread safety of double-checked locking, TOCTOU prevention, and RwLock-based concurrent access patterns under load. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add dispatcher loop guard and self-repair stuck job tests Dispatcher: test force_text mechanism prevents infinite tool call loops, verify iteration bound arithmetic guarantees termination for all configs. Self-repair: test stuck job detection, recovery within attempt limits, manual escalation when limit exceeded, graceful degradation without store/builder dependencies. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure design doc Python + Playwright framework with mock LLM server for deterministic browser-level testing of the web gateway. Covers connection/auth, chat round-trip with SSE streaming, and skills lifecycle scenarios. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure implementation plan 10-task plan covering: scaffolding, mock LLM server, helpers, conftest fixtures, connection/chat/skills test scenarios, CI workflow, README, and integration run. Co-Authored-By: Claude Opus 4.6 <[email protected]> * scaffold: E2E test project with pyproject.toml Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E helpers with DOM selectors and port discovery Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: mock OpenAI-compat LLM server for E2E tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E conftest with session fixtures for mock LLM and ironclaw Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 1 -- connection and tab navigation tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 2 -- chat message round-trip tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 3 -- skills search, install, remove tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add weekly E2E test workflow with Playwright Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: E2E test README with setup and usage instructions Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test integration fixes from first run - Use temp file DB instead of :memory: (libSQL :memory: doesn't persist tables across execute_batch) - Fix installed skills selector: #skills-list not #installed-skills - Add pytest-timeout to dependencies - Improve skills install/remove test with wait_for instead of fixed sleeps 8 passed, 1 skipped (skills install depends on ClawHub availability) Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1) Add src/tools/schema_validator.rs with validate_strict_schema() that checks tool parameter schemas against OpenAI function calling strict-mode rules: type object at top level, required keys in properties, enum type consistency, array items definitions, nested object recursion, and additionalProperties. 17 tests validate all 34+ built-in tool schemas across 5 test groups: - 9 simple tools (echo, time, json, http, shell, file read/write/list/patch) - 4 job tools (create, list, status, cancel) - 4 skill tools (list, search, install, remove) - 13 inline schemas for extension, routine, and complex job tools - 4 memory tool schemas Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test reliability for HTML injection and SSE reconnect - HTML injection: test sanitization directly via JS injection instead of depending on full LLM round-trip (avoids intermittent 404 from mock) - SSE reconnect: increase wait times for DB persistence and relax assertion to check total message count after history reload Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt formatting Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add WASM and MCP tool schema validation tests (QA 1.1) Extends the schema validator with representative WASM tool schemas (weather, HTTP client, batch processor, status), MCP tool schemas (default, file read, SQL query, strict mode), and defect detection tests for common external schema issues (missing type, typo in required, array without items, enum type mismatch). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add auth middleware and compaction module tests Auth middleware (8 new tests): valid/invalid bearer tokens, query param fallback, case sensitivity, empty tokens, whitespace handling. Compaction module (16 new tests): truncation strategy, summarize strategy with mock LLM, workspace fallback, format_turns helper, sequential compactions, coherence after compaction, token decrease verification. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add config round-trip integration tests (QA 1.2) Test the full bootstrap .env lifecycle: write via the same format as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy, and assert values match. Covers LLM backend selection, embedding disable flag, onboard completion flag, session token keys, multi-key preservation across upsert, and special characters (spaces, equals, quotes, backslashes, hashes). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4) Value estimator (14 new tests): zero/negative prices, large values, negative cost, exact margin boundaries, custom margin configuration. Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates when all tool calls fail (regression guard for PR #252 infinite loop) and when max iterations are reached. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add failover edge cases and provider chaos tests (QA 2.6/4.1) Failover edge cases (4 new tests): cooldown at zero nanos, half-open failure reopens circuit, all providers fail gracefully (no panic), single failing provider with cooldown. Provider chaos tests (15 new tests): flakey provider with retries, hanging provider with timeout, garbage provider, circuit breaker trip/recover, failover chain cascading, non-transient error stops chain, full stack integration (retry + failover + circuit breaker). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback on QA tests - Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs) - Refactor bootstrap.rs to expose path-parameterized variants so config_round_trip tests call real code instead of reimplementations - Remove deprecated event_loop fixture, use dynamic ports, minimal env, session-scoped browser, and wire HEADED=1 in E2E conftest - Add cross-referencing doc comments between schema validators - Simplify array validation logic in tool.rs - Bump e2e.yml checkout@v4 to @v6 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt and fix clippy warning in signal.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: improve E2E fixture error reporting and prevent stdin blocking - Add --no-onboard flag to prevent wizard from blocking in CI - Pipe /dev/null to stdin to prevent any stdin reads from hanging - Add RUST_BACKTRACE=1 for crash diagnostics - On server startup timeout, dump stderr to pytest output so CI logs show why the server failed to start Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set session-scoped event loop for E2E async fixtures pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to None (function scope), causing session-scoped async fixtures to be re-evaluated per test function with independent event loops. Each test then independently attempts to start the ironclaw server, times out at 120s, and wastes ~24 minutes of CI before the job is cancelled. Setting asyncio_default_fixture_loop_scope = "session" ensures all session-scoped async fixtures share a single event loop, so the server starts once and is reused across all tests. Also adds -x flag to pytest in CI to stop on first failure instead of running all 19 tests when the fixture is broken. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set test loop scope to session to match fixture loop scope With asyncio_default_fixture_loop_scope=session but asyncio_default_test_loop_scope=function (the default), tests run on a per-function event loop while fixtures produce objects (Playwright pages, browser contexts) on the session event loop. This event loop mismatch causes the test to hang indefinitely awaiting Playwright operations that are bound to the wrong loop. Setting both scopes to "session" ensures a single event loop is shared across all fixtures and tests, eliminating the deadlock. Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add roll-up jobs to match branch protection required checks Branch protection expects "Code Style (fmt + clippy)" and "Run Tests" status checks, but only individual job names were reported. Add roll-up jobs that aggregate results and report the expected names. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-27 09:09:45 +04:00