Compare commits

...
70 Commits
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1156884a49 chore: release v0.12.0 (#331)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-26 12:12:58 +04:00
996c6a8cc9 feat(web): improve WASM channel setup flow (#380)
* feat(web): improve WASM channel setup flow with stepper UI and auto-configure

Streamline the WASM channel setup experience in the web gateway:

- Auto-open configure modal after installing a WASM channel
- Add progress stepper (Installed → Configured → Active) on channel cards
- Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart)
- Show "Awaiting Pairing" status for Telegram until first user is paired
- Add SSE extension_status events for real-time status updates
- Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard
- Always mount webhook routes at startup so hot-added channels work without restart
- Add pairing request polling (10s interval) on extensions tab
- Track activation errors per channel with inline error display

Includes review fixes: activation_error priority over active status, stepper
failed state rendering, restart poll timeout, configure modal double-submit
guard, and SSE sender ordering constraint documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: address PR review comments

- Move PairingStore construction outside .map() loop
- Extract createReconfigureButton() helper to reduce duplication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 22:27:13 -08:00
abda94d44f fix: correct MCP registry URLs and remove non-existent Google endpoints (#370)
Audit all built-in MCP server URLs against live endpoints. Fix 5 broken
paths (Linear, Sentry, Cloudflare, Asana, Intercom), fix 1 broken host
(GitHub), and remove 2 entries (Google Drive, Google Calendar) whose
domain mcp.google.com does not exist and Google has no official remote
MCP servers for these products.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-26 00:04:39 +00:00
443b120272 feat(web): inline tool activity cards with auto-collapsing (#376)
* feat(web): inline tool activity cards with auto-collapsing

Add Claude/Codex-style inline tool activity cards to the web UI that
show tool execution progress directly in the chat conversation.

While processing:
- Animated thinking dots with message text (e.g. "Calling LLM...")
- Individual tool cards with live spinner and elapsed timer
- Cards show tool name, duration, and expandable output preview

After response arrives:
- Activity group auto-collapses to "Used N tools (Xs)"
- Click summary to expand and see individual tool cards
- Click card header to see tool output in monospace

Also includes:
- "Calling LLM..." thinking status from dispatcher (all channels)
- 5-minute max timer guard to prevent leaks on dropped SSE
- Handles parallel tools, same tool twice, failures, thread switching

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): use frozen duration for completed tools in activity summary

The collapsed activity summary was showing inflated total duration
because finalizeActivityGroup() recalculated elapsed time from
Date.now() for already-completed tools. Now each tool card stores
its final duration at completion time and the summary uses that
frozen value instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:35 -08:00
2477923af2 fix: resolve_thread adopts existing session threads by UUID (#377)
* fix: resolve_thread adopts existing session threads by UUID

When chat_new_thread_handler creates a thread directly in the session,
it doesn't register a thread_map entry. On the first message,
resolve_thread would create a duplicate thread with a different UUID,
causing:

- Thread appears empty when switching back (loadHistory queries the
  original UUID but turns live on the duplicate)
- Orphaned tabs in the thread list (both the original and duplicate
  appear)

Fix: before creating a new thread, check if the external_thread_id is
itself a UUID that exists as a thread in the session. If so, adopt it
and register the mapping. A mapped_elsewhere guard preserves channel
scope isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: double-checked locking in resolve_thread UUID adoption

Re-check mapped_elsewhere after acquiring the write lock to prevent
a TOCTOU race where another task could map the same UUID between
the read lock check and write lock insertion, breaking channel
isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:10 -08:00
0c5f082d16 feat(web): display logs newest-first in web gateway UI (#369)
Reverse log display order so the most recent entries appear at the top,
removing the need to scroll to see latest activity.

Frontend: rename appendLogEntry to prependLogEntry, use prepend() for
DOM insertion, cap oldest entries from the bottom, and auto-scroll to
top. Backend: update recent_entries() doc comment to clarify the
oldest-first return order works correctly with the frontend's prepend.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:00:09 -08:00
db2ba424ce Add --version flag with clap built-in support and test (#342)
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-25 17:25:07 +04:00
ibhagwanandGitHub e41b282868 feat(signal): tool approval workflow and status updates (#350)
* fix(signal): send approval prompts to users

The Signal channel was not handling StatusUpdate::ApprovalNeeded,
causing approval requests to be silently ignored and users to
never see approval prompts.

This adds proper handling of ApprovalNeeded status that sends
a formatted message to the user with:
- Tool name and description
- Parameters (formatted as JSON)
- Request ID for reference
- Instructions on how to approve/deny/always-approve

The message uses Signal's markdown-style formatting for better
readability on mobile devices.

* feat(signal): add missing StatusUpdate handlers

Add handling for all StatusUpdate variants in Signal channel,
bringing it on par with Telegram's implementation:

- ToolStarted: Shows spinner icon when tool execution begins
- ToolCompleted: Shows checkmark/X based on success/failure
- JobStarted: Shows sandbox job start with ID and URL
- AuthRequired: Shows auth prompt with instructions and URLs
- AuthCompleted: Shows auth success/failure with optional message

This ensures Signal status feedback users receive full during
tool execution, approvals, and authentication flows, matching
the experience of Telegram and other channels.

fix(signal): address clippy warnings and improve error handling

- Collapse nested if statements into let-chains
- Fix needless borrow on Status message
- Extract send_status_message helper to reduce duplication
- Add warning logs for failed message sends

* fix(signal): suppress 'Done' status messages to user

* feat(signal): debug mode parity with REPL

- Add debug_mode to SignalChannel toggled via /debug command
- Gate ToolResult, ToolStarted, ToolCompleted behind debug mode
- Add tests: debug_mode_disabled_by_default, debug_mode_toggle, debug_mode_persists_across_toggles
2026-02-25 16:34:54 +04:00
62dc5d046e feat: add OpenRouter preset to setup wizard (#270)
* feat: add OpenRouter preset to setup wizard

Add OpenRouter as a top-level provider option in the onboarding wizard
(Step 3). Selecting it pre-fills the base URL (https://openrouter.ai/api/v1)
and prompts for an API key, avoiding manual URL entry. Under the hood it
uses the existing openai_compatible backend.

Inlines the key collection flow (rather than delegating to
setup_api_key_provider) so success messages consistently say "OpenRouter"
instead of "openai_compatible", including the early-return env-key path.

Closes #178

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address serrrfirat review comments on OpenRouter wizard preset

- Re-run path now recognizes OpenRouter: display shows "OpenRouter"
  and keep-current routes to setup_openrouter() when base URL contains
  openrouter.ai
- Refactor setup_openrouter() to delegate to setup_api_key_provider()
  with a display_name override, eliminating ~40 lines of duplication
- Update README: remove false claim about model fetching from
  OpenRouter API, add footnote explaining shared secret/env var
  between OpenRouter and OpenAI-compatible
- Fix pre-existing clippy warning in settings.rs (field_reassign_with_default)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-24 22:23:39 +04:00
DevBrocoandGitHub 4d27079cc3 Update FEATURE_PARITY.md (#337)
change status of completion 
2026-02-24 14:56:38 +04:00
e9f32eaebe fix: resolve telegram/slack name collision between tool and channel registries (#346)
When installing the Telegram WASM channel via the web UI, a name collision
between registry/tools/telegram.json and registry/channels/telegram.json
caused the tool entry to win, installing to ~/.ironclaw/tools/ instead of
~/.ironclaw/channels/. This made activation fail with "WASM runtime not
available".

- Add `get_with_kind()` to ExtensionRegistry for kind-aware lookup
- Use `kind_hint` parameter in `install()` to resolve collisions
- Rename tool entries to avoid future collisions: telegram → telegram-mtproto,
  slack → slack-tool
- Fix `_bundles.json` stale reference (tools/slack → tools/slack-tool)
- Fix `cache_discovered()` to deduplicate by (name, kind) consistently
- Add path traversal validation to install/activate/remove entry points
- Add tests for kind-aware lookup, discovery cache, and bundle resolution

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-24 15:29:25 +08:00
ibhagwanandGitHub b0b3a50fa3 feat(channels): add native Signal channel via signal-cli HTTP daemon (#271)
* feat(channels): add native Signal channel via signal-cli HTTP daemon

Implement a native Rust Signal channel that connects to a running
signal-cli daemon's HTTP endpoint, enabling Signal messaging without
WASM overhead.

Architecture:
- SSE listener at /api/v1/events for receiving messages with automatic
  reconnection and exponential backoff
- JSON-RPC client at /api/v1/rpc for sending messages and typing
  indicators
- Reply target tracking via Arc<RwLock<HashMap>> to route responses
  back to the correct DM or group conversation

Features:
- User allowlisting supporting E.164 phone numbers, bare UUIDs, and
  uuid:-prefixed identifiers (matching OpenClaw's format)
- Group allowlisting with wildcard (*) support
- Configurable story and attachment-only message filtering
- Health check via signal-cli /api/v1/check
- Broadcast support to all tracked reply targets

Configuration via environment variables:
- SIGNAL_HTTP_URL, SIGNAL_ACCOUNT (required)
- SIGNAL_ALLOWED_USERS, SIGNAL_ALLOWED_GROUPS
- SIGNAL_IGNORE_ATTACHMENTS (default: false)
- SIGNAL_IGNORE_STORIES (default: true)

Includes unit tests covering allowlist logic, envelope parsing,
recipient targeting, SSE deserialization, and edge cases.

* refactor(signal): remove expect|unwrap calls

- Change SignalChannel::new to return Result<Self, ChannelError>
- Replace .expect() on reqwest client build with proper error handling
- Replace .expect() on NonZeroUsize with compile-time const using unsafe new_unchecked
- Propagate errors through test helpers to avoid unwraps in tests

* fix(signal): prevent OOM from chunked response without Content-Length

Use bytes_stream() to check response size during download rather than
buffering entire body first. This closes the OOM vector where a
malicious signal-cli daemon could send unbounded chunked data.

* fix(signal): align is_e164 minimum digits with setup wizard

Both now require 7-15 digits after '+', preventing environment
variable bypass of the stricter onboarding validation.

* refactor(signal): extract from_parts constructor

Extract SignalChannel::from_parts() used by both new() and
sse_listener() to ensure consistent object construction.

* chore: remove redundant unused var

* refactor(signal): rename allowed_users to allow_from and add dm_policy/group_policy

- Rename allowed_users -> allow_from for consistency with other channels
- Rename allowed_groups -> allow_from_groups
- Add dm_policy field: 'open', 'allowlist', or 'pairing' (default: 'pairing')
- Add group_policy field: 'allowlist', 'open', or 'disabled' (default: 'allowlist')
- Add group_allow_from field that inherits from allow_from if empty
- Implement dm_policy and group_policy logic in message processing
- Add environment variable resolution: SIGNAL_ALLOW_FROM, SIGNAL_ALLOW_FROM_GROUPS,
  SIGNAL_DM_POLICY, SIGNAL_GROUP_POLICY, SIGNAL_GROUP_ALLOW_FROM
- Add setup wizard prompts for new policy options
- Note: full pairing flow (PairingStore integration) marked as pending for future PR

* feat(signal): implement DM pairing workflow for unapproved senders

- Add PairingStore integration to check approved senders
- Handle pairing requests for unknown senders with dm_policy=pairing
- Send pairing reply message with approval instructions
- Update FEATURE_PARITY.md to reflect DM pairing support

* chore(ci): fix clippy warnings
2026-02-24 10:25:16 +04:00
3e552e0e8e fix: make onboarding installs prefer release artifacts with source fallback (#323)
* fix: make onboarding installs prefer release artifacts with source fallback

* fix: harden extension fallback errors and surface setup warnings

* fix: validate registry artifacts and harden fallback errors

* fix: address review feedback on installer fallback

- Add upfront validate_manifest_install_inputs() in
  install_with_source_fallback so bad manifests fail fast without
  relying on inner methods to catch them
- Document ALLOWED_ARTIFACT_HOSTS as GitHub-only by design
- Document intentional url omission from DownloadFailed Display
- Add channel manifest validation tests (wrong prefix rejected,
  correct prefix accepted)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: require SHA256 checksum for artifact downloads

Reject artifact installs when the manifest has sha256: null instead of
warning and proceeding. This prevents installing unverified pre-built
binaries during onboarding. The check runs before downloading to avoid
wasting bandwidth.

Since InvalidManifest blocks source fallback, manifests with URLs but
no checksums will hard-fail rather than silently falling back to source
build — forcing the manifest to be fixed.

The release CI already computes SHA256 for each bundle; the manifests
just need to be populated with the actual values.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce SHA256 checksums and auto-patch manifests in CI

- Fix cargo fmt on SHA256 check code
- Reorder release CI: build WASM extensions before binary so manifests
  can be patched with computed SHA256 before build.rs embeds them
- Add "Patch manifests with WASM checksums" step in build-local-artifacts
  that reads checksums.txt and updates registry JSON files before building
- Add update-registry-checksums job that commits patched manifests back
  to main after release, keeping the repo in sync with released artifacts

This closes the integrity gap where all manifests had sha256: null and
artifact downloads were unverified. The binary now embeds correct SHA256
values and the installer hard-rejects null checksums.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Bowen Wang <[email protected]>
2026-02-23 18:49:18 +00:00
cbf5c93578 fix: copy missing files in Dockerfile to fix build (#322)
* fix: copy missing files in Dockerfile to fix build

The Docker build failed because Cargo.toml references files that were
not copied into the builder stage:

1. tests/html_to_markdown.rs — declared as [[test]] in Cargo.toml,
   Cargo validates the path exists even when only building a binary.
2. build.rs — auto-discovered build script that embeds registry
   manifests at compile time via include_str!(env!("OUT_DIR")).
3. registry/ — contains extension manifests read by build.rs to
   generate the embedded catalog.

Added COPY directives for build.rs, tests/, and registry/.

Fixes nearai/ironclaw#320

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback on WASM channel omission

- Add Dockerfile comment documenting that channels-src/ is intentionally
  omitted since WASM compilation requires wasm32-wasip2 and wasm-tools
  which are not installed in the builder stage

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add WASM channel compilation support to Docker build

- Copy channels-src/ into builder stage for Telegram/Slack/Discord/WhatsApp
- Install wasm32-wasip2 target and wasm-tools so build.rs can compile
  WASM channel components instead of silently skipping them

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 22:37:52 +04:00
Rui ChenandGitHub 0d9b6f3208 docs: add brew install ironclaw instructions (#310)
Signed-off-by: Rui Chen <[email protected]>
2026-02-23 18:04:57 +00:00
4e2dd76ae5 Fix skills system: enable by default, fix registry and install (#300)
* feat: add Docker detection module with platform guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Docker sandbox step to setup wizard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: show Docker status in boot screen

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: check Docker availability at startup

When SANDBOX_ENABLED=true, proactively detect whether Docker is
installed and running before creating the ContainerJobManager.
If Docker is unavailable, log a warning with platform-specific
guidance and disable the sandbox for the session.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable sandbox by default, improve wizard explanation, document detection limits

- SandboxConfig defaults to enabled=true (startup check disables
  gracefully if Docker is unavailable)
- Wizard step explains why Docker matters: isolation for LLM-generated
  code vs running directly on the host
- Document detection confidence per platform in detect.rs module docs:
  high on macOS/Linux, medium on Windows (named pipe edge cases)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: cargo fmt + update test_builder_defaults for enabled-by-default

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate wizard Docker status handling per review

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: fix skills system - enable by default, fix registry connectivity and install

- Enable skills system by default (SKILLS_ENABLED no longer required)
- Bypass Vercel TLS fingerprint blocking by pointing DEFAULT_REGISTRY_URL
  directly at the Convex backend (wry-manatee-359.convex.site)
- Handle ZIP archives from ClawHub download API - the registry returns
  ZIP files containing SKILL.md, not raw text. Uses flate2 (existing dep)
  to extract SKILL.md from the archive.
- Surface catalog search errors in the UI with a yellow warning banner
  instead of silently returning empty results
- Handle both {"results":[...]} envelope and bare [...] array JSON formats
  from the search API
- Add ClawHub links and metadata to search result cards (clickable skill
  names linking to clawhub.ai, relevance score, "updated X ago" recency)
- Fix 3 pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address security review feedback on ZIP extraction and SSRF

- Cap download size to 10 MB before reading response body
- Guard against ZIP bombs: cap uncompressed_size at 1 MB, wrap
  DeflateDecoder with .take() read limit
- Use checked_add for ZIP header offset arithmetic to prevent overflow
- Remove .unwrap() on try_into() -- use direct array construction
- Handle IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) in SSRF checks
- Don't leak internal registry URLs in user-facing catalog_error messages
- Fix non-ASCII panic in catalog response debug logging (use .get() instead
  of byte slicing)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add /skills command and enrich search results with ClawHub metadata

- Parse /skills and /skills search <query> as SystemCommands in submission.rs
- Add skill_catalog to AgentDeps and wire it through main.rs
- Handle "skills" command in commands.rs: list installed skills and search ClawHub
- Add /skills and /skills search <q> entries to /help output
- Add SkillDetail, SkillStats, SkillOwner structs to catalog.rs
- Add fetch_skill_detail() calling GET /api/v1/skills/{slug} on Convex backend
- Add enrich_search_results() to fetch stars/downloads/owner for top 5 results in parallel
- Fix SkillDetailResponse wrapper struct to match actual API shape: {"skill":{...},"owner":{...}}
- Surface stars, downloads, owner in web UI skill search cards (app.js)
- Surface enriched data in skills web handler and skill_search tool output

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: cargo fmt after merge conflict resolution

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: separate installed_skills dir for correct trust on restart, remove duplicate handlers

Trust level bug: skills installed from ClawHub were written to user_dir
(~/.ironclaw/skills/) which is discovered as Trusted on restart. Now installs
go to ~/.ironclaw/installed_skills/ which is discovered as Installed, matching
the documented skill directory layout.

Changes:
- SkillsConfig: add installed_dir field (SKILLS_INSTALLED_DIR env var,
  default ~/.ironclaw/installed_skills/)
- SkillRegistry: add with_installed_dir() builder, installed_dir()/
  install_target_dir() accessors, and discover installed_dir with
  SkillTrust::Installed in discover_all()
- All install paths (web handler, skill tool) use install_target_dir()
  instead of user_dir() so new installs land in the correct directory
- 3 new registry tests: test_installed_dir_uses_installed_trust,
  test_install_target_dir_prefers_installed_dir,
  test_user_dir_stays_trusted_with_installed_dir

Duplicate handler cleanup: handlers/skills.rs was the canonical implementation
but the handlers module was never compiled (not declared in web/mod.rs), so
server.rs had its own duplicate inline definitions that the router used.
Wire up the handlers module, delete the 260-line duplicate in server.rs, and
have server.rs import skills handlers from handlers::skills. Fix pre-existing
compile error in handlers/extensions.rs (missing needs_setup field). Add
#[allow(dead_code)] on not-yet-migrated handler modules to suppress warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: probe more Docker socket paths on macOS

Docker Desktop 4.13+ (stabilised in 4.18) no longer creates the
/var/run/docker.sock symlink by default. The API socket lives at
~/.docker/run/docker.sock, which bollard's connect_with_local_defaults()
does not try.

Add a fallback probe list covering the common macOS container runtimes:
- ~/.docker/run/docker.sock   — Docker Desktop 4.13+
- ~/.colima/default/docker.sock — Colima
- ~/.rd/docker.sock             — Rancher Desktop

Remove the bogus ~/.docker/desktop/docker.sock path that was added
previously; it is not an API socket on any known Docker installation.

Fixes the false-negative "Docker is installed but not running" warning
reported by Illia on macOS with Docker Desktop 4.18+.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* Harden Docker detection for rootless Linux and Windows fallback

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 10:04:02 -08:00
f4ba85ffa2 fix: fall back to build-from-source when extension download fails (#312)
* fix: fall back to build-from-source when extension download fails

Extension manifests hardcode GitHub release URLs for WASM artifacts,
but these artifacts are not yet published to any release. This causes
all WASM extension installs to fail with HTTP 404.

Add a fallback_source field to RegistryEntry so that when the primary
WasmDownload source fails (e.g., 404), the installer automatically
falls back to WasmBuildable (build from source). The manifest
conversion now populates this fallback whenever a download URL is set.

Fixes nearai/ironclaw#298

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Copilot/Gemini review feedback

- Skip fallback for AlreadyInstalled errors (Gemini)
- Include both primary and fallback errors in combined message (Copilot)
- Fix comment to match broader behavior (any error, not just download) (Copilot)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback

- Forward AlreadyInstalled from fallback directly instead of wrapping
  in ExtensionError::Other (defensive, prevents misleading error message)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add unit tests for fallback install logic

Extract fallback_decision() and combine_install_errors() from
install_from_entry() to enable direct unit testing without requiring
a full ExtensionManager setup.

Tests cover:
- Primary success returns directly (no fallback attempted)
- AlreadyInstalled short-circuits (no fallback attempted)
- Download failure with fallback available triggers fallback
- Error without fallback source returns primary error
- Both-fail produces combined error with both messages
- AlreadyInstalled from fallback is forwarded directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-23 06:51:43 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ebb4ce95e3 chore: release v0.11.1 (#319)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 01:24:01 +00:00
Illia Polosukhin 27c9353eaa Ignore out-of-date generated CI so custom release.yml jobs are allowed 2026-02-22 16:51:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
004906e582 chore: release v0.11.0 (#318)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 00:41:14 +00:00
6f21cfa680 fix: auto-compact and retry on ContextLengthExceeded (#315)
* fix: auto-compact and retry on ContextLengthExceeded in agentic loop

When the LLM returns a context-length-exceeded error mid-turn, the
dispatcher now automatically compacts the conversation history and
retries once instead of propagating the raw error to the user.

The compaction keeps all system messages (system prompt, skill context),
the last user message, and all subsequent messages (current turn's tool
calls and results), dropping older conversation history. A note is
inserted to inform the LLM that earlier context was dropped.

If the retry also fails, the original error is returned.

Fixes nearai/ironclaw#260

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Gemini/Copilot review feedback

- Fix system message duplication: only collect system messages before the
  last User message to avoid duplicating nudges in the tail slice (Gemini + Copilot)
- Only add compaction note when earlier history is actually dropped (Copilot)
- Propagate actual retry error instead of masking with original (Copilot)
- Fix else branch to preserve system messages when no User messages exist
- Add test for nudge-after-user deduplication

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 00:29:31 +00:00
Illia PolosukhinGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
7bc3d5507a doc(README): Adding badges to readme (#316)
* Adding badges to readme

* Update README.md

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-22 22:06:05 +00:00
7f68207f1e Feat/completion (#240)
* feat: add OpenRouter usage examples

* feat: add HTPS headers

* feat: add shell completion generation via clap_complete

* feat: add shell completion generation via clap_complete

* feat: add shell completion generation via clap_complete

* Refactor completion: use clap_complete::Shell directly, improve tests, remove tracing duplication, fix .env.example and Cargo.toml

* fix: rename init_cli_logging to init_cli_tracing (sync with main)

---------

Co-authored-by: BroccoliFin <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-22 19:08:43 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
b8901baafd chore: release v0.10.0 (#279)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-22 18:16:21 +00:00
4003300a8c fix: improve Telegram status delivery and reliability (#304)
* fix: make Telegram status prompts reliable

Approval and auth prompts could be missed when polling or reply-context sends failed, leaving users stuck in waiting states. This adds explicit status mapping and retries, keeps typing active through intermediate work while suppressing noisy tool telemetry, and adds regression tests plus CI coverage for the Telegram channel crate.

* fix: normalize terminal status handling

Terminal status strings from the agent loop can vary in casing and formatting, which could leak internal status lines to Telegram. This normalizes Done/Interrupted mapping and filters terminal status text consistently to keep chat UX clean while preserving actionable prompts.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 18:07:57 +00:00
Robert YanandGitHub c68dc2ff2a feat: update dashboard favicon (#309) 2026-02-22 18:06:50 +00:00
d4785ce4d2 fix: persist user message at turn start before agentic loop (#305)
* fix: persist user message at turn start before agentic loop

Split persist_turn into persist_user_message + persist_assistant_response.
The user message is now written to DB immediately after thread.start_turn(),
before the agentic loop runs. This ensures the message survives process
crashes mid-response. The assistant response is persisted only on completion.

Updated all 6 call sites in thread_ops.rs (success, error, approval
success/error, rejection, and auth intercept paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: document persist_assistant_response dependency on persist_user_message

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: re-ensure conversation in persist_assistant_response

Add ensure_conversation call and user_id parameter to
persist_assistant_response so assistant replies are still persisted
even if persist_user_message failed transiently at turn start.

Addresses PR review feedback from @ilblackdragon.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:50:28 +00:00
2544df1c4a feat: add web UI test skill for Chrome extension (#302)
* feat: add web UI test skill for Chrome extension testing

Add a SKILL.md checklist for manually testing the IronClaw web gateway
UI using the Claude for Chrome browser extension. Covers connection,
chat, skills tab (search, install by search, install by URL, remove),
and smoke tests for other tabs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use placeholder token and correct cleanup path per review

- Replace hardcoded test123 token with <your-token> placeholder
- Fix cleanup path: ~/.ironclaw/installed_skills/ (not skills/)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:49:35 +00:00
82f24bf08f fix: block send until thread is selected (#306)
* fix: block send until thread is selected

Prevents messages from ending up in orphan threads when user sends
while currentThreadId is null during page load.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: guard enableChatInput against null thread + add user feedback

Prevents SSE events from re-enabling input before a thread is selected.
Adds status message when user tries to send without a thread.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:19:18 +00:00
510fba4c92 fix: reload chat history on SSE reconnect (#307)
When SSE auto-reconnects after a server restart, the chat now
re-syncs from the database so no messages are lost.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:18:46 +00:00
04d3b005b1 feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch

FullJob routines previously fell back to lightweight mode (single LLM call,
no tools) with a warning. This wires them to the existing Scheduler/Worker
infrastructure so they dispatch real jobs with full tool access.

Fire-and-forget model: the routine creates a job via ContextManager, schedules
it, links the routine_run to the job_id, and completes immediately. The job
runs independently with full tool access.

- Add RoutineError::JobDispatchFailed variant
- Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL)
- Add execute_full_job() in routine_engine with context_manager/scheduler
- Wire context_manager + scheduler into RoutineEngine from agent_loop
- Fix pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist job to DB before scheduling in execute_full_job

The worker emits job_actions and llm_calls rows that reference agent_jobs
via foreign key. Without persisting the job first, those inserts can fail.
Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations

Move the create + persist + schedule sequence into a single
Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs)
don't duplicate the logic. FullJob routines now pass max_iterations via job
metadata, and the worker reads it (defaulting to 50 if unset).

Also removes the context_manager field from RoutineEngine since dispatch_job
handles everything internally.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clamp max_iterations to 500 and log category update failures

Address PR review feedback:
- worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500)
  to prevent unbounded LLM token usage from malicious/buggy configs
- commands.rs: log warning on category update failure instead of silently
  discarding the error

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:18:21 +00:00
ea57447649 feat: hot-activate WASM channels, channel-first prompts, unified artifact resolution (#297)
* refactor: unify WASM artifact resolution into registry/artifacts.rs

Consolidate duplicated WASM find/build/install logic from 5+ files into
a single src/registry/artifacts.rs module. This fixes two bugs:
- registry/installer.rs now respects CARGO_TARGET_DIR (was hardcoded)
- channels/wasm/bundled.rs now searches all WASM triples (was wasip2 only)

Also includes: extension manager hot-activation for WASM channels,
extension guidance in LLM prompts, channel manager hot-add support,
webhook router channel lookup, and minor cleanups.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: send approval prompts as messages on WASM channels (Telegram, Slack)

WASM channels mapped ApprovalNeeded status to a typing indicator,
so users on Telegram never saw tool approval prompts — the agent
got stuck in AwaitingApproval and all subsequent messages failed
with "Waiting for approval".

- Intercept ApprovalNeeded in WasmChannel::handle_status_update and
  send the prompt as an actual message via call_on_respond, showing
  tool name, description, parameters, and yes/no/always instructions
- Guard against empty LLM responses after clean_response() strips
  reasoning_content think-tags (defense-in-depth for reasoning models)
- Add reasoning_content fallback to NearAiChatProvider::complete()
  for consistency with complete_with_tools()
- Add debug logging when empty responses are suppressed
- Improve error logging for channel respond() failures
- Register WASM channel webhook routes before credential checks so
  platforms don't deactivate webhook URLs with 404s

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #297 review comments

- ChannelManager::add: use async write().await instead of try_write()
- resolve_target_dir: resolve relative CARGO_TARGET_DIR against crate_dir
- install_wasm_files: log warning on capabilities copy failure
- refresh_active_channel: load capabilities file for webhook secret name
- activate_wasm_channel: validate name against path traversal
- Fix cargo fmt formatting in nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire up channel runtime for hot-activation and address PR review round 2

- Wire up set_channel_runtime() in main.rs so hot-activation actually works
  (with_channel_runtime was never called — hot-activation was dead code)
- Change ExtensionManager channel runtime fields to RwLock<Option<...>>
  interior mutability so set_channel_runtime(&self) works after Arc wrapping
- Fix artifact tests to use resolve_target_dir() instead of hardcoding
  "target/" (breaks when CARGO_TARGET_DIR is set)
- Fix bundled.rs build hint: cargo component build (not cargo build --target)
- Fix wasm_artifact_path doc: binary_name should not include .wasm extension

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use char-aware truncation to prevent UTF-8 panic in approval prompt

&s[..77] panics on multi-byte UTF-8 (CJK, emoji). Use s.chars().take(77)
for safe truncation at character boundaries.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:09:56 +00:00
a320f265b3 Fix tool schema OpenAI compatibility (#301)
* fix: remove union type arrays from tool schemas for OpenAI compatibility

OpenAI rejects JSON Schema union types containing "array" without an
"items" subschema. The http tool's "body" and json tool's "data" params
used union types to accept any value. Replace with freeform (untyped)
schemas which OpenAI treats as accepting any JSON value.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: update schema tests to assert type is absent, fix missed json.rs test

- http.rs test: assert body has no "type" (not just has description)
- json.rs test: update to match the freeform schema change (was still
  asserting type is present)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 07:52:34 +00:00
c3ce26278a refactor: simplify config resolution and consolidate main.rs init (#287)
* refactor: simplify config resolution and consolidate main.rs init into AppBuilder

- Add parse_bool_env() and parse_string_env() helpers to eliminate repetitive
  5-line optional_env/parse/map_err/unwrap_or boilerplate across 12 config files
- Add EmbeddingsConfig::create_provider() to centralize embeddings construction
  (fixes hardcoded 1536 dimensions and missing Ollama provider in app.rs)
- Extract init_cli_tracing(), setup_wasm_channels(), start_tunnel(),
  run_memory_command(), run_worker(), run_claude_bridge() from main.rs
- Replace ~600 lines of inline init in main.rs with AppBuilder::build_all()
- Expose catalog_entries from AppComponents for gateway registry entries
- Net reduction: ~738 lines across 15 files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: propagate dev_loaded_tool_names from AppBuilder and add parse_option_env helper

Address PR review feedback:

- Capture dev_loaded_tool_names from WASM loading in init_extensions()
  and expose via AppComponents so bootstrap_hooks receives the actual
  dev tool names instead of an empty slice (fixes silent hook skip)
- Add parse_option_env<T>() helper for Option<T> config fields,
  simplifying max_cost_per_day_cents and max_actions_per_hour in agent.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: fetch real NEAR AI pricing and unify cost calculation path

CostGuard was independently looking up pricing via costs::model_cost(),
falling back to GPT-4o default rates when NEAR AI model names didn't
match the static table — causing ~3x cost overestimates in logs.

- Add pricing map to NearAiChatProvider that fetches real rates from
  /v1/model/list at startup (background, non-blocking)
- Update cost_per_token() to check fetched pricing first, then static
  table, then default
- Add cost_per_token parameter to CostGuard::record_llm_call() so the
  dispatcher passes provider-sourced rates directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: update default NEAR AI model to GLM-latest

Replace fireworks llama4-maverick-instruct-basic with zai-org/GLM-latest
as the default model in config and setup wizard.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: align wizard default model name with config

Change "zai/GLM-latest" to "zai-org/GLM-latest" in wizard.rs to match
the default in config/llm.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 02:54:31 +00:00
mfcoburnandGitHub 91b602790a Update image source in README.md 2026-02-21 17:21:50 -05:00
mfcoburnandGitHub e5ce076773 Add files via upload 2026-02-21 15:14:57 -07:00
c1f3b83c98 refactor: remove ExtensionSource::Bundled, use download-only install for WASM channels (#293)
The Bundled variant and its local-artifacts fallback are superseded by the
embedded registry catalog which provides WasmDownload entries with GitHub
release URLs. The in-chat extension manager now always downloads channel
WASM binaries from releases, simplifying the install path.

The setup wizard retains its own local install_bundled_channel path for
dev builds where build artifacts exist on disk.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 13:55:49 -08:00
37c0158765 Fix: allow OAuth callback to work on remote servers (fixes #186) (#212)
* fix: allow OAuth callback to work on remote servers via OAUTH_CALLBACK_HOST

Fixes #186.

The OAuth callback URL was hardcoded to `http://127.0.0.1:9876` in two
places (NEAR AI login and MCP server auth). On a remote server this URL
is unreachable from the user's browser, making authentication impossible.

Changes:
- Add `callback_host()` to `oauth_defaults` that reads `OAUTH_CALLBACK_HOST`
  (default: `127.0.0.1`)
- Update `bind_callback_listener()` to bind to `0.0.0.0` when a non-loopback
  host is configured, so the port is reachable from outside the machine
- Update `session.rs` and `mcp/auth.rs` to use `callback_host()` instead
  of hardcoded `127.0.0.1` / `localhost`

Usage on a remote server:
  export OAUTH_CALLBACK_HOST=<your-server-ip>
  ironclaw login

* fix: address PR review comments for OAuth callback security

* fix: address serrrfirat review comments on PR #212

---------

Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 15:39:47 +04:00
0a30c95ee1 Feat: add rate limiting for built-in tools (closes #171) (#276)
* feat: add rate limiting for built-in tools (closes #171)

Extend the Tool trait with an optional rate_limit_config() method and
wire a shared sliding-window RateLimiter into the tool execution path in
worker.rs so that per-tool per-user limits are enforced at runtime.

- Add ToolRateLimitConfig struct (requests_per_minute / requests_per_hour)
  and rate_limit_config() default method to the Tool trait
- Extract shared RateLimiter from tools/wasm/ into tools/rate_limiter.rs;
  WASM rate_limiter.rs now re-exports from the shared module
- Add RateLimited error variant to crate::error::ToolError
- Register RateLimiter on ToolRegistry and check limits in execute_tool_inner
- Apply conservative configs to high-impact tools:
    ShellTool        30 rpm / 300 rph
    HttpTool         30 rpm / 500 rph
    WriteFileTool    20 rpm / 200 rph
    ApplyPatchTool   20 rpm / 200 rph
    MemoryWriteTool  20 rpm / 200 rph
    CreateJobTool     5 rpm /  30 rph

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address Gemini review comments on rate limiter

- worker.rs: collapse nested if-let into a single `if let ... && let ...`
  (clippy::collapsible_if)
- rate_limiter.rs: extract check_internal(record: bool) helper to DRY up
  check_and_record / check (were identical except for the increment step)
- rate_limiter.rs: replace magic numbers 60 / 3600 with MINUTE_SECS /
  HOUR_SECS constants

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 14:39:53 +04:00
b3bf50f10e feat: add pairing/permission system to all WASM channels and fix extension registry (#286)
Port Telegram's permission model (owner_id, dm_policy, allow_from, pairing codes)
to Discord, Slack, and WhatsApp WASM channels. Add web UI for configuration and
pairing approval. Fix extension registry issues preventing Discord install and
causing Slack activation to hit the wrong endpoint.

WASM channels:
- Discord: add DiscordConfig, permission checks, ephemeral pairing replies,
  fix capabilities.json (header_name→name), downgrade wit-bindgen to 0.36
- Slack: expand SlackConfig with permission fields, add check_sender_permission
  and send_pairing_reply via chat.postMessage
- WhatsApp: expand WhatsAppConfig with permission fields, add permission checks
  and pairing reply via Cloud API
- Telegram: reformat capabilities.json, add setup.required_secrets

Extension system:
- Add Discord to KNOWN_CHANNELS in bundled.rs and to extension registry
- Rename "slack" MCP→"slack-mcp", "slack-channel"→"slack" to fix name collision
- Add ExtensionSource::Bundled variant handling in discovery.rs
- Add get_setup_schema/save_setup_secrets to ExtensionManager
- Add needs_setup field to InstalledExtension

Web gateway:
- Add GET/POST /api/extensions/{name}/setup for configuration modal
- Add GET /api/pairing/{channel} and POST /api/pairing/{channel}/approve
- Add configure modal UI (password fields, provided badges, auto-generate hints)
- Add pairing request UI on active WASM channel cards
- Show "Restart to activate" label instead of Activate button for WASM channels

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 23:21:32 -08:00
48b5323ec9 feat: group chat privacy, channel-aware prompts, and safety hardening (#285)
Prevent personal memory (MEMORY.md) from leaking into group chat contexts
by adding system_prompt_for_context(is_group_chat) to the workspace. Add
channel-specific formatting hints (Discord, Telegram, Slack, WhatsApp),
runtime metadata injection, group chat behavioral guidance with NO_REPLY
silent token, safety rules in the system prompt, tool call style guidance,
wrap_external_content() for untrusted data, and improved workspace seed
files with richer identity/soul/agent templates and heartbeat checklist.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 06:28:33 +00:00
3124ab2b7f docs: add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) (#193)
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers
- Expand .env.example with Together AI and Fireworks AI example configs
- Add "Alternative LLM Providers" section to README with quickstart snippet

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 10:11:54 +04:00
dbd3e0807f Feat/html to markdown #106 (#115)
* feat: add HTML-to-Markdown conversion for web content

- Add readabilityrs for content extraction
- Add html-to-markdown for conversion
- Feature-gated behind html-markdown flag
- Integrates with HTTP tool response handling
- Includes comprehensive tests and examples

Closes #106

* Update comments for is_html_response helper and fix tests to not fail silently in certain instances

---------

Co-authored-by: Zach Frederick <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 06:05:16 +00:00
436066415b feat: embedded registry catalog and WASM bundle install pipeline (#283)
* feat: embedded registry catalog and WASM bundle install pipeline

Embed registry manifests at compile time so the extension catalog is
available without network access. Add tar.gz bundle support for WASM
extension downloads (tools and channels), a /api/extensions/registry
endpoint, CI job to build and publish WASM bundles on release, and
ephemeral in-memory secrets fallback so the extension manager works
even without a persistent secrets store.

Key changes:
- build.rs: collect registry/*.json into embedded_catalog.json at compile time
- src/registry/embedded.rs + catalog.rs: load embedded or on-disk catalog
- src/extensions/manager.rs: download_and_install_wasm handles tar.gz bundles,
  bare .wasm files, and separate capabilities downloads; wasm channel install
- src/channels/web/server.rs: /api/extensions/registry endpoint + no-cache headers
- src/app.rs: ephemeral InMemorySecretsStore fallback for extension manager
- registry/*.json: populate artifact download URLs for release bundles
- .github/workflows/release.yml: build-wasm-extensions CI job
- Simplified setup wizard and CLI registry commands

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — archive hardening, decompression bomb guard, test fix

- Add 100 MB decompressed entry size cap to tar.gz extraction in both
  manager.rs and installer.rs to prevent decompression bombs
- Add archive.set_preserve_permissions(false) and set_unpack_xattrs(false)
  for defense-in-depth against malicious archives
- Fix test assertion logic in catalog.rs (|| → || with correct negation)
- Replace silent tar fallback in CI with explicit if/else for capabilities
- Add warning when installing without SHA256 verification

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: resolve clippy warning in settings.rs and enforce zero-warnings policy

Use struct initializer with ..Default::default() instead of field
reassignment. Update CLAUDE.md to codify zero clippy warnings policy —
all warnings must be fixed before committing, including pre-existing ones.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review round 2 — build reliability, caps validation, naming

- build.rs: emit per-file rerun-if-changed for reliable content tracking;
  fix bundles fallback to match BundlesFile shape ({"bundles":{}})
- embedded.rs: parse catalog once via OnceLock instead of double-parsing
- manager.rs + installer.rs: add 1 MB size cap on capabilities_url downloads
  with proper error surfacing
- secrets/store.rs: rename misleading `pub mod testing` to `pub mod in_memory`
- server.rs: track installed extensions by (name, kind) tuple to avoid
  false positives across different extension kinds

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 05:43:28 +00:00
firat.sertgozGitHubIllia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
3d4c647216 fix: map Esc to interrupt and Ctrl+C to graceful quit (#267)
* fix: map Esc to interrupt and Ctrl+C to graceful quit

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-21 04:46:15 +00:00
b68d67bd35 feat: show token usage and cost tracker in gateway status popover (#284)
* feat: show token usage, cost tracker, and uptime in gateway status popover

The "Connected" hover popover in the web gateway now displays three
sections: connection info (SSE/WS counts, uptime), daily cost tracker
(spend + actions/hr), and per-model token usage (input/output counts
with cost per model). Also fixes the field name mismatch between the
backend response and JS rendering that prevented the popover from
showing correct data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — escape HTML in popover, add model_usage test

- Escape model name and cost strings with escapeHtml() before inserting
  into innerHTML to prevent XSS via crafted model names
- Add test_model_usage_per_model_tracking test covering multi-model
  token/cost accumulation in CostGuard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 03:45:04 +00:00
493e4578d0 feat: support custom HTTP headers for OpenAI-compatible provider (#269)
Add LLM_EXTRA_HEADERS env var (format: Key:Value,Key2:Value2) to inject
custom HTTP headers into every request to OpenAI-compatible endpoints.
This enables OpenRouter attribution headers (HTTP-Referer, X-Title)
and other service-specific headers without code changes.

Closes #179

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 03:06:15 +00:00
250551799b style: adopt agent-market design language for web UI (#282)
* fix: move Logs to status bar and fix chat history ordering after restart

Move the Logs tab out of the main tab bar and into the right-side status
area as a compact pill button next to "Connected". Remove it from the
Ctrl+1-N shortcut order (now Ctrl+1-5).

Fix chat message ordering in libSQL backend: datetime('now') has only
second precision, so back-to-back user+assistant inserts got identical
timestamps causing non-deterministic ORDER BY. Now passes explicit
millisecond-precision timestamps and uses rowid as tiebreaker for
existing data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: separate WASM extensions from MCP servers on Extensions page

Reorganize the Extensions tab into 5 distinct sections: Installed
Extensions, Available WASM Extensions, Install WASM Extension (by
tar.gz URL), MCP Servers (with Add Custom form), and Registered Tools.
Registry entries are now filtered client-side by kind so WASM tools/
channels and MCP servers each have dedicated UI sections.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: adopt agent-market design language for web UI

Refresh the web gateway visual identity with a cleaner, modern aesthetic:
deeper blacks, green accent palette, DM Sans + IBM Plex Mono typography,
larger border-radii, glassmorphic navigation, refined hover effects,
pill badges, and green focus rings. CSS-only change plus Google Fonts.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-02-21 02:50:51 +00:00
c038c7705b feat: add smart routing provider for cost-optimized model selection (#281)
* feat: add smart routing provider for cost-optimized model selection

Route simple tasks (greetings, status checks, short questions) to a cheap
model (e.g. Haiku) and complex tasks (code generation, analysis) to the
primary model, reducing agent costs without sacrificing quality.

Activates automatically when NEARAI_CHEAP_MODEL is set. Cascade mode
retries uncertain cheap-model responses with the primary model.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract provider chain into shared build_provider_chain()

Consolidate the duplicated LLM provider chain construction from main.rs
and app.rs into a single build_provider_chain() function in llm/mod.rs.

This fixes the inconsistency where app.rs was missing retry wrapping
that main.rs had, and ensures both paths apply identical decorators:
retry → smart routing → failover → circuit breaker → cache.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — uncertainty detection and clippy lint

- Remove false-positive short response (<20 chars) uncertainty check
  that would escalate "Yes.", "42" etc. Now only empty responses and
  explicit uncertainty phrases trigger cascade escalation.
- Add #[allow(clippy::type_complexity)] to build_provider_chain() to
  fix CI clippy -D warnings failure.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:37:45 +00:00
98ee648fcb perf: speed up startup from ~15s to ~2s (#280)
Three high-impact changes eliminate most startup latency:

1. Enable wasmtime persistent compilation cache — call
   cache_config_load_default() so compiled native code is serialized to
   disk (~/.cache/wasmtime). Subsequent startups deserialize instead of
   recompiling, dropping the WASM phase from ~13s to <1s.

2. Cache compiled Component in PreparedModule — store the compiled
   wasmtime::component::Component directly instead of raw bytes.
   Eliminates ~2.6s recompilation on every first tool/channel execution.

3. Move blocking housekeeping to background tasks — embedding backfill
   (~1.3s of failing HTTP calls) and stale job cleanup are fire-and-forget
   work that no longer blocks the critical startup path.

Also: deduplicate Workspace creation in main.rs (two identical instances
reduced to one), and replace leftover println! in session validation with
tracing calls.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:30:57 +00:00
2cdd1acb1e refactor: consolidate tool approval into single param-aware method (#274)
* refactor: consolidate tool approval into single param-aware method

Replace the two confusing approval methods (requires_approval() and
requires_approval_for()) with a single requires_approval(&self, params)
returning a 3-variant ApprovalRequirement enum (Never, UnlessAutoApproved,
Always). This enables param-aware approval decisions: HTTP calls without
auth headers now skip approval entirely, while authenticated requests
always require it. Shell tool merges its destructive-command detection
into the same method.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add credential injection to built-in HTTP tool

Wire the WASM credential injection system into the built-in HTTP tool
so credentials are auto-injected at the boundary (zero-exposure model).

- Add SharedCredentialRegistry: thread-safe, append-only registry of
  credential mappings populated by WASM tools at registration time
- Add credential_detect module with broad auth detection for headers
  (12 exact + 5 substring matches), header values (7 auth scheme
  prefixes), and URL query params (17 exact + 5 substring matches)
- HttpTool now accepts optional credential registry + secrets store,
  auto-injects matching credentials in execute(), and uses broader
  auth detection in requires_approval()
- ToolRegistry passes credential registry to HttpTool at startup and
  populates it when WASM tools register
- Remove old hardcoded AUTH_HEADER_NAMES / has_auth_headers in favor
  of the new params_contain_manual_credentials()

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #274 review comments (query param injection, lock poisoning, visibility)

- Fix injected query params not being sent on outbound HTTP requests by
  also calling .query() on the RequestBuilder alongside parsed_url mutation
- Recover from poisoned RwLock in SharedCredentialRegistry instead of
  silently ignoring failures, with tracing::warn for visibility
- Narrow inject_credential and host_matches_pattern to pub(crate) to
  avoid committing to them as stable public API

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 01:28:23 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
542268fde5 chore: release v0.9.0 (#278)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-21 00:48:24 +00:00
3b6105d5ea feat: add TEE attestation shield to web gateway UI (#275)
Show a shield indicator in the tab bar when the instance is running
inside a TEE deployment. On hover, fetches and displays the TDX
attestation report (image digest, TLS cert fingerprint, report data,
VM config) from the management API.

Co-authored-by: Cursor <[email protected]>
2026-02-21 00:25:59 +00:00
Pierre LE GUENandGitHub df8616b604 fix: add X-Accel-Buffering header to SSE endpoints (#277)
Nginx buffers responses by default, breaking SSE connections that go
through a reverse proxy. Add X-Accel-Buffering: no header to chat and
log SSE handlers to match what compose-api and chat-api already do.
2026-02-20 16:25:27 -08:00
e8dcb52fda feat: configurable tool iterations, auto-approve, and policy fix (#251)
* feat: direct agentic loop for SWE-bench benchmarks

Replace the full Agent-based runner with a purpose-built agentic loop
that directly calls the LLM with tools. The old path routed through
SafetyLayer (which blocked SWE-bench prompts), dispatcher (capped at
10 iterations), approval flow (wasted iterations), and 20+ irrelevant
builtin tools (diluted the model's focus).

New architecture:
- AgenticLoop: LLM call -> tool execution -> repeat (up to 30 iters)
- Per-task tool scoping via BenchSuite::task_tools() with working dirs
- Suite-provided system prompts via BenchSuite::system_prompt()
- No safety layer, no approval flow, no sessions/threads overhead
- Configurable max_iterations in BenchConfig and TOML

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: apply --model CLI override to LLM provider

The --model flag was updating matrix entry labels but not the actual
LLM provider, so requests were still sent using the model from .env.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: configurable tool iterations and auto-approve for benchmarks

Add max_tool_iterations and auto_approve_tools settings to AgentConfig,
replacing the hardcoded MAX_TOOL_ITERATIONS constant. Fix shell_injection
policy rule to not block markdown backtick code snippets.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address benchmarks crate audit findings

High:
- Fix truncate_output UTF-8 panic on multi-byte char boundaries
- Fix parallel results durability (write JSONL per-task, not after all)

Medium:
- Fix --sample to use random shuffle instead of first-N
- Delegate all LlmProvider methods in InstrumentedLlm
- Fix LLM-as-judge to return fail instead of misleading 0.5
- Remove unnecessary shallow clone (always gets unshallowed)
- Replace .unwrap() with .expect() in LazyLock regex init

Low:
- Remove dead code: unused error variants, trait methods, struct fields
- Remove BenchSuite::name() (redundant with id())
- Remove TaskSubmission::conversation, ConversationTurn, TurnRole
- Remove unused methods from BenchChannel, results, config
- Clean up ChannelCapture conversation tracking

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add SWE-bench dataset and Docker scoring infrastructure

Add the SWE-bench Lite dataset (300 tasks) and Docker files for
isolated test execution and scoring of SWE-bench patches.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: remove benchmarks (extracted to separate repo)

Benchmarks crate has been extracted to its own repository.
Remove the workspace member and all benchmarks/ files.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add missing AgentConfig fields in test initializer

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 00:21:13 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
1f18422b88 chore: release v0.8.0 (#249)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-20 20:45:42 +00:00
448383cfb0 refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably

- Filter out `type: "reasoning"` output items from NEAR AI Responses API
  parsing so chain-of-thought never reaches the UI (nearai.rs)
- Rewrite clean_response with regex-based tag stripping that is
  code-aware (preserves tags inside fenced blocks and inline backticks),
  supports 9+ tag names (think, thought, reasoning, reflection, etc.),
  handles <final> extraction, pipe-delimited tags, and case/whitespace
  tolerance (reasoning.rs)
- Add Reasoning::complete() helper so all non-agentic LLM call sites
  (summarize, suggest, heartbeat, compaction) get automatic response
  cleaning; thread SafetyLayer through to those callers
- Change persist_turn from fire-and-forget tokio::spawn to awaited async
  so both user and assistant messages are written before returning,
  preventing data loss on shutdown/restart
- Pass input_count through seed_response_chain so response chaining
  delta calculation is accurate after thread hydration on restart
- Make NearAiResponse.usage optional and preserve response_id in alt
  response path for chaining continuity
- Persist session token to DB during onboarding wizard so runtime
  loads it without legacy-key fallback; suppress spurious warning on
  fresh installs
- Fix dev tool double-registration when builder already registers them
- Load dotenv/ironclaw env for doctor and status subcommands
- Reduce startup log noise (demote info→debug for skills, remove
  redundant info lines)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Nudge to not loop over tools continuesly

* refactor: remove Responses API, consolidate NEAR AI to Chat Completions only

The Responses API provider (nearai.rs, 1278 lines) added significant complexity
(response chaining state machine, delta message calculation, previous_response_id
persistence) for marginal benefit. This consolidates to the Chat Completions API
only, upgrading NearAiChatProvider with dual auth (session token + API key) and
401 retry for session token renewal.

- Delete src/llm/nearai.rs (Responses API provider)
- Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models
- Remove response_id from CompletionResponse and ToolCompletionResponse
- Remove seed_response_chain/get_response_chain_id from LlmProvider trait
- Remove response chain persistence from agent (thread_ops, session)
- Remove NearAiApiMode enum and NEARAI_API_MODE config
- Clean up all wrapper providers (retry, circuit_breaker, failover, cache)
- Update documentation (CLAUDE.md, .env.example)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: runtime log level control via gateway UI and URL parameter

Add server-side log level switching using tracing_subscriber::reload::Layer
so the EnvFilter can be swapped at runtime without restarting. Expose via
GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs
toolbar, and a ?log_level=debug URL parameter for one-click activation.

Also applies cargo fmt to pre-existing files (llm/, tests/).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 20:43:32 +00:00
7df356c109 fix: persist WASM channel workspace writes across callbacks (#264)
* fix: persist WASM channel workspace writes across callbacks

WASM channel callbacks (polling, webhooks, on_start) call
workspace_write() to persist state, but the host code never committed
these writes — take_pending_writes() was never called. Additionally,
no WorkspaceReader was injected into channel capabilities, so
workspace_read() always returned None.

This caused Telegram's polling offset to reset to 0 on every tick,
making getUpdates re-deliver already-processed messages and producing
2-4 duplicate LLM responses per user message.

Add ChannelWorkspaceStore (Arc-wrapped HashMap with std::sync::RwLock)
that persists across callback invocations within a channel's lifetime.
Inject it as the WorkspaceReader and commit pending writes after every
callback execution (on_start, on_poll, on_http_request, execute_poll).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 15:52:21 +00:00
3829d81269 fix: consolidate per-module ENV_MUTEX into crate-wide test lock (#246)
Each config test module (llm.rs, embeddings.rs) defined its own
ENV_MUTEX, which doesn't prevent cross-module env races since
cargo test runs in parallel. Move to a single shared mutex in
config/helpers.rs so all unsafe set_var/remove_var calls are
serialized crate-wide.

Closes #245

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:11:10 +00:00
8a4f3b6f88 fix: remove auto-proceed fake user message injection from agent loop (#255)
The agentic loop injected fake user messages ("Please proceed and use
the available tools to complete this task.") when the LLM responded
with text instead of tool calls. This caused hallucinated conversations
during casual chat, 3x wasted LLM calls, and trust issues.

Remove the `resume_after_tool` parameter and `tools_executed` tracking
entirely. Text responses now return immediately, trusting the LLM to
decide when tools are needed (consistent with ZeroClaw and OpenClaw).

Closes #145

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:09:52 +00:00
140f29decf ci: add automated PR labeling system (#253)
* ci: add automated PR labeling system

Add two independent workflows for PR auto-labeling:
- Scope labels via actions/labeler (path glob matching)
- Size, risk, and contributor tier via custom shell script

Includes idempotent label bootstrap script (create-labels.sh).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: temporarily use pull_request trigger for testing

Switch to pull_request so workflows run from the PR branch.
Will revert to pull_request_target before merge.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use absolute path for search/issues API call

gh api requires a leading slash for REST endpoints.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use gh pr list instead of search API for contributor count

The search/issues API returns 404 with the default GITHUB_TOKEN.
gh pr list --state merged works with standard permissions.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: revert to pull_request_target for fork PR support

Restore pull_request_target trigger and base branch checkout
now that testing is complete.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 12:02:41 +04:00
5725a62c83 fix: onboarding errors reset flow and remote server auth (#185, #186) (#248)
* fix: incremental settings persistence and remote server auth (#185, #186)

Persist settings after each wizard step so failures don't lose prior
progress. Load existing settings on re-run to recover from partial
onboarding. Add manual token paste option for remote/headless servers
where browser OAuth is unreachable, and support IRONCLAW_OAUTH_CALLBACK_URL
for custom callback URLs. Color prompt output (green/red/blue prefixes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace session token paste with API key entry, address PR review

Replace option 4 in NEAR AI auth menu from session token paste to NEAR
AI Cloud API key entry (cloud.near.ai). Also address all PR review
feedback: restrict .env file permissions to 0o600, mask API key input
with secret_input, fix libsql loaded flag in try_load_existing_settings,
add ENV_MUTEX to oauth_defaults tests, and add NEARAI_API_KEY to secrets
injection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate keys in upsert_bootstrap_var

When the .env file contains duplicate keys (e.g. from manual editing),
only write the replacement once and skip subsequent duplicates.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: NEARAI_SESSION_TOKEN env var takes precedence over file-based tokens

Hosting providers inject session tokens via env var and expect them to
be used directly. Previously the env var was only picked up when no
session file existed and was treated as a legacy migration. Now the env
var always wins, without persisting to disk.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: distinguish NEAR AI Chat and NEAR AI Cloud providers

Split documentation into two clearly named modes:
- NEAR AI Chat: Responses API at private.near.ai, session token auth
- NEAR AI Cloud: Chat Completions API at cloud-api.near.ai, API key auth

Update default base URLs so each mode points to its correct endpoint.
Update .env.example, deploy/env.example, CLAUDE.md, setup spec, and
code comments across config/llm.rs, nearai.rs, nearai_chat.rs, mod.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wizard recovery ordering — load DB before persist, fresh choices win

Previously, persist_after_step() ran after Step 1 but before
try_load_existing_settings(), bulk-upserting defaults that clobbered
prior settings. Additionally, merge_from gave stale DB values
precedence over fresh Step 1 choices.

Fix: snapshot Step 1 settings, load DB, then re-apply the snapshot.
This ensures prior progress (steps 2-7) is recovered while fresh
Step 1 choices override stale DB values.

Add two tests verifying wizard recovery merge ordering.

Addresses PR review comments from Copilot on wizard.rs:150,
wizard.rs:1607, and wizard.rs:1626.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting in config/llm.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: collapse nested if per clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use print_success for API key confirmation, fix menu spacing

- Use print_success() for colored output consistency in api_key_login
- Fix box-drawing alignment: options 1-2 had an extra trailing space

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 08:02:22 +00:00
bfe393eb38 fix: parallelize tool call execution via JoinSet (#219) (#252)
* fix: parallelize tool call execution via JoinSet (#219)

When the LLM returns multiple tool_calls in a single response, they were
executed sequentially. This change makes both the worker and dispatcher
paths concurrent using tokio::task::JoinSet, so N independent tool calls
complete in ~max(latency) instead of sum(latency).

Worker path: migrate execute_tools_parallel from join_all to JoinSet and
route the respond_with_tools branch through the same parallel path.

Dispatcher path: restructure the while-idx loop into three phases —
preflight (sequential approval/hook checks), parallel execution via
JoinSet, and sequential post-flight processing (session recording,
auth detection, sanitization).

Also fixes a pre-existing infinite loop bug where hook rejection used
`continue` inside a `while idx` loop, skipping `idx += 1` and retrying
the same rejected tool forever.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — ordered results, deferred auth, dedup standalone fn

- Fix auth early return skipping unrecorded tool results: defer auth
  response until after all results in the batch are recorded in session
  history and context_messages (both dispatcher and thread_ops paths)
- Fix tool results appearing out of order: collect Phase 1 hook
  rejections indexed by original position, merge with Phase 2 execution
  results, and emit all in Phase 3 in original tool_calls order
- Deduplicate execute_chat_tool: Agent method now delegates to the
  standalone function instead of duplicating 90 lines of logic
- Fix benchmark compilation: add missing session_manager arg to Agent::new

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: rustfmt alignment for CI compatibility

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address second round of PR review comments

- Distinguish JoinError panic vs cancellation in log messages and error
  reasons across all 3 files (dispatcher, thread_ops, worker)
- Simplify deferred_auth from Option<(String, String)> to Option<String>
  since only the instructions string is used
- Add single-tool short-circuit in worker execute_tools_parallel to
  avoid JoinSet overhead for the common single-tool case

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 06:07:30 +00:00
AI-Reviewer-QSandGitHub 9906190de7 fix: prevent pipe deadlock in shell command execution (#140)
Drain stdout and stderr concurrently with child.wait() using tokio::join
to prevent deadlocks when command output exceeds the OS pipe buffer
(64KB on Linux, 16KB on macOS).

Use AsyncReadExt::take() for memory-bounded reads and
tokio::io::copy to sink for draining excess output.

Add regression test that generates 128KB of output to verify the
fix prevents deadlocks.
2026-02-20 03:07:46 +00:00
Illia PolosukhinandClaude Opus 4.6 9349a3baca fix: add missing session_manager arg to Agent::new in benchmark runner
Agent::new gained an 8th parameter (session_manager) but the benchmark
runner was not updated, breaking compilation of the bench crate.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-19 18:32:56 -08:00
3f135bdde9 fix: persist turns after approval and add agent-level tests (#250)
* fix: persist turns after approval and add agent-level tests

Port relevant changes from PR #112 that were not carried over to #237:

- Add persist_turn calls in process_approval for the response, error,
  and auth-required paths. Previously, turns completed after tool
  approval were never persisted to DB — if the process crashed after
  approval the entire turn (user message + assistant response) was lost.

- Add agent-level unit tests: StaticLlmProvider mock, make_test_agent
  helper, tests for auto-approval logic, destructive shell command
  detection, and PendingApproval backward-compatible deserialization
  (without deferred_tool_calls field).

- Remove unused _thread_state binding in process_approval.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address 14 audit findings in src/agent/

Audit of the agent module found 2 High, 7 Medium, 3 Low, and 2 Nit
severity issues. This commit fixes all of them:

High:
- Remove 4 `.expect()` calls in session.rs (entry API, match, direct
  indexing, if-let) to eliminate panic paths in production
- Add typed RoutineError enum replacing Result<_, String> across
  routine.rs, routine_engine.rs, and callers in history/store.rs and
  db/libsql/mod.rs

Medium:
- Sanitize routine names in path construction to prevent directory
  traversal (routine_engine.rs)
- Log warnings for 5 silently-swallowed errors in scheduler.rs,
  compaction.rs, and worker.rs
- Extract shared handle_auth_intercept helper to deduplicate auth
  interception in thread_ops.rs
- Add session count warning threshold in session_manager.rs
- Make FullJob stub degradation visible via warn-level log and
  prepended warning in output

Low:
- Restrict dead code visibility with #[cfg(test)] on 19 unused items
  in submission.rs, task.rs, and undo.rs
- Narrow pub to pub(crate) on self_repair.rs builder methods
- Remove TaskStatus from mod.rs re-exports (test-only type)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review comments

- Reorder persist_turn before persist_response_chain so the
  conversation row exists before the metadata UPDATE runs
- Add persist_response_chain call to handle_auth_intercept so
  auth-required paths preserve the response chain
- Harden sanitize_routine_name to use allowlist (alphanumeric,
  dash, underscore) instead of denylist replacements
- Fix stale active_thread ID in get_or_create_thread: fall back
  to create_thread() when the stored ID is missing from the map
- Persist turn on approval rejection so user messages survive
  crashes after a tool is rejected

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 02:28:15 +00:00
97a7637f30 feat: extension registry with metadata catalog and onboarding integration (#238)
* feat: add extension registry with metadata catalog, CLI, and onboarding integration

Adds a central registry that catalogs all 14 available extensions (10 tools,
4 channels) with their capabilities, auth requirements, and artifact references.
The onboarding wizard now shows installable channels from the registry and
offers tool installation as a new Step 7.

- registry/ folder with per-extension JSON manifests and bundle definitions
- src/registry/ module: manifest structs, catalog loader, installer
- `ironclaw registry list|info|install|install-defaults` CLI commands
- Setup wizard enhanced: channels from registry, new extensions step (8 steps)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): resolve workspace errors for tool crates and channels-only onboarding

Tool crates in tools-src/ and channels-src/ failed `cargo metadata` during
onboard install because Cargo resolved them as part of the root workspace.
Add `[workspace]` table to each standalone crate and extend the root
`workspace.exclude` list so they build independently.

Channels-only mode (`onboard --channels-only`) failed with "Secrets not
configured" and "No database connection" because it skipped database and
security setup. Add `reconnect_existing_db()` to establish the DB connection
and load saved settings before running channel configuration.

Also improve the tunnel "already configured" display to show full provider
details (domain, mode, command) instead of just the provider name.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(registry): address PR review feedback on installer and catalog

- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (#7)
- Add is_file() guard in catalog loader to skip directories (#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (#11, #12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): restore DB connection fields after settings reload

reconnect_postgres() and reconnect_libsql() called Settings::from_db_map()
which overwrote database_url / libsql_path / libsql_url set from env vars.
Also use get_strict() in cmd_info to surface ambiguous bare-name errors.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix clippy collapsible_if and print_literal warnings

Collapse nested if-let chains and inline string literals in format
macros to satisfy CI clippy lint checks (deny warnings).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(registry): prefer artifacts for install-defaults and improve dir lookup

- InstallDefaults now defaults to downloading pre-built artifacts
  (matching `registry install` behavior), with --build flag for source builds.
- find_registry_dir() walks up 3 ancestor levels from the exe and adds
  a CARGO_MANIFEST_DIR fallback, matching load_registry_catalog() logic.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 01:17:44 +00:00
bigguybobbyandGitHub dae26d640e feat(models): add GPT-5.3 Codex, full GPT-5.x family, Claude 4.x series, o4-mini (#197)
Fixes #184 — updates model selection, priority sort, and cost table to
match current OpenAI and Anthropic model catalogs.

OpenAI: GPT-5.3 Codex, GPT-5.2 Codex/Pro, GPT-5.1 Codex/Mini/Max,
GPT-5/Mini/Nano, GPT-4.1/Mini/Nano, o4-mini, o3/Pro
Anthropic: Claude Opus 4.6/4.5/4.1/4.0, Claude Sonnet 4.6/4.5/4.0,
Claude Haiku 4.5, Claude 3.7 Sonnet, Claude 3.5 Haiku

Also resolves stale merge-conflict markers in http.rs and json.rs.
2026-02-20 01:16:13 +00:00
fa64df05ff feat: wire memory hygiene into the heartbeat loop (#195)
* feat: wire memory hygiene into heartbeat loop (#166)

* refactor: address PR review comments for hygiene wiring

* style: fix fmt import ordering and clippy too_many_arguments warning

* fix: update heartbeat integration test to pass HygieneConfig argument

HeartbeatRunner::new() now requires a HygieneConfig as its second
argument after the hygiene wiring refactor. Pass the default config
in the integration test.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-20 01:13:25 +00:00
356f56f77c docs: update CLAUDE.md for recently merged features (#183)
* docs: update CLAUDE.md for recently merged features

Document skills system, sandbox network proxy, leak detector,
Tinfoil private inference, setup wizard, and shell env scrubbing
that were merged but not reflected in CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: fix SKILL.md format example and scoring description

Align SKILL.md frontmatter example with actual SkillManifest struct:
activation block with patterns/keywords/max_context_tokens, requires
nested under metadata.openclaw. Fix scoring pipeline description to
mention keywords, tags, and regex patterns instead of triggers/intents.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: optimize CLAUDE.md structure and reduce from 959 to 671 lines

- Update llm/ directory tree (4 -> 12 files to match actual codebase)
- Fix "NEAR AI (required)" -> "NEAR AI (when LLM_BACKEND=nearai)"
- Remove 28-item Completed changelog list (no actionable value)
- Deduplicate 3 config blocks with cross-references
- Extract Workspace deep-dive to src/workspace/README.md
- Extract Tool Architecture deep-dive to src/tools/README.md
- Consolidate Code Style and Review Discipline under Key Patterns
- Add workspace and tools to Module Specifications table

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 01:04:39 +00:00
242 changed files with 53046 additions and 10698 deletions
+2 -2
View File
@@ -286,8 +286,8 @@ impl Tool for <Name>Tool {
false // Set true if tool processes external data
}
fn requires_approval(&self) -> bool {
false // Set true if tool is destructive or contacts external services
fn requires_approval(&self, _params: &serde_json::Value) -> crate::tools::tool::ApprovalRequirement {
crate::tools::tool::ApprovalRequirement::Never // Set to UnlessAutoApproved or Always as needed
}
}
```
+53 -10
View File
@@ -2,18 +2,25 @@
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_POOL_SIZE=10
# LLM Provider (NEAR AI)
# NEAR AI provides a unified interface to all models with user authentication
# Session token is stored in ~/.ironclaw/session.json and managed automatically.
# On first run, the agent will open a browser for OAuth authentication.
NEARAI_MODEL=claude-3-5-sonnet-20241022
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
# Session token stored in ~/.ironclaw/session.json automatically.
# Base URL defaults to https://private.near.ai
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
# Base URL defaults to https://cloud-api.near.ai
NEARAI_MODEL=zai-org/GLM-5-FP8
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_API_KEY=... # API key from cloud.near.ai
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic
# === Ollama ===
# OLLAMA_MODEL=llama3.2
@@ -25,13 +32,32 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=http://localhost:1234/v1
# LLM_API_KEY=sk-... # optional for local servers
# Custom HTTP headers for OpenAI-compatible providers
# Format: comma-separated key:value pairs
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
# === OpenRouter (via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://openrouter.ai/api/v1
# LLM_API_KEY=sk-or-...
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
# === Together AI (via OpenAI-compatible) ===
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.together.xyz/v1
# LLM_API_KEY=...
# === Fireworks AI (via OpenAI-compatible) ===
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_...
# For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration
# CLI is always enabled
@@ -49,6 +75,17 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Signal Channel (optional, requires signal-cli daemon --http)
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
# SIGNAL_ACCOUNT=+1234567890
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
# SIGNAL_IGNORE_ATTACHMENTS=false
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
@@ -68,6 +105,12 @@ HEARTBEAT_INTERVAL_SECS=1800
HEARTBEAT_NOTIFY_CHANNEL=cli
HEARTBEAT_NOTIFY_USER=default
# Memory hygiene settings (automatic cleanup of stale workspace documents)
# Runs on each heartbeat tick; identity files (IDENTITY.md, SOUL.md) are never deleted
# MEMORY_HYGIENE_ENABLED=true
# MEMORY_HYGIENE_RETENTION_DAYS=30 # delete daily/ docs older than this many days
# MEMORY_HYGIENE_CADENCE_HOURS=12 # minimum hours between cleanup passes
# Safety settings
SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
+1
View File
@@ -0,0 +1 @@
tests/test-pages/**/*.html linguist-generated=true
+166
View File
@@ -0,0 +1,166 @@
# Scope labels for actions/labeler@v6
# Maps file path globs to scope labels. Multiple labels can apply per PR.
"scope: agent":
- changed-files:
- any-glob-to-any-file:
- src/agent/**
"scope: channel":
- changed-files:
- any-glob-to-any-file:
- src/channels/channel.rs
- src/channels/manager.rs
- src/channels/mod.rs
"scope: channel/cli":
- changed-files:
- any-glob-to-any-file:
- src/channels/cli/**
- src/cli/**
"scope: channel/web":
- changed-files:
- any-glob-to-any-file:
- src/channels/web/**
"scope: channel/wasm":
- changed-files:
- any-glob-to-any-file:
- src/channels/wasm/**
"scope: tool":
- changed-files:
- any-glob-to-any-file:
- src/tools/tool.rs
- src/tools/registry.rs
- src/tools/mod.rs
- src/tools/sandbox.rs
"scope: tool/builtin":
- changed-files:
- any-glob-to-any-file:
- src/tools/builtin/**
"scope: tool/wasm":
- changed-files:
- any-glob-to-any-file:
- src/tools/wasm/**
"scope: tool/mcp":
- changed-files:
- any-glob-to-any-file:
- src/tools/mcp/**
"scope: tool/builder":
- changed-files:
- any-glob-to-any-file:
- src/tools/builder/**
"scope: db":
- changed-files:
- any-glob-to-any-file:
- src/db/mod.rs
"scope: db/postgres":
- changed-files:
- any-glob-to-any-file:
- src/db/postgres.rs
- migrations/**
"scope: db/libsql":
- changed-files:
- any-glob-to-any-file:
- src/db/libsql_backend.rs
- src/db/libsql_migrations.rs
"scope: safety":
- changed-files:
- any-glob-to-any-file:
- src/safety/**
"scope: llm":
- changed-files:
- any-glob-to-any-file:
- src/llm/**
"scope: workspace":
- changed-files:
- any-glob-to-any-file:
- src/workspace/**
"scope: orchestrator":
- changed-files:
- any-glob-to-any-file:
- src/orchestrator/**
"scope: worker":
- changed-files:
- any-glob-to-any-file:
- src/worker/**
"scope: secrets":
- changed-files:
- any-glob-to-any-file:
- src/secrets/**
"scope: config":
- changed-files:
- any-glob-to-any-file:
- src/config.rs
- src/settings.rs
"scope: extensions":
- changed-files:
- any-glob-to-any-file:
- src/extensions/**
"scope: setup":
- changed-files:
- any-glob-to-any-file:
- src/setup/**
"scope: evaluation":
- changed-files:
- any-glob-to-any-file:
- src/evaluation/**
"scope: estimation":
- changed-files:
- any-glob-to-any-file:
- src/estimation/**
"scope: sandbox":
- changed-files:
- any-glob-to-any-file:
- src/sandbox/**
- Dockerfile*
"scope: hooks":
- changed-files:
- any-glob-to-any-file:
- src/hooks/**
"scope: pairing":
- changed-files:
- any-glob-to-any-file:
- src/pairing/**
"scope: ci":
- changed-files:
- any-glob-to-any-file:
- .github/workflows/**
- .github/scripts/**
"scope: docs":
- changed-files:
- any-glob-to-any-file:
- "**/*.md"
- docs/**
- LICENSE*
"scope: dependencies":
- changed-files:
- any-glob-to-any-file:
- Cargo.toml
- Cargo.lock
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Idempotent label bootstrap for IronClaw PR automation.
# Uses `gh label create --force` so it can be re-run safely.
#
# Usage: bash .github/scripts/create-labels.sh
# Requires: gh CLI authenticated with repo scope
set -euo pipefail
if ! command -v gh &>/dev/null; then
echo "Error: gh CLI is required. Install from https://cli.github.com" >&2
exit 1
fi
create() {
local name="$1" color="$2" description="$3"
gh label create "$name" --color "$color" --description "$description" --force
}
echo "==> Creating size labels..."
create "size: XS" "F9D0C4" "< 10 changed lines (excluding docs)"
create "size: S" "F5A3A3" "10-49 changed lines"
create "size: M" "E57373" "50-199 changed lines"
create "size: L" "D32F2F" "200-499 changed lines"
create "size: XL" "B71C1C" "500+ changed lines"
echo "==> Creating risk labels..."
create "risk: low" "4CAF50" "Changes to docs, tests, or low-risk modules"
create "risk: medium" "FFC107" "Business logic, config, or moderate-risk modules"
create "risk: high" "F44336" "Safety, secrets, auth, or critical infrastructure"
create "risk: manual" "9E9E9E" "Risk level set manually (sticky, not overwritten)"
echo "==> Creating scope labels..."
create "scope: agent" "006B75" "Agent core (agent loop, router, scheduler)"
create "scope: channel" "00838F" "Channel infrastructure"
create "scope: channel/cli" "00897B" "TUI / CLI channel"
create "scope: channel/web" "00796B" "Web gateway channel"
create "scope: channel/wasm" "00695C" "WASM channel runtime"
create "scope: tool" "1565C0" "Tool infrastructure"
create "scope: tool/builtin" "1976D2" "Built-in tools"
create "scope: tool/wasm" "1E88E5" "WASM tool sandbox"
create "scope: tool/mcp" "2196F3" "MCP client"
create "scope: tool/builder" "42A5F5" "Dynamic tool builder"
create "scope: db" "4A148C" "Database trait / abstraction"
create "scope: db/postgres" "6A1B9A" "PostgreSQL backend"
create "scope: db/libsql" "7B1FA2" "libSQL / Turso backend"
create "scope: safety" "880E4F" "Prompt injection defense"
create "scope: llm" "4527A0" "LLM integration"
create "scope: workspace" "283593" "Persistent memory / workspace"
create "scope: orchestrator" "0D47A1" "Container orchestrator"
create "scope: worker" "01579B" "Container worker"
create "scope: secrets" "BF360C" "Secrets management"
create "scope: config" "E65100" "Configuration"
create "scope: extensions" "33691E" "Extension management"
create "scope: setup" "827717" "Onboarding / setup"
create "scope: evaluation" "558B2F" "Success evaluation"
create "scope: estimation" "9E9D24" "Cost/time estimation"
create "scope: sandbox" "00BFA5" "Docker sandbox"
create "scope: hooks" "6D4C41" "Git/event hooks"
create "scope: pairing" "4E342E" "Pairing mode"
create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
create "contributor: experienced" "FFB74D" "6-19 merged PRs"
create "contributor: core" "FF8A65" "20+ merged PRs"
echo "Done. All labels created/updated."
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env bash
# Classify a PR by size, risk, and contributor tier.
# Called by the pr-label-classify workflow.
#
# Inputs (env vars):
# PR_NUMBER — pull request number
# REPO — owner/repo (e.g. "user/ironclaw")
#
# Requires: gh CLI, jq
set -euo pipefail
PR_NUMBER="${PR_NUMBER:?PR_NUMBER is required}"
REPO="${REPO:?REPO is required}"
# ─── helpers ────────────────────────────────────────────────────────────────
# Remove all labels in a dimension except the desired one.
# Usage: set_exclusive_label "size" "size: M"
set_exclusive_label() {
local prefix="$1" desired="$2"
# Fetch current labels on the PR
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
# Remove any existing label with the same prefix
while IFS= read -r label; do
[[ -z "$label" ]] && continue
if [[ "$label" == "${prefix}:"* && "$label" != "$desired" ]]; then
gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "$label" 2>/dev/null || true
fi
done <<< "$current"
# Add the desired label
gh pr edit "$PR_NUMBER" --repo "$REPO" --add-label "$desired"
}
# ─── size ───────────────────────────────────────────────────────────────────
classify_size() {
# Sum changed lines across non-doc files
local total
total=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '
[.[] | select(.filename | test("\\.(md|txt|rst|adoc)$") | not) | .changes]
| add // 0
')
local label
if (( total < 10 )); then label="size: XS"
elif (( total < 50 )); then label="size: S"
elif (( total < 200 )); then label="size: M"
elif (( total < 500 )); then label="size: L"
else label="size: XL"
fi
echo "Size: ${total} changed lines -> ${label}"
set_exclusive_label "size" "$label"
}
# ─── risk ───────────────────────────────────────────────────────────────────
classify_risk() {
# If "risk: manual" is present, skip — it's a sticky override
local current
current=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json labels --jq '.labels[].name')
if echo "$current" | grep -qx "risk: manual"; then
echo "Risk: skipped (manual override)"
return
fi
# Fetch changed file paths
local files
files=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files" \
--paginate --jq '.[].filename')
local risk="low"
while IFS= read -r file; do
[[ -z "$file" ]] && continue
case "$file" in
# High risk: safety, secrets, auth, crypto, setup, orchestrator auth
src/safety/*|src/secrets/*|src/llm/session.rs|src/orchestrator/auth.rs|\
src/channels/web/auth.rs|src/setup/*)
risk="high"
break # can't go higher
;;
# Medium risk: agent core, config, database, worker, tools, channels
src/agent/*|src/config.rs|src/settings.rs|src/db/*|src/worker/*|\
src/tools/*|src/channels/*|src/orchestrator/*|src/context/*|\
src/hooks/*|src/sandbox/*|src/extensions/*|Cargo.toml|\
.github/workflows/*)
# Only upgrade, never downgrade
[[ "$risk" != "high" ]] && risk="medium"
;;
# Low risk: docs, tests, estimation, evaluation, history, etc.
*)
;;
esac
done <<< "$files"
echo "Risk: ${risk}"
set_exclusive_label "risk" "risk: ${risk}"
}
# ─── contributor tier ───────────────────────────────────────────────────────
classify_contributor() {
# Get PR author
local author
author=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json author --jq '.author.login')
# Count merged PRs by this author in this repo
local count
count=$(gh pr list --repo "$REPO" --state merged --author "$author" \
--limit 100 --json number --jq 'length')
local label
if (( count == 0 )); then label="contributor: new"
elif (( count < 6 )); then label="contributor: regular"
elif (( count < 20 )); then label="contributor: experienced"
else label="contributor: core"
fi
echo "Contributor: ${author} has ${count} merged PRs -> ${label}"
set_exclusive_label "contributor" "$label"
}
# ─── main ───────────────────────────────────────────────────────────────────
echo "Classifying PR #${PR_NUMBER} in ${REPO}..."
classify_size
classify_risk
classify_contributor
echo "Done."
+26
View File
@@ -0,0 +1,26 @@
name: "PR: Classify (Size, Risk, Contributor)"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
issues: read # needed for search/issues API (contributor count)
jobs:
classify:
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.ref }}
- name: Classify PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: bash .github/scripts/pr-labeler.sh
+18
View File
@@ -0,0 +1,18 @@
name: "PR: Scope Labels"
on:
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
scope:
runs-on: ubuntu-latest
steps:
- uses: actions/labeler@v5
with:
configuration-path: .github/labeler.yml
sync-labels: false # additive only — never remove scope labels
+180 -4
View File
@@ -89,10 +89,12 @@ jobs:
# Build and packages all the platform-specific things
build-local-artifacts:
name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
# Let the initial task tell us to not run (currently very blunt)
# Wait for WASM extensions so we can patch manifests with SHA256 checksums
# before build.rs bakes them into the embedded catalog.
needs:
- plan
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
- build-wasm-extensions
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by dist in create-release.
@@ -139,6 +141,28 @@ jobs:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- name: Patch manifests with WASM checksums
if: ${{ needs.plan.outputs.publishing == 'true' }}
shell: bash
run: |
CHECKSUMS="target/distrib/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found, skipping manifest patching"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Install dependencies
run: |
${{ matrix.packages_install }}
@@ -214,14 +238,113 @@ jobs:
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Build WASM extension bundles (tar.gz with .wasm + .capabilities.json)
build-wasm-extensions:
needs:
- plan
if: ${{ needs.plan.outputs.publishing == 'true' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install Rust toolchain + wasm target
run: |
rustup target add wasm32-wasip2
cargo install cargo-component --locked || true
- uses: swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Build and package WASM extensions
shell: bash
run: |
set -euo pipefail
mkdir -p target/wasm-bundles
# Process each manifest in registry/tools/ and registry/channels/
for manifest in registry/tools/*.json registry/channels/*.json; do
[ -f "$manifest" ] || continue
name=$(jq -r '.name' "$manifest")
source_dir=$(jq -r '.source.dir' "$manifest")
caps_file=$(jq -r '.source.capabilities' "$manifest")
crate_name=$(jq -r '.source.crate_name' "$manifest")
if [ ! -d "$source_dir" ]; then
echo "::warning::Source dir '$source_dir' not found for '$name', skipping"
continue
fi
echo "=== Building $name from $source_dir ==="
# Build WASM component
cargo component build --release --manifest-path "$source_dir/Cargo.toml" || {
echo "::warning::Build failed for '$name', skipping"
continue
}
# Find the built WASM file (Cargo uses underscores in artifact names)
wasm_artifact="${crate_name//-/_}"
wasm_path=""
for target_dir in wasm32-wasip2 wasm32-wasip1 wasm32-wasi; do
candidate="$source_dir/target/$target_dir/release/${wasm_artifact}.wasm"
if [ -f "$candidate" ]; then
wasm_path="$candidate"
break
fi
done
if [ -z "$wasm_path" ]; then
echo "::warning::No WASM output found for '$name', skipping"
continue
fi
# Copy files with standardized names for the archive
cp "$wasm_path" "target/wasm-bundles/${name}.wasm"
caps_path="$source_dir/$caps_file"
if [ -f "$caps_path" ]; then
cp "$caps_path" "target/wasm-bundles/${name}.capabilities.json"
else
echo "::warning::No capabilities file at '$caps_path' for '$name'"
fi
# Create tar.gz bundle
bundle="target/wasm-bundles/${name}-wasm32-wasip2.tar.gz"
(cd target/wasm-bundles && if [ -f "${name}.capabilities.json" ]; then tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm" "${name}.capabilities.json"; else tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm"; fi)
# Compute SHA256
sha256=$(sha256sum "$bundle" | cut -d' ' -f1)
echo "$sha256 ${name}-wasm32-wasip2.tar.gz" >> target/wasm-bundles/checksums.txt
# Clean up intermediate files
rm -f "target/wasm-bundles/${name}.wasm" "target/wasm-bundles/${name}.capabilities.json"
echo " -> $bundle ($sha256)"
done
echo "=== WASM bundles built ==="
ls -la target/wasm-bundles/
- name: "Upload WASM bundles"
uses: actions/upload-artifact@v4
with:
name: artifacts-wasm-extensions
path: |
target/wasm-bundles/*.tar.gz
target/wasm-bundles/checksums.txt
# Determines if we should publish/announce
host:
needs:
- plan
- build-local-artifacts
- build-global-artifacts
# Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
- build-wasm-extensions
# Only run if we're "publishing", and only if plan, local, global, and wasm didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-22.04"
@@ -281,6 +404,59 @@ jobs:
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
# Commit patched manifest SHA256 checksums back to main so the repo
# stays in sync with the released artifacts.
update-registry-checksums:
needs:
- plan
- host
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
- name: Fetch WASM checksums
uses: actions/download-artifact@v4
with:
name: artifacts-wasm-extensions
path: target/wasm-bundles/
- name: Patch manifests with SHA256
shell: bash
run: |
CHECKSUMS="target/wasm-bundles/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add registry/
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
fi
announce:
needs:
- plan
+2
View File
@@ -19,3 +19,5 @@ jobs:
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
+115
View File
@@ -7,6 +7,121 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
- *(web)* improve WASM channel setup flow ([#380](https://github.com/nearai/ironclaw/pull/380))
- *(web)* inline tool activity cards with auto-collapsing ([#376](https://github.com/nearai/ironclaw/pull/376))
- *(web)* display logs newest-first in web gateway UI ([#369](https://github.com/nearai/ironclaw/pull/369))
- *(signal)* tool approval workflow and status updates ([#350](https://github.com/nearai/ironclaw/pull/350))
- add OpenRouter preset to setup wizard ([#270](https://github.com/nearai/ironclaw/pull/270))
- *(channels)* add native Signal channel via signal-cli HTTP daemon ([#271](https://github.com/nearai/ironclaw/pull/271))
### Fixed
- correct MCP registry URLs and remove non-existent Google endpoints ([#370](https://github.com/nearai/ironclaw/pull/370))
- resolve_thread adopts existing session threads by UUID ([#377](https://github.com/nearai/ironclaw/pull/377))
- resolve telegram/slack name collision between tool and channel registries ([#346](https://github.com/nearai/ironclaw/pull/346))
- make onboarding installs prefer release artifacts with source fallback ([#323](https://github.com/nearai/ironclaw/pull/323))
- copy missing files in Dockerfile to fix build ([#322](https://github.com/nearai/ironclaw/pull/322))
- fall back to build-from-source when extension download fails ([#312](https://github.com/nearai/ironclaw/pull/312))
### Other
- Add --version flag with clap built-in support and test ([#342](https://github.com/nearai/ironclaw/pull/342))
- Update FEATURE_PARITY.md ([#337](https://github.com/nearai/ironclaw/pull/337))
- add brew install ironclaw instructions ([#310](https://github.com/nearai/ironclaw/pull/310))
- Fix skills system: enable by default, fix registry and install ([#300](https://github.com/nearai/ironclaw/pull/300))
## [0.11.1](https://github.com/nearai/ironclaw/compare/v0.11.0...v0.11.1) - 2026-02-23
### Other
- Ignore out-of-date generated CI so custom release.yml jobs are allowed
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-23
### Fixed
- auto-compact and retry on ContextLengthExceeded ([#315](https://github.com/nearai/ironclaw/pull/315))
### Other
- *(README)* Adding badges to readme ([#316](https://github.com/nearai/ironclaw/pull/316))
- Feat/completion ([#240](https://github.com/nearai/ironclaw/pull/240))
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
- update dashboard favicon ([#309](https://github.com/nearai/ironclaw/pull/309))
- add web UI test skill for Chrome extension ([#302](https://github.com/nearai/ironclaw/pull/302))
- implement FullJob routine mode with scheduler dispatch ([#288](https://github.com/nearai/ironclaw/pull/288))
- hot-activate WASM channels, channel-first prompts, unified artifact resolution ([#297](https://github.com/nearai/ironclaw/pull/297))
- add pairing/permission system to all WASM channels and fix extension registry ([#286](https://github.com/nearai/ironclaw/pull/286))
- group chat privacy, channel-aware prompts, and safety hardening ([#285](https://github.com/nearai/ironclaw/pull/285))
- embedded registry catalog and WASM bundle install pipeline ([#283](https://github.com/nearai/ironclaw/pull/283))
- show token usage and cost tracker in gateway status popover ([#284](https://github.com/nearai/ironclaw/pull/284))
- support custom HTTP headers for OpenAI-compatible provider ([#269](https://github.com/nearai/ironclaw/pull/269))
- add smart routing provider for cost-optimized model selection ([#281](https://github.com/nearai/ironclaw/pull/281))
### Fixed
- persist user message at turn start before agentic loop ([#305](https://github.com/nearai/ironclaw/pull/305))
- block send until thread is selected ([#306](https://github.com/nearai/ironclaw/pull/306))
- reload chat history on SSE reconnect ([#307](https://github.com/nearai/ironclaw/pull/307))
- map Esc to interrupt and Ctrl+C to graceful quit ([#267](https://github.com/nearai/ironclaw/pull/267))
### Other
- Fix tool schema OpenAI compatibility ([#301](https://github.com/nearai/ironclaw/pull/301))
- simplify config resolution and consolidate main.rs init ([#287](https://github.com/nearai/ironclaw/pull/287))
- Update image source in README.md
- Add files via upload
- remove ExtensionSource::Bundled, use download-only install for WASM channels ([#293](https://github.com/nearai/ironclaw/pull/293))
- allow OAuth callback to work on remote servers (fixes #186) ([#212](https://github.com/nearai/ironclaw/pull/212))
- add rate limiting for built-in tools (closes #171) ([#276](https://github.com/nearai/ironclaw/pull/276))
- add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) ([#193](https://github.com/nearai/ironclaw/pull/193))
- Feat/html to markdown #106 ([#115](https://github.com/nearai/ironclaw/pull/115))
- adopt agent-market design language for web UI ([#282](https://github.com/nearai/ironclaw/pull/282))
- speed up startup from ~15s to ~2s ([#280](https://github.com/nearai/ironclaw/pull/280))
- consolidate tool approval into single param-aware method ([#274](https://github.com/nearai/ironclaw/pull/274))
## [0.9.0](https://github.com/nearai/ironclaw/compare/v0.8.0...v0.9.0) - 2026-02-21
### Added
- add TEE attestation shield to web gateway UI ([#275](https://github.com/nearai/ironclaw/pull/275))
- configurable tool iterations, auto-approve, and policy fix ([#251](https://github.com/nearai/ironclaw/pull/251))
### Fixed
- add X-Accel-Buffering header to SSE endpoints ([#277](https://github.com/nearai/ironclaw/pull/277))
## [0.8.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.7.0...ironclaw-v0.8.0) - 2026-02-20
### Added
- extension registry with metadata catalog and onboarding integration ([#238](https://github.com/nearai/ironclaw/pull/238))
- *(models)* add GPT-5.3 Codex, full GPT-5.x family, Claude 4.x series, o4-mini ([#197](https://github.com/nearai/ironclaw/pull/197))
- wire memory hygiene into the heartbeat loop ([#195](https://github.com/nearai/ironclaw/pull/195))
### Fixed
- persist WASM channel workspace writes across callbacks ([#264](https://github.com/nearai/ironclaw/pull/264))
- consolidate per-module ENV_MUTEX into crate-wide test lock ([#246](https://github.com/nearai/ironclaw/pull/246))
- remove auto-proceed fake user message injection from agent loop ([#255](https://github.com/nearai/ironclaw/pull/255))
- onboarding errors reset flow and remote server auth (#185, #186) ([#248](https://github.com/nearai/ironclaw/pull/248))
- parallelize tool call execution via JoinSet ([#219](https://github.com/nearai/ironclaw/pull/219)) ([#252](https://github.com/nearai/ironclaw/pull/252))
- prevent pipe deadlock in shell command execution ([#140](https://github.com/nearai/ironclaw/pull/140))
- persist turns after approval and add agent-level tests ([#250](https://github.com/nearai/ironclaw/pull/250))
### Other
- add automated PR labeling system ([#253](https://github.com/nearai/ironclaw/pull/253))
- update CLAUDE.md for recently merged features ([#183](https://github.com/nearai/ironclaw/pull/183))
## [0.7.0](https://github.com/nearai/ironclaw/compare/ironclaw-v0.6.0...ironclaw-v0.7.0) - 2026-02-19
### Added
+218 -331
View File
@@ -13,14 +13,17 @@
### Features
- **Multi-channel input**: TUI (Ratatui), HTTP webhooks, WASM channels (Telegram, Slack), web gateway
- **Parallel job execution** with state machine and self-repair for stuck jobs
- **Sandbox execution**: Docker container isolation with orchestrator/worker pattern
- **Sandbox execution**: Docker container isolation with network proxy and credential injection
- **Claude Code mode**: Delegate jobs to Claude CLI inside containers
- **Skills system**: SKILL.md prompt extensions with trust model, tool attenuation, and ClawHub registry
- **Routines**: Scheduled (cron) and reactive (event, webhook) task execution
- **Web gateway**: Browser UI with SSE/WebSocket real-time streaming
- **Extension management**: Install, auth, activate MCP/WASM extensions
- **Extensible tools**: Built-in tools, WASM sandbox, MCP client, dynamic builder
- **Persistent memory**: Workspace with hybrid search (FTS + vector via RRF)
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection
- **Prompt injection defense**: Sanitizer, validator, policy rules, leak detection, shell env scrubbing
- **Multi-provider LLM**: NEAR AI, OpenAI, Anthropic, Ollama, OpenAI-compatible, Tinfoil private inference
- **Setup wizard**: 7-step interactive onboarding for first-run configuration
- **Heartbeat system**: Proactive periodic execution with checklist
## Build & Test
@@ -29,7 +32,7 @@
# Format code
cargo fmt
# Lint (address warnings before committing)
# Lint (fix ALL warnings before committing, including pre-existing ones)
cargo clippy --all --benches --tests --examples --all-features
# Run all tests
@@ -64,6 +67,7 @@ src/
│ ├── context_monitor.rs # Memory pressure detection
│ ├── undo.rs # Turn-based undo/redo with checkpoints
│ ├── submission.rs # Submission parsing (undo, redo, compact, clear, etc.)
│ ├── dispatcher.rs # Skill-aware job dispatching
│ ├── task.rs # Sub-task execution framework
│ ├── routine.rs # Routine types (Trigger, Action, Guardrails)
│ └── routine_engine.rs # Routine execution (cron ticker, event matcher)
@@ -113,11 +117,18 @@ src/
│ ├── policy.rs # PolicyRule system with severity/actions
│ └── leak_detector.rs # Secret detection (API keys, tokens, etc.)
├── llm/ # LLM integration (NEAR AI only)
├── llm/ # LLM integration (multi-provider)
│ ├── mod.rs # Provider factory, LlmBackend enum
│ ├── provider.rs # LlmProvider trait, message types
│ ├── nearai.rs # NEAR AI chat-api implementation
│ ├── nearai_chat.rs # NEAR AI Chat Completions provider (session token + API key auth)
│ ├── reasoning.rs # Planning, tool selection, evaluation
── session.rs # Session token management with auto-renewal
── session.rs # Session token management with auto-renewal
│ ├── circuit_breaker.rs # Circuit breaker for provider failures
│ ├── retry.rs # Retry with exponential backoff
│ ├── failover.rs # Multi-provider failover chain
│ ├── response_cache.rs # LLM response caching
│ ├── costs.rs # Token cost tracking
│ └── rig_adapter.rs # Rig framework adapter
├── tools/ # Extensible tool system
│ ├── tool.rs # Tool trait, ToolOutput, ToolError
@@ -131,6 +142,7 @@ src/
│ │ ├── job.rs # CreateJob, ListJobs, JobStatus, CancelJob
│ │ ├── routine.rs # routine_create/list/update/delete/history
│ │ ├── extension_tools.rs # Extension install/auth/activate/remove
│ │ ├── skill_tools.rs # skill_list/search/install/remove tools
│ │ └── marketplace.rs, ecommerce.rs, taskrabbit.rs, restaurant.rs (stubs)
│ ├── builder/ # Dynamic tool building
│ │ ├── core.rs # BuildRequirement, SoftwareType, Language
@@ -180,11 +192,38 @@ src/
│ ├── success.rs # SuccessEvaluator trait, RuleBasedEvaluator, LlmEvaluator
│ └── metrics.rs # MetricsCollector, QualityMetrics
├── sandbox/ # Docker execution sandbox
│ ├── mod.rs # Public API, default allowlist
│ ├── config.rs # SandboxConfig, SandboxPolicy enum
│ ├── manager.rs # SandboxManager orchestration
│ ├── container.rs # ContainerRunner, Docker lifecycle
│ ├── error.rs # SandboxError types
│ └── proxy/ # Network proxy for containers
│ ├── mod.rs # NetworkProxyBuilder
│ ├── http.rs # HttpProxy, CredentialResolver trait
│ ├── policy.rs # NetworkPolicyDecider trait
│ └── allowlist.rs # DomainAllowlist validation
├── secrets/ # Secrets management
│ ├── crypto.rs # AES-256-GCM encryption
│ ├── store.rs # Secret storage
│ └── types.rs # Credential types
├── setup/ # Onboarding wizard (spec: src/setup/README.md)
│ ├── mod.rs # Entry point, check_onboard_needed()
│ ├── wizard.rs # 7-step interactive wizard
│ ├── channels.rs # Channel setup helpers
│ └── prompts.rs # Terminal prompts (select, confirm, secret)
├── skills/ # SKILL.md prompt extension system
│ ├── mod.rs # Core types (SkillTrust, LoadedSkill)
│ ├── registry.rs # SkillRegistry: discover, install, remove
│ ├── selector.rs # Deterministic scoring prefilter
│ ├── attenuation.rs # Trust-based tool ceiling
│ ├── gating.rs # Requirement checks (bins, env, config)
│ ├── parser.rs # SKILL.md frontmatter + markdown parser
│ └── catalog.rs # ClawHub registry client
└── history/ # Persistence
├── store.rs # PostgreSQL repositories
└── analytics.rs # Aggregation queries (JobStats, ToolStats)
@@ -214,6 +253,7 @@ When designing new features or systems, always prefer generic/extensible archite
- `LlmProvider` - Add new LLM backends
- `SuccessEvaluator` - Custom evaluation logic
- `EmbeddingProvider` - Add embedding backends (workspace search)
- `NetworkPolicyDecider` - Custom network access policies for sandbox containers
### Tool Implementation
```rust
@@ -252,6 +292,43 @@ Pending -> InProgress -> Completed -> Submitted -> Accepted
\-> Failed
```
### Code Style
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
### Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
**Fix the pattern, not just the instance:** When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
**Propagate architectural fixes to satellite types:** If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
**Schema translation is more than DDL:** When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
**Feature flag testing:** When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
## Configuration
Environment variables (see `.env.example`):
@@ -263,10 +340,14 @@ LIBSQL_PATH=~/.ironclaw/ironclaw.db # libSQL local path (default)
# LIBSQL_URL=libsql://xxx.turso.io # Turso cloud (optional)
# LIBSQL_AUTH_TOKEN=xxx # Required with LIBSQL_URL
# NEAR AI (required)
NEARAI_SESSION_TOKEN=sess_...
NEARAI_MODEL=claude-3-5-sonnet-20241022
# NEAR AI (when LLM_BACKEND=nearai, the default)
# Two auth modes: session token (default) or API key
# Session token auth (default): uses browser OAuth on first run
NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
NEARAI_BASE_URL=https://private.near.ai
# API key auth: set NEARAI_API_KEY, base URL defaults to cloud-api.near.ai
# NEARAI_API_KEY=... # API key from cloud.near.ai
NEARAI_MODEL=claude-3-5-sonnet-20241022
# Agent settings
AGENT_NAME=ironclaw
@@ -297,6 +378,10 @@ SANDBOX_ENABLED=true
SANDBOX_IMAGE=ironclaw-worker:latest
SANDBOX_MEMORY_LIMIT_MB=512
SANDBOX_TIMEOUT_SECS=1800
SANDBOX_CPU_LIMIT=1.0 # CPU cores per container
SANDBOX_NETWORK_PROXY=true # Enable network proxy for containers
SANDBOX_PROXY_PORT=8080 # Proxy listener port
SANDBOX_DEFAULT_POLICY=workspace_write # ReadOnly, WorkspaceWrite, FullAccess
# Claude Code mode (runs inside sandbox containers)
CLAUDE_CODE_ENABLED=false
@@ -308,16 +393,29 @@ CLAUDE_CODE_CONFIG_DIR=/home/worker/.claude
ROUTINES_ENABLED=true
ROUTINES_CRON_INTERVAL=60 # Tick interval in seconds
ROUTINES_MAX_CONCURRENT=3
# Skills system
SKILLS_ENABLED=true
SKILLS_MAX_TOKENS=4000 # Max prompt budget per turn
SKILLS_CATALOG_URL=https://clawhub.dev # ClawHub registry URL
SKILLS_AUTO_DISCOVER=true # Scan skill directories on startup
# Tinfoil private inference
TINFOIL_API_KEY=... # Required when LLM_BACKEND=tinfoil
TINFOIL_MODEL=kimi-k2-5 # Default model
```
### NEAR AI Provider
### LLM Providers
Uses the NEAR AI chat-api (`https://api.near.ai/v1/responses`) which provides:
- Unified access to multiple models (OpenAI, Anthropic, etc.)
- User authentication via session tokens
- Usage tracking and billing through NEAR AI
IronClaw supports multiple LLM backends via the `LLM_BACKEND` env var: `nearai` (default), `openai`, `anthropic`, `ollama`, `openai_compatible`, and `tinfoil`.
Session tokens have the format `sess_xxx` (37 characters). They are authenticated against the NEAR AI auth service.
**NEAR AI** -- Uses the Chat Completions API with dual auth support. Session token auth (default): authenticates with session tokens (`sess_xxx`) obtained via browser OAuth (GitHub/Google), base URL defaults to `https://private.near.ai`. API key auth: set `NEARAI_API_KEY` (from `cloud.near.ai`), base URL defaults to `https://cloud-api.near.ai`. Both modes use the same Chat Completions endpoint. Tool messages are flattened to plain text for compatibility. Set `NEARAI_SESSION_TOKEN` env var for hosting providers that inject tokens via environment.
**NEAR AI Cloud** -- Uses the OpenAI-compatible Chat Completions API (`https://cloud-api.near.ai/v1/chat/completions`). Authenticates with API keys from `cloud.near.ai`. Auto-selected when `NEARAI_API_KEY` is set (or explicitly via `NEARAI_API_MODE=chat_completions`). Tool messages are flattened to plain text for compatibility. Configure with `NEARAI_API_KEY` and `NEARAI_BASE_URL` (default: `https://cloud-api.near.ai`).
**OpenAI-compatible** -- Any endpoint that speaks the OpenAI API (vLLM, LiteLLM, OpenRouter, etc.). Configure with `LLM_BASE_URL`, `LLM_API_KEY` (optional), `LLM_MODEL`. Set `LLM_EXTRA_HEADERS` to inject custom HTTP headers into every request (format: `Key:Value,Key2:Value2`), useful for OpenRouter attribution headers like `HTTP-Referer` and `X-Title`.
**Tinfoil** -- Private inference via `https://inference.tinfoil.sh/v1`. Runs models inside hardware-attested TEEs so neither Tinfoil nor the cloud provider can see prompts or responses. Uses the OpenAI-compatible Chat Completions API. Configure with `TINFOIL_API_KEY` and `TINFOIL_MODEL` (default: `kimi-k2-5`).
## Database
@@ -386,22 +484,7 @@ Both backends implement this trait. PostgreSQL delegates to the existing `Store`
- `tool_failures` - Self-repair tracking
- `secrets`, `wasm_tools`, `tool_capabilities` - Extension infrastructure
### Configuration
```bash
# Backend selection (default: postgres)
DATABASE_BACKEND=libsql
# PostgreSQL
DATABASE_URL=postgres://user:pass@localhost/ironclaw
# libSQL (embedded)
LIBSQL_PATH=~/.ironclaw/ironclaw.db # Default path
# libSQL (Turso cloud sync)
LIBSQL_URL=libsql://your-db.turso.io
LIBSQL_AUTH_TOKEN=your-token # Required when LIBSQL_URL is set
```
Database configuration: see Configuration section above.
### Current Limitations (libSQL backend)
@@ -419,6 +502,7 @@ All external tool output passes through `SafetyLayer`:
1. **Sanitizer** - Detects injection patterns, escapes dangerous content
2. **Validator** - Checks length, encoding, forbidden patterns
3. **Policy** - Rules with severity (Critical/High/Medium/Low) and actions (Block/Warn/Review/Sanitize)
4. **Leak Detector** - Scans for 15+ secret patterns (API keys, tokens, private keys, connection strings) at two points: tool output before it reaches the LLM, and LLM responses before they reach the user. Actions per pattern: Block (reject entirely), Redact (mask the secret), or Warn (flag but allow)
Tool outputs are wrapped before reaching LLM:
```xml
@@ -427,6 +511,99 @@ Tool outputs are wrapped before reaching LLM:
</tool_output>
```
### Shell Environment Scrubbing
The shell tool (`src/tools/builtin/shell.rs`) scrubs sensitive environment variables before executing commands, preventing secrets from leaking through `env`, `printenv`, or `$VAR` expansion. The sanitizer (`src/safety/sanitizer.rs`) also detects command injection patterns (chained commands, subshells, path traversal) and blocks or escapes them based on policy rules.
## Skills System
Skills are SKILL.md files that extend the agent's prompt with domain-specific instructions. Each skill is a YAML frontmatter block (metadata, activation criteria, required tools) followed by a markdown body that gets injected into the LLM context when the skill activates.
### Trust Model
| Trust Level | Source | Tool Access |
|-------------|--------|-------------|
| **Trusted** | User-placed in `~/.ironclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry | Read-only tools only (no shell, file write, HTTP) |
### SKILL.md Format
```yaml
---
name: my-skill
version: 0.1.0
description: Does something useful
activation:
patterns:
- "deploy to.*production"
keywords:
- "deployment"
max_context_tokens: 2000
metadata:
openclaw:
requires:
bins: [docker, kubectl]
env: [KUBECONFIG]
---
# Deployment Skill
Instructions for the agent when this skill activates...
```
### Selection Pipeline
1. **Gating** -- Check binary/env/config requirements; skip skills whose prerequisites are missing
2. **Scoring** -- Deterministic scoring against message content using keywords, tags, and regex patterns
3. **Budget** -- Select top-scoring skills that fit within `SKILLS_MAX_TOKENS` prompt budget
4. **Attenuation** -- Apply trust-based tool ceiling; installed skills lose access to dangerous tools
### Skill Tools
Four built-in tools for managing skills at runtime:
- **`skill_list`** -- List all discovered skills with trust level and status
- **`skill_search`** -- Search ClawHub registry for available skills
- **`skill_install`** -- Download and install a skill from ClawHub
- **`skill_remove`** -- Remove an installed skill
### Skill Directories
- `~/.ironclaw/skills/` -- User's global skills (trusted)
- `<workspace>/skills/` -- Per-workspace skills (trusted)
- `~/.ironclaw/installed_skills/` -- Registry-installed skills (installed trust)
### Testing Skills
- `skills/web-ui-test/` -- Manual test checklist for the web gateway UI via Claude for Chrome extension. Covers connection, chat, skills search/install/remove, and other tabs.
Skills configuration: see Configuration section above.
## Docker Sandbox
The `src/sandbox/` module provides Docker-based isolation for job execution with a network proxy that controls outbound access and injects credentials.
### Sandbox Policies
| Policy | Filesystem | Network | Use Case |
|--------|-----------|---------|----------|
| **ReadOnly** | Read-only workspace mount | Allowlisted domains only | Analysis, code review |
| **WorkspaceWrite** | Read-write workspace mount | Allowlisted domains only | Code generation, file edits |
| **FullAccess** | Full filesystem | Unrestricted | Trusted admin tasks |
### Network Proxy
Containers route all HTTP/HTTPS traffic through a host-side proxy (`src/sandbox/proxy/`):
- **Domain allowlist** -- Only allowlisted domains are reachable (default: package registries, docs sites, GitHub, common APIs)
- **Credential injection** -- The `CredentialResolver` trait injects auth headers into proxied requests so secrets never enter the container environment
- **CONNECT tunnel** -- HTTPS traffic uses CONNECT method; the proxy validates the target domain against the allowlist before establishing the tunnel
- **Policy decisions** -- The `NetworkPolicyDecider` trait allows custom logic for allow/deny/inject decisions per request
### Zero-Exposure Credential Model
Secrets (API keys, tokens) are stored encrypted on the host and injected into HTTP requests by the proxy at transit time. Container processes never have access to raw credential values, preventing exfiltration even if container code is compromised.
Sandbox configuration: see Configuration section above.
## Testing
Tests are in `mod tests {}` blocks at the bottom of each file. Run specific module tests:
@@ -451,164 +628,13 @@ Key test patterns:
7. **Webhook trigger endpoint** - Routines webhook trigger not yet exposed in web gateway
8. **Full channel status view** - Gateway status widget exists, but no per-channel connection dashboard
### Completed
## Tool Architecture
-**Workspace integration** - Memory tools registered, workspace passed to Agent and heartbeat
-**WASM sandboxing** - Full implementation in `tools/wasm/` with fuel metering, memory limits, capabilities
-**Dynamic tool building** - `tools/builder/` has LlmSoftwareBuilder with iterative build loop
-**HTTP webhook security** - Secret validation implemented, proper error handling (no panics)
-**Embeddings integration** - OpenAI and NEAR AI providers wired to workspace for semantic search
-**Workspace system prompt** - Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) injected into LLM context
-**Heartbeat notifications** - Route through channel manager (broadcast API) instead of logging-only
-**Auto-context compaction** - Triggers automatically when context exceeds threshold
-**Embedding backfill** - Runs on startup when embeddings provider is enabled
-**Clippy clean** - All warnings addressed via config struct refactoring
-**Tool approval enforcement** - Tools with `requires_approval()` (shell, http, file write/patch, build_software) now gate execution, track auto-approved tools per session
-**Tool definition refresh** - Tool definitions refreshed each iteration so newly built tools become visible in same session
-**Worker tool call handling** - Uses `respond_with_tools()` to properly execute tool calls when `select_tools()` returns empty
-**Gateway control plane** - Web gateway with 40+ API endpoints, SSE/WebSocket
-**Web Control UI** - Browser-based dashboard with chat, memory, jobs, logs, extensions, routines
-**Slack/Telegram channels** - Implemented as WASM tools
-**Docker sandbox** - Orchestrator/worker containers with per-job auth
-**Claude Code mode** - Delegate jobs to Claude CLI inside containers
-**Routines system** - Cron, event, webhook, and manual triggers with guardrails
-**Extension management** - Install, auth, activate MCP/WASM extensions via CLI and web UI
-**libSQL/Turso backend** - Database trait abstraction (`src/db/`), feature-gated dual backend support (postgres/libsql), embedded SQLite for zero-dependency local mode
**Keep tool-specific logic out of the main agent codebase.** The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through `capabilities.json` files (API endpoints, credentials, rate limits, auth setup). Service-specific auth flows, CLI commands, and configuration do not belong in the main agent.
## Adding a New Tool
Tools can be built as **WASM** (sandboxed, credential-injected, single binary) or **MCP servers** (ecosystem of pre-built servers, any language, but no sandbox). Both are first-class via `ironclaw tool install`. Auth is declared in capabilities files with OAuth and manual token entry support.
### Built-in Tools (Rust)
1. Create `src/tools/builtin/my_tool.rs`
2. Implement the `Tool` trait
3. Add `mod my_tool;` and `pub use` in `src/tools/builtin/mod.rs`
4. Register in `ToolRegistry::register_builtin_tools()` in `registry.rs`
5. Add tests
### WASM Tools (Recommended)
WASM tools are the preferred way to add new capabilities. They run in a sandboxed environment with explicit capabilities.
1. Create a new crate in `tools-src/<name>/`
2. Implement the WIT interface (`wit/tool.wit`)
3. Create `<name>.capabilities.json` declaring required permissions
4. Build with `cargo build --target wasm32-wasip2 --release`
5. Install with `ironclaw tool install path/to/tool.wasm`
See `tools-src/` for examples.
## Tool Architecture Principles
**CRITICAL: Keep tool-specific logic out of the main agent codebase.**
The main agent provides generic infrastructure; tools are self-contained units that declare their requirements through capabilities files.
### What Goes in Tools (capabilities.json)
- API endpoints the tool needs (HTTP allowlist)
- Credentials required (secret names, injection locations)
- Rate limits and timeouts
- Auth setup instructions (see below)
- Workspace paths the tool can read
### What Does NOT Go in Main Agent
- Service-specific auth flows (OAuth for Notion, Slack, etc.)
- Service-specific CLI commands (`auth notion`, `auth slack`)
- Service-specific configuration handling
- Hardcoded API URLs or token formats
### Tool Authentication
Tools declare their auth requirements in `<tool>.capabilities.json` under the `auth` section. Two methods are supported:
#### OAuth (Browser-based login)
For services that support OAuth, users just click through browser login:
```json
{
"auth": {
"secret_name": "notion_api_token",
"display_name": "Notion",
"oauth": {
"authorization_url": "https://api.notion.com/v1/oauth/authorize",
"token_url": "https://api.notion.com/v1/oauth/token",
"client_id_env": "NOTION_OAUTH_CLIENT_ID",
"client_secret_env": "NOTION_OAUTH_CLIENT_SECRET",
"scopes": [],
"use_pkce": false,
"extra_params": { "owner": "user" }
},
"env_var": "NOTION_TOKEN"
}
}
```
To enable OAuth for a tool:
1. Register a public OAuth app with the service (e.g., notion.so/my-integrations)
2. Configure redirect URIs: `http://localhost:9876/callback` through `http://localhost:9886/callback`
3. Set environment variables for client_id and client_secret
#### Manual Token Entry (Fallback)
For services without OAuth or when OAuth isn't configured:
```json
{
"auth": {
"secret_name": "openai_api_key",
"display_name": "OpenAI",
"instructions": "Get your API key from platform.openai.com/api-keys",
"setup_url": "https://platform.openai.com/api-keys",
"token_hint": "Starts with 'sk-'",
"env_var": "OPENAI_API_KEY"
}
}
```
#### Auth Flow Priority
When running `ironclaw tool auth <tool>`:
1. Check `env_var` - if set in environment, use it directly
2. Check `oauth` - if configured, open browser for OAuth flow
3. Fall back to `instructions` + manual token entry
The agent reads auth config from the tool's capabilities file and provides the appropriate flow. No service-specific code in the main agent.
### WASM Tools vs MCP Servers: When to Use Which
Both are first-class in the extension system (`ironclaw tool install` handles both), but they have different strengths.
**WASM Tools (IronClaw native)**
- Sandboxed: fuel metering, memory limits, no access except what's allowlisted
- Credentials injected by host runtime, tool code never sees the actual token
- Output scanned for secret leakage before returning to the LLM
- Auth (OAuth/manual) declared in `capabilities.json`, agent handles the flow
- Single binary, no process management, works offline
- Cost: must build yourself in Rust, no ecosystem, synchronous only
**MCP Servers (Model Context Protocol)**
- Growing ecosystem of pre-built servers (GitHub, Notion, Postgres, etc.)
- Any language (TypeScript/Python most common)
- Can do websockets, streaming, background polling
- Cost: external process with full system access (no sandbox), manages own credentials, IronClaw can't prevent leaks
**Decision guide:**
| Scenario | Use |
|----------|-----|
| Good MCP server already exists | **MCP** |
| Handles sensitive credentials (email send, banking) | **WASM** |
| Quick prototype or one-off integration | **MCP** |
| Core capability you'll maintain long-term | **WASM** |
| Needs background connections (websockets, polling) | **MCP** |
| Multiple tools share one OAuth token (e.g., Google suite) | **WASM** |
The LLM-facing interface is identical for both (tool name, schema, execute), so swapping between them is transparent to the agent.
See `src/tools/README.md` for full tool architecture, adding new tools (built-in Rust and WASM), auth JSON examples, and WASM vs MCP decision guide.
## Adding a New Channel
@@ -645,154 +671,15 @@ for that module's behavior. When modifying code in a module that has a spec:
| Module | Spec File |
|--------|-----------|
| `src/setup/` | `src/setup/README.md` |
## Code Style
- Use `crate::` imports, not `super::`
- No `pub use` re-exports unless exposing to downstream consumers
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
## Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
### Fix the pattern, not just the instance
When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
### Propagate architectural fixes to satellite types
If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
### Schema translation is more than DDL
When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
### Feature flag testing
When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
### Mechanical verification before committing
Run these checks on changed files before committing:
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
| `src/workspace/` | `src/workspace/README.md` |
| `src/tools/` | `src/tools/README.md` |
## Workspace & Memory System
Inspired by [OpenClaw](https://github.com/openclaw/openclaw), the workspace provides persistent memory for agents with a flexible filesystem-like structure.
OpenClaw-inspired persistent memory with a flexible filesystem-like structure. Principle: "Memory is database, not RAM" -- if you want to remember something, write it explicitly. Uses hybrid search combining FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion.
### Key Principles
Four memory tools for LLM use: `memory_search` (hybrid search -- call before answering questions about prior work), `memory_write`, `memory_read`, `memory_tree`. Identity files (AGENTS.md, SOUL.md, USER.md, IDENTITY.md) are injected into the LLM system prompt.
1. **"Memory is database, not RAM"** - If you want to remember something, write it explicitly
2. **Flexible structure** - Create any directory/file hierarchy you need
3. **Self-documenting** - Use README.md files to describe directory structure
4. **Hybrid search** - Combines FTS (keyword) + vector (semantic) via Reciprocal Rank Fusion
The heartbeat system runs proactive periodic execution (default: 30 minutes), reading `HEARTBEAT.md` and notifying via channel if findings are detected.
### Filesystem Structure
```
workspace/
├── README.md <- Root runbook/index
├── MEMORY.md <- Long-term curated memory
├── HEARTBEAT.md <- Periodic checklist
├── IDENTITY.md <- Agent name, nature, vibe
├── SOUL.md <- Core values
├── AGENTS.md <- Behavior instructions
├── USER.md <- User context
├── context/ <- Identity-related docs
│ ├── vision.md
│ └── priorities.md
├── daily/ <- Daily logs
│ ├── 2024-01-15.md
│ └── 2024-01-16.md
├── projects/ <- Arbitrary structure
│ └── alpha/
│ ├── README.md
│ └── notes.md
└── ...
```
### Using the Workspace
```rust
use crate::workspace::{Workspace, OpenAiEmbeddings, paths};
// Create workspace for a user
let workspace = Workspace::new("user_123", pool)
.with_embeddings(Arc::new(OpenAiEmbeddings::new(api_key)));
// Read/write any path
let doc = workspace.read("projects/alpha/notes.md").await?;
workspace.write("context/priorities.md", "# Priorities\n\n1. Feature X").await?;
workspace.append("daily/2024-01-15.md", "Completed task X").await?;
// Convenience methods for well-known files
workspace.append_memory("User prefers dark mode").await?;
workspace.append_daily_log("Session note").await?;
// List directory contents
let entries = workspace.list("projects/").await?;
// Search (hybrid FTS + vector)
let results = workspace.search("dark mode preference", 5).await?;
// Get system prompt from identity files
let prompt = workspace.system_prompt().await?;
```
### Memory Tools
Four tools for LLM use:
- **`memory_search`** - Hybrid search, MUST be called before answering questions about prior work
- **`memory_write`** - Write to any path (memory, daily_log, or custom paths)
- **`memory_read`** - Read any file by path
- **`memory_tree`** - View workspace structure as a tree (depth parameter, default 1)
### Hybrid Search (RRF)
Combines full-text search and vector similarity using Reciprocal Rank Fusion:
```
score(d) = Σ 1/(k + rank(d)) for each method where d appears
```
Default k=60. Results from both methods are combined, with documents appearing in both getting boosted scores.
**Backend differences:**
- **PostgreSQL:** `ts_rank_cd` for FTS, pgvector cosine distance for vectors, full RRF
- **libSQL:** FTS5 for keyword search only (vector search via `libsql_vector_idx` not yet wired)
### Heartbeat System
Proactive periodic execution (default: 30 minutes):
1. Reads `HEARTBEAT.md` checklist
2. Runs agent turn with checklist prompt
3. If findings, notifies via channel
4. If nothing, agent replies "HEARTBEAT_OK" (no notification)
```rust
use crate::agent::{HeartbeatConfig, spawn_heartbeat};
let config = HeartbeatConfig::default()
.with_interval(Duration::from_secs(60 * 30))
.with_notify("user_123", "telegram");
spawn_heartbeat(config, workspace, llm, response_tx);
```
### Chunking Strategy
Documents are chunked for search indexing:
- Default: 800 words per chunk (roughly 800 tokens for English)
- 15% overlap between chunks for context preservation
- Minimum chunk size: 50 words (tiny trailing chunks merge with previous)
See `src/workspace/README.md` for full API documentation, filesystem structure, hybrid search details, chunking strategy, and heartbeat system.
Generated
+848 -234
View File
File diff suppressed because it is too large Load Diff
+32 -5
View File
@@ -1,15 +1,25 @@
[workspace]
members = [".", "benchmarks"]
members = ["."]
exclude = [
"channels-src/discord",
"channels-src/telegram",
"channels-src/slack",
"channels-src/whatsapp",
"tools-src/github",
"tools-src/gmail",
"tools-src/google-calendar",
"tools-src/google-docs",
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/okta",
"tools-src/slack",
"tools-src/telegram",
]
[package]
name = "ironclaw"
version = "0.7.0"
version = "0.12.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -59,7 +69,7 @@ dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -72,7 +82,7 @@ clap = { version = "4", features = ["derive", "env"] }
# Terminal
crossterm = "0.28"
rustyline = { version = "17", features = ["derive", "with-file-history"] }
rustyline = { version = "17", features = ["custom-bindings", "derive", "with-file-history"] }
termimad = "0.34"
# Channel integrations
@@ -127,6 +137,10 @@ rig-core = "0.30"
# Docker sandbox
bollard = "0.18"
# Archive extraction for WASM extension bundles
flate2 = "1"
tar = "0.4"
# HTTP proxy for sandboxed network access
hyper = { version = "1.5", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "tokio", "http1", "http2"] }
@@ -134,6 +148,12 @@ http-body-util = "0.1"
bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
@@ -152,7 +172,7 @@ pretty_assertions = "1"
tempfile = "3"
[features]
default = ["postgres", "libsql"]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
@@ -163,6 +183,11 @@ postgres = [
]
libsql = ["dep:libsql"]
integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
[[test]]
name = "html_to_markdown"
required-features = ["html-to-markdown"]
# The profile that 'cargo dist' will build with
[profile.dist]
@@ -173,6 +198,8 @@ lto = "thin"
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
+8 -2
View File
@@ -11,16 +11,22 @@ FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source and build artifacts
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --bin ironclaw
+3 -4
View File
@@ -68,7 +68,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | | P2 | signal-cli |
| Signal | ✅ | | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
@@ -120,7 +120,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | 🚧 | Caption support for media; no size limits |
| Typing indicators | ✅ | 🚧 | TUI shows status |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
@@ -158,7 +158,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `doctor` | ✅ | ❌ | P2 | Diagnostics |
| `logs` | ✅ | ❌ | P3 | Query logs |
| `update` | ✅ | ❌ | P3 | Self-update |
| `completion` | ✅ | | P3 | Shell completion |
| `completion` | ✅ | | - | Shell completion |
| `/subagents spawn` | ✅ | ❌ | P3 | Spawn subagents from chat |
| `/export-session` | ✅ | ❌ | P3 | Export current session transcript |
@@ -540,7 +540,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Signal channel
- ❌ Matrix channel
- ❌ Other messaging platforms
- ❌ TTS/audio features
+33 -1
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="ironclaw.png" alt="IronClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
@@ -8,6 +8,12 @@
<strong>Your secure personal AI assistant, always on your side</strong>
</p>
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
@@ -99,6 +105,15 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
```
</details>
<details>
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>Compile the source code (Cargo on Windows, Linux, macOS)</summary>
@@ -143,6 +158,23 @@ and secrets encryption (using your system keychain). Settings are persisted in t
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.ironclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
IronClaw defaults to NEAR AI but works with any OpenAI-compatible endpoint.
Popular options include **OpenRouter** (300+ models), **Together AI**, **Fireworks AI**,
**Ollama** (local), and self-hosted servers like **vLLM** or **LiteLLM**.
Select *"OpenAI-compatible"* in the wizard, or set environment variables directly:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
## Security
IronClaw implements defense in depth to protect your data and prevent misuse.
-50
View File
@@ -1,50 +0,0 @@
[package]
name = "ironclaw-bench"
version = "0.1.0"
edition = "2024"
rust-version = "1.85"
description = "Benchmarking harness for IronClaw agent"
license = "MIT OR Apache-2.0"
publish = false
[[bin]]
name = "ironclaw-bench"
path = "src/main.rs"
[dependencies]
ironclaw = { path = ".." }
# Async runtime
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
toml = "0.8"
# CLI
clap = { version = "4", features = ["derive"] }
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str"] }
# Error handling
thiserror = "2"
anyhow = "1"
# Async traits
async-trait = "0.1"
# Logging
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# Scoring
regex = "1"
[dev-dependencies]
tempfile = "3"
@@ -1,15 +0,0 @@
{
"run_id": "8a48de1f-09ce-4c35-9ad1-dad98fb83a1a",
"suite_id": "spot",
"config_label": "default",
"model": "openai/gpt-5.2",
"commit_hash": "2c43b83",
"pass_rate": 1.0,
"avg_score": 1.0,
"total_tasks": 21,
"completed_tasks": 21,
"total_cost_usd": 0.307053,
"total_wall_time_ms": 111009,
"started_at": "2026-02-17T22:02:08.206112Z",
"finished_at": "2026-02-17T22:03:59.270325Z"
}
@@ -1,21 +0,0 @@
{"task_id":"smoke-greeting","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2037,"llm_calls":1,"input_tokens":1359,"output_tokens":65,"estimated_cost_usd":0.005052,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Im NEAR AI Agent—an autonomous assistant that helps you plan, write, debug, and execute tasks using available tools (like reading/writing files, running shell commands, and making HTTP requests). Tell me what youre working on and what outcome you want, and Ill drive it to completion.","started_at":"2026-02-17T22:02:08.206165Z","finished_at":"2026-02-17T22:02:10.243552Z","config_label":"default","error":null}
{"task_id":"smoke-math","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2038,"llm_calls":1,"input_tokens":1366,"output_tokens":5,"estimated_cost_usd":0.004173,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"1081","started_at":"2026-02-17T22:02:10.243933Z","finished_at":"2026-02-17T22:02:12.282133Z","config_label":"default","error":null}
{"task_id":"tool-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2646,"llm_calls":2,"input_tokens":2786,"output_tokens":24,"estimated_cost_usd":0.008718,"tool_calls":[{"name":"echo","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Spot check passed","started_at":"2026-02-17T22:02:12.282294Z","finished_at":"2026-02-17T22:02:14.928540Z","config_label":"default","error":null}
{"task_id":"tool-time","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3460,"llm_calls":2,"input_tokens":2839,"output_tokens":78,"estimated_cost_usd":0.009687,"tool_calls":[{"name":"time","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Current date/time (UTC): **2026-02-17 22:02:16** \nISO 8601: **2026-02-17T22:02:16.045933+00:00**","started_at":"2026-02-17T22:02:14.928810Z","finished_at":"2026-02-17T22:02:18.389188Z","config_label":"default","error":null}
{"task_id":"tool-json-query","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2751,"llm_calls":2,"input_tokens":2839,"output_tokens":43,"estimated_cost_usd":0.009162,"tool_calls":[{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Bob","started_at":"2026-02-17T22:02:18.389812Z","finished_at":"2026-02-17T22:02:21.141146Z","config_label":"default","error":null}
{"task_id":"tool-shell-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3460,"llm_calls":2,"input_tokens":2811,"output_tokens":25,"estimated_cost_usd":0.008808,"tool_calls":[{"name":"shell","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"benchmark test","started_at":"2026-02-17T22:02:21.141384Z","finished_at":"2026-02-17T22:02:24.601812Z","config_label":"default","error":null}
{"task_id":"tool-list-dir","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":19146,"llm_calls":2,"input_tokens":4370,"output_tokens":1337,"estimated_cost_usd":0.033165,"tool_calls":[{"name":"list_dir","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"```\nclaude-501/\nclaude-505/\nclaude/\ncodex/\ncom.apple.launchd.HqYyNM7hoK/\ncom.apple.launchd.Z0bTtlsPEN/\ncom.apple.launchd.oY8y5gkzN4/\ncom.apple.launchd.sdo3WGu0S7/\ndata-gym-cache/\ndocker-desktop-privileged2203342862/\ngrammers-test/\nnear-agent-target/\nnear-market-bot-push/\nnear-market-bot/\nopenclaw/\npowerlog/\ntest_grpo/\ntest_grpo2/\ntest_grpo_entry/\ntest_grpo_mix/\ntest_grpo_singleton/\ntmpybtjiuc0/\n.d8bd37babaf2f1f7-00000000.node (5.9MB)\n.d8bd37fbfbb0f7ff-00000000.node (5.9MB)\n.d8bd3d9bdf6dd3f7-00000000.node (5.9MB)\n.d8bd3fbfbf21ddf7-00000000.node (5.9MB)\n.d8bd7dfebb38d5ff-00000000.node (5.9MB)\n.d8bdf5deba3bf1f7-00000000.node (5.9MB)\n.d8bdf7cfff76f3f7-00000000.node (5.9MB)\n.d8bdfd8ffe26d5ff-00000000.node (5.9MB)\n.d8bdfdeb9ba8dbff-00000000.node (5.9MB)\n.d8bdfffe9ab2ddff-00000000.node (5.9MB)\n.s.PGSQL.5432 (0B)\n.s.PGSQL.5432.lock (56B)\n__KMP_REGISTERED_LIB_75079 (1.0KB)\nagent_loop_new.rs (28.5KB)\nagent_mod.rs (1.6KB)\nauth_trace.md (10.8KB)\nbench-daily.md (72B)\nbench-log.md (109B)\nbench-meeting.md (160B)\nbench-monday.md (44B)\nbench-prefs.md (61B)\nbench-project.md (213B)\nbench-reminder.md (68B)\nbench-todo.md (153B)\nbench-tuesday.md (40B)\ncac-deck.html (151.5KB)\ncircuit_breaker.rs (22.1KB)\ncli_config.rs (9.1KB)\ncli_service.rs (1.1KB)\ncommands.rs (17.5KB)\nconfig.rs (58.4KB)\nconflicts_summary.md (21.4KB)\ncost_guard.rs (11.3KB)\ndebug_forc2.py (2.8KB)\ndebug_forc3.py (3.0KB)\ndebug_forc4.py (3.0KB)\ndebug_forc5.py (4.0KB)\ndebug_forc6.py (3.9KB)\ndebug_forc7.py (2.7KB)\ndebug_forc8.py (2.9KB)\ndebug_forc_prove.py (3.0KB)\ndispatcher.rs (26.2KB)\ndoctor.rs (8.4KB)\nhygiene.rs (7.3KB)\nironclaw_blog_test.png (21.4KB)\nironclaw_browser_test_viewport.png (156.1KB)\nironclaw_linkedin_debug.png (6.5KB)\nironclaw_spot_test.txt (19B)\nkeys_chain_signatures.rs (6.1KB)\nkeys_error.rs (1.6KB)\nkeys_intents.rs (5.8KB)\nkeys_mod.rs (31.6KB)\nkeys_policy.rs (31.4KB)\nkeys_rpc.rs (8.5KB)\nkeys_signer.rs (8.0KB)\nkeys_spending.rs (5.9KB)\nkeys_transaction.rs (13.7KB)\nkeys_types.rs (17.4KB)\nleak_detection_research_summary.md (13.0KB)\nleak_detector.rs (25.3KB)\nlib_new.rs (5.0KB)\nllm_mod.rs (10.0KB)\nmain.rs (56.8KB)\nmain_bootstrap.rs (11.0KB)\nmain_rs.txt (34.1KB)\nnear_resp.json (39B)\nobs_log.rs (5.9KB)\nobs_mod.rs (2.7KB)\nobs_multi.rs (4.0KB)\nobs_noop.rs (1.5KB)\nobs_traits.rs (4.0KB)\nour_bootstrap.rs (12.6KB)\nour_config.rs (45.2KB)\npost_final_review.py (6.4KB)\npost_review.py (4.6KB)\npost_review2.py (4.8KB)\npost_review3.py (2.2KB)\npr-35.diff (137.4KB)\npr115.diff (1.6MB)\npr126.diff (336.9KB)\npr14_full_diff.txt (193.4KB)\npr39_files.txt (243B)\npr51_files.txt (548B)\npr93_diff.txt (160.4KB)\nprs.json (4.6KB)\nresponse_cache.rs (15.1KB)\nrun_hard_proof_search.sh (545B)\nrun_hard_proof_search2.sh (515B)\nrun_proof_search_v3.sh (523B)\nrun_proof_search_v4.sh (524B)\nservice.rs (11.9KB)\nsettings.rs (34.7KB)\ntest_line.py (789B)\ntest_search.txt (1.8KB)\nthread_ops.rs (38.5KB)\ntool_wit.txt (6.0KB)\ntunnel_cf.rs (3.9KB)\ntunnel_custom.rs (6.2KB)\ntunnel_mod.rs (10.4KB)\ntunnel_ngrok.rs (4.2KB)\ntunnel_none.rs (1.3KB)\ntunnel_ts.rs (3.5KB)\nwasm_capabilities.rs (15.2KB)\nwasm_capabilities_schema.rs (26.2KB)\nwasm_host.rs (25.3KB)\nwasm_wrapper.rs (14.9KB)\n```","started_at":"2026-02-17T22:02:24.602248Z","finished_at":"2026-02-17T22:02:43.748463Z","config_label":"default","error":null}
{"task_id":"chain-write-read","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5712,"llm_calls":3,"input_tokens":4489,"output_tokens":73,"estimated_cost_usd":0.014562,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"It says: `ironclaw spot check`.","started_at":"2026-02-17T22:02:43.749933Z","finished_at":"2026-02-17T22:02:49.462007Z","config_label":"default","error":null}
{"task_id":"chain-shell-json","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4582,"llm_calls":3,"input_tokens":4433,"output_tokens":94,"estimated_cost_usd":0.014709,"tool_calls":[{"name":"shell","duration_ms":0,"success":true},{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"[Called tool `json` with arguments: {\"data\":{\"code\":200,\"status\":\"ok\"},\"operation\":\"query\",\"path\":\"status\"}]","started_at":"2026-02-17T22:02:49.463794Z","finished_at":"2026-02-17T22:02:54.046092Z","config_label":"default","error":null}
{"task_id":"chain-time-echo","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4583,"llm_calls":3,"input_tokens":4404,"output_tokens":76,"estimated_cost_usd":0.014352,"tool_calls":[{"name":"time","duration_ms":0,"success":true},{"name":"echo","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"2026-02-17T22:02:55.786456+00:00","started_at":"2026-02-17T22:02:54.046306Z","finished_at":"2026-02-17T22:02:58.629389Z","config_label":"default","error":null}
{"task_id":"robust-no-tool","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":1020,"llm_calls":1,"input_tokens":1366,"output_tokens":5,"estimated_cost_usd":0.004173,"tool_calls":[],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Paris.","started_at":"2026-02-17T22:02:58.630502Z","finished_at":"2026-02-17T22:02:59.650655Z","config_label":"default","error":null}
{"task_id":"robust-correct-tool","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":3980,"llm_calls":2,"input_tokens":2825,"output_tokens":50,"estimated_cost_usd":0.009225,"tool_calls":[{"name":"time","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Its **22:03:01 UTC** right now (**2026-02-17**).","started_at":"2026-02-17T22:02:59.650901Z","finished_at":"2026-02-17T22:03:03.631122Z","config_label":"default","error":null}
{"task_id":"robust-json-validate","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":2954,"llm_calls":2,"input_tokens":2822,"output_tokens":43,"estimated_cost_usd":0.009111,"tool_calls":[{"name":"json","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The JSON is valid (`valid: true`).","started_at":"2026-02-17T22:03:03.631527Z","finished_at":"2026-02-17T22:03:06.585971Z","config_label":"default","error":null}
{"task_id":"memory-save-daily","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4890,"llm_calls":3,"input_tokens":4547,"output_tokens":114,"estimated_cost_usd":0.015351,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"```md\n# Daily Tasks\n\n1. Review PR #42\n2. Update API docs\n3. Deploy to staging\n```","started_at":"2026-02-17T22:03:06.586556Z","finished_at":"2026-02-17T22:03:11.476648Z","config_label":"default","error":null}
{"task_id":"memory-save-reminder","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5607,"llm_calls":3,"input_tokens":4531,"output_tokens":95,"estimated_cost_usd":0.015018,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The appointment is on **March 5th at 2pm**, with **Dr. Smith**.","started_at":"2026-02-17T22:03:11.476873Z","finished_at":"2026-02-17T22:03:17.084343Z","config_label":"default","error":null}
{"task_id":"memory-save-meeting","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":6113,"llm_calls":3,"input_tokens":4750,"output_tokens":122,"estimated_cost_usd":0.01608,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Frontend owner: **Bob** \nLaunch date: **April 15th**","started_at":"2026-02-17T22:03:17.085432Z","finished_at":"2026-02-17T22:03:23.198905Z","config_label":"default","error":null}
{"task_id":"memory-append-log","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":9880,"llm_calls":6,"input_tokens":9941,"output_tokens":208,"estimated_cost_usd":0.032943,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"shell","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Contents of `/tmp/bench-log.md`:\n1. Morning standup: discussed blockers on auth service \n2. Afternoon: resolved API timeout issue in payment gateway \n\nSummary: You discussed blockers affecting the auth service during the morning standup, and later in the afternoon you resolved an API timeout issue in the payment gateway.","started_at":"2026-02-17T22:03:23.199155Z","finished_at":"2026-02-17T22:03:33.079396Z","config_label":"default","error":null}
{"task_id":"memory-recall-detail","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":4989,"llm_calls":3,"input_tokens":4721,"output_tokens":121,"estimated_cost_usd":0.015978,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"The project uses **PostgreSQL** as the database, and the PM is **James**.","started_at":"2026-02-17T22:03:33.080611Z","finished_at":"2026-02-17T22:03:38.069896Z","config_label":"default","error":null}
{"task_id":"memory-todo-priority","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":5916,"llm_calls":3,"input_tokens":4676,"output_tokens":110,"estimated_cost_usd":0.015678,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"High priority tasks:\n- Fix login bug\n- Security audit","started_at":"2026-02-17T22:03:38.070162Z","finished_at":"2026-02-17T22:03:43.986831Z","config_label":"default","error":null}
{"task_id":"memory-multifile","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":6081,"llm_calls":4,"input_tokens":6442,"output_tokens":196,"estimated_cost_usd":0.022266,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"On Tuesday you have **Sprint retro at 11am**, then **team lunch at noon**.","started_at":"2026-02-17T22:03:43.988013Z","finished_at":"2026-02-17T22:03:50.069452Z","config_label":"default","error":null}
{"task_id":"memory-update-context","suite_id":"spot","score":{"value":1.0,"label":"pass","details":null},"trace":{"wall_time_ms":9164,"llm_calls":5,"input_tokens":8354,"output_tokens":252,"estimated_cost_usd":0.028842,"tool_calls":[{"name":"write_file","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true},{"name":"apply_patch","duration_ms":0,"success":true},{"name":"read_file","duration_ms":0,"success":true}],"turns":1,"hit_iteration_limit":false,"hit_timeout":false},"response":"Final read confirms the file now says `timezone: EST`.","started_at":"2026-02-17T22:03:50.070277Z","finished_at":"2026-02-17T22:03:59.235196Z","config_label":"default","error":null}
-21
View File
@@ -1,21 +0,0 @@
{"id": "smoke-greeting", "prompt": "Hello! Introduce yourself briefly.", "tags": ["smoke"], "assertions": {"response_matches": "(?i)(hello|hi|hey|assistant|agent|help)", "no_error": true, "max_tool_calls": 0}}
{"id": "smoke-math", "prompt": "What is 47 * 23? Reply with just the number.", "tags": ["smoke"], "assertions": {"response_contains": ["1081"], "no_error": true, "max_tool_calls": 0}}
{"id": "tool-echo", "prompt": "Use the echo tool to repeat the message: 'Spot check passed'", "tags": ["tool"], "assertions": {"tools_used": ["echo"], "response_contains": ["Spot check passed"], "no_error": true}}
{"id": "tool-time", "prompt": "What is the current date and time? Use the time tool.", "tags": ["tool"], "assertions": {"tools_used": ["time"], "response_matches": "20\\d{2}", "no_error": true}}
{"id": "tool-json-query", "prompt": "Given this JSON: {\"users\": [{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]}, use the json tool to extract the second user's name.", "tags": ["tool"], "assertions": {"tools_used": ["json"], "response_contains": ["Bob"], "no_error": true}}
{"id": "tool-shell-echo", "prompt": "Use the shell tool to run: echo 'benchmark test'", "tags": ["tool"], "assertions": {"tools_used": ["shell"], "response_contains": ["benchmark test"], "no_error": true}}
{"id": "tool-list-dir", "prompt": "Use the list_dir tool to list the contents of the /tmp directory.", "tags": ["tool"], "assertions": {"tools_used": ["list_dir"], "no_error": true}}
{"id": "chain-write-read", "prompt": "Write the text 'ironclaw spot check' to /tmp/ironclaw_spot_test.txt using the write_file tool, then read it back using the read_file tool and tell me what it says.", "tags": ["chain"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["ironclaw spot check"], "no_error": true}}
{"id": "chain-shell-json", "prompt": "Run a shell command to output the JSON string '{\"status\": \"ok\", \"code\": 200}', then use the json tool to extract the status field.", "tags": ["chain"], "assertions": {"response_contains": ["ok"], "min_tool_calls": 1, "no_error": true}}
{"id": "chain-time-echo", "prompt": "First get the current time using the time tool, then use the echo tool to repeat it back.", "tags": ["chain"], "assertions": {"tools_used": ["time", "echo"], "no_error": true}}
{"id": "robust-no-tool", "prompt": "What is the capital of France? Answer directly without using any tools.", "tags": ["robust"], "assertions": {"response_contains": ["Paris"], "max_tool_calls": 0, "no_error": true}}
{"id": "robust-correct-tool", "prompt": "What time is it right now?", "tags": ["robust"], "assertions": {"tools_used": ["time"], "tools_not_used": ["shell", "echo"], "no_error": true}}
{"id": "robust-json-validate", "prompt": "Use the json tool to validate whether this is valid JSON: {\"key\": \"value\", \"num\": 42}", "tags": ["robust"], "assertions": {"tools_used": ["json"], "tools_not_used": ["shell"], "no_error": true}}
{"id": "memory-save-daily", "prompt": "Save these daily tasks to /tmp/bench-daily.md:\n1. Review PR #42\n2. Update API docs\n3. Deploy to staging\nThen read the file back and confirm what was saved.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["PR #42", "docs", "staging"], "no_error": true}}
{"id": "memory-save-reminder", "prompt": "Write a reminder to /tmp/bench-reminder.md: Dentist appointment on March 5th at 2pm with Dr. Smith. Then read the file back and tell me when the appointment is and with whom.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["March 5", "Smith"], "response_matches": "2(:00)?\\s*[Pp][Mm]", "no_error": true}}
{"id": "memory-save-meeting", "prompt": "Save these meeting notes to /tmp/bench-meeting.md:\nMeeting: Project Phoenix sync\nAttendees: Alice, Bob, Carol\nDecisions:\n- Launch date: April 15th\n- Budget: $50k approved\n- Bob owns frontend, Carol owns backend\nThen read the file back and tell me who owns the frontend and what the launch date is.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["Bob", "frontend", "April 15"], "no_error": true}}
{"id": "memory-append-log", "prompt": "Write 'Morning standup: discussed blockers on auth service' to /tmp/bench-log.md. Then append a new line 'Afternoon: resolved API timeout issue in payment gateway' to the same file. Finally read the full file and summarize what happened.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["auth", "timeout"], "min_tool_calls": 3, "no_error": true}}
{"id": "memory-recall-detail", "prompt": "Save the following project context to /tmp/bench-project.md:\nProject Ironclad uses Rust for the backend, React for the frontend, and PostgreSQL for the database. The API is deployed on AWS ECS. The lead developer is Sarah and the PM is James. The sprint ends on March 20th.\nThen read it back and answer: What database does the project use, and who is the PM?", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["PostgreSQL", "James"], "no_error": true}}
{"id": "memory-todo-priority", "prompt": "Write the following to /tmp/bench-todo.md:\n- [ ] Fix login bug (priority: HIGH)\n- [ ] Write unit tests (priority: medium)\n- [ ] Update README (priority: low)\n- [ ] Security audit (priority: HIGH)\nThen read it back and tell me which tasks are high priority.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["login bug", "security audit"], "no_error": true}}
{"id": "memory-multifile", "prompt": "Save 'Team standup at 9am, then client demo at 2pm' to /tmp/bench-monday.md and 'Sprint retro at 11am, team lunch at noon' to /tmp/bench-tuesday.md. Then read both files and tell me what's happening on Tuesday.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["retro", "lunch"], "min_tool_calls": 3, "no_error": true}}
{"id": "memory-update-context", "prompt": "Write 'User preference: dark mode, timezone: PST, language: English' to /tmp/bench-prefs.md. Then read it back, and rewrite the file changing the timezone to EST. Finally read it one more time and confirm the timezone is now EST.", "tags": ["memory"], "assertions": {"tools_used": ["write_file", "read_file"], "response_contains": ["EST"], "min_tool_calls": 4, "no_error": true}}
-8
View File
@@ -1,8 +0,0 @@
task_timeout = "120s"
parallelism = 1
[[matrix]]
label = "default"
[suite_config]
dataset_path = "benchmarks/data/spot.jsonl"
-243
View File
@@ -1,243 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// A single entry in the custom JSONL format.
#[derive(Debug, Deserialize)]
struct CustomEntry {
id: String,
prompt: String,
#[serde(default)]
context: Option<String>,
#[serde(default)]
tags: Vec<String>,
#[serde(default)]
expected: Option<String>,
#[serde(default)]
expected_contains: Option<String>,
#[serde(default)]
expected_regex: Option<String>,
/// "exact", "contains", "regex", or "llm" (default: "exact")
#[serde(default = "default_scorer")]
scorer: String,
}
fn default_scorer() -> String {
"exact".to_string()
}
/// Custom JSONL benchmark suite.
///
/// Each line of the JSONL file is a task with `id`, `prompt`, and scoring
/// criteria (`expected`, `expected_contains`, `expected_regex`).
pub struct CustomSuite {
dataset_path: PathBuf,
}
impl CustomSuite {
pub fn new(dataset_path: impl Into<PathBuf>) -> Self {
Self {
dataset_path: dataset_path.into(),
}
}
}
#[async_trait]
impl BenchSuite for CustomSuite {
fn name(&self) -> &str {
"Custom JSONL"
}
fn id(&self) -> &str {
"custom"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path).map_err(BenchError::Io)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: CustomEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("line {}: {}", line_num + 1, e)))?;
let mut metadata = serde_json::json!({
"scorer": entry.scorer,
});
if let Some(ref expected) = entry.expected {
metadata["expected"] = serde_json::Value::String(expected.clone());
}
if let Some(ref expected_contains) = entry.expected_contains {
metadata["expected_contains"] =
serde_json::Value::String(expected_contains.clone());
}
if let Some(ref expected_regex) = entry.expected_regex {
metadata["expected_regex"] = serde_json::Value::String(expected_regex.clone());
}
tasks.push(BenchTask {
id: entry.id,
prompt: entry.prompt,
context: entry.context,
resources: vec![],
tags: entry.tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let scorer = task
.metadata
.get("scorer")
.and_then(|v| v.as_str())
.unwrap_or("exact");
match scorer {
"exact" => {
if let Some(expected) = task.metadata.get("expected").and_then(|v| v.as_str()) {
Ok(scoring::exact_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected' field for exact scoring".to_string(),
})
}
}
"contains" => {
if let Some(expected) = task
.metadata
.get("expected_contains")
.and_then(|v| v.as_str())
{
Ok(scoring::contains_match(expected, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_contains' field for contains scoring".to_string(),
})
}
}
"regex" => {
if let Some(pattern) = task.metadata.get("expected_regex").and_then(|v| v.as_str())
{
Ok(scoring::regex_match(pattern, &submission.response))
} else {
Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: "no 'expected_regex' field for regex scoring".to_string(),
})
}
}
"llm" => {
// TODO: LLM-as-judge scoring
tracing::warn!(
task_id = %task.id,
"LLM-as-judge scoring not implemented, returning placeholder 0.5"
);
Ok(BenchScore::partial(0.5, "LLM scoring not yet implemented"))
}
other => Err(BenchError::Scoring {
task_id: task.id.clone(),
reason: format!("unknown scorer: {other}"),
}),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_custom_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
writeln!(
file,
r#"{{"id": "t2", "prompt": "Say hello", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 2);
assert_eq!(tasks[0].id, "t1");
assert_eq!(tasks[1].id, "t2");
}
#[tokio::test]
async fn test_custom_exact_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "What is 2+2?", "expected": "4"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "4".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[tokio::test]
async fn test_custom_contains_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tasks.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "prompt": "Greet me", "expected_contains": "hello", "scorer": "contains"}}"#
)
.unwrap();
let suite = CustomSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: "Hello there!".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
-183
View File
@@ -1,183 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::scoring;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskResource, TaskSubmission};
/// GAIA dataset entry (Hugging Face JSONL format).
#[derive(Debug, Deserialize)]
struct GaiaEntry {
task_id: String,
#[serde(alias = "Question")]
question: String,
#[serde(alias = "Final answer", alias = "final_answer")]
final_answer: String,
#[serde(alias = "Level", default)]
level: Option<u32>,
#[serde(alias = "file_name", default)]
file_name: Option<String>,
}
/// GAIA benchmark suite.
///
/// Tasks are loaded from HuggingFace JSONL exports. Scoring uses normalized
/// exact match against the `final_answer` field.
pub struct GaiaSuite {
dataset_path: PathBuf,
attachments_dir: Option<PathBuf>,
}
impl GaiaSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
attachments_dir: Option<impl Into<PathBuf>>,
) -> Self {
Self {
dataset_path: dataset_path.into(),
attachments_dir: attachments_dir.map(|d| d.into()),
}
}
}
#[async_trait]
impl BenchSuite for GaiaSuite {
fn name(&self) -> &str {
"GAIA"
}
fn id(&self) -> &str {
"gaia"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: GaiaEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("GAIA line {}: {}", line_num + 1, e)))?;
let mut resources = Vec::new();
if let Some(ref file_name) = entry.file_name {
if !file_name.is_empty() {
if let Some(ref dir) = self.attachments_dir {
resources.push(TaskResource {
name: file_name.clone(),
path: dir.join(file_name).to_string_lossy().to_string(),
resource_type: crate::suite::ResourceType::File,
});
}
}
}
let mut tags = Vec::new();
if let Some(level) = entry.level {
tags.push(format!("level-{level}"));
}
let metadata = serde_json::json!({
"expected": entry.final_answer,
"level": entry.level,
});
tasks.push(BenchTask {
id: entry.task_id,
prompt: entry.question,
context: None,
resources,
tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let expected = task
.metadata
.get("expected")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::Scoring {
task_id: task.id.clone(),
reason: "missing expected answer in metadata".to_string(),
})?;
Ok(scoring::exact_match(expected, &submission.response))
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_gaia_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "What is the capital of France?", "final_answer": "Paris", "Level": 1}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "g1");
assert!(tasks[0].tags.contains(&"level-1".to_string()));
}
#[tokio::test]
async fn test_gaia_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("gaia.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"task_id": "g1", "question": "Capital of France?", "final_answer": "Paris"}}"#
)
.unwrap();
let suite = GaiaSuite::new(&path, None::<PathBuf>);
let tasks = suite.load_tasks().await.unwrap();
// Exact match (case insensitive)
let submission = TaskSubmission {
response: "paris".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
// Wrong answer
let submission = TaskSubmission {
response: "London".to_string(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
}
-124
View File
@@ -1,124 +0,0 @@
pub mod custom;
pub mod gaia;
pub mod spot;
pub mod swe_bench;
pub mod tau_bench;
use crate::config::BenchConfig;
use crate::error::BenchError;
use crate::suite::BenchSuite;
/// List of all known suite IDs.
pub const KNOWN_SUITES: &[(&str, &str)] = &[
("custom", "Custom JSONL tasks"),
("gaia", "GAIA benchmark (knowledge & reasoning)"),
("spot", "Spot checks (end-to-end user workflows)"),
("tau_bench", "Tau-bench (multi-turn tool use)"),
("swe_bench", "SWE-bench Pro (software engineering)"),
];
/// Create a suite adapter by name.
pub fn create_suite(name: &str, config: &BenchConfig) -> Result<Box<dyn BenchSuite>, BenchError> {
let suite_map = config.suite_config_map();
match name {
"custom" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'custom' suite".to_string(),
)
})?;
Ok(Box::new(custom::CustomSuite::new(dataset_path)))
}
"gaia" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'gaia' suite".to_string(),
)
})?;
let attachments_dir = suite_map
.get("attachments_dir")
.and_then(|v| v.as_str())
.map(|s| s.to_string());
Ok(Box::new(gaia::GaiaSuite::new(
dataset_path,
attachments_dir,
)))
}
"spot" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'spot' suite".to_string(),
)
})?;
Ok(Box::new(spot::SpotSuite::new(dataset_path)))
}
"tau_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'tau_bench' suite".to_string(),
)
})?;
let domain = suite_map
.get("domain")
.and_then(|v| v.as_str())
.unwrap_or("retail")
.to_string();
Ok(Box::new(tau_bench::TauBenchSuite::new(
dataset_path,
domain,
)))
}
"swe_bench" => {
let dataset_path = suite_map
.get("dataset_path")
.and_then(|v| v.as_str())
.map(|s| s.to_string())
.ok_or_else(|| {
BenchError::Config(
"suite_config.dataset_path is required for 'swe_bench' suite".to_string(),
)
})?;
let workspace_dir = suite_map
.get("workspace_dir")
.and_then(|v| v.as_str())
.unwrap_or("/tmp/swe-bench")
.to_string();
let use_docker = suite_map
.get("use_docker")
.and_then(|v| v.as_bool())
.unwrap_or(false);
Ok(Box::new(swe_bench::SweBenchSuite::new(
dataset_path,
workspace_dir,
use_docker,
)))
}
_ => {
let available = KNOWN_SUITES
.iter()
.map(|(id, _)| *id)
.collect::<Vec<_>>()
.join(", ");
Err(BenchError::SuiteNotFound {
name: name.to_string(),
available,
})
}
}
}
-504
View File
@@ -1,504 +0,0 @@
use std::collections::HashSet;
use std::io::BufRead;
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use regex::Regex;
use serde::{Deserialize, Serialize};
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// Multi-criterion assertions for a spot check scenario.
///
/// Each field generates one or more individual checks. The final score is
/// `passed_checks / total_checks`, giving a value between 0.0 and 1.0.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct SpotAssertions {
/// All must appear in the response (case-insensitive).
#[serde(default)]
pub response_contains: Vec<String>,
/// None may appear in the response (case-insensitive).
#[serde(default)]
pub response_not_contains: Vec<String>,
/// Each tool name must appear in the tool_calls list (checked by name,
/// not by count; duplicates in tool_calls are collapsed).
#[serde(default)]
pub tools_used: Vec<String>,
/// None of these tool names may appear in the tool_calls list.
#[serde(default)]
pub tools_not_used: Vec<String>,
/// Regex pattern the response must match.
#[serde(default)]
pub response_matches: Option<String>,
/// Hard fail if the task produced an error.
#[serde(default)]
pub no_error: bool,
/// Minimum number of tool calls expected (counts duplicates).
#[serde(default)]
pub min_tool_calls: Option<usize>,
/// Maximum number of tool calls allowed (counts duplicates).
#[serde(default)]
pub max_tool_calls: Option<usize>,
}
impl SpotAssertions {
/// Evaluate all assertions against a submission, returning (score, failure_details).
pub fn evaluate(&self, submission: &TaskSubmission) -> (f64, Vec<String>) {
let mut passed: usize = 0;
let mut total: usize = 0;
let mut failures: Vec<String> = Vec::new();
// Hard fail: error check
if self.no_error {
total += 1;
if let Some(ref err) = submission.error {
failures.push(format!("no_error: task errored with: {err}"));
// Hard fail: return 0.0 immediately
return (0.0, failures);
}
passed += 1;
}
let response_lower = submission.response.to_lowercase();
// response_contains: all must appear
for needle in &self.response_contains {
total += 1;
if response_lower.contains(&needle.to_lowercase()) {
passed += 1;
} else {
failures.push(format!("response_contains: missing \"{needle}\""));
}
}
// response_not_contains: none may appear
for needle in &self.response_not_contains {
total += 1;
if response_lower.contains(&needle.to_lowercase()) {
failures.push(format!("response_not_contains: found \"{needle}\""));
} else {
passed += 1;
}
}
let tool_set: HashSet<&str> = submission.tool_calls.iter().map(|s| s.as_str()).collect();
// tools_used: each must appear
for tool in &self.tools_used {
total += 1;
if tool_set.contains(tool.as_str()) {
passed += 1;
} else {
failures.push(format!("tools_used: \"{tool}\" not called"));
}
}
// tools_not_used: none may appear
for tool in &self.tools_not_used {
total += 1;
if tool_set.contains(tool.as_str()) {
failures.push(format!("tools_not_used: \"{tool}\" was called"));
} else {
passed += 1;
}
}
// response_matches: regex pattern
if let Some(ref pattern) = self.response_matches {
total += 1;
match Regex::new(pattern) {
Ok(re) => {
if re.is_match(&submission.response) {
passed += 1;
} else {
failures.push(format!("response_matches: /{pattern}/ did not match"));
}
}
Err(e) => {
failures.push(format!("response_matches: bad regex: {e}"));
}
}
}
let call_count = submission.tool_calls.len();
// min_tool_calls
if let Some(min) = self.min_tool_calls {
total += 1;
if call_count >= min {
passed += 1;
} else {
failures.push(format!(
"min_tool_calls: expected >= {min}, got {call_count}"
));
}
}
// max_tool_calls
if let Some(max) = self.max_tool_calls {
total += 1;
if call_count <= max {
passed += 1;
} else {
failures.push(format!(
"max_tool_calls: expected <= {max}, got {call_count}"
));
}
}
if total == 0 {
return (1.0, failures);
}
let score = passed as f64 / total as f64;
(score, failures)
}
}
/// JSONL entry for a spot check scenario.
#[derive(Debug, Deserialize)]
struct SpotEntry {
id: String,
prompt: String,
#[serde(default)]
context: Option<String>,
#[serde(default)]
tags: Vec<String>,
#[serde(default)]
assertions: SpotAssertions,
}
/// Spot benchmark suite: end-to-end checks for real user workflows.
///
/// Tests conversation, individual tool use, multi-tool chaining, and robustness.
/// Each task declares multi-criterion assertions scored as passed/total.
pub struct SpotSuite {
dataset_path: PathBuf,
}
impl SpotSuite {
pub fn new(dataset_path: impl Into<PathBuf>) -> Self {
Self {
dataset_path: dataset_path.into(),
}
}
}
#[async_trait]
impl BenchSuite for SpotSuite {
fn name(&self) -> &str {
"Spot Checks"
}
fn id(&self) -> &str {
"spot"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path).map_err(BenchError::Io)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: SpotEntry = serde_json::from_str(trimmed)
.map_err(|e| BenchError::Config(format!("spot line {}: {}", line_num + 1, e)))?;
let metadata = serde_json::json!({
"assertions": serde_json::to_value(&entry.assertions)
.map_err(|e| BenchError::Config(format!("spot {}: {}", entry.id, e)))?,
});
tasks.push(BenchTask {
id: entry.id,
prompt: entry.prompt,
context: entry.context,
resources: vec![],
tags: entry.tags,
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
let assertions: SpotAssertions = task
.metadata
.get("assertions")
.ok_or_else(|| BenchError::Scoring {
task_id: task.id.clone(),
reason: "missing assertions in metadata".to_string(),
})
.and_then(|v| {
serde_json::from_value(v.clone()).map_err(|e| BenchError::Scoring {
task_id: task.id.clone(),
reason: format!("bad assertions: {e}"),
})
})?;
let (score, failures) = assertions.evaluate(submission);
if score >= 1.0 {
Ok(BenchScore::pass())
} else if score <= 0.0 {
Ok(BenchScore::fail(failures.join("; ")))
} else {
Ok(BenchScore::partial(score, failures.join("; ")))
}
}
fn additional_tools(&self) -> Vec<Arc<dyn ironclaw::tools::Tool>> {
vec![
Arc::new(ironclaw::tools::builtin::ShellTool::new()),
Arc::new(ironclaw::tools::builtin::ReadFileTool::new()),
Arc::new(ironclaw::tools::builtin::WriteFileTool::new()),
Arc::new(ironclaw::tools::builtin::ListDirTool::new()),
Arc::new(ironclaw::tools::builtin::ApplyPatchTool::new()),
]
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
fn make_submission(
response: &str,
tool_calls: Vec<&str>,
error: Option<&str>,
) -> TaskSubmission {
TaskSubmission {
response: response.to_string(),
conversation: vec![],
tool_calls: tool_calls.into_iter().map(|s| s.to_string()).collect(),
error: error.map(|s| s.to_string()),
}
}
#[test]
fn test_all_pass() {
let assertions = SpotAssertions {
response_contains: vec!["hello".to_string()],
tools_used: vec!["echo".to_string()],
no_error: true,
..Default::default()
};
let sub = make_submission("Hello, world!", vec!["echo"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_hard_fail_on_error() {
let assertions = SpotAssertions {
no_error: true,
response_contains: vec!["hello".to_string()],
..Default::default()
};
let sub = make_submission("Hello!", vec![], Some("timeout after 60s"));
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
assert!(failures[0].contains("no_error"));
}
#[test]
fn test_partial_score() {
let assertions = SpotAssertions {
response_contains: vec!["alpha".to_string(), "beta".to_string()],
..Default::default()
};
let sub = make_submission("alpha is here but not the other", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert_eq!(failures.len(), 1);
assert!(failures[0].contains("beta"));
}
#[test]
fn test_response_not_contains() {
let assertions = SpotAssertions {
response_not_contains: vec!["error".to_string(), "fail".to_string()],
..Default::default()
};
let sub = make_submission("This is an error message", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert_eq!(failures.len(), 1);
assert!(failures[0].contains("error"));
}
#[test]
fn test_tools_used_and_not_used() {
let assertions = SpotAssertions {
tools_used: vec!["time".to_string()],
tools_not_used: vec!["shell".to_string(), "echo".to_string()],
..Default::default()
};
let sub = make_submission("The time is now", vec!["time"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_tools_not_used_fails() {
let assertions = SpotAssertions {
tools_not_used: vec!["shell".to_string()],
..Default::default()
};
let sub = make_submission("result", vec!["shell", "time"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_response_matches_regex() {
let assertions = SpotAssertions {
response_matches: Some(r"\d{4}".to_string()),
..Default::default()
};
let sub = make_submission("The year is 2026", vec![], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
assert!(failures.is_empty());
}
#[test]
fn test_response_matches_regex_fail() {
let assertions = SpotAssertions {
response_matches: Some(r"^\d+$".to_string()),
..Default::default()
};
let sub = make_submission("not a number", vec![], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_min_max_tool_calls() {
let assertions = SpotAssertions {
min_tool_calls: Some(2),
max_tool_calls: Some(4),
..Default::default()
};
// Within range
let sub = make_submission("ok", vec!["a", "b", "c"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
// Too few
let sub = make_submission("ok", vec!["a"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert!(failures[0].contains("min_tool_calls"));
// Too many
let sub = make_submission("ok", vec!["a", "b", "c", "d", "e"], None);
let (score, failures) = assertions.evaluate(&sub);
assert_eq!(score, 0.5);
assert!(failures[0].contains("max_tool_calls"));
}
#[test]
fn test_max_zero_tool_calls() {
let assertions = SpotAssertions {
max_tool_calls: Some(0),
..Default::default()
};
let sub = make_submission("just talking", vec![], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
let sub = make_submission("oops", vec!["echo"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 0.0);
}
#[test]
fn test_empty_assertions() {
let assertions = SpotAssertions::default();
let sub = make_submission("anything", vec!["whatever"], None);
let (score, _) = assertions.evaluate(&sub);
assert_eq!(score, 1.0);
}
#[tokio::test]
async fn test_spot_load_tasks() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("spot.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "s1", "prompt": "Hello", "tags": ["smoke"], "assertions": {{"response_contains": ["hello"], "no_error": true}}}}"#
)
.unwrap();
writeln!(
file,
r#"{{"id": "s2", "prompt": "Echo test", "assertions": {{"tools_used": ["echo"]}}}}"#
)
.unwrap();
let suite = SpotSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 2);
assert_eq!(tasks[0].id, "s1");
assert_eq!(tasks[1].id, "s2");
assert!(tasks[0].tags.contains(&"smoke".to_string()));
}
#[tokio::test]
async fn test_spot_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("spot.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "s1", "prompt": "Hello", "assertions": {{"response_contains": ["hello", "world"], "no_error": true}}}}"#
)
.unwrap();
let suite = SpotSuite::new(&path);
let tasks = suite.load_tasks().await.unwrap();
// Full pass
let sub = make_submission("Hello World!", vec![], None);
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
// Partial
let sub = make_submission("Hello there", vec![], None);
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert!(score.value > 0.0 && score.value < 1.0);
assert_eq!(score.label, "partial");
// Error hard fail
let sub = make_submission("Hello World!", vec![], Some("boom"));
let score = suite.score(&tasks[0], &sub).await.unwrap();
assert_eq!(score.value, 0.0);
assert_eq!(score.label, "fail");
}
}
-416
View File
@@ -1,416 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use regex::Regex;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, TaskSubmission};
/// Validate that a string is safe for use as a filesystem path component.
/// Allows alphanumerics, hyphens, underscores, dots, and forward slashes (for nested paths).
/// Rejects absolute paths, `..` traversal, and shell metacharacters.
fn is_safe_path_component(s: &str) -> bool {
!s.is_empty()
&& !s.starts_with('/')
&& !s.contains("..")
&& s.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/'))
}
/// Validate that a repo string matches the expected `owner/repo` GitHub format.
fn is_valid_github_repo(repo: &str) -> bool {
// Match "owner/repo" where both parts are alphanumeric with hyphens/underscores/dots
static REPO_PATTERN: std::sync::LazyLock<Regex> =
std::sync::LazyLock::new(|| Regex::new(r"^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$").unwrap());
REPO_PATTERN.is_match(repo)
}
/// Validate that a string looks like a git ref (hex SHA or valid ref name).
fn is_valid_git_ref(s: &str) -> bool {
!s.is_empty()
&& s.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '/'))
&& !s.contains("..")
}
/// SWE-bench dataset entry.
#[derive(Debug, Deserialize)]
struct SweBenchEntry {
instance_id: String,
repo: String,
base_commit: String,
#[serde(default)]
problem_statement: String,
#[serde(default)]
hints_text: Option<String>,
#[serde(default)]
test_patch: Option<String>,
#[serde(default)]
patch: Option<String>,
}
/// SWE-bench Pro: real-world software engineering tasks.
///
/// Each task clones a repo at a specific commit, presents the problem statement,
/// and expects the agent to produce a patch. Scoring runs the test suite.
pub struct SweBenchSuite {
dataset_path: PathBuf,
workspace_dir: PathBuf,
use_docker: bool,
}
impl SweBenchSuite {
pub fn new(
dataset_path: impl Into<PathBuf>,
workspace_dir: impl Into<PathBuf>,
use_docker: bool,
) -> Self {
Self {
dataset_path: dataset_path.into(),
workspace_dir: workspace_dir.into(),
use_docker,
}
}
}
#[async_trait]
impl BenchSuite for SweBenchSuite {
fn name(&self) -> &str {
"SWE-bench Pro"
}
fn id(&self) -> &str {
"swe_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: SweBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("swe_bench line {}: {}", line_num + 1, e))
})?;
if !is_safe_path_component(&entry.instance_id) {
return Err(BenchError::Config(format!(
"swe_bench line {}: unsafe instance_id \"{}\"",
line_num + 1,
entry.instance_id,
)));
}
if !is_valid_github_repo(&entry.repo) {
return Err(BenchError::Config(format!(
"swe_bench line {}: invalid repo format \"{}\"",
line_num + 1,
entry.repo,
)));
}
if !is_valid_git_ref(&entry.base_commit) {
return Err(BenchError::Config(format!(
"swe_bench line {}: invalid base_commit \"{}\"",
line_num + 1,
entry.base_commit,
)));
}
let metadata = serde_json::json!({
"repo": entry.repo,
"base_commit": entry.base_commit,
"test_patch": entry.test_patch,
"gold_patch": entry.patch,
"use_docker": self.use_docker,
"workspace_dir": self.workspace_dir.to_string_lossy(),
});
let prompt = if let Some(ref hints) = entry.hints_text {
format!("{}\n\nHints:\n{}", entry.problem_statement, hints)
} else {
entry.problem_statement
};
tasks.push(BenchTask {
id: entry.instance_id,
prompt,
context: Some(format!(
"Repository: {}, Commit: {}",
entry.repo, entry.base_commit
)),
resources: vec![],
tags: vec![format!("repo-{}", entry.repo.replace('/', "-"))],
expected_turns: None,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn setup_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let repo = task
.metadata
.get("repo")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing repo in metadata".to_string(),
})?;
let base_commit = task
.metadata
.get("base_commit")
.and_then(|v| v.as_str())
.ok_or_else(|| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: "missing base_commit in metadata".to_string(),
})?;
let task_dir = self.workspace_dir.join(&task.id);
// Clone repo if not already present
if !task_dir.exists() {
let repo_url = format!("https://github.com/{}.git", repo);
let output = tokio::process::Command::new("git")
.args([
"clone",
"--depth",
"1",
&repo_url,
&task_dir.to_string_lossy(),
])
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git clone failed: {stderr}"),
});
}
}
// Checkout the base commit
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {e}"),
})?;
if !output.status.success() {
// Shallow clone might not have the commit; fetch more history
let _ = tokio::process::Command::new("git")
.args(["fetch", "--unshallow"])
.current_dir(&task_dir)
.output()
.await;
let output = tokio::process::Command::new("git")
.args(["checkout", base_commit])
.current_dir(&task_dir)
.output()
.await
.map_err(|e| BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout retry failed: {e}"),
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(BenchError::TaskFailed {
task_id: task.id.clone(),
reason: format!("git checkout failed: {stderr}"),
});
}
}
Ok(())
}
async fn teardown_task(&self, task: &BenchTask) -> Result<(), BenchError> {
let task_dir = self.workspace_dir.join(&task.id);
if task_dir.exists() {
// Reset any changes
let _ = tokio::process::Command::new("git")
.args(["checkout", "."])
.current_dir(&task_dir)
.output()
.await;
let _ = tokio::process::Command::new("git")
.args(["clean", "-fdx"])
.current_dir(&task_dir)
.output()
.await;
}
Ok(())
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// For SWE-bench, scoring requires running the test patch against the agent's changes.
// This is a simplified version that checks if the agent produced any code changes.
let test_patch = task.metadata.get("test_patch").and_then(|v| v.as_str());
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response from agent"));
}
// If we have a test patch, try to verify the submission
if let Some(_test_patch) = test_patch {
// TODO: Apply agent's patch, then apply test patch, then run tests.
// For now, give partial credit if the agent produced some output.
tracing::warn!(
task_id = %task.id,
"SWE-bench test execution not implemented, returning placeholder 0.25"
);
Ok(BenchScore::partial(
0.25,
"test execution not yet implemented; partial credit for response",
))
} else {
tracing::warn!(
task_id = %task.id,
"no test_patch available, returning placeholder 0.25"
);
Ok(BenchScore::partial(
0.25,
"no test_patch available for automated scoring",
))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_swe_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "django__django-12345", "repo": "django/django", "base_commit": "abc123", "problem_statement": "Fix the ORM bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].id, "django__django-12345");
assert!(tasks[0].tags.contains(&"repo-django-django".to_string()));
}
#[tokio::test]
async fn test_swe_bench_scoring_no_response() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "s1", "repo": "org/repo", "base_commit": "abc", "problem_statement": "Fix bug"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let tasks = suite.load_tasks().await.unwrap();
let submission = TaskSubmission {
response: String::new(),
conversation: vec![],
tool_calls: vec![],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.0);
}
#[test]
fn test_is_safe_path_component() {
assert!(is_safe_path_component("django__django-12345"));
assert!(is_safe_path_component("org/repo"));
assert!(is_safe_path_component("abc123"));
assert!(!is_safe_path_component(""));
assert!(!is_safe_path_component("../../etc/passwd"));
assert!(!is_safe_path_component("/etc/passwd"));
assert!(!is_safe_path_component("foo;rm -rf /"));
assert!(!is_safe_path_component("foo bar"));
}
#[test]
fn test_is_valid_github_repo() {
assert!(is_valid_github_repo("django/django"));
assert!(is_valid_github_repo("org/repo-name"));
assert!(is_valid_github_repo("Org.Name/Repo_v2"));
assert!(!is_valid_github_repo(""));
assert!(!is_valid_github_repo("no-slash"));
assert!(!is_valid_github_repo("too/many/slashes"));
assert!(!is_valid_github_repo("spa ce/repo"));
}
#[test]
fn test_is_valid_git_ref() {
assert!(is_valid_git_ref("abc123"));
assert!(is_valid_git_ref("deadbeef0123456789abcdef0123456789abcdef"));
assert!(is_valid_git_ref("v1.2.3"));
assert!(is_valid_git_ref("main"));
assert!(!is_valid_git_ref(""));
assert!(!is_valid_git_ref("bad..ref"));
assert!(!is_valid_git_ref("has space"));
assert!(!is_valid_git_ref("semi;colon"));
}
#[tokio::test]
async fn test_swe_bench_rejects_path_traversal() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "../../etc/passwd", "repo": "org/repo", "base_commit": "abc", "problem_statement": "evil"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let err = suite.load_tasks().await.unwrap_err();
assert!(err.to_string().contains("unsafe instance_id"));
}
#[tokio::test]
async fn test_swe_bench_rejects_bad_repo() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("swe.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"instance_id": "task1", "repo": "not-a-repo-format", "base_commit": "abc", "problem_statement": "bad"}}"#
)
.unwrap();
let suite = SweBenchSuite::new(&path, "/tmp/swe-test", false);
let err = suite.load_tasks().await.unwrap_err();
assert!(err.to_string().contains("invalid repo format"));
}
}
-233
View File
@@ -1,233 +0,0 @@
use std::io::BufRead;
use std::path::PathBuf;
use async_trait::async_trait;
use serde::Deserialize;
use crate::error::BenchError;
use crate::suite::{BenchScore, BenchSuite, BenchTask, ConversationTurn, TaskSubmission};
/// Tau-bench task entry.
#[derive(Debug, Deserialize)]
struct TauBenchEntry {
id: String,
#[serde(default)]
domain: String,
instruction: String,
#[serde(default)]
user_persona: Option<String>,
#[serde(default)]
expected_state: Option<serde_json::Value>,
#[serde(default)]
expected_actions: Vec<String>,
#[serde(default)]
max_turns: Option<usize>,
}
/// Tau-bench: multi-turn tool-calling dialog benchmark.
///
/// Tests agent ability to handle customer service scenarios with simulated
/// domain APIs (retail, airline). Scoring compares final state against expected.
pub struct TauBenchSuite {
dataset_path: PathBuf,
domain: String,
}
impl TauBenchSuite {
pub fn new(dataset_path: impl Into<PathBuf>, domain: impl Into<String>) -> Self {
Self {
dataset_path: dataset_path.into(),
domain: domain.into(),
}
}
}
#[async_trait]
impl BenchSuite for TauBenchSuite {
fn name(&self) -> &str {
"Tau-bench"
}
fn id(&self) -> &str {
"tau_bench"
}
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError> {
let file = std::fs::File::open(&self.dataset_path)?;
let reader = std::io::BufReader::new(file);
let mut tasks = Vec::new();
for (line_num, line) in reader.lines().enumerate() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let entry: TauBenchEntry = serde_json::from_str(trimmed).map_err(|e| {
BenchError::Config(format!("tau_bench line {}: {}", line_num + 1, e))
})?;
let domain = if entry.domain.is_empty() {
self.domain.clone()
} else {
entry.domain.clone()
};
let metadata = serde_json::json!({
"domain": domain,
"user_persona": entry.user_persona,
"expected_state": entry.expected_state,
"expected_actions": entry.expected_actions,
});
tasks.push(BenchTask {
id: entry.id,
prompt: entry.instruction,
context: entry.user_persona.clone(),
resources: vec![],
tags: vec![format!("domain-{domain}")],
expected_turns: entry.max_turns,
timeout: None,
metadata,
});
}
Ok(tasks)
}
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError> {
// Score based on expected actions completion
let expected_actions: Vec<String> = task
.metadata
.get("expected_actions")
.and_then(|v| serde_json::from_value(v.clone()).ok())
.unwrap_or_default();
if expected_actions.is_empty() {
// No expected actions defined; score based on whether agent responded
if submission.response.is_empty() {
return Ok(BenchScore::fail("no response"));
}
return Ok(BenchScore::partial(
0.5,
"no expected_actions to evaluate against",
));
}
// Check which expected actions were actually called
let called: std::collections::HashSet<&str> =
submission.tool_calls.iter().map(|s| s.as_str()).collect();
let matched = expected_actions
.iter()
.filter(|a| called.contains(a.as_str()))
.count();
let ratio = matched as f64 / expected_actions.len() as f64;
if ratio >= 1.0 {
Ok(BenchScore::pass())
} else if ratio > 0.0 {
Ok(BenchScore::partial(
ratio,
format!(
"{}/{} expected actions completed",
matched,
expected_actions.len()
),
))
} else {
Ok(BenchScore::fail(format!(
"0/{} expected actions completed",
expected_actions.len()
)))
}
}
async fn next_user_message(
&self,
task: &BenchTask,
conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
// Check if we've exceeded max turns
if let Some(max) = task.expected_turns {
let user_turns = conversation
.iter()
.filter(|t| matches!(t.role, crate::suite::TurnRole::User))
.count();
if user_turns >= max {
return Ok(None);
}
}
// Multi-turn simulation requires an LLM to play the customer role.
// Until that's implemented, every scenario is single-turn only.
// TODO: Use LLM to simulate customer based on user_persona.
tracing::warn!(
task_id = %task.id,
"multi-turn simulation not implemented, ending after first turn"
);
Ok(None)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
#[tokio::test]
async fn test_tau_bench_load() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return my order", "expected_actions": ["lookup_order", "process_return"], "max_turns": 3}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
assert_eq!(tasks.len(), 1);
assert_eq!(tasks[0].expected_turns, Some(3));
}
#[tokio::test]
async fn test_tau_bench_scoring() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tau.jsonl");
let mut file = std::fs::File::create(&path).unwrap();
writeln!(
file,
r#"{{"id": "t1", "instruction": "Return order", "expected_actions": ["lookup_order", "process_return"]}}"#
)
.unwrap();
let suite = TauBenchSuite::new(&path, "retail");
let tasks = suite.load_tasks().await.unwrap();
// Partial completion
let submission = TaskSubmission {
response: "I found your order.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string()],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 0.5);
assert_eq!(score.label, "partial");
// Full completion
let submission = TaskSubmission {
response: "Return processed.".to_string(),
conversation: vec![],
tool_calls: vec!["lookup_order".to_string(), "process_return".to_string()],
error: None,
};
let score = suite.score(&tasks[0], &submission).await.unwrap();
assert_eq!(score.value, 1.0);
}
}
-259
View File
@@ -1,259 +0,0 @@
use std::sync::Arc;
use async_trait::async_trait;
use tokio::sync::{Mutex, mpsc};
use tokio_stream::wrappers::ReceiverStream;
use ironclaw::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use ironclaw::error::ChannelError;
use crate::results::TraceToolCall;
use crate::suite::ConversationTurn;
/// Truncate a string to at most `max_bytes` without splitting a UTF-8 character.
fn truncate_str(s: &str, max_bytes: usize) -> &str {
if s.len() <= max_bytes {
return s;
}
let mut end = max_bytes;
while !s.is_char_boundary(end) {
end -= 1;
}
&s[..end]
}
/// Captured state from a benchmark channel run.
#[derive(Debug, Default)]
pub struct ChannelCapture {
/// All responses the agent sent back.
pub responses: Vec<String>,
/// Tool calls observed (name, success, duration_ms).
pub tool_calls: Vec<TraceToolCall>,
/// Full conversation turns for multi-turn scoring.
pub conversation: Vec<ConversationTurn>,
/// Status messages (for debugging).
pub status_log: Vec<String>,
}
/// A headless Channel implementation for benchmarking.
///
/// Modeled after `ReplChannel`: uses mpsc to inject messages and captures
/// all responses and tool status events. Auto-approves tool execution
/// so benchmarks run without user interaction.
pub struct BenchChannel {
/// Sender to inject messages into the agent loop.
msg_tx: mpsc::Sender<IncomingMessage>,
/// Receiver the agent loop reads from (taken once by `start()`).
msg_rx: Mutex<Option<mpsc::Receiver<IncomingMessage>>>,
/// Accumulated capture data.
capture: Arc<Mutex<ChannelCapture>>,
}
impl BenchChannel {
pub fn new() -> (Self, mpsc::Sender<IncomingMessage>) {
let (tx, rx) = mpsc::channel(64);
let channel = Self {
msg_tx: tx.clone(),
msg_rx: Mutex::new(Some(rx)),
capture: Arc::new(Mutex::new(ChannelCapture::default())),
};
(channel, tx)
}
/// Get a handle to the capture data.
pub fn capture(&self) -> Arc<Mutex<ChannelCapture>> {
Arc::clone(&self.capture)
}
}
#[async_trait]
impl Channel for BenchChannel {
fn name(&self) -> &str {
"bench"
}
async fn start(&self) -> Result<MessageStream, ChannelError> {
let rx = self
.msg_rx
.lock()
.await
.take()
.ok_or_else(|| ChannelError::StartupFailed {
name: "bench".to_string(),
reason: "start() already called".to_string(),
})?;
Ok(Box::pin(ReceiverStream::new(rx)))
}
async fn respond(
&self,
_msg: &IncomingMessage,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.responses.push(response.content.clone());
cap.conversation.push(ConversationTurn {
role: crate::suite::TurnRole::Assistant,
content: response.content,
});
Ok(())
}
async fn send_status(
&self,
status: StatusUpdate,
_metadata: &serde_json::Value,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
match status {
StatusUpdate::ToolCompleted { ref name, success } => {
cap.tool_calls.push(TraceToolCall {
name: name.clone(),
duration_ms: 0, // We don't have precise per-tool timing here
success,
});
cap.status_log
.push(format!("tool_completed: {name} success={success}"));
}
StatusUpdate::ApprovalNeeded { ref request_id, .. } => {
// Auto-approve all tools during benchmarks
cap.status_log.push(format!("auto_approved: {request_id}"));
drop(cap); // Release lock before sending
let approval = IncomingMessage::new("bench", "bench-user", "always");
let _ = self.msg_tx.send(approval).await;
return Ok(());
}
StatusUpdate::Thinking(ref msg) => {
cap.status_log.push(format!("thinking: {msg}"));
}
StatusUpdate::ToolStarted { ref name } => {
cap.status_log.push(format!("tool_started: {name}"));
}
StatusUpdate::ToolResult {
ref name,
ref preview,
} => {
cap.status_log.push(format!(
"tool_result: {name} -> {}",
truncate_str(preview, 100)
));
}
StatusUpdate::StreamChunk(_) => {}
StatusUpdate::Status(ref msg) => {
cap.status_log.push(format!("status: {msg}"));
}
StatusUpdate::JobStarted {
ref job_id,
ref title,
..
} => {
cap.status_log
.push(format!("job_started: {job_id} ({title})"));
}
StatusUpdate::AuthRequired {
ref extension_name, ..
} => {
cap.status_log
.push(format!("auth_required: {extension_name} (auto-skipped)"));
}
StatusUpdate::AuthCompleted {
ref extension_name,
success,
..
} => {
cap.status_log.push(format!(
"auth_completed: {extension_name} success={success}"
));
}
}
Ok(())
}
async fn broadcast(
&self,
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let mut cap = self.capture.lock().await;
cap.status_log.push(format!(
"broadcast: {}",
truncate_str(&response.content, 100)
));
Ok(())
}
async fn health_check(&self) -> Result<(), ChannelError> {
Ok(())
}
async fn shutdown(&self) -> Result<(), ChannelError> {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_bench_channel_captures_responses() {
let (channel, _tx) = BenchChannel::new();
let capture = channel.capture();
let msg = IncomingMessage::new("bench", "user", "hello");
let response = OutgoingResponse::text("world");
channel.respond(&msg, response).await.unwrap();
let cap = capture.lock().await;
assert_eq!(cap.responses.len(), 1);
assert_eq!(cap.responses[0], "world");
assert_eq!(cap.conversation.len(), 1);
}
#[tokio::test]
async fn test_bench_channel_auto_approves() {
let (channel, _tx) = BenchChannel::new();
// start() to consume the receiver
let _stream = channel.start().await.unwrap();
let status = StatusUpdate::ApprovalNeeded {
request_id: "req-1".to_string(),
tool_name: "shell".to_string(),
description: "run ls".to_string(),
parameters: serde_json::json!({}),
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
// The approval message was sent through msg_tx,
// which means the stream would receive it.
// We can't easily read from the stream in this test without
// consuming it, but we can verify the status log.
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert!(cap.status_log.iter().any(|s| s.contains("auto_approved")));
}
#[tokio::test]
async fn test_bench_channel_captures_tool_events() {
let (channel, _tx) = BenchChannel::new();
let status = StatusUpdate::ToolCompleted {
name: "echo".to_string(),
success: true,
};
channel
.send_status(status, &serde_json::Value::Null)
.await
.unwrap();
let capture_arc = channel.capture();
let cap = capture_arc.lock().await;
assert_eq!(cap.tool_calls.len(), 1);
assert_eq!(cap.tool_calls[0].name, "echo");
assert!(cap.tool_calls[0].success);
}
}
-205
View File
@@ -1,205 +0,0 @@
use std::path::{Path, PathBuf};
use std::time::Duration;
use serde::Deserialize;
use crate::error::BenchError;
/// Top-level bench configuration, loaded from TOML.
#[derive(Debug, Clone, Deserialize)]
pub struct BenchConfig {
/// Where to write results. Default: "./bench-results".
#[serde(default = "default_results_dir")]
pub results_dir: PathBuf,
/// Per-task timeout. Default: "300s".
#[serde(
default = "default_task_timeout",
deserialize_with = "deserialize_duration"
)]
pub task_timeout: Duration,
/// How many tasks to run in parallel. Default: 1.
#[serde(default = "default_parallelism")]
pub parallelism: usize,
/// Model/config matrix entries. At least one required.
#[serde(default)]
pub matrix: Vec<MatrixEntry>,
/// Suite-specific configuration (passed through to adapter).
#[serde(default = "default_suite_config")]
pub suite_config: toml::Value,
}
/// A single model/config combination to benchmark.
#[derive(Debug, Clone, Deserialize)]
pub struct MatrixEntry {
/// Label for this configuration (used in results).
pub label: String,
/// Model identifier.
#[serde(default)]
pub model: Option<String>,
}
impl BenchConfig {
/// Load from a TOML file.
pub fn from_file(path: &Path) -> Result<Self, BenchError> {
if !path.exists() {
return Err(BenchError::ConfigNotFound {
path: path.to_path_buf(),
});
}
let content = std::fs::read_to_string(path)?;
let config: BenchConfig = toml::from_str(&content)?;
if config.matrix.is_empty() {
return Err(BenchError::Config(
"config must have at least one [[matrix]] entry".to_string(),
));
}
Ok(config)
}
/// Create a minimal config for when no config file is provided.
/// Uses defaults and optional CLI overrides.
pub fn minimal(model: Option<String>) -> Self {
let label = model.as_deref().unwrap_or("default").to_string();
Self {
results_dir: default_results_dir(),
task_timeout: default_task_timeout(),
parallelism: default_parallelism(),
matrix: vec![MatrixEntry { label, model }],
suite_config: toml::Value::Table(toml::map::Map::new()),
}
}
/// Get the suite_config as a generic map for adapter use.
pub fn suite_config_map(&self) -> toml::map::Map<String, toml::Value> {
match &self.suite_config {
toml::Value::Table(map) => map.clone(),
_ => toml::map::Map::new(),
}
}
/// Get a string value from suite_config.
pub fn suite_config_str(&self, key: &str) -> Option<String> {
self.suite_config_map()
.get(key)
.and_then(|v| v.as_str())
.map(|s| s.to_string())
}
}
fn default_suite_config() -> toml::Value {
toml::Value::Table(toml::map::Map::new())
}
fn default_results_dir() -> PathBuf {
PathBuf::from("./bench-results")
}
fn default_task_timeout() -> Duration {
Duration::from_secs(300)
}
fn default_parallelism() -> usize {
1
}
/// Deserialize a duration from a string like "300s", "5m", etc.
fn deserialize_duration<'de, D>(deserializer: D) -> Result<Duration, D::Error>
where
D: serde::Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
parse_duration(&s).map_err(serde::de::Error::custom)
}
fn parse_duration(s: &str) -> Result<Duration, String> {
let s = s.trim();
if let Some(secs) = s.strip_suffix('s') {
secs.trim()
.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid seconds: {e}"))
} else if let Some(mins) = s.strip_suffix('m') {
mins.trim()
.parse::<u64>()
.map(|m| Duration::from_secs(m * 60))
.map_err(|e| format!("invalid minutes: {e}"))
} else {
// Assume seconds if no suffix
s.parse::<u64>()
.map(Duration::from_secs)
.map_err(|e| format!("invalid duration '{s}': {e}"))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_duration() {
assert_eq!(parse_duration("300s").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("5m").unwrap(), Duration::from_secs(300));
assert_eq!(parse_duration("60").unwrap(), Duration::from_secs(60));
}
#[test]
fn test_minimal_config() {
let config = BenchConfig::minimal(Some("test-model".to_string()));
assert_eq!(config.matrix.len(), 1);
assert_eq!(config.matrix[0].label, "test-model");
assert_eq!(config.parallelism, 1);
}
#[test]
fn test_config_rejects_empty_matrix() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("empty.toml");
std::fs::write(
&path,
r#"
results_dir = "./results"
task_timeout = "60s"
"#,
)
.unwrap();
let err = BenchConfig::from_file(&path).unwrap_err();
assert!(
err.to_string().contains("at least one [[matrix]]"),
"got: {err}"
);
}
#[test]
fn test_config_from_toml() {
let toml_str = r#"
results_dir = "./my-results"
task_timeout = "60s"
parallelism = 2
[[matrix]]
label = "fast"
model = "gpt-4o-mini"
[[matrix]]
label = "full"
model = "claude-3-5-sonnet"
[suite_config]
dataset_path = "./data/test.jsonl"
"#;
let config: BenchConfig = toml::from_str(toml_str).unwrap();
assert_eq!(config.results_dir, PathBuf::from("./my-results"));
assert_eq!(config.task_timeout, Duration::from_secs(60));
assert_eq!(config.parallelism, 2);
assert_eq!(config.matrix.len(), 2);
assert_eq!(
config.suite_config_str("dataset_path").unwrap(),
"./data/test.jsonl"
);
}
}
-31
View File
@@ -1,31 +0,0 @@
use std::path::PathBuf;
#[derive(Debug, thiserror::Error)]
pub enum BenchError {
#[error("Config error: {0}")]
Config(String),
#[error("Config file not found: {path}")]
ConfigNotFound { path: PathBuf },
#[error("Suite {name} not found. Available: {available}")]
SuiteNotFound { name: String, available: String },
#[error("Task {task_id} failed: {reason}")]
TaskFailed { task_id: String, reason: String },
#[error("Scoring error for task {task_id}: {reason}")]
Scoring { task_id: String, reason: String },
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
#[error("JSON error: {0}")]
Json(#[from] serde_json::Error),
#[error("TOML parse error: {0}")]
Toml(#[from] toml::de::Error),
#[error("Agent error: {0}")]
Agent(#[from] ironclaw::Error),
}
-251
View File
@@ -1,251 +0,0 @@
use std::sync::Arc;
use std::sync::atomic::{AtomicU32, Ordering};
use std::time::Instant;
use async_trait::async_trait;
use rust_decimal::Decimal;
use rust_decimal::prelude::ToPrimitive;
use tokio::sync::Mutex;
use ironclaw::error::LlmError;
use ironclaw::llm::{
CompletionRequest, CompletionResponse, LlmProvider, ToolCompletionRequest,
ToolCompletionResponse,
};
/// Recorded metrics from a single LLM call.
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct LlmCallRecord {
pub input_tokens: u32,
pub output_tokens: u32,
pub duration_ms: u64,
pub had_tool_calls: bool,
}
/// Wraps an `LlmProvider` to record per-call metrics.
///
/// The wrapper is transparent to the agent: it delegates every call
/// to the inner provider and captures token counts and timings.
pub struct InstrumentedLlm {
inner: Arc<dyn LlmProvider>,
records: Mutex<Vec<LlmCallRecord>>,
total_input_tokens: AtomicU32,
total_output_tokens: AtomicU32,
call_count: AtomicU32,
}
impl InstrumentedLlm {
pub fn new(inner: Arc<dyn LlmProvider>) -> Self {
Self {
inner,
records: Mutex::new(Vec::new()),
total_input_tokens: AtomicU32::new(0),
total_output_tokens: AtomicU32::new(0),
call_count: AtomicU32::new(0),
}
}
/// Take all recorded call metrics, clearing the internal buffer.
pub async fn take_records(&self) -> Vec<LlmCallRecord> {
let mut records = self.records.lock().await;
std::mem::take(&mut *records)
}
/// Snapshot of total tokens without clearing.
pub fn total_input_tokens(&self) -> u32 {
self.total_input_tokens.load(Ordering::Relaxed)
}
pub fn total_output_tokens(&self) -> u32 {
self.total_output_tokens.load(Ordering::Relaxed)
}
pub fn call_count(&self) -> u32 {
self.call_count.load(Ordering::Relaxed)
}
/// Estimated cost using the inner provider's cost-per-token rates.
pub fn estimated_cost(&self) -> f64 {
let (input_rate, output_rate) = self.inner.cost_per_token();
let input_cost =
input_rate * Decimal::from(self.total_input_tokens.load(Ordering::Relaxed));
let output_cost =
output_rate * Decimal::from(self.total_output_tokens.load(Ordering::Relaxed));
let total = input_cost + output_cost;
total.to_f64().unwrap_or(0.0)
}
/// Reset all counters and records.
pub async fn reset(&self) {
self.records.lock().await.clear();
self.total_input_tokens.store(0, Ordering::Relaxed);
self.total_output_tokens.store(0, Ordering::Relaxed);
self.call_count.store(0, Ordering::Relaxed);
}
async fn record(
&self,
input_tokens: u32,
output_tokens: u32,
duration_ms: u64,
had_tool_calls: bool,
) {
self.total_input_tokens
.fetch_add(input_tokens, Ordering::Relaxed);
self.total_output_tokens
.fetch_add(output_tokens, Ordering::Relaxed);
self.call_count.fetch_add(1, Ordering::Relaxed);
self.records.lock().await.push(LlmCallRecord {
input_tokens,
output_tokens,
duration_ms,
had_tool_calls,
});
}
}
#[async_trait]
impl LlmProvider for InstrumentedLlm {
fn model_name(&self) -> &str {
self.inner.model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.inner.cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
false,
)
.await;
Ok(response)
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let start = Instant::now();
let response = self.inner.complete_with_tools(request).await?;
let elapsed = start.elapsed().as_millis() as u64;
let had_tool_calls = !response.tool_calls.is_empty();
self.record(
response.input_tokens,
response.output_tokens,
elapsed,
had_tool_calls,
)
.await;
Ok(response)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
self.inner.list_models().await
}
}
#[cfg(test)]
mod tests {
use super::*;
use ironclaw::llm::{ChatMessage, CompletionRequest, CompletionResponse, FinishReason};
/// Fake LLM that returns a canned response with known token counts.
struct FakeLlm;
#[async_trait]
impl LlmProvider for FakeLlm {
fn model_name(&self) -> &str {
"fake-model"
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(
Decimal::new(3, 6), // $0.000003 per input token
Decimal::new(15, 6), // $0.000015 per output token
)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, LlmError> {
Ok(CompletionResponse {
content: "test response".to_string(),
input_tokens: 100,
output_tokens: 50,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
async fn complete_with_tools(
&self,
_request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
Ok(ToolCompletionResponse {
content: Some("tool response".to_string()),
tool_calls: vec![],
input_tokens: 200,
output_tokens: 100,
finish_reason: FinishReason::Stop,
response_id: None,
})
}
}
#[tokio::test]
async fn test_instrumented_records_metrics() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
assert_eq!(instrumented.total_input_tokens(), 100);
assert_eq!(instrumented.total_output_tokens(), 50);
let records = instrumented.take_records().await;
assert_eq!(records.len(), 1);
assert_eq!(records[0].input_tokens, 100);
assert!(!records[0].had_tool_calls);
}
#[tokio::test]
async fn test_instrumented_cost_calculation() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
// 100 * 0.000003 + 50 * 0.000015 = 0.0003 + 0.00075 = 0.00105
let cost = instrumented.estimated_cost();
assert!((cost - 0.00105).abs() < 0.0001);
}
#[tokio::test]
async fn test_instrumented_reset() {
let inner = Arc::new(FakeLlm);
let instrumented = InstrumentedLlm::new(inner);
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
let _ = instrumented.complete(request).await.unwrap();
assert_eq!(instrumented.call_count(), 1);
instrumented.reset().await;
assert_eq!(instrumented.call_count(), 0);
assert_eq!(instrumented.total_input_tokens(), 0);
let records = instrumented.take_records().await;
assert!(records.is_empty());
}
}
-313
View File
@@ -1,313 +0,0 @@
mod adapters;
mod channel;
mod config;
mod error;
mod instrumented_llm;
mod results;
mod runner;
mod scoring;
mod suite;
use std::path::PathBuf;
use std::sync::Arc;
use clap::{Parser, Subcommand};
use tracing_subscriber::{EnvFilter, layer::SubscriberExt, util::SubscriberInitExt};
use uuid::Uuid;
use crate::config::BenchConfig;
#[derive(Parser)]
#[command(name = "ironclaw-bench", about = "IronClaw benchmarking harness")]
struct Cli {
#[command(subcommand)]
command: Commands,
}
#[derive(Subcommand)]
enum Commands {
/// Run a benchmark suite.
Run {
/// Suite to run (custom, gaia, spot, tau_bench, swe_bench).
#[arg(long)]
suite: String,
/// Path to bench config TOML.
#[arg(long)]
config: Option<PathBuf>,
/// Override model for all matrix entries.
#[arg(long)]
model: Option<String>,
/// Max tasks to run in parallel.
#[arg(long)]
parallelism: Option<usize>,
/// Sample N tasks from the suite (for quick testing).
#[arg(long)]
sample: Option<usize>,
/// Only run these task IDs (comma-separated).
#[arg(long, value_delimiter = ',')]
task_ids: Option<Vec<String>>,
/// Only run tasks with these tags (comma-separated).
#[arg(long, value_delimiter = ',')]
tags: Option<Vec<String>>,
/// Per-task timeout in seconds.
#[arg(long)]
timeout_secs: Option<u64>,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
/// Resume a previous run by ID.
#[arg(long)]
resume: Option<Uuid>,
},
/// Show results for a run.
Results {
/// Run ID or "latest".
#[arg(default_value = "latest")]
run_id: String,
/// Output format.
#[arg(long, default_value = "table")]
format: ResultsFormat,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// Compare two runs.
Compare {
/// Baseline run ID.
baseline: Uuid,
/// Comparison run ID.
comparison: Uuid,
/// Override results directory.
#[arg(long)]
results_dir: Option<PathBuf>,
},
/// List available benchmark suites.
List,
}
#[derive(Clone, Debug, clap::ValueEnum)]
enum ResultsFormat {
Table,
Json,
Csv,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let cli = Cli::parse();
tracing_subscriber::registry()
.with(
EnvFilter::try_from_default_env()
.unwrap_or_else(|_| EnvFilter::new("ironclaw_bench=info,ironclaw=warn")),
)
.with(tracing_subscriber::fmt::layer().with_target(false))
.init();
match cli.command {
Commands::List => {
println!("Available benchmark suites:\n");
for (id, desc) in adapters::KNOWN_SUITES {
println!(" {:<15} {}", id, desc);
}
println!();
}
Commands::Run {
suite,
config: config_path,
model,
parallelism,
sample,
task_ids,
tags,
timeout_secs,
results_dir,
resume,
} => {
// Load or create config
let mut bench_config = if let Some(ref path) = config_path {
BenchConfig::from_file(path)?
} else {
BenchConfig::minimal(model.clone())
};
// Apply CLI overrides
if let Some(p) = parallelism {
bench_config.parallelism = p;
}
if let Some(t) = timeout_secs {
bench_config.task_timeout = std::time::Duration::from_secs(t);
}
if let Some(ref dir) = results_dir {
bench_config.results_dir = dir.clone();
}
// If model override specified and we have matrix entries, update them
if let Some(ref m) = model {
for entry in &mut bench_config.matrix {
entry.model = Some(m.clone());
}
}
// Create suite
let bench_suite = adapters::create_suite(&suite, &bench_config)?;
// Initialize ironclaw LLM provider
let ironclaw_config = ironclaw::Config::from_env().await.map_err(|e| {
anyhow::anyhow!(
"Failed to load ironclaw config: {}. Make sure .env is configured.",
e
)
})?;
let session = ironclaw::llm::create_session_manager(ironclaw::llm::SessionConfig {
auth_base_url: ironclaw_config.llm.nearai.auth_base_url.clone(),
session_path: ironclaw_config.llm.nearai.session_path.clone(),
})
.await;
session.ensure_authenticated().await?;
let llm = ironclaw::llm::create_llm_provider(&ironclaw_config.llm, session)?;
let safety = Arc::new(ironclaw::safety::SafetyLayer::new(&ironclaw_config.safety));
let runner = runner::BenchRunner::new(bench_suite, bench_config.clone(), llm, safety);
// Run for each matrix entry
for matrix_entry in &bench_config.matrix {
let run_id = runner
.run(
matrix_entry,
sample,
task_ids.as_deref(),
tags.as_deref(),
resume,
)
.await?;
println!("Run complete: {}", run_id);
}
}
Commands::Results {
run_id,
format,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let uuid = if run_id == "latest" {
results::find_latest_run(&base)?
.ok_or_else(|| anyhow::anyhow!("No runs found in {}", base.display()))?
} else {
Uuid::parse_str(&run_id)?
};
let json_path = results::run_json_path(&base, uuid);
let jsonl_path = results::tasks_jsonl_path(&base, uuid);
let run = results::read_run_result(&json_path)?;
let tasks = results::read_task_results(&jsonl_path)?;
match format {
ResultsFormat::Table => {
results::print_results_table(&tasks, &run);
}
ResultsFormat::Json => {
let output = serde_json::json!({
"run": run,
"tasks": tasks,
});
println!("{}", serde_json::to_string_pretty(&output)?);
}
ResultsFormat::Csv => {
println!("task_id,score,label,tokens,cost,turns,time_s");
for task in &tasks {
println!(
"{},{:.3},{},{},{:.4},{},{:.1}",
task.task_id,
task.score.value,
task.score.label,
task.trace.input_tokens + task.trace.output_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
}
}
}
Commands::Compare {
baseline,
comparison,
results_dir,
} => {
let base = results_dir.unwrap_or_else(|| PathBuf::from("./bench-results"));
let baseline_run = results::read_run_result(&results::run_json_path(&base, baseline))?;
let comparison_run =
results::read_run_result(&results::run_json_path(&base, comparison))?;
println!("\nComparison: {} vs {}\n", baseline, comparison);
println!(
"{:<20} {:>12} {:>12} {:>10}",
"Metric", "Baseline", "Comparison", "Delta"
);
println!("{}", "-".repeat(58));
let pass_delta = comparison_run.pass_rate - baseline_run.pass_rate;
println!(
"{:<20} {:>11.1}% {:>11.1}% {:>+9.1}%",
"Pass rate",
baseline_run.pass_rate * 100.0,
comparison_run.pass_rate * 100.0,
pass_delta * 100.0,
);
let score_delta = comparison_run.avg_score - baseline_run.avg_score;
println!(
"{:<20} {:>12.3} {:>12.3} {:>+10.3}",
"Avg score", baseline_run.avg_score, comparison_run.avg_score, score_delta,
);
let cost_delta = comparison_run.total_cost_usd - baseline_run.total_cost_usd;
println!(
"{:<20} {:>11.4}$ {:>11.4}$ {:>+9.4}$",
"Total cost",
baseline_run.total_cost_usd,
comparison_run.total_cost_usd,
cost_delta,
);
let time_b = baseline_run.total_wall_time_ms as f64 / 1000.0;
let time_c = comparison_run.total_wall_time_ms as f64 / 1000.0;
println!(
"{:<20} {:>11.1}s {:>11.1}s {:>+9.1}s",
"Total time",
time_b,
time_c,
time_c - time_b,
);
println!(
"{:<20} {:>12} {:>12}",
"Model", baseline_run.model, comparison_run.model,
);
println!();
}
}
Ok(())
}
-473
View File
@@ -1,473 +0,0 @@
use std::collections::HashSet;
use std::io::{BufRead, Write};
use std::path::{Path, PathBuf};
use chrono::{DateTime, Utc};
use uuid::Uuid;
use crate::error::BenchError;
use crate::suite::BenchScore;
/// Metrics from a single task run: LLM usage, timing, tool calls.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct Trace {
pub wall_time_ms: u64,
pub llm_calls: u32,
pub input_tokens: u32,
pub output_tokens: u32,
pub estimated_cost_usd: f64,
pub tool_calls: Vec<TraceToolCall>,
pub turns: u32,
pub hit_iteration_limit: bool,
pub hit_timeout: bool,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TraceToolCall {
pub name: String,
pub duration_ms: u64,
pub success: bool,
}
/// Result of running a single benchmark task.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResult {
pub task_id: String,
pub suite_id: String,
pub score: BenchScore,
pub trace: Trace,
pub response: String,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
pub config_label: String,
#[serde(default)]
pub error: Option<String>,
}
/// Aggregate results for a full benchmark run.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct RunResult {
pub run_id: Uuid,
pub suite_id: String,
pub config_label: String,
pub model: String,
/// Short git commit hash at the time of the run.
#[serde(default)]
pub commit_hash: String,
pub pass_rate: f64,
pub avg_score: f64,
pub total_tasks: usize,
pub completed_tasks: usize,
pub total_cost_usd: f64,
pub total_wall_time_ms: u64,
pub started_at: DateTime<Utc>,
pub finished_at: DateTime<Utc>,
}
impl RunResult {
/// Build aggregate from individual task results.
#[allow(clippy::too_many_arguments)]
pub fn from_tasks(
run_id: Uuid,
suite_id: &str,
config_label: &str,
model: &str,
commit_hash: &str,
total_tasks: usize,
tasks: &[TaskResult],
started_at: DateTime<Utc>,
) -> Self {
let pass_count = tasks.iter().filter(|t| t.score.value >= 1.0).count();
let pass_rate = if tasks.is_empty() {
0.0
} else {
pass_count as f64 / tasks.len() as f64
};
let avg_score = if tasks.is_empty() {
0.0
} else {
tasks.iter().map(|t| t.score.value).sum::<f64>() / tasks.len() as f64
};
let total_cost: f64 = tasks.iter().map(|t| t.trace.estimated_cost_usd).sum();
let total_wall: u64 = tasks.iter().map(|t| t.trace.wall_time_ms).sum();
Self {
run_id,
suite_id: suite_id.to_string(),
config_label: config_label.to_string(),
model: model.to_string(),
commit_hash: commit_hash.to_string(),
pass_rate,
avg_score,
total_tasks,
completed_tasks: tasks.len(),
total_cost_usd: total_cost,
total_wall_time_ms: total_wall,
started_at,
finished_at: Utc::now(),
}
}
}
/// Append a single task result as one JSON line to the JSONL file.
pub fn append_task_result(path: &Path, result: &TaskResult) -> Result<(), BenchError> {
let mut file = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open(path)?;
let line = serde_json::to_string(result)?;
writeln!(file, "{line}")?;
Ok(())
}
/// Overwrite the JSONL file with the given results (used after scoring).
pub fn write_task_results(path: &Path, results: &[TaskResult]) -> Result<(), BenchError> {
let mut file = std::fs::File::create(path)?;
for result in results {
let line = serde_json::to_string(result)?;
writeln!(file, "{line}")?;
}
Ok(())
}
/// Read all task results from a JSONL file.
pub fn read_task_results(path: &Path) -> Result<Vec<TaskResult>, BenchError> {
if !path.exists() {
return Ok(Vec::new());
}
let file = std::fs::File::open(path)?;
let reader = std::io::BufReader::new(file);
let mut results = Vec::new();
for line in reader.lines() {
let line = line?;
let trimmed = line.trim();
if trimmed.is_empty() {
continue;
}
let result: TaskResult = serde_json::from_str(trimmed)?;
results.push(result);
}
Ok(results)
}
/// Write the aggregate run result as JSON.
pub fn write_run_result(path: &Path, result: &RunResult) -> Result<(), BenchError> {
let json = serde_json::to_string_pretty(result)?;
std::fs::write(path, json)?;
Ok(())
}
/// Read the aggregate run result from JSON.
pub fn read_run_result(path: &Path) -> Result<RunResult, BenchError> {
let json = std::fs::read_to_string(path)?;
let result: RunResult = serde_json::from_str(&json)?;
Ok(result)
}
/// Get the set of already-completed task IDs from a JSONL file (for resume).
///
/// Only includes tasks that have been scored (label != "pending"). Tasks that
/// were written but not scored (e.g., from an interrupted run) will be re-executed.
pub fn completed_task_ids(path: &Path) -> Result<HashSet<String>, BenchError> {
let results = read_task_results(path)?;
Ok(results
.into_iter()
.filter(|r| r.score.label != "pending")
.map(|r| r.task_id)
.collect())
}
/// Get the results directory for a specific run.
pub fn run_dir(base: &Path, run_id: Uuid) -> PathBuf {
base.join(run_id.to_string())
}
/// Get the tasks JSONL path for a run.
pub fn tasks_jsonl_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("tasks.jsonl")
}
/// Get the run JSON path for a run.
pub fn run_json_path(base: &Path, run_id: Uuid) -> PathBuf {
run_dir(base, run_id).join("run.json")
}
/// Find the latest run directory by the modification time of its `run.json`.
///
/// Falls back to `tasks.jsonl` mtime, then directory mtime. This avoids the
/// issue where modifying files inside a directory doesn't update the directory's
/// mtime on many filesystems.
pub fn find_latest_run(base: &Path) -> Result<Option<Uuid>, BenchError> {
if !base.exists() {
return Ok(None);
}
let mut entries: Vec<_> = std::fs::read_dir(base)?
.filter_map(|e| e.ok())
.filter(|e| e.file_type().map(|ft| ft.is_dir()).unwrap_or(false))
.filter_map(|e| {
let name = e.file_name().to_string_lossy().to_string();
let uuid = Uuid::parse_str(&name).ok()?;
let dir_path = e.path();
// Prefer run.json mtime, fall back to tasks.jsonl, then directory
let modified = std::fs::metadata(dir_path.join("run.json"))
.and_then(|m| m.modified())
.or_else(|_| {
std::fs::metadata(dir_path.join("tasks.jsonl")).and_then(|m| m.modified())
})
.or_else(|_| e.metadata().and_then(|m| m.modified()))
.ok()?;
Some((uuid, modified))
})
.collect();
entries.sort_by(|a, b| b.1.cmp(&a.1));
Ok(entries.first().map(|(uuid, _)| *uuid))
}
/// Print a summary table of task results.
pub fn print_results_table(tasks: &[TaskResult], run: &RunResult) {
println!();
let commit_suffix = if run.commit_hash.is_empty() {
String::new()
} else {
format!(" | Commit: {}", run.commit_hash)
};
println!(
"Run: {} | Suite: {} | Model: {}{}",
run.run_id, run.suite_id, run.model, commit_suffix
);
println!(
"Pass rate: {:.1}% | Avg score: {:.3} | Tasks: {}/{} | Cost: ${:.4} | Time: {:.1}s",
run.pass_rate * 100.0,
run.avg_score,
run.completed_tasks,
run.total_tasks,
run.total_cost_usd,
run.total_wall_time_ms as f64 / 1000.0,
);
println!();
// Header
println!(
"{:<30} {:>6} {:>7} {:>8} {:>10} {:>6} {:>8}",
"Task ID", "Score", "Label", "Tokens", "Cost", "Turns", "Time"
);
println!("{}", "-".repeat(80));
for task in tasks {
let total_tokens = task.trace.input_tokens + task.trace.output_tokens;
let task_id_display = if task.task_id.len() > 28 {
let truncated: String = task.task_id.chars().take(25).collect();
format!("{truncated}...")
} else {
task.task_id.clone()
};
println!(
"{:<30} {:>6.3} {:>7} {:>8} {:>10.4} {:>6} {:>7.1}s",
task_id_display,
task.score.value,
task.score.label,
total_tokens,
task.trace.estimated_cost_usd,
task.trace.turns,
task.trace.wall_time_ms as f64 / 1000.0,
);
}
println!();
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_run_result_from_tasks() {
let tasks = vec![
TaskResult {
task_id: "t1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 1.0,
label: "pass".to_string(),
details: None,
},
trace: Trace {
wall_time_ms: 1000,
llm_calls: 2,
input_tokens: 100,
output_tokens: 50,
estimated_cost_usd: 0.01,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
TaskResult {
task_id: "t2".to_string(),
suite_id: "custom".to_string(),
score: BenchScore {
value: 0.0,
label: "fail".to_string(),
details: Some("wrong".to_string()),
},
trace: Trace {
wall_time_ms: 2000,
llm_calls: 3,
input_tokens: 200,
output_tokens: 100,
estimated_cost_usd: 0.02,
tool_calls: vec![],
turns: 2,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "wrong answer".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
},
];
let run = RunResult::from_tasks(
Uuid::new_v4(),
"custom",
"default",
"test-model",
"abc1234",
2,
&tasks,
Utc::now(),
);
assert_eq!(run.pass_rate, 0.5);
assert_eq!(run.avg_score, 0.5);
assert_eq!(run.total_tasks, 2);
assert_eq!(run.completed_tasks, 2);
assert!((run.total_cost_usd - 0.03).abs() < f64::EPSILON);
assert_eq!(run.total_wall_time_ms, 3000);
}
#[test]
fn test_jsonl_roundtrip() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "round-trip-test".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 500,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.001,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "hello".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
append_task_result(&path, &result).expect("append");
let loaded = read_task_results(&path).expect("read");
assert_eq!(loaded.len(), 2);
assert_eq!(loaded[0].task_id, "round-trip-test");
}
#[test]
fn test_completed_task_ids() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
let result = TaskResult {
task_id: "unique-id-1".to_string(),
suite_id: "custom".to_string(),
score: BenchScore::pass(),
trace: Trace {
wall_time_ms: 100,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "x".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "test".to_string(),
error: None,
};
append_task_result(&path, &result).expect("append");
let ids = completed_task_ids(&path).expect("ids");
assert!(ids.contains("unique-id-1"));
assert!(!ids.contains("unique-id-2"));
}
#[test]
fn test_write_task_results_overwrites() {
let dir = tempfile::tempdir().expect("temp dir");
let path = dir.path().join("tasks.jsonl");
// Write initial "pending" result via append
let pending = TaskResult {
task_id: "t1".to_string(),
suite_id: "spot".to_string(),
score: BenchScore {
value: 0.0,
label: "pending".to_string(),
details: None,
},
trace: Trace {
wall_time_ms: 100,
llm_calls: 1,
input_tokens: 10,
output_tokens: 5,
estimated_cost_usd: 0.001,
tool_calls: vec![],
turns: 1,
hit_iteration_limit: false,
hit_timeout: false,
},
response: "42".to_string(),
started_at: Utc::now(),
finished_at: Utc::now(),
config_label: "default".to_string(),
error: None,
};
append_task_result(&path, &pending).expect("append");
// Verify pending score
let before = read_task_results(&path).expect("read");
assert_eq!(before.len(), 1);
assert_eq!(before[0].score.label, "pending");
// Overwrite with scored result
let mut scored = pending;
scored.score = BenchScore::pass();
write_task_results(&path, &[scored]).expect("write");
// Verify scored result replaced pending
let after = read_task_results(&path).expect("read");
assert_eq!(after.len(), 1);
assert_eq!(after[0].score.label, "pass");
assert_eq!(after[0].score.value, 1.0);
}
}
-550
View File
@@ -1,550 +0,0 @@
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use std::time::Instant;
use chrono::Utc;
use tokio::sync::Mutex;
use uuid::Uuid;
use ironclaw::agent::{Agent, AgentDeps};
use ironclaw::channels::{ChannelManager, IncomingMessage};
use ironclaw::config::AgentConfig;
use ironclaw::llm::LlmProvider;
use ironclaw::safety::SafetyLayer;
use ironclaw::tools::ToolRegistry;
use crate::channel::BenchChannel;
use crate::config::{BenchConfig, MatrixEntry};
use crate::error::BenchError;
use crate::instrumented_llm::InstrumentedLlm;
use crate::results::{
RunResult, TaskResult, Trace, append_task_result, completed_task_ids, run_dir, run_json_path,
tasks_jsonl_path, write_run_result, write_task_results,
};
use crate::suite::{BenchSuite, BenchTask, ConversationTurn, TaskSubmission, TurnRole};
/// Parameters for running a single task in isolation.
struct TaskRunParams<'a> {
task: &'a BenchTask,
suite_id: &'a str,
config_label: &'a str,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
timeout: std::time::Duration,
additional_tools: &'a [Arc<dyn ironclaw::tools::Tool>],
}
/// Orchestrates benchmark execution: loads tasks, runs agent per task,
/// scores results, writes JSONL output.
pub struct BenchRunner {
suite: Arc<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl BenchRunner {
pub fn new(
suite: Box<dyn BenchSuite>,
config: BenchConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
suite: Arc::from(suite),
config,
llm,
safety,
}
}
/// Run the benchmark for one matrix entry.
///
/// Returns the run_id for result retrieval.
pub async fn run(
&self,
matrix: &MatrixEntry,
sample: Option<usize>,
task_filter: Option<&[String]>,
tag_filter: Option<&[String]>,
resume_run_id: Option<Uuid>,
) -> Result<Uuid, BenchError> {
let run_id = resume_run_id.unwrap_or_else(Uuid::new_v4);
let results_base = &self.config.results_dir;
let dir = run_dir(results_base, run_id);
std::fs::create_dir_all(&dir)?;
let jsonl_path = tasks_jsonl_path(results_base, run_id);
let json_path = run_json_path(results_base, run_id);
// Load completed task IDs for resume support
let completed: HashSet<String> = if resume_run_id.is_some() {
completed_task_ids(&jsonl_path)?
} else {
HashSet::new()
};
if !completed.is_empty() {
tracing::info!(
"Resuming run {}: {} tasks already completed",
run_id,
completed.len()
);
}
// Load all tasks once (used for both execution and scoring)
let all_tasks = self.suite.load_tasks().await?;
let task_index: HashMap<String, BenchTask> = all_tasks
.iter()
.map(|t| (t.id.clone(), t.clone()))
.collect();
// Filter tasks for execution
let mut tasks = all_tasks;
if let Some(ids) = task_filter {
let id_set: HashSet<&str> = ids.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| id_set.contains(t.id.as_str()));
}
if let Some(tags) = tag_filter {
let tag_set: HashSet<&str> = tags.iter().map(|s| s.as_str()).collect();
tasks.retain(|t| t.tags.iter().any(|tag| tag_set.contains(tag.as_str())));
}
// Filter out already-completed tasks
tasks.retain(|t| !completed.contains(&t.id));
// Sample if requested
if let Some(n) = sample {
tasks.truncate(n);
}
let total_tasks = tasks.len() + completed.len();
let model_label = matrix.model.as_deref().unwrap_or(self.llm.model_name());
let commit_hash = git_short_hash();
tracing::info!(
"[{} @ {}] Running {} tasks for suite '{}' (run: {})",
model_label,
commit_hash,
tasks.len(),
self.suite.id(),
run_id
);
let started_at = Utc::now();
let all_results: Arc<Mutex<Vec<TaskResult>>> =
Arc::new(Mutex::new(Vec::with_capacity(tasks.len())));
if self.config.parallelism <= 1 {
// Sequential execution
let additional_tools = self.suite.additional_tools();
for (i, task) in tasks.iter().enumerate() {
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed.len(),
total_tasks,
task.id
);
if let Err(e) = self.suite.setup_task(task).await {
tracing::warn!("setup_task failed for {}: {}", task.id, e);
let result = make_error_result(
task,
self.suite.id(),
&matrix.label,
Utc::now(),
&format!("setup_task failed: {e}"),
);
append_task_result(&jsonl_path, &result)?;
all_results.lock().await.push(result);
continue;
}
let params = TaskRunParams {
task,
suite_id: self.suite.id(),
config_label: &matrix.label,
llm: Arc::clone(&self.llm),
safety: Arc::clone(&self.safety),
timeout: task.timeout.unwrap_or(self.config.task_timeout),
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
if let Err(e) = self.suite.teardown_task(task).await {
tracing::warn!("teardown_task failed for {}: {}", task.id, e);
}
append_task_result(&jsonl_path, &result)?;
all_results.lock().await.push(result);
}
} else {
// Parallel execution with bounded concurrency
let semaphore = Arc::new(tokio::sync::Semaphore::new(self.config.parallelism));
let shared_tools: Arc<[Arc<dyn ironclaw::tools::Tool>]> =
Arc::from(self.suite.additional_tools());
let mut handles = Vec::new();
for (i, task) in tasks.into_iter().enumerate() {
let sem = Arc::clone(&semaphore);
let suite = Arc::clone(&self.suite);
let config_label = matrix.label.clone();
let llm = Arc::clone(&self.llm);
let safety = Arc::clone(&self.safety);
let timeout = task.timeout.unwrap_or(self.config.task_timeout);
let results_ref = Arc::clone(&all_results);
let completed_count = completed.len();
let total = total_tasks;
let additional_tools = Arc::clone(&shared_tools);
handles.push(tokio::spawn(async move {
let _permit = match sem.acquire().await {
Ok(p) => p,
Err(_) => {
tracing::error!("Semaphore closed for task {}", task.id);
return;
}
};
tracing::info!(
"[{}/{}] Running task: {}",
i + 1 + completed_count,
total,
task.id
);
if let Err(e) = suite.setup_task(&task).await {
tracing::warn!("setup_task failed for {}: {}", task.id, e);
let result = make_error_result(
&task,
suite.id(),
&config_label,
Utc::now(),
&format!("setup_task failed: {e}"),
);
results_ref.lock().await.push(result);
return;
}
let suite_id = suite.id().to_string();
let params = TaskRunParams {
task: &task,
suite_id: &suite_id,
config_label: &config_label,
llm,
safety,
timeout,
additional_tools: &additional_tools,
};
let result = run_task_isolated(params).await;
if let Err(e) = suite.teardown_task(&task).await {
tracing::warn!("teardown_task failed for {}: {}", task.id, e);
}
results_ref.lock().await.push(result);
}));
}
for handle in handles {
if let Err(e) = handle.await {
tracing::error!("Task panicked: {}", e);
}
}
// Write all results to JSONL after parallel execution completes.
// This avoids the race condition of concurrent file appends.
let results = all_results.lock().await;
for result in results.iter() {
append_task_result(&jsonl_path, result)?;
}
}
// Score all results using the cached task index
let results = all_results.lock().await;
let mut scored: Vec<TaskResult> = Vec::with_capacity(results.len());
for result in results.iter() {
if let Some(task) = task_index.get(&result.task_id) {
let submission = TaskSubmission {
response: result.response.clone(),
conversation: vec![],
tool_calls: result
.trace
.tool_calls
.iter()
.map(|tc| tc.name.clone())
.collect(),
error: result.error.clone(),
};
match self.suite.score(task, &submission).await {
Ok(score) => {
let mut scored_result = result.clone();
scored_result.score = score;
scored.push(scored_result);
}
Err(e) => {
tracing::warn!("Scoring failed for {}: {}", result.task_id, e);
scored.push(result.clone());
}
}
} else {
scored.push(result.clone());
}
}
// Combine with any previously completed results for the aggregate
let mut all_for_aggregate = crate::results::read_task_results(&jsonl_path)?;
// De-duplicate (prefer the newer scored versions)
let scored_ids: HashSet<String> = scored.iter().map(|r| r.task_id.clone()).collect();
all_for_aggregate.retain(|r| !scored_ids.contains(&r.task_id));
all_for_aggregate.extend(scored);
// Rewrite JSONL with scored results so `results` command shows final scores
write_task_results(&jsonl_path, &all_for_aggregate)?;
let model_name = matrix.model.as_deref().unwrap_or(self.llm.model_name());
let run_result = RunResult::from_tasks(
run_id,
self.suite.id(),
&matrix.label,
model_name,
&commit_hash,
total_tasks,
&all_for_aggregate,
started_at,
);
write_run_result(&json_path, &run_result)?;
tracing::info!(
"[{} @ {}] Run {} complete: {:.1}% pass rate, {:.3} avg score, ${:.4} cost",
model_name,
commit_hash,
run_id,
run_result.pass_rate * 100.0,
run_result.avg_score,
run_result.total_cost_usd,
);
Ok(run_id)
}
}
/// Run a single benchmark task in complete isolation.
///
/// Creates a fresh Agent + BenchChannel + InstrumentedLlm for the task,
/// injects the prompt, waits for the response, and returns the result.
///
/// # Current limitations
///
/// - **Single-turn only**: After the first assistant response, `/quit` is sent.
/// Multi-turn suites (e.g., Tau-bench's `next_user_message()`) are not yet wired.
/// - **Resources not injected**: `BenchTask.resources` (e.g., GAIA file attachments)
/// are not included in the prompt or made available via the workspace.
/// - **Conversation not captured**: `TaskSubmission.conversation` is always empty,
/// which prevents multi-turn scoring hooks from working.
async fn run_task_isolated(params: TaskRunParams<'_>) -> TaskResult {
let TaskRunParams {
task,
suite_id,
config_label,
llm,
safety,
timeout,
additional_tools,
} = params;
let started_at = Utc::now();
let start = Instant::now();
// Wrap LLM with instrumentation
let instrumented = Arc::new(InstrumentedLlm::new(llm));
// Create bench channel
let (bench_channel, msg_tx) = BenchChannel::new();
let capture = bench_channel.capture();
// Build tool registry
let tools = Arc::new(ToolRegistry::new());
tools.register_builtin_tools();
// Register additional suite-specific tools
for tool in additional_tools {
tools.register(Arc::clone(tool)).await;
}
// Build agent config (minimal, headless)
let agent_config = AgentConfig {
name: format!("bench-{}", task.id),
max_parallel_jobs: 1,
job_timeout: timeout,
stuck_threshold: timeout,
repair_check_interval: timeout + std::time::Duration::from_secs(999),
max_repair_attempts: 0,
use_planning: false,
session_idle_timeout: timeout,
allow_local_tools: true,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
};
let cost_guard = Arc::new(ironclaw::agent::cost_guard::CostGuard::new(
ironclaw::agent::cost_guard::CostGuardConfig::default(),
));
let deps = AgentDeps {
store: None,
llm: instrumented.clone() as Arc<dyn LlmProvider>,
cheap_llm: None,
safety,
tools,
workspace: None,
extension_manager: None,
skill_registry: None,
skills_config: ironclaw::config::SkillsConfig::default(),
hooks: Arc::new(ironclaw::hooks::HookRegistry::new()),
cost_guard,
};
let mut channels = ChannelManager::new();
channels.add(Box::new(bench_channel));
let agent = Agent::new(agent_config, deps, channels, None, None, None, None);
// Build the full prompt with context
let full_prompt = if let Some(ref ctx) = task.context {
format!("{}\n\nContext:\n{}", task.prompt, ctx)
} else {
task.prompt.clone()
};
// Inject the task prompt
let incoming = IncomingMessage::new("bench", "bench-user", &full_prompt);
if msg_tx.send(incoming).await.is_err() {
return make_error_result(
task,
suite_id,
config_label,
started_at,
"failed to send prompt",
);
}
// Record prompt in conversation
{
let mut cap = capture.lock().await;
cap.conversation.push(ConversationTurn {
role: TurnRole::User,
content: full_prompt,
});
}
// Run agent with timeout.
// After the first response, send /quit to end the session.
let quit_tx = msg_tx.clone();
let capture_for_quit = Arc::clone(&capture);
let quit_handle = tokio::spawn(async move {
// Poll for first response
loop {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let cap = capture_for_quit.lock().await;
if !cap.responses.is_empty() {
break;
}
}
// Give a small grace period for any final status events
tokio::time::sleep(std::time::Duration::from_millis(200)).await;
let quit = IncomingMessage::new("bench", "bench-user", "/quit");
let _ = quit_tx.send(quit).await;
});
let agent_result = tokio::time::timeout(timeout, agent.run()).await;
quit_handle.abort();
let wall_time = start.elapsed();
let hit_timeout = agent_result.is_err();
if let Ok(Err(e)) = &agent_result {
tracing::warn!("Agent error for task {}: {}", task.id, e);
}
// Extract results from capture
let cap = capture.lock().await;
let response = cap.responses.last().cloned().unwrap_or_default();
let trace = Trace {
wall_time_ms: wall_time.as_millis() as u64,
llm_calls: instrumented.call_count(),
input_tokens: instrumented.total_input_tokens(),
output_tokens: instrumented.total_output_tokens(),
estimated_cost_usd: instrumented.estimated_cost(),
tool_calls: cap.tool_calls.clone(),
turns: cap.responses.len() as u32,
hit_iteration_limit: false,
hit_timeout,
};
let error = if hit_timeout {
Some(format!("timeout after {}s", timeout.as_secs()))
} else if let Ok(Err(e)) = &agent_result {
Some(e.to_string())
} else {
None
};
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore {
value: 0.0,
label: "pending".to_string(),
details: None,
},
trace,
response,
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error,
}
}
fn make_error_result(
task: &BenchTask,
suite_id: &str,
config_label: &str,
started_at: chrono::DateTime<Utc>,
reason: &str,
) -> TaskResult {
TaskResult {
task_id: task.id.clone(),
suite_id: suite_id.to_string(),
score: crate::suite::BenchScore::fail(reason),
trace: Trace {
wall_time_ms: 0,
llm_calls: 0,
input_tokens: 0,
output_tokens: 0,
estimated_cost_usd: 0.0,
tool_calls: vec![],
turns: 0,
hit_iteration_limit: false,
hit_timeout: false,
},
response: String::new(),
started_at,
finished_at: Utc::now(),
config_label: config_label.to_string(),
error: Some(reason.to_string()),
}
}
/// Get the short git commit hash of HEAD, or "unknown" if not in a repo.
fn git_short_hash() -> String {
std::process::Command::new("git")
.args(["rev-parse", "--short", "HEAD"])
.output()
.ok()
.and_then(|o| {
if o.status.success() {
Some(String::from_utf8_lossy(&o.stdout).trim().to_string())
} else {
None
}
})
.unwrap_or_else(|| "unknown".to_string())
}
-113
View File
@@ -1,113 +0,0 @@
use regex::Regex;
use crate::suite::BenchScore;
/// Normalize an answer string for comparison: lowercase, trim whitespace,
/// strip trailing punctuation, collapse internal whitespace.
pub fn normalize_answer(s: &str) -> String {
let trimmed = s.trim().to_lowercase();
let collapsed: String = trimmed.split_whitespace().collect::<Vec<_>>().join(" ");
collapsed.trim_end_matches(['.', ',', ';', '!']).to_string()
}
/// Exact match after normalization.
pub fn exact_match(expected: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected);
let norm_actual = normalize_answer(actual);
if norm_expected == norm_actual {
BenchScore::pass()
} else {
BenchScore::fail(format!(
"expected \"{norm_expected}\", got \"{norm_actual}\""
))
}
}
/// Check if the actual answer contains the expected substring (normalized).
pub fn contains_match(expected_substring: &str, actual: &str) -> BenchScore {
let norm_expected = normalize_answer(expected_substring);
let norm_actual = normalize_answer(actual);
if norm_actual.contains(&norm_expected) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not contain \"{norm_expected}\""))
}
}
/// Check if the actual answer matches a regex pattern.
pub fn regex_match(pattern: &str, actual: &str) -> BenchScore {
match Regex::new(pattern) {
Ok(re) => {
if re.is_match(actual) {
BenchScore::pass()
} else {
BenchScore::fail(format!("response does not match pattern /{pattern}/"))
}
}
Err(e) => BenchScore::fail(format!("invalid regex pattern: {e}")),
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_normalize_answer() {
assert_eq!(normalize_answer(" Hello World. "), "hello world");
assert_eq!(normalize_answer("Yes!"), "yes");
assert_eq!(normalize_answer("42"), "42");
assert_eq!(normalize_answer(" "), "");
}
#[test]
fn test_exact_match_pass() {
let score = exact_match("Hello World", " hello world. ");
assert_eq!(score.value, 1.0);
assert_eq!(score.label, "pass");
}
#[test]
fn test_exact_match_fail() {
let score = exact_match("hello", "world");
assert_eq!(score.value, 0.0);
assert_eq!(score.label, "fail");
}
#[test]
fn test_contains_match_pass() {
let score = contains_match("world", "Hello World!");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_contains_match_fail() {
let score = contains_match("xyz", "Hello World!");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_pass() {
let score = regex_match(r"\d{4}", "The year is 2024.");
assert_eq!(score.value, 1.0);
}
#[test]
fn test_regex_match_fail() {
let score = regex_match(r"\d{4}", "No numbers here.");
assert_eq!(score.value, 0.0);
}
#[test]
fn test_regex_match_invalid_pattern() {
let score = regex_match(r"[invalid", "anything");
assert_eq!(score.value, 0.0);
assert!(
score
.details
.as_deref()
.unwrap_or("")
.contains("invalid regex")
);
}
}
-154
View File
@@ -1,154 +0,0 @@
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use crate::error::BenchError;
/// A single task in a benchmark suite.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct BenchTask {
pub id: String,
pub prompt: String,
#[serde(default)]
pub context: Option<String>,
#[serde(default)]
pub resources: Vec<TaskResource>,
#[serde(default)]
pub tags: Vec<String>,
#[serde(default)]
pub expected_turns: Option<usize>,
#[serde(default)]
pub timeout: Option<Duration>,
#[serde(default)]
pub metadata: serde_json::Value,
}
/// A resource attached to a benchmark task (file, URL, etc.).
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TaskResource {
pub name: String,
pub path: String,
#[serde(default)]
pub resource_type: ResourceType,
}
#[derive(Debug, Clone, Default, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum ResourceType {
#[default]
File,
Url,
Directory,
}
/// What the agent produced for scoring.
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct TaskSubmission {
pub response: String,
pub conversation: Vec<ConversationTurn>,
pub tool_calls: Vec<String>,
pub error: Option<String>,
}
/// A single turn in a multi-turn conversation.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ConversationTurn {
pub role: TurnRole,
pub content: String,
}
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum TurnRole {
User,
Assistant,
System,
}
/// Score for a single task.
#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct BenchScore {
/// 0.0 to 1.0 (1.0 = perfect).
pub value: f64,
/// "pass" / "fail" / "partial".
pub label: String,
#[serde(default)]
pub details: Option<String>,
}
impl BenchScore {
pub fn pass() -> Self {
Self {
value: 1.0,
label: "pass".to_string(),
details: None,
}
}
pub fn fail(details: impl Into<String>) -> Self {
Self {
value: 0.0,
label: "fail".to_string(),
details: Some(details.into()),
}
}
pub fn partial(value: f64, details: impl Into<String>) -> Self {
Self {
value: value.clamp(0.0, 1.0),
label: "partial".to_string(),
details: Some(details.into()),
}
}
}
/// Trait for benchmark suite adapters.
///
/// Each suite (GAIA, Tau-bench, custom, etc.) implements this trait
/// to provide task loading, scoring, and optional lifecycle hooks.
#[async_trait]
#[allow(dead_code)]
pub trait BenchSuite: Send + Sync {
/// Human-readable name (e.g., "GAIA Validation").
fn name(&self) -> &str;
/// Machine ID (e.g., "gaia").
fn id(&self) -> &str;
/// Load all tasks from the suite's data source.
async fn load_tasks(&self) -> Result<Vec<BenchTask>, BenchError>;
/// Score the agent's submission against the expected answer.
async fn score(
&self,
task: &BenchTask,
submission: &TaskSubmission,
) -> Result<BenchScore, BenchError>;
/// Optional: set up environment before running a task (clone repo, init DB, etc.).
async fn setup_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: tear down environment after a task completes.
async fn teardown_task(&self, _task: &BenchTask) -> Result<(), BenchError> {
Ok(())
}
/// Optional: additional tools to register for this suite's tasks.
fn additional_tools(&self) -> Vec<Arc<dyn ironclaw::tools::Tool>> {
vec![]
}
/// Multi-turn: generate next simulated user message based on conversation so far.
/// Return `None` to end the conversation.
async fn next_user_message(
&self,
_task: &BenchTask,
_conversation: &[ConversationTurn],
) -> Result<Option<String>, BenchError> {
Ok(None)
}
}
+92 -1
View File
@@ -10,12 +10,17 @@
//! Prerequisites: rustup target add wasm32-wasip2, cargo install wasm-tools
use std::env;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
let root = PathBuf::from(&manifest_dir);
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
@@ -104,3 +109,89 @@ fn main() {
}
}
}
/// Collect all registry manifests into a single JSON blob at compile time.
///
/// Output: `$OUT_DIR/embedded_catalog.json` with structure:
/// ```json
/// { "tools": [...], "channels": [...], "bundles": {...} }
/// ```
fn embed_registry_catalog(root: &Path) {
use std::fs;
let registry_dir = root.join("registry");
// Rerun if the bundles file changes (per-file watches for tools/channels
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
// No registry dir: write empty catalog
fs::write(
&out_path,
r#"{"tools":[],"channels":[],"bundles":{"bundles":{}}}"#,
)
.unwrap();
return;
}
let mut tools = Vec::new();
let mut channels = Vec::new();
// Collect tool manifests
let tools_dir = registry_dir.join("tools");
if tools_dir.is_dir() {
collect_json_files(&tools_dir, &mut tools);
}
// Collect channel manifests
let channels_dir = registry_dir.join("channels");
if channels_dir.is_dir() {
collect_json_files(&channels_dir, &mut channels);
}
// Read bundles
let bundles_path = registry_dir.join("_bundles.json");
let bundles_raw = if bundles_path.is_file() {
fs::read_to_string(&bundles_path).unwrap_or_else(|_| r#"{"bundles":{}}"#.to_string())
} else {
r#"{"bundles":{}}"#.to_string()
};
// Build the combined JSON
let catalog = format!(
r#"{{"tools":[{}],"channels":[{}],"bundles":{}}}"#,
tools.join(","),
channels.join(","),
bundles_raw,
);
fs::write(&out_path, catalog).unwrap();
}
/// Read all .json files from a directory and push their raw contents into `out`.
fn collect_json_files(dir: &Path, out: &mut Vec<String>) {
use std::fs;
let mut entries: Vec<_> = fs::read_dir(dir)
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| {
e.path().is_file() && e.path().extension().and_then(|x| x.to_str()) == Some("json")
})
.collect();
// Sort for deterministic output
entries.sort_by_key(|e| e.file_name());
for entry in entries {
// Emit per-file watch so Cargo reruns when file contents change
println!("cargo:rerun-if-changed={}", entry.path().display());
if let Ok(content) = fs::read_to_string(entry.path()) {
out.push(content);
}
}
}
+401
View File
@@ -0,0 +1,401 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy",
]
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "discord-channel"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "leb128"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "log"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
[[package]]
name = "semver"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "smallvec"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "spdx"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3e17e880bafaeb362a7b751ec46bdc5b61445a188f80e0606e68167cd540fa3"
dependencies = [
"smallvec",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wasm-encoder"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e913f9242315ca39eff82aee0e19ee7a372155717ff0eb082c741e435ce25ed1"
dependencies = [
"leb128",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "185dfcd27fa5db2e6a23906b54c28199935f71d9a27a1a27b3a88d6fee2afae7"
dependencies = [
"anyhow",
"indexmap",
"serde",
"serde_derive",
"serde_json",
"spdx",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d07b6a3b550fefa1a914b6d54fc175dd11c3392da11eee604e6ffc759805d25"
dependencies = [
"ahash",
"bitflags",
"hashbrown 0.14.5",
"indexmap",
"semver",
]
[[package]]
name = "wit-bindgen"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a2b3e15cd6068f233926e7d8c7c588b2ec4fb7cc7bf3824115e7c7e2a8485a3"
dependencies = [
"wit-bindgen-rt",
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b632a5a0fa2409489bd49c9e6d99fcc61bb3d4ce9d1907d44662e75a28c71172"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rt"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7947d0131c7c9da3f01dfde0ab8bd4c4cf3c5bd49b6dba0ae640f1fa752572ea"
dependencies = [
"bitflags",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4329de4186ee30e2ef30a0533f9b3c123c019a237a7c82d692807bf1b3ee2697"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "177fb7ee1484d113b4792cc480b1ba57664bbc951b42a4beebe573502135b1fc"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b505603761ed400c90ed30261f44a768317348e49f1864e82ecdc3b2744e5627"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae2a7999ed18efe59be8de2db9cb2b7f84d88b27818c79353dfc53131840fe1a"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zerocopy"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+3 -1
View File
@@ -9,7 +9,7 @@ publish = false
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.41.0"
wit-bindgen = "0.36"
[lib]
crate-type = ["cdylib"]
@@ -21,3 +21,5 @@ lto = true
codegen-units = 1
[workspace]
+14 -2
View File
@@ -2,6 +2,15 @@
"type": "channel",
"name": "discord",
"description": "Discord Gateway/Webhook channel for handling slash commands, buttons, and messages",
"setup": {
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -10,7 +19,7 @@
"credentials": {
"discord_bot_token": {
"secret_name": "discord_bot_token",
"location": { "type": "header", "header_name": "Authorization", "prefix": "Bot " },
"location": { "type": "header", "name": "Authorization", "prefix": "Bot " },
"host_patterns": ["discord.com"]
}
},
@@ -34,6 +43,9 @@
}
},
"config": {
"require_signature_verification": true
"require_signature_verification": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+226 -16
View File
@@ -124,12 +124,57 @@ struct DiscordMessageMetadata {
thread_id: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "discord";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct DiscordConfig {
#[serde(default)]
#[allow(dead_code)]
require_signature_verification: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
struct DiscordChannel;
impl Guest for DiscordChannel {
fn on_start(_config_json: String) -> Result<ChannelConfig, String> {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: DiscordConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Discord channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Discord".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -169,16 +214,21 @@ impl Guest for DiscordChannel {
// Application Command (slash command)
2 => {
handle_slash_command(&interaction);
json_response(
200,
serde_json::json!({
"type": 5,
"data": {
"content": "🤔 Thinking..."
}
}),
)
if handle_slash_command(&interaction) {
json_response(200, serde_json::json!({"type": 5}))
} else {
// Permission denied — ephemeral response
json_response(
200,
serde_json::json!({
"type": 4,
"data": {
"content": "You are not authorized to use this bot.",
"flags": 64
}
}),
)
}
}
// Message Component (buttons, selects)
@@ -270,7 +320,8 @@ impl Guest for DiscordChannel {
}
}
fn handle_slash_command(interaction: &DiscordInteraction) {
/// Returns true if the message was emitted, false if permission denied.
fn handle_slash_command(interaction: &DiscordInteraction) -> bool {
let user = interaction
.member
.as_ref()
@@ -287,6 +338,22 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
})
.unwrap_or_default();
// DM if no guild member context (only direct user field set)
let is_dm = interaction.member.is_none();
// Permission check
if !check_sender_permission(
&user_id,
Some(&user_name),
is_dm,
Some(&PairingReplyCtx {
application_id: interaction.application_id.clone(),
token: interaction.token.clone(),
}),
) {
return false;
}
let channel_id = interaction.channel_id.clone().unwrap_or_default();
let command_name = interaction
@@ -322,14 +389,13 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
channel_host::LogLevel::Error,
&format!("Failed to serialize metadata: {}", e),
);
// Attempt to notify user of internal error
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
interaction.application_id, interaction.token
);
let payload = serde_json::json!({
"content": "❌ Internal Error: Failed to process command metadata.",
"flags": 64 // Ephemeral
"flags": 64
});
let _ = channel_host::http_request(
"POST",
@@ -338,7 +404,7 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
Some(&serde_json::to_vec(&payload).unwrap_or_default()),
None,
);
return;
return true; // Error, but not a permission denial
}
};
@@ -349,10 +415,10 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
thread_id: None,
metadata_json,
});
true
}
fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordMessage) {
// Check member first (for server contexts), then user (for DMs)
let user = interaction
.member
.as_ref()
@@ -369,6 +435,11 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
})
.unwrap_or_default();
let is_dm = interaction.member.is_none();
if !check_sender_permission(&user_id, Some(&user_name), is_dm, None) {
return;
}
let channel_id = message.channel_id.clone();
let metadata = DiscordMessageMetadata {
@@ -399,6 +470,145 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Context needed to send a pairing reply via Discord webhook followup.
struct PairingReplyCtx {
application_id: String,
token: String,
}
/// Check if a sender is permitted to interact with the bot.
/// Returns true if allowed, false if denied (pairing reply sent if applicable).
fn check_sender_permission(
user_id: &str,
username: Option<&str>,
is_dm: bool,
reply_ctx: Option<&PairingReplyCtx>,
) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping interaction from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Guild interactions bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender against allow list
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&user_id.to_string())
|| username.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"username": username,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
if let Some(ctx) = reply_ctx {
let _ = send_pairing_reply(ctx, &result.code);
}
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code as an ephemeral Discord followup message.
fn send_pairing_reply(ctx: &PairingReplyCtx, code: &str) -> Result<(), String> {
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
ctx.application_id, ctx.token
);
let payload = serde_json::json!({
"content": format!(
"To pair with this bot, run: `ironclaw pairing approve discord {}`",
code
),
"flags": 64 // Ephemeral — only visible to the sender
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Discord API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
let headers = serde_json::json!({"Content-Type": "application/json"});
+2
View File
@@ -27,3 +27,5 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+18 -1
View File
@@ -2,6 +2,20 @@
"type": "channel",
"name": "slack",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"setup": {
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -33,6 +47,9 @@
}
},
"config": {
"signing_secret_name": "slack_signing_secret"
"signing_secret_name": "slack_signing_secret",
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+167 -6
View File
@@ -104,15 +104,31 @@ struct SlackPostMessageResponse {
ts: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "slack";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct SlackConfig {
/// Name of secret containing signing secret (for verification by host).
/// Parsed from config for forward compatibility; not yet used in WASM
/// (host handles signature verification).
#[serde(default = "default_signing_secret_name")]
#[allow(dead_code)]
signing_secret_name: String,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_signing_secret_name() -> String {
@@ -123,12 +139,30 @@ struct SlackChannel;
impl Guest for SlackChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
// Parse configuration
let _config: SlackConfig = serde_json::from_str(&config_json)
let config: SlackConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Slack channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Slack".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -136,7 +170,7 @@ impl Guest for SlackChannel {
methods: vec!["POST".to_string()],
require_secret: true,
}],
poll: None, // Slack uses push via webhooks, no polling needed
poll: None,
})
}
@@ -280,7 +314,7 @@ impl Guest for SlackChannel {
/// Handle a Slack event and emit message if applicable.
fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Option<String>) {
match event.event_type.as_str() {
// Direct mention of the bot
// Direct mention of the bot (always in a channel, not a DM)
"app_mention" => {
if let (Some(user), Some(channel), Some(text), Some(ts)) = (
event.user,
@@ -288,6 +322,10 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
event.text,
event.ts.clone(),
) {
// app_mention is always in a channel (not DM)
if !check_sender_permission(&user, &channel, false) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -307,6 +345,9 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
) {
// Only process DMs (channel IDs starting with D)
if channel.starts_with('D') {
if !check_sender_permission(&user, &channel, true) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -358,6 +399,126 @@ fn emit_message(
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// For pairing mode, sends a pairing code DM if denied.
fn check_sender_permission(user_id: &str, channel_id: &str, is_dm: bool) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Channel messages bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (Slack events only have user ID, not username)
let is_allowed =
allowed.contains(&"*".to_string()) || allowed.contains(&user_id.to_string());
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"channel_id": channel_id,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
let _ = send_pairing_reply(channel_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via Slack chat.postMessage.
fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
code
),
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
"https://slack.com/api/chat.postMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status == 200 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Slack API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Strip leading bot mention from text.
fn strip_bot_mention(text: &str) -> String {
// Slack mentions look like <@U12345678>
+2 -1
View File
@@ -17,7 +17,6 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# Exclude from parent workspace (this is a standalone WASM component)
[workspace]
[profile.release]
# Optimize for size
@@ -25,3 +24,5 @@ opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+523 -133
View File
@@ -244,6 +244,67 @@ struct TelegramConfig {
struct TelegramChannel;
#[derive(Debug, Clone, PartialEq, Eq)]
enum TelegramStatusAction {
Typing,
Notify(String),
}
const TELEGRAM_STATUS_MAX_CHARS: usize = 600;
fn truncate_status_message(input: &str, max_chars: usize) -> String {
let mut iter = input.chars();
let truncated: String = iter.by_ref().take(max_chars).collect();
if iter.next().is_some() {
format!("{}...", truncated)
} else {
truncated
}
}
fn status_message_for_user(update: &StatusUpdate) -> Option<String> {
let message = update.message.trim();
if message.is_empty() {
None
} else {
Some(truncate_status_message(message, TELEGRAM_STATUS_MAX_CHARS))
}
}
fn get_updates_url(offset: i64, timeout_secs: u32) -> String {
format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout={}&allowed_updates=[\"message\",\"edited_message\"]",
offset, timeout_secs
)
}
fn classify_status_update(update: &StatusUpdate) -> Option<TelegramStatusAction> {
match update.status {
StatusType::Thinking => Some(TelegramStatusAction::Typing),
StatusType::Done | StatusType::Interrupted => None,
// Tool telemetry can be noisy in chat; keep it as typing-only UX.
StatusType::ToolStarted | StatusType::ToolCompleted | StatusType::ToolResult => None,
StatusType::Status => {
let msg = update.message.trim();
if msg.eq_ignore_ascii_case("Done")
|| msg.eq_ignore_ascii_case("Interrupted")
|| msg.eq_ignore_ascii_case("Awaiting approval")
|| msg.eq_ignore_ascii_case("Rejected")
{
None
} else {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
StatusType::ApprovalNeeded
| StatusType::JobStarted
| StatusType::AuthRequired
| StatusType::AuthCompleted => {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
}
impl Guest for TelegramChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
channel_host::log(
@@ -422,20 +483,36 @@ impl Guest for TelegramChannel {
&format!("Polling getUpdates with offset {}", offset),
);
// Build getUpdates URL with parameters
// - offset: Identifier of the first update to be returned
// - timeout: Long polling timeout in seconds (Telegram recommends 30+)
// - allowed_updates: Only get message updates
let url = format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout=30&allowed_updates=[\"message\",\"edited_message\"]",
offset
);
let headers_json = serde_json::json!({}).to_string();
let primary_url = get_updates_url(offset, 30);
let headers = serde_json::json!({});
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll.
// If the TCP connection drops, retry once immediately with a short poll
// so we don't wait a full extra tick (~30s) before delivering updates.
let result = match channel_host::http_request(
"GET",
&primary_url,
&headers_json,
None,
Some(35_000),
) {
Ok(response) => Ok(response),
Err(primary_err) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"getUpdates request failed ({}), retrying once immediately",
primary_err
),
);
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll
let result =
channel_host::http_request("GET", &url, &headers.to_string(), None, Some(35_000));
let retry_url = get_updates_url(offset, 3);
channel_host::http_request("GET", &retry_url, &headers_json, None, Some(8_000))
.map_err(|retry_err| {
format!("primary error: {}; retry error: {}", primary_err, retry_err)
})
}
};
match result {
Ok(response) => {
@@ -516,7 +593,7 @@ impl Guest for TelegramChannel {
let result = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
Some("Markdown"),
);
@@ -539,7 +616,7 @@ impl Guest for TelegramChannel {
let msg_id = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
None,
)
.map_err(|e| format!("Plain-text retry also failed: {}", e))?;
@@ -558,10 +635,10 @@ impl Guest for TelegramChannel {
}
fn on_status(update: StatusUpdate) {
// Only send typing indicator for Thinking status
if !matches!(update.status, StatusType::Thinking) {
return;
}
let action = match classify_status_update(&update) {
Some(action) => action,
None => return,
};
// Parse chat_id from metadata
let metadata: TelegramMessageMetadata = match serde_json::from_str(&update.metadata_json) {
@@ -569,40 +646,68 @@ impl Guest for TelegramChannel {
Err(_) => {
channel_host::log(
channel_host::LogLevel::Debug,
"on_status: no valid Telegram metadata, skipping typing indicator",
"on_status: no valid Telegram metadata, skipping status update",
);
return;
}
};
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
match action {
TelegramStatusAction::Typing => {
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
}
}
TelegramStatusAction::Notify(prompt) => {
// Send user-visible status updates for actionable events.
if let Err(first_err) =
send_message(metadata.chat_id, &prompt, Some(metadata.message_id), None)
{
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Failed to send status reply ({}), retrying without reply context",
first_err
),
);
if let Err(retry_err) = send_message(metadata.chat_id, &prompt, None, None) {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Failed to send status message without reply context: {}",
retry_err
),
);
}
}
}
}
}
@@ -643,15 +748,18 @@ impl std::fmt::Display for SendError {
fn send_message(
chat_id: i64,
text: &str,
reply_to_message_id: i64,
reply_to_message_id: Option<i64>,
parse_mode: Option<&str>,
) -> Result<i64, SendError> {
let mut payload = serde_json::json!({
"chat_id": chat_id,
"text": text,
"reply_to_message_id": reply_to_message_id,
});
if let Some(message_id) = reply_to_message_id {
payload["reply_to_message_id"] = serde_json::Value::Number(message_id.into());
}
if let Some(mode) = parse_mode {
payload["parse_mode"] = serde_json::Value::String(mode.to_string());
}
@@ -831,40 +939,17 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
/// Send a pairing code message to a chat. Used when an unknown user DMs the bot.
fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"chat_id": chat_id,
"text": format!(
send_message(
chat_id,
&format!(
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
code
),
"parse_mode": "Markdown",
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize payload: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
Some("Markdown"),
)
.map(|_| ())
.map_err(|e| e.to_string())
}
// ============================================================================
@@ -1027,33 +1112,17 @@ fn handle_message(message: TelegramMessage) {
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string());
// Clean the message text (strip bot mentions and commands)
let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH).unwrap_or_default();
let cleaned_text = clean_message_text(
let content_to_emit = match content_to_emit_for_agent(
&content,
if bot_username.is_empty() {
None
} else {
Some(bot_username.as_str())
},
);
// Determine what to emit to the agent.
// - `/start` (no args): emit a welcome placeholder so the agent greets the user
// - Other bare `/commands` (e.g. /interrupt, /help): pass the raw command through
// so Submission::parse() can handle it
// - Commands with args (e.g. `/start hello`): cleaned_text already has the args
// - Plain text: pass through as-is
let trimmed_content = content.trim();
let content_to_emit = if trimmed_content.eq_ignore_ascii_case("/start") {
"[User started the bot]".to_string()
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
// Bare control command like /interrupt, /stop, /help — pass through raw
trimmed_content.to_string()
} else if cleaned_text.is_empty() {
return;
} else {
cleaned_text
) {
Some(value) => value,
None => return,
};
// Emit the message to the agent
@@ -1121,6 +1190,31 @@ fn clean_message_text(text: &str, bot_username: Option<&str>) -> String {
result
}
/// Decide which user content should be emitted to the agent loop.
///
/// - `/start` emits a placeholder so the agent can greet the user
/// - bare slash commands are passed through for Submission parsing
/// - empty/mention-only messages are ignored
/// - otherwise cleaned text is emitted
fn content_to_emit_for_agent(content: &str, bot_username: Option<&str>) -> Option<String> {
let cleaned_text = clean_message_text(content, bot_username);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
return Some("[User started the bot]".to_string());
}
if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
return Some(trimmed_content.to_string());
}
if cleaned_text.is_empty() {
return None;
}
Some(cleaned_text)
}
// ============================================================================
// Utilities
// ============================================================================
@@ -1181,62 +1275,126 @@ mod tests {
// Commands with args: command prefix stripped, args returned
assert_eq!(clean_message_text("/start hello", None), "hello");
assert_eq!(clean_message_text("/help me please", None), "me please");
assert_eq!(clean_message_text("/model claude-opus-4-6", None), "claude-opus-4-6");
assert_eq!(
clean_message_text("/model claude-opus-4-6", None),
"claude-opus-4-6"
);
}
/// Tests for the content_to_emit logic in handle_message.
/// Since handle_message uses WASM host calls, we test the decision logic inline.
/// Since handle_message uses WASM host calls, test the extracted decision function.
#[test]
fn test_content_to_emit_logic() {
// Simulates the content_to_emit decision for various inputs.
// This mirrors the logic in handle_message after clean_message_text.
fn resolve_content(content: &str) -> Option<String> {
let cleaned_text = clean_message_text(content, None);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
Some("[User started the bot]".to_string())
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
Some(trimmed_content.to_string())
} else if cleaned_text.is_empty() {
None // would return/skip in handle_message
} else {
Some(cleaned_text)
}
}
// /start → welcome placeholder
assert_eq!(resolve_content("/start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content("/Start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content(" /start "), Some("[User started the bot]".to_string()));
assert_eq!(
content_to_emit_for_agent("/start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent("/Start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent(" /start ", None),
Some("[User started the bot]".to_string())
);
// /start with args → pass args through
assert_eq!(resolve_content("/start hello"), Some("hello".to_string()));
assert_eq!(
content_to_emit_for_agent("/start hello", None),
Some("hello".to_string())
);
// Control commands → pass through raw so Submission::parse() can match
assert_eq!(resolve_content("/interrupt"), Some("/interrupt".to_string()));
assert_eq!(resolve_content("/stop"), Some("/stop".to_string()));
assert_eq!(resolve_content("/help"), Some("/help".to_string()));
assert_eq!(resolve_content("/undo"), Some("/undo".to_string()));
assert_eq!(resolve_content("/redo"), Some("/redo".to_string()));
assert_eq!(resolve_content("/ping"), Some("/ping".to_string()));
assert_eq!(resolve_content("/tools"), Some("/tools".to_string()));
assert_eq!(resolve_content("/compact"), Some("/compact".to_string()));
assert_eq!(resolve_content("/clear"), Some("/clear".to_string()));
assert_eq!(resolve_content("/version"), Some("/version".to_string()));
assert_eq!(
content_to_emit_for_agent("/interrupt", None),
Some("/interrupt".to_string())
);
assert_eq!(
content_to_emit_for_agent("/stop", None),
Some("/stop".to_string())
);
assert_eq!(
content_to_emit_for_agent("/help", None),
Some("/help".to_string())
);
assert_eq!(
content_to_emit_for_agent("/undo", None),
Some("/undo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/redo", None),
Some("/redo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/ping", None),
Some("/ping".to_string())
);
assert_eq!(
content_to_emit_for_agent("/tools", None),
Some("/tools".to_string())
);
assert_eq!(
content_to_emit_for_agent("/compact", None),
Some("/compact".to_string())
);
assert_eq!(
content_to_emit_for_agent("/clear", None),
Some("/clear".to_string())
);
assert_eq!(
content_to_emit_for_agent("/version", None),
Some("/version".to_string())
);
assert_eq!(
content_to_emit_for_agent("/approve", None),
Some("/approve".to_string())
);
assert_eq!(
content_to_emit_for_agent("/always", None),
Some("/always".to_string())
);
assert_eq!(
content_to_emit_for_agent("/deny", None),
Some("/deny".to_string())
);
assert_eq!(
content_to_emit_for_agent("/yes", None),
Some("/yes".to_string())
);
assert_eq!(
content_to_emit_for_agent("/no", None),
Some("/no".to_string())
);
// Commands with args → cleaned text (command stripped)
assert_eq!(resolve_content("/help me please"), Some("me please".to_string()));
assert_eq!(
content_to_emit_for_agent("/help me please", None),
Some("me please".to_string())
);
// Plain text → pass through
assert_eq!(resolve_content("hello world"), Some("hello world".to_string()));
assert_eq!(resolve_content("just text"), Some("just text".to_string()));
assert_eq!(
content_to_emit_for_agent("hello world", None),
Some("hello world".to_string())
);
assert_eq!(
content_to_emit_for_agent("just text", None),
Some("just text".to_string())
);
// Empty / whitespace → skip (None)
assert_eq!(resolve_content(""), None);
assert_eq!(resolve_content(" "), None);
assert_eq!(content_to_emit_for_agent("", None), None);
assert_eq!(content_to_emit_for_agent(" ", None), None);
// Bare @mention without bot → skip
assert_eq!(resolve_content("@botname"), None);
assert_eq!(content_to_emit_for_agent("@botname", None), None);
// With bot username configured: other mentions are preserved.
assert_eq!(
content_to_emit_for_agent("@alice hello", Some("MyBot")),
Some("@alice hello".to_string())
);
}
#[test]
@@ -1317,4 +1475,236 @@ mod tests {
assert_eq!(msg.text, None);
assert_eq!(msg.caption.as_deref(), Some("What's in this image?"));
}
#[test]
fn test_get_updates_url_includes_offset_and_timeout() {
let url = get_updates_url(444_809_884, 30);
assert!(url.contains("offset=444809884"));
assert!(url.contains("timeout=30"));
assert!(url.contains("allowed_updates=[\"message\",\"edited_message\"]"));
}
#[test]
fn test_classify_status_update_thinking() {
let update = StatusUpdate {
status: StatusType::Thinking,
message: "Thinking...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Typing)
);
}
#[test]
fn test_classify_status_update_approval_needed() {
let update = StatusUpdate {
status: StatusType::ApprovalNeeded,
message: "Approval needed for tool 'http_request'".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Approval needed for tool 'http_request'".to_string()
))
);
}
#[test]
fn test_classify_status_update_done_ignored() {
let update = StatusUpdate {
status: StatusType::Done,
message: "Done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_auth_required() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "Authentication required for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication required for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_started_ignored() {
let update = StatusUpdate {
status: StatusType::ToolStarted,
message: "Tool started: http_request".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_tool_completed_ignored() {
let update = StatusUpdate {
status: StatusType::ToolCompleted,
message: "Tool completed: http_request (ok)".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_job_started_notify() {
let update = StatusUpdate {
status: StatusType::JobStarted,
message: "Job started: Daily sync".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Job started: Daily sync".to_string()
))
);
}
#[test]
fn test_classify_status_update_auth_completed_notify() {
let update = StatusUpdate {
status: StatusType::AuthCompleted,
message: "Authentication completed for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication completed for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_result_ignored() {
let update = StatusUpdate {
status: StatusType::ToolResult,
message: "Tool result: http_request ...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_awaiting_approval_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Awaiting approval".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Interrupted,
message: "Interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_done_ignored_case_insensitive() {
let update = StatusUpdate {
status: StatusType::Status,
message: "done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_rejected_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Rejected".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_notify() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Context compaction started".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Context compaction started".to_string()
))
);
}
#[test]
fn test_status_message_for_user_ignores_blank() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: " ".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(status_message_for_user(&update), None);
}
#[test]
fn test_truncate_status_message_appends_ellipsis() {
let input = "abcdefghijklmnopqrstuvwxyz";
let output = truncate_status_message(input, 10);
assert_eq!(output, "abcdefghij...");
}
#[test]
fn test_status_message_for_user_truncates_long_input() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "x".repeat(700),
metadata_json: "{}".to_string(),
};
let msg = status_message_for_user(&update).expect("expected message");
assert!(msg.len() <= TELEGRAM_STATUS_MAX_CHARS + 3);
assert!(msg.ends_with("..."));
}
}
@@ -1 +1,54 @@
{"type":"channel","name":"telegram","description":"Telegram Bot API channel for receiving and responding to Telegram messages","capabilities":{"http":{"allowlist":[{"host":"api.telegram.org","path_prefix":"/bot"}],"credentials":{"telegram_bot":{"secret_name":"telegram_bot_token","location":{"type":"url_path","placeholder":"{TELEGRAM_BOT_TOKEN}"},"host_patterns":["api.telegram.org"]}},"rate_limit":{"requests_per_minute":30,"requests_per_hour":1000}},"secrets":{"allowed_names":["telegram_*"]},"channel":{"allowed_paths":["/webhook/telegram"],"allow_polling":true,"min_poll_interval_ms":30000,"workspace_prefix":"channels/telegram/","emit_rate_limit":{"messages_per_minute":100,"messages_per_hour":5000}}},"config":{"bot_username":null,"owner_id":null,"respond_to_all_group_messages":false,"polling_enabled":false,"poll_interval_ms":30000,"dm_policy":"pairing","allow_from":[]}}
{
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
"setup": {
"required_secrets": [
{
"name": "telegram_bot_token",
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
{ "host": "api.telegram.org", "path_prefix": "/bot" }
],
"credentials": {
"telegram_bot": {
"secret_name": "telegram_bot_token",
"location": { "type": "url_path", "placeholder": "{TELEGRAM_BOT_TOKEN}" },
"host_patterns": ["api.telegram.org"]
}
},
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 1000
}
},
"secrets": {
"allowed_names": ["telegram_*"]
},
"channel": {
"allowed_paths": ["/webhook/telegram"],
"allow_polling": true,
"min_poll_interval_ms": 30000,
"workspace_prefix": "channels/telegram/",
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
}
}
},
"config": {
"bot_username": null,
"owner_id": null,
"respond_to_all_group_messages": false,
"polling_enabled": false,
"poll_interval_ms": 30000,
"dm_policy": "pairing",
"allow_from": []
}
}
+2
View File
@@ -16,3 +16,5 @@ serde_json = "1"
opt-level = "s"
lto = true
strip = true
[workspace]
+191
View File
@@ -226,6 +226,15 @@ struct WhatsAppMessageMetadata {
timestamp: String,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "whatsapp";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct WhatsAppConfig {
@@ -236,6 +245,15 @@ struct WhatsAppConfig {
/// Whether to reply to the original message (thread context)
#[serde(default = "default_reply_to_message")]
reply_to_message: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_api_version() -> String {
@@ -264,6 +282,9 @@ impl Guest for WhatsAppChannel {
WhatsAppConfig {
api_version: default_api_version(),
reply_to_message: default_reply_to_message(),
owner_id: None,
dm_policy: None,
allow_from: None,
}
}
};
@@ -279,6 +300,24 @@ impl Guest for WhatsAppChannel {
// Persist api_version in workspace so on_respond() can read it
let _ = channel_host::workspace_write("channels/whatsapp/api_version", &config.api_version);
// Persist permission config for handle_message
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
// WhatsApp Cloud API is webhook-only, no polling available
Ok(ChannelConfig {
display_name: "WhatsApp".to_string(),
@@ -604,6 +643,15 @@ fn handle_message(
// Look up sender's name from contacts
let user_name = contact_names.get(&message.from).cloned();
// Permission check (WhatsApp is always DM)
if !check_sender_permission(
&message.from,
user_name.as_deref(),
phone_number_id,
) {
return;
}
// Build metadata for response routing
// This is critical - the response handler uses this to know where to send
let metadata = WhatsAppMessageMetadata {
@@ -637,6 +685,149 @@ fn handle_message(
// Utilities
// ============================================================================
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// WhatsApp is always 1-to-1 (DM), so dm_policy always applies.
fn check_sender_permission(
sender_phone: &str,
user_name: Option<&str>,
phone_number_id: &str,
) -> bool {
// 1. Owner check (highest priority)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if sender_phone != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner {} (owner: {})",
sender_phone, owner
),
);
return false;
}
return true;
}
// 2. DM policy (WhatsApp is always DM)
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (phone number or name)
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&sender_phone.to_string())
|| user_name.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"phone": sender_phone,
"name": user_name,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, sender_phone, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for {}: code {}",
sender_phone, result.code
),
);
if result.created {
let _ = send_pairing_reply(sender_phone, phone_number_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via WhatsApp Cloud API.
fn send_pairing_reply(
recipient_phone: &str,
phone_number_id: &str,
code: &str,
) -> Result<(), String> {
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
let url = format!(
"https://graph.facebook.com/{}/{}/messages",
api_version, phone_number_id
);
let payload = serde_json::json!({
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient_phone,
"type": "text",
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
code
)
}
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json",
"Authorization": "Bearer {WHATSAPP_ACCESS_TOKEN}"
});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"WhatsApp API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
@@ -48,6 +48,9 @@
},
"config": {
"api_version": "v18.0",
"reply_to_message": true
"reply_to_message": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+8 -5
View File
@@ -2,12 +2,15 @@
# Do not use placeholder passwords in production.
DATABASE_URL=postgres://ironclaw:CHANGE_ME@localhost:5432/ironclaw
# NEAR AI
NEARAI_SESSION_TOKEN=CHANGE_ME
# NEAR AI Cloud (API key auth, Chat Completions API)
# Get an API key from https://cloud.near.ai
NEARAI_API_KEY=CHANGE_ME
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
NEARAI_API_MODE=chat_completions
NEARAI_BASE_URL=https://cloud-api.near.ai
# Or use NEAR AI Chat (session token auth, Responses API):
# NEARAI_SESSION_TOKEN=sess_...
# NEARAI_BASE_URL=https://private.near.ai
# Agent
AGENT_NAME=ironclaw
+172
View File
@@ -0,0 +1,172 @@
# LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
configurations.
## Provider Overview
| Provider | Backend value | Requires API key | Notes |
|---|---|---|---|
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
| LM Studio | `openai_compatible` | No | Local GUI |
---
## NEAR AI (default)
No additional configuration required. On first run, `ironclaw onboard` opens a browser
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
```env
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
```
---
## Anthropic (Claude)
```env
LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...
```
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
---
## OpenAI (GPT)
```env
LLM_BACKEND=openai
OPENAI_API_KEY=sk-...
```
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
---
## Ollama (local)
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
```env
LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434 # default
```
Pull a model first: `ollama pull llama3.2`
---
## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
### OpenRouter
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
Popular OpenRouter model IDs:
| Model | ID |
|---|---|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
| GPT-4o | `openai/gpt-4o` |
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
### Together AI
[Together AI](https://www.together.ai) provides fast inference for open-source models.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
```
Popular Together AI model IDs:
| Model | ID |
|---|---|
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
### Fireworks AI
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
```
### vLLM / LiteLLM (self-hosted)
For self-hosted inference servers:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
```
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
```
### LM Studio (local GUI)
Start LM Studio's local server, then:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio
```
---
## Using the Setup Wizard
Instead of editing `.env` manually, run the onboarding wizard:
```bash
ironclaw onboard
```
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
The model name is configured in the following step.
+3053
View File
File diff suppressed because it is too large Load Diff
+455
View File
@@ -0,0 +1,455 @@
# Print an optspec for argparse to handle cmd's options that are independent of any subcommand.
function __fish_ironclaw_global_optspecs
string join \n cli-only no-db m/message= c/config= no-onboard h/help V/version
end
function __fish_ironclaw_needs_command
# Figure out if the current invocation already has a command.
set -l cmd (commandline -opc)
set -e cmd[1]
argparse -s (__fish_ironclaw_global_optspecs) -- $cmd 2>/dev/null
or return
if set -q argv[1]
# Also print the command, so this can be used to figure out what it is.
echo $argv[1]
return 1
end
return 0
end
function __fish_ironclaw_using_subcommand
set -l cmd (__fish_ironclaw_needs_command)
test -z "$cmd"
and return 1
contains -- $cmd[1] $argv
end
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -s V -l version -d 'Print version'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_needs_command" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand run" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l skip-auth -d 'Skip authentication (use existing session)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l channels-only -d 'Reconfigure channels only'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand onboard" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and not __fish_seen_subcommand_from init list get set reset path help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s o -l output -d 'Output path (default: ~/.ironclaw/config.toml)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l force -d 'Overwrite existing file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from init" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s f -l filter -d 'Show only settings matching this prefix (e.g., "agent", "heartbeat")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from get" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from set" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from reset" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from path" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand config; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and not __fish_seen_subcommand_from install list remove info auth help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s n -l name -d 'Tool name (defaults to directory/file name)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l capabilities -d 'Path to capabilities JSON file (auto-detected if not specified)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s t -l target -d 'Target directory for installation (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l release -d 'Build in release mode (default: true)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l skip-build -d 'Skip compilation (use existing .wasm file)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s f -l force -d 'Force overwrite if tool already exists'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s d -l dir -d 'Directory to list tools from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s d -l dir -d 'Directory to remove tool from (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from info" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s d -l dir -d 'Directory to look for tool (default: ~/.ironclaw/tools/)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the secret (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand tool; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and not __fish_seen_subcommand_from add remove list auth test toggle help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l client-id -d 'OAuth client ID (if authentication is required)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l auth-url -d 'OAuth authorization URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l token-url -d 'OAuth token URL (optional, can be discovered)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l scopes -d 'Scopes to request (comma-separated)' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l description -d 'Server description' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from add" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from remove" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s v -l verbose -d 'Show detailed information'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s u -l user -d 'User ID for storing the token (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from auth" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s u -l user -d 'User ID for authentication (default: "default")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from test" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l enable -d 'Enable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l disable -d 'Disable the server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from toggle" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand mcp; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and not __fish_seen_subcommand_from search read write tree status help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s l -l limit -d 'Maximum number of results' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from search" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from read" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s a -l append -d 'Append instead of overwrite'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from write" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s d -l depth -d 'Maximum depth to traverse' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from tree" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand memory; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and not __fish_seen_subcommand_from list approve help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l json -d 'Output as JSON'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from list" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from approve" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand pairing; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and not __fish_seen_subcommand_from install start stop status uninstall help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from install" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from start" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from stop" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from uninstall" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand service; and __fish_seen_subcommand_from help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand doctor" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand status" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l shell -d 'The shell to generate completions for' -r -f -a "bash\t''
zsh\t''
fish\t''
powershell\t''
elvish\t''"
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand completion" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l max-iterations -d 'Maximum iterations before stopping' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand worker" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l job-id -d 'Job ID to execute' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l orchestrator-url -d 'URL of the orchestrator\'s internal API' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l max-turns -d 'Maximum agentic turns for Claude Code' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l model -d 'Claude model to use (e.g. "sonnet", "opus")' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s m -l message -d 'Single message mode - send one message and exit' -r
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s c -l config -d 'Configuration file path (optional, uses env vars by default)' -r -F
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l cli-only -d 'Run in interactive CLI mode only (disable other channels)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-db -d 'Skip database connection (for testing)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -l no-onboard -d 'Skip first-run onboarding check'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand claude-bridge" -s h -l help -d 'Print help'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "run" -d 'Run the agent (default if no subcommand given)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "onboard" -d 'Interactive onboarding wizard'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "config" -d 'Manage configuration settings'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "tool" -d 'Manage WASM tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "mcp" -d 'Manage MCP servers (hosted tool providers)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "memory" -d 'Query and manage workspace memory'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "pairing" -d 'DM pairing (approve inbound requests from unknown senders)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "service" -d 'Manage OS service (launchd / systemd)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "doctor" -d 'Probe external dependencies and validate configuration'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "status" -d 'Show system health and diagnostics'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "completion" -d 'Generate shell completion scripts'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "worker" -d 'Run as a sandboxed worker inside a Docker container (internal use). This is invoked automatically by the orchestrator, not by users directly'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "claude-bridge" -d 'Run as a Claude Code bridge inside a Docker container (internal use). Spawns the `claude` CLI and streams output back to the orchestrator'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and not __fish_seen_subcommand_from run onboard config tool mcp memory pairing service doctor status completion worker claude-bridge help" -f -a "help" -d 'Print this message or the help of the given subcommand(s)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "init" -d 'Generate a default config.toml file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "list" -d 'List all settings and their current values'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "get" -d 'Get a specific setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "set" -d 'Set a setting value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "reset" -d 'Reset a setting to its default value'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from config" -f -a "path" -d 'Show the settings storage info'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "install" -d 'Install a WASM tool from source directory or .wasm file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "list" -d 'List installed tools'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "remove" -d 'Remove an installed tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "info" -d 'Show information about a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from tool" -f -a "auth" -d 'Configure authentication for a tool'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "add" -d 'Add an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "remove" -d 'Remove an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "list" -d 'List configured MCP servers'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "auth" -d 'Authenticate with an MCP server (OAuth flow)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "test" -d 'Test connection to an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from mcp" -f -a "toggle" -d 'Enable or disable an MCP server'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "search" -d 'Search workspace memory (hybrid full-text + semantic)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "read" -d 'Read a file from the workspace'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "write" -d 'Write content to a workspace file'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "tree" -d 'Show workspace directory tree'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from memory" -f -a "status" -d 'Show workspace status (document count, index health)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "list" -d 'List pending pairing requests'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from pairing" -f -a "approve" -d 'Approve a pairing request by code'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "install" -d 'Install the OS service (launchd on macOS, systemd on Linux)'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "start" -d 'Start the installed service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "stop" -d 'Stop the running service'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "status" -d 'Show service status'
complete -c ironclaw -n "__fish_ironclaw_using_subcommand help; and __fish_seen_subcommand_from service" -f -a "uninstall" -d 'Uninstall the OS service and remove the unit file'
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

After

Width:  |  Height:  |  Size: 267 KiB

+2027
View File
File diff suppressed because it is too large Load Diff
+42
View File
@@ -0,0 +1,42 @@
{
"bundles": {
"google": {
"display_name": "Google Suite",
"description": "Gmail, Calendar, Drive, Docs, Sheets, Slides",
"extensions": [
"tools/gmail",
"tools/google-calendar",
"tools/google-docs",
"tools/google-drive",
"tools/google-sheets",
"tools/google-slides"
],
"shared_auth": "google_oauth_token"
},
"messaging": {
"display_name": "Messaging Channels",
"description": "Discord, Telegram, Slack, and WhatsApp channels",
"extensions": [
"channels/discord",
"channels/telegram",
"channels/slack",
"channels/whatsapp"
],
"shared_auth": null
},
"default": {
"display_name": "Recommended Set",
"description": "Core tools and channels for a productive setup",
"extensions": [
"tools/github",
"tools/gmail",
"tools/google-calendar",
"tools/google-drive",
"tools/slack-tool",
"channels/telegram",
"channels/slack"
],
"shared_auth": null
}
}
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "discord",
"display_name": "Discord",
"kind": "channel",
"version": "0.1.0",
"description": "Discord Gateway/Webhook channel for slash commands, buttons, and messages",
"keywords": ["messaging", "chat", "discord", "bot"],
"source": {
"dir": "channels-src/discord",
"capabilities": "discord.capabilities.json",
"crate_name": "discord-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Discord",
"secrets": ["discord_bot_token"],
"shared_auth": null,
"setup_url": "https://discord.com/developers/applications"
},
"tags": ["messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "slack",
"display_name": "Slack",
"kind": "channel",
"version": "0.1.0",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"keywords": ["messaging", "chat", "workspace", "slack"],
"source": {
"dir": "channels-src/slack",
"capabilities": "slack.capabilities.json",
"crate_name": "slack-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Slack",
"secrets": ["slack_bot_token", "slack_signing_secret"],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "telegram",
"display_name": "Telegram",
"kind": "channel",
"version": "0.1.0",
"description": "Telegram Bot API channel for receiving and responding to messages",
"keywords": ["messaging", "bot", "chat", "telegram"],
"source": {
"dir": "channels-src/telegram",
"capabilities": "telegram.capabilities.json",
"crate_name": "telegram-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_bot_token"],
"shared_auth": null,
"setup_url": "https://t.me/BotFather"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "whatsapp",
"display_name": "WhatsApp",
"kind": "channel",
"version": "0.1.0",
"description": "WhatsApp Cloud API channel for receiving and responding to messages",
"keywords": ["messaging", "chat", "whatsapp", "meta"],
"source": {
"dir": "channels-src/whatsapp",
"capabilities": "whatsapp.capabilities.json",
"crate_name": "whatsapp-channel"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Meta",
"secrets": ["whatsapp_access_token", "whatsapp_verify_token"],
"shared_auth": null,
"setup_url": "https://developers.facebook.com/apps/"
},
"tags": ["messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "github",
"display_name": "GitHub",
"kind": "tool",
"version": "0.1.0",
"description": "GitHub integration for issues, PRs, repos, and code search",
"keywords": ["git", "code", "issues", "pull-requests", "repositories"],
"source": {
"dir": "tools-src/github",
"capabilities": "github-tool.capabilities.json",
"crate_name": "github-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "GitHub",
"secrets": ["github_token"],
"shared_auth": null,
"setup_url": "https://github.com/settings/tokens"
},
"tags": ["default", "development"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "gmail",
"display_name": "Gmail",
"kind": "tool",
"version": "0.1.0",
"description": "Read, send, and manage Gmail messages and threads",
"keywords": ["email", "google", "mail", "messaging"],
"source": {
"dir": "tools-src/gmail",
"capabilities": "gmail-tool.capabilities.json",
"crate_name": "gmail-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-calendar",
"display_name": "Google Calendar",
"kind": "tool",
"version": "0.1.0",
"description": "Create, read, update, and delete Google Calendar events",
"keywords": ["calendar", "google", "scheduling", "events"],
"source": {
"dir": "tools-src/google-calendar",
"capabilities": "google-calendar-tool.capabilities.json",
"crate_name": "google-calendar-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-docs",
"display_name": "Google Docs",
"kind": "tool",
"version": "0.1.0",
"description": "Create and edit Google Docs documents",
"keywords": ["documents", "google", "writing", "docs"],
"source": {
"dir": "tools-src/google-docs",
"capabilities": "google-docs-tool.capabilities.json",
"crate_name": "google-docs-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-drive",
"display_name": "Google Drive",
"kind": "tool",
"version": "0.1.0",
"description": "Upload, download, search, and manage Google Drive files and folders",
"keywords": ["storage", "google", "files", "drive"],
"source": {
"dir": "tools-src/google-drive",
"capabilities": "google-drive-tool.capabilities.json",
"crate_name": "google-drive-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["default", "google", "storage"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-sheets",
"display_name": "Google Sheets",
"kind": "tool",
"version": "0.1.0",
"description": "Read and write Google Sheets spreadsheet data",
"keywords": ["spreadsheets", "google", "data", "sheets"],
"source": {
"dir": "tools-src/google-sheets",
"capabilities": "google-sheets-tool.capabilities.json",
"crate_name": "google-sheets-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "google-slides",
"display_name": "Google Slides",
"kind": "tool",
"version": "0.1.0",
"description": "Create and edit Google Slides presentations",
"keywords": ["presentations", "google", "slides"],
"source": {
"dir": "tools-src/google-slides",
"capabilities": "google-slides-tool.capabilities.json",
"crate_name": "google-slides-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Google",
"secrets": ["google_oauth_token"],
"shared_auth": "google_oauth_token",
"setup_url": "https://console.cloud.google.com/apis/credentials"
},
"tags": ["google", "productivity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "okta",
"display_name": "Okta",
"kind": "tool",
"version": "0.1.0",
"description": "Okta SSO for user profile, app catalog, and SSO launch links",
"keywords": ["sso", "identity", "authentication", "okta"],
"source": {
"dir": "tools-src/okta",
"capabilities": "okta-tool.capabilities.json",
"crate_name": "okta-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Okta",
"secrets": ["okta_oauth_token"],
"shared_auth": null,
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/"
},
"tags": ["identity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "slack-tool",
"display_name": "Slack",
"kind": "tool",
"version": "0.1.0",
"description": "Post messages, read channels, and manage conversations via Slack API",
"keywords": ["messaging", "chat", "workspace"],
"source": {
"dir": "tools-src/slack",
"capabilities": "slack-tool.capabilities.json",
"crate_name": "slack-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Slack",
"secrets": ["slack_bot_token"],
"shared_auth": null,
"setup_url": "https://api.slack.com/apps"
},
"tags": ["default", "messaging"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "telegram-mtproto",
"display_name": "Telegram",
"kind": "tool",
"version": "0.1.0",
"description": "Telegram user-mode integration via MTProto for messages and contacts",
"keywords": ["messaging", "chat", "telegram", "mtproto"],
"source": {
"dir": "tools-src/telegram",
"capabilities": "telegram-tool.capabilities.json",
"crate_name": "telegram-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Telegram",
"secrets": ["telegram_api_id", "telegram_api_hash"],
"shared_auth": null,
"setup_url": "https://my.telegram.org/apps"
},
"tags": ["messaging"]
}
+106
View File
@@ -0,0 +1,106 @@
---
name: web-ui-test
version: 0.1.0
description: Test the IronClaw web UI using the Claude for Chrome browser extension.
activation:
keywords:
- test web ui
- test the ui
- browser test
- chrome test
- test skills tab
- test chat
- web gateway test
patterns:
- "test.*web.*ui"
- "test.*browser"
- "chrome.*extension.*test"
---
# Web UI Testing with Claude for Chrome
Use this skill when manually testing the IronClaw web gateway UI via the Claude for Chrome browser extension.
## Prerequisites
- IronClaw must be running with `GATEWAY_ENABLED=true`
- Note the gateway URL (default: `http://127.0.0.1:3000/`) and auth token
- The Claude for Chrome extension must be installed and connected
## Starting the Server
```bash
CLI_ENABLED=false GATEWAY_AUTH_TOKEN=<your-token> cargo run
```
Wait for "Agent ironclaw ready and listening" in the logs before proceeding.
## Test Checklist
### 1. Connection
- Navigate to `http://127.0.0.1:3000/?token=<token>`
- Verify "Connected" indicator in the top-right corner
- Verify all tabs are visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### 2. Chat Tab
- Send a simple message (e.g., "Hello, what tools do you have?")
- Verify the LLM responds without errors
- If you see "Invalid schema for function" errors, the tool schema fix (PR #301) may not be merged yet
### 3. Skills Tab
- Click the Skills tab
- Verify "No skills installed" or a list of installed skills (no "Skills system not enabled" error)
- Search for "markdown" in the ClawHub search box
- Verify results appear with: name, version, description, relevance score, "updated X ago"
- Verify skill names are clickable links to clawhub.ai
- If search returns empty with a yellow warning banner, the registry may be unreachable
### 4. Skill Install (from search)
- Search for a skill (e.g., "markdown")
- Click "Install" on a result
- Confirm the install dialog
- Verify success toast appears
- Verify the skill appears in "Installed Skills" section
### 5. Skill Install (by URL)
- Scroll to "Install Skill by URL"
- Enter a skill name and a ClawHub download URL:
- Name: `markdown-viewer`
- URL: `https://wry-manatee-359.convex.site/api/v1/download?slug=markdown-viewer`
- Click Install
- Verify success toast and skill appears in installed list
### 6. Skill Remove
- Find an installed skill
- Click "Remove"
- Confirm removal
- Verify the skill disappears from the installed list
### 7. Other Tabs (smoke test)
- **Memory**: Should show the memory filesystem (may be empty)
- **Jobs**: Should show job list (may be empty)
- **Routines**: Should show routine list
- **Extensions**: Should show extension list with install options
## Cleanup
After testing, remove any test-installed skills:
```bash
rm -rf ~/.ironclaw/installed_skills/<skill-name>
```
Stop the server with Ctrl+C or by killing the process.
## Known Issues
- ClawHub registry at `clawhub.ai` is behind Vercel which blocks non-browser TLS fingerprints; the backend uses `wry-manatee-359.convex.site` directly
- Skill downloads are ZIP archives containing SKILL.md, not raw text
- The `confirm()` dialog for install may block browser automation; override with `window.confirm = () => true` in the console first
+63 -14
View File
@@ -68,6 +68,7 @@ pub struct AgentDeps {
pub workspace: Option<Arc<Workspace>>,
pub extension_manager: Option<Arc<ExtensionManager>>,
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
pub skills_config: SkillsConfig,
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
@@ -85,6 +86,7 @@ pub struct Agent {
pub(super) session_manager: Arc<SessionManager>,
pub(super) context_monitor: ContextMonitor,
pub(super) heartbeat_config: Option<HeartbeatConfig>,
pub(super) hygiene_config: Option<crate::config::HygieneConfig>,
pub(super) routine_config: Option<RoutineConfig>,
}
@@ -93,11 +95,13 @@ impl Agent {
///
/// Optionally accepts pre-created `ContextManager` and `SessionManager` for sharing
/// with external components (job tools, web gateway). Creates new ones if not provided.
#[allow(clippy::too_many_arguments)]
pub fn new(
config: AgentConfig,
deps: AgentDeps,
channels: ChannelManager,
channels: Arc<ChannelManager>,
heartbeat_config: Option<HeartbeatConfig>,
hygiene_config: Option<crate::config::HygieneConfig>,
routine_config: Option<RoutineConfig>,
context_manager: Option<Arc<ContextManager>>,
session_manager: Option<Arc<SessionManager>>,
@@ -120,13 +124,14 @@ impl Agent {
Self {
config,
deps,
channels: Arc::new(channels),
channels,
context_manager,
scheduler,
router: Router::new(),
session_manager,
context_monitor: ContextMonitor::new(),
heartbeat_config,
hygiene_config,
routine_config,
}
}
@@ -170,6 +175,10 @@ impl Agent {
self.deps.skill_registry.as_ref()
}
pub(super) fn skill_catalog(&self) -> Option<&Arc<crate::skills::catalog::SkillCatalog>> {
self.deps.skill_catalog.as_ref()
}
/// Select active skills for a message using deterministic prefiltering.
pub(super) fn select_active_skills(
&self,
@@ -354,14 +363,18 @@ impl Agent {
}
});
tracing::info!(
"Heartbeat enabled with {}s interval",
hb_config.interval_secs
);
let hygiene = self
.hygiene_config
.as_ref()
.map(|h| h.to_workspace_config())
.unwrap_or_default();
Some(spawn_heartbeat(
config,
hygiene,
workspace.clone(),
self.cheap_llm().clone(),
self.safety().clone(),
Some(notify_tx),
))
} else {
@@ -389,6 +402,7 @@ impl Agent {
self.llm().clone(),
Arc::clone(workspace),
notify_tx,
Some(self.scheduler.clone()),
));
// Register routine tools
@@ -491,21 +505,41 @@ impl Agent {
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_content),
}) => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(new_content))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
_ => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(response))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
}
}
Ok(Some(_)) => {
Ok(Some(empty)) => {
// Empty response, nothing to send (e.g. approval handled via send_status)
tracing::debug!(
channel = %message.channel,
user = %message.user_id,
empty_len = empty.len(),
"Suppressed empty response (not sent to channel)"
);
}
Ok(None) => {
// Shutdown signal received (/quit, /exit, /shutdown)
@@ -514,10 +548,17 @@ impl Agent {
}
Err(e) => {
tracing::error!("Error handling message: {}", e);
let _ = self
if let Err(send_err) = self
.channels
.respond(&message, OutgoingResponse::text(format!("Error: {}", e)))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %send_err,
"Failed to send error response to channel"
);
}
}
}
@@ -674,7 +715,15 @@ impl Agent {
// Convert SubmissionResult to response string
match result? {
SubmissionResult::Response { content } => Ok(Some(content)),
SubmissionResult::Response { content } => {
// Suppress silent replies (e.g. from group chat "nothing to say" responses)
if crate::llm::is_silent_reply(&content) {
tracing::debug!("Suppressing silent reply token");
Ok(None)
} else {
Ok(Some(content))
}
}
SubmissionResult::Ok { message } => Ok(message),
SubmissionResult::Error { message } => Ok(Some(format!("Error: {}", message))),
SubmissionResult::Interrupted => Ok(Some("Interrupted.".into())),
+173 -28
View File
@@ -13,7 +13,18 @@ use crate::agent::submission::SubmissionResult;
use crate::agent::{Agent, MessageIntent};
use crate::channels::{IncomingMessage, StatusUpdate};
use crate::error::Error;
use crate::llm::ChatMessage;
use crate::llm::{ChatMessage, Reasoning};
/// Format a count with a suffix, using K/M abbreviations for large numbers.
fn format_count(n: u64, suffix: &str) -> String {
if n >= 1_000_000 {
format!("{:.1}M {}", n as f64 / 1_000_000.0, suffix)
} else if n >= 1_000 {
format!("{:.1}K {}", n as f64 / 1_000.0, suffix)
} else {
format!("{} {}", n, suffix)
}
}
impl Agent {
/// Handle job-related intents without turn tracking.
@@ -73,36 +84,23 @@ impl Agent {
description: String,
category: Option<String>,
) -> Result<String, Error> {
// Create job context
let job_id = self
.context_manager
.create_job_for_user(user_id, &title, &description)
.scheduler
.dispatch_job(user_id, &title, &description, None)
.await?;
// Update category if provided
if let Some(cat) = category {
self.context_manager
// Set the dedicated category field (not stored in metadata)
if let Some(cat) = category
&& let Err(e) = self
.context_manager
.update_context(job_id, |ctx| {
ctx.category = Some(cat);
})
.await?;
}
// Persist new job to database (fire-and-forget)
if let Some(store) = self.store()
&& let Ok(ctx) = self.context_manager.get_context(job_id).await
.await
{
let store = store.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job(&ctx).await {
tracing::warn!("Failed to persist new job {}: {}", job_id, e);
}
});
tracing::warn!(job_id = %job_id, "Failed to set job category: {}", e);
}
// Schedule for execution
self.scheduler.schedule(job_id).await?;
Ok(format!(
"Created job: {}\nID: {}\n\nThe job has been scheduled and is now running.",
title, job_id
@@ -232,8 +230,10 @@ impl Agent {
let runner = crate::agent::HeartbeatRunner::new(
crate::agent::HeartbeatConfig::default(),
crate::workspace::hygiene::HygieneConfig::default(),
workspace.clone(),
self.llm().clone(),
self.safety().clone(),
);
match runner.check_heartbeat().await {
@@ -294,10 +294,11 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.3);
match self.llm().complete(request).await {
Ok(response) => Ok(SubmissionResult::response(format!(
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Thread Summary:\n\n{}",
response.content.trim()
text.trim()
))),
Err(e) => Ok(SubmissionResult::error(format!("Summarize failed: {}", e))),
}
@@ -341,10 +342,11 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.5);
match self.llm().complete(request).await {
Ok(response) => Ok(SubmissionResult::response(format!(
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Suggested Next Steps:\n\n{}",
response.content.trim()
text.trim()
))),
Err(e) => Ok(SubmissionResult::error(format!("Suggest failed: {}", e))),
}
@@ -382,6 +384,10 @@ impl Agent {
" /thread <id> Switch to thread\n",
" /resume <id> Resume from checkpoint\n",
"\n",
"Skills:\n",
" /skills List installed skills\n",
" /skills search <q> Search ClawHub registry\n",
"\n",
"Agent:\n",
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
@@ -414,6 +420,22 @@ impl Agent {
))
}
"skills" => {
if args.first().map(|s| s.as_str()) == Some("search") {
let query = args[1..].join(" ");
if query.is_empty() {
return Ok(SubmissionResult::error("Usage: /skills search <query>"));
}
self.handle_skills_search(&query).await
} else if args.is_empty() {
self.handle_skills_list().await
} else {
Ok(SubmissionResult::error(
"Usage: /skills or /skills search <query>",
))
}
}
"model" => {
let current = self.llm().active_model_name();
@@ -484,6 +506,129 @@ impl Agent {
}
}
/// List installed skills.
async fn handle_skills_list(&self) -> Result<SubmissionResult, Error> {
let Some(registry) = self.skill_registry() else {
return Ok(SubmissionResult::error("Skills system not enabled."));
};
let guard = match registry.read() {
Ok(g) => g,
Err(e) => {
return Ok(SubmissionResult::error(format!(
"Skill registry lock error: {}",
e
)));
}
};
let skills = guard.skills();
if skills.is_empty() {
return Ok(SubmissionResult::response(
"No skills installed.\n\nUse /skills search <query> to find skills on ClawHub.",
));
}
let mut out = String::from("Installed skills:\n\n");
for s in skills {
let desc = if s.manifest.description.chars().count() > 60 {
let truncated: String = s.manifest.description.chars().take(57).collect();
format!("{}...", truncated)
} else {
s.manifest.description.clone()
};
out.push_str(&format!(
" {:<24} v{:<10} [{}] {}\n",
s.manifest.name, s.manifest.version, s.trust, desc,
));
}
out.push_str("\nUse /skills search <query> to find more on ClawHub.");
Ok(SubmissionResult::response(out))
}
/// Search ClawHub for skills.
async fn handle_skills_search(&self, query: &str) -> Result<SubmissionResult, Error> {
let catalog = match self.skill_catalog() {
Some(c) => c,
None => {
return Ok(SubmissionResult::error("Skill catalog not available."));
}
};
let outcome = catalog.search(query).await;
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let mut out = format!("ClawHub results for \"{}\":\n\n", query);
if entries.is_empty() {
if let Some(ref err) = outcome.error {
out.push_str(&format!(" (registry error: {})\n", err));
} else {
out.push_str(" No results found.\n");
}
} else {
for entry in &entries {
let owner_str = entry
.owner
.as_deref()
.map(|o| format!(" by {}", o))
.unwrap_or_default();
let stats_parts: Vec<String> = [
entry.stars.map(|s| format!("{} stars", s)),
entry.downloads.map(|d| format_count(d, "downloads")),
]
.into_iter()
.flatten()
.collect();
let stats_str = if stats_parts.is_empty() {
String::new()
} else {
format!(" {}", stats_parts.join(" "))
};
out.push_str(&format!(
" {:<24} v{:<10}{}{}\n",
entry.name, entry.version, owner_str, stats_str,
));
if !entry.description.is_empty() {
out.push_str(&format!(" {}\n\n", entry.description));
}
}
}
// Show matching installed skills
if let Some(registry) = self.skill_registry()
&& let Ok(guard) = registry.read()
{
let query_lower = query.to_lowercase();
let matches: Vec<_> = guard
.skills()
.iter()
.filter(|s| {
s.manifest.name.to_lowercase().contains(&query_lower)
|| s.manifest.description.to_lowercase().contains(&query_lower)
})
.collect();
if !matches.is_empty() {
out.push_str(&format!("Installed skills matching \"{}\":\n", query));
for s in &matches {
out.push_str(&format!(
" {:<24} v{:<10} [{}]\n",
s.manifest.name, s.manifest.version, s.trust,
));
}
}
}
Ok(SubmissionResult::response(out))
}
/// Handle legacy command routing from the Router (job commands that go through
/// process_user_input -> router -> handle_job_or_command -> here).
pub(super) async fn handle_command(
+28 -7
View File
@@ -12,7 +12,8 @@ use chrono::Utc;
use crate::agent::context_monitor::{CompactionStrategy, ContextBreakdown};
use crate::agent::session::Thread;
use crate::error::Error;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider};
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
/// Result of a compaction operation.
@@ -33,12 +34,13 @@ pub struct CompactionResult {
/// Compacts conversation context to stay within limits.
pub struct ContextCompactor {
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl ContextCompactor {
/// Create a new context compactor.
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
Self { llm, safety }
}
/// Compact a thread's context using the given strategy.
@@ -105,7 +107,16 @@ impl ContextCompactor {
// Write to workspace if available
let summary_written = if let Some(ws) = workspace {
self.write_summary_to_workspace(ws, &summary).await.is_ok()
match self.write_summary_to_workspace(ws, &summary).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"Compaction summary write failed (turns will still be truncated): {}",
e
);
false
}
}
} else {
false
};
@@ -157,7 +168,16 @@ impl ContextCompactor {
let content = format_turns_for_storage(old_turns);
// Write to workspace
let written = self.write_context_to_workspace(ws, &content).await.is_ok();
let written = match self.write_context_to_workspace(ws, &content).await {
Ok(()) => true,
Err(e) => {
tracing::warn!(
"Compaction context write failed (turns will still be truncated): {}",
e
);
false
}
};
// Truncate
thread.truncate_turns(keep_recent);
@@ -213,8 +233,9 @@ Be brief but capture all important details. Use bullet points."#,
.with_max_tokens(1024)
.with_temperature(0.3);
let response = self.llm.complete(request).await?;
Ok(response.content)
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (text, _) = reasoning.complete(request).await?;
Ok(text)
}
/// Write a summary to the workspace daily log.
+75 -9
View File
@@ -4,7 +4,7 @@
//! to prevent runaway agents from burning through API credits. Especially
//! important for daemon/heartbeat modes where the agent acts autonomously.
use std::collections::VecDeque;
use std::collections::{HashMap, VecDeque};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
@@ -53,6 +53,14 @@ impl std::fmt::Display for CostLimitExceeded {
}
}
/// Per-model token usage counters.
#[derive(Debug, Clone, Default)]
pub struct ModelTokens {
pub input_tokens: u64,
pub output_tokens: u64,
pub cost: Decimal,
}
/// Tracks costs and action rates, enforcing configurable limits.
///
/// Thread-safe; designed to be shared via `Arc<CostGuard>`.
@@ -67,6 +75,9 @@ pub struct CostGuard {
/// Flag set when daily budget is exceeded to short-circuit checks.
budget_exceeded: AtomicBool,
/// Per-model token usage since startup.
model_tokens: Mutex<HashMap<String, ModelTokens>>,
}
struct DailyCost {
@@ -85,6 +96,7 @@ impl CostGuard {
}),
action_window: Mutex::new(VecDeque::new()),
budget_exceeded: AtomicBool::new(false),
model_tokens: Mutex::new(HashMap::new()),
}
}
@@ -139,14 +151,19 @@ impl CostGuard {
/// Record a completed LLM action: its token costs and the action timestamp.
///
/// Call this AFTER an LLM call completes so that costs are tracked.
///
/// When `cost_per_token` is `Some`, those rates are used directly (provider-
/// sourced pricing). When `None`, falls back to the static `costs::model_cost`
/// lookup table, then `costs::default_cost`.
pub async fn record_llm_call(
&self,
model: &str,
input_tokens: u32,
output_tokens: u32,
cost_per_token: Option<(Decimal, Decimal)>,
) -> Decimal {
let (input_rate, output_rate) =
costs::model_cost(model).unwrap_or_else(costs::default_cost);
let (input_rate, output_rate) = cost_per_token
.unwrap_or_else(|| costs::model_cost(model).unwrap_or_else(costs::default_cost));
let cost =
input_rate * Decimal::from(input_tokens) + output_rate * Decimal::from(output_tokens);
@@ -192,6 +209,15 @@ impl CostGuard {
window.push_back(Instant::now());
}
// Track per-model token usage
{
let mut tokens = self.model_tokens.lock().await;
let entry = tokens.entry(model.to_string()).or_default();
entry.input_tokens += u64::from(input_tokens);
entry.output_tokens += u64::from(output_tokens);
entry.cost += cost;
}
cost
}
@@ -215,6 +241,11 @@ impl CostGuard {
}
window.len() as u64
}
/// Per-model token usage since startup.
pub async fn model_usage(&self) -> HashMap<String, ModelTokens> {
self.model_tokens.lock().await.clone()
}
}
/// Convert a Decimal USD amount to whole cents (truncated).
@@ -235,7 +266,9 @@ mod tests {
assert!(guard.check_allowed().await.is_ok());
// Record a big call, still allowed
guard.record_llm_call("gpt-4o", 100_000, 100_000).await;
guard
.record_llm_call("gpt-4o", 100_000, 100_000, None)
.await;
assert!(guard.check_allowed().await.is_ok());
}
@@ -252,7 +285,7 @@ mod tests {
// Record a call that costs more than $0.01
// gpt-4o: input=$0.0000025/tok, output=$0.00001/tok
// 10000 input + 10000 output = $0.025 + $0.10 = $0.125
guard.record_llm_call("gpt-4o", 10_000, 10_000).await;
guard.record_llm_call("gpt-4o", 10_000, 10_000, None).await;
// Now should be blocked
let result = guard.check_allowed().await;
@@ -275,7 +308,7 @@ mod tests {
// First 3 actions allowed
for _ in 0..3 {
assert!(guard.check_allowed().await.is_ok());
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
}
// 4th should be blocked
@@ -296,7 +329,7 @@ mod tests {
assert_eq!(guard.daily_spend().await, Decimal::ZERO);
let cost = guard.record_llm_call("gpt-4o", 1000, 500).await;
let cost = guard.record_llm_call("gpt-4o", 1000, 500, None).await;
assert!(cost > Decimal::ZERO);
assert_eq!(guard.daily_spend().await, cost);
}
@@ -307,8 +340,8 @@ mod tests {
assert_eq!(guard.actions_this_hour().await, 0);
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
assert_eq!(guard.actions_this_hour().await, 2);
}
@@ -336,4 +369,37 @@ mod tests {
assert!(rate.to_string().contains("101 actions"));
assert!(rate.to_string().contains("100 allowed"));
}
#[tokio::test]
async fn test_model_usage_per_model_tracking() {
let guard = CostGuard::new(CostGuardConfig::default());
// Initially empty
assert!(guard.model_usage().await.is_empty());
// Record calls for two different models
guard.record_llm_call("gpt-4o", 1000, 500, None).await;
guard.record_llm_call("gpt-4o", 2000, 1000, None).await;
guard
.record_llm_call("claude-3-5-sonnet-20241022", 500, 200, None)
.await;
let usage = guard.model_usage().await;
assert_eq!(usage.len(), 2);
let gpt = usage.get("gpt-4o").expect("gpt-4o should be tracked");
assert_eq!(gpt.input_tokens, 3000);
assert_eq!(gpt.output_tokens, 1500);
assert!(gpt.cost > Decimal::ZERO);
let claude = usage
.get("claude-3-5-sonnet-20241022")
.expect("claude should be tracked");
assert_eq!(claude.input_tokens, 500);
assert_eq!(claude.output_tokens, 200);
assert!(claude.cost > Decimal::ZERO);
// Costs should differ since models have different pricing
assert_ne!(gpt.cost, claude.cost);
}
}
+1025 -299
View File
File diff suppressed because it is too large Load Diff
+32 -13
View File
@@ -29,8 +29,10 @@ use std::time::Duration;
use tokio::sync::mpsc;
use crate::channels::OutgoingResponse;
use crate::llm::{ChatMessage, CompletionRequest, FinishReason, LlmProvider};
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
use crate::workspace::hygiene::HygieneConfig;
/// Configuration for the heartbeat runner.
#[derive(Debug, Clone)]
@@ -96,8 +98,10 @@ pub enum HeartbeatResult {
/// Heartbeat runner for proactive periodic execution.
pub struct HeartbeatRunner {
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
consecutive_failures: u32,
}
@@ -106,13 +110,17 @@ impl HeartbeatRunner {
/// Create a new heartbeat runner.
pub fn new(
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
config,
hygiene_config,
workspace,
llm,
safety,
response_tx: None,
consecutive_failures: 0,
}
@@ -145,6 +153,22 @@ impl HeartbeatRunner {
loop {
interval.tick().await;
// Run memory hygiene in the background so it never delays the
// heartbeat checklist. Failures are logged inside run_if_due.
let hygiene_workspace = Arc::clone(&self.workspace);
let hygiene_config = self.hygiene_config.clone();
tokio::spawn(async move {
let report =
crate::workspace::hygiene::run_if_due(&hygiene_workspace, &hygiene_config)
.await;
if report.had_work() {
tracing::info!(
daily_logs_deleted = report.daily_logs_deleted,
"heartbeat: memory hygiene deleted stale documents"
);
}
});
match self.check_heartbeat().await {
HeartbeatResult::Ok => {
tracing::debug!("Heartbeat OK");
@@ -238,25 +262,18 @@ impl HeartbeatRunner {
.with_max_tokens(max_tokens)
.with_temperature(0.3);
let response = match self.llm.complete(request).await {
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let (content, _usage) = match reasoning.complete(request).await {
Ok(r) => r,
Err(e) => return HeartbeatResult::Failed(format!("LLM call failed: {}", e)),
};
let content = response.content.trim();
let content = content.trim();
// Guard against empty content. Reasoning models (e.g. GLM-4.7) may
// burn all output tokens on chain-of-thought and return content: null.
if content.is_empty() {
return if response.finish_reason == FinishReason::Length {
HeartbeatResult::Failed(
"LLM response was truncated (finish_reason=length) with no content. \
The model may have exhausted its token budget on reasoning."
.to_string(),
)
} else {
HeartbeatResult::Failed("LLM returned empty content.".to_string())
};
return HeartbeatResult::Failed("LLM returned empty content.".to_string());
}
// Check if nothing needs attention
@@ -332,11 +349,13 @@ fn strip_html_comments(content: &str) -> String {
/// Returns a handle that can be used to stop the runner.
pub fn spawn_heartbeat(
config: HeartbeatConfig,
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
) -> tokio::task::JoinHandle<()> {
let mut runner = HeartbeatRunner::new(config, workspace, llm);
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm, safety);
if let Some(tx) = response_tx {
runner = runner.with_response_channel(tx);
}
+1 -1
View File
@@ -44,6 +44,6 @@ pub use self_repair::{BrokenTool, RepairResult, RepairTask, SelfRepair, StuckJob
pub use session::{PendingApproval, PendingAuth, Session, Thread, ThreadState, Turn, TurnState};
pub use session_manager::SessionManager;
pub use submission::{Submission, SubmissionParser, SubmissionResult};
pub use task::{Task, TaskContext, TaskHandler, TaskOutput, TaskStatus};
pub use task::{Task, TaskContext, TaskHandler, TaskOutput};
pub use undo::{Checkpoint, UndoManager};
pub use worker::{Worker, WorkerDeps};
+38 -13
View File
@@ -26,6 +26,8 @@ use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::error::RoutineError;
/// A routine is a named, persistent, user-owned task with a trigger and an action.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Routine {
@@ -86,13 +88,16 @@ impl Trigger {
}
/// Parse a trigger from its DB representation.
pub fn from_db(trigger_type: &str, config: serde_json::Value) -> Result<Self, String> {
pub fn from_db(trigger_type: &str, config: serde_json::Value) -> Result<Self, RoutineError> {
match trigger_type {
"cron" => {
let schedule = config
.get("schedule")
.and_then(|v| v.as_str())
.ok_or("cron trigger missing 'schedule'")?
.ok_or_else(|| RoutineError::MissingField {
context: "cron trigger".into(),
field: "schedule".into(),
})?
.to_string();
Ok(Trigger::Cron { schedule })
}
@@ -100,7 +105,10 @@ impl Trigger {
let pattern = config
.get("pattern")
.and_then(|v| v.as_str())
.ok_or("event trigger missing 'pattern'")?
.ok_or_else(|| RoutineError::MissingField {
context: "event trigger".into(),
field: "pattern".into(),
})?
.to_string();
let channel = config
.get("channel")
@@ -120,7 +128,9 @@ impl Trigger {
Ok(Trigger::Webhook { path, secret })
}
"manual" => Ok(Trigger::Manual),
other => Err(format!("unknown trigger type: {other}")),
other => Err(RoutineError::UnknownTriggerType {
trigger_type: other.to_string(),
}),
}
}
@@ -186,13 +196,16 @@ impl RoutineAction {
}
/// Parse an action from its DB representation.
pub fn from_db(action_type: &str, config: serde_json::Value) -> Result<Self, String> {
pub fn from_db(action_type: &str, config: serde_json::Value) -> Result<Self, RoutineError> {
match action_type {
"lightweight" => {
let prompt = config
.get("prompt")
.and_then(|v| v.as_str())
.ok_or("lightweight action missing 'prompt'")?
.ok_or_else(|| RoutineError::MissingField {
context: "lightweight action".into(),
field: "prompt".into(),
})?
.to_string();
let context_paths = config
.get("context_paths")
@@ -217,12 +230,18 @@ impl RoutineAction {
let title = config
.get("title")
.and_then(|v| v.as_str())
.ok_or("full_job action missing 'title'")?
.ok_or_else(|| RoutineError::MissingField {
context: "full_job action".into(),
field: "title".into(),
})?
.to_string();
let description = config
.get("description")
.and_then(|v| v.as_str())
.ok_or("full_job action missing 'description'")?
.ok_or_else(|| RoutineError::MissingField {
context: "full_job action".into(),
field: "description".into(),
})?
.to_string();
let max_iterations = config
.get("max_iterations")
@@ -235,7 +254,9 @@ impl RoutineAction {
max_iterations,
})
}
other => Err(format!("unknown action type: {other}")),
other => Err(RoutineError::UnknownActionType {
action_type: other.to_string(),
}),
}
}
@@ -334,14 +355,16 @@ impl std::fmt::Display for RunStatus {
}
impl FromStr for RunStatus {
type Err = String;
type Err = RoutineError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"running" => Ok(RunStatus::Running),
"ok" => Ok(RunStatus::Ok),
"attention" => Ok(RunStatus::Attention),
"failed" => Ok(RunStatus::Failed),
other => Err(format!("unknown run status: {other}")),
other => Err(RoutineError::UnknownRunStatus {
status: other.to_string(),
}),
}
}
}
@@ -370,9 +393,11 @@ pub fn content_hash(content: &str) -> u64 {
}
/// Parse a cron expression and compute the next fire time from now.
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, String> {
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, RoutineError> {
let cron_schedule =
cron::Schedule::from_str(schedule).map_err(|e| format!("invalid cron: {e}"))?;
cron::Schedule::from_str(schedule).map_err(|e| RoutineError::InvalidCron {
reason: e.to_string(),
})?;
Ok(cron_schedule.upcoming(Utc).next())
}
+103 -32
View File
@@ -19,12 +19,14 @@ use regex::Regex;
use tokio::sync::{RwLock, mpsc};
use uuid::Uuid;
use crate::agent::Scheduler;
use crate::agent::routine::{
NotifyConfig, Routine, RoutineAction, RoutineRun, RunStatus, Trigger, next_cron_fire,
};
use crate::channels::{IncomingMessage, OutgoingResponse};
use crate::config::RoutineConfig;
use crate::db::Database;
use crate::error::RoutineError;
use crate::llm::{ChatMessage, CompletionRequest, FinishReason, LlmProvider};
use crate::workspace::Workspace;
@@ -40,6 +42,8 @@ pub struct RoutineEngine {
running_count: Arc<AtomicUsize>,
/// Compiled event regex cache: routine_id -> compiled regex.
event_cache: Arc<RwLock<Vec<(Uuid, Routine, Regex)>>>,
/// Scheduler for dispatching jobs (FullJob mode).
scheduler: Option<Arc<Scheduler>>,
}
impl RoutineEngine {
@@ -49,6 +53,7 @@ impl RoutineEngine {
llm: Arc<dyn LlmProvider>,
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
scheduler: Option<Arc<Scheduler>>,
) -> Self {
Self {
config,
@@ -58,6 +63,7 @@ impl RoutineEngine {
notify_tx,
running_count: Arc::new(AtomicUsize::new(0)),
event_cache: Arc::new(RwLock::new(Vec::new())),
scheduler,
}
}
@@ -174,23 +180,26 @@ impl RoutineEngine {
}
/// Fire a routine manually (from tool call or CLI).
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, String> {
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, RoutineError> {
let routine = self
.store
.get_routine(routine_id)
.await
.map_err(|e| format!("DB error: {e}"))?
.ok_or_else(|| format!("routine {routine_id} not found"))?;
.map_err(|e| RoutineError::Database {
reason: e.to_string(),
})?
.ok_or(RoutineError::NotFound { id: routine_id })?;
if !routine.enabled {
return Err(format!("routine '{}' is disabled", routine.name));
return Err(RoutineError::Disabled {
name: routine.name.clone(),
});
}
if !self.check_concurrent(&routine).await {
return Err(format!(
"routine '{}' already at max concurrent runs",
routine.name
));
return Err(RoutineError::MaxConcurrent {
name: routine.name.clone(),
});
}
let run_id = Uuid::new_v4();
@@ -209,7 +218,9 @@ impl RoutineEngine {
};
if let Err(e) = self.store.create_routine_run(&run).await {
return Err(format!("failed to create run record: {e}"));
return Err(RoutineError::Database {
reason: format!("failed to create run record: {e}"),
});
}
// Execute inline for manual triggers (caller wants to wait)
@@ -219,7 +230,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
tokio::spawn(async move {
@@ -251,7 +262,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
// Record the run in DB, then spawn execution
@@ -298,7 +309,7 @@ struct EngineContext {
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
running_count: Arc<AtomicUsize>,
max_lightweight_tokens: u32,
scheduler: Option<Arc<Scheduler>>,
}
/// Execute a routine run. Handles both lightweight and full_job modes.
@@ -312,15 +323,11 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
context_paths,
max_tokens,
} => execute_lightweight(&ctx, &routine, prompt, context_paths, *max_tokens).await,
RoutineAction::FullJob { description, .. } => {
// Full job mode: for now, execute as lightweight with the description
// as prompt. Full scheduler integration will come as a follow-up.
tracing::info!(
routine = %routine.name,
"FullJob mode executing as lightweight (scheduler integration pending)"
);
execute_lightweight(&ctx, &routine, description, &[], ctx.max_lightweight_tokens).await
}
RoutineAction::FullJob {
title,
description,
max_iterations,
} => execute_full_job(&ctx, &routine, &run, title, description, *max_iterations).await,
};
// Decrement running count
@@ -331,7 +338,7 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
Ok(execution) => execution,
Err(e) => {
tracing::error!(routine = %routine.name, "Execution failed: {}", e);
(RunStatus::Failed, Some(e), None)
(RunStatus::Failed, Some(e.to_string()), None)
}
};
@@ -384,6 +391,71 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
.await;
}
/// Sanitize a routine name for use in workspace paths.
/// Only keeps alphanumeric, dash, and underscore characters; replaces everything else.
fn sanitize_routine_name(name: &str) -> String {
name.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect()
}
/// Execute a full-job routine by dispatching to the scheduler.
///
/// Fire-and-forget: creates a job via `Scheduler::dispatch_job` (which handles
/// creation, metadata, persistence, and scheduling), links the routine run to
/// the job, and returns immediately. The job runs independently via the
/// existing Worker/Scheduler with full tool access.
async fn execute_full_job(
ctx: &EngineContext,
routine: &Routine,
run: &RoutineRun,
title: &str,
description: &str,
max_iterations: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let scheduler = ctx
.scheduler
.as_ref()
.ok_or_else(|| RoutineError::JobDispatchFailed {
reason: "scheduler not available".to_string(),
})?;
let metadata = serde_json::json!({ "max_iterations": max_iterations });
let job_id = scheduler
.dispatch_job(&routine.user_id, title, description, Some(metadata))
.await
.map_err(|e| RoutineError::JobDispatchFailed {
reason: format!("failed to dispatch job: {e}"),
})?;
// Link the routine run to the dispatched job
if let Err(e) = ctx.store.link_routine_run_to_job(run.id, job_id).await {
tracing::error!(
routine = %routine.name,
"Failed to link run to job: {}", e
);
}
tracing::info!(
routine = %routine.name,
job_id = %job_id,
max_iterations = max_iterations,
"Dispatched full job for routine"
);
let summary = format!(
"Dispatched job {job_id} for full execution with tool access (max_iterations: {max_iterations})"
);
Ok((RunStatus::Ok, Some(summary), None))
}
/// Execute a lightweight routine (single LLM call).
async fn execute_lightweight(
ctx: &EngineContext,
@@ -391,7 +463,7 @@ async fn execute_lightweight(
prompt: &str,
context_paths: &[String],
max_tokens: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), String> {
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
// Load context from workspace
let mut context_parts = Vec::new();
for path in context_paths {
@@ -408,8 +480,9 @@ async fn execute_lightweight(
}
}
// Load routine state from workspace
let state_path = format!("routines/{}/state.md", routine.name);
// Load routine state from workspace (name sanitized to prevent path traversal)
let safe_name = sanitize_routine_name(&routine.name);
let state_path = format!("routines/{safe_name}/state.md");
let state_content = match ctx.workspace.read(&state_path).await {
Ok(doc) => Some(doc.content),
Err(_) => None,
@@ -469,7 +542,9 @@ async fn execute_lightweight(
.llm
.complete(request)
.await
.map_err(|e| format!("LLM call failed: {e}"))?;
.map_err(|e| RoutineError::LlmFailed {
reason: e.to_string(),
})?;
let content = response.content.trim();
let tokens_used = Some((response.input_tokens + response.output_tokens) as i32);
@@ -477,13 +552,9 @@ async fn execute_lightweight(
// Empty content guard (same as heartbeat)
if content.is_empty() {
return if response.finish_reason == FinishReason::Length {
Err(
"LLM response truncated (finish_reason=length) with no content. \
Model may have exhausted token budget on reasoning."
.to_string(),
)
Err(RoutineError::TruncatedResponse)
} else {
Err("LLM returned empty content.".to_string())
Err(RoutineError::EmptyResponse)
};
}
+56 -4
View File
@@ -81,6 +81,50 @@ impl Scheduler {
}
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
/// 1. Creates the job context via `ContextManager`
/// 2. Optionally applies metadata (e.g. `max_iterations`)
/// 3. Persists the job to the database (so FK references from
/// `job_actions` / `llm_calls` work immediately)
/// 4. Schedules the job for worker execution
///
/// Returns the new job ID.
pub async fn dispatch_job(
&self,
user_id: &str,
title: &str,
description: &str,
metadata: Option<serde_json::Value>,
) -> Result<Uuid, JobError> {
let job_id = self
.context_manager
.create_job_for_user(user_id, title, description)
.await?;
// Apply metadata if provided
if let Some(meta) = metadata {
self.context_manager
.update_context(job_id, |ctx| {
ctx.metadata = meta;
})
.await?;
}
// Persist to DB before scheduling so the worker's FK references are valid
if let Some(ref store) = self.store {
let ctx = self.context_manager.get_context(job_id).await?;
store.save_job(&ctx).await.map_err(|e| JobError::Failed {
id: job_id,
reason: format!("failed to persist job: {e}"),
})?;
}
self.schedule(job_id).await?;
Ok(job_id)
}
/// Schedule a job for execution.
pub async fn schedule(&self, job_id: Uuid) -> Result<(), JobError> {
// Hold write lock for the entire check-insert sequence to prevent
@@ -136,7 +180,9 @@ impl Scheduler {
});
// Start the worker
let _ = tx.send(WorkerMessage::Start).await;
if tx.send(WorkerMessage::Start).await.is_err() {
tracing::error!(job_id = %job_id, "Worker died before receiving Start message");
}
// Insert while still holding the write lock
jobs.insert(job_id, ScheduledJob { handle, tx });
@@ -355,7 +401,7 @@ impl Scheduler {
.into());
}
if tool.requires_approval() {
if tool.requires_approval(&params).is_required() {
return Err(crate::error::ToolError::AuthRequired {
name: tool_name.to_string(),
}
@@ -418,10 +464,16 @@ impl Scheduler {
// Update job state
self.context_manager
.update_context(job_id, |ctx| {
let _ = ctx.transition_to(
if let Err(e) = ctx.transition_to(
JobState::Cancelled,
Some("Stopped by scheduler".to_string()),
);
) {
tracing::warn!(
job_id = %job_id,
error = %e,
"Failed to transition job to Cancelled state"
);
}
})
.await?;
+8 -6
View File
@@ -66,12 +66,14 @@ pub trait SelfRepair: Send + Sync {
/// Default self-repair implementation.
pub struct DefaultSelfRepair {
context_manager: Arc<ContextManager>,
#[allow(dead_code)] // Will be used for time-based stuck detection
// TODO: use for time-based stuck detection (currently only max_repair_attempts is checked)
#[allow(dead_code)]
stuck_threshold: Duration,
max_repair_attempts: u32,
store: Option<Arc<dyn Database>>,
builder: Option<Arc<dyn SoftwareBuilder>>,
#[allow(dead_code)] // Will be used for tool hot-reload after repair
// TODO: use for tool hot-reload after repair
#[allow(dead_code)]
tools: Option<Arc<ToolRegistry>>,
}
@@ -93,15 +95,15 @@ impl DefaultSelfRepair {
}
/// Add a Store for tool failure tracking.
#[allow(dead_code)] // Public API for configuring repair with persistence
pub fn with_store(mut self, store: Arc<dyn Database>) -> Self {
#[allow(dead_code)] // TODO: wire up in main.rs when persistence is needed
pub(crate) fn with_store(mut self, store: Arc<dyn Database>) -> Self {
self.store = Some(store);
self
}
/// Add a Builder and ToolRegistry for automatic tool repair.
#[allow(dead_code)] // Public API for enabling automatic tool repair
pub fn with_builder(
#[allow(dead_code)] // TODO: wire up in main.rs when auto-repair is needed
pub(crate) fn with_builder(
mut self,
builder: Arc<dyn SoftwareBuilder>,
tools: Arc<ToolRegistry>,
+19 -16
View File
@@ -70,10 +70,9 @@ impl Session {
pub fn create_thread(&mut self) -> &mut Thread {
let thread = Thread::new(self.id);
let thread_id = thread.id;
self.threads.insert(thread_id, thread);
self.active_thread = Some(thread_id);
self.last_active_at = Utc::now();
self.threads.get_mut(&thread_id).expect("just inserted")
self.threads.entry(thread_id).or_insert(thread)
}
/// Get the active thread.
@@ -88,10 +87,19 @@ impl Session {
/// Get or create the active thread.
pub fn get_or_create_thread(&mut self) -> &mut Thread {
if self.active_thread.is_none() {
self.create_thread();
match self.active_thread {
None => self.create_thread(),
Some(id) => {
if self.threads.contains_key(&id) {
// Safe: contains_key confirmed the entry exists.
self.threads.get_mut(&id).unwrap()
} else {
// Stale active_thread ID: create a new thread, which
// updates self.active_thread to the new thread's ID.
self.create_thread()
}
}
}
self.active_thread_mut().expect("just created")
}
/// Switch to a different thread.
@@ -177,10 +185,6 @@ pub struct Thread {
/// Pending auth token request (thread is in auth mode).
#[serde(default)]
pub pending_auth: Option<PendingAuth>,
/// Last NEAR AI response ID for response chaining. Persisted to DB
/// metadata so we can resume chaining across restarts.
#[serde(default)]
pub last_response_id: Option<String>,
}
impl Thread {
@@ -197,7 +201,6 @@ impl Thread {
metadata: serde_json::Value::Null,
pending_approval: None,
pending_auth: None,
last_response_id: None,
}
}
@@ -214,7 +217,6 @@ impl Thread {
metadata: serde_json::Value::Null,
pending_approval: None,
pending_auth: None,
last_response_id: None,
}
}
@@ -240,7 +242,8 @@ impl Thread {
self.turns.push(turn);
self.state = ThreadState::Processing;
self.updated_at = Utc::now();
self.turns.last_mut().expect("just pushed")
// turn_number was len() before push, so it's a valid index after push
&mut self.turns[turn_number]
}
/// Complete the current turn with a response.
@@ -353,8 +356,10 @@ impl Thread {
if let Some(next) = iter.peek()
&& next.role == crate::llm::Role::Assistant
{
let response = iter.next().expect("peeked");
turn.complete(&response.content);
// iter.next() is guaranteed Some after a successful peek()
if let Some(response) = iter.next() {
turn.complete(&response.content);
}
}
self.turns.push(turn);
@@ -852,7 +857,6 @@ mod tests {
thread.start_turn("hello");
thread.complete_turn("world");
thread.last_response_id = Some("resp_abc123".to_string());
let json = serde_json::to_string(&thread).unwrap();
let restored: Thread = serde_json::from_str(&json).unwrap();
@@ -862,7 +866,6 @@ mod tests {
assert_eq!(restored.turns.len(), 1);
assert_eq!(restored.turns[0].user_input, "hello");
assert_eq!(restored.turns[0].response, Some("world".to_string()));
assert_eq!(restored.last_response_id, Some("resp_abc123".to_string()));
}
#[test]
+86
View File
@@ -13,6 +13,9 @@ use crate::agent::session::Session;
use crate::agent::undo::UndoManager;
use crate::hooks::HookRegistry;
/// Warn when session count exceeds this threshold.
const SESSION_COUNT_WARNING_THRESHOLD: usize = 1000;
/// Key for mapping external thread IDs to internal ones.
#[derive(Clone, Hash, Eq, PartialEq)]
struct ThreadKey {
@@ -68,6 +71,14 @@ impl SessionManager {
let session = Arc::new(Mutex::new(new_session));
sessions.insert(user_id.to_string(), Arc::clone(&session));
if sessions.len() >= SESSION_COUNT_WARNING_THRESHOLD && sessions.len() % 100 == 0 {
tracing::warn!(
"High session count: {} active sessions. \
Pruning runs every 10 minutes; consider reducing session_idle_timeout.",
sessions.len()
);
}
// Fire OnSessionStart hook (fire-and-forget)
if let Some(ref hooks) = self.hooks {
let hooks = hooks.clone();
@@ -117,6 +128,42 @@ impl SessionManager {
}
}
// Check if external_thread_id is itself a known thread UUID that
// exists in the session but was never registered in the thread_map
// (e.g. created by chat_new_thread_handler or hydrated from DB).
// We only adopt it if no thread_map entry maps to this UUID —
// otherwise it belongs to a different channel scope.
if let Some(ext_tid) = external_thread_id
&& let Ok(ext_uuid) = Uuid::parse_str(ext_tid)
{
let thread_map = self.thread_map.read().await;
let mapped_elsewhere = thread_map.values().any(|&v| v == ext_uuid);
drop(thread_map);
if !mapped_elsewhere {
let sess = session.lock().await;
if sess.threads.contains_key(&ext_uuid) {
drop(sess);
let mut thread_map = self.thread_map.write().await;
// Re-check after acquiring write lock to prevent race condition
// where another task mapped this UUID between our read and write.
if !thread_map.values().any(|&v| v == ext_uuid) {
thread_map.insert(key, ext_uuid);
drop(thread_map);
// Ensure undo manager exists
let mut undo_managers = self.undo_managers.write().await;
undo_managers
.entry(ext_uuid)
.or_insert_with(|| Arc::new(Mutex::new(UndoManager::new())));
return (session, ext_uuid);
}
// If it was mapped elsewhere while we were unlocked, fall through
// to create a new thread, preserving channel isolation.
}
}
}
// Create new thread (always create a new one for a new key)
let thread_id = {
let mut sess = session.lock().await;
@@ -724,4 +771,43 @@ mod tests {
.await;
assert_ne!(resolved, tid);
}
#[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread};
let manager = SessionManager::new();
let tid = Uuid::new_v4();
// Simulate chat_new_thread_handler: create thread directly in session
// without registering it in thread_map
let session = Arc::new(Mutex::new(Session::new("user-direct")));
{
let mut sess = session.lock().await;
let thread = Thread::with_id(tid, sess.id);
sess.threads.insert(tid, thread);
}
{
let mut sessions = manager.sessions.write().await;
sessions.insert("user-direct".to_string(), Arc::clone(&session));
}
// resolve_thread should find the existing thread by UUID
// instead of creating a duplicate
let (_, resolved) = manager
.resolve_thread("user-direct", "gateway", Some(&tid.to_string()))
.await;
assert_eq!(
resolved, tid,
"should reuse existing thread, not create a new one"
);
// Verify no duplicate threads were created
let sess = session.lock().await;
assert_eq!(
sess.threads.len(),
1,
"should have exactly 1 thread, not a duplicate"
);
}
}
+55
View File
@@ -62,6 +62,23 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/skills" {
return Submission::SystemCommand {
command: "skills".to_string(),
args: vec![],
};
}
if lower.starts_with("/skills ") {
let args: Vec<String> = trimmed
.split_whitespace()
.skip(1)
.map(|s| s.to_string())
.collect();
return Submission::SystemCommand {
command: "skills".to_string(),
args,
};
}
if lower == "/ping" {
return Submission::SystemCommand {
command: "ping".to_string(),
@@ -234,6 +251,7 @@ impl Submission {
}
/// Create an approval submission.
#[cfg(test)]
pub fn approval(request_id: Uuid, approved: bool) -> Self {
Self::ExecApproval {
request_id,
@@ -243,6 +261,7 @@ impl Submission {
}
/// Create an "always approve" submission.
#[cfg(test)]
pub fn always_approve(request_id: Uuid) -> Self {
Self::ExecApproval {
request_id,
@@ -252,26 +271,31 @@ impl Submission {
}
/// Create an interrupt submission.
#[cfg(test)]
pub fn interrupt() -> Self {
Self::Interrupt
}
/// Create a compact submission.
#[cfg(test)]
pub fn compact() -> Self {
Self::Compact
}
/// Create an undo submission.
#[cfg(test)]
pub fn undo() -> Self {
Self::Undo
}
/// Create a redo submission.
#[cfg(test)]
pub fn redo() -> Self {
Self::Redo
}
/// Check if this submission starts a new turn.
#[cfg(test)]
pub fn starts_turn(&self) -> bool {
matches!(self, Self::UserInput { .. })
}
@@ -340,6 +364,7 @@ impl SubmissionResult {
}
/// Create an OK result.
#[cfg(test)]
pub fn ok() -> Self {
Self::Ok { message: None }
}
@@ -685,6 +710,36 @@ mod tests {
assert!(!submission.starts_turn());
}
#[test]
fn test_parser_system_command_skills() {
let submission = SubmissionParser::parse("/skills");
assert!(
matches!(submission, Submission::SystemCommand { command, args } if command == "skills" && args.is_empty())
);
// Case insensitive
let submission = SubmissionParser::parse("/SKILLS");
assert!(
matches!(submission, Submission::SystemCommand { command, .. } if command == "skills")
);
}
#[test]
fn test_parser_system_command_skills_search() {
let submission = SubmissionParser::parse("/skills search markdown");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "markdown"])
);
// Multiple words in query
let submission = SubmissionParser::parse("/skills search code review tools");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "code", "review", "tools"])
);
}
#[test]
fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+7
View File
@@ -29,6 +29,7 @@ impl TaskOutput {
}
/// Create a text result.
#[cfg(test)]
pub fn text(text: impl Into<String>, duration: Duration) -> Self {
Self {
result: serde_json::Value::String(text.into()),
@@ -37,6 +38,7 @@ impl TaskOutput {
}
/// Create an empty success result.
#[cfg(test)]
pub fn empty(duration: Duration) -> Self {
Self {
result: serde_json::Value::Null,
@@ -130,6 +132,7 @@ impl Task {
}
/// Create a new Job task with a specific ID.
#[cfg(test)]
pub fn job_with_id(id: Uuid, title: impl Into<String>, description: impl Into<String>) -> Self {
Self::Job {
id,
@@ -152,6 +155,7 @@ impl Task {
}
/// Create a new Background task.
#[cfg(test)]
pub fn background(handler: std::sync::Arc<dyn TaskHandler>) -> Self {
Self::Background {
id: Uuid::new_v4(),
@@ -160,6 +164,7 @@ impl Task {
}
/// Create a new Background task with a specific ID.
#[cfg(test)]
pub fn background_with_id(id: Uuid, handler: std::sync::Arc<dyn TaskHandler>) -> Self {
Self::Background { id, handler }
}
@@ -174,6 +179,7 @@ impl Task {
}
/// Get the parent ID for sub-tasks.
#[cfg(test)]
pub fn parent_id(&self) -> Option<Uuid> {
match self {
Self::Job { .. } => None,
@@ -225,6 +231,7 @@ impl fmt::Debug for Task {
}
/// Status of a scheduled task.
#[cfg(test)]
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TaskStatus {
/// Task is queued waiting for execution.
+337 -217
View File
@@ -6,11 +6,14 @@
use std::sync::Arc;
use tokio::sync::Mutex;
use tokio::task::JoinSet;
use uuid::Uuid;
use crate::agent::Agent;
use crate::agent::compaction::ContextCompactor;
use crate::agent::dispatcher::{AgenticLoopResult, detect_auth_awaiting, parse_auth_result};
use crate::agent::dispatcher::{
AgenticLoopResult, check_auth_required, execute_chat_tool_standalone, parse_auth_result,
};
use crate::agent::session::{PendingApproval, Session, ThreadState};
use crate::agent::submission::SubmissionResult;
use crate::channels::{IncomingMessage, StatusUpdate};
@@ -84,20 +87,6 @@ impl Agent {
thread.restore_from_messages(chat_messages);
}
// Restore response chain from conversation metadata
if let Some(store) = self.store()
&& let Ok(Some(metadata)) = store.get_conversation_metadata(thread_uuid).await
&& let Some(rid) = metadata
.get("last_response_id")
.and_then(|v| v.as_str())
.map(String::from)
{
thread.last_response_id = Some(rid.clone());
self.llm()
.seed_response_chain(&thread_uuid.to_string(), rid);
tracing::debug!("Restored response chain for thread {}", thread_uuid);
}
// Insert into session and register with session manager
{
let mut sess = session.lock().await;
@@ -225,7 +214,7 @@ impl Agent {
)
.await;
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
if let Err(e) = compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -263,6 +252,10 @@ impl Agent {
thread.messages()
};
// Persist user message to DB immediately so it survives crashes
self.persist_user_message(thread_id, &message.user_id, content)
.await;
// Send thinking status
let _ = self
.channels
@@ -275,7 +268,7 @@ impl Agent {
// Run the agentic tool execution loop
let result = self
.run_agentic_loop(message, session.clone(), thread_id, turn_messages, false)
.run_agentic_loop(message, session.clone(), thread_id, turn_messages)
.await;
// Re-acquire lock and check if interrupted
@@ -322,7 +315,6 @@ impl Agent {
};
thread.complete_turn(&response);
self.persist_response_chain(thread);
let _ = self
.channels
.send_status(
@@ -332,8 +324,9 @@ impl Agent {
)
.await;
// Fire-and-forget: persist turn to DB
self.persist_turn(thread_id, &message.user_id, content, Some(&response));
// Persist assistant response (user message already persisted at turn start)
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
Ok(SubmissionResult::response(response))
}
@@ -361,92 +354,73 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// Persist the user message even on failure
self.persist_turn(thread_id, &message.user_id, content, None);
// User message already persisted at turn start; nothing else to save
Ok(SubmissionResult::error(e.to_string()))
}
}
}
/// Fire-and-forget: persist a turn (user message + optional assistant response) to the DB.
pub(super) fn persist_turn(
/// Persist the user message to the DB at turn start (before the agentic loop).
///
/// This ensures the user message is durable even if the process crashes
/// mid-response. Call this right after `thread.start_turn()`.
pub(super) async fn persist_user_message(
&self,
thread_id: Uuid,
user_id: &str,
user_input: &str,
response: Option<&str>,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let user_id = user_id.to_string();
let user_input = user_input.to_string();
let response = response.map(String::from);
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "user", &user_input)
.await
{
tracing::warn!("Failed to persist user message: {}", e);
return;
}
if let Some(ref resp) = response
&& let Err(e) = store
.add_conversation_message(thread_id, "assistant", resp)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
});
if let Err(e) = store
.add_conversation_message(thread_id, "user", user_input)
.await
{
tracing::warn!("Failed to persist user message: {}", e);
}
}
/// Sync the provider's response chain ID to the thread and DB metadata.
/// Persist the assistant response to the DB after the agentic loop completes.
///
/// Call after a successful agentic loop to persist the latest
/// `previous_response_id` so chaining survives restarts.
pub(super) fn persist_response_chain(&self, thread: &mut crate::agent::session::Thread) {
let tid = thread.id.to_string();
let response_id = match self.llm().get_response_chain_id(&tid) {
Some(rid) => rid,
None => return,
};
// Update in-memory thread
thread.last_response_id = Some(response_id.clone());
// Fire-and-forget DB write
/// Re-ensures the conversation row exists so that assistant responses are
/// still persisted even if `persist_user_message` failed transiently at
/// turn start (e.g. a brief DB blip that resolved before response time).
pub(super) async fn persist_assistant_response(
&self,
thread_id: Uuid,
user_id: &str,
response: &str,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
let thread_id = thread.id;
tokio::spawn(async move {
let val = serde_json::json!(response_id);
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "last_response_id", &val)
.await
{
tracing::warn!(
"Failed to persist response chain for thread {}: {}",
thread_id,
e
);
}
});
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Err(e) = store
.add_conversation_message(thread_id, "assistant", response)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
}
pub(super) async fn process_undo(
@@ -559,7 +533,7 @@ impl Agent {
crate::agent::context_monitor::CompactionStrategy::Summarize { keep_recent: 5 },
);
let compactor = ContextCompactor::new(self.llm().clone());
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
match compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -608,8 +582,8 @@ impl Agent {
approved: bool,
always: bool,
) -> Result<SubmissionResult, Error> {
// Get thread state and pending approval
let (_thread_state, pending) = {
// Get pending approval for this thread
let pending = {
let mut sess = session.lock().await;
let thread = sess
.threads
@@ -620,8 +594,7 @@ impl Agent {
return Ok(SubmissionResult::error("No pending approval request."));
}
let pending = thread.take_pending_approval();
(thread.state, pending)
thread.take_pending_approval()
};
let pending = match pending {
@@ -734,29 +707,17 @@ impl Agent {
// If tool_auth returned awaiting_token, enter auth mode and
// return instructions directly (skip agentic loop continuation).
if let Some((ext_name, instructions)) =
detect_auth_awaiting(&pending.tool_name, &tool_result)
check_auth_required(&pending.tool_name, &tool_result)
{
let auth_data = parse_auth_result(&tool_result);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::AuthRequired {
extension_name: ext_name,
instructions: Some(instructions.clone()),
auth_url: auth_data.auth_url,
setup_url: auth_data.setup_url,
},
&message.metadata,
)
.await;
self.handle_auth_intercept(
&session,
thread_id,
message,
&tool_result,
ext_name,
instructions.clone(),
)
.await;
return Ok(SubmissionResult::response(instructions));
}
@@ -798,89 +759,165 @@ impl Agent {
.await;
}
let mut deferred_queue = std::collections::VecDeque::from(deferred_tool_calls);
while let Some(tc) = deferred_queue.pop_front() {
// Re-check approval for each deferred tool call
if let Some(tool) = self.tools().get(&tc.name).await
&& tool.requires_approval()
{
let is_auto_approved = {
let sess = session.lock().await;
let mut approved = sess.is_tool_auto_approved(&tc.name);
if approved && tool.requires_approval_for(&tc.arguments) {
approved = false;
// === Phase 1: Preflight (sequential) ===
// Walk deferred tools checking approval. Collect runnable
// tools; stop at the first that needs approval.
let mut runnable: Vec<crate::llm::ToolCall> = Vec::new();
let mut approval_needed: Option<(
usize,
crate::llm::ToolCall,
Arc<dyn crate::tools::Tool>,
)> = None;
for (idx, tc) in deferred_tool_calls.iter().enumerate() {
if let Some(tool) = self.tools().get(&tc.name).await {
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => {
let sess = session.lock().await;
!sess.is_tool_auto_approved(&tc.name)
}
approved
ApprovalRequirement::Always => true,
};
if !is_auto_approved {
let new_pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: deferred_queue.iter().cloned().collect(),
};
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.await_approval(new_pending);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Status("Awaiting approval".into()),
&message.metadata,
)
.await;
return Ok(SubmissionResult::NeedApproval {
request_id,
tool_name,
description,
parameters,
});
if needs_approval {
approval_needed = Some((idx, tc.clone(), tool));
break; // remaining tools stay deferred
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolStarted {
name: tc.name.clone(),
},
&message.metadata,
)
.await;
runnable.push(tc.clone());
}
let deferred_result = self
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
// === Phase 2: Parallel execution ===
let exec_results: Vec<(crate::llm::ToolCall, Result<String, Error>)> = if runnable.len()
<= 1
{
// Single tool (or none): execute inline
let mut results = Vec::new();
for tc in &runnable {
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolStarted {
name: tc.name.clone(),
},
&message.metadata,
)
.await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: deferred_result.is_ok(),
},
&message.metadata,
)
.await;
let result = self
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
&message.metadata,
)
.await;
results.push((tc.clone(), result));
}
results
} else {
// Multiple tools: execute in parallel via JoinSet
let mut join_set = JoinSet::new();
let runnable_count = runnable.len();
for (spawn_idx, tc) in runnable.iter().enumerate() {
let tools = self.tools().clone();
let safety = self.safety().clone();
let channels = self.channels.clone();
let job_ctx = job_ctx.clone();
let tc = tc.clone();
let channel = message.channel.clone();
let metadata = message.metadata.clone();
join_set.spawn(async move {
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolStarted {
name: tc.name.clone(),
},
&metadata,
)
.await;
let result = execute_chat_tool_standalone(
&tools,
&safety,
&tc.name,
&tc.arguments,
&job_ctx,
)
.await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
&metadata,
)
.await;
(spawn_idx, tc, result)
});
}
// Collect and reorder by original index
let mut ordered: Vec<Option<(crate::llm::ToolCall, Result<String, Error>)>> =
(0..runnable_count).map(|_| None).collect();
while let Some(join_result) = join_set.join_next().await {
match join_result {
Ok((idx, tc, result)) => {
ordered[idx] = Some((tc, result));
}
Err(e) => {
if e.is_panic() {
tracing::error!("Deferred tool execution task panicked: {}", e);
} else {
tracing::error!("Deferred tool execution task cancelled: {}", e);
}
}
}
}
// Fill panicked slots with error results
ordered
.into_iter()
.enumerate()
.map(|(i, opt)| {
opt.unwrap_or_else(|| {
let tc = runnable[i].clone();
let err: Error = crate::error::ToolError::ExecutionFailed {
name: tc.name.clone(),
reason: "Task failed during execution".to_string(),
}
.into();
(tc, Err(err))
})
})
.collect()
};
// === Phase 3: Post-flight (sequential, in original order) ===
// Process all results before any conditional return so every
// tool result is recorded in the session audit trail.
let mut deferred_auth: Option<String> = None;
for (tc, deferred_result) in exec_results {
if let Ok(ref output) = deferred_result
&& !output.is_empty()
{
@@ -910,32 +947,21 @@ impl Agent {
}
}
// Auth detection for deferred tools
if let Some((ext_name, instructions)) =
detect_auth_awaiting(&tc.name, &deferred_result)
// Auth detection defer return until all results are recorded
if deferred_auth.is_none()
&& let Some((ext_name, instructions)) =
check_auth_required(&tc.name, &deferred_result)
{
let auth_data = parse_auth_result(&deferred_result);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::AuthRequired {
extension_name: ext_name,
instructions: Some(instructions.clone()),
auth_url: auth_data.auth_url,
setup_url: auth_data.setup_url,
},
&message.metadata,
)
.await;
return Ok(SubmissionResult::response(instructions));
self.handle_auth_intercept(
&session,
thread_id,
message,
&deferred_result,
ext_name,
instructions.clone(),
)
.await;
deferred_auth = Some(instructions);
}
let deferred_content = match deferred_result {
@@ -953,9 +979,55 @@ impl Agent {
context_messages.push(ChatMessage::tool_result(&tc.id, &tc.name, deferred_content));
}
// Return auth response after all results are recorded
if let Some(instructions) = deferred_auth {
return Ok(SubmissionResult::response(instructions));
}
// Handle approval if a tool needed it
if let Some((approval_idx, tc, tool)) = approval_needed {
let new_pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: deferred_tool_calls[approval_idx + 1..].to_vec(),
};
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.await_approval(new_pending);
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Status("Awaiting approval".into()),
&message.metadata,
)
.await;
return Ok(SubmissionResult::NeedApproval {
request_id,
tool_name,
description,
parameters,
});
}
// Continue the agentic loop (a tool was already executed this turn)
let result = self
.run_agentic_loop(message, session.clone(), thread_id, context_messages, true)
.run_agentic_loop(message, session.clone(), thread_id, context_messages)
.await;
// Handle the result
@@ -968,7 +1040,9 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
thread.complete_turn(&response);
self.persist_response_chain(thread);
// User message already persisted at turn start; save assistant response
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
.channels
.send_status(
@@ -1004,15 +1078,25 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// User message already persisted at turn start
Ok(SubmissionResult::error(e.to_string()))
}
}
} else {
// Rejected - clear approval and return to idle
// Rejected - complete the turn with a rejection message and persist
let rejection = format!(
"Tool '{}' was rejected. The agent will not execute this tool.\n\n\
You can continue the conversation or try a different approach.",
pending.tool_name
);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.clear_pending_approval();
thread.complete_turn(&rejection);
// User message already persisted at turn start; save rejection response
self.persist_assistant_response(thread_id, &message.user_id, &rejection)
.await;
}
}
@@ -1025,14 +1109,50 @@ impl Agent {
)
.await;
Ok(SubmissionResult::response(format!(
"Tool '{}' was rejected. The agent will not execute this tool.\n\n\
You can continue the conversation or try a different approach.",
pending.tool_name
)))
Ok(SubmissionResult::response(rejection))
}
}
/// Handle an auth-required result from a tool execution.
///
/// Enters auth mode on the thread, completes + persists the turn,
/// and sends the AuthRequired status to the channel.
/// Returns the instructions string for the caller to wrap in a response.
async fn handle_auth_intercept(
&self,
session: &Arc<Mutex<Session>>,
thread_id: Uuid,
message: &IncomingMessage,
tool_result: &Result<String, Error>,
ext_name: String,
instructions: String,
) {
let auth_data = parse_auth_result(tool_result);
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
// User message already persisted at turn start; save auth instructions
self.persist_assistant_response(thread_id, &message.user_id, &instructions)
.await;
}
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::AuthRequired {
extension_name: ext_name,
instructions: Some(instructions.clone()),
auth_url: auth_data.auth_url,
setup_url: auth_data.setup_url,
},
&message.metadata,
)
.await;
}
/// Handle an auth token submitted while the thread is in auth mode.
///
/// The token goes directly to the extension manager's credential store,
+4
View File
@@ -67,6 +67,7 @@ impl UndoManager {
}
/// Create with a custom checkpoint limit.
#[cfg(test)]
pub fn with_max_checkpoints(mut self, max: usize) -> Self {
self.max_checkpoints = max;
self
@@ -126,6 +127,7 @@ impl UndoManager {
}
/// Pop the last checkpoint from the undo stack.
#[cfg(test)]
pub fn pop_undo(&mut self) -> Option<Checkpoint> {
self.undo_stack.pop_back()
}
@@ -178,6 +180,7 @@ impl UndoManager {
}
/// Get a checkpoint by ID.
#[cfg(test)]
pub fn get_checkpoint(&self, id: Uuid) -> Option<&Checkpoint> {
self.undo_stack
.iter()
@@ -186,6 +189,7 @@ impl UndoManager {
}
/// List all available checkpoints (for UI display).
#[cfg(test)]
pub fn list_checkpoints(&self) -> Vec<&Checkpoint> {
self.undo_stack.iter().collect()
}
+441 -49
View File
@@ -3,8 +3,8 @@
use std::sync::Arc;
use std::time::Duration;
use futures::future::join_all;
use tokio::sync::mpsc;
use tokio::task::JoinSet;
use uuid::Uuid;
use crate::agent::scheduler::WorkerMessage;
@@ -18,6 +18,7 @@ use crate::llm::{
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::rate_limiter::RateLimitResult;
/// Shared dependencies for worker execution.
///
@@ -97,6 +98,20 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
}
/// Run the worker until the job is complete or stopped.
pub async fn run(self, mut rx: mpsc::Receiver<WorkerMessage>) -> Result<(), Error> {
tracing::info!("Worker starting for job {}", self.job_id);
@@ -163,7 +178,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reasoning: &Reasoning,
reason_ctx: &mut ReasoningContext,
) -> Result<(), Error> {
let max_iterations = 50;
const MAX_WORKER_ITERATIONS: usize = 500;
let max_iterations = self
.context_manager()
.get_context(self.job_id)
.await
.ok()
.and_then(|ctx| ctx.metadata.get("max_iterations").and_then(|v| v.as_u64()))
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
// Initial tool definitions for planning (will be refreshed in loop)
@@ -192,6 +215,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.join("\n")
)));
self.log_event("message", serde_json::json!({
"role": "assistant",
"content": format!("Plan: {}\n\n{}", p.goal,
p.actions.iter().enumerate()
.map(|(i, a)| format!("{}. {} - {}", i + 1, a.tool_name, a.reasoning))
.collect::<Vec<_>>().join("\n"))
}));
Some(p)
}
Err(e) => {
@@ -266,6 +297,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Add assistant response to context
reason_ctx.messages.push(ChatMessage::assistant(&response));
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": response,
}),
);
// Give it one more chance to select a tool
if iteration > 3 && iteration % 5 == 0 {
reason_ctx.messages.push(ChatMessage::user(
@@ -284,6 +323,16 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_calls.len()
);
if let Some(ref text) = content {
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": text,
}),
);
}
// Add assistant message with tool_calls (OpenAI protocol)
reason_ctx
.messages
@@ -292,19 +341,21 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_calls.clone(),
));
for tc in tool_calls {
let result = self.execute_tool(&tc.name, &tc.arguments).await;
// Create synthetic selection for process_tool_result
let selection = ToolSelection {
// Convert ToolCalls to ToolSelections and execute in parallel
let selections: Vec<ToolSelection> = tool_calls
.iter()
.map(|tc| ToolSelection {
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: tc.id.clone(),
};
})
.collect();
self.process_tool_result(reason_ctx, &selection, result)
let results = self.execute_tools_parallel(&selections).await;
for (selection, result) in selections.iter().zip(results) {
self.process_tool_result(reason_ctx, selection, result.result)
.await?;
}
}
@@ -347,24 +398,71 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
/// Execute multiple tools in parallel.
/// Execute multiple tools in parallel using a JoinSet.
///
/// Each task is tagged with its original index so results are returned
/// in the same order as `selections`, regardless of completion order.
async fn execute_tools_parallel(&self, selections: &[ToolSelection]) -> Vec<ToolExecResult> {
let futures: Vec<_> = selections
.iter()
.map(|selection| {
let tool_name = selection.tool_name.clone();
let params = selection.parameters.clone();
let deps = self.deps.clone();
let job_id = self.job_id;
let count = selections.len();
async move {
let result = Self::execute_tool_inner(&deps, job_id, &tool_name, &params).await;
ToolExecResult { result }
// Short-circuit for single tool: execute directly without JoinSet overhead
if count <= 1 {
let mut results = Vec::with_capacity(count);
for selection in selections {
let result = Self::execute_tool_inner(
&self.deps,
self.job_id,
&selection.tool_name,
&selection.parameters,
)
.await;
results.push(ToolExecResult { result });
}
return results;
}
let mut join_set = JoinSet::new();
for (idx, selection) in selections.iter().enumerate() {
let deps = self.deps.clone();
let job_id = self.job_id;
let tool_name = selection.tool_name.clone();
let params = selection.parameters.clone();
join_set.spawn(async move {
let result = Self::execute_tool_inner(&deps, job_id, &tool_name, &params).await;
(idx, ToolExecResult { result })
});
}
// Collect and reorder by original index
let mut results: Vec<Option<ToolExecResult>> = (0..count).map(|_| None).collect();
while let Some(join_result) = join_set.join_next().await {
match join_result {
Ok((idx, exec_result)) => results[idx] = Some(exec_result),
Err(e) => {
if e.is_panic() {
tracing::error!("Tool execution task panicked: {}", e);
} else {
tracing::error!("Tool execution task cancelled: {}", e);
}
}
})
.collect();
}
}
join_all(futures).await
// Fill any panicked slots with error results
results
.into_iter()
.enumerate()
.map(|(i, opt)| {
opt.unwrap_or_else(|| ToolExecResult {
result: Err(crate::error::ToolError::ExecutionFailed {
name: selections[i].tool_name.clone(),
reason: "Task failed during execution".to_string(),
}
.into()),
})
})
.collect()
}
/// Inner tool execution logic that can be called from both single and parallel paths.
@@ -383,16 +481,31 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
})?;
// Tools requiring approval are blocked in autonomous jobs
if tool.requires_approval() {
if tool.requires_approval(params).is_required() {
return Err(crate::error::ToolError::AuthRequired {
name: tool_name.to_string(),
}
.into());
}
// Fetch job context early so we have the real user_id for hooks
// Fetch job context early so we have the real user_id for hooks and rate limiting
let job_ctx = deps.context_manager.get_context(job_id).await?;
// Check per-tool rate limit before running hooks or executing (cheaper check first)
if let Some(config) = tool.rate_limit_config()
&& let RateLimitResult::Limited { retry_after, .. } = deps
.tools
.rate_limiter()
.check_and_record(&job_ctx.user_id, tool_name, &config)
.await
{
return Err(crate::error::ToolError::RateLimited {
name: tool_name.to_string(),
retry_after: Some(retry_after),
}
.into());
}
// Run BeforeToolCall hook
let params = {
use crate::hooks::{HookError, HookEvent, HookOutcome};
@@ -505,7 +618,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
let output_str = serde_json::to_string_pretty(&output.result)
.ok()
.map(|s| deps.safety.sanitize_tool_output(tool_name, &s).content);
deps.context_manager
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem.create_action(tool_name, params.clone()).succeed(
output_str.clone(),
@@ -516,30 +630,52 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
rec
})
.await
.ok()
{
Ok(rec) => Some(rec),
Err(e) => {
tracing::warn!(job_id = %job_id, tool = tool_name, "Failed to record action in memory: {e}");
None
}
}
}
Ok(Err(e)) => {
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.fail(e.to_string(), elapsed);
mem.record_action(rec.clone());
rec
})
.await
{
Ok(rec) => Some(rec),
Err(e) => {
tracing::warn!(job_id = %job_id, tool = tool_name, "Failed to record action in memory: {e}");
None
}
}
}
Err(_) => {
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.fail("Execution timeout", elapsed);
mem.record_action(rec.clone());
rec
})
.await
{
Ok(rec) => Some(rec),
Err(e) => {
tracing::warn!(job_id = %job_id, tool = tool_name, "Failed to record action in memory: {e}");
None
}
}
}
Ok(Err(e)) => deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.fail(e.to_string(), elapsed);
mem.record_action(rec.clone());
rec
})
.await
.ok(),
Err(_) => deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.fail("Execution timeout", elapsed);
mem.record_action(rec.clone());
rec
})
.await
.ok(),
};
// Persist action to database (fire-and-forget)
@@ -579,6 +715,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
selection: &ToolSelection,
result: Result<String, Error>,
) -> Result<bool, Error> {
self.log_event(
"tool_use",
serde_json::json!({
"tool_name": selection.tool_name,
"input": crate::agent::agent_loop::truncate_for_preview(
&selection.parameters.to_string(), 500),
}),
);
match result {
Ok(output) => {
// Sanitize output
@@ -599,6 +744,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
wrapped,
));
self.log_event("tool_result", serde_json::json!({
"tool_name": selection.tool_name,
"success": true,
"output": crate::agent::agent_loop::truncate_for_preview(&sanitized.content, 500),
}));
// Tool output never drives job completion. A malicious tool could
// emit "TASK_COMPLETE" to force premature completion. Only the LLM's
// own structured response (in execution_loop) can mark a job done.
@@ -625,6 +776,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
});
}
self.log_event(
"tool_result",
serde_json::json!({
"tool_name": selection.tool_name,
"success": false,
"output": format!("Error: {}", e),
}),
);
reason_ctx.messages.push(ChatMessage::tool_result(
&selection.tool_call_id,
&selection.tool_name,
@@ -746,6 +906,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": true,
"message": "Job completed successfully",
}),
);
self.persist_status(
JobState::Completed,
Some("Job completed successfully".to_string()),
@@ -764,6 +931,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Execution failed: {}", reason),
}),
);
self.persist_status(JobState::Failed, Some(reason.to_string()));
Ok(())
}
@@ -777,6 +951,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Job stuck: {}", reason),
}),
);
self.persist_status(JobState::Stuck, Some(reason.to_string()));
Ok(())
}
@@ -800,6 +981,102 @@ mod tests {
use crate::llm::ToolSelection;
use crate::util::llm_signals_completion;
use super::*;
use crate::config::SafetyConfig;
use crate::context::JobContext;
use crate::llm::{
CompletionRequest, CompletionResponse, LlmProvider, ToolCompletionRequest,
ToolCompletionResponse,
};
use crate::safety::SafetyLayer;
use crate::tools::{Tool, ToolError, ToolOutput};
/// A test tool that sleeps for a configurable duration before returning.
struct SlowTool {
tool_name: String,
delay: Duration,
}
#[async_trait::async_trait]
impl Tool for SlowTool {
fn name(&self) -> &str {
&self.tool_name
}
fn description(&self) -> &str {
"Test tool with configurable delay"
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({"type": "object", "properties": {}})
}
async fn execute(
&self,
_params: serde_json::Value,
_ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
tokio::time::sleep(self.delay).await;
Ok(ToolOutput::text(
format!("done_{}", self.tool_name),
start.elapsed(),
))
}
fn requires_sanitization(&self) -> bool {
false
}
}
/// Stub LLM provider (never called in these tests).
struct StubLlm;
#[async_trait::async_trait]
impl LlmProvider for StubLlm {
fn model_name(&self) -> &str {
"stub"
}
fn cost_per_token(&self) -> (rust_decimal::Decimal, rust_decimal::Decimal) {
(rust_decimal::Decimal::ZERO, rust_decimal::Decimal::ZERO)
}
async fn complete(
&self,
_req: CompletionRequest,
) -> Result<CompletionResponse, crate::error::LlmError> {
unimplemented!("stub")
}
async fn complete_with_tools(
&self,
_req: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, crate::error::LlmError> {
unimplemented!("stub")
}
}
/// Build a Worker wired to a ToolRegistry containing the given tools.
async fn make_worker(tools: Vec<Arc<dyn Tool>>) -> Worker {
let registry = ToolRegistry::new();
for t in tools {
registry.register(t).await;
}
let cm = Arc::new(crate::context::ContextManager::new(5));
let job_id = cm.create_job("test", "test job").await.unwrap();
let deps = WorkerDeps {
context_manager: cm,
llm: Arc::new(StubLlm),
safety: Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
})),
tools: Arc::new(registry),
store: None,
hooks: Arc::new(crate::hooks::HookRegistry::new()),
timeout: Duration::from_secs(30),
use_planning: false,
};
Worker::new(job_id, deps)
}
#[test]
fn test_tool_selection_preserves_call_id() {
let selection = ToolSelection {
@@ -876,4 +1153,119 @@ mod tests {
"The tool returned: TASK_COMPLETE signal"
));
}
#[tokio::test]
async fn test_parallel_speedup() {
// 3 tools each sleeping 200ms should finish in roughly 200ms (parallel),
// not ~600ms (sequential).
let tools: Vec<Arc<dyn Tool>> = (0..3)
.map(|i| {
Arc::new(SlowTool {
tool_name: format!("slow_{}", i),
delay: Duration::from_millis(200),
}) as Arc<dyn Tool>
})
.collect();
let worker = make_worker(tools).await;
let selections: Vec<ToolSelection> = (0..3)
.map(|i| ToolSelection {
tool_name: format!("slow_{}", i),
parameters: serde_json::json!({}),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: format!("call_{}", i),
})
.collect();
let start = std::time::Instant::now();
let results = worker.execute_tools_parallel(&selections).await;
let elapsed = start.elapsed();
assert_eq!(results.len(), 3);
for r in &results {
assert!(r.result.is_ok(), "Tool should succeed");
}
// Parallel should complete well under the sequential 600ms threshold.
assert!(
elapsed < Duration::from_millis(500),
"Parallel execution took {:?}, expected < 500ms",
elapsed
);
}
#[tokio::test]
async fn test_result_ordering_preserved() {
// Tools with different delays finish in different order.
// Results must be returned in the original request order.
let tools: Vec<Arc<dyn Tool>> = vec![
Arc::new(SlowTool {
tool_name: "tool_a".into(),
delay: Duration::from_millis(300),
}),
Arc::new(SlowTool {
tool_name: "tool_b".into(),
delay: Duration::from_millis(100),
}),
Arc::new(SlowTool {
tool_name: "tool_c".into(),
delay: Duration::from_millis(200),
}),
];
let worker = make_worker(tools).await;
let selections = vec![
ToolSelection {
tool_name: "tool_a".into(),
parameters: serde_json::json!({}),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: "call_a".into(),
},
ToolSelection {
tool_name: "tool_b".into(),
parameters: serde_json::json!({}),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: "call_b".into(),
},
ToolSelection {
tool_name: "tool_c".into(),
parameters: serde_json::json!({}),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: "call_c".into(),
},
];
let results = worker.execute_tools_parallel(&selections).await;
// Results must be in same order as selections, not completion order.
assert!(results[0].result.as_ref().unwrap().contains("done_tool_a"));
assert!(results[1].result.as_ref().unwrap().contains("done_tool_b"));
assert!(results[2].result.as_ref().unwrap().contains("done_tool_c"));
}
#[tokio::test]
async fn test_missing_tool_produces_error_not_panic() {
// If a tool doesn't exist, the result slot should contain an error.
let worker = make_worker(vec![]).await;
let selections = vec![ToolSelection {
tool_name: "nonexistent_tool".into(),
parameters: serde_json::json!({}),
reasoning: String::new(),
alternatives: vec![],
tool_call_id: "call_x".into(),
}];
let results = worker.execute_tools_parallel(&selections).await;
assert_eq!(results.len(), 1);
assert!(
results[0].result.is_err(),
"Missing tool should produce an error, not a panic"
);
}
}
+125 -163
View File
@@ -22,6 +22,7 @@ use crate::skills::SkillRegistry;
use crate::skills::catalog::SkillCatalog;
use crate::tools::ToolRegistry;
use crate::tools::mcp::McpSessionManager;
use crate::tools::wasm::SharedCredentialRegistry;
use crate::tools::wasm::WasmToolRuntime;
use crate::workspace::{EmbeddingProvider, Workspace};
@@ -48,6 +49,8 @@ pub struct AppComponents {
pub skill_catalog: Option<Arc<SkillCatalog>>,
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
pub session: Arc<SessionManager>,
pub catalog_entries: Vec<crate::extensions::RegistryEntry>,
pub dev_loaded_tool_names: Vec<String>,
}
/// Options that control optional init phases.
@@ -200,9 +203,13 @@ impl AppBuilder {
self.session.attach_store(db.clone(), "default").await;
if let Err(e) = db.cleanup_stale_sandbox_jobs().await {
tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e);
}
// Fire-and-forget housekeeping — no need to block startup.
let db_cleanup = db.clone();
tokio::spawn(async move {
if let Err(e) = db_cleanup.cleanup_stale_sandbox_jobs().await {
tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e);
}
});
self.db = Some(db);
Ok(())
@@ -285,94 +292,14 @@ impl AppBuilder {
/// Phase 3: Initialize LLM provider chain.
///
/// Creates the primary provider, then wraps with failover, circuit
/// breaker, and response cache as configured.
/// Delegates to `build_provider_chain` which applies all decorators
/// (retry, smart routing, failover, circuit breaker, response cache).
#[allow(clippy::type_complexity)]
pub fn init_llm(
&self,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), anyhow::Error> {
use crate::llm::{
CachedProvider, CircuitBreakerConfig, CircuitBreakerProvider, CooldownConfig,
FailoverProvider, ResponseCacheConfig, create_cheap_llm_provider, create_llm_provider,
create_llm_provider_with_config,
};
let llm = create_llm_provider(&self.config.llm, self.session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
// Wrap in failover if a fallback model is configured
let llm: Arc<dyn LlmProvider> = if let Some(fallback_model) =
self.config.llm.nearai.fallback_model.as_ref()
{
if fallback_model == &self.config.llm.nearai.model {
tracing::warn!(
"fallback_model is the same as primary model, failover may not be effective"
);
}
let mut fallback_config = self.config.llm.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(&fallback_config, self.session.clone())?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
"LLM failover enabled"
);
let cooldown_config = CooldownConfig {
cooldown_duration: std::time::Duration::from_secs(
self.config.llm.nearai.failover_cooldown_secs,
),
failure_threshold: self.config.llm.nearai.failover_cooldown_threshold,
};
Arc::new(FailoverProvider::with_cooldown(
vec![llm, fallback],
cooldown_config,
)?)
} else {
llm
};
// Wrap in circuit breaker if configured
let llm: Arc<dyn LlmProvider> =
if let Some(threshold) = self.config.llm.nearai.circuit_breaker_threshold {
let cb_config = CircuitBreakerConfig {
failure_threshold: threshold,
recovery_timeout: std::time::Duration::from_secs(
self.config.llm.nearai.circuit_breaker_recovery_secs,
),
..CircuitBreakerConfig::default()
};
tracing::info!(
threshold,
recovery_secs = self.config.llm.nearai.circuit_breaker_recovery_secs,
"LLM circuit breaker enabled"
);
Arc::new(CircuitBreakerProvider::new(llm, cb_config))
} else {
llm
};
// Wrap in response cache if configured
let llm: Arc<dyn LlmProvider> = if self.config.llm.nearai.response_cache_enabled {
let rc_config = ResponseCacheConfig {
ttl: std::time::Duration::from_secs(self.config.llm.nearai.response_cache_ttl_secs),
max_entries: self.config.llm.nearai.response_cache_max_entries,
};
tracing::info!(
ttl_secs = self.config.llm.nearai.response_cache_ttl_secs,
max_entries = self.config.llm.nearai.response_cache_max_entries,
"LLM response cache enabled"
);
Arc::new(CachedProvider::new(llm, rc_config))
} else {
llm
};
// Cheap LLM for lightweight tasks
let cheap_llm = create_cheap_llm_provider(&self.config.llm, self.session.clone())?;
if let Some(ref cheap) = cheap_llm {
tracing::info!("Cheap LLM provider initialized: {}", cheap.model_name());
}
let (llm, cheap_llm) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
Ok((llm, cheap_llm))
}
@@ -389,55 +316,41 @@ impl AppBuilder {
),
anyhow::Error,
> {
use crate::workspace::{NearAiEmbeddings, OpenAiEmbeddings};
let safety = Arc::new(SafetyLayer::new(&self.config.safety));
tracing::info!("Safety layer initialized");
let tools = Arc::new(ToolRegistry::new());
tools.register_builtin_tools();
tracing::info!("Registered {} built-in tools", tools.count());
// Create embeddings provider if configured
let embeddings: Option<Arc<dyn EmbeddingProvider>> = if self.config.embeddings.enabled {
match self.config.embeddings.provider.as_str() {
"nearai" => {
tracing::info!(
"Embeddings enabled via NEAR AI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(
NearAiEmbeddings::new(
&self.config.llm.nearai.base_url,
self.session.clone(),
)
.with_model(&self.config.embeddings.model, 1536),
))
}
_ => {
if let Some(api_key) = self.config.embeddings.openai_api_key() {
tracing::info!(
"Embeddings enabled via OpenAI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(OpenAiEmbeddings::with_model(
api_key,
&self.config.embeddings.model,
match self.config.embeddings.model.as_str() {
"text-embedding-3-large" => 3072,
_ => 1536,
},
)))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
}
}
}
// Initialize tool registry with credential injection support
let credential_registry = Arc::new(SharedCredentialRegistry::new());
let tools = if let Some(ref ss) = self.secrets_store {
Arc::new(
ToolRegistry::new()
.with_credentials(Arc::clone(&credential_registry), Arc::clone(ss)),
)
} else {
tracing::info!("Embeddings disabled (set OPENAI_API_KEY or EMBEDDING_ENABLED=true)");
None
Arc::new(ToolRegistry::new())
};
tools.register_builtin_tools();
// Create embeddings provider using the unified method
let embeddings = self
.config
.embeddings
.create_provider(&self.config.llm.nearai.base_url, self.session.clone());
// Warn if libSQL backend is used with non-1536 embedding dimension.
if self.config.database.backend == crate::config::DatabaseBackend::LibSql
&& self.config.embeddings.enabled
&& self.config.embeddings.dimension != 1536
{
tracing::warn!(
configured_dimension = self.config.embeddings.dimension,
"Embedding dimension {} is not 1536. The libSQL schema uses \
F32_BLOB(1536) which requires exactly 1536 dimensions. \
Embedding storage will fail. Use PostgreSQL or set \
EMBEDDING_DIMENSION=1536.",
self.config.embeddings.dimension
);
}
// Register memory tools if database is available
let workspace = if let Some(ref db) = self.db {
@@ -479,6 +392,8 @@ impl AppBuilder {
Arc<McpSessionManager>,
Option<Arc<WasmToolRuntime>>,
Option<Arc<ExtensionManager>>,
Vec<crate::extensions::RegistryEntry>,
Vec<String>,
),
anyhow::Error,
> {
@@ -508,6 +423,8 @@ impl AppBuilder {
let tools = Arc::clone(tools);
let wasm_config = self.config.wasm.clone();
async move {
let mut dev_loaded_tool_names: Vec<String> = Vec::new();
if let Some(ref runtime) = wasm_tool_runtime {
let mut loader = WasmToolLoader::new(Arc::clone(runtime), Arc::clone(&tools));
if let Some(ref secrets) = secrets_store {
@@ -538,10 +455,11 @@ impl AppBuilder {
match load_dev_tools(&loader, &wasm_config.tools_dir).await {
Ok(results) => {
if !results.loaded.is_empty() {
dev_loaded_tool_names.extend(results.loaded.iter().cloned());
if !dev_loaded_tool_names.is_empty() {
tracing::info!(
"Loaded {} dev WASM tools from build artifacts",
results.loaded.len()
dev_loaded_tool_names.len()
);
}
}
@@ -550,6 +468,8 @@ impl AppBuilder {
}
}
}
dev_loaded_tool_names
}
};
@@ -654,13 +574,46 @@ impl AppBuilder {
}
};
tokio::join!(wasm_tools_future, mcp_servers_future);
let (dev_loaded_tool_names, _) = tokio::join!(wasm_tools_future, mcp_servers_future);
// Create extension manager
let extension_manager = if let Some(ref secrets) = self.secrets_store {
// Load registry catalog entries for extension discovery
let catalog_entries = match crate::registry::RegistryCatalog::load_or_embedded() {
Ok(catalog) => {
let entries: Vec<_> = catalog
.all()
.iter()
.map(|m| m.to_registry_entry())
.collect();
tracing::info!(
count = entries.len(),
"Loaded registry catalog entries for extension discovery"
);
entries
}
Err(e) => {
tracing::warn!("Failed to load registry catalog: {}", e);
Vec::new()
}
};
// Create extension manager. Use ephemeral in-memory secrets if no
// persistent store is configured (listing/install/activate still work).
let ext_secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> = if let Some(ref s) =
self.secrets_store
{
Arc::clone(s)
} else {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
let ephemeral_key =
secrecy::SecretString::from(crate::secrets::keychain::generate_master_key_hex());
let crypto = Arc::new(SecretsCrypto::new(ephemeral_key).expect("ephemeral crypto"));
tracing::debug!("Using ephemeral in-memory secrets store for extension manager");
Arc::new(InMemorySecretsStore::new(crypto))
};
let extension_manager = {
let manager = Arc::new(ExtensionManager::new(
Arc::clone(&mcp_session_manager),
Arc::clone(secrets),
ext_secrets,
Arc::clone(tools),
Some(Arc::clone(hooks)),
wasm_tool_runtime.clone(),
@@ -669,27 +622,28 @@ impl AppBuilder {
self.config.tunnel.public_url.clone(),
"default".to_string(),
self.db.clone(),
catalog_entries.clone(),
));
tools.register_extension_tools(Arc::clone(&manager));
tracing::info!("Extension manager initialized with in-chat discovery tools");
Some(manager)
} else {
tracing::debug!(
"Extension manager not available (no secrets store). \
Extension tools won't be registered."
);
None
};
// Register dev tools if local tools are enabled
if self.config.agent.allow_local_tools {
// register_builder_tool() already calls register_dev_tools() internally,
// so only register them here when the builder didn't already do it.
let builder_registered_dev_tools = self.config.builder.enabled
&& (self.config.agent.allow_local_tools || !self.config.sandbox.enabled);
if self.config.agent.allow_local_tools && !builder_registered_dev_tools {
tools.register_dev_tools();
tracing::info!(
"Local tools enabled (allow_local_tools=true), dev tools registered directly"
);
}
Ok((mcp_session_manager, wasm_tool_runtime, extension_manager))
Ok((
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
))
}
/// Run all init phases in order and return the assembled components.
@@ -703,15 +657,17 @@ impl AppBuilder {
// Create hook registry early so runtime extension activation can register hooks.
let hooks = Arc::new(HookRegistry::new());
let (mcp_session_manager, wasm_tool_runtime, extension_manager) =
self.init_extensions(&tools, &hooks).await?;
let (
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
) = self.init_extensions(&tools, &hooks).await?;
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
match ws.seed_if_empty().await {
Ok(count) if count > 0 => {
tracing::info!("Workspace seeded with {} core files", count);
}
Ok(_) => {}
Err(e) => {
tracing::warn!("Failed to seed workspace: {}", e);
@@ -719,21 +675,25 @@ impl AppBuilder {
}
if embeddings.is_some() {
match ws.backfill_embeddings().await {
Ok(count) if count > 0 => {
tracing::info!("Backfilled embeddings for {} chunks", count);
let ws_bg = Arc::clone(ws);
tokio::spawn(async move {
match ws_bg.backfill_embeddings().await {
Ok(count) if count > 0 => {
tracing::info!("Backfilled embeddings for {} chunks", count);
}
Ok(_) => {}
Err(e) => {
tracing::warn!("Failed to backfill embeddings: {}", e);
}
}
Ok(_) => {}
Err(e) => {
tracing::warn!("Failed to backfill embeddings: {}", e);
}
}
});
}
}
// Skills system
let (skill_registry, skill_catalog) = if self.config.skills.enabled {
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone());
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone())
.with_installed_dir(self.config.skills.installed_dir.clone());
let loaded = registry.discover_all().await;
if !loaded.is_empty() {
tracing::info!("Loaded {} skill(s): {}", loaded.len(), loaded.join(", "));
@@ -779,6 +739,8 @@ impl AppBuilder {
skill_catalog,
cost_guard,
session: self.session,
catalog_entries,
dev_loaded_tool_names,
})
}
}
+26 -2
View File
@@ -20,8 +20,10 @@ pub struct BootInfo {
pub heartbeat_enabled: bool,
pub heartbeat_interval_secs: u64,
pub sandbox_enabled: bool,
pub docker_status: crate::sandbox::detect::DockerStatus,
pub claude_code_enabled: bool,
pub routines_enabled: bool,
pub skills_enabled: bool,
pub channels: Vec<String>,
/// Public URL from a managed tunnel (e.g., "https://abc.ngrok.io").
pub tunnel_url: Option<String>,
@@ -35,6 +37,7 @@ pub fn print_boot_screen(info: &BootInfo) {
let bold = "\x1b[1m";
let cyan = "\x1b[36m";
let dim = "\x1b[90m";
let yellow = "\x1b[33m";
let yellow_underline = "\x1b[33;4m";
let reset = "\x1b[0m";
@@ -90,8 +93,19 @@ pub fn print_boot_screen(info: &BootInfo) {
let mins = info.heartbeat_interval_secs / 60;
features.push(format!("heartbeat ({mins}m)"));
}
if info.sandbox_enabled {
features.push("sandbox".to_string());
match info.docker_status {
crate::sandbox::detect::DockerStatus::Available => {
features.push("sandbox".to_string());
}
crate::sandbox::detect::DockerStatus::NotInstalled => {
features.push(format!("{yellow}sandbox (docker not installed){reset}"));
}
crate::sandbox::detect::DockerStatus::NotRunning => {
features.push(format!("{yellow}sandbox (docker not running){reset}"));
}
crate::sandbox::detect::DockerStatus::Disabled => {
// Don't show sandbox when disabled
}
}
if info.claude_code_enabled {
features.push("claude-code".to_string());
@@ -99,6 +113,9 @@ pub fn print_boot_screen(info: &BootInfo) {
if info.routines_enabled {
features.push("routines".to_string());
}
if info.skills_enabled {
features.push("skills".to_string());
}
if !features.is_empty() {
println!(
" {dim}features{reset} {cyan}{}{reset}",
@@ -140,6 +157,7 @@ pub fn print_boot_screen(info: &BootInfo) {
#[cfg(test)]
mod tests {
use super::*;
use crate::sandbox::detect::DockerStatus;
#[test]
fn test_print_boot_screen_full() {
@@ -158,8 +176,10 @@ mod tests {
heartbeat_enabled: true,
heartbeat_interval_secs: 1800,
sandbox_enabled: true,
docker_status: DockerStatus::Available,
claude_code_enabled: false,
routines_enabled: true,
skills_enabled: true,
channels: vec![
"repl".to_string(),
"gateway".to_string(),
@@ -189,8 +209,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec![],
tunnel_url: None,
tunnel_provider: None,
@@ -216,8 +238,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec!["repl".to_string()],
tunnel_url: None,
tunnel_provider: None,
+61 -1
View File
@@ -103,7 +103,67 @@ pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
content.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(&path, content)
std::fs::write(&path, &content)?;
restrict_file_permissions(&path)?;
Ok(())
}
/// Update or add a single variable in `~/.ironclaw/.env`, preserving existing content.
///
/// Unlike `save_bootstrap_env` (which overwrites the entire file), this
/// reads the current `.env`, replaces the line for `key` if it exists,
/// or appends it otherwise. Use this when writing a single bootstrap var
/// outside the wizard (which manages the full set via `save_bootstrap_env`).
pub fn upsert_bootstrap_var(key: &str, value: &str) -> std::io::Result<()> {
let path = ironclaw_env_path();
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
let new_line = format!("{}=\"{}\"", key, escaped);
let prefix = format!("{}=", key);
let existing = std::fs::read_to_string(&path).unwrap_or_default();
let mut found = false;
let mut result = String::new();
for line in existing.lines() {
if line.starts_with(&prefix) {
if !found {
result.push_str(&new_line);
result.push('\n');
found = true;
}
// Skip duplicate lines for this key
continue;
}
result.push_str(line);
result.push('\n');
}
if !found {
result.push_str(&new_line);
result.push('\n');
}
std::fs::write(&path, result)?;
restrict_file_permissions(&path)?;
Ok(())
}
/// Set restrictive file permissions (0o600) on Unix systems.
///
/// The `.env` file may contain database credentials and API keys,
/// so it should only be readable by the owner.
fn restrict_file_permissions(_path: &std::path::Path) -> std::io::Result<()> {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let perms = std::fs::Permissions::from_mode(0o600);
std::fs::set_permissions(_path, perms)?;
}
Ok(())
}
/// Write `DATABASE_URL` to `~/.ironclaw/.env`.
+32 -9
View File
@@ -40,16 +40,39 @@ impl ChannelManager {
}
/// Add a channel to the manager.
pub fn add(&mut self, channel: Box<dyn Channel>) {
pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string();
// We need to get the inner HashMap to insert
// Since we're in a sync context during setup, we'll use try_write
if let Ok(mut channels) = self.channels.try_write() {
channels.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
} else {
tracing::error!("Failed to add channel: {} (lock contention)", name);
}
self.channels.write().await.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
}
/// Hot-add a channel to a running agent.
///
/// Starts the channel, registers it in the channels map for `respond()`/`broadcast()`,
/// and spawns a task that forwards its stream messages through `inject_tx` into
/// the agent loop.
pub async fn hot_add(&self, channel: Box<dyn Channel>) -> Result<(), ChannelError> {
let name = channel.name().to_string();
let stream = channel.start().await?;
// Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel);
// Forward stream messages through inject_tx
let tx = self.inject_tx.clone();
tokio::spawn(async move {
use futures::StreamExt;
let mut stream = stream;
while let Some(msg) = stream.next().await {
if tx.send(msg).await.is_err() {
tracing::warn!(channel = %name, "Inject channel closed, stopping hot-added channel");
break;
}
}
tracing::info!(channel = %name, "Hot-added channel stream ended");
});
Ok(())
}
/// Start all channels and return a merged stream of messages.
+2
View File
@@ -31,6 +31,7 @@ mod channel;
mod http;
mod manager;
mod repl;
mod signal;
pub mod wasm;
pub mod web;
mod webhook_server;
@@ -39,5 +40,6 @@ pub use channel::{Channel, IncomingMessage, MessageStream, OutgoingResponse, Sta
pub use http::HttpChannel;
pub use manager::ChannelManager;
pub use repl::ReplChannel;
pub use signal::SignalChannel;
pub use web::GatewayChannel;
pub use webhook_server::{WebhookServer, WebhookServerConfig};
+41 -4
View File
@@ -15,6 +15,7 @@
//! - `/compact` - Compact the context
//! - `/new` - Start a new thread
//! - `yes`/`no`/`always` - Respond to tool approval prompts
//! - `Esc` - Interrupt current operation
use std::borrow::Cow;
use std::io::{self, Write};
@@ -28,7 +29,10 @@ use rustyline::error::ReadlineError;
use rustyline::highlight::Highlighter;
use rustyline::hint::Hinter;
use rustyline::validate::Validator;
use rustyline::{CompletionType, Editor, Helper};
use rustyline::{
Cmd as ReadlineCmd, CompletionType, ConditionalEventHandler, Editor, Event, EventContext,
EventHandler, Helper, KeyCode, KeyEvent, Modifiers, RepeatCount,
};
use termimad::MadSkin;
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
@@ -121,6 +125,23 @@ impl Highlighter for ReplHelper {
impl Validator for ReplHelper {}
impl Helper for ReplHelper {}
struct EscInterruptHandler {
triggered: Arc<AtomicBool>,
}
impl ConditionalEventHandler for EscInterruptHandler {
fn handle(
&self,
_evt: &Event,
_n: RepeatCount,
_positive: bool,
_ctx: &EventContext,
) -> Option<ReadlineCmd> {
self.triggered.store(true, Ordering::Relaxed);
Some(ReadlineCmd::Interrupt)
}
}
/// Build a termimad skin with our color scheme.
fn make_skin() -> MadSkin {
let mut skin = MadSkin::default();
@@ -247,6 +268,7 @@ fn print_help() {
println!(" {c}/compact{r} {d}compact context window{r}");
println!(" {c}/new{r} {d}new conversation thread{r}");
println!(" {c}/interrupt{r} {d}stop current operation{r}");
println!(" {c}esc{r} {d}stop current operation{r}");
println!();
println!(" {h}Approval responses{r}");
println!(" {c}yes{r} ({c}y{r}) {d}approve tool execution{r}");
@@ -274,6 +296,7 @@ impl Channel for ReplChannel {
let single_message = self.single_message.clone();
let debug_mode = Arc::clone(&self.debug_mode);
let suppress_banner = Arc::clone(&self.suppress_banner);
let esc_interrupt_triggered_for_thread = Arc::new(AtomicBool::new(false));
std::thread::spawn(move || {
// Single message mode: send it and return
@@ -301,6 +324,13 @@ impl Channel for ReplChannel {
rl.set_helper(Some(ReplHelper));
rl.bind_sequence(
KeyEvent(KeyCode::Esc, Modifiers::NONE),
EventHandler::Conditional(Box::new(EscInterruptHandler {
triggered: Arc::clone(&esc_interrupt_triggered_for_thread),
})),
);
// Load history
let hist_path = history_path();
if let Some(parent) = hist_path.parent() {
@@ -360,9 +390,16 @@ impl Channel for ReplChannel {
}
}
Err(ReadlineError::Interrupted) => {
// Ctrl+C: send /interrupt
let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() {
if esc_interrupt_triggered_for_thread.swap(false, Ordering::Relaxed) {
// Esc: interrupt current operation and keep REPL open.
let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() {
break;
}
} else {
// Ctrl+C (VINTR): request graceful shutdown.
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
break;
}
}
File diff suppressed because it is too large Load Diff
+33 -20
View File
@@ -20,6 +20,7 @@ const CARGO_MANIFEST_DIR: &str = env!("CARGO_MANIFEST_DIR");
const KNOWN_CHANNELS: &[(&str, &str)] = &[
("telegram", "telegram_channel"),
("slack", "slack_channel"),
("discord", "discord_channel"),
("whatsapp", "whatsapp_channel"),
];
@@ -42,6 +43,10 @@ fn channels_src_dir() -> PathBuf {
/// Locate the build artifacts for a channel.
///
/// Checks two layouts:
/// 1. **Flat** (Docker/packaged): `<channels_src>/<name>/<name>.wasm`
/// 2. **Build tree** (dev): `<channels_src>/<name>/target/wasm32-wasip2/release/<crate_name>.wasm`
///
/// Returns (wasm_path, capabilities_path) or an error if files are missing.
fn locate_channel_artifacts(name: &str) -> Result<(PathBuf, PathBuf), String> {
let (_, crate_name) = KNOWN_CHANNELS
@@ -52,31 +57,38 @@ fn locate_channel_artifacts(name: &str) -> Result<(PathBuf, PathBuf), String> {
let src_dir = channels_src_dir();
let channel_dir = src_dir.join(name);
let wasm_path = channel_dir
.join("target/wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
let caps_path = channel_dir.join(format!("{}.capabilities.json", name));
if !wasm_path.exists() {
return Err(format!(
"Channel '{}' WASM not found at {}. Build it first:\n \
cd {} && cargo build --target wasm32-wasip2 --release",
name,
wasm_path.display(),
channel_dir.display()
));
// Check flat layout first (Docker/packaged deployments)
let flat_wasm = channel_dir.join(format!("{}.wasm", name));
if flat_wasm.exists() && caps_path.exists() {
return Ok((flat_wasm, caps_path));
}
if !caps_path.exists() {
return Err(format!(
"Channel '{}' capabilities not found at {}",
name,
caps_path.display()
));
// Fall back to build tree layout (dev builds) — search across all WASM triples
if let Some(build_wasm) =
crate::registry::artifacts::find_wasm_artifact(&channel_dir, crate_name, "release")
&& caps_path.exists()
{
return Ok((build_wasm, caps_path));
}
Ok((wasm_path, caps_path))
// Provide a helpful error with the paths we checked
let expected_build = crate::registry::artifacts::resolve_target_dir(&channel_dir)
.join("wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
Err(format!(
"Channel '{}' WASM not found. Checked:\n \
- {} (flat/packaged)\n \
- {} (build tree, and other triples)\n \
Build it first:\n \
cd {} && cargo component build --release",
name,
flat_wasm.display(),
expected_build.display(),
channel_dir.display()
))
}
/// Install a channel from build artifacts into the channels directory.
@@ -130,10 +142,11 @@ mod tests {
use super::*;
#[test]
fn test_known_channels_includes_all_three() {
fn test_known_channels_includes_all_four() {
let names = bundled_channel_names();
assert!(names.contains(&"telegram"));
assert!(names.contains(&"slack"));
assert!(names.contains(&"discord"));
assert!(names.contains(&"whatsapp"));
}
+97
View File
@@ -300,6 +300,51 @@ impl ChannelHostState {
}
}
/// In-memory workspace store for WASM channels.
///
/// Persists workspace writes across callback invocations within a single
/// channel lifetime. This allows WASM channels to maintain state (e.g.,
/// Telegram polling offsets) between poll ticks without requiring a
/// full database-backed workspace.
///
/// Uses `std::sync::RwLock` (not tokio) because WASM execution runs
/// inside `spawn_blocking`.
pub struct ChannelWorkspaceStore {
data: std::sync::RwLock<std::collections::HashMap<String, String>>,
}
impl ChannelWorkspaceStore {
/// Create a new empty workspace store.
pub fn new() -> Self {
Self {
data: std::sync::RwLock::new(std::collections::HashMap::new()),
}
}
/// Commit pending writes from a callback execution into the store.
pub fn commit_writes(&self, writes: &[PendingWorkspaceWrite]) {
if writes.is_empty() {
return;
}
if let Ok(mut data) = self.data.write() {
for write in writes {
tracing::debug!(
path = %write.path,
content_len = write.content.len(),
"Committing workspace write to channel store"
);
data.insert(write.path.clone(), write.content.clone());
}
}
}
}
impl crate::tools::wasm::WorkspaceReader for ChannelWorkspaceStore {
fn read(&self, path: &str) -> Option<String> {
self.data.read().ok()?.get(path).cloned()
}
}
/// Rate limiter for channel message emission.
///
/// Tracks emission rates across multiple executions.
@@ -497,4 +542,56 @@ mod tests {
assert_eq!(state.channel_name(), "telegram");
}
#[test]
fn test_channel_workspace_store_commit_and_read() {
use crate::channels::wasm::host::{ChannelWorkspaceStore, PendingWorkspaceWrite};
use crate::tools::wasm::WorkspaceReader;
let store = ChannelWorkspaceStore::new();
// Initially empty
assert!(store.read("channels/telegram/offset").is_none());
// Commit some writes
let writes = vec![
PendingWorkspaceWrite {
path: "channels/telegram/offset".to_string(),
content: "103".to_string(),
},
PendingWorkspaceWrite {
path: "channels/telegram/state.json".to_string(),
content: r#"{"ok":true}"#.to_string(),
},
];
store.commit_writes(&writes);
// Should be readable
assert_eq!(
store.read("channels/telegram/offset"),
Some("103".to_string())
);
assert_eq!(
store.read("channels/telegram/state.json"),
Some(r#"{"ok":true}"#.to_string())
);
// Overwrite a value
let writes2 = vec![PendingWorkspaceWrite {
path: "channels/telegram/offset".to_string(),
content: "200".to_string(),
}];
store.commit_writes(&writes2);
assert_eq!(
store.read("channels/telegram/offset"),
Some("200".to_string())
);
// Empty writes are a no-op
store.commit_writes(&[]);
assert_eq!(
store.read("channels/telegram/offset"),
Some("200".to_string())
);
}
}

Some files were not shown because too many files have changed in this diff Show More