Compare commits

..
Author SHA1 Message Date
serrrfiratandClaude Opus 4.6 bcae0852df feat(llm): add OpenAI Codex provider (Responses API + OAuth)
Add LlmBackend::OpenAiCodex with dual auth support:
- API key mode (api.openai.com, pay-per-token)
- OAuth mode (chatgpt.com via Codex CLI tokens, subscription billing)

Implements the Responses API wire format (/v1/responses) with flat
input/output arrays, instructions field, and flat tool definitions.
Includes token manager with 401 retry and disk reload, setup wizard
integration, config resolution, and 30 unit tests.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-27 07:46:27 +04:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1156884a49 chore: release v0.12.0 (#331)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-26 12:12:58 +04:00
996c6a8cc9 feat(web): improve WASM channel setup flow (#380)
* feat(web): improve WASM channel setup flow with stepper UI and auto-configure

Streamline the WASM channel setup experience in the web gateway:

- Auto-open configure modal after installing a WASM channel
- Add progress stepper (Installed → Configured → Active) on channel cards
- Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart)
- Show "Awaiting Pairing" status for Telegram until first user is paired
- Add SSE extension_status events for real-time status updates
- Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard
- Always mount webhook routes at startup so hot-added channels work without restart
- Add pairing request polling (10s interval) on extensions tab
- Track activation errors per channel with inline error display

Includes review fixes: activation_error priority over active status, stepper
failed state rendering, restart poll timeout, configure modal double-submit
guard, and SSE sender ordering constraint documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: address PR review comments

- Move PairingStore construction outside .map() loop
- Extract createReconfigureButton() helper to reduce duplication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 22:27:13 -08:00
abda94d44f fix: correct MCP registry URLs and remove non-existent Google endpoints (#370)
Audit all built-in MCP server URLs against live endpoints. Fix 5 broken
paths (Linear, Sentry, Cloudflare, Asana, Intercom), fix 1 broken host
(GitHub), and remove 2 entries (Google Drive, Google Calendar) whose
domain mcp.google.com does not exist and Google has no official remote
MCP servers for these products.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-26 00:04:39 +00:00
443b120272 feat(web): inline tool activity cards with auto-collapsing (#376)
* feat(web): inline tool activity cards with auto-collapsing

Add Claude/Codex-style inline tool activity cards to the web UI that
show tool execution progress directly in the chat conversation.

While processing:
- Animated thinking dots with message text (e.g. "Calling LLM...")
- Individual tool cards with live spinner and elapsed timer
- Cards show tool name, duration, and expandable output preview

After response arrives:
- Activity group auto-collapses to "Used N tools (Xs)"
- Click summary to expand and see individual tool cards
- Click card header to see tool output in monospace

Also includes:
- "Calling LLM..." thinking status from dispatcher (all channels)
- 5-minute max timer guard to prevent leaks on dropped SSE
- Handles parallel tools, same tool twice, failures, thread switching

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): use frozen duration for completed tools in activity summary

The collapsed activity summary was showing inflated total duration
because finalizeActivityGroup() recalculated elapsed time from
Date.now() for already-completed tools. Now each tool card stores
its final duration at completion time and the summary uses that
frozen value instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:35 -08:00
2477923af2 fix: resolve_thread adopts existing session threads by UUID (#377)
* fix: resolve_thread adopts existing session threads by UUID

When chat_new_thread_handler creates a thread directly in the session,
it doesn't register a thread_map entry. On the first message,
resolve_thread would create a duplicate thread with a different UUID,
causing:

- Thread appears empty when switching back (loadHistory queries the
  original UUID but turns live on the duplicate)
- Orphaned tabs in the thread list (both the original and duplicate
  appear)

Fix: before creating a new thread, check if the external_thread_id is
itself a UUID that exists as a thread in the session. If so, adopt it
and register the mapping. A mapped_elsewhere guard preserves channel
scope isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: double-checked locking in resolve_thread UUID adoption

Re-check mapped_elsewhere after acquiring the write lock to prevent
a TOCTOU race where another task could map the same UUID between
the read lock check and write lock insertion, breaking channel
isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:01:10 -08:00
0c5f082d16 feat(web): display logs newest-first in web gateway UI (#369)
Reverse log display order so the most recent entries appear at the top,
removing the need to scroll to see latest activity.

Frontend: rename appendLogEntry to prependLogEntry, use prepend() for
DOM insertion, cap oldest entries from the bottom, and auto-scroll to
top. Backend: update recent_entries() doc comment to clarify the
oldest-first return order works correctly with the frontend's prepend.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-25 16:00:09 -08:00
db2ba424ce Add --version flag with clap built-in support and test (#342)
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-25 17:25:07 +04:00
ibhagwanandGitHub e41b282868 feat(signal): tool approval workflow and status updates (#350)
* fix(signal): send approval prompts to users

The Signal channel was not handling StatusUpdate::ApprovalNeeded,
causing approval requests to be silently ignored and users to
never see approval prompts.

This adds proper handling of ApprovalNeeded status that sends
a formatted message to the user with:
- Tool name and description
- Parameters (formatted as JSON)
- Request ID for reference
- Instructions on how to approve/deny/always-approve

The message uses Signal's markdown-style formatting for better
readability on mobile devices.

* feat(signal): add missing StatusUpdate handlers

Add handling for all StatusUpdate variants in Signal channel,
bringing it on par with Telegram's implementation:

- ToolStarted: Shows spinner icon when tool execution begins
- ToolCompleted: Shows checkmark/X based on success/failure
- JobStarted: Shows sandbox job start with ID and URL
- AuthRequired: Shows auth prompt with instructions and URLs
- AuthCompleted: Shows auth success/failure with optional message

This ensures Signal status feedback users receive full during
tool execution, approvals, and authentication flows, matching
the experience of Telegram and other channels.

fix(signal): address clippy warnings and improve error handling

- Collapse nested if statements into let-chains
- Fix needless borrow on Status message
- Extract send_status_message helper to reduce duplication
- Add warning logs for failed message sends

* fix(signal): suppress 'Done' status messages to user

* feat(signal): debug mode parity with REPL

- Add debug_mode to SignalChannel toggled via /debug command
- Gate ToolResult, ToolStarted, ToolCompleted behind debug mode
- Add tests: debug_mode_disabled_by_default, debug_mode_toggle, debug_mode_persists_across_toggles
2026-02-25 16:34:54 +04:00
62dc5d046e feat: add OpenRouter preset to setup wizard (#270)
* feat: add OpenRouter preset to setup wizard

Add OpenRouter as a top-level provider option in the onboarding wizard
(Step 3). Selecting it pre-fills the base URL (https://openrouter.ai/api/v1)
and prompts for an API key, avoiding manual URL entry. Under the hood it
uses the existing openai_compatible backend.

Inlines the key collection flow (rather than delegating to
setup_api_key_provider) so success messages consistently say "OpenRouter"
instead of "openai_compatible", including the early-return env-key path.

Closes #178

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address serrrfirat review comments on OpenRouter wizard preset

- Re-run path now recognizes OpenRouter: display shows "OpenRouter"
  and keep-current routes to setup_openrouter() when base URL contains
  openrouter.ai
- Refactor setup_openrouter() to delegate to setup_api_key_provider()
  with a display_name override, eliminating ~40 lines of duplication
- Update README: remove false claim about model fetching from
  OpenRouter API, add footnote explaining shared secret/env var
  between OpenRouter and OpenAI-compatible
- Fix pre-existing clippy warning in settings.rs (field_reassign_with_default)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-24 22:23:39 +04:00
DevBrocoandGitHub 4d27079cc3 Update FEATURE_PARITY.md (#337)
change status of completion 
2026-02-24 14:56:38 +04:00
e9f32eaebe fix: resolve telegram/slack name collision between tool and channel registries (#346)
When installing the Telegram WASM channel via the web UI, a name collision
between registry/tools/telegram.json and registry/channels/telegram.json
caused the tool entry to win, installing to ~/.ironclaw/tools/ instead of
~/.ironclaw/channels/. This made activation fail with "WASM runtime not
available".

- Add `get_with_kind()` to ExtensionRegistry for kind-aware lookup
- Use `kind_hint` parameter in `install()` to resolve collisions
- Rename tool entries to avoid future collisions: telegram → telegram-mtproto,
  slack → slack-tool
- Fix `_bundles.json` stale reference (tools/slack → tools/slack-tool)
- Fix `cache_discovered()` to deduplicate by (name, kind) consistently
- Add path traversal validation to install/activate/remove entry points
- Add tests for kind-aware lookup, discovery cache, and bundle resolution

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-24 15:29:25 +08:00
ibhagwanandGitHub b0b3a50fa3 feat(channels): add native Signal channel via signal-cli HTTP daemon (#271)
* feat(channels): add native Signal channel via signal-cli HTTP daemon

Implement a native Rust Signal channel that connects to a running
signal-cli daemon's HTTP endpoint, enabling Signal messaging without
WASM overhead.

Architecture:
- SSE listener at /api/v1/events for receiving messages with automatic
  reconnection and exponential backoff
- JSON-RPC client at /api/v1/rpc for sending messages and typing
  indicators
- Reply target tracking via Arc<RwLock<HashMap>> to route responses
  back to the correct DM or group conversation

Features:
- User allowlisting supporting E.164 phone numbers, bare UUIDs, and
  uuid:-prefixed identifiers (matching OpenClaw's format)
- Group allowlisting with wildcard (*) support
- Configurable story and attachment-only message filtering
- Health check via signal-cli /api/v1/check
- Broadcast support to all tracked reply targets

Configuration via environment variables:
- SIGNAL_HTTP_URL, SIGNAL_ACCOUNT (required)
- SIGNAL_ALLOWED_USERS, SIGNAL_ALLOWED_GROUPS
- SIGNAL_IGNORE_ATTACHMENTS (default: false)
- SIGNAL_IGNORE_STORIES (default: true)

Includes unit tests covering allowlist logic, envelope parsing,
recipient targeting, SSE deserialization, and edge cases.

* refactor(signal): remove expect|unwrap calls

- Change SignalChannel::new to return Result<Self, ChannelError>
- Replace .expect() on reqwest client build with proper error handling
- Replace .expect() on NonZeroUsize with compile-time const using unsafe new_unchecked
- Propagate errors through test helpers to avoid unwraps in tests

* fix(signal): prevent OOM from chunked response without Content-Length

Use bytes_stream() to check response size during download rather than
buffering entire body first. This closes the OOM vector where a
malicious signal-cli daemon could send unbounded chunked data.

* fix(signal): align is_e164 minimum digits with setup wizard

Both now require 7-15 digits after '+', preventing environment
variable bypass of the stricter onboarding validation.

* refactor(signal): extract from_parts constructor

Extract SignalChannel::from_parts() used by both new() and
sse_listener() to ensure consistent object construction.

* chore: remove redundant unused var

* refactor(signal): rename allowed_users to allow_from and add dm_policy/group_policy

- Rename allowed_users -> allow_from for consistency with other channels
- Rename allowed_groups -> allow_from_groups
- Add dm_policy field: 'open', 'allowlist', or 'pairing' (default: 'pairing')
- Add group_policy field: 'allowlist', 'open', or 'disabled' (default: 'allowlist')
- Add group_allow_from field that inherits from allow_from if empty
- Implement dm_policy and group_policy logic in message processing
- Add environment variable resolution: SIGNAL_ALLOW_FROM, SIGNAL_ALLOW_FROM_GROUPS,
  SIGNAL_DM_POLICY, SIGNAL_GROUP_POLICY, SIGNAL_GROUP_ALLOW_FROM
- Add setup wizard prompts for new policy options
- Note: full pairing flow (PairingStore integration) marked as pending for future PR

* feat(signal): implement DM pairing workflow for unapproved senders

- Add PairingStore integration to check approved senders
- Handle pairing requests for unknown senders with dm_policy=pairing
- Send pairing reply message with approval instructions
- Update FEATURE_PARITY.md to reflect DM pairing support

* chore(ci): fix clippy warnings
2026-02-24 10:25:16 +04:00
3e552e0e8e fix: make onboarding installs prefer release artifacts with source fallback (#323)
* fix: make onboarding installs prefer release artifacts with source fallback

* fix: harden extension fallback errors and surface setup warnings

* fix: validate registry artifacts and harden fallback errors

* fix: address review feedback on installer fallback

- Add upfront validate_manifest_install_inputs() in
  install_with_source_fallback so bad manifests fail fast without
  relying on inner methods to catch them
- Document ALLOWED_ARTIFACT_HOSTS as GitHub-only by design
- Document intentional url omission from DownloadFailed Display
- Add channel manifest validation tests (wrong prefix rejected,
  correct prefix accepted)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: require SHA256 checksum for artifact downloads

Reject artifact installs when the manifest has sha256: null instead of
warning and proceeding. This prevents installing unverified pre-built
binaries during onboarding. The check runs before downloading to avoid
wasting bandwidth.

Since InvalidManifest blocks source fallback, manifests with URLs but
no checksums will hard-fail rather than silently falling back to source
build — forcing the manifest to be fixed.

The release CI already computes SHA256 for each bundle; the manifests
just need to be populated with the actual values.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: enforce SHA256 checksums and auto-patch manifests in CI

- Fix cargo fmt on SHA256 check code
- Reorder release CI: build WASM extensions before binary so manifests
  can be patched with computed SHA256 before build.rs embeds them
- Add "Patch manifests with WASM checksums" step in build-local-artifacts
  that reads checksums.txt and updates registry JSON files before building
- Add update-registry-checksums job that commits patched manifests back
  to main after release, keeping the repo in sync with released artifacts

This closes the integrity gap where all manifests had sha256: null and
artifact downloads were unverified. The binary now embeds correct SHA256
values and the installer hard-rejects null checksums.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Bowen Wang <[email protected]>
2026-02-23 18:49:18 +00:00
cbf5c93578 fix: copy missing files in Dockerfile to fix build (#322)
* fix: copy missing files in Dockerfile to fix build

The Docker build failed because Cargo.toml references files that were
not copied into the builder stage:

1. tests/html_to_markdown.rs — declared as [[test]] in Cargo.toml,
   Cargo validates the path exists even when only building a binary.
2. build.rs — auto-discovered build script that embeds registry
   manifests at compile time via include_str!(env!("OUT_DIR")).
3. registry/ — contains extension manifests read by build.rs to
   generate the embedded catalog.

Added COPY directives for build.rs, tests/, and registry/.

Fixes nearai/ironclaw#320

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback on WASM channel omission

- Add Dockerfile comment documenting that channels-src/ is intentionally
  omitted since WASM compilation requires wasm32-wasip2 and wasm-tools
  which are not installed in the builder stage

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add WASM channel compilation support to Docker build

- Copy channels-src/ into builder stage for Telegram/Slack/Discord/WhatsApp
- Install wasm32-wasip2 target and wasm-tools so build.rs can compile
  WASM channel components instead of silently skipping them

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 22:37:52 +04:00
Rui ChenandGitHub 0d9b6f3208 docs: add brew install ironclaw instructions (#310)
Signed-off-by: Rui Chen <[email protected]>
2026-02-23 18:04:57 +00:00
4e2dd76ae5 Fix skills system: enable by default, fix registry and install (#300)
* feat: add Docker detection module with platform guidance

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Docker sandbox step to setup wizard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: show Docker status in boot screen

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: check Docker availability at startup

When SANDBOX_ENABLED=true, proactively detect whether Docker is
installed and running before creating the ContainerJobManager.
If Docker is unavailable, log a warning with platform-specific
guidance and disable the sandbox for the session.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable sandbox by default, improve wizard explanation, document detection limits

- SandboxConfig defaults to enabled=true (startup check disables
  gracefully if Docker is unavailable)
- Wizard step explains why Docker matters: isolation for LLM-generated
  code vs running directly on the host
- Document detection confidence per platform in detect.rs module docs:
  high on macOS/Linux, medium on Windows (named pipe edge cases)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: cargo fmt + update test_builder_defaults for enabled-by-default

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate wizard Docker status handling per review

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: fix skills system - enable by default, fix registry connectivity and install

- Enable skills system by default (SKILLS_ENABLED no longer required)
- Bypass Vercel TLS fingerprint blocking by pointing DEFAULT_REGISTRY_URL
  directly at the Convex backend (wry-manatee-359.convex.site)
- Handle ZIP archives from ClawHub download API - the registry returns
  ZIP files containing SKILL.md, not raw text. Uses flate2 (existing dep)
  to extract SKILL.md from the archive.
- Surface catalog search errors in the UI with a yellow warning banner
  instead of silently returning empty results
- Handle both {"results":[...]} envelope and bare [...] array JSON formats
  from the search API
- Add ClawHub links and metadata to search result cards (clickable skill
  names linking to clawhub.ai, relevance score, "updated X ago" recency)
- Fix 3 pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address security review feedback on ZIP extraction and SSRF

- Cap download size to 10 MB before reading response body
- Guard against ZIP bombs: cap uncompressed_size at 1 MB, wrap
  DeflateDecoder with .take() read limit
- Use checked_add for ZIP header offset arithmetic to prevent overflow
- Remove .unwrap() on try_into() -- use direct array construction
- Handle IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) in SSRF checks
- Don't leak internal registry URLs in user-facing catalog_error messages
- Fix non-ASCII panic in catalog response debug logging (use .get() instead
  of byte slicing)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add /skills command and enrich search results with ClawHub metadata

- Parse /skills and /skills search <query> as SystemCommands in submission.rs
- Add skill_catalog to AgentDeps and wire it through main.rs
- Handle "skills" command in commands.rs: list installed skills and search ClawHub
- Add /skills and /skills search <q> entries to /help output
- Add SkillDetail, SkillStats, SkillOwner structs to catalog.rs
- Add fetch_skill_detail() calling GET /api/v1/skills/{slug} on Convex backend
- Add enrich_search_results() to fetch stars/downloads/owner for top 5 results in parallel
- Fix SkillDetailResponse wrapper struct to match actual API shape: {"skill":{...},"owner":{...}}
- Surface stars, downloads, owner in web UI skill search cards (app.js)
- Surface enriched data in skills web handler and skill_search tool output

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: cargo fmt after merge conflict resolution

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: separate installed_skills dir for correct trust on restart, remove duplicate handlers

Trust level bug: skills installed from ClawHub were written to user_dir
(~/.ironclaw/skills/) which is discovered as Trusted on restart. Now installs
go to ~/.ironclaw/installed_skills/ which is discovered as Installed, matching
the documented skill directory layout.

Changes:
- SkillsConfig: add installed_dir field (SKILLS_INSTALLED_DIR env var,
  default ~/.ironclaw/installed_skills/)
- SkillRegistry: add with_installed_dir() builder, installed_dir()/
  install_target_dir() accessors, and discover installed_dir with
  SkillTrust::Installed in discover_all()
- All install paths (web handler, skill tool) use install_target_dir()
  instead of user_dir() so new installs land in the correct directory
- 3 new registry tests: test_installed_dir_uses_installed_trust,
  test_install_target_dir_prefers_installed_dir,
  test_user_dir_stays_trusted_with_installed_dir

Duplicate handler cleanup: handlers/skills.rs was the canonical implementation
but the handlers module was never compiled (not declared in web/mod.rs), so
server.rs had its own duplicate inline definitions that the router used.
Wire up the handlers module, delete the 260-line duplicate in server.rs, and
have server.rs import skills handlers from handlers::skills. Fix pre-existing
compile error in handlers/extensions.rs (missing needs_setup field). Add
#[allow(dead_code)] on not-yet-migrated handler modules to suppress warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: probe more Docker socket paths on macOS

Docker Desktop 4.13+ (stabilised in 4.18) no longer creates the
/var/run/docker.sock symlink by default. The API socket lives at
~/.docker/run/docker.sock, which bollard's connect_with_local_defaults()
does not try.

Add a fallback probe list covering the common macOS container runtimes:
- ~/.docker/run/docker.sock   — Docker Desktop 4.13+
- ~/.colima/default/docker.sock — Colima
- ~/.rd/docker.sock             — Rancher Desktop

Remove the bogus ~/.docker/desktop/docker.sock path that was added
previously; it is not an API socket on any known Docker installation.

Fixes the false-negative "Docker is installed but not running" warning
reported by Illia on macOS with Docker Desktop 4.18+.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* Harden Docker detection for rootless Linux and Windows fallback

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-23 10:04:02 -08:00
f4ba85ffa2 fix: fall back to build-from-source when extension download fails (#312)
* fix: fall back to build-from-source when extension download fails

Extension manifests hardcode GitHub release URLs for WASM artifacts,
but these artifacts are not yet published to any release. This causes
all WASM extension installs to fail with HTTP 404.

Add a fallback_source field to RegistryEntry so that when the primary
WasmDownload source fails (e.g., 404), the installer automatically
falls back to WasmBuildable (build from source). The manifest
conversion now populates this fallback whenever a download URL is set.

Fixes nearai/ironclaw#298

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address Copilot/Gemini review feedback

- Skip fallback for AlreadyInstalled errors (Gemini)
- Include both primary and fallback errors in combined message (Copilot)
- Fix comment to match broader behavior (any error, not just download) (Copilot)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address serrrfirat review feedback

- Forward AlreadyInstalled from fallback directly instead of wrapping
  in ExtensionError::Other (defensive, prevents misleading error message)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add unit tests for fallback install logic

Extract fallback_decision() and combine_install_errors() from
install_from_entry() to enable direct unit testing without requiring
a full ExtensionManager setup.

Tests cover:
- Primary success returns directly (no fallback attempted)
- AlreadyInstalled short-circuits (no fallback attempted)
- Download failure with fallback available triggers fallback
- Error without fallback source returns primary error
- Both-fail produces combined error with both messages
- AlreadyInstalled from fallback is forwarded directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-23 06:51:43 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
ebb4ce95e3 chore: release v0.11.1 (#319)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 01:24:01 +00:00
Illia Polosukhin 27c9353eaa Ignore out-of-date generated CI so custom release.yml jobs are allowed 2026-02-22 16:51:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
004906e582 chore: release v0.11.0 (#318)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-02-23 00:41:14 +00:00
65 changed files with 8699 additions and 618 deletions
+24 -1
View File
@@ -4,7 +4,7 @@ DATABASE_POOL_SIZE=10
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil, openai_codex
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
@@ -57,6 +57,18 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_...
# === OpenAI Codex (Responses API) ===
# Two auth modes:
# 1. API key: Standard OpenAI billing (api.openai.com/v1/responses)
# 2. Codex CLI OAuth: ChatGPT subscription billing (chatgpt.com)
# Reads token from ~/.codex/auth.json (or $CODEX_HOME/auth.json)
# OPENAI_CODEX_MODEL=gpt-5.3-codex
# LLM_BACKEND=openai_codex
# OPENAI_CODEX_API_KEY=sk-... # API key mode
# CODEX_AUTH_PATH=~/.codex/auth.json # OAuth mode (default path)
# OPENAI_CODEX_ACCOUNT_ID=... # Required for ChatGPT endpoint
# OPENAI_CODEX_BASE_URL=... # Override base URL
# For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration
@@ -75,6 +87,17 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Signal Channel (optional, requires signal-cli daemon --http)
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
# SIGNAL_ACCOUNT=+1234567890
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
# SIGNAL_IGNORE_ATTACHMENTS=false
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
+79 -2
View File
@@ -89,10 +89,12 @@ jobs:
# Build and packages all the platform-specific things
build-local-artifacts:
name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
# Let the initial task tell us to not run (currently very blunt)
# Wait for WASM extensions so we can patch manifests with SHA256 checksums
# before build.rs bakes them into the embedded catalog.
needs:
- plan
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
- build-wasm-extensions
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by dist in create-release.
@@ -139,6 +141,28 @@ jobs:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
- name: Patch manifests with WASM checksums
if: ${{ needs.plan.outputs.publishing == 'true' }}
shell: bash
run: |
CHECKSUMS="target/distrib/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found, skipping manifest patching"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Install dependencies
run: |
${{ matrix.packages_install }}
@@ -380,6 +404,59 @@ jobs:
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
# Commit patched manifest SHA256 checksums back to main so the repo
# stays in sync with the released artifacts.
update-registry-checksums:
needs:
- plan
- host
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
- name: Fetch WASM checksums
uses: actions/download-artifact@v4
with:
name: artifacts-wasm-extensions
path: target/wasm-bundles/
- name: Patch manifests with SHA256
shell: bash
run: |
CHECKSUMS="target/wasm-bundles/checksums.txt"
if [ ! -f "$CHECKSUMS" ]; then
echo "No checksums.txt found"
exit 0
fi
while IFS= read -r line; do
sha256=$(echo "$line" | awk '{print $1}')
filename=$(echo "$line" | awk '{print $2}')
name=$(echo "$filename" | sed 's/-wasm32-wasip2\.tar\.gz$//')
for manifest in registry/tools/${name}.json registry/channels/${name}.json; do
if [ -f "$manifest" ]; then
jq --arg sha "$sha256" '.artifacts["wasm32-wasip2"].sha256 = $sha' "$manifest" > "${manifest}.tmp" && mv "${manifest}.tmp" "$manifest"
echo "Patched $manifest with sha256=$sha256"
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add registry/
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
fi
announce:
needs:
- plan
+38 -1
View File
@@ -7,7 +7,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-22
## [0.12.0](https://github.com/nearai/ironclaw/compare/v0.11.1...v0.12.0) - 2026-02-26
### Added
- *(web)* improve WASM channel setup flow ([#380](https://github.com/nearai/ironclaw/pull/380))
- *(web)* inline tool activity cards with auto-collapsing ([#376](https://github.com/nearai/ironclaw/pull/376))
- *(web)* display logs newest-first in web gateway UI ([#369](https://github.com/nearai/ironclaw/pull/369))
- *(signal)* tool approval workflow and status updates ([#350](https://github.com/nearai/ironclaw/pull/350))
- add OpenRouter preset to setup wizard ([#270](https://github.com/nearai/ironclaw/pull/270))
- *(channels)* add native Signal channel via signal-cli HTTP daemon ([#271](https://github.com/nearai/ironclaw/pull/271))
### Fixed
- correct MCP registry URLs and remove non-existent Google endpoints ([#370](https://github.com/nearai/ironclaw/pull/370))
- resolve_thread adopts existing session threads by UUID ([#377](https://github.com/nearai/ironclaw/pull/377))
- resolve telegram/slack name collision between tool and channel registries ([#346](https://github.com/nearai/ironclaw/pull/346))
- make onboarding installs prefer release artifacts with source fallback ([#323](https://github.com/nearai/ironclaw/pull/323))
- copy missing files in Dockerfile to fix build ([#322](https://github.com/nearai/ironclaw/pull/322))
- fall back to build-from-source when extension download fails ([#312](https://github.com/nearai/ironclaw/pull/312))
### Other
- Add --version flag with clap built-in support and test ([#342](https://github.com/nearai/ironclaw/pull/342))
- Update FEATURE_PARITY.md ([#337](https://github.com/nearai/ironclaw/pull/337))
- add brew install ironclaw instructions ([#310](https://github.com/nearai/ironclaw/pull/310))
- Fix skills system: enable by default, fix registry and install ([#300](https://github.com/nearai/ironclaw/pull/300))
## [0.11.1](https://github.com/nearai/ironclaw/compare/v0.11.0...v0.11.1) - 2026-02-23
### Other
- Ignore out-of-date generated CI so custom release.yml jobs are allowed
## [0.11.0](https://github.com/nearai/ironclaw/compare/v0.10.0...v0.11.0) - 2026-02-23
### Fixed
- auto-compact and retry on ContextLengthExceeded ([#315](https://github.com/nearai/ironclaw/pull/315))
### Other
Generated
+9 -1
View File
@@ -2700,7 +2700,7 @@ dependencies = [
[[package]]
name = "ironclaw"
version = "0.11.0"
version = "0.12.0"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -2728,6 +2728,7 @@ dependencies = [
"hyper 1.8.1",
"hyper-util",
"libsql",
"lru",
"mime_guess",
"open",
"pgvector",
@@ -5031,6 +5032,12 @@ dependencies = [
"digest",
]
[[package]]
name = "sha1_smol"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d"
[[package]]
name = "sha2"
version = "0.10.9"
@@ -6206,6 +6213,7 @@ dependencies = [
"getrandom 0.4.1",
"js-sys",
"serde_core",
"sha1_smol",
"wasm-bindgen",
]
+5 -2
View File
@@ -19,7 +19,7 @@ exclude = [
[package]
name = "ironclaw"
version = "0.11.0"
version = "0.12.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -69,7 +69,7 @@ dotenvy = "0.15"
toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "serde"] }
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -149,6 +149,7 @@ bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
clap_complete = "4.5.0"
lru = "0.16.3"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
@@ -197,6 +198,8 @@ lto = "thin"
[workspace.metadata.dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# Ignore out-of-date generated CI so custom release.yml jobs are allowed
allow-dirty = ["ci"]
# CI backends to support
ci = "github"
# The installers to generate for each app
+8 -2
View File
@@ -11,16 +11,22 @@ FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/*
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source and build artifacts
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --bin ironclaw
+2 -3
View File
@@ -68,7 +68,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | | P2 | signal-cli |
| Signal | ✅ | | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
@@ -158,7 +158,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `doctor` | ✅ | ❌ | P2 | Diagnostics |
| `logs` | ✅ | ❌ | P3 | Query logs |
| `update` | ✅ | ❌ | P3 | Self-update |
| `completion` | ✅ | | P3 | Shell completion |
| `completion` | ✅ | | - | Shell completion |
| `/subagents spawn` | ✅ | ❌ | P3 | Spawn subagents from chat |
| `/export-session` | ✅ | ❌ | P3 | Export current session transcript |
@@ -540,7 +540,6 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Signal channel
- ❌ Matrix channel
- ❌ Other messaging platforms
- ❌ TTS/audio features
+9
View File
@@ -105,6 +105,15 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
```
</details>
<details>
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
```
</details>
<details>
<summary>Compile the source code (Cargo on Windows, Linux, macOS)</summary>
+1 -1
View File
@@ -32,7 +32,7 @@
"tools/gmail",
"tools/google-calendar",
"tools/google-drive",
"tools/slack",
"tools/slack-tool",
"channels/telegram",
"channels/slack"
],
+1 -1
View File
@@ -1,5 +1,5 @@
{
"name": "slack",
"name": "slack-tool",
"display_name": "Slack",
"kind": "tool",
"version": "0.1.0",
+1 -1
View File
@@ -1,5 +1,5 @@
{
"name": "telegram",
"name": "telegram-mtproto",
"display_name": "Telegram",
"kind": "tool",
"version": "0.1.0",
+5
View File
@@ -68,6 +68,7 @@ pub struct AgentDeps {
pub workspace: Option<Arc<Workspace>>,
pub extension_manager: Option<Arc<ExtensionManager>>,
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
pub skills_config: SkillsConfig,
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
@@ -174,6 +175,10 @@ impl Agent {
self.deps.skill_registry.as_ref()
}
pub(super) fn skill_catalog(&self) -> Option<&Arc<crate::skills::catalog::SkillCatalog>> {
self.deps.skill_catalog.as_ref()
}
/// Select active skills for a message using deterministic prefiltering.
pub(super) fn select_active_skills(
&self,
+154
View File
@@ -15,6 +15,17 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning};
/// Format a count with a suffix, using K/M abbreviations for large numbers.
fn format_count(n: u64, suffix: &str) -> String {
if n >= 1_000_000 {
format!("{:.1}M {}", n as f64 / 1_000_000.0, suffix)
} else if n >= 1_000 {
format!("{:.1}K {}", n as f64 / 1_000.0, suffix)
} else {
format!("{} {}", n, suffix)
}
}
impl Agent {
/// Handle job-related intents without turn tracking.
pub(super) async fn handle_job_or_command(
@@ -373,6 +384,10 @@ impl Agent {
" /thread <id> Switch to thread\n",
" /resume <id> Resume from checkpoint\n",
"\n",
"Skills:\n",
" /skills List installed skills\n",
" /skills search <q> Search ClawHub registry\n",
"\n",
"Agent:\n",
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
@@ -405,6 +420,22 @@ impl Agent {
))
}
"skills" => {
if args.first().map(|s| s.as_str()) == Some("search") {
let query = args[1..].join(" ");
if query.is_empty() {
return Ok(SubmissionResult::error("Usage: /skills search <query>"));
}
self.handle_skills_search(&query).await
} else if args.is_empty() {
self.handle_skills_list().await
} else {
Ok(SubmissionResult::error(
"Usage: /skills or /skills search <query>",
))
}
}
"model" => {
let current = self.llm().active_model_name();
@@ -475,6 +506,129 @@ impl Agent {
}
}
/// List installed skills.
async fn handle_skills_list(&self) -> Result<SubmissionResult, Error> {
let Some(registry) = self.skill_registry() else {
return Ok(SubmissionResult::error("Skills system not enabled."));
};
let guard = match registry.read() {
Ok(g) => g,
Err(e) => {
return Ok(SubmissionResult::error(format!(
"Skill registry lock error: {}",
e
)));
}
};
let skills = guard.skills();
if skills.is_empty() {
return Ok(SubmissionResult::response(
"No skills installed.\n\nUse /skills search <query> to find skills on ClawHub.",
));
}
let mut out = String::from("Installed skills:\n\n");
for s in skills {
let desc = if s.manifest.description.chars().count() > 60 {
let truncated: String = s.manifest.description.chars().take(57).collect();
format!("{}...", truncated)
} else {
s.manifest.description.clone()
};
out.push_str(&format!(
" {:<24} v{:<10} [{}] {}\n",
s.manifest.name, s.manifest.version, s.trust, desc,
));
}
out.push_str("\nUse /skills search <query> to find more on ClawHub.");
Ok(SubmissionResult::response(out))
}
/// Search ClawHub for skills.
async fn handle_skills_search(&self, query: &str) -> Result<SubmissionResult, Error> {
let catalog = match self.skill_catalog() {
Some(c) => c,
None => {
return Ok(SubmissionResult::error("Skill catalog not available."));
}
};
let outcome = catalog.search(query).await;
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let mut out = format!("ClawHub results for \"{}\":\n\n", query);
if entries.is_empty() {
if let Some(ref err) = outcome.error {
out.push_str(&format!(" (registry error: {})\n", err));
} else {
out.push_str(" No results found.\n");
}
} else {
for entry in &entries {
let owner_str = entry
.owner
.as_deref()
.map(|o| format!(" by {}", o))
.unwrap_or_default();
let stats_parts: Vec<String> = [
entry.stars.map(|s| format!("{} stars", s)),
entry.downloads.map(|d| format_count(d, "downloads")),
]
.into_iter()
.flatten()
.collect();
let stats_str = if stats_parts.is_empty() {
String::new()
} else {
format!(" {}", stats_parts.join(" "))
};
out.push_str(&format!(
" {:<24} v{:<10}{}{}\n",
entry.name, entry.version, owner_str, stats_str,
));
if !entry.description.is_empty() {
out.push_str(&format!(" {}\n\n", entry.description));
}
}
}
// Show matching installed skills
if let Some(registry) = self.skill_registry()
&& let Ok(guard) = registry.read()
{
let query_lower = query.to_lowercase();
let matches: Vec<_> = guard
.skills()
.iter()
.filter(|s| {
s.manifest.name.to_lowercase().contains(&query_lower)
|| s.manifest.description.to_lowercase().contains(&query_lower)
})
.collect();
if !matches.is_empty() {
out.push_str(&format!("Installed skills matching \"{}\":\n", query));
for s in &matches {
out.push_str(&format!(
" {:<24} v{:<10} [{}]\n",
s.manifest.name, s.manifest.version, s.trust,
));
}
}
}
Ok(SubmissionResult::response(out))
}
/// Handle legacy command routing from the Router (job commands that go through
/// process_user_input -> router -> handle_job_or_command -> here).
pub(super) async fn handle_command(
+10
View File
@@ -212,6 +212,15 @@ impl Agent {
);
}
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::Thinking("Calling LLM...".into()),
&message.metadata,
)
.await;
let output = match reasoning.respond_with_tools(&context).await {
Ok(output) => output,
Err(crate::error::LlmError::ContextLengthExceeded { used, limit }) => {
@@ -960,6 +969,7 @@ mod tests {
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
+75
View File
@@ -128,6 +128,42 @@ impl SessionManager {
}
}
// Check if external_thread_id is itself a known thread UUID that
// exists in the session but was never registered in the thread_map
// (e.g. created by chat_new_thread_handler or hydrated from DB).
// We only adopt it if no thread_map entry maps to this UUID —
// otherwise it belongs to a different channel scope.
if let Some(ext_tid) = external_thread_id
&& let Ok(ext_uuid) = Uuid::parse_str(ext_tid)
{
let thread_map = self.thread_map.read().await;
let mapped_elsewhere = thread_map.values().any(|&v| v == ext_uuid);
drop(thread_map);
if !mapped_elsewhere {
let sess = session.lock().await;
if sess.threads.contains_key(&ext_uuid) {
drop(sess);
let mut thread_map = self.thread_map.write().await;
// Re-check after acquiring write lock to prevent race condition
// where another task mapped this UUID between our read and write.
if !thread_map.values().any(|&v| v == ext_uuid) {
thread_map.insert(key, ext_uuid);
drop(thread_map);
// Ensure undo manager exists
let mut undo_managers = self.undo_managers.write().await;
undo_managers
.entry(ext_uuid)
.or_insert_with(|| Arc::new(Mutex::new(UndoManager::new())));
return (session, ext_uuid);
}
// If it was mapped elsewhere while we were unlocked, fall through
// to create a new thread, preserving channel isolation.
}
}
}
// Create new thread (always create a new one for a new key)
let thread_id = {
let mut sess = session.lock().await;
@@ -735,4 +771,43 @@ mod tests {
.await;
assert_ne!(resolved, tid);
}
#[tokio::test]
async fn test_resolve_thread_finds_existing_session_thread_by_uuid() {
use crate::agent::session::{Session, Thread};
let manager = SessionManager::new();
let tid = Uuid::new_v4();
// Simulate chat_new_thread_handler: create thread directly in session
// without registering it in thread_map
let session = Arc::new(Mutex::new(Session::new("user-direct")));
{
let mut sess = session.lock().await;
let thread = Thread::with_id(tid, sess.id);
sess.threads.insert(tid, thread);
}
{
let mut sessions = manager.sessions.write().await;
sessions.insert("user-direct".to_string(), Arc::clone(&session));
}
// resolve_thread should find the existing thread by UUID
// instead of creating a duplicate
let (_, resolved) = manager
.resolve_thread("user-direct", "gateway", Some(&tid.to_string()))
.await;
assert_eq!(
resolved, tid,
"should reuse existing thread, not create a new one"
);
// Verify no duplicate threads were created
let sess = session.lock().await;
assert_eq!(
sess.threads.len(),
1,
"should have exactly 1 thread, not a duplicate"
);
}
}
+47
View File
@@ -62,6 +62,23 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/skills" {
return Submission::SystemCommand {
command: "skills".to_string(),
args: vec![],
};
}
if lower.starts_with("/skills ") {
let args: Vec<String> = trimmed
.split_whitespace()
.skip(1)
.map(|s| s.to_string())
.collect();
return Submission::SystemCommand {
command: "skills".to_string(),
args,
};
}
if lower == "/ping" {
return Submission::SystemCommand {
command: "ping".to_string(),
@@ -693,6 +710,36 @@ mod tests {
assert!(!submission.starts_turn());
}
#[test]
fn test_parser_system_command_skills() {
let submission = SubmissionParser::parse("/skills");
assert!(
matches!(submission, Submission::SystemCommand { command, args } if command == "skills" && args.is_empty())
);
// Case insensitive
let submission = SubmissionParser::parse("/SKILLS");
assert!(
matches!(submission, Submission::SystemCommand { command, .. } if command == "skills")
);
}
#[test]
fn test_parser_system_command_skills_search() {
let submission = SubmissionParser::parse("/skills search markdown");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "markdown"])
);
// Multiple words in query
let submission = SubmissionParser::parse("/skills search code review tools");
assert!(
matches!(submission, Submission::SystemCommand { command, args }
if command == "skills" && args == vec!["search", "code", "review", "tools"])
);
}
#[test]
fn test_parser_quit() {
assert!(matches!(SubmissionParser::parse("/quit"), Submission::Quit));
+2 -1
View File
@@ -692,7 +692,8 @@ impl AppBuilder {
// Skills system
let (skill_registry, skill_catalog) = if self.config.skills.enabled {
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone());
let mut registry = SkillRegistry::new(self.config.skills.local_dir.clone())
.with_installed_dir(self.config.skills.installed_dir.clone());
let loaded = registry.discover_all().await;
if !loaded.is_empty() {
tracing::info!("Loaded {} skill(s): {}", loaded.len(), loaded.join(", "));
+26 -2
View File
@@ -20,8 +20,10 @@ pub struct BootInfo {
pub heartbeat_enabled: bool,
pub heartbeat_interval_secs: u64,
pub sandbox_enabled: bool,
pub docker_status: crate::sandbox::detect::DockerStatus,
pub claude_code_enabled: bool,
pub routines_enabled: bool,
pub skills_enabled: bool,
pub channels: Vec<String>,
/// Public URL from a managed tunnel (e.g., "https://abc.ngrok.io").
pub tunnel_url: Option<String>,
@@ -35,6 +37,7 @@ pub fn print_boot_screen(info: &BootInfo) {
let bold = "\x1b[1m";
let cyan = "\x1b[36m";
let dim = "\x1b[90m";
let yellow = "\x1b[33m";
let yellow_underline = "\x1b[33;4m";
let reset = "\x1b[0m";
@@ -90,8 +93,19 @@ pub fn print_boot_screen(info: &BootInfo) {
let mins = info.heartbeat_interval_secs / 60;
features.push(format!("heartbeat ({mins}m)"));
}
if info.sandbox_enabled {
features.push("sandbox".to_string());
match info.docker_status {
crate::sandbox::detect::DockerStatus::Available => {
features.push("sandbox".to_string());
}
crate::sandbox::detect::DockerStatus::NotInstalled => {
features.push(format!("{yellow}sandbox (docker not installed){reset}"));
}
crate::sandbox::detect::DockerStatus::NotRunning => {
features.push(format!("{yellow}sandbox (docker not running){reset}"));
}
crate::sandbox::detect::DockerStatus::Disabled => {
// Don't show sandbox when disabled
}
}
if info.claude_code_enabled {
features.push("claude-code".to_string());
@@ -99,6 +113,9 @@ pub fn print_boot_screen(info: &BootInfo) {
if info.routines_enabled {
features.push("routines".to_string());
}
if info.skills_enabled {
features.push("skills".to_string());
}
if !features.is_empty() {
println!(
" {dim}features{reset} {cyan}{}{reset}",
@@ -140,6 +157,7 @@ pub fn print_boot_screen(info: &BootInfo) {
#[cfg(test)]
mod tests {
use super::*;
use crate::sandbox::detect::DockerStatus;
#[test]
fn test_print_boot_screen_full() {
@@ -158,8 +176,10 @@ mod tests {
heartbeat_enabled: true,
heartbeat_interval_secs: 1800,
sandbox_enabled: true,
docker_status: DockerStatus::Available,
claude_code_enabled: false,
routines_enabled: true,
skills_enabled: true,
channels: vec![
"repl".to_string(),
"gateway".to_string(),
@@ -189,8 +209,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec![],
tunnel_url: None,
tunnel_provider: None,
@@ -216,8 +238,10 @@ mod tests {
heartbeat_enabled: false,
heartbeat_interval_secs: 0,
sandbox_enabled: false,
docker_status: DockerStatus::Disabled,
claude_code_enabled: false,
routines_enabled: false,
skills_enabled: false,
channels: vec!["repl".to_string()],
tunnel_url: None,
tunnel_provider: None,
+2
View File
@@ -31,6 +31,7 @@ mod channel;
mod http;
mod manager;
mod repl;
mod signal;
pub mod wasm;
pub mod web;
mod webhook_server;
@@ -39,5 +40,6 @@ pub use channel::{Channel, IncomingMessage, MessageStream, OutgoingResponse, Sta
pub use http::HttpChannel;
pub use manager::ChannelManager;
pub use repl::ReplChannel;
pub use signal::SignalChannel;
pub use web::GatewayChannel;
pub use webhook_server::{WebhookServer, WebhookServerConfig};
File diff suppressed because it is too large Load Diff
+35 -8
View File
@@ -24,16 +24,43 @@ pub async fn extensions_list_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let pairing_store = crate::pairing::PairingStore::new();
let extensions = installed
.into_iter()
.map(|ext| ExtensionInfo {
name: ext.name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
.map(|ext| {
let activation_status = if ext.kind == crate::extensions::ExtensionKind::WasmChannel {
Some(if ext.activation_error.is_some() {
"failed".to_string()
} else if !ext.authenticated {
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
.unwrap_or(false);
if has_paired {
"active".to_string()
} else {
"pairing".to_string()
}
} else {
"configured".to_string()
})
} else {
None
};
ExtensionInfo {
name: ext.name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
activation_status,
activation_error: ext.activation_error,
}
})
.collect();
+22 -17
View File
@@ -1,23 +1,28 @@
//! Handler modules for the web gateway API.
//!
//! Each module groups related endpoint handlers by domain.
//!
//! # Migration status
//!
//! `skills` is the canonical implementation used by `server.rs`.
//! The remaining modules are in-progress migrations from inline server.rs
//! handlers; their functions are not yet wired up, hence the `dead_code` allow.
pub mod chat;
pub mod extensions;
pub mod jobs;
pub mod memory;
pub mod routines;
pub mod settings;
pub mod skills;
pub mod static_files;
// Re-export all handler functions so `server.rs` can reference them
// as `handlers::chat_send_handler`, etc.
pub use chat::*;
pub use extensions::*;
pub use jobs::*;
pub use memory::*;
pub use routines::*;
pub use settings::*;
pub use skills::*;
pub use static_files::*;
// Modules not yet wired into server.rs router -- suppress dead_code until
// they replace their inline counterparts.
#[allow(dead_code)]
pub mod chat;
#[allow(dead_code)]
pub mod extensions;
#[allow(dead_code)]
pub mod jobs;
#[allow(dead_code)]
pub mod memory;
#[allow(dead_code)]
pub mod routines;
#[allow(dead_code)]
pub mod settings;
#[allow(dead_code)]
pub mod static_files;
+15 -4
View File
@@ -58,8 +58,14 @@ pub async fn skills_search_handler(
))?;
// Search ClawHub catalog
let catalog_results = catalog.search(&req.query).await;
let catalog_json: Vec<serde_json::Value> = catalog_results
let catalog_outcome = catalog.search(&req.query).await;
let catalog_error = catalog_outcome.error.clone();
// Enrich top results with detail data (stars, downloads, owner)
let mut entries = catalog_outcome.results;
catalog.enrich_search_results(&mut entries, 5).await;
let catalog_json: Vec<serde_json::Value> = entries
.into_iter()
.map(|e| {
serde_json::json!({
@@ -68,6 +74,10 @@ pub async fn skills_search_handler(
"description": e.description,
"version": e.version,
"score": e.score,
"updatedAt": e.updated_at,
"stars": e.stars,
"downloads": e.downloads,
"owner": e.owner,
})
})
.collect();
@@ -103,6 +113,7 @@ pub async fn skills_search_handler(
catalog: catalog_json,
installed,
registry_url: catalog.registry_url().to_string(),
catalog_error,
}))
}
@@ -147,7 +158,7 @@ pub async fn skills_install_handler(
)));
};
// Parse, check duplicates, and get user_dir under a brief read lock.
// Parse, check duplicates, and get install_dir under a brief read lock.
let (user_dir, skill_name_from_parse) = {
let guard = registry.read().map_err(|e| {
(
@@ -168,7 +179,7 @@ pub async fn skills_install_handler(
))));
}
(guard.user_dir().to_path_buf(), skill_name)
(guard.install_target_dir().to_path_buf(), skill_name)
};
// Perform async I/O (write to disk, load) with no lock held.
+3
View File
@@ -90,6 +90,9 @@ impl LogBroadcaster {
}
/// Snapshot of recent entries for replaying to a new subscriber.
///
/// Returns entries oldest-first so that the frontend's `prepend()`
/// naturally places the newest entry at the top of the DOM.
pub fn recent_entries(&self) -> Vec<LogEntry> {
self.recent
.lock()
+3
View File
@@ -15,6 +15,7 @@
//! ```
pub mod auth;
pub(crate) mod handlers;
pub mod log_layer;
pub mod openai_compat;
pub mod server;
@@ -92,6 +93,7 @@ impl GatewayChannel {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
Self {
@@ -125,6 +127,7 @@ impl GatewayChannel {
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
restart_requested: std::sync::atomic::AtomicBool::new(false),
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
+79 -257
View File
@@ -28,6 +28,9 @@ use uuid::Uuid;
use crate::agent::SessionManager;
use crate::channels::IncomingMessage;
use crate::channels::web::auth::{AuthState, auth_middleware};
use crate::channels::web::handlers::skills::{
skills_install_handler, skills_list_handler, skills_remove_handler, skills_search_handler,
};
use crate::channels::web::log_layer::LogBroadcaster;
use crate::channels::web::sse::SseManager;
use crate::channels::web::types::*;
@@ -155,6 +158,8 @@ pub struct GatewayState {
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
/// Flag set when a restart has been requested via the API.
pub restart_requested: std::sync::atomic::AtomicBool,
}
/// Start the gateway HTTP server.
@@ -235,6 +240,8 @@ pub async fn start_server(
"/api/extensions/{name}/setup",
get(extensions_setup_handler).post(extensions_setup_submit_handler),
)
// Gateway management
.route("/api/gateway/restart", post(gateway_restart_handler))
// Pairing
.route("/api/pairing/{channel}", get(pairing_list_handler))
.route(
@@ -1719,17 +1726,46 @@ async fn extensions_list_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let pairing_store = crate::pairing::PairingStore::new();
let extensions = installed
.into_iter()
.map(|ext| ExtensionInfo {
name: ext.name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
.map(|ext| {
let activation_status = if ext.kind == crate::extensions::ExtensionKind::WasmChannel {
Some(if ext.activation_error.is_some() {
"failed".to_string()
} else if !ext.authenticated {
// No credentials configured yet.
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
// Telegram: check pairing status (end-to-end setup via web UI).
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
.unwrap_or(false);
if has_paired {
"active".to_string()
} else {
"pairing".to_string()
}
} else {
// Authenticated but not fully active (or non-Telegram).
"configured".to_string()
})
} else {
None
};
ExtensionInfo {
name: ext.name,
kind: ext.kind.to_string(),
description: ext.description,
url: ext.url,
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
activation_status,
activation_error: ext.activation_error,
}
})
.collect();
@@ -2034,11 +2070,44 @@ async fn extensions_setup_submit_handler(
))?;
match ext_mgr.save_setup_secrets(&name, &req.secrets).await {
Ok(message) => Ok(Json(ActionResponse::ok(message))),
Ok(result) => {
let mut resp = ActionResponse::ok(result.message);
resp.activated = Some(result.activated);
if !result.activated {
resp.needs_restart = Some(true);
}
Ok(Json(resp))
}
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Gateway management handlers ---
async fn gateway_restart_handler(State(state): State<Arc<GatewayState>>) -> Json<ActionResponse> {
// Idempotency guard: only allow one restart at a time.
if state
.restart_requested
.compare_exchange(
false,
true,
std::sync::atomic::Ordering::SeqCst,
std::sync::atomic::Ordering::SeqCst,
)
.is_err()
{
return Json(ActionResponse::ok("Restart already in progress"));
}
// Take the shutdown sender and trigger graceful shutdown.
if let Some(tx) = state.shutdown_tx.write().await.take() {
let _ = tx.send(());
tracing::info!("Gateway restart requested via API");
}
Json(ActionResponse::ok("Restarting..."))
}
// --- Pairing handlers ---
async fn pairing_list_handler(
@@ -2086,253 +2155,6 @@ async fn pairing_approve_handler(
}
}
// --- Skills handlers ---
async fn skills_list_handler(
State(state): State<Arc<GatewayState>>,
) -> Result<Json<super::types::SkillListResponse>, (StatusCode, String)> {
let registry = state.skill_registry.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Skills system not enabled".to_string(),
))?;
let guard = registry.read().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
let skills: Vec<super::types::SkillInfo> = guard
.skills()
.iter()
.map(|s| super::types::SkillInfo {
name: s.manifest.name.clone(),
description: s.manifest.description.clone(),
version: s.manifest.version.clone(),
trust: s.trust.to_string(),
source: format!("{:?}", s.source),
keywords: s.manifest.activation.keywords.clone(),
})
.collect();
let count = skills.len();
Ok(Json(super::types::SkillListResponse { skills, count }))
}
async fn skills_search_handler(
State(state): State<Arc<GatewayState>>,
Json(req): Json<super::types::SkillSearchRequest>,
) -> Result<Json<super::types::SkillSearchResponse>, (StatusCode, String)> {
let registry = state.skill_registry.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Skills system not enabled".to_string(),
))?;
let catalog = state.skill_catalog.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Skill catalog not available".to_string(),
))?;
// Search ClawHub catalog
let catalog_results = catalog.search(&req.query).await;
let catalog_json: Vec<serde_json::Value> = catalog_results
.into_iter()
.map(|e| {
serde_json::json!({
"slug": e.slug,
"name": e.name,
"description": e.description,
"version": e.version,
"score": e.score,
})
})
.collect();
// Search local skills
let query_lower = req.query.to_lowercase();
let installed: Vec<super::types::SkillInfo> = {
let guard = registry.read().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
guard
.skills()
.iter()
.filter(|s| {
s.manifest.name.to_lowercase().contains(&query_lower)
|| s.manifest.description.to_lowercase().contains(&query_lower)
})
.map(|s| super::types::SkillInfo {
name: s.manifest.name.clone(),
description: s.manifest.description.clone(),
version: s.manifest.version.clone(),
trust: s.trust.to_string(),
source: format!("{:?}", s.source),
keywords: s.manifest.activation.keywords.clone(),
})
.collect()
};
Ok(Json(super::types::SkillSearchResponse {
catalog: catalog_json,
installed,
registry_url: catalog.registry_url().to_string(),
}))
}
async fn skills_install_handler(
State(state): State<Arc<GatewayState>>,
headers: axum::http::HeaderMap,
Json(req): Json<super::types::SkillInstallRequest>,
) -> Result<Json<ActionResponse>, (StatusCode, String)> {
// Require explicit confirmation header to prevent accidental installs.
// Chat tools have requires_approval(); this is the equivalent for the web API.
if headers
.get("x-confirm-action")
.and_then(|v| v.to_str().ok())
!= Some("true")
{
return Err((
StatusCode::BAD_REQUEST,
"Skill install requires X-Confirm-Action: true header".to_string(),
));
}
let registry = state.skill_registry.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Skills system not enabled".to_string(),
))?;
let content = if let Some(ref raw) = req.content {
raw.clone()
} else if let Some(ref url) = req.url {
// Fetch from explicit URL (with SSRF protection)
crate::tools::builtin::skill_tools::fetch_skill_content(url)
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
} else if let Some(ref catalog) = state.skill_catalog {
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), &req.name);
crate::tools::builtin::skill_tools::fetch_skill_content(&url)
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, e.to_string()))?
} else {
return Ok(Json(ActionResponse::fail(
"Provide 'content' or 'url' to install a skill".to_string(),
)));
};
// Parse, check duplicates, and get user_dir under a brief read lock.
let (user_dir, skill_name_from_parse) = {
let guard = registry.read().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
let normalized = crate::skills::normalize_line_endings(&content);
let parsed = crate::skills::parser::parse_skill_md(&normalized)
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?;
let skill_name = parsed.manifest.name.clone();
if guard.has(&skill_name) {
return Ok(Json(ActionResponse::fail(format!(
"Skill '{}' already exists",
skill_name
))));
}
(guard.user_dir().to_path_buf(), skill_name)
};
// Perform async I/O (write to disk, load) with no lock held.
let normalized = crate::skills::normalize_line_endings(&content);
let (skill_name, loaded_skill) =
crate::skills::registry::SkillRegistry::prepare_install_to_disk(
&user_dir,
&skill_name_from_parse,
&normalized,
)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Commit: brief write lock for in-memory addition
let mut guard = registry.write().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
match guard.commit_install(&skill_name, loaded_skill) {
Ok(()) => Ok(Json(ActionResponse::ok(format!(
"Skill '{}' installed",
skill_name
)))),
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
async fn skills_remove_handler(
State(state): State<Arc<GatewayState>>,
headers: axum::http::HeaderMap,
Path(name): Path<String>,
) -> Result<Json<ActionResponse>, (StatusCode, String)> {
// Require explicit confirmation header to prevent accidental removals.
if headers
.get("x-confirm-action")
.and_then(|v| v.to_str().ok())
!= Some("true")
{
return Err((
StatusCode::BAD_REQUEST,
"Skill removal requires X-Confirm-Action: true header".to_string(),
));
}
let registry = state.skill_registry.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Skills system not enabled".to_string(),
))?;
// Validate removal under a brief read lock
let skill_path = {
let guard = registry.read().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
guard
.validate_remove(&name)
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
};
// Delete files from disk (async I/O, no lock held)
crate::skills::registry::SkillRegistry::delete_skill_files(&skill_path)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Remove from in-memory registry under a brief write lock
let mut guard = registry.write().map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Skill registry lock poisoned: {}", e),
)
})?;
match guard.commit_remove(&name) {
Ok(()) => Ok(Json(ActionResponse::ok(format!(
"Skill '{}' removed",
name
)))),
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Routines handlers ---
async fn routines_list_handler(
+6
View File
@@ -42,6 +42,11 @@ impl SseManager {
let _ = self.tx.send(event);
}
/// Get a clone of the broadcast sender for use by other components.
pub fn sender(&self) -> broadcast::Sender<SseEvent> {
self.tx.clone()
}
/// Get current number of active connections.
pub fn connection_count(&self) -> u64 {
self.connection_count.load(Ordering::Relaxed)
@@ -120,6 +125,7 @@ impl SseManager {
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::Heartbeat => "heartbeat",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
Ok(Event::default().event(event_type).data(data))
});
File diff suppressed because it is too large Load Diff
+29
View File
@@ -42,6 +42,7 @@
<button data-tab="jobs">Jobs</button>
<button data-tab="routines">Routines</button>
<button data-tab="extensions">Extensions</button>
<button data-tab="skills">Skills</button>
<div class="spacer"></div>
<button class="status-logs-btn" data-tab="logs" title="Logs">Logs</button>
<div class="tee-shield" id="tee-shield" style="display:none" title="Running in a Trusted Execution Environment">
@@ -231,6 +232,34 @@
</div>
</div>
</div>
<!-- Skills Tab -->
<div class="tab-panel" id="tab-skills">
<div class="extensions-container">
<div class="extensions-section">
<h3>Search ClawHub</h3>
<div class="skill-search-box">
<input type="text" id="skill-search-input" placeholder="Search for skills...">
<button onclick="searchClawHub()">Search</button>
</div>
<div class="extensions-list" id="skill-search-results"></div>
</div>
<div class="extensions-section">
<h3>Installed Skills</h3>
<div class="extensions-list" id="skills-list">
<div class="empty-state">Loading skills...</div>
</div>
</div>
<div class="extensions-section">
<h3>Install Skill by URL</h3>
<div class="ext-install-form">
<input type="text" id="skill-install-name" placeholder="Skill name or slug">
<input type="text" id="skill-install-url" placeholder="HTTPS URL to SKILL.md (optional)">
<button onclick="installSkillFromForm()">Install</button>
</div>
</div>
</div>
</div>
</div>
<div id="toasts"></div>
+449
View File
@@ -483,6 +483,7 @@ body {
to { transform: rotate(360deg); }
}
.scroll-load-spinner {
display: flex;
align-items: center;
@@ -502,6 +503,225 @@ body {
animation: spin 0.6s linear infinite;
}
/* === Tool Activity Cards === */
.activity-group {
align-self: flex-start;
max-width: 80%;
padding: 4px 0 4px 12px;
border-left: 2px solid var(--border);
margin: 4px 0;
display: flex;
flex-direction: column;
gap: 2px;
}
.activity-group.collapsed {
border-left-color: transparent;
padding-left: 0;
}
/* Thinking indicator */
.activity-thinking {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 8px;
font-size: 13px;
color: var(--text-secondary);
}
.activity-thinking-dots {
display: flex;
gap: 3px;
}
.activity-thinking-dot {
width: 5px;
height: 5px;
border-radius: 50%;
background: var(--text-secondary);
animation: thinkingPulse 1.4s ease-in-out infinite;
}
.activity-thinking-dot:nth-child(2) { animation-delay: 0.2s; }
.activity-thinking-dot:nth-child(3) { animation-delay: 0.4s; }
@keyframes thinkingPulse {
0%, 80%, 100% { opacity: 0.3; transform: scale(0.8); }
40% { opacity: 1; transform: scale(1); }
}
.activity-thinking-text {
font-style: italic;
}
/* Tool card */
.activity-tool-card {
background: var(--bg-secondary);
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
transition: border-color 0.2s;
}
.activity-tool-card[data-status="running"] {
border-color: rgba(52, 211, 153, 0.3);
}
.activity-tool-card[data-status="fail"] {
border-color: rgba(230, 76, 76, 0.3);
}
.activity-tool-header {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 10px;
cursor: pointer;
user-select: none;
transition: background 0.15s;
}
.activity-tool-header:hover {
background: var(--bg-tertiary);
}
.activity-tool-icon {
display: flex;
align-items: center;
justify-content: center;
width: 16px;
height: 16px;
flex-shrink: 0;
}
.activity-tool-icon .spinner {
width: 12px;
height: 12px;
border: 2px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
.activity-icon-success {
color: var(--success);
font-size: 14px;
font-weight: 700;
line-height: 1;
}
.activity-icon-fail {
color: var(--danger);
font-size: 14px;
font-weight: 700;
line-height: 1;
}
.activity-tool-name {
font-size: 13px;
font-family: var(--font-mono);
font-weight: 500;
color: var(--text);
flex: 1;
}
.activity-tool-duration {
font-size: 11px;
font-family: var(--font-mono);
color: var(--text-secondary);
min-width: 36px;
text-align: right;
}
.activity-tool-chevron {
font-size: 10px;
color: var(--text-secondary);
transition: transform 0.15s ease;
width: 12px;
text-align: center;
}
.activity-tool-chevron.expanded {
transform: rotate(90deg);
}
.activity-tool-body {
border-top: 1px solid var(--border);
}
.activity-tool-output {
margin: 0;
padding: 8px 10px;
font-family: var(--font-mono);
font-size: 12px;
line-height: 1.4;
color: var(--text-secondary);
background: var(--code-bg);
max-height: 200px;
overflow-y: auto;
white-space: pre-wrap;
word-break: break-all;
}
/* Collapsed summary */
.activity-summary {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 10px;
cursor: pointer;
user-select: none;
font-size: 13px;
color: var(--text-secondary);
border-radius: var(--radius);
transition: background 0.15s;
}
.activity-summary:hover {
background: var(--bg-tertiary);
}
.activity-summary-chevron {
font-size: 10px;
transition: transform 0.15s ease;
width: 12px;
text-align: center;
}
.activity-summary-chevron.expanded {
transform: rotate(90deg);
}
.activity-summary-text {
font-weight: 500;
}
.activity-summary-duration {
font-family: var(--font-mono);
font-size: 11px;
opacity: 0.7;
}
.activity-cards-container {
display: flex;
flex-direction: column;
gap: 2px;
padding-left: 12px;
border-left: 2px solid var(--border);
margin-top: 2px;
}
@media (max-width: 768px) {
.activity-group {
max-width: 95%;
}
}
/* Approval card (inline in chat) */
.approval-card {
align-self: flex-start;
@@ -1936,6 +2156,159 @@ body {
font-weight: 500;
}
/* WASM channel setup stepper */
.ext-stepper {
display: flex;
align-items: center;
gap: 0;
margin: 8px 0 4px;
}
.stepper-step {
display: flex;
align-items: center;
gap: 4px;
}
.stepper-circle {
width: 20px;
height: 20px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
font-size: 11px;
font-weight: 700;
flex-shrink: 0;
}
.stepper-label {
font-size: 11px;
white-space: nowrap;
}
.stepper-step.completed .stepper-circle {
background: var(--success);
color: #000;
}
.stepper-step.completed .stepper-label {
color: var(--success);
}
.stepper-step.failed .stepper-circle {
background: var(--danger);
color: #fff;
}
.stepper-step.failed .stepper-label {
color: var(--danger);
}
.stepper-step.in-progress .stepper-circle {
background: var(--warning);
color: #000;
animation: pulse-glow 1.5s ease-in-out infinite;
}
.stepper-step.in-progress .stepper-label {
color: var(--warning);
}
@keyframes pulse-glow {
0%, 100% { opacity: 1; }
50% { opacity: 0.6; }
}
.stepper-step.pending .stepper-circle {
background: var(--bg-tertiary);
border: 1px solid var(--border);
color: var(--text-secondary);
}
.stepper-step.pending .stepper-label {
color: var(--text-secondary);
}
.ext-pairing-label {
font-size: 12px;
color: var(--warning);
font-weight: 500;
}
.stepper-connector {
width: 20px;
height: 2px;
background: var(--border);
margin: 0 4px;
flex-shrink: 0;
}
.stepper-connector.completed {
background: var(--success);
}
.ext-error {
font-size: 11px;
color: var(--danger);
background: rgba(230, 76, 76, 0.1);
border: 1px solid rgba(230, 76, 76, 0.2);
border-radius: var(--radius);
padding: 6px 8px;
margin-top: 6px;
}
.ext-note {
font-size: 11px;
color: var(--text-secondary);
background: rgba(255, 255, 255, 0.04);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 6px 8px;
margin-top: 6px;
}
/* Restart overlay */
.restart-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.8);
z-index: 2000;
display: flex;
align-items: center;
justify-content: center;
}
.restart-message {
text-align: center;
color: var(--text);
}
.restart-message h2 {
margin: 16px 0 8px;
}
.restart-message p {
color: var(--text-secondary);
}
.restart-spinner {
width: 40px;
height: 40px;
border: 3px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin: 0 auto;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.btn-ext {
padding: 4px 10px;
border-radius: var(--radius);
@@ -2809,6 +3182,82 @@ mark {
transform: scale(0.98);
}
/* --- Skills tab --- */
.skill-search-box {
display: flex;
gap: 8px;
align-items: center;
margin-bottom: 12px;
}
.skill-search-box input {
flex: 1;
padding: 8px 12px;
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius);
color: var(--text);
font-size: 13px;
}
.skill-search-box input:focus {
outline: none;
border-color: var(--accent);
box-shadow: 0 0 0 3px rgba(52, 211, 153, 0.1);
}
.skill-search-box button {
padding: 8px 20px;
background: var(--accent);
color: #09090b;
border: none;
border-radius: var(--radius);
cursor: pointer;
font-size: 13px;
font-weight: 600;
transition: background 0.2s, transform 0.2s;
}
.skill-search-box button:hover {
background: var(--accent-hover);
transform: translateY(-1px);
}
.skill-trust {
font-size: 10px;
padding: 2px 6px;
border-radius: 8px;
font-weight: 500;
text-transform: uppercase;
letter-spacing: 0.3px;
}
.skill-trust.trust-trusted {
background: rgba(52, 211, 153, 0.15);
color: var(--success);
}
.skill-trust.trust-installed {
background: rgba(96, 165, 250, 0.15);
color: #60a5fa;
}
.skill-version {
font-size: 11px;
color: var(--text-secondary);
font-family: var(--font-mono);
}
@keyframes skillFadeIn {
from { opacity: 0; transform: translateY(8px); }
to { opacity: 1; transform: translateY(0); }
}
.skill-search-result {
animation: skillFadeIn 0.3s ease-out both;
}
/* --- Activity toolbar --- */
.activity-toolbar {
+29
View File
@@ -193,6 +193,15 @@ pub enum SseEvent {
#[serde(skip_serializing_if = "Option::is_none")]
session_id: Option<String>,
},
/// Extension activation status change (WASM channels).
#[serde(rename = "extension_status")]
ExtensionStatus {
extension_name: String,
status: String,
#[serde(skip_serializing_if = "Option::is_none")]
message: Option<String>,
},
}
// --- Memory ---
@@ -349,6 +358,12 @@ pub struct ExtensionInfo {
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
/// WASM channel activation status: "installed", "configured", "active", "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_status: Option<String>,
/// Human-readable error when activation_status is "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_error: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -412,6 +427,12 @@ pub struct ActionResponse {
/// Instructions for manual token entry.
#[serde(skip_serializing_if = "Option::is_none")]
pub instructions: Option<String>,
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a gateway restart is needed (activation failed).
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
}
impl ActionResponse {
@@ -422,6 +443,8 @@ impl ActionResponse {
auth_url: None,
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
@@ -432,6 +455,8 @@ impl ActionResponse {
auth_url: None,
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
}
@@ -508,6 +533,9 @@ pub struct SkillSearchResponse {
pub catalog: Vec<serde_json::Value>,
pub installed: Vec<SkillInfo>,
pub registry_url: String,
/// If the catalog registry was unreachable or errored, a human-readable message.
#[serde(skip_serializing_if = "Option::is_none")]
pub catalog_error: Option<String>,
}
#[derive(Debug, Deserialize)]
@@ -609,6 +637,7 @@ impl WsServerMessage {
SseEvent::JobToolResult { .. } => "job_tool_result",
SseEvent::JobStatus { .. } => "job_status",
SseEvent::JobResult { .. } => "job_result",
SseEvent::ExtensionStatus { .. } => "extension_status",
};
let data = serde_json::to_value(event).unwrap_or(serde_json::Value::Null);
WsServerMessage::Event {
+1
View File
@@ -493,6 +493,7 @@ mod tests {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
}
}
}
+15
View File
@@ -166,3 +166,18 @@ impl Cli {
matches!(self.command, None | Some(Command::Run))
}
}
#[cfg(test)]
mod tests {
use super::*;
use clap::CommandFactory;
#[test]
fn test_version() {
let cmd = Cli::command();
assert_eq!(
cmd.get_version().unwrap_or("unknown"),
env!("CARGO_PKG_VERSION")
);
}
}
+97
View File
@@ -12,6 +12,7 @@ pub struct ChannelsConfig {
pub cli: CliConfig,
pub http: Option<HttpConfig>,
pub gateway: Option<GatewayConfig>,
pub signal: Option<SignalConfig>,
/// Directory containing WASM channel modules (default: ~/.ironclaw/channels/).
pub wasm_channels_dir: std::path::PathBuf,
/// Whether WASM channels are enabled.
@@ -43,6 +44,49 @@ pub struct GatewayConfig {
pub user_id: String,
}
/// Signal channel configuration (signal-cli daemon HTTP/JSON-RPC).
#[derive(Debug, Clone)]
pub struct SignalConfig {
/// Base URL of the signal-cli daemon HTTP endpoint (e.g. `http://127.0.0.1:8080`).
pub http_url: String,
/// Signal account identifier (E.164 phone number, e.g. `+1234567890`).
pub account: String,
/// Users allowed to interact with the bot in DMs.
///
/// Each entry is one of:
/// - `*` — allow everyone
/// - E.164 phone number (e.g. `+1234567890`)
/// - bare UUID (e.g. `a1b2c3d4-e5f6-7890-abcd-ef1234567890`)
/// - `uuid:<id>` prefix form (e.g. `uuid:a1b2c3d4-e5f6-7890-abcd-ef1234567890`)
///
/// An empty list denies all senders (secure by default).
pub allow_from: Vec<String>,
/// Groups allowed to interact with the bot.
///
/// - Empty list — deny all group messages (DMs only, secure by default).
/// - `*` — allow all groups.
/// - Specific group IDs — allow only those groups.
pub allow_from_groups: Vec<String>,
/// DM policy: "open", "allowlist", or "pairing". Default: "pairing".
///
/// - "open" — allow all DM senders (ignores allow_from for DMs)
/// - "allowlist" — only allow senders in allow_from list
/// - "pairing" — allowlist + send pairing reply to unknown users
pub dm_policy: String,
/// Group policy: "allowlist", "open", or "disabled". Default: "allowlist".
///
/// - "disabled" — deny all group messages
/// - "allowlist" — check allow_from_groups and group_allow_from
/// - "open" — accept all group messages (respects allow_from_groups for group ID)
pub group_policy: String,
/// Allow list for group message senders. If empty, inherits from allow_from.
pub group_allow_from: Vec<String>,
/// Skip messages that contain only attachments (no text).
pub ignore_attachments: bool,
/// Skip story messages.
pub ignore_stories: bool,
}
impl ChannelsConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
let http = if optional_env("HTTP_PORT")?.is_some() || optional_env("HTTP_HOST")?.is_some() {
@@ -68,6 +112,58 @@ impl ChannelsConfig {
None
};
let signal = if let Some(http_url) = optional_env("SIGNAL_HTTP_URL")? {
let account = optional_env("SIGNAL_ACCOUNT")?.ok_or(ConfigError::InvalidValue {
key: "SIGNAL_ACCOUNT".to_string(),
message: "SIGNAL_ACCOUNT is required when SIGNAL_HTTP_URL is set".to_string(),
})?;
let allow_from = match std::env::var_os("SIGNAL_ALLOW_FROM") {
None => vec![account.clone()],
Some(val) => {
let s = val.to_string_lossy();
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
}
};
let dm_policy =
optional_env("SIGNAL_DM_POLICY")?.unwrap_or_else(|| "pairing".to_string());
let group_policy =
optional_env("SIGNAL_GROUP_POLICY")?.unwrap_or_else(|| "allowlist".to_string());
Some(SignalConfig {
http_url,
account,
allow_from,
allow_from_groups: optional_env("SIGNAL_ALLOW_FROM_GROUPS")?
.map(|s| {
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_default(),
dm_policy,
group_policy,
group_allow_from: optional_env("SIGNAL_GROUP_ALLOW_FROM")?
.map(|s| {
s.split(',')
.map(|e| e.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_default(),
ignore_attachments: optional_env("SIGNAL_IGNORE_ATTACHMENTS")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(false),
ignore_stories: optional_env("SIGNAL_IGNORE_STORIES")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(true),
})
} else {
None
};
let cli_enabled = optional_env("CLI_ENABLED")?
.map(|s| s.to_lowercase() != "false" && s != "0")
.unwrap_or(true);
@@ -78,6 +174,7 @@ impl ChannelsConfig {
},
http,
gateway,
signal,
wasm_channels_dir: optional_env("WASM_CHANNELS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
+263 -1
View File
@@ -25,6 +25,8 @@ pub enum LlmBackend {
OpenAiCompatible,
/// Tinfoil private inference
Tinfoil,
/// OpenAI Codex via Responses API (ChatGPT OAuth or API key)
OpenAiCodex,
}
impl std::str::FromStr for LlmBackend {
@@ -38,8 +40,9 @@ impl std::str::FromStr for LlmBackend {
"ollama" => Ok(Self::Ollama),
"openai_compatible" | "openai-compatible" | "compatible" => Ok(Self::OpenAiCompatible),
"tinfoil" => Ok(Self::Tinfoil),
"openai_codex" | "codex" => Ok(Self::OpenAiCodex),
_ => Err(format!(
"invalid LLM backend '{}', expected one of: nearai, openai, anthropic, ollama, openai_compatible, tinfoil",
"invalid LLM backend '{}', expected one of: nearai, openai, anthropic, ollama, openai_compatible, tinfoil, openai_codex",
s
)),
}
@@ -55,6 +58,7 @@ impl std::fmt::Display for LlmBackend {
Self::Ollama => write!(f, "ollama"),
Self::OpenAiCompatible => write!(f, "openai_compatible"),
Self::Tinfoil => write!(f, "tinfoil"),
Self::OpenAiCodex => write!(f, "openai_codex"),
}
}
}
@@ -102,6 +106,29 @@ pub struct TinfoilConfig {
pub model: String,
}
/// Configuration for OpenAI Codex via Responses API.
///
/// Supports two auth modes:
/// - **API key**: Standard OpenAI billing via `api.openai.com/v1/responses`
/// - **OAuth**: ChatGPT subscription billing via `chatgpt.com/backend-api/codex/responses`,
/// using tokens from the Codex CLI (`~/.codex/auth.json`)
#[derive(Debug, Clone)]
pub struct OpenAiCodexConfig {
/// Model name (default: "gpt-5.3-codex").
pub model: String,
/// Base URL. Defaults based on auth mode:
/// - API key: `https://api.openai.com/v1`
/// - OAuth: `https://chatgpt.com/backend-api/codex`
pub base_url: String,
/// API key for api.openai.com (standard billing).
pub api_key: Option<SecretString>,
/// Path to Codex CLI auth.json for OAuth tokens.
/// Default: `~/.codex/auth.json` (or `$CODEX_HOME/auth.json`).
pub auth_path: PathBuf,
/// OpenAI account ID (required for ChatGPT endpoint).
pub account_id: Option<String>,
}
/// LLM provider configuration.
///
/// NEAR AI remains the default backend. Users can switch to other providers
@@ -122,6 +149,8 @@ pub struct LlmConfig {
pub openai_compatible: Option<OpenAiCompatibleConfig>,
/// Tinfoil config (populated when backend=tinfoil)
pub tinfoil: Option<TinfoilConfig>,
/// OpenAI Codex config (populated when backend=openai_codex)
pub openai_codex: Option<OpenAiCodexConfig>,
}
/// NEAR AI configuration.
@@ -325,6 +354,32 @@ impl LlmConfig {
None
};
let openai_codex = if backend == LlmBackend::OpenAiCodex {
let api_key = optional_env("OPENAI_CODEX_API_KEY")?.map(SecretString::from);
let model =
optional_env("OPENAI_CODEX_MODEL")?.unwrap_or_else(|| "gpt-5.3-codex".to_string());
let auth_path = optional_env("CODEX_AUTH_PATH")?
.map(PathBuf::from)
.unwrap_or_else(default_codex_auth_path);
let account_id = optional_env("OPENAI_CODEX_ACCOUNT_ID")?;
let base_url = optional_env("OPENAI_CODEX_BASE_URL")?.unwrap_or_else(|| {
if api_key.is_some() {
"https://api.openai.com/v1".to_string()
} else {
"https://chatgpt.com/backend-api/codex".to_string()
}
});
Some(OpenAiCodexConfig {
model,
base_url,
api_key,
auth_path,
account_id,
})
} else {
None
};
Ok(Self {
backend,
nearai,
@@ -333,6 +388,7 @@ impl LlmConfig {
ollama,
openai_compatible,
tinfoil,
openai_codex,
})
}
}
@@ -371,6 +427,49 @@ fn parse_extra_headers(val: &str) -> Result<Vec<(String, String)>, ConfigError>
Ok(headers)
}
/// Get the default Codex CLI auth.json path.
///
/// Respects `$CODEX_HOME` if set, otherwise defaults to `~/.codex/auth.json`.
fn default_codex_auth_path() -> PathBuf {
if let Ok(codex_home) = std::env::var("CODEX_HOME") {
return PathBuf::from(codex_home).join("auth.json");
}
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".codex")
.join("auth.json")
}
/// Extract an OAuth access token from a Codex CLI `auth.json` file.
///
/// Tries fields in order: `tokens.access_token`, `token`, `api_key`, `access_token`.
/// Returns `None` on any failure (file not found, parse error, no matching field).
pub fn extract_codex_oauth_token(auth_path: &std::path::Path) -> Option<String> {
let content = std::fs::read_to_string(auth_path).ok()?;
let json: serde_json::Value = serde_json::from_str(&content).ok()?;
// Try nested tokens.access_token first (Codex CLI format)
if let Some(token) = json
.get("tokens")
.and_then(|t| t.get("access_token"))
.and_then(|v| v.as_str())
&& !token.is_empty()
{
return Some(token.to_string());
}
// Try top-level fields
for field in &["token", "api_key", "access_token"] {
if let Some(val) = json.get(field).and_then(|v| v.as_str())
&& !val.is_empty()
{
return Some(val.to_string());
}
}
None
}
/// Get the default session file path (~/.ironclaw/session.json).
fn default_session_path() -> PathBuf {
dirs::home_dir()
@@ -508,4 +607,167 @@ mod tests {
]
);
}
/// Clear codex-related env vars for testing.
fn clear_codex_env() {
// SAFETY: Only called under ENV_MUTEX in tests.
unsafe {
std::env::remove_var("LLM_BACKEND");
std::env::remove_var("OPENAI_CODEX_API_KEY");
std::env::remove_var("OPENAI_CODEX_MODEL");
std::env::remove_var("OPENAI_CODEX_BASE_URL");
std::env::remove_var("OPENAI_CODEX_ACCOUNT_ID");
std::env::remove_var("CODEX_AUTH_PATH");
}
}
#[test]
fn codex_defaults_model_and_oauth_base_url() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env();
let settings = Settings {
llm_backend: Some("openai_codex".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present");
assert_eq!(codex.model, "gpt-5.3-codex");
// No API key → OAuth mode → ChatGPT base URL
assert!(codex.api_key.is_none());
assert_eq!(codex.base_url, "https://chatgpt.com/backend-api/codex");
assert!(codex.auth_path.to_string_lossy().contains("auth.json"));
}
#[test]
fn codex_api_key_sets_openai_base_url() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("OPENAI_CODEX_API_KEY", "sk-test-key");
}
let settings = Settings {
llm_backend: Some("openai_codex".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present");
assert!(codex.api_key.is_some());
assert_eq!(codex.base_url, "https://api.openai.com/v1");
// Cleanup
unsafe {
std::env::remove_var("OPENAI_CODEX_API_KEY");
}
}
#[test]
fn codex_env_vars_override_defaults() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("OPENAI_CODEX_MODEL", "gpt-5.1-codex");
std::env::set_var("OPENAI_CODEX_BASE_URL", "https://custom.example.com/v1");
std::env::set_var("OPENAI_CODEX_ACCOUNT_ID", "acct_123");
std::env::set_var("CODEX_AUTH_PATH", "/tmp/test-auth.json");
}
let settings = Settings {
llm_backend: Some("openai_codex".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let codex = cfg.openai_codex.expect("codex config should be present");
assert_eq!(codex.model, "gpt-5.1-codex");
assert_eq!(codex.base_url, "https://custom.example.com/v1");
assert_eq!(codex.account_id.as_deref(), Some("acct_123"));
assert_eq!(
codex.auth_path,
std::path::PathBuf::from("/tmp/test-auth.json")
);
// Cleanup
unsafe {
std::env::remove_var("OPENAI_CODEX_MODEL");
std::env::remove_var("OPENAI_CODEX_BASE_URL");
std::env::remove_var("OPENAI_CODEX_ACCOUNT_ID");
std::env::remove_var("CODEX_AUTH_PATH");
}
}
#[test]
fn codex_not_populated_for_other_backends() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_codex_env();
let settings = Settings {
llm_backend: Some("nearai".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
assert!(cfg.openai_codex.is_none());
}
#[test]
fn test_extract_codex_oauth_token_nested() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-nested.json");
std::fs::write(
&path,
r#"{"tokens":{"access_token":"oauth-tok-123","refresh_token":"rt_456"}}"#,
)
.expect("write test file");
let token = extract_codex_oauth_token(&path);
assert_eq!(token, Some("oauth-tok-123".to_string()));
let _ = std::fs::remove_file(&path);
}
#[test]
fn test_extract_codex_oauth_token_flat() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-flat.json");
std::fs::write(&path, r#"{"token":"flat-tok-789"}"#).expect("write test file");
let token = extract_codex_oauth_token(&path);
assert_eq!(token, Some("flat-tok-789".to_string()));
let _ = std::fs::remove_file(&path);
}
#[test]
fn test_extract_codex_oauth_token_missing_file() {
let path = std::path::Path::new("/tmp/ironclaw-nonexistent-auth.json");
assert!(extract_codex_oauth_token(path).is_none());
}
#[test]
fn test_extract_codex_oauth_token_empty_fields() {
let dir = std::env::temp_dir().join("ironclaw-test-codex");
let _ = std::fs::create_dir_all(&dir);
let path = dir.join("auth-empty.json");
std::fs::write(
&path,
r#"{"tokens":{"access_token":""},"token":"","api_key":""}"#,
)
.expect("write test file");
let token = extract_codex_oauth_token(&path);
assert!(token.is_none());
let _ = std::fs::remove_file(&path);
}
}
+4 -3
View File
@@ -31,14 +31,14 @@ use crate::settings::Settings;
// Re-export all public types so `crate::config::FooConfig` continues to work.
pub use self::agent::AgentConfig;
pub use self::builder::BuilderModeConfig;
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig};
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig, SignalConfig};
pub use self::database::{DatabaseBackend, DatabaseConfig, default_libsql_path};
pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig;
pub use self::llm::{
AnthropicDirectConfig, LlmBackend, LlmConfig, NearAiConfig, OllamaConfig,
OpenAiCompatibleConfig, OpenAiDirectConfig, TinfoilConfig,
AnthropicDirectConfig, LlmBackend, LlmConfig, NearAiConfig, OllamaConfig, OpenAiCodexConfig,
OpenAiCompatibleConfig, OpenAiDirectConfig, TinfoilConfig, extract_codex_oauth_token,
};
pub use self::routines::RoutineConfig;
pub use self::safety::SafetyConfig;
@@ -220,6 +220,7 @@ pub async fn inject_llm_keys_from_secrets(
("llm_anthropic_api_key", "ANTHROPIC_API_KEY"),
("llm_compatible_api_key", "LLM_API_KEY"),
("llm_nearai_api_key", "NEARAI_API_KEY"),
("llm_codex_api_key", "OPENAI_CODEX_API_KEY"),
];
let mut injected = HashMap::new();
+20 -4
View File
@@ -8,8 +8,12 @@ use crate::error::ConfigError;
pub struct SkillsConfig {
/// Whether the skills system is enabled.
pub enabled: bool,
/// Directory containing local skills (default: ~/.ironclaw/skills/).
/// Directory containing user-placed skills (default: ~/.ironclaw/skills/).
/// Skills here are loaded with `Trusted` trust level.
pub local_dir: PathBuf,
/// Directory containing registry-installed skills (default: ~/.ironclaw/installed_skills/).
/// Skills here are loaded with `Installed` trust level and get read-only tool access.
pub installed_dir: PathBuf,
/// Maximum number of skills that can be active simultaneously.
pub max_active_skills: usize,
/// Maximum total context tokens allocated to skill prompts.
@@ -19,15 +23,16 @@ pub struct SkillsConfig {
impl Default for SkillsConfig {
fn default() -> Self {
Self {
enabled: false,
enabled: true,
local_dir: default_skills_dir(),
installed_dir: default_installed_skills_dir(),
max_active_skills: 3,
max_context_tokens: 4000,
}
}
}
/// Get the default skills directory (~/.ironclaw/skills/).
/// Get the default user skills directory (~/.ironclaw/skills/).
fn default_skills_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
@@ -35,13 +40,24 @@ fn default_skills_dir() -> PathBuf {
.join("skills")
}
/// Get the default installed skills directory (~/.ironclaw/installed_skills/).
fn default_installed_skills_dir() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("installed_skills")
}
impl SkillsConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: parse_bool_env("SKILLS_ENABLED", false)?,
enabled: parse_bool_env("SKILLS_ENABLED", true)?,
local_dir: optional_env("SKILLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_skills_dir),
installed_dir: optional_env("SKILLS_INSTALLED_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_installed_skills_dir),
max_active_skills: parse_optional_env("SKILLS_MAX_ACTIVE", 3)?,
max_context_tokens: parse_optional_env("SKILLS_MAX_CONTEXT_TOKENS", 4000)?,
})
+2
View File
@@ -104,6 +104,7 @@ impl OnlineDiscovery {
source: ExtensionSource::McpUrl {
url: url.to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
})
} else {
@@ -178,6 +179,7 @@ impl OnlineDiscovery {
.unwrap_or_else(|| format!("MCP server from GitHub: {}", item.full_name)),
keywords: item.topics,
source: ExtensionSource::Discovered { url },
fallback_source: None,
auth_hint: AuthHint::Dcr,
})
})
+252 -27
View File
@@ -52,6 +52,14 @@ struct ChannelRuntimeState {
telegram_owner_id: Option<i64>,
}
/// Result of saving setup secrets and attempting activation.
pub struct SetupResult {
/// Human-readable status message.
pub message: String,
/// Whether the channel was successfully activated after saving secrets.
pub activated: bool,
}
/// Central manager for extension lifecycle operations.
pub struct ExtensionManager {
registry: ExtensionRegistry,
@@ -82,6 +90,11 @@ pub struct ExtensionManager {
store: Option<Arc<dyn crate::db::Database>>,
/// Names of WASM channels that were successfully loaded at startup.
active_channel_names: RwLock<HashSet<String>>,
/// Last activation error for each WASM channel (ephemeral, cleared on success).
activation_errors: RwLock<HashMap<String, String>>,
/// SSE broadcast sender (set post-construction via `set_sse_sender()`).
sse_sender:
RwLock<Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>>,
}
impl ExtensionManager {
@@ -121,6 +134,8 @@ impl ExtensionManager {
user_id,
store,
active_channel_names: RwLock::new(HashSet::new()),
activation_errors: RwLock::new(HashMap::new()),
sse_sender: RwLock::new(None),
}
}
@@ -153,6 +168,25 @@ impl ExtensionManager {
active.extend(names);
}
/// Set the SSE broadcast sender for pushing extension status events to the web UI.
pub async fn set_sse_sender(
&self,
sender: tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>,
) {
*self.sse_sender.write().await = Some(sender);
}
/// Broadcast an extension status change to the web UI via SSE.
async fn broadcast_extension_status(&self, name: &str, status: &str, message: Option<&str>) {
if let Some(ref sender) = *self.sse_sender.read().await {
let _ = sender.send(crate::channels::web::types::SseEvent::ExtensionStatus {
extension_name: name.to_string(),
status: status.to_string(),
message: message.map(|m| m.to_string()),
});
}
}
/// Search for extensions. If `discover` is true, also searches online.
pub async fn search(
&self,
@@ -191,9 +225,10 @@ impl ExtensionManager {
kind_hint: Option<ExtensionKind>,
) -> Result<InstallResult, ExtensionError> {
tracing::info!(extension = %name, url = ?url, kind = ?kind_hint, "Installing extension");
Self::validate_extension_name(name)?;
// If we have a registry entry, use it
if let Some(entry) = self.registry.get(name).await {
// If we have a registry entry, use it (prefer kind_hint to resolve collisions)
if let Some(entry) = self.registry.get_with_kind(name, kind_hint).await {
return self.install_from_entry(&entry).await.map_err(|e| {
tracing::error!(extension = %name, error = %e, "Extension install failed");
e
@@ -245,6 +280,7 @@ impl ExtensionManager {
/// Activate an installed (and optionally authenticated) extension.
pub async fn activate(&self, name: &str) -> Result<ActivateResult, ExtensionError> {
Self::validate_extension_name(name)?;
let kind = self.determine_installed_kind(name).await?;
match kind {
@@ -297,6 +333,7 @@ impl ExtensionManager {
tools,
needs_setup: false,
installed: true,
activation_error: None,
});
}
}
@@ -325,6 +362,7 @@ impl ExtensionManager {
tools: if active { vec![name] } else { Vec::new() },
needs_setup: false,
installed: true,
activation_error: None,
});
}
}
@@ -341,10 +379,12 @@ impl ExtensionManager {
match crate::channels::wasm::discover_channels(&self.wasm_channels_dir).await {
Ok(channels) => {
let active_names = self.active_channel_names.read().await;
let errors = self.activation_errors.read().await;
for (name, _discovered) in channels {
let active = active_names.contains(&name);
let (authenticated, needs_setup) =
self.check_channel_auth_status(&name).await;
let activation_error = errors.get(&name).cloned();
extensions.push(InstalledExtension {
name,
kind: ExtensionKind::WasmChannel,
@@ -355,6 +395,7 @@ impl ExtensionManager {
tools: Vec::new(),
needs_setup,
installed: true,
activation_error,
});
}
}
@@ -390,6 +431,7 @@ impl ExtensionManager {
tools: Vec::new(),
needs_setup: false,
installed: false,
activation_error: None,
});
}
}
@@ -399,6 +441,7 @@ impl ExtensionManager {
/// Remove an installed extension.
pub async fn remove(&self, name: &str) -> Result<String, ExtensionError> {
Self::validate_extension_name(name)?;
let kind = self.determine_installed_kind(name).await?;
match kind {
@@ -545,10 +588,41 @@ impl ExtensionManager {
async fn install_from_entry(
&self,
entry: &RegistryEntry,
) -> Result<InstallResult, ExtensionError> {
let primary_result = self.try_install_from_source(entry, &entry.source).await;
match fallback_decision(&primary_result, &entry.fallback_source) {
FallbackDecision::Return => primary_result,
FallbackDecision::TryFallback => {
let primary_err = primary_result.unwrap_err();
let fallback = entry.fallback_source.as_ref().unwrap();
tracing::info!(
extension = %entry.name,
primary_error = %primary_err,
"Primary install failed, trying fallback source"
);
self.try_install_from_source(entry, fallback)
.await
.map_err(|fallback_err| {
tracing::error!(
extension = %entry.name,
fallback_error = %fallback_err,
"Fallback install also failed"
);
combine_install_errors(&primary_err, fallback_err)
})
}
}
}
/// Attempt to install an extension using a specific source.
async fn try_install_from_source(
&self,
entry: &RegistryEntry,
source: &ExtensionSource,
) -> Result<InstallResult, ExtensionError> {
match entry.kind {
ExtensionKind::McpServer => {
let url = match &entry.source {
let url = match source {
ExtensionSource::McpUrl { url } => url.clone(),
ExtensionSource::Discovered { url } => url.clone(),
_ => {
@@ -559,7 +633,7 @@ impl ExtensionManager {
};
self.install_mcp_from_url(&entry.name, &url).await
}
ExtensionKind::WasmTool => match &entry.source {
ExtensionKind::WasmTool => match source {
ExtensionSource::WasmDownload {
wasm_url,
capabilities_url,
@@ -586,10 +660,10 @@ impl ExtensionManager {
.await
}
_ => Err(ExtensionError::InstallFailed(
"WASM tool entry has no download URL".to_string(),
"WASM tool entry has no download URL or build info".to_string(),
)),
},
ExtensionKind::WasmChannel => match &entry.source {
ExtensionKind::WasmChannel => match source {
ExtensionSource::WasmDownload {
wasm_url,
capabilities_url,
@@ -616,7 +690,7 @@ impl ExtensionManager {
.await
}
_ => Err(ExtensionError::InstallFailed(
"WASM channel entry has no download URL".to_string(),
"WASM channel entry has no download URL or build info".to_string(),
)),
},
}
@@ -1701,14 +1775,6 @@ impl ExtensionManager {
)));
}
// Validate name to prevent path traversal
if name.contains('/') || name.contains('\\') || name.contains("..") || name.contains('\0') {
return Err(ExtensionError::ActivationFailed(format!(
"Invalid channel name '{}': contains path separator or traversal characters",
name
)));
}
// Load the channel from files
let wasm_path = self.wasm_channels_dir.join(format!("{}.wasm", name));
let cap_path = self
@@ -2002,6 +2068,17 @@ impl ExtensionManager {
)))
}
/// Reject names containing path separators or traversal sequences.
fn validate_extension_name(name: &str) -> Result<(), ExtensionError> {
if name.contains('/') || name.contains('\\') || name.contains("..") || name.contains('\0') {
return Err(ExtensionError::InstallFailed(format!(
"Invalid extension name '{}': contains path separator or traversal characters",
name
)));
}
Ok(())
}
async fn cleanup_expired_auths(&self) {
let mut pending = self.pending_auth.write().await;
pending.retain(|_, auth| auth.created_at.elapsed() < std::time::Duration::from_secs(300));
@@ -2050,11 +2127,14 @@ impl ExtensionManager {
}
/// Save setup secrets for an extension, validating names against the capabilities schema.
///
/// After saving, attempts to hot-activate the channel. Returns a [`SetupResult`]
/// indicating whether activation succeeded (so the frontend can show appropriate UI).
pub async fn save_setup_secrets(
&self,
name: &str,
secrets: &std::collections::HashMap<String, String>,
) -> Result<String, ExtensionError> {
) -> Result<SetupResult, ExtensionError> {
let kind = self.determine_installed_kind(name).await?;
if kind != ExtensionKind::WasmChannel {
return Err(ExtensionError::Other(
@@ -2137,21 +2217,38 @@ impl ExtensionManager {
// Try to hot-activate the channel now that secrets are saved
match self.activate_wasm_channel(name).await {
Ok(result) => Ok(format!(
"Configuration saved and channel '{}' activated. {}",
name, result.message
)),
Ok(result) => {
self.activation_errors.write().await.remove(name);
self.broadcast_extension_status(name, "active", None).await;
Ok(SetupResult {
message: format!(
"Configuration saved and channel '{}' activated. {}",
name, result.message
),
activated: true,
})
}
Err(e) => {
let error_msg = e.to_string();
tracing::warn!(
channel = name,
error = %e,
"Saved configuration but hot-activation failed, restart may be needed"
);
Ok(format!(
"Configuration saved for '{}'. \
Automatic activation failed ({}), restart IronClaw to activate.",
name, e
))
self.activation_errors
.write()
.await
.insert(name.to_string(), error_msg.clone());
self.broadcast_extension_status(name, "failed", Some(&error_msg))
.await;
Ok(SetupResult {
message: format!(
"Configuration saved for '{}'. \
Automatic activation failed ({}), restart IronClaw to activate.",
name, e
),
activated: false,
})
}
}
}
@@ -2228,10 +2325,56 @@ fn infer_kind_from_url(url: &str) -> ExtensionKind {
}
}
/// Decision from `fallback_decision`: should we try the fallback source or
/// return the primary result as-is?
enum FallbackDecision {
/// Return the primary result directly (success or non-retriable error).
Return,
/// Primary failed with a retriable error and a fallback source is available.
TryFallback,
}
/// Decide whether to attempt a fallback install based on the primary result
/// and the availability of a fallback source.
fn fallback_decision(
primary_result: &Result<InstallResult, ExtensionError>,
fallback_source: &Option<Box<ExtensionSource>>,
) -> FallbackDecision {
match (primary_result, fallback_source) {
// Success — no fallback needed
(Ok(_), _) => FallbackDecision::Return,
// AlreadyInstalled — don't try building from source
(Err(ExtensionError::AlreadyInstalled(_)), _) => FallbackDecision::Return,
// Failed with a fallback available — try it
(Err(_), Some(_)) => FallbackDecision::TryFallback,
// Failed with no fallback — return the error
(Err(_), None) => FallbackDecision::Return,
}
}
/// Combine primary and fallback errors into a single error.
///
/// Preserves `AlreadyInstalled` from the fallback directly; otherwise wraps
/// both error messages into `ExtensionError::Other`.
fn combine_install_errors(
primary_err: &ExtensionError,
fallback_err: ExtensionError,
) -> ExtensionError {
if matches!(fallback_err, ExtensionError::AlreadyInstalled(_)) {
return fallback_err;
}
ExtensionError::Other(format!(
"Primary install failed: {}; fallback install also failed: {}",
primary_err, fallback_err
))
}
#[cfg(test)]
mod tests {
use crate::extensions::ExtensionKind;
use crate::extensions::manager::infer_kind_from_url;
use crate::extensions::manager::{
FallbackDecision, combine_install_errors, fallback_decision, infer_kind_from_url,
};
use crate::extensions::{ExtensionError, ExtensionKind, ExtensionSource, InstallResult};
#[test]
fn test_infer_kind_from_url() {
@@ -2252,4 +2395,86 @@ mod tests {
ExtensionKind::McpServer
);
}
// ---- fallback install logic tests ----
fn make_ok_result() -> Result<InstallResult, ExtensionError> {
Ok(InstallResult {
name: "test".to_string(),
kind: ExtensionKind::WasmTool,
message: "Installed".to_string(),
})
}
fn make_fallback_source() -> Option<Box<ExtensionSource>> {
Some(Box::new(ExtensionSource::WasmBuildable {
repo_url: "tools-src/test".to_string(),
build_dir: Some("tools-src/test".to_string()),
crate_name: Some("test-tool".to_string()),
}))
}
#[test]
fn test_fallback_decision_success_returns_directly() {
let result = make_ok_result();
let fallback = make_fallback_source();
assert!(matches!(
fallback_decision(&result, &fallback),
FallbackDecision::Return
));
}
#[test]
fn test_fallback_decision_already_installed_skips_fallback() {
let result: Result<InstallResult, ExtensionError> =
Err(ExtensionError::AlreadyInstalled("test".to_string()));
let fallback = make_fallback_source();
assert!(matches!(
fallback_decision(&result, &fallback),
FallbackDecision::Return
));
}
#[test]
fn test_fallback_decision_download_failed_triggers_fallback() {
let result: Result<InstallResult, ExtensionError> =
Err(ExtensionError::DownloadFailed("404 Not Found".to_string()));
let fallback = make_fallback_source();
assert!(matches!(
fallback_decision(&result, &fallback),
FallbackDecision::TryFallback
));
}
#[test]
fn test_fallback_decision_error_without_fallback_returns() {
let result: Result<InstallResult, ExtensionError> =
Err(ExtensionError::DownloadFailed("404 Not Found".to_string()));
let fallback = None;
assert!(matches!(
fallback_decision(&result, &fallback),
FallbackDecision::Return
));
}
#[test]
fn test_combine_errors_includes_both_messages() {
let primary = ExtensionError::DownloadFailed("404 Not Found".to_string());
let fallback = ExtensionError::InstallFailed("cargo not found".to_string());
let combined = combine_install_errors(&primary, fallback);
let msg = combined.to_string();
assert!(msg.contains("404 Not Found"), "missing primary: {msg}");
assert!(msg.contains("cargo not found"), "missing fallback: {msg}");
}
#[test]
fn test_combine_errors_forwards_already_installed_from_fallback() {
let primary = ExtensionError::DownloadFailed("404".to_string());
let fallback = ExtensionError::AlreadyInstalled("test".to_string());
let combined = combine_install_errors(&primary, fallback);
assert!(
matches!(combined, ExtensionError::AlreadyInstalled(ref name) if name == "test"),
"Expected AlreadyInstalled, got: {combined:?}"
);
}
}
+6
View File
@@ -64,6 +64,9 @@ pub struct RegistryEntry {
pub keywords: Vec<String>,
/// Where to get this extension.
pub source: ExtensionSource,
/// Fallback source when the primary source fails (e.g., download 404 → build from source).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub fallback_source: Option<Box<ExtensionSource>>,
/// How authentication works.
pub auth_hint: AuthHint,
}
@@ -200,6 +203,9 @@ pub struct InstalledExtension {
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
/// Last activation error for WASM channels.
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_error: Option<String>,
}
/// Error type for extension operations.
+190 -45
View File
@@ -103,11 +103,14 @@ impl ExtensionRegistry {
}
}
scored.sort_by(|a, b| b.1.cmp(&a.1));
scored.sort_by_key(|b| std::cmp::Reverse(b.1));
scored.into_iter().map(|(r, _)| r).collect()
}
/// Look up an entry by exact name.
///
/// NOTE: Prefer [`get_with_kind`] when a kind hint is available, to avoid
/// returning the wrong entry when two entries share a name but differ in kind.
pub async fn get(&self, name: &str) -> Option<RegistryEntry> {
if let Some(entry) = self.entries.iter().find(|e| e.name == name) {
return Some(entry.clone());
@@ -116,6 +119,35 @@ impl ExtensionRegistry {
cache.iter().find(|e| e.name == name).cloned()
}
/// Look up an entry by exact name, filtering by kind when provided.
///
/// When `kind` is `Some(...)`, only returns an entry matching both name and
/// kind — never falls back to a different kind. When `kind` is `None`,
/// returns the first name match (same as [`get`]).
pub async fn get_with_kind(
&self,
name: &str,
kind: Option<ExtensionKind>,
) -> Option<RegistryEntry> {
if let Some(kind) = kind {
if let Some(entry) = self
.entries
.iter()
.find(|e| e.name == name && e.kind == kind)
{
return Some(entry.clone());
}
let cache = self.discovery_cache.read().await;
if let Some(entry) = cache.iter().find(|e| e.name == name && e.kind == kind) {
return Some(entry.clone());
}
// Kind was specified but no entry matches — don't fall back to a
// different kind, as that would silently misroute the install.
return None;
}
self.get(name).await
}
/// Return all registry entries (builtins + cached discoveries).
pub async fn all_entries(&self) -> Vec<RegistryEntry> {
let mut entries = self.entries.clone();
@@ -135,8 +167,11 @@ impl ExtensionRegistry {
pub async fn cache_discovered(&self, entries: Vec<RegistryEntry>) {
let mut cache = self.discovery_cache.write().await;
for entry in entries {
// Deduplicate by name
if !cache.iter().any(|e| e.name == entry.name) {
// Deduplicate by (name, kind) — same pair as new_with_catalog()
if !cache
.iter()
.any(|e| e.name == entry.name && e.kind == entry.kind)
{
cache.push(entry);
}
}
@@ -208,6 +243,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.notion.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -225,44 +261,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"bugs".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.linear.app".to_string(),
},
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
name: "google-calendar".to_string(),
display_name: "Google Calendar".to_string(),
kind: ExtensionKind::McpServer,
description: "Connect to Google Calendar for managing events, schedules, and reminders"
.to_string(),
keywords: vec![
"calendar".into(),
"events".into(),
"schedule".into(),
"meetings".into(),
"google".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.google.com/calendar".to_string(),
},
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
name: "google-drive".to_string(),
display_name: "Google Drive".to_string(),
kind: ExtensionKind::McpServer,
description: "Connect to Google Drive for file management, search, and document access"
.to_string(),
keywords: vec![
"drive".into(),
"files".into(),
"documents".into(),
"storage".into(),
"google".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.google.com/drive".to_string(),
url: "https://mcp.linear.app/sse".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -280,8 +281,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"issues".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.github.com".to_string(),
url: "https://api.githubcopilot.com/mcp/".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -301,6 +303,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.slack.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -318,8 +321,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"performance".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.sentry.dev/sse".to_string(),
url: "https://mcp.sentry.dev/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -339,6 +343,7 @@ fn builtin_entries() -> Vec<RegistryEntry> {
source: ExtensionSource::McpUrl {
url: "https://mcp.stripe.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -356,8 +361,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"infrastructure".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.cloudflare.com/sse".to_string(),
url: "https://mcp.cloudflare.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -373,8 +379,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"team".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.asana.com".to_string(),
url: "https://mcp.asana.com/v2/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
@@ -391,8 +398,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
"helpdesk".into(),
],
source: ExtensionSource::McpUrl {
url: "https://mcp.intercom.com".to_string(),
url: "https://mcp.intercom.com/mcp".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
},
// WASM channels (telegram, slack, discord, whatsapp) come from the embedded
@@ -417,6 +425,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -439,6 +448,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -461,6 +471,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -483,6 +494,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -546,6 +558,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://custom.example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
};
@@ -571,6 +584,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://example.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::None,
};
@@ -595,6 +609,7 @@ mod tests {
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
// This shares a name with the builtin slack-mcp but has a different kind, so both should appear
@@ -609,6 +624,7 @@ mod tests {
build_dir: Some("tools-src/slack".to_string()),
crate_name: Some("slack-tool".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
@@ -644,6 +660,7 @@ mod tests {
source: ExtensionSource::McpUrl {
url: "https://other.slack.com".to_string(),
},
fallback_source: None,
auth_hint: AuthHint::Dcr,
}];
@@ -655,6 +672,134 @@ mod tests {
assert_eq!(entry.unwrap().display_name, "Slack MCP");
}
#[tokio::test]
async fn test_get_with_kind_resolves_collision() {
// Two entries with the same name but different kinds (the telegram collision scenario)
let catalog_entries = vec![
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "Telegram MTProto tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/telegram".to_string(),
build_dir: Some("tools-src/telegram".to_string()),
crate_name: Some("telegram-tool".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
fallback_source: None,
auth_hint: AuthHint::CapabilitiesAuth,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
// Without kind hint, get() returns the first match (WasmTool)
let entry = registry.get("telegram").await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmTool);
// With kind hint for WasmChannel, get_with_kind() returns the channel entry
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::WasmChannel))
.await;
assert!(entry.is_some());
let entry = entry.unwrap();
assert_eq!(entry.kind, ExtensionKind::WasmChannel);
assert_eq!(entry.display_name, "Telegram Channel");
// With kind hint for WasmTool, get_with_kind() returns the tool entry
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::WasmTool))
.await;
assert!(entry.is_some());
let entry = entry.unwrap();
assert_eq!(entry.kind, ExtensionKind::WasmTool);
assert_eq!(entry.display_name, "Telegram Tool");
// Without kind hint (None), get_with_kind() falls back to first match
let entry = registry.get_with_kind("telegram", None).await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().kind, ExtensionKind::WasmTool);
// Kind mismatch: no McpServer named "telegram" exists — must return None,
// not silently fall back to the WasmTool entry.
let entry = registry
.get_with_kind("telegram", Some(ExtensionKind::McpServer))
.await;
assert!(
entry.is_none(),
"Should return None when kind doesn't match, not fall back to wrong kind"
);
}
#[tokio::test]
async fn test_get_with_kind_discovery_cache() {
let registry = ExtensionRegistry::new();
// Add two entries with the same name but different kinds to the discovery cache
let tool_entry = RegistryEntry {
name: "cached-ext".to_string(),
display_name: "Cached Tool".to_string(),
kind: ExtensionKind::WasmTool,
description: "A cached tool".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/cached".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
};
let channel_entry = RegistryEntry {
name: "cached-ext".to_string(),
display_name: "Cached Channel".to_string(),
kind: ExtensionKind::WasmChannel,
description: "A cached channel".to_string(),
keywords: vec![],
source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/cached".to_string(),
build_dir: None,
crate_name: None,
},
fallback_source: None,
auth_hint: AuthHint::None,
};
registry
.cache_discovered(vec![tool_entry, channel_entry])
.await;
// Kind-aware lookup should find the channel in the cache
let entry = registry
.get_with_kind("cached-ext", Some(ExtensionKind::WasmChannel))
.await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().display_name, "Cached Channel");
// Kind-aware lookup should find the tool in the cache
let entry = registry
.get_with_kind("cached-ext", Some(ExtensionKind::WasmTool))
.await;
assert!(entry.is_some());
assert_eq!(entry.unwrap().display_name, "Cached Tool");
}
// Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog
// to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage.
}
+26
View File
@@ -11,6 +11,7 @@ pub mod circuit_breaker;
pub mod costs;
pub mod failover;
mod nearai_chat;
pub mod openai_codex;
mod provider;
mod reasoning;
pub mod response_cache;
@@ -43,6 +44,7 @@ use secrecy::ExposeSecret;
use crate::config::{LlmBackend, LlmConfig, NearAiConfig};
use crate::error::LlmError;
use crate::llm::openai_codex::OpenAiCodexProvider;
/// Create an LLM provider based on configuration.
///
@@ -60,6 +62,7 @@ pub fn create_llm_provider(
LlmBackend::Ollama => create_ollama_provider(config),
LlmBackend::OpenAiCompatible => create_openai_compatible_provider(config),
LlmBackend::Tinfoil => create_tinfoil_provider(config),
LlmBackend::OpenAiCodex => create_openai_codex_provider(config),
}
}
@@ -265,6 +268,28 @@ fn create_openai_compatible_provider(config: &LlmConfig) -> Result<Arc<dyn LlmPr
Ok(Arc::new(RigAdapter::new(model, &compat.model)))
}
fn create_openai_codex_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
let codex = config
.openai_codex
.as_ref()
.ok_or_else(|| LlmError::AuthFailed {
provider: "openai_codex".to_string(),
})?;
let auth_mode = if codex.api_key.is_some() {
"API key"
} else {
"OAuth (Codex CLI)"
};
tracing::info!(
model = %codex.model,
base_url = %codex.base_url,
auth = auth_mode,
"Using OpenAI Codex (Responses API)"
);
Ok(Arc::new(OpenAiCodexProvider::new(codex.clone())?))
}
/// Create a cheap/fast LLM provider for lightweight tasks (heartbeat, routing, evaluation).
///
/// Uses `NEARAI_CHEAP_MODEL` if set, otherwise falls back to the main provider.
@@ -472,6 +497,7 @@ mod tests {
ollama: None,
openai_compatible: None,
tinfoil: None,
openai_codex: None,
}
}
File diff suppressed because it is too large Load Diff
+125 -50
View File
@@ -9,7 +9,7 @@ use ironclaw::{
agent::{Agent, AgentDeps},
app::{AppBuilder, AppBuilderFlags},
channels::{
ChannelManager, GatewayChannel, HttpChannel, ReplChannel, WebhookServer,
ChannelManager, GatewayChannel, HttpChannel, ReplChannel, SignalChannel, WebhookServer,
WebhookServerConfig,
wasm::{
RegisteredEndpoint, SharedWasmChannel, WasmChannelLoader, WasmChannelRouter,
@@ -220,9 +220,35 @@ async fn main() -> anyhow::Result<()> {
// ── Orchestrator / container job manager ────────────────────────────
// Proactive Docker detection
let docker_status = if config.sandbox.enabled {
let detection = ironclaw::sandbox::check_docker().await;
match detection.status {
ironclaw::sandbox::DockerStatus::Available => {
tracing::info!("Docker is available");
}
ironclaw::sandbox::DockerStatus::NotInstalled => {
tracing::warn!(
"Docker is not installed -- sandbox disabled for this session. {}",
detection.platform.install_hint()
);
}
ironclaw::sandbox::DockerStatus::NotRunning => {
tracing::warn!(
"Docker is installed but not running -- sandbox disabled for this session. {}",
detection.platform.start_hint()
);
}
ironclaw::sandbox::DockerStatus::Disabled => {}
}
detection.status
} else {
ironclaw::sandbox::DockerStatus::Disabled
};
let job_event_tx: Option<
tokio::sync::broadcast::Sender<(uuid::Uuid, ironclaw::channels::web::types::SseEvent)>,
> = if config.sandbox.enabled {
> = if config.sandbox.enabled && docker_status.is_ok() {
let (tx, _) = tokio::sync::broadcast::channel(256);
Some(tx)
} else {
@@ -233,51 +259,52 @@ async fn main() -> anyhow::Result<()> {
std::collections::VecDeque<ironclaw::orchestrator::api::PendingPrompt>,
>::new()));
let container_job_manager: Option<Arc<ContainerJobManager>> = if config.sandbox.enabled {
let token_store = TokenStore::new();
let job_config = ContainerJobConfig {
image: config.sandbox.image.clone(),
memory_limit_mb: config.sandbox.memory_limit_mb,
cpu_shares: config.sandbox.cpu_shares,
orchestrator_port: 50051,
claude_code_api_key: std::env::var("ANTHROPIC_API_KEY").ok(),
claude_code_oauth_token: ironclaw::config::ClaudeCodeConfig::extract_oauth_token(),
claude_code_model: config.claude_code.model.clone(),
claude_code_max_turns: config.claude_code.max_turns,
claude_code_memory_limit_mb: config.claude_code.memory_limit_mb,
claude_code_allowed_tools: config.claude_code.allowed_tools.clone(),
};
let jm = Arc::new(ContainerJobManager::new(job_config, token_store.clone()));
let container_job_manager: Option<Arc<ContainerJobManager>> =
if config.sandbox.enabled && docker_status.is_ok() {
let token_store = TokenStore::new();
let job_config = ContainerJobConfig {
image: config.sandbox.image.clone(),
memory_limit_mb: config.sandbox.memory_limit_mb,
cpu_shares: config.sandbox.cpu_shares,
orchestrator_port: 50051,
claude_code_api_key: std::env::var("ANTHROPIC_API_KEY").ok(),
claude_code_oauth_token: ironclaw::config::ClaudeCodeConfig::extract_oauth_token(),
claude_code_model: config.claude_code.model.clone(),
claude_code_max_turns: config.claude_code.max_turns,
claude_code_memory_limit_mb: config.claude_code.memory_limit_mb,
claude_code_allowed_tools: config.claude_code.allowed_tools.clone(),
};
let jm = Arc::new(ContainerJobManager::new(job_config, token_store.clone()));
// Start the orchestrator internal API in the background
let orchestrator_state = OrchestratorState {
llm: components.llm.clone(),
job_manager: Arc::clone(&jm),
token_store,
job_event_tx: job_event_tx.clone(),
prompt_queue: Arc::clone(&prompt_queue),
store: components.db.clone(),
secrets_store: components.secrets_store.clone(),
user_id: "default".to_string(),
};
// Start the orchestrator internal API in the background
let orchestrator_state = OrchestratorState {
llm: components.llm.clone(),
job_manager: Arc::clone(&jm),
token_store,
job_event_tx: job_event_tx.clone(),
prompt_queue: Arc::clone(&prompt_queue),
store: components.db.clone(),
secrets_store: components.secrets_store.clone(),
user_id: "default".to_string(),
};
tokio::spawn(async move {
if let Err(e) = OrchestratorApi::start(orchestrator_state, 50051).await {
tracing::error!("Orchestrator API failed: {}", e);
tokio::spawn(async move {
if let Err(e) = OrchestratorApi::start(orchestrator_state, 50051).await {
tracing::error!("Orchestrator API failed: {}", e);
}
});
if config.claude_code.enabled {
tracing::info!(
"Claude Code sandbox mode available (model: {}, max_turns: {})",
config.claude_code.model,
config.claude_code.max_turns
);
}
});
if config.claude_code.enabled {
tracing::info!(
"Claude Code sandbox mode available (model: {}, max_turns: {})",
config.claude_code.model,
config.claude_code.max_turns
);
}
Some(jm)
} else {
None
};
Some(jm)
} else {
None
};
// ── Channel setup ──────────────────────────────────────────────────
@@ -341,6 +368,25 @@ async fn main() -> anyhow::Result<()> {
}
}
// Add Signal channel if configured and not CLI-only mode.
if !cli.cli_only
&& let Some(ref signal_config) = config.channels.signal
{
let signal_channel = SignalChannel::new(signal_config.clone())?;
channel_names.push("signal".to_string());
channels.add(Box::new(signal_channel)).await;
let safe_url = SignalChannel::redact_url(&signal_config.http_url);
tracing::info!(
url = %safe_url,
"Signal channel enabled"
);
if signal_config.allow_from.is_empty() {
tracing::warn!(
"Signal channel has empty allow_from list - ALL messages will be DENIED."
);
}
}
// Add HTTP channel if configured and not CLI-only mode.
let mut webhook_server_addr: Option<std::net::SocketAddr> = None;
if !cli.cli_only
@@ -428,6 +474,11 @@ async fn main() -> anyhow::Result<()> {
// ── Gateway channel ────────────────────────────────────────────────
let mut gateway_url: Option<String> = None;
let mut sse_sender: Option<
tokio::sync::broadcast::Sender<ironclaw::channels::web::types::SseEvent>,
> = None;
let mut gateway_state: Option<std::sync::Arc<ironclaw::channels::web::server::GatewayState>> =
None;
if let Some(ref gw_config) = config.channels.gateway {
let mut gw =
GatewayChannel::new(gw_config.clone()).with_llm_provider(Arc::clone(&components.llm));
@@ -480,6 +531,12 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("Web UI: http://{}:{}/", gw_config.host, gw_config.port);
// Capture SSE sender before moving gw into channels.
// IMPORTANT: This must come after all `with_*` calls since `rebuild_state`
// creates a new SseManager, which would orphan this sender.
sse_sender = Some(gw.state().sse.sender());
gateway_state = Some(Arc::clone(gw.state()));
channel_names.push("gateway".to_string());
channels.add(Box::new(gw)).await;
}
@@ -517,8 +574,10 @@ async fn main() -> anyhow::Result<()> {
heartbeat_enabled: config.heartbeat.enabled,
heartbeat_interval_secs: config.heartbeat.interval_secs,
sandbox_enabled: config.sandbox.enabled,
docker_status,
claude_code_enabled: config.claude_code.enabled,
routines_enabled: config.routines.enabled,
skills_enabled: config.skills.enabled,
channels: channel_names,
tunnel_url: active_tunnel
.as_ref()
@@ -549,6 +608,13 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("Channel runtime wired into extension manager for hot-activation");
}
// Wire SSE sender into extension manager for broadcasting status events.
if let Some(ref ext_mgr) = components.extension_manager
&& let Some(sender) = sse_sender
{
ext_mgr.set_sse_sender(sender).await;
}
let deps = AgentDeps {
store: components.db,
llm: components.llm,
@@ -558,6 +624,7 @@ async fn main() -> anyhow::Result<()> {
workspace: components.workspace,
extension_manager: components.extension_manager,
skill_registry: components.skill_registry,
skill_catalog: components.skill_catalog,
skills_config: config.skills.clone(),
hooks: components.hooks,
cost_guard: components.cost_guard,
@@ -590,6 +657,17 @@ async fn main() -> anyhow::Result<()> {
}
tracing::info!("Agent shutdown complete");
// Check if a restart was requested via the gateway API.
if let Some(ref gw_state) = gateway_state
&& gw_state
.restart_requested
.load(std::sync::atomic::Ordering::Relaxed)
{
eprintln!("Restarting IronClaw (exit code 75)...");
std::process::exit(75);
}
Ok(())
}
@@ -802,7 +880,6 @@ async fn setup_wasm_channels(
};
let wasm_router = Arc::new(WasmChannelRouter::new());
let mut has_webhook_channels = false;
let mut channels: Vec<(String, Box<dyn ironclaw::channels::Channel>)> = Vec::new();
let mut channel_names: Vec<String> = Vec::new();
@@ -885,8 +962,6 @@ async fn setup_wasm_channels(
secret_header,
)
.await;
has_webhook_channels = true;
if let Some(secrets) = secrets_store {
match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await {
Ok(count) => {
@@ -915,13 +990,13 @@ async fn setup_wasm_channels(
tracing::warn!("Failed to load WASM channel {}: {}", path.display(), err);
}
let webhook_routes = if has_webhook_channels {
// Always create webhook routes (even with no channels loaded) so that
// channels hot-added at runtime can receive webhooks without a restart.
let webhook_routes = {
Some(create_wasm_channel_router(
Arc::clone(&wasm_router),
extension_manager.map(Arc::clone),
))
} else {
None
};
Some(WasmChannelSetup {
+1 -1
View File
@@ -7,4 +7,4 @@
mod store;
pub use store::{PairingRequest, PairingStore, PairingStoreError};
pub use store::{PairingRequest, PairingStore, PairingStoreError, UpsertResult};
+61 -6
View File
@@ -27,9 +27,42 @@ pub enum RegistryError {
path: std::path::PathBuf,
},
#[error("Download failed for {url}: {reason}")]
// `url` is stored for programmatic access (logs, retries) but intentionally
// omitted from the Display message to avoid leaking internal artifact URLs
// to end users.
#[error("Artifact download failed: {reason}")]
DownloadFailed { url: String, reason: String },
#[error("Invalid extension manifest for '{name}' field '{field}': {reason}")]
InvalidManifest {
name: String,
field: &'static str,
reason: String,
},
#[error("Checksum verification failed: expected {expected_sha256}, got {actual_sha256}")]
ChecksumMismatch {
url: String,
expected_sha256: String,
actual_sha256: String,
},
#[error(
"Source fallback unavailable for '{name}' after artifact install failed. Retry artifact download or run from a repository checkout."
)]
SourceFallbackUnavailable {
name: String,
source_dir: PathBuf,
artifact_error: Box<RegistryError>,
},
#[error("Artifact install and source fallback both failed for '{name}'.")]
InstallFallbackFailed {
name: String,
artifact_error: Box<RegistryError>,
source_error: Box<RegistryError>,
},
#[error(
"Ambiguous name '{name}': exists as both {kind_a} and {kind_b}. Use '{prefix_a}/{name}' or '{prefix_b}/{name}'."
)]
@@ -54,7 +87,7 @@ pub enum RegistryError {
/// Central catalog loaded from the `registry/` directory.
#[derive(Debug, Clone)]
pub struct RegistryCatalog {
/// All loaded manifests, keyed by "<kind>/<name>" (e.g. "tools/slack").
/// All loaded manifests, keyed by "<kind>/<name>" (e.g. "tools/github").
manifests: HashMap<String, ExtensionManifest>,
/// Bundle definitions from `_bundles.json`.
@@ -241,11 +274,11 @@ impl RegistryCatalog {
results
}
/// Get a manifest by name. Tries exact key match first ("tools/slack"),
/// then searches by bare name ("slack").
/// Get a manifest by name. Tries exact key match first ("tools/github"),
/// then searches by bare name ("github").
///
/// If a bare name matches both a tool and a channel, returns `None`.
/// Use a qualified key ("tools/slack" or "channels/slack") to disambiguate.
/// Use a qualified key ("tools/github" or "channels/telegram") to disambiguate.
pub fn get(&self, name: &str) -> Option<&ExtensionManifest> {
// Try exact key first
if let Some(m) = self.manifests.get(name) {
@@ -289,7 +322,7 @@ impl RegistryCatalog {
}
}
/// Get the full key ("tools/slack" or "channels/telegram") for a manifest.
/// Get the full key ("tools/github" or "channels/telegram") for a manifest.
pub fn key_for(&self, name: &str) -> Option<String> {
if self.manifests.contains_key(name) {
return Some(name.to_string());
@@ -649,4 +682,26 @@ mod tests {
// At minimum, the embedded catalog from the repo should have entries
assert!(!catalog.all().is_empty() || !catalog.bundle_names().is_empty());
}
#[test]
fn test_bundle_entries_resolve_against_real_registry() {
// Load the actual registry/ directory (catches stale bundle refs after renames)
let catalog = RegistryCatalog::load_or_embedded().unwrap();
for bundle_name in catalog.bundle_names() {
let (manifests, missing) = catalog.resolve_bundle(bundle_name).unwrap();
assert!(
missing.is_empty(),
"Bundle '{}' has unresolved entries: {:?}. \
Check that _bundles.json entries match manifest name fields.",
bundle_name,
missing
);
assert!(
!manifests.is_empty(),
"Bundle '{}' resolved to zero manifests",
bundle_name
);
}
}
}
+476 -17
View File
@@ -1,12 +1,148 @@
//! Install extensions from the registry: build-from-source or download pre-built artifacts.
use std::path::{Path, PathBuf};
use std::net::IpAddr;
use std::path::{Component, Path, PathBuf};
use tokio::fs;
use crate::registry::catalog::RegistryError;
use crate::registry::manifest::{BundleDefinition, ExtensionManifest, ManifestKind};
// GitHub-only by design. New trusted hosts (e.g. a NEAR AI CDN) must be
// explicitly added here; unknown hosts fall back to source build with a
// warning rather than surfacing a clear "host not allowed" error.
const ALLOWED_ARTIFACT_HOSTS: &[&str] = &[
"github.com",
"objects.githubusercontent.com",
"github-releases.githubusercontent.com",
"raw.githubusercontent.com",
];
fn should_attempt_source_fallback(err: &RegistryError) -> bool {
!matches!(
err,
RegistryError::AlreadyInstalled { .. }
| RegistryError::ChecksumMismatch { .. }
| RegistryError::InvalidManifest { .. }
)
}
fn is_allowed_artifact_host(host: &str) -> bool {
ALLOWED_ARTIFACT_HOSTS
.iter()
.any(|allowed| host.eq_ignore_ascii_case(allowed))
|| host.ends_with(".githubusercontent.com")
}
fn validate_artifact_url(
manifest_name: &str,
field: &'static str,
url: &str,
) -> Result<(), RegistryError> {
let parsed = reqwest::Url::parse(url).map_err(|e| RegistryError::InvalidManifest {
name: manifest_name.to_string(),
field,
reason: format!("invalid URL: {}", e),
})?;
if parsed.scheme() != "https" {
return Err(RegistryError::InvalidManifest {
name: manifest_name.to_string(),
field,
reason: "URL must use https".to_string(),
});
}
let host = parsed
.host_str()
.ok_or_else(|| RegistryError::InvalidManifest {
name: manifest_name.to_string(),
field,
reason: "URL host is missing".to_string(),
})?;
if host.parse::<IpAddr>().is_ok() || !is_allowed_artifact_host(host) {
return Err(RegistryError::InvalidManifest {
name: manifest_name.to_string(),
field,
reason: format!("host '{}' is not allowed", host),
});
}
Ok(())
}
fn validate_manifest_install_inputs(manifest: &ExtensionManifest) -> Result<(), RegistryError> {
let is_valid_name = !manifest.name.is_empty()
&& manifest
.name
.chars()
.all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-' || c == '_');
if !is_valid_name {
return Err(RegistryError::InvalidManifest {
name: manifest.name.clone(),
field: "name",
reason: "name must contain only lowercase letters, digits, '-' or '_'".to_string(),
});
}
let expected_prefix = match manifest.kind {
ManifestKind::Tool => "tools-src/",
ManifestKind::Channel => "channels-src/",
};
if !manifest.source.dir.starts_with(expected_prefix) {
return Err(RegistryError::InvalidManifest {
name: manifest.name.clone(),
field: "source.dir",
reason: format!("must start with '{}'", expected_prefix),
});
}
let source_path = Path::new(&manifest.source.dir);
let has_unsafe_component = source_path.components().any(|component| {
matches!(
component,
Component::ParentDir | Component::RootDir | Component::Prefix(_) | Component::CurDir
)
});
if source_path.is_absolute() || has_unsafe_component {
return Err(RegistryError::InvalidManifest {
name: manifest.name.clone(),
field: "source.dir",
reason: "must be a safe relative path without traversal segments".to_string(),
});
}
let has_path_separator = manifest.source.capabilities.contains('/')
|| manifest.source.capabilities.contains('\\')
|| manifest.source.capabilities.contains("..");
if has_path_separator {
return Err(RegistryError::InvalidManifest {
name: manifest.name.clone(),
field: "source.capabilities",
reason: "must be a file name without path separators".to_string(),
});
}
Ok(())
}
fn download_failure_reason(error: &reqwest::Error) -> String {
if error.is_timeout() {
"request timed out".to_string()
} else if error.is_connect() {
"connection failed".to_string()
} else if error.is_request() {
"request failed".to_string()
} else {
"network error".to_string()
}
}
/// Result of installing a single extension from the registry.
#[derive(Debug)]
pub struct InstallOutcome {
@@ -57,6 +193,8 @@ impl RegistryInstaller {
manifest: &ExtensionManifest,
force: bool,
) -> Result<InstallOutcome, RegistryError> {
validate_manifest_install_inputs(manifest)?;
let source_dir = self.repo_root.join(&manifest.source.dir);
if !source_dir.exists() {
return Err(RegistryError::ManifestRead {
@@ -137,6 +275,67 @@ impl RegistryInstaller {
})
}
pub async fn install_with_source_fallback(
&self,
manifest: &ExtensionManifest,
force: bool,
) -> Result<InstallOutcome, RegistryError> {
// Validate upfront so we fail fast on bad manifests regardless of
// which install path runs, without relying on inner methods to
// catch it first.
validate_manifest_install_inputs(manifest)?;
let has_artifact = manifest
.artifacts
.get("wasm32-wasip2")
.and_then(|a| a.url.as_ref())
.is_some();
if !has_artifact {
return self.install_from_source(manifest, force).await;
}
let source_dir = self.repo_root.join(&manifest.source.dir);
match self.install_from_artifact(manifest, force).await {
Ok(outcome) => Ok(outcome),
Err(artifact_err) => {
if !should_attempt_source_fallback(&artifact_err) {
return Err(artifact_err);
}
if !source_dir.is_dir() {
return Err(RegistryError::SourceFallbackUnavailable {
name: manifest.name.clone(),
source_dir,
artifact_error: Box::new(artifact_err),
});
}
tracing::warn!(
extension = %manifest.name,
error = %artifact_err,
"Artifact install failed; falling back to build-from-source"
);
match self.install_from_source(manifest, force).await {
Ok(mut outcome) => {
outcome.warnings.push(format!(
"Artifact install failed ({}); installed via source fallback.",
artifact_err
));
Ok(outcome)
}
Err(source_err) => Err(RegistryError::InstallFallbackFailed {
name: manifest.name.clone(),
artifact_error: Box::new(artifact_err),
source_error: Box::new(source_err),
}),
}
}
}
}
/// Download and install a pre-built artifact.
///
/// Supports two formats:
@@ -147,6 +346,8 @@ impl RegistryInstaller {
manifest: &ExtensionManifest,
force: bool,
) -> Result<InstallOutcome, RegistryError> {
validate_manifest_install_inputs(manifest)?;
let artifact = manifest.artifacts.get("wasm32-wasip2").ok_or_else(|| {
RegistryError::ExtensionNotFound(format!(
"No wasm32-wasip2 artifact for '{}'",
@@ -161,6 +362,21 @@ impl RegistryInstaller {
))
})?;
validate_artifact_url(&manifest.name, "artifacts.wasm32-wasip2.url", url)?;
// Require SHA256 — refuse to install unverified binaries. Check before
// downloading to avoid wasting bandwidth on manifests that are missing
// checksums.
let expected_sha =
artifact
.sha256
.as_ref()
.ok_or_else(|| RegistryError::InvalidManifest {
name: manifest.name.clone(),
field: "artifacts.wasm32-wasip2.sha256",
reason: "sha256 is required for artifact downloads".to_string(),
})?;
let target_dir = match manifest.kind {
ManifestKind::Tool => &self.tools_dir,
ManifestKind::Channel => &self.channels_dir,
@@ -185,16 +401,7 @@ impl RegistryInstaller {
manifest.kind, manifest.display_name
);
let bytes = download_artifact(url).await?;
// Verify SHA256 if provided, warn otherwise
if let Some(expected_sha) = &artifact.sha256 {
verify_sha256(&bytes, expected_sha, url)?;
} else {
println!(
"WARNING: No SHA256 checksum for '{}'; download is not cryptographically verified.",
manifest.name
);
}
verify_sha256(&bytes, expected_sha, url)?;
let target_caps = target_dir.join(format!("{}.capabilities.json", manifest.name));
@@ -214,6 +421,11 @@ impl RegistryInstaller {
// 1. Separate capabilities_url in the artifact
// 2. Source tree (legacy, requires repo)
if let Some(ref caps_url) = artifact.capabilities_url {
validate_artifact_url(
&manifest.name,
"artifacts.wasm32-wasip2.capabilities_url",
caps_url,
)?;
const MAX_CAPS_SIZE: usize = 1024 * 1024; // 1 MB
match download_artifact(caps_url).await {
Ok(caps_bytes) if caps_bytes.len() <= MAX_CAPS_SIZE => {
@@ -360,14 +572,18 @@ async fn download_artifact(url: &str) -> Result<bytes::Bytes, RegistryError> {
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("request failed: {}", e),
reason: download_failure_reason(&e),
})?;
let response = response
.error_for_status()
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: e.to_string(),
reason: format!(
"http status {}",
e.status()
.map_or("unknown".to_string(), |status| status.as_u16().to_string())
),
})?;
response
@@ -375,7 +591,7 @@ async fn download_artifact(url: &str) -> Result<bytes::Bytes, RegistryError> {
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read body: {}", e),
reason: format!("failed to read response body: {}", e),
})
}
@@ -387,9 +603,10 @@ fn verify_sha256(bytes: &[u8], expected: &str, url: &str) -> Result<(), Registry
let actual = format!("{:x}", hasher.finalize());
if actual != expected {
return Err(RegistryError::DownloadFailed {
return Err(RegistryError::ChecksumMismatch {
url: url.to_string(),
reason: format!("SHA256 mismatch: expected {}, got {}", expected, actual),
expected_sha256: expected.to_string(),
actual_sha256: actual,
});
}
Ok(())
@@ -510,6 +727,55 @@ fn extract_tar_gz(
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use crate::registry::manifest::{ArtifactSpec, SourceSpec};
fn test_manifest(
name: &str,
source_dir: &str,
artifact_url: Option<String>,
sha256: Option<&str>,
) -> ExtensionManifest {
test_manifest_with_kind(name, source_dir, artifact_url, sha256, ManifestKind::Tool)
}
fn test_manifest_with_kind(
name: &str,
source_dir: &str,
artifact_url: Option<String>,
sha256: Option<&str>,
kind: ManifestKind,
) -> ExtensionManifest {
let mut artifacts = HashMap::new();
if artifact_url.is_some() || sha256.is_some() {
artifacts.insert(
"wasm32-wasip2".to_string(),
ArtifactSpec {
url: artifact_url,
sha256: sha256.map(ToString::to_string),
capabilities_url: None,
},
);
}
ExtensionManifest {
name: name.to_string(),
display_name: name.to_string(),
kind,
version: "0.1.0".to_string(),
description: "test manifest".to_string(),
keywords: Vec::new(),
source: SourceSpec {
dir: source_dir.to_string(),
capabilities: format!("{}.capabilities.json", name),
crate_name: name.to_string(),
},
artifacts,
auth_summary: None,
tags: Vec::new(),
}
}
#[test]
fn test_installer_creation() {
@@ -541,7 +807,140 @@ mod tests {
#[test]
fn test_verify_sha256_invalid() {
assert!(verify_sha256(b"data", "0000", "test://url").is_err());
let err = verify_sha256(b"data", "0000", "test://url").expect_err("checksum mismatch");
assert!(matches!(err, RegistryError::ChecksumMismatch { .. }));
}
#[tokio::test]
async fn test_install_from_source_rejects_path_traversal_name() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
let manifest = test_manifest("../evil", "tools-src/evil", None, None);
let result = installer.install_from_source(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, .. }) => {
assert_eq!(field, "name");
}
other => panic!("unexpected result: {:?}", other),
}
}
#[tokio::test]
async fn test_install_from_artifact_rejects_non_https_url() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
let manifest = test_manifest(
"demo",
"tools-src/demo",
Some(
"http://github.com/nearai/ironclaw/releases/latest/download/demo.wasm".to_string(),
),
None,
);
let result = installer.install_from_artifact(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, .. }) => {
assert_eq!(field, "artifacts.wasm32-wasip2.url");
}
other => panic!("unexpected result: {:?}", other),
}
}
#[tokio::test]
async fn test_install_from_artifact_rejects_disallowed_host() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
let manifest = test_manifest(
"demo",
"tools-src/demo",
Some("https://169.254.169.254/latest/meta-data".to_string()),
None,
);
let result = installer.install_from_artifact(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, .. }) => {
assert_eq!(field, "artifacts.wasm32-wasip2.url");
}
other => panic!("unexpected result: {:?}", other),
}
}
#[tokio::test]
async fn test_install_from_artifact_rejects_null_sha256() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
// Valid URL but no sha256 — should be rejected before any download attempt
let manifest = test_manifest(
"demo",
"tools-src/demo",
Some(
"https://github.com/nearai/ironclaw/releases/latest/download/demo-wasm32-wasip2.tar.gz".to_string(),
),
None, // sha256 = null
);
let result = installer.install_from_artifact(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, reason, .. }) => {
assert_eq!(field, "artifacts.wasm32-wasip2.sha256");
assert!(reason.contains("required"), "reason: {}", reason);
}
other => panic!("unexpected result: {:?}", other),
}
}
#[test]
fn test_should_attempt_source_fallback_policy() {
let download = RegistryError::DownloadFailed {
url: "https://github.com/nearai/ironclaw/releases/latest/download/demo.wasm"
.to_string(),
reason: "http status 404".to_string(),
};
assert!(should_attempt_source_fallback(&download));
let already = RegistryError::AlreadyInstalled {
name: "demo".to_string(),
path: PathBuf::from("/tmp/demo.wasm"),
};
assert!(!should_attempt_source_fallback(&already));
let checksum = RegistryError::ChecksumMismatch {
url: "https://github.com/nearai/ironclaw/releases/latest/download/demo.wasm"
.to_string(),
expected_sha256: "deadbeef".to_string(),
actual_sha256: "feedface".to_string(),
};
assert!(!should_attempt_source_fallback(&checksum));
let invalid = RegistryError::InvalidManifest {
name: "demo".to_string(),
field: "artifacts.wasm32-wasip2.url",
reason: "host not allowed".to_string(),
};
assert!(!should_attempt_source_fallback(&invalid));
}
#[test]
@@ -587,6 +986,66 @@ mod tests {
assert!(result.has_capabilities);
}
#[tokio::test]
async fn test_install_from_source_rejects_wrong_prefix_for_channel() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
// Channel manifest with tools-src/ prefix should be rejected
let manifest = test_manifest_with_kind(
"telegram",
"tools-src/telegram",
None,
None,
ManifestKind::Channel,
);
let result = installer.install_from_source(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, reason, .. }) => {
assert_eq!(field, "source.dir");
assert!(reason.contains("channels-src/"), "reason: {}", reason);
}
other => panic!("unexpected result: {:?}", other),
}
}
#[tokio::test]
async fn test_install_from_source_accepts_correct_channel_prefix() {
let temp = tempfile::tempdir().expect("tempdir");
let installer = RegistryInstaller::new(
temp.path().to_path_buf(),
temp.path().join("tools"),
temp.path().join("channels"),
);
// Channel manifest with channels-src/ prefix should pass validation
// (will fail later because source dir doesn't exist, which is fine)
let manifest = test_manifest_with_kind(
"telegram",
"channels-src/telegram",
None,
None,
ManifestKind::Channel,
);
let result = installer.install_from_source(&manifest, false).await;
match result {
Err(RegistryError::ManifestRead { reason, .. }) => {
assert!(
reason.contains("source directory does not exist"),
"reason: {}",
reason
);
}
other => panic!("unexpected result: {:?}", other),
}
}
#[test]
fn test_extract_tar_gz_missing_wasm() {
use flate2::Compression;
+139 -16
View File
@@ -154,26 +154,29 @@ impl ExtensionManifest {
/// Convert this manifest into a [`RegistryEntry`] for use with the in-chat
/// extension discovery system.
pub fn to_registry_entry(&self) -> RegistryEntry {
// Prefer pre-built artifact download when a URL is available
let source = if let Some(artifact) = self.artifacts.get("wasm32-wasip2") {
let buildable = ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()),
};
// Prefer pre-built artifact download when a URL is available,
// with build-from-source as fallback in case the download fails (e.g., 404).
let (source, fallback_source) = if let Some(artifact) = self.artifacts.get("wasm32-wasip2")
{
if let Some(ref url) = artifact.url {
ExtensionSource::WasmDownload {
wasm_url: url.clone(),
capabilities_url: artifact.capabilities_url.clone(),
}
(
ExtensionSource::WasmDownload {
wasm_url: url.clone(),
capabilities_url: artifact.capabilities_url.clone(),
},
Some(Box::new(buildable)),
)
} else {
ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()),
}
(buildable, None)
}
} else {
ExtensionSource::WasmBuildable {
repo_url: self.source.dir.clone(),
build_dir: Some(self.source.dir.clone()),
crate_name: Some(self.source.crate_name.clone()),
}
(buildable, None)
};
let auth_hint = match self.auth_summary.as_ref().and_then(|a| a.method.as_deref()) {
@@ -190,6 +193,7 @@ impl ExtensionManifest {
description: self.description.clone(),
keywords: self.keywords.clone(),
source,
fallback_source,
auth_hint,
}
}
@@ -292,4 +296,123 @@ mod tests {
assert_eq!(ManifestKind::Tool.to_string(), "tool");
assert_eq!(ManifestKind::Channel.to_string(), "channel");
}
/// When a manifest has a download URL in artifacts, to_registry_entry()
/// should set WasmDownload as primary source and WasmBuildable as fallback.
#[test]
fn test_manifest_with_download_url_has_buildable_fallback() {
let json = r#"{
"name": "gmail",
"display_name": "Gmail",
"kind": "tool",
"version": "0.1.0",
"description": "Gmail tool",
"keywords": ["email"],
"source": {
"dir": "tools-src/gmail",
"capabilities": "gmail-tool.capabilities.json",
"crate_name": "gmail-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"tags": ["default"]
}"#;
let manifest: ExtensionManifest = serde_json::from_str(json).expect("parse manifest");
let entry = manifest.to_registry_entry();
// Primary source should be WasmDownload
assert!(
matches!(&entry.source, ExtensionSource::WasmDownload { .. }),
"Primary source should be WasmDownload, got {:?}",
entry.source
);
// Fallback should be WasmBuildable with the source dir info
let fallback = entry
.fallback_source
.as_ref()
.expect("Should have fallback_source when download URL is set");
match fallback.as_ref() {
ExtensionSource::WasmBuildable {
build_dir,
crate_name,
..
} => {
assert_eq!(build_dir.as_deref(), Some("tools-src/gmail"));
assert_eq!(crate_name.as_deref(), Some("gmail-tool"));
}
other => panic!("Fallback should be WasmBuildable, got {:?}", other),
}
}
/// When a manifest has null URL in artifacts, the primary source should be
/// WasmBuildable with no fallback.
#[test]
fn test_manifest_with_null_url_no_fallback() {
let json = r#"{
"name": "slack",
"display_name": "Slack",
"kind": "tool",
"version": "0.1.0",
"description": "Slack tool",
"keywords": [],
"source": {
"dir": "tools-src/slack",
"capabilities": "slack-tool.capabilities.json",
"crate_name": "slack-tool"
},
"artifacts": {
"wasm32-wasip2": { "url": null, "sha256": null }
},
"tags": []
}"#;
let manifest: ExtensionManifest = serde_json::from_str(json).expect("parse manifest");
let entry = manifest.to_registry_entry();
assert!(
matches!(&entry.source, ExtensionSource::WasmBuildable { .. }),
"Should use WasmBuildable when URL is null"
);
assert!(
entry.fallback_source.is_none(),
"Should have no fallback when already using WasmBuildable"
);
}
/// When a manifest has no artifacts section, should use WasmBuildable with no fallback.
#[test]
fn test_manifest_no_artifacts_no_fallback() {
let json = r#"{
"name": "custom",
"display_name": "Custom",
"kind": "tool",
"version": "0.1.0",
"description": "Custom tool",
"keywords": [],
"source": {
"dir": "tools-src/custom",
"capabilities": "custom.capabilities.json",
"crate_name": "custom-tool"
},
"tags": []
}"#;
let manifest: ExtensionManifest = serde_json::from_str(json).expect("parse manifest");
let entry = manifest.to_registry_entry();
assert!(
matches!(&entry.source, ExtensionSource::WasmBuildable { .. }),
"Should use WasmBuildable when no artifacts"
);
assert!(
entry.fallback_source.is_none(),
"Should have no fallback when already using WasmBuildable"
);
}
}
+1 -1
View File
@@ -225,7 +225,7 @@ impl Sanitizer {
}
// Sort warnings by severity (critical first)
warnings.sort_by(|a, b| b.severity.cmp(&a.severity));
warnings.sort_by_key(|b| std::cmp::Reverse(b.severity));
// Determine if we need to modify content
let has_critical = warnings.iter().any(|w| w.severity == Severity::Critical);
+1 -1
View File
@@ -28,7 +28,7 @@ pub struct SandboxConfig {
impl Default for SandboxConfig {
fn default() -> Self {
Self {
enabled: false, // Disabled by default until Docker is confirmed available
enabled: true, // Startup check disables gracefully if Docker unavailable
policy: SandboxPolicy::ReadOnly,
timeout: Duration::from_secs(120),
memory_limit_mb: 2048,
+83 -15
View File
@@ -26,7 +26,7 @@
//! ```
use std::collections::HashMap;
use std::path::Path;
use std::path::{Path, PathBuf};
use std::time::Duration;
use bollard::Docker;
@@ -490,40 +490,108 @@ impl ContainerRunner {
///
/// Tries these locations in order:
/// 1. `DOCKER_HOST` env var (bollard default)
/// 2. `/var/run/docker.sock` (Linux default)
/// 3. `~/.docker/run/docker.sock` (Docker Desktop on macOS)
/// 2. `/var/run/docker.sock` (Linux default; also used by OrbStack and Podman Desktop on macOS)
/// 3. `~/.docker/run/docker.sock` (Docker Desktop 4.13+ on macOS — primary user-owned socket)
/// 4. `~/.colima/default/docker.sock` (Colima — popular lightweight Docker Desktop alternative)
/// 5. `~/.rd/docker.sock` (Rancher Desktop on macOS)
/// 6. `$XDG_RUNTIME_DIR/docker.sock` (common rootless Docker socket on Linux)
/// 7. `/run/user/$UID/docker.sock` (rootless Docker fallback on Linux)
pub async fn connect_docker() -> Result<Docker> {
// First try bollard defaults (checks DOCKER_HOST, then /var/run/docker.sock)
// First try bollard defaults (checks DOCKER_HOST env var, then /var/run/docker.sock).
// This covers Linux, OrbStack (updates the /var/run symlink), and any user with
// DOCKER_HOST set to their runtime's socket.
if let Ok(docker) = Docker::connect_with_local_defaults()
&& docker.ping().await.is_ok()
{
return Ok(docker);
}
// Try Docker Desktop socket (macOS)
if let Some(home) = std::env::var_os("HOME") {
let desktop_sock = std::path::Path::new(&home).join(".docker/run/docker.sock");
if desktop_sock.exists() {
let sock_str = desktop_sock.to_string_lossy();
if let Ok(docker) =
Docker::connect_with_socket(&sock_str, 120, bollard::API_DEFAULT_VERSION)
&& docker.ping().await.is_ok()
{
return Ok(docker);
#[cfg(unix)]
{
// Try well-known user-owned socket locations for desktop and rootless runtimes.
// Docker Desktop 4.13+ (stabilised in 4.18) stopped creating the
// /var/run/docker.sock symlink by default and moved the API socket
// to ~/.docker/run/docker.sock.
for sock in unix_socket_candidates() {
if sock.exists() {
let sock_str = sock.to_string_lossy();
if let Ok(docker) =
Docker::connect_with_socket(&sock_str, 120, bollard::API_DEFAULT_VERSION)
&& docker.ping().await.is_ok()
{
return Ok(docker);
}
}
}
}
Err(SandboxError::DockerNotAvailable {
reason: "Could not connect to Docker. Tried: default socket, ~/.docker/run/docker.sock"
reason: "Could not connect to Docker daemon. Tried: $DOCKER_HOST, \
/var/run/docker.sock, ~/.docker/run/docker.sock, \
~/.colima/default/docker.sock, ~/.rd/docker.sock, \
$XDG_RUNTIME_DIR/docker.sock, /run/user/$UID/docker.sock"
.to_string(),
})
}
#[cfg(unix)]
fn unix_socket_candidates() -> Vec<PathBuf> {
unix_socket_candidates_from_env(
std::env::var_os("HOME").map(PathBuf::from),
std::env::var_os("XDG_RUNTIME_DIR").map(PathBuf::from),
std::env::var("UID").ok(),
)
}
#[cfg(unix)]
fn unix_socket_candidates_from_env(
home: Option<PathBuf>,
xdg_runtime_dir: Option<PathBuf>,
uid: Option<String>,
) -> Vec<PathBuf> {
let mut candidates = Vec::new();
let mut push_unique = |path: PathBuf| {
if !candidates.iter().any(|existing| existing == &path) {
candidates.push(path);
}
};
if let Some(home) = home {
push_unique(home.join(".docker/run/docker.sock")); // Docker Desktop 4.13+
push_unique(home.join(".colima/default/docker.sock")); // Colima
push_unique(home.join(".rd/docker.sock")); // Rancher Desktop
}
if let Some(xdg_runtime_dir) = xdg_runtime_dir {
push_unique(xdg_runtime_dir.join("docker.sock"));
}
if let Some(uid) = uid.filter(|value| !value.is_empty()) {
push_unique(PathBuf::from(format!("/run/user/{uid}/docker.sock")));
}
candidates
}
#[cfg(test)]
mod tests {
use super::*;
#[cfg(unix)]
#[test]
fn test_unix_socket_candidates_include_rootless_paths() {
let candidates = unix_socket_candidates_from_env(
Some(PathBuf::from("/home/tester")),
Some(PathBuf::from("/run/user/1000")),
Some("1000".to_string()),
);
assert!(candidates.contains(&PathBuf::from("/home/tester/.docker/run/docker.sock")));
assert!(candidates.contains(&PathBuf::from("/home/tester/.colima/default/docker.sock")));
assert!(candidates.contains(&PathBuf::from("/home/tester/.rd/docker.sock")));
assert!(candidates.contains(&PathBuf::from("/run/user/1000/docker.sock")));
}
#[tokio::test]
async fn test_docker_connection() {
// This test requires Docker to be running
+233
View File
@@ -0,0 +1,233 @@
//! Proactive Docker detection with platform-specific guidance.
//!
//! Checks whether Docker is both installed (binary on PATH) and running
//! (daemon responding to ping), and provides platform-appropriate
//! installation or startup instructions when it is not.
//!
//! # Detection Limitations
//!
//! - **macOS**: High confidence. Detects both standard Docker Desktop socket
//! (`~/.docker/run/docker.sock`) and the default `/var/run/docker.sock`.
//!
//! - **Linux**: High confidence for standard installs. Rootless Docker uses
//! a different socket path (`/run/user/$UID/docker.sock`) which is now
//! checked by the fallback in `connect_docker()`. If `DOCKER_HOST` is set,
//! bollard's default connection still takes precedence.
//!
//! - **Windows**: Medium confidence. Binary detection uses `where.exe` which
//! works reliably. Daemon detection relies on bollard's default named pipe
//! connection (`//./pipe/docker_engine`) which works with Docker Desktop.
//! The Unix socket fallback in `connect_docker()` is a no-op on Windows,
//! so detection also probes `docker version`/`docker info` via CLI if the
//! named pipe is unavailable.
/// Docker daemon availability status.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DockerStatus {
/// Docker binary found on PATH and daemon responding to ping.
Available,
/// `docker` binary not found on PATH.
NotInstalled,
/// Binary found but daemon not responding.
NotRunning,
/// Sandbox feature not enabled (no check performed).
Disabled,
}
impl DockerStatus {
/// Returns true if Docker is available and ready.
pub fn is_ok(&self) -> bool {
matches!(self, DockerStatus::Available)
}
/// Human-readable status string.
pub fn as_str(&self) -> &'static str {
match self {
DockerStatus::Available => "available",
DockerStatus::NotInstalled => "not installed",
DockerStatus::NotRunning => "not running",
DockerStatus::Disabled => "disabled",
}
}
}
/// Host platform for install guidance.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Platform {
MacOS,
Linux,
Windows,
}
impl Platform {
/// Detect the current platform.
pub fn current() -> Self {
match std::env::consts::OS {
"macos" => Platform::MacOS,
"windows" => Platform::Windows,
_ => Platform::Linux,
}
}
/// Installation instructions for Docker on this platform.
pub fn install_hint(&self) -> &'static str {
match self {
Platform::MacOS => {
"Install Docker Desktop: https://docs.docker.com/desktop/install/mac-install/"
}
Platform::Linux => "Install Docker Engine: https://docs.docker.com/engine/install/",
Platform::Windows => {
"Install Docker Desktop: https://docs.docker.com/desktop/install/windows-install/"
}
}
}
/// Instructions to start the Docker daemon on this platform.
pub fn start_hint(&self) -> &'static str {
match self {
Platform::MacOS => "Start Docker Desktop from Applications, or run: open -a Docker",
Platform::Linux => "Start the Docker daemon: sudo systemctl start docker",
Platform::Windows => "Start Docker Desktop from the Start menu",
}
}
}
/// Result of a Docker detection check.
pub struct DockerDetection {
pub status: DockerStatus,
pub platform: Platform,
}
/// Check whether Docker is installed and running.
///
/// 1. Checks if `docker` binary exists on PATH
/// 2. If found, tries to connect and ping the Docker daemon via `connect_docker()`
/// 3. Returns `Available`, `NotInstalled`, or `NotRunning`
pub async fn check_docker() -> DockerDetection {
let platform = Platform::current();
// Step 1: Check if docker binary is on PATH
if !docker_binary_exists() {
return DockerDetection {
status: DockerStatus::NotInstalled,
platform,
};
}
// Step 2: Try to connect to the daemon
if crate::sandbox::connect_docker().await.is_ok() {
return DockerDetection {
status: DockerStatus::Available,
platform,
};
}
// Windows fallback: if the named pipe probe fails but docker CLI can still
// reach the daemon/server, treat Docker as available.
#[cfg(windows)]
if docker_cli_daemon_reachable() {
return DockerDetection {
status: DockerStatus::Available,
platform,
};
}
DockerDetection {
status: DockerStatus::NotRunning,
platform,
}
}
/// Check if the `docker` binary exists on PATH.
fn docker_binary_exists() -> bool {
#[cfg(unix)]
{
std::process::Command::new("which")
.arg("docker")
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.is_ok_and(|s| s.success())
}
#[cfg(windows)]
{
std::process::Command::new("where")
.arg("docker")
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.is_ok_and(|s| s.success())
}
}
#[cfg(windows)]
fn docker_cli_daemon_reachable() -> bool {
let stdout = std::process::Stdio::null();
let stderr = std::process::Stdio::null();
// `docker version` requires daemon reachability for server fields.
let version_ok = std::process::Command::new("docker")
.args(["version", "--format", "{{.Server.Version}}"])
.stdout(stdout)
.stderr(stderr)
.status()
.is_ok_and(|s| s.success());
if version_ok {
return true;
}
// Fallback for environments where `docker version --format` behaves differently.
std::process::Command::new("docker")
.args(["info", "--format", "{{.ServerVersion}}"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.is_ok_and(|s| s.success())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_detect_platform() {
let platform = Platform::current();
match platform {
Platform::MacOS | Platform::Linux | Platform::Windows => {}
}
}
#[test]
fn test_install_hint_not_empty() {
for platform in [Platform::MacOS, Platform::Linux, Platform::Windows] {
assert!(!platform.install_hint().is_empty());
assert!(!platform.start_hint().is_empty());
}
}
#[test]
fn test_docker_status_display() {
assert_eq!(DockerStatus::Available.as_str(), "available");
assert_eq!(DockerStatus::NotInstalled.as_str(), "not installed");
assert_eq!(DockerStatus::NotRunning.as_str(), "not running");
assert_eq!(DockerStatus::Disabled.as_str(), "disabled");
}
#[test]
fn test_docker_status_is_ok() {
assert!(DockerStatus::Available.is_ok());
assert!(!DockerStatus::NotInstalled.is_ok());
assert!(!DockerStatus::NotRunning.is_ok());
assert!(!DockerStatus::Disabled.is_ok());
}
#[tokio::test]
async fn test_check_docker_returns_valid_status() {
let result = check_docker().await;
match result.status {
DockerStatus::Available | DockerStatus::NotInstalled | DockerStatus::NotRunning => {}
DockerStatus::Disabled => panic!("check_docker should never return Disabled"),
}
}
}
+1 -1
View File
@@ -460,7 +460,7 @@ mod tests {
#[test]
fn test_builder_defaults() {
let manager = SandboxManagerBuilder::new().build();
assert!(!manager.config.enabled); // Disabled by default
assert!(manager.config.enabled); // Enabled by default (startup check disables if Docker unavailable)
}
#[test]
+2
View File
@@ -87,12 +87,14 @@
pub mod config;
pub mod container;
pub mod detect;
pub mod error;
pub mod manager;
pub mod proxy;
pub use config::{ResourceLimits, SandboxConfig, SandboxPolicy};
pub use container::{ContainerOutput, ContainerRunner, connect_docker};
pub use detect::{DockerDetection, DockerStatus, Platform, check_docker};
pub use error::{Result, SandboxError};
pub use manager::{ExecOutput, SandboxManager, SandboxManagerBuilder};
pub use proxy::{
+35
View File
@@ -212,6 +212,41 @@ pub struct ChannelSettings {
#[serde(default)]
pub http_host: Option<String>,
/// Whether Signal channel is enabled.
#[serde(default)]
pub signal_enabled: bool,
/// Signal HTTP URL (signal-cli daemon endpoint).
#[serde(default)]
pub signal_http_url: Option<String>,
/// Signal account (E.164 phone number).
#[serde(default)]
pub signal_account: Option<String>,
/// Signal allow from list for DMs (comma-separated E.164 phone numbers).
/// Comma-separated identifiers: E.164 phone numbers, `*`, bare UUIDs, or `uuid:<id>` entries.
/// Defaults to the configured account.
#[serde(default)]
pub signal_allow_from: Option<String>,
/// Signal allow from groups (comma-separated group IDs).
#[serde(default)]
pub signal_allow_from_groups: Option<String>,
/// Signal DM policy: "open", "allowlist", or "pairing". Default: "pairing".
#[serde(default)]
pub signal_dm_policy: Option<String>,
/// Signal group policy: "allowlist", "open", or "disabled". Default: "allowlist".
#[serde(default)]
pub signal_group_policy: Option<String>,
/// Signal group allow from (comma-separated group member IDs).
/// If empty, inherits from signal_allow_from.
#[serde(default)]
pub signal_group_allow_from: Option<String>,
/// Telegram owner user ID. When set, the bot only responds to this user.
/// Captured during setup by having the user message the bot.
#[serde(default)]
+17 -5
View File
@@ -172,7 +172,18 @@ env-var mode or skipped secrets.
| Anthropic | API key | `anthropic_api_key` | `ANTHROPIC_API_KEY` |
| OpenAI | API key | `openai_api_key` | `OPENAI_API_KEY` |
| Ollama | None | - | - |
| OpenAI-compatible | Optional API key | `llm_compatible_api_key` | `LLM_API_KEY` |
| OpenRouter¹ | API key | `llm_compatible_api_key` | `LLM_API_KEY` |
| OpenAI-compatible¹ | Optional API key | `llm_compatible_api_key` | `LLM_API_KEY` |
¹ OpenRouter and OpenAI-compatible share the same secret name and env var because
OpenRouter is stored as `llm_backend = "openai_compatible"` under the hood.
Switching between them overwrites the same credential slot.
**OpenRouter** (`setup_openrouter`):
- Pre-configured OpenAI-compatible preset with base URL `https://openrouter.ai/api/v1`
- Delegates to `setup_api_key_provider()` with a display name override ("OpenRouter")
- Sets `llm_backend = "openai_compatible"` and `openai_compatible_base_url` automatically
- Clears `selected_model` so Step 4 prompts for a model name (manual text input, no API-based model fetching)
**API-key providers** (`setup_api_key_provider`):
1. Check env var → if set, ask to reuse, persist to secrets store
@@ -258,7 +269,7 @@ key first, then falls back to the standard env var.
6c. Build channel options: discovered + bundled + registry catalog
6d. Multi-select: CLI/TUI, HTTP, all available channels
6e. Install missing bundled channels (copy WASM binaries)
6f. Install missing registry channels (build from source)
6f. Install missing registry channels (download artifacts, fallback to source build)
6g. Initialize SecretsContext (for token storage)
6h. Setup HTTP webhook (if selected)
6i. Setup each WASM channel (secrets, owner binding)
@@ -267,7 +278,7 @@ key first, then falls back to the standard env var.
**Channel sources** (priority order for installation):
1. Already installed in `~/.ironclaw/channels/`
2. Bundled channels (pre-compiled in `channels-src/`)
3. Registry channels (`registry/channels/*.json`, built from source)
3. Registry channels (`registry/channels/*.json`, download-first with source fallback)
**Tunnel setup** (`setup_tunnel`):
- Options: ngrok, Cloudflare Tunnel, localtunnel, custom URL
@@ -305,8 +316,9 @@ key first, then falls back to the standard env var.
4. Discover already-installed tools in `~/.ironclaw/tools/`
5. Multi-select: show all registry tools with display name, auth method,
and description. Pre-check tools tagged `"default"` and already installed.
6. For each selected tool not yet installed, build from source via
`RegistryInstaller::install_from_source()`
6. For each selected tool not yet installed, install via
`RegistryInstaller::install_with_source_fallback()` (download-first,
fallback to source build)
7. Print consolidated auth hints (deduplicated by provider, e.g. one hint
for all Google tools sharing `google_oauth_token`)
+197
View File
@@ -11,6 +11,8 @@ use std::sync::Arc;
use reqwest::Client;
use secrecy::{ExposeSecret, SecretString};
use serde::Deserialize;
use url::Url;
use uuid::Uuid;
#[cfg(feature = "postgres")]
use crate::secrets::SecretsCrypto;
@@ -639,6 +641,19 @@ pub struct HttpSetupResult {
pub host: String,
}
/// Result of Signal channel setup.
#[derive(Debug, Clone)]
pub struct SignalSetupResult {
pub enabled: bool,
pub http_url: String,
pub account: String,
pub allow_from: String,
pub allow_from_groups: String,
pub dm_policy: String,
pub group_policy: String,
pub group_allow_from: String,
}
/// Set up HTTP webhook channel.
pub async fn setup_http(secrets: &SecretsContext) -> Result<HttpSetupResult, ChannelSetupError> {
println!("HTTP Webhook Setup:");
@@ -684,6 +699,188 @@ pub fn generate_webhook_secret() -> String {
generate_secret_with_length(32)
}
fn validate_e164(account: &str) -> Result<(), String> {
if !account.starts_with('+') {
return Err("E.164 account must start with '+'".to_string());
}
let digits = &account[1..];
if digits.is_empty() {
return Err("E.164 account must have digits after '+'".to_string());
}
if !digits.chars().all(|c| c.is_ascii_digit()) {
return Err("E.164 account must contain only digits after '+'".to_string());
}
if digits.len() < 7 || digits.len() > 15 {
return Err("E.164 account must be 7-15 digits after '+'".to_string());
}
Ok(())
}
fn validate_allow_from_list(list: &str) -> Result<(), String> {
if list.is_empty() {
return Ok(());
}
for (i, item) in list.split(',').enumerate() {
let trimmed = item.trim();
if trimmed.is_empty() {
continue;
}
if trimmed == "*" {
continue;
}
if let Some(uuid_part) = trimmed.strip_prefix("uuid:") {
if Uuid::parse_str(uuid_part).is_err() {
return Err(format!(
"allow_from[{}]: '{}' is not a valid UUID (after 'uuid:' prefix)",
i, trimmed
));
}
continue;
}
if validate_e164(trimmed).is_ok() {
continue;
}
if Uuid::parse_str(trimmed).is_ok() {
continue;
}
return Err(format!(
"allow_from[{}]: '{}' must be '*', E.164 phone number, UUID, or 'uuid:<id>'",
i, trimmed
));
}
Ok(())
}
fn validate_allow_from_groups_list(list: &str) -> Result<(), String> {
if list.is_empty() {
return Ok(());
}
for (i, item) in list.split(',').enumerate() {
let trimmed = item.trim();
if trimmed.is_empty() {
continue;
}
if trimmed == "*" {
continue;
}
if trimmed.is_empty() {
return Err(format!(
"allow_from_groups[{}]: group ID cannot be empty",
i
));
}
}
Ok(())
}
/// Set up Signal channel.
/// `Settings` is reserved for future use
pub async fn setup_signal(_settings: &Settings) -> Result<SignalSetupResult, ChannelSetupError> {
println!("Signal Channel Setup:");
println!();
print_info("Signal channel connects to a signal-cli daemon running in HTTP mode.");
println!();
let http_url = input("Signal-cli HTTP URL")?;
match Url::parse(&http_url) {
Ok(url) if url.scheme() == "http" || url.scheme() == "https" => {}
Ok(_) => {
print_error("URL must use http or https scheme");
return Err(ChannelSetupError::Validation(
"Invalid HTTP URL: must use http or https scheme".to_string(),
));
}
Err(e) => {
print_error(&format!("Invalid URL: {}", e));
return Err(ChannelSetupError::Validation(format!(
"Invalid HTTP URL: {}",
e
)));
}
}
let account = input("Signal account (E.164)")?;
if let Err(e) = validate_e164(&account) {
print_error(&e);
return Err(ChannelSetupError::Validation(e));
}
let allow_from = optional_input(
"Allow from (comma-separated: E.164 numbers, '*' for anyone, UUIDs or 'uuid:<id>'; empty for self-only)",
Some(&format!("default: {} (self-only)", account)),
)?
.unwrap_or_else(|| account.clone());
let dm_policy = optional_input(
"DM policy (open, allowlist, pairing)",
Some("default: pairing"),
)?
.unwrap_or_else(|| "pairing".to_string());
let allow_from_groups = optional_input(
"Allow from groups (comma-separated group IDs, '*' for any group; empty for none)",
Some("default: (none)"),
)?
.unwrap_or_default();
let group_policy = optional_input(
"Group policy (allowlist, open, disabled)",
Some("default: allowlist"),
)?
.unwrap_or_else(|| "allowlist".to_string());
let group_allow_from = optional_input(
"Group allow from (comma-separated member IDs; empty to inherit from allow_from)",
Some("default: (inherit from allow_from)"),
)?
.unwrap_or_default();
if let Err(e) = validate_allow_from_list(&allow_from) {
print_error(&e);
return Err(ChannelSetupError::Validation(e));
}
if let Err(e) = validate_allow_from_groups_list(&allow_from_groups) {
print_error(&e);
return Err(ChannelSetupError::Validation(e));
}
println!();
print_success(&format!(
"Signal channel configured for account: {}",
account
));
print_info(&format!("HTTP URL: {}", http_url));
if allow_from == account {
print_info("Allow from: self-only");
} else {
print_info(&format!("Allow from: {}", allow_from));
}
print_info(&format!("DM policy: {}", dm_policy));
if allow_from_groups.is_empty() {
print_info("Allow from groups: (none)");
} else {
print_info(&format!("Allow from groups: {}", allow_from_groups));
}
print_info(&format!("Group policy: {}", group_policy));
if group_allow_from.is_empty() {
print_info("Group allow from: (inherits from allow_from)");
} else {
print_info(&format!("Group allow from: {}", group_allow_from));
}
Ok(SignalSetupResult {
enabled: true,
http_url,
account,
allow_from,
allow_from_groups,
dm_policy,
group_policy,
group_allow_from,
})
}
/// Result of WASM channel setup.
#[derive(Debug, Clone)]
pub struct WasmChannelSetupResult {
+322 -29
View File
@@ -8,7 +8,8 @@
//! 5. Embeddings
//! 6. Channel configuration
//! 7. Extensions (tool installation from registry)
//! 8. Heartbeat (background tasks)
//! 8. Docker sandbox
//! 9. Heartbeat (background tasks)
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
@@ -26,13 +27,18 @@ use crate::llm::{SessionConfig, SessionManager};
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::settings::{KeySource, Settings};
use crate::setup::channels::{
SecretsContext, setup_http, setup_telegram, setup_tunnel, setup_wasm_channel,
SecretsContext, setup_http, setup_signal, setup_telegram, setup_tunnel, setup_wasm_channel,
};
use crate::setup::prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
print_success, secret_input, select_many, select_one,
};
// unused const, keep commented for clarity / future use
// const CHANNEL_INDEX_CLI: usize = 0;
const CHANNEL_INDEX_HTTP: usize = 1;
const CHANNEL_INDEX_SIGNAL: usize = 2;
/// Setup wizard error.
#[derive(Debug, thiserror::Error)]
pub enum SetupError {
@@ -140,7 +146,7 @@ impl SetupWizard {
print_step(1, 1, "Channel Configuration");
self.step_channels().await?;
} else {
let total_steps = 8;
let total_steps = 9;
// Step 1: Database
print_step(1, total_steps, "Database Connection");
@@ -191,8 +197,13 @@ impl SetupWizard {
print_step(7, total_steps, "Extensions");
self.step_extensions().await?;
// Step 8: Heartbeat
print_step(8, total_steps, "Background Tasks");
// Step 8: Docker Sandbox
print_step(8, total_steps, "Docker Sandbox");
self.step_docker_sandbox().await?;
self.persist_after_step().await;
// Step 9: Heartbeat
print_step(9, total_steps, "Background Tasks");
self.step_heartbeat()?;
self.persist_after_step().await;
}
@@ -724,30 +735,46 @@ impl SetupWizard {
async fn step_inference_provider(&mut self) -> Result<(), SetupError> {
// Show current provider if already configured
if let Some(ref current) = self.settings.llm_backend {
let display = match current.as_str() {
"nearai" => "NEAR AI",
"anthropic" => "Anthropic (Claude)",
"openai" => "OpenAI",
"ollama" => "Ollama (local)",
"openai_compatible" => "OpenAI-compatible endpoint",
other => other,
let is_openrouter = current == "openai_compatible"
&& self
.settings
.openai_compatible_base_url
.as_deref()
.is_some_and(|u| u.contains("openrouter.ai"));
let display = if is_openrouter {
"OpenRouter"
} else {
match current.as_str() {
"nearai" => "NEAR AI",
"anthropic" => "Anthropic (Claude)",
"openai" => "OpenAI",
"ollama" => "Ollama (local)",
"openai_compatible" => "OpenAI-compatible endpoint",
"openai_codex" => "OpenAI Codex (Responses API)",
other => other,
}
};
print_info(&format!("Current provider: {}", display));
println!();
let is_known = matches!(
current.as_str(),
"nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible"
"nearai" | "anthropic" | "openai" | "ollama" | "openai_compatible" | "openai_codex"
);
if is_known && confirm("Keep current provider?", true).map_err(SetupError::Io)? {
// Still run the auth sub-flow in case they need to update keys
if is_openrouter {
return self.setup_openrouter().await;
}
match current.as_str() {
"nearai" => return self.setup_nearai().await,
"anthropic" => return self.setup_anthropic().await,
"openai" => return self.setup_openai().await,
"ollama" => return self.setup_ollama(),
"openai_compatible" => return self.setup_openai_compatible().await,
"openai_codex" => return self.setup_openai_codex().await,
_ => {
return Err(SetupError::Config(format!(
"Unhandled provider: {}",
@@ -773,7 +800,9 @@ impl SetupWizard {
"Anthropic - Claude models (direct API key)",
"OpenAI - GPT models (direct API key)",
"Ollama - local models, no API key needed",
"OpenAI-compatible - custom endpoint (vLLM, LiteLLM, Together, etc.)",
"OpenRouter - 200+ models via single API key",
"OpenAI-compatible - custom endpoint (vLLM, LiteLLM, etc.)",
"OpenAI Codex - Responses API (ChatGPT OAuth or API key)",
];
let choice = select_one("Provider:", options).map_err(SetupError::Io)?;
@@ -783,7 +812,9 @@ impl SetupWizard {
1 => self.setup_anthropic().await?,
2 => self.setup_openai().await?,
3 => self.setup_ollama()?,
4 => self.setup_openai_compatible().await?,
4 => self.setup_openrouter().await?,
5 => self.setup_openai_compatible().await?,
6 => self.setup_openai_codex().await?,
_ => return Err(SetupError::Config("Invalid provider selection".to_string())),
}
@@ -857,6 +888,7 @@ impl SetupWizard {
"llm_anthropic_api_key",
"Anthropic API key",
"https://console.anthropic.com/settings/keys",
None,
)
.await
}
@@ -869,11 +901,12 @@ impl SetupWizard {
"llm_openai_api_key",
"OpenAI API key",
"https://platform.openai.com/api-keys",
None,
)
.await
}
/// Shared setup flow for API-key-based providers (Anthropic, OpenAI).
/// Shared setup flow for API-key-based providers (Anthropic, OpenAI, OpenRouter).
async fn setup_api_key_provider(
&mut self,
backend: &str,
@@ -881,12 +914,13 @@ impl SetupWizard {
secret_name: &str,
prompt_label: &str,
hint_url: &str,
override_display_name: Option<&str>,
) -> Result<(), SetupError> {
let display_name = match backend {
let display_name = override_display_name.unwrap_or(match backend {
"anthropic" => "Anthropic",
"openai" => "OpenAI",
other => other,
};
});
self.settings.llm_backend = Some(backend.to_string());
if self.settings.selected_model.is_some() {
@@ -966,6 +1000,24 @@ impl SetupWizard {
Ok(())
}
/// OpenRouter provider setup: pre-configured OpenAI-compatible endpoint.
///
/// Sets the base URL to `https://openrouter.ai/api/v1` and delegates
/// API key collection to `setup_api_key_provider` with a display name
/// override so messages say "OpenRouter" instead of "openai_compatible".
async fn setup_openrouter(&mut self) -> Result<(), SetupError> {
self.settings.openai_compatible_base_url = Some("https://openrouter.ai/api/v1".to_string());
self.setup_api_key_provider(
"openai_compatible",
"LLM_API_KEY",
"llm_compatible_api_key",
"OpenRouter API key",
"https://openrouter.ai/settings/keys",
Some("OpenRouter"),
)
.await
}
/// OpenAI-compatible provider setup: base URL + optional API key.
async fn setup_openai_compatible(&mut self) -> Result<(), SetupError> {
self.settings.llm_backend = Some("openai_compatible".to_string());
@@ -1018,6 +1070,100 @@ impl SetupWizard {
Ok(())
}
/// OpenAI Codex provider setup: API key or Codex CLI OAuth.
async fn setup_openai_codex(&mut self) -> Result<(), SetupError> {
self.settings.llm_backend = Some("openai_codex".to_string());
if self.settings.selected_model.is_some() {
self.settings.selected_model = None;
}
let auth_options = &[
"API key - standard OpenAI billing (api.openai.com)",
"Codex CLI OAuth - ChatGPT subscription billing (~/.codex/auth.json)",
];
let auth_choice =
select_one("Authentication mode:", auth_options).map_err(SetupError::Io)?;
match auth_choice {
0 => {
// API key mode — delegate to shared helper
self.setup_api_key_provider(
"openai_codex",
"OPENAI_CODEX_API_KEY",
"llm_codex_api_key",
"OpenAI API key (for Codex)",
"https://platform.openai.com/api-keys",
Some("OpenAI Codex"),
)
.await?;
}
1 => {
// OAuth mode — read from Codex CLI auth.json
let auth_path = std::env::var("CODEX_AUTH_PATH").unwrap_or_else(|_| {
dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".codex")
.join("auth.json")
.to_string_lossy()
.to_string()
});
let path = std::path::Path::new(&auth_path);
match crate::config::extract_codex_oauth_token(path) {
Some(token) => {
print_info(&format!(
"Found Codex OAuth token: {}",
mask_api_key(&token)
));
if !confirm("Use this token?", true).map_err(SetupError::Io)? {
return Err(SetupError::Cancelled);
}
print_success("OpenAI Codex configured (OAuth from Codex CLI)");
}
None => {
print_error(&format!("No Codex OAuth token found at {}", path.display()));
print_info(
"Run `npx codex --full-setup` to authenticate, then retry setup.",
);
if confirm("Retry after authenticating?", true).map_err(SetupError::Io)? {
// Check again after user's action
match crate::config::extract_codex_oauth_token(path) {
Some(_) => {
print_success("OpenAI Codex configured (OAuth from Codex CLI)");
}
None => {
return Err(SetupError::Auth(format!(
"Still no token found at {}. Run Codex CLI setup first.",
path.display()
)));
}
}
} else {
return Err(SetupError::Cancelled);
}
}
}
// Prompt for account ID (required for ChatGPT endpoint)
let account_id = optional_input(
"OpenAI account ID (for ChatGPT endpoint, optional)",
Some("leave blank if unknown"),
)
.map_err(SetupError::Io)?;
if let Some(ref id) = account_id
&& !id.is_empty()
{
print_info(&format!("Account ID: {}", id));
}
}
_ => return Err(SetupError::Config("Invalid auth choice".to_string())),
}
Ok(())
}
/// Step 4: Model selection.
///
/// Branches on the selected LLM backend and fetches models from the
@@ -1079,6 +1225,14 @@ impl SetupWizard {
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
}
"openai_codex" => {
// OAuth tokens can't call /v1/models — use hardcoded list
let models: Vec<(String, String)> = crate::llm::openai_codex::CODEX_MODELS
.iter()
.map(|(id, desc)| (id.to_string(), desc.to_string()))
.collect();
self.select_from_model_list(&models)?;
}
_ => {
// NEAR AI: use existing provider list_models()
let fetched = self.fetch_nearai_models().await;
@@ -1182,6 +1336,7 @@ impl SetupWizard {
ollama: None,
openai_compatible: None,
tinfoil: None,
openai_codex: None,
};
match create_llm_provider(&config, session) {
@@ -1437,8 +1592,11 @@ impl SetupWizard {
"HTTP webhook".to_string(),
self.settings.channels.http_enabled,
),
("Signal".to_string(), self.settings.channels.signal_enabled),
];
let non_wasm_count = options.len();
// Add available WASM channels (installed + bundled + registry)
for name in &wasm_channel_names {
let is_enabled = self.settings.channels.wasm_channels.contains(name);
@@ -1460,7 +1618,7 @@ impl SetupWizard {
.iter()
.enumerate()
.filter_map(|(idx, name)| {
if selected.contains(&(idx + 2)) {
if selected.contains(&(non_wasm_count + idx)) {
Some(name.clone())
} else {
None
@@ -1487,7 +1645,6 @@ impl SetupWizard {
any_installed = true;
}
// Then try registry channels (build from source for any still missing)
let installed_from_registry = install_selected_registry_channels(
&channels_dir,
&selected_wasm_channels,
@@ -1509,7 +1666,8 @@ impl SetupWizard {
}
// Determine if we need secrets context
let needs_secrets = selected.contains(&1) || !selected_wasm_channels.is_empty();
let needs_secrets =
selected.contains(&CHANNEL_INDEX_HTTP) || !selected_wasm_channels.is_empty();
let secrets = if needs_secrets {
match self.init_secrets_context().await {
Ok(ctx) => Some(ctx),
@@ -1523,8 +1681,8 @@ impl SetupWizard {
None
};
// HTTP is index 1
if selected.contains(&1) {
// HTTP channel
if selected.contains(&CHANNEL_INDEX_HTTP) {
println!();
if let Some(ref ctx) = secrets {
let result = setup_http(ctx).await?;
@@ -1539,6 +1697,29 @@ impl SetupWizard {
self.settings.channels.http_enabled = false;
}
// Signal channel
if selected.contains(&CHANNEL_INDEX_SIGNAL) {
println!();
let result = setup_signal(&self.settings).await?;
self.settings.channels.signal_enabled = result.enabled;
self.settings.channels.signal_http_url = Some(result.http_url);
self.settings.channels.signal_account = Some(result.account);
self.settings.channels.signal_allow_from = Some(result.allow_from);
self.settings.channels.signal_allow_from_groups = Some(result.allow_from_groups);
self.settings.channels.signal_dm_policy = Some(result.dm_policy);
self.settings.channels.signal_group_policy = Some(result.group_policy);
self.settings.channels.signal_group_allow_from = Some(result.group_allow_from);
} else {
self.settings.channels.signal_enabled = false;
self.settings.channels.signal_http_url = None;
self.settings.channels.signal_account = None;
self.settings.channels.signal_allow_from = None;
self.settings.channels.signal_allow_from_groups = None;
self.settings.channels.signal_dm_policy = None;
self.settings.channels.signal_group_policy = None;
self.settings.channels.signal_group_allow_from = None;
}
let discovered_by_name: HashMap<String, ChannelCapabilitiesFile> =
discovered_channels.into_iter().collect();
@@ -1679,9 +1860,12 @@ impl SetupWizard {
continue; // Already installed, skip
}
match installer.install_from_source(tool, false).await {
match installer.install_with_source_fallback(tool, false).await {
Ok(outcome) => {
print_success(&format!("Installed {}", outcome.name));
for warning in &outcome.warnings {
print_info(&format!("{}: {}", outcome.name, warning));
}
installed_count += 1;
// Track auth needs
@@ -1722,7 +1906,85 @@ impl SetupWizard {
Ok(())
}
/// Step 8: Heartbeat configuration.
/// Step 8: Docker Sandbox -- check Docker installation and availability.
async fn step_docker_sandbox(&mut self) -> Result<(), SetupError> {
print_info("IronClaw can execute code, run builds, and use tools inside Docker");
print_info("containers. This keeps your system safe -- commands from the LLM run");
print_info("in an isolated sandbox with no access to your credentials, limited");
print_info("filesystem access, and network traffic restricted to an allowlist.");
println!();
print_info("Without Docker, code execution tools (shell, file write) run directly");
print_info("on your machine with no isolation.");
println!();
if !confirm("Enable Docker sandbox?", false).map_err(SetupError::Io)? {
self.settings.sandbox.enabled = false;
print_info("Sandbox disabled. You can enable it later with SANDBOX_ENABLED=true.");
return Ok(());
}
// Check Docker availability
let detection = crate::sandbox::detect::check_docker().await;
match detection.status {
crate::sandbox::detect::DockerStatus::Available => {
self.settings.sandbox.enabled = true;
print_success("Docker is installed and running. Sandbox enabled.");
}
crate::sandbox::detect::DockerStatus::NotInstalled
| crate::sandbox::detect::DockerStatus::NotRunning => {
println!();
let not_installed =
detection.status == crate::sandbox::detect::DockerStatus::NotInstalled;
if not_installed {
print_error("Docker is not installed.");
print_info(detection.platform.install_hint());
} else {
print_error("Docker is installed but not running.");
print_info(detection.platform.start_hint());
}
println!();
let retry_prompt = if not_installed {
"Retry after installing Docker?"
} else {
"Retry after starting Docker?"
};
if confirm(retry_prompt, false).map_err(SetupError::Io)? {
let retry = crate::sandbox::detect::check_docker().await;
if retry.status.is_ok() {
self.settings.sandbox.enabled = true;
print_success(if not_installed {
"Docker is now available. Sandbox enabled."
} else {
"Docker is now running. Sandbox enabled."
});
} else {
self.settings.sandbox.enabled = false;
print_info(if not_installed {
"Docker still not available. Sandbox disabled for now."
} else {
"Docker still not responding. Sandbox disabled for now."
});
}
} else {
self.settings.sandbox.enabled = false;
print_info(if not_installed {
"Sandbox disabled. Install Docker and set SANDBOX_ENABLED=true later."
} else {
"Sandbox disabled. Start Docker and set SANDBOX_ENABLED=true later."
});
}
}
crate::sandbox::detect::DockerStatus::Disabled => {
self.settings.sandbox.enabled = false;
}
}
Ok(())
}
/// Step 9: Heartbeat configuration.
fn step_heartbeat(&mut self) -> Result<(), SetupError> {
print_info("Heartbeat runs periodic background tasks (e.g., checking your calendar,");
print_info("monitoring for notifications, running scheduled workflows).");
@@ -1853,6 +2115,33 @@ impl SetupWizard {
env_vars.push(("ONBOARD_COMPLETED", "true".to_string()));
}
// Signal channel env vars (chicken-and-egg: config resolves before DB).
if let Some(ref url) = self.settings.channels.signal_http_url {
env_vars.push(("SIGNAL_HTTP_URL", url.clone()));
}
if let Some(ref account) = self.settings.channels.signal_account {
env_vars.push(("SIGNAL_ACCOUNT", account.clone()));
}
if let Some(ref allow_from) = self.settings.channels.signal_allow_from {
env_vars.push(("SIGNAL_ALLOW_FROM", allow_from.clone()));
}
if let Some(ref allow_from_groups) = self.settings.channels.signal_allow_from_groups
&& !allow_from_groups.is_empty()
{
env_vars.push(("SIGNAL_ALLOW_FROM_GROUPS", allow_from_groups.clone()));
}
if let Some(ref dm_policy) = self.settings.channels.signal_dm_policy {
env_vars.push(("SIGNAL_DM_POLICY", dm_policy.clone()));
}
if let Some(ref group_policy) = self.settings.channels.signal_group_policy {
env_vars.push(("SIGNAL_GROUP_POLICY", group_policy.clone()));
}
if let Some(ref group_allow_from) = self.settings.channels.signal_group_allow_from
&& !group_allow_from.is_empty()
{
env_vars.push(("SIGNAL_GROUP_ALLOW_FROM", group_allow_from.clone()));
}
if !env_vars.is_empty() {
let pairs: Vec<(&str, &str)> = env_vars.iter().map(|(k, v)| (*k, v.as_str())).collect();
crate::bootstrap::save_bootstrap_env(&pairs).map_err(|e| {
@@ -2573,8 +2862,6 @@ fn load_registry_catalog() -> Option<crate::registry::catalog::RegistryCatalog>
/// Install selected channels from the registry that aren't already on disk
/// and weren't handled by the bundled installer.
///
/// This builds channels from source using `cargo component build`.
async fn install_selected_registry_channels(
channels_dir: &std::path::Path,
selected_channels: &[String],
@@ -2619,8 +2906,14 @@ async fn install_selected_registry_channels(
channels_dir.to_path_buf(),
);
match installer.install_from_source(manifest, false).await {
Ok(_) => {
match installer
.install_with_source_fallback(manifest, false)
.await
{
Ok(outcome) => {
for warning in &outcome.warnings {
crate::setup::prompts::print_info(&format!("{}: {}", name, warning));
}
installed.push(name.clone());
}
Err(e) => {
+310 -28
View File
@@ -5,7 +5,7 @@
//! up-to-date with the registry.
//!
//! Configuration:
//! - `CLAWHUB_REGISTRY` env var overrides the default base URL (`https://clawhub.ai`)
//! - `CLAWHUB_REGISTRY` env var overrides the default base URL
use std::sync::Arc;
use std::time::{Duration, Instant};
@@ -14,7 +14,10 @@ use serde::{Deserialize, Serialize};
use tokio::sync::RwLock;
/// Default ClawHub registry URL.
const DEFAULT_REGISTRY_URL: &str = "https://clawhub.ai";
///
/// Points directly at the Convex backend, bypassing Vercel's edge which
/// rejects non-browser TLS fingerprints (JA3/JA4 filtering).
const DEFAULT_REGISTRY_URL: &str = "https://wry-manatee-359.convex.site";
/// How long cached search results remain valid (5 minutes).
const CACHE_TTL: Duration = Duration::from_secs(300);
@@ -25,6 +28,15 @@ const MAX_RESULTS: usize = 25;
/// HTTP request timeout for catalog queries.
const REQUEST_TIMEOUT: Duration = Duration::from_secs(10);
/// Result of a catalog search, carrying both results and any error that occurred.
#[derive(Debug, Clone)]
pub struct CatalogSearchOutcome {
/// Skill entries returned by the search (empty on error).
pub results: Vec<CatalogEntry>,
/// If the registry was unreachable or returned an error, a human-readable message.
pub error: Option<String>,
}
/// A skill entry from the ClawHub catalog.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CatalogEntry {
@@ -41,18 +53,102 @@ pub struct CatalogEntry {
/// Relevance score from the search API.
#[serde(default)]
pub score: f64,
/// Last updated timestamp (epoch milliseconds from registry).
#[serde(default)]
pub updated_at: Option<u64>,
/// Star count (populated via detail enrichment).
#[serde(default)]
pub stars: Option<u64>,
/// Total download count (populated via detail enrichment).
#[serde(default)]
pub downloads: Option<u64>,
/// Current install count (populated via detail enrichment).
#[serde(default)]
pub installs_current: Option<u64>,
/// Owner handle (populated via detail enrichment).
#[serde(default)]
pub owner: Option<String>,
}
/// Top-level wrapper from the ClawHub `/api/v1/skills/{slug}` response.
///
/// The API returns `{"skill": {...}, "owner": {...}, "latestVersion": {...}}`.
#[derive(Debug, Clone, Deserialize)]
struct SkillDetailResponse {
skill: SkillDetailInner,
#[serde(default)]
owner: Option<SkillOwner>,
}
/// Inner `skill` object within `SkillDetailResponse`.
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase")]
struct SkillDetailInner {
pub slug: String,
#[serde(default)]
pub display_name: Option<String>,
#[serde(default)]
pub summary: Option<String>,
#[serde(default)]
pub stats: Option<SkillStats>,
#[serde(default)]
pub updated_at: Option<u64>,
}
/// Detailed skill information from the ClawHub `/api/v1/skills/{slug}` endpoint.
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SkillDetail {
pub slug: String,
#[serde(default)]
pub display_name: Option<String>,
#[serde(default)]
pub summary: Option<String>,
#[serde(default)]
pub version: Option<String>,
#[serde(default)]
pub stats: Option<SkillStats>,
#[serde(default)]
pub owner: Option<SkillOwner>,
#[serde(default)]
pub updated_at: Option<u64>,
}
/// Statistics for a skill from the ClawHub detail endpoint.
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SkillStats {
#[serde(default)]
pub stars: Option<u64>,
#[serde(default)]
pub downloads: Option<u64>,
#[serde(default)]
pub installs_current: Option<u64>,
#[serde(default)]
pub installs_all_time: Option<u64>,
#[serde(default)]
pub versions: Option<u64>,
}
/// Owner information for a skill.
#[derive(Debug, Clone, Deserialize)]
pub struct SkillOwner {
#[serde(default)]
pub handle: Option<String>,
#[serde(default, rename = "displayName")]
pub display_name: Option<String>,
}
/// Cached search result with TTL.
struct CachedSearch {
query: String,
results: Vec<CatalogEntry>,
outcome: CatalogSearchOutcome,
fetched_at: Instant,
}
/// Runtime skill catalog that queries ClawHub's API.
pub struct SkillCatalog {
/// Base URL for the registry (e.g. `https://clawhub.ai`).
/// Base URL for the registry.
registry_url: String,
/// HTTP client (reused across requests).
client: reqwest::Client,
@@ -64,7 +160,7 @@ impl SkillCatalog {
/// Create a new catalog.
///
/// Reads `CLAWHUB_REGISTRY` (or legacy `CLAWDHUB_REGISTRY`) from the
/// environment, falling back to `https://clawhub.ai`.
/// environment, falling back to the Convex backend.
pub fn new() -> Self {
let registry_url = std::env::var("CLAWHUB_REGISTRY")
.or_else(|_| std::env::var("CLAWDHUB_REGISTRY"))
@@ -102,9 +198,10 @@ impl SkillCatalog {
/// Search for skills in the catalog.
///
/// First checks the in-memory cache. If not cached or expired, fetches
/// from the ClawHub API. Returns an empty Vec on network errors (catalog
/// search is best-effort, never blocks the agent).
pub async fn search(&self, query: &str) -> Vec<CatalogEntry> {
/// from the ClawHub API. Returns a [`CatalogSearchOutcome`] that carries
/// both results and any error that occurred (catalog search is best-effort,
/// never blocks the agent).
pub async fn search(&self, query: &str) -> CatalogSearchOutcome {
let query_lower = query.to_lowercase();
// Check cache
@@ -113,12 +210,12 @@ impl SkillCatalog {
if let Some(cached) = cache.iter().find(|c| c.query == query_lower)
&& cached.fetched_at.elapsed() < CACHE_TTL
{
return cached.results.clone();
return cached.outcome.clone();
}
}
// Fetch from API
let results = self.fetch_search(&query_lower).await;
let outcome = self.fetch_search(&query_lower).await;
// Update cache
{
@@ -131,43 +228,75 @@ impl SkillCatalog {
}
cache.push(CachedSearch {
query: query_lower,
results: results.clone(),
outcome: outcome.clone(),
fetched_at: Instant::now(),
});
}
results
outcome
}
/// Fetch search results from the ClawHub API.
async fn fetch_search(&self, query: &str) -> Vec<CatalogEntry> {
async fn fetch_search(&self, query: &str) -> CatalogSearchOutcome {
let url = format!("{}/api/v1/search", self.registry_url);
let response = match self.client.get(&url).query(&[("q", query)]).send().await {
Ok(resp) => resp,
Err(e) => {
tracing::debug!("Catalog search failed (network): {}", e);
return Vec::new();
tracing::warn!("Catalog search failed (network): {}", e);
return CatalogSearchOutcome {
results: Vec::new(),
error: Some("Registry unreachable".to_string()),
};
}
};
if !response.status().is_success() {
let status = response.status();
tracing::debug!(
"Catalog search returned status {}: {}",
response.status(),
status,
response
.text()
.await
.unwrap_or_else(|_| "(no body)".to_string())
);
return Vec::new();
return CatalogSearchOutcome {
results: Vec::new(),
error: Some(format!("Registry returned status {status}")),
};
}
// Parse the response -- ClawHub returns an array of results.
// We try the v1 format first (with slug, displayName, version, score),
// then fall back to a simpler format.
match response.json::<Vec<CatalogSearchResult>>().await {
Ok(results) => results
// Parse the response body as text first so we can try multiple formats.
let body = match response.text().await {
Ok(b) => b,
Err(e) => {
tracing::debug!("Catalog search: failed to read response body: {}", e);
return CatalogSearchOutcome {
results: Vec::new(),
error: Some("Failed to read registry response".to_string()),
};
}
};
// Try wrapped format first: {"results": [...]}
// Then fall back to bare array: [...]
let raw_results = if let Ok(envelope) = serde_json::from_str::<CatalogSearchEnvelope>(&body)
{
envelope.results
} else if let Ok(arr) = serde_json::from_str::<Vec<CatalogSearchResult>>(&body) {
arr
} else {
let preview = body.get(..200).unwrap_or(&body);
tracing::debug!("Catalog search: failed to parse response: {}", preview);
return CatalogSearchOutcome {
results: Vec::new(),
error: Some("Invalid response from registry".to_string()),
};
};
CatalogSearchOutcome {
results: raw_results
.into_iter()
.take(MAX_RESULTS)
.map(|r| CatalogEntry {
@@ -176,11 +305,78 @@ impl SkillCatalog {
description: r.summary.unwrap_or_default(),
version: r.version.unwrap_or_default(),
score: r.score.unwrap_or(0.0),
updated_at: r.updated_at,
stars: None,
downloads: None,
installs_current: None,
owner: None,
})
.collect(),
Err(e) => {
tracing::debug!("Catalog search: failed to parse response: {}", e);
Vec::new()
error: None,
}
}
/// Fetch detailed information for a single skill by slug.
///
/// Calls `GET /api/v1/skills/{slug}` and returns the detail if available.
/// Returns `None` on any network or parse error (best-effort).
pub async fn fetch_skill_detail(&self, slug: &str) -> Option<SkillDetail> {
let url = format!(
"{}/api/v1/skills/{}",
self.registry_url,
urlencoding::encode(slug)
);
let response = self.client.get(&url).send().await.ok()?;
if !response.status().is_success() {
tracing::debug!(
"Skill detail for '{}' returned status {}",
slug,
response.status()
);
return None;
}
let wrapper = response.json::<SkillDetailResponse>().await.ok()?;
let inner = wrapper.skill;
Some(SkillDetail {
slug: inner.slug,
display_name: inner.display_name,
summary: inner.summary,
version: None, // not returned in detail response
stats: inner.stats,
owner: wrapper.owner,
updated_at: inner.updated_at,
})
}
/// Enrich catalog entries with detail data (stars, downloads, owner).
///
/// Fetches detail for up to `max` entries in parallel. Best-effort: entries
/// that fail to enrich keep their `None` values.
pub async fn enrich_search_results(&self, entries: &mut [CatalogEntry], max: usize) {
let count = entries.len().min(max);
if count == 0 {
return;
}
let futures: Vec<_> = entries[..count]
.iter()
.map(|e| self.fetch_skill_detail(&e.slug))
.collect();
let details = futures::future::join_all(futures).await;
for (entry, detail) in entries[..count].iter_mut().zip(details.into_iter()) {
if let Some(detail) = detail {
if let Some(ref stats) = detail.stats {
entry.stars = stats.stars;
entry.downloads = stats.downloads;
entry.installs_current = stats.installs_current;
}
if let Some(ref owner) = detail.owner {
entry.owner = owner.handle.clone().or_else(|| owner.display_name.clone());
}
}
}
}
@@ -202,6 +398,12 @@ impl Default for SkillCatalog {
}
}
/// Wrapper for ClawHub's `{"results": [...]}` envelope.
#[derive(Debug, Deserialize)]
struct CatalogSearchEnvelope {
results: Vec<CatalogSearchResult>,
}
/// Internal type matching ClawHub's `/api/v1/search` response items.
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -215,6 +417,8 @@ struct CatalogSearchResult {
summary: Option<String>,
#[serde(default)]
score: Option<f64>,
#[serde(default)]
updated_at: Option<u64>,
}
/// Construct the download URL for a skill's SKILL.md from the registry.
@@ -252,11 +456,13 @@ mod tests {
}
#[tokio::test]
async fn test_search_returns_empty_on_network_error() {
async fn test_search_returns_error_on_network_failure() {
// Point at an invalid URL to trigger a network error
let catalog = SkillCatalog::with_url("http://127.0.0.1:1");
let results = catalog.search("test").await;
assert!(results.is_empty());
let outcome = catalog.search("test").await;
assert!(outcome.results.is_empty());
assert!(outcome.error.is_some());
assert!(outcome.error.unwrap().contains("Registry unreachable"));
}
#[tokio::test]
@@ -295,6 +501,77 @@ mod tests {
assert!(url.contains("slug=foo%26bar%3Dbaz%23frag"));
}
#[test]
fn test_parse_wrapped_response() {
// ClawHub returns {"results": [...]} format
let json = r#"{"results":[{"slug":"markdown","displayName":"Markdown","summary":"A skill","version":"1.0.0","score":3.5}]}"#;
let envelope: CatalogSearchEnvelope = serde_json::from_str(json).unwrap();
assert_eq!(envelope.results.len(), 1);
assert_eq!(envelope.results[0].slug, "markdown");
assert_eq!(
envelope.results[0].display_name.as_deref(),
Some("Markdown")
);
}
#[test]
fn test_parse_bare_array_response() {
// Fallback: bare array format
let json = r#"[{"slug":"markdown","displayName":"Markdown","summary":"A skill","version":"1.0.0","score":3.5}]"#;
let results: Vec<CatalogSearchResult> = serde_json::from_str(json).unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].slug, "markdown");
}
#[test]
fn test_parse_skill_detail() {
// Response format matches the actual ClawHub API: {"skill": {...}, "owner": {...}}
let json = r#"{
"skill": {
"slug": "steipete/markdown-writer",
"displayName": "Markdown Writer",
"summary": "Write markdown docs",
"stats": {
"stars": 142,
"downloads": 8400,
"installsCurrent": 55,
"installsAllTime": 200,
"versions": 5
},
"updatedAt": 1700000000000
},
"owner": {
"handle": "steipete",
"displayName": "Peter S."
},
"latestVersion": {
"version": "1.2.3",
"createdAt": 1700000000000,
"changelog": ""
}
}"#;
let wrapper: SkillDetailResponse = serde_json::from_str(json).unwrap();
let inner = &wrapper.skill;
assert_eq!(inner.slug, "steipete/markdown-writer");
assert_eq!(inner.display_name.as_deref(), Some("Markdown Writer"));
let stats = inner.stats.as_ref().unwrap();
assert_eq!(stats.stars, Some(142));
assert_eq!(stats.downloads, Some(8400));
assert_eq!(stats.installs_current, Some(55));
let owner = wrapper.owner.as_ref().unwrap();
assert_eq!(owner.handle.as_deref(), Some("steipete"));
}
#[tokio::test]
async fn test_fetch_skill_detail_returns_none_on_error() {
let catalog = SkillCatalog::with_url("http://127.0.0.1:1");
let result = catalog.fetch_skill_detail("nonexistent/skill").await;
assert!(result.is_none());
}
#[test]
fn test_catalog_entry_serde() {
let entry = CatalogEntry {
@@ -303,6 +580,11 @@ mod tests {
description: "A test".to_string(),
version: "1.0.0".to_string(),
score: 0.95,
updated_at: Some(1700000000000),
stars: Some(42),
downloads: Some(1000),
installs_current: None,
owner: Some("tester".to_string()),
};
let json = serde_json::to_string(&entry).unwrap();
let parsed: CatalogEntry = serde_json::from_str(&json).unwrap();
+115 -1
View File
@@ -68,8 +68,10 @@ pub enum SkillRegistryError {
pub struct SkillRegistry {
/// All loaded skills.
skills: Vec<LoadedSkill>,
/// User skills directory (~/.ironclaw/skills/).
/// User skills directory (~/.ironclaw/skills/). Skills here are Trusted.
user_dir: PathBuf,
/// Registry-installed skills directory (~/.ironclaw/installed_skills/). Skills here are Installed.
installed_dir: Option<PathBuf>,
/// Optional workspace skills directory.
workspace_dir: Option<PathBuf>,
}
@@ -80,10 +82,22 @@ impl SkillRegistry {
Self {
skills: Vec::new(),
user_dir,
installed_dir: None,
workspace_dir: None,
}
}
/// Set the registry-installed skills directory.
///
/// Skills installed via ClawHub or the skill tools are written here and
/// loaded with `SkillTrust::Installed` (read-only tool access). This
/// directory is separate from the user dir so that trust levels survive
/// restarts correctly.
pub fn with_installed_dir(mut self, dir: PathBuf) -> Self {
self.installed_dir = Some(dir);
self
}
/// Set a workspace skills directory.
pub fn with_workspace_dir(mut self, dir: PathBuf) -> Self {
self.workspace_dir = Some(dir);
@@ -95,6 +109,7 @@ impl SkillRegistry {
/// Discovery order (earlier wins on name collision):
/// 1. Workspace skills directory (if set) -- Trusted
/// 2. User skills directory -- Trusted
/// 3. Installed skills directory (if set) -- Installed
pub async fn discover_all(&mut self) -> Vec<String> {
let mut loaded_names: Vec<String> = Vec::new();
let mut seen: HashSet<String> = HashSet::new();
@@ -129,6 +144,25 @@ impl SkillRegistry {
self.skills.push(skill);
}
// 3. Installed skills (registry-installed, lowest priority)
if let Some(inst_dir) = self.installed_dir.clone() {
let inst_skills = self
.discover_from_dir(&inst_dir, SkillTrust::Installed, SkillSource::User)
.await;
for (name, skill) in inst_skills {
if seen.contains(&name) {
tracing::debug!(
"Skipping installed skill '{}' (overridden by user/workspace)",
name
);
continue;
}
seen.insert(name.clone());
loaded_names.push(name);
self.skills.push(skill);
}
}
loaded_names
}
@@ -424,6 +458,20 @@ impl SkillRegistry {
pub fn user_dir(&self) -> &Path {
&self.user_dir
}
/// Get the installed skills directory path, if configured.
pub fn installed_dir(&self) -> Option<&Path> {
self.installed_dir.as_deref()
}
/// Get the directory where new registry installs should be written.
///
/// Returns the installed_dir if configured (preferred), otherwise falls
/// back to user_dir. In practice, the installed_dir is always set when
/// the app is running; the fallback exists for test registries.
pub fn install_target_dir(&self) -> &Path {
self.installed_dir.as_deref().unwrap_or(&self.user_dir)
}
}
/// Load and validate a single SKILL.md file from disk.
@@ -948,4 +996,70 @@ mod tests {
let h2 = compute_hash("world");
assert_ne!(h1, h2);
}
/// Skills in the installed_dir are discovered with SkillTrust::Installed,
/// not Trusted. This ensures registry-installed skills do not gain full
/// tool access after an agent restart.
#[tokio::test]
async fn test_installed_dir_uses_installed_trust() {
let user_dir = tempfile::tempdir().unwrap();
let inst_dir = tempfile::tempdir().unwrap();
// Place a skill in the installed dir
let skill_dir = inst_dir.path().join("registry-skill");
fs::create_dir(&skill_dir).unwrap();
fs::write(
skill_dir.join("SKILL.md"),
"---\nname: registry-skill\nversion: \"1.2.3\"\n---\n\nInstalled prompt.\n",
)
.unwrap();
let mut registry = SkillRegistry::new(user_dir.path().to_path_buf())
.with_installed_dir(inst_dir.path().to_path_buf());
let loaded = registry.discover_all().await;
assert_eq!(loaded, vec!["registry-skill"]);
let skill = registry.find_by_name("registry-skill").unwrap();
assert_eq!(
skill.trust,
SkillTrust::Installed,
"installed_dir skills must be Installed"
);
assert_eq!(skill.manifest.version, "1.2.3");
}
/// install_target_dir() returns installed_dir when set, user_dir otherwise.
#[test]
fn test_install_target_dir_prefers_installed_dir() {
let user_dir = PathBuf::from("/tmp/user-skills");
let inst_dir = PathBuf::from("/tmp/installed-skills");
let registry = SkillRegistry::new(user_dir.clone()).with_installed_dir(inst_dir.clone());
assert_eq!(registry.install_target_dir(), inst_dir.as_path());
let registry_no_inst = SkillRegistry::new(user_dir.clone());
assert_eq!(registry_no_inst.install_target_dir(), user_dir.as_path());
}
/// User skills (user_dir) remain Trusted even when installed_dir is set.
#[tokio::test]
async fn test_user_dir_stays_trusted_with_installed_dir() {
let user_dir = tempfile::tempdir().unwrap();
let inst_dir = tempfile::tempdir().unwrap();
let skill_dir = user_dir.path().join("my-skill");
fs::create_dir(&skill_dir).unwrap();
fs::write(
skill_dir.join("SKILL.md"),
"---\nname: my-skill\n---\n\nUser prompt.\n",
)
.unwrap();
let mut registry = SkillRegistry::new(user_dir.path().to_path_buf())
.with_installed_dir(inst_dir.path().to_path_buf());
registry.discover_all().await;
let skill = registry.find_by_name("my-skill").unwrap();
assert_eq!(skill.trust, SkillTrust::Trusted);
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ pub fn prefilter_skills<'a>(
.collect();
// Sort by score descending
scored.sort_by(|a, b| b.score.cmp(&a.score));
scored.sort_by_key(|b| std::cmp::Reverse(b.score));
// Apply candidate limit and context budget
let mut result = Vec::new();
+1
View File
@@ -289,6 +289,7 @@ impl TestHarnessBuilder {
workspace: None,
extension_manager: None,
skill_registry: None,
skill_catalog: None,
skills_config: SkillsConfig::default(),
hooks,
cost_guard,
+230 -15
View File
@@ -155,7 +155,14 @@ impl Tool for SkillSearchTool {
let query = require_str(&params, "query")?;
// Search the ClawHub catalog (async, best-effort)
let catalog_results = self.catalog.search(query).await;
let catalog_outcome = self.catalog.search(query).await;
let catalog_error = catalog_outcome.error.clone();
// Enrich top results with detail data (stars, downloads, owner)
let mut catalog_entries = catalog_outcome.results;
self.catalog
.enrich_search_results(&mut catalog_entries, 5)
.await;
// Search locally loaded skills
let installed_names: Vec<String> = {
@@ -171,7 +178,7 @@ impl Tool for SkillSearchTool {
};
// Mark catalog entries that are already installed
let catalog_json: Vec<serde_json::Value> = catalog_results
let catalog_json: Vec<serde_json::Value> = catalog_entries
.iter()
.map(|entry| {
let is_installed = installed_names.iter().any(|n| {
@@ -185,6 +192,9 @@ impl Tool for SkillSearchTool {
"version": entry.version,
"score": entry.score,
"installed": is_installed,
"stars": entry.stars,
"downloads": entry.downloads,
"owner": entry.owner,
})
})
.collect();
@@ -218,13 +228,16 @@ impl Tool for SkillSearchTool {
.collect()
};
let output = serde_json::json!({
let mut output = serde_json::json!({
"catalog": catalog_json,
"catalog_count": catalog_json.len(),
"installed": local_matches,
"installed_count": local_matches.len(),
"registry_url": self.catalog.registry_url(),
});
if let Some(err) = catalog_error {
output["catalog_error"] = serde_json::Value::String(err);
}
Ok(ToolOutput::success(output, start.elapsed()))
}
@@ -298,7 +311,7 @@ impl Tool for SkillInstallTool {
fetch_skill_content(&download_url).await?
};
// Check for duplicates and get user_dir under a brief read lock.
// Check for duplicates and get install_dir under a brief read lock.
let (user_dir, skill_name_from_parse) = {
let guard = self
.registry
@@ -318,7 +331,7 @@ impl Tool for SkillInstallTool {
)));
}
(guard.user_dir().to_path_buf(), skill_name)
(guard.install_target_dir().to_path_buf(), skill_name)
};
// Perform async I/O (write to disk, validate round-trip) with no lock held.
@@ -383,14 +396,23 @@ pub fn validate_fetch_url(url_str: &str) -> Result<(), ToolError> {
.host_str()
.ok_or_else(|| ToolError::ExecutionFailed("URL has no host".to_string()))?;
// Check if host is an IP address and reject private ranges
if let Ok(ip) = host.parse::<std::net::IpAddr>()
&& (ip.is_loopback() || ip.is_unspecified() || is_private_ip(&ip) || is_link_local_ip(&ip))
{
return Err(ToolError::ExecutionFailed(format!(
"URL points to a private/loopback/link-local address: {}",
host
)));
// Check if host is an IP address and reject private ranges.
// Unwrap IPv4-mapped IPv6 addresses (e.g. ::ffff:192.168.1.1) to catch
// SSRF bypasses that encode private IPv4 addresses as IPv6.
if let Ok(raw_ip) = host.parse::<std::net::IpAddr>() {
let ip = match raw_ip {
std::net::IpAddr::V6(v6) => v6
.to_ipv4_mapped()
.map(std::net::IpAddr::V4)
.unwrap_or(std::net::IpAddr::V6(v6)),
other => other,
};
if ip.is_loopback() || ip.is_unspecified() || is_private_ip(&ip) || is_link_local_ip(&ip) {
return Err(ToolError::ExecutionFailed(format!(
"URL points to a private/loopback/link-local address: {}",
host
)));
}
}
// Reject common internal hostnames
@@ -435,6 +457,11 @@ fn is_link_local_ip(ip: &std::net::IpAddr) -> bool {
}
/// Fetch SKILL.md content from a URL with SSRF protection.
///
/// The ClawHub registry returns skill downloads as ZIP archives containing
/// `SKILL.md` and `_meta.json`. This function detects ZIP responses (by the
/// `PK\x03\x04` magic bytes) and extracts `SKILL.md` automatically. Plain
/// text responses are returned as-is.
pub async fn fetch_skill_content(url: &str) -> Result<String, ToolError> {
validate_fetch_url(url)?;
@@ -457,10 +484,28 @@ pub async fn fetch_skill_content(url: &str) -> Result<String, ToolError> {
)));
}
let content = response
.text()
// Limit download size to prevent memory exhaustion from large responses.
const MAX_DOWNLOAD_BYTES: usize = 10 * 1024 * 1024; // 10 MB
let bytes = response
.bytes()
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Failed to read response body: {}", e)))?;
if bytes.len() > MAX_DOWNLOAD_BYTES {
return Err(ToolError::ExecutionFailed(format!(
"Response too large: {} bytes (max {} bytes)",
bytes.len(),
MAX_DOWNLOAD_BYTES
)));
}
// Detect ZIP archive (PK\x03\x04 magic) and extract SKILL.md
let content = if bytes.starts_with(b"PK\x03\x04") {
extract_skill_from_zip(&bytes)?
} else {
String::from_utf8(bytes.to_vec()).map_err(|e| {
ToolError::ExecutionFailed(format!("Response is not valid UTF-8: {}", e))
})?
};
// Basic size check
if content.len() as u64 > crate::skills::MAX_PROMPT_FILE_SIZE {
@@ -474,6 +519,102 @@ pub async fn fetch_skill_content(url: &str) -> Result<String, ToolError> {
Ok(content)
}
/// Extract `SKILL.md` from a ZIP archive returned by the ClawHub download API.
///
/// Walks ZIP local file headers looking for an entry named `SKILL.md`.
/// Supports Store (method 0) and Deflate (method 8) compression.
fn extract_skill_from_zip(data: &[u8]) -> Result<String, ToolError> {
use flate2::read::DeflateDecoder;
use std::io::Read;
// SKILL.md files should never be larger than 1 MB.
const MAX_DECOMPRESSED: usize = 1_024 * 1_024;
let mut offset = 0;
while offset + 30 <= data.len() {
// Local file header signature = PK\x03\x04
if data[offset..offset + 4] != [0x50, 0x4B, 0x03, 0x04] {
break;
}
let compression = u16::from_le_bytes([data[offset + 8], data[offset + 9]]);
let compressed_size = u32::from_le_bytes([
data[offset + 18],
data[offset + 19],
data[offset + 20],
data[offset + 21],
]) as usize;
let uncompressed_size = u32::from_le_bytes([
data[offset + 22],
data[offset + 23],
data[offset + 24],
data[offset + 25],
]) as usize;
let name_len = u16::from_le_bytes([data[offset + 26], data[offset + 27]]) as usize;
let extra_len = u16::from_le_bytes([data[offset + 28], data[offset + 29]]) as usize;
let name_start = offset + 30;
let name_end = name_start + name_len;
if name_end > data.len() {
break;
}
let file_name = std::str::from_utf8(&data[name_start..name_end]).unwrap_or("");
let data_start = name_end
.checked_add(extra_len)
.ok_or_else(|| ToolError::ExecutionFailed("ZIP header offset overflow".to_string()))?;
let data_end = data_start
.checked_add(compressed_size)
.ok_or_else(|| ToolError::ExecutionFailed("ZIP header size overflow".to_string()))?;
if file_name == "SKILL.md" {
if data_end > data.len() {
return Err(ToolError::ExecutionFailed(
"ZIP archive truncated".to_string(),
));
}
if uncompressed_size > MAX_DECOMPRESSED {
return Err(ToolError::ExecutionFailed(
"ZIP entry too large to decompress safely".to_string(),
));
}
let raw = &data[data_start..data_end];
let decompressed = match compression {
0 => raw.to_vec(), // Store
8 => {
// Deflate -- wrap with a read limit to guard against ZIP bombs
// where the declared size is small but decompressed output is huge.
let mut decoder = DeflateDecoder::new(raw).take(MAX_DECOMPRESSED as u64);
let mut buf = Vec::with_capacity(uncompressed_size.min(MAX_DECOMPRESSED));
decoder.read_to_end(&mut buf).map_err(|e| {
ToolError::ExecutionFailed(format!("Failed to decompress SKILL.md: {}", e))
})?;
buf
}
other => {
return Err(ToolError::ExecutionFailed(format!(
"Unsupported ZIP compression method: {}",
other
)));
}
};
return String::from_utf8(decompressed).map_err(|e| {
ToolError::ExecutionFailed(format!("SKILL.md in archive is not valid UTF-8: {}", e))
});
}
// Skip to next entry
offset = data_end;
}
Err(ToolError::ExecutionFailed(
"ZIP archive does not contain SKILL.md".to_string(),
))
}
// ── skill_remove ────────────────────────────────────────────────────────
pub struct SkillRemoveTool {
@@ -675,4 +816,78 @@ mod tests {
let err = super::validate_fetch_url("file:///etc/passwd").unwrap_err();
assert!(err.to_string().contains("Only HTTPS"));
}
#[test]
fn test_extract_skill_from_zip_deflate() {
// Build a real ZIP with flate2 + manual header construction.
use flate2::Compression;
use flate2::write::DeflateEncoder;
use std::io::Write;
let skill_md = b"---\nname: test\n---\n# Test Skill\n";
let mut encoder = DeflateEncoder::new(Vec::new(), Compression::default());
encoder.write_all(skill_md).unwrap();
let compressed = encoder.finish().unwrap();
let mut zip = Vec::new();
// Local file header
zip.extend_from_slice(&[0x50, 0x4B, 0x03, 0x04]); // signature
zip.extend_from_slice(&[0x14, 0x00]); // version needed (2.0)
zip.extend_from_slice(&[0x00, 0x00]); // flags
zip.extend_from_slice(&[0x08, 0x00]); // compression: deflate
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // mod time/date
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // crc32 (unused)
zip.extend_from_slice(&(compressed.len() as u32).to_le_bytes()); // compressed size
zip.extend_from_slice(&(skill_md.len() as u32).to_le_bytes()); // uncompressed size
zip.extend_from_slice(&8u16.to_le_bytes()); // filename length
zip.extend_from_slice(&0u16.to_le_bytes()); // extra field length
zip.extend_from_slice(b"SKILL.md");
zip.extend_from_slice(&compressed);
let result = super::extract_skill_from_zip(&zip).unwrap();
assert_eq!(result, "---\nname: test\n---\n# Test Skill\n");
}
#[test]
fn test_extract_skill_from_zip_store() {
let skill_md = b"---\nname: stored\n---\n# Stored\n";
let mut zip = Vec::new();
// Local file header
zip.extend_from_slice(&[0x50, 0x4B, 0x03, 0x04]);
zip.extend_from_slice(&[0x0A, 0x00]); // version needed (1.0)
zip.extend_from_slice(&[0x00, 0x00]); // flags
zip.extend_from_slice(&[0x00, 0x00]); // compression: store
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // mod time/date
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // crc32
zip.extend_from_slice(&(skill_md.len() as u32).to_le_bytes()); // compressed = uncompressed
zip.extend_from_slice(&(skill_md.len() as u32).to_le_bytes());
zip.extend_from_slice(&8u16.to_le_bytes()); // filename length
zip.extend_from_slice(&0u16.to_le_bytes()); // extra field length
zip.extend_from_slice(b"SKILL.md");
zip.extend_from_slice(skill_md);
let result = super::extract_skill_from_zip(&zip).unwrap();
assert_eq!(result, "---\nname: stored\n---\n# Stored\n");
}
#[test]
fn test_extract_skill_from_zip_missing_skill_md() {
let mut zip = Vec::new();
zip.extend_from_slice(&[0x50, 0x4B, 0x03, 0x04]);
zip.extend_from_slice(&[0x0A, 0x00]); // version
zip.extend_from_slice(&[0x00, 0x00]); // flags
zip.extend_from_slice(&[0x00, 0x00]); // compression: store
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // mod time/date
zip.extend_from_slice(&[0x00, 0x00, 0x00, 0x00]); // crc32
zip.extend_from_slice(&2u32.to_le_bytes()); // compressed size
zip.extend_from_slice(&2u32.to_le_bytes()); // uncompressed size
zip.extend_from_slice(&10u16.to_le_bytes()); // filename length
zip.extend_from_slice(&0u16.to_le_bytes()); // extra field length
zip.extend_from_slice(b"_meta.json");
zip.extend_from_slice(b"{}");
let err = super::extract_skill_from_zip(&zip).unwrap_err();
assert!(err.to_string().contains("does not contain SKILL.md"));
}
}
+2
View File
@@ -201,6 +201,7 @@ async fn start_test_server_with_provider(
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
@@ -689,6 +690,7 @@ async fn test_no_llm_provider_returns_503() {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
+1
View File
@@ -59,6 +59,7 @@ async fn start_test_server() -> (
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();