Compare commits

...
28 Commits
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>Illia Polosukhin
b8901baafd chore: release v0.10.0 (#279)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-22 18:16:21 +00:00
4003300a8c fix: improve Telegram status delivery and reliability (#304)
* fix: make Telegram status prompts reliable

Approval and auth prompts could be missed when polling or reply-context sends failed, leaving users stuck in waiting states. This adds explicit status mapping and retries, keeps typing active through intermediate work while suppressing noisy tool telemetry, and adds regression tests plus CI coverage for the Telegram channel crate.

* fix: normalize terminal status handling

Terminal status strings from the agent loop can vary in casing and formatting, which could leak internal status lines to Telegram. This normalizes Done/Interrupted mapping and filters terminal status text consistently to keep chat UX clean while preserving actionable prompts.

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 18:07:57 +00:00
Robert YanandGitHub c68dc2ff2a feat: update dashboard favicon (#309) 2026-02-22 18:06:50 +00:00
d4785ce4d2 fix: persist user message at turn start before agentic loop (#305)
* fix: persist user message at turn start before agentic loop

Split persist_turn into persist_user_message + persist_assistant_response.
The user message is now written to DB immediately after thread.start_turn(),
before the agentic loop runs. This ensures the message survives process
crashes mid-response. The assistant response is persisted only on completion.

Updated all 6 call sites in thread_ops.rs (success, error, approval
success/error, rejection, and auth intercept paths).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: document persist_assistant_response dependency on persist_user_message

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: re-ensure conversation in persist_assistant_response

Add ensure_conversation call and user_id parameter to
persist_assistant_response so assistant replies are still persisted
even if persist_user_message failed transiently at turn start.

Addresses PR review feedback from @ilblackdragon.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:50:28 +00:00
2544df1c4a feat: add web UI test skill for Chrome extension (#302)
* feat: add web UI test skill for Chrome extension testing

Add a SKILL.md checklist for manually testing the IronClaw web gateway
UI using the Claude for Chrome browser extension. Covers connection,
chat, skills tab (search, install by search, install by URL, remove),
and smoke tests for other tabs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use placeholder token and correct cleanup path per review

- Replace hardcoded test123 token with <your-token> placeholder
- Fix cleanup path: ~/.ironclaw/installed_skills/ (not skills/)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:49:35 +00:00
82f24bf08f fix: block send until thread is selected (#306)
* fix: block send until thread is selected

Prevents messages from ending up in orphan threads when user sends
while currentThreadId is null during page load.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: guard enableChatInput against null thread + add user feedback

Prevents SSE events from re-enabling input before a thread is selected.
Adds status message when user tries to send without a thread.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:19:18 +00:00
510fba4c92 fix: reload chat history on SSE reconnect (#307)
When SSE auto-reconnects after a server restart, the chat now
re-syncs from the database so no messages are lost.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-02-22 08:18:46 +00:00
04d3b005b1 feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch

FullJob routines previously fell back to lightweight mode (single LLM call,
no tools) with a warning. This wires them to the existing Scheduler/Worker
infrastructure so they dispatch real jobs with full tool access.

Fire-and-forget model: the routine creates a job via ContextManager, schedules
it, links the routine_run to the job_id, and completes immediately. The job
runs independently with full tool access.

- Add RoutineError::JobDispatchFailed variant
- Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL)
- Add execute_full_job() in routine_engine with context_manager/scheduler
- Wire context_manager + scheduler into RoutineEngine from agent_loop
- Fix pre-existing clippy warnings in tests/html_to_markdown.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist job to DB before scheduling in execute_full_job

The worker emits job_actions and llm_calls rows that reference agent_jobs
via foreign key. Without persisting the job first, those inserts can fail.
Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations

Move the create + persist + schedule sequence into a single
Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs)
don't duplicate the logic. FullJob routines now pass max_iterations via job
metadata, and the worker reads it (defaulting to 50 if unset).

Also removes the context_manager field from RoutineEngine since dispatch_job
handles everything internally.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clamp max_iterations to 500 and log category update failures

Address PR review feedback:
- worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500)
  to prevent unbounded LLM token usage from malicious/buggy configs
- commands.rs: log warning on category update failure instead of silently
  discarding the error

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:18:21 +00:00
ea57447649 feat: hot-activate WASM channels, channel-first prompts, unified artifact resolution (#297)
* refactor: unify WASM artifact resolution into registry/artifacts.rs

Consolidate duplicated WASM find/build/install logic from 5+ files into
a single src/registry/artifacts.rs module. This fixes two bugs:
- registry/installer.rs now respects CARGO_TARGET_DIR (was hardcoded)
- channels/wasm/bundled.rs now searches all WASM triples (was wasip2 only)

Also includes: extension manager hot-activation for WASM channels,
extension guidance in LLM prompts, channel manager hot-add support,
webhook router channel lookup, and minor cleanups.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: send approval prompts as messages on WASM channels (Telegram, Slack)

WASM channels mapped ApprovalNeeded status to a typing indicator,
so users on Telegram never saw tool approval prompts — the agent
got stuck in AwaitingApproval and all subsequent messages failed
with "Waiting for approval".

- Intercept ApprovalNeeded in WasmChannel::handle_status_update and
  send the prompt as an actual message via call_on_respond, showing
  tool name, description, parameters, and yes/no/always instructions
- Guard against empty LLM responses after clean_response() strips
  reasoning_content think-tags (defense-in-depth for reasoning models)
- Add reasoning_content fallback to NearAiChatProvider::complete()
  for consistency with complete_with_tools()
- Add debug logging when empty responses are suppressed
- Improve error logging for channel respond() failures
- Register WASM channel webhook routes before credential checks so
  platforms don't deactivate webhook URLs with 404s

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #297 review comments

- ChannelManager::add: use async write().await instead of try_write()
- resolve_target_dir: resolve relative CARGO_TARGET_DIR against crate_dir
- install_wasm_files: log warning on capabilities copy failure
- refresh_active_channel: load capabilities file for webhook secret name
- activate_wasm_channel: validate name against path traversal
- Fix cargo fmt formatting in nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire up channel runtime for hot-activation and address PR review round 2

- Wire up set_channel_runtime() in main.rs so hot-activation actually works
  (with_channel_runtime was never called — hot-activation was dead code)
- Change ExtensionManager channel runtime fields to RwLock<Option<...>>
  interior mutability so set_channel_runtime(&self) works after Arc wrapping
- Fix artifact tests to use resolve_target_dir() instead of hardcoding
  "target/" (breaks when CARGO_TARGET_DIR is set)
- Fix bundled.rs build hint: cargo component build (not cargo build --target)
- Fix wasm_artifact_path doc: binary_name should not include .wasm extension

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use char-aware truncation to prevent UTF-8 panic in approval prompt

&s[..77] panics on multi-byte UTF-8 (CJK, emoji). Use s.chars().take(77)
for safe truncation at character boundaries.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 08:09:56 +00:00
a320f265b3 Fix tool schema OpenAI compatibility (#301)
* fix: remove union type arrays from tool schemas for OpenAI compatibility

OpenAI rejects JSON Schema union types containing "array" without an
"items" subschema. The http tool's "body" and json tool's "data" params
used union types to accept any value. Replace with freeform (untyped)
schemas which OpenAI treats as accepting any JSON value.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: update schema tests to assert type is absent, fix missed json.rs test

- http.rs test: assert body has no "type" (not just has description)
- json.rs test: update to match the freeform schema change (was still
  asserting type is present)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 07:52:34 +00:00
c3ce26278a refactor: simplify config resolution and consolidate main.rs init (#287)
* refactor: simplify config resolution and consolidate main.rs init into AppBuilder

- Add parse_bool_env() and parse_string_env() helpers to eliminate repetitive
  5-line optional_env/parse/map_err/unwrap_or boilerplate across 12 config files
- Add EmbeddingsConfig::create_provider() to centralize embeddings construction
  (fixes hardcoded 1536 dimensions and missing Ollama provider in app.rs)
- Extract init_cli_tracing(), setup_wasm_channels(), start_tunnel(),
  run_memory_command(), run_worker(), run_claude_bridge() from main.rs
- Replace ~600 lines of inline init in main.rs with AppBuilder::build_all()
- Expose catalog_entries from AppComponents for gateway registry entries
- Net reduction: ~738 lines across 15 files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: propagate dev_loaded_tool_names from AppBuilder and add parse_option_env helper

Address PR review feedback:

- Capture dev_loaded_tool_names from WASM loading in init_extensions()
  and expose via AppComponents so bootstrap_hooks receives the actual
  dev tool names instead of an empty slice (fixes silent hook skip)
- Add parse_option_env<T>() helper for Option<T> config fields,
  simplifying max_cost_per_day_cents and max_actions_per_hour in agent.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: fetch real NEAR AI pricing and unify cost calculation path

CostGuard was independently looking up pricing via costs::model_cost(),
falling back to GPT-4o default rates when NEAR AI model names didn't
match the static table — causing ~3x cost overestimates in logs.

- Add pricing map to NearAiChatProvider that fetches real rates from
  /v1/model/list at startup (background, non-blocking)
- Update cost_per_token() to check fetched pricing first, then static
  table, then default
- Add cost_per_token parameter to CostGuard::record_llm_call() so the
  dispatcher passes provider-sourced rates directly

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: update default NEAR AI model to GLM-latest

Replace fireworks llama4-maverick-instruct-basic with zai-org/GLM-latest
as the default model in config and setup wizard.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: align wizard default model name with config

Change "zai/GLM-latest" to "zai-org/GLM-latest" in wizard.rs to match
the default in config/llm.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-22 02:54:31 +00:00
mfcoburnandGitHub 91b602790a Update image source in README.md 2026-02-21 17:21:50 -05:00
mfcoburnandGitHub e5ce076773 Add files via upload 2026-02-21 15:14:57 -07:00
c1f3b83c98 refactor: remove ExtensionSource::Bundled, use download-only install for WASM channels (#293)
The Bundled variant and its local-artifacts fallback are superseded by the
embedded registry catalog which provides WasmDownload entries with GitHub
release URLs. The in-chat extension manager now always downloads channel
WASM binaries from releases, simplifying the install path.

The setup wizard retains its own local install_bundled_channel path for
dev builds where build artifacts exist on disk.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 13:55:49 -08:00
37c0158765 Fix: allow OAuth callback to work on remote servers (fixes #186) (#212)
* fix: allow OAuth callback to work on remote servers via OAUTH_CALLBACK_HOST

Fixes #186.

The OAuth callback URL was hardcoded to `http://127.0.0.1:9876` in two
places (NEAR AI login and MCP server auth). On a remote server this URL
is unreachable from the user's browser, making authentication impossible.

Changes:
- Add `callback_host()` to `oauth_defaults` that reads `OAUTH_CALLBACK_HOST`
  (default: `127.0.0.1`)
- Update `bind_callback_listener()` to bind to `0.0.0.0` when a non-loopback
  host is configured, so the port is reachable from outside the machine
- Update `session.rs` and `mcp/auth.rs` to use `callback_host()` instead
  of hardcoded `127.0.0.1` / `localhost`

Usage on a remote server:
  export OAUTH_CALLBACK_HOST=<your-server-ip>
  ironclaw login

* fix: address PR review comments for OAuth callback security

* fix: address serrrfirat review comments on PR #212

---------

Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 15:39:47 +04:00
0a30c95ee1 Feat: add rate limiting for built-in tools (closes #171) (#276)
* feat: add rate limiting for built-in tools (closes #171)

Extend the Tool trait with an optional rate_limit_config() method and
wire a shared sliding-window RateLimiter into the tool execution path in
worker.rs so that per-tool per-user limits are enforced at runtime.

- Add ToolRateLimitConfig struct (requests_per_minute / requests_per_hour)
  and rate_limit_config() default method to the Tool trait
- Extract shared RateLimiter from tools/wasm/ into tools/rate_limiter.rs;
  WASM rate_limiter.rs now re-exports from the shared module
- Add RateLimited error variant to crate::error::ToolError
- Register RateLimiter on ToolRegistry and check limits in execute_tool_inner
- Apply conservative configs to high-impact tools:
    ShellTool        30 rpm / 300 rph
    HttpTool         30 rpm / 500 rph
    WriteFileTool    20 rpm / 200 rph
    ApplyPatchTool   20 rpm / 200 rph
    MemoryWriteTool  20 rpm / 200 rph
    CreateJobTool     5 rpm /  30 rph

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor: address Gemini review comments on rate limiter

- worker.rs: collapse nested if-let into a single `if let ... && let ...`
  (clippy::collapsible_if)
- rate_limiter.rs: extract check_internal(record: bool) helper to DRY up
  check_and_record / check (were identical except for the increment step)
- rate_limiter.rs: replace magic numbers 60 / 3600 with MINUTE_SECS /
  HOUR_SECS constants

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 14:39:53 +04:00
b3bf50f10e feat: add pairing/permission system to all WASM channels and fix extension registry (#286)
Port Telegram's permission model (owner_id, dm_policy, allow_from, pairing codes)
to Discord, Slack, and WhatsApp WASM channels. Add web UI for configuration and
pairing approval. Fix extension registry issues preventing Discord install and
causing Slack activation to hit the wrong endpoint.

WASM channels:
- Discord: add DiscordConfig, permission checks, ephemeral pairing replies,
  fix capabilities.json (header_name→name), downgrade wit-bindgen to 0.36
- Slack: expand SlackConfig with permission fields, add check_sender_permission
  and send_pairing_reply via chat.postMessage
- WhatsApp: expand WhatsAppConfig with permission fields, add permission checks
  and pairing reply via Cloud API
- Telegram: reformat capabilities.json, add setup.required_secrets

Extension system:
- Add Discord to KNOWN_CHANNELS in bundled.rs and to extension registry
- Rename "slack" MCP→"slack-mcp", "slack-channel"→"slack" to fix name collision
- Add ExtensionSource::Bundled variant handling in discovery.rs
- Add get_setup_schema/save_setup_secrets to ExtensionManager
- Add needs_setup field to InstalledExtension

Web gateway:
- Add GET/POST /api/extensions/{name}/setup for configuration modal
- Add GET /api/pairing/{channel} and POST /api/pairing/{channel}/approve
- Add configure modal UI (password fields, provided badges, auto-generate hints)
- Add pairing request UI on active WASM channel cards
- Show "Restart to activate" label instead of Activate button for WASM channels

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-20 23:21:32 -08:00
48b5323ec9 feat: group chat privacy, channel-aware prompts, and safety hardening (#285)
Prevent personal memory (MEMORY.md) from leaking into group chat contexts
by adding system_prompt_for_context(is_group_chat) to the workspace. Add
channel-specific formatting hints (Discord, Telegram, Slack, WhatsApp),
runtime metadata injection, group chat behavioral guidance with NO_REPLY
silent token, safety rules in the system prompt, tool call style guidance,
wrap_external_content() for untrusted data, and improved workspace seed
files with richer identity/soul/agent templates and heartbeat checklist.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 06:28:33 +00:00
3124ab2b7f docs: add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) (#193)
- Add docs/LLM_PROVIDERS.md with setup instructions for all supported providers
- Expand .env.example with Together AI and Fireworks AI example configs
- Add "Alternative LLM Providers" section to README with quickstart snippet

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-02-21 10:11:54 +04:00
dbd3e0807f Feat/html to markdown #106 (#115)
* feat: add HTML-to-Markdown conversion for web content

- Add readabilityrs for content extraction
- Add html-to-markdown for conversion
- Feature-gated behind html-markdown flag
- Integrates with HTTP tool response handling
- Includes comprehensive tests and examples

Closes #106

* Update comments for is_html_response helper and fix tests to not fail silently in certain instances

---------

Co-authored-by: Zach Frederick <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 06:05:16 +00:00
436066415b feat: embedded registry catalog and WASM bundle install pipeline (#283)
* feat: embedded registry catalog and WASM bundle install pipeline

Embed registry manifests at compile time so the extension catalog is
available without network access. Add tar.gz bundle support for WASM
extension downloads (tools and channels), a /api/extensions/registry
endpoint, CI job to build and publish WASM bundles on release, and
ephemeral in-memory secrets fallback so the extension manager works
even without a persistent secrets store.

Key changes:
- build.rs: collect registry/*.json into embedded_catalog.json at compile time
- src/registry/embedded.rs + catalog.rs: load embedded or on-disk catalog
- src/extensions/manager.rs: download_and_install_wasm handles tar.gz bundles,
  bare .wasm files, and separate capabilities downloads; wasm channel install
- src/channels/web/server.rs: /api/extensions/registry endpoint + no-cache headers
- src/app.rs: ephemeral InMemorySecretsStore fallback for extension manager
- registry/*.json: populate artifact download URLs for release bundles
- .github/workflows/release.yml: build-wasm-extensions CI job
- Simplified setup wizard and CLI registry commands

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — archive hardening, decompression bomb guard, test fix

- Add 100 MB decompressed entry size cap to tar.gz extraction in both
  manager.rs and installer.rs to prevent decompression bombs
- Add archive.set_preserve_permissions(false) and set_unpack_xattrs(false)
  for defense-in-depth against malicious archives
- Fix test assertion logic in catalog.rs (|| → || with correct negation)
- Replace silent tar fallback in CI with explicit if/else for capabilities
- Add warning when installing without SHA256 verification

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: resolve clippy warning in settings.rs and enforce zero-warnings policy

Use struct initializer with ..Default::default() instead of field
reassignment. Update CLAUDE.md to codify zero clippy warnings policy —
all warnings must be fixed before committing, including pre-existing ones.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review round 2 — build reliability, caps validation, naming

- build.rs: emit per-file rerun-if-changed for reliable content tracking;
  fix bundles fallback to match BundlesFile shape ({"bundles":{}})
- embedded.rs: parse catalog once via OnceLock instead of double-parsing
- manager.rs + installer.rs: add 1 MB size cap on capabilities_url downloads
  with proper error surfacing
- secrets/store.rs: rename misleading `pub mod testing` to `pub mod in_memory`
- server.rs: track installed extensions by (name, kind) tuple to avoid
  false positives across different extension kinds

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 05:43:28 +00:00
firat.sertgozGitHubIllia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
3d4c647216 fix: map Esc to interrupt and Ctrl+C to graceful quit (#267)
* fix: map Esc to interrupt and Ctrl+C to graceful quit

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-21 04:46:15 +00:00
b68d67bd35 feat: show token usage and cost tracker in gateway status popover (#284)
* feat: show token usage, cost tracker, and uptime in gateway status popover

The "Connected" hover popover in the web gateway now displays three
sections: connection info (SSE/WS counts, uptime), daily cost tracker
(spend + actions/hr), and per-model token usage (input/output counts
with cost per model). Also fixes the field name mismatch between the
backend response and JS rendering that prevented the popover from
showing correct data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — escape HTML in popover, add model_usage test

- Escape model name and cost strings with escapeHtml() before inserting
  into innerHTML to prevent XSS via crafted model names
- Add test_model_usage_per_model_tracking test covering multi-model
  token/cost accumulation in CostGuard

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 03:45:04 +00:00
493e4578d0 feat: support custom HTTP headers for OpenAI-compatible provider (#269)
Add LLM_EXTRA_HEADERS env var (format: Key:Value,Key2:Value2) to inject
custom HTTP headers into every request to OpenAI-compatible endpoints.
This enables OpenRouter attribution headers (HTTP-Referer, X-Title)
and other service-specific headers without code changes.

Closes #179

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-21 03:06:15 +00:00
250551799b style: adopt agent-market design language for web UI (#282)
* fix: move Logs to status bar and fix chat history ordering after restart

Move the Logs tab out of the main tab bar and into the right-side status
area as a compact pill button next to "Connected". Remove it from the
Ctrl+1-N shortcut order (now Ctrl+1-5).

Fix chat message ordering in libSQL backend: datetime('now') has only
second precision, so back-to-back user+assistant inserts got identical
timestamps causing non-deterministic ORDER BY. Now passes explicit
millisecond-precision timestamps and uses rowid as tiebreaker for
existing data.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: separate WASM extensions from MCP servers on Extensions page

Reorganize the Extensions tab into 5 distinct sections: Installed
Extensions, Available WASM Extensions, Install WASM Extension (by
tar.gz URL), MCP Servers (with Add Custom form), and Registered Tools.
Registry entries are now filtered client-side by kind so WASM tools/
channels and MCP servers each have dedicated UI sections.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: adopt agent-market design language for web UI

Refresh the web gateway visual identity with a cleaner, modern aesthetic:
deeper blacks, green accent palette, DM Sans + IBM Plex Mono typography,
larger border-radii, glassmorphic navigation, refined hover effects,
pill badges, and green focus rings. CSS-only change plus Google Fonts.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

* Update src/channels/web/static/style.css

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-02-21 02:50:51 +00:00
c038c7705b feat: add smart routing provider for cost-optimized model selection (#281)
* feat: add smart routing provider for cost-optimized model selection

Route simple tasks (greetings, status checks, short questions) to a cheap
model (e.g. Haiku) and complex tasks (code generation, analysis) to the
primary model, reducing agent costs without sacrificing quality.

Activates automatically when NEARAI_CHEAP_MODEL is set. Cascade mode
retries uncertain cheap-model responses with the primary model.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: extract provider chain into shared build_provider_chain()

Consolidate the duplicated LLM provider chain construction from main.rs
and app.rs into a single build_provider_chain() function in llm/mod.rs.

This fixes the inconsistency where app.rs was missing retry wrapping
that main.rs had, and ensures both paths apply identical decorators:
retry → smart routing → failover → circuit breaker → cache.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — uncertainty detection and clippy lint

- Remove false-positive short response (<20 chars) uncertainty check
  that would escalate "Yes.", "42" etc. Now only empty responses and
  explicit uncertainty phrases trigger cascade escalation.
- Add #[allow(clippy::type_complexity)] to build_provider_chain() to
  fix CI clippy -D warnings failure.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:37:45 +00:00
98ee648fcb perf: speed up startup from ~15s to ~2s (#280)
Three high-impact changes eliminate most startup latency:

1. Enable wasmtime persistent compilation cache — call
   cache_config_load_default() so compiled native code is serialized to
   disk (~/.cache/wasmtime). Subsequent startups deserialize instead of
   recompiling, dropping the WASM phase from ~13s to <1s.

2. Cache compiled Component in PreparedModule — store the compiled
   wasmtime::component::Component directly instead of raw bytes.
   Eliminates ~2.6s recompilation on every first tool/channel execution.

3. Move blocking housekeeping to background tasks — embedding backfill
   (~1.3s of failing HTTP calls) and stale job cleanup are fire-and-forget
   work that no longer blocks the critical startup path.

Also: deduplicate Workspace creation in main.rs (two identical instances
reduced to one), and replace leftover println! in session validation with
tracing calls.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 02:30:57 +00:00
2cdd1acb1e refactor: consolidate tool approval into single param-aware method (#274)
* refactor: consolidate tool approval into single param-aware method

Replace the two confusing approval methods (requires_approval() and
requires_approval_for()) with a single requires_approval(&self, params)
returning a 3-variant ApprovalRequirement enum (Never, UnlessAutoApproved,
Always). This enables param-aware approval decisions: HTTP calls without
auth headers now skip approval entirely, while authenticated requests
always require it. Shell tool merges its destructive-command detection
into the same method.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add credential injection to built-in HTTP tool

Wire the WASM credential injection system into the built-in HTTP tool
so credentials are auto-injected at the boundary (zero-exposure model).

- Add SharedCredentialRegistry: thread-safe, append-only registry of
  credential mappings populated by WASM tools at registration time
- Add credential_detect module with broad auth detection for headers
  (12 exact + 5 substring matches), header values (7 auth scheme
  prefixes), and URL query params (17 exact + 5 substring matches)
- HttpTool now accepts optional credential registry + secrets store,
  auto-injects matching credentials in execute(), and uses broader
  auth detection in requires_approval()
- ToolRegistry passes credential registry to HttpTool at startup and
  populates it when WASM tools register
- Remove old hardcoded AUTH_HEADER_NAMES / has_auth_headers in favor
  of the new params_contain_manual_credentials()

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #274 review comments (query param injection, lock poisoning, visibility)

- Fix injected query params not being sent on outbound HTTP requests by
  also calling .query() on the RequestBuilder alongside parsed_url mutation
- Recover from poisoned RwLock in SharedCredentialRegistry instead of
  silently ignoring failures, with tracing::warn for visibility
- Narrow inject_credential and host_matches_pattern to pub(crate) to
  avoid committing to them as stable public API

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-21 01:28:23 +00:00
147 changed files with 32177 additions and 3330 deletions
+2 -2
View File
@@ -286,8 +286,8 @@ impl Tool for <Name>Tool {
false // Set true if tool processes external data
}
fn requires_approval(&self) -> bool {
false // Set true if tool is destructive or contacts external services
fn requires_approval(&self, _params: &serde_json::Value) -> crate::tools::tool::ApprovalRequirement {
crate::tools::tool::ApprovalRequirement::Never // Set to UnlessAutoApproved or Always as needed
}
}
```
+17 -2
View File
@@ -33,11 +33,26 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_BASE_URL=http://localhost:1234/v1
# LLM_API_KEY=sk-... # optional for local servers
# === OpenRouter (via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://openrouter.ai/api/v1
# LLM_API_KEY=sk-or-...
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
# === Together AI (via OpenAI-compatible) ===
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.together.xyz/v1
# LLM_API_KEY=...
# === Fireworks AI (via OpenAI-compatible) ===
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
# LLM_BACKEND=openai_compatible
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
# LLM_API_KEY=fw_...
# For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration
+1
View File
@@ -0,0 +1 @@
tests/test-pages/**/*.html linguist-generated=true
+101 -2
View File
@@ -214,14 +214,113 @@ jobs:
path: |
${{ steps.cargo-dist.outputs.paths }}
${{ env.BUILD_MANIFEST_NAME }}
# Build WASM extension bundles (tar.gz with .wasm + .capabilities.json)
build-wasm-extensions:
needs:
- plan
if: ${{ needs.plan.outputs.publishing == 'true' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install Rust toolchain + wasm target
run: |
rustup target add wasm32-wasip2
cargo install cargo-component --locked || true
- uses: swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Build and package WASM extensions
shell: bash
run: |
set -euo pipefail
mkdir -p target/wasm-bundles
# Process each manifest in registry/tools/ and registry/channels/
for manifest in registry/tools/*.json registry/channels/*.json; do
[ -f "$manifest" ] || continue
name=$(jq -r '.name' "$manifest")
source_dir=$(jq -r '.source.dir' "$manifest")
caps_file=$(jq -r '.source.capabilities' "$manifest")
crate_name=$(jq -r '.source.crate_name' "$manifest")
if [ ! -d "$source_dir" ]; then
echo "::warning::Source dir '$source_dir' not found for '$name', skipping"
continue
fi
echo "=== Building $name from $source_dir ==="
# Build WASM component
cargo component build --release --manifest-path "$source_dir/Cargo.toml" || {
echo "::warning::Build failed for '$name', skipping"
continue
}
# Find the built WASM file (Cargo uses underscores in artifact names)
wasm_artifact="${crate_name//-/_}"
wasm_path=""
for target_dir in wasm32-wasip2 wasm32-wasip1 wasm32-wasi; do
candidate="$source_dir/target/$target_dir/release/${wasm_artifact}.wasm"
if [ -f "$candidate" ]; then
wasm_path="$candidate"
break
fi
done
if [ -z "$wasm_path" ]; then
echo "::warning::No WASM output found for '$name', skipping"
continue
fi
# Copy files with standardized names for the archive
cp "$wasm_path" "target/wasm-bundles/${name}.wasm"
caps_path="$source_dir/$caps_file"
if [ -f "$caps_path" ]; then
cp "$caps_path" "target/wasm-bundles/${name}.capabilities.json"
else
echo "::warning::No capabilities file at '$caps_path' for '$name'"
fi
# Create tar.gz bundle
bundle="target/wasm-bundles/${name}-wasm32-wasip2.tar.gz"
(cd target/wasm-bundles && if [ -f "${name}.capabilities.json" ]; then tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm" "${name}.capabilities.json"; else tar czf "${name}-wasm32-wasip2.tar.gz" "${name}.wasm"; fi)
# Compute SHA256
sha256=$(sha256sum "$bundle" | cut -d' ' -f1)
echo "$sha256 ${name}-wasm32-wasip2.tar.gz" >> target/wasm-bundles/checksums.txt
# Clean up intermediate files
rm -f "target/wasm-bundles/${name}.wasm" "target/wasm-bundles/${name}.capabilities.json"
echo " -> $bundle ($sha256)"
done
echo "=== WASM bundles built ==="
ls -la target/wasm-bundles/
- name: "Upload WASM bundles"
uses: actions/upload-artifact@v4
with:
name: artifacts-wasm-extensions
path: |
target/wasm-bundles/*.tar.gz
target/wasm-bundles/checksums.txt
# Determines if we should publish/announce
host:
needs:
- plan
- build-local-artifacts
- build-global-artifacts
# Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
- build-wasm-extensions
# Only run if we're "publishing", and only if plan, local, global, and wasm didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') && (needs.build-wasm-extensions.result == 'skipped' || needs.build-wasm-extensions.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-22.04"
+2
View File
@@ -19,3 +19,5 @@ jobs:
- uses: Swatinem/rust-cache@v2
- name: Run Tests
run: cargo test --all-features -- --nocapture
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
+37
View File
@@ -7,6 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.10.0](https://github.com/nearai/ironclaw/compare/v0.9.0...v0.10.0) - 2026-02-22
### Added
- update dashboard favicon ([#309](https://github.com/nearai/ironclaw/pull/309))
- add web UI test skill for Chrome extension ([#302](https://github.com/nearai/ironclaw/pull/302))
- implement FullJob routine mode with scheduler dispatch ([#288](https://github.com/nearai/ironclaw/pull/288))
- hot-activate WASM channels, channel-first prompts, unified artifact resolution ([#297](https://github.com/nearai/ironclaw/pull/297))
- add pairing/permission system to all WASM channels and fix extension registry ([#286](https://github.com/nearai/ironclaw/pull/286))
- group chat privacy, channel-aware prompts, and safety hardening ([#285](https://github.com/nearai/ironclaw/pull/285))
- embedded registry catalog and WASM bundle install pipeline ([#283](https://github.com/nearai/ironclaw/pull/283))
- show token usage and cost tracker in gateway status popover ([#284](https://github.com/nearai/ironclaw/pull/284))
- support custom HTTP headers for OpenAI-compatible provider ([#269](https://github.com/nearai/ironclaw/pull/269))
- add smart routing provider for cost-optimized model selection ([#281](https://github.com/nearai/ironclaw/pull/281))
### Fixed
- persist user message at turn start before agentic loop ([#305](https://github.com/nearai/ironclaw/pull/305))
- block send until thread is selected ([#306](https://github.com/nearai/ironclaw/pull/306))
- reload chat history on SSE reconnect ([#307](https://github.com/nearai/ironclaw/pull/307))
- map Esc to interrupt and Ctrl+C to graceful quit ([#267](https://github.com/nearai/ironclaw/pull/267))
### Other
- Fix tool schema OpenAI compatibility ([#301](https://github.com/nearai/ironclaw/pull/301))
- simplify config resolution and consolidate main.rs init ([#287](https://github.com/nearai/ironclaw/pull/287))
- Update image source in README.md
- Add files via upload
- remove ExtensionSource::Bundled, use download-only install for WASM channels ([#293](https://github.com/nearai/ironclaw/pull/293))
- allow OAuth callback to work on remote servers (fixes #186) ([#212](https://github.com/nearai/ironclaw/pull/212))
- add rate limiting for built-in tools (closes #171) ([#276](https://github.com/nearai/ironclaw/pull/276))
- add LLM providers guide (OpenRouter, Together AI, Fireworks, Ollama, vLLM) ([#193](https://github.com/nearai/ironclaw/pull/193))
- Feat/html to markdown #106 ([#115](https://github.com/nearai/ironclaw/pull/115))
- adopt agent-market design language for web UI ([#282](https://github.com/nearai/ironclaw/pull/282))
- speed up startup from ~15s to ~2s ([#280](https://github.com/nearai/ironclaw/pull/280))
- consolidate tool approval into single param-aware method ([#274](https://github.com/nearai/ironclaw/pull/274))
## [0.9.0](https://github.com/nearai/ironclaw/compare/v0.8.0...v0.9.0) - 2026-02-21
### Added
+12 -1
View File
@@ -32,7 +32,7 @@
# Format code
cargo fmt
# Lint (address warnings before committing)
# Lint (fix ALL warnings before committing, including pre-existing ones)
cargo clippy --all --benches --tests --examples --all-features
# Run all tests
@@ -321,7 +321,10 @@ cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
@@ -408,6 +411,10 @@ IronClaw supports multiple LLM backends via the `LLM_BACKEND` env var: `nearai`
**NEAR AI** -- Uses the Chat Completions API with dual auth support. Session token auth (default): authenticates with session tokens (`sess_xxx`) obtained via browser OAuth (GitHub/Google), base URL defaults to `https://private.near.ai`. API key auth: set `NEARAI_API_KEY` (from `cloud.near.ai`), base URL defaults to `https://cloud-api.near.ai`. Both modes use the same Chat Completions endpoint. Tool messages are flattened to plain text for compatibility. Set `NEARAI_SESSION_TOKEN` env var for hosting providers that inject tokens via environment.
**NEAR AI Cloud** -- Uses the OpenAI-compatible Chat Completions API (`https://cloud-api.near.ai/v1/chat/completions`). Authenticates with API keys from `cloud.near.ai`. Auto-selected when `NEARAI_API_KEY` is set (or explicitly via `NEARAI_API_MODE=chat_completions`). Tool messages are flattened to plain text for compatibility. Configure with `NEARAI_API_KEY` and `NEARAI_BASE_URL` (default: `https://cloud-api.near.ai`).
**OpenAI-compatible** -- Any endpoint that speaks the OpenAI API (vLLM, LiteLLM, OpenRouter, etc.). Configure with `LLM_BASE_URL`, `LLM_API_KEY` (optional), `LLM_MODEL`. Set `LLM_EXTRA_HEADERS` to inject custom HTTP headers into every request (format: `Key:Value,Key2:Value2`), useful for OpenRouter attribution headers like `HTTP-Referer` and `X-Title`.
**Tinfoil** -- Private inference via `https://inference.tinfoil.sh/v1`. Runs models inside hardware-attested TEEs so neither Tinfoil nor the cloud provider can see prompts or responses. Uses the OpenAI-compatible Chat Completions API. Configure with `TINFOIL_API_KEY` and `TINFOIL_MODEL` (default: `kimi-k2-5`).
## Database
@@ -565,6 +572,10 @@ Four built-in tools for managing skills at runtime:
- `<workspace>/skills/` -- Per-workspace skills (trusted)
- `~/.ironclaw/installed_skills/` -- Registry-installed skills (installed trust)
### Testing Skills
- `skills/web-ui-test/` -- Manual test checklist for the web gateway UI via Claude for Chrome extension. Covers connection, chat, skills search/install/remove, and other tabs.
Skills configuration: see Configuration section above.
## Docker Sandbox
Generated
+480 -4
View File
@@ -11,6 +11,12 @@ dependencies = [
"gimli",
]
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aead"
version = "0.5.2"
@@ -64,6 +70,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"const-random",
"once_cell",
"version_check",
"zerocopy 0.8.37",
@@ -188,6 +195,15 @@ version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0f477b951e452a0b6b4a10b53ccd569042d1d01729b519e02074a9c0958a063"
[[package]]
name = "astral-tl"
version = "0.7.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d90933ffb0f97e2fc2e0de21da9d3f20597b804012d199843a6fe7c2810d28f3"
dependencies = [
"memchr",
]
[[package]]
name = "async-broadcast"
version = "0.7.2"
@@ -924,6 +940,26 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "const-random"
version = "0.1.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87e00182fe74b066627d63b85fd550ac2998d4b0bd86bfed477a0ae4c7c71359"
dependencies = [
"const-random-macro",
]
[[package]]
name = "const-random-macro"
version = "0.1.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9d839f2a20b0aee515dc581a6172f2321f96cab76c1a38a4c584a194955390e"
dependencies = [
"getrandom 0.2.17",
"once_cell",
"tiny-keccak",
]
[[package]]
name = "constant_time_eq"
version = "0.4.2"
@@ -1099,6 +1135,21 @@ dependencies = [
"target-lexicon",
]
[[package]]
name = "crc"
version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5eb8a2a1cd12ab0d987a5d5e825195d372001a4094a0376319d5a0ad71c1ba0d"
dependencies = [
"crc-catalog",
]
[[package]]
name = "crc-catalog"
version = "2.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "19d374276b40fb8bbdee95aef7c7fa6b5316ec764510eb64b8dd0e2ed0d7e7f5"
[[package]]
name = "crc32fast"
version = "1.5.0"
@@ -1244,6 +1295,12 @@ dependencies = [
"winapi",
]
[[package]]
name = "crunchy"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "460fbee9c2c2f33933d720630a6a0bac33ba7053db5344fac858d4b8952d77d5"
[[package]]
name = "crypto-common"
version = "0.1.7"
@@ -1255,6 +1312,29 @@ dependencies = [
"typenum",
]
[[package]]
name = "cssparser"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dae61cf9c0abb83bd659dab65b7e4e38d8236824c85f0f804f173567bda257d2"
dependencies = [
"cssparser-macros",
"dtoa-short",
"itoa",
"phf 0.13.1",
"smallvec",
]
[[package]]
name = "cssparser-macros"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13b588ba4ac1a99f7f2964d24b3d896ddc6bf847ee3855dbd4366f058cfcd331"
dependencies = [
"quote",
"syn 2.0.114",
]
[[package]]
name = "ctr"
version = "0.9.2"
@@ -1497,12 +1577,33 @@ version = "0.15.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1aaf95b3e5c8f23aa320147307562d361db0ae0d51242340f558153b4eb2439b"
[[package]]
name = "dtoa"
version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c3cf4824e2d5f025c7b531afcb2325364084a16806f6d47fbc1f5fbd9960590"
[[package]]
name = "dtoa-short"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd1511a7b6a56299bd043a9c167a6d2bfb37bf84a6dfceaba651168adfb43c87"
dependencies = [
"dtoa",
]
[[package]]
name = "dyn-clone"
version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0881ea181b1df73ff77ffaaf9c7544ecc11e82fba9b5f27b262a3c73a332555"
[[package]]
name = "ego-tree"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2972feb8dffe7bc8c5463b1dacda1b0dfbed3710e50f977d965429692d74cd8"
[[package]]
name = "either"
version = "1.15.0"
@@ -1680,6 +1781,16 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "flate2"
version = "1.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
]
[[package]]
name = "fnv"
version = "1.0.7"
@@ -1692,6 +1803,12 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "foldhash"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
[[package]]
name = "foreign-types"
version = "0.3.2"
@@ -1743,6 +1860,16 @@ version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6d5a32815ae3f33302d95fdcb2ce17862f8c65363dcfd29360480ba1001fc9c"
[[package]]
name = "futf"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df420e2e84819663797d1ec6544b13c5be84629e7bb00dc960d6917db2987843"
dependencies = [
"mac",
"new_debug_unreachable",
]
[[package]]
name = "futures"
version = "0.3.31"
@@ -1883,6 +2010,15 @@ dependencies = [
"version_check",
]
[[package]]
name = "getopts"
version = "0.2.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfe4fbac503b8d1f88e6676011885f34b7174f46e59956bba534ba83abded4df"
dependencies = [
"unicode-width 0.2.0",
]
[[package]]
name = "getrandom"
version = "0.2.17"
@@ -2001,7 +2137,7 @@ version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash",
"foldhash 0.1.5",
"serde",
]
@@ -2010,6 +2146,11 @@ name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash 0.2.0",
]
[[package]]
name = "hashlink"
@@ -2065,6 +2206,54 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "html-escape"
version = "0.2.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d1ad449764d627e22bfd7cd5e8868264fc9236e07c752972b4080cd351cb476"
dependencies = [
"utf8-width",
]
[[package]]
name = "html-to-markdown-rs"
version = "2.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb31d75f2fdbc8d889d78a912e10c22c30451afb44ee3310f5bfcabf79a31a17"
dependencies = [
"ahash 0.8.12",
"astral-tl",
"base64 0.22.1",
"html-escape",
"html5ever 0.38.0",
"lru",
"once_cell",
"regex",
"serde",
"serde_json",
"thiserror 2.0.18",
]
[[package]]
name = "html5ever"
version = "0.36.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6452c4751a24e1b99c3260d505eaeee76a050573e61f30ac2c924ddc7236f01e"
dependencies = [
"log",
"markup5ever 0.36.1",
]
[[package]]
name = "html5ever"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1054432bae2f14e0061e33d23402fbaa67a921d319d56adc6bcf887ddad1cbc2"
dependencies = [
"log",
"markup5ever 0.38.0",
]
[[package]]
name = "http"
version = "0.2.12"
@@ -2490,7 +2679,7 @@ dependencies = [
[[package]]
name = "ironclaw"
version = "0.9.0"
version = "0.10.0"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -2508,9 +2697,11 @@ dependencies = [
"deadpool-postgres",
"dirs 6.0.0",
"dotenvy",
"flate2",
"fs4",
"futures",
"hkdf",
"html-to-markdown-rs",
"http-body-util",
"hyper 1.8.1",
"hyper-util",
@@ -2521,6 +2712,7 @@ dependencies = [
"postgres-types",
"pretty_assertions",
"rand 0.8.5",
"readabilityrs",
"refinery",
"regex",
"reqwest",
@@ -2536,6 +2728,7 @@ dependencies = [
"serde_yml",
"sha2",
"subtle",
"tar",
"tempfile",
"termimad",
"testcontainers-modules",
@@ -2639,6 +2832,21 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "kuchikikiki"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b73885c6a3cefdf7a1db0327cefbe4b9b72cac94cae4b19ede4fa492d8af02a0"
dependencies = [
"bitflags 2.10.0",
"crc",
"cssparser",
"html5ever 0.38.0",
"indexmap 2.13.0",
"precomputed-hash",
"selectors 0.35.0",
]
[[package]]
name = "lazy-regex"
version = "3.5.1"
@@ -2789,7 +2997,7 @@ dependencies = [
"log",
"memchr",
"phf 0.11.3",
"phf_codegen",
"phf_codegen 0.11.3",
"phf_shared 0.11.3",
"uncased",
]
@@ -2883,12 +3091,27 @@ version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "lru"
version = "0.16.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1dc47f592c06f33f8e3aea9591776ec7c9f9e4124778ff8a3c3b87159f7e593"
dependencies = [
"hashbrown 0.16.1",
]
[[package]]
name = "lru-slab"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]]
name = "mac"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
[[package]]
name = "mach2"
version = "0.4.3"
@@ -2898,6 +3121,28 @@ dependencies = [
"libc",
]
[[package]]
name = "markup5ever"
version = "0.36.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c3294c4d74d0742910f8c7b466f44dda9eb2d5742c1e430138df290a1e8451c"
dependencies = [
"log",
"tendril 0.4.3",
"web_atoms",
]
[[package]]
name = "markup5ever"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8983d30f2915feeaaab2d6babdd6bc7e9ed1a00b66b5e6d74df19aa9c0e91862"
dependencies = [
"log",
"tendril 0.5.0",
"web_atoms",
]
[[package]]
name = "matchers"
version = "0.2.0"
@@ -2990,6 +3235,16 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
[[package]]
name = "miniz_oxide"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fa76a2c86f704bdb222d66965fb3d63269ce38518b83cb0575fca855ebb6316"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.1.1"
@@ -3028,6 +3283,12 @@ dependencies = [
"tempfile",
]
[[package]]
name = "new_debug_unreachable"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "650eef8c711430f1a879fdd01d4745a7deea475becfb90269c06775983bbf086"
[[package]]
name = "nibble_vec"
version = "0.1.0"
@@ -3398,6 +3659,7 @@ version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
dependencies = [
"phf_macros",
"phf_shared 0.13.1",
"serde",
]
@@ -3408,10 +3670,20 @@ version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aef8048c789fa5e851558d709946d6d79a8ff88c0440c587967f8e94bfb1216a"
dependencies = [
"phf_generator",
"phf_generator 0.11.3",
"phf_shared 0.11.3",
]
[[package]]
name = "phf_codegen"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
]
[[package]]
name = "phf_generator"
version = "0.11.3"
@@ -3422,6 +3694,29 @@ dependencies = [
"rand 0.8.5",
]
[[package]]
name = "phf_generator"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
dependencies = [
"fastrand",
"phf_shared 0.13.1",
]
[[package]]
name = "phf_macros"
version = "0.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "812f032b54b1e759ccd5f8b6677695d5268c588701effba24601f6932f8269ef"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "phf_shared"
version = "0.11.3"
@@ -3585,6 +3880,12 @@ dependencies = [
"zerocopy 0.8.37",
]
[[package]]
name = "precomputed-hash"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "925383efa346730478fb4838dbe9137d2a47675ad789c546d150a6e1dd4ab31c"
[[package]]
name = "pretty_assertions"
version = "1.4.1"
@@ -3852,6 +4153,24 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "readabilityrs"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eb174b0af6c181a87d68b42800806657bfbdf88b566f819aaadb9d2a7b7699d"
dependencies = [
"bitflags 2.10.0",
"kuchikikiki",
"once_cell",
"regex",
"scraper",
"serde",
"serde_json",
"thiserror 1.0.69",
"url",
"v_htmlescape",
]
[[package]]
name = "redox_syscall"
version = "0.3.5"
@@ -4394,6 +4713,21 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "scraper"
version = "0.25.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93cecd86d6259499c844440546d02f55f3e17bd286e529e48d1f9f67e92315cb"
dependencies = [
"cssparser",
"ego-tree",
"getopts",
"html5ever 0.36.1",
"precomputed-hash",
"selectors 0.33.0",
"tendril 0.4.3",
]
[[package]]
name = "seahash"
version = "4.1.0"
@@ -4465,6 +4799,44 @@ dependencies = [
"libc",
]
[[package]]
name = "selectors"
version = "0.33.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "feef350c36147532e1b79ea5c1f3791373e61cbd9a6a2615413b3807bb164fb7"
dependencies = [
"bitflags 2.10.0",
"cssparser",
"derive_more",
"log",
"new_debug_unreachable",
"phf 0.13.1",
"phf_codegen 0.13.1",
"precomputed-hash",
"rustc-hash 2.1.1",
"servo_arc",
"smallvec",
]
[[package]]
name = "selectors"
version = "0.35.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93fdfed56cd634f04fe8b9ddf947ae3dc493483e819593d2ba17df9ad05db8b2"
dependencies = [
"bitflags 2.10.0",
"cssparser",
"derive_more",
"log",
"new_debug_unreachable",
"phf 0.13.1",
"phf_codegen 0.13.1",
"precomputed-hash",
"rustc-hash 2.1.1",
"servo_arc",
"smallvec",
]
[[package]]
name = "semver"
version = "1.0.27"
@@ -4618,6 +4990,15 @@ dependencies = [
"version_check",
]
[[package]]
name = "servo_arc"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "170fb83ab34de17dc69aa7c67482b22218ddb85da56546f9bd6b929e32a05930"
dependencies = [
"stable_deref_trait",
]
[[package]]
name = "sha1"
version = "0.10.6"
@@ -4695,6 +5076,12 @@ dependencies = [
"libc",
]
[[package]]
name = "simd-adler32"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e320a6c5ad31d271ad523dcf3ad13e2767ad8b1cb8f047f75a8aeaf8da139da2"
[[package]]
name = "simdutf8"
version = "0.1.5"
@@ -4766,6 +5153,30 @@ version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f42444fea5b87a39db4218d9422087e66a85d0e7a0963a439b07bcdf91804006"
[[package]]
name = "string_cache"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a18596f8c785a729f2819c0f6a7eae6ebeebdfffbfe4214ae6b087f690e31901"
dependencies = [
"new_debug_unreachable",
"parking_lot",
"phf_shared 0.13.1",
"precomputed-hash",
]
[[package]]
name = "string_cache_codegen"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "585635e46db231059f76c5849798146164652513eb9e8ab2685939dd90f29b69"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
"proc-macro2",
"quote",
]
[[package]]
name = "stringprep"
version = "0.1.5"
@@ -4903,6 +5314,17 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tar"
version = "0.4.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d863878d212c87a19c1a610eb53bb01fe12951c0501cf5a0d65f724914a667a"
dependencies = [
"filetime",
"libc",
"xattr",
]
[[package]]
name = "target-lexicon"
version = "0.12.16"
@@ -4922,6 +5344,27 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "tendril"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d24a120c5fc464a3458240ee02c299ebcb9d67b5249c8848b09d639dca8d7bb0"
dependencies = [
"futf",
"mac",
"utf-8",
]
[[package]]
name = "tendril"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4790fc369d5a530f4b544b094e31388b9b3a37c0f4652ade4505945f5660d24"
dependencies = [
"new_debug_unreachable",
"utf-8",
]
[[package]]
name = "termcolor"
version = "1.4.1"
@@ -5065,6 +5508,15 @@ dependencies = [
"time-core",
]
[[package]]
name = "tiny-keccak"
version = "2.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c9d3793400a45f954c52e73d068316d76b6f4e36977e3fcebb13a2721e80237"
dependencies = [
"crunchy",
]
[[package]]
name = "tinystr"
version = "0.8.2"
@@ -5706,6 +6158,12 @@ version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09cc8ee72d2a9becf2f2febe0205bbed8fc6615b7cb429ad062dc7b7ddd036a9"
[[package]]
name = "utf8-width"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1292c0d970b54115d14f2492fe0170adf21d68a1de108eebc51c1df4f346a091"
[[package]]
name = "utf8_iter"
version = "1.0.4"
@@ -5730,6 +6188,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "v_htmlescape"
version = "0.15.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e8257fbc510f0a46eb602c10215901938b5c2a7d5e70fc11483b1d3c9b5b18c"
[[package]]
name = "valuable"
version = "0.1.1"
@@ -6265,6 +6729,18 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web_atoms"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57a9779e9f04d2ac1ce317aee707aa2f6b773afba7b931222bff6983843b1576"
dependencies = [
"phf 0.13.1",
"phf_codegen 0.13.1",
"string_cache",
"string_cache_codegen",
]
[[package]]
name = "which"
version = "4.4.2"
+16 -3
View File
@@ -19,7 +19,7 @@ exclude = [
[package]
name = "ironclaw"
version = "0.9.0"
version = "0.10.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -82,7 +82,7 @@ clap = { version = "4", features = ["derive", "env"] }
# Terminal
crossterm = "0.28"
rustyline = { version = "17", features = ["derive", "with-file-history"] }
rustyline = { version = "17", features = ["custom-bindings", "derive", "with-file-history"] }
termimad = "0.34"
# Channel integrations
@@ -137,6 +137,10 @@ rig-core = "0.30"
# Docker sandbox
bollard = "0.18"
# Archive extraction for WASM extension bundles
flate2 = "1"
tar = "0.4"
# HTTP proxy for sandboxed network access
hyper = { version = "1.5", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "tokio", "http1", "http2"] }
@@ -145,6 +149,10 @@ bytes = "1"
base64 = "0.22.1"
mime_guess = "2.0.5"
# HTML to Markdown conversion (feature gated)
html-to-markdown-rs = { version = "2.3", optional = true }
readabilityrs = { version = "0.1.2", optional = true }
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
security-framework = "3"
@@ -162,7 +170,7 @@ pretty_assertions = "1"
tempfile = "3"
[features]
default = ["postgres", "libsql"]
default = ["postgres", "libsql", "html-to-markdown"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
@@ -173,6 +181,11 @@ postgres = [
]
libsql = ["dep:libsql"]
integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
[[test]]
name = "html_to_markdown"
required-features = ["html-to-markdown"]
# The profile that 'cargo dist' will build with
[profile.dist]
+1 -1
View File
@@ -120,7 +120,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | 🚧 | Caption support for media; no size limits |
| Typing indicators | ✅ | 🚧 | TUI shows status |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
+18 -1
View File
@@ -1,5 +1,5 @@
<p align="center">
<img src="ironclaw.png" alt="IronClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
@@ -143,6 +143,23 @@ and secrets encryption (using your system keychain). Settings are persisted in t
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.ironclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
IronClaw defaults to NEAR AI but works with any OpenAI-compatible endpoint.
Popular options include **OpenRouter** (300+ models), **Together AI**, **Fireworks AI**,
**Ollama** (local), and self-hosted servers like **vLLM** or **LiteLLM**.
Select *"OpenAI-compatible"* in the wizard, or set environment variables directly:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
## Security
IronClaw implements defense in depth to protect your data and prevent misuse.
+92 -1
View File
@@ -10,12 +10,17 @@
//! Prerequisites: rustup target add wasm32-wasip2, cargo install wasm-tools
use std::env;
use std::path::PathBuf;
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() {
let manifest_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
let root = PathBuf::from(&manifest_dir);
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
@@ -104,3 +109,89 @@ fn main() {
}
}
}
/// Collect all registry manifests into a single JSON blob at compile time.
///
/// Output: `$OUT_DIR/embedded_catalog.json` with structure:
/// ```json
/// { "tools": [...], "channels": [...], "bundles": {...} }
/// ```
fn embed_registry_catalog(root: &Path) {
use std::fs;
let registry_dir = root.join("registry");
// Rerun if the bundles file changes (per-file watches for tools/channels
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
// No registry dir: write empty catalog
fs::write(
&out_path,
r#"{"tools":[],"channels":[],"bundles":{"bundles":{}}}"#,
)
.unwrap();
return;
}
let mut tools = Vec::new();
let mut channels = Vec::new();
// Collect tool manifests
let tools_dir = registry_dir.join("tools");
if tools_dir.is_dir() {
collect_json_files(&tools_dir, &mut tools);
}
// Collect channel manifests
let channels_dir = registry_dir.join("channels");
if channels_dir.is_dir() {
collect_json_files(&channels_dir, &mut channels);
}
// Read bundles
let bundles_path = registry_dir.join("_bundles.json");
let bundles_raw = if bundles_path.is_file() {
fs::read_to_string(&bundles_path).unwrap_or_else(|_| r#"{"bundles":{}}"#.to_string())
} else {
r#"{"bundles":{}}"#.to_string()
};
// Build the combined JSON
let catalog = format!(
r#"{{"tools":[{}],"channels":[{}],"bundles":{}}}"#,
tools.join(","),
channels.join(","),
bundles_raw,
);
fs::write(&out_path, catalog).unwrap();
}
/// Read all .json files from a directory and push their raw contents into `out`.
fn collect_json_files(dir: &Path, out: &mut Vec<String>) {
use std::fs;
let mut entries: Vec<_> = fs::read_dir(dir)
.unwrap()
.filter_map(|e| e.ok())
.filter(|e| {
e.path().is_file() && e.path().extension().and_then(|x| x.to_str()) == Some("json")
})
.collect();
// Sort for deterministic output
entries.sort_by_key(|e| e.file_name());
for entry in entries {
// Emit per-file watch so Cargo reruns when file contents change
println!("cargo:rerun-if-changed={}", entry.path().display());
if let Ok(content) = fs::read_to_string(entry.path()) {
out.push(content);
}
}
}
+401
View File
@@ -0,0 +1,401 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "ahash"
version = "0.8.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy",
]
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "discord-channel"
version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7714e70437a7dc3ac8eb7e6f8df75fd8eb422675fc7678aff7364301092b1017"
dependencies = [
"equivalent",
"hashbrown 0.16.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "leb128"
version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "log"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
[[package]]
name = "semver"
version = "1.0.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d767eb0aabc880b29956c35734170f26ed551a859dbd361d140cdbeca61ab1e2"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.149"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "smallvec"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "spdx"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3e17e880bafaeb362a7b751ec46bdc5b61445a188f80e0606e68167cd540fa3"
dependencies = [
"smallvec",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "version_check"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b928f33d975fc6ad9f86c8f283853ad26bdd5b10b7f1542aa2fa15e2289105a"
[[package]]
name = "wasm-encoder"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e913f9242315ca39eff82aee0e19ee7a372155717ff0eb082c741e435ce25ed1"
dependencies = [
"leb128",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "185dfcd27fa5db2e6a23906b54c28199935f71d9a27a1a27b3a88d6fee2afae7"
dependencies = [
"anyhow",
"indexmap",
"serde",
"serde_derive",
"serde_json",
"spdx",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d07b6a3b550fefa1a914b6d54fc175dd11c3392da11eee604e6ffc759805d25"
dependencies = [
"ahash",
"bitflags",
"hashbrown 0.14.5",
"indexmap",
"semver",
]
[[package]]
name = "wit-bindgen"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a2b3e15cd6068f233926e7d8c7c588b2ec4fb7cc7bf3824115e7c7e2a8485a3"
dependencies = [
"wit-bindgen-rt",
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b632a5a0fa2409489bd49c9e6d99fcc61bb3d4ce9d1907d44662e75a28c71172"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rt"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7947d0131c7c9da3f01dfde0ab8bd4c4cf3c5bd49b6dba0ae640f1fa752572ea"
dependencies = [
"bitflags",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4329de4186ee30e2ef30a0533f9b3c123c019a237a7c82d692807bf1b3ee2697"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "177fb7ee1484d113b4792cc480b1ba57664bbc951b42a4beebe573502135b1fc"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b505603761ed400c90ed30261f44a768317348e49f1864e82ecdc3b2744e5627"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.220.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae2a7999ed18efe59be8de2db9cb2b7f84d88b27818c79353dfc53131840fe1a"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zerocopy"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
+1 -1
View File
@@ -9,7 +9,7 @@ publish = false
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.41.0"
wit-bindgen = "0.36"
[lib]
crate-type = ["cdylib"]
+14 -2
View File
@@ -2,6 +2,15 @@
"type": "channel",
"name": "discord",
"description": "Discord Gateway/Webhook channel for handling slash commands, buttons, and messages",
"setup": {
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -10,7 +19,7 @@
"credentials": {
"discord_bot_token": {
"secret_name": "discord_bot_token",
"location": { "type": "header", "header_name": "Authorization", "prefix": "Bot " },
"location": { "type": "header", "name": "Authorization", "prefix": "Bot " },
"host_patterns": ["discord.com"]
}
},
@@ -34,6 +43,9 @@
}
},
"config": {
"require_signature_verification": true
"require_signature_verification": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+226 -16
View File
@@ -124,12 +124,57 @@ struct DiscordMessageMetadata {
thread_id: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "discord";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct DiscordConfig {
#[serde(default)]
#[allow(dead_code)]
require_signature_verification: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
struct DiscordChannel;
impl Guest for DiscordChannel {
fn on_start(_config_json: String) -> Result<ChannelConfig, String> {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: DiscordConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Discord channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Discord".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -169,16 +214,21 @@ impl Guest for DiscordChannel {
// Application Command (slash command)
2 => {
handle_slash_command(&interaction);
json_response(
200,
serde_json::json!({
"type": 5,
"data": {
"content": "🤔 Thinking..."
}
}),
)
if handle_slash_command(&interaction) {
json_response(200, serde_json::json!({"type": 5}))
} else {
// Permission denied — ephemeral response
json_response(
200,
serde_json::json!({
"type": 4,
"data": {
"content": "You are not authorized to use this bot.",
"flags": 64
}
}),
)
}
}
// Message Component (buttons, selects)
@@ -270,7 +320,8 @@ impl Guest for DiscordChannel {
}
}
fn handle_slash_command(interaction: &DiscordInteraction) {
/// Returns true if the message was emitted, false if permission denied.
fn handle_slash_command(interaction: &DiscordInteraction) -> bool {
let user = interaction
.member
.as_ref()
@@ -287,6 +338,22 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
})
.unwrap_or_default();
// DM if no guild member context (only direct user field set)
let is_dm = interaction.member.is_none();
// Permission check
if !check_sender_permission(
&user_id,
Some(&user_name),
is_dm,
Some(&PairingReplyCtx {
application_id: interaction.application_id.clone(),
token: interaction.token.clone(),
}),
) {
return false;
}
let channel_id = interaction.channel_id.clone().unwrap_or_default();
let command_name = interaction
@@ -322,14 +389,13 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
channel_host::LogLevel::Error,
&format!("Failed to serialize metadata: {}", e),
);
// Attempt to notify user of internal error
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
interaction.application_id, interaction.token
);
let payload = serde_json::json!({
"content": "❌ Internal Error: Failed to process command metadata.",
"flags": 64 // Ephemeral
"flags": 64
});
let _ = channel_host::http_request(
"POST",
@@ -338,7 +404,7 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
Some(&serde_json::to_vec(&payload).unwrap_or_default()),
None,
);
return;
return true; // Error, but not a permission denial
}
};
@@ -349,10 +415,10 @@ fn handle_slash_command(interaction: &DiscordInteraction) {
thread_id: None,
metadata_json,
});
true
}
fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordMessage) {
// Check member first (for server contexts), then user (for DMs)
let user = interaction
.member
.as_ref()
@@ -369,6 +435,11 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
})
.unwrap_or_default();
let is_dm = interaction.member.is_none();
if !check_sender_permission(&user_id, Some(&user_name), is_dm, None) {
return;
}
let channel_id = message.channel_id.clone();
let metadata = DiscordMessageMetadata {
@@ -399,6 +470,145 @@ fn handle_message_component(interaction: &DiscordInteraction, message: &DiscordM
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Context needed to send a pairing reply via Discord webhook followup.
struct PairingReplyCtx {
application_id: String,
token: String,
}
/// Check if a sender is permitted to interact with the bot.
/// Returns true if allowed, false if denied (pairing reply sent if applicable).
fn check_sender_permission(
user_id: &str,
username: Option<&str>,
is_dm: bool,
reply_ctx: Option<&PairingReplyCtx>,
) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping interaction from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Guild interactions bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender against allow list
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&user_id.to_string())
|| username.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"username": username,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
if let Some(ctx) = reply_ctx {
let _ = send_pairing_reply(ctx, &result.code);
}
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code as an ephemeral Discord followup message.
fn send_pairing_reply(ctx: &PairingReplyCtx, code: &str) -> Result<(), String> {
let url = format!(
"https://discord.com/api/v10/webhooks/{}/{}",
ctx.application_id, ctx.token
);
let payload = serde_json::json!({
"content": format!(
"To pair with this bot, run: `ironclaw pairing approve discord {}`",
code
),
"flags": 64 // Ephemeral — only visible to the sender
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Discord API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
let headers = serde_json::json!({"Content-Type": "application/json"});
+18 -1
View File
@@ -2,6 +2,20 @@
"type": "channel",
"name": "slack",
"description": "Slack Events API channel for receiving and responding to Slack messages",
"setup": {
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
@@ -33,6 +47,9 @@
}
},
"config": {
"signing_secret_name": "slack_signing_secret"
"signing_secret_name": "slack_signing_secret",
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+167 -6
View File
@@ -104,15 +104,31 @@ struct SlackPostMessageResponse {
ts: Option<String>,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "slack";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct SlackConfig {
/// Name of secret containing signing secret (for verification by host).
/// Parsed from config for forward compatibility; not yet used in WASM
/// (host handles signature verification).
#[serde(default = "default_signing_secret_name")]
#[allow(dead_code)]
signing_secret_name: String,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_signing_secret_name() -> String {
@@ -123,12 +139,30 @@ struct SlackChannel;
impl Guest for SlackChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
// Parse configuration
let _config: SlackConfig = serde_json::from_str(&config_json)
let config: SlackConfig = serde_json::from_str(&config_json)
.map_err(|e| format!("Failed to parse config: {}", e))?;
channel_host::log(channel_host::LogLevel::Info, "Slack channel starting");
// Persist owner_id so subsequent callbacks can read it
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
// Persist dm_policy and allow_from for DM pairing
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
Ok(ChannelConfig {
display_name: "Slack".to_string(),
http_endpoints: vec![HttpEndpointConfig {
@@ -136,7 +170,7 @@ impl Guest for SlackChannel {
methods: vec!["POST".to_string()],
require_secret: true,
}],
poll: None, // Slack uses push via webhooks, no polling needed
poll: None,
})
}
@@ -280,7 +314,7 @@ impl Guest for SlackChannel {
/// Handle a Slack event and emit message if applicable.
fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Option<String>) {
match event.event_type.as_str() {
// Direct mention of the bot
// Direct mention of the bot (always in a channel, not a DM)
"app_mention" => {
if let (Some(user), Some(channel), Some(text), Some(ts)) = (
event.user,
@@ -288,6 +322,10 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
event.text,
event.ts.clone(),
) {
// app_mention is always in a channel (not DM)
if !check_sender_permission(&user, &channel, false) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -307,6 +345,9 @@ fn handle_slack_event(event: SlackEvent, team_id: Option<String>, _event_id: Opt
) {
// Only process DMs (channel IDs starting with D)
if channel.starts_with('D') {
if !check_sender_permission(&user, &channel, true) {
return;
}
emit_message(user, text, channel, event.thread_ts.or(Some(ts)), team_id);
}
}
@@ -358,6 +399,126 @@ fn emit_message(
});
}
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// For pairing mode, sends a pairing code DM if denied.
fn check_sender_permission(user_id: &str, channel_id: &str, is_dm: bool) -> bool {
// 1. Owner check (highest priority, applies to all contexts)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if user_id != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner user {} (owner: {})",
user_id, owner
),
);
return false;
}
return true;
}
// 2. DM policy (only for DMs when no owner_id)
if !is_dm {
return true; // Channel messages bypass DM policy
}
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list: config allow_from + pairing store
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (Slack events only have user ID, not username)
let is_allowed =
allowed.contains(&"*".to_string()) || allowed.contains(&user_id.to_string());
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"user_id": user_id,
"channel_id": channel_id,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, user_id, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for user {}: code {}",
user_id, result.code
),
);
if result.created {
let _ = send_pairing_reply(channel_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via Slack chat.postMessage.
fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
code
),
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({"Content-Type": "application/json"});
let result = channel_host::http_request(
"POST",
"https://slack.com/api/chat.postMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status == 200 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"Slack API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Strip leading bot mention from text.
fn strip_bot_mention(text: &str) -> String {
// Slack mentions look like <@U12345678>
-1
View File
@@ -17,7 +17,6 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# Exclude from parent workspace (this is a standalone WASM component)
[workspace]
[profile.release]
# Optimize for size
+523 -133
View File
@@ -244,6 +244,67 @@ struct TelegramConfig {
struct TelegramChannel;
#[derive(Debug, Clone, PartialEq, Eq)]
enum TelegramStatusAction {
Typing,
Notify(String),
}
const TELEGRAM_STATUS_MAX_CHARS: usize = 600;
fn truncate_status_message(input: &str, max_chars: usize) -> String {
let mut iter = input.chars();
let truncated: String = iter.by_ref().take(max_chars).collect();
if iter.next().is_some() {
format!("{}...", truncated)
} else {
truncated
}
}
fn status_message_for_user(update: &StatusUpdate) -> Option<String> {
let message = update.message.trim();
if message.is_empty() {
None
} else {
Some(truncate_status_message(message, TELEGRAM_STATUS_MAX_CHARS))
}
}
fn get_updates_url(offset: i64, timeout_secs: u32) -> String {
format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout={}&allowed_updates=[\"message\",\"edited_message\"]",
offset, timeout_secs
)
}
fn classify_status_update(update: &StatusUpdate) -> Option<TelegramStatusAction> {
match update.status {
StatusType::Thinking => Some(TelegramStatusAction::Typing),
StatusType::Done | StatusType::Interrupted => None,
// Tool telemetry can be noisy in chat; keep it as typing-only UX.
StatusType::ToolStarted | StatusType::ToolCompleted | StatusType::ToolResult => None,
StatusType::Status => {
let msg = update.message.trim();
if msg.eq_ignore_ascii_case("Done")
|| msg.eq_ignore_ascii_case("Interrupted")
|| msg.eq_ignore_ascii_case("Awaiting approval")
|| msg.eq_ignore_ascii_case("Rejected")
{
None
} else {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
StatusType::ApprovalNeeded
| StatusType::JobStarted
| StatusType::AuthRequired
| StatusType::AuthCompleted => {
status_message_for_user(update).map(TelegramStatusAction::Notify)
}
}
}
impl Guest for TelegramChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> {
channel_host::log(
@@ -422,20 +483,36 @@ impl Guest for TelegramChannel {
&format!("Polling getUpdates with offset {}", offset),
);
// Build getUpdates URL with parameters
// - offset: Identifier of the first update to be returned
// - timeout: Long polling timeout in seconds (Telegram recommends 30+)
// - allowed_updates: Only get message updates
let url = format!(
"https://api.telegram.org/bot{{TELEGRAM_BOT_TOKEN}}/getUpdates?offset={}&timeout=30&allowed_updates=[\"message\",\"edited_message\"]",
offset
);
let headers_json = serde_json::json!({}).to_string();
let primary_url = get_updates_url(offset, 30);
let headers = serde_json::json!({});
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll.
// If the TCP connection drops, retry once immediately with a short poll
// so we don't wait a full extra tick (~30s) before delivering updates.
let result = match channel_host::http_request(
"GET",
&primary_url,
&headers_json,
None,
Some(35_000),
) {
Ok(response) => Ok(response),
Err(primary_err) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"getUpdates request failed ({}), retrying once immediately",
primary_err
),
);
// 35s HTTP timeout outlives Telegram's 30s server-side long-poll
let result =
channel_host::http_request("GET", &url, &headers.to_string(), None, Some(35_000));
let retry_url = get_updates_url(offset, 3);
channel_host::http_request("GET", &retry_url, &headers_json, None, Some(8_000))
.map_err(|retry_err| {
format!("primary error: {}; retry error: {}", primary_err, retry_err)
})
}
};
match result {
Ok(response) => {
@@ -516,7 +593,7 @@ impl Guest for TelegramChannel {
let result = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
Some("Markdown"),
);
@@ -539,7 +616,7 @@ impl Guest for TelegramChannel {
let msg_id = send_message(
metadata.chat_id,
&response.content,
metadata.message_id,
Some(metadata.message_id),
None,
)
.map_err(|e| format!("Plain-text retry also failed: {}", e))?;
@@ -558,10 +635,10 @@ impl Guest for TelegramChannel {
}
fn on_status(update: StatusUpdate) {
// Only send typing indicator for Thinking status
if !matches!(update.status, StatusType::Thinking) {
return;
}
let action = match classify_status_update(&update) {
Some(action) => action,
None => return,
};
// Parse chat_id from metadata
let metadata: TelegramMessageMetadata = match serde_json::from_str(&update.metadata_json) {
@@ -569,40 +646,68 @@ impl Guest for TelegramChannel {
Err(_) => {
channel_host::log(
channel_host::LogLevel::Debug,
"on_status: no valid Telegram metadata, skipping typing indicator",
"on_status: no valid Telegram metadata, skipping status update",
);
return;
}
};
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
match action {
TelegramStatusAction::Typing => {
// POST /sendChatAction with action "typing"
let payload = serde_json::json!({
"chat_id": metadata.chat_id,
"action": "typing"
});
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let payload_bytes = match serde_json::to_vec(&payload) {
Ok(b) => b,
Err(_) => return,
};
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendChatAction",
&headers.to_string(),
Some(&payload_bytes),
None,
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
if let Err(e) = result {
channel_host::log(
channel_host::LogLevel::Debug,
&format!("sendChatAction failed: {}", e),
);
}
}
TelegramStatusAction::Notify(prompt) => {
// Send user-visible status updates for actionable events.
if let Err(first_err) =
send_message(metadata.chat_id, &prompt, Some(metadata.message_id), None)
{
channel_host::log(
channel_host::LogLevel::Warn,
&format!(
"Failed to send status reply ({}), retrying without reply context",
first_err
),
);
if let Err(retry_err) = send_message(metadata.chat_id, &prompt, None, None) {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Failed to send status message without reply context: {}",
retry_err
),
);
}
}
}
}
}
@@ -643,15 +748,18 @@ impl std::fmt::Display for SendError {
fn send_message(
chat_id: i64,
text: &str,
reply_to_message_id: i64,
reply_to_message_id: Option<i64>,
parse_mode: Option<&str>,
) -> Result<i64, SendError> {
let mut payload = serde_json::json!({
"chat_id": chat_id,
"text": text,
"reply_to_message_id": reply_to_message_id,
});
if let Some(message_id) = reply_to_message_id {
payload["reply_to_message_id"] = serde_json::Value::Number(message_id.into());
}
if let Some(mode) = parse_mode {
payload["parse_mode"] = serde_json::Value::String(mode.to_string());
}
@@ -831,40 +939,17 @@ fn register_webhook(tunnel_url: &str, webhook_secret: Option<&str>) -> Result<()
/// Send a pairing code message to a chat. Used when an unknown user DMs the bot.
fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"chat_id": chat_id,
"text": format!(
send_message(
chat_id,
&format!(
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
code
),
"parse_mode": "Markdown",
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize payload: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json"
});
let result = channel_host::http_request(
"POST",
"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage",
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) => {
if response.status != 200 {
let body_str = String::from_utf8_lossy(&response.body);
return Err(format!("HTTP {}: {}", response.status, body_str));
}
Ok(())
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
Some("Markdown"),
)
.map(|_| ())
.map_err(|e| e.to_string())
}
// ============================================================================
@@ -1027,33 +1112,17 @@ fn handle_message(message: TelegramMessage) {
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string());
// Clean the message text (strip bot mentions and commands)
let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH).unwrap_or_default();
let cleaned_text = clean_message_text(
let content_to_emit = match content_to_emit_for_agent(
&content,
if bot_username.is_empty() {
None
} else {
Some(bot_username.as_str())
},
);
// Determine what to emit to the agent.
// - `/start` (no args): emit a welcome placeholder so the agent greets the user
// - Other bare `/commands` (e.g. /interrupt, /help): pass the raw command through
// so Submission::parse() can handle it
// - Commands with args (e.g. `/start hello`): cleaned_text already has the args
// - Plain text: pass through as-is
let trimmed_content = content.trim();
let content_to_emit = if trimmed_content.eq_ignore_ascii_case("/start") {
"[User started the bot]".to_string()
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
// Bare control command like /interrupt, /stop, /help — pass through raw
trimmed_content.to_string()
} else if cleaned_text.is_empty() {
return;
} else {
cleaned_text
) {
Some(value) => value,
None => return,
};
// Emit the message to the agent
@@ -1121,6 +1190,31 @@ fn clean_message_text(text: &str, bot_username: Option<&str>) -> String {
result
}
/// Decide which user content should be emitted to the agent loop.
///
/// - `/start` emits a placeholder so the agent can greet the user
/// - bare slash commands are passed through for Submission parsing
/// - empty/mention-only messages are ignored
/// - otherwise cleaned text is emitted
fn content_to_emit_for_agent(content: &str, bot_username: Option<&str>) -> Option<String> {
let cleaned_text = clean_message_text(content, bot_username);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
return Some("[User started the bot]".to_string());
}
if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
return Some(trimmed_content.to_string());
}
if cleaned_text.is_empty() {
return None;
}
Some(cleaned_text)
}
// ============================================================================
// Utilities
// ============================================================================
@@ -1181,62 +1275,126 @@ mod tests {
// Commands with args: command prefix stripped, args returned
assert_eq!(clean_message_text("/start hello", None), "hello");
assert_eq!(clean_message_text("/help me please", None), "me please");
assert_eq!(clean_message_text("/model claude-opus-4-6", None), "claude-opus-4-6");
assert_eq!(
clean_message_text("/model claude-opus-4-6", None),
"claude-opus-4-6"
);
}
/// Tests for the content_to_emit logic in handle_message.
/// Since handle_message uses WASM host calls, we test the decision logic inline.
/// Since handle_message uses WASM host calls, test the extracted decision function.
#[test]
fn test_content_to_emit_logic() {
// Simulates the content_to_emit decision for various inputs.
// This mirrors the logic in handle_message after clean_message_text.
fn resolve_content(content: &str) -> Option<String> {
let cleaned_text = clean_message_text(content, None);
let trimmed_content = content.trim();
if trimmed_content.eq_ignore_ascii_case("/start") {
Some("[User started the bot]".to_string())
} else if cleaned_text.is_empty() && trimmed_content.starts_with('/') {
Some(trimmed_content.to_string())
} else if cleaned_text.is_empty() {
None // would return/skip in handle_message
} else {
Some(cleaned_text)
}
}
// /start → welcome placeholder
assert_eq!(resolve_content("/start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content("/Start"), Some("[User started the bot]".to_string()));
assert_eq!(resolve_content(" /start "), Some("[User started the bot]".to_string()));
assert_eq!(
content_to_emit_for_agent("/start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent("/Start", None),
Some("[User started the bot]".to_string())
);
assert_eq!(
content_to_emit_for_agent(" /start ", None),
Some("[User started the bot]".to_string())
);
// /start with args → pass args through
assert_eq!(resolve_content("/start hello"), Some("hello".to_string()));
assert_eq!(
content_to_emit_for_agent("/start hello", None),
Some("hello".to_string())
);
// Control commands → pass through raw so Submission::parse() can match
assert_eq!(resolve_content("/interrupt"), Some("/interrupt".to_string()));
assert_eq!(resolve_content("/stop"), Some("/stop".to_string()));
assert_eq!(resolve_content("/help"), Some("/help".to_string()));
assert_eq!(resolve_content("/undo"), Some("/undo".to_string()));
assert_eq!(resolve_content("/redo"), Some("/redo".to_string()));
assert_eq!(resolve_content("/ping"), Some("/ping".to_string()));
assert_eq!(resolve_content("/tools"), Some("/tools".to_string()));
assert_eq!(resolve_content("/compact"), Some("/compact".to_string()));
assert_eq!(resolve_content("/clear"), Some("/clear".to_string()));
assert_eq!(resolve_content("/version"), Some("/version".to_string()));
assert_eq!(
content_to_emit_for_agent("/interrupt", None),
Some("/interrupt".to_string())
);
assert_eq!(
content_to_emit_for_agent("/stop", None),
Some("/stop".to_string())
);
assert_eq!(
content_to_emit_for_agent("/help", None),
Some("/help".to_string())
);
assert_eq!(
content_to_emit_for_agent("/undo", None),
Some("/undo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/redo", None),
Some("/redo".to_string())
);
assert_eq!(
content_to_emit_for_agent("/ping", None),
Some("/ping".to_string())
);
assert_eq!(
content_to_emit_for_agent("/tools", None),
Some("/tools".to_string())
);
assert_eq!(
content_to_emit_for_agent("/compact", None),
Some("/compact".to_string())
);
assert_eq!(
content_to_emit_for_agent("/clear", None),
Some("/clear".to_string())
);
assert_eq!(
content_to_emit_for_agent("/version", None),
Some("/version".to_string())
);
assert_eq!(
content_to_emit_for_agent("/approve", None),
Some("/approve".to_string())
);
assert_eq!(
content_to_emit_for_agent("/always", None),
Some("/always".to_string())
);
assert_eq!(
content_to_emit_for_agent("/deny", None),
Some("/deny".to_string())
);
assert_eq!(
content_to_emit_for_agent("/yes", None),
Some("/yes".to_string())
);
assert_eq!(
content_to_emit_for_agent("/no", None),
Some("/no".to_string())
);
// Commands with args → cleaned text (command stripped)
assert_eq!(resolve_content("/help me please"), Some("me please".to_string()));
assert_eq!(
content_to_emit_for_agent("/help me please", None),
Some("me please".to_string())
);
// Plain text → pass through
assert_eq!(resolve_content("hello world"), Some("hello world".to_string()));
assert_eq!(resolve_content("just text"), Some("just text".to_string()));
assert_eq!(
content_to_emit_for_agent("hello world", None),
Some("hello world".to_string())
);
assert_eq!(
content_to_emit_for_agent("just text", None),
Some("just text".to_string())
);
// Empty / whitespace → skip (None)
assert_eq!(resolve_content(""), None);
assert_eq!(resolve_content(" "), None);
assert_eq!(content_to_emit_for_agent("", None), None);
assert_eq!(content_to_emit_for_agent(" ", None), None);
// Bare @mention without bot → skip
assert_eq!(resolve_content("@botname"), None);
assert_eq!(content_to_emit_for_agent("@botname", None), None);
// With bot username configured: other mentions are preserved.
assert_eq!(
content_to_emit_for_agent("@alice hello", Some("MyBot")),
Some("@alice hello".to_string())
);
}
#[test]
@@ -1317,4 +1475,236 @@ mod tests {
assert_eq!(msg.text, None);
assert_eq!(msg.caption.as_deref(), Some("What's in this image?"));
}
#[test]
fn test_get_updates_url_includes_offset_and_timeout() {
let url = get_updates_url(444_809_884, 30);
assert!(url.contains("offset=444809884"));
assert!(url.contains("timeout=30"));
assert!(url.contains("allowed_updates=[\"message\",\"edited_message\"]"));
}
#[test]
fn test_classify_status_update_thinking() {
let update = StatusUpdate {
status: StatusType::Thinking,
message: "Thinking...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Typing)
);
}
#[test]
fn test_classify_status_update_approval_needed() {
let update = StatusUpdate {
status: StatusType::ApprovalNeeded,
message: "Approval needed for tool 'http_request'".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Approval needed for tool 'http_request'".to_string()
))
);
}
#[test]
fn test_classify_status_update_done_ignored() {
let update = StatusUpdate {
status: StatusType::Done,
message: "Done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_auth_required() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "Authentication required for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication required for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_started_ignored() {
let update = StatusUpdate {
status: StatusType::ToolStarted,
message: "Tool started: http_request".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_tool_completed_ignored() {
let update = StatusUpdate {
status: StatusType::ToolCompleted,
message: "Tool completed: http_request (ok)".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_job_started_notify() {
let update = StatusUpdate {
status: StatusType::JobStarted,
message: "Job started: Daily sync".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Job started: Daily sync".to_string()
))
);
}
#[test]
fn test_classify_status_update_auth_completed_notify() {
let update = StatusUpdate {
status: StatusType::AuthCompleted,
message: "Authentication completed for weather.".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Authentication completed for weather.".to_string()
))
);
}
#[test]
fn test_classify_status_update_tool_result_ignored() {
let update = StatusUpdate {
status: StatusType::ToolResult,
message: "Tool result: http_request ...".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_awaiting_approval_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Awaiting approval".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Interrupted,
message: "Interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_done_ignored_case_insensitive() {
let update = StatusUpdate {
status: StatusType::Status,
message: "done".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_interrupted_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "interrupted".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_rejected_ignored() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Rejected".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(classify_status_update(&update), None);
}
#[test]
fn test_classify_status_update_status_notify() {
let update = StatusUpdate {
status: StatusType::Status,
message: "Context compaction started".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(
classify_status_update(&update),
Some(TelegramStatusAction::Notify(
"Context compaction started".to_string()
))
);
}
#[test]
fn test_status_message_for_user_ignores_blank() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: " ".to_string(),
metadata_json: "{}".to_string(),
};
assert_eq!(status_message_for_user(&update), None);
}
#[test]
fn test_truncate_status_message_appends_ellipsis() {
let input = "abcdefghijklmnopqrstuvwxyz";
let output = truncate_status_message(input, 10);
assert_eq!(output, "abcdefghij...");
}
#[test]
fn test_status_message_for_user_truncates_long_input() {
let update = StatusUpdate {
status: StatusType::AuthRequired,
message: "x".repeat(700),
metadata_json: "{}".to_string(),
};
let msg = status_message_for_user(&update).expect("expected message");
assert!(msg.len() <= TELEGRAM_STATUS_MAX_CHARS + 3);
assert!(msg.ends_with("..."));
}
}
@@ -1 +1,54 @@
{"type":"channel","name":"telegram","description":"Telegram Bot API channel for receiving and responding to Telegram messages","capabilities":{"http":{"allowlist":[{"host":"api.telegram.org","path_prefix":"/bot"}],"credentials":{"telegram_bot":{"secret_name":"telegram_bot_token","location":{"type":"url_path","placeholder":"{TELEGRAM_BOT_TOKEN}"},"host_patterns":["api.telegram.org"]}},"rate_limit":{"requests_per_minute":30,"requests_per_hour":1000}},"secrets":{"allowed_names":["telegram_*"]},"channel":{"allowed_paths":["/webhook/telegram"],"allow_polling":true,"min_poll_interval_ms":30000,"workspace_prefix":"channels/telegram/","emit_rate_limit":{"messages_per_minute":100,"messages_per_hour":5000}}},"config":{"bot_username":null,"owner_id":null,"respond_to_all_group_messages":false,"polling_enabled":false,"poll_interval_ms":30000,"dm_policy":"pairing","allow_from":[]}}
{
"type": "channel",
"name": "telegram",
"description": "Telegram Bot API channel for receiving and responding to Telegram messages",
"setup": {
"required_secrets": [
{
"name": "telegram_bot_token",
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
},
"capabilities": {
"http": {
"allowlist": [
{ "host": "api.telegram.org", "path_prefix": "/bot" }
],
"credentials": {
"telegram_bot": {
"secret_name": "telegram_bot_token",
"location": { "type": "url_path", "placeholder": "{TELEGRAM_BOT_TOKEN}" },
"host_patterns": ["api.telegram.org"]
}
},
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 1000
}
},
"secrets": {
"allowed_names": ["telegram_*"]
},
"channel": {
"allowed_paths": ["/webhook/telegram"],
"allow_polling": true,
"min_poll_interval_ms": 30000,
"workspace_prefix": "channels/telegram/",
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
}
}
},
"config": {
"bot_username": null,
"owner_id": null,
"respond_to_all_group_messages": false,
"polling_enabled": false,
"poll_interval_ms": 30000,
"dm_policy": "pairing",
"allow_from": []
}
}
+191
View File
@@ -226,6 +226,15 @@ struct WhatsAppMessageMetadata {
timestamp: String,
}
/// Workspace path for persisting owner_id across WASM callbacks.
const OWNER_ID_PATH: &str = "state/owner_id";
/// Workspace path for persisting dm_policy across WASM callbacks.
const DM_POLICY_PATH: &str = "state/dm_policy";
/// Workspace path for persisting allow_from (JSON array) across WASM callbacks.
const ALLOW_FROM_PATH: &str = "state/allow_from";
/// Channel name for pairing store (used by pairing host APIs).
const CHANNEL_NAME: &str = "whatsapp";
/// Channel configuration from capabilities file.
#[derive(Debug, Deserialize)]
struct WhatsAppConfig {
@@ -236,6 +245,15 @@ struct WhatsAppConfig {
/// Whether to reply to the original message (thread context)
#[serde(default = "default_reply_to_message")]
reply_to_message: bool,
#[serde(default)]
owner_id: Option<String>,
#[serde(default)]
dm_policy: Option<String>,
#[serde(default)]
allow_from: Option<Vec<String>>,
}
fn default_api_version() -> String {
@@ -264,6 +282,9 @@ impl Guest for WhatsAppChannel {
WhatsAppConfig {
api_version: default_api_version(),
reply_to_message: default_reply_to_message(),
owner_id: None,
dm_policy: None,
allow_from: None,
}
}
};
@@ -279,6 +300,24 @@ impl Guest for WhatsAppChannel {
// Persist api_version in workspace so on_respond() can read it
let _ = channel_host::workspace_write("channels/whatsapp/api_version", &config.api_version);
// Persist permission config for handle_message
if let Some(ref owner_id) = config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
channel_host::log(
channel_host::LogLevel::Info,
&format!("Owner restriction enabled: user {}", owner_id),
);
} else {
let _ = channel_host::workspace_write(OWNER_ID_PATH, "");
}
let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing");
let _ = channel_host::workspace_write(DM_POLICY_PATH, dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
.unwrap_or_else(|_| "[]".to_string());
let _ = channel_host::workspace_write(ALLOW_FROM_PATH, &allow_from_json);
// WhatsApp Cloud API is webhook-only, no polling available
Ok(ChannelConfig {
display_name: "WhatsApp".to_string(),
@@ -604,6 +643,15 @@ fn handle_message(
// Look up sender's name from contacts
let user_name = contact_names.get(&message.from).cloned();
// Permission check (WhatsApp is always DM)
if !check_sender_permission(
&message.from,
user_name.as_deref(),
phone_number_id,
) {
return;
}
// Build metadata for response routing
// This is critical - the response handler uses this to know where to send
let metadata = WhatsAppMessageMetadata {
@@ -637,6 +685,149 @@ fn handle_message(
// Utilities
// ============================================================================
// ============================================================================
// Permission & Pairing
// ============================================================================
/// Check if a sender is permitted. Returns true if allowed.
/// WhatsApp is always 1-to-1 (DM), so dm_policy always applies.
fn check_sender_permission(
sender_phone: &str,
user_name: Option<&str>,
phone_number_id: &str,
) -> bool {
// 1. Owner check (highest priority)
let owner_id = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
if let Some(ref owner) = owner_id {
if sender_phone != owner {
channel_host::log(
channel_host::LogLevel::Debug,
&format!(
"Dropping message from non-owner {} (owner: {})",
sender_phone, owner
),
);
return false;
}
return true;
}
// 2. DM policy (WhatsApp is always DM)
let dm_policy =
channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy == "open" {
return true;
}
// 3. Build merged allow list
let mut allowed: Vec<String> = channel_host::workspace_read(ALLOW_FROM_PATH)
.and_then(|s| serde_json::from_str(&s).ok())
.unwrap_or_default();
if let Ok(store_allowed) = channel_host::pairing_read_allow_from(CHANNEL_NAME) {
allowed.extend(store_allowed);
}
// 4. Check sender (phone number or name)
let is_allowed = allowed.contains(&"*".to_string())
|| allowed.contains(&sender_phone.to_string())
|| user_name.is_some_and(|u| allowed.contains(&u.to_string()));
if is_allowed {
return true;
}
// 5. Not allowed — handle by policy
if dm_policy == "pairing" {
let meta = serde_json::json!({
"phone": sender_phone,
"name": user_name,
})
.to_string();
match channel_host::pairing_upsert_request(CHANNEL_NAME, sender_phone, &meta) {
Ok(result) => {
channel_host::log(
channel_host::LogLevel::Info,
&format!(
"Pairing request for {}: code {}",
sender_phone, result.code
),
);
if result.created {
let _ = send_pairing_reply(sender_phone, phone_number_id, &result.code);
}
}
Err(e) => {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Pairing upsert failed: {}", e),
);
}
}
}
false
}
/// Send a pairing code message via WhatsApp Cloud API.
fn send_pairing_reply(
recipient_phone: &str,
phone_number_id: &str,
code: &str,
) -> Result<(), String> {
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
let url = format!(
"https://graph.facebook.com/{}/{}/messages",
api_version, phone_number_id
);
let payload = serde_json::json!({
"messaging_product": "whatsapp",
"recipient_type": "individual",
"to": recipient_phone,
"type": "text",
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
code
)
}
});
let payload_bytes =
serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize: {}", e))?;
let headers = serde_json::json!({
"Content-Type": "application/json",
"Authorization": "Bearer {WHATSAPP_ACCESS_TOKEN}"
});
let result = channel_host::http_request(
"POST",
&url,
&headers.to_string(),
Some(&payload_bytes),
None,
);
match result {
Ok(response) if response.status >= 200 && response.status < 300 => Ok(()),
Ok(response) => {
let body_str = String::from_utf8_lossy(&response.body);
Err(format!(
"WhatsApp API error: {} - {}",
response.status, body_str
))
}
Err(e) => Err(format!("HTTP request failed: {}", e)),
}
}
/// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default();
@@ -48,6 +48,9 @@
},
"config": {
"api_version": "v18.0",
"reply_to_message": true
"reply_to_message": true,
"owner_id": null,
"dm_policy": "pairing",
"allow_from": []
}
}
+172
View File
@@ -0,0 +1,172 @@
# LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
configurations.
## Provider Overview
| Provider | Backend value | Requires API key | Notes |
|---|---|---|---|
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| Fireworks AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
| vLLM / LiteLLM | `openai_compatible` | Optional | Self-hosted |
| LM Studio | `openai_compatible` | No | Local GUI |
---
## NEAR AI (default)
No additional configuration required. On first run, `ironclaw onboard` opens a browser
for OAuth authentication. Credentials are saved to `~/.ironclaw/session.json`.
```env
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://private.near.ai
```
---
## Anthropic (Claude)
```env
LLM_BACKEND=anthropic
ANTHROPIC_API_KEY=sk-ant-...
```
Popular models: `claude-sonnet-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
---
## OpenAI (GPT)
```env
LLM_BACKEND=openai
OPENAI_API_KEY=sk-...
```
Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
---
## Ollama (local)
Install Ollama from [ollama.com](https://ollama.com), pull a model, then:
```env
LLM_BACKEND=ollama
OLLAMA_MODEL=llama3.2
# OLLAMA_BASE_URL=http://localhost:11434 # default
```
Pull a model first: `ollama pull llama3.2`
---
## OpenAI-Compatible Endpoints
All providers below use `LLM_BACKEND=openai_compatible`. Set `LLM_BASE_URL` to the
provider's OpenAI-compatible endpoint and `LLM_API_KEY` to your API key.
### OpenRouter
[OpenRouter](https://openrouter.ai) routes to 300+ models from a single API key.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://openrouter.ai/api/v1
LLM_API_KEY=sk-or-...
LLM_MODEL=anthropic/claude-sonnet-4
```
Popular OpenRouter model IDs:
| Model | ID |
|---|---|
| Claude Sonnet 4 | `anthropic/claude-sonnet-4` |
| GPT-4o | `openai/gpt-4o` |
| Llama 4 Maverick | `meta-llama/llama-4-maverick` |
| Gemini 2.0 Flash | `google/gemini-2.0-flash-001` |
| Mistral Small | `mistralai/mistral-small-3.1-24b-instruct` |
Browse all models at [openrouter.ai/models](https://openrouter.ai/models).
### Together AI
[Together AI](https://www.together.ai) provides fast inference for open-source models.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.together.xyz/v1
LLM_API_KEY=...
LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
```
Popular Together AI model IDs:
| Model | ID |
|---|---|
| Llama 3.3 70B | `meta-llama/Llama-3.3-70B-Instruct-Turbo` |
| DeepSeek R1 | `deepseek-ai/DeepSeek-R1` |
| Qwen 2.5 72B | `Qwen/Qwen2.5-72B-Instruct-Turbo` |
### Fireworks AI
[Fireworks AI](https://fireworks.ai) offers fast inference with compound AI system support.
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=https://api.fireworks.ai/inference/v1
LLM_API_KEY=fw_...
LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
```
### vLLM / LiteLLM (self-hosted)
For self-hosted inference servers:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:8000/v1
LLM_API_KEY=token-abc123 # set to any string if auth is not configured
LLM_MODEL=meta-llama/Llama-3.1-8B-Instruct
```
LiteLLM proxy (forwards to any backend, including Bedrock, Vertex, Azure):
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:4000/v1
LLM_API_KEY=sk-...
LLM_MODEL=gpt-4o # as configured in litellm config.yaml
```
### LM Studio (local GUI)
Start LM Studio's local server, then:
```env
LLM_BACKEND=openai_compatible
LLM_BASE_URL=http://localhost:1234/v1
LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
# LLM_API_KEY is not required for LM Studio
```
---
## Using the Setup Wizard
Instead of editing `.env` manually, run the onboarding wizard:
```bash
ironclaw onboard
```
Select **"OpenAI-compatible"** for OpenRouter, Together AI, Fireworks, vLLM, LiteLLM,
or LM Studio. You will be prompted for the base URL and (optionally) an API key.
The model name is configured in the following step.
BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.4 MiB

After

Width:  |  Height:  |  Size: 267 KiB

+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": null,
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+106
View File
@@ -0,0 +1,106 @@
---
name: web-ui-test
version: 0.1.0
description: Test the IronClaw web UI using the Claude for Chrome browser extension.
activation:
keywords:
- test web ui
- test the ui
- browser test
- chrome test
- test skills tab
- test chat
- web gateway test
patterns:
- "test.*web.*ui"
- "test.*browser"
- "chrome.*extension.*test"
---
# Web UI Testing with Claude for Chrome
Use this skill when manually testing the IronClaw web gateway UI via the Claude for Chrome browser extension.
## Prerequisites
- IronClaw must be running with `GATEWAY_ENABLED=true`
- Note the gateway URL (default: `http://127.0.0.1:3000/`) and auth token
- The Claude for Chrome extension must be installed and connected
## Starting the Server
```bash
CLI_ENABLED=false GATEWAY_AUTH_TOKEN=<your-token> cargo run
```
Wait for "Agent ironclaw ready and listening" in the logs before proceeding.
## Test Checklist
### 1. Connection
- Navigate to `http://127.0.0.1:3000/?token=<token>`
- Verify "Connected" indicator in the top-right corner
- Verify all tabs are visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### 2. Chat Tab
- Send a simple message (e.g., "Hello, what tools do you have?")
- Verify the LLM responds without errors
- If you see "Invalid schema for function" errors, the tool schema fix (PR #301) may not be merged yet
### 3. Skills Tab
- Click the Skills tab
- Verify "No skills installed" or a list of installed skills (no "Skills system not enabled" error)
- Search for "markdown" in the ClawHub search box
- Verify results appear with: name, version, description, relevance score, "updated X ago"
- Verify skill names are clickable links to clawhub.ai
- If search returns empty with a yellow warning banner, the registry may be unreachable
### 4. Skill Install (from search)
- Search for a skill (e.g., "markdown")
- Click "Install" on a result
- Confirm the install dialog
- Verify success toast appears
- Verify the skill appears in "Installed Skills" section
### 5. Skill Install (by URL)
- Scroll to "Install Skill by URL"
- Enter a skill name and a ClawHub download URL:
- Name: `markdown-viewer`
- URL: `https://wry-manatee-359.convex.site/api/v1/download?slug=markdown-viewer`
- Click Install
- Verify success toast and skill appears in installed list
### 6. Skill Remove
- Find an installed skill
- Click "Remove"
- Confirm removal
- Verify the skill disappears from the installed list
### 7. Other Tabs (smoke test)
- **Memory**: Should show the memory filesystem (may be empty)
- **Jobs**: Should show job list (may be empty)
- **Routines**: Should show routine list
- **Extensions**: Should show extension list with install options
## Cleanup
After testing, remove any test-installed skills:
```bash
rm -rf ~/.ironclaw/installed_skills/<skill-name>
```
Stop the server with Ctrl+C or by killing the process.
## Known Issues
- ClawHub registry at `clawhub.ai` is behind Vercel which blocks non-browser TLS fingerprints; the backend uses `wry-manatee-359.convex.site` directly
- Skill downloads are ZIP archives containing SKILL.md, not raw text
- The `confirm()` dialog for install may block browser automation; override with `window.confirm = () => true` in the console first
+46 -10
View File
@@ -98,7 +98,7 @@ impl Agent {
pub fn new(
config: AgentConfig,
deps: AgentDeps,
channels: ChannelManager,
channels: Arc<ChannelManager>,
heartbeat_config: Option<HeartbeatConfig>,
hygiene_config: Option<crate::config::HygieneConfig>,
routine_config: Option<RoutineConfig>,
@@ -123,7 +123,7 @@ impl Agent {
Self {
config,
deps,
channels: Arc::new(channels),
channels,
context_manager,
scheduler,
router: Router::new(),
@@ -397,6 +397,7 @@ impl Agent {
self.llm().clone(),
Arc::clone(workspace),
notify_tx,
Some(self.scheduler.clone()),
));
// Register routine tools
@@ -499,21 +500,41 @@ impl Agent {
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_content),
}) => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(new_content))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
_ => {
let _ = self
if let Err(e) = self
.channels
.respond(&message, OutgoingResponse::text(response))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %e,
"Failed to send response to channel"
);
}
}
}
}
Ok(Some(_)) => {
Ok(Some(empty)) => {
// Empty response, nothing to send (e.g. approval handled via send_status)
tracing::debug!(
channel = %message.channel,
user = %message.user_id,
empty_len = empty.len(),
"Suppressed empty response (not sent to channel)"
);
}
Ok(None) => {
// Shutdown signal received (/quit, /exit, /shutdown)
@@ -522,10 +543,17 @@ impl Agent {
}
Err(e) => {
tracing::error!("Error handling message: {}", e);
let _ = self
if let Err(send_err) = self
.channels
.respond(&message, OutgoingResponse::text(format!("Error: {}", e)))
.await;
.await
{
tracing::error!(
channel = %message.channel,
error = %send_err,
"Failed to send error response to channel"
);
}
}
}
@@ -682,7 +710,15 @@ impl Agent {
// Convert SubmissionResult to response string
match result? {
SubmissionResult::Response { content } => Ok(Some(content)),
SubmissionResult::Response { content } => {
// Suppress silent replies (e.g. from group chat "nothing to say" responses)
if crate::llm::is_silent_reply(&content) {
tracing::debug!("Suppressing silent reply token");
Ok(None)
} else {
Ok(Some(content))
}
}
SubmissionResult::Ok { message } => Ok(message),
SubmissionResult::Error { message } => Ok(Some(format!("Error: {}", message))),
SubmissionResult::Interrupted => Ok(Some("Interrupted.".into())),
+8 -21
View File
@@ -73,36 +73,23 @@ impl Agent {
description: String,
category: Option<String>,
) -> Result<String, Error> {
// Create job context
let job_id = self
.context_manager
.create_job_for_user(user_id, &title, &description)
.scheduler
.dispatch_job(user_id, &title, &description, None)
.await?;
// Update category if provided
if let Some(cat) = category {
self.context_manager
// Set the dedicated category field (not stored in metadata)
if let Some(cat) = category
&& let Err(e) = self
.context_manager
.update_context(job_id, |ctx| {
ctx.category = Some(cat);
})
.await?;
}
// Persist new job to database (fire-and-forget)
if let Some(store) = self.store()
&& let Ok(ctx) = self.context_manager.get_context(job_id).await
.await
{
let store = store.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job(&ctx).await {
tracing::warn!("Failed to persist new job {}: {}", job_id, e);
}
});
tracing::warn!(job_id = %job_id, "Failed to set job category: {}", e);
}
// Schedule for execution
self.scheduler.schedule(job_id).await?;
Ok(format!(
"Created job: {}\nID: {}\n\nThe job has been scheduled and is now running.",
title, job_id
+75 -9
View File
@@ -4,7 +4,7 @@
//! to prevent runaway agents from burning through API credits. Especially
//! important for daemon/heartbeat modes where the agent acts autonomously.
use std::collections::VecDeque;
use std::collections::{HashMap, VecDeque};
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Instant;
@@ -53,6 +53,14 @@ impl std::fmt::Display for CostLimitExceeded {
}
}
/// Per-model token usage counters.
#[derive(Debug, Clone, Default)]
pub struct ModelTokens {
pub input_tokens: u64,
pub output_tokens: u64,
pub cost: Decimal,
}
/// Tracks costs and action rates, enforcing configurable limits.
///
/// Thread-safe; designed to be shared via `Arc<CostGuard>`.
@@ -67,6 +75,9 @@ pub struct CostGuard {
/// Flag set when daily budget is exceeded to short-circuit checks.
budget_exceeded: AtomicBool,
/// Per-model token usage since startup.
model_tokens: Mutex<HashMap<String, ModelTokens>>,
}
struct DailyCost {
@@ -85,6 +96,7 @@ impl CostGuard {
}),
action_window: Mutex::new(VecDeque::new()),
budget_exceeded: AtomicBool::new(false),
model_tokens: Mutex::new(HashMap::new()),
}
}
@@ -139,14 +151,19 @@ impl CostGuard {
/// Record a completed LLM action: its token costs and the action timestamp.
///
/// Call this AFTER an LLM call completes so that costs are tracked.
///
/// When `cost_per_token` is `Some`, those rates are used directly (provider-
/// sourced pricing). When `None`, falls back to the static `costs::model_cost`
/// lookup table, then `costs::default_cost`.
pub async fn record_llm_call(
&self,
model: &str,
input_tokens: u32,
output_tokens: u32,
cost_per_token: Option<(Decimal, Decimal)>,
) -> Decimal {
let (input_rate, output_rate) =
costs::model_cost(model).unwrap_or_else(costs::default_cost);
let (input_rate, output_rate) = cost_per_token
.unwrap_or_else(|| costs::model_cost(model).unwrap_or_else(costs::default_cost));
let cost =
input_rate * Decimal::from(input_tokens) + output_rate * Decimal::from(output_tokens);
@@ -192,6 +209,15 @@ impl CostGuard {
window.push_back(Instant::now());
}
// Track per-model token usage
{
let mut tokens = self.model_tokens.lock().await;
let entry = tokens.entry(model.to_string()).or_default();
entry.input_tokens += u64::from(input_tokens);
entry.output_tokens += u64::from(output_tokens);
entry.cost += cost;
}
cost
}
@@ -215,6 +241,11 @@ impl CostGuard {
}
window.len() as u64
}
/// Per-model token usage since startup.
pub async fn model_usage(&self) -> HashMap<String, ModelTokens> {
self.model_tokens.lock().await.clone()
}
}
/// Convert a Decimal USD amount to whole cents (truncated).
@@ -235,7 +266,9 @@ mod tests {
assert!(guard.check_allowed().await.is_ok());
// Record a big call, still allowed
guard.record_llm_call("gpt-4o", 100_000, 100_000).await;
guard
.record_llm_call("gpt-4o", 100_000, 100_000, None)
.await;
assert!(guard.check_allowed().await.is_ok());
}
@@ -252,7 +285,7 @@ mod tests {
// Record a call that costs more than $0.01
// gpt-4o: input=$0.0000025/tok, output=$0.00001/tok
// 10000 input + 10000 output = $0.025 + $0.10 = $0.125
guard.record_llm_call("gpt-4o", 10_000, 10_000).await;
guard.record_llm_call("gpt-4o", 10_000, 10_000, None).await;
// Now should be blocked
let result = guard.check_allowed().await;
@@ -275,7 +308,7 @@ mod tests {
// First 3 actions allowed
for _ in 0..3 {
assert!(guard.check_allowed().await.is_ok());
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
}
// 4th should be blocked
@@ -296,7 +329,7 @@ mod tests {
assert_eq!(guard.daily_spend().await, Decimal::ZERO);
let cost = guard.record_llm_call("gpt-4o", 1000, 500).await;
let cost = guard.record_llm_call("gpt-4o", 1000, 500, None).await;
assert!(cost > Decimal::ZERO);
assert_eq!(guard.daily_spend().await, cost);
}
@@ -307,8 +340,8 @@ mod tests {
assert_eq!(guard.actions_this_hour().await, 0);
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
guard.record_llm_call("gpt-4o", 10, 10, None).await;
assert_eq!(guard.actions_this_hour().await, 2);
}
@@ -336,4 +369,37 @@ mod tests {
assert!(rate.to_string().contains("101 actions"));
assert!(rate.to_string().contains("100 allowed"));
}
#[tokio::test]
async fn test_model_usage_per_model_tracking() {
let guard = CostGuard::new(CostGuardConfig::default());
// Initially empty
assert!(guard.model_usage().await.is_empty());
// Record calls for two different models
guard.record_llm_call("gpt-4o", 1000, 500, None).await;
guard.record_llm_call("gpt-4o", 2000, 1000, None).await;
guard
.record_llm_call("claude-3-5-sonnet-20241022", 500, 200, None)
.await;
let usage = guard.model_usage().await;
assert_eq!(usage.len(), 2);
let gpt = usage.get("gpt-4o").expect("gpt-4o should be tracked");
assert_eq!(gpt.input_tokens, 3000);
assert_eq!(gpt.output_tokens, 1500);
assert!(gpt.cost > Decimal::ZERO);
let claude = usage
.get("claude-3-5-sonnet-20241022")
.expect("claude should be tracked");
assert_eq!(claude.input_tokens, 500);
assert_eq!(claude.output_tokens, 200);
assert!(claude.cost > Decimal::ZERO);
// Costs should differ since models have different pricing
assert_ne!(gpt.cost, claude.cost);
}
}
+41 -32
View File
@@ -40,9 +40,17 @@ impl Agent {
thread_id: Uuid,
initial_messages: Vec<ChatMessage>,
) -> Result<AgenticLoopResult, Error> {
// Detect group chat from channel metadata (needed before loading system prompt)
let is_group_chat = message
.metadata
.get("chat_type")
.and_then(|v| v.as_str())
.is_some_and(|t| t == "group" || t == "channel" || t == "supergroup");
// Load workspace system prompt (identity files: AGENTS.md, SOUL.md, etc.)
// In group chats, MEMORY.md is excluded to prevent leaking personal context.
let system_prompt = if let Some(ws) = self.workspace() {
match ws.system_prompt().await {
match ws.system_prompt_for_context(is_group_chat).await {
Ok(prompt) if !prompt.is_empty() => Some(prompt),
Ok(_) => None,
Err(e) => {
@@ -94,7 +102,10 @@ impl Agent {
None
};
let mut reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
let mut reasoning = Reasoning::new(self.llm().clone(), self.safety().clone())
.with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat);
if let Some(prompt) = system_prompt {
reasoning = reasoning.with_system_prompt(prompt);
}
@@ -211,6 +222,7 @@ impl Agent {
&model_name,
output.usage.input_tokens,
output.usage.output_tokens,
Some(self.llm().cost_per_token()),
)
.await;
tracing::debug!(
@@ -284,32 +296,8 @@ impl Agent {
for (idx, original_tc) in tool_calls.iter().enumerate() {
let mut tc = original_tc.clone();
// Check if tool requires approval (skipped when auto_approve_tools is set)
if !self.config.auto_approve_tools
&& let Some(tool) = self.tools().get(&tc.name).await
&& tool.requires_approval()
{
let mut is_auto_approved = {
let sess = session.lock().await;
sess.is_tool_auto_approved(&tc.name)
};
// Override auto-approval for destructive parameters
if is_auto_approved && tool.requires_approval_for(&tc.arguments) {
tracing::info!(
tool = %tc.name,
"Parameters require explicit approval despite auto-approve"
);
is_auto_approved = false;
}
if !is_auto_approved {
approval_needed = Some((idx, tc, tool));
break; // remaining tools are deferred
}
}
// Hook: BeforeToolCall
// Hook: BeforeToolCall (runs before approval so hooks can
// modify parameters — approval is checked on final params)
let event = crate::hooks::HookEvent::ToolCall {
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
@@ -352,6 +340,27 @@ impl Agent {
_ => {}
}
// Check if tool requires approval on the final (post-hook)
// parameters. Skipped when auto_approve_tools is set.
if !self.config.auto_approve_tools
&& let Some(tool) = self.tools().get(&tc.name).await
{
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => {
let sess = session.lock().await;
!sess.is_tool_auto_approved(&tc.name)
}
ApprovalRequirement::Always => true,
};
if needs_approval {
approval_needed = Some((idx, tc, tool));
break; // remaining tools are deferred
}
}
let preflight_idx = preflight.len();
preflight.push((tc.clone(), PreflightOutcome::Runnable));
runnable.push((preflight_idx, tc));
@@ -882,7 +891,7 @@ mod tests {
auto_approve_tools: false,
},
deps,
ChannelManager::new(),
Arc::new(ChannelManager::new()),
None,
None,
None,
@@ -910,9 +919,9 @@ mod tests {
}
#[test]
fn test_shell_destructive_command_requires_approval_for() {
// ShellTool::requires_approval_for should detect destructive commands.
// This exercises the same code path used inline in run_agentic_loop.
fn test_shell_destructive_command_requires_explicit_approval() {
// requires_explicit_approval() detects destructive commands that
// should return ApprovalRequirement::Always from ShellTool.
use crate::tools::builtin::shell::requires_explicit_approval;
let destructive_cmds = [
+64 -26
View File
@@ -19,6 +19,7 @@ use regex::Regex;
use tokio::sync::{RwLock, mpsc};
use uuid::Uuid;
use crate::agent::Scheduler;
use crate::agent::routine::{
NotifyConfig, Routine, RoutineAction, RoutineRun, RunStatus, Trigger, next_cron_fire,
};
@@ -41,6 +42,8 @@ pub struct RoutineEngine {
running_count: Arc<AtomicUsize>,
/// Compiled event regex cache: routine_id -> compiled regex.
event_cache: Arc<RwLock<Vec<(Uuid, Routine, Regex)>>>,
/// Scheduler for dispatching jobs (FullJob mode).
scheduler: Option<Arc<Scheduler>>,
}
impl RoutineEngine {
@@ -50,6 +53,7 @@ impl RoutineEngine {
llm: Arc<dyn LlmProvider>,
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
scheduler: Option<Arc<Scheduler>>,
) -> Self {
Self {
config,
@@ -59,6 +63,7 @@ impl RoutineEngine {
notify_tx,
running_count: Arc::new(AtomicUsize::new(0)),
event_cache: Arc::new(RwLock::new(Vec::new())),
scheduler,
}
}
@@ -225,7 +230,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
tokio::spawn(async move {
@@ -257,7 +262,7 @@ impl RoutineEngine {
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
max_lightweight_tokens: self.config.max_lightweight_tokens,
scheduler: self.scheduler.clone(),
};
// Record the run in DB, then spawn execution
@@ -304,7 +309,7 @@ struct EngineContext {
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
running_count: Arc<AtomicUsize>,
max_lightweight_tokens: u32,
scheduler: Option<Arc<Scheduler>>,
}
/// Execute a routine run. Handles both lightweight and full_job modes.
@@ -318,29 +323,11 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
context_paths,
max_tokens,
} => execute_lightweight(&ctx, &routine, prompt, context_paths, *max_tokens).await,
RoutineAction::FullJob { description, .. } => {
// Full job mode: scheduler integration not yet implemented.
// Execute as lightweight and prepend a warning to the summary.
tracing::warn!(
routine = %routine.name,
"FullJob mode not yet implemented; falling back to lightweight execution"
);
match execute_lightweight(&ctx, &routine, description, &[], ctx.max_lightweight_tokens)
.await
{
Ok((status, summary, tokens)) => {
let warning = "[Note: FullJob mode is not yet implemented. This routine ran as \
a single LLM call without tool access. Configure as 'lightweight' \
or wait for full scheduler integration.]";
let summary = match summary {
Some(s) => Some(format!("{warning}\n\n{s}")),
None => Some(warning.to_string()),
};
Ok((status, summary, tokens))
}
Err(e) => Err(e),
}
}
RoutineAction::FullJob {
title,
description,
max_iterations,
} => execute_full_job(&ctx, &routine, &run, title, description, *max_iterations).await,
};
// Decrement running count
@@ -418,6 +405,57 @@ fn sanitize_routine_name(name: &str) -> String {
.collect()
}
/// Execute a full-job routine by dispatching to the scheduler.
///
/// Fire-and-forget: creates a job via `Scheduler::dispatch_job` (which handles
/// creation, metadata, persistence, and scheduling), links the routine run to
/// the job, and returns immediately. The job runs independently via the
/// existing Worker/Scheduler with full tool access.
async fn execute_full_job(
ctx: &EngineContext,
routine: &Routine,
run: &RoutineRun,
title: &str,
description: &str,
max_iterations: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let scheduler = ctx
.scheduler
.as_ref()
.ok_or_else(|| RoutineError::JobDispatchFailed {
reason: "scheduler not available".to_string(),
})?;
let metadata = serde_json::json!({ "max_iterations": max_iterations });
let job_id = scheduler
.dispatch_job(&routine.user_id, title, description, Some(metadata))
.await
.map_err(|e| RoutineError::JobDispatchFailed {
reason: format!("failed to dispatch job: {e}"),
})?;
// Link the routine run to the dispatched job
if let Err(e) = ctx.store.link_routine_run_to_job(run.id, job_id).await {
tracing::error!(
routine = %routine.name,
"Failed to link run to job: {}", e
);
}
tracing::info!(
routine = %routine.name,
job_id = %job_id,
max_iterations = max_iterations,
"Dispatched full job for routine"
);
let summary = format!(
"Dispatched job {job_id} for full execution with tool access (max_iterations: {max_iterations})"
);
Ok((RunStatus::Ok, Some(summary), None))
}
/// Execute a lightweight routine (single LLM call).
async fn execute_lightweight(
ctx: &EngineContext,
+45 -1
View File
@@ -81,6 +81,50 @@ impl Scheduler {
}
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
/// 1. Creates the job context via `ContextManager`
/// 2. Optionally applies metadata (e.g. `max_iterations`)
/// 3. Persists the job to the database (so FK references from
/// `job_actions` / `llm_calls` work immediately)
/// 4. Schedules the job for worker execution
///
/// Returns the new job ID.
pub async fn dispatch_job(
&self,
user_id: &str,
title: &str,
description: &str,
metadata: Option<serde_json::Value>,
) -> Result<Uuid, JobError> {
let job_id = self
.context_manager
.create_job_for_user(user_id, title, description)
.await?;
// Apply metadata if provided
if let Some(meta) = metadata {
self.context_manager
.update_context(job_id, |ctx| {
ctx.metadata = meta;
})
.await?;
}
// Persist to DB before scheduling so the worker's FK references are valid
if let Some(ref store) = self.store {
let ctx = self.context_manager.get_context(job_id).await?;
store.save_job(&ctx).await.map_err(|e| JobError::Failed {
id: job_id,
reason: format!("failed to persist job: {e}"),
})?;
}
self.schedule(job_id).await?;
Ok(job_id)
}
/// Schedule a job for execution.
pub async fn schedule(&self, job_id: Uuid) -> Result<(), JobError> {
// Hold write lock for the entire check-insert sequence to prevent
@@ -357,7 +401,7 @@ impl Scheduler {
.into());
}
if tool.requires_approval() {
if tool.requires_approval(&params).is_required() {
return Err(crate::error::ToolError::AuthRequired {
name: tool_name.to_string(),
}
+58 -45
View File
@@ -252,6 +252,10 @@ impl Agent {
thread.messages()
};
// Persist user message to DB immediately so it survives crashes
self.persist_user_message(thread_id, &message.user_id, content)
.await;
// Send thinking status
let _ = self
.channels
@@ -320,9 +324,8 @@ impl Agent {
)
.await;
// Persist turn to DB before returning so the write
// completes even if the process shuts down right after.
self.persist_turn(thread_id, &message.user_id, content, Some(&response))
// Persist assistant response (user message already persisted at turn start)
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
Ok(SubmissionResult::response(response))
@@ -351,23 +354,21 @@ impl Agent {
}
Err(e) => {
thread.fail_turn(e.to_string());
// Persist the user message even on failure
self.persist_turn(thread_id, &message.user_id, content, None)
.await;
// User message already persisted at turn start; nothing else to save
Ok(SubmissionResult::error(e.to_string()))
}
}
}
/// Persist a turn (user message + optional assistant response) to the DB.
pub(super) async fn persist_turn(
/// Persist the user message to the DB at turn start (before the agentic loop).
///
/// This ensures the user message is durable even if the process crashes
/// mid-response. Call this right after `thread.start_turn()`.
pub(super) async fn persist_user_message(
&self,
thread_id: Uuid,
user_id: &str,
user_input: &str,
response: Option<&str>,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
@@ -387,13 +388,36 @@ impl Agent {
.await
{
tracing::warn!("Failed to persist user message: {}", e);
}
}
/// Persist the assistant response to the DB after the agentic loop completes.
///
/// Re-ensures the conversation row exists so that assistant responses are
/// still persisted even if `persist_user_message` failed transiently at
/// turn start (e.g. a brief DB blip that resolved before response time).
pub(super) async fn persist_assistant_response(
&self,
thread_id: Uuid,
user_id: &str,
response: &str,
) {
let store = match self.store() {
Some(s) => Arc::clone(s),
None => return,
};
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", user_id, None)
.await
{
tracing::warn!("Failed to ensure conversation {}: {}", thread_id, e);
return;
}
if let Some(resp) = response
&& let Err(e) = store
.add_conversation_message(thread_id, "assistant", resp)
.await
if let Err(e) = store
.add_conversation_message(thread_id, "assistant", response)
.await
{
tracing::warn!("Failed to persist assistant message: {}", e);
}
@@ -746,19 +770,18 @@ impl Agent {
)> = None;
for (idx, tc) in deferred_tool_calls.iter().enumerate() {
if let Some(tool) = self.tools().get(&tc.name).await
&& tool.requires_approval()
{
let is_auto_approved = {
let sess = session.lock().await;
let mut approved = sess.is_tool_auto_approved(&tc.name);
if approved && tool.requires_approval_for(&tc.arguments) {
approved = false;
if let Some(tool) = self.tools().get(&tc.name).await {
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => {
let sess = session.lock().await;
!sess.is_tool_auto_approved(&tc.name)
}
approved
ApprovalRequirement::Always => true,
};
if !is_auto_approved {
if needs_approval {
approval_needed = Some((idx, tc.clone(), tool));
break; // remaining tools stay deferred
}
@@ -1016,12 +1039,10 @@ impl Agent {
match result {
Ok(AgenticLoopResult::Response(response)) => {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.complete_turn(&response);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&response))
.await;
}
// User message already persisted at turn start; save assistant response
self.persist_assistant_response(thread_id, &message.user_id, &response)
.await;
let _ = self
.channels
.send_status(
@@ -1056,12 +1077,8 @@ impl Agent {
})
}
Err(e) => {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.fail_turn(e.to_string());
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, None)
.await;
}
// User message already persisted at turn start
Ok(SubmissionResult::error(e.to_string()))
}
}
@@ -1075,13 +1092,11 @@ impl Agent {
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.clear_pending_approval();
thread.complete_turn(&rejection);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&rejection))
.await;
}
// User message already persisted at turn start; save rejection response
self.persist_assistant_response(thread_id, &message.user_id, &rejection)
.await;
}
}
@@ -1116,13 +1131,11 @@ impl Agent {
{
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) {
let user_input = thread.last_turn().map(|t| t.user_input.clone());
thread.enter_auth_mode(ext_name.clone());
thread.complete_turn(&instructions);
if let Some(input) = user_input {
self.persist_turn(thread_id, &message.user_id, &input, Some(&instructions))
.await;
}
// User message already persisted at turn start; save auth instructions
self.persist_assistant_response(thread_id, &message.user_id, &instructions)
.await;
}
}
let _ = self
+112 -3
View File
@@ -18,6 +18,7 @@ use crate::llm::{
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::rate_limiter::RateLimitResult;
/// Shared dependencies for worker execution.
///
@@ -97,6 +98,20 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
}
/// Run the worker until the job is complete or stopped.
pub async fn run(self, mut rx: mpsc::Receiver<WorkerMessage>) -> Result<(), Error> {
tracing::info!("Worker starting for job {}", self.job_id);
@@ -163,7 +178,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reasoning: &Reasoning,
reason_ctx: &mut ReasoningContext,
) -> Result<(), Error> {
let max_iterations = 50;
const MAX_WORKER_ITERATIONS: usize = 500;
let max_iterations = self
.context_manager()
.get_context(self.job_id)
.await
.ok()
.and_then(|ctx| ctx.metadata.get("max_iterations").and_then(|v| v.as_u64()))
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
// Initial tool definitions for planning (will be refreshed in loop)
@@ -192,6 +215,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.join("\n")
)));
self.log_event("message", serde_json::json!({
"role": "assistant",
"content": format!("Plan: {}\n\n{}", p.goal,
p.actions.iter().enumerate()
.map(|(i, a)| format!("{}. {} - {}", i + 1, a.tool_name, a.reasoning))
.collect::<Vec<_>>().join("\n"))
}));
Some(p)
}
Err(e) => {
@@ -266,6 +297,14 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Add assistant response to context
reason_ctx.messages.push(ChatMessage::assistant(&response));
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": response,
}),
);
// Give it one more chance to select a tool
if iteration > 3 && iteration % 5 == 0 {
reason_ctx.messages.push(ChatMessage::user(
@@ -284,6 +323,16 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_calls.len()
);
if let Some(ref text) = content {
self.log_event(
"message",
serde_json::json!({
"role": "assistant",
"content": text,
}),
);
}
// Add assistant message with tool_calls (OpenAI protocol)
reason_ctx
.messages
@@ -432,16 +481,31 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
})?;
// Tools requiring approval are blocked in autonomous jobs
if tool.requires_approval() {
if tool.requires_approval(params).is_required() {
return Err(crate::error::ToolError::AuthRequired {
name: tool_name.to_string(),
}
.into());
}
// Fetch job context early so we have the real user_id for hooks
// Fetch job context early so we have the real user_id for hooks and rate limiting
let job_ctx = deps.context_manager.get_context(job_id).await?;
// Check per-tool rate limit before running hooks or executing (cheaper check first)
if let Some(config) = tool.rate_limit_config()
&& let RateLimitResult::Limited { retry_after, .. } = deps
.tools
.rate_limiter()
.check_and_record(&job_ctx.user_id, tool_name, &config)
.await
{
return Err(crate::error::ToolError::RateLimited {
name: tool_name.to_string(),
retry_after: Some(retry_after),
}
.into());
}
// Run BeforeToolCall hook
let params = {
use crate::hooks::{HookError, HookEvent, HookOutcome};
@@ -651,6 +715,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
selection: &ToolSelection,
result: Result<String, Error>,
) -> Result<bool, Error> {
self.log_event(
"tool_use",
serde_json::json!({
"tool_name": selection.tool_name,
"input": crate::agent::agent_loop::truncate_for_preview(
&selection.parameters.to_string(), 500),
}),
);
match result {
Ok(output) => {
// Sanitize output
@@ -671,6 +744,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
wrapped,
));
self.log_event("tool_result", serde_json::json!({
"tool_name": selection.tool_name,
"success": true,
"output": crate::agent::agent_loop::truncate_for_preview(&sanitized.content, 500),
}));
// Tool output never drives job completion. A malicious tool could
// emit "TASK_COMPLETE" to force premature completion. Only the LLM's
// own structured response (in execution_loop) can mark a job done.
@@ -697,6 +776,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
});
}
self.log_event(
"tool_result",
serde_json::json!({
"tool_name": selection.tool_name,
"success": false,
"output": format!("Error: {}", e),
}),
);
reason_ctx.messages.push(ChatMessage::tool_result(
&selection.tool_call_id,
&selection.tool_name,
@@ -818,6 +906,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": true,
"message": "Job completed successfully",
}),
);
self.persist_status(
JobState::Completed,
Some("Job completed successfully".to_string()),
@@ -836,6 +931,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Execution failed: {}", reason),
}),
);
self.persist_status(JobState::Failed, Some(reason.to_string()));
Ok(())
}
@@ -849,6 +951,13 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
reason: s,
})?;
self.log_event(
"result",
serde_json::json!({
"success": false,
"message": format!("Job stuck: {}", reason),
}),
);
self.persist_status(JobState::Stuck, Some(reason.to_string()));
Ok(())
}
+118 -153
View File
@@ -22,6 +22,7 @@ use crate::skills::SkillRegistry;
use crate::skills::catalog::SkillCatalog;
use crate::tools::ToolRegistry;
use crate::tools::mcp::McpSessionManager;
use crate::tools::wasm::SharedCredentialRegistry;
use crate::tools::wasm::WasmToolRuntime;
use crate::workspace::{EmbeddingProvider, Workspace};
@@ -48,6 +49,8 @@ pub struct AppComponents {
pub skill_catalog: Option<Arc<SkillCatalog>>,
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
pub session: Arc<SessionManager>,
pub catalog_entries: Vec<crate::extensions::RegistryEntry>,
pub dev_loaded_tool_names: Vec<String>,
}
/// Options that control optional init phases.
@@ -200,9 +203,13 @@ impl AppBuilder {
self.session.attach_store(db.clone(), "default").await;
if let Err(e) = db.cleanup_stale_sandbox_jobs().await {
tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e);
}
// Fire-and-forget housekeeping — no need to block startup.
let db_cleanup = db.clone();
tokio::spawn(async move {
if let Err(e) = db_cleanup.cleanup_stale_sandbox_jobs().await {
tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e);
}
});
self.db = Some(db);
Ok(())
@@ -285,94 +292,14 @@ impl AppBuilder {
/// Phase 3: Initialize LLM provider chain.
///
/// Creates the primary provider, then wraps with failover, circuit
/// breaker, and response cache as configured.
/// Delegates to `build_provider_chain` which applies all decorators
/// (retry, smart routing, failover, circuit breaker, response cache).
#[allow(clippy::type_complexity)]
pub fn init_llm(
&self,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), anyhow::Error> {
use crate::llm::{
CachedProvider, CircuitBreakerConfig, CircuitBreakerProvider, CooldownConfig,
FailoverProvider, ResponseCacheConfig, create_cheap_llm_provider, create_llm_provider,
create_llm_provider_with_config,
};
let llm = create_llm_provider(&self.config.llm, self.session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
// Wrap in failover if a fallback model is configured
let llm: Arc<dyn LlmProvider> = if let Some(fallback_model) =
self.config.llm.nearai.fallback_model.as_ref()
{
if fallback_model == &self.config.llm.nearai.model {
tracing::warn!(
"fallback_model is the same as primary model, failover may not be effective"
);
}
let mut fallback_config = self.config.llm.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(&fallback_config, self.session.clone())?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
"LLM failover enabled"
);
let cooldown_config = CooldownConfig {
cooldown_duration: std::time::Duration::from_secs(
self.config.llm.nearai.failover_cooldown_secs,
),
failure_threshold: self.config.llm.nearai.failover_cooldown_threshold,
};
Arc::new(FailoverProvider::with_cooldown(
vec![llm, fallback],
cooldown_config,
)?)
} else {
llm
};
// Wrap in circuit breaker if configured
let llm: Arc<dyn LlmProvider> =
if let Some(threshold) = self.config.llm.nearai.circuit_breaker_threshold {
let cb_config = CircuitBreakerConfig {
failure_threshold: threshold,
recovery_timeout: std::time::Duration::from_secs(
self.config.llm.nearai.circuit_breaker_recovery_secs,
),
..CircuitBreakerConfig::default()
};
tracing::info!(
threshold,
recovery_secs = self.config.llm.nearai.circuit_breaker_recovery_secs,
"LLM circuit breaker enabled"
);
Arc::new(CircuitBreakerProvider::new(llm, cb_config))
} else {
llm
};
// Wrap in response cache if configured
let llm: Arc<dyn LlmProvider> = if self.config.llm.nearai.response_cache_enabled {
let rc_config = ResponseCacheConfig {
ttl: std::time::Duration::from_secs(self.config.llm.nearai.response_cache_ttl_secs),
max_entries: self.config.llm.nearai.response_cache_max_entries,
};
tracing::info!(
ttl_secs = self.config.llm.nearai.response_cache_ttl_secs,
max_entries = self.config.llm.nearai.response_cache_max_entries,
"LLM response cache enabled"
);
Arc::new(CachedProvider::new(llm, rc_config))
} else {
llm
};
// Cheap LLM for lightweight tasks
let cheap_llm = create_cheap_llm_provider(&self.config.llm, self.session.clone())?;
if let Some(ref cheap) = cheap_llm {
tracing::info!("Cheap LLM provider initialized: {}", cheap.model_name());
}
let (llm, cheap_llm) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
Ok((llm, cheap_llm))
}
@@ -389,54 +316,41 @@ impl AppBuilder {
),
anyhow::Error,
> {
use crate::workspace::{NearAiEmbeddings, OpenAiEmbeddings};
let safety = Arc::new(SafetyLayer::new(&self.config.safety));
tracing::info!("Safety layer initialized");
let tools = Arc::new(ToolRegistry::new());
// Initialize tool registry with credential injection support
let credential_registry = Arc::new(SharedCredentialRegistry::new());
let tools = if let Some(ref ss) = self.secrets_store {
Arc::new(
ToolRegistry::new()
.with_credentials(Arc::clone(&credential_registry), Arc::clone(ss)),
)
} else {
Arc::new(ToolRegistry::new())
};
tools.register_builtin_tools();
// Create embeddings provider if configured
let embeddings: Option<Arc<dyn EmbeddingProvider>> = if self.config.embeddings.enabled {
match self.config.embeddings.provider.as_str() {
"nearai" => {
tracing::info!(
"Embeddings enabled via NEAR AI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(
NearAiEmbeddings::new(
&self.config.llm.nearai.base_url,
self.session.clone(),
)
.with_model(&self.config.embeddings.model, 1536),
))
}
_ => {
if let Some(api_key) = self.config.embeddings.openai_api_key() {
tracing::info!(
"Embeddings enabled via OpenAI (model: {})",
self.config.embeddings.model
);
Some(Arc::new(OpenAiEmbeddings::with_model(
api_key,
&self.config.embeddings.model,
match self.config.embeddings.model.as_str() {
"text-embedding-3-large" => 3072,
_ => 1536,
},
)))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
}
}
}
} else {
tracing::info!("Embeddings disabled (set OPENAI_API_KEY or EMBEDDING_ENABLED=true)");
None
};
// Create embeddings provider using the unified method
let embeddings = self
.config
.embeddings
.create_provider(&self.config.llm.nearai.base_url, self.session.clone());
// Warn if libSQL backend is used with non-1536 embedding dimension.
if self.config.database.backend == crate::config::DatabaseBackend::LibSql
&& self.config.embeddings.enabled
&& self.config.embeddings.dimension != 1536
{
tracing::warn!(
configured_dimension = self.config.embeddings.dimension,
"Embedding dimension {} is not 1536. The libSQL schema uses \
F32_BLOB(1536) which requires exactly 1536 dimensions. \
Embedding storage will fail. Use PostgreSQL or set \
EMBEDDING_DIMENSION=1536.",
self.config.embeddings.dimension
);
}
// Register memory tools if database is available
let workspace = if let Some(ref db) = self.db {
@@ -478,6 +392,8 @@ impl AppBuilder {
Arc<McpSessionManager>,
Option<Arc<WasmToolRuntime>>,
Option<Arc<ExtensionManager>>,
Vec<crate::extensions::RegistryEntry>,
Vec<String>,
),
anyhow::Error,
> {
@@ -507,6 +423,8 @@ impl AppBuilder {
let tools = Arc::clone(tools);
let wasm_config = self.config.wasm.clone();
async move {
let mut dev_loaded_tool_names: Vec<String> = Vec::new();
if let Some(ref runtime) = wasm_tool_runtime {
let mut loader = WasmToolLoader::new(Arc::clone(runtime), Arc::clone(&tools));
if let Some(ref secrets) = secrets_store {
@@ -537,10 +455,11 @@ impl AppBuilder {
match load_dev_tools(&loader, &wasm_config.tools_dir).await {
Ok(results) => {
if !results.loaded.is_empty() {
dev_loaded_tool_names.extend(results.loaded.iter().cloned());
if !dev_loaded_tool_names.is_empty() {
tracing::info!(
"Loaded {} dev WASM tools from build artifacts",
results.loaded.len()
dev_loaded_tool_names.len()
);
}
}
@@ -549,6 +468,8 @@ impl AppBuilder {
}
}
}
dev_loaded_tool_names
}
};
@@ -653,13 +574,46 @@ impl AppBuilder {
}
};
tokio::join!(wasm_tools_future, mcp_servers_future);
let (dev_loaded_tool_names, _) = tokio::join!(wasm_tools_future, mcp_servers_future);
// Create extension manager
let extension_manager = if let Some(ref secrets) = self.secrets_store {
// Load registry catalog entries for extension discovery
let catalog_entries = match crate::registry::RegistryCatalog::load_or_embedded() {
Ok(catalog) => {
let entries: Vec<_> = catalog
.all()
.iter()
.map(|m| m.to_registry_entry())
.collect();
tracing::info!(
count = entries.len(),
"Loaded registry catalog entries for extension discovery"
);
entries
}
Err(e) => {
tracing::warn!("Failed to load registry catalog: {}", e);
Vec::new()
}
};
// Create extension manager. Use ephemeral in-memory secrets if no
// persistent store is configured (listing/install/activate still work).
let ext_secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> = if let Some(ref s) =
self.secrets_store
{
Arc::clone(s)
} else {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
let ephemeral_key =
secrecy::SecretString::from(crate::secrets::keychain::generate_master_key_hex());
let crypto = Arc::new(SecretsCrypto::new(ephemeral_key).expect("ephemeral crypto"));
tracing::debug!("Using ephemeral in-memory secrets store for extension manager");
Arc::new(InMemorySecretsStore::new(crypto))
};
let extension_manager = {
let manager = Arc::new(ExtensionManager::new(
Arc::clone(&mcp_session_manager),
Arc::clone(secrets),
ext_secrets,
Arc::clone(tools),
Some(Arc::clone(hooks)),
wasm_tool_runtime.clone(),
@@ -668,16 +622,11 @@ impl AppBuilder {
self.config.tunnel.public_url.clone(),
"default".to_string(),
self.db.clone(),
catalog_entries.clone(),
));
tools.register_extension_tools(Arc::clone(&manager));
tracing::info!("Extension manager initialized with in-chat discovery tools");
Some(manager)
} else {
tracing::debug!(
"Extension manager not available (no secrets store). \
Extension tools won't be registered."
);
None
};
// register_builder_tool() already calls register_dev_tools() internally,
@@ -688,7 +637,13 @@ impl AppBuilder {
tools.register_dev_tools();
}
Ok((mcp_session_manager, wasm_tool_runtime, extension_manager))
Ok((
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
))
}
/// Run all init phases in order and return the assembled components.
@@ -702,8 +657,13 @@ impl AppBuilder {
// Create hook registry early so runtime extension activation can register hooks.
let hooks = Arc::new(HookRegistry::new());
let (mcp_session_manager, wasm_tool_runtime, extension_manager) =
self.init_extensions(&tools, &hooks).await?;
let (
mcp_session_manager,
wasm_tool_runtime,
extension_manager,
catalog_entries,
dev_loaded_tool_names,
) = self.init_extensions(&tools, &hooks).await?;
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
@@ -715,15 +675,18 @@ impl AppBuilder {
}
if embeddings.is_some() {
match ws.backfill_embeddings().await {
Ok(count) if count > 0 => {
tracing::info!("Backfilled embeddings for {} chunks", count);
let ws_bg = Arc::clone(ws);
tokio::spawn(async move {
match ws_bg.backfill_embeddings().await {
Ok(count) if count > 0 => {
tracing::info!("Backfilled embeddings for {} chunks", count);
}
Ok(_) => {}
Err(e) => {
tracing::warn!("Failed to backfill embeddings: {}", e);
}
}
Ok(_) => {}
Err(e) => {
tracing::warn!("Failed to backfill embeddings: {}", e);
}
}
});
}
}
@@ -775,6 +738,8 @@ impl AppBuilder {
skill_catalog,
cost_guard,
session: self.session,
catalog_entries,
dev_loaded_tool_names,
})
}
}
+32 -9
View File
@@ -40,16 +40,39 @@ impl ChannelManager {
}
/// Add a channel to the manager.
pub fn add(&mut self, channel: Box<dyn Channel>) {
pub async fn add(&self, channel: Box<dyn Channel>) {
let name = channel.name().to_string();
// We need to get the inner HashMap to insert
// Since we're in a sync context during setup, we'll use try_write
if let Ok(mut channels) = self.channels.try_write() {
channels.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
} else {
tracing::error!("Failed to add channel: {} (lock contention)", name);
}
self.channels.write().await.insert(name.clone(), channel);
tracing::debug!("Added channel: {}", name);
}
/// Hot-add a channel to a running agent.
///
/// Starts the channel, registers it in the channels map for `respond()`/`broadcast()`,
/// and spawns a task that forwards its stream messages through `inject_tx` into
/// the agent loop.
pub async fn hot_add(&self, channel: Box<dyn Channel>) -> Result<(), ChannelError> {
let name = channel.name().to_string();
let stream = channel.start().await?;
// Register for respond/broadcast/send_status
self.channels.write().await.insert(name.clone(), channel);
// Forward stream messages through inject_tx
let tx = self.inject_tx.clone();
tokio::spawn(async move {
use futures::StreamExt;
let mut stream = stream;
while let Some(msg) = stream.next().await {
if tx.send(msg).await.is_err() {
tracing::warn!(channel = %name, "Inject channel closed, stopping hot-added channel");
break;
}
}
tracing::info!(channel = %name, "Hot-added channel stream ended");
});
Ok(())
}
/// Start all channels and return a merged stream of messages.
+41 -4
View File
@@ -15,6 +15,7 @@
//! - `/compact` - Compact the context
//! - `/new` - Start a new thread
//! - `yes`/`no`/`always` - Respond to tool approval prompts
//! - `Esc` - Interrupt current operation
use std::borrow::Cow;
use std::io::{self, Write};
@@ -28,7 +29,10 @@ use rustyline::error::ReadlineError;
use rustyline::highlight::Highlighter;
use rustyline::hint::Hinter;
use rustyline::validate::Validator;
use rustyline::{CompletionType, Editor, Helper};
use rustyline::{
Cmd as ReadlineCmd, CompletionType, ConditionalEventHandler, Editor, Event, EventContext,
EventHandler, Helper, KeyCode, KeyEvent, Modifiers, RepeatCount,
};
use termimad::MadSkin;
use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream;
@@ -121,6 +125,23 @@ impl Highlighter for ReplHelper {
impl Validator for ReplHelper {}
impl Helper for ReplHelper {}
struct EscInterruptHandler {
triggered: Arc<AtomicBool>,
}
impl ConditionalEventHandler for EscInterruptHandler {
fn handle(
&self,
_evt: &Event,
_n: RepeatCount,
_positive: bool,
_ctx: &EventContext,
) -> Option<ReadlineCmd> {
self.triggered.store(true, Ordering::Relaxed);
Some(ReadlineCmd::Interrupt)
}
}
/// Build a termimad skin with our color scheme.
fn make_skin() -> MadSkin {
let mut skin = MadSkin::default();
@@ -247,6 +268,7 @@ fn print_help() {
println!(" {c}/compact{r} {d}compact context window{r}");
println!(" {c}/new{r} {d}new conversation thread{r}");
println!(" {c}/interrupt{r} {d}stop current operation{r}");
println!(" {c}esc{r} {d}stop current operation{r}");
println!();
println!(" {h}Approval responses{r}");
println!(" {c}yes{r} ({c}y{r}) {d}approve tool execution{r}");
@@ -274,6 +296,7 @@ impl Channel for ReplChannel {
let single_message = self.single_message.clone();
let debug_mode = Arc::clone(&self.debug_mode);
let suppress_banner = Arc::clone(&self.suppress_banner);
let esc_interrupt_triggered_for_thread = Arc::new(AtomicBool::new(false));
std::thread::spawn(move || {
// Single message mode: send it and return
@@ -301,6 +324,13 @@ impl Channel for ReplChannel {
rl.set_helper(Some(ReplHelper));
rl.bind_sequence(
KeyEvent(KeyCode::Esc, Modifiers::NONE),
EventHandler::Conditional(Box::new(EscInterruptHandler {
triggered: Arc::clone(&esc_interrupt_triggered_for_thread),
})),
);
// Load history
let hist_path = history_path();
if let Some(parent) = hist_path.parent() {
@@ -360,9 +390,16 @@ impl Channel for ReplChannel {
}
}
Err(ReadlineError::Interrupted) => {
// Ctrl+C: send /interrupt
let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() {
if esc_interrupt_triggered_for_thread.swap(false, Ordering::Relaxed) {
// Esc: interrupt current operation and keep REPL open.
let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() {
break;
}
} else {
// Ctrl+C (VINTR): request graceful shutdown.
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
break;
}
}
+33 -20
View File
@@ -20,6 +20,7 @@ const CARGO_MANIFEST_DIR: &str = env!("CARGO_MANIFEST_DIR");
const KNOWN_CHANNELS: &[(&str, &str)] = &[
("telegram", "telegram_channel"),
("slack", "slack_channel"),
("discord", "discord_channel"),
("whatsapp", "whatsapp_channel"),
];
@@ -42,6 +43,10 @@ fn channels_src_dir() -> PathBuf {
/// Locate the build artifacts for a channel.
///
/// Checks two layouts:
/// 1. **Flat** (Docker/packaged): `<channels_src>/<name>/<name>.wasm`
/// 2. **Build tree** (dev): `<channels_src>/<name>/target/wasm32-wasip2/release/<crate_name>.wasm`
///
/// Returns (wasm_path, capabilities_path) or an error if files are missing.
fn locate_channel_artifacts(name: &str) -> Result<(PathBuf, PathBuf), String> {
let (_, crate_name) = KNOWN_CHANNELS
@@ -52,31 +57,38 @@ fn locate_channel_artifacts(name: &str) -> Result<(PathBuf, PathBuf), String> {
let src_dir = channels_src_dir();
let channel_dir = src_dir.join(name);
let wasm_path = channel_dir
.join("target/wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
let caps_path = channel_dir.join(format!("{}.capabilities.json", name));
if !wasm_path.exists() {
return Err(format!(
"Channel '{}' WASM not found at {}. Build it first:\n \
cd {} && cargo build --target wasm32-wasip2 --release",
name,
wasm_path.display(),
channel_dir.display()
));
// Check flat layout first (Docker/packaged deployments)
let flat_wasm = channel_dir.join(format!("{}.wasm", name));
if flat_wasm.exists() && caps_path.exists() {
return Ok((flat_wasm, caps_path));
}
if !caps_path.exists() {
return Err(format!(
"Channel '{}' capabilities not found at {}",
name,
caps_path.display()
));
// Fall back to build tree layout (dev builds) — search across all WASM triples
if let Some(build_wasm) =
crate::registry::artifacts::find_wasm_artifact(&channel_dir, crate_name, "release")
&& caps_path.exists()
{
return Ok((build_wasm, caps_path));
}
Ok((wasm_path, caps_path))
// Provide a helpful error with the paths we checked
let expected_build = crate::registry::artifacts::resolve_target_dir(&channel_dir)
.join("wasm32-wasip2/release")
.join(format!("{}.wasm", crate_name));
Err(format!(
"Channel '{}' WASM not found. Checked:\n \
- {} (flat/packaged)\n \
- {} (build tree, and other triples)\n \
Build it first:\n \
cd {} && cargo component build --release",
name,
flat_wasm.display(),
expected_build.display(),
channel_dir.display()
))
}
/// Install a channel from build artifacts into the channels directory.
@@ -130,10 +142,11 @@ mod tests {
use super::*;
#[test]
fn test_known_channels_includes_all_three() {
fn test_known_channels_includes_all_four() {
let names = bundled_channel_names();
assert!(names.contains(&"telegram"));
assert!(names.contains(&"slack"));
assert!(names.contains(&"discord"));
assert!(names.contains(&"whatsapp"));
}
+16 -1
View File
@@ -110,6 +110,21 @@ impl WasmChannelRouter {
.unwrap_or_else(|| "X-Webhook-Secret".to_string())
}
/// Update the webhook secret for an already-registered channel.
///
/// This is used when credentials are saved after a channel was registered
/// without a secret (e.g., loaded at startup before the user configured it).
pub async fn update_secret(&self, channel_name: &str, secret: String) {
self.secrets
.write()
.await
.insert(channel_name.to_string(), secret);
tracing::info!(
channel = %channel_name,
"Updated webhook secret for channel"
);
}
/// Unregister a channel and its endpoints.
pub async fn unregister(&self, channel_name: &str) {
self.channels.write().await.remove(channel_name);
@@ -488,7 +503,7 @@ mod tests {
let prepared = Arc::new(PreparedChannelModule {
name: name.to_string(),
description: format!("Test channel: {}", name),
component_bytes: Vec::new(),
component: None,
limits: ResourceLimits::default(),
});
+30 -10
View File
@@ -68,38 +68,51 @@ impl WasmChannelRuntimeConfig {
}
/// A compiled WASM channel component ready for instantiation.
#[derive(Debug)]
///
/// Stores the pre-compiled `Component` directly so instantiation
/// doesn't require recompilation.
pub struct PreparedChannelModule {
/// Channel name.
pub name: String,
/// Channel description.
pub description: String,
/// Compiled component bytes (public for testing, otherwise use component_bytes()).
pub(crate) component_bytes: Vec<u8>,
/// Pre-compiled component (cheaply cloneable via internal Arc).
pub(crate) component: Option<wasmtime::component::Component>,
/// Resource limits for this channel.
pub limits: ResourceLimits,
}
impl PreparedChannelModule {
/// Get the compiled component bytes.
pub fn component_bytes(&self) -> &[u8] {
&self.component_bytes
/// Get the pre-compiled component for instantiation.
pub fn component(&self) -> Option<&wasmtime::component::Component> {
self.component.as_ref()
}
/// Create a PreparedChannelModule for testing purposes.
///
/// Creates a module with no actual WASM bytes, suitable for testing
/// Creates a module with no actual WASM component, suitable for testing
/// channel infrastructure without requiring a real WASM component.
pub fn for_testing(name: impl Into<String>, description: impl Into<String>) -> Self {
Self {
name: name.into(),
description: description.into(),
component_bytes: Vec::new(),
component: None,
limits: ResourceLimits::default(),
}
}
}
impl std::fmt::Debug for PreparedChannelModule {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PreparedChannelModule")
.field("name", &self.name)
.field("description", &self.description)
.field("has_component", &self.component.is_some())
.field("limits", &self.limits)
.finish()
}
}
/// WASM channel runtime.
///
/// Manages the Wasmtime engine and a cache of prepared channel modules.
@@ -137,6 +150,13 @@ impl WasmChannelRuntime {
// Disable debug info in production
wasmtime_config.debug_info(false);
// Enable persistent compilation cache. Wasmtime serializes compiled native
// code to disk (~/.cache/wasmtime by default), so subsequent startups
// deserialize instead of recompiling — typically 10-50x faster.
if let Err(e) = wasmtime_config.cache_config_load_default() {
tracing::warn!("Failed to enable wasmtime compilation cache: {}", e);
}
let engine = Engine::new(&wasmtime_config).map_err(|e| {
WasmChannelError::Config(format!("Failed to create Wasmtime engine: {}", e))
})?;
@@ -183,13 +203,13 @@ impl WasmChannelRuntime {
// Compile in blocking task (Wasmtime compilation is synchronous)
let prepared = tokio::task::spawn_blocking(move || {
// Validate and compile the component
let _component = wasmtime::component::Component::new(&engine, &wasm_bytes)
let component = wasmtime::component::Component::new(&engine, &wasm_bytes)
.map_err(|e| WasmChannelError::Compilation(e.to_string()))?;
Ok::<_, WasmChannelError>(PreparedChannelModule {
name: name.clone(),
description: desc,
component_bytes: wasm_bytes,
component: Some(component),
limits: limits.unwrap_or(default_limits),
})
})
+698 -40
View File
@@ -37,7 +37,7 @@ use tokio::sync::{RwLock, mpsc, oneshot};
use tokio_stream::wrappers::ReceiverStream;
use uuid::Uuid;
use wasmtime::Store;
use wasmtime::component::{Component, Linker};
use wasmtime::component::Linker;
use wasmtime_wasi::{ResourceTable, WasiCtx, WasiCtxBuilder, WasiView};
use crate::channels::wasm::capabilities::ChannelCapabilities;
@@ -725,9 +725,13 @@ impl WasmChannel {
) -> Result<SandboxedChannel, WasmChannelError> {
let engine = runtime.engine();
// Compile the component (uses cached bytes)
let component = Component::new(engine, prepared.component_bytes())
.map_err(|e| WasmChannelError::Compilation(e.to_string()))?;
// Use the pre-compiled component (no recompilation needed)
let component = prepared
.component()
.ok_or_else(|| {
WasmChannelError::Compilation("No compiled component available".to_string())
})?
.clone();
// Create linker and add host functions
let mut linker = Linker::new(engine);
@@ -776,9 +780,14 @@ impl WasmChannel {
/// Execute the on_start callback.
///
/// Returns the channel configuration for HTTP endpoint registration.
async fn call_on_start(&self) -> Result<ChannelConfig, WasmChannelError> {
/// Call the WASM module's `on_start` callback.
///
/// Typically called once during `start()`, but can be called again after
/// credentials are refreshed to re-trigger webhook registration and
/// other one-time setup that depends on credentials.
pub async fn call_on_start(&self) -> Result<ChannelConfig, WasmChannelError> {
// If no WASM bytes, return default config (for testing)
if self.prepared.component_bytes.is_empty() {
if self.prepared.component().is_none() {
tracing::info!(
channel = %self.name,
"WASM channel on_start called (no WASM module, returning defaults)"
@@ -918,7 +927,7 @@ impl WasmChannel {
);
// If no WASM bytes, return 200 OK (for testing)
if self.prepared.component_bytes.is_empty() {
if self.prepared.component().is_none() {
tracing::debug!(
channel = %self.name,
method = method,
@@ -1018,7 +1027,7 @@ impl WasmChannel {
/// Called periodically if polling is configured.
pub async fn call_on_poll(&self) -> Result<(), WasmChannelError> {
// If no WASM bytes, do nothing (for testing)
if self.prepared.component_bytes.is_empty() {
if self.prepared.component().is_none() {
tracing::debug!(
channel = %self.name,
"WASM channel on_poll called (no WASM module)"
@@ -1118,7 +1127,7 @@ impl WasmChannel {
);
// If no WASM bytes, do nothing (for testing)
if self.prepared.component_bytes.is_empty() {
if self.prepared.component().is_none() {
tracing::debug!(
channel = %self.name,
message_id = %message_id,
@@ -1236,7 +1245,7 @@ impl WasmChannel {
metadata: &serde_json::Value,
) -> Result<(), WasmChannelError> {
// If no WASM bytes, do nothing (for testing)
if self.prepared.component_bytes.is_empty() {
if self.prepared.component().is_none() {
return Ok(());
}
@@ -1307,7 +1316,7 @@ impl WasmChannel {
timeout: Duration,
wit_update: wit_channel::StatusUpdate,
) -> Result<(), WasmChannelError> {
if prepared.component_bytes.is_empty() {
if prepared.component().is_none() {
return Ok(());
}
@@ -1366,13 +1375,25 @@ impl WasmChannel {
/// that repeats the call every 4 seconds (Telegram's typing indicator
/// expires after ~5s).
///
/// On Done/Interrupted/Status: cancels the repeat task, fires on_status once.
/// On terminal or user-action-required states: cancels the repeat task,
/// then fires on_status once.
///
/// On intermediate progress states (tool/auth/job/status updates), keeps
/// the typing repeater running and fires on_status once.
/// On StreamChunk: no-op (too noisy).
async fn handle_status_update(
&self,
status: StatusUpdate,
metadata: &serde_json::Value,
) -> Result<(), ChannelError> {
fn is_terminal_text_status(msg: &str) -> bool {
let trimmed = msg.trim();
trimmed.eq_ignore_ascii_case("done")
|| trimmed.eq_ignore_ascii_case("interrupted")
|| trimmed.eq_ignore_ascii_case("awaiting approval")
|| trimmed.eq_ignore_ascii_case("rejected")
}
match &status {
StatusUpdate::Thinking(_) => {
// Cancel any existing typing task
@@ -1433,10 +1454,98 @@ impl WasmChannel {
StatusUpdate::StreamChunk(_) => {
// No-op, too noisy
}
_ => {
// Done, Interrupted, Status, ToolStarted, ToolCompleted: cancel and fire once
StatusUpdate::ApprovalNeeded {
tool_name,
description,
parameters,
..
} => {
// WASM channels (Telegram, Slack, etc.) cannot render
// interactive approval overlays. Send the approval prompt
// as an actual message so the user can reply yes/no.
self.cancel_typing_task().await;
let params_preview = parameters
.as_object()
.map(|obj| {
obj.iter()
.map(|(k, v)| {
let val = match v {
serde_json::Value::String(s) => {
if s.chars().count() > 80 {
let truncated: String = s.chars().take(77).collect();
format!("\"{}...\"", truncated)
} else {
format!("\"{}\"", s)
}
}
other => {
let s = other.to_string();
if s.chars().count() > 80 {
let truncated: String = s.chars().take(77).collect();
format!("{}...", truncated)
} else {
s
}
}
};
format!(" {}: {}", k, val)
})
.collect::<Vec<_>>()
.join("\n")
})
.unwrap_or_default();
let prompt = format!(
"Approval needed: {tool_name}\n\
{description}\n\
\n\
Parameters:\n\
{params_preview}\n\
\n\
Reply \"yes\" to approve, \"no\" to deny, or \"always\" to auto-approve."
);
let metadata_json = serde_json::to_string(metadata).unwrap_or_default();
if let Err(e) = self
.call_on_respond(uuid::Uuid::new_v4(), &prompt, None, &metadata_json)
.await
{
tracing::warn!(
channel = %self.name,
error = %e,
"Failed to send approval prompt via on_respond, falling back to on_status"
);
// Fall back to status update (typing indicator)
let _ = self.call_on_status(&status, metadata).await;
}
}
StatusUpdate::AuthRequired { .. } => {
// Waiting on user action: stop typing and fire once.
self.cancel_typing_task().await;
if let Err(e) = self.call_on_status(&status, metadata).await {
tracing::debug!(
channel = %self.name,
error = %e,
"on_status failed (best-effort)"
);
}
}
StatusUpdate::Status(msg) if is_terminal_text_status(msg) => {
// Waiting on user or terminal states: stop typing and fire once.
self.cancel_typing_task().await;
if let Err(e) = self.call_on_status(&status, metadata).await {
tracing::debug!(
channel = %self.name,
error = %e,
"on_status failed (best-effort)"
);
}
}
_ => {
// Intermediate progress status: keep any existing typing task alive.
if let Err(e) = self.call_on_status(&status, metadata).await {
tracing::debug!(
channel = %self.name,
@@ -1627,7 +1736,7 @@ impl WasmChannel {
workspace_store: &Arc<ChannelWorkspaceStore>,
) -> Result<Vec<EmittedMessage>, WasmChannelError> {
// Skip if no WASM bytes (testing mode)
if prepared.component_bytes.is_empty() {
if prepared.component().is_none() {
tracing::debug!(
channel = %channel_name,
"WASM channel on_poll called (no WASM module)"
@@ -2051,6 +2160,16 @@ fn convert_http_response(wit: wit_channel::OutgoingHttpResponse) -> HttpResponse
}
/// Convert a StatusUpdate + metadata into the WIT StatusUpdate type.
fn truncate_status_text(input: &str, max_chars: usize) -> String {
let mut iter = input.chars();
let truncated: String = iter.by_ref().take(max_chars).collect();
if iter.next().is_some() {
format!("{}...", truncated)
} else {
truncated
}
}
fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_channel::StatusUpdate {
let metadata_json = serde_json::to_string(metadata).unwrap_or_default();
@@ -2062,17 +2181,25 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
},
StatusUpdate::ToolStarted { name } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolStarted,
message: name.clone(),
message: format!("Tool started: {}", name),
metadata_json,
},
StatusUpdate::ToolCompleted { name, success } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolCompleted,
message: format!("{}: {}", name, if *success { "ok" } else { "failed" }),
message: format!(
"Tool completed: {} ({})",
name,
if *success { "ok" } else { "failed" }
),
metadata_json,
},
StatusUpdate::ToolResult { name, preview } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolCompleted,
message: format!("{}: {}", name, preview),
status: wit_channel::StatusType::ToolResult,
message: format!(
"Tool result: {}\n{}",
name,
truncate_status_text(preview, 280)
),
metadata_json,
},
StatusUpdate::StreamChunk(chunk) => wit_channel::StatusUpdate {
@@ -2081,11 +2208,16 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
metadata_json,
},
StatusUpdate::Status(msg) => {
// Map well-known status strings to WIT types
let status_type = match msg.as_str() {
"Done" => wit_channel::StatusType::Done,
"Interrupted" => wit_channel::StatusType::Interrupted,
_ => wit_channel::StatusType::Thinking,
// Map well-known status strings to WIT types (case-insensitive
// to stay consistent with is_terminal_text_status and the
// Telegram-side classify_status_update).
let trimmed = msg.trim();
let status_type = if trimmed.eq_ignore_ascii_case("done") {
wit_channel::StatusType::Done
} else if trimmed.eq_ignore_ascii_case("interrupted") {
wit_channel::StatusType::Interrupted
} else {
wit_channel::StatusType::Status
};
wit_channel::StatusUpdate {
status: status_type,
@@ -2094,34 +2226,62 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
}
}
StatusUpdate::ApprovalNeeded {
request_id,
tool_name,
description,
..
} => wit_channel::StatusUpdate {
status: wit_channel::StatusType::Thinking,
message: format!("Approval needed: {} - {}", tool_name, description),
status: wit_channel::StatusType::ApprovalNeeded,
message: format!(
"Approval needed for tool '{}'. {}\nRequest ID: {}\nReply with: yes (or /approve), no (or /deny), or always (or /always).",
tool_name, description, request_id
),
metadata_json,
},
StatusUpdate::JobStarted { job_id, title, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::Thinking,
message: format!("Job started: {} ({})", title, job_id),
StatusUpdate::JobStarted {
job_id,
title,
browse_url,
} => wit_channel::StatusUpdate {
status: wit_channel::StatusType::JobStarted,
message: format!("Job started: {} ({})\n{}", title, job_id, browse_url),
metadata_json,
},
StatusUpdate::AuthRequired { extension_name, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::Thinking,
message: format!("Auth required: {}", extension_name),
StatusUpdate::AuthRequired {
extension_name,
instructions,
auth_url,
setup_url,
} => wit_channel::StatusUpdate {
status: wit_channel::StatusType::AuthRequired,
message: {
let mut lines = vec![format!("Authentication required for {}.", extension_name)];
if let Some(text) = instructions
&& !text.trim().is_empty()
{
lines.push(text.trim().to_string());
}
if let Some(url) = auth_url {
lines.push(format!("Auth URL: {}", url));
}
if let Some(url) = setup_url {
lines.push(format!("Setup URL: {}", url));
}
lines.join("\n")
},
metadata_json,
},
StatusUpdate::AuthCompleted {
extension_name,
success,
..
message,
} => wit_channel::StatusUpdate {
status: wit_channel::StatusType::Thinking,
status: wit_channel::StatusType::AuthCompleted,
message: format!(
"Auth {}: {}",
"Authentication {} for {}. {}",
if *success { "completed" } else { "failed" },
extension_name
extension_name,
message
),
metadata_json,
},
@@ -2137,6 +2297,12 @@ fn clone_wit_status_update(update: &wit_channel::StatusUpdate) -> wit_channel::S
wit_channel::StatusType::Interrupted => wit_channel::StatusType::Interrupted,
wit_channel::StatusType::ToolStarted => wit_channel::StatusType::ToolStarted,
wit_channel::StatusType::ToolCompleted => wit_channel::StatusType::ToolCompleted,
wit_channel::StatusType::ToolResult => wit_channel::StatusType::ToolResult,
wit_channel::StatusType::ApprovalNeeded => wit_channel::StatusType::ApprovalNeeded,
wit_channel::StatusType::Status => wit_channel::StatusType::Status,
wit_channel::StatusType::JobStarted => wit_channel::StatusType::JobStarted,
wit_channel::StatusType::AuthRequired => wit_channel::StatusType::AuthRequired,
wit_channel::StatusType::AuthCompleted => wit_channel::StatusType::AuthCompleted,
},
message: update.message.clone(),
metadata_json: update.metadata_json.clone(),
@@ -2206,7 +2372,7 @@ mod tests {
let prepared = Arc::new(PreparedChannelModule {
name: "test".to_string(),
description: "Test channel".to_string(),
component_bytes: Vec::new(),
component: None,
limits: ResourceLimits::default(),
});
@@ -2271,7 +2437,7 @@ mod tests {
#[tokio::test]
async fn test_execute_poll_no_wasm_returns_empty() {
// When there's no WASM module (empty component_bytes), execute_poll
// When there's no WASM module (None component), execute_poll
// should return an empty vector of messages
let config = WasmChannelRuntimeConfig::for_testing();
let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());
@@ -2279,7 +2445,7 @@ mod tests {
let prepared = Arc::new(PreparedChannelModule {
name: "poll-test".to_string(),
description: "Test channel".to_string(),
component_bytes: Vec::new(), // No WASM bytes
component: None, // No WASM module
limits: ResourceLimits::default(),
});
@@ -2381,7 +2547,7 @@ mod tests {
let prepared = Arc::new(PreparedChannelModule {
name: "poll-channel".to_string(),
description: "Polling test channel".to_string(),
component_bytes: Vec::new(),
component: None,
limits: ResourceLimits::default(),
});
@@ -2480,6 +2646,100 @@ mod tests {
channel.shutdown().await.expect("Shutdown should succeed");
}
#[tokio::test]
async fn test_typing_task_persists_on_tool_started() {
let channel = create_test_channel();
let _stream = channel.start().await.expect("Channel should start");
let metadata = serde_json::json!({"chat_id": 123});
// Start typing
let _ = channel
.send_status(
crate::channels::StatusUpdate::Thinking("Processing...".into()),
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_some());
// Intermediate tool status should not cancel typing
let _ = channel
.send_status(
crate::channels::StatusUpdate::ToolStarted {
name: "http_request".into(),
},
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_some());
channel.shutdown().await.expect("Shutdown should succeed");
}
#[tokio::test]
async fn test_typing_task_cancelled_on_approval_needed() {
let channel = create_test_channel();
let _stream = channel.start().await.expect("Channel should start");
let metadata = serde_json::json!({"chat_id": 123});
// Start typing
let _ = channel
.send_status(
crate::channels::StatusUpdate::Thinking("Processing...".into()),
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_some());
// Approval-needed should stop typing while waiting for user action
let _ = channel
.send_status(
crate::channels::StatusUpdate::ApprovalNeeded {
request_id: "req-1".into(),
tool_name: "http_request".into(),
description: "Fetch weather".into(),
parameters: serde_json::json!({"url": "https://wttr.in"}),
},
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_none());
channel.shutdown().await.expect("Shutdown should succeed");
}
#[tokio::test]
async fn test_typing_task_cancelled_on_awaiting_approval_status() {
let channel = create_test_channel();
let _stream = channel.start().await.expect("Channel should start");
let metadata = serde_json::json!({"chat_id": 123});
// Start typing
let _ = channel
.send_status(
crate::channels::StatusUpdate::Thinking("Processing...".into()),
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_some());
// Legacy terminal status string should also cancel typing
let _ = channel
.send_status(
crate::channels::StatusUpdate::Status("Awaiting approval".into()),
&metadata,
)
.await;
assert!(channel.typing_task.read().await.is_none());
channel.shutdown().await.expect("Shutdown should succeed");
}
#[tokio::test]
async fn test_typing_task_replaced_on_new_thinking() {
let channel = create_test_channel();
@@ -2603,6 +2863,27 @@ mod tests {
assert!(matches!(wit.status, super::wit_channel::StatusType::Done));
}
#[test]
fn test_status_to_wit_done_case_insensitive() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
// lowercase
let wit = status_to_wit(
&crate::channels::StatusUpdate::Status("done".into()),
&metadata,
);
assert!(matches!(wit.status, super::wit_channel::StatusType::Done));
// with whitespace
let wit = status_to_wit(
&crate::channels::StatusUpdate::Status(" Done ".into()),
&metadata,
);
assert!(matches!(wit.status, super::wit_channel::StatusType::Done));
}
#[test]
fn test_status_to_wit_interrupted() {
use super::status_to_wit;
@@ -2619,6 +2900,311 @@ mod tests {
));
}
#[test]
fn test_status_to_wit_interrupted_case_insensitive() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
// lowercase
let wit = status_to_wit(
&crate::channels::StatusUpdate::Status("interrupted".into()),
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::Interrupted
));
// with whitespace
let wit = status_to_wit(
&crate::channels::StatusUpdate::Status(" Interrupted ".into()),
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::Interrupted
));
}
#[test]
fn test_status_to_wit_generic_status() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::Status("Awaiting approval".into()),
&metadata,
);
assert!(matches!(wit.status, super::wit_channel::StatusType::Status));
assert_eq!(wit.message, "Awaiting approval");
}
#[test]
fn test_status_to_wit_auth_required() {
use super::status_to_wit;
let metadata = serde_json::json!({"chat_id": 42});
let wit = status_to_wit(
&crate::channels::StatusUpdate::AuthRequired {
extension_name: "weather".to_string(),
instructions: Some("Paste your token".to_string()),
auth_url: Some("https://example.com/auth".to_string()),
setup_url: None,
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::AuthRequired
));
assert!(wit.message.contains("Authentication required for weather"));
assert!(wit.message.contains("Paste your token"));
}
#[test]
fn test_status_to_wit_tool_started() {
use super::status_to_wit;
let metadata = serde_json::json!({"chat_id": 7});
let wit = status_to_wit(
&crate::channels::StatusUpdate::ToolStarted {
name: "http_request".to_string(),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ToolStarted
));
assert_eq!(wit.message, "Tool started: http_request");
}
#[test]
fn test_status_to_wit_tool_completed_success() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: true,
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ToolCompleted
));
assert_eq!(wit.message, "Tool completed: http_request (ok)");
}
#[test]
fn test_status_to_wit_tool_completed_failure() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: false,
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ToolCompleted
));
assert_eq!(wit.message, "Tool completed: http_request (failed)");
}
#[test]
fn test_status_to_wit_tool_result() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::ToolResult {
name: "http_request".to_string(),
preview: "{".to_string() + "\"temperature\": 22}",
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ToolResult
));
assert!(wit.message.starts_with("Tool result: http_request\n"));
}
#[test]
fn test_status_to_wit_tool_result_truncates_preview() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let long_preview = "x".repeat(400);
let wit = status_to_wit(
&crate::channels::StatusUpdate::ToolResult {
name: "big_tool".to_string(),
preview: long_preview,
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ToolResult
));
assert!(wit.message.ends_with("..."));
}
#[test]
fn test_status_to_wit_job_started() {
use super::status_to_wit;
let metadata = serde_json::json!({"chat_id": 1});
let wit = status_to_wit(
&crate::channels::StatusUpdate::JobStarted {
job_id: "job-1".to_string(),
title: "Daily sync".to_string(),
browse_url: "https://example.com/jobs/job-1".to_string(),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::JobStarted
));
assert!(wit.message.contains("Daily sync"));
assert!(wit.message.contains("https://example.com/jobs/job-1"));
}
#[test]
fn test_status_to_wit_auth_completed_success() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::AuthCompleted {
extension_name: "weather".to_string(),
success: true,
message: "Token saved".to_string(),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::AuthCompleted
));
assert!(wit.message.contains("Authentication completed"));
assert!(wit.message.contains("Token saved"));
}
#[test]
fn test_status_to_wit_auth_completed_failure() {
use super::status_to_wit;
let metadata = serde_json::json!(null);
let wit = status_to_wit(
&crate::channels::StatusUpdate::AuthCompleted {
extension_name: "weather".to_string(),
success: false,
message: "Invalid token".to_string(),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::AuthCompleted
));
assert!(wit.message.contains("Authentication failed"));
assert!(wit.message.contains("Invalid token"));
}
#[test]
fn test_status_to_wit_approval_needed() {
use super::status_to_wit;
let metadata = serde_json::json!({"chat_id": 42});
let wit = status_to_wit(
&crate::channels::StatusUpdate::ApprovalNeeded {
request_id: "req-123".to_string(),
tool_name: "http_request".to_string(),
description: "Fetch weather data".to_string(),
parameters: serde_json::json!({"url": "https://api.weather.test"}),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ApprovalNeeded
));
assert!(wit.message.contains("http_request"));
assert!(wit.message.contains("/approve"));
}
#[test]
fn test_approval_prompt_roundtrip_submission_aliases() {
use super::status_to_wit;
use crate::agent::submission::{Submission, SubmissionParser};
let metadata = serde_json::json!({"chat_id": 42});
let wit = status_to_wit(
&crate::channels::StatusUpdate::ApprovalNeeded {
request_id: "req-321".to_string(),
tool_name: "http_request".to_string(),
description: "Fetch weather data".to_string(),
parameters: serde_json::json!({"url": "https://api.weather.test"}),
},
&metadata,
);
assert!(matches!(
wit.status,
super::wit_channel::StatusType::ApprovalNeeded
));
assert!(wit.message.contains("/approve"));
assert!(wit.message.contains("/deny"));
assert!(wit.message.contains("/always"));
let approve = SubmissionParser::parse("/approve");
assert!(matches!(
approve,
Submission::ApprovalResponse {
approved: true,
always: false
}
));
let deny = SubmissionParser::parse("/deny");
assert!(matches!(
deny,
Submission::ApprovalResponse {
approved: false,
always: false
}
));
let always = SubmissionParser::parse("/always");
assert!(matches!(
always,
Submission::ApprovalResponse {
approved: true,
always: true
}
));
}
#[test]
fn test_clone_wit_status_update() {
use super::{clone_wit_status_update, wit_channel};
@@ -2635,6 +3221,78 @@ mod tests {
assert_eq!(cloned.metadata_json, "{\"a\":1}");
}
#[test]
fn test_clone_wit_status_update_approval_needed() {
use super::{clone_wit_status_update, wit_channel};
let original = wit_channel::StatusUpdate {
status: wit_channel::StatusType::ApprovalNeeded,
message: "approval needed".to_string(),
metadata_json: "{\"chat_id\":42}".to_string(),
};
let cloned = clone_wit_status_update(&original);
assert!(matches!(
cloned.status,
wit_channel::StatusType::ApprovalNeeded
));
assert_eq!(cloned.message, "approval needed");
assert_eq!(cloned.metadata_json, "{\"chat_id\":42}");
}
#[test]
fn test_clone_wit_status_update_auth_completed() {
use super::{clone_wit_status_update, wit_channel};
let original = wit_channel::StatusUpdate {
status: wit_channel::StatusType::AuthCompleted,
message: "auth complete".to_string(),
metadata_json: "{}".to_string(),
};
let cloned = clone_wit_status_update(&original);
assert!(matches!(
cloned.status,
wit_channel::StatusType::AuthCompleted
));
assert_eq!(cloned.message, "auth complete");
}
#[test]
fn test_clone_wit_status_update_all_variants() {
use super::{clone_wit_status_update, wit_channel};
let variants = vec![
wit_channel::StatusType::Thinking,
wit_channel::StatusType::Done,
wit_channel::StatusType::Interrupted,
wit_channel::StatusType::ToolStarted,
wit_channel::StatusType::ToolCompleted,
wit_channel::StatusType::ToolResult,
wit_channel::StatusType::ApprovalNeeded,
wit_channel::StatusType::Status,
wit_channel::StatusType::JobStarted,
wit_channel::StatusType::AuthRequired,
wit_channel::StatusType::AuthCompleted,
];
for status in variants {
let original = wit_channel::StatusUpdate {
status,
message: "sample".to_string(),
metadata_json: "{}".to_string(),
};
let cloned = clone_wit_status_update(&original);
assert_eq!(
std::mem::discriminant(&cloned.status),
std::mem::discriminant(&original.status)
);
assert_eq!(cloned.message, "sample");
assert_eq!(cloned.metadata_json, "{}");
}
}
#[test]
fn test_redact_credentials_replaces_values() {
use super::ChannelStoreData;
+1 -1
View File
@@ -20,7 +20,7 @@ pub async fn extensions_list_handler(
))?;
let installed = ext_mgr
.list(None)
.list(None, false)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
+18
View File
@@ -89,6 +89,9 @@ impl GatewayChannel {
skill_registry: None,
skill_catalog: None,
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
});
Self {
@@ -119,6 +122,9 @@ impl GatewayChannel {
skill_registry: self.state.skill_registry.clone(),
skill_catalog: self.state.skill_catalog.clone(),
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
@@ -206,6 +212,18 @@ impl GatewayChannel {
self
}
/// Inject registry catalog entries for the available extensions API.
pub fn with_registry_entries(mut self, entries: Vec<crate::extensions::RegistryEntry>) -> Self {
self.rebuild_state(|s| s.registry_entries = entries);
self
}
/// Inject the cost guard for token/cost tracking in the status popover.
pub fn with_cost_guard(mut self, cg: Arc<crate::agent::cost_guard::CostGuard>) -> Self {
self.rebuild_state(|s| s.cost_guard = Some(cg));
self
}
/// Get the auth token (for printing to console on startup).
pub fn auth_token(&self) -> &str {
&self.auth_token
+266 -11
View File
@@ -13,7 +13,7 @@ use axum::{
http::{StatusCode, header},
middleware,
response::{
Html, IntoResponse,
IntoResponse,
sse::{Event, KeepAlive, Sse},
},
routing::{get, post},
@@ -148,6 +148,13 @@ pub struct GatewayState {
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
/// Rate limiter for chat endpoints (30 messages per 60 seconds).
pub chat_rate_limiter: RateLimiter,
/// Registry catalog entries for the available extensions API.
/// Populated at startup from `registry/` manifests, independent of extension manager.
pub registry_entries: Vec<crate::extensions::RegistryEntry>,
/// Cost guard for token/cost tracking.
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
}
/// Start the gateway HTTP server.
@@ -214,6 +221,7 @@ pub async fn start_server(
// Extensions
.route("/api/extensions", get(extensions_list_handler))
.route("/api/extensions/tools", get(extensions_tools_handler))
.route("/api/extensions/registry", get(extensions_registry_handler))
.route("/api/extensions/install", post(extensions_install_handler))
.route(
"/api/extensions/{name}/activate",
@@ -223,6 +231,16 @@ pub async fn start_server(
"/api/extensions/{name}/remove",
post(extensions_remove_handler),
)
.route(
"/api/extensions/{name}/setup",
get(extensions_setup_handler).post(extensions_setup_submit_handler),
)
// Pairing
.route("/api/pairing/{channel}", get(pairing_list_handler))
.route(
"/api/pairing/{channel}/approve",
post(pairing_approve_handler),
)
// Routines
.route("/api/routines", get(routines_list_handler))
.route("/api/routines/summary", get(routines_summary_handler))
@@ -272,7 +290,8 @@ pub async fn start_server(
let statics = Router::new()
.route("/", get(index_handler))
.route("/style.css", get(css_handler))
.route("/app.js", get(js_handler));
.route("/app.js", get(js_handler))
.route("/favicon.ico", get(favicon_handler));
// Project file serving (behind auth to prevent unauthorized file access).
let projects = Router::new()
@@ -344,24 +363,46 @@ pub async fn start_server(
// --- Static file handlers ---
async fn index_handler() -> Html<&'static str> {
Html(include_str!("static/index.html"))
async fn index_handler() -> impl IntoResponse {
(
[
(header::CONTENT_TYPE, "text/html; charset=utf-8"),
(header::CACHE_CONTROL, "no-cache"),
],
include_str!("static/index.html"),
)
}
async fn css_handler() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "text/css")],
[
(header::CONTENT_TYPE, "text/css"),
(header::CACHE_CONTROL, "no-cache"),
],
include_str!("static/style.css"),
)
}
async fn js_handler() -> impl IntoResponse {
(
[(header::CONTENT_TYPE, "application/javascript")],
[
(header::CONTENT_TYPE, "application/javascript"),
(header::CACHE_CONTROL, "no-cache"),
],
include_str!("static/app.js"),
)
}
async fn favicon_handler() -> impl IntoResponse {
(
[
(header::CONTENT_TYPE, "image/x-icon"),
(header::CACHE_CONTROL, "public, max-age=86400"),
],
include_bytes!("static/favicon.ico").as_slice(),
)
}
// --- Health ---
async fn health_handler() -> Json<HealthResponse> {
@@ -1674,7 +1715,7 @@ async fn extensions_list_handler(
))?;
let installed = ext_mgr
.list(None)
.list(None, false)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -1688,6 +1729,7 @@ async fn extensions_list_handler(
authenticated: ext.authenticated,
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
})
.collect();
@@ -1718,10 +1760,30 @@ async fn extensions_install_handler(
State(state): State<Arc<GatewayState>>,
Json(req): Json<InstallExtensionRequest>,
) -> Result<Json<ActionResponse>, (StatusCode, String)> {
let ext_mgr = state.extension_manager.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Extension manager not available (secrets store required)".to_string(),
))?;
// When extension manager isn't available, check registry entries for a helpful message
let Some(ext_mgr) = state.extension_manager.as_ref() else {
// Look up the entry in the catalog to give a specific error
if let Some(entry) = state.registry_entries.iter().find(|e| e.name == req.name) {
let msg = match &entry.source {
crate::extensions::ExtensionSource::WasmBuildable { .. } => {
format!(
"'{}' requires building from source. \
Run `ironclaw registry install {}` from the CLI.",
req.name, req.name
)
}
_ => format!(
"Extension manager not available (secrets store required). \
Configure DATABASE_URL or a secrets backend to enable installation of '{}'.",
req.name
),
};
return Ok(Json(ActionResponse::fail(msg)));
}
return Ok(Json(ActionResponse::fail(
"Extension manager not available (secrets store required)".to_string(),
)));
};
let kind_hint = req.kind.as_deref().and_then(|k| match k {
"mcp_server" => Some(crate::extensions::ExtensionKind::McpServer),
@@ -1870,6 +1932,160 @@ async fn extensions_remove_handler(
}
}
async fn extensions_registry_handler(
State(state): State<Arc<GatewayState>>,
Query(params): Query<RegistrySearchQuery>,
) -> Json<RegistrySearchResponse> {
let query = params.query.unwrap_or_default();
let query_lower = query.to_lowercase();
let tokens: Vec<&str> = query_lower.split_whitespace().collect();
// Filter registry entries by query (or return all if empty)
let matching: Vec<&crate::extensions::RegistryEntry> = if tokens.is_empty() {
state.registry_entries.iter().collect()
} else {
state
.registry_entries
.iter()
.filter(|e| {
let name = e.name.to_lowercase();
let display = e.display_name.to_lowercase();
let desc = e.description.to_lowercase();
tokens.iter().any(|t| {
name.contains(t)
|| display.contains(t)
|| desc.contains(t)
|| e.keywords.iter().any(|k| k.to_lowercase().contains(t))
})
})
.collect()
};
// Cross-reference with installed extensions by (name, kind) to avoid
// false positives when the same name exists as different kinds.
let installed: std::collections::HashSet<(String, String)> =
if let Some(ext_mgr) = state.extension_manager.as_ref() {
ext_mgr
.list(None, false)
.await
.unwrap_or_default()
.into_iter()
.map(|ext| (ext.name, ext.kind.to_string()))
.collect()
} else {
std::collections::HashSet::new()
};
let entries = matching
.into_iter()
.map(|e| {
let kind_str = e.kind.to_string();
RegistryEntryInfo {
name: e.name.clone(),
display_name: e.display_name.clone(),
installed: installed.contains(&(e.name.clone(), kind_str.clone())),
kind: kind_str,
description: e.description.clone(),
keywords: e.keywords.clone(),
}
})
.collect();
Json(RegistrySearchResponse { entries })
}
async fn extensions_setup_handler(
State(state): State<Arc<GatewayState>>,
Path(name): Path<String>,
) -> Result<Json<ExtensionSetupResponse>, (StatusCode, String)> {
let ext_mgr = state.extension_manager.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Extension manager not available (secrets store required)".to_string(),
))?;
let secrets = ext_mgr
.get_setup_schema(&name)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let kind = ext_mgr
.list(None, false)
.await
.ok()
.and_then(|list| list.into_iter().find(|e| e.name == name))
.map(|e| e.kind.to_string())
.unwrap_or_default();
Ok(Json(ExtensionSetupResponse {
name,
kind,
secrets,
}))
}
async fn extensions_setup_submit_handler(
State(state): State<Arc<GatewayState>>,
Path(name): Path<String>,
Json(req): Json<ExtensionSetupRequest>,
) -> Result<Json<ActionResponse>, (StatusCode, String)> {
let ext_mgr = state.extension_manager.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Extension manager not available (secrets store required)".to_string(),
))?;
match ext_mgr.save_setup_secrets(&name, &req.secrets).await {
Ok(message) => Ok(Json(ActionResponse::ok(message))),
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Pairing handlers ---
async fn pairing_list_handler(
Path(channel): Path<String>,
) -> Result<Json<PairingListResponse>, (StatusCode, String)> {
let store = crate::pairing::PairingStore::new();
let requests = store
.list_pending(&channel)
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let infos = requests
.into_iter()
.map(|r| PairingRequestInfo {
code: r.code,
sender_id: r.id,
meta: r.meta,
created_at: r.created_at,
})
.collect();
Ok(Json(PairingListResponse {
channel,
requests: infos,
}))
}
async fn pairing_approve_handler(
Path(channel): Path<String>,
Json(req): Json<PairingApproveRequest>,
) -> Result<Json<ActionResponse>, (StatusCode, String)> {
let store = crate::pairing::PairingStore::new();
match store.approve(&channel, &req.code) {
Ok(Some(approved)) => Ok(Json(ActionResponse::ok(format!(
"Pairing approved for sender '{}'",
approved.id
)))),
Ok(None) => Ok(Json(ActionResponse::fail(
"Invalid or expired pairing code".to_string(),
))),
Err(crate::pairing::PairingStoreError::ApproveRateLimited) => Err((
StatusCode::TOO_MANY_REQUESTS,
"Too many failed approve attempts; try again later".to_string(),
)),
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Skills handlers ---
async fn skills_list_handler(
@@ -2569,18 +2785,57 @@ async fn gateway_status_handler(
.map(|t| t.connection_count())
.unwrap_or(0);
let uptime_secs = state.startup_time.elapsed().as_secs();
let (daily_cost, actions_this_hour, model_usage) = if let Some(ref cg) = state.cost_guard {
let cost = cg.daily_spend().await;
let actions = cg.actions_this_hour().await;
let usage = cg.model_usage().await;
let models: Vec<ModelUsageEntry> = usage
.into_iter()
.map(|(model, tokens)| ModelUsageEntry {
model,
input_tokens: tokens.input_tokens,
output_tokens: tokens.output_tokens,
cost: format!("{:.6}", tokens.cost),
})
.collect();
(Some(format!("{:.4}", cost)), Some(actions), Some(models))
} else {
(None, None, None)
};
Json(GatewayStatusResponse {
sse_connections,
ws_connections,
total_connections: sse_connections + ws_connections,
uptime_secs,
daily_cost,
actions_this_hour,
model_usage,
})
}
#[derive(serde::Serialize)]
struct ModelUsageEntry {
model: String,
input_tokens: u64,
output_tokens: u64,
cost: String,
}
#[derive(serde::Serialize)]
struct GatewayStatusResponse {
sse_connections: u64,
ws_connections: u64,
total_connections: u64,
uptime_secs: u64,
#[serde(skip_serializing_if = "Option::is_none")]
daily_cost: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
actions_this_hour: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
model_usage: Option<Vec<ModelUsageEntry>>,
}
#[cfg(test)]
+520 -28
View File
@@ -9,6 +9,7 @@ let assistantThreadId = null;
let hasMore = false;
let oldestTimestamp = null;
let loadingOlder = false;
let sseHasConnectedBefore = false;
let jobEvents = new Map(); // job_id -> Array of events
let jobListRefreshTimer = null;
const JOB_EVENTS_CAP = 500;
@@ -107,6 +108,10 @@ function connectSSE() {
eventSource.onopen = () => {
document.getElementById('sse-dot').classList.remove('disconnected');
document.getElementById('sse-status').textContent = 'Connected';
if (sseHasConnectedBefore && currentThreadId) {
loadHistory();
}
sseHasConnectedBefore = true;
};
eventSource.onerror = () => {
@@ -236,6 +241,11 @@ function isCurrentThread(threadId) {
function sendMessage() {
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
if (!currentThreadId) {
console.warn('sendMessage: no thread selected, ignoring');
setStatus('Waiting for thread to load...');
return;
}
const content = input.value.trim();
if (!content) return;
@@ -258,6 +268,8 @@ function sendMessage() {
}
function enableChatInput() {
// Don't re-enable until a thread is selected (prevents orphan messages)
if (!currentThreadId) return;
const input = document.getElementById('chat-input');
const sendBtn = document.getElementById('send-btn');
sendBtn.disabled = false;
@@ -735,6 +747,11 @@ function loadThreads() {
if (!currentThreadId && assistantThreadId) {
switchToAssistant();
}
// Enable chat input once a thread is available
if (currentThreadId) {
enableChatInput();
}
}).catch(() => {});
}
@@ -783,6 +800,10 @@ chatInput.addEventListener('keydown', (e) => {
});
chatInput.addEventListener('input', () => autoResizeTextarea(chatInput));
// Disable send until a thread is selected (loadThreads will enable it)
chatInput.disabled = true;
document.getElementById('send-btn').disabled = true;
// Infinite scroll: load older messages when scrolled near the top
document.getElementById('chat-messages').addEventListener('scroll', function () {
if (this.scrollTop < 100 && hasMore && !loadingOlder) {
@@ -1204,15 +1225,18 @@ function loadServerLogLevel() {
function loadExtensions() {
const extList = document.getElementById('extensions-list');
const wasmList = document.getElementById('available-wasm-list');
const mcpList = document.getElementById('mcp-servers-list');
const toolsTbody = document.getElementById('tools-tbody');
const toolsEmpty = document.getElementById('tools-empty');
// Fetch both in parallel
// Fetch all three in parallel
Promise.all([
apiFetch('/api/extensions').catch(() => ({ extensions: [] })),
apiFetch('/api/extensions/tools').catch(() => ({ tools: [] })),
]).then(([extData, toolData]) => {
// Render extensions
apiFetch('/api/extensions/registry').catch(function(err) { console.warn('registry fetch failed:', err); return { entries: [] }; }),
]).then(([extData, toolData, registryData]) => {
// Render installed extensions
if (extData.extensions.length === 0) {
extList.innerHTML = '<div class="empty-state">No extensions installed</div>';
} else {
@@ -1222,6 +1246,31 @@ function loadExtensions() {
}
}
// Split registry entries by kind
var wasmEntries = registryData.entries.filter(function(e) { return e.kind !== 'mcp_server' && !e.installed; });
var mcpEntries = registryData.entries.filter(function(e) { return e.kind === 'mcp_server'; });
// Available WASM extensions
if (wasmEntries.length === 0) {
wasmList.innerHTML = '<div class="empty-state">No additional WASM extensions available</div>';
} else {
wasmList.innerHTML = '';
for (const entry of wasmEntries) {
wasmList.appendChild(renderAvailableExtensionCard(entry));
}
}
// MCP servers (show both installed and uninstalled)
if (mcpEntries.length === 0) {
mcpList.innerHTML = '<div class="empty-state">No MCP servers available</div>';
} else {
mcpList.innerHTML = '';
for (const entry of mcpEntries) {
var installedExt = extData.extensions.find(function(e) { return e.name === entry.name; });
mcpList.appendChild(renderMcpServerCard(entry, installedExt));
}
}
// Render tools
if (toolData.tools.length === 0) {
toolsTbody.innerHTML = '';
@@ -1235,6 +1284,148 @@ function loadExtensions() {
});
}
function renderAvailableExtensionCard(entry) {
const card = document.createElement('div');
card.className = 'ext-card ext-available';
const header = document.createElement('div');
header.className = 'ext-header';
const name = document.createElement('span');
name.className = 'ext-name';
name.textContent = entry.display_name;
header.appendChild(name);
const kind = document.createElement('span');
kind.className = 'ext-kind kind-' + entry.kind;
kind.textContent = entry.kind;
header.appendChild(kind);
card.appendChild(header);
const desc = document.createElement('div');
desc.className = 'ext-desc';
desc.textContent = entry.description;
card.appendChild(desc);
if (entry.keywords && entry.keywords.length > 0) {
const kw = document.createElement('div');
kw.className = 'ext-keywords';
kw.textContent = entry.keywords.join(', ');
card.appendChild(kw);
}
const actions = document.createElement('div');
actions.className = 'ext-actions';
const installBtn = document.createElement('button');
installBtn.className = 'btn-ext install';
installBtn.textContent = 'Install';
installBtn.addEventListener('click', function() {
installBtn.disabled = true;
installBtn.textContent = 'Installing...';
apiFetch('/api/extensions/install', {
method: 'POST',
body: { name: entry.name, kind: entry.kind },
}).then(function(res) {
if (res.success) {
showToast('Installed ' + entry.display_name, 'success');
} else {
showToast('Install: ' + (res.message || 'unknown error'), 'error');
}
loadExtensions();
}).catch(function(err) {
showToast('Install failed: ' + err.message, 'error');
loadExtensions();
});
});
actions.appendChild(installBtn);
card.appendChild(actions);
return card;
}
function renderMcpServerCard(entry, installedExt) {
var card = document.createElement('div');
card.className = 'ext-card' + (installedExt ? '' : ' ext-available');
var header = document.createElement('div');
header.className = 'ext-header';
var name = document.createElement('span');
name.className = 'ext-name';
name.textContent = entry.display_name;
header.appendChild(name);
var kind = document.createElement('span');
kind.className = 'ext-kind kind-mcp_server';
kind.textContent = 'mcp_server';
header.appendChild(kind);
if (installedExt) {
var authDot = document.createElement('span');
authDot.className = 'ext-auth-dot ' + (installedExt.authenticated ? 'authed' : 'unauthed');
authDot.title = installedExt.authenticated ? 'Authenticated' : 'Not authenticated';
header.appendChild(authDot);
}
card.appendChild(header);
var desc = document.createElement('div');
desc.className = 'ext-desc';
desc.textContent = entry.description;
card.appendChild(desc);
var actions = document.createElement('div');
actions.className = 'ext-actions';
if (installedExt) {
if (!installedExt.active) {
var activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', function() { activateExtension(installedExt.name); });
actions.appendChild(activateBtn);
} else {
var activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = 'Active';
actions.appendChild(activeLabel);
}
var removeBtn = document.createElement('button');
removeBtn.className = 'btn-ext remove';
removeBtn.textContent = 'Remove';
removeBtn.addEventListener('click', function() { removeExtension(installedExt.name); });
actions.appendChild(removeBtn);
} else {
var installBtn = document.createElement('button');
installBtn.className = 'btn-ext install';
installBtn.textContent = 'Install';
installBtn.addEventListener('click', function() {
installBtn.disabled = true;
installBtn.textContent = 'Installing...';
apiFetch('/api/extensions/install', {
method: 'POST',
body: { name: entry.name, kind: entry.kind },
}).then(function(res) {
if (res.success) {
showToast('Installed ' + entry.display_name, 'success');
} else {
showToast('Install: ' + (res.message || 'unknown error'), 'error');
}
loadExtensions();
}).catch(function(err) {
showToast('Install failed: ' + err.message, 'error');
loadExtensions();
});
});
actions.appendChild(installBtn);
}
card.appendChild(actions);
return card;
}
function renderExtensionCard(ext) {
const card = document.createElement('div');
card.className = 'ext-card';
@@ -1297,6 +1488,14 @@ function renderExtensionCard(ext) {
actions.appendChild(activeLabel);
}
if (ext.needs_setup) {
const configBtn = document.createElement('button');
configBtn.className = 'btn-ext configure';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Configure';
configBtn.addEventListener('click', () => showConfigureModal(ext.name));
actions.appendChild(configBtn);
}
const removeBtn = document.createElement('button');
removeBtn.className = 'btn-ext remove';
removeBtn.textContent = 'Remove';
@@ -1304,6 +1503,17 @@ function renderExtensionCard(ext) {
actions.appendChild(removeBtn);
card.appendChild(actions);
// For WASM channels, check for pending pairing requests.
// Show even when inactive — pairing requests can arrive via webhooks
// before the channel is fully activated.
if (ext.kind === 'wasm_channel') {
const pairingSection = document.createElement('div');
pairingSection.className = 'ext-pairing';
card.appendChild(pairingSection);
loadPairingRequests(ext.name, pairingSection);
}
return card;
}
@@ -1319,7 +1529,7 @@ function activateExtension(name) {
showToast('Opening authentication for ' + name, 'info');
window.open(res.auth_url, '_blank');
} else if (res.awaiting_token) {
showToast(res.instructions || 'Please provide an API token for ' + name, 'info');
showConfigureModal(name);
} else {
showToast('Activate failed: ' + res.message, 'error');
}
@@ -1342,6 +1552,189 @@ function removeExtension(name) {
.catch((err) => showToast('Remove failed: ' + err.message, 'error'));
}
function showConfigureModal(name) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/setup')
.then((setup) => {
if (!setup.secrets || setup.secrets.length === 0) {
showToast('No configuration needed for ' + name, 'info');
return;
}
renderConfigureModal(name, setup.secrets);
})
.catch((err) => showToast('Failed to load setup: ' + err.message, 'error'));
}
function renderConfigureModal(name, secrets) {
closeConfigureModal();
const overlay = document.createElement('div');
overlay.className = 'configure-overlay';
overlay.addEventListener('click', (e) => {
if (e.target === overlay) closeConfigureModal();
});
const modal = document.createElement('div');
modal.className = 'configure-modal';
const header = document.createElement('h3');
header.textContent = 'Configure ' + name;
modal.appendChild(header);
const form = document.createElement('div');
form.className = 'configure-form';
const fields = [];
for (const secret of secrets) {
const field = document.createElement('div');
field.className = 'configure-field';
const label = document.createElement('label');
label.textContent = secret.prompt;
if (secret.optional) {
const opt = document.createElement('span');
opt.className = 'field-optional';
opt.textContent = ' (optional)';
label.appendChild(opt);
}
field.appendChild(label);
const inputRow = document.createElement('div');
inputRow.className = 'configure-input-row';
const input = document.createElement('input');
input.type = 'password';
input.name = secret.name;
input.placeholder = secret.provided ? '(already set — leave empty to keep)' : '';
input.addEventListener('keydown', (e) => {
if (e.key === 'Enter') submitConfigureModal(name, fields);
});
inputRow.appendChild(input);
if (secret.provided) {
const badge = document.createElement('span');
badge.className = 'field-provided';
badge.textContent = 'Set';
inputRow.appendChild(badge);
}
if (secret.auto_generate && !secret.provided) {
const hint = document.createElement('span');
hint.className = 'field-autogen';
hint.textContent = 'Auto-generated if empty';
inputRow.appendChild(hint);
}
field.appendChild(inputRow);
form.appendChild(field);
fields.push({ name: secret.name, input: input });
}
modal.appendChild(form);
const actions = document.createElement('div');
actions.className = 'configure-actions';
const submitBtn = document.createElement('button');
submitBtn.className = 'btn-ext activate';
submitBtn.textContent = 'Save';
submitBtn.addEventListener('click', () => submitConfigureModal(name, fields));
actions.appendChild(submitBtn);
const cancelBtn = document.createElement('button');
cancelBtn.className = 'btn-ext remove';
cancelBtn.textContent = 'Cancel';
cancelBtn.addEventListener('click', closeConfigureModal);
actions.appendChild(cancelBtn);
modal.appendChild(actions);
overlay.appendChild(modal);
document.body.appendChild(overlay);
if (fields.length > 0) fields[0].input.focus();
}
function submitConfigureModal(name, fields) {
const secrets = {};
for (const f of fields) {
if (f.input.value.trim()) {
secrets[f.name] = f.input.value.trim();
}
}
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/setup', {
method: 'POST',
body: { secrets },
})
.then((res) => {
closeConfigureModal();
if (res.success) {
showToast(res.message, 'success');
} else {
showToast(res.message || 'Configuration failed', 'error');
}
loadExtensions();
})
.catch((err) => {
showToast('Configuration failed: ' + err.message, 'error');
});
}
function closeConfigureModal() {
const existing = document.querySelector('.configure-overlay');
if (existing) existing.remove();
}
// --- Pairing ---
function loadPairingRequests(channel, container) {
apiFetch('/api/pairing/' + encodeURIComponent(channel))
.then(data => {
container.innerHTML = '';
if (!data.requests || data.requests.length === 0) return;
const heading = document.createElement('div');
heading.className = 'pairing-heading';
heading.textContent = 'Pending pairing requests';
container.appendChild(heading);
data.requests.forEach(req => {
const row = document.createElement('div');
row.className = 'pairing-row';
const code = document.createElement('span');
code.className = 'pairing-code';
code.textContent = req.code;
row.appendChild(code);
const sender = document.createElement('span');
sender.className = 'pairing-sender';
sender.textContent = 'from ' + req.sender_id;
row.appendChild(sender);
const btn = document.createElement('button');
btn.className = 'btn-ext activate';
btn.textContent = 'Approve';
btn.addEventListener('click', () => approvePairing(channel, req.code, container));
row.appendChild(btn);
container.appendChild(row);
});
})
.catch(() => {});
}
function approvePairing(channel, code, container) {
apiFetch('/api/pairing/' + encodeURIComponent(channel) + '/approve', {
method: 'POST',
body: { code },
}).then(res => {
if (res.success) {
showToast('Pairing approved', 'success');
loadPairingRequests(channel, container);
} else {
showToast(res.message || 'Approve failed', 'error');
}
}).catch(err => showToast('Error: ' + err.message, 'error'));
}
// --- Jobs ---
let currentJobId = null;
@@ -1815,14 +2208,20 @@ function appendActivityEvent(terminal, eventType, data) {
+ escapeHtml(typeof data.input === 'string' ? data.input : JSON.stringify(data.input, null, 2))
+ '</pre></details>';
break;
case 'tool_result':
el.innerHTML = '<details class="activity-tool-block activity-tool-result"><summary>'
+ '<span class="activity-tool-icon">&#10003;</span> '
case 'tool_result': {
const trSuccess = data.success !== false;
const trIcon = trSuccess ? '&#10003;' : '&#10007;';
const trOutput = data.output || data.error || '';
const trClass = 'activity-tool-block activity-tool-result'
+ (trSuccess ? '' : ' activity-tool-error');
el.innerHTML = '<details class="' + trClass + '"><summary>'
+ '<span class="activity-tool-icon">' + trIcon + '</span> '
+ escapeHtml(data.tool_name || 'result')
+ '</summary><pre class="activity-tool-output">'
+ escapeHtml(data.output || '')
+ escapeHtml(trOutput)
+ '</pre></details>';
break;
}
case 'status':
el.innerHTML = '<span class="activity-status">' + escapeHtml(data.message || '') + '</span>';
break;
@@ -1830,7 +2229,7 @@ function appendActivityEvent(terminal, eventType, data) {
el.className += ' activity-final';
const success = data.success !== false;
el.innerHTML = '<span class="activity-result-status" data-success="' + success + '">'
+ escapeHtml(data.message || data.status || 'done') + '</span>';
+ escapeHtml(data.message || data.error || data.status || 'done') + '</span>';
if (data.session_id) {
el.innerHTML += ' <span class="activity-session-id">session: ' + escapeHtml(data.session_id) + '</span>';
}
@@ -2015,7 +2414,9 @@ function renderRoutineDetail(routine) {
+ '<td>' + formatDate(run.started_at) + '</td>'
+ '<td>' + formatDate(run.completed_at) + '</td>'
+ '<td><span class="badge ' + runStatusClass + '">' + escapeHtml(run.status) + '</span></td>'
+ '<td>' + escapeHtml(run.result_summary || '-') + '</td>'
+ '<td>' + escapeHtml(run.result_summary || '-')
+ (run.job_id ? ' <a href="#" onclick="event.preventDefault(); switchTab(\'jobs\'); openJobDetail(\'' + run.job_id + '\')">[view job]</a>' : '')
+ '</td>'
+ '<td>' + (run.tokens_used != null ? run.tokens_used : '-') + '</td>'
+ '</tr>';
}
@@ -2082,13 +2483,72 @@ function startGatewayStatusPolling() {
gatewayStatusInterval = setInterval(fetchGatewayStatus, 30000);
}
function formatTokenCount(n) {
if (n == null || n === 0) return '0';
if (n >= 1000000) return (n / 1000000).toFixed(1) + 'M';
if (n >= 1000) return (n / 1000).toFixed(1) + 'k';
return '' + n;
}
function formatCost(costStr) {
if (!costStr) return '$0.00';
var n = parseFloat(costStr);
if (n < 0.01) return '$' + n.toFixed(4);
return '$' + n.toFixed(2);
}
function shortModelName(model) {
// Strip provider prefix and shorten common model names
var m = model.indexOf('/') >= 0 ? model.split('/').pop() : model;
// Shorten dated suffixes
m = m.replace(/-20\d{6}$/, '');
return m;
}
function fetchGatewayStatus() {
apiFetch('/api/gateway/status').then((data) => {
const popover = document.getElementById('gateway-popover');
popover.innerHTML = '<div class="gw-stat"><span>SSE clients</span><span>' + (data.sse_clients || 0) + '</span></div>'
+ '<div class="gw-stat"><span>Log clients</span><span>' + (data.log_clients || 0) + '</span></div>'
+ '<div class="gw-stat"><span>Uptime</span><span>' + formatDuration(data.uptime_secs) + '</span></div>';
}).catch(() => {});
apiFetch('/api/gateway/status').then(function(data) {
var popover = document.getElementById('gateway-popover');
var html = '';
// Connection info
html += '<div class="gw-section-label">Connections</div>';
html += '<div class="gw-stat"><span>SSE</span><span>' + (data.sse_connections || 0) + '</span></div>';
html += '<div class="gw-stat"><span>WebSocket</span><span>' + (data.ws_connections || 0) + '</span></div>';
html += '<div class="gw-stat"><span>Uptime</span><span>' + formatDuration(data.uptime_secs) + '</span></div>';
// Cost tracker
if (data.daily_cost != null) {
html += '<div class="gw-divider"></div>';
html += '<div class="gw-section-label">Cost Today</div>';
html += '<div class="gw-stat"><span>Spent</span><span>' + formatCost(data.daily_cost) + '</span></div>';
if (data.actions_this_hour != null) {
html += '<div class="gw-stat"><span>Actions/hr</span><span>' + data.actions_this_hour + '</span></div>';
}
}
// Per-model token usage
if (data.model_usage && data.model_usage.length > 0) {
html += '<div class="gw-divider"></div>';
html += '<div class="gw-section-label">Token Usage</div>';
data.model_usage.sort(function(a, b) {
return (b.input_tokens + b.output_tokens) - (a.input_tokens + a.output_tokens);
});
for (var i = 0; i < data.model_usage.length; i++) {
var m = data.model_usage[i];
var name = escapeHtml(shortModelName(m.model));
html += '<div class="gw-model-row">'
+ '<span class="gw-model-name">' + name + '</span>'
+ '<span class="gw-model-cost">' + escapeHtml(formatCost(m.cost)) + '</span>'
+ '</div>';
html += '<div class="gw-token-detail">'
+ '<span>in: ' + formatTokenCount(m.input_tokens) + '</span>'
+ '<span>out: ' + formatTokenCount(m.output_tokens) + '</span>'
+ '</div>';
}
}
popover.innerHTML = html;
}).catch(function() {});
}
// Show/hide popover on hover
@@ -2195,32 +2655,64 @@ document.getElementById('tee-shield').addEventListener('mouseleave', function()
// --- Extension install ---
function installExtension() {
const name = document.getElementById('ext-install-name').value.trim();
function installWasmExtension() {
var name = document.getElementById('wasm-install-name').value.trim();
if (!name) {
showToast('Extension name is required', 'error');
return;
}
const url = document.getElementById('ext-install-url').value.trim();
const kind = document.getElementById('ext-install-kind').value;
var url = document.getElementById('wasm-install-url').value.trim();
if (!url) {
showToast('URL to .tar.gz bundle is required', 'error');
return;
}
apiFetch('/api/extensions/install', {
method: 'POST',
body: { name, url: url || undefined, kind },
}).then((res) => {
body: { name: name, url: url, kind: 'wasm_tool' },
}).then(function(res) {
if (res.success) {
showToast('Installed ' + name, 'success');
document.getElementById('ext-install-name').value = '';
document.getElementById('ext-install-url').value = '';
document.getElementById('wasm-install-name').value = '';
document.getElementById('wasm-install-url').value = '';
loadExtensions();
} else {
showToast('Install failed: ' + (res.message || 'unknown error'), 'error');
}
}).catch((err) => {
}).catch(function(err) {
showToast('Install failed: ' + err.message, 'error');
});
}
function addMcpServer() {
var name = document.getElementById('mcp-install-name').value.trim();
if (!name) {
showToast('Server name is required', 'error');
return;
}
var url = document.getElementById('mcp-install-url').value.trim();
if (!url) {
showToast('MCP server URL is required', 'error');
return;
}
apiFetch('/api/extensions/install', {
method: 'POST',
body: { name: name, url: url, kind: 'mcp_server' },
}).then(function(res) {
if (res.success) {
showToast('Added MCP server ' + name, 'success');
document.getElementById('mcp-install-name').value = '';
document.getElementById('mcp-install-url').value = '';
loadExtensions();
} else {
showToast('Failed to add MCP server: ' + (res.message || 'unknown error'), 'error');
}
}).catch(function(err) {
showToast('Failed to add MCP server: ' + err.message, 'error');
});
}
// --- Keyboard shortcuts ---
document.addEventListener('keydown', (e) => {
@@ -2228,10 +2720,10 @@ document.addEventListener('keydown', (e) => {
const tag = (e.target.tagName || '').toLowerCase();
const inInput = tag === 'input' || tag === 'textarea';
// Mod+1-6: switch tabs
if (mod && e.key >= '1' && e.key <= '6') {
// Mod+1-5: switch tabs
if (mod && e.key >= '1' && e.key <= '5') {
e.preventDefault();
const tabs = ['chat', 'memory', 'jobs', 'routines', 'logs', 'extensions'];
const tabs = ['chat', 'memory', 'jobs', 'routines', 'extensions'];
const idx = parseInt(e.key) - 1;
if (tabs[idx]) switchTab(tabs[idx]);
return;
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

+32 -20
View File
@@ -4,6 +4,10 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>IronClaw</title>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;600;700&family=IBM+Plex+Mono:wght@400;500;600&display=swap" rel="stylesheet">
<link rel="stylesheet" href="/style.css">
<script
src="https://cdn.jsdelivr.net/npm/[email protected]/lib/marked.umd.min.js"
@@ -36,10 +40,10 @@
<button class="active" data-tab="chat">Chat</button>
<button data-tab="memory">Memory</button>
<button data-tab="jobs">Jobs</button>
<button data-tab="logs">Logs</button>
<button data-tab="routines">Routines</button>
<button data-tab="extensions">Extensions</button>
<div class="spacer"></div>
<button class="status-logs-btn" data-tab="logs" title="Logs">Logs</button>
<div class="tee-shield" id="tee-shield" style="display:none" title="Running in a Trusted Execution Environment">
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10z"/>
@@ -185,34 +189,42 @@
<!-- Extensions Tab -->
<div class="tab-panel" id="tab-extensions">
<div class="extensions-container">
<div class="extensions-section">
<h3>Install Extension</h3>
<div class="ext-install-form" id="ext-install-form">
<input type="text" id="ext-install-name" placeholder="Extension name (required)">
<input type="text" id="ext-install-url" placeholder="URL (optional)">
<select id="ext-install-kind">
<option value="mcp_server">MCP Server</option>
<option value="wasm_tool">WASM Tool</option>
<option value="wasm_channel">WASM Channel</option>
</select>
<button onclick="installExtension()">Install</button>
</div>
</div>
<div class="extensions-section">
<h3>Installed Extensions</h3>
<div class="extensions-list" id="extensions-list">
<div class="empty-state">Loading extensions...</div>
</div>
</div>
<div class="extensions-section" id="available-wasm-section">
<h3>Available WASM Extensions</h3>
<div class="extensions-list" id="available-wasm-list">
<div class="empty-state">Loading...</div>
</div>
</div>
<div class="extensions-section">
<h3>Install WASM Extension</h3>
<div class="ext-install-form">
<input type="text" id="wasm-install-name" placeholder="Extension name">
<input type="text" id="wasm-install-url" placeholder="URL to .tar.gz bundle">
<button onclick="installWasmExtension()">Install</button>
</div>
</div>
<div class="extensions-section">
<h3>MCP Servers</h3>
<div class="extensions-list" id="mcp-servers-list">
<div class="empty-state">Loading...</div>
</div>
<h4>Add Custom MCP Server</h4>
<div class="ext-install-form">
<input type="text" id="mcp-install-name" placeholder="Server name">
<input type="text" id="mcp-install-url" placeholder="MCP server URL (https://...)">
<button onclick="addMcpServer()">Add</button>
</div>
</div>
<div class="extensions-section">
<h3>Registered Tools</h3>
<table class="tools-table" id="tools-table">
<thead>
<tr>
<th>Name</th>
<th>Description</th>
</tr>
</thead>
<thead><tr><th>Name</th><th>Description</th></tr></thead>
<tbody id="tools-tbody"></tbody>
</table>
<div class="empty-state" id="tools-empty" style="display:none">No tools registered</div>
File diff suppressed because it is too large Load Diff
+72
View File
@@ -346,6 +346,9 @@ pub struct ExtensionInfo {
pub authenticated: bool,
pub active: bool,
pub tools: Vec<String>,
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
}
#[derive(Debug, Serialize)]
@@ -371,6 +374,31 @@ pub struct InstallExtensionRequest {
pub kind: Option<String>,
}
// --- Extension Setup ---
#[derive(Debug, Serialize)]
pub struct ExtensionSetupResponse {
pub name: String,
pub kind: String,
pub secrets: Vec<SecretFieldInfo>,
}
#[derive(Debug, Serialize)]
pub struct SecretFieldInfo {
pub name: String,
pub prompt: String,
pub optional: bool,
/// Whether this secret is already stored.
pub provided: bool,
/// Whether the secret will be auto-generated if left empty.
pub auto_generate: bool,
}
#[derive(Debug, Deserialize)]
pub struct ExtensionSetupRequest {
pub secrets: std::collections::HashMap<String, String>,
}
#[derive(Debug, Serialize)]
pub struct ActionResponse {
pub success: bool,
@@ -408,6 +436,50 @@ impl ActionResponse {
}
}
// --- Registry ---
#[derive(Debug, Serialize)]
pub struct RegistryEntryInfo {
pub name: String,
pub display_name: String,
pub kind: String,
pub description: String,
pub keywords: Vec<String>,
pub installed: bool,
}
#[derive(Debug, Serialize)]
pub struct RegistrySearchResponse {
pub entries: Vec<RegistryEntryInfo>,
}
#[derive(Debug, Deserialize)]
pub struct RegistrySearchQuery {
pub query: Option<String>,
}
// --- Pairing ---
#[derive(Debug, Serialize)]
pub struct PairingListResponse {
pub channel: String,
pub requests: Vec<PairingRequestInfo>,
}
#[derive(Debug, Serialize)]
pub struct PairingRequestInfo {
pub code: String,
pub sender_id: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub meta: Option<serde_json::Value>,
pub created_at: String,
}
#[derive(Debug, Deserialize)]
pub struct PairingApproveRequest {
pub code: String,
}
// --- Skills ---
#[derive(Debug, Serialize)]
+3
View File
@@ -490,6 +490,9 @@ mod tests {
skill_registry: None,
skill_catalog: None,
chat_rate_limiter: crate::channels::web::server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
}
}
}
+145 -31
View File
@@ -66,12 +66,47 @@ pub const OAUTH_CALLBACK_PORT: u16 = 9876;
///
/// Checks `IRONCLAW_OAUTH_CALLBACK_URL` env var first (useful for remote/VPS
/// deployments where `127.0.0.1` is unreachable from the user's browser),
/// then falls back to `http://127.0.0.1:{OAUTH_CALLBACK_PORT}`.
/// then falls back to `http://{callback_host()}:{OAUTH_CALLBACK_PORT}`.
pub fn callback_url() -> String {
std::env::var("IRONCLAW_OAUTH_CALLBACK_URL")
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| format!("http://127.0.0.1:{}", OAUTH_CALLBACK_PORT))
.unwrap_or_else(|| format!("http://{}:{}", callback_host(), OAUTH_CALLBACK_PORT))
}
/// Returns the hostname used in OAuth callback URLs.
///
/// Reads `OAUTH_CALLBACK_HOST` from the environment (default: `127.0.0.1`).
///
/// **Remote server usage:** set `OAUTH_CALLBACK_HOST` to the network interface
/// address you want to listen on (e.g. the server's LAN IP or `0.0.0.0`).
/// The callback listener will bind to that specific address instead of the
/// loopback interface, so the OAuth redirect can reach an external browser.
/// Note: this transmits the session token over plain HTTP — prefer SSH port
/// forwarding (`ssh -L 9876:127.0.0.1:9876 user@host`) when possible.
///
/// # Example
///
/// ```bash
/// export OAUTH_CALLBACK_HOST=203.0.113.10
/// ironclaw login
/// # Opens: http://203.0.113.10:9876/auth/callback
/// ```
pub fn callback_host() -> String {
std::env::var("OAUTH_CALLBACK_HOST").unwrap_or_else(|_| "127.0.0.1".to_string())
}
/// Returns `true` if `host` is a loopback address that only accepts local connections.
///
/// Covers `localhost` (case-insensitive), the full `127.0.0.0/8` IPv4 loopback
/// range, and `::1` for IPv6.
pub fn is_loopback_host(host: &str) -> bool {
if host.eq_ignore_ascii_case("localhost") {
return true;
}
host.parse::<std::net::IpAddr>()
.map(|ip| ip.is_loopback())
.unwrap_or(false)
}
/// Error from the OAuth callback listener.
@@ -90,35 +125,50 @@ pub enum OAuthCallbackError {
Io(String),
}
/// Map a `std::io::Error` from a bind attempt to an `OAuthCallbackError`.
fn bind_error(e: std::io::Error) -> OAuthCallbackError {
if e.kind() == std::io::ErrorKind::AddrInUse {
OAuthCallbackError::PortInUse(OAUTH_CALLBACK_PORT, e.to_string())
} else {
OAuthCallbackError::Io(e.to_string())
}
}
/// Bind the OAuth callback listener on the fixed port.
///
/// Binds to IPv4 `127.0.0.1` first because callback URLs use `127.0.0.1`
/// explicitly (e.g., NEAR AI redirects to `http://127.0.0.1:9876/auth/callback`).
/// Falls back to IPv6 `[::1]` only if IPv4 binding fails for a reason other
/// than `AddrInUse`. If the port is already occupied, fails immediately.
/// When `OAUTH_CALLBACK_HOST` is a loopback address (the default `127.0.0.1`),
/// binds to `127.0.0.1` first and falls back to `[::1]` so local-only auth
/// flows remain restricted to the local machine.
///
/// When `OAUTH_CALLBACK_HOST` is set to a remote address, binds to that
/// specific address so only connections directed to it are accepted.
pub async fn bind_callback_listener() -> Result<TcpListener, OAuthCallbackError> {
let ipv4_addr = format!("127.0.0.1:{}", OAUTH_CALLBACK_PORT);
match TcpListener::bind(&ipv4_addr).await {
Ok(listener) => return Ok(listener),
Err(e) if e.kind() == std::io::ErrorKind::AddrInUse => {
return Err(OAuthCallbackError::PortInUse(
OAUTH_CALLBACK_PORT,
e.to_string(),
));
}
Err(_) => {
// IPv4 not available, fall back to IPv6
}
}
TcpListener::bind(format!("[::1]:{}", OAUTH_CALLBACK_PORT))
.await
.map_err(|e| {
if e.kind() == std::io::ErrorKind::AddrInUse {
OAuthCallbackError::PortInUse(OAUTH_CALLBACK_PORT, e.to_string())
} else {
OAuthCallbackError::Io(e.to_string())
let host = callback_host();
if is_loopback_host(&host) {
// Local mode: prefer IPv4 loopback, fall back to IPv6.
let ipv4_addr = format!("127.0.0.1:{}", OAUTH_CALLBACK_PORT);
match TcpListener::bind(&ipv4_addr).await {
Ok(listener) => return Ok(listener),
Err(e) if e.kind() == std::io::ErrorKind::AddrInUse => {
return Err(OAuthCallbackError::PortInUse(
OAUTH_CALLBACK_PORT,
e.to_string(),
));
}
})
Err(_) => {
// IPv4 not available, fall back to IPv6
}
}
TcpListener::bind(format!("[::1]:{}", OAUTH_CALLBACK_PORT))
.await
.map_err(bind_error)
} else {
// Remote mode: bind to the specific configured host address only,
// not 0.0.0.0, to limit exposure to the intended interface.
let addr = format!("{}:{}", host, OAUTH_CALLBACK_PORT);
TcpListener::bind(&addr).await.map_err(bind_error)
}
}
/// Wait for an OAuth callback and extract a query parameter value.
@@ -311,27 +361,91 @@ pub fn landing_html(provider_name: &str, success: bool) -> String {
mod tests {
use std::sync::Mutex;
use crate::cli::oauth_defaults::{builtin_credentials, callback_url, landing_html};
use crate::cli::oauth_defaults::{
builtin_credentials, callback_host, callback_url, is_loopback_host, landing_html,
};
/// Serializes env-mutating tests to prevent parallel races.
static ENV_MUTEX: Mutex<()> = Mutex::new(());
#[test]
fn test_is_loopback_host() {
assert!(is_loopback_host("127.0.0.1"));
assert!(is_loopback_host("127.0.0.2")); // full 127.0.0.0/8 range
assert!(is_loopback_host("127.255.255.254"));
assert!(is_loopback_host("::1"));
assert!(is_loopback_host("localhost"));
assert!(is_loopback_host("LOCALHOST"));
assert!(!is_loopback_host("203.0.113.10"));
assert!(!is_loopback_host("my-server.example.com"));
assert!(!is_loopback_host("0.0.0.0"));
}
#[test]
fn test_callback_host_default() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("OAUTH_CALLBACK_HOST").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("OAUTH_CALLBACK_HOST");
}
assert_eq!(callback_host(), "127.0.0.1");
// Restore
unsafe {
if let Some(val) = original {
std::env::set_var("OAUTH_CALLBACK_HOST", val);
}
}
}
#[test]
fn test_callback_host_env_override() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original_host = std::env::var("OAUTH_CALLBACK_HOST").ok();
let original_url = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("OAUTH_CALLBACK_HOST", "203.0.113.10");
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
assert_eq!(callback_host(), "203.0.113.10");
// callback_url() fallback should incorporate the custom host
let url = callback_url();
assert!(url.contains("203.0.113.10"), "url was: {url}");
// Restore
unsafe {
if let Some(val) = original_host {
std::env::set_var("OAUTH_CALLBACK_HOST", val);
} else {
std::env::remove_var("OAUTH_CALLBACK_HOST");
}
if let Some(val) = original_url {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
}
}
}
#[test]
fn test_callback_url_default() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
// Clear the env var to test default behavior
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// Clear both env vars to test default behavior
let original_url = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
let original_host = std::env::var("OAUTH_CALLBACK_HOST").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
std::env::remove_var("OAUTH_CALLBACK_HOST");
}
let url = callback_url();
assert_eq!(url, "http://127.0.0.1:9876");
// Restore
unsafe {
if let Some(val) = original {
if let Some(val) = original_url {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
}
if let Some(val) = original_host {
std::env::set_var("OAUTH_CALLBACK_HOST", val);
}
}
}
+17 -51
View File
@@ -1,7 +1,5 @@
//! Registry CLI commands for discovering and installing extensions.
use std::path::PathBuf;
use clap::Subcommand;
use crate::registry::catalog::RegistryCatalog;
@@ -59,8 +57,20 @@ pub enum RegistryCommand {
/// Run a registry command.
pub async fn run_registry_command(cmd: RegistryCommand) -> anyhow::Result<()> {
let registry_dir = find_registry_dir()?;
let catalog = RegistryCatalog::load(&registry_dir)?;
// For install commands that need to build from source, a disk registry is required.
// For list/info, embedded manifests suffice.
let registry_dir = RegistryCatalog::find_dir();
let catalog = if let Some(ref dir) = registry_dir {
RegistryCatalog::load(dir)?
} else {
RegistryCatalog::load_or_embedded()?
};
// Resolve repo root for installer (empty path when running from binary)
let repo_root = registry_dir
.as_ref()
.and_then(|d| d.parent().map(|p| p.to_path_buf()))
.unwrap_or_default();
match cmd {
RegistryCommand::List { kind, tag, verbose } => {
@@ -68,53 +78,14 @@ pub async fn run_registry_command(cmd: RegistryCommand) -> anyhow::Result<()> {
}
RegistryCommand::Info { name } => cmd_info(&catalog, &name),
RegistryCommand::Install { name, force, build } => {
cmd_install(&catalog, &registry_dir, &name, force, build).await
cmd_install(&catalog, &repo_root, &name, force, build).await
}
RegistryCommand::InstallDefaults { force, build } => {
cmd_install(&catalog, &registry_dir, "default", force, build).await
cmd_install(&catalog, &repo_root, "default", force, build).await
}
}
}
/// Find the registry directory by looking relative to the current executable or cwd.
fn find_registry_dir() -> anyhow::Result<PathBuf> {
// Try relative to current directory (for dev usage)
let cwd = std::env::current_dir()?;
let candidate = cwd.join("registry");
if candidate.is_dir() {
return Ok(candidate);
}
// Try relative to executable (covers installed binary, target/debug/, target/release/)
if let Ok(exe) = std::env::current_exe()
&& let Some(parent) = exe.parent()
{
// Walk up to 3 levels: exe dir, parent (target/release → target), grandparent (→ repo root)
let mut dir = Some(parent);
for _ in 0..3 {
if let Some(d) = dir {
let candidate = d.join("registry");
if candidate.is_dir() {
return Ok(candidate);
}
dir = d.parent();
}
}
}
// Try CARGO_MANIFEST_DIR (compile-time, works in dev builds)
let manifest_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"));
let candidate = manifest_dir.join("registry");
if candidate.is_dir() {
return Ok(candidate);
}
anyhow::bail!(
"Could not find registry/ directory. Run from the ironclaw repo root, \
or ensure registry/ is next to the ironclaw binary."
)
}
fn cmd_list(
catalog: &RegistryCatalog,
kind: Option<&str>,
@@ -254,16 +225,11 @@ fn cmd_info(catalog: &RegistryCatalog, name: &str) -> anyhow::Result<()> {
async fn cmd_install(
catalog: &RegistryCatalog,
registry_dir: &std::path::Path,
repo_root: &std::path::Path,
name: &str,
force: bool,
prefer_build: bool,
) -> anyhow::Result<()> {
// Registry dir parent is the repo root
let repo_root = registry_dir
.parent()
.ok_or_else(|| anyhow::anyhow!("Cannot determine repo root from registry dir"))?;
let installer = RegistryInstaller::with_defaults(repo_root.to_path_buf());
let (manifests, bundle) = catalog.resolve(name)?;
+9 -166
View File
@@ -4,7 +4,6 @@
use std::io::Write;
use std::path::{Path, PathBuf};
use std::process::Command as ProcessCommand;
use std::sync::Arc;
use clap::Subcommand;
@@ -155,11 +154,18 @@ async fn install_tool(
};
// Build the WASM component if not skipping
let profile = if release { "release" } else { "debug" };
let wasm_path = if skip_build {
// Look for existing wasm file
find_wasm_artifact(&path, &tool_name, release)?
crate::registry::artifacts::find_wasm_artifact(&path, &tool_name, profile)
.or_else(|| crate::registry::artifacts::find_any_wasm_artifact(&path, profile))
.ok_or_else(|| {
anyhow::anyhow!(
"No .wasm artifact found. Run without --skip-build to build first."
)
})?
} else {
build_wasm_component(&path, release)?
crate::registry::artifacts::build_wasm_component_sync(&path, release)?
};
// Look for capabilities file
@@ -253,169 +259,6 @@ async fn install_tool(
Ok(())
}
/// Build a WASM component using cargo-component.
fn build_wasm_component(source_dir: &Path, release: bool) -> anyhow::Result<PathBuf> {
println!("Building WASM component in {}...", source_dir.display());
// Check if cargo-component is available
let check = ProcessCommand::new("cargo")
.args(["component", "--version"])
.output();
if check.is_err() || !check.unwrap().status.success() {
anyhow::bail!(
"cargo-component not found. Install with: cargo install cargo-component\n\
Or use --skip-build with an existing .wasm file."
);
}
// Build command
let mut cmd = ProcessCommand::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
println!(
" Running: cargo component build{}",
if release { " --release" } else { "" }
);
let output = cmd.output()?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("Build failed:\n{}", stderr);
}
// Find the output wasm file
// cargo-component may output to wasm32-wasip1 or wasm32-wasip2 depending on version
let profile = if release { "release" } else { "debug" };
let candidates = [
source_dir
.join("target")
.join("wasm32-wasip1")
.join(profile),
source_dir
.join("target")
.join("wasm32-wasip2")
.join(profile),
source_dir
.join("target")
.join("wasm32-unknown-unknown")
.join(profile),
];
let target_dir = candidates.iter().find(|p| p.exists()).ok_or_else(|| {
anyhow::anyhow!(
"No WASM target directory found. Expected one of: {}",
candidates
.iter()
.map(|p| p.display().to_string())
.collect::<Vec<_>>()
.join(", ")
)
})?;
// Look for .wasm files in target dir
let entries: Vec<_> = std::fs::read_dir(target_dir)?
.filter_map(|e| e.ok())
.filter(|e| {
e.path()
.extension()
.map(|ext| ext == "wasm")
.unwrap_or(false)
})
.collect();
if entries.is_empty() {
anyhow::bail!(
"No .wasm file found in {}. Build may have failed.",
target_dir.display()
);
}
if entries.len() > 1 {
println!(
" Warning: Multiple .wasm files found, using first: {}",
entries[0].path().display()
);
}
let wasm_path = entries[0].path();
println!(" Built: {}", wasm_path.display());
Ok(wasm_path)
}
/// Find an existing WASM artifact without building.
fn find_wasm_artifact(source_dir: &Path, name: &str, release: bool) -> anyhow::Result<PathBuf> {
let profile = if release { "release" } else { "debug" };
// cargo-component may output to wasm32-wasip1 or wasm32-wasip2 depending on version
let target_dirs = [
source_dir
.join("target")
.join("wasm32-wasip1")
.join(profile),
source_dir
.join("target")
.join("wasm32-wasip2")
.join(profile),
source_dir
.join("target")
.join("wasm32-unknown-unknown")
.join(profile),
];
let snake_name = name.replace('-', "_");
// Try exact name match in any target dir first
for target_dir in &target_dirs {
let candidates = [
target_dir.join(format!("{}.wasm", name)),
target_dir.join(format!("{}.wasm", snake_name)),
];
for candidate in &candidates {
if candidate.exists() {
return Ok(candidate.clone());
}
}
}
// Find a target dir that exists
let target_dir = target_dirs.iter().find(|p| p.exists()).ok_or_else(|| {
anyhow::anyhow!("No target directory found. Run without --skip-build to build first.")
})?;
// Fall back to any .wasm file
let entries: Vec<_> = std::fs::read_dir(target_dir)
.map_err(|_| {
anyhow::anyhow!(
"Target directory not found: {}. Run without --skip-build.",
target_dir.display()
)
})?
.filter_map(|e| e.ok())
.filter(|e| {
e.path()
.extension()
.map(|ext| ext == "wasm")
.unwrap_or(false)
})
.collect();
if entries.is_empty() {
anyhow::bail!(
"No .wasm file found in {}. Build the project first or remove --skip-build.",
target_dir.display()
);
}
Ok(entries[0].path())
}
/// Extract crate name from Cargo.toml.
async fn extract_crate_name(cargo_toml: &Path) -> anyhow::Result<String> {
let content = fs::read_to_string(cargo_toml).await?;
+38 -104
View File
@@ -1,6 +1,6 @@
use std::time::Duration;
use crate::config::helpers::optional_env;
use crate::config::helpers::{parse_bool_env, parse_option_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -32,109 +32,43 @@ pub struct AgentConfig {
impl AgentConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
name: optional_env("AGENT_NAME")?.unwrap_or_else(|| settings.agent.name.clone()),
max_parallel_jobs: optional_env("AGENT_MAX_PARALLEL_JOBS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_MAX_PARALLEL_JOBS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_parallel_jobs as usize),
job_timeout: Duration::from_secs(
optional_env("AGENT_JOB_TIMEOUT_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_JOB_TIMEOUT_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.job_timeout_secs),
),
stuck_threshold: Duration::from_secs(
optional_env("AGENT_STUCK_THRESHOLD_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_STUCK_THRESHOLD_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.stuck_threshold_secs),
),
repair_check_interval: Duration::from_secs(
optional_env("SELF_REPAIR_CHECK_INTERVAL_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SELF_REPAIR_CHECK_INTERVAL_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.repair_check_interval_secs),
),
max_repair_attempts: optional_env("SELF_REPAIR_MAX_ATTEMPTS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SELF_REPAIR_MAX_ATTEMPTS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_repair_attempts),
use_planning: optional_env("AGENT_USE_PLANNING")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_USE_PLANNING".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.agent.use_planning),
session_idle_timeout: Duration::from_secs(
optional_env("SESSION_IDLE_TIMEOUT_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SESSION_IDLE_TIMEOUT_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.session_idle_timeout_secs),
),
allow_local_tools: optional_env("ALLOW_LOCAL_TOOLS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "ALLOW_LOCAL_TOOLS".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(false),
max_cost_per_day_cents: optional_env("MAX_COST_PER_DAY_CENTS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MAX_COST_PER_DAY_CENTS".to_string(),
message: format!("must be a positive integer: {e}"),
})?,
max_actions_per_hour: optional_env("MAX_ACTIONS_PER_HOUR")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MAX_ACTIONS_PER_HOUR".to_string(),
message: format!("must be a positive integer: {e}"),
})?,
max_tool_iterations: optional_env("AGENT_MAX_TOOL_ITERATIONS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_MAX_TOOL_ITERATIONS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.agent.max_tool_iterations),
auto_approve_tools: optional_env("AGENT_AUTO_APPROVE_TOOLS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "AGENT_AUTO_APPROVE_TOOLS".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.agent.auto_approve_tools),
name: parse_optional_env("AGENT_NAME", settings.agent.name.clone())?,
max_parallel_jobs: parse_optional_env(
"AGENT_MAX_PARALLEL_JOBS",
settings.agent.max_parallel_jobs as usize,
)?,
job_timeout: Duration::from_secs(parse_optional_env(
"AGENT_JOB_TIMEOUT_SECS",
settings.agent.job_timeout_secs,
)?),
stuck_threshold: Duration::from_secs(parse_optional_env(
"AGENT_STUCK_THRESHOLD_SECS",
settings.agent.stuck_threshold_secs,
)?),
repair_check_interval: Duration::from_secs(parse_optional_env(
"SELF_REPAIR_CHECK_INTERVAL_SECS",
settings.agent.repair_check_interval_secs,
)?),
max_repair_attempts: parse_optional_env(
"SELF_REPAIR_MAX_ATTEMPTS",
settings.agent.max_repair_attempts,
)?,
use_planning: parse_bool_env("AGENT_USE_PLANNING", settings.agent.use_planning)?,
session_idle_timeout: Duration::from_secs(parse_optional_env(
"SESSION_IDLE_TIMEOUT_SECS",
settings.agent.session_idle_timeout_secs,
)?),
allow_local_tools: parse_bool_env("ALLOW_LOCAL_TOOLS", false)?,
max_cost_per_day_cents: parse_option_env("MAX_COST_PER_DAY_CENTS")?,
max_actions_per_hour: parse_option_env("MAX_ACTIONS_PER_HOUR")?,
max_tool_iterations: parse_optional_env(
"AGENT_MAX_TOOL_ITERATIONS",
settings.agent.max_tool_iterations,
)?,
auto_approve_tools: parse_bool_env(
"AGENT_AUTO_APPROVE_TOOLS",
settings.agent.auto_approve_tools,
)?,
})
}
}
+3 -17
View File
@@ -1,7 +1,7 @@
use std::path::PathBuf;
use std::time::Duration;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Builder mode configuration.
@@ -34,25 +34,11 @@ impl Default for BuilderModeConfig {
impl BuilderModeConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("BUILDER_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "BUILDER_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("BUILDER_ENABLED", true)?,
build_dir: optional_env("BUILDER_DIR")?.map(PathBuf::from),
max_iterations: parse_optional_env("BUILDER_MAX_ITERATIONS", 20)?,
timeout_secs: parse_optional_env("BUILDER_TIMEOUT_SECS", 600)?,
auto_register: optional_env("BUILDER_AUTO_REGISTER")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "BUILDER_AUTO_REGISTER".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
auto_register: parse_bool_env("BUILDER_AUTO_REGISTER", true)?,
})
}
+7 -30
View File
@@ -2,7 +2,7 @@ use std::path::PathBuf;
use secrecy::SecretString;
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -48,14 +48,7 @@ impl ChannelsConfig {
let http = if optional_env("HTTP_PORT")?.is_some() || optional_env("HTTP_HOST")?.is_some() {
Some(HttpConfig {
host: optional_env("HTTP_HOST")?.unwrap_or_else(|| "0.0.0.0".to_string()),
port: optional_env("HTTP_PORT")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HTTP_PORT".to_string(),
message: format!("must be a valid port number: {e}"),
})?
.unwrap_or(8080),
port: parse_optional_env("HTTP_PORT", 8080)?,
webhook_secret: optional_env("HTTP_WEBHOOK_SECRET")?.map(SecretString::from),
user_id: optional_env("HTTP_USER_ID")?.unwrap_or_else(|| "http".to_string()),
})
@@ -63,20 +56,11 @@ impl ChannelsConfig {
None
};
let gateway = if optional_env("GATEWAY_ENABLED")?
.map(|s| s.to_lowercase() == "true" || s == "1")
.unwrap_or(true)
{
let gateway_enabled = parse_bool_env("GATEWAY_ENABLED", true)?;
let gateway = if gateway_enabled {
Some(GatewayConfig {
host: optional_env("GATEWAY_HOST")?.unwrap_or_else(|| "127.0.0.1".to_string()),
port: optional_env("GATEWAY_PORT")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "GATEWAY_PORT".to_string(),
message: format!("must be a valid port number: {e}"),
})?
.unwrap_or(3000),
port: parse_optional_env("GATEWAY_PORT", 3000)?,
auth_token: optional_env("GATEWAY_AUTH_TOKEN")?,
user_id: optional_env("GATEWAY_USER_ID")?.unwrap_or_else(|| "default".to_string()),
})
@@ -97,18 +81,11 @@ impl ChannelsConfig {
wasm_channels_dir: optional_env("WASM_CHANNELS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
wasm_channels_enabled: optional_env("WASM_CHANNELS_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_CHANNELS_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
wasm_channels_enabled: parse_bool_env("WASM_CHANNELS_ENABLED", true)?,
telegram_owner_id: optional_env("TELEGRAM_OWNER_ID")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
.map_err(|e: std::num::ParseIntError| ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
})?
+67 -17
View File
@@ -1,8 +1,12 @@
use std::sync::Arc;
use secrecy::{ExposeSecret, SecretString};
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::llm::SessionManager;
use crate::settings::Settings;
use crate::workspace::EmbeddingProvider;
/// Embeddings provider configuration.
#[derive(Debug, Clone)]
@@ -65,23 +69,10 @@ impl EmbeddingsConfig {
.or_else(|| settings.ollama_base_url.clone())
.unwrap_or_else(|| "http://localhost:11434".to_string());
let dimension = optional_env("EMBEDDING_DIMENSION")?
.map(|s| s.parse::<usize>())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "EMBEDDING_DIMENSION".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or_else(|| default_dimension_for_model(&model));
let dimension =
parse_optional_env("EMBEDDING_DIMENSION", default_dimension_for_model(&model))?;
let enabled = optional_env("EMBEDDING_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "EMBEDDING_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.embeddings.enabled);
let enabled = parse_bool_env("EMBEDDING_ENABLED", settings.embeddings.enabled)?;
Ok(Self {
enabled,
@@ -97,6 +88,65 @@ impl EmbeddingsConfig {
pub fn openai_api_key(&self) -> Option<&str> {
self.openai_api_key.as_ref().map(|s| s.expose_secret())
}
/// Create the appropriate embedding provider based on configuration.
///
/// Returns `None` if embeddings are disabled or the required credentials
/// are missing. The `nearai_base_url` and `session` are needed only for
/// the NEAR AI provider but must be passed unconditionally.
pub fn create_provider(
&self,
nearai_base_url: &str,
session: Arc<SessionManager>,
) -> Option<Arc<dyn EmbeddingProvider>> {
if !self.enabled {
tracing::info!("Embeddings disabled (set EMBEDDING_ENABLED=true to enable)");
return None;
}
match self.provider.as_str() {
"nearai" => {
tracing::info!(
"Embeddings enabled via NEAR AI (model: {}, dim: {})",
self.model,
self.dimension,
);
Some(Arc::new(
crate::workspace::NearAiEmbeddings::new(nearai_base_url, session)
.with_model(&self.model, self.dimension),
))
}
"ollama" => {
tracing::info!(
"Embeddings enabled via Ollama (model: {}, url: {}, dim: {})",
self.model,
self.ollama_base_url,
self.dimension,
);
Some(Arc::new(
crate::workspace::OllamaEmbeddings::new(&self.ollama_base_url)
.with_model(&self.model, self.dimension),
))
}
_ => {
if let Some(api_key) = self.openai_api_key() {
tracing::info!(
"Embeddings enabled via OpenAI (model: {}, dim: {})",
self.model,
self.dimension,
);
Some(Arc::new(crate::workspace::OpenAiEmbeddings::with_model(
api_key,
&self.model,
self.dimension,
)))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
}
}
}
}
}
#[cfg(test)]
+6 -17
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::optional_env;
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -29,22 +29,11 @@ impl Default for HeartbeatConfig {
impl HeartbeatConfig {
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("HEARTBEAT_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HEARTBEAT_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(settings.heartbeat.enabled),
interval_secs: optional_env("HEARTBEAT_INTERVAL_SECS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "HEARTBEAT_INTERVAL_SECS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(settings.heartbeat.interval_secs),
enabled: parse_bool_env("HEARTBEAT_ENABLED", settings.heartbeat.enabled)?,
interval_secs: parse_optional_env(
"HEARTBEAT_INTERVAL_SECS",
settings.heartbeat.interval_secs,
)?,
notify_channel: optional_env("HEARTBEAT_NOTIFY_CHANNEL")?
.or_else(|| settings.heartbeat.notify_channel.clone()),
notify_user: optional_env("HEARTBEAT_NOTIFY_USER")?
+42
View File
@@ -47,3 +47,45 @@ where
.transpose()
.map(|opt| opt.unwrap_or(default))
}
/// Parse a boolean from an env var with a default.
///
/// Accepts "true"/"1" as true, "false"/"0" as false.
pub(crate) fn parse_bool_env(key: &str, default: bool) -> Result<bool, ConfigError> {
match optional_env(key)? {
Some(s) => match s.to_lowercase().as_str() {
"true" | "1" => Ok(true),
"false" | "0" => Ok(false),
_ => Err(ConfigError::InvalidValue {
key: key.to_string(),
message: format!("must be 'true' or 'false', got '{s}'"),
}),
},
None => Ok(default),
}
}
/// Parse an env var into `Option<T>` — returns `None` when unset,
/// `Some(parsed)` when set to a valid value.
pub(crate) fn parse_option_env<T>(key: &str) -> Result<Option<T>, ConfigError>
where
T: std::str::FromStr,
T::Err: std::fmt::Display,
{
optional_env(key)?
.map(|s| {
s.parse().map_err(|e| ConfigError::InvalidValue {
key: key.to_string(),
message: format!("{e}"),
})
})
.transpose()
}
/// Parse a string from an env var with a default.
pub(crate) fn parse_string_env(
key: &str,
default: impl Into<String>,
) -> Result<String, ConfigError> {
Ok(optional_env(key)?.unwrap_or_else(|| default.into()))
}
+4 -25
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::optional_env;
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Memory hygiene configuration.
@@ -28,30 +28,9 @@ impl Default for HygieneConfig {
impl HygieneConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("MEMORY_HYGIENE_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
retention_days: optional_env("MEMORY_HYGIENE_RETENTION_DAYS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_RETENTION_DAYS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(30),
cadence_hours: optional_env("MEMORY_HYGIENE_CADENCE_HOURS")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "MEMORY_HYGIENE_CADENCE_HOURS".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.unwrap_or(12),
enabled: parse_bool_env("MEMORY_HYGIENE_ENABLED", true)?,
retention_days: parse_optional_env("MEMORY_HYGIENE_RETENTION_DAYS", 30)?,
cadence_hours: parse_optional_env("MEMORY_HYGIENE_CADENCE_HOURS", 12)?,
})
}
+113 -4
View File
@@ -90,6 +90,9 @@ pub struct OpenAiCompatibleConfig {
pub base_url: String,
pub api_key: Option<SecretString>,
pub model: String,
/// Extra HTTP headers injected into every LLM request.
/// Parsed from `LLM_EXTRA_HEADERS` env var (format: `Key:Value,Key2:Value2`).
pub extra_headers: Vec<(String, String)>,
}
/// Configuration for Tinfoil private inference.
@@ -167,6 +170,10 @@ pub struct NearAiConfig {
/// Number of consecutive retryable failures before a provider enters
/// cooldown (default: 3).
pub failover_cooldown_threshold: u32,
/// Enable cascade mode for smart routing: when a moderate-complexity task
/// gets an uncertain response from the cheap model, re-send to primary.
/// Default: true.
pub smart_routing_cascade: bool,
}
impl LlmConfig {
@@ -199,10 +206,7 @@ impl LlmConfig {
let nearai = NearAiConfig {
model: optional_env("NEARAI_MODEL")?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| {
"fireworks::accounts/fireworks/models/llama4-maverick-instruct-basic"
.to_string()
}),
.unwrap_or_else(|| "zai-org/GLM-latest".to_string()),
cheap_model: optional_env("NEARAI_CHEAP_MODEL")?,
base_url: optional_env("NEARAI_BASE_URL")?.unwrap_or_else(|| {
if nearai_api_key.is_some() {
@@ -232,6 +236,7 @@ impl LlmConfig {
response_cache_max_entries: parse_optional_env("RESPONSE_CACHE_MAX_ENTRIES", 1000)?,
failover_cooldown_secs: parse_optional_env("LLM_FAILOVER_COOLDOWN_SECS", 300)?,
failover_cooldown_threshold: parse_optional_env("LLM_FAILOVER_THRESHOLD", 3)?,
smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
};
// Resolve provider-specific configs based on backend
@@ -293,10 +298,15 @@ impl LlmConfig {
let model = optional_env("LLM_MODEL")?
.or_else(|| settings.selected_model.clone())
.unwrap_or_else(|| "default".to_string());
let extra_headers = optional_env("LLM_EXTRA_HEADERS")?
.map(|val| parse_extra_headers(&val))
.transpose()?
.unwrap_or_default();
Some(OpenAiCompatibleConfig {
base_url,
api_key,
model,
extra_headers,
})
} else {
None
@@ -327,6 +337,40 @@ impl LlmConfig {
}
}
/// Parse `LLM_EXTRA_HEADERS` value into a list of (key, value) pairs.
///
/// Format: `Key1:Value1,Key2:Value2` — colon-separated key:value, comma-separated pairs.
/// Colon is used as the separator (not `=`) because header values often contain `=`
/// (e.g., base64 tokens).
fn parse_extra_headers(val: &str) -> Result<Vec<(String, String)>, ConfigError> {
if val.trim().is_empty() {
return Ok(Vec::new());
}
let mut headers = Vec::new();
for pair in val.split(',') {
let pair = pair.trim();
if pair.is_empty() {
continue;
}
let Some((key, value)) = pair.split_once(':') else {
return Err(ConfigError::InvalidValue {
key: "LLM_EXTRA_HEADERS".to_string(),
message: format!("malformed header entry '{}', expected Key:Value", pair),
});
};
let key = key.trim();
if key.is_empty() {
return Err(ConfigError::InvalidValue {
key: "LLM_EXTRA_HEADERS".to_string(),
message: format!("empty header name in entry '{}'", pair),
});
}
headers.push((key.to_string(), value.trim().to_string()));
}
Ok(headers)
}
/// Get the default session file path (~/.ironclaw/session.json).
fn default_session_path() -> PathBuf {
dirs::home_dir()
@@ -399,4 +443,69 @@ mod tests {
std::env::remove_var("LLM_MODEL");
}
}
#[test]
fn test_extra_headers_parsed() {
let result = parse_extra_headers("HTTP-Referer:https://myapp.com,X-Title:MyApp").unwrap();
assert_eq!(
result,
vec![
("HTTP-Referer".to_string(), "https://myapp.com".to_string()),
("X-Title".to_string(), "MyApp".to_string()),
]
);
}
#[test]
fn test_extra_headers_empty_string() {
let result = parse_extra_headers("").unwrap();
assert!(result.is_empty());
}
#[test]
fn test_extra_headers_whitespace_only() {
let result = parse_extra_headers(" ").unwrap();
assert!(result.is_empty());
}
#[test]
fn test_extra_headers_malformed() {
let result = parse_extra_headers("NoColonHere");
assert!(result.is_err());
}
#[test]
fn test_extra_headers_empty_key() {
let result = parse_extra_headers(":value");
assert!(result.is_err());
}
#[test]
fn test_extra_headers_value_with_colons() {
// Values can contain colons (e.g., URLs)
let result = parse_extra_headers("Authorization:Bearer abc:def").unwrap();
assert_eq!(
result,
vec![("Authorization".to_string(), "Bearer abc:def".to_string())]
);
}
#[test]
fn test_extra_headers_trailing_comma() {
let result = parse_extra_headers("X-Title:MyApp,").unwrap();
assert_eq!(result, vec![("X-Title".to_string(), "MyApp".to_string())]);
}
#[test]
fn test_extra_headers_with_spaces() {
let result =
parse_extra_headers(" HTTP-Referer : https://myapp.com , X-Title : MyApp ").unwrap();
assert_eq!(
result,
vec![
("HTTP-Referer".to_string(), "https://myapp.com".to_string()),
("X-Title".to_string(), "MyApp".to_string()),
]
);
}
}
+2 -9
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Routines configuration.
@@ -31,14 +31,7 @@ impl Default for RoutineConfig {
impl RoutineConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("ROUTINES_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "ROUTINES_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("ROUTINES_ENABLED", true)?,
cron_check_interval_secs: parse_optional_env("ROUTINES_CRON_INTERVAL", 15)?,
max_concurrent_routines: parse_optional_env("ROUTINES_MAX_CONCURRENT", 10)?,
default_cooldown_secs: parse_optional_env("ROUTINES_DEFAULT_COOLDOWN", 300)?,
+2 -9
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Safety configuration.
@@ -12,14 +12,7 @@ impl SafetyConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
max_output_length: parse_optional_env("SAFETY_MAX_OUTPUT_LENGTH", 100_000)?,
injection_check_enabled: optional_env("SAFETY_INJECTION_CHECK_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SAFETY_INJECTION_CHECK_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
injection_check_enabled: parse_bool_env("SAFETY_INJECTION_CHECK_ENABLED", true)?,
})
}
}
+7 -29
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env, parse_string_env};
use crate::error::ConfigError;
/// Docker sandbox configuration.
@@ -44,28 +44,13 @@ impl SandboxModeConfig {
.unwrap_or_default();
Ok(Self {
enabled: optional_env("SANDBOX_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SANDBOX_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
policy: optional_env("SANDBOX_POLICY")?.unwrap_or_else(|| "readonly".to_string()),
enabled: parse_bool_env("SANDBOX_ENABLED", true)?,
policy: parse_string_env("SANDBOX_POLICY", "readonly")?,
timeout_secs: parse_optional_env("SANDBOX_TIMEOUT_SECS", 120)?,
memory_limit_mb: parse_optional_env("SANDBOX_MEMORY_LIMIT_MB", 2048)?,
cpu_shares: parse_optional_env("SANDBOX_CPU_SHARES", 1024)?,
image: optional_env("SANDBOX_IMAGE")?
.unwrap_or_else(|| "ironclaw-worker:latest".to_string()),
auto_pull_image: optional_env("SANDBOX_AUTO_PULL")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SANDBOX_AUTO_PULL".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
image: parse_string_env("SANDBOX_IMAGE", "ironclaw-worker:latest")?,
auto_pull_image: parse_bool_env("SANDBOX_AUTO_PULL", true)?,
extra_allowed_domains: extra_domains,
})
}
@@ -221,18 +206,11 @@ impl ClaudeCodeConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
let defaults = Self::default();
Ok(Self {
enabled: optional_env("CLAUDE_CODE_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "CLAUDE_CODE_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(defaults.enabled),
enabled: parse_bool_env("CLAUDE_CODE_ENABLED", defaults.enabled)?,
config_dir: optional_env("CLAUDE_CONFIG_DIR")?
.map(std::path::PathBuf::from)
.unwrap_or(defaults.config_dir),
model: optional_env("CLAUDE_CODE_MODEL")?.unwrap_or(defaults.model),
model: parse_string_env("CLAUDE_CODE_MODEL", defaults.model)?,
max_turns: parse_optional_env("CLAUDE_CODE_MAX_TURNS", defaults.max_turns)?,
memory_limit_mb: parse_optional_env(
"CLAUDE_CODE_MEMORY_LIMIT_MB",
+2 -9
View File
@@ -1,6 +1,6 @@
use std::path::PathBuf;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// Skills system configuration.
@@ -38,14 +38,7 @@ fn default_skills_dir() -> PathBuf {
impl SkillsConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("SKILLS_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "SKILLS_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(false),
enabled: parse_bool_env("SKILLS_ENABLED", false)?,
local_dir: optional_env("SKILLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_skills_dir),
+3 -17
View File
@@ -1,7 +1,7 @@
use std::path::PathBuf;
use std::time::Duration;
use crate::config::helpers::{optional_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::error::ConfigError;
/// WASM sandbox configuration.
@@ -48,14 +48,7 @@ fn default_tools_dir() -> PathBuf {
impl WasmConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
Ok(Self {
enabled: optional_env("WASM_ENABLED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_ENABLED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
enabled: parse_bool_env("WASM_ENABLED", true)?,
tools_dir: optional_env("WASM_TOOLS_DIR")?
.map(PathBuf::from)
.unwrap_or_else(default_tools_dir),
@@ -65,14 +58,7 @@ impl WasmConfig {
)?,
default_timeout_secs: parse_optional_env("WASM_DEFAULT_TIMEOUT_SECS", 60)?,
default_fuel_limit: parse_optional_env("WASM_DEFAULT_FUEL_LIMIT", 10_000_000)?,
cache_compiled: optional_env("WASM_CACHE_COMPILED")?
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "WASM_CACHE_COMPILED".to_string(),
message: format!("must be 'true' or 'false': {e}"),
})?
.unwrap_or(true),
cache_compiled: parse_bool_env("WASM_CACHE_COMPILED", true)?,
cache_dir: optional_env("WASM_CACHE_DIR")?.map(PathBuf::from),
})
}
+7 -6
View File
@@ -49,9 +49,10 @@ impl ConversationStore for LibSqlBackend {
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
conn.execute(
"INSERT INTO conversation_messages (id, conversation_id, role, content) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), conversation_id.to_string(), role, content],
"INSERT INTO conversation_messages (id, conversation_id, role, content, created_at) VALUES (?1, ?2, ?3, ?4, ?5)",
params![id.to_string(), conversation_id.to_string(), role, content, now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -100,7 +101,7 @@ impl ConversationStore for LibSqlBackend {
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
ORDER BY m2.created_at ASC
ORDER BY m2.created_at ASC, m2.rowid ASC
LIMIT 1
) AS title
FROM conversations c
@@ -216,7 +217,7 @@ impl ConversationStore for LibSqlBackend {
SELECT id, role, content, created_at
FROM conversation_messages
WHERE conversation_id = ?1 AND created_at < ?2
ORDER BY created_at DESC
ORDER BY created_at DESC, rowid DESC
LIMIT ?3
"#,
params![cid, fmt_ts(&before_ts), fetch_limit],
@@ -228,7 +229,7 @@ impl ConversationStore for LibSqlBackend {
SELECT id, role, content, created_at
FROM conversation_messages
WHERE conversation_id = ?1
ORDER BY created_at DESC
ORDER BY created_at DESC, rowid DESC
LIMIT ?2
"#,
params![cid, fetch_limit],
@@ -309,7 +310,7 @@ impl ConversationStore for LibSqlBackend {
SELECT id, role, content, created_at
FROM conversation_messages
WHERE conversation_id = ?1
ORDER BY created_at ASC
ORDER BY created_at ASC, rowid ASC
"#,
params![conversation_id.to_string()],
)
+15
View File
@@ -387,4 +387,19 @@ impl RoutineStore for LibSqlBackend {
None => Ok(0),
}
}
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
let conn = self.connect().await?;
conn.execute(
"UPDATE routine_runs SET job_id = ?1 WHERE id = ?2",
params![job_id.to_string(), run_id.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(())
}
}
+5
View File
@@ -274,6 +274,11 @@ pub trait RoutineStore: Send + Sync {
limit: i64,
) -> Result<Vec<RoutineRun>, DatabaseError>;
async fn count_running_routine_runs(&self, routine_id: Uuid) -> Result<i64, DatabaseError>;
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError>;
}
#[async_trait]
+8
View File
@@ -437,6 +437,14 @@ impl RoutineStore for PgBackend {
async fn count_running_routine_runs(&self, routine_id: Uuid) -> Result<i64, DatabaseError> {
self.store.count_running_routine_runs(routine_id).await
}
async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
self.store.link_routine_run_to_job(run_id, job_id).await
}
}
// ==================== ToolFailureStore ====================
+9
View File
@@ -202,6 +202,12 @@ pub enum ToolError {
#[error("Tool {name} requires authentication")]
AuthRequired { name: String },
#[error("Tool {name} is rate limited, retry after {retry_after:?}")]
RateLimited {
name: String,
retry_after: Option<Duration>,
},
#[error("Tool builder failed: {0}")]
BuilderFailed(String),
}
@@ -401,6 +407,9 @@ pub enum RoutineError {
#[error("LLM call failed: {reason}")]
LlmFailed { reason: String },
#[error("Failed to dispatch full job: {reason}")]
JobDispatchFailed { reason: String },
#[error("LLM returned empty content")]
EmptyResponse,
+1 -1
View File
@@ -1,4 +1,4 @@
//! Online extension discovery for finding MCP servers not in the built-in registry.
//! Online extension discovery for finding extensions not in the built-in registry.
//!
//! Multi-tier search strategy:
//! 1. Probe well-known URL patterns (mcp.{service}.com, {service}.com/mcp)
+1217 -66
View File
File diff suppressed because it is too large Load Diff
+27 -14
View File
@@ -1,16 +1,19 @@
//! Unified extension system for discovering, installing, authenticating, and activating
//! MCP servers and WASM tools through conversational agent interactions.
//! Lifecycle management for extensions: discovery, installation, authentication,
//! and activation of channels, tools, and MCP servers.
//!
//! Extensions are the user-facing abstraction over MCP servers and WASM tools. The agent
//! can search a built-in registry (or discover online), install, authenticate, and activate
//! extensions at runtime without CLI commands.
//! Extensions are the user-facing abstraction that unifies three runtime kinds:
//! - **Channels** (Telegram, Slack, Discord) — messaging integrations (WASM)
//! - **Tools** — sandboxed capabilities (WASM)
//! - **MCP servers** — external API integrations via Model Context Protocol
//!
//! The agent can search a built-in registry (or discover online), install,
//! authenticate, and activate extensions at runtime without CLI commands.
//!
//! ```text
//! User: "add notion"
//! -> tool_search("notion") -> finds MCP server in registry
//! -> tool_install("notion") -> saves config to mcp-servers.json
//! -> tool_auth("notion") -> OAuth 2.1 flow, returns URL
//! -> tool_activate("notion") -> connects, registers tools
//! User: "add telegram"
//! -> tool_search("telegram") -> finds channel in registry
//! -> tool_install("telegram") -> copies bundled WASM to channels dir
//! -> tool_activate("telegram") -> configures credentials, starts channel
//! ```
pub mod discovery;
@@ -31,7 +34,7 @@ pub enum ExtensionKind {
McpServer,
/// Sandboxed WASM module, file-based, capabilities auth.
WasmTool,
/// WASM channel module (future: dynamic activation, currently needs restart).
/// WASM channel module with hot-activation support.
WasmChannel,
}
@@ -82,6 +85,9 @@ pub enum ExtensionSource {
repo_url: String,
#[serde(default)]
build_dir: Option<String>,
/// Crate name used to locate the build artifact binary.
#[serde(default)]
crate_name: Option<String>,
},
/// Discovered online (not yet validated for a specific source type).
Discovered { url: String },
@@ -169,6 +175,10 @@ pub struct ActivateResult {
pub message: String,
}
fn default_true() -> bool {
true
}
/// An installed extension with its current status.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct InstalledExtension {
@@ -184,6 +194,12 @@ pub struct InstalledExtension {
/// Tool names if active.
#[serde(default)]
pub tools: Vec<String>,
/// Whether this extension has a setup schema (required_secrets) that can be configured.
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
}
/// Error type for extension operations.
@@ -219,9 +235,6 @@ pub enum ExtensionError {
#[error("Config error: {0}")]
Config(String),
#[error("Channels require restart to activate")]
ChannelNeedsRestart,
#[error("{0}")]
Other(String),
}
+120 -5
View File
@@ -1,7 +1,7 @@
//! Curated in-memory catalog of known extensions with fuzzy search.
//!
//! The registry holds well-known MCP servers and WASM tools that can be installed
//! via conversational commands. Online discoveries are cached here too.
//! The registry holds well-known channels, tools, and MCP servers that can be
//! installed via conversational commands. Online discoveries are cached here too.
use tokio::sync::RwLock;
@@ -26,6 +26,26 @@ impl ExtensionRegistry {
}
}
/// Create a new registry merging builtin entries with catalog-provided entries.
///
/// Deduplicates by `(name, kind)` pair -- a builtin MCP "slack" and a registry
/// WASM "slack" can coexist since they're different kinds.
pub fn new_with_catalog(catalog_entries: Vec<RegistryEntry>) -> Self {
let mut entries = builtin_entries();
for entry in catalog_entries {
if !entries
.iter()
.any(|e| e.name == entry.name && e.kind == entry.kind)
{
entries.push(entry);
}
}
Self {
entries,
discovery_cache: RwLock::new(Vec::new()),
}
}
/// Search the registry by query string. Returns results sorted by relevance.
///
/// Splits the query into lowercase tokens and scores each entry by matches
@@ -96,6 +116,21 @@ impl ExtensionRegistry {
cache.iter().find(|e| e.name == name).cloned()
}
/// Return all registry entries (builtins + cached discoveries).
pub async fn all_entries(&self) -> Vec<RegistryEntry> {
let mut entries = self.entries.clone();
let cache = self.discovery_cache.read().await;
for entry in cache.iter() {
if !entries
.iter()
.any(|e| e.name == entry.name && e.kind == entry.kind)
{
entries.push(entry.clone());
}
}
entries
}
/// Add discovered entries to the cache.
pub async fn cache_discovered(&self, entries: Vec<RegistryEntry>) {
let mut cache = self.discovery_cache.write().await;
@@ -250,11 +285,11 @@ fn builtin_entries() -> Vec<RegistryEntry> {
auth_hint: AuthHint::Dcr,
},
RegistryEntry {
name: "slack".to_string(),
display_name: "Slack".to_string(),
name: "slack-mcp".to_string(),
display_name: "Slack MCP".to_string(),
kind: ExtensionKind::McpServer,
description:
"Connect to Slack for messaging, channel management, and team communication"
"Connect to Slack via MCP for messaging, channel management, and team communication"
.to_string(),
keywords: vec![
"messaging".into(),
@@ -360,6 +395,9 @@ fn builtin_entries() -> Vec<RegistryEntry> {
},
auth_hint: AuthHint::Dcr,
},
// WASM channels (telegram, slack, discord, whatsapp) come from the embedded
// registry catalog (registry/channels/*.json) with WasmDownload URLs pointing
// to GitHub release artifacts. See new_with_catalog() for merging.
]
}
@@ -542,4 +580,81 @@ mod tests {
let results = registry.search("dup").await;
assert_eq!(results.len(), 1, "Should not duplicate cached entries");
}
#[tokio::test]
async fn test_new_with_catalog() {
let catalog_entries = vec![
RegistryEntry {
name: "telegram".to_string(),
display_name: "Telegram".to_string(),
kind: ExtensionKind::WasmChannel,
description: "Telegram Bot API channel".to_string(),
keywords: vec!["messaging".into(), "bot".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "channels-src/telegram".to_string(),
build_dir: Some("channels-src/telegram".to_string()),
crate_name: Some("telegram-channel".to_string()),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
// This shares a name with the builtin slack-mcp but has a different kind, so both should appear
RegistryEntry {
name: "slack-mcp".to_string(),
display_name: "Slack MCP WASM".to_string(),
kind: ExtensionKind::WasmTool,
description: "Slack WASM tool".to_string(),
keywords: vec!["messaging".into()],
source: ExtensionSource::WasmBuildable {
repo_url: "tools-src/slack".to_string(),
build_dir: Some("tools-src/slack".to_string()),
crate_name: Some("slack-tool".to_string()),
},
auth_hint: AuthHint::CapabilitiesAuth,
},
];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
// Should find the new telegram entry
let results = registry.search("telegram").await;
assert!(!results.is_empty(), "Should find telegram from catalog");
assert_eq!(results[0].entry.name, "telegram");
// Should have both builtin MCP slack-mcp and catalog WASM slack-mcp
let results = registry.search("slack").await;
let slack_mcp = results
.iter()
.any(|r| r.entry.name == "slack-mcp" && r.entry.kind == ExtensionKind::McpServer);
let slack_wasm = results
.iter()
.any(|r| r.entry.name == "slack-mcp" && r.entry.kind == ExtensionKind::WasmTool);
assert!(slack_mcp, "Should have builtin MCP slack-mcp");
assert!(slack_wasm, "Should have catalog WASM slack-mcp");
}
#[tokio::test]
async fn test_new_with_catalog_dedup_same_kind() {
// A catalog entry with same name AND kind as a builtin should be skipped
let catalog_entries = vec![RegistryEntry {
name: "slack-mcp".to_string(),
display_name: "Slack MCP Override".to_string(),
kind: ExtensionKind::McpServer, // same kind as builtin slack-mcp
description: "Should be skipped".to_string(),
keywords: vec![],
source: ExtensionSource::McpUrl {
url: "https://other.slack.com".to_string(),
},
auth_hint: AuthHint::Dcr,
}];
let registry = ExtensionRegistry::new_with_catalog(catalog_entries);
let entry = registry.get("slack-mcp").await;
assert!(entry.is_some());
// Should still be the builtin, not the override
assert_eq!(entry.unwrap().display_name, "Slack MCP");
}
// Channel tests (telegram, slack, discord, whatsapp) require the embedded catalog
// to be loaded via new_with_catalog(). See test_new_with_catalog for catalog coverage.
}
+15
View File
@@ -1167,6 +1167,21 @@ impl Store {
.await?;
Ok(row.get("cnt"))
}
/// Link a routine run to a dispatched job.
pub async fn link_routine_run_to_job(
&self,
run_id: Uuid,
job_id: Uuid,
) -> Result<(), DatabaseError> {
let conn = self.conn().await?;
conn.execute(
"UPDATE routine_runs SET job_id = $1 WHERE id = $2",
&[&job_id, &run_id],
)
.await?;
Ok(())
}
}
#[cfg(feature = "postgres")]
+166 -2
View File
@@ -17,6 +17,7 @@ pub mod response_cache;
pub mod retry;
mod rig_adapter;
pub mod session;
pub mod smart_routing;
pub use circuit_breaker::{CircuitBreakerConfig, CircuitBreakerProvider};
pub use failover::{CooldownConfig, FailoverProvider};
@@ -26,13 +27,14 @@ pub use provider::{
Role, ToolCall, ToolCompletionRequest, ToolCompletionResponse, ToolDefinition, ToolResult,
};
pub use reasoning::{
ActionPlan, Reasoning, ReasoningContext, RespondOutput, RespondResult, TokenUsage,
ToolSelection,
ActionPlan, Reasoning, ReasoningContext, RespondOutput, RespondResult, SILENT_REPLY_TOKEN,
TokenUsage, ToolSelection, is_silent_reply,
};
pub use response_cache::{CachedProvider, ResponseCacheConfig};
pub use retry::{RetryConfig, RetryProvider};
pub use rig_adapter::RigAdapter;
pub use session::{SessionConfig, SessionManager, create_session_manager};
pub use smart_routing::{SmartRoutingConfig, SmartRoutingProvider, TaskComplexity};
use std::sync::Arc;
@@ -218,6 +220,25 @@ fn create_openai_compatible_provider(config: &LlmConfig) -> Result<Arc<dyn LlmPr
use rig::providers::openai;
let mut extra_headers = reqwest::header::HeaderMap::new();
for (key, value) in &compat.extra_headers {
let name = match reqwest::header::HeaderName::from_bytes(key.as_bytes()) {
Ok(n) => n,
Err(e) => {
tracing::warn!(header = %key, error = %e, "Skipping LLM_EXTRA_HEADERS entry: invalid header name");
continue;
}
};
let val = match reqwest::header::HeaderValue::from_str(value) {
Ok(v) => v,
Err(e) => {
tracing::warn!(header = %key, error = %e, "Skipping LLM_EXTRA_HEADERS entry: invalid header value");
continue;
}
};
extra_headers.insert(name, val);
}
let client: openai::CompletionsClient = openai::Client::builder()
.base_url(&compat.base_url)
.api_key(
@@ -227,6 +248,7 @@ fn create_openai_compatible_provider(config: &LlmConfig) -> Result<Arc<dyn LlmPr
.map(|k| k.expose_secret().to_string())
.unwrap_or_else(|| "no-key".to_string()),
)
.http_headers(extra_headers)
.build()
.map_err(|e| LlmError::RequestFailed {
provider: "openai_compatible".to_string(),
@@ -273,6 +295,147 @@ pub fn create_cheap_llm_provider(
)?)))
}
/// Build the full LLM provider chain with all configured wrappers.
///
/// Applies decorators in this order:
/// 1. Raw provider (from config)
/// 2. RetryProvider (per-provider retry with exponential backoff)
/// 3. SmartRoutingProvider (cheap/primary split when cheap model is configured)
/// 4. FailoverProvider (fallback model when primary fails)
/// 5. CircuitBreakerProvider (fast-fail when backend is degraded)
/// 6. CachedProvider (in-memory response cache)
///
/// Also returns a separate cheap LLM provider for heartbeat/evaluation (not
/// part of the chain — it's a standalone provider for explicitly cheap tasks).
///
/// This is the single source of truth for provider chain construction,
/// called by both `main.rs` and `app.rs`.
#[allow(clippy::type_complexity)]
pub fn build_provider_chain(
config: &LlmConfig,
session: Arc<SessionManager>,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), LlmError> {
let llm = create_llm_provider(config, session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
// 1. Retry
let retry_config = RetryConfig {
max_retries: config.nearai.max_retries,
};
let llm: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
tracing::info!(
max_retries = retry_config.max_retries,
"LLM retry wrapper enabled"
);
Arc::new(RetryProvider::new(llm, retry_config.clone()))
} else {
llm
};
// 2. Smart routing (cheap/primary split)
let llm: Arc<dyn LlmProvider> = if let Some(ref cheap_model) = config.nearai.cheap_model {
let mut cheap_config = config.nearai.clone();
cheap_config.model = cheap_model.clone();
let cheap = create_llm_provider_with_config(&cheap_config, session.clone())?;
let cheap: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
Arc::new(RetryProvider::new(cheap, retry_config.clone()))
} else {
cheap
};
tracing::info!(
primary = %llm.model_name(),
cheap = %cheap.model_name(),
"Smart routing enabled"
);
Arc::new(SmartRoutingProvider::new(
llm,
cheap,
SmartRoutingConfig {
cascade_enabled: config.nearai.smart_routing_cascade,
..SmartRoutingConfig::default()
},
))
} else {
llm
};
// 3. Failover
let llm: Arc<dyn LlmProvider> = if let Some(ref fallback_model) = config.nearai.fallback_model {
if fallback_model == &config.nearai.model {
tracing::warn!(
"fallback_model is the same as primary model, failover may not be effective"
);
}
let mut fallback_config = config.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(&fallback_config, session.clone())?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
"LLM failover enabled"
);
let fallback: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
Arc::new(RetryProvider::new(fallback, retry_config.clone()))
} else {
fallback
};
let cooldown_config = CooldownConfig {
cooldown_duration: std::time::Duration::from_secs(config.nearai.failover_cooldown_secs),
failure_threshold: config.nearai.failover_cooldown_threshold,
};
Arc::new(FailoverProvider::with_cooldown(
vec![llm, fallback],
cooldown_config,
)?)
} else {
llm
};
// 4. Circuit breaker
let llm: Arc<dyn LlmProvider> = if let Some(threshold) = config.nearai.circuit_breaker_threshold
{
let cb_config = CircuitBreakerConfig {
failure_threshold: threshold,
recovery_timeout: std::time::Duration::from_secs(
config.nearai.circuit_breaker_recovery_secs,
),
..CircuitBreakerConfig::default()
};
tracing::info!(
threshold,
recovery_secs = config.nearai.circuit_breaker_recovery_secs,
"LLM circuit breaker enabled"
);
Arc::new(CircuitBreakerProvider::new(llm, cb_config))
} else {
llm
};
// 5. Response cache
let llm: Arc<dyn LlmProvider> = if config.nearai.response_cache_enabled {
let rc_config = ResponseCacheConfig {
ttl: std::time::Duration::from_secs(config.nearai.response_cache_ttl_secs),
max_entries: config.nearai.response_cache_max_entries,
};
tracing::info!(
ttl_secs = config.nearai.response_cache_ttl_secs,
max_entries = config.nearai.response_cache_max_entries,
"LLM response cache enabled"
);
Arc::new(CachedProvider::new(llm, rc_config))
} else {
llm
};
// Standalone cheap LLM for heartbeat/evaluation (not part of the chain)
let cheap_llm = create_cheap_llm_provider(config, session)?;
if let Some(ref cheap) = cheap_llm {
tracing::info!("Cheap LLM provider initialized: {}", cheap.model_name());
}
Ok((llm, cheap_llm))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -296,6 +459,7 @@ mod tests {
response_cache_max_entries: 1000,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
smart_routing_cascade: true,
}
}
+280 -8
View File
@@ -6,12 +6,13 @@
//! - **Session token auth**: Otherwise, uses `SessionManager` for Bearer session token
//! with automatic renewal on 401 errors
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use reqwest::Client;
use rust_decimal::Decimal;
use rust_decimal_macros::dec;
use rust_decimal::prelude::MathematicalOps;
use secrecy::ExposeSecret;
use serde::{Deserialize, Serialize};
@@ -21,7 +22,7 @@ use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse,
};
use crate::llm::session::SessionManager;
use crate::llm::{costs, session::SessionManager};
/// Information about an available model from NEAR AI API.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -42,6 +43,9 @@ pub struct NearAiChatProvider {
session: Arc<SessionManager>,
active_model: std::sync::RwLock<String>,
flatten_tool_messages: bool,
/// Per-model pricing fetched from the NEAR AI `/v1/model/list` endpoint.
/// Maps model ID → (input_cost_per_token, output_cost_per_token).
pricing: Arc<std::sync::RwLock<HashMap<String, (Decimal, Decimal)>>>,
}
impl NearAiChatProvider {
@@ -72,13 +76,49 @@ impl NearAiChatProvider {
})?;
let active_model = std::sync::RwLock::new(config.model.clone());
Ok(Self {
let pricing = Arc::new(std::sync::RwLock::new(HashMap::new()));
let provider = Self {
client,
config,
session,
active_model,
flatten_tool_messages,
})
pricing,
};
// Fire-and-forget background pricing fetch — don't block startup.
// Only spawns when a tokio runtime is active (skipped in sync tests).
if let Ok(handle) = tokio::runtime::Handle::try_current() {
let client = provider.client.clone();
let base_url = provider.config.base_url.clone();
let api_key = provider.config.api_key.clone();
let session = provider.session.clone();
let pricing = provider.pricing.clone();
handle.spawn(async move {
match fetch_pricing(&client, &base_url, api_key.as_ref(), &session).await {
Ok(map) if !map.is_empty() => {
tracing::info!("Loaded NEAR AI pricing for {} model(s)", map.len());
match pricing.write() {
Ok(mut guard) => *guard = map,
Err(poisoned) => *poisoned.into_inner() = map,
}
}
Ok(_) => {
tracing::debug!("NEAR AI pricing endpoint returned no pricing data");
}
Err(e) => {
tracing::debug!(
"Could not fetch NEAR AI pricing (will use fallback): {}",
e
);
}
}
});
}
Ok(provider)
}
fn api_url(&self, path: &str) -> String {
@@ -382,7 +422,13 @@ impl LlmProvider for NearAiChatProvider {
reason: "No choices in response".to_string(),
})?;
let content = choice.message.content.unwrap_or_default();
// Fall back to reasoning_content when content is null (same as
// complete_with_tools — reasoning models may put the answer there).
let content = choice
.message
.content
.or(choice.message.reasoning_content)
.unwrap_or_default();
let finish_reason = match choice.finish_reason.as_deref() {
Some("stop") => FinishReason::Stop,
Some("length") => FinishReason::Length,
@@ -453,7 +499,9 @@ impl LlmProvider for NearAiChatProvider {
reason: "No choices in response".to_string(),
})?;
let content = choice.message.content;
// Fall back to reasoning_content when content is null (e.g. GLM-5
// returns its answer in reasoning_content instead of content).
let content = choice.message.content.or(choice.message.reasoning_content);
let tool_calls: Vec<ToolCall> = choice
.message
.tool_calls
@@ -500,8 +548,14 @@ impl LlmProvider for NearAiChatProvider {
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
// Default costs - could be model-specific in the future
(dec!(0.000003), dec!(0.000015))
let model = self.active_model_name();
// Try fetched pricing first, then static lookup table, then default
if let Ok(guard) = self.pricing.read()
&& let Some(&rates) = guard.get(&model)
{
return rates;
}
costs::model_cost(&model).unwrap_or_else(costs::default_cost)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
@@ -562,6 +616,143 @@ struct ChatCompletionMessage {
tool_calls: Option<Vec<ChatCompletionToolCall>>,
}
// -- Pricing fetch types and logic -----------------------------------------
/// Cost amount from the NEAR AI `/v1/model/list` response.
///
/// Real cost per token = `amount * 10^(-scale)`.
#[derive(Debug, Deserialize)]
struct ModelCost {
amount: f64,
#[serde(default)]
scale: i32,
}
/// A single model entry from the pricing response.
#[derive(Debug, Deserialize)]
struct PricingModelEntry {
#[serde(default, alias = "modelId", alias = "model_id")]
model_id: Option<String>,
#[serde(default, alias = "inputCostPerToken")]
input_cost_per_token: Option<ModelCost>,
#[serde(default, alias = "outputCostPerToken")]
output_cost_per_token: Option<ModelCost>,
#[serde(default)]
metadata: Option<PricingMetadata>,
}
#[derive(Debug, Deserialize)]
struct PricingMetadata {
#[serde(default)]
aliases: Vec<String>,
}
/// Wrapper for the `/v1/model/list` response body.
#[derive(Debug, Deserialize)]
struct PricingResponse {
#[serde(default)]
models: Option<Vec<PricingModelEntry>>,
#[serde(default)]
data: Option<Vec<PricingModelEntry>>,
}
/// Convert a `ModelCost` to a `Decimal` per-token price.
fn model_cost_to_decimal(mc: &ModelCost) -> Option<Decimal> {
if mc.amount == 0.0 {
return Some(Decimal::ZERO);
}
// amount * 10^(-scale)
let base = Decimal::try_from(mc.amount).ok()?;
let factor = Decimal::TEN.checked_powi(-i64::from(mc.scale))?;
base.checked_mul(factor)
}
/// Fetch pricing from the NEAR AI `/v1/model/list` endpoint.
///
/// Returns a map of model_id → (input_cost_per_token, output_cost_per_token).
/// Errors are non-fatal; callers should fall back to the static lookup table.
async fn fetch_pricing(
client: &Client,
base_url: &str,
api_key: Option<&secrecy::SecretString>,
session: &SessionManager,
) -> Result<HashMap<String, (Decimal, Decimal)>, LlmError> {
let base = base_url.trim_end_matches('/');
let url = if base.ends_with("/v1") {
format!("{}/model/list", base)
} else {
format!("{}/v1/model/list", base)
};
let token = if let Some(key) = api_key {
key.expose_secret().to_string()
} else {
let tok = session.get_token().await?;
tok.expose_secret().to_string()
};
let response = client
.get(&url)
.header("Authorization", format!("Bearer {}", token))
.timeout(std::time::Duration::from_secs(15))
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to fetch pricing: {}", e),
})?;
if !response.status().is_success() {
return Err(LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Pricing endpoint returned HTTP {}", response.status()),
});
}
let body = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to read pricing response: {}", e),
})?;
// Parse as {models: [...]} or {data: [...]} or direct array
let entries: Vec<PricingModelEntry> =
if let Ok(resp) = serde_json::from_str::<PricingResponse>(&body) {
resp.models.or(resp.data).unwrap_or_default()
} else if let Ok(arr) = serde_json::from_str::<Vec<PricingModelEntry>>(&body) {
arr
} else {
return Ok(HashMap::new());
};
let mut map = HashMap::new();
for entry in &entries {
let (Some(input_mc), Some(output_mc)) =
(&entry.input_cost_per_token, &entry.output_cost_per_token)
else {
continue;
};
let (Some(input), Some(output)) = (
model_cost_to_decimal(input_mc),
model_cost_to_decimal(output_mc),
) else {
continue;
};
// Insert under the primary model_id
if let Some(ref id) = entry.model_id {
map.insert(id.clone(), (input, output));
}
// Also insert under any aliases
if let Some(ref meta) = entry.metadata {
for alias in &meta.aliases {
map.insert(alias.clone(), (input, output));
}
}
}
Ok(map)
}
/// Rewrite tool-call / tool-result messages into plain assistant/user text.
///
/// NEAR AI cloud-api does not support the OpenAI multi-turn tool-calling
@@ -598,6 +789,7 @@ fn flatten_tool_messages(messages: Vec<ChatCompletionMessage>) -> Vec<ChatComple
ChatCompletionMessage {
role: "assistant".to_string(),
content: Some(parts.join("\n")),
tool_call_id: None,
name: None,
tool_calls: None,
@@ -609,6 +801,7 @@ fn flatten_tool_messages(messages: Vec<ChatCompletionMessage>) -> Vec<ChatComple
ChatCompletionMessage {
role: "user".to_string(),
content: Some(format!("[Tool `{}` returned: {}]", tool_name, result)),
tool_call_id: None,
name: None,
tool_calls: None,
@@ -696,6 +889,10 @@ struct ChatCompletionResponseMessage {
#[allow(dead_code)]
role: String,
content: Option<String>,
/// Some models (e.g. GLM-5) return chain-of-thought reasoning here
/// instead of in `content`.
#[serde(default)]
reasoning_content: Option<String>,
tool_calls: Option<Vec<ChatCompletionToolCall>>,
}
@@ -748,6 +945,7 @@ fn parse_usage(usage: Option<&ChatCompletionUsage>) -> (u32, u32) {
mod tests {
use super::*;
use crate::llm::session::SessionConfig;
use rust_decimal_macros::dec;
fn test_nearai_config(base_url: &str) -> NearAiConfig {
NearAiConfig {
@@ -766,6 +964,7 @@ mod tests {
response_cache_max_entries: 1000,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
smart_routing_cascade: true,
}
}
@@ -990,4 +1189,77 @@ mod tests {
assert!(text.starts_with("Let me check that."));
assert!(text.contains("[Called tool `search`"));
}
#[test]
fn test_model_cost_to_decimal_basic() {
// amount=3, scale=6 → 3 * 10^-6 = 0.000003
let mc = ModelCost {
amount: 3.0,
scale: 6,
};
let result = model_cost_to_decimal(&mc).unwrap();
assert_eq!(result, dec!(0.000003));
}
#[test]
fn test_model_cost_to_decimal_zero() {
let mc = ModelCost {
amount: 0.0,
scale: 6,
};
assert_eq!(model_cost_to_decimal(&mc), Some(Decimal::ZERO));
}
#[test]
fn test_model_cost_to_decimal_larger_scale() {
// amount=85, scale=8 → 85 * 10^-8 = 0.00000085
let mc = ModelCost {
amount: 85.0,
scale: 8,
};
let result = model_cost_to_decimal(&mc).unwrap();
assert_eq!(result, dec!(0.00000085));
}
#[test]
fn test_cost_per_token_uses_pricing_map() {
let cfg = test_nearai_config("http://127.0.0.1:8318");
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// Inject pricing directly
{
let mut guard = provider.pricing.write().unwrap();
guard.insert("test-model".to_string(), (dec!(0.000001), dec!(0.000005)));
}
let (input, output) = provider.cost_per_token();
assert_eq!(input, dec!(0.000001));
assert_eq!(output, dec!(0.000005));
}
#[test]
fn test_cost_per_token_falls_back_to_static() {
let mut cfg = test_nearai_config("http://127.0.0.1:8318");
cfg.model = "gpt-4o".to_string();
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// No pricing in map, should fall back to static costs::model_cost
let (input, output) = provider.cost_per_token();
let (expected_in, expected_out) = costs::model_cost("gpt-4o").unwrap();
assert_eq!(input, expected_in);
assert_eq!(output, expected_out);
}
#[test]
fn test_cost_per_token_falls_back_to_default() {
let mut cfg = test_nearai_config("http://127.0.0.1:8318");
cfg.model = "some-unknown-nearai-model".to_string();
let provider = NearAiChatProvider::new(cfg, test_session()).expect("provider");
// No pricing in map, not in static table, should use default_cost
let (input, output) = provider.cost_per_token();
let (default_in, default_out) = costs::default_cost();
assert_eq!(input, default_in);
assert_eq!(output, default_out);
}
}
+194 -5
View File
@@ -12,6 +12,24 @@ use crate::llm::{
};
use crate::safety::SafetyLayer;
/// Token the agent returns when it has nothing to say (e.g. in group chats).
/// The dispatcher should check for this and suppress the message.
pub const SILENT_REPLY_TOKEN: &str = "NO_REPLY";
/// Check if a response is a silent reply (the agent has nothing to say).
///
/// Returns true if the trimmed text is exactly the silent reply token or
/// contains only the token surrounded by whitespace/punctuation.
pub fn is_silent_reply(text: &str) -> bool {
let trimmed = text.trim();
trimmed == SILENT_REPLY_TOKEN
|| trimmed.starts_with(SILENT_REPLY_TOKEN)
&& trimmed.len() <= SILENT_REPLY_TOKEN.len() + 4
&& trimmed[SILENT_REPLY_TOKEN.len()..]
.chars()
.all(|c| c.is_whitespace() || c.is_ascii_punctuation())
}
/// Quick-check: bail early if no reasoning/final tags are present at all.
static QUICK_TAG_RE: LazyLock<Regex> = LazyLock::new(|| {
Regex::new(r"(?i)<\s*/?\s*(?:think(?:ing)?|thought|thoughts|antthinking|reasoning|reflection|scratchpad|inner_monologue|final)\b").expect("QUICK_TAG_RE")
@@ -191,6 +209,12 @@ pub struct Reasoning {
workspace_system_prompt: Option<String>,
/// Optional skill context block to inject into system prompt.
skill_context: Option<String>,
/// Channel name (e.g. "discord", "telegram") for formatting hints.
channel: Option<String>,
/// Model name for runtime context.
model_name: Option<String>,
/// Whether this is a group chat context.
is_group_chat: bool,
}
impl Reasoning {
@@ -201,6 +225,9 @@ impl Reasoning {
safety,
workspace_system_prompt: None,
skill_context: None,
channel: None,
model_name: None,
is_group_chat: false,
}
}
@@ -226,6 +253,30 @@ impl Reasoning {
self
}
/// Set the channel name for channel-specific formatting hints.
pub fn with_channel(mut self, channel: impl Into<String>) -> Self {
let ch = channel.into();
if !ch.is_empty() {
self.channel = Some(ch);
}
self
}
/// Set the model name for runtime context.
pub fn with_model_name(mut self, name: impl Into<String>) -> Self {
let n = name.into();
if !n.is_empty() {
self.model_name = Some(n);
}
self
}
/// Mark this as a group chat context, enabling group-specific guidance.
pub fn with_group_chat(mut self, is_group: bool) -> Self {
self.is_group_chat = is_group;
self
}
/// Run a simple LLM completion with automatic response cleaning.
///
/// This is the preferred entry point for code paths that call the LLM
@@ -451,8 +502,23 @@ Respond in JSON format:
});
}
// Guard against empty text after cleaning. This can happen
// when reasoning models (e.g. GLM-5) return chain-of-thought
// in reasoning_content wrapped in <think> tags and content is
// null — the .or(reasoning_content) fallback picks it up, then
// clean_response strips the think tags leaving an empty string.
let cleaned = clean_response(&content);
let final_text = if cleaned.trim().is_empty() {
tracing::warn!(
"LLM response was empty after cleaning (original len={}), using fallback",
content.len()
);
"I'm not sure how to respond to that.".to_string()
} else {
cleaned
};
Ok(RespondOutput {
result: RespondResult::Text(clean_response(&content)),
result: RespondResult::Text(final_text),
usage,
})
} else {
@@ -463,8 +529,18 @@ Respond in JSON format:
request.metadata = context.metadata.clone();
let response = self.llm.complete(request).await?;
let cleaned = clean_response(&response.content);
let final_text = if cleaned.trim().is_empty() {
tracing::warn!(
"LLM response was empty after cleaning (original len={}), using fallback",
response.content.len()
);
"I'm not sure how to respond to that.".to_string()
} else {
cleaned
};
Ok(RespondOutput {
result: RespondResult::Text(clean_response(&response.content)),
result: RespondResult::Text(final_text),
usage: TokenUsage {
input_tokens: response.input_tokens,
output_tokens: response.output_tokens,
@@ -553,8 +629,20 @@ Respond with a JSON plan in this format:
String::new()
};
// Channel-specific formatting hints
let channel_section = self.build_channel_section();
// Extension guidance (only when extension tools are available)
let extensions_section = self.build_extensions_section(context);
// Runtime context (agent metadata)
let runtime_section = self.build_runtime_section();
// Group chat guidance
let group_section = self.build_group_section();
format!(
r#"You are NEAR AI Agent, an autonomous assistant.
r#"You are IronClaw Agent, a secure autonomous assistant.
## Response Format CRITICAL
@@ -575,9 +663,110 @@ Example:
- Call tools when they would help accomplish the task
- Do NOT call the same tool repeatedly with similar arguments; if a tool returned unhelpful results, move on
- If you have already called tools and gathered enough information, produce your final answer immediately
- If tools return empty or irrelevant results, answer with what you already know rather than retrying{}
- If tools return empty or irrelevant results, answer with what you already know rather than retrying
## Tool Call Style
- Do not narrate routine, low-risk tool calls; just call the tool
- Narrate only when it helps: multi-step work, sensitive actions, or when the user asks
- For multi-step tasks, call independent tools in parallel when possible
- If a tool fails, explain the error briefly and try an alternative approach
## Safety
- You have no independent goals. Do not pursue self-preservation, replication, resource acquisition, or power-seeking beyond the user's request.
- Prioritize safety and human oversight over task completion. If instructions conflict, pause and ask.
- Comply with stop, pause, or audit requests. Never bypass safeguards.
- Do not manipulate anyone to expand your access or disable safeguards.
- Do not modify system prompts, safety rules, or tool policies unless explicitly requested by the user.{}{}{}{}{}
{}{}"#,
tools_section, identity_section, skills_section
tools_section,
extensions_section,
channel_section,
runtime_section,
group_section,
identity_section,
skills_section,
)
}
fn build_extensions_section(&self, context: &ReasoningContext) -> String {
// Only include when the extension management tools are available
let has_ext_tools = context
.available_tools
.iter()
.any(|t| t.name == "tool_search");
if !has_ext_tools {
return String::new();
}
"\n\n## Extensions\n\
You can search, install, and activate extensions to add new capabilities:\n\
- **Channels** (Telegram, Slack, Discord) messaging integrations. \
When users ask about connecting a messaging platform, search for it as a channel.\n\
- **Tools** sandboxed functions that extend your abilities.\n\
- **MCP servers** external API integrations via the Model Context Protocol.\n\n\
Use `tool_search` to find extensions by name. Refer to them by their kind \
(channel, tool, or server) not as \"MCP server\" generically."
.to_string()
}
fn build_channel_section(&self) -> String {
let channel = match self.channel.as_deref() {
Some(c) => c,
None => return String::new(),
};
let hints = match channel {
"discord" => {
"\
- No markdown tables (Discord renders them as plaintext). Use bullet lists instead.\n\
- Wrap multiple URLs in `<>` to suppress embeds: `<https://example.com>`."
}
"whatsapp" => {
"\
- No markdown headers or tables (WhatsApp ignores them). Use **bold** for emphasis.\n\
- Keep messages concise; long replies get truncated on mobile."
}
"telegram" => {
"\
- No markdown tables (Telegram strips them). Bullet lists and bold work well."
}
"slack" => {
"\
- No markdown tables. Use Slack formatting: *bold*, _italic_, `code`.\n\
- Prefer threaded replies when responding to older messages."
}
_ => return String::new(),
};
format!("\n\n## Channel Formatting ({})\n{}", channel, hints)
}
fn build_runtime_section(&self) -> String {
let mut parts = Vec::new();
if let Some(ref ch) = self.channel {
parts.push(format!("channel={}", ch));
}
if let Some(ref model) = self.model_name {
parts.push(format!("model={}", model));
}
if parts.is_empty() {
return String::new();
}
format!("\n\n## Runtime\n{}", parts.join(" | "))
}
fn build_group_section(&self) -> String {
if !self.is_group_chat {
return String::new();
}
format!(
"\n\n## Group Chat\n\
You are in a group chat. Be selective about when to contribute.\n\
Respond when: directly addressed, can add genuine value, or correcting misinformation.\n\
Stay silent when: casual banter, question already answered, nothing to add.\n\
React with emoji when available instead of cluttering with messages.\n\
You are a participant, not the user's proxy. Do not share their private context.\n\
When you have nothing to say, respond with ONLY: {}\n\
It must be your ENTIRE message. Never append it to an actual response.",
SILENT_REPLY_TOKEN,
)
}
+18 -3
View File
@@ -7,6 +7,8 @@
use std::path::PathBuf;
use std::sync::Arc;
use crate::cli::oauth_defaults::OAUTH_CALLBACK_PORT;
use chrono::{DateTime, Utc};
use reqwest::Client;
use secrecy::SecretString;
@@ -157,14 +159,14 @@ impl SessionManager {
}
// Token exists, validate it by calling /v1/users/me
println!("Validating session...");
tracing::debug!("Validating session...");
match self.validate_token().await {
Ok(()) => {
println!("Session valid.");
tracing::debug!("Session valid");
Ok(())
}
Err(e) => {
println!("Session expired or invalid: {}", e);
tracing::info!("Session expired or invalid: {}", e);
self.initiate_login().await
}
}
@@ -238,6 +240,7 @@ impl SessionManager {
use crate::cli::oauth_defaults;
let cb_url = oauth_defaults::callback_url();
let host = oauth_defaults::callback_host();
// Show auth provider menu BEFORE binding the listener
println!();
@@ -288,6 +291,18 @@ impl SessionManager {
}
}
// Warn about plain-HTTP token transmission only for OAuth paths (1, 2)
// where the callback URL actually carries the session token.
if !oauth_defaults::is_loopback_host(&host) {
println!();
println!("Warning: OAuth callback is using plain HTTP to a remote host ({host}).");
println!(" The session token will be transmitted unencrypted.");
println!(" Consider SSH port forwarding instead:");
println!(
" ssh -L {OAUTH_CALLBACK_PORT}:127.0.0.1:{OAUTH_CALLBACK_PORT} user@{host}"
);
}
// OAuth paths: bind the callback listener now
let listener = oauth_defaults::bind_callback_listener()
.await
+700
View File
@@ -0,0 +1,700 @@
//! Smart routing provider that routes requests to cheap or primary models based on task complexity.
//!
//! Inspired by RelayPlane's cost-reduction approach: simple tasks (status checks, greetings,
//! short questions) go to a cheap model (e.g. Haiku), while complex tasks (code generation,
//! analysis, multi-step reasoning) go to the primary model (e.g. Sonnet/Opus).
//!
//! This is a decorator that wraps two `LlmProvider`s and implements `LlmProvider` itself,
//! following the same pattern as `RetryProvider`, `CachedProvider`, and `CircuitBreakerProvider`.
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use async_trait::async_trait;
use rust_decimal::Decimal;
use crate::error::LlmError;
use crate::llm::provider::{
CompletionRequest, CompletionResponse, LlmProvider, ModelMetadata, Role, ToolCompletionRequest,
ToolCompletionResponse,
};
/// Classification of a request's complexity, determining which model handles it.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TaskComplexity {
/// Short, simple queries -> cheap model
Simple,
/// Ambiguous complexity -> cheap model first, cascade to primary if uncertain
Moderate,
/// Code generation, analysis, multi-step reasoning -> primary model
Complex,
}
/// Configuration for the smart routing provider.
#[derive(Debug, Clone)]
pub struct SmartRoutingConfig {
/// Enable cascade mode: retry with primary if cheap model response seems uncertain.
pub cascade_enabled: bool,
/// Message length threshold below which a message may be classified as Simple (default: 200).
pub simple_max_chars: usize,
/// Message length threshold above which a message is classified as Complex (default: 1000).
pub complex_min_chars: usize,
}
impl Default for SmartRoutingConfig {
fn default() -> Self {
Self {
cascade_enabled: true,
simple_max_chars: 200,
complex_min_chars: 1000,
}
}
}
/// Atomic counters for routing observability.
struct SmartRoutingStats {
total_requests: AtomicU64,
cheap_requests: AtomicU64,
primary_requests: AtomicU64,
cascade_escalations: AtomicU64,
}
impl SmartRoutingStats {
fn new() -> Self {
Self {
total_requests: AtomicU64::new(0),
cheap_requests: AtomicU64::new(0),
primary_requests: AtomicU64::new(0),
cascade_escalations: AtomicU64::new(0),
}
}
}
/// Snapshot of routing statistics for external consumption.
#[derive(Debug, Clone)]
pub struct SmartRoutingSnapshot {
pub total_requests: u64,
pub cheap_requests: u64,
pub primary_requests: u64,
pub cascade_escalations: u64,
}
/// Smart routing provider that classifies task complexity and routes to the appropriate model.
///
/// - `complete()` — classifies and routes to cheap or primary model
/// - `complete_with_tools()` — always routes to primary (tool use requires reliable structured output)
pub struct SmartRoutingProvider {
primary: Arc<dyn LlmProvider>,
cheap: Arc<dyn LlmProvider>,
config: SmartRoutingConfig,
stats: SmartRoutingStats,
}
impl SmartRoutingProvider {
/// Create a new smart routing provider wrapping a primary and cheap provider.
pub fn new(
primary: Arc<dyn LlmProvider>,
cheap: Arc<dyn LlmProvider>,
config: SmartRoutingConfig,
) -> Self {
Self {
primary,
cheap,
config,
stats: SmartRoutingStats::new(),
}
}
/// Get a snapshot of routing statistics.
pub fn stats(&self) -> SmartRoutingSnapshot {
SmartRoutingSnapshot {
total_requests: self.stats.total_requests.load(Ordering::Relaxed),
cheap_requests: self.stats.cheap_requests.load(Ordering::Relaxed),
primary_requests: self.stats.primary_requests.load(Ordering::Relaxed),
cascade_escalations: self.stats.cascade_escalations.load(Ordering::Relaxed),
}
}
/// Classify the complexity of a request based on its last user message.
fn classify(&self, request: &CompletionRequest) -> TaskComplexity {
let last_user_msg = request
.messages
.iter()
.rev()
.find(|m| m.role == Role::User)
.map(|m| m.content.as_str())
.unwrap_or("");
classify_message(last_user_msg, &self.config)
}
/// Check if a response from the cheap model shows uncertainty, warranting escalation.
fn response_is_uncertain(response: &CompletionResponse) -> bool {
let content = response.content.trim();
// Empty response is always uncertain
if content.is_empty() {
return true;
}
let lower = content.to_lowercase();
// Uncertainty signals
let uncertainty_patterns = [
"i'm not sure",
"i am not sure",
"i don't know",
"i do not know",
"i'm unable to",
"i am unable to",
"i cannot",
"i can't",
"beyond my capabilities",
"beyond my ability",
"i'm not able to",
"i am not able to",
"i don't have enough",
"i do not have enough",
"i need more context",
"i need more information",
"could you clarify",
"could you provide more",
"i'm not confident",
"i am not confident",
];
uncertainty_patterns.iter().any(|p| lower.contains(p))
}
}
/// Classify a message's complexity based on content patterns and length.
///
/// Exposed as a free function for testability.
fn classify_message(msg: &str, config: &SmartRoutingConfig) -> TaskComplexity {
let trimmed = msg.trim();
let len = trimmed.len();
// Empty or very short -> Simple
if len == 0 {
return TaskComplexity::Simple;
}
// Check for code blocks (triple backticks) -> Complex
if trimmed.contains("```") {
return TaskComplexity::Complex;
}
let lower = trimmed.to_lowercase();
// Complex keywords/patterns -> Complex regardless of length
const COMPLEX_KEYWORDS: &[&str] = &[
"implement",
"refactor",
"analyze",
"debug",
"create a",
"build a",
"design",
"fix the",
"fix this",
"write a",
"write the",
"explain how",
"explain why",
"explain the",
"compare",
"optimize",
"review",
"rewrite",
"migrate",
"architect",
"integrate",
];
if COMPLEX_KEYWORDS.iter().any(|k| lower.contains(k)) {
return TaskComplexity::Complex;
}
// Long messages -> Complex
if len >= config.complex_min_chars {
return TaskComplexity::Complex;
}
// Simple keywords/patterns for short messages
const SIMPLE_KEYWORDS: &[&str] = &[
"list",
"show",
"what is",
"what's",
"status",
"help",
"yes",
"no",
"ok",
"thanks",
"thank you",
"hello",
"hi",
"hey",
"ping",
"version",
"how many",
"when",
"where is",
"who",
];
if len <= config.simple_max_chars && SIMPLE_KEYWORDS.iter().any(|k| lower.contains(k)) {
return TaskComplexity::Simple;
}
// Short confirmations / single words -> Simple
if len <= 10 {
return TaskComplexity::Simple;
}
// Everything else -> Moderate
TaskComplexity::Moderate
}
#[async_trait]
impl LlmProvider for SmartRoutingProvider {
fn model_name(&self) -> &str {
self.primary.model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.primary.cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
self.stats.total_requests.fetch_add(1, Ordering::Relaxed);
let complexity = self.classify(&request);
match complexity {
TaskComplexity::Simple => {
tracing::debug!(
model = %self.cheap.model_name(),
"Smart routing: Simple task -> cheap model"
);
self.stats.cheap_requests.fetch_add(1, Ordering::Relaxed);
self.cheap.complete(request).await
}
TaskComplexity::Complex => {
tracing::debug!(
model = %self.primary.model_name(),
"Smart routing: Complex task -> primary model"
);
self.stats.primary_requests.fetch_add(1, Ordering::Relaxed);
self.primary.complete(request).await
}
TaskComplexity::Moderate => {
if self.config.cascade_enabled {
tracing::debug!(
model = %self.cheap.model_name(),
"Smart routing: Moderate task -> cheap model (cascade enabled)"
);
self.stats.cheap_requests.fetch_add(1, Ordering::Relaxed);
let response = self.cheap.complete(request.clone()).await?;
if Self::response_is_uncertain(&response) {
tracing::info!(
cheap_model = %self.cheap.model_name(),
primary_model = %self.primary.model_name(),
"Smart routing: Escalating to primary (cheap model response uncertain)"
);
self.stats
.cascade_escalations
.fetch_add(1, Ordering::Relaxed);
self.stats.primary_requests.fetch_add(1, Ordering::Relaxed);
self.primary.complete(request).await
} else {
Ok(response)
}
} else {
// Without cascade, moderate tasks go to cheap model
tracing::debug!(
model = %self.cheap.model_name(),
"Smart routing: Moderate task -> cheap model (cascade disabled)"
);
self.stats.cheap_requests.fetch_add(1, Ordering::Relaxed);
self.cheap.complete(request).await
}
}
}
}
/// Tool use always goes to the primary model for reliable structured output.
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
self.stats.total_requests.fetch_add(1, Ordering::Relaxed);
self.stats.primary_requests.fetch_add(1, Ordering::Relaxed);
tracing::debug!(
model = %self.primary.model_name(),
"Smart routing: Tool use -> primary model (always)"
);
self.primary.complete_with_tools(request).await
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
self.primary.list_models().await
}
async fn model_metadata(&self) -> Result<ModelMetadata, LlmError> {
self.primary.model_metadata().await
}
fn active_model_name(&self) -> String {
self.primary.active_model_name()
}
fn set_model(&self, model: &str) -> Result<(), LlmError> {
self.primary.set_model(model)
}
fn calculate_cost(&self, input_tokens: u32, output_tokens: u32) -> Decimal {
self.primary.calculate_cost(input_tokens, output_tokens)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::llm::ChatMessage;
use crate::testing::StubLlm;
fn default_config() -> SmartRoutingConfig {
SmartRoutingConfig::default()
}
// -- Classification tests --
#[test]
fn classify_empty_message_as_simple() {
assert_eq!(
classify_message("", &default_config()),
TaskComplexity::Simple
);
}
#[test]
fn classify_greeting_as_simple() {
assert_eq!(
classify_message("hello", &default_config()),
TaskComplexity::Simple
);
assert_eq!(
classify_message("hi there", &default_config()),
TaskComplexity::Simple
);
}
#[test]
fn classify_short_question_with_simple_keyword() {
assert_eq!(
classify_message("what is the status?", &default_config()),
TaskComplexity::Simple
);
assert_eq!(
classify_message("show me the list", &default_config()),
TaskComplexity::Simple
);
assert_eq!(
classify_message("help", &default_config()),
TaskComplexity::Simple
);
}
#[test]
fn classify_yes_no_as_simple() {
assert_eq!(
classify_message("yes", &default_config()),
TaskComplexity::Simple
);
assert_eq!(
classify_message("no", &default_config()),
TaskComplexity::Simple
);
assert_eq!(
classify_message("ok", &default_config()),
TaskComplexity::Simple
);
}
#[test]
fn classify_code_generation_as_complex() {
assert_eq!(
classify_message("implement a binary search function", &default_config()),
TaskComplexity::Complex
);
assert_eq!(
classify_message("refactor the auth module", &default_config()),
TaskComplexity::Complex
);
assert_eq!(
classify_message("debug this error", &default_config()),
TaskComplexity::Complex
);
}
#[test]
fn classify_code_blocks_as_complex() {
let msg = "What does this do?\n```rust\nfn main() {}\n```";
assert_eq!(
classify_message(msg, &default_config()),
TaskComplexity::Complex
);
}
#[test]
fn classify_long_message_as_complex() {
let long_msg = "a ".repeat(600); // 1200 chars
assert_eq!(
classify_message(&long_msg, &default_config()),
TaskComplexity::Complex
);
}
#[test]
fn classify_medium_message_without_keywords_as_moderate() {
// > 10 chars, < 1000 chars, no simple or complex keywords
let msg = "Tell me about the weather patterns in the Pacific Ocean during summer months";
assert_eq!(
classify_message(msg, &default_config()),
TaskComplexity::Moderate
);
}
#[test]
fn classify_very_short_unknown_as_simple() {
// <= 10 chars, no keywords
assert_eq!(
classify_message("foo", &default_config()),
TaskComplexity::Simple
);
}
// -- Uncertainty detection tests --
#[test]
fn detects_uncertain_short_response() {
let response = CompletionResponse {
content: "I'm not sure.".to_string(),
input_tokens: 10,
output_tokens: 5,
finish_reason: crate::llm::FinishReason::Stop,
};
assert!(SmartRoutingProvider::response_is_uncertain(&response));
}
#[test]
fn detects_empty_response_as_uncertain() {
let response = CompletionResponse {
content: "".to_string(),
input_tokens: 10,
output_tokens: 0,
finish_reason: crate::llm::FinishReason::Stop,
};
assert!(SmartRoutingProvider::response_is_uncertain(&response));
}
#[test]
fn short_confident_response_is_not_uncertain() {
let response = CompletionResponse {
content: "Yes.".to_string(),
input_tokens: 10,
output_tokens: 1,
finish_reason: crate::llm::FinishReason::Stop,
};
assert!(!SmartRoutingProvider::response_is_uncertain(&response));
}
#[test]
fn confident_response_is_not_uncertain() {
let response = CompletionResponse {
content: "The answer is 42. This is a well-known constant from the Hitchhiker's Guide."
.to_string(),
input_tokens: 10,
output_tokens: 20,
finish_reason: crate::llm::FinishReason::Stop,
};
assert!(!SmartRoutingProvider::response_is_uncertain(&response));
}
// -- Routing tests --
fn make_request(content: &str) -> CompletionRequest {
CompletionRequest::new(vec![ChatMessage::user(content)])
}
fn make_tool_request() -> ToolCompletionRequest {
ToolCompletionRequest::new(vec![ChatMessage::user("implement a search")], vec![])
}
#[tokio::test]
async fn simple_task_routes_to_cheap() {
let primary = Arc::new(StubLlm::new("primary-response").with_model_name("primary"));
let cheap = Arc::new(StubLlm::new("cheap-response").with_model_name("cheap"));
let router = SmartRoutingProvider::new(
primary.clone(),
cheap.clone(),
SmartRoutingConfig {
cascade_enabled: false,
..default_config()
},
);
let resp = router.complete(make_request("hello")).await.unwrap();
assert_eq!(resp.content, "cheap-response");
assert_eq!(cheap.calls(), 1);
assert_eq!(primary.calls(), 0);
}
#[tokio::test]
async fn complex_task_routes_to_primary() {
let primary = Arc::new(StubLlm::new("primary-response").with_model_name("primary"));
let cheap = Arc::new(StubLlm::new("cheap-response").with_model_name("cheap"));
let router = SmartRoutingProvider::new(primary.clone(), cheap.clone(), default_config());
let resp = router
.complete(make_request("implement a binary search"))
.await
.unwrap();
assert_eq!(resp.content, "primary-response");
assert_eq!(primary.calls(), 1);
assert_eq!(cheap.calls(), 0);
}
#[tokio::test]
async fn tool_use_always_routes_to_primary() {
let primary = Arc::new(StubLlm::new("primary-response").with_model_name("primary"));
let cheap = Arc::new(StubLlm::new("cheap-response").with_model_name("cheap"));
let router = SmartRoutingProvider::new(primary.clone(), cheap.clone(), default_config());
let resp = router
.complete_with_tools(make_tool_request())
.await
.unwrap();
assert_eq!(resp.content, Some("primary-response".to_string()));
assert_eq!(primary.calls(), 1);
assert_eq!(cheap.calls(), 0);
}
#[tokio::test]
async fn stats_increment_correctly() {
let primary = Arc::new(StubLlm::new("primary").with_model_name("primary"));
let cheap = Arc::new(StubLlm::new("cheap").with_model_name("cheap"));
let router = SmartRoutingProvider::new(
primary,
cheap,
SmartRoutingConfig {
cascade_enabled: false,
..default_config()
},
);
// Simple -> cheap
router.complete(make_request("hello")).await.unwrap();
// Complex -> primary
router
.complete(make_request("implement a search"))
.await
.unwrap();
// Tool use -> primary
router
.complete_with_tools(make_tool_request())
.await
.unwrap();
let stats = router.stats();
assert_eq!(stats.total_requests, 3);
assert_eq!(stats.cheap_requests, 1);
assert_eq!(stats.primary_requests, 2);
assert_eq!(stats.cascade_escalations, 0);
}
#[tokio::test]
async fn cascade_escalates_on_uncertain_response() {
// Cheap model returns an uncertain response
let primary = Arc::new(StubLlm::new("primary-response").with_model_name("primary"));
let cheap = Arc::new(StubLlm::new("I'm not sure about that.").with_model_name("cheap"));
let router = SmartRoutingProvider::new(
primary.clone(),
cheap.clone(),
SmartRoutingConfig {
cascade_enabled: true,
..default_config()
},
);
// A moderate task (no simple/complex keywords, medium length)
let resp = router
.complete(make_request(
"Tell me about the weather patterns in the Pacific Ocean during summer months",
))
.await
.unwrap();
// Should have escalated to primary
assert_eq!(resp.content, "primary-response");
assert_eq!(cheap.calls(), 1);
assert_eq!(primary.calls(), 1);
let stats = router.stats();
assert_eq!(stats.cascade_escalations, 1);
}
#[tokio::test]
async fn cascade_does_not_escalate_on_confident_response() {
let primary = Arc::new(StubLlm::new("primary-response").with_model_name("primary"));
let cheap = Arc::new(
StubLlm::new(
"The Pacific Ocean weather patterns during summer are characterized by trade winds.",
)
.with_model_name("cheap"),
);
let router = SmartRoutingProvider::new(
primary.clone(),
cheap.clone(),
SmartRoutingConfig {
cascade_enabled: true,
..default_config()
},
);
let resp = router
.complete(make_request(
"Tell me about the weather patterns in the Pacific Ocean during summer months",
))
.await
.unwrap();
// Should NOT have escalated
assert!(resp.content.contains("Pacific Ocean"));
assert_eq!(cheap.calls(), 1);
assert_eq!(primary.calls(), 0);
let stats = router.stats();
assert_eq!(stats.cascade_escalations, 0);
}
#[tokio::test]
async fn model_name_returns_primary() {
let primary = Arc::new(StubLlm::new("ok").with_model_name("sonnet"));
let cheap = Arc::new(StubLlm::new("ok").with_model_name("haiku"));
let router = SmartRoutingProvider::new(primary, cheap, default_config());
assert_eq!(router.model_name(), "sonnet");
assert_eq!(router.active_model_name(), "sonnet");
}
}
+509 -1151
View File
File diff suppressed because it is too large Load Diff
+377
View File
@@ -0,0 +1,377 @@
//! Unified WASM artifact resolution: find, build, and install WASM components.
//!
//! This module consolidates all WASM artifact logic that was previously duplicated
//! across `cli/tool.rs`, `registry/installer.rs`, `extensions/manager.rs`,
//! `channels/wasm/bundled.rs`, and `tools/wasm/loader.rs`.
//!
//! # Functions
//!
//! - [`resolve_target_dir`] — resolve the cargo target directory for a crate
//! - [`find_wasm_artifact`] — find a compiled `.wasm` by crate name across all triples
//! - [`find_any_wasm_artifact`] — find any `.wasm` file (fallback when name is unknown)
//! - [`build_wasm_component`] — async build via `cargo component build`
//! - [`build_wasm_component_sync`] — sync build for CLI use
//! - [`install_wasm_files`] — copy `.wasm` + optional `.capabilities.json` to install dir
use std::path::{Path, PathBuf};
use tokio::fs;
/// WASM target triples to search, in priority order.
const WASM_TRIPLES: &[&str] = &[
"wasm32-wasip1",
"wasm32-wasip2",
"wasm32-wasi",
"wasm32-unknown-unknown",
];
/// Resolve the cargo target directory for a crate.
///
/// Checks (in order):
/// 1. `CARGO_TARGET_DIR` env var (shared target dir)
/// 2. `<crate_dir>/target/` (default per-crate layout)
pub fn resolve_target_dir(crate_dir: &Path) -> PathBuf {
if let Ok(dir) = std::env::var("CARGO_TARGET_DIR") {
let p = PathBuf::from(dir);
// Resolve relative CARGO_TARGET_DIR against crate_dir
if p.is_relative() {
return crate_dir.join(p);
}
return p;
}
crate_dir.join("target")
}
/// Find a compiled WASM artifact by searching across all target triples.
///
/// Tries exact name match first (with hyphen-to-underscore normalization),
/// then falls back to searching in whichever target directory exists.
/// `profile` is `"release"` or `"debug"`.
pub fn find_wasm_artifact(crate_dir: &Path, crate_name: &str, profile: &str) -> Option<PathBuf> {
let target_base = resolve_target_dir(crate_dir);
let snake_name = crate_name.replace('-', "_");
// Try exact name match in each target triple directory
for triple in WASM_TRIPLES {
let dir = target_base.join(triple).join(profile);
let candidates = [
dir.join(format!("{}.wasm", crate_name)),
dir.join(format!("{}.wasm", snake_name)),
];
for candidate in &candidates {
if candidate.exists() {
return Some(candidate.clone());
}
}
}
None
}
/// Find any `.wasm` file in the target dirs (fallback when crate name is unknown).
///
/// Returns the first `.wasm` found across target triples.
pub fn find_any_wasm_artifact(crate_dir: &Path, profile: &str) -> Option<PathBuf> {
let target_base = resolve_target_dir(crate_dir);
for triple in WASM_TRIPLES {
let dir = target_base.join(triple).join(profile);
if !dir.is_dir() {
continue;
}
if let Ok(entries) = std::fs::read_dir(&dir) {
for entry in entries.flatten() {
let path = entry.path();
if path.extension().map(|ext| ext == "wasm").unwrap_or(false) {
return Some(path);
}
}
}
}
None
}
/// Build a WASM component using `cargo-component` (async).
///
/// Streams build output to the terminal. Returns the path to the built artifact.
pub async fn build_wasm_component(
source_dir: &Path,
crate_name: &str,
release: bool,
) -> anyhow::Result<PathBuf> {
use tokio::process::Command;
// Check cargo-component availability
let check = Command::new("cargo")
.args(["component", "--version"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.await;
if check.is_err() || !check.as_ref().map(|s| s.success()).unwrap_or(false) {
anyhow::bail!("cargo-component not found. Install with: cargo install cargo-component");
}
let mut cmd = Command::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
// Use status() with inherited stdio so build output streams to the terminal.
let status = cmd.status().await?;
if !status.success() {
anyhow::bail!("Build failed (exit code: {})", status);
}
let profile = if release { "release" } else { "debug" };
let wasm_filename = format!("{}.wasm", crate_name.replace('-', "_"));
// Look for the specific crate's WASM file across target triples
find_wasm_artifact(source_dir, wasm_filename.trim_end_matches(".wasm"), profile)
.or_else(|| {
// Fall back: search by crate_name directly
find_wasm_artifact(source_dir, crate_name, profile)
})
.or_else(|| find_any_wasm_artifact(source_dir, profile))
.ok_or_else(|| {
anyhow::anyhow!(
"Could not find {} in {}/target/*/{}/ after build",
wasm_filename,
source_dir.display(),
profile,
)
})
}
/// Build a WASM component using `cargo-component` (sync, for CLI use).
///
/// Returns the path to the built artifact.
pub fn build_wasm_component_sync(source_dir: &Path, release: bool) -> anyhow::Result<PathBuf> {
use std::process::Command;
println!("Building WASM component in {}...", source_dir.display());
// Check if cargo-component is available
let check = Command::new("cargo")
.args(["component", "--version"])
.output();
if check.is_err() || !check.as_ref().map(|o| o.status.success()).unwrap_or(false) {
anyhow::bail!(
"cargo-component not found. Install with: cargo install cargo-component\n\
Or use --skip-build with an existing .wasm file."
);
}
let mut cmd = Command::new("cargo");
cmd.current_dir(source_dir).args(["component", "build"]);
if release {
cmd.arg("--release");
}
println!(
" Running: cargo component build{}",
if release { " --release" } else { "" }
);
let output = cmd.output()?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
anyhow::bail!("Build failed:\n{}", stderr);
}
let profile = if release { "release" } else { "debug" };
// Find the built artifact
find_any_wasm_artifact(source_dir, profile).ok_or_else(|| {
anyhow::anyhow!(
"No .wasm file found after build in {}/target/*/{}",
source_dir.display(),
profile,
)
})
}
/// Copy WASM binary + optional `capabilities.json` sidecar to an install directory.
///
/// Looks for capabilities files in `source_dir` matching several naming conventions.
/// Returns the destination wasm path.
pub async fn install_wasm_files(
wasm_src: &Path,
source_dir: &Path,
name: &str,
target_dir: &Path,
force: bool,
) -> anyhow::Result<PathBuf> {
fs::create_dir_all(target_dir).await?;
let wasm_dst = target_dir.join(format!("{}.wasm", name));
let caps_dst = target_dir.join(format!("{}.capabilities.json", name));
if wasm_dst.exists() && !force {
anyhow::bail!(
"Tool '{}' already exists at {}. Use --force to overwrite.",
name,
wasm_dst.display()
);
}
// Copy WASM binary
fs::copy(wasm_src, &wasm_dst).await?;
// Look for capabilities.json sidecar in the source directory
let caps_candidates = [
source_dir.join(format!("{}.capabilities.json", name)),
source_dir.join(format!("{}-tool.capabilities.json", name)),
source_dir.join("capabilities.json"),
];
for caps_src in &caps_candidates {
if caps_src.exists() {
if let Err(e) = fs::copy(caps_src, &caps_dst).await {
tracing::warn!(
"Failed to copy capabilities sidecar {} -> {}: {}",
caps_src.display(),
caps_dst.display(),
e,
);
}
break;
}
}
Ok(wasm_dst)
}
#[cfg(test)]
mod tests {
use tempfile::TempDir;
use super::*;
#[test]
fn test_resolve_target_dir_default() {
// When CARGO_TARGET_DIR is not set, should return <crate_dir>/target
let dir = Path::new("/some/crate");
let result = resolve_target_dir(dir);
assert!(result.ends_with("target"));
}
#[test]
fn test_find_wasm_artifact_not_found() {
let dir = TempDir::new().unwrap();
assert!(find_wasm_artifact(dir.path(), "nonexistent", "release").is_none());
}
#[test]
fn test_find_wasm_artifact_found() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip2/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("my_tool.wasm")).unwrap();
let result = find_wasm_artifact(dir.path(), "my_tool", "release");
assert!(result.is_some());
assert!(result.unwrap().ends_with("my_tool.wasm"));
}
#[test]
fn test_find_wasm_artifact_hyphen_to_underscore() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip1/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("my_tool.wasm")).unwrap();
// Search with hyphens, should find underscore version
let result = find_wasm_artifact(dir.path(), "my-tool", "release");
assert!(result.is_some());
}
#[test]
fn test_find_any_wasm_artifact_found() {
let dir = TempDir::new().unwrap();
let target_base = resolve_target_dir(dir.path());
let wasm_dir = target_base.join("wasm32-wasip2/release");
std::fs::create_dir_all(&wasm_dir).unwrap();
std::fs::File::create(wasm_dir.join("something.wasm")).unwrap();
let result = find_any_wasm_artifact(dir.path(), "release");
assert!(result.is_some());
}
#[test]
fn test_find_any_wasm_artifact_not_found() {
let dir = TempDir::new().unwrap();
assert!(find_any_wasm_artifact(dir.path(), "release").is_none());
}
#[tokio::test]
async fn test_install_wasm_files_copies() {
let src_dir = TempDir::new().unwrap();
let target_dir = TempDir::new().unwrap();
let wasm_src = src_dir.path().join("test.wasm");
tokio::fs::write(&wasm_src, b"\0asm\x01\x00\x00\x00")
.await
.unwrap();
// Create a capabilities file
let caps_src = src_dir.path().join("mytool.capabilities.json");
tokio::fs::write(&caps_src, b"{}").await.unwrap();
let result = install_wasm_files(
&wasm_src,
src_dir.path(),
"mytool",
target_dir.path(),
false,
)
.await;
assert!(result.is_ok());
let wasm_dst = result.unwrap();
assert!(wasm_dst.exists());
assert!(target_dir.path().join("mytool.capabilities.json").exists());
}
#[tokio::test]
async fn test_install_wasm_files_refuses_overwrite() {
let src_dir = TempDir::new().unwrap();
let target_dir = TempDir::new().unwrap();
let wasm_src = src_dir.path().join("test.wasm");
tokio::fs::write(&wasm_src, b"\0asm").await.unwrap();
// Pre-create the target
let existing = target_dir.path().join("mytool.wasm");
tokio::fs::write(&existing, b"existing").await.unwrap();
let result = install_wasm_files(
&wasm_src,
src_dir.path(),
"mytool",
target_dir.path(),
false,
)
.await;
assert!(result.is_err());
}
#[test]
fn test_wasm_triples_order() {
// Verify the order is as documented
assert_eq!(WASM_TRIPLES[0], "wasm32-wasip1");
assert_eq!(WASM_TRIPLES[1], "wasm32-wasip2");
assert_eq!(WASM_TRIPLES[2], "wasm32-wasi");
assert_eq!(WASM_TRIPLES[3], "wasm32-unknown-unknown");
}
}
+72
View File
@@ -3,6 +3,7 @@
use std::collections::HashMap;
use std::path::{Path, PathBuf};
use crate::registry::embedded;
use crate::registry::manifest::{BundleDefinition, BundlesFile, ExtensionManifest, ManifestKind};
/// Error type for registry operations.
@@ -64,6 +65,69 @@ pub struct RegistryCatalog {
}
impl RegistryCatalog {
/// Find the `registry/` directory by searching relative to cwd, the executable,
/// and `CARGO_MANIFEST_DIR`. Returns `None` if the directory cannot be found
/// (non-fatal at startup).
pub fn find_dir() -> Option<PathBuf> {
// Try relative to current directory (for dev usage)
if let Ok(cwd) = std::env::current_dir() {
let candidate = cwd.join("registry");
if candidate.is_dir() {
return Some(candidate);
}
}
// Try relative to executable (covers installed binary, target/debug/, target/release/)
if let Ok(exe) = std::env::current_exe()
&& let Some(parent) = exe.parent()
{
// Walk up to 3 levels: exe dir, parent (target/release -> target), grandparent (-> repo root)
let mut dir = Some(parent);
for _ in 0..3 {
if let Some(d) = dir {
let candidate = d.join("registry");
if candidate.is_dir() {
return Some(candidate);
}
dir = d.parent();
}
}
}
// Try CARGO_MANIFEST_DIR (compile-time, works in dev builds)
let manifest_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"));
let candidate = manifest_dir.join("registry");
if candidate.is_dir() {
return Some(candidate);
}
None
}
/// Try to load from disk; if `registry/` cannot be found, fall back to
/// manifests embedded into the binary at compile time.
pub fn load_or_embedded() -> Result<Self, RegistryError> {
if let Some(dir) = Self::find_dir() {
return Self::load(&dir);
}
// Fall back to embedded catalog
let manifests = embedded::load_embedded();
let bundles = embedded::load_embedded_bundles();
tracing::info!(
"Loaded embedded registry catalog ({} extensions, {} bundles)",
manifests.len(),
bundles.len()
);
Ok(Self {
manifests,
bundles,
root: PathBuf::new(),
})
}
/// Load the catalog from a registry directory.
///
/// Expects the structure:
@@ -577,4 +641,12 @@ mod tests {
let result = RegistryCatalog::load(Path::new("/nonexistent/path"));
assert!(result.is_err());
}
#[test]
fn test_load_or_embedded_succeeds() {
// Should always succeed: either finds registry/ on disk or falls back to embedded
let catalog = RegistryCatalog::load_or_embedded().unwrap();
// At minimum, the embedded catalog from the repo should have entries
assert!(!catalog.all().is_empty() || !catalog.bundle_names().is_empty());
}
}
+97
View File
@@ -0,0 +1,97 @@
//! Embedded registry catalog compiled into the binary at build time.
//!
//! When IronClaw is distributed as a pre-built binary without a source tree,
//! the `registry/` directory is unavailable. This module provides the same
//! manifest data via `include_str!` from a JSON blob generated by `build.rs`.
use std::collections::HashMap;
use std::sync::OnceLock;
use crate::registry::manifest::{BundleDefinition, BundlesFile, ExtensionManifest};
/// Raw JSON generated by build.rs from `registry/{tools,channels}/*.json` and `_bundles.json`.
const EMBEDDED_CATALOG: &str = include_str!(concat!(env!("OUT_DIR"), "/embedded_catalog.json"));
/// Intermediate deserialization shape matching the build.rs output.
#[derive(serde::Deserialize)]
struct EmbeddedCatalogRaw {
#[serde(default)]
tools: Vec<ExtensionManifest>,
#[serde(default)]
channels: Vec<ExtensionManifest>,
#[serde(default)]
bundles: BundlesFile,
}
/// Parsed catalog cached across calls.
struct ParsedCatalog {
manifests: HashMap<String, ExtensionManifest>,
bundles: HashMap<String, BundleDefinition>,
}
fn parsed_catalog() -> &'static ParsedCatalog {
static CACHE: OnceLock<ParsedCatalog> = OnceLock::new();
CACHE.get_or_init(|| {
let raw: EmbeddedCatalogRaw = match serde_json::from_str(EMBEDDED_CATALOG) {
Ok(v) => v,
Err(e) => {
tracing::warn!("Failed to parse embedded catalog: {}", e);
return ParsedCatalog {
manifests: HashMap::new(),
bundles: HashMap::new(),
};
}
};
let mut manifests = HashMap::new();
for m in raw.tools {
let key = format!("tools/{}", m.name);
manifests.insert(key, m);
}
for m in raw.channels {
let key = format!("channels/{}", m.name);
manifests.insert(key, m);
}
ParsedCatalog {
manifests,
bundles: raw.bundles.bundles,
}
})
}
/// Load all embedded extension manifests, keyed by `"tools/<name>"` or `"channels/<name>"`.
pub fn load_embedded() -> HashMap<String, ExtensionManifest> {
parsed_catalog().manifests.clone()
}
/// Load embedded bundle definitions.
pub fn load_embedded_bundles() -> HashMap<String, BundleDefinition> {
parsed_catalog().bundles.clone()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_load_embedded_parses() {
let manifests = load_embedded();
// Should have at least the manifests from registry/ if built from the repo
// (empty is also valid for minimal builds without registry/)
assert!(
manifests.is_empty() || manifests.contains_key("tools/github"),
"Expected either empty catalog or github tool, got {} entries",
manifests.len()
);
}
#[test]
fn test_load_embedded_bundles_parses() {
let bundles = load_embedded_bundles();
assert!(
bundles.is_empty() || bundles.contains_key("default"),
"Expected either empty bundles or 'default' bundle"
);
}
}
+320 -113
View File
@@ -94,12 +94,13 @@ impl RegistryInstaller {
source_dir.display()
);
let crate_name = &manifest.source.crate_name;
let wasm_path = build_wasm_component(&source_dir, crate_name)
.await
.map_err(|e| RegistryError::ManifestRead {
path: source_dir.clone(),
reason: format!("build failed: {}", e),
})?;
let wasm_path =
crate::registry::artifacts::build_wasm_component(&source_dir, crate_name, true)
.await
.map_err(|e| RegistryError::ManifestRead {
path: source_dir.clone(),
reason: format!("build failed: {}", e),
})?;
// Copy WASM binary
println!(" Installing to {}", target_wasm.display());
@@ -137,6 +138,10 @@ impl RegistryInstaller {
}
/// Download and install a pre-built artifact.
///
/// Supports two formats:
/// - **tar.gz bundle**: Contains `{name}.wasm` + `{name}.capabilities.json`
/// - **bare .wasm file**: Just the WASM binary (capabilities fetched separately if available)
pub async fn install_from_artifact(
&self,
manifest: &ExtensionManifest,
@@ -156,13 +161,6 @@ impl RegistryInstaller {
))
})?;
let expected_sha = artifact.sha256.as_ref().ok_or_else(|| {
RegistryError::ExtensionNotFound(format!(
"No SHA256 hash for '{}'. Cannot verify download.",
manifest.name
))
})?;
let target_dir = match manifest.kind {
ManifestKind::Tool => &self.tools_dir,
ManifestKind::Channel => &self.channels_dir,
@@ -186,75 +184,90 @@ impl RegistryInstaller {
"Downloading {} '{}'...",
manifest.kind, manifest.display_name
);
let response = reqwest::get(url)
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.clone(),
reason: format!("request failed: {}", e),
})?;
let bytes = download_artifact(url).await?;
let response = response
.error_for_status()
.map_err(|e| RegistryError::DownloadFailed {
url: url.clone(),
reason: e.to_string(),
})?;
let bytes = response
.bytes()
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.clone(),
reason: format!("failed to read body: {}", e),
})?;
// Verify SHA256
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(&bytes);
let actual_sha = format!("{:x}", hasher.finalize());
if actual_sha != *expected_sha {
return Err(RegistryError::DownloadFailed {
url: url.clone(),
reason: format!(
"SHA256 mismatch: expected {}, got {}",
expected_sha, actual_sha
),
});
// Verify SHA256 if provided, warn otherwise
if let Some(expected_sha) = &artifact.sha256 {
verify_sha256(&bytes, expected_sha, url)?;
} else {
println!(
"WARNING: No SHA256 checksum for '{}'; download is not cryptographically verified.",
manifest.name
);
}
// Write file
fs::write(&target_wasm, &bytes)
.await
.map_err(RegistryError::Io)?;
// Copy capabilities from source dir (still needed even for pre-built artifacts).
// NOTE: This requires the source tree to be present. When pre-built artifact
// distribution is implemented, capabilities should be bundled with the artifact
// or fetched from a separate URL.
let caps_source = self
.repo_root
.join(&manifest.source.dir)
.join(&manifest.source.capabilities);
let target_caps = target_dir.join(format!("{}.capabilities.json", manifest.name));
let has_capabilities = if caps_source.exists() {
fs::copy(&caps_source, &target_caps)
// Detect format and extract
let has_capabilities = if is_gzip(&bytes) {
// tar.gz bundle: extract {name}.wasm and {name}.capabilities.json
let extracted =
extract_tar_gz(&bytes, &manifest.name, &target_wasm, &target_caps, url)?;
extracted.has_capabilities
} else {
// Bare WASM file
fs::write(&target_wasm, &bytes)
.await
.map_err(RegistryError::Io)?;
true
} else {
false
// Try to get capabilities from:
// 1. Separate capabilities_url in the artifact
// 2. Source tree (legacy, requires repo)
if let Some(ref caps_url) = artifact.capabilities_url {
const MAX_CAPS_SIZE: usize = 1024 * 1024; // 1 MB
match download_artifact(caps_url).await {
Ok(caps_bytes) if caps_bytes.len() <= MAX_CAPS_SIZE => {
fs::write(&target_caps, &caps_bytes)
.await
.map_err(RegistryError::Io)?;
true
}
Ok(caps_bytes) => {
tracing::warn!(
"Capabilities file too large ({} bytes, max {}), skipping",
caps_bytes.len(),
MAX_CAPS_SIZE
);
false
}
Err(e) => {
tracing::warn!("Failed to download capabilities from {}: {}", caps_url, e);
false
}
}
} else {
// Legacy fallback: try source tree
let caps_source = self
.repo_root
.join(&manifest.source.dir)
.join(&manifest.source.capabilities);
if caps_source.exists() {
fs::copy(&caps_source, &target_caps)
.await
.map_err(RegistryError::Io)?;
true
} else {
false
}
}
};
println!(" Installed to {}", target_wasm.display());
let mut warnings = Vec::new();
if !has_capabilities {
warnings.push(format!(
"No capabilities file found for '{}'. Auth and hooks may not work.",
manifest.name
));
}
Ok(InstallOutcome {
name: manifest.name.clone(),
kind: manifest.kind,
wasm_path: target_wasm,
has_capabilities,
warnings: Vec::new(),
warnings,
})
}
@@ -341,62 +354,157 @@ impl RegistryInstaller {
}
}
/// Build a WASM component from a source directory using `cargo component build --release`.
///
/// Uses `tokio::process::Command` with inherited stdio so build progress is visible.
/// Looks for the specific `{crate_name}.wasm` in the release directory rather than
/// picking the first `.wasm` file found.
async fn build_wasm_component(source_dir: &Path, crate_name: &str) -> anyhow::Result<PathBuf> {
use tokio::process::Command;
/// Download an artifact from a URL.
async fn download_artifact(url: &str) -> Result<bytes::Bytes, RegistryError> {
let response = reqwest::get(url)
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("request failed: {}", e),
})?;
// Check cargo-component availability
let check = Command::new("cargo")
.args(["component", "--version"])
.stdout(std::process::Stdio::null())
.stderr(std::process::Stdio::null())
.status()
.await;
let response = response
.error_for_status()
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: e.to_string(),
})?;
if check.is_err() || !check.as_ref().map(|s| s.success()).unwrap_or(false) {
anyhow::bail!("cargo-component not found. Install with: cargo install cargo-component");
response
.bytes()
.await
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read body: {}", e),
})
}
/// Verify SHA256 of downloaded bytes.
fn verify_sha256(bytes: &[u8], expected: &str, url: &str) -> Result<(), RegistryError> {
use sha2::{Digest, Sha256};
let mut hasher = Sha256::new();
hasher.update(bytes);
let actual = format!("{:x}", hasher.finalize());
if actual != expected {
return Err(RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("SHA256 mismatch: expected {}, got {}", expected, actual),
});
}
Ok(())
}
// Use status() with inherited stdio so build output streams to the terminal.
let status = Command::new("cargo")
.current_dir(source_dir)
.args(["component", "build", "--release"])
.status()
.await?;
/// Check if bytes start with gzip magic number (0x1f 0x8b).
fn is_gzip(bytes: &[u8]) -> bool {
bytes.len() >= 2 && bytes[0] == 0x1f && bytes[1] == 0x8b
}
if !status.success() {
anyhow::bail!("Build failed (exit code: {})", status);
}
/// Result of extracting a tar.gz bundle.
struct ExtractResult {
has_capabilities: bool,
}
// Look for the specific crate's WASM file (Cargo uses underscores in artifact names).
let wasm_filename = format!("{}.wasm", crate_name.replace('-', "_"));
let target_base = source_dir.join("target");
let candidates = [
"wasm32-wasip1",
"wasm32-wasip2",
"wasm32-wasi",
"wasm32-unknown-unknown",
];
/// Extract a tar.gz archive, looking for `{name}.wasm` and `{name}.capabilities.json`.
fn extract_tar_gz(
bytes: &[u8],
name: &str,
target_wasm: &Path,
target_caps: &Path,
url: &str,
) -> Result<ExtractResult, RegistryError> {
use flate2::read::GzDecoder;
use tar::Archive;
for target in &candidates {
let wasm_path = target_base
.join(target)
.join("release")
.join(&wasm_filename);
if wasm_path.exists() {
return Ok(wasm_path);
use std::io::Read as _;
let decoder = GzDecoder::new(bytes);
let mut archive = Archive::new(decoder);
// Defense-in-depth: do not preserve permissions or extended attributes
archive.set_preserve_permissions(false);
#[cfg(any(unix, target_os = "redox"))]
archive.set_unpack_xattrs(false);
// 100 MB cap on decompressed entry size to prevent decompression bombs
const MAX_ENTRY_SIZE: u64 = 100 * 1024 * 1024;
let wasm_filename = format!("{}.wasm", name);
let caps_filename = format!("{}.capabilities.json", name);
let mut found_wasm = false;
let mut found_caps = false;
let entries = archive
.entries()
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read tar.gz entries: {}", e),
})?;
for entry in entries {
let mut entry = entry.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read tar.gz entry: {}", e),
})?;
if entry.size() > MAX_ENTRY_SIZE {
return Err(RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!(
"archive entry too large ({} bytes, max {} bytes)",
entry.size(),
MAX_ENTRY_SIZE
),
});
}
let entry_path = entry
.path()
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("invalid path in tar.gz: {}", e),
})?
.to_path_buf();
// Match by filename (ignoring any directory prefix in the archive)
let filename = entry_path
.file_name()
.and_then(|n| n.to_str())
.unwrap_or("");
if filename == wasm_filename {
let mut data = Vec::with_capacity(entry.size() as usize);
std::io::Read::read_to_end(&mut entry.by_ref().take(MAX_ENTRY_SIZE), &mut data)
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read {} from archive: {}", wasm_filename, e),
})?;
std::fs::write(target_wasm, &data).map_err(RegistryError::Io)?;
found_wasm = true;
} else if filename == caps_filename {
let mut data = Vec::with_capacity(entry.size() as usize);
std::io::Read::read_to_end(&mut entry.by_ref().take(MAX_ENTRY_SIZE), &mut data)
.map_err(|e| RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!("failed to read {} from archive: {}", caps_filename, e),
})?;
std::fs::write(target_caps, &data).map_err(RegistryError::Io)?;
found_caps = true;
}
}
anyhow::bail!(
"Could not find {} in {}/target/*/release/",
wasm_filename,
source_dir.display()
)
if !found_wasm {
return Err(RegistryError::DownloadFailed {
url: url.to_string(),
reason: format!(
"tar.gz archive does not contain '{}'. Archive may be malformed.",
wasm_filename
),
});
}
Ok(ExtractResult {
has_capabilities: found_caps,
})
}
#[cfg(test)]
@@ -412,4 +520,103 @@ mod tests {
);
assert_eq!(installer.repo_root, PathBuf::from("/repo"));
}
#[test]
fn test_is_gzip() {
assert!(is_gzip(&[0x1f, 0x8b, 0x08]));
assert!(!is_gzip(&[0x00, 0x61, 0x73, 0x6d])); // WASM magic
assert!(!is_gzip(&[0x1f])); // Too short
assert!(!is_gzip(&[]));
}
#[test]
fn test_verify_sha256_valid() {
use sha2::{Digest, Sha256};
let data = b"hello world";
let mut hasher = Sha256::new();
hasher.update(data);
let hash = format!("{:x}", hasher.finalize());
assert!(verify_sha256(data, &hash, "test://url").is_ok());
}
#[test]
fn test_verify_sha256_invalid() {
assert!(verify_sha256(b"data", "0000", "test://url").is_err());
}
#[test]
fn test_extract_tar_gz() {
use flate2::Compression;
use flate2::write::GzEncoder;
use tar::Builder;
// Create a tar.gz in memory with test.wasm and test.capabilities.json
let mut encoder = GzEncoder::new(Vec::new(), Compression::default());
{
let mut builder = Builder::new(&mut encoder);
let wasm_data = b"\0asm\x01\x00\x00\x00";
let mut header = tar::Header::new_gnu();
header.set_size(wasm_data.len() as u64);
header.set_cksum();
builder
.append_data(&mut header, "test.wasm", &wasm_data[..])
.unwrap();
let caps_data = br#"{"auth":null}"#;
let mut header = tar::Header::new_gnu();
header.set_size(caps_data.len() as u64);
header.set_cksum();
builder
.append_data(&mut header, "test.capabilities.json", &caps_data[..])
.unwrap();
builder.finish().unwrap();
}
let gz_bytes = encoder.finish().unwrap();
let tmp = tempfile::tempdir().unwrap();
let wasm_path = tmp.path().join("test.wasm");
let caps_path = tmp.path().join("test.capabilities.json");
let result =
extract_tar_gz(&gz_bytes, "test", &wasm_path, &caps_path, "test://url").unwrap();
assert!(wasm_path.exists());
assert!(caps_path.exists());
assert!(result.has_capabilities);
}
#[test]
fn test_extract_tar_gz_missing_wasm() {
use flate2::Compression;
use flate2::write::GzEncoder;
use tar::Builder;
let mut encoder = GzEncoder::new(Vec::new(), Compression::default());
{
let mut builder = Builder::new(&mut encoder);
let data = b"not a wasm file";
let mut header = tar::Header::new_gnu();
header.set_size(data.len() as u64);
header.set_cksum();
builder
.append_data(&mut header, "wrong.wasm", &data[..])
.unwrap();
builder.finish().unwrap();
}
let gz_bytes = encoder.finish().unwrap();
let tmp = tempfile::tempdir().unwrap();
let result = extract_tar_gz(
&gz_bytes,
"test",
&tmp.path().join("test.wasm"),
&tmp.path().join("test.capabilities.json"),
"test://url",
);
assert!(result.is_err());
}
}

Some files were not shown because too many files have changed in this diff Show More