Compare commits

..
Author SHA1 Message Date
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
3615967f92 chore: release v0.15.0 (#526)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 16:37:10 -08:00
704d63f16a feat(oauth): route callbacks through web gateway for hosted instances (#555)
* feat: route OAuth callbacks through web gateway for hosted instances

On hosted instances (e.g., NEAR AI), OAuth callbacks can't reach the
local TCP listener on port 9876. This adds a gateway-routed OAuth flow
that works behind reverse proxies and load balancers.

Backend changes:
- Add /oauth/callback as a public route on the web gateway
- PendingOAuthFlow registry shared between ExtensionManager and handler
- Gateway mode auto-detected via IRONCLAW_OAUTH_CALLBACK_URL env var
- Platform state format (instance:nonce) for nginx routing
- Token exchange proxy support via IRONCLAW_OAUTH_EXCHANGE_URL
- Local TCP listener mode preserved as backward-compatible fallback

UX improvements:
- Hide Configure button for tools with auto-resolved OAuth credentials
  (builtin defaults or platform-injected env vars)
- Skip client_id/client_secret fields in setup schema when auto-resolved
- Show Reconfigure only after successful authentication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(oauth): harden gateway callback and refactor AuthResult

- Add 60s timeout to exchange_via_proxy HTTP client (matching exchange_oauth_code)
- Read GATEWAY_AUTH_TOKEN once at ExtensionManager construction instead of
  per-flow from env (prevents coupling and clarifies token provenance)
- Extract oauth_error_page() helper to deduplicate error landing pages
- Remove IRONCLAW_FORCE_GATEWAY_CALLBACK env var (auto-detection suffices)
- Refactor AuthResult into typed AuthStatus enum with constructors,
  eliminating stringly-typed status and Option fields that were always None
- Adapt all handlers (chat, extensions, ws) to new AuthResult/AuthStatus API
- Use setup_url (not validation_endpoint) for awaiting_token responses

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(oauth): address review feedback — empty token guard, test flakiness, doc typos

- Fail early in exchange_via_proxy() when gateway_token is empty instead
  of sending an unauthenticated request to the exchange proxy
- Fix test_oauth_callback_strips_instance_prefix to use an expired flow
  so it never attempts a real HTTP token exchange (prevents CI flakiness)
- Fix doc comments: /auth/callback → /oauth/callback in PendingOAuthFlow
  and ExtensionManager pending_oauth_flows docs

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: clarify strip_instance_prefix safety, wrapper credential fix, test assertion

- Add comment to strip_instance_prefix noting nonces are base64url (no colons)
- Expand wrapper.rs comment explaining the credential_user_id bug fix
- Fix test_oauth_callback_strips_instance_prefix assertion: landing_html
  does not include provider_name on error pages

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 23:47:45 +00:00
902492bcdb feat(web): show error details for failed tool calls (#490)
* feat(web): show error details and input params for failed tool calls

Failed tool calls in the gateway UI previously showed only a red X icon
with an empty expandable body. This change:

- Adds optional `error` and `parameters` fields to `ToolCompleted` SSE
  events so the browser receives failure details in real-time
- Auto-expands failed tool cards to make errors immediately visible
- Adds `StatusUpdate::tool_completed()` constructor that centralizes
  the 5 duplicated construction sites and applies `redact_params()` to
  prevent sensitive values (e.g. secret_save's "value" param) from
  leaking through SSE broadcasts
- Adds `sensitive_params()` trait method to `Tool` for declaring which
  parameters must be redacted before logging, hooks, and UI display
- Adds `redact_params()` utility and wires it through hooks, approvals,
  ActionRecord storage, and debug logs in dispatcher/worker
- Adds `SecretListTool` and `SecretDeleteTool` for LLM-driven secret
  management (values never returned, only names/metadata)
- Fixes auth flow: setup-only extensions show configure modal instead
  of OAuth card; auth_completed SSE dismisses both UI paths
- CI: release workflow creates PR instead of pushing directly to main
- Registry: MissingChecksum error enables source fallback for
  bootstrapping when checksums haven't been populated yet

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: keep original params in PendingApproval for execution, redact only for display

Address two PR review comments:

1. execute_chat_tool_standalone now redacts sensitive params before logging,
   matching the pattern already used in worker.rs.

2. PendingApproval previously stored redacted parameters, which meant
   approved tool calls received "[REDACTED]" instead of the actual values.
   Add a display_parameters field for UI/logs and keep parameters as the
   original values used for execution.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- worker.rs: redact sensitive params before BeforeToolCall hook, matching
  dispatcher.rs — hooks in the autonomous job path now receive redacted
  params instead of raw values
- registry.rs: fix docstring for register_secrets_tools (list, delete,
  not save/list/delete — no SecretSaveTool is registered)
- app.js: fix double toast/loadExtensions in submitConfigureModal —
  for non-OAuth success the auth_completed SSE already handles both,
  so skip them in the HTTP response handler to avoid duplicates

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 15:38:26 -08:00
13697976db feat(extensions): improve auth UX and add load-time validation (#536)
* feat(extensions): add load-time validation for auth capabilities

Catch common misconfigurations (missing auth section, missing setup_url,
short prompts) at startup via tracing::warn instead of silently failing
at auth time.

* feat(extensions): improve auth prompts, setup_url, and showAuthCard

Add setup_url and descriptive prompts to channel and tool capabilities
files. Fix showAuthCard in web gateway and improve extension manager
auth flow messaging.

* refactor(extensions): extract MIN_PROMPT_LENGTH constant in validate()

Address review feedback: replace magic number 30 with a named constant
for readability and maintainability.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 13:57:10 -08:00
cbcd5adcc0 fix(security): restrict query-token auth to SSE endpoints only (#528)
* fix(security): restrict query-token auth to SSE endpoints only

Query-string `?token=xxx` auth was accepted on all endpoints, exposing
the main auth token in server logs, Referer headers, and browser history
for state-changing routes. Now only GET /api/chat/events and
GET /api/logs/events accept query tokens; all other endpoints require
the Authorization header.

Supersedes #364.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add WebSocket endpoint to query-token allowlist, add URL-encoding tests

The WS upgrade at /api/chat/ws also can't set custom headers, so it
needs query-token auth like the SSE endpoints. Also adds tests for
URL-encoded token values to cover the form_urlencoded parser.

Addresses review feedback from Gemini (partially, /api/jobs/{id}/events
is a JSON endpoint not SSE, so it correctly stays excluded) and Copilot
(URL-encoded token test).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:06:51 +00:00
e24c33ff90 fix(ci): flush profraw coverage data in E2E teardown (#550)
The ironclaw binary only handles SIGINT (via tokio::signal::ctrl_c),
not SIGTERM. When conftest.py sent SIGTERM during teardown, the OS
killed the process immediately without running atexit handlers, so
LLVM never flushed .profraw files. cargo llvm-cov report then found
zero profraw files and failed.

- Send SIGINT instead of SIGTERM so the existing ctrl_c handler
  triggers graceful shutdown → main() returns → atexit runs → profraw
  flushed
- Increase shutdown wait from 5s to 10s for graceful cleanup
- Add a diagnostic step to verify profraw files exist before the
  report step, making future issues visible in CI logs

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:05:46 +00:00
f99991d27b fix(wasm): coerce string parameters to schema-declared types (#498)
* fix(wasm): coerce string parameters to schema-declared types

LLMs frequently pass numeric values as JSON strings ("5" instead of 5)
or booleans as strings ("true" instead of true). The WASM module's
serde deserializer rejects these type mismatches. This adds a
coerce_params_to_schema() helper that walks the params JSON object
and converts string values to their schema-declared types (number,
integer, boolean) before passing to the WASM module.

Adds 5 unit tests covering number, integer, boolean coercion,
already-correct types, and unparseable strings.

Closes #486

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use in-place mutation and case-insensitive boolean coercion

Address review feedback:
- Use get_mut instead of clone+insert to avoid allocations
- Make boolean coercion case-insensitive (handles "True", "FALSE", etc.)
- Expand boolean test to cover false and mixed-case values

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: collapse nested if-let to satisfy clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 18:19:01 +00:00
89600e2b5c fix(agent): strip leaked [Called tool ...] text from responses (#497)
* fix(agent): strip leaked [Called tool ...] text from agent responses

When the NEAR AI provider flattens tool_call messages to plain text,
markers like [Called tool ...] and [Tool ... returned: ...] can leak
into the user-visible response if the LLM echoes them back. This adds
a sanitization step in the agentic loop's text response path that
strips these internal markers before returning. If stripping leaves
the response empty, a generic fallback message is returned instead.

Closes #487

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use fold instead of collect+join to avoid heap allocation

Address review feedback: replace Vec collect + join with fold to build
the filtered string directly, avoiding an intermediate heap allocation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-03-04 18:16:26 +00:00
e4e78d8a87 fix(web): reset job list UI on restart failure (#499)
* fix(web): reset job list UI on restart failure

The restartJob() catch handler was missing a loadJobs() call, so the
job row stayed in a stale highlighted state after a failed restart
attempt. Add loadJobs() to match the success path behavior.

Closes #485

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use .finally() for loadJobs() instead of duplicating

Move loadJobs() to a .finally() block so it runs on both success and
failure without duplication.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 10:05:42 -08:00
b9446712e9 fix(telegram): add missing webhook section to capabilities.json (#381)
The Telegram channel capabilities file was missing the `webhook`
block inside `capabilities.channel`, causing the router to fall back
to the default `X-Webhook-Secret` header instead of the Telegram-
specific `X-Telegram-Bot-Api-Secret-Token`.

When a webhook secret is configured (via `telegram_webhook_secret`),
incoming updates are rejected with 401 because Telegram sends the
token in `X-Telegram-Bot-Api-Secret-Token` but the router looks for
`X-Webhook-Secret`.

The existing test in `schema.rs` already expects the correct header
name, confirming this is an oversight in the shipped capabilities
file.

Co-authored-by: SMKRV <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-03-04 14:40:28 +00:00
LawyeredandGitHub 31a4330f24 Fix UTF-8 unsafe truncation in sandbox log capture (#359) 2026-03-04 15:25:26 +01:00
9b47dbbaed fix(security): replace .unwrap() panics in pairing store with proper error handling (#515)
The pairing store called .unwrap() on path.parent() in three locations
(upsert_request, record_failed_approve, add_allow_from). If a path has
no parent (root path or empty), this panics — a potential denial-of-service
vector if an attacker can influence the path.

Added InvalidPath variant to PairingStoreError and replaced all three
.unwrap() calls with ok_or_else error propagation. This follows the
project's no-panics-in-production policy.

Locations fixed:
- upsert_request (line ~227)
- record_failed_approve (line ~322)
- add_allow_from (line ~465)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-04 08:21:19 +00:00
ac3c928853 ci: enhance coverage with feature matrix, postgres, and E2E (#523)
* ci: enhance coverage workflow with feature matrix, postgres, and E2E

Replace single-config coverage job with a multi-job pipeline:

- Mirror test.yml's 3-config feature matrix (all-features, default, libsql-only)
- Add PostgreSQL service (pgvector/pgvector:pg16) with migrations for
  postgres configs so integration tests actually run instead of skipping
- Add E2E coverage job using cargo-llvm-cov instrumented binary with
  Playwright browser tests
- Add coverage-gate roll-up job for branch protection
- Upload per-config flags to Codecov (all-features, default, libsql-only, e2e)
- Forward LLVM coverage env vars in E2E conftest.py so profraw data
  lands where cargo-llvm-cov report expects it

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on coverage workflow

- Avoid setting DATABASE_URL to empty string for libsql-only config;
  use $GITHUB_ENV conditional step so the var is unset entirely
- Add set -euo pipefail and psql -v ON_ERROR_STOP=1 to migrations
  so SQL errors fail the job immediately

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 06:23:53 +00:00
Pierre LE GUENandGitHub bf2a08be94 feat: add local-test skill and Dockerfile.test for web gateway testing (#524)
Add Dockerfile.test as reusable infrastructure for spinning up local
test instances with libsql (no PostgreSQL dependency). Defaults to
port 3003 to avoid conflict with dev server.

Add local-test workspace skill that teaches the agent how to build,
run, and test against local Docker containers using Chrome MCP browser
automation tools. Covers LLM backend configuration, multi-instance
testing, cleanup, and troubleshooting.
2026-03-04 05:53:35 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
308758c27c chore: release v0.14.0 (#480)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 05:02:33 +00:00
f60c91e9a7 ci: enforce regression tests for fix commits (#517)
* ci: enforce regression tests for fix commits

Add a commit-msg hook and CI workflow that require test changes
alongside bug fix commits, ensuring every fix includes a regression
test that would have caught the bug.

- scripts/commit-msg-regression.sh: local git hook (blocks fix commits
  without test changes; exempts static/docs-only; bypass via
  [skip-regression-check] marker)
- .github/workflows/regression-test-check.yml: CI mirror on PRs
  (checks title + commit messages; skip via label)
- scripts/dev-setup.sh: install hook in step 6
- .github/scripts/create-labels.sh: add skip-regression-check label
- CLAUDE.md: document regression test policy

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on regression test enforcement

- Use here-strings instead of echo|grep to avoid misinterpreting
  special characters in variables
- Use git diff -W (whole-function context) to detect edits inside
  existing test functions, not just new #[test] attributes
- Honor [skip-regression-check] in commit messages in CI (not just
  the PR label)
- Use git rev-parse --git-path hooks for worktree-safe hook install

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update .github/workflows/regression-test-check.yml

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-03-04 04:35:54 +00:00
a22d44f2b2 ci: add code coverage with cargo-llvm-cov and Codecov (#511)
* ci: add code coverage with cargo-llvm-cov and Codecov

Add a Coverage workflow that runs on PRs and pushes to main using
cargo-llvm-cov with --all-features, uploading LCOV results to Codecov.
Include codecov.yml config with project/patch targets and ignore rules
for stub files.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: switch Codecov upload to OIDC (tokenless)

Use GitHub OIDC tokens instead of CODECOV_TOKEN secret so coverage
uploads work for fork PRs where secrets are not available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: fail coverage upload strictly on push, leniently on PRs

Use a conditional so pushes to main fail if Codecov upload breaks
(preventing silent reporting gaps) while PRs stay lenient to avoid
blocking fork PRs where OIDC may not be available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: disable Codecov auto-detection to suppress warnings

We provide lcov.info explicitly, so disable auto-search for gcov,
coverage.py, and Xcode formats that produce noisy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: include channels-src and tools-src in coverage reporting

These WASM source directories should be tracked for test coverage
rather than ignored.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: remove stale ignore entries from codecov.yml

The marketplace, ecommerce, taskrabbit, and restaurant stub files
no longer exist in the codebase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: run coverage on push to main only

Avoids running tests twice on PRs (once in test.yml, once for coverage).
Coverage runs on merge to main instead. Simplify fail_ci_if_error to
always true since it only runs on push now.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 02:14:15 +00:00
a181c8b384 fix(web): mobile browser bar obscures chat input (#508)
* fix(web): use dvh units to prevent mobile browser bar from obscuring chat input

On mobile browsers (Brave/Android, Safari/iOS), the bottom navigation bar
covers the chat input because 100vh includes space behind browser chrome.
Switch to 100dvh (dynamic viewport height) with vh fallback for older
browsers, and add safe-area-inset padding for notched devices.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Fix padding declaration in chat input style

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-04 01:55:52 +00:00
35a79caf87 fix(web): assign unique thread_id to manual routine triggers (#500)
* fix(web): assign unique thread_id to manual routine triggers

Manual routine triggers via the web API created an IncomingMessage
without a thread_id, causing session_manager.resolve_thread() to
route the output to whatever thread was last associated with the
(user, "gateway", None) key. This sets a unique thread_id of the
form "routine-{id}-{timestamp}" so each manual trigger gets its own
dedicated thread.

Closes #484

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add ownership check to routine trigger handler (IDOR)

Address review feedback: verify routine.user_id matches the
authenticated user before allowing the trigger, preventing
unauthorized cross-user routine execution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:41:41 +00:00
85999b25a8 fix(web): refresh routine UI after Run Now trigger (#501)
* fix(web): refresh routine UI after "Run Now" trigger

triggerRoutine() only showed a toast but did not refresh the routine
data after triggering. This adds openRoutineDetail() / loadRoutines()
calls after the toast, matching the pattern used by toggleRoutine().

Closes #483

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: only refresh detail view if triggered routine matches current view

Check currentRoutineId === id before refreshing the detail panel to
avoid refreshing the wrong routine's view.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:38:48 +00:00
b60e5e907a fix(skills): use slug for skill download URL from ClawHub (#502)
* fix(web): use slug for skill download URL from ClawHub

The skill install handler was using req.name (display name like
"Markdown Converter") instead of the slug (like "owner/markdown-converter")
when constructing the download URL. The registry endpoint expects a slug,
so display names caused 502 errors.

- Add optional `slug` field to SkillInstallRequest
- Prefer slug over name when building the download URL
- JS installSkill() now sends slug from search results

Closes #482

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard against empty slug string in skill download URL

Filter out empty slug strings so we fall back to name instead of
constructing an invalid download URL.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:53:16 +00:00
d562dc8d90 fix(workspace): thread document path through search results (#503)
* fix(workspace): thread document path through search results

Memory search results were showing chunk UUIDs instead of source file
paths. Thread document_path through RankedResult, SearchResult, and the
RRF fusion pipeline so handlers can display the actual file path.

Fixes #481

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use into_iter to move values instead of cloning

Address review feedback: consume results with into_iter() to move
String fields directly instead of cloning them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:31:30 +00:00
Nick PismenkovandGitHub c239a4fc2a feat: remove the okta tool (#506) 2026-03-03 21:30:54 +00:00
944968bf76 fix(workspace): import custom templates before seeding defaults (#505)
Swap the order of import_from_directory() and seed_if_empty() so that
custom workspace templates from WORKSPACE_IMPORT_DIR take priority
over generic seeds. Previously, seed_if_empty() ran first and created
all default files, causing import_from_directory() to skip everything
since the files already existed in the DB.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 11:14:02 -08:00
18b59ae9a7 feat: add OAuth support for WASM tools in web gateway (#489)
* feat: add OAuth support for WASM tools in web gateway

Extract reusable OAuth functions (build_oauth_url, exchange_oauth_code,
store_oauth_tokens, validate_oauth_token) from CLI into shared
oauth_defaults module, then wire them into the web gateway's
ExtensionManager.

Key changes:
- Install auto-activates WASM tools (no separate Activate button)
- Configure button triggers OAuth flow via save_setup_secrets
- Scope merging: installing a second Google tool triggers re-auth with
  merged scopes from all tools sharing the same secret_name
- Cancel-and-retry: aborting stale OAuth listeners prevents port conflicts
- Post-auth validation: wrong account detected via validation_endpoint
- Reconfigure always re-auths (deletes old token before starting fresh)
- UI shows error toast on OAuth failure, refreshes extension list

Flow: Install → Active → Configure (enter client_id/secret) → Save →
OAuth popup → authorize → done. Second Google tool install auto-triggers
scope expansion OAuth.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Add custom headers support to ValidationEndpointSchema (fixes
  missing Notion-Version header regression)
- Guard activate handler auth check with status == "awaiting_authorization"
  to prevent unexpected OAuth popups
- Add window dimensions to OAuth popup in activateExtension()
- Simplify UTF-8 truncation boundary check

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address Copilot PR review comments (security, UX, bugs)

- Add CSRF state parameter to OAuth flow (random state in auth URL, validated in callback)
- Restore MCP server Activate button in web UI (was hidden for all non-channel extensions)
- Abort JoinHandle in cleanup_expired_auths to prevent port 9876 conflicts
- Fix Google-specific error message for non-Google OAuth providers
- Add has_auth field to ExtensionInfo API response (fixes Configure button visibility)
- Use oauth_defaults::callback_url() instead of hardcoded redirect_uri (both CLI and manager)
- Update auth check comment to match actual behavior (scope expansion + first-time auth)
- Add unit tests for build_oauth_url (basic, PKCE, extra params, state uniqueness)
- Check all required setup secrets (client_id + client_secret) before starting OAuth

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 01:08:40 +08:00
f4855962fc fix: use std::sync::RwLock in MessageTool to avoid runtime panic (#411)
* fix: use std::sync::RwLock in MessageTool to avoid runtime panic

The `requires_approval` method is synchronous but was using
`tokio::sync::RwLock` with `.await` which requires blocking the
runtime. This caused a panic:
"Cannot block the current thread from within a runtime"

Changes:
- Replace `tokio::sync::RwLock` with `std::sync::RwLock` for
  `default_channel` and `default_target` fields
- Use `unwrap_or_else(|e| e.into_inner())` to gracefully handle
  poisoned locks (recovers instead of panicking)
- Update all usages from `.read().await` to `.read().unwrap_or_else()`

The locks are short-held (just cloning strings), making std::sync::RwLock
appropriate for sync methods called from async contexts.

Fixes: "Cannot block the current thread from within a runtime" panic
when the LLM tries to send a message via the message tool.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address code review feedback for MessageTool RwLock fix

- Fix formatting (long lines broken up per rustfmt)
- Add regression test that demonstrates the panic with tokio::sync::RwLock
  and passes with std::sync::RwLock when calling requires_approval()
  (sync method) from async context

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 16:16:02 +00:00
f18fb5173b feat(web): fix jobs UI parity for non-sandbox mode (#491)
* feat(web): fix jobs UI parity for non-sandbox mode

The web gateway Jobs UI was built primarily for sandbox (Docker) jobs.
When running without sandbox (common for NEAR AI hosted envs), multiple
features were broken. This change fixes all of them:

- Agent jobs now broadcast live SSE events to the web UI (Activity tab)
- Agent job restart via scheduler.dispatch_job (not chat message)
- Follow-up prompts for agent jobs via WorkerMessage injection
- Capability flags (can_restart, can_prompt, job_kind) in job detail API
- Rate-limit retry with cap (10 consecutive) and Retry-After header parsing
- Plan interruption on user message (breaks out of plan, re-evaluates)
- Correct SSE status field in mark_completed/mark_failed/mark_stuck
- SseManager preserved across rebuild_state calls

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: fix formatting in db/mod.rs and nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:23:30 +08:00
78878ad7ef Remove restart infrastructure, generalize WASM channel setup (#493)
* refactor: remove restart infrastructure and generalize Telegram-specific code

Remove the gateway restart mechanism (hot-activation works, restart won't
fix activation failures) and generalize Telegram-specific hardcoded checks
so all WASM channels get equal treatment.

Part 1 - Remove restart infrastructure:
- Remove needs_restart from ActionResponse, restart_requested from GatewayState
- Remove gateway_restart_handler, /api/gateway/restart route, exit code 75
- Remove restart overlay JS/CSS (dead code - restartGateway() never called)
- Surface actual activation errors instead of suggesting restart

Part 2 - Generalize Telegram-specific code:
- Replace telegram_owner_id: Option<i64> with generic
  wasm_channel_owner_ids: HashMap<String, i64> (backwards-compatible
  via TELEGRAM_OWNER_ID env var)
- Pairing status check now applies to all active WASM channels
- All channels get 3-step stepper in web UI, remove "coming soon" note
- Remove dead setup_telegram() code (~700 lines) - Telegram's
  capabilities.json declares required_secrets, so the generic
  setup_wasm_channel() path handles it

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add Settings::set() test for wasm_channel_owner_ids

Addresses review feedback: verify that setting per-channel owner IDs
via the dotted-path Settings::set() API works correctly with the new
HashMap<String, i64> type.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): refresh extension stepper after pairing approval

loadPairingRequests only refreshed the pairing section, not the
stepper status. Call loadExtensions() instead so the stepper updates
from "Awaiting Pairing" to "Active" immediately after approval.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:10:32 +08:00
5f841554d5 feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import (#477)
* feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import

Add two new OpenClaw-compatible workspace markdown files:

- TOOLS.md: Environment-specific tool notes (SSH hosts, device names,
  etc.) injected into the system prompt under "## Tool Notes". Seeded
  as comment-only (like HEARTBEAT.md) so it's effectively empty until
  the user adds real content. Not write-protected — the agent can
  update it as it learns the environment.

- BOOTSTRAP.md: First-run onboarding ritual. Injected FIRST in the
  system prompt when present. Guides the agent through introducing
  itself, learning about the user, and updating workspace files.
  Only seeded on truly fresh workspaces (no existing identity files)
  to avoid triggering the ritual on existing deployments. Agent clears
  it via `memory_write(target="bootstrap")` when done.

Add `Workspace::import_from_directory()` for disk-to-DB import:

- Scans a directory for *.md files and imports any that don't already
  exist in the database (never overwrites user edits)
- Controlled by WORKSPACE_IMPORT_DIR env var, runs after seed_if_empty()
- Enables Docker images / deployment scripts to ship customized
  workspace templates that override generic seeds
- Backwards compatible: no-op when env var is unset

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Use stable `path.extension() != Some(OsStr::new("md"))` instead of
  unstable `is_none_or` (nightly-only)
- Use `tokio::join!` for concurrent DB reads in fresh-workspace check
- Skip unreadable directory entries instead of failing the entire import
- Skip unreadable files instead of failing the entire import

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 19:00:21 -08:00
6adf95b6d1 fix: wire secrets store into all WASM runtime activation paths (#479)
WASM tools and channels activated at runtime (via web UI or CLI) were
missing secrets store wiring, causing credential injection to silently
fail. Tools like web-search would get 401s from APIs even though the
user had configured their API key.

Four bugs fixed:
- activate_wasm_tool(): WasmToolLoader created without .with_secrets_store()
- register_wasm_from_storage(): hardcoded secrets_store: None
- WasmChannelLoader: no secrets_store field at all (added field + builder)
- activate_wasm_channel() and startup path: both missed wiring secrets

The startup path in app.rs was correct; all runtime paths now match it.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-02 16:56:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8530f44630 chore: release v0.13.1 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 21:49:13 +00:00
5257fecca1 feat: add Brave Web Search WASM tool (#474)
* feat: add Brave Web Search WASM tool

Add a new WASM tool for searching the web via the Brave Search API.
Follows the same architecture as the GitHub WASM tool with zero-exposure
credential injection (X-Subscription-Token header).

Features:
- Full Brave Search API support (query, count, country, search_lang,
  ui_lang, freshness)
- Input validation on all parameters
- Retry logic for 429/5xx transient errors
- RFC 3986 percent-encoding
- Registry manifest for Extensions tab discovery

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: avoid Vec allocation in is_valid_ui_lang

Use iterator-based destructuring instead of collecting into a Vec,
avoiding a heap allocation in the WASM sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:07:18 -08:00
20073ccf57 fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown
- Make Enter complete the first matching command when autocomplete is
  visible, instead of requiring explicit arrow-key navigation first

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:06:55 -08:00
1a26b1e57f fix: correct download URLs for telegram-mtproto and slack-tool extensions (#470)
The tool manifests pointed to channel bundle URLs (telegram-wasm32-wasip2.tar.gz,
slack-wasm32-wasip2.tar.gz) instead of the tool bundles (telegram-mtproto-...,
slack-tool-...). This caused install to fail because the archive contents
didn't match the expected .wasm filename.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:37:50 -08:00
98 changed files with 6421 additions and 2060 deletions
+3
View File
@@ -62,6 +62,9 @@ create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating workflow labels..."
create "skip-regression-check" "9E9E9E" "Acknowledged: fix without regression test"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
+178
View File
@@ -0,0 +1,178 @@
name: Code Coverage
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
jobs:
coverage:
name: Coverage (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
has_postgres: true
- name: default
flags: ""
has_postgres: true
- name: libsql-only
flags: "--no-default-features --features libsql"
has_postgres: false
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ironclaw_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
key: coverage-${{ matrix.name }}
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Run database migrations
if: matrix.has_postgres
run: |
set -euo pipefail
for f in migrations/V*.sql; do
echo "Applying $f..."
psql -v ON_ERROR_STOP=1 -f "$f"
done
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: ironclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: lcov.info
flags: ${{ matrix.name }}
disable_search: true
use_oidc: true
fail_ci_if_error: true
e2e-coverage:
name: E2E Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
key: e2e-coverage
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Set up coverage instrumentation
run: |
source <(cargo llvm-cov show-env --export-prefix)
# Persist env vars for subsequent steps
echo "RUSTFLAGS=${RUSTFLAGS}" >> "$GITHUB_ENV"
echo "LLVM_PROFILE_FILE=${LLVM_PROFILE_FILE}" >> "$GITHUB_ENV"
echo "CARGO_LLVM_COV=1" >> "$GITHUB_ENV"
echo "CARGO_LLVM_COV_SHOW_ENV=1" >> "$GITHUB_ENV"
echo "CARGO_LLVM_COV_TARGET_DIR=${CARGO_LLVM_COV_TARGET_DIR}" >> "$GITHUB_ENV"
cargo llvm-cov clean --workspace
- name: Build instrumented binary
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: |
pytest tests/e2e/ -v -x --timeout=120
env:
RUST_LOG: ironclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
if: always()
run: |
echo "LLVM_PROFILE_FILE=${LLVM_PROFILE_FILE}"
echo "CARGO_LLVM_COV_TARGET_DIR=${CARGO_LLVM_COV_TARGET_DIR}"
profraw_count=$(find target/ -name '*.profraw' 2>/dev/null | wc -l)
echo "Found ${profraw_count} .profraw files under target/"
find target/ -name '*.profraw' 2>/dev/null || true
if [ "$profraw_count" -eq 0 ]; then
echo "::warning::No .profraw files found — coverage report will fail"
fi
- name: Generate coverage report
if: always()
run: cargo llvm-cov report --lcov --output-path e2e-coverage.info
- name: Upload to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: e2e-coverage.info
flags: e2e
disable_search: true
use_oidc: true
fail_ci_if_error: true
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
coverage-gate:
name: Coverage
runs-on: ubuntu-latest
if: always()
needs: [coverage, e2e-coverage]
steps:
- run: |
if [[ "${{ needs.coverage.result }}" != "success" || "${{ needs.e2e-coverage.result }}" != "success" ]]; then
echo "One or more coverage jobs failed"
exit 1
fi
+107
View File
@@ -0,0 +1,107 @@
name: Regression Test Check
on:
pull_request:
jobs:
regression-test:
name: Regression test enforcement
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for regression tests
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: |
set -euo pipefail
BASE_REF="origin/${{ github.event.pull_request.base.ref }}"
# --- 1. Is this a fix PR? Check title first, then commit messages ---
IS_FIX=false
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$PR_TITLE"; then
IS_FIX=true
fi
if [ "$IS_FIX" = false ]; then
COMMITS=$(git log --format='%s' "${BASE_REF}..HEAD")
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$COMMITS"; then
IS_FIX=true
fi
fi
if [ "$IS_FIX" = false ]; then
echo "Not a fix PR — skipping regression test check."
exit 0
fi
echo "Fix PR detected."
# --- 2. Skip label or commit message marker ---
if grep -qF ',skip-regression-check,' <<< ",$PR_LABELS,"; then
echo "skip-regression-check label present — skipping."
exit 0
fi
COMMIT_BODIES=$(git log --format='%B' "${BASE_REF}..HEAD")
if grep -qF '[skip-regression-check]' <<< "$COMMIT_BODIES"; then
echo "[skip-regression-check] found in commit message — skipping."
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
CHANGED_FILES=$(git diff --name-only "${BASE_REF}...HEAD")
if [ -z "$CHANGED_FILES" ]; then
echo "No changed files — skipping."
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$CHANGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
echo "All changes are static assets or docs — skipping."
exit 0
fi
# --- 4. Look for test changes ---
# Fast path: new test attributes or test modules in added lines.
if git diff "${BASE_REF}...HEAD" -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
echo "Test changes found in .rs files."
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
if git diff "${BASE_REF}...HEAD" -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
echo "Test changes found in existing test functions."
exit 0
fi
if grep -qE '^tests/' <<< "$CHANGED_FILES"; then
echo "Test file changes found under tests/."
exit 0
fi
# --- 5. No tests found ---
echo "::warning::This PR looks like a bug fix but contains no test changes. Every fix should include a regression test. Add a #[test] or #[tokio::test], or apply the 'skip-regression-check' label if not feasible."
exit 1
+12 -2
View File
@@ -413,6 +413,9 @@ jobs:
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
permissions:
contents: write
pull-requests: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
@@ -445,7 +448,7 @@ jobs:
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
- name: Create PR with updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
@@ -453,8 +456,15 @@ jobs:
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
BRANCH="chore/update-checksums-$(date +%s)"
git checkout -b "$BRANCH"
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
git push origin "$BRANCH"
gh pr create \
--title "chore: update WASM artifact SHA256 checksums" \
--body "Auto-generated by release CI. Updates SHA256 checksums in registry manifests to match the released WASM artifacts." \
--base main \
--head "$BRANCH"
fi
announce:
+60
View File
@@ -7,6 +7,66 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.15.0](https://github.com/nearai/ironclaw/compare/v0.14.0...v0.15.0) - 2026-03-04
### Added
- *(oauth)* route callbacks through web gateway for hosted instances ([#555](https://github.com/nearai/ironclaw/pull/555))
- *(web)* show error details for failed tool calls ([#490](https://github.com/nearai/ironclaw/pull/490))
- *(extensions)* improve auth UX and add load-time validation ([#536](https://github.com/nearai/ironclaw/pull/536))
- add local-test skill and Dockerfile.test for web gateway testing ([#524](https://github.com/nearai/ironclaw/pull/524))
### Fixed
- *(security)* restrict query-token auth to SSE endpoints only ([#528](https://github.com/nearai/ironclaw/pull/528))
- *(ci)* flush profraw coverage data in E2E teardown ([#550](https://github.com/nearai/ironclaw/pull/550))
- *(wasm)* coerce string parameters to schema-declared types ([#498](https://github.com/nearai/ironclaw/pull/498))
- *(agent)* strip leaked [Called tool ...] text from responses ([#497](https://github.com/nearai/ironclaw/pull/497))
- *(web)* reset job list UI on restart failure ([#499](https://github.com/nearai/ironclaw/pull/499))
- *(security)* replace .unwrap() panics in pairing store with proper error handling ([#515](https://github.com/nearai/ironclaw/pull/515))
### Other
- Fix UTF-8 unsafe truncation in sandbox log capture ([#359](https://github.com/nearai/ironclaw/pull/359))
- enhance coverage with feature matrix, postgres, and E2E ([#523](https://github.com/nearai/ironclaw/pull/523))
## [0.14.0](https://github.com/nearai/ironclaw/compare/v0.13.1...v0.14.0) - 2026-03-04
### Added
- remove the okta tool ([#506](https://github.com/nearai/ironclaw/pull/506))
- add OAuth support for WASM tools in web gateway ([#489](https://github.com/nearai/ironclaw/pull/489))
- *(web)* fix jobs UI parity for non-sandbox mode ([#491](https://github.com/nearai/ironclaw/pull/491))
- *(workspace)* add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import ([#477](https://github.com/nearai/ironclaw/pull/477))
### Fixed
- *(web)* mobile browser bar obscures chat input ([#508](https://github.com/nearai/ironclaw/pull/508))
- *(web)* assign unique thread_id to manual routine triggers ([#500](https://github.com/nearai/ironclaw/pull/500))
- *(web)* refresh routine UI after Run Now trigger ([#501](https://github.com/nearai/ironclaw/pull/501))
- *(skills)* use slug for skill download URL from ClawHub ([#502](https://github.com/nearai/ironclaw/pull/502))
- *(workspace)* thread document path through search results ([#503](https://github.com/nearai/ironclaw/pull/503))
- *(workspace)* import custom templates before seeding defaults ([#505](https://github.com/nearai/ironclaw/pull/505))
- use std::sync::RwLock in MessageTool to avoid runtime panic ([#411](https://github.com/nearai/ironclaw/pull/411))
- wire secrets store into all WASM runtime activation paths ([#479](https://github.com/nearai/ironclaw/pull/479))
### Other
- enforce regression tests for fix commits ([#517](https://github.com/nearai/ironclaw/pull/517))
- add code coverage with cargo-llvm-cov and Codecov ([#511](https://github.com/nearai/ironclaw/pull/511))
- Remove restart infrastructure, generalize WASM channel setup ([#493](https://github.com/nearai/ironclaw/pull/493))
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
+3
View File
@@ -321,6 +321,8 @@ cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Regression test with every fix:** Every bug fix must include a test that would have caught the bug. Add a `#[test]` or `#[tokio::test]` that reproduces the original failure. Exempt: changes limited to `src/channels/web/static/` or `.md` files. Use `[skip-regression-check]` in commit message or PR label if genuinely not feasible. The `commit-msg` hook and CI workflow enforce this automatically.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
@@ -328,6 +330,7 @@ Dead code behind the wrong `#[cfg]` gate will only show up when building with a
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
## Configuration
Generated
+1 -1
View File
@@ -2828,7 +2828,7 @@ dependencies = [
[[package]]
name = "ironclaw"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"aes-gcm",
"aho-corasick",
+1 -2
View File
@@ -12,14 +12,13 @@ exclude = [
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/okta",
"tools-src/slack",
"tools-src/telegram",
]
[package]
name = "ironclaw"
version = "0.13.0"
version = "0.15.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
+57
View File
@@ -0,0 +1,57 @@
# Lightweight test Dockerfile for IronClaw web gateway testing.
#
# Build:
# docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
#
# Run (each on a different port):
# docker run --rm -p 3003:3003 ironclaw-test
# docker run --rm -p 3004:3003 ironclaw-test
# docker run --rm -p 3005:3003 ironclaw-test
# Stage 1: Build (libsql only — no PostgreSQL dependency)
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --no-default-features --features libsql --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
WORKDIR /home/ironclaw
EXPOSE 3003
ENV RUST_LOG=ironclaw=info \
GATEWAY_ENABLED=true \
GATEWAY_HOST=0.0.0.0 \
GATEWAY_PORT=3003 \
GATEWAY_AUTH_TOKEN=test \
DATABASE_BACKEND=libsql \
LIBSQL_PATH=/home/ironclaw/test.db \
SANDBOX_ENABLED=false
ENTRYPOINT ["ironclaw", "--no-onboard"]
@@ -6,15 +6,16 @@
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"prompt": "Enter your Discord Bot Token. Find it under Bot > Token in your Discord Application settings.",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (from Developer Portal > General Information)",
"prompt": "Enter your Discord Application Public Key (found under General Information in your Discord Application settings).",
"optional": false
}
]
],
"setup_url": "https://discord.com/developers/applications"
},
"capabilities": {
"http": {
+4 -3
View File
@@ -6,15 +6,16 @@
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"prompt": "Enter your Slack Bot User OAuth Token (starts with xoxb-). Find it under OAuth & Permissions in your Slack App settings.",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"prompt": "Enter your Slack App Signing Secret (found under Basic Information > App Credentials in your Slack App settings).",
"optional": false
}
]
],
"setup_url": "https://api.slack.com/apps"
},
"capabilities": {
"http": {
@@ -9,7 +9,8 @@
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
],
"setup_url": "https://t.me/BotFather"
},
"capabilities": {
"http": {
@@ -39,6 +40,10 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token",
"secret_name": "telegram_webhook_secret"
}
}
},
@@ -6,7 +6,7 @@
"required_secrets": [
{
"name": "whatsapp_access_token",
"prompt": "Enter your WhatsApp Cloud API access token (from Meta Developer Portal)",
"prompt": "Enter your WhatsApp Cloud API permanent access token (from the Meta Developer Portal under your app's WhatsApp > API Setup).",
"validation": "^[A-Za-z0-9_-]+$"
},
{
@@ -16,7 +16,8 @@
"auto_generate": { "length": 32 }
}
],
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}"
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}",
"setup_url": "https://developers.facebook.com/apps"
},
"capabilities": {
"http": {
+10
View File
@@ -0,0 +1,10 @@
coverage:
status:
project:
default:
target: auto
threshold: 1%
patch:
default:
target: 80%
threshold: 5%
-31
View File
@@ -1,31 +0,0 @@
{
"name": "okta",
"display_name": "Okta",
"kind": "tool",
"version": "0.1.0",
"description": "Okta SSO for user profile, app catalog, and SSO launch links",
"keywords": ["sso", "identity", "authentication", "okta"],
"source": {
"dir": "tools-src/okta",
"capabilities": "okta-tool.capabilities.json",
"crate_name": "okta-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Okta",
"secrets": ["okta_oauth_token"],
"shared_auth": null,
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/"
},
"tags": ["identity"]
}
+31
View File
@@ -0,0 +1,31 @@
{
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.1.0",
"description": "Search the web using Brave Search API",
"keywords": ["search", "web", "brave", "internet"],
"source": {
"dir": "tools-src/web-search",
"capabilities": "web-search-tool.capabilities.json",
"crate_name": "web-search-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Brave",
"secrets": ["brave_api_key"],
"shared_auth": null,
"setup_url": "https://brave.com/search/api/"
},
"tags": ["default", "search"]
}
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# commit-msg hook: require regression tests for fix commits.
#
# Installed by scripts/dev-setup.sh as .git/hooks/commit-msg.
# Bypass with [skip-regression-check] in the commit message.
set -euo pipefail
MSG_FILE="$1"
FIRST_LINE=$(head -1 "$MSG_FILE")
# --- 1. Is this a fix commit? ---
if ! grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$FIRST_LINE"; then
exit 0
fi
# --- 2. Skip marker ---
if grep -qF '[skip-regression-check]' "$MSG_FILE"; then
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
# Get staged files (commit-msg runs after staging is finalized).
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACMR)
if [ -z "$STAGED_FILES" ]; then
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$STAGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
exit 0
fi
# --- 4. Look for test changes in staged .rs files ---
# Fast path: new test attributes or test modules in added lines.
if git diff --cached -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
# -W shows the full enclosing function, so #[test] appears in context
# lines when changes are inside a test function.
if git diff --cached -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
exit 0
fi
# Also check for new/modified files under tests/
if grep -qE '^tests/' <<< "$STAGED_FILES"; then
exit 0
fi
# --- 5. No test found — block the commit ---
echo ""
echo "╔══════════════════════════════════════════════════════════════╗"
echo "║ REGRESSION TEST REQUIRED ║"
echo "║ ║"
echo "║ This commit looks like a bug fix but has no test changes. ║"
echo "║ Every fix should include a test that reproduces the bug. ║"
echo "║ ║"
echo "║ Options: ║"
echo "║ • Add a #[test] or #[tokio::test] that catches the bug ║"
echo "║ • Add [skip-regression-check] to your commit message ║"
echo "╚══════════════════════════════════════════════════════════════╝"
echo ""
exit 1
+17 -5
View File
@@ -24,14 +24,14 @@ if ! command -v rustup &>/dev/null; then
echo "ERROR: rustup not found. Install from https://rustup.rs"
exit 1
fi
echo "[1/5] rustup found: $(rustup --version 2>/dev/null | head -1)"
echo "[1/6] rustup found: $(rustup --version 2>/dev/null | head -1)"
# 2. Add WASM target (required by build.rs for channel compilation)
echo "[2/5] Adding wasm32-wasip2 target..."
echo "[2/6] Adding wasm32-wasip2 target..."
rustup target add wasm32-wasip2
# 3. Install wasm-tools (required by build.rs for WASM component model)
echo "[3/5] Installing wasm-tools..."
echo "[3/6] Installing wasm-tools..."
if command -v wasm-tools &>/dev/null; then
echo " wasm-tools already installed: $(wasm-tools --version)"
else
@@ -39,13 +39,25 @@ else
fi
# 4. Verify the project compiles
echo "[4/5] Running cargo check..."
echo "[4/6] Running cargo check..."
cargo check
# 5. Run tests using libsql temp DB (no Docker/external DB needed)
echo "[5/5] Running tests (no external DB required)..."
echo "[5/6] Running tests (no external DB required)..."
cargo test
# 6. Install git hooks
echo "[6/6] Installing git hooks..."
HOOKS_DIR=$(git rev-parse --git-path hooks 2>/dev/null) || true
if [ -n "$HOOKS_DIR" ]; then
mkdir -p "$HOOKS_DIR"
SCRIPT_ABS="$(cd "$(dirname "$0")" && pwd)/commit-msg-regression.sh"
ln -sf "$SCRIPT_ABS" "$HOOKS_DIR/commit-msg"
echo " commit-msg hook installed (regression test enforcement)"
else
echo " Skipped: not a git repository"
fi
echo ""
echo "=== Setup complete ==="
echo ""
+225
View File
@@ -0,0 +1,225 @@
---
name: local-test
version: 0.1.0
description: Build, run, and test IronClaw locally using Docker containers and Chrome MCP browser automation.
activation:
keywords:
- test locally
- local test
- docker test
- test my changes
- test in docker
- test web gateway
- spin up test
- test container
patterns:
- "test.*local"
- "docker.*test"
- "spin.*up.*test"
- "test.*changes.*docker"
max_context_tokens: 3000
---
# Local Testing with Docker + Chrome MCP
Use this skill to build, run, and test IronClaw web gateway changes locally using `Dockerfile.test` and Chrome MCP browser automation tools.
## Quick Start
```bash
# Build the test image (libsql-only, no PostgreSQL needed)
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
# Run on port 3003 (default)
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<key> \
ironclaw-test
# Open in browser
# http://localhost:3003/?token=test
```
## Building the Image
The test Dockerfile uses a two-stage build: Rust compilation with `--features libsql` (no PostgreSQL dependency), then a minimal Debian runtime image.
```bash
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
```
Build takes ~5-10 minutes on first run (cached subsequent builds are faster). The `--platform linux/amd64` flag avoids QEMU warnings on Apple Silicon but can be omitted if targeting native architecture.
## Running Containers
### Required Environment Variables
| Variable | Purpose | Default in Dockerfile |
|----------|---------|----------------------|
| `ONBOARD_COMPLETED=true` | Skip onboarding wizard (exits immediately otherwise) | not set |
| `CLI_ENABLED=false` | Disable TUI/REPL (causes EOF shutdown otherwise) | not set |
### LLM Backend Configuration
Pick ONE of these configurations:
**NEAR AI (API key mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<your-key> \
ironclaw-test
```
**NEAR AI (session token mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_SESSION_TOKEN=<sess_xxx> \
-e NEARAI_BASE_URL=https://private.near.ai \
ironclaw-test
```
**OpenAI:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=openai \
-e OPENAI_API_KEY=<your-key> \
ironclaw-test
```
**Anthropic:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=anthropic \
-e ANTHROPIC_API_KEY=<your-key> \
ironclaw-test
```
**Dummy run (no LLM, just test the UI loads):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=dummy \
ironclaw-test
```
### Common Overrides
| Variable | Purpose | Example |
|----------|---------|---------|
| `GATEWAY_PORT` | Change the listen port | `3003` (default) |
| `GATEWAY_AUTH_TOKEN` | Auth token for API | `test` (default) |
| `NEARAI_MODEL` | Override LLM model | `claude-3-5-sonnet-20241022` |
| `RUST_LOG` | Logging verbosity | `ironclaw=debug` |
| `ROUTINES_ENABLED` | Enable routines | `true`/`false` |
| `SKILLS_ENABLED` | Enable skills system | `true` (default) |
### Multi-Instance Testing
Run multiple containers on different host ports:
```bash
docker run --rm -d --name ic-test-a -p 3003:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
docker run --rm -d --name ic-test-b -p 3004:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
```
## Chrome MCP Testing Workflow
Use the Claude for Chrome browser automation tools to test the web UI.
### Step 1: Get Browser Context
```
mcp__claude-in-chrome__tabs_context_mcp
```
Always start here to see current tabs and get fresh tab IDs.
### Step 2: Open the Gateway
```
mcp__claude-in-chrome__tabs_create_mcp url=http://localhost:3003/?token=test
```
### Step 3: Verify the Page
```
mcp__claude-in-chrome__read_page
```
Check for:
- "Connected" indicator in top-right
- All tabs visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### Step 4: Take Screenshots
```
mcp__claude-in-chrome__computer action=screenshot
```
### Step 5: Test Mobile Viewport
```
mcp__claude-in-chrome__resize_window width=375 height=812
mcp__claude-in-chrome__computer action=screenshot
```
Reset to desktop:
```
mcp__claude-in-chrome__resize_window width=1280 height=800
```
### Step 6: Run JavaScript Checks
```
mcp__claude-in-chrome__javascript_tool script="document.querySelector('.connection-status')?.textContent"
```
### Step 7: Test Interactions
Click tabs, send messages, search skills — use `computer` tool with `action=click` and coordinate-based clicks, or use `find` + `form_input` for text entry.
## Cleanup
```bash
# Stop a specific container
docker stop ic-test-a
# Stop all test containers
docker ps --filter ancestor=ironclaw-test -q | xargs -r docker stop
# Remove the test image
docker rmi ironclaw-test
```
## Troubleshooting
### Container exits immediately
- **Missing `ONBOARD_COMPLETED=true`**: The onboarding wizard tries to read stdin, gets EOF, and exits.
- **Missing `CLI_ENABLED=false`**: The REPL channel reads stdin, gets EOF, and shuts down the agent.
### "Model not found" or LLM errors
- Check that your API key/token is valid and the model name is correct.
- For NEAR AI session token mode, you also need `NEARAI_BASE_URL=https://private.near.ai`.
### Platform mismatch warnings on Apple Silicon
- The `--platform linux/amd64` flag causes QEMU emulation warnings — these are harmless.
- Alternatively, omit the flag and build natively if your dependencies support ARM64.
### Port already in use
- The dev server defaults to port 3001; the test Dockerfile defaults to 3003 to avoid conflicts.
- Use a different host port: `-p 3005:3003`.
### Cannot connect from browser
- Verify `GATEWAY_HOST=0.0.0.0` (set by default in Dockerfile).
- Check the container logs: `docker logs <container-id>`.
- Make sure you include the token query param: `?token=test`.
+8 -2
View File
@@ -73,6 +73,8 @@ pub struct AgentDeps {
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>,
}
/// The main agent that coordinates all components.
@@ -111,7 +113,7 @@ impl Agent {
let session_manager = session_manager.unwrap_or_else(|| Arc::new(SessionManager::new()));
let scheduler = Arc::new(Scheduler::new(
let mut scheduler = Scheduler::new(
config.clone(),
context_manager.clone(),
deps.llm.clone(),
@@ -119,7 +121,11 @@ impl Agent {
deps.tools.clone(),
deps.store.clone(),
deps.hooks.clone(),
));
);
if let Some(ref tx) = deps.sse_tx {
scheduler.set_sse_sender(tx.clone());
}
let scheduler = Arc::new(scheduler);
Self {
config,
+133 -18
View File
@@ -15,6 +15,7 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning, ReasoningContext, RespondResult};
use crate::tools::redact_params;
/// Result of the agentic loop execution.
pub(super) enum AgenticLoopResult {
@@ -291,7 +292,11 @@ impl Agent {
match output.result {
RespondResult::Text(text) => {
return Ok(AgenticLoopResult::Response(text));
// Strip internal "[Called tool ...]" text that can leak when
// provider flattening (e.g. NEAR AI) converts tool_calls to
// plain text and the LLM echoes it back.
let sanitized = strip_internal_tool_call_text(&text);
return Ok(AgenticLoopResult::Response(sanitized));
}
RespondResult::ToolCalls {
tool_calls,
@@ -317,14 +322,25 @@ impl Agent {
)
.await;
// Record tool calls in the thread
// Record tool calls in the thread with sensitive params redacted.
// Look up each tool's sensitive_params before acquiring the session lock.
{
let mut redacted_args: Vec<serde_json::Value> =
Vec::with_capacity(tool_calls.len());
for tc in &tool_calls {
let safe = if let Some(tool) = self.tools().get(&tc.name).await {
redact_params(&tc.arguments, tool.sensitive_params())
} else {
tc.arguments.clone()
};
redacted_args.push(safe);
}
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
for tc in &tool_calls {
turn.record_tool_call(&tc.name, tc.arguments.clone());
for (tc, safe_args) in tool_calls.iter().zip(redacted_args) {
turn.record_tool_call(&tc.name, safe_args);
}
}
}
@@ -353,11 +369,22 @@ impl Agent {
for (idx, original_tc) in tool_calls.iter().enumerate() {
let mut tc = original_tc.clone();
// Fetch the tool upfront so we can redact sensitive params
// before they touch hooks or approval display.
let tool_opt = self.tools().get(&tc.name).await;
let sensitive = tool_opt
.as_ref()
.map(|t| t.sensitive_params())
.unwrap_or(&[]);
// Hook: BeforeToolCall (runs before approval so hooks can
// modify parameters — approval is checked on final params)
// modify parameters — approval is checked on final params).
// Hooks receive redacted params so sensitive values are not
// exposed to hook handlers or their logs.
let hook_params = redact_params(&tc.arguments, sensitive);
let event = crate::hooks::HookEvent::ToolCall {
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
parameters: hook_params,
user_id: message.user_id.clone(),
context: "chat".to_string(),
};
@@ -384,8 +411,20 @@ impl Agent {
}
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_params),
}) => match serde_json::from_str(&new_params) {
Ok(parsed) => tc.arguments = parsed,
}) => match serde_json::from_str::<serde_json::Value>(&new_params) {
Ok(mut parsed) => {
// Restore original sensitive param values so a hook
// cannot overwrite them (they were sent as [REDACTED]).
if let Some(obj) = parsed.as_object_mut() {
for key in sensitive {
if let Some(orig_val) = original_tc.arguments.get(*key)
{
obj.insert((*key).to_string(), orig_val.clone());
}
}
}
tc.arguments = parsed;
}
Err(e) => {
tracing::warn!(
tool = %tc.name,
@@ -400,7 +439,7 @@ impl Agent {
// Check if tool requires approval on the final (post-hook)
// parameters. Skipped when auto_approve_tools is set.
if !self.config.auto_approve_tools
&& let Some(tool) = self.tools().get(&tc.name).await
&& let Some(tool) = tool_opt
{
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
@@ -447,14 +486,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let disp_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
disp_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -495,13 +537,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -671,10 +716,15 @@ impl Agent {
// Handle approval if a tool needed it
if let Some((approval_idx, tc, tool)) = approval_needed {
// Show redacted params in the approval UI — the user already knows
// the sensitive value (they provided it); showing it again is
// unnecessary and creates a leakage path through channel logs.
let display_params = redact_params(&tc.arguments, tool.sensitive_params());
let pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: display_params,
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -734,9 +784,10 @@ pub(super) async fn execute_chat_tool_standalone(
.into());
}
let safe_params = redact_params(params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
"Tool call started"
);
@@ -900,6 +951,38 @@ fn compact_messages_for_retry(messages: &[ChatMessage]) -> Vec<ChatMessage> {
compacted
}
/// Strip internal `[Called tool ...]` and `[Tool ... returned: ...]` markers
/// from a response string. These markers are inserted by provider-level message
/// flattening (e.g. NEAR AI) and can leak into the user-visible response when
/// the LLM echoes them back.
fn strip_internal_tool_call_text(text: &str) -> String {
// Remove lines that are purely internal tool-call markers.
// Pattern: lines matching `[Called tool <name>(...)]` or `[Tool <name> returned: ...]`
let result = text
.lines()
.filter(|line| {
let trimmed = line.trim();
!((trimmed.starts_with("[Called tool ") && trimmed.ends_with(']'))
|| (trimmed.starts_with("[Tool ")
&& trimmed.contains(" returned:")
&& trimmed.ends_with(']')))
})
.fold(String::new(), |mut acc, s| {
if !acc.is_empty() {
acc.push('\n');
}
acc.push_str(s);
acc
});
let result = result.trim();
if result.is_empty() {
"I wasn't able to complete that request. Could you try rephrasing or providing more details?".to_string()
} else {
result.to_string()
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -982,6 +1065,7 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
};
Agent::new(
@@ -1085,6 +1169,7 @@ mod tests {
request_id: uuid::Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "echo hi"}),
display_parameters: serde_json::json!({"command": "echo hi"}),
description: "Run shell command".to_string(),
tool_call_id: "call_1".to_string(),
context_messages: vec![],
@@ -1719,6 +1804,7 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
};
Agent::new(
@@ -1830,6 +1916,7 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
};
Agent::new(
@@ -1899,4 +1986,32 @@ mod tests {
}
}
}
#[test]
fn test_strip_internal_tool_call_text_removes_markers() {
let input = "[Called tool search({\"query\": \"test\"})]\nHere is the answer.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Here is the answer.");
}
#[test]
fn test_strip_internal_tool_call_text_removes_returned_markers() {
let input = "[Tool search returned: some result]\nSummary of findings.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Summary of findings.");
}
#[test]
fn test_strip_internal_tool_call_text_all_markers_yields_fallback() {
let input = "[Called tool search({\"query\": \"test\"})]\n[Tool search returned: error]";
let result = super::strip_internal_tool_call_text(input);
assert!(result.contains("wasn't able to complete"));
}
#[test]
fn test_strip_internal_tool_call_text_preserves_normal_text() {
let input = "This is a normal response with [brackets] inside.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, input);
}
}
+32
View File
@@ -10,6 +10,7 @@ use uuid::Uuid;
use crate::agent::task::{Task, TaskContext, TaskOutput};
use crate::agent::worker::{Worker, WorkerDeps};
use crate::channels::web::types::SseEvent;
use crate::config::AgentConfig;
use crate::context::{ContextManager, JobContext, JobState};
use crate::db::Database;
@@ -28,6 +29,8 @@ pub enum WorkerMessage {
Stop,
/// Check health.
Ping,
/// Inject a follow-up user message into the worker's reasoning context.
UserMessage(String),
}
/// Status of a scheduled job.
@@ -51,6 +54,8 @@ pub struct Scheduler {
tools: Arc<ToolRegistry>,
store: Option<Arc<dyn Database>>,
hooks: Arc<HookRegistry>,
/// SSE broadcast sender for live job event streaming.
sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
/// Running jobs (main LLM-driven jobs).
jobs: Arc<RwLock<HashMap<Uuid, ScheduledJob>>>,
/// Running sub-tasks (tool executions, background tasks).
@@ -76,11 +81,17 @@ impl Scheduler {
tools,
store,
hooks,
sse_tx: None,
jobs: Arc::new(RwLock::new(HashMap::new())),
subtasks: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Set the SSE broadcast sender for live job event streaming.
pub fn set_sse_sender(&mut self, tx: tokio::sync::broadcast::Sender<SseEvent>) {
self.sse_tx = Some(tx);
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
@@ -169,6 +180,7 @@ impl Scheduler {
hooks: self.hooks.clone(),
timeout: self.config.job_timeout,
use_planning: self.config.use_planning,
sse_tx: self.sse_tx.clone(),
};
let worker = Worker::new(job_id, deps);
@@ -500,6 +512,26 @@ impl Scheduler {
Ok(())
}
/// Send a follow-up user message to a running job.
///
/// Returns `Ok(())` if the message was queued, `Err` if the job is not running.
pub async fn send_message(&self, job_id: Uuid, content: String) -> Result<(), JobError> {
// Clone the sender while holding the lock, then release before the
// async send to avoid blocking scheduler writes during backpressure.
let tx = {
let jobs = self.jobs.read().await;
let scheduled = jobs.get(&job_id).ok_or(JobError::NotFound { id: job_id })?;
scheduled.tx.clone()
};
tx.send(WorkerMessage::UserMessage(content))
.await
.map_err(|_| JobError::Failed {
id: job_id,
reason: "Worker channel closed".to_string(),
})?;
Ok(())
}
/// Check if a job is running.
pub async fn is_running(&self, job_id: Uuid) -> bool {
self.jobs.read().await.contains_key(&job_id)
+7 -1
View File
@@ -148,8 +148,12 @@ pub struct PendingApproval {
pub request_id: Uuid,
/// Tool name requiring approval.
pub tool_name: String,
/// Tool parameters.
/// Tool parameters (original values, used for execution).
pub parameters: serde_json::Value,
/// Redacted tool parameters (sensitive values replaced with `[REDACTED]`).
/// Used for display in approval UI, logs, and SSE broadcasts.
#[serde(default)]
pub display_parameters: serde_json::Value,
/// Description of what the tool will do.
pub description: String,
/// Tool call ID from LLM (for proper context continuation).
@@ -950,6 +954,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "rm -rf /"}),
display_parameters: serde_json::json!({"command": "rm -rf /"}),
description: "dangerous command".to_string(),
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
@@ -974,6 +979,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "http".to_string(),
parameters: serde_json::json!({}),
display_parameters: serde_json::json!({}),
description: "test".to_string(),
tool_call_id: "call_456".to_string(),
context_messages: vec![],
+31 -20
View File
@@ -21,6 +21,7 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::ChatMessage;
use crate::tools::redact_params;
impl Agent {
/// Hydrate a historical thread from DB into memory if not already present.
@@ -357,7 +358,7 @@ impl Agent {
let request_id = pending.request_id;
let tool_name = pending.tool_name.clone();
let description = pending.description.clone();
let parameters = pending.parameters.clone();
let parameters = pending.display_parameters.clone();
thread.await_approval(pending);
let _ = self
.channels
@@ -751,14 +752,17 @@ impl Agent {
.execute_chat_tool(&pending.tool_name, &pending.parameters, &job_ctx)
.await;
let tool_ref = self.tools().get(&pending.tool_name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: pending.tool_name.clone(),
success: tool_result.is_ok(),
},
StatusUpdate::tool_completed(
pending.tool_name.clone(),
&tool_result,
&pending.display_parameters,
tool_ref.as_deref(),
),
&message.metadata,
)
.await;
@@ -908,14 +912,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let deferred_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
deferred_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -957,13 +964,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -1086,6 +1096,7 @@ impl Agent {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: redact_params(&tc.arguments, tool.sensitive_params()),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -1095,7 +1106,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
{
let mut sess = session.lock().await;
@@ -1162,7 +1173,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
thread.await_approval(new_pending);
let _ = self
.channels
@@ -1284,7 +1295,7 @@ impl Agent {
};
match ext_mgr.auth(&pending.extension_name, Some(token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
tracing::info!(
"Extension '{}' authenticated via auth mode",
pending.extension_name
@@ -1353,8 +1364,8 @@ impl Agent {
}
}
let msg = result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token. Please try again.".to_string());
// Re-emit AuthRequired so web UI re-shows the card
let _ = self
@@ -1364,8 +1375,8 @@ impl Agent {
StatusUpdate::AuthRequired {
extension_name: pending.extension_name.clone(),
instructions: Some(msg.clone()),
auth_url: result.auth_url,
setup_url: result.setup_url,
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
},
&message.metadata,
)
+220 -25
View File
@@ -9,6 +9,7 @@ use uuid::Uuid;
use crate::agent::scheduler::WorkerMessage;
use crate::agent::task::TaskOutput;
use crate::channels::web::types::SseEvent;
use crate::context::{ContextManager, JobState};
use crate::db::Database;
use crate::error::Error;
@@ -17,8 +18,8 @@ use crate::llm::{
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolSelection,
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::rate_limiter::RateLimitResult;
use crate::tools::{ToolRegistry, redact_params};
/// Shared dependencies for worker execution.
///
@@ -34,6 +35,8 @@ pub struct WorkerDeps {
pub hooks: Arc<HookRegistry>,
pub timeout: Duration,
pub use_planning: bool,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
}
/// Worker that executes a single job.
@@ -98,18 +101,90 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
/// Fire-and-forget persistence of a job event and SSE broadcast.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
let job_id = self.job_id;
// Persist to DB
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
let et = event_type.to_string();
let d = data.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
if let Err(e) = store.save_job_event(job_id, &et, &d).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
// Broadcast SSE for live web UI updates
if let Some(ref tx) = self.deps.sse_tx {
let job_id_str = job_id.to_string();
let event = match event_type {
"message" => Some(SseEvent::JobMessage {
job_id: job_id_str,
role: data
.get("role")
.and_then(|v| v.as_str())
.unwrap_or("assistant")
.to_string(),
content: data
.get("content")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"tool_use" => Some(SseEvent::JobToolUse {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
input: data
.get("input")
.cloned()
.unwrap_or(serde_json::Value::Null),
}),
"tool_result" => Some(SseEvent::JobToolResult {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
output: data
.get("output")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"status" => Some(SseEvent::JobStatus {
job_id: job_id_str,
message: data
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"result" => Some(SseEvent::JobResult {
job_id: job_id_str,
status: data
.get("status")
.and_then(|v| v.as_str())
.unwrap_or("completed")
.to_string(),
session_id: data
.get("session_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
}),
_ => None,
};
if let Some(event) = event {
let _ = tx.send(event);
}
}
}
/// Run the worker until the job is complete or stopped.
@@ -123,7 +198,7 @@ impl Worker {
tracing::debug!("Worker for job {} stopped before starting", self.job_id);
return Ok(());
}
Some(WorkerMessage::Ping) => {}
Some(WorkerMessage::Ping) | Some(WorkerMessage::UserMessage(_)) => {}
}
// Get job context
@@ -219,6 +294,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
const MAX_CONSECUTIVE_RATE_LIMITS: usize = 10;
let mut consecutive_rate_limits = 0usize;
// Initial tool definitions for planning (will be refreshed in loop)
reason_ctx.available_tools = self.tools().tool_definitions().await;
@@ -269,15 +346,27 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
None
};
// If we have a plan, execute it
// If we have a plan, execute it. Two exit paths:
// 1. Plan ran to completion → job is Completed or needs continuation
// (check state and only fall through if not terminal)
// 2. Plan was interrupted by UserMessage → fall through to direct loop
if let Some(ref plan) = plan {
return self.execute_plan(rx, reasoning, reason_ctx, plan).await;
self.execute_plan(rx, reasoning, reason_ctx, plan).await?;
// If the plan marked the job terminal, we're done. Only fall
// through to the direct selection loop if the plan was
// interrupted or explicitly left the job in-progress.
if let Ok(ctx) = self.context_manager().get_context(self.job_id).await
&& (ctx.state.is_terminal() || ctx.state == JobState::Stuck)
{
return Ok(());
}
}
// Otherwise, use direct tool selection loop
// Direct tool selection loop (also used as fallback after plan interruption)
loop {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!("Worker for job {} received stop signal", self.job_id);
@@ -287,6 +376,20 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"Worker received follow-up user message"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
}
}
}
@@ -307,12 +410,64 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Refresh tool definitions so newly built tools become visible
reason_ctx.available_tools = self.tools().tool_definitions().await;
// Select next tool(s) to use
let selections = reasoning.select_tools(reason_ctx).await?;
// Select next tool(s) to use, with rate-limit retry.
let selections = match reasoning.select_tools(reason_ctx).await {
Ok(s) => s,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during tool selection, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
if selections.is_empty() {
// No tools from select_tools, ask LLM directly (may still return tool calls)
let respond_output = reasoning.respond_with_tools(reason_ctx).await?;
let respond_output = match reasoning.respond_with_tools(reason_ctx).await {
Ok(o) => o,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during respond_with_tools, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
match respond_output.result {
RespondResult::Text(response) => {
@@ -424,6 +579,11 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
// Reset rate-limit counter after a successful iteration (all LLM
// calls succeeded). Placed here so alternating success/fail between
// select_tools and respond_with_tools cannot bypass the cap.
consecutive_rate_limits = 0;
// Small delay between iterations
tokio::time::sleep(Duration::from_millis(100)).await;
}
@@ -540,9 +700,10 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Run BeforeToolCall hook
let params = {
use crate::hooks::{HookError, HookEvent, HookOutcome};
let hook_params = redact_params(params, tool.sensitive_params());
let event = HookEvent::ToolCall {
tool_name: tool_name.to_string(),
parameters: params.clone(),
parameters: hook_params,
user_id: job_ctx.user_id.clone(),
context: format!("job:{}", job_id),
};
@@ -598,9 +759,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.into());
}
// Redact sensitive parameter values (e.g. secret_save's "value") before
// they touch any observability or audit path.
let safe_params = redact_params(&params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
job = %job_id,
"Tool call started"
);
@@ -652,7 +816,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem.create_action(tool_name, params.clone()).succeed(
let rec = mem.create_action(tool_name, safe_params.clone()).succeed(
output_str.clone(),
output.result.clone(),
elapsed,
@@ -674,7 +838,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail(e.to_string(), elapsed);
mem.record_action(rec.clone());
rec
@@ -693,7 +857,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail("Execution timeout", elapsed);
mem.record_action(rec.clone());
rec
@@ -836,8 +1000,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
plan: &ActionPlan,
) -> Result<(), Error> {
for (i, action) in plan.actions.iter().enumerate() {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!(
@@ -850,6 +1014,29 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"User message received during plan execution, abandoning plan"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
self.log_event(
"status",
serde_json::json!({
"message": "Plan interrupted by user message, re-evaluating...",
}),
);
// Return Ok to break out of plan; caller falls through to
// the direct selection loop for LLM re-evaluation.
return Ok(());
}
}
}
@@ -902,14 +1089,18 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
if crate::util::llm_signals_completion(&response) {
self.mark_completed().await?;
} else {
// Job not complete, could re-plan or fall back to direct selection
// Job not complete — return Ok without marking terminal so the
// caller falls through to the direct selection loop for continuation.
tracing::info!(
"Job {} plan completed but work remains, falling back to direct selection",
self.job_id
);
// Continue with standard execution loop by returning (will be picked up by main loop)
self.mark_stuck("Plan completed but job incomplete - needs re-planning")
.await?;
self.log_event(
"status",
serde_json::json!({
"message": "Plan completed but job needs more work, continuing...",
}),
);
}
Ok(())
@@ -940,6 +1131,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "completed",
"success": true,
"message": "Job completed successfully",
}),
@@ -965,6 +1157,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "failed",
"success": false,
"message": format!("Execution failed: {}", reason),
}),
@@ -985,6 +1178,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "stuck",
"success": false,
"message": format!("Job stuck: {}", reason),
}),
@@ -1103,6 +1297,7 @@ mod tests {
hooks: Arc::new(crate::hooks::HookRegistry::new()),
timeout: Duration::from_secs(30),
use_planning: false,
sse_tx: None,
};
Worker::new(job_id, deps)
+29
View File
@@ -331,6 +331,10 @@ impl AppBuilder {
};
tools.register_builtin_tools();
if let Some(ref ss) = self.secrets_store {
tools.register_secrets_tools(Arc::clone(ss));
}
// Create embeddings provider using the unified method
let embeddings = self
.config
@@ -665,6 +669,31 @@ impl AppBuilder {
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
// Import workspace files from disk FIRST if WORKSPACE_IMPORT_DIR is set.
// This lets Docker images / deployment scripts ship customized
// workspace templates (e.g., AGENTS.md, TOOLS.md) that override
// the generic seeds. Only imports files that don't already exist
// in the database — never overwrites user edits.
//
// Runs before seed_if_empty() so that custom templates take priority
// over generic seeds. seed_if_empty() then fills any remaining gaps.
if let Ok(import_dir) = std::env::var("WORKSPACE_IMPORT_DIR") {
let import_path = std::path::Path::new(&import_dir);
match ws.import_from_directory(import_path).await {
Ok(count) if count > 0 => {
tracing::info!("Imported {} workspace file(s) from {}", count, import_dir);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
"Failed to import workspace files from {}: {}",
import_dir,
e
);
}
}
}
match ws.seed_if_empty().await {
Ok(_) => {}
Err(e) => {
+174 -1
View File
@@ -117,7 +117,20 @@ pub enum StatusUpdate {
/// Tool execution started.
ToolStarted { name: String },
/// Tool execution completed.
ToolCompleted { name: String, success: bool },
///
/// Use [`StatusUpdate::tool_completed`] to construct this variant — it
/// handles redaction of sensitive parameters and keeps the 9-line pattern
/// in one place.
ToolCompleted {
name: String,
success: bool,
/// Error message when success is false.
error: Option<String>,
/// Tool input parameters (JSON string) for display on failure.
/// Only populated when `success` is `false`. Values listed in the
/// tool's `sensitive_params()` are replaced with `"[REDACTED]"`.
parameters: Option<String>,
},
/// Brief preview of tool execution output.
ToolResult { name: String, preview: String },
/// Streaming text chunk.
@@ -152,6 +165,38 @@ pub enum StatusUpdate {
},
}
impl StatusUpdate {
/// Build a `ToolCompleted` status with redacted parameters.
///
/// On failure, serializes the tool's input parameters as pretty JSON after
/// replacing any keys listed in the tool's `sensitive_params()` with
/// `"[REDACTED]"`. On success, no parameters or error are included.
///
/// Pass the resolved `Tool` reference (if available) so this method can
/// query `sensitive_params()` directly — callers don't need to manage the
/// borrow lifetime of the sensitive slice.
pub fn tool_completed(
name: String,
result: &Result<String, crate::error::Error>,
params: &serde_json::Value,
tool: Option<&dyn crate::tools::Tool>,
) -> Self {
let success = result.is_ok();
let sensitive = tool.map(|t| t.sensitive_params()).unwrap_or(&[]);
Self::ToolCompleted {
name,
success,
error: result.as_ref().err().map(|e| e.to_string()),
parameters: if !success {
let safe = crate::tools::redact_params(params, sensitive);
Some(serde_json::to_string_pretty(&safe).unwrap_or_else(|_| safe.to_string()))
} else {
None
},
}
}
}
/// Trait for message channels.
///
/// Channels receive messages from external sources and convert them to
@@ -223,3 +268,131 @@ pub trait Channel: Send + Sync {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Stub tool that marks `"value"` as sensitive.
struct SecretTool;
#[async_trait]
impl crate::tools::Tool for SecretTool {
fn name(&self) -> &str {
"secret_save"
}
fn description(&self) -> &str {
"stub"
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({"type": "object", "properties": {}})
}
async fn execute(
&self,
_params: serde_json::Value,
_ctx: &crate::context::JobContext,
) -> Result<crate::tools::ToolOutput, crate::tools::ToolError> {
unreachable!()
}
fn sensitive_params(&self) -> &[&str] {
&["value"]
}
}
#[test]
fn tool_completed_redacts_sensitive_params_on_failure() {
let params = serde_json::json!({"name": "api_key", "value": "sk-secret-123"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "secret_save".into(),
reason: "db error".into(),
}
.into());
let tool = SecretTool;
let status = StatusUpdate::tool_completed(
"secret_save".into(),
&err,
&params,
Some(&tool as &dyn crate::tools::Tool),
);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(!success);
let err_msg = error.as_deref().expect("should have error");
assert!(err_msg.contains("db error"), "error: {}", err_msg);
let param_str = parameters
.as_ref()
.expect("should have parameters on failure");
assert!(
param_str.contains("[REDACTED]"),
"sensitive value should be redacted: {}",
param_str
);
assert!(
!param_str.contains("sk-secret-123"),
"raw secret should not appear: {}",
param_str
);
assert!(
param_str.contains("api_key"),
"non-sensitive params should be preserved: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_params_on_success() {
let params = serde_json::json!({"name": "key", "value": "secret"});
let ok: Result<String, crate::error::Error> = Ok("done".into());
let status = StatusUpdate::tool_completed("secret_save".into(), &ok, &params, None);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(success);
assert!(error.is_none());
assert!(parameters.is_none(), "no params should be sent on success");
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_tool_passes_params_unredacted() {
let params = serde_json::json!({"cmd": "ls -la"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "shell".into(),
reason: "timeout".into(),
}
.into());
let status = StatusUpdate::tool_completed("shell".into(), &err, &params, None);
if let StatusUpdate::ToolCompleted { parameters, .. } = &status {
let param_str = parameters.as_ref().expect("should have parameters");
assert!(
param_str.contains("ls -la"),
"non-sensitive params should pass through: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
}
+1 -1
View File
@@ -466,7 +466,7 @@ impl Channel for ReplChannel {
StatusUpdate::ToolStarted { name } => {
eprintln!(" \x1b[33m\u{25CB} {name}\x1b[0m");
}
StatusUpdate::ToolCompleted { name, success } => {
StatusUpdate::ToolCompleted { name, success, .. } => {
if success {
eprintln!(" \x1b[32m\u{25CF} {name}\x1b[0m");
} else {
+1 -1
View File
@@ -974,7 +974,7 @@ impl Channel for SignalChannel {
// Send tool completed notification (debug mode only)
if self.is_debug()
&& let StatusUpdate::ToolCompleted { name, success } = &status
&& let StatusUpdate::ToolCompleted { name, success, .. } = &status
&& let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str())
{
let (icon, color) = if *success {
+14 -1
View File
@@ -19,12 +19,14 @@ use crate::channels::wasm::schema::ChannelCapabilitiesFile;
use crate::channels::wasm::wrapper::WasmChannel;
use crate::db::SettingsStore;
use crate::pairing::PairingStore;
use crate::secrets::SecretsStore;
/// Loads WASM channels from the filesystem.
pub struct WasmChannelLoader {
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
impl WasmChannelLoader {
@@ -38,9 +40,16 @@ impl WasmChannelLoader {
runtime,
pairing_store,
settings_store,
secrets_store: None,
}
}
/// Set the secrets store for host-based credential injection in WASM channels.
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Load a single WASM channel from a file pair.
///
/// Expects:
@@ -72,6 +81,7 @@ impl WasmChannelLoader {
let cap_bytes = fs::read(cap_path).await?;
let cap_file = ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| WasmChannelError::InvalidCapabilities(e.to_string()))?;
cap_file.validate();
// Debug: log raw capabilities
tracing::debug!(
@@ -127,7 +137,7 @@ impl WasmChannelLoader {
.await?;
// Create the channel
let channel = WasmChannel::new(
let mut channel = WasmChannel::new(
self.runtime.clone(),
prepared,
capabilities,
@@ -135,6 +145,9 @@ impl WasmChannelLoader {
self.pairing_store.clone(),
self.settings_store.clone(),
);
if let Some(ref secrets) = self.secrets_store {
channel = channel.with_secrets_store(Arc::clone(secrets));
}
tracing::info!(
name = name,
+94
View File
@@ -90,6 +90,37 @@ impl ChannelCapabilitiesFile {
serde_json::from_slice(bytes)
}
/// Validate the capabilities file and emit warnings for common misconfigurations.
///
/// Called once at load time to catch issues early. Warnings are emitted via
/// `tracing::warn` so they show up in startup logs without blocking loading.
pub fn validate(&self) {
const MIN_PROMPT_LENGTH: usize = 30;
// Check for short prompts in required_secrets
for secret in &self.setup.required_secrets {
if secret.prompt.len() < MIN_PROMPT_LENGTH {
tracing::warn!(
channel = self.name,
secret = secret.name,
prompt = secret.prompt,
"setup.required_secrets prompt is shorter than {} chars — \
consider a more descriptive prompt that tells the user where to find this value",
MIN_PROMPT_LENGTH
);
}
}
// Has required_secrets but no setup_url
if !self.setup.required_secrets.is_empty() && self.setup.setup_url.is_none() {
tracing::warn!(
channel = self.name,
"setup.required_secrets defined but no setup.setup_url — \
user has no link to obtain credentials"
);
}
}
/// Convert to runtime ChannelCapabilities.
pub fn to_capabilities(&self) -> ChannelCapabilities {
self.capabilities.to_channel_capabilities(&self.name)
@@ -262,6 +293,10 @@ pub struct SetupSchema {
/// Placeholders like {secret_name} are replaced with actual values.
#[serde(default)]
pub validation_endpoint: Option<String>,
/// User-facing URL where they can create/manage credentials.
#[serde(default)]
pub setup_url: Option<String>,
}
/// Configuration for a secret required during setup.
@@ -605,6 +640,65 @@ mod tests {
// ── Category 5: Discord Capabilities Setup & Configuration ──────────
#[test]
fn test_validate_channel_short_prompt() {
// prompt < 30 chars — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{ "name": "bot_token", "prompt": "Bot token" }
],
"setup_url": "https://example.com"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for short prompt
file.validate();
}
#[test]
fn test_validate_channel_missing_setup_url() {
// required_secrets without setup_url — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from the developer portal settings"
}
]
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for missing setup_url
file.validate();
}
#[test]
fn test_validate_clean_channel() {
// Well-configured channel — should not panic or warn
let json = r#"{
"name": "good-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from https://example.com/bot-settings"
}
],
"setup_url": "https://example.com/bot-settings"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic and emits no warnings
file.validate();
}
#[test]
fn test_discord_capabilities_has_public_key_secret() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
+5 -1
View File
@@ -2479,7 +2479,7 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
message: format!("Tool started: {}", name),
metadata_json,
},
StatusUpdate::ToolCompleted { name, success } => wit_channel::StatusUpdate {
StatusUpdate::ToolCompleted { name, success, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolCompleted,
message: format!(
"Tool completed: {} ({})",
@@ -3387,6 +3387,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: true,
error: None,
parameters: None,
},
&metadata,
);
@@ -3407,6 +3409,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: false,
error: Some("connection refused".to_string()),
parameters: None,
},
&metadata,
);
+148 -30
View File
@@ -2,7 +2,7 @@
use axum::{
extract::{Request, State},
http::{HeaderMap, StatusCode},
http::{HeaderMap, Method, StatusCode},
middleware::Next,
response::{IntoResponse, Response},
};
@@ -14,10 +14,44 @@ pub struct AuthState {
pub token: String,
}
/// Whether query-string token auth is allowed for this request.
///
/// Only GET requests to streaming endpoints may use `?token=xxx`. This
/// minimizes token-in-URL exposure on state-changing routes, where the token
/// would leak via server logs, Referer headers, and browser history.
///
/// Allowed endpoints:
/// - SSE: `/api/chat/events`, `/api/logs/events` (EventSource can't set headers)
/// - WebSocket: `/api/chat/ws` (WS upgrade can't set custom headers)
///
/// If you add a new SSE or WebSocket endpoint, add its path here.
fn allows_query_token_auth(request: &Request) -> bool {
if request.method() != Method::GET {
return false;
}
matches!(
request.uri().path(),
"/api/chat/events" | "/api/logs/events" | "/api/chat/ws"
)
}
/// Extract the `token` query parameter value, URL-decoded.
fn query_token(request: &Request) -> Option<String> {
let query = request.uri().query()?;
url::form_urlencoded::parse(query.as_bytes()).find_map(|(k, v)| {
if k == "token" {
Some(v.into_owned())
} else {
None
}
})
}
/// Auth middleware that validates bearer token from header or query param.
///
/// SSE connections can't set headers from `EventSource`, so we also accept
/// `?token=xxx` as a query parameter.
/// `?token=xxx` as a query parameter, but only on SSE endpoints.
pub async fn auth_middleware(
State(auth): State<AuthState>,
headers: HeaderMap,
@@ -35,15 +69,12 @@ pub async fn auth_middleware(
return next.run(request).await;
}
// Fall back to query parameter for SSE EventSource (constant-time comparison)
if let Some(query) = request.uri().query() {
for pair in query.split('&') {
if let Some(token) = pair.strip_prefix("token=")
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
}
// Fall back to query parameter, but only for SSE endpoints (constant-time comparison).
if allows_query_token_auth(&request)
&& let Some(token) = query_token(&request)
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
(StatusCode::UNAUTHORIZED, "Invalid or missing auth token").into_response()
@@ -62,24 +93,28 @@ mod tests {
assert_eq!(cloned.token, "test-token");
}
// === QA Plan - Web gateway auth tests ===
use axum::Router;
use axum::body::Body;
use axum::middleware;
use axum::routing::get;
use axum::routing::{get, post};
use tower::ServiceExt;
async fn dummy_handler() -> &'static str {
"ok"
}
/// Router with streaming endpoints (query auth allowed) and regular
/// endpoints (query auth rejected).
fn test_app(token: &str) -> Router {
let state = AuthState {
token: token.to_string(),
};
Router::new()
.route("/test", get(dummy_handler))
.route("/api/chat/events", get(dummy_handler))
.route("/api/logs/events", get(dummy_handler))
.route("/api/chat/ws", get(dummy_handler))
.route("/api/chat/history", get(dummy_handler))
.route("/api/chat/send", post(dummy_handler))
.layer(middleware::from_fn_with_state(state, auth_middleware))
}
@@ -87,7 +122,7 @@ mod tests {
async fn test_valid_bearer_token_passes() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
@@ -99,7 +134,7 @@ mod tests {
async fn test_invalid_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer wrong-token")
.body(Body::empty())
.unwrap();
@@ -108,10 +143,10 @@ mod tests {
}
#[tokio::test]
async fn test_missing_auth_header_falls_through_to_query() {
async fn test_query_token_allowed_for_chat_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=secret-token")
.uri("/api/chat/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
@@ -119,10 +154,80 @@ mod tests {
}
#[tokio::test]
async fn test_query_param_invalid_token_rejected() {
async fn test_query_token_allowed_for_logs_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=wrong-token")
.uri("/api/logs/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_allowed_for_ws_upgrade() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/ws?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded() {
// Token with characters that get percent-encoded in URLs.
let raw_token = "tok+en/with spaces";
let app = test_app(raw_token);
let req = Request::builder()
.uri("/api/chat/events?token=tok%2Ben%2Fwith%20spaces")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded_mismatch() {
let app = test_app("real-token");
// Encoded value decodes to "wrong-token", not "real-token".
let req = Request::builder()
.uri("/api/chat/events?token=wrong%2Dtoken")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_non_sse_get() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/history?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.method(Method::POST)
.uri("/api/chat/send?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_invalid_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events?token=wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
@@ -132,17 +237,32 @@ mod tests {
#[tokio::test]
async fn test_no_auth_at_all_rejected() {
let app = test_app("secret-token");
let req = Request::builder().uri("/test").body(Body::empty()).unwrap();
let req = Request::builder()
.uri("/api/chat/events")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
// RFC 6750 Section 2.1: auth-scheme comparison must be case-insensitive.
async fn test_bearer_header_works_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.method(Method::POST)
.uri("/api/chat/send")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "bearer secret-token")
.body(Body::empty())
.unwrap();
@@ -154,7 +274,7 @@ mod tests {
async fn test_bearer_prefix_mixed_case() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "BEARER secret-token")
.body(Body::empty())
.unwrap();
@@ -166,7 +286,7 @@ mod tests {
async fn test_empty_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer ")
.body(Body::empty())
.unwrap();
@@ -176,11 +296,9 @@ mod tests {
#[tokio::test]
async fn test_token_with_whitespace_rejected() {
// Extra space after "Bearer " means the token value starts with a space,
// which should not match the expected token.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
+6 -5
View File
@@ -142,7 +142,7 @@ pub async fn chat_auth_token_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if result.status == "authenticated" {
if result.is_authenticated() {
// Auto-activate so tools are available immediately
let msg = match ext_mgr.activate(&req.extension_name).await {
Ok(r) => format!(
@@ -170,13 +170,14 @@ pub async fn chat_auth_token_handler(
// Re-emit auth_required for retry
state.sse.broadcast(SseEvent::AuthRequired {
extension_name: req.extension_name.clone(),
instructions: result.instructions.clone(),
auth_url: result.auth_url.clone(),
setup_url: result.setup_url.clone(),
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
Ok(Json(ActionResponse::fail(
result
.instructions
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token".to_string()),
)))
}
+13 -8
View File
@@ -33,7 +33,7 @@ pub async fn extensions_list_handler(
"failed".to_string()
} else if !ext.authenticated {
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
} else if ext.active {
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
@@ -59,6 +59,7 @@ pub async fn extensions_list_handler(
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
has_auth: ext.has_auth,
activation_status,
activation_error: ext.activation_error,
}
@@ -123,7 +124,11 @@ pub async fn extensions_activate_handler(
))?;
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
// Activation just loads the WASM module. Auth (OAuth/manual) is
// triggered separately via save_setup_secrets or the auth endpoint.
Ok(Json(ActionResponse::ok(result.message)))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
@@ -136,7 +141,7 @@ pub async fn extensions_activate_handler(
// Activation failed due to auth; try authenticating first.
match ext_mgr.auth(&name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded, retry activation.
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
@@ -147,13 +152,13 @@ pub async fn extensions_activate_handler(
// Auth in progress (OAuth URL or awaiting manual token).
let mut resp = ActionResponse::fail(
auth_result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| format!("'{}' requires authentication.", name)),
);
resp.auth_url = auth_result.auth_url;
resp.awaiting_token = Some(auth_result.awaiting_token);
resp.instructions = auth_result.instructions;
resp.auth_url = auth_result.auth_url().map(String::from);
resp.awaiting_token = Some(auth_result.is_awaiting_token());
resp.instructions = auth_result.instructions().map(String::from);
Ok(Json(resp))
}
Err(auth_err) => Ok(Json(ActionResponse::fail(format!(
+202 -94
View File
@@ -181,6 +181,9 @@ pub async fn jobs_detail_handler(
});
}
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
let is_claude_code = mode.as_deref() == Some("claude_code");
return Ok(Json(JobDetailResponse {
id: job.id,
title: job.task.clone(),
@@ -193,11 +196,11 @@ pub async fn jobs_detail_handler(
elapsed_secs,
project_dir: Some(job.project_dir.clone()),
browse_url: Some(format!("/projects/{}/", browse_id)),
job_mode: {
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
mode.filter(|m| m != "worker")
},
job_mode: mode.filter(|m| m != "worker"),
transitions,
can_restart: state.job_manager.is_some(),
can_prompt: is_claude_code && state.prompt_queue.is_some(),
job_kind: Some("sandbox".to_string()),
}));
}
@@ -208,6 +211,12 @@ pub async fn jobs_detail_handler(
(end - start).num_seconds().max(0) as u64
});
// Only show prompt bar for jobs that have a running worker (Pending/InProgress).
// Stuck jobs have no active worker loop, so messages would be silently dropped.
let is_promptable = matches!(
ctx.state,
crate::context::JobState::Pending | crate::context::JobState::InProgress
);
return Ok(Json(JobDetailResponse {
id: ctx.job_id,
title: ctx.title.clone(),
@@ -222,6 +231,9 @@ pub async fn jobs_detail_handler(
browse_url: None,
job_mode: None,
transitions: Vec::new(),
can_restart: state.scheduler.is_some(),
can_prompt: is_promptable && state.scheduler.is_some(),
job_kind: Some("agent".to_string()),
}));
}
@@ -295,108 +307,164 @@ pub async fn jobs_restart_handler(
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
let old_job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let old_job = store
.get_sandbox_job(old_job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Try sandbox job restart first.
if let Ok(Some(old_job)) = store.get_sandbox_job(old_job_id).await {
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
}
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
// Enrich the task with failure context.
let task = if let Some(ref reason) = old_job.failure_reason {
format!(
"Previous attempt failed: {}. Retry: {}",
reason, old_job.task
)
} else {
old_job.task.clone()
};
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => {
crate::orchestrator::job_manager::JobMode::ClaudeCode
}
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
)
})?;
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
// Create a new job with the same task and project_dir.
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
// Try agent job restart: dispatch a new job via the scheduler.
if let Ok(Some(old_job)) = store.get_job(old_job_id).await {
if old_job.state.is_active() {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.state),
));
}
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: old_job.task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let slot = state.scheduler.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Scheduler not available".to_string(),
))?;
let scheduler_guard = slot.read().await;
let scheduler = scheduler_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Agent not started yet".to_string(),
))?;
// Look up the original job's mode so the restart uses the same mode.
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => crate::orchestrator::job_manager::JobMode::ClaudeCode,
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
// Look up failure reason (O(1) point lookup).
let failure_reason = store
.get_agent_job_failure_reason(old_job_id)
.await
.ok()
.flatten()
.unwrap_or_default();
// Restore credential grants from the original job so the restarted container
// has access to the same secrets.
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&old_job.task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
let title = if !failure_reason.is_empty() {
format!(
"Previous attempt failed: {}. Retry: {}",
failure_reason, old_job.title
)
})?;
} else {
old_job.title.clone()
};
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let new_job_id = scheduler
.dispatch_job(&old_job.user_id, &title, &old_job.description, None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})))
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
/// Submit a follow-up prompt to a running Claude Code sandbox job.
/// Submit a follow-up prompt to a running job.
///
/// Routes to the appropriate backend:
/// - Claude Code sandbox jobs → prompt queue (polled by the bridge)
/// - Agent (non-sandbox) jobs → WorkerMessage injection via scheduler
/// - Worker-mode sandbox jobs → not supported (no mechanism to inject)
pub async fn jobs_prompt_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Json(body): Json<serde_json::Value>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let job_id: uuid::Uuid = id
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
@@ -412,17 +480,57 @@ pub async fn jobs_prompt_handler(
let done = body.get("done").and_then(|v| v.as_bool()).unwrap_or(false);
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
// Try sandbox job path: check if we have a sandbox record for this ID.
if let Some(ref s) = state.store
&& let Ok(Some(_)) = s.get_sandbox_job(job_id).await
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
// It's a sandbox job. Check if Claude Code mode.
let mode = s.get_sandbox_job_mode(job_id).await.ok().flatten();
if mode.as_deref() == Some("claude_code") {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
}
return Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})));
} else {
return Err((
StatusCode::NOT_IMPLEMENTED,
"Follow-up prompts are not supported for worker-mode sandbox jobs".to_string(),
));
}
}
Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})))
// Try agent job path: send via scheduler.
let slot = state.scheduler.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Agent job prompts require the scheduler to be configured".to_string(),
))?;
let scheduler_guard = slot.read().await;
if let Some(ref scheduler) = *scheduler_guard
&& scheduler.is_running(job_id).await
{
scheduler
.send_message(job_id, content)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "sent",
"job_id": job_id.to_string(),
})));
}
Err((
StatusCode::NOT_FOUND,
"Job not found or not running".to_string(),
))
}
/// Load persisted job events for a job (for history replay on page open).
+3 -3
View File
@@ -159,10 +159,10 @@ pub async fn memory_search_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let hits: Vec<SearchHit> = results
.iter()
.into_iter()
.map(|r| SearchHit {
path: r.document_id.to_string(),
content: r.content.clone(),
path: r.document_path,
content: r.content,
score: r.score as f64,
})
.collect();
+10 -1
View File
@@ -147,6 +147,10 @@ pub async fn routines_trigger_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
if routine.user_id != state.user_id {
return Err((StatusCode::FORBIDDEN, "Access denied".to_string()));
}
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
@@ -156,7 +160,12 @@ pub async fn routines_trigger_handler(
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let msg = IncomingMessage::new("gateway", &state.user_id, content);
let thread_id = format!(
"routine-{}-{}",
routine_id,
chrono::Utc::now().timestamp_millis()
);
let msg = IncomingMessage::new("gateway", &state.user_id, content).with_thread(thread_id);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
+8 -1
View File
@@ -148,7 +148,14 @@ pub async fn skills_install_handler(
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
} else if let Some(ref catalog) = state.skill_catalog {
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), &req.name);
// Prefer slug (e.g. "owner/skill-name") over display name for the
// download URL, since the registry endpoint expects a slug.
let download_key = req
.slug
.as_deref()
.filter(|s| !s.is_empty())
.unwrap_or(&req.name);
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), download_key);
crate::tools::builtin::skill_tools::fetch_skill_content(&url)
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, e.to_string()))?
+18 -4
View File
@@ -84,6 +84,7 @@ impl GatewayChannel {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: config.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(ws::WsConnectionTracker::new())),
@@ -94,7 +95,6 @@ impl GatewayChannel {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
Self {
@@ -108,7 +108,8 @@ impl GatewayChannel {
fn rebuild_state(&mut self, mutate: impl FnOnce(&mut GatewayState)) {
let mut new_state = GatewayState {
msg_tx: tokio::sync::RwLock::new(None),
sse: SseManager::new(),
// Preserve the existing broadcast channel so sender handles remain valid.
sse: SseManager::from_sender(self.state.sse.sender()),
workspace: self.state.workspace.clone(),
session_manager: self.state.session_manager.clone(),
log_broadcaster: self.state.log_broadcaster.clone(),
@@ -118,6 +119,7 @@ impl GatewayChannel {
store: self.state.store.clone(),
job_manager: self.state.job_manager.clone(),
prompt_queue: self.state.prompt_queue.clone(),
scheduler: self.state.scheduler.clone(),
user_id: self.state.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: self.state.ws_tracker.clone(),
@@ -128,7 +130,6 @@ impl GatewayChannel {
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
restart_requested: std::sync::atomic::AtomicBool::new(false),
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
@@ -198,6 +199,12 @@ impl GatewayChannel {
self
}
/// Inject the scheduler for sending follow-up messages to agent jobs.
pub fn with_scheduler(mut self, slot: crate::tools::builtin::SchedulerSlot) -> Self {
self.rebuild_state(|s| s.scheduler = Some(slot));
self
}
/// Inject the skill registry for skill management API.
pub fn with_skill_registry(mut self, sr: Arc<std::sync::RwLock<SkillRegistry>>) -> Self {
self.rebuild_state(|s| s.skill_registry = Some(sr));
@@ -297,9 +304,16 @@ impl Channel for GatewayChannel {
name,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolCompleted { name, success } => SseEvent::ToolCompleted {
StatusUpdate::ToolCompleted {
name,
success,
error,
parameters,
} => SseEvent::ToolCompleted {
name,
success,
error,
parameters,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolResult { name, preview } => SseEvent::ToolResult {
+583 -50
View File
@@ -156,6 +156,8 @@ pub struct GatewayState {
pub skill_registry: Option<Arc<std::sync::RwLock<crate::skills::SkillRegistry>>>,
/// Skill catalog for searching the ClawHub registry.
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
/// Scheduler for sending follow-up messages to running agent jobs.
pub scheduler: Option<crate::tools::builtin::SchedulerSlot>,
/// Rate limiter for chat endpoints (30 messages per 60 seconds).
pub chat_rate_limiter: RateLimiter,
/// Registry catalog entries for the available extensions API.
@@ -165,8 +167,6 @@ pub struct GatewayState {
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
/// Flag set when a restart has been requested via the API.
pub restart_requested: std::sync::atomic::AtomicBool,
}
/// Start the gateway HTTP server.
@@ -192,7 +192,9 @@ pub async fn start_server(
})?;
// Public routes (no auth)
let public = Router::new().route("/api/health", get(health_handler));
let public = Router::new()
.route("/api/health", get(health_handler))
.route("/oauth/callback", get(oauth_callback_handler));
// Protected routes (require auth)
let auth_state = AuthState { token: auth_token };
@@ -247,8 +249,6 @@ pub async fn start_server(
"/api/extensions/{name}/setup",
get(extensions_setup_handler).post(extensions_setup_submit_handler),
)
// Gateway management
.route("/api/gateway/restart", post(gateway_restart_handler))
// Pairing
.route("/api/pairing/{channel}", get(pairing_list_handler))
.route(
@@ -426,6 +426,180 @@ async fn health_handler() -> Json<HealthResponse> {
})
}
/// Return an OAuth error landing page response.
fn oauth_error_page(label: &str) -> axum::response::Response {
let html = crate::cli::oauth_defaults::landing_html(label, false);
axum::response::Html(html).into_response()
}
/// OAuth callback handler for the web gateway.
///
/// This is a PUBLIC route (no Bearer token required) because OAuth providers
/// redirect the user's browser here. The `state` query parameter correlates
/// the callback with a pending OAuth flow registered by `start_wasm_oauth()`.
///
/// Used on hosted instances where `IRONCLAW_OAUTH_CALLBACK_URL` points to
/// the gateway (e.g., `https://kind-deer.agent1.near.ai/oauth/callback`).
/// Local/desktop mode continues to use the TCP listener on port 9876.
async fn oauth_callback_handler(
State(state): State<Arc<GatewayState>>,
Query(params): Query<std::collections::HashMap<String, String>>,
) -> impl IntoResponse {
use crate::cli::oauth_defaults;
// Check for error from OAuth provider (e.g., user denied consent)
if let Some(error) = params.get("error") {
let description = params
.get("error_description")
.cloned()
.unwrap_or_else(|| error.clone());
return oauth_error_page(&description);
}
let state_param = match params.get("state") {
Some(s) if !s.is_empty() => s.clone(),
_ => return oauth_error_page("IronClaw"),
};
let code = match params.get("code") {
Some(c) if !c.is_empty() => c.clone(),
_ => return oauth_error_page("IronClaw"),
};
// Look up the pending flow by CSRF state (atomic remove prevents replay)
let ext_mgr = match state.extension_manager.as_ref() {
Some(mgr) => mgr,
None => return oauth_error_page("IronClaw"),
};
// Strip instance prefix from state for registry lookup.
// Platform nginx sends `state=instance:nonce` but flows are keyed by nonce only.
let lookup_key = oauth_defaults::strip_instance_prefix(&state_param);
let flow = ext_mgr
.pending_oauth_flows()
.write()
.await
.remove(lookup_key);
let flow = match flow {
Some(f) => f,
None => {
tracing::warn!(
state = %state_param,
lookup_key = %lookup_key,
"OAuth callback received with unknown or expired state"
);
return oauth_error_page("IronClaw");
}
};
// Check flow expiry (5 minutes, matching TCP listener timeout)
if flow.created_at.elapsed() > oauth_defaults::OAUTH_FLOW_EXPIRY {
tracing::warn!(
extension = %flow.extension_name,
"OAuth flow expired"
);
return oauth_error_page(&flow.display_name);
}
// Exchange the authorization code for tokens.
// Use the platform exchange proxy when configured (keeps client_secret off container),
// otherwise call the provider's token URL directly.
let exchange_proxy_url = std::env::var("IRONCLAW_OAUTH_EXCHANGE_URL").ok();
let result: Result<(), String> = async {
let token_response = if let Some(ref proxy_url) = exchange_proxy_url {
let gateway_token = flow.gateway_token.as_deref().unwrap_or_default();
oauth_defaults::exchange_via_proxy(
proxy_url,
gateway_token,
&code,
&flow.redirect_uri,
flow.code_verifier.as_deref(),
&flow.access_token_field,
)
.await
.map_err(|e| e.to_string())?
} else {
oauth_defaults::exchange_oauth_code(
&flow.token_url,
&flow.client_id,
flow.client_secret.as_deref(),
&code,
&flow.redirect_uri,
flow.code_verifier.as_deref(),
&flow.access_token_field,
)
.await
.map_err(|e| e.to_string())?
};
// Validate the token before storing (catches wrong account, etc.)
if let Some(ref validation) = flow.validation_endpoint {
oauth_defaults::validate_oauth_token(&token_response.access_token, validation)
.await
.map_err(|e| e.to_string())?;
}
// Store tokens encrypted in the secrets store
oauth_defaults::store_oauth_tokens(
flow.secrets.as_ref(),
&flow.user_id,
&flow.secret_name,
flow.provider.as_deref(),
&token_response.access_token,
token_response.refresh_token.as_deref(),
token_response.expires_in,
&flow.scopes,
)
.await
.map_err(|e| e.to_string())?;
Ok(())
}
.await;
let (success, message) = match &result {
Ok(()) => (
true,
format!("{} authenticated successfully", flow.display_name),
),
Err(e) => (
false,
format!("{} authentication failed: {}", flow.display_name, e),
),
};
match &result {
Ok(()) => {
tracing::info!(
extension = %flow.extension_name,
"OAuth completed successfully via gateway callback"
);
}
Err(e) => {
tracing::warn!(
extension = %flow.extension_name,
error = %e,
"OAuth failed via gateway callback"
);
}
}
// Broadcast SSE event to notify the web UI
if let Some(ref sender) = flow.sse_sender {
let _ = sender.send(SseEvent::AuthCompleted {
extension_name: flow.extension_name,
success,
message,
});
}
let html = oauth_defaults::landing_html(&flow.display_name, success);
axum::response::Html(html).into_response()
}
// --- Chat handlers ---
async fn chat_send_handler(
@@ -554,7 +728,7 @@ async fn chat_auth_token_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if result.status == "authenticated" {
if result.is_authenticated() {
// Auto-activate so tools are available immediately
let msg = match ext_mgr.activate(&req.extension_name).await {
Ok(r) => format!(
@@ -582,13 +756,14 @@ async fn chat_auth_token_handler(
// Re-emit auth_required for retry
state.sse.broadcast(SseEvent::AuthRequired {
extension_name: req.extension_name.clone(),
instructions: result.instructions.clone(),
auth_url: result.auth_url.clone(),
setup_url: result.setup_url.clone(),
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
Ok(Json(ActionResponse::fail(
result
.instructions
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token".to_string()),
)))
}
@@ -1218,8 +1393,8 @@ async fn extensions_list_handler(
} else if !ext.authenticated {
// No credentials configured yet.
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
// Telegram: check pairing status (end-to-end setup via web UI).
} else if ext.active {
// Check pairing status for active channels.
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
@@ -1230,7 +1405,7 @@ async fn extensions_list_handler(
"pairing".to_string()
}
} else {
// Authenticated but not fully active (or non-Telegram).
// Authenticated but not yet active.
"configured".to_string()
})
} else {
@@ -1246,6 +1421,7 @@ async fn extensions_list_handler(
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
has_auth: ext.has_auth,
activation_status,
activation_error: ext.activation_error,
}
@@ -1315,7 +1491,34 @@ async fn extensions_install_handler(
.install(&req.name, req.url.as_deref(), kind_hint)
.await
{
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
let mut resp = ActionResponse::ok(result.message);
// Auto-activate WASM tools after install (install = active).
if result.kind == crate::extensions::ExtensionKind::WasmTool {
if let Err(e) = ext_mgr.activate(&req.name).await {
tracing::debug!(
extension = %req.name,
error = %e,
"Auto-activation after install failed"
);
}
// Check auth after activation. This may initiate OAuth both for scope
// expansion and for first-time auth when credentials are already
// configured (e.g., built-in providers). We only surface an auth_url
// when the extension reports it is awaiting authorization.
match ext_mgr.auth(&req.name, None).await {
Ok(auth_result) if auth_result.auth_url().is_some() => {
// Scope expansion or initial OAuth: user needs to authorize
resp.auth_url = auth_result.auth_url().map(String::from);
}
_ => {}
}
}
Ok(Json(resp))
}
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
@@ -1330,7 +1533,19 @@ async fn extensions_activate_handler(
))?;
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
// Activation loaded the WASM module. Check if the tool needs
// OAuth scope expansion (e.g., adding google-docs when gmail
// already has a token but missing the documents scope).
// Initial OAuth setup is triggered via save_setup_secrets.
let mut resp = ActionResponse::ok(result.message);
if let Ok(auth_result) = ext_mgr.auth(&name, None).await
&& auth_result.auth_url().is_some()
{
resp.auth_url = auth_result.auth_url().map(String::from);
}
Ok(Json(resp))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
@@ -1343,7 +1558,7 @@ async fn extensions_activate_handler(
// Activation failed due to auth; try authenticating first.
match ext_mgr.auth(&name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded, retry activation.
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
@@ -1354,13 +1569,13 @@ async fn extensions_activate_handler(
// Auth in progress (OAuth URL or awaiting manual token).
let mut resp = ActionResponse::fail(
auth_result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| format!("'{}' requires authentication.", name)),
);
resp.auth_url = auth_result.auth_url;
resp.awaiting_token = Some(auth_result.awaiting_token);
resp.instructions = auth_result.instructions;
resp.auth_url = auth_result.auth_url().map(String::from);
resp.awaiting_token = Some(auth_result.is_awaiting_token());
resp.instructions = auth_result.instructions().map(String::from);
Ok(Json(resp))
}
Err(auth_err) => Ok(Json(ActionResponse::fail(format!(
@@ -1550,43 +1765,22 @@ async fn extensions_setup_submit_handler(
match ext_mgr.save_setup_secrets(&name, &req.secrets).await {
Ok(result) => {
// Broadcast auth_completed so the chat UI can dismiss any in-progress
// auth card or setup modal that was triggered by tool_auth/tool_activate.
state.sse.broadcast(SseEvent::AuthCompleted {
extension_name: name.clone(),
success: true,
message: result.message.clone(),
});
let mut resp = ActionResponse::ok(result.message);
resp.activated = Some(result.activated);
if !result.activated {
resp.needs_restart = Some(true);
}
resp.auth_url = result.auth_url;
Ok(Json(resp))
}
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Gateway management handlers ---
async fn gateway_restart_handler(State(state): State<Arc<GatewayState>>) -> Json<ActionResponse> {
// Idempotency guard: only allow one restart at a time.
if state
.restart_requested
.compare_exchange(
false,
true,
std::sync::atomic::Ordering::SeqCst,
std::sync::atomic::Ordering::SeqCst,
)
.is_err()
{
return Json(ActionResponse::ok("Restart already in progress"));
}
// Take the shutdown sender and trigger graceful shutdown.
if let Some(tx) = state.shutdown_tx.write().await.take() {
let _ = tx.send(());
tracing::info!("Gateway restart requested via API");
}
Json(ActionResponse::ok("Restarting..."))
}
// --- Pairing handlers ---
async fn pairing_list_handler(
@@ -2218,4 +2412,343 @@ mod tests {
let turns = build_turns_from_db_messages(&[]);
assert!(turns.is_empty());
}
// --- OAuth callback handler tests ---
/// Build a minimal `GatewayState` for testing the OAuth callback handler.
fn test_gateway_state(ext_mgr: Option<Arc<ExtensionManager>>) -> Arc<GatewayState> {
Arc::new(GatewayState {
msg_tx: tokio::sync::RwLock::new(None),
sse: SseManager::new(),
workspace: None,
session_manager: None,
log_broadcaster: None,
log_level_handle: None,
extension_manager: ext_mgr,
tool_registry: None,
store: None,
job_manager: None,
prompt_queue: None,
user_id: "test".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: None,
llm_provider: None,
skill_registry: None,
skill_catalog: None,
scheduler: None,
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: vec![],
cost_guard: None,
startup_time: std::time::Instant::now(),
})
}
/// Build a test router with just the OAuth callback route.
fn test_oauth_router(state: Arc<GatewayState>) -> Router {
Router::new()
.route("/oauth/callback", get(oauth_callback_handler))
.with_state(state)
}
#[tokio::test]
async fn test_oauth_callback_missing_params() {
use axum::body::Body;
use tower::ServiceExt;
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_error_from_provider() {
use axum::body::Body;
use tower::ServiceExt;
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?error=access_denied&error_description=access_denied")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_unknown_state() {
use axum::body::Body;
use tower::ServiceExt;
// Build an ExtensionManager so the handler can look up flows
let secrets = Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets,
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=unknown_state_value")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_expired_flow() {
use axum::body::Body;
use tower::ServiceExt;
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
// Insert an expired flow (created 10 minutes ago)
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "test_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: None,
gateway_token: None,
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
.pending_oauth_flows()
.write()
.await
.insert("expired_state".to_string(), flow);
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=expired_state")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
// Expired flow → error landing page
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_no_extension_manager() {
use axum::body::Body;
use tower::ServiceExt;
// No extension manager set → graceful error
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=some_state")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_strips_instance_prefix() {
use axum::body::Body;
use tower::ServiceExt;
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
// Insert a flow keyed by raw nonce "test_nonce" (without instance prefix).
// Use an expired flow so the handler exits before attempting a real HTTP
// token exchange — we only need to verify that the instance prefix was
// stripped and the flow was found by the raw nonce.
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "test_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: None,
gateway_token: None,
// Expired — handler will reject after lookup (no network I/O)
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
.pending_oauth_flows()
.write()
.await
.insert("test_nonce".to_string(), flow);
let state = test_gateway_state(Some(ext_mgr.clone()));
let app = test_oauth_router(state);
// Send callback with instance prefix: "myinstance:test_nonce"
// The handler should strip "myinstance:" and find the flow keyed by "test_nonce"
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=fake_code&state=myinstance:test_nonce")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
// The flow was found (stripped prefix matched) but is expired, so the
// handler returns an error landing page. The flow being consumed from
// the registry (checked below) proves the prefix was stripped correctly.
assert!(
html.contains("Authorization Failed"),
"Expected error page, html was: {}",
&html[..html.len().min(500)]
);
// Verify the flow was consumed (removed from registry)
assert!(
ext_mgr
.pending_oauth_flows()
.read()
.await
.get("test_nonce")
.is_none()
);
}
}
+17
View File
@@ -36,6 +36,23 @@ impl SseManager {
}
}
/// Create an SSE manager that reuses an existing broadcast sender.
///
/// This preserves the broadcast channel across `rebuild_state` calls so
/// that sender handles captured by other components remain valid.
///
/// **Important:** The connection counter is reset to zero. This method must
/// only be called before the server starts accepting connections (i.e.,
/// during startup wiring). Calling it after connections are established
/// will break connection tracking and allow exceeding `MAX_CONNECTIONS`.
pub fn from_sender(tx: broadcast::Sender<SseEvent>) -> Self {
Self {
tx,
connection_count: Arc::new(AtomicU64::new(0)),
max_connections: MAX_CONNECTIONS,
}
}
/// Broadcast an event to all connected clients.
pub fn broadcast(&self, event: SseEvent) {
// Ignore send errors (no receivers is fine)
+105 -104
View File
@@ -180,7 +180,7 @@ function connectSSE() {
eventSource.addEventListener('tool_completed', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
completeToolCard(data.name, data.success);
completeToolCard(data.name, data.success, data.error, data.parameters);
});
eventSource.addEventListener('tool_result', (e) => {
@@ -222,13 +222,24 @@ function connectSSE() {
eventSource.addEventListener('auth_required', (e) => {
const data = JSON.parse(e.data);
showAuthCard(data);
if (data.auth_url) {
// OAuth flow: show the auth card with an OAuth button + optional token paste field.
showAuthCard(data);
} else {
// Setup flow: fetch the extension's credential schema and show the multi-field
// configure modal (the same UI used by the Extensions tab "Setup" button).
showConfigureModal(data.extension_name);
}
});
eventSource.addEventListener('auth_completed', (e) => {
const data = JSON.parse(e.data);
// Dismiss whichever UI path was active: auth card (OAuth) or configure modal (setup).
removeAuthCard(data.extension_name);
showToast(data.message, 'success');
closeConfigureModal();
showToast(data.message, data.success ? 'success' : 'error');
// Refresh extensions list so status indicators update
if (currentTab === 'extensions') loadExtensions();
enableChatInput();
});
@@ -359,6 +370,9 @@ function selectSlashItem(cmd) {
function updateSlashHighlight() {
const items = document.querySelectorAll('#slash-autocomplete .slash-ac-item');
items.forEach((el, i) => el.classList.toggle('selected', i === _slashSelected));
if (_slashSelected >= 0 && items[_slashSelected]) {
items[_slashSelected].scrollIntoView({ block: 'nearest' });
}
}
function filterSlashCommands(value) {
@@ -581,7 +595,7 @@ function addToolCard(name) {
container.scrollTop = container.scrollHeight;
}
function completeToolCard(name, success) {
function completeToolCard(name, success, error, parameters) {
const entries = _activeToolCards[name];
if (!entries || entries.length === 0) return;
// Find first running card
@@ -602,6 +616,27 @@ function completeToolCard(name, success) {
? '<span class="activity-icon-success">&#10003;</span>'
: '<span class="activity-icon-fail">&#10007;</span>';
entry.card.setAttribute('data-status', success ? 'success' : 'fail');
// For failed tools, populate the body with error details and auto-expand
if (!success && (error || parameters)) {
const output = entry.card.querySelector('.activity-tool-output');
if (output) {
let detail = '';
if (parameters) {
detail += 'Input:\n' + parameters + '\n\n';
}
if (error) {
detail += 'Error:\n' + error;
}
output.textContent = detail;
// Auto-expand so the error is immediately visible
const body = entry.card.querySelector('.activity-tool-body');
const chevron = entry.card.querySelector('.activity-tool-chevron');
if (body) body.style.display = 'block';
if (chevron) chevron.classList.add('expanded');
}
}
}
function setToolCardOutput(name, preview) {
@@ -865,7 +900,7 @@ function showAuthCard(data) {
const tokenInput = document.createElement('input');
tokenInput.type = 'password';
tokenInput.placeholder = 'Paste your API key or token';
tokenInput.placeholder = data.instructions || 'Paste your API key or token';
tokenInput.addEventListener('keydown', (e) => {
if (e.key === 'Enter') submitAuthToken(data.extension_name, tokenInput.value);
});
@@ -1196,7 +1231,7 @@ chatInput.addEventListener('keydown', (e) => {
updateSlashHighlight();
return;
}
if (e.key === 'Tab' || (e.key === 'Enter' && _slashSelected >= 0)) {
if (e.key === 'Tab' || e.key === 'Enter') {
e.preventDefault();
const pick = _slashSelected >= 0 ? _slashMatches[_slashSelected] : _slashMatches[0];
if (pick) selectSlashItem(pick.cmd);
@@ -1757,6 +1792,11 @@ function renderAvailableExtensionCard(entry) {
}).then(function(res) {
if (res.success) {
showToast('Installed ' + entry.display_name, 'success');
// OAuth popup if auth started during install (builtin creds)
if (res.auth_url) {
showToast('Opening authentication for ' + entry.display_name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
}
loadExtensions();
// Auto-open configure for WASM channels
if (entry.kind === 'wasm_channel') {
@@ -1928,14 +1968,6 @@ function renderExtensionCard(ext) {
card.appendChild(errorDiv);
}
// Show "coming soon" note for non-Telegram channels that are configured but not fully supported yet
if (ext.kind === 'wasm_channel' && ext.name !== 'telegram'
&& (ext.activation_status === 'configured' || ext.active)) {
const noteDiv = document.createElement('div');
noteDiv.className = 'ext-note';
noteDiv.textContent = 'Full integration coming soon. Use the CLI to complete setup.';
card.appendChild(noteDiv);
}
const actions = document.createElement('div');
actions.className = 'ext-actions';
@@ -1966,24 +1998,29 @@ function renderExtensionCard(ext) {
actions.appendChild(setupBtn);
}
} else {
// Non-WASM-channel extensions: original behavior
if (!ext.active) {
// WASM tools / MCP servers
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = ext.active ? 'Active' : 'Installed';
actions.appendChild(activeLabel);
// MCP servers may be installed but inactive — show Activate button
if (ext.kind === 'mcp_server' && !ext.active) {
const activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', () => activateExtension(ext.name));
actions.appendChild(activateBtn);
} else {
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = 'Active';
actions.appendChild(activeLabel);
}
if (ext.needs_setup) {
// Show Configure/Reconfigure button when there are secrets to enter.
// Skip when has_auth is true but needs_setup is false and not yet authenticated —
// this means OAuth credentials resolve automatically (builtin/env) and the user
// just needs to complete the OAuth flow, not fill in a config form.
if (ext.needs_setup || (ext.has_auth && ext.authenticated)) {
const configBtn = document.createElement('button');
configBtn.className = 'btn-ext configure';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Setup';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Configure';
configBtn.addEventListener('click', () => showConfigureModal(ext.name));
actions.appendChild(configBtn);
}
@@ -2013,6 +2050,11 @@ function activateExtension(name) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/activate', { method: 'POST' })
.then((res) => {
if (res.success) {
// Even on success, the tool may need OAuth (e.g., WASM loaded but no token yet)
if (res.auth_url) {
showToast('Opening authentication for ' + name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
}
loadExtensions();
return;
}
@@ -2163,17 +2205,19 @@ function submitConfigureModal(name, fields) {
.then((res) => {
closeConfigureModal();
if (res.success) {
if (res.activated) {
showToast('Configured and activated ' + name, 'success');
} else if (res.needs_restart) {
showToast('Configured ' + name + '. Use Reconfigure to re-enter credentials and activate.', 'info');
} else {
showToast(res.message, 'success');
if (res.auth_url) {
// OAuth flow started — open consent popup. The auth_completed SSE will
// not arrive immediately (it fires after OAuth callback), so show a toast now.
showToast('Opening OAuth authorization for ' + name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
loadExtensions();
}
// For non-OAuth success: the server always broadcasts auth_completed SSE,
// which will show the toast and refresh extensions — no need to do it here too.
} else {
showToast(res.message || 'Configuration failed', 'error');
loadExtensions();
}
loadExtensions();
})
.catch((err) => {
btns.forEach(function(b) { b.disabled = false; });
@@ -2232,7 +2276,7 @@ function approvePairing(channel, code, container) {
}).then(res => {
if (res.success) {
showToast('Pairing approved', 'success');
loadPairingRequests(channel, container);
loadExtensions();
} else {
showToast(res.message || 'Approve failed', 'error');
}
@@ -2255,53 +2299,6 @@ function stopPairingPoll() {
}
}
// --- Gateway restart ---
function restartGateway() {
if (!confirm('Restart IronClaw gateway? Active connections will be dropped.')) return;
apiFetch('/api/gateway/restart', { method: 'POST' })
.then(function() {
showRestartOverlay();
})
.catch(function() {
showRestartOverlay();
});
}
function showRestartOverlay() {
var overlay = document.createElement('div');
overlay.className = 'restart-overlay';
overlay.innerHTML = '<div class="restart-message">'
+ '<div class="restart-spinner"></div>'
+ '<h2>Restarting IronClaw...</h2>'
+ '<p>Waiting for server to come back online</p>'
+ '</div>';
document.body.appendChild(overlay);
var pollCount = 0;
var pollTimer = setInterval(function() {
pollCount++;
if (pollCount > 30) { // 60 seconds
clearInterval(pollTimer);
overlay.querySelector('h2').textContent = 'Restart timed out';
overlay.querySelector('p').textContent = 'Server did not come back within 60 seconds. Check logs.';
overlay.querySelector('.restart-spinner').style.display = 'none';
return;
}
fetch('/api/gateway/status', {
headers: { 'Authorization': 'Bearer ' + token },
})
.then(function(r) {
if (r.ok) {
clearInterval(pollTimer);
window.location.reload();
}
})
.catch(function() { /* still restarting */ });
}, 2000);
}
// --- WASM channel stepper ---
function renderWasmChannelStepper(ext) {
@@ -2309,23 +2306,17 @@ function renderWasmChannelStepper(ext) {
stepper.className = 'ext-stepper';
var status = ext.activation_status || 'installed';
var isTelegram = ext.name === 'telegram';
// Telegram gets a 3-step stepper (Installed → Configured → Active/Pairing).
// Other channels only get 2 steps (Installed → Configured) since full
// integration isn't available in the web UI yet.
var steps = [
{ label: 'Installed', key: 'installed' },
{ label: 'Configured', key: 'configured' },
{ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' },
];
if (isTelegram) {
steps.push({ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' });
}
var reachedIdx;
if (status === 'active') reachedIdx = isTelegram ? 2 : 1;
if (status === 'active') reachedIdx = 2;
else if (status === 'pairing') reachedIdx = 2;
else if (status === 'failed') reachedIdx = isTelegram ? 2 : 1;
else if (status === 'failed') reachedIdx = 2;
else if (status === 'configured') reachedIdx = 1;
else reachedIdx = 0;
@@ -2436,9 +2427,8 @@ function renderJobsList(jobs) {
let actionBtns = '';
if (job.state === 'pending' || job.state === 'in_progress') {
actionBtns = '<button class="btn-cancel" onclick="event.stopPropagation(); cancelJob(\'' + job.id + '\')">Cancel</button>';
} else if (job.state === 'failed' || job.state === 'interrupted') {
actionBtns = '<button class="btn-restart" onclick="event.stopPropagation(); restartJob(\'' + job.id + '\')">Restart</button>';
}
// Retry is only shown in the detail view where can_restart is available.
return '<tr class="job-row" onclick="openJobDetail(\'' + job.id + '\')">'
+ '<td title="' + escapeHtml(job.id) + '">' + shortId + '</td>'
@@ -2467,10 +2457,12 @@ function restartJob(jobId) {
apiFetch('/api/jobs/' + jobId + '/restart', { method: 'POST' })
.then((res) => {
showToast('Job restarted as ' + (res.new_job_id || '').substring(0, 8), 'success');
loadJobs();
})
.catch((err) => {
showToast('Failed to restart job: ' + err.message, 'error');
})
.finally(() => {
loadJobs();
});
}
@@ -2505,8 +2497,8 @@ function renderJobDetail(job) {
+ '<h2>' + escapeHtml(job.title) + '</h2>'
+ '<span class="badge ' + stateClass + '">' + escapeHtml(job.state) + '</span>';
if (job.state === 'failed' || job.state === 'interrupted') {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Restart</button>';
if ((job.state === 'failed' || job.state === 'interrupted') && job.can_restart === true) {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Retry</button>';
}
if (job.browse_url) {
headerHtml += '<a class="btn-browse" href="' + escapeHtml(job.browse_url) + '" target="_blank">Browse Files</a>';
@@ -2753,7 +2745,7 @@ function renderJobActivity(container, job) {
activityCurrentJobId = job ? job.id : null;
activityRenderedLiveIndex = 0;
container.innerHTML = '<div class="activity-toolbar">'
let html = '<div class="activity-toolbar">'
+ '<select id="activity-type-filter">'
+ '<option value="all">All Events</option>'
+ '<option value="message">Messages</option>'
@@ -2762,12 +2754,17 @@ function renderJobActivity(container, job) {
+ '</select>'
+ '<label class="logs-checkbox"><input type="checkbox" id="activity-autoscroll" checked> Auto-scroll</label>'
+ '</div>'
+ '<div class="activity-terminal" id="activity-terminal"></div>'
+ '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
+ '<div class="activity-terminal" id="activity-terminal"></div>';
if (job && job.can_prompt === true) {
html += '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
}
container.innerHTML = html;
document.getElementById('activity-type-filter').addEventListener('change', applyActivityFilter);
@@ -2776,9 +2773,9 @@ function renderJobActivity(container, job) {
const sendBtn = document.getElementById('activity-send-btn');
const doneBtn = document.getElementById('activity-done-btn');
sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
input.addEventListener('keydown', (e) => {
if (sendBtn) sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
if (doneBtn) doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
if (input) input.addEventListener('keydown', (e) => {
if (e.key === 'Enter') sendJobPrompt(job.id, false);
});
@@ -3065,7 +3062,11 @@ function renderRoutineDetail(routine) {
function triggerRoutine(id) {
apiFetch('/api/routines/' + id + '/trigger', { method: 'POST' })
.then(() => showToast('Routine triggered', 'success'))
.then(() => {
showToast('Routine triggered', 'success');
if (currentRoutineId === id) openRoutineDetail(id);
else loadRoutines();
})
.catch((err) => showToast('Trigger failed: ' + err.message, 'error'));
}
@@ -3615,7 +3616,7 @@ function formatTimeAgo(epochMs) {
}
function installSkill(nameOrSlug, url, btn) {
var body = { name: nameOrSlug };
var body = { name: nameOrSlug, slug: nameOrSlug };
if (url) body.url = url;
apiFetch('/api/skills/install', {
+1 -1
View File
@@ -2,7 +2,7 @@
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<title>IronClaw</title>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="preconnect" href="https://fonts.googleapis.com">
+9 -38
View File
@@ -30,6 +30,7 @@ body {
background: var(--bg);
color: var(--text);
height: 100vh;
height: 100dvh;
display: flex;
flex-direction: column;
overflow: hidden;
@@ -41,6 +42,7 @@ body {
align-items: center;
justify-content: center;
height: 100vh;
height: 100dvh;
}
.auth-card-login {
@@ -141,6 +143,7 @@ body {
display: none;
flex-direction: column;
height: 100vh;
height: 100dvh;
}
/* Tab Bar */
@@ -550,6 +553,10 @@ body {
border-color: rgba(230, 76, 76, 0.3);
}
.activity-tool-card[data-status="fail"] .activity-tool-name {
color: var(--danger);
}
.activity-tool-header {
display: flex;
align-items: center;
@@ -987,7 +994,7 @@ body {
/* Chat input */
.chat-input {
display: flex;
padding: 12px 16px;
padding: 12px 16px max(12px, env(safe-area-inset-bottom)) 16px;
gap: 8px;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
@@ -1808,6 +1815,7 @@ body {
.job-files {
display: flex;
height: calc(100vh - 280px);
height: calc(100dvh - 280px);
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
@@ -2312,43 +2320,6 @@ body {
margin-top: 6px;
}
/* Restart overlay */
.restart-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.8);
z-index: 2000;
display: flex;
align-items: center;
justify-content: center;
}
.restart-message {
text-align: center;
color: var(--text);
}
.restart-message h2 {
margin: 16px 0 8px;
}
.restart-message p {
color: var(--text-secondary);
}
.restart-spinner {
width: 40px;
height: 40px;
border: 3px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin: 0 auto;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
+19 -5
View File
@@ -123,6 +123,10 @@ pub enum SseEvent {
name: String,
success: bool,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
parameters: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
thread_id: Option<String>,
},
#[serde(rename = "tool_result")]
@@ -332,6 +336,15 @@ pub struct JobDetailResponse {
#[serde(skip_serializing_if = "Option::is_none")]
pub job_mode: Option<String>,
pub transitions: Vec<TransitionInfo>,
/// Whether this job can be restarted from the UI.
#[serde(default)]
pub can_restart: bool,
/// Whether follow-up prompts can be sent to this job.
#[serde(default)]
pub can_prompt: bool,
/// The kind of job: "sandbox" or "agent".
#[serde(skip_serializing_if = "Option::is_none")]
pub job_kind: Option<String>,
}
// --- Project Files ---
@@ -379,6 +392,9 @@ pub struct ExtensionInfo {
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
#[serde(default)]
pub has_auth: bool,
/// WASM channel activation status: "installed", "configured", "active", "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_status: Option<String>,
@@ -451,9 +467,6 @@ pub struct ActionResponse {
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a gateway restart is needed (activation failed).
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
}
impl ActionResponse {
@@ -465,7 +478,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
@@ -477,7 +489,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
}
@@ -562,6 +573,9 @@ pub struct SkillSearchResponse {
#[derive(Debug, Deserialize)]
pub struct SkillInstallRequest {
pub name: String,
/// Registry slug (e.g. "owner/skill-name"). Preferred over `name` for
/// constructing the download URL when fetching from ClawHub.
pub slug: Option<String>,
pub url: Option<String>,
pub content: Option<String>,
}
+5 -5
View File
@@ -242,7 +242,7 @@ async fn handle_client_message(
} => {
if let Some(ref ext_mgr) = state.extension_manager {
match ext_mgr.auth(&extension_name, Some(&token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
let msg = match ext_mgr.activate(&extension_name).await {
Ok(r) => format!(
"{} authenticated ({} tools loaded)",
@@ -268,9 +268,9 @@ async fn handle_client_message(
.sse
.broadcast(crate::channels::web::types::SseEvent::AuthRequired {
extension_name,
instructions: result.instructions,
auth_url: result.auth_url,
setup_url: result.setup_url,
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
}
Err(e) => {
@@ -483,6 +483,7 @@ mod tests {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -493,7 +494,6 @@ mod tests {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
}
}
}
+812 -13
View File
@@ -17,10 +17,18 @@
//! - **Runtime**: Users can set GOOGLE_OAUTH_CLIENT_ID / GOOGLE_OAUTH_CLIENT_SECRET
//! env vars, which take priority over built-in defaults.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore;
use sha2::{Digest, Sha256};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener;
use tokio::sync::RwLock;
use crate::secrets::{CreateSecretParams, SecretsStore};
// ── Built-in credentials ────────────────────────────────────────────────
@@ -121,6 +129,9 @@ pub enum OAuthCallbackError {
#[error("Timed out waiting for authorization")]
Timeout,
#[error("CSRF state mismatch: expected {expected}, got {actual}")]
StateMismatch { expected: String, actual: String },
#[error("IO error: {0}")]
Io(String),
}
@@ -177,16 +188,22 @@ pub async fn bind_callback_listener() -> Result<TcpListener, OAuthCallbackError>
/// extracts the value of `param_name` (e.g., "code" or "token"), and shows a branded
/// landing page using `display_name` (e.g., "Google", "Notion", "NEAR AI").
///
/// When `expected_state` is `Some`, the callback's `state` query parameter is validated
/// against it to prevent CSRF attacks. If the state doesn't match, the callback is
/// rejected with an error page.
///
/// Times out after 5 minutes.
pub async fn wait_for_callback(
listener: TcpListener,
path_prefix: &str,
param_name: &str,
display_name: &str,
expected_state: Option<&str>,
) -> Result<String, OAuthCallbackError> {
let path_prefix = path_prefix.to_string();
let param_name = param_name.to_string();
let display_name = display_name.to_string();
let expected_state = expected_state.map(String::from);
tokio::time::timeout(Duration::from_secs(300), async move {
loop {
@@ -221,17 +238,29 @@ pub async fn wait_for_callback(
return Err(OAuthCallbackError::Denied);
}
// Look for the target parameter
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == param_name {
let value = urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned();
// Parse all query params into a map for validation
let params: HashMap<&str, String> = query
.split('&')
.filter_map(|p| {
let mut parts = p.splitn(2, '=');
let key = parts.next()?;
let val = parts.next().unwrap_or("");
Some((
key,
urlencoding::decode(val)
.unwrap_or_else(|_| val.into())
.into_owned(),
))
})
.collect();
let html = landing_html(&display_name, true);
// Validate CSRF state parameter
if let Some(ref expected) = expected_state {
let actual = params.get("state").cloned().unwrap_or_default();
if actual != *expected {
let html = landing_html(&display_name, false);
let response = format!(
"HTTP/1.1 200 OK\r\n\
"HTTP/1.1 403 Forbidden\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
@@ -239,11 +268,29 @@ pub async fn wait_for_callback(
html
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(value);
return Err(OAuthCallbackError::StateMismatch {
expected: expected.clone(),
actual,
});
}
}
// Look for the target parameter
if let Some(value) = params.get(param_name.as_str()) {
let html = landing_html(&display_name, true);
let response = format!(
"HTTP/1.1 200 OK\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
{}",
html
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(value.clone());
}
}
// Not the callback we're looking for
@@ -271,7 +318,288 @@ fn html_escape(s: &str) -> String {
out
}
/// HTML landing page shown in the browser after an OAuth redirect.
// ── Shared OAuth flow steps ─────────────────────────────────────────
/// Response from the OAuth token exchange.
pub struct OAuthTokenResponse {
pub access_token: String,
pub refresh_token: Option<String>,
pub expires_in: Option<u64>,
}
/// Result of building an OAuth 2.0 authorization URL.
pub struct OAuthUrlResult {
/// The full authorization URL to redirect the user to.
pub url: String,
/// PKCE code verifier (must be sent with the token exchange request).
pub code_verifier: Option<String>,
/// Random state parameter for CSRF protection (must be validated in callback).
pub state: String,
}
/// Build an OAuth 2.0 authorization URL with optional PKCE and CSRF state.
///
/// Returns an `OAuthUrlResult` containing the authorization URL, optional PKCE
/// code verifier, and a random `state` parameter for CSRF protection. The caller
/// must validate the `state` value in the callback before exchanging the code.
pub fn build_oauth_url(
authorization_url: &str,
client_id: &str,
redirect_uri: &str,
scopes: &[String],
use_pkce: bool,
extra_params: &HashMap<String, String>,
) -> OAuthUrlResult {
// Generate PKCE verifier and challenge
let (code_verifier, code_challenge) = if use_pkce {
let mut verifier_bytes = [0u8; 32];
rand::thread_rng().fill_bytes(&mut verifier_bytes);
let verifier = URL_SAFE_NO_PAD.encode(verifier_bytes);
let mut hasher = Sha256::new();
hasher.update(verifier.as_bytes());
let challenge = URL_SAFE_NO_PAD.encode(hasher.finalize());
(Some(verifier), Some(challenge))
} else {
(None, None)
};
// Generate random state for CSRF protection
let mut state_bytes = [0u8; 32];
rand::thread_rng().fill_bytes(&mut state_bytes);
let state = URL_SAFE_NO_PAD.encode(state_bytes);
// Build authorization URL
let mut auth_url = format!(
"{}?client_id={}&response_type=code&redirect_uri={}&state={}",
authorization_url,
urlencoding::encode(client_id),
urlencoding::encode(redirect_uri),
urlencoding::encode(&state),
);
if !scopes.is_empty() {
auth_url.push_str(&format!(
"&scope={}",
urlencoding::encode(&scopes.join(" "))
));
}
if let Some(ref challenge) = code_challenge {
auth_url.push_str(&format!(
"&code_challenge={}&code_challenge_method=S256",
challenge
));
}
for (key, value) in extra_params {
auth_url.push_str(&format!(
"&{}={}",
urlencoding::encode(key),
urlencoding::encode(value)
));
}
OAuthUrlResult {
url: auth_url,
code_verifier,
state,
}
}
/// Exchange an OAuth authorization code for tokens.
///
/// POSTs to `token_url` with the authorization code and optional PKCE verifier.
/// If `client_secret` is provided, uses HTTP Basic auth; otherwise includes
/// `client_id` in the form body (for public clients).
pub async fn exchange_oauth_code(
token_url: &str,
client_id: &str,
client_secret: Option<&str>,
code: &str,
redirect_uri: &str,
code_verifier: Option<&str>,
access_token_field: &str,
) -> Result<OAuthTokenResponse, OAuthCallbackError> {
let client = reqwest::Client::new();
let mut token_params = vec![
("grant_type", "authorization_code".to_string()),
("code", code.to_string()),
("redirect_uri", redirect_uri.to_string()),
];
if let Some(verifier) = code_verifier {
token_params.push(("code_verifier", verifier.to_string()));
}
let mut request = client.post(token_url);
if let Some(secret) = client_secret {
request = request.basic_auth(client_id, Some(secret));
} else {
token_params.push(("client_id", client_id.to_string()));
}
let token_response = request
.form(&token_params)
.send()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Token exchange request failed: {}", e)))?;
if !token_response.status().is_success() {
let status = token_response.status();
let body = token_response.text().await.unwrap_or_default();
return Err(OAuthCallbackError::Io(format!(
"Token exchange failed: {} - {}",
status, body
)));
}
let token_data: serde_json::Value = token_response
.json()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to parse token response: {}", e)))?;
let access_token = token_data
.get(access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
// Log only the field names present, not values (which may contain tokens)
let fields: Vec<&str> = token_data
.as_object()
.map(|o| o.keys().map(|k| k.as_str()).collect())
.unwrap_or_default();
OAuthCallbackError::Io(format!(
"No '{}' field in token response (fields present: {:?})",
access_token_field, fields
))
})?
.to_string();
let refresh_token = token_data
.get("refresh_token")
.and_then(|v| v.as_str())
.map(String::from);
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
Ok(OAuthTokenResponse {
access_token,
refresh_token,
expires_in,
})
}
/// Store OAuth tokens (access + refresh) in the secrets store.
///
/// Also stores the granted scopes as `{secret_name}_scopes` so that scope
/// expansion can be detected on subsequent activations.
#[allow(clippy::too_many_arguments)]
pub async fn store_oauth_tokens(
store: &(dyn SecretsStore + Send + Sync),
user_id: &str,
secret_name: &str,
provider: Option<&str>,
access_token: &str,
refresh_token: Option<&str>,
expires_in: Option<u64>,
scopes: &[String],
) -> Result<(), OAuthCallbackError> {
let mut params = CreateSecretParams::new(secret_name, access_token);
if let Some(prov) = provider {
params = params.with_provider(prov);
}
if let Some(secs) = expires_in {
let expires_at = chrono::Utc::now() + chrono::Duration::seconds(secs as i64);
params = params.with_expiry(expires_at);
}
store
.create(user_id, params)
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to save token: {}", e)))?;
// Store refresh token separately (no expiry, it's long-lived)
if let Some(rt) = refresh_token {
let refresh_name = format!("{}_refresh_token", secret_name);
let mut refresh_params = CreateSecretParams::new(&refresh_name, rt);
if let Some(prov) = provider {
refresh_params = refresh_params.with_provider(prov);
}
store
.create(user_id, refresh_params)
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to save refresh token: {}", e)))?;
}
// Store granted scopes for scope expansion detection
if !scopes.is_empty() {
let scopes_name = format!("{}_scopes", secret_name);
let scopes_value = scopes.join(" ");
let scopes_params = CreateSecretParams::new(&scopes_name, &scopes_value);
// Best-effort: scope tracking failure shouldn't block auth
let _ = store.create(user_id, scopes_params).await;
}
Ok(())
}
/// Validate an OAuth token against a tool's validation endpoint.
///
/// Sends a request to the configured endpoint with the token as a Bearer header.
/// Returns `Ok(())` if the response status matches the expected success status,
/// or an error with details if validation fails (wrong account, expired token, etc.).
pub async fn validate_oauth_token(
token: &str,
validation: &crate::tools::wasm::ValidationEndpointSchema,
) -> Result<(), OAuthCallbackError> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.build()
.map_err(|e| OAuthCallbackError::Io(format!("Failed to build HTTP client: {}", e)))?;
let request = match validation.method.to_uppercase().as_str() {
"POST" => client.post(&validation.url),
_ => client.get(&validation.url),
};
let mut request = request.header("Authorization", format!("Bearer {}", token));
// Add custom headers from the validation schema (e.g., Notion-Version)
for (key, value) in &validation.headers {
request = request.header(key, value);
}
let response = request
.send()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Validation request failed: {}", e)))?;
if response.status().as_u16() == validation.success_status {
Ok(())
} else {
let status = response.status();
let body = response.text().await.unwrap_or_default();
let truncated: String = if body.len() > 200 {
let mut end = 200;
while end > 0 && !body.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &body[..end])
} else {
body
};
Err(OAuthCallbackError::Io(format!(
"Token validation failed: HTTP {} (expected {}): {}",
status, validation.success_status, truncated
)))
}
}
// ── Landing pages ───────────────────────────────────────────────────
pub fn landing_html(provider_name: &str, success: bool) -> String {
let safe_name = html_escape(provider_name);
let (icon, heading, subtitle, accent) = if success {
@@ -357,6 +685,219 @@ pub fn landing_html(provider_name: &str, success: bool) -> String {
)
}
// ── Gateway callback support ─────────────────────────────────────────
/// State for an in-progress OAuth flow, keyed by CSRF `state` parameter.
///
/// Created by `start_wasm_oauth()` and consumed by the web gateway's
/// `/oauth/callback` handler when running in hosted mode.
pub struct PendingOAuthFlow {
/// Extension name (e.g., "google_calendar").
pub extension_name: String,
/// Human-readable display name (e.g., "Google Calendar").
pub display_name: String,
/// OAuth token exchange URL.
pub token_url: String,
/// OAuth client ID.
pub client_id: String,
/// OAuth client secret (optional for PKCE-only flows).
pub client_secret: Option<String>,
/// The redirect_uri used in the authorization request.
pub redirect_uri: String,
/// PKCE code verifier (must match the code_challenge sent in the auth URL).
pub code_verifier: Option<String>,
/// Field name in token response containing the access token.
pub access_token_field: String,
/// Secret name for storage (e.g., "google_oauth_token").
pub secret_name: String,
/// Provider hint (e.g., "google").
pub provider: Option<String>,
/// Token validation endpoint (optional).
pub validation_endpoint: Option<crate::tools::wasm::ValidationEndpointSchema>,
/// Scopes that were requested.
pub scopes: Vec<String>,
/// User ID for secret storage.
pub user_id: String,
/// Secrets store reference for token persistence.
pub secrets: Arc<dyn SecretsStore + Send + Sync>,
/// SSE broadcast sender for notifying the web UI.
pub sse_sender: Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>,
/// Gateway auth token for authenticating with the platform token exchange proxy.
pub gateway_token: Option<String>,
/// When this flow was created (for expiry).
pub created_at: std::time::Instant,
}
impl std::fmt::Debug for PendingOAuthFlow {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PendingOAuthFlow")
.field("extension_name", &self.extension_name)
.field("display_name", &self.display_name)
.field("secret_name", &self.secret_name)
.field("created_at", &self.created_at)
.finish_non_exhaustive()
}
}
/// Thread-safe registry of pending OAuth flows, keyed by CSRF `state` parameter.
pub type PendingOAuthRegistry = Arc<RwLock<HashMap<String, PendingOAuthFlow>>>;
/// Create a new empty pending OAuth flow registry.
pub fn new_pending_oauth_registry() -> PendingOAuthRegistry {
Arc::new(RwLock::new(HashMap::new()))
}
/// Returns `true` if OAuth callbacks should be routed through the web gateway
/// instead of the local TCP listener.
///
/// This is the case when `IRONCLAW_OAUTH_CALLBACK_URL` is set to a non-loopback
/// URL, meaning the user's browser will redirect to a hosted gateway rather than
/// localhost.
pub fn use_gateway_callback() -> bool {
std::env::var("IRONCLAW_OAUTH_CALLBACK_URL")
.ok()
.filter(|v| !v.is_empty())
.map(|raw| {
url::Url::parse(&raw)
.ok()
.and_then(|u| u.host_str().map(String::from))
.map(|host| !is_loopback_host(&host))
.unwrap_or(false)
})
.unwrap_or(false)
}
/// Maximum age for pending OAuth flows (5 minutes, matching TCP listener timeout).
pub const OAUTH_FLOW_EXPIRY: Duration = Duration::from_secs(300);
/// Remove expired flows from the registry.
///
/// Called when inserting new flows to prevent accumulation from abandoned
/// OAuth attempts.
pub async fn sweep_expired_flows(registry: &PendingOAuthRegistry) {
let mut flows = registry.write().await;
flows.retain(|_, flow| flow.created_at.elapsed() < OAUTH_FLOW_EXPIRY);
}
// ── Platform routing helpers ────────────────────────────────────────
/// Prepend instance name to CSRF state for platform routing.
///
/// The NEAR AI platform nginx proxy at `auth.DOMAIN` parses the instance name
/// from the `state` query parameter (format: `instance:nonce`) to route the
/// OAuth callback to the correct container.
///
/// Returns the nonce unchanged when `IRONCLAW_INSTANCE_NAME` is not set
/// (local/non-platform mode).
pub fn build_platform_state(nonce: &str) -> String {
let instance = std::env::var("IRONCLAW_INSTANCE_NAME")
.or_else(|_| std::env::var("OPENCLAW_INSTANCE_NAME"))
.ok()
.filter(|v| !v.is_empty());
match instance {
Some(name) => format!("{}:{}", name, nonce),
None => nonce.to_string(),
}
}
/// Strip the instance prefix from a state parameter to recover the lookup nonce.
///
/// `"myinstance:abc123"` → `"abc123"`, `"abc123"` → `"abc123"` (no prefix).
///
/// Safe because nonces are base64url-encoded (`[A-Za-z0-9_-]`, no colons).
pub fn strip_instance_prefix(state: &str) -> &str {
state
.split_once(':')
.map(|(_, nonce)| nonce)
.unwrap_or(state)
}
/// Exchange an OAuth authorization code via the platform's token exchange proxy.
///
/// The proxy holds `client_secret` server-side so the container never sees it.
/// Authenticated via the gateway auth token (Bearer header).
///
/// The proxy expects form params `{code, redirect_uri, code_verifier}` and
/// returns a standard Google token response `{access_token, refresh_token, expires_in}`.
pub async fn exchange_via_proxy(
proxy_url: &str,
gateway_token: &str,
code: &str,
redirect_uri: &str,
code_verifier: Option<&str>,
access_token_field: &str,
) -> Result<OAuthTokenResponse, OAuthCallbackError> {
if gateway_token.is_empty() {
return Err(OAuthCallbackError::Io(
"Gateway auth token is required for proxy token exchange".to_string(),
));
}
let exchange_url = format!("{}/oauth/exchange", proxy_url.trim_end_matches('/'));
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(60))
.build()
.map_err(|e| OAuthCallbackError::Io(format!("Failed to build HTTP client: {}", e)))?;
let mut params = vec![
("code", code.to_string()),
("redirect_uri", redirect_uri.to_string()),
];
if let Some(verifier) = code_verifier {
params.push(("code_verifier", verifier.to_string()));
}
let response = client
.post(&exchange_url)
.bearer_auth(gateway_token)
.form(&params)
.send()
.await
.map_err(|e| {
OAuthCallbackError::Io(format!("Token exchange proxy request failed: {}", e))
})?;
if !response.status().is_success() {
let status = response.status();
let body = response.text().await.unwrap_or_default();
return Err(OAuthCallbackError::Io(format!(
"Token exchange proxy failed: {} - {}",
status, body
)));
}
let token_data: serde_json::Value = response
.json()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to parse proxy response: {}", e)))?;
let access_token = token_data
.get(access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
let fields: Vec<&str> = token_data
.as_object()
.map(|o| o.keys().map(|k| k.as_str()).collect())
.unwrap_or_default();
OAuthCallbackError::Io(format!(
"No '{}' field in proxy response (fields present: {:?})",
access_token_field, fields
))
})?
.to_string();
let refresh_token = token_data
.get("refresh_token")
.and_then(|v| v.as_str())
.map(String::from);
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
Ok(OAuthTokenResponse {
access_token,
refresh_token,
expires_in,
})
}
#[cfg(test)]
mod tests {
use std::sync::Mutex;
@@ -512,4 +1053,262 @@ mod tests {
assert!(html.contains("#ef4444")); // red accent
assert!(!html.contains("Connected"));
}
#[test]
fn test_build_oauth_url_basic() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result = build_oauth_url(
"https://accounts.google.com/o/oauth2/auth",
"my-client-id",
"http://localhost:9876/callback",
&["openid".to_string(), "email".to_string()],
false,
&HashMap::new(),
);
assert!(
result
.url
.starts_with("https://accounts.google.com/o/oauth2/auth?")
);
assert!(result.url.contains("client_id=my-client-id"));
assert!(result.url.contains("response_type=code"));
assert!(result.url.contains("redirect_uri="));
assert!(result.url.contains("scope=openid%20email"));
assert!(result.url.contains("state="));
assert!(result.code_verifier.is_none());
assert!(!result.state.is_empty());
}
#[test]
fn test_build_oauth_url_with_pkce() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result = build_oauth_url(
"https://auth.example.com/authorize",
"client-123",
"http://localhost:9876/callback",
&[],
true,
&HashMap::new(),
);
assert!(result.url.contains("code_challenge="));
assert!(result.url.contains("code_challenge_method=S256"));
assert!(result.code_verifier.is_some());
let verifier = result.code_verifier.unwrap();
assert!(!verifier.is_empty());
}
#[test]
fn test_build_oauth_url_with_extra_params() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let mut extra = HashMap::new();
extra.insert("access_type".to_string(), "offline".to_string());
extra.insert("prompt".to_string(), "consent".to_string());
let result = build_oauth_url(
"https://auth.example.com/authorize",
"client-123",
"http://localhost:9876/callback",
&["read".to_string()],
false,
&extra,
);
assert!(result.url.contains("access_type=offline"));
assert!(result.url.contains("prompt=consent"));
}
#[test]
fn test_build_oauth_url_state_is_unique() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result1 = build_oauth_url(
"https://auth.example.com/authorize",
"client",
"http://localhost:9876/callback",
&[],
false,
&HashMap::new(),
);
let result2 = build_oauth_url(
"https://auth.example.com/authorize",
"client",
"http://localhost:9876/callback",
&[],
false,
&HashMap::new(),
);
// State should be different each time (random)
assert_ne!(result1.state, result2.state);
}
#[test]
fn test_use_gateway_callback_false_by_default() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
}
}
}
#[test]
fn test_use_gateway_callback_true_for_hosted() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var(
"IRONCLAW_OAUTH_CALLBACK_URL",
"https://kind-deer.agent1.near.ai",
);
}
assert!(crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_use_gateway_callback_false_for_localhost() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", "http://127.0.0.1:3001");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_use_gateway_callback_false_for_empty() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", "");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_build_platform_state_with_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_INSTANCE_NAME", "kind-deer");
}
assert_eq!(build_platform_state("abc123"), "kind-deer:abc123");
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
} else {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
}
}
}
#[test]
fn test_build_platform_state_without_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
let original_oc = std::env::var("OPENCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
std::env::remove_var("OPENCLAW_INSTANCE_NAME");
}
assert_eq!(build_platform_state("abc123"), "abc123");
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
}
if let Some(val) = original_oc {
std::env::set_var("OPENCLAW_INSTANCE_NAME", val);
}
}
}
#[test]
fn test_build_platform_state_with_openclaw_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original_ic = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
let original_oc = std::env::var("OPENCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
std::env::set_var("OPENCLAW_INSTANCE_NAME", "quiet-lion");
}
assert_eq!(build_platform_state("xyz789"), "quiet-lion:xyz789");
unsafe {
if let Some(val) = original_ic {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
}
if let Some(val) = original_oc {
std::env::set_var("OPENCLAW_INSTANCE_NAME", val);
} else {
std::env::remove_var("OPENCLAW_INSTANCE_NAME");
}
}
}
#[test]
fn test_strip_instance_prefix_with_colon() {
use crate::cli::oauth_defaults::strip_instance_prefix;
assert_eq!(strip_instance_prefix("kind-deer:abc123"), "abc123");
assert_eq!(strip_instance_prefix("my-instance:xyz"), "xyz");
}
#[test]
fn test_strip_instance_prefix_without_colon() {
use crate::cli::oauth_defaults::strip_instance_prefix;
assert_eq!(strip_instance_prefix("abc123"), "abc123");
assert_eq!(strip_instance_prefix(""), "");
}
}
+58 -184
View File
@@ -782,11 +782,7 @@ async fn auth_tool_oauth(
auth: &crate::tools::wasm::AuthCapabilitySchema,
oauth: &crate::tools::wasm::OAuthConfigSchema,
) -> anyhow::Result<()> {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore;
use sha2::{Digest, Sha256};
use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
use crate::cli::oauth_defaults;
let display_name = auth.display_name.as_deref().unwrap_or(&auth.secret_name);
@@ -827,142 +823,69 @@ async fn auth_tool_oauth(
println!();
let listener = oauth_defaults::bind_callback_listener().await?;
let redirect_uri = format!("http://localhost:{}/callback", OAUTH_CALLBACK_PORT);
let redirect_uri = format!("{}/callback", oauth_defaults::callback_url());
// Generate PKCE verifier and challenge
let (code_verifier, code_challenge) = if oauth.use_pkce {
let mut verifier_bytes = [0u8; 32];
rand::thread_rng().fill_bytes(&mut verifier_bytes);
let verifier = URL_SAFE_NO_PAD.encode(verifier_bytes);
let mut hasher = Sha256::new();
hasher.update(verifier.as_bytes());
let challenge = URL_SAFE_NO_PAD.encode(hasher.finalize());
(Some(verifier), Some(challenge))
} else {
(None, None)
};
// Build authorization URL
let mut auth_url = format!(
"{}?client_id={}&response_type=code&redirect_uri={}",
oauth.authorization_url,
urlencoding::encode(&client_id),
urlencoding::encode(&redirect_uri)
// Build authorization URL with PKCE and CSRF state
let oauth_result = oauth_defaults::build_oauth_url(
&oauth.authorization_url,
&client_id,
&redirect_uri,
&oauth.scopes,
oauth.use_pkce,
&oauth.extra_params,
);
if !oauth.scopes.is_empty() {
auth_url.push_str(&format!(
"&scope={}",
urlencoding::encode(&oauth.scopes.join(" "))
));
}
if let Some(ref challenge) = code_challenge {
auth_url.push_str(&format!(
"&code_challenge={}&code_challenge_method=S256",
challenge
));
}
// Add extra params
for (key, value) in &oauth.extra_params {
auth_url.push_str(&format!(
"&{}={}",
urlencoding::encode(key),
urlencoding::encode(value)
));
}
let code_verifier = oauth_result.code_verifier;
println!(" Opening browser for {} login...", display_name);
println!();
if let Err(e) = open::that(&auth_url) {
if let Err(e) = open::that(&oauth_result.url) {
println!(" Could not open browser: {}", e);
println!(" Please open this URL manually:");
println!(" {}", auth_url);
println!(" {}", oauth_result.url);
}
println!(" Waiting for authorization...");
let code =
oauth_defaults::wait_for_callback(listener, "/callback", "code", display_name).await?;
let code = oauth_defaults::wait_for_callback(
listener,
"/callback",
"code",
display_name,
Some(&oauth_result.state),
)
.await?;
println!();
println!(" Exchanging code for token...");
// Exchange code for token
let client = reqwest::Client::new();
let mut token_params = vec![
("grant_type", "authorization_code".to_string()),
("code", code),
("redirect_uri", redirect_uri),
];
if let Some(ref verifier) = code_verifier {
token_params.push(("code_verifier", verifier.to_string()));
}
// Build token request
let mut request = client.post(&oauth.token_url);
// Use Basic auth if client_secret is provided, otherwise include client_id in body
if let Some(ref secret) = client_secret {
request = request.basic_auth(&client_id, Some(secret));
} else {
token_params.push(("client_id", client_id));
}
let token_response = request.form(&token_params).send().await?;
if !token_response.status().is_success() {
let status = token_response.status();
let body = token_response.text().await.unwrap_or_default();
return Err(anyhow::anyhow!(
"Token exchange failed: {} - {}",
status,
body
));
}
let token_data: serde_json::Value = token_response.json().await?;
let access_token = token_data
.get(&oauth.access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
anyhow::anyhow!(
"No {} in token response: {:?}",
oauth.access_token_field,
token_data
)
})?;
let refresh_token = token_data.get("refresh_token").and_then(|v| v.as_str());
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
// Save the token (with refresh token and expiry if provided)
save_token(
store,
user_id,
auth,
access_token,
refresh_token,
expires_in,
let token_response = oauth_defaults::exchange_oauth_code(
&oauth.token_url,
&client_id,
client_secret.as_deref(),
&code,
&redirect_uri,
code_verifier.as_deref(),
&oauth.access_token_field,
)
.await?;
// Extract any additional info for display
let workspace_name = token_data
.get("workspace_name")
.and_then(|v| v.as_str())
.or_else(|| token_data.get("team_name").and_then(|v| v.as_str()));
// Save tokens (access + refresh + scopes)
oauth_defaults::store_oauth_tokens(
store,
user_id,
&auth.secret_name,
auth.provider.as_deref(),
&token_response.access_token,
token_response.refresh_token.as_deref(),
token_response.expires_in,
&oauth.scopes,
)
.await?;
println!();
println!("{} connected!", display_name);
if let Some(workspace) = workspace_name {
println!(" Workspace: {}", workspace);
}
println!();
println!(" The tool can now access the API.");
println!();
@@ -1107,46 +1030,15 @@ async fn validate_token(
validation: &crate::tools::wasm::ValidationEndpointSchema,
_secret_name: &str,
) -> anyhow::Result<()> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()?;
// Build request based on method
let request = match validation.method.to_uppercase().as_str() {
"GET" => client.get(&validation.url),
"POST" => client.post(&validation.url),
_ => client.get(&validation.url),
};
// Add authorization header (assume Bearer for now, could be extended)
let response = request
.header("Authorization", format!("Bearer {}", token))
.header("Notion-Version", "2022-06-28") // Notion-specific, but harmless for others
.send()
.await?;
if response.status().as_u16() == validation.success_status {
Ok(())
} else {
let status = response.status();
let body = response.text().await.unwrap_or_default();
Err(anyhow::anyhow!(
"HTTP {} (expected {}): {}",
status,
validation.success_status,
if body.len() > 100 {
format!("{}...", &body[..100])
} else {
body
}
))
}
crate::cli::oauth_defaults::validate_oauth_token(token, validation)
.await
.map_err(|e| anyhow::anyhow!("{}", e))
}
/// Save token to secrets store.
///
/// Optionally stores a refresh token (as `{secret_name}_refresh_token`) and
/// sets `expires_at` on the access token so the runtime can auto-refresh.
/// Delegates to the shared `store_oauth_tokens` for OAuth tokens, or stores
/// directly for manual/env-var tokens (no scopes or refresh token).
async fn save_token(
store: &(dyn SecretsStore + Send + Sync),
user_id: &str,
@@ -1155,36 +1047,18 @@ async fn save_token(
refresh_token: Option<&str>,
expires_in: Option<u64>,
) -> anyhow::Result<()> {
let mut params = CreateSecretParams::new(&auth.secret_name, token);
if let Some(ref provider) = auth.provider {
params = params.with_provider(provider);
}
if let Some(secs) = expires_in {
let expires_at = chrono::Utc::now() + chrono::Duration::seconds(secs as i64);
params = params.with_expiry(expires_at);
}
store
.create(user_id, params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save token: {}", e))?;
// Store refresh token separately (no expiry, it's long-lived)
if let Some(rt) = refresh_token {
let refresh_name = format!("{}_refresh_token", auth.secret_name);
let mut refresh_params = CreateSecretParams::new(&refresh_name, rt);
if let Some(ref provider) = auth.provider {
refresh_params = refresh_params.with_provider(provider);
}
store
.create(user_id, refresh_params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save refresh token: {}", e))?;
}
Ok(())
crate::cli::oauth_defaults::store_oauth_tokens(
store,
user_id,
&auth.secret_name,
auth.provider.as_deref(),
token,
refresh_token,
expires_in,
&[], // No scopes for manual/env-var tokens
)
.await
.map_err(|e| anyhow::anyhow!("{}", e))
}
/// Print success message.
+18 -10
View File
@@ -1,3 +1,4 @@
use std::collections::HashMap;
use std::path::PathBuf;
use secrecy::SecretString;
@@ -18,8 +19,9 @@ pub struct ChannelsConfig {
pub wasm_channels_dir: std::path::PathBuf,
/// Whether WASM channels are enabled.
pub wasm_channels_enabled: bool,
/// Telegram owner user ID. When set, the bot only responds to this user.
pub telegram_owner_id: Option<i64>,
/// Per-channel owner user IDs. When set, the channel only responds to this user.
/// Key: channel name (e.g., "telegram"), Value: owner user ID.
pub wasm_channel_owner_ids: HashMap<String, i64>,
}
#[derive(Debug, Clone)]
@@ -180,14 +182,20 @@ impl ChannelsConfig {
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
wasm_channels_enabled: parse_bool_env("WASM_CHANNELS_ENABLED", true)?,
telegram_owner_id: optional_env("TELEGRAM_OWNER_ID")?
.map(|s| s.parse())
.transpose()
.map_err(|e: std::num::ParseIntError| ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
})?
.or(settings.channels.telegram_owner_id),
wasm_channel_owner_ids: {
let mut ids = settings.channels.wasm_channel_owner_ids.clone();
// Backwards compat: TELEGRAM_OWNER_ID env var
if let Some(id_str) = optional_env("TELEGRAM_OWNER_ID")? {
let id: i64 = id_str.parse().map_err(|e: std::num::ParseIntError| {
ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
}
})?;
ids.insert("telegram".to_string(), id);
}
ids
},
})
}
}
+24
View File
@@ -213,6 +213,30 @@ impl JobStore for LibSqlBackend {
Ok(jobs)
}
async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
"SELECT failure_reason FROM agent_jobs WHERE id = ?1",
[id.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
Ok(get_opt_text(&row, 0))
} else {
Ok(None)
}
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
+6 -4
View File
@@ -515,7 +515,7 @@ impl WorkspaceStore for LibSqlBackend {
let mut rows = conn
.query(
r#"
SELECT c.id, c.document_id, c.content
SELECT c.id, c.document_id, d.path, c.content
FROM memory_chunks_fts fts
JOIN memory_chunks c ON c._rowid = fts.rowid
JOIN memory_documents d ON d.id = c.document_id
@@ -542,7 +542,8 @@ impl WorkspaceStore for LibSqlBackend {
results.push(RankedResult {
chunk_id: get_text(&row, 0).parse().unwrap_or_default(),
document_id: get_text(&row, 1).parse().unwrap_or_default(),
content: get_text(&row, 2),
document_path: get_text(&row, 2),
content: get_text(&row, 3),
rank: results.len() as u32 + 1,
});
}
@@ -563,7 +564,7 @@ impl WorkspaceStore for LibSqlBackend {
let mut rows = conn
.query(
r#"
SELECT c.id, c.document_id, c.content
SELECT c.id, c.document_id, d.path, c.content
FROM vector_top_k('idx_memory_chunks_embedding', vector(?1), ?2) AS top_k
JOIN memory_chunks c ON c._rowid = top_k.id
JOIN memory_documents d ON d.id = c.document_id
@@ -587,7 +588,8 @@ impl WorkspaceStore for LibSqlBackend {
results.push(RankedResult {
chunk_id: get_text(&row, 0).parse().unwrap_or_default(),
document_id: get_text(&row, 1).parse().unwrap_or_default(),
content: get_text(&row, 2),
document_path: get_text(&row, 2),
content: get_text(&row, 3),
rank: results.len() as u32 + 1,
});
}
+3
View File
@@ -177,6 +177,9 @@ pub trait JobStore: Send + Sync {
async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>;
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError>;
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError>;
/// Get the failure reason for a single agent job (O(1) lookup).
async fn get_agent_job_failure_reason(&self, id: Uuid)
-> Result<Option<String>, DatabaseError>;
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>;
async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>;
async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>;
+7
View File
@@ -223,6 +223,13 @@ impl JobStore for PgBackend {
self.store.agent_job_summary().await
}
async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
self.store.get_agent_job_failure_reason(id).await
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
self.store.save_action(job_id, action).await
}
+3
View File
@@ -331,6 +331,9 @@ pub enum WorkspaceError {
#[error("Heartbeat error: {reason}")]
HeartbeatError { reason: String },
#[error("I/O error: {reason}")]
IoError { reason: String },
}
/// Orchestrator errors (internal API, container management).
+789 -265
View File
File diff suppressed because it is too large Load Diff
+382 -18
View File
@@ -24,6 +24,7 @@ pub use discovery::OnlineDiscovery;
pub use manager::ExtensionManager;
pub use registry::ExtensionRegistry;
use serde::ser::SerializeMap;
use serde::{Deserialize, Serialize};
/// The kind of extension, determining how it's installed, authenticated, and activated.
@@ -145,28 +146,267 @@ pub struct InstallResult {
pub message: String,
}
/// Auth readiness state for the extensions list UI.
///
/// Used by `check_tool_auth_status` and `check_channel_auth_status` to
/// communicate a tool's credential state to the list handler without
/// ambiguous `(bool, bool)` tuples.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolAuthState {
/// Token/credentials are present — ready to use.
Ready,
/// Auth section exists but the access token is missing (OAuth not completed).
NeedsAuth,
/// Setup credentials (client_id/secret) must be configured before OAuth can start.
NeedsSetup,
/// No auth configuration at all (no capabilities or auth section).
NoAuth,
}
/// The typed auth status, carrying only the data relevant to each state.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AuthStatus {
/// Authentication is complete; no further action needed.
Authenticated,
/// No authentication is required for this extension.
NoAuthRequired,
/// OAuth flow started — user must open `auth_url` in their browser.
AwaitingAuthorization {
auth_url: String,
callback_type: String,
},
/// Waiting for user to provide a token/key manually.
AwaitingToken {
instructions: String,
setup_url: Option<String>,
},
/// OAuth client credentials need to be configured before auth can proceed.
NeedsSetup {
instructions: String,
setup_url: Option<String>,
},
}
impl AuthStatus {
/// The wire-format status string (backward-compatible with JS consumers).
pub fn as_str(&self) -> &'static str {
match self {
AuthStatus::Authenticated => "authenticated",
AuthStatus::NoAuthRequired => "no_auth_required",
AuthStatus::AwaitingAuthorization { .. } => "awaiting_authorization",
AuthStatus::AwaitingToken { .. } => "awaiting_token",
AuthStatus::NeedsSetup { .. } => "needs_setup",
}
}
}
/// Result of authenticating an extension.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone)]
pub struct AuthResult {
pub name: String,
pub kind: ExtensionKind,
/// OAuth URL to open (for OAuth flows).
#[serde(skip_serializing_if = "Option::is_none")]
pub auth_url: Option<String>,
/// Whether using local or remote callback.
#[serde(skip_serializing_if = "Option::is_none")]
pub callback_type: Option<String>,
/// Instructions for manual token entry (for WASM tools).
#[serde(skip_serializing_if = "Option::is_none")]
pub instructions: Option<String>,
/// URL for manual token setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub setup_url: Option<String>,
/// Whether the tool is waiting for a token from the user.
#[serde(default)]
pub awaiting_token: bool,
/// Current auth status.
pub status: String,
pub status: AuthStatus,
}
impl AuthResult {
// ── Constructors ──────────────────────────────────────────────────
pub fn authenticated(name: impl Into<String>, kind: ExtensionKind) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::Authenticated,
}
}
pub fn no_auth_required(name: impl Into<String>, kind: ExtensionKind) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::NoAuthRequired,
}
}
pub fn awaiting_authorization(
name: impl Into<String>,
kind: ExtensionKind,
auth_url: String,
callback_type: String,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::AwaitingAuthorization {
auth_url,
callback_type,
},
}
}
pub fn awaiting_token(
name: impl Into<String>,
kind: ExtensionKind,
instructions: String,
setup_url: Option<String>,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::AwaitingToken {
instructions,
setup_url,
},
}
}
pub fn needs_setup(
name: impl Into<String>,
kind: ExtensionKind,
instructions: String,
setup_url: Option<String>,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::NeedsSetup {
instructions,
setup_url,
},
}
}
// ── Accessors ─────────────────────────────────────────────────────
pub fn is_authenticated(&self) -> bool {
matches!(self.status, AuthStatus::Authenticated)
}
pub fn auth_url(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingAuthorization { auth_url, .. } => Some(auth_url),
_ => None,
}
}
pub fn callback_type(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingAuthorization { callback_type, .. } => Some(callback_type),
_ => None,
}
}
pub fn instructions(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingToken { instructions, .. }
| AuthStatus::NeedsSetup { instructions, .. } => Some(instructions),
_ => None,
}
}
pub fn setup_url(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingToken { setup_url, .. }
| AuthStatus::NeedsSetup { setup_url, .. } => setup_url.as_deref(),
_ => None,
}
}
pub fn is_awaiting_token(&self) -> bool {
matches!(self.status, AuthStatus::AwaitingToken { .. })
}
pub fn status_str(&self) -> &'static str {
self.status.as_str()
}
}
/// Serialize `AuthResult` to the same flat JSON shape the JS frontend expects.
impl Serialize for AuthResult {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
// Count fields: name + kind + status + optional fields
let optional_count = self.auth_url().is_some() as usize
+ self.callback_type().is_some() as usize
+ self.instructions().is_some() as usize
+ self.setup_url().is_some() as usize;
let mut map = serializer.serialize_map(Some(4 + optional_count))?;
map.serialize_entry("name", &self.name)?;
map.serialize_entry("kind", &self.kind)?;
if let Some(url) = self.auth_url() {
map.serialize_entry("auth_url", url)?;
}
if let Some(cb) = self.callback_type() {
map.serialize_entry("callback_type", cb)?;
}
if let Some(inst) = self.instructions() {
map.serialize_entry("instructions", inst)?;
}
if let Some(url) = self.setup_url() {
map.serialize_entry("setup_url", url)?;
}
map.serialize_entry("awaiting_token", &self.is_awaiting_token())?;
map.serialize_entry("status", self.status_str())?;
map.end()
}
}
/// Deserialize from the flat JSON shape back into the typed enum.
impl<'de> Deserialize<'de> for AuthResult {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
/// Flat helper matching the old JSON shape.
#[derive(Deserialize)]
#[allow(dead_code)]
struct Raw {
name: String,
kind: ExtensionKind,
#[serde(default)]
auth_url: Option<String>,
#[serde(default)]
callback_type: Option<String>,
#[serde(default)]
instructions: Option<String>,
#[serde(default)]
setup_url: Option<String>,
#[serde(default)]
awaiting_token: bool,
status: String,
}
let raw = Raw::deserialize(deserializer)?;
let status = match raw.status.as_str() {
"authenticated" => AuthStatus::Authenticated,
"no_auth_required" => AuthStatus::NoAuthRequired,
"awaiting_authorization" => AuthStatus::AwaitingAuthorization {
auth_url: raw.auth_url.unwrap_or_default(),
callback_type: raw.callback_type.unwrap_or_default(),
},
"awaiting_token" => AuthStatus::AwaitingToken {
instructions: raw.instructions.unwrap_or_default(),
setup_url: raw.setup_url,
},
"needs_setup" => AuthStatus::NeedsSetup {
instructions: raw.instructions.unwrap_or_default(),
setup_url: raw.setup_url,
},
other => {
return Err(serde::de::Error::unknown_variant(
other,
&[
"authenticated",
"no_auth_required",
"awaiting_authorization",
"awaiting_token",
"needs_setup",
],
));
}
};
Ok(AuthResult {
name: raw.name,
kind: raw.kind,
status,
})
}
}
/// Result of activating an extension.
@@ -204,6 +444,9 @@ pub struct InstalledExtension {
/// Whether this extension has a setup schema (required_secrets) that can be configured.
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
#[serde(default)]
pub has_auth: bool,
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
@@ -254,3 +497,124 @@ pub enum ExtensionError {
#[error("{0}")]
Other(String),
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn auth_result_authenticated_round_trip() {
let result = AuthResult::authenticated("gmail", ExtensionKind::WasmTool);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "authenticated");
assert_eq!(json["name"], "gmail");
assert_eq!(json["kind"], "wasm_tool");
assert_eq!(json["awaiting_token"], false);
assert!(json.get("auth_url").is_none());
assert!(json.get("instructions").is_none());
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(back.is_authenticated());
assert!(back.auth_url().is_none());
}
#[test]
fn auth_result_awaiting_authorization_round_trip() {
let result = AuthResult::awaiting_authorization(
"google-drive",
ExtensionKind::WasmTool,
"https://accounts.google.com/o/oauth2/v2/auth?state=abc".to_string(),
"local".to_string(),
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "awaiting_authorization");
assert_eq!(
json["auth_url"],
"https://accounts.google.com/o/oauth2/v2/auth?state=abc"
);
assert_eq!(json["callback_type"], "local");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert_eq!(
back.auth_url(),
Some("https://accounts.google.com/o/oauth2/v2/auth?state=abc")
);
assert_eq!(back.callback_type(), Some("local"));
assert!(!back.is_authenticated());
}
#[test]
fn auth_result_awaiting_token_round_trip() {
let result = AuthResult::awaiting_token(
"telegram",
ExtensionKind::WasmChannel,
"Enter your bot token".to_string(),
None,
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "awaiting_token");
assert_eq!(json["instructions"], "Enter your bot token");
assert_eq!(json["awaiting_token"], true);
assert!(json.get("auth_url").is_none());
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(back.is_awaiting_token());
assert_eq!(back.instructions(), Some("Enter your bot token"));
}
#[test]
fn auth_result_needs_setup_round_trip() {
let result = AuthResult::needs_setup(
"custom-tool",
ExtensionKind::WasmTool,
"Configure OAuth credentials in the Setup tab.".to_string(),
Some("https://console.cloud.google.com".to_string()),
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "needs_setup");
assert_eq!(json["setup_url"], "https://console.cloud.google.com");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(!back.is_authenticated());
assert!(!back.is_awaiting_token());
assert_eq!(back.setup_url(), Some("https://console.cloud.google.com"));
}
#[test]
fn auth_result_no_auth_required_round_trip() {
let result = AuthResult::no_auth_required("echo", ExtensionKind::WasmTool);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "no_auth_required");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(!back.is_authenticated());
assert_eq!(back.status, AuthStatus::NoAuthRequired);
}
#[test]
fn auth_status_type_safety() {
// AwaitingAuthorization always has auth_url
let result = AuthResult::awaiting_authorization(
"test",
ExtensionKind::WasmTool,
"https://example.com".to_string(),
"local".to_string(),
);
assert!(result.auth_url().is_some());
assert!(!result.is_awaiting_token());
// Authenticated never has auth_url
let result = AuthResult::authenticated("test", ExtensionKind::WasmTool);
assert!(result.auth_url().is_none());
assert!(result.instructions().is_none());
assert!(result.setup_url().is_none());
}
}
+15
View File
@@ -821,6 +821,21 @@ impl Store {
.collect())
}
/// Get the failure reason for a single agent job.
pub async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
let conn = self.conn().await?;
let row = conn
.query_opt(
"SELECT failure_reason FROM agent_jobs WHERE id = $1",
&[&id],
)
.await?;
Ok(row.and_then(|r| r.get::<_, Option<String>>("failure_reason")))
}
/// Summary counts for agent (non-sandbox) jobs.
pub async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.conn().await?;
+24 -1
View File
@@ -199,6 +199,29 @@ impl NearAiChatProvider {
})?;
let status = response.status();
// Extract Retry-After header before consuming the response body.
// Supports both delay-seconds (RFC 7231 §7.1.3) and HTTP-date formats.
let retry_after_header = response
.headers()
.get("retry-after")
.and_then(|v| v.to_str().ok())
.and_then(|v| {
// Try delay-seconds first (most common from API providers)
if let Ok(secs) = v.trim().parse::<u64>() {
return Some(std::time::Duration::from_secs(secs));
}
// Try HTTP-date (e.g. "Mon, 02 Mar 2026 18:00:00 GMT")
if let Ok(dt) = chrono::DateTime::parse_from_rfc2822(v.trim()) {
let now = chrono::Utc::now();
let delta = dt.signed_duration_since(now);
// Use max(0) so past/present dates yield Duration::ZERO
// rather than None (which would cause an immediate retry).
return Some(std::time::Duration::from_secs(
delta.num_seconds().max(0) as u64
));
}
None
});
let response_text = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to read response body: {}", e),
@@ -230,7 +253,7 @@ impl NearAiChatProvider {
if status_code == 429 {
return Err(LlmError::RateLimited {
provider: "nearai_chat".to_string(),
retry_after: None,
retry_after: retry_after_header,
});
}
+1 -1
View File
@@ -347,7 +347,7 @@ impl SessionManager {
// The NEAR AI API redirects to: {frontend_callback}/auth/callback?token=X&...
let session_token =
oauth_defaults::wait_for_callback(listener, "/auth/callback", "token", "NEAR AI")
oauth_defaults::wait_for_callback(listener, "/auth/callback", "token", "NEAR AI", None)
.await
.map_err(|e| LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
+14 -20
View File
@@ -484,8 +484,6 @@ async fn async_main() -> anyhow::Result<()> {
let mut sse_sender: Option<
tokio::sync::broadcast::Sender<ironclaw::channels::web::types::SseEvent>,
> = None;
let mut gateway_state: Option<std::sync::Arc<ironclaw::channels::web::server::GatewayState>> =
None;
if let Some(ref gw_config) = config.channels.gateway {
let mut gw =
GatewayChannel::new(gw_config.clone()).with_llm_provider(Arc::clone(&components.llm));
@@ -508,6 +506,7 @@ async fn async_main() -> anyhow::Result<()> {
if let Some(ref jm) = container_job_manager {
gw = gw.with_job_manager(Arc::clone(jm));
}
gw = gw.with_scheduler(scheduler_slot.clone());
if let Some(ref sr) = components.skill_registry {
gw = gw.with_skill_registry(Arc::clone(sr));
}
@@ -542,7 +541,6 @@ async fn async_main() -> anyhow::Result<()> {
// IMPORTANT: This must come after all `with_*` calls since `rebuild_state`
// creates a new SseManager, which would orphan this sender.
sse_sender = Some(gw.state().sse.sender());
gateway_state = Some(Arc::clone(gw.state()));
channel_names.push("gateway".to_string());
channels.add(Box::new(gw)).await;
@@ -618,7 +616,7 @@ async fn async_main() -> anyhow::Result<()> {
rt,
ps,
router,
config.channels.telegram_owner_id,
config.channels.wasm_channel_owner_ids.clone(),
)
.await;
tracing::info!("Channel runtime wired into extension manager for hot-activation");
@@ -649,9 +647,9 @@ async fn async_main() -> anyhow::Result<()> {
// Wire SSE sender into extension manager for broadcasting status events.
if let Some(ref ext_mgr) = components.extension_manager
&& let Some(sender) = sse_sender
&& let Some(ref sender) = sse_sender
{
ext_mgr.set_sse_sender(sender).await;
ext_mgr.set_sse_sender(sender.clone()).await;
}
let deps = AgentDeps {
@@ -667,6 +665,7 @@ async fn async_main() -> anyhow::Result<()> {
skills_config: config.skills.clone(),
hooks: components.hooks,
cost_guard: components.cost_guard,
sse_tx: sse_sender,
};
let agent = Agent::new(
@@ -700,16 +699,6 @@ async fn async_main() -> anyhow::Result<()> {
tracing::info!("Agent shutdown complete");
// Check if a restart was requested via the gateway API.
if let Some(ref gw_state) = gateway_state
&& gw_state
.restart_requested
.load(std::sync::atomic::Ordering::Relaxed)
{
eprintln!("Restarting IronClaw (exit code 75)...");
std::process::exit(75);
}
Ok(())
}
@@ -911,11 +900,14 @@ async fn setup_wasm_channels(
let pairing_store = Arc::new(PairingStore::new());
let settings_store: Option<Arc<dyn ironclaw::db::SettingsStore>> =
database.map(|db| Arc::clone(db) as Arc<dyn ironclaw::db::SettingsStore>);
let loader = WasmChannelLoader::new(
let mut loader = WasmChannelLoader::new(
Arc::clone(&runtime),
Arc::clone(&pairing_store),
settings_store,
);
if let Some(secrets) = secrets_store {
loader = loader.with_secrets_store(Arc::clone(secrets));
}
let results = match loader
.load_from_dir(&config.channels.wasm_channels_dir)
@@ -979,9 +971,11 @@ async fn setup_wasm_channels(
);
}
// Inject owner_id for Telegram so the bot only responds to the bound user.
if channel_name == "telegram"
&& let Some(owner_id) = config.channels.telegram_owner_id
// Inject owner_id if configured for this channel.
if let Some(&owner_id) = config
.channels
.wasm_channel_owner_ids
.get(channel_name.as_str())
{
config_updates.insert("owner_id".to_string(), serde_json::json!(owner_id));
}
+15 -3
View File
@@ -30,6 +30,9 @@ pub enum PairingStoreError {
#[error("Invalid channel: {0}")]
InvalidChannel(String),
#[error("Invalid path: {0}")]
InvalidPath(String),
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
@@ -224,7 +227,10 @@ impl PairingStore {
meta: Option<serde_json::Value>,
) -> Result<UpsertResult, PairingStoreError> {
let path = pairing_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
let mut file = fs::OpenOptions::new()
.read(true)
@@ -319,7 +325,10 @@ impl PairingStore {
fn record_failed_approve(&self, channel: &str) -> Result<(), PairingStoreError> {
let path = approve_attempts_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
// Open (or create) and lock before reading so concurrent callers
// don't clobber each other's writes.
@@ -462,7 +471,10 @@ impl PairingStore {
}
let path = allow_from_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
let file = fs::OpenOptions::new()
.read(true)
+3
View File
@@ -47,6 +47,9 @@ pub enum RegistryError {
actual_sha256: String,
},
#[error("Missing SHA256 checksum for '{name}' artifact. Use --build to build from source.")]
MissingChecksum { name: String },
#[error(
"Source fallback unavailable for '{name}' after artifact install failed. Retry artifact download or run from a repository checkout."
)]
+17 -8
View File
@@ -20,6 +20,10 @@ const ALLOWED_ARTIFACT_HOSTS: &[&str] = &[
];
fn should_attempt_source_fallback(err: &RegistryError) -> bool {
// MissingChecksum is intentionally allowed here — it's a bootstrapping issue
// (no release has populated checksums yet), not a security concern. Source
// builds use local trusted code. ChecksumMismatch (tampered artifact) and
// InvalidManifest (structural problem) remain blocked.
!matches!(
err,
RegistryError::AlreadyInstalled { .. }
@@ -367,15 +371,15 @@ impl RegistryInstaller {
// Require SHA256 — refuse to install unverified binaries. Check before
// downloading to avoid wasting bandwidth on manifests that are missing
// checksums.
// checksums. Uses MissingChecksum (not InvalidManifest) so that
// install_with_source_fallback can fall back to building from source
// when checksums haven't been populated yet (bootstrapping).
let expected_sha =
artifact
.sha256
.as_ref()
.ok_or_else(|| RegistryError::InvalidManifest {
.ok_or_else(|| RegistryError::MissingChecksum {
name: manifest.name.clone(),
field: "artifacts.wasm32-wasip2.sha256",
reason: "sha256 is required for artifact downloads".to_string(),
})?;
let target_dir = match manifest.kind {
@@ -500,7 +504,7 @@ impl RegistryInstaller {
if prefer_build || !has_artifact {
self.install_from_source(manifest, force).await
} else {
self.install_from_artifact(manifest, force).await
self.install_with_source_fallback(manifest, force).await
}
}
@@ -905,9 +909,8 @@ mod tests {
let result = installer.install_from_artifact(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, reason, .. }) => {
assert_eq!(field, "artifacts.wasm32-wasip2.sha256");
assert!(reason.contains("required"), "reason: {}", reason);
Err(RegistryError::MissingChecksum { name }) => {
assert_eq!(name, "demo");
}
other => panic!("unexpected result: {:?}", other),
}
@@ -942,6 +945,12 @@ mod tests {
reason: "host not allowed".to_string(),
};
assert!(!should_attempt_source_fallback(&invalid));
// MissingChecksum SHOULD allow source fallback (bootstrapping)
let missing = RegistryError::MissingChecksum {
name: "demo".to_string(),
};
assert!(should_attempt_source_fallback(&missing));
}
#[test]
+51 -28
View File
@@ -63,6 +63,29 @@ pub struct ContainerRunner {
proxy_port: u16,
}
/// Append `text` into `buffer` up to `limit` bytes without breaking UTF-8.
///
/// Returns `true` when truncation occurred.
fn append_with_limit(buffer: &mut String, text: &str, limit: usize) -> bool {
if text.is_empty() {
return false;
}
if buffer.len() >= limit {
return true;
}
let remaining = limit - buffer.len();
if text.len() <= remaining {
buffer.push_str(text);
return false;
}
let end = crate::util::floor_char_boundary(text, remaining);
buffer.push_str(&text[..end]);
true
}
impl ContainerRunner {
/// Create a new container runner.
pub fn new(docker: Docker, image: String, proxy_port: u16) -> Self {
@@ -393,23 +416,11 @@ impl ContainerRunner {
match result {
Ok(LogOutput::StdOut { message }) => {
let text = String::from_utf8_lossy(&message);
if stdout.len() + text.len() > half_max {
truncated = true;
let remaining = half_max.saturating_sub(stdout.len());
stdout.push_str(&text[..remaining.min(text.len())]);
} else {
stdout.push_str(&text);
}
truncated |= append_with_limit(&mut stdout, &text, half_max);
}
Ok(LogOutput::StdErr { message }) => {
let text = String::from_utf8_lossy(&message);
if stderr.len() + text.len() > half_max {
truncated = true;
let remaining = half_max.saturating_sub(stderr.len());
stderr.push_str(&text[..remaining.min(text.len())]);
} else {
stderr.push_str(&text);
}
truncated |= append_with_limit(&mut stderr, &text, half_max);
}
Ok(_) => {}
Err(e) => {
@@ -439,23 +450,11 @@ impl ContainerRunner {
match result {
Ok(LogOutput::StdOut { message }) => {
let text = String::from_utf8_lossy(&message);
if stdout.len() < half_max {
let remaining = half_max.saturating_sub(stdout.len());
stdout.push_str(&text[..remaining.min(text.len())]);
if text.len() > remaining {
truncated = true;
}
}
truncated |= append_with_limit(&mut stdout, &text, half_max);
}
Ok(LogOutput::StdErr { message }) => {
let text = String::from_utf8_lossy(&message);
if stderr.len() < half_max {
let remaining = half_max.saturating_sub(stderr.len());
stderr.push_str(&text[..remaining.min(text.len())]);
if text.len() > remaining {
truncated = true;
}
}
truncated |= append_with_limit(&mut stderr, &text, half_max);
}
Ok(_) => {}
Err(e) => {
@@ -577,6 +576,30 @@ fn unix_socket_candidates_from_env(
mod tests {
use super::*;
#[test]
fn append_with_limit_truncates_on_utf8_boundary() {
let mut out = String::new();
let truncated = append_with_limit(&mut out, "ab🙂cd", 5);
assert!(truncated);
assert_eq!(out, "ab");
}
#[test]
fn append_with_limit_marks_truncated_when_full() {
let mut out = "abc".to_string();
let truncated = append_with_limit(&mut out, "z", 3);
assert!(truncated);
assert_eq!(out, "abc");
}
#[test]
fn append_with_limit_appends_without_truncation() {
let mut out = String::new();
let truncated = append_with_limit(&mut out, "hello", 10);
assert!(!truncated);
assert_eq!(out, "hello");
}
#[cfg(unix)]
#[test]
fn test_unix_socket_candidates_include_rootless_paths() {
+28 -15
View File
@@ -249,10 +249,10 @@ pub struct ChannelSettings {
#[serde(default)]
pub signal_group_allow_from: Option<String>,
/// Telegram owner user ID. When set, the bot only responds to this user.
/// Captured during setup by having the user message the bot.
/// Per-channel owner user IDs. When set, the channel only responds to this user.
/// Key: channel name (e.g., "telegram"), Value: owner user ID.
#[serde(default)]
pub telegram_owner_id: Option<i64>,
pub wasm_channel_owner_ids: std::collections::HashMap<String, i64>,
/// Enabled WASM channels by name.
/// Channels not in this list but present in the channels directory will still load.
@@ -1049,28 +1049,37 @@ mod tests {
}
#[test]
fn test_telegram_owner_id_db_round_trip() {
fn test_wasm_channel_owner_ids_db_round_trip() {
let mut settings = Settings::default();
settings.channels.telegram_owner_id = Some(123456789);
settings
.channels
.wasm_channel_owner_ids
.insert("telegram".to_string(), 123456789);
let map = settings.to_db_map();
let restored = Settings::from_db_map(&map);
assert_eq!(restored.channels.telegram_owner_id, Some(123456789));
assert_eq!(
restored.channels.wasm_channel_owner_ids.get("telegram"),
Some(&123456789)
);
}
#[test]
fn test_telegram_owner_id_default_none() {
fn test_wasm_channel_owner_ids_default_empty() {
let settings = Settings::default();
assert_eq!(settings.channels.telegram_owner_id, None);
assert!(settings.channels.wasm_channel_owner_ids.is_empty());
}
#[test]
fn test_telegram_owner_id_via_set() {
fn test_wasm_channel_owner_ids_via_set() {
let mut settings = Settings::default();
settings
.set("channels.telegram_owner_id", "987654321")
.set("channels.wasm_channel_owner_ids.telegram", "987654321")
.unwrap();
assert_eq!(settings.channels.telegram_owner_id, Some(987654321));
assert_eq!(
settings.channels.wasm_channel_owner_ids.get("telegram"),
Some(&987654321)
);
}
#[test]
@@ -1406,7 +1415,11 @@ mod tests {
channels: ChannelSettings {
http_enabled: true,
http_port: Some(9090),
telegram_owner_id: Some(12345),
wasm_channel_owner_ids: {
let mut m = std::collections::HashMap::new();
m.insert("telegram".to_string(), 12345);
m
},
..Default::default()
},
heartbeat: HeartbeatSettings {
@@ -1473,9 +1486,9 @@ mod tests {
assert!(restored.channels.http_enabled, "http_enabled lost");
assert_eq!(restored.channels.http_port, Some(9090), "http_port lost");
assert_eq!(
restored.channels.telegram_owner_id,
Some(12345),
"telegram_owner_id lost"
restored.channels.wasm_channel_owner_ids.get("telegram"),
Some(&12345),
"wasm_channel_owner_ids lost"
);
assert!(restored.heartbeat.enabled, "heartbeat.enabled lost");
assert_eq!(
+2 -337
View File
@@ -1,6 +1,6 @@
//! Channel-specific setup flows.
//! Channel setup flows.
//!
//! Each channel (Telegram, HTTP, etc.) has its own setup function that:
//! Each channel (HTTP, Signal, WASM, etc.) has its own setup function that:
//! 1. Displays setup instructions
//! 2. Collects configuration (tokens, ports, etc.)
//! 3. Validates the configuration
@@ -9,9 +9,7 @@
use std::sync::Arc;
use base64::Engine;
use reqwest::Client;
use secrecy::{ExposeSecret, SecretString};
use serde::Deserialize;
use url::Url;
use uuid::Uuid;
@@ -105,261 +103,6 @@ impl SecretsContext {
}
}
/// Result of Telegram setup.
#[derive(Debug, Clone)]
pub struct TelegramSetupResult {
pub enabled: bool,
pub bot_username: Option<String>,
pub webhook_secret: Option<String>,
pub owner_id: Option<i64>,
}
/// Telegram Bot API response for getMe.
#[derive(Debug, Deserialize)]
struct TelegramGetMeResponse {
ok: bool,
result: Option<TelegramUser>,
}
#[derive(Debug, Deserialize)]
struct TelegramUser {
username: Option<String>,
#[allow(dead_code)]
first_name: String,
}
/// Telegram Bot API response for getUpdates.
#[derive(Debug, Deserialize)]
struct TelegramGetUpdatesResponse {
ok: bool,
result: Vec<TelegramUpdate>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdate {
update_id: i64,
message: Option<TelegramUpdateMessage>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdateMessage {
from: Option<TelegramUpdateUser>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdateUser {
id: i64,
first_name: String,
username: Option<String>,
}
/// Set up Telegram bot channel.
///
/// Guides the user through:
/// 1. Creating a bot with @BotFather
/// 2. Entering the bot token
/// 3. Validating the token
/// 4. Saving the token to the database
pub async fn setup_telegram(
secrets: &SecretsContext,
settings: &Settings,
) -> Result<TelegramSetupResult, ChannelSetupError> {
println!("Telegram Setup:");
println!();
print_info("To create a Telegram bot:");
print_info("1. Open Telegram and message @BotFather");
print_info("2. Send /newbot and follow the prompts");
print_info("3. Copy the bot token (looks like 123456:ABC-DEF...)");
println!();
// Check if token already exists
if secrets.secret_exists("telegram_bot_token").await {
print_info("Existing Telegram token found in database.");
if !confirm("Replace existing token?", false)? {
// Still offer to configure webhook secret and owner binding
let webhook_secret = setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
let owner_id = bind_telegram_owner_flow(secrets, settings).await?;
return Ok(TelegramSetupResult {
enabled: true,
bot_username: None,
webhook_secret,
owner_id,
});
}
}
loop {
let token = secret_input("Bot token (from @BotFather)")?;
// Validate the token
print_info("Validating bot token...");
match validate_telegram_token(&token).await {
Ok(username) => {
print_success(&format!(
"Bot validated: @{}",
username.as_deref().unwrap_or("unknown")
));
// Save to database
secrets.save_secret("telegram_bot_token", &token).await?;
print_success("Token saved to database");
// Bind bot to owner's Telegram account
let owner_id = bind_telegram_owner(&token).await?;
// Offer webhook secret configuration
let webhook_secret =
setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
return Ok(TelegramSetupResult {
enabled: true,
bot_username: username,
webhook_secret,
owner_id,
});
}
Err(e) => {
print_error(&format!("Token validation failed: {}", e));
if !confirm("Try again?", true)? {
return Ok(TelegramSetupResult {
enabled: false,
bot_username: None,
webhook_secret: None,
owner_id: None,
});
}
}
}
}
}
/// Bind the bot to the owner's Telegram account by having them send a message.
///
/// Polls `getUpdates` until a message arrives, then captures the sender's user ID.
/// Returns `None` if the user declines or the flow times out.
async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, ChannelSetupError> {
println!();
print_info("Account Binding (recommended):");
print_info("Binding restricts the bot so only YOU can use it.");
print_info("Without this, anyone who finds your bot can send it messages.");
println!();
if !confirm("Bind bot to your Telegram account?", true)? {
print_info("Skipping account binding. Bot will accept messages from all users.");
return Ok(None);
}
print_info("Send any message (e.g. /start) to your bot in Telegram.");
print_info("Waiting for your message (up to 120 seconds)...");
let client = Client::builder()
.timeout(std::time::Duration::from_secs(35))
.build()
.map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
// Clear any existing webhook so getUpdates works
let delete_url = format!(
"https://api.telegram.org/bot{}/deleteWebhook",
token.expose_secret()
);
if let Err(e) = client.post(&delete_url).send().await {
tracing::warn!("Failed to delete webhook (getUpdates may not work): {e}");
}
let updates_url = format!(
"https://api.telegram.org/bot{}/getUpdates",
token.expose_secret()
);
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(120);
while std::time::Instant::now() < deadline {
let response = client
.get(&updates_url)
.query(&[("timeout", "30"), ("allowed_updates", "[\"message\"]")])
.send()
.await
.map_err(|e| ChannelSetupError::Network(format!("getUpdates request failed: {}", e)))?;
if !response.status().is_success() {
return Err(ChannelSetupError::Network(format!(
"getUpdates returned status {}",
response.status()
)));
}
let body: TelegramGetUpdatesResponse = response.json().await.map_err(|e| {
ChannelSetupError::Network(format!("Failed to parse getUpdates response: {}", e))
})?;
if !body.ok {
return Err(ChannelSetupError::Network(
"Telegram API returned error for getUpdates".to_string(),
));
}
// Find the first message with a sender
for update in &body.result {
if let Some(ref msg) = update.message
&& let Some(ref from) = msg.from
{
let display_name = from
.username
.as_ref()
.map(|u| format!("@{}", u))
.unwrap_or_else(|| from.first_name.clone());
print_success(&format!(
"Received message from {} (ID: {})",
display_name, from.id
));
// Acknowledge the update so it doesn't pile up
let ack_url = format!(
"https://api.telegram.org/bot{}/getUpdates",
token.expose_secret()
);
if let Err(e) = client
.get(&ack_url)
.query(&[("offset", &(update.update_id + 1).to_string())])
.send()
.await
{
tracing::warn!("Failed to acknowledge Telegram update: {e}");
}
return Ok(Some(from.id));
}
}
}
print_error("Timed out waiting for a message. You can re-run setup to try again.");
print_info("Bot will accept messages from all users until owner is bound.");
Ok(None)
}
/// Bind flow when the token already exists (reads from secrets store).
///
/// Retrieves the saved bot token and delegates to `bind_telegram_owner`.
async fn bind_telegram_owner_flow(
secrets: &SecretsContext,
settings: &Settings,
) -> Result<Option<i64>, ChannelSetupError> {
if settings.channels.telegram_owner_id.is_some() {
print_info("Bot is already bound to a Telegram account.");
if !confirm("Re-bind to a different account?", false)? {
return Ok(settings.channels.telegram_owner_id);
}
}
// We need the token to poll getUpdates
let token = secrets.get_secret("telegram_bot_token").await?;
bind_telegram_owner(&token).await
}
/// Set up a tunnel for exposing the agent to the internet.
///
/// This is shared across all channels that need webhook endpoints.
@@ -725,84 +468,6 @@ fn setup_tunnel_static() -> Result<TunnelSettings, ChannelSetupError> {
})
}
/// Set up Telegram webhook secret for signature validation.
///
/// Returns the webhook secret if configured.
async fn setup_telegram_webhook_secret(
secrets: &SecretsContext,
tunnel: &TunnelSettings,
) -> Result<Option<String>, ChannelSetupError> {
if tunnel.public_url.is_none() {
print_info("");
print_info("No tunnel configured. Telegram will use polling mode (30s+ delay).");
print_info("Run setup again to configure a tunnel for instant delivery.");
return Ok(None);
}
println!();
print_info("Telegram Webhook Security:");
print_info("A webhook secret adds an extra layer of security by validating");
print_info("that requests actually come from Telegram's servers.");
if !confirm("Generate a webhook secret?", true)? {
return Ok(None);
}
let secret = generate_webhook_secret();
secrets
.save_secret(
"telegram_webhook_secret",
&SecretString::from(secret.clone()),
)
.await?;
print_success("Webhook secret generated and saved");
Ok(Some(secret))
}
/// Validate a Telegram bot token by calling the getMe API.
///
/// Returns the bot's username if valid.
pub async fn validate_telegram_token(
token: &SecretString,
) -> Result<Option<String>, ChannelSetupError> {
let client = Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
let url = format!(
"https://api.telegram.org/bot{}/getMe",
token.expose_secret()
);
let response = client
.get(&url)
.send()
.await
.map_err(|e| ChannelSetupError::Network(format!("Request failed: {}", e)))?;
if !response.status().is_success() {
return Err(ChannelSetupError::Network(format!(
"API returned status {}",
response.status()
)));
}
let body: TelegramGetMeResponse = response
.json()
.await
.map_err(|e| ChannelSetupError::Network(format!("Failed to parse response: {}", e)))?;
if body.ok {
Ok(body.result.and_then(|u| u.username))
} else {
Err(ChannelSetupError::Network(
"Telegram API returned error".to_string(),
))
}
}
/// Result of HTTP webhook setup.
#[derive(Debug, Clone)]
pub struct HttpSetupResult {
+1 -4
View File
@@ -24,10 +24,7 @@ mod prompts;
#[cfg(any(feature = "postgres", feature = "libsql"))]
mod wizard;
pub use channels::{
ChannelSetupError, SecretsContext, setup_http, setup_telegram, setup_tunnel,
validate_telegram_token,
};
pub use channels::{ChannelSetupError, SecretsContext, setup_http, setup_tunnel};
pub use prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
print_success, secret_input, select_many, select_one,
+1 -10
View File
@@ -26,7 +26,7 @@ use crate::llm::{SessionConfig, SessionManager};
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::settings::{KeySource, Settings};
use crate::setup::channels::{
SecretsContext, setup_http, setup_signal, setup_telegram, setup_tunnel, setup_wasm_channel,
SecretsContext, setup_http, setup_signal, setup_tunnel, setup_wasm_channel,
};
use crate::setup::prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
@@ -1670,15 +1670,6 @@ impl SetupWizard {
let result = if let Some(cap_file) = discovered_by_name.get(&channel_name) {
if !cap_file.setup.required_secrets.is_empty() {
setup_wasm_channel(ctx, &channel_name, &cap_file.setup).await?
} else if channel_name == "telegram" {
let telegram_result = setup_telegram(ctx, &self.settings).await?;
if let Some(owner_id) = telegram_result.owner_id {
self.settings.channels.telegram_owner_id = Some(owner_id);
}
crate::setup::channels::WasmChannelSetupResult {
enabled: telegram_result.enabled,
channel_name: "telegram".to_string(),
}
} else {
print_info(&format!(
"No setup configuration found for {}",
+1
View File
@@ -293,6 +293,7 @@ impl TestHarnessBuilder {
skills_config: SkillsConfig::default(),
hooks,
cost_guard,
sse_tx: None,
};
TestHarness {
+2 -2
View File
@@ -218,7 +218,7 @@ impl Tool for ToolAuthTool {
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
// Auto-activate after successful auth so tools are available immediately
if result.status == "authenticated" {
if result.is_authenticated() {
match self.manager.activate(name).await {
Ok(activate_result) => {
let output = serde_json::json!({
@@ -324,7 +324,7 @@ impl Tool for ToolActivateTool {
// Activation failed due to missing auth; initiate auth flow
// so the agent loop can show the auth card.
match self.manager.auth(name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded (e.g. env var was set); retry activation.
let result = self
.manager
+31 -9
View File
@@ -95,15 +95,17 @@ impl Tool for MemorySearchTool {
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Search failed: {}", e)))?;
let result_count = results.len();
let output = serde_json::json!({
"query": query,
"results": results.iter().map(|r| serde_json::json!({
"results": results.into_iter().map(|r| serde_json::json!({
"content": r.content,
"score": r.score,
"path": r.document_path,
"document_id": r.document_id.to_string(),
"is_hybrid_match": r.is_hybrid(),
})).collect::<Vec<_>>(),
"result_count": results.len(),
"result_count": result_count,
});
Ok(ToolOutput::success(output, start.elapsed()))
@@ -140,7 +142,8 @@ impl Tool for MemoryWriteTool {
Use for important facts, decisions, preferences, or lessons learned that should \
be remembered across sessions. Targets: 'memory' for curated long-term facts, \
'daily_log' for timestamped session notes, 'heartbeat' for the periodic \
checklist (HEARTBEAT.md), or provide a custom path for arbitrary file creation."
checklist (HEARTBEAT.md), 'bootstrap' to clear the first-run ritual file, \
or provide a custom path for arbitrary file creation."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -153,7 +156,7 @@ impl Tool for MemoryWriteTool {
},
"target": {
"type": "string",
"description": "Where to write: 'memory' for MEMORY.md, 'daily_log' for today's log, 'heartbeat' for HEARTBEAT.md checklist, or a path like 'projects/alpha/notes.md'",
"description": "Where to write: 'memory' for MEMORY.md, 'daily_log' for today's log, 'heartbeat' for HEARTBEAT.md checklist, 'bootstrap' to clear BOOTSTRAP.md (content is ignored; the file is always cleared), or a path like 'projects/alpha/notes.md'",
"default": "daily_log"
},
"append": {
@@ -175,17 +178,36 @@ impl Tool for MemoryWriteTool {
let content = require_str(&params, "content")?;
let target = params
.get("target")
.and_then(|v| v.as_str())
.unwrap_or("daily_log");
// Bootstrap target: clear BOOTSTRAP.md to mark first-run ritual complete.
// Handled early because it accepts empty content (unlike other targets).
if target == "bootstrap" {
// Write empty content to effectively disable the bootstrap injection.
// system_prompt_for_context() skips empty files.
self.workspace
.write(paths::BOOTSTRAP, "")
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Write failed: {}", e)))?;
let output = serde_json::json!({
"status": "cleared",
"path": paths::BOOTSTRAP,
"message": "BOOTSTRAP.md cleared. First-run ritual will not repeat.",
});
return Ok(ToolOutput::success(output, start.elapsed()));
}
if content.trim().is_empty() {
return Err(ToolError::InvalidParameters(
"content cannot be empty".to_string(),
));
}
let target = params
.get("target")
.and_then(|v| v.as_str())
.unwrap_or("daily_log");
// Reject writes to identity files that are loaded into the system prompt.
// An attacker could use prompt injection to trick the agent into overwriting
// these, poisoning future conversations.
+72 -17
View File
@@ -3,10 +3,9 @@
//! Allows the agent to proactively message users on any connected channel.
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::{Arc, RwLock};
use async_trait::async_trait;
use tokio::sync::RwLock;
use crate::bootstrap::ironclaw_base_dir;
use crate::channels::{ChannelManager, OutgoingResponse};
@@ -19,6 +18,7 @@ use crate::tools::tool::{
pub struct MessageTool {
channel_manager: Arc<ChannelManager>,
/// Default channel for current conversation (set per-turn).
/// Uses std::sync::RwLock because requires_approval() is sync and called from async context.
default_channel: Arc<RwLock<Option<String>>>,
/// Default target (user_id or group_id) for current conversation (set per-turn).
default_target: Arc<RwLock<Option<String>>>,
@@ -48,8 +48,14 @@ impl MessageTool {
/// Set the default channel and target for the current conversation turn.
/// Call this before each agent turn with the incoming message's channel/target.
pub async fn set_context(&self, channel: Option<String>, target: Option<String>) {
*self.default_channel.write().await = channel;
*self.default_target.write().await = target;
*self
.default_channel
.write()
.unwrap_or_else(|e| e.into_inner()) = channel;
*self
.default_target
.write()
.unwrap_or_else(|e| e.into_inner()) = target;
}
}
@@ -106,24 +112,32 @@ impl Tool for MessageTool {
let channel = if let Some(c) = params.get("channel").and_then(|v| v.as_str()) {
c.to_string()
} else {
self.default_channel.read().await.clone().ok_or_else(|| {
ToolError::ExecutionFailed(
"No channel specified and no active conversation. Provide channel parameter."
.to_string(),
)
})?
self.default_channel
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
.ok_or_else(|| {
ToolError::ExecutionFailed(
"No channel specified and no active conversation. Provide channel parameter."
.to_string(),
)
})?
};
// Get target: use param or fall back to default
let target = if let Some(t) = params.get("target").and_then(|v| v.as_str()) {
t.to_string()
} else {
self.default_target.read().await.clone().ok_or_else(|| {
ToolError::ExecutionFailed(
"No target specified and no active conversation. Provide target parameter."
.to_string(),
)
})?
self.default_target
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
.ok_or_else(|| {
ToolError::ExecutionFailed(
"No target specified and no active conversation. Provide target parameter."
.to_string(),
)
})?
};
let attachments: Vec<String> = match params.get("attachments") {
@@ -199,7 +213,10 @@ impl Tool for MessageTool {
let param_channel = params.get("channel").and_then(|v| v.as_str());
if let Some(channel) = param_channel {
// Check if it differs from the default channel
let default_channel = self.default_channel.blocking_read();
let default_channel = self
.default_channel
.read()
.unwrap_or_else(|e| e.into_inner());
if let Some(default) = default_channel.as_ref()
&& channel != default
{
@@ -515,4 +532,42 @@ mod tests {
err
);
}
/// Regression test: requires_approval() is a sync method called from async context.
/// With tokio::sync::RwLock, this would panic with:
/// "Cannot block the current thread from within a runtime"
/// because blocking_read() cannot be called inside an async runtime.
/// With std::sync::RwLock, it works correctly since std locks are safe
/// for short-held locks in sync methods called from async contexts.
#[tokio::test]
async fn requires_approval_works_from_async_context() {
let tool = MessageTool::new(Arc::new(ChannelManager::new()));
// Set context asynchronously (simulating real usage pattern)
tool.set_context(Some("signal".to_string()), Some("+1234567890".to_string()))
.await;
// Call requires_approval (sync method) from async context.
// This is the critical test: with tokio::sync::RwLock::blocking_read(),
// this would panic. With std::sync::RwLock::read(), it works.
let approval = tool.requires_approval(&serde_json::json!({
"content": "hello",
"channel": "telegram"
}));
// Different channel from default -> Always
assert!(matches!(approval, ApprovalRequirement::Always));
// No channel specified (uses default) -> UnlessAutoApproved
let approval = tool.requires_approval(&serde_json::json!({
"content": "hello"
}));
assert!(matches!(approval, ApprovalRequirement::UnlessAutoApproved));
// Explicit channel (even if same as default) -> Always
let approval = tool.requires_approval(&serde_json::json!({
"content": "hello",
"channel": "signal"
}));
assert!(matches!(approval, ApprovalRequirement::Always));
}
}
+2
View File
@@ -10,6 +10,7 @@ mod memory;
mod message;
pub mod path_utils;
pub mod routine;
pub mod secrets_tools;
pub(crate) mod shell;
pub mod skill_tools;
mod time;
@@ -31,6 +32,7 @@ pub use message::MessageTool;
pub use routine::{
RoutineCreateTool, RoutineDeleteTool, RoutineHistoryTool, RoutineListTool, RoutineUpdateTool,
};
pub use secrets_tools::{SecretDeleteTool, SecretListTool};
pub use shell::ShellTool;
pub use skill_tools::{SkillInstallTool, SkillListTool, SkillRemoveTool, SkillSearchTool};
pub use time::TimeTool;
+222
View File
@@ -0,0 +1,222 @@
//! Agent-callable tools for inspecting user secrets.
//!
//! These tools allow the LLM to query and manage secrets on behalf of the
//! user. The zero-exposure model is preserved throughout:
//!
//! - `secret_list` returns only names and metadata (no values).
//! - `secret_delete` removes a secret by name.
//!
//! Storing secrets is handled via the extensions setup flow — the user types
//! values directly into the secure UI, which submits them to
//! `/api/extensions/{name}/setup`. Values never appear in the LLM conversation,
//! logs, or ActionRecords.
use std::sync::Arc;
use async_trait::async_trait;
use crate::context::JobContext;
use crate::secrets::SecretsStore;
use crate::tools::tool::{ApprovalRequirement, Tool, ToolError, ToolOutput, require_str};
// ── secret_list ──────────────────────────────────────────────────────────────
pub struct SecretListTool {
store: Arc<dyn SecretsStore + Send + Sync>,
}
impl SecretListTool {
pub fn new(store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
Self { store }
}
}
#[async_trait]
impl Tool for SecretListTool {
fn name(&self) -> &str {
"secret_list"
}
fn description(&self) -> &str {
"List all stored secrets by name. Never returns values — only names and \
optional provider metadata. Use this to check what credentials are available \
before attempting a task that requires them."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {}
})
}
async fn execute(
&self,
_params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let refs = self
.store
.list(&ctx.user_id)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
let secrets: Vec<serde_json::Value> = refs
.into_iter()
.map(|r| {
serde_json::json!({
"name": r.name,
"provider": r.provider,
})
})
.collect();
let count = secrets.len();
let output = serde_json::json!({
"secrets": secrets,
"count": count,
});
Ok(ToolOutput::success(output, start.elapsed()))
}
}
// ── secret_delete ─────────────────────────────────────────────────────────────
pub struct SecretDeleteTool {
store: Arc<dyn SecretsStore + Send + Sync>,
}
impl SecretDeleteTool {
pub fn new(store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
Self { store }
}
}
#[async_trait]
impl Tool for SecretDeleteTool {
fn name(&self) -> &str {
"secret_delete"
}
fn description(&self) -> &str {
"Permanently delete a stored secret by name. This cannot be undone."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the secret to delete."
}
},
"required": ["name"]
})
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = require_str(&params, "name")?;
let deleted = self
.store
.delete(&ctx.user_id, name)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
let output = if deleted {
serde_json::json!({
"status": "deleted",
"name": name,
})
} else {
serde_json::json!({
"status": "not_found",
"name": name,
"message": format!("No secret named '{}' found.", name),
})
};
Ok(ToolOutput::success(output, start.elapsed()))
}
fn requires_approval(&self, _params: &serde_json::Value) -> ApprovalRequirement {
ApprovalRequirement::UnlessAutoApproved
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use secrecy::SecretString;
use super::*;
use crate::context::JobContext;
use crate::secrets::{CreateSecretParams, InMemorySecretsStore, SecretsCrypto};
fn test_store() -> Arc<InMemorySecretsStore> {
let key = "0123456789abcdef0123456789abcdef";
let crypto = Arc::new(SecretsCrypto::new(SecretString::from(key.to_string())).unwrap());
Arc::new(InMemorySecretsStore::new(crypto))
}
fn test_ctx() -> JobContext {
JobContext::new("test", "test job")
}
#[tokio::test]
async fn test_secret_list() {
let store = test_store();
let list = SecretListTool::new(Arc::clone(&store) as Arc<dyn SecretsStore + Send + Sync>);
let ctx = test_ctx();
store
.create(
&ctx.user_id,
CreateSecretParams::new("openai_key", "sk-test"),
)
.await
.unwrap();
let list_result = list.execute(serde_json::json!({}), &ctx).await.unwrap();
assert_eq!(list_result.result["count"], 1);
assert_eq!(list_result.result["secrets"][0]["name"], "openai_key");
assert!(list_result.result["secrets"][0].get("value").is_none());
}
#[tokio::test]
async fn test_secret_delete() {
let store = test_store();
let delete =
SecretDeleteTool::new(Arc::clone(&store) as Arc<dyn SecretsStore + Send + Sync>);
let ctx = test_ctx();
store
.create(&ctx.user_id, CreateSecretParams::new("to_delete", "secret"))
.await
.unwrap();
let result = delete
.execute(serde_json::json!({"name": "to_delete"}), &ctx)
.await
.unwrap();
assert_eq!(result.result["status"], "deleted");
// Deleting again returns not_found
let result2 = delete
.execute(serde_json::json!({"name": "to_delete"}), &ctx)
.await
.unwrap();
assert_eq!(result2.result["status"], "not_found");
}
}
+4 -1
View File
@@ -531,7 +531,7 @@ pub async fn wait_for_authorization_callback(
listener: TcpListener,
server_name: &str,
) -> Result<String, AuthError> {
oauth_defaults::wait_for_callback(listener, "/callback", "code", server_name)
oauth_defaults::wait_for_callback(listener, "/callback", "code", server_name, None)
.await
.map_err(|e| match e {
oauth_defaults::OAuthCallbackError::Denied => AuthError::AuthorizationDenied,
@@ -539,6 +539,9 @@ pub async fn wait_for_authorization_callback(
oauth_defaults::OAuthCallbackError::PortInUse(_, msg) => {
AuthError::Http(format!("Port error: {}", msg))
}
oauth_defaults::OAuthCallbackError::StateMismatch { .. } => {
AuthError::Http("CSRF state mismatch in OAuth callback".to_string())
}
oauth_defaults::OAuthCallbackError::Io(msg) => AuthError::Http(msg),
})
}
+1 -1
View File
@@ -26,5 +26,5 @@ pub use rate_limiter::RateLimiter;
pub use registry::ToolRegistry;
pub use tool::{
ApprovalRequirement, Tool, ToolDomain, ToolError, ToolOutput, ToolRateLimitConfig,
validate_tool_schema,
redact_params, validate_tool_schema,
};
+15 -1
View File
@@ -346,6 +346,20 @@ impl ToolRegistry {
tracing::info!("Registered {} job management tools", job_tool_count);
}
/// Register secret management tools (list, delete).
///
/// These allow the LLM to persist API keys and tokens encrypted in the database.
/// Values are never returned to the LLM; only names and metadata are exposed.
pub fn register_secrets_tools(
&self,
store: Arc<dyn crate::secrets::SecretsStore + Send + Sync>,
) {
use crate::tools::builtin::{SecretDeleteTool, SecretListTool};
self.register_sync(Arc::new(SecretListTool::new(Arc::clone(&store))));
self.register_sync(Arc::new(SecretDeleteTool::new(store)));
tracing::info!("Registered 2 secret management tools (list, delete)");
}
/// Register extension management tools (search, install, auth, activate, list, remove).
///
/// These allow the LLM to manage MCP servers and WASM tools through conversation.
@@ -585,7 +599,7 @@ impl ToolRegistry {
limits: None,
description: Some(&tool_with_binary.tool.description),
schema: Some(tool_with_binary.tool.parameters_schema.clone()),
secrets_store: None,
secrets_store: self.secrets_store.clone(),
oauth_refresh: None,
})
.await
+75
View File
@@ -239,6 +239,23 @@ pub trait Tool: Send + Sync {
ToolDomain::Orchestrator
}
/// Parameter names whose values must be redacted before logging, hooks, and approvals.
///
/// The agent framework replaces these parameter values with `"[REDACTED]"` before:
/// - Writing to debug logs
/// - Storing in `ActionRecord` (in-memory job history)
/// - Recording in `TurnToolCall` (session state)
/// - Sending to `BeforeToolCall` hooks
/// - Displaying in the approval UI
///
/// **The `execute()` method still receives the original, unredacted parameters.**
/// Redaction only applies to the observability and audit paths, not execution.
///
/// Use this for tools that accept plaintext secrets as parameters (e.g. `secret_save`).
fn sensitive_params(&self) -> &[&str] {
&[]
}
/// Per-invocation rate limit for this tool.
///
/// Return `Some(config)` to throttle how often this tool can be called per user.
@@ -287,6 +304,33 @@ pub fn require_param<'a>(
.ok_or_else(|| ToolError::InvalidParameters(format!("missing '{}' parameter", name)))
}
/// Replace sensitive parameter values with `"[REDACTED]"`.
///
/// Returns a new JSON value with the specified keys replaced. Non-object params
/// and unknown keys are passed through unchanged. The original value is cloned
/// only if there are sensitive params to redact; otherwise it is cloned once
/// (cheap — callers own the result).
///
/// Used by the agent framework before logging, hook dispatch, approval display,
/// and `ActionRecord` storage so plaintext secrets never reach those paths.
pub fn redact_params(params: &serde_json::Value, sensitive: &[&str]) -> serde_json::Value {
if sensitive.is_empty() {
return params.clone();
}
let mut redacted = params.clone();
if let Some(obj) = redacted.as_object_mut() {
for key in sensitive {
if obj.contains_key(*key) {
obj.insert(
(*key).to_string(),
serde_json::Value::String("[REDACTED]".into()),
);
}
}
}
redacted
}
/// Lenient runtime validation of a tool's `parameters_schema()`.
///
/// Use this function at tool-registration time to catch structural mistakes
@@ -500,6 +544,37 @@ mod tests {
assert!(ApprovalRequirement::Always.is_required());
}
#[test]
fn test_redact_params_replaces_sensitive_key() {
let params = serde_json::json!({"name": "openai_key", "value": "sk-secret"});
let redacted = redact_params(&params, &["value"]);
assert_eq!(redacted["name"], "openai_key");
assert_eq!(redacted["value"], "[REDACTED]");
// Original unchanged
assert_eq!(params["value"], "sk-secret");
}
#[test]
fn test_redact_params_empty_sensitive_is_noop() {
let params = serde_json::json!({"name": "key", "value": "secret"});
let redacted = redact_params(&params, &[]);
assert_eq!(redacted, params);
}
#[test]
fn test_redact_params_missing_key_is_noop() {
let params = serde_json::json!({"name": "key"});
let redacted = redact_params(&params, &["value"]);
assert_eq!(redacted, params);
}
#[test]
fn test_redact_params_non_object_is_passthrough() {
let params = serde_json::json!("just a string");
let redacted = redact_params(&params, &["value"]);
assert_eq!(redacted, params);
}
#[test]
fn test_validate_schema_valid() {
let schema = serde_json::json!({
+112
View File
@@ -105,6 +105,59 @@ impl CapabilitiesFile {
self
}
/// Validate the capabilities file and emit warnings for common misconfigurations.
///
/// Called once at load time to catch issues early. Warnings are emitted via
/// `tracing::warn` so they show up in startup logs without blocking loading.
pub fn validate(&self, name: &str) {
const MIN_PROMPT_LENGTH: usize = 30;
// setup.required_secrets present but no auth section → auth card won't display
if let Some(setup) = &self.setup {
if !setup.required_secrets.is_empty() && self.auth.is_none() {
tracing::warn!(
tool = name,
"setup.required_secrets defined but no 'auth' section — \
chat-based auth card will not display for this tool"
);
}
// Check for short prompts
for secret in &setup.required_secrets {
if secret.prompt.len() < MIN_PROMPT_LENGTH {
tracing::warn!(
tool = name,
secret = secret.name,
prompt = secret.prompt,
"setup.required_secrets prompt is shorter than {} chars — \
consider a more descriptive prompt that tells the user where to find this value",
MIN_PROMPT_LENGTH
);
}
}
}
// Manual auth (no OAuth) checks
if let Some(auth) = &self.auth
&& auth.oauth.is_none()
{
if auth.setup_url.is_none() {
tracing::warn!(
tool = name,
"auth section has no OAuth and no setup_url — \
user has no link to obtain credentials"
);
}
if auth.instructions.is_none() {
tracing::warn!(
tool = name,
"auth section has no OAuth and no instructions — \
user has no guidance on how to obtain credentials"
);
}
}
}
/// Convert to runtime Capabilities.
pub fn to_capabilities(&self) -> Capabilities {
let mut caps = Capabilities::default();
@@ -512,6 +565,11 @@ pub struct ValidationEndpointSchema {
/// Expected HTTP status code for success (defaults to 200).
#[serde(default = "default_success_status")]
pub success_status: u16,
/// Additional headers to send with the validation request.
/// Used for service-specific requirements (e.g., Notion-Version for Notion API).
#[serde(default)]
pub headers: HashMap<String, String>,
}
fn default_method() -> String {
@@ -1051,6 +1109,60 @@ mod tests {
assert_eq!(caps.setup.unwrap().required_secrets[0].name, "my_secret");
}
#[test]
fn test_validate_setup_without_auth_warns() {
// setup.required_secrets with no auth section — should not panic
let json = r#"{
"setup": {
"required_secrets": [
{ "name": "api_key", "prompt": "Enter your API key from the provider dashboard settings page" }
]
}
}"#;
let caps = CapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning is emitted via tracing
caps.validate("test-tool");
}
#[test]
fn test_validate_manual_auth_missing_fields() {
// auth without OAuth, missing setup_url and instructions
let json = r#"{
"auth": {
"secret_name": "my_api_key"
}
}"#;
let caps = CapabilitiesFile::from_json(json).unwrap();
// Should not panic; warnings emitted for missing setup_url and instructions
caps.validate("test-tool");
}
#[test]
fn test_validate_clean_tool() {
// Well-configured tool with auth, setup_url, instructions, and good prompts
let json = r#"{
"auth": {
"secret_name": "my_api_key",
"setup_url": "https://example.com/api-keys",
"instructions": "Go to example.com/api-keys and create a new key"
},
"setup": {
"required_secrets": [
{
"name": "my_api_key",
"prompt": "Enter your API key from https://example.com/api-keys"
}
]
}
}"#;
let caps = CapabilitiesFile::from_json(json).unwrap();
// Should not panic and emits no warnings (has auth, setup_url, instructions, long prompt)
caps.validate("clean-tool");
}
#[test]
fn test_resolve_nested_empty_capabilities_noop() {
// Empty inner capabilities should not clobber outer http
+1
View File
@@ -126,6 +126,7 @@ impl WasmToolLoader {
let cap_bytes = fs::read(cap_path).await?;
let cap_file = CapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| WasmLoadError::InvalidCapabilities(e.to_string()))?;
cap_file.validate(name);
let caps = cap_file.to_capabilities();
let oauth = resolve_oauth_refresh_config(&cap_file);
(caps, oauth)
+146 -1
View File
@@ -592,6 +592,10 @@ impl WasmToolWrapper {
let instance = SandboxedTool::instantiate(&mut store, &component, &linker)
.map_err(|e| WasmError::InstantiationFailed(e.to_string()))?;
// Coerce string-encoded values to their schema-declared types.
// LLMs frequently pass numeric values as strings (e.g. "5" instead of 5).
let params = coerce_params_to_schema(params, &self.schema);
// Prepare the request
let params_json = serde_json::to_string(&params)
.map_err(|e| WasmError::InvalidResponseJson(e.to_string()))?;
@@ -652,10 +656,17 @@ impl Tool for WasmToolWrapper {
// Pre-resolve host credentials from secrets store (async, before blocking task).
// This decrypts the secrets once so the sync http_request() host function
// can inject them without needing async access.
//
// BUG FIX: ExtensionManager stores OAuth tokens under user_id "default"
// (hardcoded at construction in app.rs), but this was previously looking
// them up under ctx.user_id — which could be a Telegram user ID, web
// gateway user, etc. — causing credential resolution to silently fail.
// Must match the storage key until per-user credential isolation is added.
let credential_user_id = "default";
let host_credentials = resolve_host_credentials(
&self.capabilities,
self.secrets_store.as_deref(),
&ctx.user_id,
credential_user_id,
self.oauth_refresh.as_ref(),
)
.await;
@@ -1083,6 +1094,61 @@ fn is_private_ip(ip: std::net::IpAddr) -> bool {
}
}
/// Coerce parameter values to match their JSON Schema-declared types.
///
/// LLMs frequently send numeric values as strings (e.g. `"5"` instead of `5`)
/// or booleans as strings (`"true"` instead of `true`). This walks the params
/// object and converts string values where the schema expects a different type.
fn coerce_params_to_schema(
mut params: serde_json::Value,
schema: &serde_json::Value,
) -> serde_json::Value {
let properties = schema.get("properties").and_then(|p| p.as_object());
let properties = match properties {
Some(p) => p,
None => return params,
};
let obj = match params.as_object_mut() {
Some(o) => o,
None => return params,
};
for (key, prop_schema) in properties {
let declared_type = prop_schema.get("type").and_then(|t| t.as_str());
let declared_type = match declared_type {
Some(t) => t,
None => continue,
};
if let Some(current_value) = obj.get_mut(key)
&& let Some(s) = current_value.as_str()
{
if declared_type == "string" {
continue;
}
let coerced = match declared_type {
"number" => s.parse::<f64>().ok().map(serde_json::Value::from),
"integer" => s.parse::<i64>().ok().map(serde_json::Value::from),
"boolean" => match s.to_lowercase().as_str() {
"true" => Some(serde_json::json!(true)),
"false" => Some(serde_json::json!(false)),
_ => None,
},
_ => None,
};
if let Some(new_val) = coerced {
*current_value = new_val;
}
}
}
params
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -1588,4 +1654,83 @@ mod tests {
let result = super::reject_private_ip("https://8.8.8.8/dns-query");
assert!(result.is_ok());
}
#[test]
fn test_coerce_params_string_to_number() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"count": { "type": "number" },
"name": { "type": "string" }
}
});
let params = serde_json::json!({"count": "5", "name": "test"});
let result = super::coerce_params_to_schema(params, &schema);
assert_eq!(result["count"], serde_json::json!(5.0));
assert_eq!(result["name"], serde_json::json!("test"));
}
#[test]
fn test_coerce_params_string_to_integer() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"limit": { "type": "integer" }
}
});
let params = serde_json::json!({"limit": "10"});
let result = super::coerce_params_to_schema(params, &schema);
assert_eq!(result["limit"], serde_json::json!(10));
}
#[test]
fn test_coerce_params_string_to_boolean() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"a": { "type": "boolean" },
"b": { "type": "boolean" },
"c": { "type": "boolean" },
"d": { "type": "boolean" }
}
});
let params = serde_json::json!({
"a": "true",
"b": "false",
"c": "True",
"d": "FALSE"
});
let result = super::coerce_params_to_schema(params, &schema);
assert_eq!(result["a"], serde_json::json!(true));
assert_eq!(result["b"], serde_json::json!(false));
assert_eq!(result["c"], serde_json::json!(true));
assert_eq!(result["d"], serde_json::json!(false));
}
#[test]
fn test_coerce_params_already_correct_type() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"count": { "type": "number" }
}
});
let params = serde_json::json!({"count": 5});
let result = super::coerce_params_to_schema(params, &schema);
assert_eq!(result["count"], serde_json::json!(5));
}
#[test]
fn test_coerce_params_invalid_string_not_coerced() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"count": { "type": "number" }
}
});
let params = serde_json::json!({"count": "not-a-number"});
let result = super::coerce_params_to_schema(params, &schema);
// Should remain as string since it can't be parsed
assert_eq!(result["count"], serde_json::json!("not-a-number"));
}
}
+2
View File
@@ -20,6 +20,8 @@ workspace/
├── SOUL.md <- Core values
├── AGENTS.md <- Behavior instructions
├── USER.md <- User context
├── TOOLS.md <- Environment-specific tool notes
├── BOOTSTRAP.md <- First-run ritual (deleted after onboarding)
├── context/ <- Identity-related docs
│ ├── vision.md
│ └── priorities.md
+4
View File
@@ -27,6 +27,10 @@ pub mod paths {
pub const DAILY_DIR: &str = "daily/";
/// Context directory (for identity-related docs).
pub const CONTEXT_DIR: &str = "context/";
/// User-editable notes for environment-specific tool guidance.
pub const TOOLS: &str = "TOOLS.md";
/// First-run ritual file; self-deletes after onboarding completes.
pub const BOOTSTRAP: &str = "BOOTSTRAP.md";
}
/// A memory document stored in the database.
+182
View File
@@ -271,6 +271,53 @@ const HEARTBEAT_SEED: &str = "\
- Clean up context/ documents that are outdated
-->";
/// Default template seeded into TOOLS.md on first access.
///
/// TOOLS.md does not control tool availability; it is user guidance
/// for how to use external tools. The agent may update this file as it
/// learns environment-specific details (SSH hostnames, device names, etc.).
const TOOLS_SEED: &str = "\
<!-- TOOLS.md Environment-specific tool notes.
This file does not control which tools are available; it is guidance only.
The agent can update this file as it learns your setup.
Examples:
- SSH hosts: dev-box (Ubuntu 22.04, username: alice)
- Camera: Canon R6 mounted at /Volumes/EOS_R
- Default shell on remote: bash, no zsh
Add your environment notes below (outside the comment block).
-->";
/// First-run ritual seeded into BOOTSTRAP.md on initial workspace setup.
///
/// The agent reads this file at the start of every session when it exists.
/// After completing the ritual the agent must delete this file so it is
/// never repeated. It is NOT a protected file; the agent needs write access.
const BOOTSTRAP_SEED: &str = "\
# Bootstrap
You are starting up for the first time. Follow these steps before anything else.
## Steps
1. **Say hello.** Greet the user warmly and introduce yourself briefly.
2. **Get to know the user.** Ask a few questions to understand who they are, \
what they work on, and what they want from an AI assistant. Take notes.
3. **Save what you learned.**
- Write any environment-specific tool details the user mentions to `TOOLS.md` \
using `memory_write` with target set to the path.
- Write a summary of the conversation and key facts to `MEMORY.md` \
using `memory_write` with target `memory`.
- Note: `USER.md`, `IDENTITY.md`, `SOUL.md`, and `AGENTS.md` are protected \
from tool writes for security. Tell the user what you'd suggest for those files \
so they can edit them directly.
4. **Delete this file.** When onboarding is complete, use `memory_write` with \
target `bootstrap` to clear this file so setup never repeats.
Keep the conversation natural. Do not read these steps aloud.
";
/// Workspace provides database-backed memory storage for an agent.
///
/// Each workspace is scoped to a user (and optionally an agent).
@@ -547,6 +594,24 @@ impl Workspace {
) -> Result<String, WorkspaceError> {
let mut parts = Vec::new();
// Bootstrap ritual: inject FIRST when present (first-run only).
// The agent must complete the ritual and then delete this file.
//
// Note: BOOTSTRAP.md is intentionally NOT write-protected so the agent
// can delete it after onboarding. This means a prompt injection attack
// could write to it, but the file is only injected on the next session
// (not the current one), limiting the blast radius.
if let Ok(doc) = self.read(paths::BOOTSTRAP).await
&& !doc.content.is_empty()
{
parts.push(format!(
"## First-Run Bootstrap\n\n\
A BOOTSTRAP.md file exists in the workspace. Read and follow it, \
then delete it when done.\n\n{}",
doc.content
));
}
// Load identity files in order of importance
let identity_files = [
(paths::AGENTS, "## Agent Instructions"),
@@ -563,6 +628,14 @@ impl Workspace {
}
}
// Tool notes: environment-specific guidance the agent or user has written.
// TOOLS.md does not control tool availability; it is guidance only.
if let Ok(doc) = self.read(paths::TOOLS).await
&& !doc.content.is_empty()
{
parts.push(format!("## Tool Notes\n\n{}", doc.content));
}
// Load MEMORY.md only in direct/main sessions (never group chats)
if !is_group_chat
&& let Ok(doc) = self.read(paths::MEMORY).await
@@ -693,6 +766,7 @@ impl Workspace {
- `SOUL.md` - Core values and behavioral boundaries\n\
- `AGENTS.md` - Session routine and operational instructions\n\
- `USER.md` - Information about you (the user)\n\
- `TOOLS.md` - Environment-specific tool notes\n\
- `HEARTBEAT.md` - Periodic background task checklist\n\
- `daily/` - Automatic daily session logs\n\
- `context/` - Additional context documents\n\n\
@@ -763,6 +837,7 @@ impl Workspace {
You can also edit this directly to provide context upfront.",
),
(paths::HEARTBEAT, HEARTBEAT_SEED),
(paths::TOOLS, TOOLS_SEED),
];
let mut count = 0;
@@ -784,12 +859,119 @@ impl Workspace {
}
}
// BOOTSTRAP.md is only seeded on truly fresh workspaces (no identity
// files exist yet). This prevents existing users from getting a
// spurious first-run ritual after upgrading.
if self.read(paths::BOOTSTRAP).await.is_err() {
let (agents_res, soul_res, user_res) = tokio::join!(
self.read(paths::AGENTS),
self.read(paths::SOUL),
self.read(paths::USER),
);
let is_fresh_workspace =
matches!(agents_res, Err(WorkspaceError::DocumentNotFound { .. }))
&& matches!(soul_res, Err(WorkspaceError::DocumentNotFound { .. }))
&& matches!(user_res, Err(WorkspaceError::DocumentNotFound { .. }));
if is_fresh_workspace {
if let Err(e) = self.write(paths::BOOTSTRAP, BOOTSTRAP_SEED).await {
tracing::warn!("Failed to seed {}: {}", paths::BOOTSTRAP, e);
} else {
count += 1;
}
}
}
if count > 0 {
tracing::info!("Seeded {} workspace files", count);
}
Ok(count)
}
/// Import markdown files from a directory on disk into the workspace DB.
///
/// Scans `dir` for `*.md` files (non-recursive) and writes each one into
/// the workspace **only if it doesn't already exist in the database**.
/// This allows Docker images or deployment scripts to ship customized
/// workspace templates that override the generic seeds.
///
/// Returns the number of files imported (0 if all already existed).
pub async fn import_from_directory(
&self,
dir: &std::path::Path,
) -> Result<usize, WorkspaceError> {
if !dir.is_dir() {
tracing::warn!(
"Workspace import directory does not exist: {}",
dir.display()
);
return Ok(0);
}
let entries = std::fs::read_dir(dir).map_err(|e| WorkspaceError::IoError {
reason: format!("failed to read directory {}: {}", dir.display(), e),
})?;
let mut count = 0;
for entry in entries {
let entry = match entry {
Ok(e) => e,
Err(e) => {
tracing::warn!("Failed to read directory entry in {}: {}", dir.display(), e);
continue;
}
};
let path = entry.path();
// Only import .md files
if path.extension() != Some(std::ffi::OsStr::new("md")) {
continue;
}
let Some(file_name) = path.file_name().and_then(|n| n.to_str()) else {
continue;
};
// Skip if already exists in DB (never overwrite user edits)
match self.read(file_name).await {
Ok(_) => continue,
Err(WorkspaceError::DocumentNotFound { .. }) => {}
Err(e) => {
tracing::warn!("Failed to check {}: {}", file_name, e);
continue;
}
}
let content = match std::fs::read_to_string(&path) {
Ok(c) => c,
Err(e) => {
tracing::warn!("Failed to read import file {}: {}", path.display(), e);
continue;
}
};
if content.trim().is_empty() {
continue;
}
if let Err(e) = self.write(file_name, &content).await {
tracing::warn!("Failed to import {}: {}", file_name, e);
} else {
tracing::info!("Imported workspace file from disk: {}", file_name);
count += 1;
}
}
if count > 0 {
tracing::info!(
"Imported {} workspace file(s) from {}",
count,
dir.display()
);
}
Ok(count)
}
/// Generate embeddings for chunks that don't have them yet.
///
/// This is useful for backfilling embeddings after enabling the provider.
+4 -2
View File
@@ -431,7 +431,7 @@ impl Repository {
let rows = conn
.query(
r#"
SELECT c.id as chunk_id, c.document_id, c.content,
SELECT c.id as chunk_id, c.document_id, d.path as document_path, c.content,
ts_rank_cd(c.content_tsv, plainto_tsquery('english', $3)) as rank
FROM memory_chunks c
JOIN memory_documents d ON d.id = c.document_id
@@ -453,6 +453,7 @@ impl Repository {
.map(|(i, row)| RankedResult {
chunk_id: row.get("chunk_id"),
document_id: row.get("document_id"),
document_path: row.get("document_path"),
content: row.get("content"),
rank: (i + 1) as u32,
})
@@ -473,7 +474,7 @@ impl Repository {
let rows = conn
.query(
r#"
SELECT c.id as chunk_id, c.document_id, c.content,
SELECT c.id as chunk_id, c.document_id, d.path as document_path, c.content,
1 - (c.embedding <=> $3) as similarity
FROM memory_chunks c
JOIN memory_documents d ON d.id = c.document_id
@@ -495,6 +496,7 @@ impl Repository {
.map(|(i, row)| RankedResult {
chunk_id: row.get("chunk_id"),
document_id: row.get("document_id"),
document_path: row.get("document_path"),
content: row.get("content"),
rank: (i + 1) as u32,
})
+9
View File
@@ -81,6 +81,8 @@ impl SearchConfig {
pub struct SearchResult {
/// Document ID containing this chunk.
pub document_id: Uuid,
/// File path of the source document.
pub document_path: String,
/// Chunk ID.
pub chunk_id: Uuid,
/// Chunk content.
@@ -115,6 +117,8 @@ impl SearchResult {
pub struct RankedResult {
pub chunk_id: Uuid,
pub document_id: Uuid,
/// File path of the source document.
pub document_path: String,
pub content: String,
pub rank: u32, // 1-based rank
}
@@ -143,6 +147,7 @@ pub fn reciprocal_rank_fusion(
// Track scores and metadata for each chunk
struct ChunkInfo {
document_id: Uuid,
document_path: String,
content: String,
score: f32,
fts_rank: Option<u32>,
@@ -162,6 +167,7 @@ pub fn reciprocal_rank_fusion(
})
.or_insert(ChunkInfo {
document_id: result.document_id,
document_path: result.document_path,
content: result.content,
score: rrf_score,
fts_rank: Some(result.rank),
@@ -180,6 +186,7 @@ pub fn reciprocal_rank_fusion(
})
.or_insert(ChunkInfo {
document_id: result.document_id,
document_path: result.document_path,
content: result.content,
score: rrf_score,
fts_rank: None,
@@ -192,6 +199,7 @@ pub fn reciprocal_rank_fusion(
.into_iter()
.map(|(chunk_id, info)| SearchResult {
document_id: info.document_id,
document_path: info.document_path,
chunk_id,
content: info.content,
score: info.score,
@@ -235,6 +243,7 @@ mod tests {
RankedResult {
chunk_id,
document_id: doc_id,
document_path: format!("docs/{}.md", doc_id),
content: format!("content for chunk {}", chunk_id),
rank,
}
+12 -2
View File
@@ -98,6 +98,13 @@ async def ironclaw_server(ironclaw_binary, mock_llm_server):
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
}
# Forward LLVM coverage instrumentation env vars when present
# (allows cargo-llvm-cov to collect profraw data from E2E runs)
for key in ("LLVM_PROFILE_FILE", "CARGO_LLVM_COV", "CARGO_LLVM_COV_SHOW_ENV",
"CARGO_LLVM_COV_TARGET_DIR"):
val = os.environ.get(key)
if val is not None:
env[key] = val
proc = await asyncio.create_subprocess_exec(
ironclaw_binary, "--no-onboard",
stdin=asyncio.subprocess.DEVNULL,
@@ -126,9 +133,12 @@ async def ironclaw_server(ironclaw_binary, mock_llm_server):
)
finally:
if proc.returncode is None:
proc.send_signal(signal.SIGTERM)
# Use SIGINT (not SIGTERM) so tokio's ctrl_c handler triggers a
# graceful shutdown. This lets the LLVM coverage runtime run its
# atexit handler and flush .profraw files for cargo-llvm-cov.
proc.send_signal(signal.SIGINT)
try:
await asyncio.wait_for(proc.wait(), timeout=5)
await asyncio.wait_for(proc.wait(), timeout=10)
except asyncio.TimeoutError:
proc.kill()
+2 -2
View File
@@ -191,6 +191,7 @@ async fn start_test_server_with_provider(
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test-user".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -201,7 +202,6 @@ async fn start_test_server_with_provider(
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
@@ -680,6 +680,7 @@ async fn test_no_llm_provider_returns_503() {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test-user".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -690,7 +691,6 @@ async fn test_no_llm_provider_returns_503() {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
+3 -1
View File
@@ -49,6 +49,7 @@ async fn start_test_server() -> (
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test-user".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -59,7 +60,6 @@ async fn start_test_server() -> (
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
@@ -312,6 +312,8 @@ async fn test_ws_multiple_events_in_sequence() {
state.sse.broadcast(SseEvent::ToolCompleted {
name: "shell".to_string(),
success: true,
error: None,
parameters: None,
thread_id: None,
});
state.sse.broadcast(SseEvent::Response {
@@ -34,11 +34,19 @@
]
}
},
"auth": {
"secret_name": "github_token",
"display_name": "GitHub",
"instructions": "Create a Personal Access Token at github.com/settings/tokens with repo scope, then paste it here.",
"setup_url": "https://github.com/settings/tokens",
"token_hint": "Starts with 'ghp_' or 'github_pat_'",
"env_var": "GITHUB_TOKEN"
},
"setup": {
"required_secrets": [
{
"name": "github_token",
"prompt": "GitHub Personal Access Token (from github.com/settings/tokens)"
"prompt": "GitHub Personal Access Token (create one at github.com/settings/tokens with 'repo' scope)"
}
]
},
-105
View File
@@ -1,105 +0,0 @@
{
"http": {
"allowlist": [
{
"host": "*.okta.com",
"path_prefix": "/api/v1/",
"methods": ["GET", "POST", "PUT"]
},
{
"host": "*.okta.com",
"path_prefix": "/idp/myaccount/",
"methods": ["GET", "PUT"]
},
{
"host": "*.okta.com",
"path_prefix": "/oauth2/v1/",
"methods": ["POST"]
},
{
"host": "*.oktapreview.com",
"path_prefix": "/api/v1/",
"methods": ["GET", "POST", "PUT"]
},
{
"host": "*.oktapreview.com",
"path_prefix": "/idp/myaccount/",
"methods": ["GET", "PUT"]
},
{
"host": "*.oktapreview.com",
"path_prefix": "/oauth2/v1/",
"methods": ["POST"]
},
{
"host": "*.okta-emea.com",
"path_prefix": "/api/v1/",
"methods": ["GET", "POST", "PUT"]
},
{
"host": "*.okta-emea.com",
"path_prefix": "/idp/myaccount/",
"methods": ["GET", "PUT"]
},
{
"host": "*.okta-emea.com",
"path_prefix": "/oauth2/v1/",
"methods": ["POST"]
}
],
"credentials": {
"okta_oauth_token": {
"secret_name": "okta_oauth_token",
"location": { "type": "bearer" },
"host_patterns": ["*.okta.com", "*.oktapreview.com", "*.okta-emea.com"]
}
},
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 500
},
"timeout_secs": 30
},
"workspace": {
"allowed_prefixes": ["okta/"]
},
"secrets": {
"allowed_names": ["okta_oauth_token"]
},
"auth": {
"secret_name": "okta_oauth_token",
"display_name": "Okta",
"oauth": {
"authorization_url": "https://{okta_domain}/oauth2/v1/authorize",
"token_url": "https://{okta_domain}/oauth2/v1/token",
"client_id_env": "OKTA_OAUTH_CLIENT_ID",
"client_secret_env": "OKTA_OAUTH_CLIENT_SECRET",
"scopes": [
"openid",
"profile",
"email",
"offline_access",
"okta.users.read.self",
"okta.users.manage.self",
"okta.apps.read"
],
"use_pkce": true
},
"instructions": "1. In your Okta Admin Console, go to Applications > Create App Integration\n2. Select 'OIDC - OpenID Connect', then 'Web Application'\n3. Set Sign-in redirect URI to http://localhost:9876/callback (through :9886)\n4. Under Okta API Scopes, grant: okta.users.read.self, okta.users.manage.self, okta.apps.read\n5. Copy the Client ID and Client Secret\n6. IMPORTANT: You must use the Org Authorization Server (not a custom one)\n7. Store your Okta domain in workspace at 'okta/domain' (e.g., 'mycompany.okta.com')\n8. For custom domains, add them to okta-tool.capabilities.json allowlist",
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/",
"token_hint": "OAuth2 access token (JWT)",
"env_var": "OKTA_OAUTH_TOKEN"
},
"setup": {
"required_secrets": [
{
"name": "okta_oauth_client_id",
"prompt": "Okta OAuth Client ID"
},
{
"name": "okta_oauth_client_secret",
"prompt": "Okta OAuth Client Secret"
}
]
}
}
-281
View File
@@ -1,281 +0,0 @@
use crate::near::agent::host;
use crate::types::*;
const WORKSPACE_DOMAIN_PATH: &str = "okta/domain";
/// Read the configured Okta domain from workspace, or return a helpful error.
fn get_domain() -> Result<String, String> {
host::workspace_read(WORKSPACE_DOMAIN_PATH).ok_or_else(|| {
"Okta domain not configured. Write your Okta domain to workspace path 'okta/domain' \
using the memory_write tool (e.g., memory_write with path='okta/domain' and \
content='mycompany.okta.com')."
.to_string()
})
}
/// Build the base URL for the Okta Management API.
fn management_base(domain: &str) -> String {
format!("https://{}/api/v1", domain)
}
/// Make an Okta API call.
fn okta_api_call(method: &str, url: &str, body: Option<&str>) -> Result<String, String> {
let headers = if body.is_some() {
r#"{"Content-Type": "application/json", "Accept": "application/json"}"#
} else {
r#"{"Accept": "application/json"}"#
};
let body_bytes = body.map(|b| b.as_bytes().to_vec());
host::log(
host::LogLevel::Debug,
&format!("Okta API: {} {}", method, url),
);
let response = host::http_request(method, url, headers, body_bytes.as_deref(), None)?;
if response.status < 200 || response.status >= 300 {
let body_text = String::from_utf8_lossy(&response.body);
// Try to extract Okta's error summary for a better message.
if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(&body_text) {
if let Some(summary) = parsed["errorSummary"].as_str() {
return Err(format!("Okta API error ({}): {}", response.status, summary));
}
}
return Err(format!(
"Okta API returned status {}: {}",
response.status, body_text
));
}
String::from_utf8(response.body).map_err(|e| format!("Invalid UTF-8: {}", e))
}
// ---------------------------------------------------------------------------
// Action implementations
// ---------------------------------------------------------------------------
/// GET /api/v1/users/me
pub fn get_profile() -> Result<String, String> {
let domain = get_domain()?;
let url = format!("{}/users/me", management_base(&domain));
let response = okta_api_call("GET", &url, None)?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let profile = parse_user_profile(&parsed)?;
serde_json::to_string(&profile).map_err(|e| e.to_string())
}
/// POST /api/v1/users/me (partial update via Management API)
pub fn update_profile(fields: &serde_json::Value) -> Result<String, String> {
let domain = get_domain()?;
let url = format!("{}/users/me", management_base(&domain));
// Wrap fields under "profile" key for Okta's expected format.
let payload = serde_json::json!({ "profile": fields });
let body = serde_json::to_string(&payload).map_err(|e| e.to_string())?;
let response = okta_api_call("POST", &url, Some(&body))?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let profile = parse_user_profile(&parsed)?;
let result = UpdateProfileResult {
success: true,
profile,
};
serde_json::to_string(&result).map_err(|e| e.to_string())
}
/// GET /api/v1/users/me/appLinks
pub fn list_apps() -> Result<String, String> {
let domain = get_domain()?;
let url = format!("{}/users/me/appLinks", management_base(&domain));
let response = okta_api_call("GET", &url, None)?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let apps = parse_app_links(&parsed)?;
let count = apps.len();
let result = ListAppsResult { apps, count };
serde_json::to_string(&result).map_err(|e| e.to_string())
}
/// Search apps by label (case-insensitive substring match).
pub fn search_apps(query: &str) -> Result<String, String> {
let domain = get_domain()?;
let url = format!("{}/users/me/appLinks", management_base(&domain));
let response = okta_api_call("GET", &url, None)?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let all_apps = parse_app_links(&parsed)?;
let query_lower = query.to_lowercase();
let apps: Vec<AppLink> = all_apps
.into_iter()
.filter(|app| {
app.label.to_lowercase().contains(&query_lower)
|| app.app_name.to_lowercase().contains(&query_lower)
})
.collect();
let count = apps.len();
let result = ListAppsResult { apps, count };
serde_json::to_string(&result).map_err(|e| e.to_string())
}
/// Find an app by ID or label and return its SSO launch link.
pub fn get_app_sso_link(app: &str) -> Result<String, String> {
let domain = get_domain()?;
let url = format!("{}/users/me/appLinks", management_base(&domain));
let response = okta_api_call("GET", &url, None)?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let all_apps = parse_app_links(&parsed)?;
let app_lower = app.to_lowercase();
// Try exact ID match first, then case-insensitive label match.
let found = all_apps
.iter()
.find(|a| a.app_instance_id == app)
.or_else(|| {
all_apps
.iter()
.find(|a| a.label.to_lowercase() == app_lower)
})
.or_else(|| {
all_apps
.iter()
.find(|a| a.label.to_lowercase().contains(&app_lower))
});
match found {
Some(app_link) => {
let result = AppSsoLinkResult {
label: app_link.label.clone(),
link_url: app_link.link_url.clone(),
app_instance_id: app_link.app_instance_id.clone(),
app_name: app_link.app_name.clone(),
};
serde_json::to_string(&result).map_err(|e| e.to_string())
}
None => {
let available: Vec<String> = all_apps.iter().map(|a| a.label.clone()).collect();
Err(format!(
"App '{}' not found. Available apps: {}",
app,
available.join(", ")
))
}
}
}
/// GET /idp/myaccount/organization
pub fn get_org_info() -> Result<String, String> {
let domain = get_domain()?;
let url = format!("https://{}/idp/myaccount/organization", domain);
// MyAccount API requires the okta-version header.
let response = okta_api_call_with_headers(
"GET",
&url,
None,
r#"{"Accept": "application/json; okta-version=1.0.0"}"#,
)?;
let parsed: serde_json::Value =
serde_json::from_str(&response).map_err(|e| format!("Failed to parse response: {}", e))?;
let result = OrgInfo {
id: parsed["id"].as_str().unwrap_or("").to_string(),
name: parsed["name"].as_str().unwrap_or("").to_string(),
subdomain: parsed["subdomain"].as_str().map(|s| s.to_string()),
website: parsed["website"].as_str().map(|s| s.to_string()),
support_phone: parsed["supportPhoneNumber"].as_str().map(|s| s.to_string()),
technical_contact: parsed["technicalContact"].as_str().map(|s| s.to_string()),
};
serde_json::to_string(&result).map_err(|e| e.to_string())
}
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/// Like `okta_api_call` but with custom headers (for MyAccount API versioning).
fn okta_api_call_with_headers(
method: &str,
url: &str,
body: Option<&str>,
headers: &str,
) -> Result<String, String> {
let body_bytes = body.map(|b| b.as_bytes().to_vec());
host::log(
host::LogLevel::Debug,
&format!("Okta API: {} {}", method, url),
);
let response = host::http_request(method, url, headers, body_bytes.as_deref(), None)?;
if response.status < 200 || response.status >= 300 {
let body_text = String::from_utf8_lossy(&response.body);
if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(&body_text) {
if let Some(summary) = parsed["errorSummary"].as_str() {
return Err(format!("Okta API error ({}): {}", response.status, summary));
}
}
return Err(format!(
"Okta API returned status {}: {}",
response.status, body_text
));
}
String::from_utf8(response.body).map_err(|e| format!("Invalid UTF-8: {}", e))
}
fn parse_user_profile(v: &serde_json::Value) -> Result<UserProfile, String> {
let p = &v["profile"];
Ok(UserProfile {
id: v["id"].as_str().unwrap_or("").to_string(),
status: v["status"].as_str().unwrap_or("").to_string(),
first_name: p["firstName"].as_str().unwrap_or("").to_string(),
last_name: p["lastName"].as_str().unwrap_or("").to_string(),
email: p["email"].as_str().unwrap_or("").to_string(),
login: p["login"].as_str().unwrap_or("").to_string(),
mobile_phone: p["mobilePhone"].as_str().map(|s| s.to_string()),
display_name: p["displayName"].as_str().map(|s| s.to_string()),
nick_name: p["nickName"].as_str().map(|s| s.to_string()),
title: p["title"].as_str().map(|s| s.to_string()),
department: p["department"].as_str().map(|s| s.to_string()),
organization: p["organization"].as_str().map(|s| s.to_string()),
timezone: p["timezone"].as_str().map(|s| s.to_string()),
locale: p["locale"].as_str().map(|s| s.to_string()),
})
}
fn parse_app_links(v: &serde_json::Value) -> Result<Vec<AppLink>, String> {
let arr = v
.as_array()
.ok_or_else(|| "Expected array of app links from Okta".to_string())?;
Ok(arr
.iter()
.map(|a| AppLink {
app_instance_id: a["appInstanceId"].as_str().unwrap_or("").to_string(),
label: a["label"].as_str().unwrap_or("").to_string(),
link_url: a["linkUrl"].as_str().unwrap_or("").to_string(),
logo_url: a["logoUrl"].as_str().map(|s| s.to_string()),
app_name: a["appName"].as_str().unwrap_or("").to_string(),
hidden: a["hidden"].as_bool().unwrap_or(false),
})
.collect())
}
-117
View File
@@ -1,117 +0,0 @@
//! Okta WASM Tool for IronClaw.
//!
//! Provides user profile management, SSO app catalog browsing, and
//! launch links for all applications under Okta single sign-on.
//!
//! # Setup
//!
//! 1. Configure OAuth2 with PKCE (see capabilities.json instructions)
//! 2. Write your Okta domain to workspace: `memory_write(path="okta/domain", content="mycompany.okta.com")`
//! 3. All actions read the domain from workspace automatically
//!
//! # Capabilities Required
//!
//! - HTTP: `*.okta.com/api/v1/*`, `*.okta.com/idp/myaccount/*` (GET, POST, PUT)
//! - Secrets: `okta_oauth_token` (injected as Bearer token)
//! - Workspace: `okta/` prefix (read-only, for domain config)
//!
//! # Supported Actions
//!
//! - `get_profile`: Fetch the current user's profile
//! - `update_profile`: Update profile fields
//! - `list_apps`: List all SSO apps assigned to the user
//! - `search_apps`: Search apps by name
//! - `get_app_sso_link`: Get the SSO launch URL for a specific app
//! - `get_org_info`: Get organization details
mod api;
mod types;
use types::OktaAction;
wit_bindgen::generate!({
world: "sandboxed-tool",
path: "../../wit/tool.wit",
});
struct OktaTool;
impl exports::near::agent::tool::Guest for OktaTool {
fn execute(req: exports::near::agent::tool::Request) -> exports::near::agent::tool::Response {
match execute_inner(&req.params) {
Ok(result) => exports::near::agent::tool::Response {
output: Some(result),
error: None,
},
Err(e) => exports::near::agent::tool::Response {
output: None,
error: Some(e),
},
}
}
fn schema() -> String {
r#"{
"type": "object",
"required": ["action"],
"properties": {
"action": {
"type": "string",
"enum": ["get_profile", "update_profile", "list_apps", "search_apps", "get_app_sso_link", "get_org_info"],
"description": "The Okta operation to perform"
},
"fields": {
"type": "object",
"description": "Profile fields to update (e.g., firstName, lastName, email, mobilePhone, displayName, title, department). Required for: update_profile"
},
"query": {
"type": "string",
"description": "Case-insensitive search query to match against app labels and names. Required for: search_apps"
},
"app": {
"type": "string",
"description": "App instance ID (e.g., '0oa1xxx') or app label (e.g., 'Google Workspace'). Required for: get_app_sso_link"
}
}
}"#
.to_string()
}
fn description() -> String {
"Okta SSO tool for managing your profile and accessing all applications under \
single sign-on. Supports viewing/updating your Okta profile, listing all assigned \
SSO apps, searching apps by name, and getting direct SSO launch links. Requires \
Okta domain in workspace at 'okta/domain' and an OAuth token with \
okta.users.read.self, okta.users.manage.self, and okta.apps.read scopes."
.to_string()
}
}
fn execute_inner(params: &str) -> Result<String, String> {
if !crate::near::agent::host::secret_exists("okta_oauth_token") {
return Err(
"Okta OAuth token not configured. Please add the 'okta_oauth_token' secret \
via OAuth2 flow or set the OKTA_OAUTH_TOKEN environment variable."
.to_string(),
);
}
let action: OktaAction =
serde_json::from_str(params).map_err(|e| format!("Invalid parameters: {}", e))?;
crate::near::agent::host::log(
crate::near::agent::host::LogLevel::Info,
&format!("Executing Okta action: {:?}", action),
);
match action {
OktaAction::GetProfile => api::get_profile(),
OktaAction::UpdateProfile { fields } => api::update_profile(&fields),
OktaAction::ListApps => api::list_apps(),
OktaAction::SearchApps { query } => api::search_apps(&query),
OktaAction::GetAppSsoLink { app } => api::get_app_sso_link(&app),
OktaAction::GetOrgInfo => api::get_org_info(),
}
}
export!(OktaTool);
-119
View File
@@ -1,119 +0,0 @@
use serde::{Deserialize, Serialize};
/// Input parameters for the Okta tool.
///
/// Actions map to Okta Management API (/api/v1/) and MyAccount API (/idp/myaccount/).
/// The tool reads the Okta domain from workspace at `okta/domain`.
#[derive(Debug, Deserialize)]
#[serde(tag = "action", rename_all = "snake_case")]
pub enum OktaAction {
/// Get the current user's Okta profile.
GetProfile,
/// Update fields on the current user's profile (partial update).
UpdateProfile {
/// Key-value pairs of profile fields to update.
/// Common fields: firstName, lastName, email, mobilePhone, displayName,
/// nickName, title, department, organization.
fields: serde_json::Value,
},
/// List all SSO applications assigned to the current user.
ListApps,
/// Search assigned apps by name (case-insensitive substring match).
SearchApps {
/// Search query to match against app labels.
query: String,
},
/// Get the SSO launch link for a specific app by its instance ID or label.
GetAppSsoLink {
/// App instance ID (e.g., "0oa1xxx") or app label to search for.
app: String,
},
/// Get information about the Okta organization.
GetOrgInfo,
}
// ---------------------------------------------------------------------------
// Response types
// ---------------------------------------------------------------------------
/// User profile from Okta.
#[derive(Debug, Serialize)]
pub struct UserProfile {
pub id: String,
pub status: String,
pub first_name: String,
pub last_name: String,
pub email: String,
pub login: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub mobile_phone: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub nick_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub department: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub organization: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub timezone: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub locale: Option<String>,
}
/// Result of a profile update.
#[derive(Debug, Serialize)]
pub struct UpdateProfileResult {
pub success: bool,
pub profile: UserProfile,
}
/// An SSO app link (chiclet) assigned to the user.
#[derive(Debug, Serialize)]
pub struct AppLink {
pub app_instance_id: String,
pub label: String,
pub link_url: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub logo_url: Option<String>,
pub app_name: String,
pub hidden: bool,
}
/// Result of listing or searching apps.
#[derive(Debug, Serialize)]
pub struct ListAppsResult {
pub apps: Vec<AppLink>,
pub count: usize,
}
/// SSO launch link for a specific app.
#[derive(Debug, Serialize)]
pub struct AppSsoLinkResult {
pub label: String,
pub link_url: String,
pub app_instance_id: String,
pub app_name: String,
}
/// Okta organization info.
#[derive(Debug, Serialize)]
pub struct OrgInfo {
pub id: String,
pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub subdomain: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub website: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub support_phone: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub technical_contact: Option<String>,
}
@@ -1,23 +1,24 @@
[package]
name = "okta-tool"
name = "web-search-tool"
version = "0.1.0"
edition = "2021"
description = "Okta SSO tool for IronClaw (WASM component) — user profile, app catalog, and SSO launch links"
description = "Brave Web Search tool for IronClaw (WASM component)"
license = "MIT OR Apache-2.0"
publish = false
[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.41.0"
[lib]
crate-type = ["cdylib"]
[dependencies]
wit-bindgen = "=0.36"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
[profile.release]
opt-level = "s"
lto = true
strip = true
codegen-units = 1
[workspace]
+478
View File
@@ -0,0 +1,478 @@
//! Brave Web Search WASM Tool for IronClaw.
//!
//! Searches the web using the Brave Search API and returns structured results.
//!
//! # Authentication
//!
//! Store your Brave Search API key:
//! `ironclaw secret set brave_api_key <key>`
//!
//! Get a key at: https://brave.com/search/api/
wit_bindgen::generate!({
world: "sandboxed-tool",
path: "../../wit/tool.wit",
});
use serde::Deserialize;
const BRAVE_SEARCH_ENDPOINT: &str = "https://api.search.brave.com/res/v1/web/search";
const MAX_COUNT: u32 = 20;
const DEFAULT_COUNT: u32 = 5;
const MAX_RETRIES: u32 = 3;
struct WebSearchTool;
impl exports::near::agent::tool::Guest for WebSearchTool {
fn execute(req: exports::near::agent::tool::Request) -> exports::near::agent::tool::Response {
match execute_inner(&req.params) {
Ok(result) => exports::near::agent::tool::Response {
output: Some(result),
error: None,
},
Err(e) => exports::near::agent::tool::Response {
output: None,
error: Some(e),
},
}
}
fn schema() -> String {
SCHEMA.to_string()
}
fn description() -> String {
"Search the web using Brave Search. Returns titles, URLs, descriptions, and \
publication dates for matching web pages. Supports filtering by country, \
language, and freshness. Authentication is handled via the 'brave_api_key' \
secret injected by the host."
.to_string()
}
}
#[derive(Debug, Deserialize)]
struct SearchParams {
query: String,
count: Option<u32>,
country: Option<String>,
search_lang: Option<String>,
ui_lang: Option<String>,
freshness: Option<String>,
}
#[derive(Debug, Deserialize)]
struct BraveSearchResponse {
web: Option<BraveWebResults>,
}
#[derive(Debug, Deserialize)]
struct BraveWebResults {
results: Option<Vec<BraveSearchResult>>,
}
#[derive(Debug, Deserialize)]
struct BraveSearchResult {
title: Option<String>,
url: Option<String>,
description: Option<String>,
age: Option<String>,
}
fn execute_inner(params: &str) -> Result<String, String> {
let params: SearchParams =
serde_json::from_str(params).map_err(|e| format!("Invalid parameters: {e}"))?;
if params.query.is_empty() {
return Err("'query' must not be empty".into());
}
if params.query.len() > 2000 {
return Err("'query' exceeds maximum length of 2000 characters".into());
}
// Validate optional parameters.
if let Some(ref lang) = params.search_lang {
if !is_valid_lang_code(lang) {
return Err(format!(
"Invalid 'search_lang': expected 2-letter code like 'en', got '{lang}'"
));
}
}
if let Some(ref country) = params.country {
if !is_valid_country_code(country) {
return Err(format!(
"Invalid 'country': expected 2-letter code like 'US', got '{country}'"
));
}
}
if let Some(ref ui_lang) = params.ui_lang {
if !is_valid_ui_lang(ui_lang) {
return Err(format!(
"Invalid 'ui_lang': expected format like 'en-US', got '{ui_lang}'"
));
}
}
if let Some(ref freshness) = params.freshness {
if !is_valid_freshness(freshness) {
return Err(format!(
"Invalid 'freshness': expected 'pd', 'pw', 'pm', 'py', or \
'YYYY-MM-DDtoYYYY-MM-DD', got '{freshness}'"
));
}
}
// Pre-flight: verify API key is available.
if !near::agent::host::secret_exists("brave_api_key") {
return Err(
"Brave API key not found in secret store. Set it with: \
ironclaw secret set brave_api_key <key>. \
Get a key at: https://brave.com/search/api/"
.into(),
);
}
let count = params.count.unwrap_or(DEFAULT_COUNT).clamp(1, MAX_COUNT);
let url = build_search_url(&params.query, count, &params);
// X-Subscription-Token is injected by the host via credential config.
let headers = serde_json::json!({
"Accept": "application/json",
"User-Agent": "IronClaw-WebSearch-Tool/0.1"
});
// Retry loop for transient errors (429 rate limit, 5xx server errors).
let response = {
let mut attempt = 0;
loop {
attempt += 1;
let resp =
near::agent::host::http_request("GET", &url, &headers.to_string(), None, None)
.map_err(|e| format!("HTTP request failed: {e}"))?;
if resp.status >= 200 && resp.status < 300 {
break resp;
}
if attempt < MAX_RETRIES && (resp.status == 429 || resp.status >= 500) {
near::agent::host::log(
near::agent::host::LogLevel::Warn,
&format!(
"Brave API error {} (attempt {}/{}). Retrying...",
resp.status, attempt, MAX_RETRIES
),
);
continue;
}
let body = String::from_utf8_lossy(&resp.body);
return Err(format!(
"Brave API error (HTTP {}): {}",
resp.status, body
));
}
};
let body =
String::from_utf8(response.body).map_err(|e| format!("Invalid UTF-8 response: {e}"))?;
let brave_response: BraveSearchResponse =
serde_json::from_str(&body).map_err(|e| format!("Failed to parse Brave response: {e}"))?;
let results = brave_response
.web
.and_then(|w| w.results)
.unwrap_or_default();
let formatted: Vec<serde_json::Value> = results
.into_iter()
.filter_map(|r| {
let title = r.title?;
let url = r.url?;
let description = r.description.unwrap_or_default();
let mut entry = serde_json::json!({
"title": title,
"url": url,
"description": description,
});
if let Some(age) = r.age {
entry["published"] = serde_json::json!(age);
}
// Extract hostname for site_name.
if let Some(host) = extract_hostname(&url) {
entry["site_name"] = serde_json::json!(host);
}
Some(entry)
})
.collect();
let output = serde_json::json!({
"query": params.query,
"result_count": formatted.len(),
"results": formatted,
});
serde_json::to_string(&output).map_err(|e| format!("Failed to serialize output: {e}"))
}
fn build_search_url(query: &str, count: u32, params: &SearchParams) -> String {
let mut url = format!(
"{}?q={}&count={}",
BRAVE_SEARCH_ENDPOINT,
url_encode(query),
count
);
if let Some(ref country) = params.country {
url.push_str(&format!("&country={}", url_encode(country)));
}
if let Some(ref search_lang) = params.search_lang {
url.push_str(&format!("&search_lang={}", url_encode(search_lang)));
}
if let Some(ref ui_lang) = params.ui_lang {
url.push_str(&format!("&ui_lang={}", url_encode(ui_lang)));
}
if let Some(ref freshness) = params.freshness {
url.push_str(&format!("&freshness={}", url_encode(freshness)));
}
url
}
/// Percent-encode a string for safe use in URL query parameters.
fn url_encode(s: &str) -> String {
let mut out = String::with_capacity(s.len() * 2);
for b in s.bytes() {
match b {
b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'_' | b'.' | b'~' => {
out.push(b as char);
}
b' ' => out.push_str("%20"),
_ => {
out.push('%');
out.push(char::from(b"0123456789ABCDEF"[(b >> 4) as usize]));
out.push(char::from(b"0123456789ABCDEF"[(b & 0xf) as usize]));
}
}
}
out
}
/// Extract hostname from a URL string without a URL parser.
fn extract_hostname(url: &str) -> Option<String> {
let after_scheme = url
.strip_prefix("https://")
.or_else(|| url.strip_prefix("http://"))?;
let host = after_scheme.split('/').next()?;
let host = host.split(':').next()?; // strip port
if host.is_empty() {
None
} else {
Some(host.to_string())
}
}
/// Validate a 2-letter language code (e.g. "en", "de").
fn is_valid_lang_code(s: &str) -> bool {
s.len() == 2 && s.bytes().all(|b| b.is_ascii_lowercase())
}
/// Validate a 2-letter country code (e.g. "US", "DE").
fn is_valid_country_code(s: &str) -> bool {
s.len() == 2 && s.bytes().all(|b| b.is_ascii_uppercase())
}
/// Validate a UI locale string (e.g. "en-US").
fn is_valid_ui_lang(s: &str) -> bool {
let mut parts = s.split('-');
if let (Some(lang), Some(country), None) = (parts.next(), parts.next(), parts.next()) {
is_valid_lang_code(lang) && is_valid_country_code(country)
} else {
false
}
}
/// Validate a freshness filter value.
fn is_valid_freshness(s: &str) -> bool {
matches!(s, "pd" | "pw" | "pm" | "py") || is_valid_date_range(s)
}
/// Check if the string is a valid date range like "2024-01-01to2024-12-31".
fn is_valid_date_range(s: &str) -> bool {
if let Some((start, end)) = s.split_once("to") {
is_date_like(start) && is_date_like(end)
} else {
false
}
}
/// Basic check for YYYY-MM-DD format.
fn is_date_like(s: &str) -> bool {
s.len() == 10
&& s.as_bytes().get(4) == Some(&b'-')
&& s.as_bytes().get(7) == Some(&b'-')
&& s.bytes()
.enumerate()
.all(|(i, b)| i == 4 || i == 7 || b.is_ascii_digit())
}
const SCHEMA: &str = r#"{
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query to look up on the web"
},
"count": {
"type": "integer",
"description": "Number of results to return (1-20, default 5)",
"minimum": 1,
"maximum": 20,
"default": 5
},
"country": {
"type": "string",
"description": "2-letter uppercase country code to bias results (e.g. 'US', 'DE', 'JP')"
},
"search_lang": {
"type": "string",
"description": "2-letter lowercase language code for search results (e.g. 'en', 'de', 'fr')"
},
"ui_lang": {
"type": "string",
"description": "Locale in language-region format (e.g. 'en-US', 'de-DE')"
},
"freshness": {
"type": "string",
"description": "Filter by discovery time: 'pd' (past day), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'"
}
},
"required": ["query"],
"additionalProperties": false
}"#;
export!(WebSearchTool);
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_url_encode() {
assert_eq!(url_encode("hello world"), "hello%20world");
assert_eq!(url_encode("foo&bar=baz"), "foo%26bar%3Dbaz");
assert_eq!(url_encode("simple"), "simple");
}
#[test]
fn test_extract_hostname() {
assert_eq!(
extract_hostname("https://example.com/path"),
Some("example.com".into())
);
assert_eq!(
extract_hostname("https://sub.example.com:8080/path"),
Some("sub.example.com".into())
);
assert_eq!(
extract_hostname("http://example.com"),
Some("example.com".into())
);
assert_eq!(extract_hostname("not-a-url"), None);
}
#[test]
fn test_is_valid_lang_code() {
assert!(is_valid_lang_code("en"));
assert!(is_valid_lang_code("de"));
assert!(!is_valid_lang_code("EN")); // must be lowercase
assert!(!is_valid_lang_code("eng")); // too long
assert!(!is_valid_lang_code("")); // empty
}
#[test]
fn test_is_valid_country_code() {
assert!(is_valid_country_code("US"));
assert!(is_valid_country_code("DE"));
assert!(!is_valid_country_code("us")); // must be uppercase
assert!(!is_valid_country_code("USA")); // too long
}
#[test]
fn test_is_valid_ui_lang() {
assert!(is_valid_ui_lang("en-US"));
assert!(is_valid_ui_lang("de-DE"));
assert!(!is_valid_ui_lang("en"));
assert!(!is_valid_ui_lang("EN-US")); // lang part must be lowercase
assert!(!is_valid_ui_lang("en-us")); // country part must be uppercase
}
#[test]
fn test_is_valid_freshness() {
assert!(is_valid_freshness("pd"));
assert!(is_valid_freshness("pw"));
assert!(is_valid_freshness("pm"));
assert!(is_valid_freshness("py"));
assert!(is_valid_freshness("2024-01-01to2024-12-31"));
assert!(!is_valid_freshness("invalid"));
assert!(!is_valid_freshness("2024-01-01")); // missing end date
}
#[test]
fn test_is_date_like() {
assert!(is_date_like("2024-01-15"));
assert!(is_date_like("2025-12-31"));
assert!(!is_date_like("2024-1-15")); // not zero-padded
assert!(!is_date_like("24-01-15")); // short year
assert!(!is_date_like("")); // empty
}
#[test]
fn test_build_search_url_minimal() {
let params = SearchParams {
query: "test query".to_string(),
count: None,
country: None,
search_lang: None,
ui_lang: None,
freshness: None,
};
let url = build_search_url("test query", 5, &params);
assert!(url.starts_with(BRAVE_SEARCH_ENDPOINT));
assert!(url.contains("q=test%20query"));
assert!(url.contains("count=5"));
assert!(!url.contains("country="));
}
#[test]
fn test_build_search_url_full() {
let params = SearchParams {
query: "rust programming".to_string(),
count: Some(10),
country: Some("US".to_string()),
search_lang: Some("en".to_string()),
ui_lang: Some("en-US".to_string()),
freshness: Some("pw".to_string()),
};
let url = build_search_url("rust programming", 10, &params);
assert!(url.contains("q=rust%20programming"));
assert!(url.contains("count=10"));
assert!(url.contains("country=US"));
assert!(url.contains("search_lang=en"));
assert!(url.contains("ui_lang=en-US"));
assert!(url.contains("freshness=pw"));
}
#[test]
fn test_url_encode_multibyte() {
assert_eq!(url_encode("café"), "caf%C3%A9");
assert_eq!(url_encode("日本語"), "%E6%97%A5%E6%9C%AC%E8%AA%9E");
}
#[test]
fn test_extract_hostname_empty() {
assert_eq!(extract_hostname("https://"), None);
assert_eq!(extract_hostname("https:///path"), None);
assert_eq!(extract_hostname(""), None);
}
}
@@ -0,0 +1,51 @@
{
"capabilities": {
"http": {
"allowlist": [
{
"host": "api.search.brave.com",
"path_prefix": "/res/v1/web/search",
"methods": [
"GET"
]
}
],
"credentials": {
"brave_api_key": {
"secret_name": "brave_api_key",
"location": {
"type": "header",
"name": "X-Subscription-Token"
},
"host_patterns": [
"api.search.brave.com"
]
}
},
"rate_limit": {
"requests_per_minute": 30,
"requests_per_hour": 500
}
},
"secrets": {
"allowed_names": [
"brave_api_key"
]
}
},
"auth": {
"secret_name": "brave_api_key",
"display_name": "Brave Search",
"instructions": "Get a free API key at brave.com/search/api/ (Free tier: 2,000 queries/month)",
"setup_url": "https://brave.com/search/api/",
"env_var": "BRAVE_API_KEY"
},
"setup": {
"required_secrets": [
{
"name": "brave_api_key",
"prompt": "Brave Search API key (from brave.com/search/api)"
}
]
}
}