Compare commits

..
Author SHA1 Message Date
ZakiandClaude Opus 4.6 bd77b8b14b chore: add __pycache__ and *.pyc to .gitignore
Prevent Python bytecode from being committed. Several open PRs
accidentally included __pycache__/ directories.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-14 11:57:42 -07:00
2b625ef3df fix(registry): bump versions for github, web-search, and discord extensions (#1106)
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-13 19:12:48 +00:00
Henry ParkandGitHub 7d745d5479 tools: improve routine schema guidance (#1089) 2026-03-13 11:24:45 -07:00
Henry ParkandGitHub 1bc10fe4ca test: add event-trigger routine e2e coverage (#1088) 2026-03-13 11:24:25 -07:00
f53c1bb10b fix(mcp): address 14 audit findings across MCP module (#1094)
* fix(mcp): address 14 audit findings across MCP module

- Replace panicking assert! in new_with_config with Result return (Critical)
- Fix initialize() race condition using tokio::sync::OnceCell (High)
- Fix localhost check bypass via proper URL parsing (High)
- Extract shared stream_transport_send() to deduplicate stdio/unix send logic
- Use atomic write (tmp+rename) for config file persistence
- Filter SSE responses by request_id to prevent wrong-response dispatch
- Share a single reqwest::Client for OAuth via fallible OnceLock
- Log notification send errors instead of silently discarding
- Fix unwrap_or(0) that could steal id=0 responses
- Store InitializeResult in OnceCell so callers can access server capabilities
- Add redirect logging in OAuth discovery
- Reuse is_localhost_url() in auth.rs
- Add McpToolWrapper unit tests and regression tests
- URL-encode PKCE challenge for consistency

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: retrigger CI with skip-regression-check label

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-13 17:37:51 +00:00
bc6725205a fix(http): replace .expect() with match in webhook handler (#1133)
* fix(http): replace .expect() with match in webhook handler

Replace `.expect("checked is_none above")` with a proper `match` on
`webhook_secret.as_ref()`. The is_none-then-expect pattern was logically
safe but violates the project rule against .expect() in production code.

Update pre-existing test to expect SERVICE_UNAVAILABLE (503) instead of
UNAUTHORIZED (401) when the secret is cleared, since the None check now
returns early before signature verification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): formatting + suppress no-panics false positive in test

- Collapse multi-line Some() to single line per rustfmt
- Add // safety: comment on test assert_eq to suppress CI grep

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-13 17:01:34 +00:00
Xing JiandGitHub 275bcfb658 fix(time): treat empty timezone string as absent (#1127)
LLMs sometimes pass "" for optional parameters instead of omitting
them. Previously, passing timezone: "" or from_timezone: "" to the
time tool would trigger a parse error ("Unknown timezone ''") rather
than falling back to the context timezone or UTC.

Fix by adding .filter(|s| !s.is_empty()) after .as_str() in
resolve_timezone_for_output and optional_timezone, so empty strings
are treated the same as a missing field.

The same pattern exists in routine.rs (cron trigger timezone and
schedule fields), where "" produces "invalid IANA timezone: ''" or a
cron parse error. That will be addressed separately once routine.rs
has a test harness in place.

Regression tests added for the now and convert operations with
empty timezone strings.

Closes #1127
2026-03-13 16:40:03 +00:00
7776d267f8 ci: enforce no .unwrap(), .expect(), or assert!() in production code (#1087)
Add a diff-based CI job and pre-commit hook check that block
panic-inducing calls (.unwrap(), .expect(), assert!, assert_eq!,
assert_ne!) from entering production Rust code. debug_assert is
excluded (compiled out in release). False positives can be suppressed
with an inline `// safety: <reason>` comment.

- pre-commit-safety.sh: add check 6 (PANIC) for staged diffs
- code_style.yml: add `no-panics` job, wire into roll-up gate
- check-boundaries.sh: extend check 2 to also catch assert!()

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-13 16:36:17 +00:00
e805ec61aa fix: 5 critical/high-priority bugs (auth bypass, relay failures, unbounded recursion, context growth) (#1083)
* fix: address 5 critical and high-priority bugs from issue tracker

- #1033: reject webhook requests when secret is cleared at runtime via
  update_secret(None), preventing auth bypass through SIGHUP hot-swap
- #908: reset consecutive_failures counter on successful SSE stream
  reconnection in relay channel, so circuit breaker counts truly
  consecutive failures
- #975: add depth limit (16) to validate_tool_schema() to prevent
  stack overflow on deeply nested schemas
- #974: add depth limit (8) to resolve_nested() to prevent stack
  overflow on deeply nested capabilities wrappers
- #826: truncate oversized tool outputs (>8KB) in routine lightweight
  loop to prevent unbounded context growth across iterations

Each fix includes a regression test.

Closes #1033, #908, #975, #974, #826

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: 5 more high-priority bugs (routine cache, job signals, input limits)

- #1077: recompute next_fire_at when re-enabling cron routines via web
  toggle, mirroring CLI behavior so cron ticker picks them up
- #1076: refresh event trigger cache after web toggle/delete operations
  so event/system_event routines reflect changes immediately
- #892: remove Stuck from check_signals() stop-states in JobDelegate
  since Stuck is recoverable (Stuck -> InProgress via self-repair)
- #976: truncate oversized description strings in CapabilitiesFile to
  4KB to prevent memory abuse from malicious capabilities files
- #977: drop oversized parameters schema JSON (>64KB) in
  CapabilitiesFile to prevent unbounded memory growth

Each fix includes regression tests where applicable.

Closes #1077, #1076, #892, #976, #977

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: prevent ReDoS in event trigger regex patterns

- #825: use RegexBuilder with 64KB size limit when compiling
  user-supplied event trigger patterns, both at creation time
  (routine tool) and at cache refresh (routine engine)

Note: Rust's regex crate already guarantees O(n) matching, so the
size limit prevents excessive memory use during compilation rather
than catastrophic backtracking at match time.

Closes #825

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Harden HTTP SSRF IP filtering

* Apply rustfmt after staging merge

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-13 16:04:55 +00:00
Henry ParkandGitHub 1e00b1fed5 fix(ci): checkout promotion PR head for metadata refresh (#1097) 2026-03-12 21:32:28 -07:00
+12
Henry ParkGitHubironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com>Nick PismenkovClaude Haiku 4.5Illia PolosukhinXing JiNick StebbingsReidUmesh Kumar Singh智方云cubecloud-iolizicanlizican123Zaki Manianreidliu41Copilotjinxinzwb1982github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>github-actions[bot] <github-actions[bot]@users.noreply.github.com>smkrvSMKRV
5e7758598f chore: periodic sync main into staging (resolved conflicts) (#1098)
* chore: promote staging to main (2026-03-10 15:19 UTC) (#865)

* fix: Channel HTTP: server doesn't start after config change (no hot-r… (#779)

* fix: Channel HTTP: server doesn't start after config change (no hot-reload)

* review fixes

* review fixes

* fix linter

* fix code style

* fix: prevent session lock contention blocking message processing (#783)

* fix: prevent session lock contention blocking message processing

## Problem
After container restart, POST /api/chat/send returns 202 ACCEPTED but messages
don't appear in conversation_messages and agent never responds. Messages get
stuck in "stale state" after restart.

Root cause: Session lock was held for entire duration of chat_threads_handler
and chat_history_handler, including during slow database queries. This blocked
the agent loop from acquiring the session lock to process incoming messages,
causing them to hang indefinitely.

## Solution
1. **Release session lock early in chat_threads_handler**: Only acquire lock
   when reading active_thread at response time, not during DB queries for
   thread list. DB operations no longer block message processing.

2. **Release session lock early in chat_history_handler**: Only acquire lock
   when accessing in-memory thread state, not during paginated DB queries or
   thread ownership checks. DB operations no longer block message processing.

3. **Add comprehensive logging**: Track message flow from receipt through
   session resolution, thread hydration, and state transitions. Helps diagnose
   future issues:
   - Message queued to agent loop (chat_send_handler)
   - Processing message from channel (handle_message)
   - Hydrating thread from DB (maybe_hydrate_thread)
   - Resolving session and thread (resolve_thread)
   - Checking thread state (process_user_input)
   - Persisting user message (persist_user_message)

## Impact
- Message processing no longer blocks on session lock contention
- API response times for thread list/history queries unaffected (DB queries
  still happen, but lock is not held)
- Better diagnostics for future debugging

## Testing
- All 2756 tests pass
- Code compiles with zero clippy warnings
- No changes to user-facing API or behavior, only lock timing

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* security: redact PII from info-level logs

Downgrade user_id and channel logging to debug level to prevent exposing
Personally Identifiable Information (PII) in production logs.

The user_id field can contain sensitive information such as phone numbers
(e.g., for Signal messages). Logging PII in cleartext at the info level
creates a security and privacy risk, as these logs may be stored in
persistent storage, indexed by log management systems, or accessible to
unauthorized personnel.

Changes:
- Info level: logs only message_id (UUID) for tracking
- Debug level: logs user_id, channel, thread_id for troubleshooting

This maintains debugging capability for developers while protecting user
privacy in production logs.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

---------

Co-authored-by: Claude Haiku 4.5 <[email protected]>

* chore: sync main into staging (#855)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)

Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).

- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini

Co-authored-by: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>

* fix: Chat input is hidden in mobile browser mode (#877)

* fix: stop XML-escaping tool output content (#598) (#874)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)

Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).

- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix: stop XML-escaping tool output content in wrap_for_llm (#598)

Remove content escaping that corrupted JSON in tool output. The
<tool_output> structural boundary is preserved but content now passes
through raw, fixing downstream parse failures.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(safety): allow empty string tool params (#848)

* fix(safety): allow empty string tool params

* fix(safety): preserve heuristic checks and add path context to tool validation

This follow-up refactor addresses PR review feedback by restoring
heuristic checks (whitespace ratio, character repetition) for tool
parameter validation and improving error reporting.

Changes:
- Restored heuristic warnings in validate_non_empty_input so they apply
  to both user input and tool parameters (when non-empty).
- Refactored check_strings to recursively build and pass JSON paths
  (e.g., "metadata.tags[1]").
- Updated validation errors to use the specific JSON path as the field
  name instead of the generic "input".
- Added regression tests for whitespace/repetition warnings and JSON
  path reporting in tool parameters.

This ensures the safety layer remains semantically neutral about empty
strings (fixing the memory_tree path: "" issue) while maintaining
rigorous protection and providing better developer ergonomics.

* style: run cargo fmt

* perf: optimize release and dist build profiles (#843)

* perf: optimize release and dist build profiles

Add [profile.release] with strip=true and panic="abort" for smaller,
faster release binaries. Upgrade [profile.dist] from lto="thin" to
lto="fat" with codegen-units=1 for maximum optimization in CI releases.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove panic=abort from release profile

Reviewers (zmanian, Copilot, Gemini) correctly flagged that panic=abort
in the release profile would kill the entire process on any tokio task
panic, breaking fault isolation for the long-running server. Removed
from release profile entirely.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: add PR template with risk assessment (#837)

* feat: add PR template with risk assessment and review tracks

Add a pull request template that includes summary, change type,
validation checklist, security/database impact sections, blast radius,
and rollback plan. Update CONTRIBUTING.md with review track definitions
(A/B/C) based on change risk level.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: expand CONTRIBUTING.md with setup, workflow, and guidelines

Add getting started, development workflow, code style summary,
database change guidance, and dependency management sections.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: add fuzzing targets for untrusted input parsers (#835)

* feat: add fuzzing targets for untrusted input parsers

Add cargo-fuzz infrastructure with 5 fuzz targets exercising
security-critical code paths:

- fuzz_safety_sanitizer: Aho-Corasick + regex injection detection
- fuzz_safety_validator: Input validation (length, encoding, patterns)
- fuzz_leak_detector: Secret leak scanning (API keys, tokens)
- fuzz_tool_params: Tool parameter JSON validation
- fuzz_config_env: TOML/JSON config parsing

Each target exercises real IronClaw business logic with invariant
assertions. Includes corpus directories and setup documentation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: improve fuzz targets to exercise real IronClaw code paths

- fuzz_config_env: exercise SafetyLayer end-to-end (sanitize, validate,
  policy check) instead of generic TOML/JSON parsing
- fuzz_tool_params: add validate_tool_schema coverage alongside
  validate_tool_params
- Add "fuzz" to workspace exclude in root Cargo.toml
- Update README descriptions to match actual target behavior

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace redundant detect() call with meaningful invariant assertion

Replace the double sanitize()+detect() call with an assertion that
critical severity warnings always trigger content modification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: rewrite fuzz_config_env to exercise IronClaw safety code directly

Replace SafetyLayer wrapper usage with direct Sanitizer, Validator, and
LeakDetector instantiation and invocation. Adds meaningful consistency
assertions (non-empty output, valid-means-no-errors, scan/clean agreement).
Removes the config construction that was only exercising struct instantiation.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(wasm): run leak scan before credential injection in tools wrapper (#791)

* fix(wasm): run leak scan before credential injection in tools wrapper

The tools WASM wrapper runs the LeakDetector on HTTP request headers
AFTER inject_host_credentials() has already substituted real secrets
(e.g., xoxb- Slack bot tokens). This causes the leak detector to
flag the tool's own legitimate outbound API calls as secret exfiltration.

Move the scan to run on raw_headers before any credential injection,
matching the fix already applied to the channels wrapper in #421.

Fixes the same class of bug as #421 (which only fixed channels/wasm/wrapper.rs).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* perf: inline leak scan to avoid Vec allocation on every HTTP request

Address review feedback: instead of cloning all header keys/values into
a Vec to pass to scan_http_request(), iterate over raw_headers directly
using scan_and_clean(). This also provides more specific error messages
(URL vs header vs body).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix cargo fmt formatting in leak scan loop

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(setup): drain residual terminal events before secret input (#747) (#849)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)

Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).

- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix: skip the regression check
[skip-regression-check]

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>

* feat(agent): add context size logging before LLM prompt (#810)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat(agent): add context size logging before LLM prompt

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>

* fix: preserve text before tool-call XML in forced-text responses (#852)

* fix: preserve text before tool-call XML in forced-text responses (#789)

Local models (Qwen3, DeepSeek, GLM) emit <tool_call> XML even when no
tools are available (force_text mode). The existing strip_xml_tag()
discards everything from an unclosed opening tag onward, producing an
empty string that triggers the "I'm not sure how to respond" fallback.

Add truncate_at_tool_tags() — a code-region-aware pre-processing step
that truncates at the first tool-call XML tag BEFORE clean_response()
runs, preserving all useful text before the tag. Protect all 7
clean_response() call sites. Case-insensitive matching handles models
that emit <TOOL_CALL> or <Tool_Call> variants.

Secondary fix: add has_native_thinking() model detection to skip
<think>/<final> system prompt injection for models with built-in
reasoning (Qwen3, QwQ, DeepSeek-R1, GLM-Z1, etc.), preventing
thinking-only responses that clean to empty.

Wire with_model_name(active_model_name()) at all 9 production sites
that construct Reasoning, so the runtime model name (not static config)
drives system prompt generation.

126 new/updated tests covering truncation edge cases, code-block
awareness, Unicode, case-insensitivity, StubLlm integration for
complete/plan/evaluate_success/respond_with_tools paths, model
detection, and conditional system prompt generation.

Closes #789

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address Copilot review — unclosed-only truncation, ASCII case folding

- truncate_at_tool_tags() now only truncates at UNCLOSED tool tags;
  properly closed tags (e.g. <tool_call>...</tool_call>) are left intact
  for clean_response() to strip normally, preserving any text after them
- Switch from to_lowercase() to to_ascii_lowercase() to prevent byte
  offset misalignment with non-ASCII characters whose lowercase form
  has different byte length (e.g. Kelvin sign U+212A)
- Add closing_tag_for() helper to derive closing tags from open patterns
- Fix doc comment: "fenced markdown code blocks or inline code spans"
  (not "indented", which find_code_regions() doesn't detect)
- Add regression tests: closed vs unclosed for each tag variant,
  Unicode + case-insensitive offset safety, and mixed closed/unclosed

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: minor review items — consistent ascii_lowercase, closing_tag_for tests

- Switch has_native_thinking() from to_lowercase() to to_ascii_lowercase()
  for consistency with truncate_at_tool_tags() approach
- Add unit tests for closing_tag_for(): standard tags, space-suffixed
  patterns, pipe-delimited tags, and exhaustive coverage of all
  TOOL_TAG_PATTERNS entries
- Add test for mixed closed+unclosed tags of different types

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* Feat/docker shell edition (#804)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(mcp): strip top-level null params before forwarding to MCP servers (#795)

* feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809)

Add declarative `unsupported_params` field to provider definitions in
providers.json. Parameters listed are stripped from requests before
sending, preventing 400 errors from providers that reject them (e.g.
gpt-5 family and kimi-k2.5 rejecting custom temperature values).

- Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig
- Propagate from registry through config resolution
- Generic strip helpers handle temperature, max_tokens, stop_sequences
- Apply filtering in RigAdapter and AnthropicOAuthProvider
- Mark openai and tinfoil providers as unsupporting temperature
- Update openai default model to gpt-5-mini

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(mcp): strip top-level null params before forwarding to MCP servers

LLMs frequently emit `"field": null` for optional parameters in tool
calls. Many MCP servers reject explicit nulls for fields that should
simply be absent — e.g. Notion returns 400 for `"sort": null` in a
search call, expecting the field to be omitted entirely.

Strip top-level null keys from the params object before calling
`call_tool()`. Only top-level keys are stripped; nested nulls are
preserved since they may be semantically meaningful.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>

* Add event-triggered routines and workflow skill templates (#756)

* Add event-triggered routines and workflow skill templates

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback for event_emit security and quality

Security fixes:
- Require approval (UnlessAutoApproved) for event_emit, matching routine_fire
- Enable sanitization on event_emit payload (external JSON reaches LLM)
- Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id

Correctness fixes:
- Rename source → event_source in event_emit for consistency with routine_create
- Use json_value_as_filter_string for filter parsing (handles numbers/booleans)
- Case-insensitive matching for event source and event_type
- Add debug logging for missing filter keys in payload
- Fix skill_install_routine_webhook_sim test missing .with_skills()
- Fix schema_validator test for event_emit payload properties

Code quality:
- Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout)
- Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs
- Add test section headers in e2e_routine_heartbeat.rs
- Clarify event_emit description to specify system_event routines only

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix: make routine_system_event_emit test create routine before emitting

- Add routine_create step to trace fixture so event_emit has a matching
  routine to fire
- Assert fired_routines > 0, not just key presence (Copilot review)
- Add .with_auto_approve_tools(true) since event_emit now requires approval

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: renumber test headers after system_event test insertion

Test 4 was duplicated (routine_cooldown and heartbeat_findings).
Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: merge staging and add missing RoutineEngine args in test

RoutineEngine::new on staging requires `tools` and `safety` params.
Update system_event_trigger_matches_and_filters test to pass them.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address new Copilot review comments

- Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim
  test so event_emit doesn't block on approval
- Fix module-level doc comment for event_emit to specify system_event trigger

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate json_value_as_string helper

Remove private `json_value_as_string` from routine_engine.rs and use
the identical public `json_value_as_filter_string` from routine.rs,
eliminating divergence risk. (Copilot review)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix: enable WASM credential injection in No-DB environments (#845)

* fix(wasm): enable credential injection in no-DB environments via env var fallback

When a secrets store is unavailable (e.g. no-DB mode), WASM channel
credentials were silently not injected, causing channels to start without
credentials. Fix by:

- Changing `inject_channel_credentials_from_secrets` to accept
  `Option<&dyn SecretsStore>` — secrets store is tried first when present
- Adding env var fallback (`inject_env_credentials`) for credentials not
  covered by the secrets store
- Enforcing a channel-name prefix security check on env var names to
  prevent WASM channels from reading unrelated host credentials
  (e.g. `AWS_SECRET_ACCESS_KEY`)
- Extracting pure `resolve_env_credentials` helper for testability
- Adding case-insensitive prefix matching for secrets store lookup

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(wasm): inject credentials at startup when no secrets store (setup.rs path)

The startup path (setup_wasm_channels -> register_channel) was guarded by
`if let Some(secrets) = secrets_store`, so in No-DB mode credentials were
never injected and the channel started without them.

Fix by:
- Changing inject_channel_credentials to accept Option<&dyn SecretsStore>
- Always calling it (removing the if-let guard) — env var fallback runs
  even when secrets_store is None
- Adding channel-name prefix security check to the env var fallback path
  (e.g. TELEGRAM_ for channel "telegram"), consistent with manager.rs

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(test): correct misleading comment on ICTEST1_UNRELATED_OTHER placeholder

* fix(wasm): guard against empty channel name in credential injection

An empty channel_name would produce prefix "_", allowing any env var
starting with "_" to pass the security check and be injected. Add an
early-return guard in resolve_env_credentials, inject_env_credentials,
and inject_channel_credentials. Add a test to cover this path.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: lizican123 <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix: promote to main (#878)

* fix: replace unsafe env::set_var with thread-safe inject_single_var in SIGHUP handler

Fixes race condition where SIGHUP handler modifies global environment variables
while other threads may be reading them via Config::from_env().

Changes:
- Replace unsafe { std::env::set_var() } with ironclaw::config::inject_single_var()
- Uses INJECTED_VARS mutex instead of unsafe global state modification
- All reads via optional_env() check the thread-safe overlay first
- Prevents data races between SIGHUP reload and concurrent config reads

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* fix: spawn webhook restart as background task to avoid blocking I/O across lock

Prevents holding Mutex lock during async I/O operations (TcpListener::bind,
task shutdown). The SIGHUP handler no longer blocks webhook processing during
listener restart.

Changes:
- Read old_addr and drop lock immediately
- Spawn restart_with_addr() as background task via tokio::spawn
- Lock is only held during the actual restart operation, not the signal handler

Benefits:
- SIGHUP handler returns immediately without blocking
- Webhook requests not delayed by listener restart I/O
- Lock contention significantly reduced

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* fix: add graceful shutdown mechanism for SIGHUP handler background task

Prevents unbounded loop without cancellation token. The SIGHUP handler now
listens for a shutdown signal and exits cleanly during graceful termination.

Changes:
- Create broadcast channel for shutdown signaling
- SIGHUP handler uses tokio::select! to wait for shutdown or SIGHUP
- Send shutdown signal to all background tasks after agent.run() completes
- Ensures clean task lifecycle and no orphaned background tasks

Benefits:
- Proper task cancellation during graceful shutdown
- Follows Tokio best practices for background task management
- No background tasks orphaned when runtime shuts down

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* refactor: replace stringly-typed parameter filtering with typed enum and single helper

Fixes DRY violation where unsupported parameter filtering was duplicated across
rig_adapter.rs and anthropic_oauth.rs using string contains checks.

Changes:
- Add UnsupportedParam typed enum in provider.rs (Temperature, MaxTokens, StopSequences)
- Create strip_unsupported_completion_params() helper function
- Create strip_unsupported_tool_params() helper function
- Update rig_adapter.rs to use shared helpers
- Update anthropic_oauth.rs to use shared helpers
- Replace 60+ lines of duplicate stringly-typed logic

Benefits:
- Type safety: parameter names checked at compile time
- Single source of truth: adding a new param updates one place
- Reduced maintenance burden: no duplicate logic to keep in sync
- Better code clarity: named enum variant is self-documenting

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* docs: clarify intentional parameter asymmetry between completion and tool requests

Add documentation explaining why strip_unsupported_tool_params does not handle
StopSequences: the field doesn't exist in ToolCompletionRequest.

Changes:
- Add clarifying comments to strip_unsupported_tool_params()
- Explain why StopSequences is only in CompletionRequest
- Note that ToolCompletionRequest only supports Temperature and MaxTokens
- Inline comment confirms no action needed for StopSequences

This addresses the appearance of incomplete implementation without changing logic,
as the asymmetry is intentional and correct (ToolCompletionRequest lacks the field).

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* perf: isolate webhook_secret to reduce lock contention on hot path

Move webhook_secret from shared HttpChannelState RwLock into its own Arc<RwLock<>>.
This eliminates contention between secret validation and other state operations.

Changes:
- Change webhook_secret field type from RwLock<Option<SecretString>> to Arc<RwLock<Option<SecretString>>>
- Update initialization in HttpChannel::new()
- Update comments to explain isolation rationale

Benefits:
- Reduce lock contention on webhook request hot path (secret validation)
- Rarely-changing field (SIGHUP only) isolated from frequent state accesses
- Other state operations (tx, pending_responses) no longer wait behind secret reads
- Minimal code change: only field declaration and initialization

The Arc wrapper allows cloning the RwLock handle to separate concerns. With this
change, every webhook request acquires its own isolated lock for secret validation,
not the shared HttpChannelState lock. This scales better under high request volume.

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* fix: prevent partial state corruption on SIGHUP restart failure

Ensure atomicity of configuration reload: if webhook listener restart fails,
secret update is skipped to prevent inconsistent state.

Changes:
- Wait for restart_with_addr() to complete (don't spawn background task)
- Track restart result with restart_failed flag
- Only update secret if restart succeeded or wasn't needed
- Ensure listener and secret stay synchronized

Problem addressed:
- Before: restart spawned as background task, secret updated immediately
- If restart failed, secret was changed but listener still on old address
- This left system in inconsistent state (partial corruption)

Solution:
- Make restart blocking (SIGHUP handler can wait, it's not on request hot path)
- Atomically update secret only after successful restart
- Flag prevents race between restart and secret update

Benefits:
- Configuration changes are atomic (both succeed or both fail together)
- No partial state corruption on restart failure
- Failed restarts don't silently leave inconsistent state
- Secret and listener address stay in sync

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* refactor: generalize hot-secret-swapping with ChannelSecretUpdater trait

Decouple SIGHUP handler from HTTP channel internals by introducing a trait
for channels that support zero-downtime secret updates.

Changes:
- Add ChannelSecretUpdater trait in channels/channel.rs
- Implement ChannelSecretUpdater for HttpChannelState
- Export trait from channels module
- Update SIGHUP handler to use trait-based secret updater collection
- Replace explicit HTTP channel knowledge with generic updater loop

Benefits:
- SIGHUP handler no longer depends on HttpChannelState details
- Tight coupling removed: main.rs doesn't need HTTP channel imports
- Extensible: new channels can opt-in by implementing the trait
- Scalable: multiple channels supported without main.rs changes
- Maintainable: adding channels requires only trait implementation, not SIGHUP handler edits

Pattern:
- ChannelSecretUpdater trait defines the interface for all updaters
- Channels that support hot-secret-swapping implement the trait
- SIGHUP handler loops through all registered updaters generically

Verification:
- All 2,787 tests pass
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* feat: validate parameter names at deserialization time, not just tests

Add custom serde deserializer for unsupported_params that validates parameter
names at runtime when loading providers.json (or user overrides).

Changes:
- Add unsupported_params_de module with custom deserializer
- Only allows: "temperature", "max_tokens", "stop_sequences"
- Invalid parameter names cause immediate deserialization error
- Update ProviderDefinition to use custom deserializer
- Enhanced test with explicit parameter name validation
- Add new test that verifies invalid parameters are rejected

Problem solved:
- Before: Invalid param names (e.g., "temperrature") silently ignored
- Now: Rejected at deserialization time with clear error message
- Prevents runtime failures caused by typos in configuration

Example error:
  unsupported parameter name 'temperrature': must be one of: temperature, max_tokens, stop_sequences

Benefits:
- Fail-fast: errors caught when loading config, not at runtime
- Clear feedback: error message lists valid parameter names
- Type safety: validators run during deserialization
- Configuration errors detected immediately, not silently ignored

Verification:
- All 2,788 tests pass (including new validation test)
- Zero clippy warnings
- Code compiles successfully

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

---------

Co-authored-by: Claude Haiku 4.5 <[email protected]>

* merge: resolve conflicts for PR #800 and #822 into staging (#881)

* fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787)

GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>

* fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794)

- Remove continue-on-error from staging-ci.yml app token steps (secrets are configured)
- Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml
  already runs tests before promoting, promotion PR gets full CI on main)
- Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802)

- Remove branches:[main] filter from code_style.yml so it runs on all PRs
- Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs)
- Update rollup job to allow skipped clippy-windows
- Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs)

Co-authored-by: Claude Opus 4.6 <[email protected]>

* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* refactor: unify three agentic loops into single AgenticLoop engine (#654)

Replace three independent copy-pasted agentic loops (dispatcher, worker,
container runtime) with a single shared engine in `agentic_loop.rs` that
all consumers customize via the `LoopDelegate` trait.

Phase 1 — Shared engine (`src/agent/agentic_loop.rs`, 205 lines):
  - `run_agentic_loop()` owns the core LLM → tool exec → repeat cycle
  - `LoopDelegate` trait (Send + Sync, &dyn dispatch) with 6 hook points
  - Tool intent nudge logic consolidated (was duplicated in 3 files)
  - Iteration limit + force-text behavior preserved

Phase 2 — Three delegate implementations:
  - `ChatDelegate` (dispatcher.rs): 3-phase approval flow, hooks, cost
    guard, context compaction, skill attenuation, interruption
  - `JobDelegate` (worker/job.rs): planning pre-loop phase, parallel
    JoinSet exec, mark_completed/stuck/failed, SSE streaming, self-repair
  - `ContainerDelegate` (worker/container.rs): sequential tool exec,
    HTTP-proxied LLM, container-safe tools, credential injection

Phase 3 — File moves and cleanup:
  - Delete `src/agent/worker.rs` — job logic moved to `src/worker/job.rs`
  - Rename `src/worker/runtime.rs` → `src/worker/container.rs`
  - Re-export `Worker`/`WorkerDeps` from `crate::worker` in `agent/mod.rs`
  - Update `scheduler.rs` imports to new worker location

Shared helpers (`src/tools/execute.rs`):
  - `execute_tool_with_safety()` replaces 4 copies of validate → timeout
    → execute → serialize
  - `process_tool_result()` replaces 3 copies of sanitize → wrap →
    ChatMessage (also used by thread_ops.rs approval resume paths)

Net result: -2,408 lines, zero duplicated loop logic, single code path
for tool intent nudge and completion detection.

Closes #654

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback from Copilot

1. scheduler.rs: Replace `unwrap_or` fallback with proper error
   propagation when parsing tool output JSON — surfaces bugs instead
   of silently changing the output type.

2. worker/job.rs: Drop MutexGuard before the cancellation `.await` in
   `check_signals()` to avoid holding a lock across an async I/O call
   (prevents `await_holding_lock` lint).

3. worker/job.rs: Restore consecutive rate-limit counter
   (MAX_CONSECUTIVE_RATE_LIMITS = 10) so sustained rate limiting marks
   the job stuck with "Persistent rate limiting" instead of silently
   burning through max_iterations.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: incorporate staging changes — token budget tracking + mark_failed

Merge staging's changes into the refactored JobDelegate:
- Add token budget tracking in call_llm (update_context/add_tokens)
- mark_stuck → mark_failed for iteration cap and rate-limit exhaustion
  (aligns with staging's #788 fix)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address zmanian's PR review — eliminate type erasure, clean up

Address all 6 review points from zmanian on PR #800:

1. Replace LoopOutcome::Custom(Box<dyn Any>) with typed
   LoopOutcome::NeedApproval(Box<PendingApproval>) — eliminates
   type erasure and downcast, resolves clippy large_enum_variant.

2. Remove dead max_tool_iterations field from ChatDelegate struct.

3. Add on_tool_intent_nudge() hook to LoopDelegate trait with
   implementations in Job and Container delegates for observability.

4. Fix SSE events in job worker to emit raw sanitized content
   instead of XML-wrapped <tool_output> tags.

5. Remove 4 duplicate completion tests from job.rs that were
   already covered by the shared util module.

6. Avoid logging full tool results — use result_size_bytes in
   debug logs (execute.rs, job.rs).

Also updates path references in CLAUDE.md, COVERAGE_PLAN.md,
and add-sse-event.md command.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(doctor): expand diagnostics from 7 to 16 health checks

* test: add unit tests for agentic_loop and execute shared modules

Add 16 tests covering the two new critical shared modules:

agentic_loop.rs (10 tests):
- Text response exits loop immediately
- Tool call → text response continuation
- LoopSignal::Stop exits before LLM call
- LoopSignal::InjectMessage adds user message to context
- Max iterations terminates with LoopOutcome::MaxIterations
- Tool intent nudge fires twice then caps
- before_llm_call early exit bypasses LLM
- truncate_for_preview: short string, long string, multibyte safety

execute.rs (6 tests):
- execute_tool_with_safety success path
- Missing tool returns ToolError::NotFound
- Tool execution failure propagates
- Per-tool timeout enforcement (50ms)
- process_tool_result XML wrapping on success
- process_tool_result error formatting

All 2,777 unit tests pass, 0 clippy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address code review — 9 issues across agentic loop, job worker, container

CRITICAL fixes:
- Rate-limit exhaustion now returns Err(LlmError::RateLimited) instead of
  Ok(Text("")), stopping the loop immediately with no ghost iteration.
  Below-threshold retries still use Text("") with an explicit empty-string
  guard in handle_text_response to skip injection.
- check_signals drains the entire message channel before returning,
  prioritizing Stop over UserMessage. Previously returned early on first
  UserMessage, silently dropping any queued Stop or additional messages.
- check_signals now detects all non-progressing job states (Cancelled,
  Failed, Stuck, Completed, Submitted, Accepted) instead of only
  Cancelled and Failed.

HIGH fixes:
- Error path in process_tool_result_job applies truncate_for_preview to
  bound error strings in SSE/DB events (was unbounded).
- Document Send+Sync lifetime constraint on LoopDelegate trait.
- Test mock before_llm_call refactored from double-lock to single lock
  acquisition, eliminating deadlock risk on refactor.

MEDIUM fixes:
- CompletionReport includes actual iteration count via shared
  Arc<Mutex<u32>> tracker (was hardcoded 0).
- process_tool_result_job return type changed from Result<bool> to
  Result<()> — the bool was always false (dead API).
- Deduplicate truncate in container.rs; now uses truncate_for_preview
  from agentic_loop.

Verified: 0 clippy warnings, 2781 tests pass, cargo fmt clean.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Henry Park <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: Umesh Kumar Singh <[email protected]>
Co-authored-by: reidliu41 <[email protected]>

* Revert "Feat/docker shell edition" + fix fmt/clippy (#886)

* Revert "Feat/docker shell edition (#804)"

This reverts commit c566faf28f.

* style: fix formatting issues from revert

Run cargo fmt to fix formatting across 7 files after the revert of
the docker shell edition feature.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* refactor: centralize test credential constants into testing::credentials (#829)

* refactor: central…

* feat(i18n): Add internationalization support with Chinese and English translations (#929) (#950)

* feat(i18n): Add internationalization support with Chinese and English translations
* fix(i18n): fix duplicate keys, broken placeholders, and dead overrides

---------

Co-authored-by: jinxin <[email protected]>
Co-authored-by: zwb1982 <[email protected]>

* chore: release v0.18.0 (#885)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* chore: update WASM artifact SHA256 checksums [skip ci] (#954)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* feat(embeddings): add EMBEDDING_BASE_URL for OpenAI-compatible embedding providers (#1082)

* feat(embeddings): add EMBEDDING_BASE_URL for OpenAI-compatible embedding providers

Allow configuring a custom base URL for OpenAI-compatible embedding
endpoints (e.g., Azure OpenAI, local proxies, vLLM) via the
EMBEDDING_BASE_URL environment variable. When unset, defaults to
https://api.openai.com.

Changes:
- Extract hardcoded OpenAI URL to OPENAI_API_BASE_URL constant
- Add base_url field to OpenAiEmbeddings with builder method with_base_url()
- Auto-prepend https:// for schemeless URLs, strip trailing slashes
- Add openai_base_url field to EmbeddingsConfig, parsed from EMBEDDING_BASE_URL
- Wire base URL through create_provider() with debug logging
- Add EMBEDDING_BASE_URL to clear_embedding_env() in tests
- Add unit tests for URL validation and env var parsing

* refactor: address Gemini review — in-place trailing slash strip, simplify config logic

- Use while/pop() instead of trim_end_matches().to_string() for zero
  extra allocation when stripping trailing slashes in with_base_url()
- Remove double openai_base_url check in create_provider() — create
  provider first, then branch on base_url for logging + configuration

---------

Co-authored-by: SMKRV <[email protected]>

---------

Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com>
Co-authored-by: Nick Pismenkov <[email protected]>
Co-authored-by: Claude Haiku 4.5 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: Xing Ji <[email protected]>
Co-authored-by: Nick Stebbings <[email protected]>
Co-authored-by: Reid <[email protected]>
Co-authored-by: Umesh Kumar Singh <[email protected]>
Co-authored-by: 智方云cubecloud-io <[email protected]>
Co-authored-by: lizican <[email protected]>
Co-authored-by: lizican123 <[email protected]>
Co-authored-by: Zaki Manian <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: jinxin <[email protected]>
Co-authored-by: zwb1982 <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: smkrv <[email protected]>
Co-authored-by: SMKRV <[email protected]>
2026-03-12 21:32:19 -07:00
c47237b9c7 fix(ci): add missing attachments field and crates/ dir to Dockerfiles (#1100)
The discord channel's poll_channel_mentions emit_message call was missing
the required `attachments: vec![]` field, causing WASM compilation failure.
Both Dockerfiles were also missing `COPY crates/ crates/` needed for the
extracted ironclaw_safety crate.

[skip-regression-check]

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-12 21:13:36 -07:00
a89cf37993 fix(registry): bump telegram channel version for capabilities change (#1064)
The validation_endpoint addition to telegram.capabilities.json requires
a version bump to pass the CI version-check gate on staging promotion.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-12 21:02:00 -07:00
Henry ParkandGitHub 3c619b6272 fix(ci): repair staging promotion workflow behavior (#1091)
* fix(ci): repair staging-ci workflow parsing

* fix(ci): chain staging promotion to latest open branch

* feat(ci): carry staging batch summaries into release PRs

* test(ci): add dry-run dispatch for promotion metadata workflows

* fix(ci): fetch only release tags for batch summaries

* fix(ci): address review feedback on batch summaries

* fix(ci): harden metadata workflows and dedupe body helpers

* fix(ci): pass repo explicitly to gh pr list
2026-03-12 20:34:27 -07:00
15c5d3e2e2 fix(wasm): address #1086 review followups -- description hint and coercion safety (#1092)
Two fixes from the review of #1086 (tool_info schema discovery):

1. Replace fragile description string mutation (append_schema_hint_if_permissive /
   strip_schema_hint) with composition at display time. The raw description stays
   clean; the tool_info hint is composed in the Tool::schema() override only when
   the advertised schema is permissive. This also includes the tool name and
   `include_schema: true` in the hint for better LLM guidance.

2. Make effective_for_coercion use the load-time extracted schema from
   PreparedModule instead of re-calling the WASM schema() export on the
   already-running instance mid-execution. This avoids potential state
   contamination from calling schema() after linear memory is initialized
   for execution.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-12 17:48:05 -07:00
Henry ParkandGitHub cd1245afc0 fix(ci): repair staging-ci workflow parsing (#1090) 2026-03-12 16:58:43 -07:00
9fbdd42988 fix(extensions): fix lifecycle bugs + comprehensive E2E tests (#1070)
* feat(extensions): unify auth and configure into single entrypoint

Refactors the extension lifecycle to eliminate the divergence between
chat and gateway paths that caused Telegram setup via chat to fail
(missing webhook secret auto-generation, no token validation).

Key changes:
- Rename save_setup_secrets() → configure(): single entrypoint for
  providing secrets to any extension (WasmChannel, WasmTool, MCP).
  Validates, stores, auto-generates, and activates.
- Add configure_token(): convenience wrapper for single-token callers
  (chat auth card, WebSocket, agent auth mode).
- Refactor auth() to pure status check: remove token parameter,
  delete token-storing branches from auth_mcp/auth_wasm_tool,
  rename auth_wasm_channel → auth_wasm_channel_status.
- Add ConfigureResult/MissingSecret types for structured responses.
- Replace hardcoded Telegram token validation with generic
  validation_endpoint from capabilities.json.
- Update all callers (9 files) to use the new interface.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: use ValidationFailed error variant instead of string matching

Replace brittle msg.contains("Invalid token") checks with a proper
ExtensionError::ValidationFailed variant. configure() now returns
this variant for token validation failures, and callers match on it
directly instead of parsing error message strings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address review — SSRF protection, error typing, missing-secret selection, WS auth

1. SSRF: call validate_fetch_url() before validation_endpoint HTTP request
2. Transport errors map to ExtensionError::Other (not ValidationFailed)
3. configure_token() picks first *missing* secret, not first non-optional
4. WebSocket error path re-emits AuthRequired on ValidationFailed

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add regression tests for extension lifecycle refactoring

- test_configure_token_picks_first_missing_secret: verifies multi-secret
  channels can be configured one secret at a time (commit ce106f4)
- test_auth_is_read_only_for_wasm_channel: verifies auth() has no side
  effects and doesn't store secrets (commit 47f8eb6)
- test_validation_failed_is_distinct_error_variant: verifies the typed
  error variant can be pattern-matched (commit a318161)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review comments — activation dispatch, dead code, caps consolidation

- Fix configure() fallthrough bug: dispatch activation by ExtensionKind
  instead of unconditionally calling activate_wasm_channel() for all
  non-WasmTool types (MCP servers and channel relays now use their
  correct activation methods)
- Remove dead MissingSecret struct and missing_secrets field (never
  populated, flagged by reviewer)
- Consolidate capabilities file parsing in configure(): parse once
  and reuse for allowed names, validation_endpoint, and auto-generation
- Fix auth() doc comment: note MCP OAuth side effects
- Fix stale save_setup_secrets reference in server.rs comment
- Add regression test for activation dispatch bug

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(extensions): fix 5 extension lifecycle bugs found during E2E testing

Bug fixes in src/extensions/manager.rs:
- Add auth guard to activate_wasm_tool() blocking activation when secrets
  are missing (NeedsSetup), matching activate_wasm_channel() behavior
- Evict WasmToolRuntime module cache on remove() so reinstall uses fresh binary
- Clear activation_errors on remove() for both WasmTool and WasmChannel
- Clean up in-progress OAuth flows on remove() (abort TCP listener, purge
  pending flow entries)

Bug fix in src/channels/web/server.rs:
- Broadcast AuthCompleted SSE event on expired OAuth callback so web UI
  doesn't stay stuck showing "auth required"

E2E test coverage:
- test_wasm_lifecycle.py: 35 tests covering install/configure/activate/
  remove/reinstall lifecycle with regression tests for bugs 1 and 3
- test_extension_oauth.py: 9 tests covering OAuth round-trip flow
- test_tool_execution.py: 5 tests for tool invocation via chat
- test_pairing.py: 4 tests for pairing request lifecycle
- Enhanced conftest.py, helpers.py, mock_llm.py for OAuth mock support

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(web): unify extension auth UX and add lifecycle regressions

* test: fix pending oauth flow fixtures after rebase

* test(e2e): fix playwright route ordering for extensions reloads

* test: address e2e review follow-ups

* test: address remaining PR review comments

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-12 16:36:08 -07:00
Henry ParkandGitHub 8a60fa2d37 fix: add tool_info schema discovery for WASM tools (#1086)
* fix: add tool_info schema discovery for WASM tools

* refactor: simplify WASM schema and hint state

* refactor: store tool_info registry reference as Weak
2026-03-12 16:30:38 -07:00
c7dec64b2d feat(ci): include commit history in staging promotion PRs (#952)
* feat(ci): include commit history in staging promotion PRs and merge commits

Promotion PRs from staging->main previously had opaque bodies showing
only the batch SHA range. Now they enumerate all non-merge commits in
each batch as a flat markdown list, visible both in the PR body and
embedded in the merge commit message via --subject/--body flags.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use unique delimiter for commit_summary output

Replace hardcoded COMMIT_SUMMARY_DELIM with a uuidgen-based delimiter
to prevent theoretical collisions with commit message content.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): use heredoc for PR body to avoid GFM code-block rendering

The inline --body string had 10 leading spaces per line (from YAML
indentation), which GitHub-flavored Markdown renders as a code block.
Move the body into a heredoc variable so content starts at column 0.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): truncate commit list at 50 and include PR number in merge subject

- Cap commit enumeration at 50 entries with a truncation note to avoid
  blowing past GitHub PR body/merge message limits on large batches.
- Prefix merge commit subject with #PR_NUMBER for traceability in git log.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): address review — shell expansion, body-file, uuidgen

1. Replace heredoc with string concatenation to prevent shell expansion
   of commit messages containing $, backticks, or backslashes
2. Use --body-file for merge commit body for robustness
3. Replace uuidgen with date +%s for portability

Addresses: https://github.com/nearai/ironclaw/pull/952#pullrequestreview-3938725460

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-12 15:33:35 -07:00
NigeandGitHub c54f739354 fix: resolve bug_bash UX/logging issues (#1054 #1055 #1058) (#1072)
* fix(web,db): improve auth UX + reduce naive timestamp log noise

* fix(clippy): keep memory test modules at end of file
2026-03-12 15:27:50 -07:00
NigeandGitHub 1ba6a83ca4 fix(http): fail closed when webhook secret is missing at runtime (#1075) 2026-03-12 15:26:40 -07:00
NigeGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
d8bcfe15cf fix(service): set CLI_ENABLED=false in macOS launchd plist (#1079)
* fix(service): set CLI_ENABLED=false in macOS launchd plist

* Update src/service.rs

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-12 15:26:37 -07:00
84 changed files with 4892 additions and 1087 deletions
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
load_commit_summary() {
local range="$1"
local max_commits="${2:-50}"
local commit_list overflow
commit_list="$(git log --oneline --no-merges --reverse "${range}" 2>/dev/null || echo "")"
if [ -n "${commit_list}" ]; then
COMMIT_COUNT="$(printf '%s\n' "${commit_list}" | wc -l | tr -d ' ')"
if [ "${COMMIT_COUNT}" -gt "${max_commits}" ]; then
COMMIT_MD="$(printf '%s\n' "${commit_list}" | head -n "${max_commits}" | sed 's/^/- /')"
overflow=$((COMMIT_COUNT - max_commits))
COMMIT_MD+=$'\n'"- ... and ${overflow} more (see compare view)"
else
COMMIT_MD="$(printf '%s\n' "${commit_list}" | sed 's/^/- /')"
fi
else
COMMIT_COUNT=0
COMMIT_MD="- (no non-merge commits in range)"
fi
}
replace_marked_section() {
local body_file="$1"
local section_file="$2"
local section_start="$3"
local section_end="$4"
local output_file="$5"
if grep -qF "${section_start}" "${body_file}" && grep -qF "${section_end}" "${body_file}"; then
awk -v start="${section_start}" -v end="${section_end}" -v replacement_file="${section_file}" '
BEGIN {
while ((getline line < replacement_file) > 0) {
replacement = replacement line ORS
}
in_block = 0
}
$0 == start {
printf "%s", replacement
in_block = 1
next
}
$0 == end {
in_block = 0
next
}
!in_block {
print
}
' "${body_file}" > "${output_file}"
else
cp "${body_file}" "${output_file}"
if [ -s "${output_file}" ]; then
printf '\n\n' >> "${output_file}"
fi
cat "${section_file}" >> "${output_file}"
fi
}
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
: "${PR_NUMBER:?PR_NUMBER is required}"
: "${REPO:?REPO is required}"
MAIN_BRANCH="${MAIN_BRANCH:-main}"
DRY_RUN="${DRY_RUN:-false}"
SECTION_START="<!-- staging-promotion-release-summary:start -->"
SECTION_END="<!-- staging-promotion-release-summary:end -->"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "${TMP_DIR}"' EXIT
# shellcheck source=.github/scripts/pr-body-utils.sh
source "$(dirname "$0")/pr-body-utils.sh"
gh pr view "${PR_NUMBER}" --repo "${REPO}" --json body > "${TMP_DIR}/pr.json"
jq -r '.body // ""' < "${TMP_DIR}/pr.json" > "${TMP_DIR}/body.md"
git fetch origin "${MAIN_BRANCH}"
git fetch origin "+refs/tags/v*:refs/tags/v*"
LAST_TAG="$(git describe --tags --match 'v*' --abbrev=0 "origin/${MAIN_BRANCH}" 2>/dev/null || true)"
if [ -n "${LAST_TAG}" ]; then
RANGE="${LAST_TAG}..origin/${MAIN_BRANCH}"
HEADER="## Staging promotion batches since ${LAST_TAG}"
EMPTY_MESSAGE="_No structured staging promotion merges found since ${LAST_TAG}._"
else
RANGE="origin/${MAIN_BRANCH}"
HEADER="## Staging promotion batches on ${MAIN_BRANCH}"
EMPTY_MESSAGE="_No structured staging promotion merges found on ${MAIN_BRANCH}._"
fi
{
echo "${SECTION_START}"
echo "${HEADER}"
echo
} > "${TMP_DIR}/section.md"
FOUND_SUMMARY=false
while IFS= read -r sha; do
[ -n "${sha}" ] || continue
BODY="$(git show -s --format=%b "${sha}")"
if ! printf '%s\n' "${BODY}" | grep -q '^staging-promotion-summary-v1$'; then
continue
fi
FOUND_SUMMARY=true
SUBJECT="$(git show -s --format=%s "${sha}")"
PR_REF="$(printf '%s\n' "${BODY}" | sed -n 's/^promotion-pr: //p' | head -n 1)"
COMMIT_COUNT="$(printf '%s\n' "${BODY}" | sed -n 's/^current-commit-count: //p' | head -n 1)"
CURRENT_RANGE="$(printf '%s\n' "${BODY}" | sed -n 's/^current-range: //p' | head -n 1)"
COMMIT_BLOCK="$(printf '%s\n' "${BODY}" | awk 'capture { print } /^Current commits in this promotion \([0-9]+\):$/ { capture = 1 }')"
{
echo "### ${SUBJECT}"
echo
if [ -n "${PR_REF}" ]; then
echo "**Promotion PR:** ${PR_REF}"
fi
if [ -n "${COMMIT_COUNT}" ]; then
echo "**Commit count:** ${COMMIT_COUNT}"
fi
if [ -n "${CURRENT_RANGE}" ]; then
echo "**Range:** \`${CURRENT_RANGE}\`"
fi
echo
if [ -n "${COMMIT_BLOCK}" ]; then
echo "${COMMIT_BLOCK}"
else
echo "- (no commit summary found)"
fi
echo
} >> "${TMP_DIR}/section.md"
done < <(git log --merges --reverse --format='%H' "${RANGE}")
if [ "${FOUND_SUMMARY}" = false ]; then
{
echo "${EMPTY_MESSAGE}"
echo
} >> "${TMP_DIR}/section.md"
fi
{
echo "*Auto-updated from structured staging promotion merge bodies on ${MAIN_BRANCH}.*"
echo "${SECTION_END}"
} >> "${TMP_DIR}/section.md"
replace_marked_section \
"${TMP_DIR}/body.md" \
"${TMP_DIR}/section.md" \
"${SECTION_START}" \
"${SECTION_END}" \
"${TMP_DIR}/new-body.md"
if [ "${DRY_RUN}" = "true" ]; then
echo "Dry run enabled. Computed PR body for #${PR_NUMBER}:"
cat "${TMP_DIR}/new-body.md"
else
gh pr edit "${PR_NUMBER}" --repo "${REPO}" --body-file "${TMP_DIR}/new-body.md"
fi
@@ -0,0 +1,53 @@
#!/usr/bin/env bash
set -euo pipefail
: "${PR_NUMBER:?PR_NUMBER is required}"
: "${REPO:?REPO is required}"
MAX_COMMITS="${MAX_COMMITS:-50}"
DRY_RUN="${DRY_RUN:-false}"
SECTION_START="<!-- staging-ci-current:start -->"
SECTION_END="<!-- staging-ci-current:end -->"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "${TMP_DIR}"' EXIT
# shellcheck source=.github/scripts/pr-body-utils.sh
source "$(dirname "$0")/pr-body-utils.sh"
gh pr view "${PR_NUMBER}" --repo "${REPO}" --json body,baseRefName,headRefName > "${TMP_DIR}/pr.json"
jq -r '.body // ""' < "${TMP_DIR}/pr.json" > "${TMP_DIR}/body.md"
BASE="$(jq -r '.baseRefName' < "${TMP_DIR}/pr.json")"
HEAD="$(jq -r '.headRefName' < "${TMP_DIR}/pr.json")"
RANGE="origin/${BASE}..origin/${HEAD}"
git fetch origin "${BASE}" "${HEAD}"
load_commit_summary "${RANGE}" "${MAX_COMMITS}"
{
echo "${SECTION_START}"
echo "### Current commits in this promotion (${COMMIT_COUNT})"
echo
echo "**Current base:** \`${BASE}\`"
echo "**Current head:** \`${HEAD}\`"
echo "**Current range:** \`${RANGE}\`"
echo
echo "${COMMIT_MD}"
echo
echo "*Auto-updated by staging promotion metadata workflow*"
echo "${SECTION_END}"
} > "${TMP_DIR}/section.md"
replace_marked_section \
"${TMP_DIR}/body.md" \
"${TMP_DIR}/section.md" \
"${SECTION_START}" \
"${SECTION_END}" \
"${TMP_DIR}/new-body.md"
if [ "${DRY_RUN}" = "true" ]; then
echo "Dry run enabled. Computed PR body for #${PR_NUMBER}:"
cat "${TMP_DIR}/new-body.md"
else
gh pr edit "${PR_NUMBER}" --repo "${REPO}" --body-file "${TMP_DIR}/new-body.md"
fi
+42 -2
View File
@@ -78,15 +78,55 @@ jobs:
- name: Check lints
run: cargo clippy --all --benches --tests --examples ${{ matrix.flags }} -- -D warnings
no-panics:
name: No panics in production code
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Check for .unwrap(), .expect(), assert!() in production code
run: |
BASE="${{ github.event.pull_request.base.sha }}"
# Get added lines in .rs files (production only, exclude tests/)
ADDED=$(git diff "$BASE"...HEAD -- 'src/**/*.rs' 'crates/**/*.rs' \
| grep -E '^\+[^+]' || true)
if [ -z "$ADDED" ]; then
echo "No production Rust changes detected."
exit 0
fi
# Match panic-inducing patterns, excluding test code and safety suppressions
VIOLATIONS=$(echo "$ADDED" \
| grep -E '\.(unwrap|expect)\(|[^_]assert(_eq|_ne)?!' \
| grep -Ev 'debug_assert|// safety:|#\[cfg\(test\)\]|#\[test\]|mod tests' \
|| true)
if [ -n "$VIOLATIONS" ]; then
echo "::error::Found .unwrap(), .expect(), or assert!() in production code."
echo "Production code must use proper error handling instead of panicking."
echo "Suppress false positives with an inline '// safety: <reason>' comment."
echo ""
echo "$VIOLATIONS" | head -20
echo ""
COUNT=$(echo "$VIOLATIONS" | wc -l | tr -d ' ')
echo "Total: $COUNT violation(s)"
exit 1
fi
echo "OK: No panic-inducing calls in changed production code."
# Roll-up job for branch protection
code-style:
name: Code Style (fmt + clippy + deny)
runs-on: ubuntu-latest
if: always()
needs: [format, clippy, clippy-windows, deny-check]
needs: [format, clippy, clippy-windows, deny-check, no-panics]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" || "${{ needs.deny-check.result }}" != "success" ]]; then
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" || "${{ needs.deny-check.result }}" != "success" || "${{ needs.no-panics.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+1 -1
View File
@@ -52,7 +52,7 @@ jobs:
- group: features
files: "tests/e2e/scenarios/test_skills.py tests/e2e/scenarios/test_tool_approval.py"
- group: extensions
files: "tests/e2e/scenarios/test_extensions.py"
files: "tests/e2e/scenarios/test_extensions.py tests/e2e/scenarios/test_extension_oauth.py tests/e2e/scenarios/test_wasm_lifecycle.py tests/e2e/scenarios/test_tool_execution.py tests/e2e/scenarios/test_pairing.py"
steps:
- uses: actions/checkout@v6
@@ -0,0 +1,44 @@
name: Release-plz Batch Summary
on:
workflow_dispatch:
inputs:
pr_number:
description: "release-plz PR number to refresh"
required: true
type: string
dry_run:
description: "Compute the body update without editing the PR"
required: false
type: boolean
default: true
pull_request_target:
types: [opened, synchronize, reopened]
permissions:
contents: read
pull-requests: write
jobs:
update-release-pr:
if: >
(github.event_name == 'pull_request_target' &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'release-plz-')) ||
github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Checkout base branch
uses: actions/checkout@v6
with:
ref: ${{ github.event_name == 'workflow_dispatch' && 'main' || github.event.pull_request.base.ref }}
fetch-depth: 0
fetch-tags: true
- name: Update release-plz PR body with staging batch summary
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number }}
REPO: ${{ github.repository }}
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || 'false' }}
run: bash .github/scripts/update-release-plz-body.sh
+7 -1
View File
@@ -58,10 +58,16 @@ jobs:
- *checkout
- *install-rust
- uses: Swatinem/rust-cache@v2
- name: Generate GitHub token
uses: actions/create-github-app-token@v2
id: generate-token
with:
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
private-key: ${{ secrets.GH_RELEASES_MANAGER_APP_PRIVATE_KEY }}
- name: Run release-plz
uses: release-plz/[email protected]
with:
command: release-pr
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ steps.generate-token.outputs.token }}
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+111 -63
View File
@@ -25,9 +25,35 @@ concurrency:
cancel-in-progress: false # Let running suites finish
jobs:
# ── Resolve promotion base branch ───────────────────────────────
resolve-promotion-base:
name: Resolve promotion base
runs-on: ubuntu-latest
outputs:
promotion_base: ${{ steps.resolve.outputs.promotion_base }}
steps:
- name: Resolve promotion base
id: resolve
env:
GH_TOKEN: ${{ github.token }}
FALLBACK_BRANCH: main
REPO: ${{ github.repository }}
run: |
LATEST=$(gh pr list --repo "${REPO}" --label staging-promotion --state open \
--json headRefName,createdAt \
--jq '[.[] | select(.headRefName | startswith("staging-promote/"))] | sort_by(.createdAt) | last | .headRefName // empty')
if [ -n "$LATEST" ]; then
echo "promotion_base=${LATEST}" >> "$GITHUB_OUTPUT"
echo "Using open promotion branch as base: ${LATEST}"
else
echo "promotion_base=${FALLBACK_BRANCH}" >> "$GITHUB_OUTPUT"
echo "No open promotion branch found. Using ${FALLBACK_BRANCH}."
fi
# ── Check for new commits ──────────────────────────────────────
check-changes:
name: Check for new commits
needs: resolve-promotion-base
runs-on: ubuntu-latest
outputs:
has_changes: ${{ steps.check.outputs.has_changes }}
@@ -44,7 +70,7 @@ jobs:
id: check
env:
FORCE_RUN: ${{ inputs.force }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
PROMOTION_BASE: ${{ needs.resolve-promotion-base.outputs.promotion_base }}
run: |
CURRENT_HEAD=$(git rev-parse HEAD)
echo "current_head=${CURRENT_HEAD}" >> "$GITHUB_OUTPUT"
@@ -66,9 +92,9 @@ jobs:
echo "Found ${COMMIT_COUNT} new commit(s) since last tested"
DIFF_RANGE="${LAST_TESTED}..${CURRENT_HEAD}"
else
git fetch origin "${DEFAULT_BRANCH}"
MERGE_BASE=$(git merge-base "origin/${DEFAULT_BRANCH}" HEAD)
echo "First run -- reviewing from merge-base ${MERGE_BASE}"
git fetch origin "${PROMOTION_BASE}"
MERGE_BASE=$(git merge-base "origin/${PROMOTION_BASE}" HEAD)
echo "First run -- reviewing from merge-base ${MERGE_BASE} against ${PROMOTION_BASE}"
DIFF_RANGE="${MERGE_BASE}..${CURRENT_HEAD}"
fi
fi
@@ -102,7 +128,7 @@ jobs:
# ── Create promotion PR (triggers claude-review.yml on the PR) ──
create-promotion-pr:
name: Create Promotion PR
needs: check-changes
needs: [resolve-promotion-base, check-changes]
if: needs.check-changes.outputs.has_changes == 'true'
runs-on: ubuntu-latest
outputs:
@@ -134,15 +160,15 @@ jobs:
id: ahead-check
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
PROMOTION_BASE: ${{ needs.resolve-promotion-base.outputs.promotion_base }}
run: |
git fetch origin "${DEFAULT_BRANCH}"
AHEAD=$(git rev-list --count "origin/${DEFAULT_BRANCH}..origin/staging")
git fetch origin "${PROMOTION_BASE}"
AHEAD=$(git rev-list --count "origin/${PROMOTION_BASE}..origin/staging")
echo "commits_ahead=${AHEAD}" >> "$GITHUB_OUTPUT"
if [ "$AHEAD" -eq 0 ]; then
echo "Staging is not ahead of ${DEFAULT_BRANCH}. Nothing to promote."
echo "Staging is not ahead of ${PROMOTION_BASE}. Nothing to promote."
else
echo "Staging is ${AHEAD} commits ahead of ${DEFAULT_BRANCH}."
echo "Staging is ${AHEAD} commits ahead of ${PROMOTION_BASE}."
fi
- name: Create promotion branch
@@ -156,54 +182,53 @@ jobs:
echo "branch=${BRANCH}" >> "$GITHUB_OUTPUT"
echo "Created promotion branch: ${BRANCH}"
- name: Find base branch
id: find-base
if: steps.ahead-check.outputs.commits_ahead != '0'
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
run: |
# Find the newest open promotion PR with a staging-promote/* head branch
LATEST=$(gh pr list --label staging-promotion --state open \
--json headRefName,createdAt \
--jq '[.[] | select(.headRefName | startswith("staging-promote/"))] | sort_by(.createdAt) | last | .headRefName // empty')
if [ -n "$LATEST" ]; then
echo "base=${LATEST}" >> "$GITHUB_OUTPUT"
echo "Chaining onto existing promotion branch: ${LATEST}"
else
echo "base=${DEFAULT_BRANCH}" >> "$GITHUB_OUTPUT"
echo "No existing promotion PR — targeting ${DEFAULT_BRANCH}"
fi
- name: Create promotion PR
id: create-pr
if: steps.ahead-check.outputs.commits_ahead != '0'
env:
GH_TOKEN: ${{ steps.token.outputs.token }}
run: |
source .github/scripts/pr-body-utils.sh
RANGE="${{ needs.check-changes.outputs.diff_range }}"
TIMESTAMP=$(date -u +"%Y-%m-%d %H:%M UTC")
BRANCH="${{ steps.branch.outputs.branch }}"
BASE="${{ steps.find-base.outputs.base }}"
BASE="${{ needs.resolve-promotion-base.outputs.promotion_base }}"
MAX_COMMITS=50
load_commit_summary "${RANGE}" "${MAX_COMMITS}"
# Build PR body via concatenation to avoid heredoc shell expansion
# (commit messages in COMMIT_MD may contain $, backticks, or backslashes)
PR_BODY="## Auto-promotion from staging CI"
PR_BODY+=$'\n\n'"**Batch range:** \`${RANGE}\`"
PR_BODY+=$'\n'"**Promotion branch:** \`${BRANCH}\`"
PR_BODY+=$'\n'"**Base:** \`${BASE}\`"
PR_BODY+=$'\n'"**Triggered by:** Staging CI batch at ${TIMESTAMP}"
PR_BODY+=$'\n\n'"### Commits in this batch (${COMMIT_COUNT}):"
PR_BODY+=$'\n'"${COMMIT_MD}"
PR_BODY+=$'\n\n'"<!-- staging-ci-current:start -->"
PR_BODY+=$'\n'"### Current commits in this promotion (${COMMIT_COUNT})"
PR_BODY+=$'\n'
PR_BODY+=$'\n'"**Current base:** \`${BASE}\`"
PR_BODY+=$'\n'"**Current head:** \`${BRANCH}\`"
PR_BODY+=$'\n'"**Current range:** \`origin/${BASE}..origin/${BRANCH}\`"
PR_BODY+=$'\n'
PR_BODY+=$'\n'"${COMMIT_MD}"
PR_BODY+=$'\n'
PR_BODY+=$'\n'"*Auto-updated by staging promotion metadata workflow*"
PR_BODY+=$'\n'"<!-- staging-ci-current:end -->"
PR_BODY+=$'\n\n'"Waiting for gates:"
PR_BODY+=$'\n'"- Tests: pending"
PR_BODY+=$'\n'"- E2E: pending"
PR_BODY+=$'\n'"- Claude Code review: pending (will post comments on this PR)"
PR_BODY+=$'\n\n'"---"
PR_BODY+=$'\n'"*Auto-created by staging-ci workflow*"
PR_URL=$(gh pr create \
--base "$BASE" \
--head "$BRANCH" \
--title "chore: promote staging to ${BASE} (${TIMESTAMP})" \
--body "## Auto-promotion from staging CI
**Batch range:** \`${RANGE}\`
**Promotion branch:** \`${BRANCH}\`
**Base:** \`${BASE}\`
**Triggered by:** Staging CI batch at ${TIMESTAMP}
Waiting for gates:
- Tests: pending
- E2E: pending
- Claude Code review: pending (will post comments on this PR)
---
*Auto-created by staging-ci workflow*" \
--body "$PR_BODY" \
--label "staging-promotion")
PR_NUM=$(echo "$PR_URL" | grep -oE '[0-9]+$')
@@ -228,7 +253,8 @@ jobs:
- uses: actions/checkout@v6
with:
ref: staging
fetch-depth: 1
# Need full history to recompute the final promoted range before merge.
fetch-depth: 0
- name: Generate GitHub App token
id: app-token
@@ -327,8 +353,10 @@ jobs:
# Use process substitution so variables propagate to parent shell
while read -r line; do
TAG=$(echo "$line" | grep -oE '^\[(CRITICAL|HIGH|MEDIUM|LOW):[0-9]+\]')
SEVERITY=$(echo "$TAG" | sed 's/\[\(.*\):\(.*\)\]/\1/')
CONFIDENCE=$(echo "$TAG" | sed 's/\[\(.*\):\(.*\)\]/\2/')
SEVERITY="${TAG#\[}"
SEVERITY="${SEVERITY%%:*}"
CONFIDENCE="${TAG##*:}"
CONFIDENCE="${CONFIDENCE%\]}"
DESC=$(echo "$line" | sed "s/\[${SEVERITY}:${CONFIDENCE}\] *//" | head -1)
echo "Found: [${SEVERITY}:${CONFIDENCE}] ${DESC}"
@@ -420,11 +448,29 @@ jobs:
GH_TOKEN: ${{ steps.token.outputs.token }}
PR_NUMBER: ${{ needs.create-promotion-pr.outputs.pr_number }}
run: |
source .github/scripts/pr-body-utils.sh
if [ -n "$PR_NUMBER" ]; then
BASE=$(gh pr view "$PR_NUMBER" --json baseRefName --jq '.baseRefName')
if [ "$BASE" = "main" ]; then
echo "Merging promotion PR #${PR_NUMBER} (targets main)"
gh pr merge "$PR_NUMBER" --merge
TITLE=$(gh pr view "$PR_NUMBER" --json title --jq '.title')
HEAD_BRANCH=$(gh pr view "$PR_NUMBER" --json headRefName --jq '.headRefName')
git fetch origin "${BASE}" "${HEAD_BRANCH}"
CURRENT_RANGE="origin/${BASE}..origin/${HEAD_BRANCH}"
MAX_COMMITS=50
load_commit_summary "${CURRENT_RANGE}" "${MAX_COMMITS}"
{
echo "staging-promotion-summary-v1"
echo "promotion-pr: #${PR_NUMBER}"
echo "base: ${BASE}"
echo "head: ${HEAD_BRANCH}"
echo "current-range: ${CURRENT_RANGE}"
echo "current-commit-count: ${COMMIT_COUNT}"
echo ""
echo "Current commits in this promotion (${COMMIT_COUNT}):"
echo "${COMMIT_MD}"
} > /tmp/staging-promotion-merge-body.md
gh pr merge "$PR_NUMBER" --merge --subject "#${PR_NUMBER} $TITLE" --body-file /tmp/staging-promotion-merge-body.md
echo "merged=true" >> "$GITHUB_OUTPUT"
else
echo "PR #${PR_NUMBER} targets '${BASE}' (not main) — leaving open for chain resolution"
@@ -464,18 +510,20 @@ jobs:
steps:
- name: Summary
run: |
echo "## Staging CI Batch Results" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "| Check | Result |" >> "$GITHUB_STEP_SUMMARY"
echo "|-------|--------|" >> "$GITHUB_STEP_SUMMARY"
echo "| Tests | ${{ needs.tests.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| E2E | ${{ needs.e2e.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Promotion PR | ${{ needs.create-promotion-pr.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Gate | ${{ needs.gate.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "| Tag Updated | ${{ needs.update-tag.result }} |" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "Range: ${{ needs.check-changes.outputs.diff_range }}" >> "$GITHUB_STEP_SUMMARY"
PR_NUM="${{ needs.create-promotion-pr.outputs.pr_number }}"
if [ -n "$PR_NUM" ]; then
echo "Promotion PR: #${PR_NUM}" >> "$GITHUB_STEP_SUMMARY"
fi
{
echo "## Staging CI Batch Results"
echo ""
echo "| Check | Result |"
echo "|-------|--------|"
echo "| Tests | ${{ needs.tests.result }} |"
echo "| E2E | ${{ needs.e2e.result }} |"
echo "| Promotion PR | ${{ needs.create-promotion-pr.result }} |"
echo "| Gate | ${{ needs.gate.result }} |"
echo "| Tag Updated | ${{ needs.update-tag.result }} |"
echo ""
echo "Range: ${{ needs.check-changes.outputs.diff_range }}"
PR_NUM="${{ needs.create-promotion-pr.outputs.pr_number }}"
if [ -n "$PR_NUM" ]; then
echo "Promotion PR: #${PR_NUM}"
fi
} >> "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,78 @@
name: Staging Promotion Metadata
on:
workflow_dispatch:
inputs:
pr_number:
description: "Staging promotion PR number to refresh"
required: true
type: string
dry_run:
description: "Compute the body update without editing the PR"
required: false
type: boolean
default: true
pull_request_target:
types: [opened, synchronize, reopened]
push:
branches:
- main
permissions:
contents: read
pull-requests: write
jobs:
refresh-single-pr:
if: >
(github.event_name == 'pull_request_target' &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'staging-promote/')) ||
github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Checkout workflow source
uses: actions/checkout@v6
with:
# For chained promotion PRs, the script lives on the trusted PR head,
# not necessarily on the older promotion branch used as the PR base.
ref: ${{ github.event_name == 'workflow_dispatch' && 'main' || github.event.pull_request.head.sha }}
fetch-depth: 0
fetch-tags: true
- name: Refresh staging promotion PR body
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event_name == 'workflow_dispatch' && inputs.pr_number || github.event.pull_request.number }}
REPO: ${{ github.repository }}
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry_run || 'false' }}
run: bash .github/scripts/update-staging-promotion-body.sh
refresh-open-prs-after-main-push:
if: github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@v6
with:
ref: main
fetch-depth: 0
fetch-tags: true
- name: Refresh all open staging promotion PR bodies
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
run: |
# ubuntu-latest uses bash 5.x, so mapfile is available here.
mapfile -t prs < <(gh pr list --repo "${REPO}" --label staging-promotion --state open \
--json number,headRefName \
--jq '.[] | select(.headRefName | startswith("staging-promote/")) | .number')
if [ "${#prs[@]}" -eq 0 ]; then
echo "No open staging promotion PRs to refresh."
exit 0
fi
for pr in "${prs[@]}"; do
echo "Refreshing staging promotion PR #${pr}"
PR_NUMBER="${pr}" bash .github/scripts/update-staging-promotion-body.sh
done
+4
View File
@@ -14,6 +14,10 @@
target/
# Python
__pycache__/
*.pyc
# Benchmark results (local runs, not committed)
bench-results/
+1
View File
@@ -19,6 +19,7 @@ WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
+1
View File
@@ -20,6 +20,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
+1 -1
View File
@@ -121,7 +121,7 @@ dependencies = [
[[package]]
name = "discord-channel"
version = "0.1.0"
version = "0.2.0"
dependencies = [
"ed25519-dalek",
"hex",
+1
View File
@@ -642,6 +642,7 @@ fn poll_channel_mentions(channel_id: &str, bot_id: &str) {
},
thread_id: None,
metadata_json,
attachments: vec![],
});
remember_processed_id(&mut recent_ids, &msg.id);
+2 -2
View File
@@ -2,7 +2,7 @@
"name": "discord",
"display_name": "Discord Channel",
"kind": "channel",
"version": "0.2.0",
"version": "0.2.1",
"wit_version": "0.3.0",
"description": "Talk to your agent in Discord",
"keywords": [
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/discord-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "efa1b9019fa33e243f8db1e1fcc732731d45836336bdd26ca19b6fe227ca8b69"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-0.2.1-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/slack-0.2.1-wasm32-wasip2.tar.gz",
"sha256": "d4667e35126986509d862bc3a0088777305d8f41c75de83c1e223b42312ede48"
}
},
+2 -2
View File
@@ -2,7 +2,7 @@
"name": "telegram",
"display_name": "Telegram Channel",
"kind": "channel",
"version": "0.2.2",
"version": "0.2.3",
"wit_version": "0.3.0",
"description": "Talk to your agent through a Telegram bot",
"keywords": [
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-0.2.2-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/telegram-0.2.3-wasm32-wasip2.tar.gz",
"sha256": "b9a83d5a2d1285ce0ec116b354336a1f245f893291ccb01dffbcaccf89d72aed"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/whatsapp-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "feb9194719d9bed796b070ab4dc30348dbfb5d3dec56f9f21e02d14137abab01"
}
},
+1 -1
View File
@@ -19,7 +19,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/github-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "da9fac56b6f20197a415489bbaec9fefb085a5cf6324cab79ea48a47eb19c13b"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/gmail-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "ee9574e02e92bc1d481f1310eb88afd99ee52bf6971074ab33bd76bf99b34b1d"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-calendar-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "2fa47150ea222e787c122182ad6f4dfa2ffaf5fe490d05e8de887a76445f8d2d"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-docs-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "40e134a1c1564f832ca861c3396895d4e33ec67b99313fc1f97baf8d971423a9"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-drive-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "002a341a1d58125563a7c69561b26fbc2629b04ea723cade744102bdc0fbb71f"
}
},
+1 -1
View File
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-sheets-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "8aa2c9d52f033edea3a6c2311b0ec694ccb6d0a54ef07e94d72bf8be1ce8009a"
}
},
+1 -1
View File
@@ -17,7 +17,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-slides-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "e931a97d4fd0b0b938e464dc7c7f2be6ea6b4d1508f5ea3cd931d44db23f05f5"
}
},
+2 -2
View File
@@ -17,8 +17,8 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "8af3f884240de8413d272845fad2164a347d7d2a502a0d148aa38425b93f62ed"
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/slack-0.2.1-wasm32-wasip2.tar.gz",
"sha256": "d4667e35126986509d862bc3a0088777305d8f41c75de83c1e223b42312ede48"
}
},
"auth_summary": {
+2 -2
View File
@@ -18,8 +18,8 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "2c66245913854be4294021fc6bb479e43f7d65830c5cec25cf6c60a71d1af468"
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/telegram-0.2.2-wasm32-wasip2.tar.gz",
"sha256": "b9a83d5a2d1285ce0ec116b354336a1f245f893291ccb01dffbcaccf89d72aed"
}
},
"auth_summary": {
+2 -2
View File
@@ -2,7 +2,7 @@
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.2.0",
"version": "0.2.1",
"wit_version": "0.3.0",
"description": "Search the web using Brave Search API",
"keywords": [
@@ -18,7 +18,7 @@
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-0.2.0-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/web-search-0.2.0-wasm32-wasip2.tar.gz",
"sha256": "56834573c54ea2a33cea1eb0f04bbdf59f1ef8d8702995cf431b0921302eeccc"
}
},
+6 -4
View File
@@ -70,19 +70,21 @@ echo
# This is a WARNING, not a hard violation.
# --------------------------------------------------------------------------
echo "--- Check 2: .unwrap() / .expect() in production code ---"
echo "--- Check 2: .unwrap() / .expect() / assert!() in production code ---"
# Collect raw matches excluding obvious test-only files and lines
raw_results=$(grep -rn '\.unwrap()\|\.expect(' src/ \
# Collect raw matches excluding obvious test-only files and lines.
# Also catches assert!(), assert_eq!(), assert_ne!() but NOT debug_assert variants.
raw_results=$(grep -rnE '\.(unwrap|expect)\(|[^_]assert(_eq|_ne)?!' src/ \
--include='*.rs' \
| grep -v 'src/main.rs' \
| grep -v 'src/testing.rs' \
| grep -v 'src/setup/' \
| grep -Ev 'debug_assert|// safety:' \
|| true)
if [ -n "$raw_results" ]; then
total=$(echo "$raw_results" | wc -l | tr -d ' ')
echo "WARNING: ~$total .unwrap()/.expect() calls found in src/ (excluding main/testing/setup)."
echo "WARNING: ~$total .unwrap()/.expect()/assert!() calls found in src/ (excluding main/testing/setup)."
echo "Many are in test modules; a per-file breakdown helps triage:"
echo
# Show per-file counts, sorted by count descending, top 15
+19
View File
@@ -10,6 +10,7 @@
# 3. Hardcoded /tmp paths in tests (flaky in parallel runs)
# 4. Tool parameters logged without redaction (secret leaks)
# 5. Multi-step DB operations without transaction wrapping
# 6. .unwrap(), .expect(), assert!() in production code (panics)
#
# Suppress individual lines with an inline "// safety: <reason>" comment.
@@ -128,6 +129,24 @@ if [ -n "$DIFF_W_OUTPUT" ]; then
fi
fi
# 6. .unwrap(), .expect(), assert!() in production code
# Matches added lines containing panic-inducing calls.
# Excludes test files, test modules, and debug_assert (compiled out in release).
# Suppress with "// safety: <reason>".
PROD_DIFF="$DIFF_OUTPUT"
# Strip hunks from test-only files (tests/ directory, *_test.rs, test_*.rs)
PROD_DIFF=$(echo "$PROD_DIFF" | grep -v '^+++ b/tests/' || true)
if echo "$PROD_DIFF" | grep -nE '^\+' \
| grep -E '\.(unwrap|expect)\(|[^_]assert(_eq|_ne)?!' \
| grep -vE 'debug_assert|// safety:|#\[cfg\(test\)\]|#\[test\]|mod tests' \
| head -5 | grep -q .; then
warn "PANIC" "Production code must not use .unwrap(), .expect(), or assert!(). Use proper error handling."
echo "$PROD_DIFF" | grep -nE '^\+' \
| grep -E '\.(unwrap|expect)\(|[^_]assert(_eq|_ne)?!' \
| grep -vE 'debug_assert|// safety:|#\[cfg\(test\)\]|#\[test\]|mod tests' \
| head -5 | sed 's/^/ /'
fi
if [ "$WARNINGS" -gt 0 ]; then
echo ""
echo "Found $WARNINGS potential issue(s). Fix them or add '// safety: <reason>' to suppress."
+31 -12
View File
@@ -93,19 +93,26 @@ impl RoutineEngine {
let mut cache = Vec::new();
for routine in routines {
match &routine.trigger {
Trigger::Event { pattern, .. } => match Regex::new(pattern) {
Ok(re) => cache.push(EventMatcher::Message {
routine: routine.clone(),
regex: re,
}),
Err(e) => {
tracing::warn!(
routine = %routine.name,
"Invalid event regex '{}': {}",
pattern, e
);
Trigger::Event { pattern, .. } => {
// Use RegexBuilder with size limit to prevent ReDoS
// from user-supplied patterns (issue #825).
match regex::RegexBuilder::new(pattern)
.size_limit(64 * 1024) // 64KB compiled size limit
.build()
{
Ok(re) => cache.push(EventMatcher::Message {
routine: routine.clone(),
regex: re,
}),
Err(e) => {
tracing::warn!(
routine = %routine.name,
"Invalid or too complex event regex '{}': {}",
pattern, e
);
}
}
},
}
Trigger::SystemEvent { .. } => {
cache.push(EventMatcher::System {
routine: routine.clone(),
@@ -973,6 +980,18 @@ async fn execute_lightweight_with_tools(
}
};
// Truncate oversized tool output to prevent unbounded context growth.
// Routine tool loops are lightweight and should not accumulate
// large payloads across iterations.
const MAX_TOOL_OUTPUT_CHARS: usize = 8192;
let result_content = if result_content.len() > MAX_TOOL_OUTPUT_CHARS {
let truncated = &result_content
[..result_content.floor_char_boundary(MAX_TOOL_OUTPUT_CHARS)];
format!("{truncated}\n... [output truncated to {MAX_TOOL_OUTPUT_CHARS} chars]")
} else {
result_content
};
// Add tool result to context
messages.push(ChatMessage::tool_result(&tc.id, &tc.name, &result_content));
}
+1
View File
@@ -290,6 +290,7 @@ impl AppBuilder {
Arc::new(ToolRegistry::new())
};
tools.register_builtin_tools();
tools.register_tool_info();
if let Some(ref ss) = self.secrets_store {
tools.register_secrets_tools(Arc::clone(ss));
+145 -76
View File
@@ -269,95 +269,108 @@ async fn webhook_handler(
let mut fallback_req = None;
{
let webhook_secret = state.webhook_secret.read().await;
if let Some(expected_secret) = webhook_secret.as_ref() {
let expected_secret = expected_secret.expose_secret();
let expected_secret = match webhook_secret.as_ref() {
Some(secret) => secret.expose_secret(),
None => {
// No secret configured — reject all requests. This guards against
// the secret being cleared at runtime via update_secret(None).
// The start() method also prevents startup without a secret, but
// this is defense-in-depth for the SIGHUP hot-swap path.
return (
StatusCode::SERVICE_UNAVAILABLE,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Webhook authentication not configured".to_string()),
}),
)
.into_response();
}
};
match headers.get("x-ironclaw-signature") {
Some(raw_signature) => match raw_signature.to_str() {
Ok(signature) => {
if !verify_hmac_signature(expected_secret, &body, signature) {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid webhook signature".to_string()),
}),
)
.into_response();
}
match headers.get("x-ironclaw-signature") {
Some(raw_signature) => match raw_signature.to_str() {
Ok(signature) => {
if !verify_hmac_signature(expected_secret, &body, signature) {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid webhook signature".to_string()),
}),
)
.into_response();
}
}
Err(_) => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid signature header encoding".to_string()),
}),
)
.into_response();
}
},
None => {
let req: WebhookRequest = match serde_json::from_slice(&body) {
Ok(req) => req,
Err(_) => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid signature header encoding".to_string()),
response: Some(
"Webhook authentication required. Provide X-IronClaw-Signature header \
(preferred) or 'secret' field in body (deprecated)."
.to_string(),
),
}),
)
.into_response();
}
},
None => {
let req: WebhookRequest = match serde_json::from_slice(&body) {
Ok(req) => req,
Err(_) => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some(
"Webhook authentication required. Provide X-IronClaw-Signature header \
(preferred) or 'secret' field in body (deprecated)."
.to_string(),
),
}),
)
.into_response();
}
};
};
match &req.secret {
Some(provided)
if bool::from(
provided.as_bytes().ct_eq(expected_secret.as_bytes()),
) =>
{
tracing::warn!(
"Webhook authenticated via deprecated 'secret' field in request body. \
Migrate to X-IronClaw-Signature header (HMAC-SHA256). \
Body secret support will be removed in a future release."
);
fallback_req = Some(req);
}
Some(_) => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid webhook secret".to_string()),
}),
)
.into_response();
}
None => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some(
"Webhook authentication required. Provide X-IronClaw-Signature header \
(preferred) or 'secret' field in body (deprecated)."
.to_string(),
),
}),
)
.into_response();
}
match &req.secret {
Some(provided)
if bool::from(provided.as_bytes().ct_eq(expected_secret.as_bytes())) =>
{
tracing::warn!(
"Webhook authenticated via deprecated 'secret' field in request body. \
Migrate to X-IronClaw-Signature header (HMAC-SHA256). \
Body secret support will be removed in a future release."
);
fallback_req = Some(req);
}
Some(_) => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some("Invalid webhook secret".to_string()),
}),
)
.into_response();
}
None => {
return (
StatusCode::UNAUTHORIZED,
Json(WebhookResponse {
message_id: Uuid::nil(),
status: "error".to_string(),
response: Some(
"Webhook authentication required. Provide X-IronClaw-Signature header \
(preferred) or 'secret' field in body (deprecated)."
.to_string(),
),
}),
)
.into_response();
}
}
}
@@ -1052,6 +1065,32 @@ mod tests {
);
}
#[tokio::test]
async fn webhook_rejects_requests_after_secret_is_cleared() {
let secret = "test-secret-123";
let channel = test_channel(Some(secret));
let _stream = channel.start().await.unwrap();
let app = channel.routes();
channel.update_secret(None).await;
let body = serde_json::json!({
"content": "hello"
});
let body_bytes = serde_json::to_vec(&body).unwrap();
let signature = compute_signature(secret, &body_bytes);
let req = Request::builder()
.method("POST")
.uri("/webhook")
.header("content-type", "application/json")
.header("x-ironclaw-signature", signature)
.body(Body::from(body_bytes))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE); // safety: test assertion
}
#[tokio::test]
async fn test_concurrent_requests_during_secret_update() {
use std::sync::Arc as StdArc;
@@ -1170,4 +1209,34 @@ mod tests {
let body = b"test body content";
assert!(!verify_hmac_signature(secret, body, "sha256=not-hex!"));
}
/// Regression test for issue #1033: when the webhook secret is cleared at
/// runtime via update_secret(None), subsequent requests must be rejected
/// instead of being processed without authentication.
#[tokio::test]
async fn webhook_rejects_when_secret_cleared_at_runtime() {
let channel = test_channel(Some("initial-secret"));
let _stream = channel.start().await.unwrap();
// Clear the secret at runtime (simulates a bad SIGHUP config reload)
channel.update_secret(None).await;
let app = channel.routes();
let body = serde_json::json!({
"content": "hello"
});
let req = Request::builder()
.method("POST")
.uri("/webhook")
.header("content-type", "application/json")
.body(Body::from(serde_json::to_vec(&body).unwrap()))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::SERVICE_UNAVAILABLE,
"requests must be rejected when webhook secret is cleared at runtime"
);
}
}
+4
View File
@@ -294,6 +294,8 @@ impl Channel for RelayChannel {
match client.connect_stream(&token, stream_timeout_secs).await {
Ok((new_stream, new_parser)) => {
tracing::info!("Relay SSE stream reconnected");
consecutive_failures = 0;
backoff_ms = backoff_initial_ms;
current_stream = new_stream;
// Abort old parser before replacing
if let Some(old) = parser_handle.write().await.take() {
@@ -312,6 +314,8 @@ impl Channel for RelayChannel {
tracing::info!(
"Relay SSE stream reconnected with new token"
);
consecutive_failures = 0;
backoff_ms = backoff_initial_ms;
current_stream = new_stream;
if let Some(old) = parser_handle.write().await.take() {
old.abort();
+24 -3
View File
@@ -190,12 +190,21 @@ pub async fn routines_toggle_handler(
None => !routine.enabled,
};
// When re-enabling a cron routine, recompute next_fire_at so the cron
// ticker can pick it up. Mirrors the CLI behavior (issue #1077).
if routine.enabled
&& !was_enabled
&& let Trigger::Cron { schedule, timezone } = &routine.trigger
&& let Trigger::Cron {
ref schedule,
ref timezone,
} = routine.trigger
{
routine.next_fire_at = next_cron_fire(schedule, timezone.as_deref())
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
routine.next_fire_at = next_cron_fire(schedule, timezone.as_deref()).map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to compute next fire: {e}"),
)
})?;
}
store
@@ -203,6 +212,12 @@ pub async fn routines_toggle_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
// Refresh the in-memory event trigger cache so event/system_event
// routines reflect the new enabled state immediately (issue #1076).
if let Some(engine) = state.routine_engine.read().await.as_ref() {
engine.refresh_event_cache().await;
}
Ok(Json(serde_json::json!({
"status": if routine.enabled { "enabled" } else { "disabled" },
"routine_id": routine_id,
@@ -227,6 +242,12 @@ pub async fn routines_delete_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if deleted {
// Refresh the in-memory event trigger cache so deleted event/system_event
// routines stop firing immediately (issue #1076).
if let Some(engine) = state.routine_engine.read().await.as_ref() {
engine.refresh_event_cache().await;
}
Ok(Json(serde_json::json!({
"status": "deleted",
"routine_id": routine_id,
+120 -72
View File
@@ -573,6 +573,14 @@ async fn oauth_callback_handler(
extension = %flow.extension_name,
"OAuth flow expired"
);
// Notify UI so auth card can show error instead of staying stuck
if let Some(ref sender) = flow.sse_sender {
let _ = sender.send(SseEvent::AuthCompleted {
extension_name: flow.extension_name.clone(),
success: false,
message: "OAuth flow expired. Please try again.".to_string(),
});
}
return oauth_error_page(&flow.display_name);
}
@@ -2706,6 +2714,7 @@ struct GatewayStatusResponse {
#[cfg(test)]
mod tests {
use super::*;
use crate::cli::oauth_defaults;
use crate::testing::credentials::TEST_GATEWAY_CRYPTO_KEY;
#[test]
@@ -2823,6 +2832,11 @@ mod tests {
.with_state(state)
}
fn expired_flow_created_at() -> Option<std::time::Instant> {
std::time::Instant::now()
.checked_sub(oauth_defaults::OAUTH_FLOW_EXPIRY + std::time::Duration::from_secs(1))
}
#[tokio::test]
async fn test_csp_header_present_on_responses() {
use std::net::SocketAddr;
@@ -2929,29 +2943,14 @@ mod tests {
use tower::ServiceExt;
// Build an ExtensionManager so the handler can look up flows
let secrets = Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
TEST_GATEWAY_CRYPTO_KEY.to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets,
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
TEST_GATEWAY_CRYPTO_KEY.to_string(),
))
.expect("crypto"),
)));
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets);
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
@@ -2985,25 +2984,13 @@ mod tests {
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets.clone());
let Some(created_at) = expired_flow_created_at() else {
eprintln!("Skipping expired OAuth flow test: monotonic uptime below expiry window");
return;
};
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
// Insert an expired flow (created 10 minutes ago)
// Insert an expired flow.
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
@@ -3023,9 +3010,7 @@ mod tests {
gateway_token: None,
resource: None,
client_id_secret_name: None,
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
created_at,
};
ext_mgr
@@ -3055,6 +3040,80 @@ mod tests {
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_expired_flow_broadcasts_auth_completed_failure() {
use axum::body::Body;
use tower::ServiceExt;
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
TEST_GATEWAY_CRYPTO_KEY.to_string(),
))
.expect("crypto"),
)));
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets.clone());
let (sender, mut receiver) = tokio::sync::broadcast::channel(4);
let Some(created_at) = expired_flow_created_at() else {
eprintln!("Skipping expired OAuth flow SSE test: monotonic uptime below expiry window");
return;
};
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "test_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: Some(sender),
gateway_token: None,
resource: None,
client_id_secret_name: None,
created_at,
};
ext_mgr
.pending_oauth_flows()
.write()
.await
.insert("expired_state".to_string(), flow);
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=expired_state")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
match receiver.recv().await.expect("auth_completed event") {
crate::channels::web::types::SseEvent::AuthCompleted {
extension_name,
success,
message,
} => {
assert_eq!(extension_name, "test_tool");
assert!(!success, "expired OAuth flow should broadcast failure");
assert_eq!(message, "OAuth flow expired. Please try again.");
}
event => panic!("expected AuthCompleted event, got {event:?}"),
}
}
#[tokio::test]
async fn test_oauth_callback_no_extension_manager() {
use axum::body::Body;
@@ -3093,28 +3152,16 @@ mod tests {
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets.clone());
// Insert a flow keyed by raw nonce "test_nonce" (without instance prefix).
// Use an expired flow so the handler exits before attempting a real HTTP
// token exchange — we only need to verify that the instance prefix was
// stripped and the flow was found by the raw nonce.
let Some(created_at) = expired_flow_created_at() else {
eprintln!("Skipping OAuth state-prefix test: monotonic uptime below expiry window");
return;
};
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
@@ -3135,9 +3182,7 @@ mod tests {
resource: None,
client_id_secret_name: None,
// Expired — handler will reject after lookup (no network I/O)
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
created_at,
};
ext_mgr
@@ -3208,24 +3253,27 @@ mod tests {
fn test_ext_mgr(
secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync>,
) -> Arc<ExtensionManager> {
) -> (Arc<ExtensionManager>, tempfile::TempDir, tempfile::TempDir) {
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let mcp_pm = Arc::new(crate::tools::mcp::process::McpProcessManager::new());
Arc::new(ExtensionManager::new(
let wasm_tools_dir = tempfile::tempdir().expect("temp wasm tools dir");
let wasm_channels_dir = tempfile::tempdir().expect("temp wasm channels dir");
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
mcp_pm,
secrets,
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
wasm_tools_dir.path().to_path_buf(),
wasm_channels_dir.path().to_path_buf(),
None,
"test".to_string(),
None,
vec![],
))
));
(ext_mgr, wasm_tools_dir, wasm_channels_dir)
}
#[tokio::test]
@@ -3234,7 +3282,7 @@ mod tests {
use tower::ServiceExt;
let secrets = test_secrets_store();
let ext_mgr = test_ext_mgr(secrets);
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets);
let state = test_gateway_state(Some(ext_mgr));
let app = test_relay_oauth_router(state);
@@ -3278,7 +3326,7 @@ mod tests {
.await
.expect("store nonce");
let ext_mgr = test_ext_mgr(secrets);
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets);
let state = test_gateway_state(Some(ext_mgr));
let app = test_relay_oauth_router(state);
@@ -3323,7 +3371,7 @@ mod tests {
.await
.expect("store nonce");
let ext_mgr = test_ext_mgr(secrets.clone());
let (ext_mgr, _wasm_tools_dir, _wasm_channels_dir) = test_ext_mgr(secrets.clone());
let state = test_gateway_state(Some(ext_mgr));
let app = test_relay_oauth_router(state);
+120 -34
View File
@@ -358,26 +358,11 @@ function connectSSE() {
});
eventSource.addEventListener('auth_required', (e) => {
const data = JSON.parse(e.data);
if (data.auth_url) {
// OAuth flow: show the auth card with an OAuth button + optional token paste field.
showAuthCard(data);
} else {
// Setup flow: fetch the extension's credential schema and show the multi-field
// configure modal (the same UI used by the Extensions tab "Setup" button).
showConfigureModal(data.extension_name);
}
handleAuthRequired(JSON.parse(e.data));
});
eventSource.addEventListener('auth_completed', (e) => {
const data = JSON.parse(e.data);
// Dismiss whichever UI path was active: auth card (OAuth) or configure modal (setup).
removeAuthCard(data.extension_name);
closeConfigureModal();
showToast(data.message, data.success ? 'success' : 'error');
// Refresh extensions list so status indicators update
if (currentTab === 'extensions') loadExtensions();
enableChatInput();
handleAuthCompleted(JSON.parse(e.data));
});
eventSource.addEventListener('extension_status', (e) => {
@@ -1001,6 +986,21 @@ function finalizeActivityGroup() {
_activeToolCards = {};
}
function humanizeToolName(rawName) {
if (!rawName) return '';
return String(rawName)
.replace(/[_-]+/g, ' ')
.replace(/([a-z0-9])([A-Z])/g, '$1 $2')
.replace(/^tool([a-zA-Z])/, 'tool $1')
.replace(/\s+/g, ' ')
.trim();
}
function shouldShowChannelConnectedMessage(extensionName, success) {
if (!success || !extensionName) return false;
return String(extensionName).toLowerCase().includes('telegram');
}
function showApproval(data) {
// Avoid duplicate cards on reconnect/history refresh.
const existing = document.querySelector('.approval-card[data-request-id="' + CSS.escape(data.request_id) + '"]');
@@ -1018,7 +1018,7 @@ function showApproval(data) {
const toolName = document.createElement('div');
toolName.className = 'approval-tool-name';
toolName.textContent = data.tool_name;
toolName.textContent = humanizeToolName(data.tool_name);
card.appendChild(toolName);
if (data.description) {
@@ -1121,13 +1121,71 @@ function showJobCard(data) {
// --- Auth card ---
function handleAuthRequired(data) {
if (data.auth_url) {
// OAuth flow: show the global auth prompt with an OAuth button + optional token paste field.
showAuthCard(data);
} else {
// Setup flow: fetch the extension's credential schema and show the multi-field
// configure modal (the same UI used by the Extensions tab "Setup" button).
showConfigureModal(data.extension_name);
}
}
function handleAuthCompleted(data) {
// Dismiss only the matching extension's UI so unrelated setup work is not interrupted.
removeAuthCard(data.extension_name);
closeConfigureModal(data.extension_name);
showToast(data.message, data.success ? 'success' : 'error');
if (shouldShowChannelConnectedMessage(data.extension_name, data.success)) {
addMessage('system', 'Telegram is now connected. You can message me there and I can send you notifications.');
}
if (currentTab === 'extensions') loadExtensions();
enableChatInput();
}
function queryByDataAttribute(selector, attributeName, attributeValue) {
if (typeof attributeValue !== 'string') return document.querySelector(selector);
if (window.CSS && typeof window.CSS.escape === 'function') {
return document.querySelector(
selector + '[' + attributeName + '="' + window.CSS.escape(attributeValue) + '"]'
);
}
const candidates = document.querySelectorAll(selector);
for (const candidate of candidates) {
if (candidate.getAttribute(attributeName) === attributeValue) return candidate;
}
return null;
}
function getAuthOverlay(extensionName) {
return queryByDataAttribute('.auth-overlay', 'data-extension-name', extensionName);
}
function getAuthCard(extensionName) {
return queryByDataAttribute('.auth-card', 'data-extension-name', extensionName);
}
function getConfigureOverlay(extensionName) {
return queryByDataAttribute('.configure-overlay', 'data-extension-name', extensionName);
}
function showAuthCard(data) {
// Remove any existing card for this extension first
removeAuthCard(data.extension_name);
// Keep a single global auth prompt so the experience is consistent across tabs.
const existing = getAuthOverlay();
if (existing) existing.remove();
const overlay = document.createElement('div');
overlay.className = 'auth-overlay';
overlay.setAttribute('data-extension-name', data.extension_name);
overlay.addEventListener('click', (e) => {
if (e.target === overlay) cancelAuth(data.extension_name);
});
const container = document.getElementById('chat-messages');
const card = document.createElement('div');
card.className = 'auth-card';
card.className = 'auth-card auth-modal';
card.setAttribute('data-extension-name', data.extension_name);
const header = document.createElement('div');
@@ -1206,21 +1264,30 @@ function showAuthCard(data) {
actions.appendChild(cancelBtn);
card.appendChild(actions);
container.appendChild(card);
container.scrollTop = container.scrollHeight;
overlay.appendChild(card);
document.body.appendChild(overlay);
tokenInput.focus();
}
function removeAuthCard(extensionName) {
const card = document.querySelector('.auth-card[data-extension-name="' + extensionName + '"]');
if (card) card.remove();
const overlay = getAuthOverlay(extensionName);
if (overlay) {
overlay.remove();
return;
}
const card = getAuthCard(extensionName);
if (card) {
const parentOverlay = card.closest('.auth-overlay');
if (parentOverlay) parentOverlay.remove();
else card.remove();
}
}
function submitAuthToken(extensionName, tokenValue) {
if (!tokenValue || !tokenValue.trim()) return;
// Disable submit button while in flight
const card = document.querySelector('.auth-card[data-extension-name="' + extensionName + '"]');
const card = getAuthCard(extensionName);
if (card) {
const btns = card.querySelectorAll('button');
btns.forEach((b) => { b.disabled = true; });
@@ -1231,8 +1298,10 @@ function submitAuthToken(extensionName, tokenValue) {
body: { extension_name: extensionName, token: tokenValue.trim() },
}).then((result) => {
if (result.success) {
// Close immediately for responsiveness; the authoritative success UX
// (toast + extensions refresh) still comes from auth_completed SSE.
removeAuthCard(extensionName);
addMessage('system', result.message);
enableChatInput();
} else {
showAuthCardError(extensionName, result.message);
}
@@ -1251,7 +1320,7 @@ function cancelAuth(extensionName) {
}
function showAuthCardError(extensionName, message) {
const card = document.querySelector('.auth-card[data-extension-name="' + extensionName + '"]');
const card = getAuthCard(extensionName);
if (!card) return;
// Re-enable buttons
const btns = card.querySelectorAll('button');
@@ -2181,6 +2250,10 @@ function renderAvailableExtensionCard(entry) {
showToast(I18n.t('extensions.installedSuccess', {name: entry.display_name}), 'success');
// OAuth popup if auth started during install (builtin creds)
if (res.auth_url) {
showAuthCard({
extension_name: entry.name,
auth_url: res.auth_url,
});
showToast('Opening authentication for ' + entry.display_name, 'info');
openOAuthUrl(res.auth_url);
}
@@ -2446,6 +2519,10 @@ function activateExtension(name) {
if (res.success) {
// Even on success, the tool may need OAuth (e.g., WASM loaded but no token yet)
if (res.auth_url) {
showAuthCard({
extension_name: name,
auth_url: res.auth_url,
});
showToast('Opening authentication for ' + name, 'info');
openOAuthUrl(res.auth_url);
}
@@ -2454,6 +2531,10 @@ function activateExtension(name) {
}
if (res.auth_url) {
showAuthCard({
extension_name: name,
auth_url: res.auth_url,
});
showToast('Opening authentication for ' + name, 'info');
openOAuthUrl(res.auth_url);
} else if (res.awaiting_token) {
@@ -2496,6 +2577,7 @@ function renderConfigureModal(name, secrets) {
closeConfigureModal();
const overlay = document.createElement('div');
overlay.className = 'configure-overlay';
overlay.setAttribute('data-extension-name', name);
overlay.addEventListener('click', (e) => {
if (e.target === overlay) closeConfigureModal();
});
@@ -2589,7 +2671,8 @@ function submitConfigureModal(name, fields) {
}
// Disable buttons to prevent double-submit
var btns = document.querySelectorAll('.configure-actions button');
const overlay = getConfigureOverlay(name) || document.querySelector('.configure-overlay');
var btns = overlay ? overlay.querySelectorAll('.configure-actions button') : [];
btns.forEach(function(b) { b.disabled = true; });
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/setup', {
@@ -2600,8 +2683,10 @@ function submitConfigureModal(name, fields) {
if (res.success) {
closeConfigureModal();
if (res.auth_url) {
// OAuth flow started — open consent popup. The auth_completed SSE will
// not arrive immediately (it fires after OAuth callback), so show a toast now.
showAuthCard({
extension_name: name,
auth_url: res.auth_url,
});
showToast('Opening OAuth authorization for ' + name, 'info');
openOAuthUrl(res.auth_url);
loadExtensions();
@@ -2620,8 +2705,9 @@ function submitConfigureModal(name, fields) {
});
}
function closeConfigureModal() {
const existing = document.querySelector('.configure-overlay');
function closeConfigureModal(extensionName) {
if (typeof extensionName !== 'string') extensionName = null;
const existing = getConfigureOverlay(extensionName);
if (existing) existing.remove();
}
+25 -1
View File
@@ -1219,7 +1219,21 @@ body {
color: var(--danger);
}
/* Auth card (inline in chat) */
/* Auth prompt */
.auth-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.6);
z-index: 1001;
display: flex;
align-items: center;
justify-content: center;
padding: 16px;
}
.auth-card {
align-self: flex-start;
max-width: 80%;
@@ -1234,6 +1248,16 @@ body {
transition: border-color 0.2s;
}
.auth-overlay .auth-card {
width: 460px;
max-width: min(460px, 90vw);
margin: 0;
align-self: auto;
background: var(--bg);
border-color: rgba(52, 211, 153, 0.35);
box-shadow: 0 24px 48px rgba(0, 0, 0, 0.35);
}
.auth-card .auth-header {
font-weight: 600;
color: var(--accent);
+63 -7
View File
@@ -23,6 +23,9 @@ pub struct EmbeddingsConfig {
pub ollama_base_url: String,
/// Embedding vector dimension. Inferred from the model name when not set explicitly.
pub dimension: usize,
/// Custom base URL for OpenAI-compatible embedding providers.
/// When set, overrides the default `https://api.openai.com`.
pub openai_base_url: Option<String>,
}
impl Default for EmbeddingsConfig {
@@ -36,6 +39,7 @@ impl Default for EmbeddingsConfig {
model,
ollama_base_url: "http://localhost:11434".to_string(),
dimension,
openai_base_url: None,
}
}
}
@@ -74,6 +78,8 @@ impl EmbeddingsConfig {
let enabled = parse_bool_env("EMBEDDING_ENABLED", settings.embeddings.enabled)?;
let openai_base_url = optional_env("EMBEDDING_BASE_URL")?;
Ok(Self {
enabled,
provider,
@@ -81,6 +87,7 @@ impl EmbeddingsConfig {
model,
ollama_base_url,
dimension,
openai_base_url,
})
}
@@ -130,16 +137,27 @@ impl EmbeddingsConfig {
}
_ => {
if let Some(api_key) = self.openai_api_key() {
tracing::debug!(
"Embeddings enabled via OpenAI (model: {}, dim: {})",
self.model,
self.dimension,
);
Some(Arc::new(crate::workspace::OpenAiEmbeddings::with_model(
let mut provider = crate::workspace::OpenAiEmbeddings::with_model(
api_key,
&self.model,
self.dimension,
)))
);
if let Some(ref base_url) = self.openai_base_url {
tracing::debug!(
"Embeddings enabled via OpenAI (model: {}, base_url: {}, dim: {})",
self.model,
base_url,
self.dimension,
);
provider = provider.with_base_url(base_url);
} else {
tracing::debug!(
"Embeddings enabled via OpenAI (model: {}, dim: {})",
self.model,
self.dimension,
);
}
Some(Arc::new(provider))
} else {
tracing::warn!("Embeddings configured but OPENAI_API_KEY not set");
None
@@ -164,6 +182,7 @@ mod tests {
std::env::remove_var("EMBEDDING_PROVIDER");
std::env::remove_var("EMBEDDING_MODEL");
std::env::remove_var("OPENAI_API_KEY");
std::env::remove_var("EMBEDDING_BASE_URL");
}
}
@@ -247,4 +266,41 @@ mod tests {
std::env::remove_var("EMBEDDING_ENABLED");
}
}
#[test]
fn embedding_base_url_parsed_from_env() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_embedding_env();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("EMBEDDING_BASE_URL", "https://custom.example.com");
}
let settings = Settings::default();
let config = EmbeddingsConfig::resolve(&settings).expect("resolve should succeed");
assert_eq!(
config.openai_base_url.as_deref(),
Some("https://custom.example.com"),
"EMBEDDING_BASE_URL env var should be parsed into openai_base_url"
);
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::remove_var("EMBEDDING_BASE_URL");
}
}
#[test]
fn embedding_base_url_defaults_to_none() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_embedding_env();
let settings = Settings::default();
let config = EmbeddingsConfig::resolve(&settings).expect("resolve should succeed");
assert!(
config.openai_base_url.is_none(),
"openai_base_url should be None when EMBEDDING_BASE_URL is not set"
);
}
}
+14 -8
View File
@@ -16,6 +16,7 @@ mod workspace;
use std::path::Path;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use async_trait::async_trait;
use chrono::{DateTime, NaiveDateTime, Utc};
@@ -32,6 +33,8 @@ use crate::workspace::MemoryDocument;
use crate::db::libsql_migrations;
static NAIVE_TIMESTAMP_LOGGED: AtomicBool = AtomicBool::new(false);
/// Explicit column list for routines table (matches positional access in `row_to_routine_libsql`).
pub(crate) const ROUTINE_COLUMNS: &str = "\
id, name, description, user_id, enabled, \
@@ -163,24 +166,27 @@ impl LibSqlBackend {
///
/// Returns an error if none of the formats match.
pub(crate) fn parse_timestamp(s: &str) -> Result<DateTime<Utc>, String> {
let log_naive_timestamp_once = || {
if !NAIVE_TIMESTAMP_LOGGED.swap(true, Ordering::Relaxed) {
tracing::debug!(
timestamp = %s,
"parsed naive timestamp without timezone; assuming UTC for backward compatibility"
);
}
};
// RFC 3339 (our canonical write format)
if let Ok(dt) = DateTime::parse_from_rfc3339(s) {
return Ok(dt.with_timezone(&Utc));
}
// Naive with fractional seconds (legacy or SQLite datetime() output)
if let Ok(ndt) = NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S%.f") {
tracing::debug!(
timestamp = %s,
"parsed naive timestamp without timezone; assuming UTC for backward compatibility"
);
log_naive_timestamp_once();
return Ok(ndt.and_utc());
}
// Naive without fractional seconds (legacy format)
if let Ok(ndt) = NaiveDateTime::parse_from_str(s, "%Y-%m-%d %H:%M:%S") {
tracing::debug!(
timestamp = %s,
"parsed naive timestamp without timezone; assuming UTC for backward compatibility"
);
log_naive_timestamp_once();
return Ok(ndt.and_utc());
}
Err(format!("unparseable timestamp: {:?}", s))
+189 -6
View File
@@ -786,6 +786,19 @@ impl ExtensionManager {
Self::validate_extension_name(name)?;
let kind = self.determine_installed_kind(name).await?;
// Clean up any in-progress OAuth flows for this extension.
// TCP mode: abort the listener task so port 9876 is freed immediately.
// Gateway mode: remove stale pending flow entries.
if let Some(pending) = self.pending_auth.write().await.remove(name)
&& let Some(handle) = pending.task_handle
{
handle.abort();
}
self.pending_oauth_flows
.write()
.await
.retain(|_, flow| flow.extension_name != name);
match kind {
ExtensionKind::McpServer => {
// Unregister tools with this server's prefix
@@ -819,6 +832,14 @@ impl ExtensionManager {
// Unregister from tool registry
self.tool_registry.unregister(name).await;
// Evict compiled module from runtime cache so reinstall uses fresh binary
if let Some(ref rt) = self.wasm_tool_runtime {
rt.remove(name).await;
}
// Clear stale activation errors so reinstall starts clean
self.activation_errors.write().await.remove(name);
// Revoke credential mappings from the shared registry
let cap_path = self
.wasm_tools_dir
@@ -859,6 +880,9 @@ impl ExtensionManager {
self.active_channel_names.write().await.remove(name);
self.persist_active_channels().await;
// Clear stale activation errors so reinstall starts clean
self.activation_errors.write().await.remove(name);
// Delete channel files
let wasm_path = self.wasm_channels_dir.join(format!("{}.wasm", name));
let cap_path = self
@@ -2860,6 +2884,17 @@ impl ExtensionManager {
});
}
// Check auth status — block activation if required secrets are missing.
// NeedsAuth (OAuth not yet completed) is allowed because configure() loads
// the tool first, then starts the OAuth flow to obtain the token.
let auth_state = self.check_tool_auth_status(name).await;
if auth_state == ToolAuthState::NeedsSetup {
return Err(ExtensionError::ActivationFailed(format!(
"Tool '{}' requires configuration. Use the setup form to provide credentials.",
name
)));
}
let runtime = self.wasm_tool_runtime.as_ref().ok_or_else(|| {
ExtensionError::ActivationFailed("WASM runtime not available".to_string())
})?;
@@ -4495,14 +4530,18 @@ mod tests {
// available" because the ExtensionManager had `wasm_tool_runtime: None`.
/// Build a minimal ExtensionManager suitable for unit tests.
fn make_test_manager(
fn make_test_manager_with_dirs(
wasm_runtime: Option<Arc<crate::tools::wasm::WasmToolRuntime>>,
tools_dir: std::path::PathBuf,
channels_dir: std::path::PathBuf,
) -> crate::extensions::manager::ExtensionManager {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
use crate::tools::mcp::process::McpProcessManager;
use crate::tools::mcp::session::McpSessionManager;
std::fs::create_dir_all(&tools_dir).ok();
std::fs::create_dir_all(&channels_dir).ok();
let key = secrecy::SecretString::from(crate::secrets::keychain::generate_master_key_hex());
let crypto = Arc::new(SecretsCrypto::new(key).expect("crypto"));
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
@@ -4517,15 +4556,22 @@ mod tests {
tools,
None, // hooks
wasm_runtime,
tools_dir.clone(),
tools_dir, // channels dir (unused here)
None, // tunnel_url
tools_dir,
channels_dir,
None, // tunnel_url
"test".to_string(),
None, // db
vec![],
)
}
fn make_test_manager(
wasm_runtime: Option<Arc<crate::tools::wasm::WasmToolRuntime>>,
tools_dir: std::path::PathBuf,
) -> crate::extensions::manager::ExtensionManager {
make_test_manager_with_dirs(wasm_runtime, tools_dir.clone(), tools_dir)
}
#[tokio::test]
async fn test_activate_wasm_tool_with_runtime_passes_runtime_check() {
// When the ExtensionManager has a WASM runtime, activation should get
@@ -4878,6 +4924,145 @@ mod tests {
);
}
#[tokio::test]
async fn test_remove_wasm_tool_clears_pending_oauth_state_and_activation_error() {
let dir = tempfile::tempdir().expect("temp dir");
let mgr = make_test_manager(None, dir.path().to_path_buf());
std::fs::write(dir.path().join("gmail.wasm"), b"fake-tool").expect("write tool");
let listener = tokio::spawn(async {
std::future::pending::<()>().await;
});
let abort_handle = listener.abort_handle();
mgr.pending_auth.write().await.insert(
"gmail".to_string(),
super::PendingAuth {
_name: "gmail".to_string(),
_kind: ExtensionKind::WasmTool,
created_at: std::time::Instant::now(),
task_handle: Some(listener),
},
);
mgr.activation_errors
.write()
.await
.insert("gmail".to_string(), "cached failure".to_string());
let secrets = Arc::clone(&mgr.secrets);
mgr.pending_oauth_flows().write().await.insert(
"gmail-state".to_string(),
crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "gmail".to_string(),
display_name: "Gmail".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "google_oauth_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets: Arc::clone(&secrets),
sse_sender: None,
gateway_token: None,
resource: None,
client_id_secret_name: None,
created_at: std::time::Instant::now(),
},
);
mgr.pending_oauth_flows().write().await.insert(
"other-state".to_string(),
crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "web-search".to_string(),
display_name: "Web Search".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client456".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "other_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: None,
gateway_token: None,
resource: None,
client_id_secret_name: None,
created_at: std::time::Instant::now(),
},
);
let result = mgr.remove("gmail").await;
assert!(result.is_ok(), "remove should succeed: {:?}", result.err());
tokio::task::yield_now().await;
assert!(
mgr.pending_auth.read().await.get("gmail").is_none(),
"pending auth entry should be removed"
);
assert!(
abort_handle.is_finished(),
"pending auth listener should be aborted"
);
assert!(
!mgr.activation_errors.read().await.contains_key("gmail"),
"stale activation error should be cleared"
);
let flows = mgr.pending_oauth_flows().read().await;
assert!(
!flows.contains_key("gmail-state"),
"gateway OAuth flow for removed extension should be cleared"
);
assert!(
flows.contains_key("other-state"),
"unrelated pending OAuth flows should be retained"
);
}
#[tokio::test]
async fn test_remove_wasm_channel_clears_activation_error_and_deletes_files() {
let dir = tempfile::tempdir().expect("temp dir");
let tools_dir = dir.path().join("tools");
let channels_dir = dir.path().join("channels");
let mgr = make_test_manager_with_dirs(None, tools_dir, channels_dir.clone());
let wasm_path = channels_dir.join("telegram.wasm");
let cap_path = channels_dir.join("telegram.capabilities.json");
std::fs::write(&wasm_path, b"fake-channel").expect("write channel");
std::fs::write(&cap_path, b"{}").expect("write capabilities");
mgr.activation_errors
.write()
.await
.insert("telegram".to_string(), "channel failed".to_string());
let result = mgr.remove("telegram").await;
assert!(result.is_ok(), "remove should succeed: {:?}", result.err());
assert!(
!mgr.activation_errors.read().await.contains_key("telegram"),
"channel activation error should be cleared on remove"
);
assert!(
!wasm_path.exists(),
"channel wasm file should be deleted on remove"
);
assert!(
!cap_path.exists(),
"channel capabilities file should be deleted on remove"
);
}
#[test]
fn test_sanitize_url_with_query_params() {
let url = "https://api.example.com/path?api_key=secret123&token=abc";
@@ -5153,7 +5338,6 @@ mod tests {
Some("https://my-gateway.example.com/oauth/callback".to_string()),
);
}
// ── Regression tests for PR #677 (unify-extension-lifecycle) ─────────
#[tokio::test]
@@ -5303,7 +5487,6 @@ mod tests {
"configure should have stored the relay stream token"
);
}
#[test]
fn test_validation_failed_is_distinct_error_variant() {
// Regression: ValidationFailed must be a distinct error variant so
+30 -10
View File
@@ -65,7 +65,20 @@ fn install_macos() -> Result<()> {
let stdout = logs_dir.join("daemon.stdout.log");
let stderr = logs_dir.join("daemon.stderr.log");
let plist = format!(
let plist = macos_plist_content(
&exe.display().to_string(),
&stdout.display().to_string(),
&stderr.display().to_string(),
);
std::fs::write(&file, plist)?;
println!("Installed launchd service: {}", file.display());
println!(" Start with: ironclaw service start");
Ok(())
}
fn macos_plist_content(exe: &str, stdout: &str, stderr: &str) -> String {
format!(
r#"<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
@@ -81,6 +94,11 @@ fn install_macos() -> Result<()> {
<true/>
<key>KeepAlive</key>
<true/>
<key>EnvironmentVariables</key>
<dict>
<key>CLI_ENABLED</key>
<string>false</string>
</dict>
<key>StandardOutPath</key>
<string>{stdout}</string>
<key>StandardErrorPath</key>
@@ -89,15 +107,10 @@ fn install_macos() -> Result<()> {
</plist>
"#,
label = SERVICE_LABEL,
exe = xml_escape(&exe.display().to_string()),
stdout = xml_escape(&stdout.display().to_string()),
stderr = xml_escape(&stderr.display().to_string()),
);
std::fs::write(&file, plist)?;
println!("Installed launchd service: {}", file.display());
println!(" Start with: ironclaw service start");
Ok(())
exe = xml_escape(exe),
stdout = xml_escape(stdout),
stderr = xml_escape(stderr),
)
}
fn install_linux() -> Result<()> {
@@ -356,4 +369,11 @@ mod tests {
let s = path.to_string_lossy();
assert!(s.ends_with(".ironclaw/logs"), "unexpected path: {s}");
}
#[test]
fn macos_plist_sets_cli_enabled_false() {
let plist = macos_plist_content("/tmp/ironclaw", "/tmp/stdout.log", "/tmp/stderr.log");
assert!(plist.contains("<key>EnvironmentVariables</key>"));
assert!(plist.contains(" <key>CLI_ENABLED</key>\n <string>false</string>"));
}
}
+3
View File
@@ -214,6 +214,7 @@ fn is_disallowed_ipv4(v4: &Ipv4Addr) -> bool {
|| v4.is_multicast()
|| v4.is_unspecified()
|| *v4 == Ipv4Addr::new(169, 254, 169, 254)
|| (v4.octets()[0] == 100 && (v4.octets()[1] & 0xC0) == 64)
}
fn is_disallowed_ip(ip: &IpAddr) -> bool {
@@ -913,6 +914,8 @@ mod tests {
assert!(is_disallowed_ip(&IpAddr::V4(Ipv4Addr::new(
169, 254, 169, 254
))));
// Carrier-grade NAT
assert!(is_disallowed_ip(&IpAddr::V4(Ipv4Addr::new(100, 64, 0, 1))));
// Public
assert!(!is_disallowed_ip(&IpAddr::V4(Ipv4Addr::new(8, 8, 8, 8))));
}
+79 -79
View File
@@ -539,88 +539,10 @@ impl Tool for MemoryTreeTool {
}
}
#[cfg(all(test, feature = "postgres"))]
#[cfg(test)]
mod tests {
use super::*;
fn make_test_workspace() -> Arc<Workspace> {
Arc::new(Workspace::new(
"test_user",
deadpool_postgres::Pool::builder(deadpool_postgres::Manager::new(
tokio_postgres::Config::new(),
tokio_postgres::NoTls,
))
.build()
.unwrap(),
))
}
#[test]
fn test_memory_search_schema() {
let workspace = make_test_workspace();
let tool = MemorySearchTool::new(workspace);
assert_eq!(tool.name(), "memory_search");
assert!(!tool.requires_sanitization());
let schema = tool.parameters_schema();
assert!(schema["properties"]["query"].is_object());
assert!(
schema["required"]
.as_array()
.unwrap()
.contains(&"query".into())
);
}
#[test]
fn test_memory_write_schema() {
let workspace = make_test_workspace();
let tool = MemoryWriteTool::new(workspace);
assert_eq!(tool.name(), "memory_write");
let schema = tool.parameters_schema();
assert!(schema["properties"]["content"].is_object());
assert!(schema["properties"]["target"].is_object());
assert!(schema["properties"]["append"].is_object());
}
#[test]
fn test_memory_read_schema() {
let workspace = make_test_workspace();
let tool = MemoryReadTool::new(workspace);
assert_eq!(tool.name(), "memory_read");
let schema = tool.parameters_schema();
assert!(schema["properties"]["path"].is_object());
assert!(
schema["required"]
.as_array()
.unwrap()
.contains(&"path".into())
);
}
#[test]
fn test_memory_tree_schema() {
let workspace = make_test_workspace();
let tool = MemoryTreeTool::new(workspace);
assert_eq!(tool.name(), "memory_tree");
let schema = tool.parameters_schema();
assert!(schema["properties"]["path"].is_object());
assert!(schema["properties"]["depth"].is_object());
assert_eq!(schema["properties"]["depth"]["default"], 1);
}
}
#[cfg(test)]
mod path_routing_tests {
use super::looks_like_filesystem_path;
#[test]
fn detects_filesystem_paths() {
assert!(looks_like_filesystem_path("/Users/nige/file.md"));
@@ -635,4 +557,82 @@ mod path_routing_tests {
assert!(!looks_like_filesystem_path("daily/2026-03-11.md"));
assert!(!looks_like_filesystem_path("projects/alpha/notes.md"));
}
#[cfg(feature = "postgres")]
mod postgres_schema_tests {
use super::*;
fn make_test_workspace() -> Arc<Workspace> {
Arc::new(Workspace::new(
"test_user",
deadpool_postgres::Pool::builder(deadpool_postgres::Manager::new(
tokio_postgres::Config::new(),
tokio_postgres::NoTls,
))
.build()
.unwrap(),
))
}
#[test]
fn test_memory_search_schema() {
let workspace = make_test_workspace();
let tool = MemorySearchTool::new(workspace);
assert_eq!(tool.name(), "memory_search");
assert!(!tool.requires_sanitization());
let schema = tool.parameters_schema();
assert!(schema["properties"]["query"].is_object());
assert!(
schema["required"]
.as_array()
.unwrap()
.contains(&"query".into())
);
}
#[test]
fn test_memory_write_schema() {
let workspace = make_test_workspace();
let tool = MemoryWriteTool::new(workspace);
assert_eq!(tool.name(), "memory_write");
let schema = tool.parameters_schema();
assert!(schema["properties"]["content"].is_object());
assert!(schema["properties"]["target"].is_object());
assert!(schema["properties"]["append"].is_object());
}
#[test]
fn test_memory_read_schema() {
let workspace = make_test_workspace();
let tool = MemoryReadTool::new(workspace);
assert_eq!(tool.name(), "memory_read");
let schema = tool.parameters_schema();
assert!(schema["properties"]["path"].is_object());
assert!(
schema["required"]
.as_array()
.unwrap()
.contains(&"path".into())
);
}
#[test]
fn test_memory_tree_schema() {
let workspace = make_test_workspace();
let tool = MemoryTreeTool::new(workspace);
assert_eq!(tool.name(), "memory_tree");
let schema = tool.parameters_schema();
assert!(schema["properties"]["path"].is_object());
assert!(schema["properties"]["depth"].is_object());
assert_eq!(schema["properties"]["depth"]["default"], 1);
}
}
}
+2
View File
@@ -15,6 +15,7 @@ pub mod secrets_tools;
pub(crate) mod shell;
pub mod skill_tools;
mod time;
mod tool_info;
pub use echo::EchoTool;
pub use extension_tools::{
@@ -39,6 +40,7 @@ pub use secrets_tools::{SecretDeleteTool, SecretListTool};
pub use shell::ShellTool;
pub use skill_tools::{SkillInstallTool, SkillListTool, SkillRemoveTool, SkillSearchTool};
pub use time::TimeTool;
pub use tool_info::ToolInfoTool;
mod html_converter;
pub mod image_analyze;
pub mod image_edit;
+252 -121
View File
@@ -24,6 +24,132 @@ use crate::context::JobContext;
use crate::db::Database;
use crate::tools::tool::{ApprovalRequirement, Tool, ToolError, ToolOutput, require_str};
pub(crate) fn routine_create_parameters_schema() -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Unique routine name, for example 'daily-pr-review'."
},
"description": {
"type": "string",
"description": "Short summary of what the routine is for."
},
"trigger_type": {
"type": "string",
"enum": ["cron", "event", "system_event", "manual"],
"description": "When the routine fires: 'cron' for schedules, 'event' for incoming messages, 'system_event' for structured emitted events, or 'manual' for explicit runs."
},
"schedule": {
"type": "string",
"description": "Cron schedule for 'cron' triggers. Uses 6 fields: second minute hour day month weekday."
},
"event_pattern": {
"type": "string",
"description": "Regex matched against incoming message text for 'event' triggers, for example '^bug\\\\b'."
},
"event_channel": {
"type": "string",
"description": "Optional platform filter for 'event' triggers, for example 'telegram'. Omit to match any channel. Not a chat or thread ID."
},
"event_source": {
"type": "string",
"description": "Structured event source for 'system_event' triggers, for example 'github'."
},
"event_type": {
"type": "string",
"description": "Structured event type for 'system_event' triggers, for example 'issue.opened'."
},
"event_filters": {
"type": "object",
"properties": {},
"additionalProperties": {
"type": ["string", "number", "boolean"]
},
"description": "Optional exact-match payload filters for 'system_event' triggers. Values can be strings, numbers, or booleans."
},
"prompt": {
"type": "string",
"description": "Instructions for what the routine should do after it fires."
},
"context_paths": {
"type": "array",
"items": { "type": "string" },
"description": "Workspace paths to load as extra context before running the routine."
},
"action_type": {
"type": "string",
"enum": ["lightweight", "full_job"],
"description": "Execution mode: 'lightweight' for one LLM turn or 'full_job' for a multi-step job with tools."
},
"use_tools": {
"type": "boolean",
"description": "Enable safe tool use in 'lightweight' mode. Ignored for 'full_job'."
},
"max_tool_rounds": {
"type": "integer",
"description": "Maximum tool-call rounds in 'lightweight' mode when 'use_tools' is true."
},
"cooldown_secs": {
"type": "integer",
"description": "Minimum seconds between fires."
},
"tool_permissions": {
"type": "array",
"items": { "type": "string" },
"description": "Pre-authorized tool names for 'full_job' routines."
},
"notify_channel": {
"type": "string",
"description": "Where routine output should be sent, for example 'telegram' or 'slack'. This does not control what triggers the routine."
},
"notify_user": {
"type": "string",
"description": "User or destination to notify, for example a username or chat ID."
},
"timezone": {
"type": "string",
"description": "IANA timezone used to evaluate 'cron' schedules, for example 'America/New_York'."
}
},
"required": ["name", "trigger_type", "prompt"]
})
}
pub(crate) fn routine_update_parameters_schema() -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the routine to update."
},
"enabled": {
"type": "boolean",
"description": "Set to true to enable the routine or false to disable it."
},
"prompt": {
"type": "string",
"description": "Replace the routine instructions for what it should do after it fires."
},
"schedule": {
"type": "string",
"description": "New cron schedule for existing 'cron' routines only. This does not convert other trigger types."
},
"timezone": {
"type": "string",
"description": "New IANA timezone for existing 'cron' routines only, for example 'America/New_York'."
},
"description": {
"type": "string",
"description": "Replace the routine summary."
}
},
"required": ["name"]
})
}
// ==================== routine_create ====================
pub struct RoutineCreateTool {
@@ -50,92 +176,7 @@ impl Tool for RoutineCreateTool {
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Unique name for the routine (e.g. 'daily-pr-review')"
},
"description": {
"type": "string",
"description": "What this routine does"
},
"trigger_type": {
"type": "string",
"enum": ["cron", "event", "system_event", "manual"],
"description": "When the routine fires"
},
"schedule": {
"type": "string",
"description": "Cron expression (for cron trigger). E.g. '0 9 * * MON-FRI' for weekdays at 9am. Uses 6-field cron (sec min hour day month weekday)."
},
"event_pattern": {
"type": "string",
"description": "Regex pattern to match messages (for event trigger)"
},
"event_channel": {
"type": "string",
"description": "Optional channel filter for event trigger (e.g. 'telegram')"
},
"event_source": {
"type": "string",
"description": "Event source for system_event triggers (e.g. 'github')"
},
"event_type": {
"type": "string",
"description": "Event type for system_event triggers (e.g. 'issue.opened')"
},
"event_filters": {
"type": "object",
"description": "Optional exact-match filters against payload fields for system_event triggers. Values can be strings, numbers, or booleans."
},
"prompt": {
"type": "string",
"description": "The prompt/instructions for the routine"
},
"context_paths": {
"type": "array",
"items": { "type": "string" },
"description": "Workspace paths to load as context (e.g. ['context/priorities.md'])"
},
"action_type": {
"type": "string",
"enum": ["lightweight", "full_job"],
"description": "Execution mode: 'lightweight' (single LLM call, default) or 'full_job' (multi-turn with tools)"
},
"use_tools": {
"type": "boolean",
"description": "Enable tool access in lightweight mode (default: false). Only safe tools (no approval required) are available. Ignored for full_job mode."
},
"max_tool_rounds": {
"type": "integer",
"description": "Max tool call rounds in lightweight mode (default: 3). Only used when use_tools is true."
},
"cooldown_secs": {
"type": "integer",
"description": "Minimum seconds between fires (default: 300)"
},
"tool_permissions": {
"type": "array",
"items": { "type": "string" },
"description": "Tool names pre-authorized for Always-approval tools in full_job mode (e.g. ['shell']). UnlessAutoApproved tools are automatically permitted in routines."
},
"notify_channel": {
"type": "string",
"description": "Channel to send results to (e.g. 'telegram', 'slack', 'tui'). Sets the default channel for message tool calls in routine jobs."
},
"notify_user": {
"type": "string",
"description": "User/target to notify (e.g. username, chat ID). Defaults to 'default'."
},
"timezone": {
"type": "string",
"description": "IANA timezone for cron schedule evaluation (e.g. 'America/New_York'). Defaults to UTC."
}
},
"required": ["name", "trigger_type", "prompt"]
})
routine_create_parameters_schema()
}
async fn execute(
@@ -199,9 +240,13 @@ impl Tool for RoutineCreateTool {
"event trigger requires 'event_pattern'".to_string(),
)
})?;
// Validate regex
regex::Regex::new(pattern)
.map_err(|e| ToolError::InvalidParameters(format!("invalid regex: {e}")))?;
// Validate regex with size limit to prevent ReDoS (issue #825)
regex::RegexBuilder::new(pattern)
.size_limit(64 * 1024)
.build()
.map_err(|e| {
ToolError::InvalidParameters(format!("invalid or too complex regex: {e}"))
})?;
let channel = params
.get("event_channel")
.and_then(|v| v.as_str())
@@ -478,41 +523,13 @@ impl Tool for RoutineUpdateTool {
}
fn description(&self) -> &str {
"Update an existing routine. Can modify trigger, prompt, schedule, or toggle enabled state. \
Pass the routine name and only the fields you want to change."
"Update an existing routine. Can change prompt, description, enabled state, or cron timing. \
Pass the routine name and only the fields you want to change. \
This does not convert one trigger type into another."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the routine to update"
},
"enabled": {
"type": "boolean",
"description": "Enable or disable the routine"
},
"prompt": {
"type": "string",
"description": "New prompt/instructions"
},
"schedule": {
"type": "string",
"description": "New cron schedule (for cron triggers)"
},
"timezone": {
"type": "string",
"description": "IANA timezone for cron schedule (e.g. 'America/New_York'). Only valid for cron triggers."
},
"description": {
"type": "string",
"description": "New description"
}
},
"required": ["name"]
})
routine_update_parameters_schema()
}
async fn execute(
@@ -953,3 +970,117 @@ impl Tool for EventEmitTool {
true
}
}
#[cfg(test)]
mod tests {
use super::{routine_create_parameters_schema, routine_update_parameters_schema};
use crate::tools::validate_tool_schema;
fn property<'a>(schema: &'a serde_json::Value, name: &str) -> &'a serde_json::Value {
schema
.get("properties")
.and_then(|props| props.get(name))
.unwrap_or_else(|| panic!("missing schema property {name}"))
}
#[test]
fn routine_create_schema_exposes_all_trigger_and_delivery_fields() {
let schema = routine_create_parameters_schema();
let errors = validate_tool_schema(&schema, "routine_create");
assert!(
errors.is_empty(),
"routine_create schema should validate cleanly: {errors:?}"
);
for field in [
"trigger_type",
"schedule",
"event_pattern",
"event_channel",
"event_source",
"event_type",
"event_filters",
"action_type",
"use_tools",
"max_tool_rounds",
"tool_permissions",
"notify_channel",
"notify_user",
"timezone",
] {
let _ = property(&schema, field);
}
}
#[test]
fn routine_create_schema_descriptions_cover_event_trigger_gotchas() {
let schema = routine_create_parameters_schema();
let trigger_type = property(&schema, "trigger_type")
.get("description")
.and_then(|value| value.as_str())
.expect("trigger_type description");
assert!(trigger_type.contains("incoming messages"));
assert!(trigger_type.contains("structured emitted events"));
let event_pattern = property(&schema, "event_pattern")
.get("description")
.and_then(|value| value.as_str())
.expect("event_pattern description");
assert!(event_pattern.contains("incoming message text"));
assert!(event_pattern.contains("^bug\\\\b"));
let event_channel = property(&schema, "event_channel")
.get("description")
.and_then(|value| value.as_str())
.expect("event_channel description");
assert!(event_channel.contains("Omit to match any channel"));
assert!(event_channel.contains("Not a chat or thread ID"));
let notify_channel = property(&schema, "notify_channel")
.get("description")
.and_then(|value| value.as_str())
.expect("notify_channel description");
assert!(notify_channel.contains("does not control what triggers"));
let prompt = property(&schema, "prompt")
.get("description")
.and_then(|value| value.as_str())
.expect("prompt description");
assert!(prompt.contains("after it fires"));
}
#[test]
fn routine_update_schema_exposes_supported_fields_and_limits() {
let schema = routine_update_parameters_schema();
let errors = validate_tool_schema(&schema, "routine_update");
assert!(
errors.is_empty(),
"routine_update schema should validate cleanly: {errors:?}"
);
for field in [
"name",
"enabled",
"prompt",
"schedule",
"timezone",
"description",
] {
let _ = property(&schema, field);
}
let schedule = property(&schema, "schedule")
.get("description")
.and_then(|value| value.as_str())
.expect("schedule description");
assert!(schedule.contains("existing 'cron' routines only"));
assert!(schedule.contains("does not convert other trigger types"));
let timezone = property(&schema, "timezone")
.get("description")
.and_then(|value| value.as_str())
.expect("timezone description");
assert!(timezone.contains("existing 'cron' routines only"));
}
}
+54 -2
View File
@@ -247,7 +247,11 @@ fn resolve_timezone_for_output(
params: &serde_json::Value,
ctx: &JobContext,
) -> Result<Option<(Tz, String)>, ToolError> {
if let Some(name) = params.get("timezone").and_then(|v| v.as_str()) {
if let Some(name) = params
.get("timezone")
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
{
let tz = parse_timezone(name)?;
return Ok(Some((tz, tz.to_string())));
}
@@ -286,7 +290,11 @@ fn context_timezone(ctx: &JobContext) -> Result<Option<(Tz, String)>, ToolError>
fn optional_timezone(params: &serde_json::Value, keys: &[&str]) -> Result<Option<Tz>, ToolError> {
for key in keys {
if let Some(value) = params.get(*key).and_then(|v| v.as_str()) {
if let Some(value) = params
.get(*key)
.and_then(|v| v.as_str())
.filter(|s| !s.is_empty())
{
return parse_timezone(value).map(Some);
}
}
@@ -534,4 +542,48 @@ mod tests {
assert_eq!(dt.to_rfc3339(), "2026-03-08T07:30:00+00:00");
}
#[tokio::test]
async fn test_now_with_empty_timezone_string_does_not_error() {
// LLMs sometimes pass "" for optional fields instead of omitting them.
// Empty timezone should be treated as absent and fall back to UTC.
let tool = TimeTool;
let ctx = JobContext::with_user("test", "chat", "test");
let output = tool
.execute(
serde_json::json!({
"operation": "now",
"timezone": ""
}),
&ctx,
)
.await
.expect("empty timezone string should not error");
assert!(output.result.get("iso").is_some(), "should have iso");
}
#[tokio::test]
async fn test_convert_with_empty_from_timezone_string_does_not_error() {
// LLMs sometimes pass "" for optional fields instead of omitting them.
// Empty from_timezone should be treated as absent.
let tool = TimeTool;
let ctx = JobContext::with_user("test", "chat", "test");
let output = tool
.execute(
serde_json::json!({
"operation": "convert",
"timestamp": "2026-03-08T12:00:00Z",
"to_timezone": "America/New_York",
"from_timezone": ""
}),
&ctx,
)
.await
.expect("empty from_timezone string should not error");
assert!(output.result.get("output").is_some(), "should have output");
}
}
+183
View File
@@ -0,0 +1,183 @@
//! On-demand tool discovery (like CLI `--help`).
//!
//! Two levels of detail:
//! - Default: name, description, parameter names (compact ~150 bytes)
//! - `include_schema: true`: adds the full typed JSON Schema
//!
//! Keeps the tools array compact (WASM tools use permissive schemas)
//! while allowing precise discovery when needed.
use std::sync::Weak;
use async_trait::async_trait;
use crate::context::JobContext;
use crate::tools::registry::ToolRegistry;
use crate::tools::tool::{Tool, ToolError, ToolOutput, require_str};
pub struct ToolInfoTool {
registry: Weak<ToolRegistry>,
}
impl ToolInfoTool {
pub fn new(registry: Weak<ToolRegistry>) -> Self {
Self { registry }
}
}
#[async_trait]
impl Tool for ToolInfoTool {
fn name(&self) -> &str {
"tool_info"
}
fn description(&self) -> &str {
"Get info about any tool: description and parameter names. \
Set include_schema to true for the full typed parameter schema."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the tool to get info about"
},
"include_schema": {
"type": "boolean",
"description": "If true, include the full typed JSON Schema for parameters (larger response). Default: false.",
"default": false
}
},
"required": ["name"]
})
}
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = require_str(&params, "name")?;
let include_schema = params
.get("include_schema")
.and_then(|v| v.as_bool())
.unwrap_or(false);
let registry = self.registry.upgrade().ok_or_else(|| {
ToolError::ExecutionFailed(
"tool registry is no longer available for tool_info".to_string(),
)
})?;
let tool = registry.get(name).await.ok_or_else(|| {
ToolError::InvalidParameters(format!("No tool named '{name}' is registered"))
})?;
let schema = tool.discovery_schema();
// Extract just param names from the schema's "properties" keys
let param_names: Vec<&str> = schema
.get("properties")
.and_then(|p| p.as_object())
.map(|props| props.keys().map(|k| k.as_str()).collect())
.unwrap_or_default();
let mut info = serde_json::json!({
"name": tool.name(),
"description": tool.description(),
"parameters": param_names,
});
if include_schema {
info["schema"] = schema;
}
Ok(ToolOutput::success(info, start.elapsed()))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::tools::builtin::EchoTool;
use std::sync::Arc;
#[tokio::test]
async fn test_tool_info_default_returns_param_names() {
let registry = Arc::new(ToolRegistry::new());
registry.register(Arc::new(EchoTool)).await;
let tool = ToolInfoTool::new(Arc::downgrade(&registry));
let ctx = JobContext::default();
let result = tool
.execute(serde_json::json!({"name": "echo"}), &ctx)
.await
.unwrap();
let info = &result.result;
assert_eq!(info["name"], "echo");
assert!(!info["description"].as_str().unwrap().is_empty());
// Default: parameters is an array of names, not the full schema
assert!(info["parameters"].is_array());
assert!(
info["parameters"]
.as_array()
.unwrap()
.iter()
.any(|v| v.as_str() == Some("message")),
"echo tool should have 'message' parameter: {:?}",
info["parameters"]
);
// No schema field by default
assert!(info.get("schema").is_none());
}
#[tokio::test]
async fn test_tool_info_with_schema() {
let registry = Arc::new(ToolRegistry::new());
registry.register(Arc::new(EchoTool)).await;
let tool = ToolInfoTool::new(Arc::downgrade(&registry));
let ctx = JobContext::default();
let result = tool
.execute(
serde_json::json!({"name": "echo", "include_schema": true}),
&ctx,
)
.await
.unwrap();
let info = &result.result;
assert_eq!(info["name"], "echo");
// With include_schema: true, schema field should be present
assert!(info["schema"].is_object());
assert!(info["schema"]["properties"].is_object());
}
#[tokio::test]
async fn test_tool_info_unknown_tool() {
let registry = Arc::new(ToolRegistry::new());
let tool = ToolInfoTool::new(Arc::downgrade(&registry));
let ctx = JobContext::default();
let result = tool
.execute(serde_json::json!({"name": "nonexistent"}), &ctx)
.await;
assert!(result.is_err());
}
#[tokio::test]
async fn test_tool_info_registry_dropped() {
let registry = Arc::new(ToolRegistry::new());
let tool = ToolInfoTool::new(Arc::downgrade(&registry));
drop(registry);
let ctx = JobContext::default();
let result = tool
.execute(serde_json::json!({"name": "echo"}), &ctx)
.await;
assert!(matches!(result, Err(ToolError::ExecutionFailed(_))));
}
}
+60 -40
View File
@@ -18,6 +18,44 @@ use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
use crate::secrets::{CreateSecretParams, SecretsStore};
use crate::tools::mcp::config::McpServerConfig;
/// Shared HTTP client for all OAuth/discovery requests.
///
/// Redirects are disabled for security (prevents redirect-based SSRF).
/// Per-request timeouts can override the default via `.timeout()` on
/// the request builder.
fn oauth_http_client() -> Result<&'static reqwest::Client, AuthError> {
static CLIENT: std::sync::OnceLock<Result<reqwest::Client, String>> =
std::sync::OnceLock::new();
CLIENT
.get_or_init(|| {
reqwest::Client::builder()
.timeout(Duration::from_secs(30))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| e.to_string())
})
.as_ref()
.map_err(|e| AuthError::Http(e.clone()))
}
/// Log a debug message when a discovery/auth response is a redirect.
/// Helps users diagnose configuration issues when legitimate servers
/// redirect and our no-redirect policy causes a failure.
fn log_redirect_if_applicable(url: &str, response: &reqwest::Response) {
if response.status().is_redirection() {
let location = response
.headers()
.get("location")
.and_then(|v| v.to_str().ok());
tracing::debug!(
"OAuth request to '{}' returned redirect {} -> {:?} (redirects disabled for security)",
url,
response.status(),
location
);
}
}
/// OAuth authorization error.
#[derive(Debug, thiserror::Error)]
pub enum AuthError {
@@ -287,10 +325,8 @@ async fn validate_url_safe(url: &str) -> Result<(), AuthError> {
)));
}
if scheme == "http" {
let host = parsed.host_str().unwrap_or("");
let is_localhost =
host == "localhost" || host == "127.0.0.1" || host == "::1" || host == "[::1]";
if !is_localhost {
if !crate::tools::mcp::config::is_localhost_url(url) {
let host = parsed.host_str().unwrap_or("");
return Err(AuthError::DiscoveryFailed(format!(
"HTTP is only allowed for localhost; use HTTPS for '{}'",
host
@@ -382,18 +418,17 @@ fn parse_resource_metadata_url(www_authenticate: &str) -> Option<String> {
async fn fetch_resource_metadata(url: &str) -> Result<ProtectedResourceMetadata, AuthError> {
validate_url_safe(url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let response = client
.get(url)
.timeout(Duration::from_secs(10))
.send()
.await
.map_err(|e| AuthError::DiscoveryFailed(e.to_string()))?;
log_redirect_if_applicable(url, &response);
if !response.status().is_success() {
return Err(AuthError::DiscoveryFailed(format!(
"HTTP {}",
@@ -411,20 +446,19 @@ async fn fetch_resource_metadata(url: &str) -> Result<ProtectedResourceMetadata,
async fn discover_via_401(server_url: &str) -> Result<AuthorizationServerMetadata, AuthError> {
validate_url_safe(server_url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let response = client
.post(server_url)
.timeout(Duration::from_secs(10))
.header("Content-Type", "application/json")
.body("{}")
.send()
.await
.map_err(|e| AuthError::DiscoveryFailed(e.to_string()))?;
log_redirect_if_applicable(server_url, &response);
if response.status().as_u16() != 401 {
return Err(AuthError::DiscoveryFailed(format!(
"Expected 401, got {}",
@@ -472,20 +506,19 @@ pub async fn discover_protected_resource(
) -> Result<ProtectedResourceMetadata, AuthError> {
validate_url_safe(server_url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let well_known_url = build_well_known_uri(server_url, "oauth-protected-resource")?;
let response = client
.get(&well_known_url)
.timeout(Duration::from_secs(10))
.send()
.await
.map_err(|e| AuthError::DiscoveryFailed(e.to_string()))?;
log_redirect_if_applicable(&well_known_url, &response);
if !response.status().is_success() {
return Err(AuthError::NotSupported);
}
@@ -502,20 +535,19 @@ pub async fn discover_authorization_server(
) -> Result<AuthorizationServerMetadata, AuthError> {
validate_url_safe(auth_server_url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let well_known_url = build_well_known_uri(auth_server_url, "oauth-authorization-server")?;
let response = client
.get(&well_known_url)
.timeout(Duration::from_secs(10))
.send()
.await
.map_err(|e| AuthError::DiscoveryFailed(e.to_string()))?;
log_redirect_if_applicable(&well_known_url, &response);
if !response.status().is_success() {
return Err(AuthError::DiscoveryFailed(format!(
"HTTP {}",
@@ -595,11 +627,7 @@ pub async fn register_client(
) -> Result<ClientRegistrationResponse, AuthError> {
validate_url_safe(registration_endpoint).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(30))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let request = ClientRegistrationRequest {
client_name: "IronClaw".to_string(),
@@ -813,7 +841,7 @@ pub fn build_authorization_url(
if let Some(pkce) = pkce {
url.push_str(&format!(
"&code_challenge={}&code_challenge_method=S256",
pkce.challenge
urlencoding::encode(&pkce.challenge)
));
}
@@ -863,11 +891,7 @@ pub async fn exchange_code_for_token(
) -> Result<AccessToken, AuthError> {
validate_url_safe(token_url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(30))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
let mut params = vec![
("grant_type", "authorization_code".to_string()),
@@ -1054,11 +1078,7 @@ pub async fn refresh_access_token(
validate_url_safe(&token_url).await?;
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(30))
.redirect(reqwest::redirect::Policy::none())
.build()
.map_err(|e| AuthError::Http(e.to_string()))?;
let client = oauth_http_client()?;
// Compute canonical resource URI for RFC 8707
let resource = canonical_resource_uri(&server_config.url);
+205 -63
View File
@@ -5,7 +5,7 @@
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::atomic::{AtomicU64, Ordering};
use async_trait::async_trait;
use tokio::sync::RwLock;
@@ -58,9 +58,10 @@ pub struct McpClient {
/// Custom headers to include in every request.
custom_headers: HashMap<String, String>,
/// Whether the MCP initialize handshake has completed.
/// Used as a local idempotency guard when no session_manager is present.
initialized: AtomicBool,
/// Ensures the MCP initialize handshake runs exactly once.
/// Uses `OnceCell` to serialize concurrent callers so only one
/// actually sends the request; subsequent calls return immediately.
initialized: tokio::sync::OnceCell<InitializeResult>,
}
impl McpClient {
@@ -83,7 +84,7 @@ impl McpClient {
user_id: "default".to_string(),
server_config: None,
custom_headers: HashMap::new(),
initialized: AtomicBool::new(false),
initialized: tokio::sync::OnceCell::new(),
}
}
@@ -106,7 +107,7 @@ impl McpClient {
user_id: "default".to_string(),
server_config: None,
custom_headers: HashMap::new(),
initialized: AtomicBool::new(false),
initialized: tokio::sync::OnceCell::new(),
}
}
@@ -114,20 +115,24 @@ impl McpClient {
///
/// Use this when you have an `McpServerConfig` with custom headers but no OAuth.
/// The config must use HTTP transport (the default); for stdio/UDS use `new_with_transport`.
pub fn new_with_config(config: McpServerConfig) -> Self {
assert!(
matches!(
config.effective_transport(),
crate::tools::mcp::config::EffectiveTransport::Http
),
"new_with_config only supports HTTP transport; use new_with_transport for stdio/UDS"
);
///
/// Returns an error if the config uses a non-HTTP transport.
pub fn new_with_config(config: McpServerConfig) -> Result<Self, ToolError> {
if !matches!(
config.effective_transport(),
crate::tools::mcp::config::EffectiveTransport::Http
) {
return Err(ToolError::InvalidParameters(
"new_with_config only supports HTTP transport; use new_with_transport for stdio/UDS"
.to_string(),
));
}
let transport = Arc::new(HttpMcpTransport::new(
config.url.clone(),
config.name.clone(),
));
Self {
Ok(Self {
transport,
server_url: config.url.clone(),
server_name: config.name.clone(),
@@ -137,9 +142,9 @@ impl McpClient {
secrets: None,
user_id: "default".to_string(),
custom_headers: config.headers.clone(),
initialized: AtomicBool::new(false),
initialized: tokio::sync::OnceCell::new(),
server_config: Some(config),
}
})
}
/// Create a new authenticated MCP client.
@@ -169,7 +174,7 @@ impl McpClient {
user_id: user_id.into(),
server_config: Some(config),
custom_headers,
initialized: AtomicBool::new(false),
initialized: tokio::sync::OnceCell::new(),
}
}
@@ -205,7 +210,7 @@ impl McpClient {
user_id: user_id.into(),
server_config,
custom_headers,
initialized: AtomicBool::new(false),
initialized: tokio::sync::OnceCell::new(),
}
}
@@ -336,53 +341,64 @@ impl McpClient {
}
/// Initialize the connection to the MCP server.
///
/// Uses `OnceCell` to guarantee that exactly one caller performs the
/// handshake, even under concurrent access. Subsequent calls return
/// immediately.
pub async fn initialize(&self) -> Result<InitializeResult, ToolError> {
// Fast path: already initialized (local flag or session manager)
if self.initialized.load(Ordering::Relaxed) {
return Ok(InitializeResult::default());
}
if let Some(ref session_manager) = self.session_manager
&& session_manager.is_initialized(&self.server_name).await
{
self.initialized.store(true, Ordering::Relaxed);
return Ok(InitializeResult::default());
}
if let Some(ref session_manager) = self.session_manager {
session_manager
.get_or_create(&self.server_name, &self.server_url)
.await;
}
let result = self
.initialized
.get_or_try_init(|| async {
if let Some(ref session_manager) = self.session_manager
&& session_manager.is_initialized(&self.server_name).await
{
return Ok(InitializeResult::default());
}
if let Some(ref session_manager) = self.session_manager {
session_manager
.get_or_create(&self.server_name, &self.server_url)
.await;
}
let request = McpRequest::initialize(self.next_request_id());
let response = self.send_request(request).await?;
let request = McpRequest::initialize(self.next_request_id());
let response = self.send_request(request).await?;
if let Some(error) = response.error {
return Err(ToolError::ExternalService(format!(
"MCP initialization error: {} (code {})",
error.message, error.code
)));
}
if let Some(error) = response.error {
return Err(ToolError::ExternalService(format!(
"MCP initialization error: {} (code {})",
error.message, error.code
)));
}
let result: InitializeResult = response
.result
.ok_or_else(|| {
ToolError::ExternalService("No result in initialize response".to_string())
let init_result: InitializeResult = response
.result
.ok_or_else(|| {
ToolError::ExternalService("No result in initialize response".to_string())
})
.and_then(|r| {
serde_json::from_value(r).map_err(|e| {
ToolError::ExternalService(format!("Invalid initialize result: {}", e))
})
})?;
if let Some(ref session_manager) = self.session_manager {
session_manager.mark_initialized(&self.server_name).await;
}
let notification = McpRequest::initialized_notification();
if let Err(e) = self.send_request(notification).await {
tracing::debug!(
"Failed to send initialized notification to '{}': {}",
self.server_name,
e
);
}
Ok(init_result)
})
.and_then(|r| {
serde_json::from_value(r).map_err(|e| {
ToolError::ExternalService(format!("Invalid initialize result: {}", e))
})
})?;
.await?;
if let Some(ref session_manager) = self.session_manager {
session_manager.mark_initialized(&self.server_name).await;
}
self.initialized.store(true, Ordering::Relaxed);
let notification = McpRequest::initialized_notification();
let _ = self.send_request(notification).await;
Ok(result)
Ok(result.clone())
}
/// List available tools from the MCP server.
@@ -471,6 +487,11 @@ impl McpClient {
}
}
/// Clone the client, resetting the tools cache and initialization state.
/// The cloned client shares the same transport and session manager, so
/// re-initialization will short-circuit via the session manager check if
/// the source was already initialized. The `next_id` counter is copied
/// so that cloned clients continue with monotonically increasing IDs.
impl Clone for McpClient {
fn clone(&self) -> Self {
Self {
@@ -484,7 +505,7 @@ impl Clone for McpClient {
user_id: self.user_id.clone(),
server_config: self.server_config.clone(),
custom_headers: self.custom_headers.clone(),
initialized: AtomicBool::new(self.initialized.load(Ordering::Relaxed)),
initialized: tokio::sync::OnceCell::new(),
}
}
}
@@ -707,7 +728,7 @@ mod tests {
headers.insert("X-Custom".to_string(), "value".to_string());
let config = McpServerConfig::new("test", "http://localhost:8080").with_headers(headers);
let client = McpClient::new_with_config(config.clone());
let client = McpClient::new_with_config(config.clone()).expect("HTTP config should work");
assert_eq!(client.server_name(), "test");
assert_eq!(client.server_url(), "http://localhost:8080");
@@ -719,7 +740,7 @@ mod tests {
#[test]
fn test_new_with_config_no_headers() {
let config = McpServerConfig::new("bare", "http://localhost:9090");
let client = McpClient::new_with_config(config);
let client = McpClient::new_with_config(config).expect("HTTP config should work");
assert_eq!(client.server_name(), "bare");
assert!(client.custom_headers.is_empty());
@@ -971,4 +992,125 @@ mod tests {
assert_eq!(obj.len(), 1);
assert!(obj["outer"]["inner"].is_null());
}
// --- Issue 1 regression: new_with_config rejects non-HTTP transport ---
#[test]
fn test_new_with_config_rejects_stdio_transport() {
let config = McpServerConfig::new_stdio(
"stdio-server",
"echo",
vec!["hello".to_string()],
HashMap::new(),
);
let result = McpClient::new_with_config(config);
let err = result
.err()
.expect("stdio config must be rejected")
.to_string();
assert!(
err.contains("new_with_config only supports HTTP"),
"error should explain the restriction: {}",
err
);
}
// --- Issue 13: McpToolWrapper unit tests ---
fn make_test_mcp_tool(destructive: bool) -> McpTool {
use crate::tools::mcp::protocol::McpToolAnnotations;
McpTool {
name: "do_thing".to_string(),
description: "Does a thing".to_string(),
input_schema: serde_json::json!({
"type": "object",
"properties": {
"input": {"type": "string"}
}
}),
annotations: if destructive {
Some(McpToolAnnotations {
destructive_hint: true,
side_effects_hint: false,
read_only_hint: false,
execution_time_hint: None,
})
} else {
None
},
}
}
#[test]
fn test_mcp_tool_wrapper_name_is_prefixed() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(false),
prefixed_name: "mcp__myserver__do_thing".to_string(),
client,
};
assert_eq!(wrapper.name(), "mcp__myserver__do_thing");
}
#[test]
fn test_mcp_tool_wrapper_description() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(false),
prefixed_name: "mcp__s__do_thing".to_string(),
client,
};
assert_eq!(wrapper.description(), "Does a thing");
}
#[test]
fn test_mcp_tool_wrapper_parameters_schema() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(false),
prefixed_name: "mcp__s__do_thing".to_string(),
client,
};
let schema = wrapper.parameters_schema();
assert_eq!(schema["type"], "object");
assert!(schema["properties"]["input"].is_object());
}
#[test]
fn test_mcp_tool_wrapper_requires_sanitization() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(false),
prefixed_name: "mcp__s__do_thing".to_string(),
client,
};
assert!(
wrapper.requires_sanitization(),
"MCP tools should always require sanitization"
);
}
#[test]
fn test_mcp_tool_wrapper_approval_destructive() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(true),
prefixed_name: "mcp__s__do_thing".to_string(),
client,
};
let approval = wrapper.requires_approval(&serde_json::json!({}));
assert_eq!(approval, ApprovalRequirement::UnlessAutoApproved);
}
#[test]
fn test_mcp_tool_wrapper_approval_non_destructive() {
let client = Arc::new(McpClient::new("http://localhost:8080"));
let wrapper = McpToolWrapper {
tool: make_test_mcp_tool(false),
prefixed_name: "mcp__s__do_thing".to_string(),
client,
};
let approval = wrapper.requires_approval(&serde_json::json!({}));
assert_eq!(approval, ApprovalRequirement::Never);
}
}
+38 -6
View File
@@ -163,10 +163,8 @@ impl McpServerConfig {
}
// Remote servers must use HTTPS (localhost is allowed for development)
let url_lower = self.url.to_lowercase();
let is_localhost =
url_lower.contains("localhost") || url_lower.contains("127.0.0.1");
if !is_localhost && !url_lower.starts_with("https://") {
let is_localhost = is_localhost_url(&self.url);
if !is_localhost && !self.url.to_lowercase().starts_with("https://") {
return Err(ConfigError::InvalidConfig {
reason: "Remote MCP servers must use HTTPS".to_string(),
});
@@ -442,7 +440,12 @@ pub async fn save_mcp_servers_to(
}
let content = serde_json::to_string_pretty(config)?;
fs::write(path, content).await?;
// Write to a temporary file first, then atomically rename to avoid
// corrupting the config if the process crashes during the write.
let tmp_path = path.with_extension("json.tmp");
fs::write(&tmp_path, content).await?;
fs::rename(&tmp_path, path).await?;
Ok(())
}
@@ -570,7 +573,7 @@ pub async fn remove_mcp_server_db(
///
/// Uses `url::Url` for proper parsing so edge cases (IPv6, userinfo, ports)
/// are handled correctly without manual string splitting.
fn is_localhost_url(url: &str) -> bool {
pub(crate) fn is_localhost_url(url: &str) -> bool {
let Ok(parsed) = url::Url::parse(url) else {
return false;
};
@@ -1125,4 +1128,33 @@ mod tests {
assert!(parsed.transport.is_none());
assert_eq!(parsed.headers.get("X-Custom").unwrap(), "value");
}
// --- Issue 3 regression: is_localhost_url rejects attacker subdomains ---
#[test]
fn test_is_localhost_url_rejects_attacker_subdomain() {
// Before the fix, url.contains("localhost") matched this.
assert!(
!is_localhost_url("http://evil.localhost.attacker.com:8080/mcp"),
"attacker subdomain containing 'localhost' must not be treated as local"
);
}
#[test]
fn test_is_localhost_url_accepts_real_localhost() {
assert!(is_localhost_url("http://localhost:8080/mcp"));
assert!(is_localhost_url("https://localhost/path"));
}
#[test]
fn test_is_localhost_url_accepts_loopback_ip() {
assert!(is_localhost_url("http://127.0.0.1:3000"));
assert!(is_localhost_url("http://[::1]:3000"));
}
#[test]
fn test_is_localhost_url_rejects_remote() {
assert!(!is_localhost_url("https://mcp.example.com"));
assert!(!is_localhost_url("http://192.168.1.1:8080"));
}
}
+10
View File
@@ -18,6 +18,8 @@ pub enum McpFactoryError {
UnixConnect { name: String, reason: String },
#[error("Unix socket transport is not supported on this platform (server '{name}')")]
UnixNotSupported { name: String },
#[error("Invalid configuration for MCP server '{name}': {reason}")]
InvalidConfig { name: String, reason: String },
}
/// Create an `McpClient` from a server configuration, dispatching on the
@@ -89,10 +91,18 @@ pub async fn create_client_from_config(
))
} else {
Ok(McpClient::new_with_config(server)
.map_err(|e| McpFactoryError::InvalidConfig {
name: server_name.clone(),
reason: e.to_string(),
})?
.with_session_manager(Arc::clone(session_manager)))
}
} else {
Ok(McpClient::new_with_config(server)
.map_err(|e| McpFactoryError::InvalidConfig {
name: server_name,
reason: e.to_string(),
})?
.with_session_manager(Arc::clone(session_manager)))
}
}
+14 -9
View File
@@ -139,7 +139,7 @@ impl McpTransport for HttpMcpTransport {
.to_string();
if content_type.contains("text/event-stream") {
self.parse_sse_response(response).await
self.parse_sse_response(response, request.id).await
} else {
response.json().await.map_err(|e| {
ToolError::ExternalService(format!(
@@ -161,11 +161,14 @@ impl McpTransport for HttpMcpTransport {
}
impl HttpMcpTransport {
/// Parse a Server-Sent Events response, returning the first valid JSON-RPC
/// `data:` line as an [`McpResponse`].
/// Parse a Server-Sent Events response, returning the JSON-RPC response
/// whose `id` matches `request_id`. Non-matching events (e.g. server
/// notifications or progress updates) are skipped so that the caller
/// receives the actual result for its request.
async fn parse_sse_response(
&self,
response: reqwest::Response,
request_id: Option<u64>,
) -> Result<McpResponse, ToolError> {
use futures::StreamExt;
@@ -202,9 +205,10 @@ impl HttpMcpTransport {
remaining_start = i + 1;
if let Some(json_str) = line.strip_prefix("data: ")
&& let Ok(response) = serde_json::from_str::<McpResponse>(json_str)
&& let Ok(resp) = serde_json::from_str::<McpResponse>(json_str)
&& resp.id == request_id
{
return Ok(response);
return Ok(resp);
}
}
}
@@ -216,14 +220,15 @@ impl HttpMcpTransport {
// Process any remaining data without a trailing newline.
if let Some(json_str) = buffer.strip_prefix("data: ")
&& let Ok(response) = serde_json::from_str::<McpResponse>(json_str.trim())
&& let Ok(resp) = serde_json::from_str::<McpResponse>(json_str.trim())
&& resp.id == request_id
{
return Ok(response);
return Ok(resp);
}
Err(ToolError::ExternalService(format!(
"[{}] No valid data in SSE response: {}",
self.server_name, buffer
"[{}] No matching response (id={:?}) in SSE stream",
self.server_name, request_id
)))
}
}
+9 -58
View File
@@ -14,7 +14,7 @@ use tokio::sync::{Mutex, oneshot};
use tokio::task::JoinHandle;
use crate::tools::mcp::protocol::{McpRequest, McpResponse};
use crate::tools::mcp::transport::{McpTransport, spawn_jsonrpc_reader, write_jsonrpc_line};
use crate::tools::mcp::transport::{McpTransport, spawn_jsonrpc_reader, stream_transport_send};
use crate::tools::tool::ToolError;
/// MCP transport that communicates with a child process over stdin/stdout.
@@ -118,63 +118,14 @@ impl McpTransport for StdioMcpTransport {
request: &McpRequest,
_headers: &HashMap<String, String>,
) -> Result<McpResponse, ToolError> {
// JSON-RPC notifications (no id) are fire-and-forget: the server
// will not send a response, so we must not wait for one.
if request.id.is_none() {
let mut stdin = self.stdin.lock().await;
write_jsonrpc_line(&mut *stdin, request).await?;
return Ok(McpResponse {
jsonrpc: "2.0".to_string(),
id: None,
result: None,
error: None,
});
}
let id = request.id.unwrap_or(0);
let (tx, rx) = oneshot::channel();
// Register the pending response handler before writing the request,
// so we don't miss a fast response from the child.
{
let mut pending = self.pending.lock().await;
pending.insert(id, tx);
}
// Write the request to stdin.
{
let mut stdin = self.stdin.lock().await;
if let Err(e) = write_jsonrpc_line(&mut *stdin, request).await {
// Remove the pending entry on write failure.
let mut pending = self.pending.lock().await;
pending.remove(&id);
return Err(e);
}
}
// Wait for the response with a timeout.
let timeout = Duration::from_secs(30);
match tokio::time::timeout(timeout, rx).await {
Ok(Ok(response)) => Ok(response),
Ok(Err(_)) => {
// Sender was dropped (reader task ended). Clean up pending entry.
let mut pending = self.pending.lock().await;
pending.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] MCP server closed connection before responding to request {:?}",
self.server_name, request.id
)))
}
Err(_) => {
// Timeout: remove the pending entry.
let mut pending = self.pending.lock().await;
pending.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] Timeout waiting for response to request {:?} after {:?}",
self.server_name, request.id, timeout
)))
}
}
stream_transport_send(
&self.stdin,
&self.pending,
request,
&self.server_name,
Duration::from_secs(30),
)
.await
}
async fn shutdown(&self) -> Result<(), ToolError> {
+105 -1
View File
@@ -97,7 +97,13 @@ pub fn spawn_jsonrpc_reader<R: AsyncBufRead + Unpin + Send + 'static>(
}
};
let id = response.id.unwrap_or(0);
let Some(id) = response.id else {
tracing::debug!(
"[{}] Received JSON-RPC notification (no id), skipping dispatch",
server_name
);
continue;
};
let mut map = pending.lock().await;
if let Some(tx) = map.remove(&id) {
// Ignore send error — the receiver may have been dropped (timeout).
@@ -115,6 +121,76 @@ pub fn spawn_jsonrpc_reader<R: AsyncBufRead + Unpin + Send + 'static>(
})
}
/// Send a JSON-RPC request over a stream-based transport (stdio / unix socket).
///
/// Handles notification fire-and-forget, pending response registration,
/// write, timeout, and cleanup. Used by both [`StdioMcpTransport`] and
/// [`UnixMcpTransport`] to avoid duplicating the send logic.
pub(crate) async fn stream_transport_send<W: AsyncWrite + Unpin>(
writer: &Mutex<W>,
pending: &Mutex<HashMap<u64, oneshot::Sender<McpResponse>>>,
request: &McpRequest,
server_name: &str,
timeout_duration: std::time::Duration,
) -> Result<McpResponse, ToolError> {
// JSON-RPC notifications (no id) are fire-and-forget: the server
// will not send a response, so we must not wait for one.
if request.id.is_none() {
let mut w = writer.lock().await;
write_jsonrpc_line(&mut *w, request).await?;
return Ok(McpResponse {
jsonrpc: "2.0".to_string(),
id: None,
result: None,
error: None,
});
}
let id = request.id.unwrap_or(0);
let (tx, rx) = oneshot::channel();
// Register the pending response handler before writing the request,
// so we don't miss a fast response from the server.
{
let mut map = pending.lock().await;
map.insert(id, tx);
}
// Write the request.
{
let mut w = writer.lock().await;
if let Err(e) = write_jsonrpc_line(&mut *w, request).await {
// Remove the pending entry on write failure.
let mut map = pending.lock().await;
map.remove(&id);
return Err(e);
}
}
// Wait for the response with a timeout.
match tokio::time::timeout(timeout_duration, rx).await {
Ok(Ok(response)) => Ok(response),
Ok(Err(_)) => {
// Sender was dropped (reader task ended). Clean up pending entry.
let mut map = pending.lock().await;
map.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] MCP server closed connection before responding to request {:?}",
server_name, request.id
)))
}
Err(_) => {
// Timeout: remove the pending entry.
let mut map = pending.lock().await;
map.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] Timeout waiting for response to request {:?} after {:?}",
server_name, request.id, timeout_duration
)))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -193,4 +269,32 @@ mod tests {
handle.await.expect("reader task should finish");
}
/// Issue 9 regression: a JSON-RPC notification (no id) must not resolve
/// a pending request keyed by id 0 (the old `unwrap_or(0)` default).
#[tokio::test]
async fn test_notification_does_not_resolve_pending_id_zero() {
// A notification response (no id), followed by a proper response for id 0.
let notification = r#"{"jsonrpc":"2.0","method":"notifications/progress","params":{}}"#;
let real_response = r#"{"jsonrpc":"2.0","id":0,"result":{"ok":true}}"#;
let input = format!("{notification}\n{real_response}\n");
let reader = std::io::Cursor::new(input.into_bytes());
let pending: Arc<Mutex<HashMap<u64, oneshot::Sender<McpResponse>>>> =
Arc::new(Mutex::new(HashMap::new()));
let (tx, rx) = oneshot::channel();
{
let mut map = pending.lock().await;
map.insert(0, tx);
}
let handle = spawn_jsonrpc_reader(reader, pending.clone(), "test".into());
let resp = rx.await.expect("should receive the real id=0 response");
assert_eq!(resp.id, Some(0));
assert!(resp.result.is_some());
handle.await.expect("reader task should finish");
}
}
+9 -58
View File
@@ -15,7 +15,7 @@ use tokio::sync::{Mutex, oneshot};
use tokio::task::JoinHandle;
use crate::tools::mcp::protocol::{McpRequest, McpResponse};
use crate::tools::mcp::transport::{McpTransport, spawn_jsonrpc_reader, write_jsonrpc_line};
use crate::tools::mcp::transport::{McpTransport, spawn_jsonrpc_reader, stream_transport_send};
use crate::tools::tool::ToolError;
/// MCP transport that communicates over a Unix domain socket.
@@ -91,63 +91,14 @@ impl McpTransport for UnixMcpTransport {
request: &McpRequest,
_headers: &HashMap<String, String>,
) -> Result<McpResponse, ToolError> {
// JSON-RPC notifications (no id) are fire-and-forget: the server
// will not send a response, so we must not wait for one.
if request.id.is_none() {
let mut writer = self.writer.lock().await;
write_jsonrpc_line(&mut *writer, request).await?;
return Ok(McpResponse {
jsonrpc: "2.0".to_string(),
id: None,
result: None,
error: None,
});
}
let id = request.id.unwrap_or(0);
let (tx, rx) = oneshot::channel();
// Register the pending response handler before writing the request,
// so we don't miss a fast response from the server.
{
let mut pending = self.pending.lock().await;
pending.insert(id, tx);
}
// Write the request to the socket.
{
let mut writer = self.writer.lock().await;
if let Err(e) = write_jsonrpc_line(&mut *writer, request).await {
// Remove the pending entry on write failure.
let mut pending = self.pending.lock().await;
pending.remove(&id);
return Err(e);
}
}
// Wait for the response with a timeout.
let timeout = Duration::from_secs(30);
match tokio::time::timeout(timeout, rx).await {
Ok(Ok(response)) => Ok(response),
Ok(Err(_)) => {
// Sender was dropped (reader task ended). Clean up pending entry.
let mut pending = self.pending.lock().await;
pending.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] MCP server closed connection before responding to request {:?}",
self.server_name, request.id
)))
}
Err(_) => {
// Timeout: remove the pending entry.
let mut pending = self.pending.lock().await;
pending.remove(&id);
Err(ToolError::ExternalService(format!(
"[{}] Timeout waiting for response to request {:?} after {:?}",
self.server_name, request.id, timeout
)))
}
}
stream_transport_send(
&self.writer,
&self.pending,
request,
&self.server_name,
Duration::from_secs(30),
)
.await
}
async fn shutdown(&self) -> Result<(), ToolError> {
+12
View File
@@ -75,6 +75,7 @@ const PROTECTED_TOOL_NAMES: &[&str] = &[
"image_generate",
"image_edit",
"image_analyze",
"tool_info",
];
/// Registry of available tools.
@@ -245,6 +246,17 @@ impl ToolRegistry {
tracing::debug!("Registered {} built-in tools", self.count());
}
/// Register the `tool_info` discovery tool.
///
/// Requires `Arc<Self>` so the tool can query the registry for other tools'
/// schemas at runtime. Call after `register_builtin_tools()`.
pub fn register_tool_info(self: &Arc<Self>) {
use crate::tools::builtin::ToolInfoTool;
let tool = ToolInfoTool::new(Arc::downgrade(self));
self.register_sync(Arc::new(tool));
tracing::debug!("Registered tool_info discovery tool");
}
/// Register only orchestrator-domain tools (safe for the main process).
///
/// This registers tools that don't touch the filesystem or run shell commands:
+2 -53
View File
@@ -558,48 +558,7 @@ mod tests {
// Routine tools
(
"routine_create",
serde_json::json!({
"type": "object",
"properties": {
"name": { "type": "string", "description": "Routine name" },
"description": { "type": "string", "description": "What it does" },
"trigger_type": {
"type": "string",
"enum": ["cron", "event", "system_event", "manual"],
"description": "When the routine fires"
},
"schedule": { "type": "string", "description": "Cron expression" },
"event_pattern": { "type": "string", "description": "Regex pattern" },
"event_channel": { "type": "string", "description": "Channel filter" },
"event_source": { "type": "string", "description": "System event source" },
"event_type": { "type": "string", "description": "System event type" },
"event_filters": {
"type": "object",
"additionalProperties": { "type": "string" },
"description": "Exact-match payload filters"
},
"prompt": { "type": "string", "description": "Instructions" },
"context_paths": {
"type": "array",
"items": { "type": "string" },
"description": "Workspace paths to load"
},
"action_type": {
"type": "string",
"enum": ["lightweight", "full_job"],
"description": "Execution mode"
},
"cooldown_secs": { "type": "integer", "description": "Min seconds between fires" },
"tool_permissions": {
"type": "array",
"items": { "type": "string" },
"description": "Pre-authorized tools for full_job mode"
},
"notify_channel": { "type": "string", "description": "Channel for message tool" },
"notify_user": { "type": "string", "description": "User/target to notify" }
},
"required": ["name", "trigger_type", "prompt"]
}),
crate::tools::builtin::routine::routine_create_parameters_schema(),
),
(
"routine_list",
@@ -611,17 +570,7 @@ mod tests {
),
(
"routine_update",
serde_json::json!({
"type": "object",
"properties": {
"name": { "type": "string", "description": "Name" },
"enabled": { "type": "boolean", "description": "Toggle" },
"prompt": { "type": "string", "description": "New prompt" },
"schedule": { "type": "string", "description": "New cron schedule" },
"description": { "type": "string", "description": "New description" }
},
"required": ["name"]
}),
crate::tools::builtin::routine::routine_update_parameters_schema(),
),
(
"routine_delete",
+59 -3
View File
@@ -336,6 +336,17 @@ pub trait Tool: Send + Sync {
None
}
/// Full parameter schema for discovery and coercion purposes.
///
/// Unlike `parameters_schema()` (which may be permissive to keep the tools
/// array compact), this returns the complete typed schema. Used by the
/// `tool_info` built-in and by WASM parameter coercion.
///
/// Default: delegates to `parameters_schema()`.
fn discovery_schema(&self) -> serde_json::Value {
self.parameters_schema()
}
/// Get the tool schema for LLM function calling.
fn schema(&self) -> ToolSchema {
ToolSchema {
@@ -419,9 +430,24 @@ pub fn redact_params(params: &serde_json::Value, sensitive: &[&str]) -> serde_js
/// Properties without a `"type"` field are allowed (freeform/any-type).
/// This is an intentional pattern used by tools like `json` and `http` for
/// OpenAI compatibility, since union types with arrays require `items`.
/// Maximum nesting depth for tool schema validation to prevent stack overflow
/// on maliciously crafted schemas.
const MAX_SCHEMA_DEPTH: usize = 16;
pub fn validate_tool_schema(schema: &serde_json::Value, path: &str) -> Vec<String> {
validate_tool_schema_inner(schema, path, 0)
}
fn validate_tool_schema_inner(schema: &serde_json::Value, path: &str, depth: usize) -> Vec<String> {
let mut errors = Vec::new();
if depth > MAX_SCHEMA_DEPTH {
errors.push(format!(
"{path}: schema nesting exceeds maximum depth of {MAX_SCHEMA_DEPTH}"
));
return errors;
}
// Rule 1: must have "type": "object" at this level
match schema.get("type").and_then(|t| t.as_str()) {
Some("object") => {}
@@ -463,14 +489,17 @@ pub fn validate_tool_schema(schema: &serde_json::Value, path: &str) -> Vec<Strin
if let Some(prop_type) = prop.get("type").and_then(|t| t.as_str()) {
match prop_type {
"object" => {
errors.extend(validate_tool_schema(prop, &prop_path));
errors.extend(validate_tool_schema_inner(prop, &prop_path, depth + 1));
}
"array" => {
if let Some(items) = prop.get("items") {
// If items is an object type, recurse
if items.get("type").and_then(|t| t.as_str()) == Some("object") {
errors
.extend(validate_tool_schema(items, &format!("{prop_path}.items")));
errors.extend(validate_tool_schema_inner(
items,
&format!("{prop_path}.items"),
depth + 1,
));
}
} else {
errors.push(format!("{prop_path}: array property missing \"items\""));
@@ -799,6 +828,33 @@ mod tests {
assert!(errors[0].contains("\"missing_field\""));
}
/// Regression test for issue #975: deeply nested schemas must not cause
/// stack overflow. The validator should stop at MAX_SCHEMA_DEPTH and
/// report an error instead of recursing infinitely.
#[test]
fn test_validate_schema_depth_limit() {
// Build a schema nested 20 levels deep (exceeds MAX_SCHEMA_DEPTH=16)
let mut schema = serde_json::json!({
"type": "object",
"properties": {
"leaf": { "type": "string" }
}
});
for _ in 0..20 {
schema = serde_json::json!({
"type": "object",
"properties": {
"nested": schema
}
});
}
let errors = validate_tool_schema(&schema, "test");
assert!(
errors.iter().any(|e| e.contains("maximum depth")),
"expected depth limit error, got: {errors:?}"
);
}
#[test]
fn test_approval_context_autonomous_allows_unless_auto_approved() {
let ctx = ApprovalContext::autonomous();
+114 -4
View File
@@ -101,24 +101,75 @@ pub struct CapabilitiesFile {
pub capabilities: Option<Box<CapabilitiesFile>>,
}
/// Maximum length for the description field to prevent memory abuse.
const MAX_DESCRIPTION_CHARS: usize = 4096;
/// Maximum serialized size of the parameters schema JSON.
const MAX_PARAMETERS_SCHEMA_BYTES: usize = 64 * 1024;
impl CapabilitiesFile {
/// Parse from JSON string.
pub fn from_json(json: &str) -> Result<Self, serde_json::Error> {
serde_json::from_str::<Self>(json).map(Self::resolve_nested)
let mut caps = serde_json::from_str::<Self>(json).map(Self::resolve_nested)?;
caps.enforce_limits();
Ok(caps)
}
/// Parse from JSON bytes.
pub fn from_bytes(bytes: &[u8]) -> Result<Self, serde_json::Error> {
serde_json::from_slice::<Self>(bytes).map(Self::resolve_nested)
let mut caps = serde_json::from_slice::<Self>(bytes).map(Self::resolve_nested)?;
caps.enforce_limits();
Ok(caps)
}
/// Truncate oversized fields to prevent unbounded memory usage.
fn enforce_limits(&mut self) {
// Truncate oversized description (issue #976)
if let Some(ref desc) = self.description
&& desc.len() > MAX_DESCRIPTION_CHARS
{
let truncated = &desc[..desc.floor_char_boundary(MAX_DESCRIPTION_CHARS)];
tracing::warn!(
"Capabilities description truncated from {} to {} chars",
desc.len(),
MAX_DESCRIPTION_CHARS,
);
self.description = Some(truncated.to_string());
}
// Drop oversized parameters schema (issue #977)
if let Some(ref params) = self.parameters {
let size = params.to_string().len();
if size > MAX_PARAMETERS_SCHEMA_BYTES {
tracing::warn!(
"Capabilities parameters schema dropped ({} bytes exceeds {} limit)",
size,
MAX_PARAMETERS_SCHEMA_BYTES,
);
self.parameters = None;
}
}
}
/// Merge nested `capabilities` wrapper into top-level fields.
///
/// Channel-level JSON nests tool capabilities under `"capabilities"`.
/// This promotes the inner fields so callers can access them uniformly.
fn resolve_nested(mut self) -> Self {
/// Maximum nesting depth for capabilities resolution.
const MAX_NESTED_DEPTH: usize = 8;
fn resolve_nested(self) -> Self {
self.resolve_nested_inner(0)
}
fn resolve_nested_inner(mut self, depth: usize) -> Self {
if depth > Self::MAX_NESTED_DEPTH {
tracing::warn!(
"Capabilities nesting exceeds maximum depth of {}, stopping resolution",
Self::MAX_NESTED_DEPTH
);
return self;
}
if let Some(inner) = self.capabilities.take() {
let inner = inner.resolve_nested();
let inner = inner.resolve_nested_inner(depth + 1);
self.description = self.description.or(inner.description);
self.parameters = self.parameters.or(inner.parameters);
self.http = self.http.or(inner.http);
@@ -1383,4 +1434,63 @@ mod tests {
"Outer description should take precedence over inner"
);
}
/// Regression test for issue #974: deeply nested capabilities wrappers
/// must not cause stack overflow. resolve_nested should stop at
/// MAX_NESTED_DEPTH and return gracefully.
#[test]
fn test_resolve_nested_depth_limit() {
// Build a capabilities file nested beyond MAX_NESTED_DEPTH (8).
// The description is at the innermost level which is beyond the limit,
// so it won't be resolved — the key assertion is no stack overflow.
let mut json = r#"{ "description": "leaf" }"#.to_string();
for _ in 0..20 {
json = format!(r#"{{ "capabilities": {json} }}"#);
}
// Should not stack overflow — this is the primary assertion.
let _caps = CapabilitiesFile::from_json(&json).unwrap();
}
/// Regression test for issue #976: oversized description strings are truncated.
#[test]
fn test_description_truncated_at_limit() {
let long_desc = "x".repeat(10_000);
let json = format!(r#"{{ "description": "{long_desc}" }}"#);
let caps = CapabilitiesFile::from_json(&json).unwrap();
let desc = caps.description.unwrap();
assert!(
desc.len() <= super::MAX_DESCRIPTION_CHARS + 50, // allow for minor overhead
"description should be truncated to ~{} chars, got {}",
super::MAX_DESCRIPTION_CHARS,
desc.len()
);
}
/// Regression test for issue #977: oversized parameters schema is dropped.
#[test]
fn test_oversized_parameters_schema_dropped() {
// Build a parameters schema larger than MAX_PARAMETERS_SCHEMA_BYTES
let mut properties = serde_json::Map::new();
for i in 0..2000 {
properties.insert(
format!("field_{i}"),
serde_json::json!({
"type": "string",
"description": "x".repeat(50)
}),
);
}
let schema = serde_json::json!({
"type": "object",
"properties": properties,
});
let json = serde_json::json!({
"parameters": schema,
});
let caps = CapabilitiesFile::from_json(&json.to_string()).unwrap();
assert!(
caps.parameters.is_none(),
"oversized parameters schema should be dropped"
);
}
}
+6 -84
View File
@@ -1,7 +1,5 @@
//! WASM sandbox error types.
use std::fmt;
use thiserror::Error;
/// Errors that can occur during WASM tool execution.
@@ -68,13 +66,13 @@ pub enum WasmError {
Timeout(std::time::Duration),
/// Component returned an error response.
/// When `hint` is non-empty it carries the tool's description and parameter
/// schema so the LLM can retry with correct arguments.
/// When `hint` is non-empty it points the LLM to `tool_info` so it can
/// fetch the tool's full parameter schema on demand.
#[error("Tool error: {message}{}", if hint.is_empty() { String::new() } else { format!("\n\nTool usage hint:\n{hint}") })]
ToolReturnedError {
/// The error message from the WASM tool.
message: String,
/// Optional description + schema hint (empty when unavailable).
/// Optional retry hint (empty when unavailable).
hint: String,
},
@@ -99,73 +97,9 @@ impl From<WasmError> for crate::tools::ToolError {
}
}
/// Details about a trap that occurred during execution.
#[derive(Debug, Clone)]
pub struct TrapInfo {
/// Human-readable trap message.
pub message: String,
/// Trap code if available.
pub code: Option<TrapCode>,
}
impl fmt::Display for TrapInfo {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match &self.code {
Some(code) => write!(f, "{}: {}", code, self.message),
None => write!(f, "{}", self.message),
}
}
}
/// Known trap codes from Wasmtime.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TrapCode {
/// Out of bounds memory access.
MemoryOutOfBounds,
/// Out of bounds table access.
TableOutOfBounds,
/// Indirect call type mismatch.
IndirectCallToNull,
/// Signature mismatch on indirect call.
BadSignature,
/// Integer overflow.
IntegerOverflow,
/// Integer division by zero.
IntegerDivisionByZero,
/// Invalid conversion to integer.
BadConversionToInteger,
/// Unreachable instruction executed.
UnreachableCodeReached,
/// Call stack exhausted.
StackOverflow,
/// Out of fuel.
OutOfFuel,
/// Unknown trap code.
Unknown,
}
impl fmt::Display for TrapCode {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let s = match self {
TrapCode::MemoryOutOfBounds => "memory out of bounds",
TrapCode::TableOutOfBounds => "table out of bounds",
TrapCode::IndirectCallToNull => "indirect call to null",
TrapCode::BadSignature => "bad signature",
TrapCode::IntegerOverflow => "integer overflow",
TrapCode::IntegerDivisionByZero => "integer division by zero",
TrapCode::BadConversionToInteger => "bad conversion to integer",
TrapCode::UnreachableCodeReached => "unreachable code reached",
TrapCode::StackOverflow => "stack overflow",
TrapCode::OutOfFuel => "out of fuel",
TrapCode::Unknown => "unknown trap",
};
write!(f, "{}", s)
}
}
#[cfg(test)]
mod tests {
use crate::tools::wasm::error::{TrapCode, TrapInfo, WasmError};
use crate::tools::wasm::error::WasmError;
#[test]
fn test_error_display() {
@@ -180,17 +114,6 @@ mod tests {
assert!(err.to_string().contains("10000000"));
}
#[test]
fn test_trap_info_display() {
let info = TrapInfo {
message: "access at offset 0x1000".to_string(),
code: Some(TrapCode::MemoryOutOfBounds),
};
let s = info.to_string();
assert!(s.contains("memory out of bounds"));
assert!(s.contains("access at offset"));
}
#[test]
fn test_conversion_to_tool_error() {
let wasm_err = WasmError::Trapped("test trap".to_string());
@@ -218,12 +141,11 @@ mod tests {
fn test_tool_returned_error_with_hint() {
let err = WasmError::ToolReturnedError {
message: "unknown action: foobar".to_string(),
hint: "Description: Gmail tool\nParameters schema: {\"type\":\"object\"}".to_string(),
hint: "Tip: call tool_info(name: \"gmail\", include_schema: true) for the full parameter schema.".to_string(),
};
let display = err.to_string();
assert!(display.contains("unknown action: foobar"));
assert!(display.contains("Tool usage hint"));
assert!(display.contains("Gmail tool"));
assert!(display.contains("Parameters schema"));
assert!(display.contains("tool_info"));
}
}
-8
View File
@@ -67,14 +67,8 @@ pub struct WasmResourceLimiter {
memory_used: u64,
/// Maximum tables allowed.
max_tables: u32,
/// Current table count.
#[allow(dead_code)] // Reserved for table limit enforcement
tables_created: u32,
/// Maximum instances allowed.
max_instances: u32,
/// Current instance count.
#[allow(dead_code)] // Reserved for instance limit enforcement
instances_created: u32,
}
impl WasmResourceLimiter {
@@ -87,9 +81,7 @@ impl WasmResourceLimiter {
memory_limit,
memory_used: 0,
max_tables: 10,
tables_created: 0,
max_instances: 10, // Component model needs multiple instances for WASI
instances_created: 0,
}
}
+1 -1
View File
@@ -96,7 +96,7 @@ pub(crate) mod storage;
mod wrapper;
// Core types
pub use error::{TrapCode, TrapInfo, WasmError};
pub use error::WasmError;
pub use host::{HostState, LogEntry, LogLevel};
pub use limits::{
DEFAULT_FUEL_LIMIT, DEFAULT_MEMORY_LIMIT, DEFAULT_TIMEOUT, FuelConfig, ResourceLimits,
+26 -41
View File
@@ -123,7 +123,9 @@ pub struct PreparedModule {
pub name: String,
/// Tool description (cached from component).
pub description: String,
/// Parameter schema JSON (cached from component).
/// Full parameter schema JSON extracted from the component.
/// Used for discovery and coercion, not necessarily for the compact
/// schema advertised in the main tools array.
pub schema: serde_json::Value,
/// Pre-compiled component (cheaply cloneable via internal Arc).
component: wasmtime::component::Component,
@@ -265,11 +267,29 @@ impl WasmToolRuntime {
let component = wasmtime::component::Component::new(&engine, &wasm_bytes)
.map_err(|e| WasmError::CompilationFailed(e.to_string()))?;
// We need to instantiate briefly to extract metadata.
// In a full implementation, we'd use WIT bindgen to get typed access.
// For now, we extract what we can from the component.
let description = extract_tool_description(&engine, &component)?;
let schema = extract_tool_schema(&engine, &component)?;
// Briefly instantiate to extract metadata (description + schema)
// from the tool's exports, analogous to MCP's list_tools().
let effective_limits = limits.clone().unwrap_or(default_limits.clone());
let (description, schema) = crate::tools::wasm::wrapper::extract_wasm_metadata(
&engine,
&component,
&effective_limits,
)
.unwrap_or_else(|e| {
tracing::warn!(
name = %name,
error = %e,
"WASM metadata extraction failed, using fallbacks"
);
(
"WASM sandboxed tool".to_string(),
serde_json::json!({
"type": "object",
"properties": {},
"additionalProperties": true
}),
)
});
Ok::<_, WasmError>(PreparedModule {
name: name.clone(),
@@ -321,41 +341,6 @@ impl WasmToolRuntime {
}
}
/// Extract tool description from a compiled component.
///
/// Returns a generic fallback. Callers should prefer loading the description
/// from the sidecar `*.capabilities.json` file and overriding via
/// `WasmToolWrapper::with_description()` or the `WasmToolRegistration::description` field.
fn extract_tool_description(
_engine: &Engine,
_component: &wasmtime::component::Component,
) -> Result<String, WasmError> {
// WIT bindgen extraction is not yet implemented (see TODO #4 in CLAUDE.md).
// Real descriptions come from the capabilities.json sidecar file, which is
// loaded by the WasmToolLoader and passed as an override at registration time.
Ok("WASM sandboxed tool".to_string())
}
/// Extract tool parameter schema from a compiled component.
///
/// Returns a permissive fallback that accepts any JSON object. Callers should
/// prefer loading the schema from the sidecar `*.capabilities.json` file and
/// overriding via `WasmToolWrapper::with_schema()` or the
/// `WasmToolRegistration::schema` field.
fn extract_tool_schema(
_engine: &Engine,
_component: &wasmtime::component::Component,
) -> Result<serde_json::Value, WasmError> {
// WIT bindgen extraction is not yet implemented (see TODO #4 in CLAUDE.md).
// Real schemas come from the capabilities.json sidecar file, which is
// loaded by the WasmToolLoader and passed as an override at registration time.
Ok(serde_json::json!({
"type": "object",
"properties": {},
"additionalProperties": true
}))
}
impl std::fmt::Debug for WasmToolRuntime {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("WasmToolRuntime")
+271 -53
View File
@@ -464,9 +464,10 @@ pub struct WasmToolWrapper {
/// Capabilities to grant to this tool.
capabilities: Capabilities,
/// Cached description (from PreparedModule or override).
/// Stored without any tool_info hints — hints are composed at display time.
description: String,
/// Cached schema (from PreparedModule or override).
schema: serde_json::Value,
/// Compact and discovery schemas for this tool.
schemas: WasmToolSchemas,
/// Injected credentials for HTTP requests (e.g., OAuth tokens).
/// Keys are placeholder names like "GOOGLE_ACCESS_TOKEN".
credentials: HashMap<String, String>,
@@ -477,6 +478,84 @@ pub struct WasmToolWrapper {
oauth_refresh: Option<OAuthRefreshConfig>,
}
#[derive(Debug, Clone)]
struct WasmToolSchemas {
/// Compact schema advertised in the main tools array.
///
/// This stays permissive by default to avoid serializing full exported
/// WASM schemas on every LLM call. Sidecars can override it explicitly.
advertised: serde_json::Value,
/// Full schema available for discovery and coercion.
///
/// Seeded from the WASM `schema()` export at registration time, unless a
/// sidecar explicitly overrides it.
discovery: serde_json::Value,
}
impl WasmToolSchemas {
fn permissive_schema() -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {},
"additionalProperties": true
})
}
fn is_permissive_schema(schema: &serde_json::Value) -> bool {
schema
.get("properties")
.and_then(|p| p.as_object())
.is_none_or(|p| p.is_empty())
}
fn new(discovery: serde_json::Value) -> Self {
Self {
advertised: Self::permissive_schema(),
discovery,
}
}
fn with_override(&self, schema: serde_json::Value) -> Self {
Self {
advertised: schema.clone(),
discovery: schema,
}
}
fn is_advertised_permissive(&self) -> bool {
Self::is_permissive_schema(&self.advertised)
}
fn advertised(&self) -> serde_json::Value {
self.advertised.clone()
}
fn discovery(&self) -> serde_json::Value {
self.discovery.clone()
}
/// Return the best schema available for type coercion.
///
/// Prefers the discovery schema when it has typed properties. Falls back
/// to the `PreparedModule` schema extracted at load time rather than
/// re-calling the WASM `schema()` export mid-execution, which could
/// interact with mutable linear memory state.
fn effective_for_coercion(&self, prepared_schema: &serde_json::Value) -> serde_json::Value {
if !Self::is_permissive_schema(&self.discovery) {
return self.discovery.clone();
}
// Fall back to the load-time extracted schema from PreparedModule.
// This avoids calling schema() on the already-running WASM instance
// where mutable state could produce inconsistent results.
if !Self::is_permissive_schema(prepared_schema) {
return prepared_schema.clone();
}
self.discovery.clone()
}
}
impl WasmToolWrapper {
/// Create a new WASM tool wrapper.
pub fn new(
@@ -486,7 +565,7 @@ impl WasmToolWrapper {
) -> Self {
Self {
description: prepared.description.clone(),
schema: prepared.schema.clone(),
schemas: WasmToolSchemas::new(prepared.schema.clone()),
runtime,
prepared,
capabilities,
@@ -504,7 +583,7 @@ impl WasmToolWrapper {
/// Override the parameter schema.
pub fn with_schema(mut self, schema: serde_json::Value) -> Self {
self.schema = schema;
self.schemas = self.schemas.with_override(schema);
self
}
@@ -615,9 +694,18 @@ impl WasmToolWrapper {
}
})?;
// Get typed interface — used for execute.
let tool_iface = instance.near_agent_tool();
// Determine effective schema for type coercion.
// Prefer the discovery schema when typed; fall back to the load-time
// extracted schema from PreparedModule rather than re-calling the WASM
// export on the already-running instance.
let effective_schema = self.schemas.effective_for_coercion(&self.prepared.schema);
// Coerce string-encoded values to their schema-declared types.
// LLMs frequently pass numeric values as strings (e.g. "5" instead of 5).
let params = coerce_params_to_schema(params, &self.schema);
let params = coerce_params_to_schema(params, &effective_schema);
// Prepare the request
let params_json = serde_json::to_string(&params)
@@ -629,7 +717,6 @@ impl WasmToolWrapper {
};
// Call execute using the generated typed interface
let tool_iface = instance.near_agent_tool();
let response = tool_iface.call_execute(&mut store, &request).map_err(|e| {
let error_str = e.to_string();
if error_str.contains("out of fuel") {
@@ -644,12 +731,13 @@ impl WasmToolWrapper {
// Get logs from host state
let logs = store.data_mut().host_state.take_logs();
// Check for tool-level error — on failure, call the WASM module's
// description() and schema() exports so the LLM can retry with the
// correct parameters without us having to include the (large) schema
// in every request's tools array.
// Check for tool-level error — point the LLM to tool_info for the
// full schema instead of dumping ~3.5KB inline.
if let Some(err) = response.error {
let hint = build_tool_hint(tool_iface, &mut store);
let hint = format!(
"Tip: call tool_info(name: \"{}\", include_schema: true) for the full parameter schema.",
self.prepared.name
);
return Err(WasmError::ToolReturnedError { message: err, hint });
}
@@ -658,47 +746,55 @@ impl WasmToolWrapper {
}
}
/// Maximum characters for the description portion of a tool hint.
const HINT_DESC_MAX: usize = 500;
/// Maximum characters for the schema portion of a tool hint.
const HINT_SCHEMA_MAX: usize = 3000;
/// Extract metadata (description + schema) from a WASM tool by briefly
/// instantiating it and calling its `description()` and `schema()` exports.
/// Analogous to MCP's `list_tools()` — discovers tool capabilities at load time.
///
/// Falls back to generic description and permissive schema on failure.
pub(super) fn extract_wasm_metadata(
engine: &wasmtime::Engine,
component: &wasmtime::component::Component,
limits: &ResourceLimits,
) -> Result<(String, serde_json::Value), WasmError> {
let store_data = StoreData::new(
limits.memory_bytes,
Capabilities::default(),
HashMap::new(),
vec![],
);
let mut store = Store::new(engine, store_data);
/// Call the WASM module's `description()` and `schema()` exports to build a
/// hint string. Returns an empty string if both calls fail or return empty.
/// Description is capped at [`HINT_DESC_MAX`] chars, schema at
/// [`HINT_SCHEMA_MAX`] chars.
fn build_tool_hint(tool_iface: &wit_tool::Guest, store: &mut Store<StoreData>) -> String {
let desc = tool_iface
.call_description(&mut *store)
// Configure fuel + epoch deadline so extraction can't hang
if let Err(e) = store.set_fuel(limits.fuel) {
tracing::debug!("Fuel not enabled for metadata extraction: {e}");
}
store.epoch_deadline_trap();
let ticks = (limits.timeout.as_millis() / EPOCH_TICK_INTERVAL.as_millis()).max(1) as u64;
store.set_epoch_deadline(ticks);
store.limiter(|data| &mut data.limiter);
// Instantiate with minimal linker
let mut linker = Linker::new(engine);
WasmToolWrapper::add_host_functions(&mut linker)?;
let instance = SandboxedTool::instantiate(&mut store, component, &linker)
.map_err(|e| WasmError::InstantiationFailed(e.to_string()))?;
let tool_iface = instance.near_agent_tool();
// Extract description (fall back to generic)
let description = tool_iface
.call_description(&mut store)
.unwrap_or_else(|_| "WASM sandboxed tool".to_string());
// Extract and parse schema (fall back to permissive)
let schema = tool_iface
.call_schema(&mut store)
.ok()
.unwrap_or_default();
let schema = tool_iface.call_schema(&mut *store).ok().unwrap_or_default();
if desc.is_empty() && schema.is_empty() {
return String::new();
}
let mut hint = String::new();
if !desc.is_empty() {
hint.push_str("Description: ");
if desc.len() > HINT_DESC_MAX {
let end = crate::util::floor_char_boundary(&desc, HINT_DESC_MAX);
hint.push_str(&desc[..end]);
hint.push('…');
} else {
hint.push_str(&desc);
}
hint.push('\n');
}
if !schema.is_empty() {
hint.push_str("Parameters schema: ");
if schema.len() > HINT_SCHEMA_MAX {
let end = crate::util::floor_char_boundary(&schema, HINT_SCHEMA_MAX);
hint.push_str(&schema[..end]);
hint.push('…');
} else {
hint.push_str(&schema);
}
}
hint
.and_then(|s| serde_json::from_str::<serde_json::Value>(&s).ok())
.unwrap_or_else(|| {
serde_json::json!({"type": "object", "properties": {}, "additionalProperties": true})
});
Ok((description, schema))
}
#[async_trait]
@@ -712,7 +808,33 @@ impl Tool for WasmToolWrapper {
}
fn parameters_schema(&self) -> serde_json::Value {
self.schema.clone()
self.schemas.advertised()
}
fn discovery_schema(&self) -> serde_json::Value {
self.schemas.discovery()
}
/// Compose the tool schema for LLM function calling.
///
/// When the advertised schema is permissive (no typed properties), appends
/// a hint to the description directing the LLM to call `tool_info` for the
/// full parameter schema. This keeps the raw description clean while still
/// guiding the LLM.
fn schema(&self) -> crate::tools::tool::ToolSchema {
let description = if self.schemas.is_advertised_permissive() {
format!(
"{} (call tool_info(name: \"{}\", include_schema: true) for parameter schema)",
self.description, self.prepared.name
)
} else {
self.description.clone()
};
crate::tools::tool::ToolSchema {
name: self.prepared.name.clone(),
description,
parameters: self.schemas.advertised(),
}
}
async fn execute(
@@ -749,7 +871,7 @@ impl Tool for WasmToolWrapper {
let prepared = Arc::clone(&self.prepared);
let capabilities = self.capabilities.clone();
let description = self.description.clone();
let schema = self.schema.clone();
let schemas = self.schemas.clone();
let credentials = self.credentials.clone();
// Execute in blocking task with timeout
@@ -759,7 +881,7 @@ impl Tool for WasmToolWrapper {
prepared,
capabilities,
description,
schema,
schemas,
credentials,
secrets_store: None, // Not needed in blocking task
oauth_refresh: None, // Already used above for pre-refresh
@@ -1232,6 +1354,7 @@ mod tests {
TEST_GOOGLE_OAUTH_TOKEN, TEST_OAUTH_CLIENT_ID, TEST_OAUTH_CLIENT_SECRET,
test_secrets_store,
};
use crate::tools::tool::Tool;
use crate::tools::wasm::capabilities::Capabilities;
use crate::tools::wasm::runtime::{WasmRuntimeConfig, WasmToolRuntime};
@@ -1246,6 +1369,84 @@ mod tests {
assert!(runtime.config().fuel_config.enabled);
}
#[tokio::test]
async fn test_advertised_schema_stays_permissive_until_sidecar_override() {
let discovery_schema = serde_json::json!({
"type": "object",
"properties": {
"query": { "type": "string" },
"limit": { "type": "integer" }
},
"required": ["query"]
});
let runtime = Arc::new(WasmToolRuntime::new(WasmRuntimeConfig::for_testing()).unwrap());
let prepared = runtime
.prepare("search", b"\0asm\x0d\0\x01\0", None)
.await
.unwrap();
let mut wrapper =
super::WasmToolWrapper::new(Arc::clone(&runtime), prepared, Capabilities::default());
wrapper.schemas = super::WasmToolSchemas::new(discovery_schema.clone());
wrapper.description = "Search documents".to_string();
// Advertised schema stays permissive; discovery holds the typed schema
assert_eq!(
wrapper.parameters_schema(),
serde_json::json!({
"type": "object",
"properties": {},
"additionalProperties": true
})
);
assert_eq!(wrapper.discovery_schema(), discovery_schema);
// Raw description is clean — no tool_info hint baked in
assert!(!wrapper.description().contains("tool_info"));
// But schema() composes the hint at display time when advertised is permissive
let schema = wrapper.schema();
assert!(
schema.description.contains("tool_info"),
"schema().description should contain tool_info hint: {}",
schema.description
);
assert!(
schema.description.contains("include_schema: true"),
"hint should mention include_schema: true: {}",
schema.description
);
// After sidecar override, both schemas match and hint disappears
let wrapper = wrapper.with_schema(serde_json::json!({
"type": "object",
"properties": {
"query": { "type": "string" }
},
"required": ["query"]
}));
assert_eq!(
wrapper.parameters_schema(),
serde_json::json!({
"type": "object",
"properties": {
"query": { "type": "string" }
},
"required": ["query"]
})
);
assert_eq!(wrapper.discovery_schema(), wrapper.parameters_schema());
// With typed schema, schema() should NOT include tool_info hint
let schema = wrapper.schema();
assert!(
!schema.description.contains("tool_info"),
"schema().description should not contain tool_info hint when typed: {}",
schema.description
);
}
#[test]
fn test_capabilities_default() {
let caps = Capabilities::default();
@@ -1788,6 +1989,23 @@ mod tests {
assert_eq!(result["count"], serde_json::json!("not-a-number"));
}
/// Regression: permissive fallback schema (empty properties) must NOT coerce.
/// This documents the bug where WASM tools with no sidecar `parameters` field
/// got the permissive fallback, causing coercion to be a no-op and LLM-provided
/// string integers to reach the WASM tool un-coerced.
#[test]
fn test_coerce_noop_with_permissive_schema() {
let permissive = serde_json::json!({
"type": "object",
"properties": {},
"additionalProperties": true
});
let params = serde_json::json!({"query": "test", "count": "10"});
let result = super::coerce_params_to_schema(params, &permissive);
// With empty properties, no coercion happens — string stays string
assert_eq!(result["count"], serde_json::json!("10"));
}
/// Regression test: leak scan must run on raw headers (before credential
/// injection), not after. If it ran post-injection, the host-injected
/// Slack bot token (`xoxb-...`) would trigger a Block and reject the
+4 -4
View File
@@ -1108,9 +1108,10 @@ impl<'a> LoopDelegate for JobDelegate<'a> {
return LoopSignal::InjectMessage(content);
}
// Check for terminal or non-progressing state. The loop should stop when the
// job has been cancelled, failed, stuck, or already completed — not just the
// three states that `is_terminal()` covers (Accepted/Failed/Cancelled).
// Check for terminal or post-completion state. The loop should stop when the
// job has been cancelled, failed, or already completed — but NOT when Stuck,
// because Stuck is recoverable (Stuck -> InProgress via self-repair).
// Stopping on Stuck would prevent recovery from resuming the worker (issue #892).
if let Ok(ctx) = self
.worker
.context_manager()
@@ -1120,7 +1121,6 @@ impl<'a> LoopDelegate for JobDelegate<'a> {
ctx.state,
JobState::Cancelled
| JobState::Failed
| JobState::Stuck
| JobState::Completed
| JobState::Submitted
| JobState::Accepted
+67 -1
View File
@@ -60,12 +60,18 @@ pub trait EmbeddingProvider: Send + Sync {
}
}
/// Default base URL for the OpenAI API.
const OPENAI_API_BASE_URL: &str = "https://api.openai.com";
/// OpenAI embedding provider using text-embedding-ada-002 or text-embedding-3-small.
///
/// Supports any OpenAI-compatible embedding endpoint via [`with_base_url`](Self::with_base_url).
pub struct OpenAiEmbeddings {
client: reqwest::Client,
api_key: String,
model: String,
dimension: usize,
base_url: String,
}
impl OpenAiEmbeddings {
@@ -78,6 +84,7 @@ impl OpenAiEmbeddings {
api_key: api_key.into(),
model: "text-embedding-3-small".to_string(),
dimension: 1536,
base_url: OPENAI_API_BASE_URL.to_string(),
}
}
@@ -88,6 +95,7 @@ impl OpenAiEmbeddings {
api_key: api_key.into(),
model: "text-embedding-ada-002".to_string(),
dimension: 1536,
base_url: OPENAI_API_BASE_URL.to_string(),
}
}
@@ -98,6 +106,7 @@ impl OpenAiEmbeddings {
api_key: api_key.into(),
model: "text-embedding-3-large".to_string(),
dimension: 3072,
base_url: OPENAI_API_BASE_URL.to_string(),
}
}
@@ -112,8 +121,35 @@ impl OpenAiEmbeddings {
api_key: api_key.into(),
model: model.into(),
dimension,
base_url: OPENAI_API_BASE_URL.to_string(),
}
}
/// Set a custom base URL for OpenAI-compatible embedding providers.
///
/// The URL must use `http://` or `https://` scheme. If no scheme is present,
/// `https://` is prepended automatically. Trailing slashes are stripped.
pub fn with_base_url(mut self, base_url: &str) -> Self {
let url = base_url.trim();
// Auto-prepend https:// if no scheme is present.
let mut url = if !url.starts_with("http://") && !url.starts_with("https://") {
tracing::debug!(
"No scheme in embedding base URL '{}', prepending https://",
url
);
format!("https://{url}")
} else {
url.to_string()
};
while url.ends_with('/') {
url.pop();
}
self.base_url = url;
self
}
}
#[derive(Debug, Serialize)]
@@ -173,9 +209,11 @@ impl EmbeddingProvider for OpenAiEmbeddings {
input: texts,
};
let url = format!("{}/v1/embeddings", self.base_url);
let response = self
.client
.post("https://api.openai.com/v1/embeddings")
.post(&url)
.header("Authorization", format!("Bearer {}", self.api_key))
.json(&request)
.send()
@@ -575,9 +613,37 @@ mod tests {
let provider = OpenAiEmbeddings::new("test-key");
assert_eq!(provider.dimension(), 1536);
assert_eq!(provider.model_name(), "text-embedding-3-small");
assert_eq!(provider.base_url, OPENAI_API_BASE_URL);
let provider = OpenAiEmbeddings::large("test-key");
assert_eq!(provider.dimension(), 3072);
assert_eq!(provider.model_name(), "text-embedding-3-large");
assert_eq!(provider.base_url, OPENAI_API_BASE_URL);
}
#[test]
fn test_openai_with_base_url_valid() {
let provider =
OpenAiEmbeddings::new("test-key").with_base_url("https://custom.example.com");
assert_eq!(provider.base_url, "https://custom.example.com");
}
#[test]
fn test_openai_with_base_url_strips_trailing_slashes() {
let provider =
OpenAiEmbeddings::new("test-key").with_base_url("https://custom.example.com///");
assert_eq!(provider.base_url, "https://custom.example.com");
}
#[test]
fn test_openai_with_base_url_http_scheme() {
let provider = OpenAiEmbeddings::new("test-key").with_base_url("http://localhost:8080");
assert_eq!(provider.base_url, "http://localhost:8080");
}
#[test]
fn test_openai_with_base_url_schemeless_prepends_https() {
let provider = OpenAiEmbeddings::new("test-key").with_base_url("custom.example.com/v1");
assert_eq!(provider.base_url, "https://custom.example.com/v1");
}
}
+77 -1
View File
@@ -20,9 +20,31 @@ from helpers import AUTH_TOKEN, wait_for_port_line, wait_for_ready
# Project root (two levels up from tests/e2e/)
ROOT = Path(__file__).resolve().parent.parent.parent
# Git main repo root (for worktree support — WASM build artifacts live
# in the main repo's tools-src/*/target/ and aren't shared across worktrees)
_MAIN_ROOT = None
try:
import subprocess as _sp
_common = _sp.check_output(
["git", "worktree", "list", "--porcelain"],
cwd=ROOT, text=True, stderr=_sp.DEVNULL,
)
for line in _common.splitlines():
if line.startswith("worktree "):
_MAIN_ROOT = Path(line.split(" ", 1)[1])
break # first entry is always the main worktree
except Exception:
pass
# Temp directory for the libSQL database file (cleaned up automatically)
_DB_TMPDIR = tempfile.TemporaryDirectory(prefix="ironclaw-e2e-")
# Temp directories for WASM extensions. These start empty and are populated by
# the install pipeline during tests; fixtures do not pre-populate dev build
# artifacts into them.
_WASM_TOOLS_TMPDIR = tempfile.TemporaryDirectory(prefix="ironclaw-e2e-wasm-tools-")
_WASM_CHANNELS_TMPDIR = tempfile.TemporaryDirectory(prefix="ironclaw-e2e-wasm-channels-")
def _find_free_port() -> int:
"""Bind to port 0 and return the OS-assigned port."""
@@ -70,7 +92,53 @@ async def mock_llm_server():
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server):
def wasm_tools_dir(_wasm_build_symlinks):
"""Empty temp dir for WASM tools.
Starts empty so the server has no pre-loaded extensions at boot.
The install API (POST /api/extensions/install) downloads and writes
WASM files here; tests exercise the full install pipeline.
NOTE on capabilities file naming: Cargo builds with underscored stems
(web_search_tool.wasm) but capabilities use hyphens (web-search-tool.
capabilities.json). The loader expects matching stems. If you pre-load
files, rename caps: web-search-tool web_search_tool.
"""
return str(Path(_WASM_TOOLS_TMPDIR.name))
@pytest.fixture(scope="session", autouse=True)
def _wasm_build_symlinks():
"""Symlink WASM build artifacts from the main repo into the worktree.
In a git worktree, tools-src/*/target/ directories don't exist because
Cargo build artifacts aren't shared. The install API's source fallback
checks these paths. Symlinking makes the fallback work without rebuilding.
"""
if _MAIN_ROOT is None or _MAIN_ROOT == ROOT:
yield
return
created = []
tools_src = ROOT / "tools-src"
main_tools_src = _MAIN_ROOT / "tools-src"
if tools_src.is_dir() and main_tools_src.is_dir():
for tool_dir in tools_src.iterdir():
if not tool_dir.is_dir():
continue
target = tool_dir / "target"
main_target = main_tools_src / tool_dir.name / "target"
if not target.exists() and main_target.is_dir():
target.symlink_to(main_target)
created.append(target)
yield
for link in created:
if link.is_symlink():
link.unlink()
@pytest.fixture(scope="session")
async def ironclaw_server(ironclaw_binary, mock_llm_server, wasm_tools_dir):
"""Start the ironclaw gateway. Yields the base URL."""
gateway_port = _find_free_port()
env = {
@@ -95,8 +163,16 @@ async def ironclaw_server(ironclaw_binary, mock_llm_server):
"ROUTINES_ENABLED": "false",
"HEARTBEAT_ENABLED": "false",
"EMBEDDING_ENABLED": "false",
# WASM tool/channel support
"WASM_ENABLED": "true",
"WASM_TOOLS_DIR": wasm_tools_dir,
"WASM_CHANNELS_DIR": _WASM_CHANNELS_TMPDIR.name,
# Prevent onboarding wizard from triggering
"ONBOARD_COMPLETED": "true",
# Force gateway OAuth callback mode (non-loopback URL) and point
# token exchange at mock_llm.py so OAuth tests work without Google.
"IRONCLAW_OAUTH_CALLBACK_URL": "https://oauth.test.example/oauth/callback",
"IRONCLAW_OAUTH_EXCHANGE_URL": mock_llm_server,
}
# Forward LLVM coverage instrumentation env vars when present
# (allows cargo-llvm-cov to collect profraw data from E2E runs).
+29
View File
@@ -133,3 +133,32 @@ async def wait_for_port_line(process, pattern: str, *, timeout: float = 60) -> i
if match := re.search(pattern, decoded):
return int(match.group(1))
raise TimeoutError(f"Port pattern '{pattern}' not found in stdout after {timeout}s")
# -- API helpers -----------------------------------------------------------
def auth_headers() -> dict[str, str]:
"""Return Authorization header dict for authenticated API calls."""
return {"Authorization": f"Bearer {AUTH_TOKEN}"}
async def api_get(base_url: str, path: str, **kwargs) -> httpx.Response:
"""Make an authenticated GET request to the ironclaw API."""
async with httpx.AsyncClient() as client:
return await client.get(
f"{base_url}{path}",
headers=auth_headers(),
timeout=kwargs.pop("timeout", 10),
**kwargs,
)
async def api_post(base_url: str, path: str, **kwargs) -> httpx.Response:
"""Make an authenticated POST request to the ironclaw API."""
async with httpx.AsyncClient() as client:
return await client.post(
f"{base_url}{path}",
headers=auth_headers(),
timeout=kwargs.pop("timeout", 10),
**kwargs,
)
+181 -56
View File
@@ -1,11 +1,16 @@
"""Mock OpenAI-compatible LLM server for E2E tests."""
"""Mock OpenAI-compatible LLM server for E2E tests.
Serves OpenAI-compatible endpoints for chat completions and model listing.
Supports both streaming and non-streaming responses, plus function calling
via TOOL_CALL_PATTERNS.
"""
import argparse
import asyncio
import json
import re
import time
import uuid
from aiohttp import web
CANNED_RESPONSES = [
@@ -13,85 +18,207 @@ CANNED_RESPONSES = [
(re.compile(r"2\s*\+\s*2|two plus two", re.IGNORECASE), "The answer is 4."),
(re.compile(r"skill|install", re.IGNORECASE), "I can help you with skills management."),
(re.compile(r"html.?test|injection.?test", re.IGNORECASE),
'Here is some content: <script>alert("xss")</script> and <img src=x onerror="alert(1)"> and <iframe src="javascript:alert(2)"></iframe> end of content.'),
'Here is some content: <script>alert("xss")</script> and <img src=x onerror="alert(1)">'
' and <iframe src="javascript:alert(2)"></iframe> end of content.'),
]
DEFAULT_RESPONSE = "I understand your request."
TOOL_CALL_PATTERNS = [
(re.compile(r"echo (.+)", re.IGNORECASE), "echo", lambda m: {"message": m.group(1)}),
(re.compile(r"what time|current time", re.IGNORECASE), "time", lambda _: {"operation": "now"}),
]
def match_response(messages: list[dict]) -> str:
"""Find canned response for the last user message."""
def _last_user_content(messages: list[dict]) -> str:
for msg in reversed(messages):
if msg.get("role") == "user":
content = msg.get("content", "")
# Handle content that may be a list (multi-modal)
if isinstance(content, list):
content = " ".join(
part.get("text", "") for part in content if part.get("type") == "text"
p.get("text", "") for p in content if p.get("type") == "text"
)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
return content
return ""
def match_response(messages: list[dict]) -> str:
content = _last_user_content(messages)
for pattern, response in CANNED_RESPONSES:
if pattern.search(content):
return response
return DEFAULT_RESPONSE
def match_tool_call(messages: list[dict], has_tools: bool) -> dict | None:
if not has_tools:
return None
content = _last_user_content(messages)
for pattern, tool_name, args_fn in TOOL_CALL_PATTERNS:
m = pattern.search(content)
if m:
return {"tool_name": tool_name, "arguments": args_fn(m)}
return None
def _extract_tool_name(msg: dict) -> str:
"""Extract tool name from a message, checking both 'name' field and XML content."""
name = msg.get("name")
if name:
return name
# ironclaw wraps tool output as <tool_output name="...">
content = msg.get("content", "")
m = re.search(r'<tool_output\s+name="([^"]+)"', content)
if m:
return m.group(1)
return "unknown"
def _find_tool_result(messages: list[dict]) -> dict | None:
"""Find a pending tool result that appears after the last user message.
Only returns a tool result if it's a fresh result the agent is waiting
for the LLM to summarize (i.e., it follows the most recent user message).
This prevents stale tool results from earlier conversation turns from
being re-processed.
"""
# Find the position of the last user message
last_user_idx = -1
for i in range(len(messages) - 1, -1, -1):
if messages[i].get("role") == "user":
last_user_idx = i
break
# Only look for tool results after the last user message
for i in range(len(messages) - 1, last_user_idx, -1):
if messages[i].get("role") == "tool":
return {"name": _extract_tool_name(messages[i]),
"content": messages[i].get("content", "")}
return None
def _make_base(completion_id: str) -> dict:
return {"id": completion_id, "object": "chat.completion.chunk",
"created": int(time.time()), "model": "mock-model"}
async def _send_sse(resp: web.StreamResponse, data: dict):
await resp.write(f"data: {json.dumps(data)}\n\n".encode())
async def chat_completions(request: web.Request) -> web.StreamResponse:
"""Handle POST /v1/chat/completions."""
"""Handle POST /v1/chat/completions and /chat/completions."""
body = await request.json()
messages = body.get("messages", [])
stream = body.get("stream", False)
response_text = match_response(messages)
completion_id = f"mock-{uuid.uuid4().hex[:8]}"
has_tools = bool(body.get("tools"))
cid = f"mock-{uuid.uuid4().hex[:8]}"
# Tool result in messages -> text summary
tr = _find_tool_result(messages)
if tr:
text = f"The {tr['name']} tool returned: {tr['content']}"
if not stream:
return _text_response(cid, text)
return await _stream_text(request, cid, text)
# Tool-call pattern match
tc = match_tool_call(messages, has_tools)
if tc:
if not stream:
return _tool_call_response(cid, tc)
return await _stream_tool_call(request, cid, tc)
# Default text response
text = match_response(messages)
if not stream:
return web.json_response({
"id": completion_id,
"object": "chat.completion",
"created": int(time.time()),
"model": "mock-model",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": response_text},
"finish_reason": "stop",
}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(response_text.split()), "total_tokens": 15},
})
return _text_response(cid, text)
return await _stream_text(request, cid, text)
# Streaming response: split into word-boundary chunks
resp = web.StreamResponse(
status=200,
headers={"Content-Type": "text/event-stream", "Cache-Control": "no-cache"},
)
await resp.prepare(request)
# First chunk: role
chunk = {
"id": completion_id,
"object": "chat.completion.chunk",
"created": int(time.time()),
def _text_response(cid: str, text: str) -> web.Response:
return web.json_response({
"id": cid, "object": "chat.completion", "created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "delta": {"role": "assistant", "content": ""}, "finish_reason": None}],
}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
"choices": [{"index": 0, "message": {"role": "assistant", "content": text},
"finish_reason": "stop"}],
"usage": {"prompt_tokens": 10, "completion_tokens": len(text.split()), "total_tokens": 15},
})
# Content chunks: split on spaces
words = response_text.split(" ")
for i, word in enumerate(words):
text = word if i == 0 else f" {word}"
chunk["choices"][0]["delta"] = {"content": text}
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
# Final chunk: finish_reason
def _tool_call_response(cid: str, tc: dict) -> web.Response:
return web.json_response({
"id": cid, "object": "chat.completion", "created": int(time.time()),
"model": "mock-model",
"choices": [{"index": 0, "message": {
"role": "assistant", "content": None,
"tool_calls": [{"id": f"call_{uuid.uuid4().hex[:8]}", "type": "function",
"function": {"name": tc["tool_name"],
"arguments": json.dumps(tc["arguments"])}}],
}, "finish_reason": "tool_calls"}],
"usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15},
})
async def _stream_text(request: web.Request, cid: str, text: str) -> web.StreamResponse:
resp = web.StreamResponse(status=200, headers={
"Content-Type": "text/event-stream", "Cache-Control": "no-cache"})
await resp.prepare(request)
base = _make_base(cid)
chunk = {**base, "choices": [{"index": 0, "delta": {"role": "assistant", "content": ""},
"finish_reason": None}]}
await _send_sse(resp, chunk)
for i, word in enumerate(text.split(" ")):
chunk["choices"][0]["delta"] = {"content": word if i == 0 else f" {word}"}
await _send_sse(resp, chunk)
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "stop"
await resp.write(f"data: {json.dumps(chunk)}\n\n".encode())
await _send_sse(resp, chunk)
await resp.write(b"data: [DONE]\n\n")
return resp
async def _stream_tool_call(request: web.Request, cid: str, tc: dict) -> web.StreamResponse:
resp = web.StreamResponse(status=200, headers={
"Content-Type": "text/event-stream", "Cache-Control": "no-cache"})
await resp.prepare(request)
call_id = f"call_{uuid.uuid4().hex[:8]}"
base = _make_base(cid)
# First chunk: role + tool call header with empty arguments
chunk = {**base, "choices": [{"index": 0, "delta": {
"role": "assistant",
"tool_calls": [{"index": 0, "id": call_id, "type": "function",
"function": {"name": tc["tool_name"], "arguments": ""}}],
}, "finish_reason": None}]}
await _send_sse(resp, chunk)
# Second chunk: arguments payload
chunk["choices"][0]["delta"] = {
"tool_calls": [{"index": 0, "function": {"arguments": json.dumps(tc["arguments"])}}]}
await _send_sse(resp, chunk)
# Final chunk: finish reason
chunk["choices"][0]["delta"] = {}
chunk["choices"][0]["finish_reason"] = "tool_calls"
await _send_sse(resp, chunk)
await resp.write(b"data: [DONE]\n\n")
return resp
async def oauth_exchange(request: web.Request) -> web.Response:
"""Mock OAuth token exchange proxy for E2E tests.
Accepts form params (code, redirect_uri, code_verifier) and returns
a fake token response. Called by ironclaw's exchange_via_proxy() when
IRONCLAW_OAUTH_EXCHANGE_URL is set.
"""
data = await request.post()
code = data.get("code", "")
return web.json_response({
"access_token": f"mock-token-{code}",
"refresh_token": "mock-refresh-token",
"expires_in": 3600,
})
async def models(_request: web.Request) -> web.Response:
"""Handle GET /v1/models."""
return web.json_response({
"object": "list",
"data": [{"id": "mock-model", "object": "model", "owned_by": "test"}],
@@ -102,23 +229,21 @@ def main():
parser = argparse.ArgumentParser()
parser.add_argument("--port", type=int, default=0)
args = parser.parse_args()
app = web.Application()
# Register both /v1/ and non-/v1/ paths (rig-core omits the /v1/ prefix)
app.router.add_post("/v1/chat/completions", chat_completions)
app.router.add_post("/chat/completions", chat_completions)
app.router.add_get("/v1/models", models)
# Use aiohttp's runner to get the actual bound port
import asyncio
app.router.add_get("/models", models)
app.router.add_post("/oauth/exchange", oauth_exchange)
async def start():
runner = web.AppRunner(app)
await runner.setup()
site = web.TCPSite(runner, "127.0.0.1", args.port)
await site.start()
# Extract the actual port from the bound socket
port = site._server.sockets[0].getsockname()[1]
print(f"MOCK_LLM_PORT={port}", flush=True)
# Block forever
await asyncio.Event().wait()
asyncio.run(start())
+264
View File
@@ -0,0 +1,264 @@
"""Extension OAuth round-trip e2e tests.
Tests the full internal OAuth callback pipeline: install gmail configure
(get auth_url) simulate OAuth callback verify token stored. Uses gateway
callback mode + mock token exchange (no real Google login).
The conftest sets IRONCLAW_OAUTH_CALLBACK_URL (non-loopback, forces gateway
mode) and IRONCLAW_OAUTH_EXCHANGE_URL (points to mock_llm.py's /oauth/exchange).
"""
from urllib.parse import parse_qs, urlparse
import httpx
import pytest
from helpers import api_get, api_post
# Module-level state
_gmail_installed = False
_auth_url = None
_csrf_state = None
def _extract_state(auth_url: str) -> str:
"""Extract the CSRF state parameter from an OAuth authorization URL."""
parsed = urlparse(auth_url)
qs = parse_qs(parsed.query)
assert "state" in qs, f"auth_url should contain state param: {auth_url}"
state = qs["state"][0]
assert len(state) > 0
return state
async def _get_extension(base_url, name):
"""Get a specific extension from the extensions list, or None."""
r = await api_get(base_url, "/api/extensions")
for ext in r.json().get("extensions", []):
if ext["name"] == name:
return ext
return None
async def _ensure_removed(base_url, name):
"""Remove extension if already installed."""
ext = await _get_extension(base_url, name)
if ext:
await api_post(base_url, f"/api/extensions/{name}/remove", timeout=30)
# ── Section A: Install + OAuth Initiation ────────────────────────────────
async def test_oauth_install_gmail(ironclaw_server):
"""Install gmail from registry for OAuth testing."""
global _gmail_installed
await _ensure_removed(ironclaw_server, "gmail")
r = await api_post(
ironclaw_server,
"/api/extensions/install",
json={"name": "gmail"},
timeout=180,
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True, f"Install failed: {data.get('message', '')}"
_gmail_installed = True
async def test_oauth_configure_returns_auth_url(ironclaw_server):
"""Configure with empty secrets returns an OAuth auth_url."""
global _auth_url, _csrf_state
if not _gmail_installed:
pytest.skip("gmail not installed")
r = await api_post(
ironclaw_server,
"/api/extensions/gmail/setup",
json={"secrets": {}},
timeout=30,
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True, f"Configure failed: {data.get('message', '')}"
_auth_url = data.get("auth_url")
assert _auth_url is not None, f"Expected auth_url in response: {data}"
assert "accounts.google.com" in _auth_url, (
f"auth_url should point to Google: {_auth_url}"
)
_csrf_state = _extract_state(_auth_url)
async def test_oauth_activate_returns_auth_url(ironclaw_server):
"""Activate on un-authenticated gmail returns auth_url."""
if not _gmail_installed:
pytest.skip("gmail not installed")
r = await api_post(
ironclaw_server, "/api/extensions/gmail/activate", timeout=30
)
assert r.status_code == 200
data = r.json()
# Activation may fail with auth_url or succeed with auth_url
auth_url = data.get("auth_url")
assert auth_url is not None, f"Expected auth_url in activate response: {data}"
# ── Section B: Internal OAuth Round-Trip ─────────────────────────────────
async def test_oauth_callback_exchanges_token(ironclaw_server):
"""Simulate OAuth callback with mock code — verifies token exchange."""
global _csrf_state
if not _csrf_state:
pytest.skip("No CSRF state from configure step")
# Re-configure to get a fresh pending flow (previous configure may have
# been consumed by the activate test above)
r = await api_post(
ironclaw_server,
"/api/extensions/gmail/setup",
json={"secrets": {}},
timeout=30,
)
data = r.json()
auth_url = data.get("auth_url")
if auth_url:
_csrf_state = _extract_state(auth_url)
# Hit the OAuth callback endpoint directly (public route, no auth header).
# The callback handler looks up the pending flow by state, calls
# exchange_via_proxy() which hits mock_llm.py's /oauth/exchange, and
# stores the returned fake token.
async with httpx.AsyncClient() as client:
r = await client.get(
f"{ironclaw_server}/oauth/callback",
params={"code": "mock_auth_code", "state": _csrf_state},
timeout=30,
follow_redirects=True,
)
assert r.status_code == 200, f"Callback returned {r.status_code}: {r.text[:300]}"
body = r.text.lower()
# The landing page says "<name> Connected" on success, "failed" on error
assert "connected" in body or "success" in body, (
f"Callback HTML should indicate success: {r.text[:500]}"
)
async def test_oauth_callback_replay_rejected(ironclaw_server):
"""Replaying the same callback is rejected (flow consumed on first use)."""
if not _csrf_state:
pytest.skip("No CSRF state")
async with httpx.AsyncClient() as client:
r = await client.get(
f"{ironclaw_server}/oauth/callback",
params={"code": "mock_auth_code", "state": _csrf_state},
timeout=10,
follow_redirects=True,
)
# Should fail — the flow was already consumed
body = r.text.lower()
assert "error" in body or "fail" in body or "expired" in body or r.status_code >= 400, (
f"Replay should be rejected, got status={r.status_code}: {r.text[:500]}"
)
async def test_oauth_callback_invalid_state(ironclaw_server):
"""Callback with bogus state is rejected."""
async with httpx.AsyncClient() as client:
r = await client.get(
f"{ironclaw_server}/oauth/callback",
params={"code": "x", "state": "totally-bogus-state-value"},
timeout=10,
follow_redirects=True,
)
body = r.text.lower()
assert "error" in body or "fail" in body or "expired" in body or r.status_code >= 400, (
f"Invalid state should be rejected, got status={r.status_code}: {r.text[:500]}"
)
async def test_oauth_extension_authenticated(ironclaw_server):
"""After OAuth callback, gmail shows authenticated=True."""
if not _gmail_installed:
pytest.skip("gmail not installed")
ext = await _get_extension(ironclaw_server, "gmail")
assert ext is not None, "gmail not in extensions list"
assert ext["authenticated"] is True, (
f"gmail should be authenticated after OAuth callback: {ext}"
)
async def test_oauth_tools_registered(ironclaw_server):
"""After OAuth authentication, gmail tools appear in tools endpoint."""
if not _gmail_installed:
pytest.skip("gmail not installed")
ext = await _get_extension(ironclaw_server, "gmail")
assert ext is not None
# Check the extension's tools array
tools = ext.get("tools", [])
assert len(tools) > 0, (
f"gmail should have tools registered after auth: {ext}"
)
async def test_remove_during_pending_oauth_invalidates_callback(ironclaw_server):
"""Removing an extension while OAuth is pending invalidates the callback state."""
if not _gmail_installed:
pytest.skip("gmail not installed")
r = await api_post(
ironclaw_server,
"/api/extensions/gmail/setup",
json={"secrets": {}},
timeout=30,
)
assert r.status_code == 200
data = r.json()
auth_url = data.get("auth_url")
assert auth_url is not None, f"Expected auth_url in response: {data}"
callback_state = _extract_state(auth_url)
remove_r = await api_post(
ironclaw_server, "/api/extensions/gmail/remove", timeout=30
)
assert remove_r.status_code == 200
assert remove_r.json().get("success") is True, (
f"Removing gmail during pending OAuth should succeed: {remove_r.text[:300]}"
)
async with httpx.AsyncClient() as client:
callback_r = await client.get(
f"{ironclaw_server}/oauth/callback",
params={"code": "mock_auth_code", "state": callback_state},
timeout=30,
follow_redirects=True,
)
assert callback_r.status_code == 200
body = callback_r.text.lower()
assert "error" in body or "fail" in body or "expired" in body, (
f"Callback after removal should fail: {callback_r.text[:500]}"
)
ext = await _get_extension(ironclaw_server, "gmail")
assert ext is None, "gmail should remain removed after invalidated callback"
# ── Section C: Cleanup ──────────────────────────────────────────────────
async def test_cleanup_gmail(ironclaw_server):
"""Remove gmail (cleanup for other test files)."""
await _ensure_removed(ironclaw_server, "gmail")
ext = await _get_extension(ironclaw_server, "gmail")
assert ext is None, "gmail should be removed"
+168 -17
View File
@@ -458,6 +458,37 @@ async def test_install_wasm_channel_triggers_configure(page):
assert await modal.is_visible()
async def test_install_with_auth_url_opens_popup_and_shows_auth_prompt(page):
"""Install responses with auth_url should surface the same auth prompt used elsewhere."""
await page.evaluate("window.open = (url) => { window._lastOpenedUrl = url; }")
await mock_ext_apis(page, registry=[_REGISTRY_WASM])
async def handle_install(route):
await route.fulfill(
status=200,
content_type="application/json",
body=json.dumps({"success": True, "auth_url": "https://example.com/oauth"}),
)
await page.route("**/api/extensions/install", handle_install)
await go_to_extensions(page)
install_btn = page.locator(SEL["available_wasm_list"]).locator(SEL["ext_install_btn"]).first
await install_btn.wait_for(state="visible", timeout=5000)
await install_btn.click()
await page.wait_for_function(
"() => window._lastOpenedUrl !== null && window._lastOpenedUrl !== undefined",
timeout=5000,
)
opened = await page.evaluate("window._lastOpenedUrl")
assert opened is not None, "window.open was not called"
assert "example.com" in opened
await page.locator(SEL["auth_card"] + '[data-extension-name="registry-tool"]').wait_for(
state="visible", timeout=5000
)
# ─── Group F: Remove flow ─────────────────────────────────────────────────────
async def test_remove_installed_extension_confirmed(page):
@@ -612,7 +643,7 @@ async def test_configure_modal_save_success(page):
async def test_configure_modal_save_oauth(page):
"""Save response with auth_url opens a popup via window.open."""
"""Save response with auth_url opens a popup and shows the global auth prompt."""
await page.evaluate("window.open = (url) => { window._lastOpenedUrl = url; }")
async def handle_setup(route):
@@ -639,6 +670,9 @@ async def test_configure_modal_save_oauth(page):
opened = await page.evaluate("window._lastOpenedUrl")
assert opened is not None, "window.open was not called"
assert "oauth" in opened or "example.com" in opened
await page.locator(SEL["auth_card"] + '[data-extension-name="test-ext"]').wait_for(
state="visible", timeout=5000
)
async def test_configure_modal_save_failure(page):
@@ -699,7 +733,7 @@ async def test_configure_modal_enter_key_submits(page):
# ─── Group H: Auth card (SSE-triggered) ───────────────────────────────────────
async def _show_auth_card(page, **kwargs):
"""Inject an auth card via JS and wait for it to appear."""
"""Inject the global auth prompt via JS and wait for it to appear."""
payload = json.dumps(kwargs)
await page.evaluate(f"showAuthCard({payload})")
await page.locator(SEL["auth_card"]).wait_for(state="visible", timeout=5000)
@@ -812,12 +846,43 @@ async def test_auth_card_replaces_existing_same_extension(page):
assert "Second" in await page.locator(SEL["auth_instructions"]).text_content()
async def test_auth_card_multiple_extensions_coexist(page):
"""Auth cards for different extensions can coexist."""
async def test_auth_card_for_different_extension_replaces_existing_prompt(page):
"""A new auth prompt replaces the previous one to keep the UX modal and global."""
await page.evaluate('showAuthCard({extension_name: "ext-a", instructions: "Token A"})')
await page.evaluate('showAuthCard({extension_name: "ext-b", instructions: "Token B"})')
await page.locator(SEL["auth_card"]).nth(1).wait_for(state="visible", timeout=3000)
assert await page.locator(SEL["auth_card"]).count() == 2
await page.locator(SEL["auth_card"]).wait_for(state="visible", timeout=3000)
assert await page.locator(SEL["auth_card"]).count() == 1
assert await page.locator(SEL["auth_card"] + '[data-extension-name="ext-a"]').count() == 0
assert await page.locator(SEL["auth_card"] + '[data-extension-name="ext-b"]').count() == 1
async def test_auth_and_configure_helpers_escape_selector_sensitive_extension_names(page):
"""Quoted extension names should not break auth/configure modal helpers."""
result = await page.evaluate(
"""({ name }) => {
showAuthCard({ extension_name: name, instructions: 'Paste token' });
showAuthCardError(name, 'Bad token');
const errorText = document.querySelector('.auth-error')?.textContent || '';
removeAuthCard(name);
const authStillPresent = Array.from(document.querySelectorAll('.auth-card'))
.some((card) => card.getAttribute('data-extension-name') === name);
const overlay = document.createElement('div');
overlay.className = 'configure-overlay';
overlay.setAttribute('data-extension-name', name);
document.body.appendChild(overlay);
closeConfigureModal(name);
const configureStillPresent = Array.from(document.querySelectorAll('.configure-overlay'))
.some((node) => node.getAttribute('data-extension-name') === name);
return { errorText, authStillPresent, configureStillPresent };
}""",
{"name": 'quoted "ext" name'},
)
assert result["errorText"] == "Bad token"
assert result["authStillPresent"] is False
assert result["configureStillPresent"] is False
async def test_auth_completed_sse_dismisses_card(page):
@@ -826,13 +891,95 @@ async def test_auth_completed_sse_dismisses_card(page):
# Simulate the auth_completed SSE event being fired
await page.evaluate("""
// Call the handler the same way the SSE listener does
removeAuthCard('myext');
handleAuthCompleted({
extension_name: 'myext',
success: true,
message: 'Authenticated!',
});
""")
assert await page.locator(SEL["auth_card"] + '[data-extension-name="myext"]').count() == 0
async def test_auth_completed_for_other_extension_keeps_configure_modal_open(page):
"""Auth completion should not close a different extension's configure modal."""
async def handle_setup(route):
await route.fulfill(
status=200,
content_type="application/json",
body=json.dumps({"secrets": [{"name": "token", "prompt": "Token", "provided": False, "optional": False, "auto_generate": False}]}),
)
await page.route("**/api/extensions/test-ext/setup", handle_setup)
await page.evaluate("showConfigureModal('test-ext')")
await page.locator(SEL["configure_modal"]).wait_for(state="visible", timeout=5000)
await page.evaluate("""
handleAuthCompleted({
extension_name: 'other-ext',
success: true,
message: 'Other extension connected.',
});
""")
assert await page.locator(SEL["configure_overlay"]).is_visible(), (
"Configure modal should remain open when another extension finishes auth"
)
async def test_auth_completed_failure_sse_shows_error_toast_and_reloads_extensions(page):
"""Failed auth_completed handling should clear stale UI and refresh extensions."""
reload_count = []
async def counting_handler(route):
path = route.request.url.split("?")[0]
if path.endswith("/api/extensions"):
reload_count.append(1)
await route.fulfill(
status=200,
content_type="application/json",
body=json.dumps({"extensions": []}),
)
else:
await route.continue_()
async def handle_tools(route):
await route.fulfill(status=200, content_type="application/json", body='{"tools":[]}')
async def handle_registry(route):
await route.fulfill(status=200, content_type="application/json", body='{"entries":[]}')
await page.route("**/api/extensions*", counting_handler)
await page.route("**/api/extensions/tools", handle_tools)
await page.route("**/api/extensions/registry", handle_registry)
await go_to_extensions(page)
count_before = len(reload_count)
await _show_auth_card(page, extension_name="gmail", auth_url="https://example.com/oauth")
assert await page.locator(SEL["auth_card"] + '[data-extension-name="gmail"]').count() == 1
await page.evaluate("""
handleAuthCompleted({
extension_name: 'gmail',
success: false,
message: 'OAuth flow expired. Please try again.',
});
""")
await wait_for_toast(page, "OAuth flow expired. Please try again.")
assert await page.locator(SEL["auth_card"] + '[data-extension-name="gmail"]').count() == 0
assert (
await page.locator(
SEL["toast_error"], has_text="OAuth flow expired. Please try again."
).count()
>= 1
)
await page.wait_for_timeout(600)
assert len(reload_count) > count_before, "Extensions list did not reload after auth failure"
# ─── Group I: Activate flow ────────────────────────────────────────────────────
async def test_activate_mcp_server_success(page):
@@ -902,8 +1049,8 @@ async def test_activate_failure_shows_error_toast(page):
await wait_for_toast(page, "Config missing")
async def test_activate_with_auth_url_opens_popup(page):
"""Activate response with auth_url calls window.open."""
async def test_activate_with_auth_url_opens_popup_and_shows_auth_prompt(page):
"""Activate response with auth_url calls window.open and shows the auth prompt."""
await page.evaluate("window.open = (url) => { window._lastOpenedUrl = url; }")
await mock_ext_apis(page, installed=[_MCP_INACTIVE])
@@ -921,6 +1068,9 @@ async def test_activate_with_auth_url_opens_popup(page):
opened = await page.evaluate("window._lastOpenedUrl")
assert opened is not None, "window.open was not called"
assert "example.com" in opened
await page.locator(
SEL["auth_card"] + '[data-extension-name="test-mcp-inactive"]'
).wait_for(state="visible", timeout=5000)
# ─── Group J: Tab reload behaviour ────────────────────────────────────────────
@@ -947,9 +1097,9 @@ async def test_extensions_tab_reloads_on_revisit(page):
async def handle_registry(route):
await route.fulfill(status=200, content_type="application/json", body='{"entries":[]}')
await page.route("**/api/extensions*", counting_handler)
await page.route("**/api/extensions/tools", handle_tools)
await page.route("**/api/extensions/registry", handle_registry)
await page.route("**/api/extensions*", counting_handler)
# First visit
await go_to_extensions(page)
@@ -990,19 +1140,20 @@ async def test_auth_completed_sse_triggers_extensions_reload(page):
async def handle_registry(route):
await route.fulfill(status=200, content_type="application/json", body='{"entries":[]}')
await page.route("**/api/extensions*", counting_handler)
await page.route("**/api/extensions/tools", handle_tools)
await page.route("**/api/extensions/registry", handle_registry)
await page.route("**/api/extensions*", counting_handler)
await go_to_extensions(page)
count_before = len(reload_count)
# Simulate auth_completed by calling loadExtensions directly (as the SSE handler does)
# Simulate auth_completed via the shared handler.
await page.evaluate("""
// Simulate what the auth_completed SSE handler does when currentTab === 'extensions'
if (typeof loadExtensions === 'function') {
loadExtensions();
}
handleAuthCompleted({
extension_name: 'reload-ext',
success: true,
message: 'Reloaded.',
});
""")
await page.wait_for_timeout(600)
+79
View File
@@ -0,0 +1,79 @@
"""DM pairing flow e2e tests.
Tests the pairing security gate for WASM channels: listing pending requests,
approving codes, and error handling.
"""
import httpx
from helpers import AUTH_TOKEN
def _headers():
return {"Authorization": f"Bearer {AUTH_TOKEN}"}
async def test_pairing_list_returns_empty_for_unknown_channel(ironclaw_server):
"""GET /api/pairing/{channel} returns empty list or 404 for non-existent channel."""
async with httpx.AsyncClient() as client:
r = await client.get(
f"{ironclaw_server}/api/pairing/nonexistent-channel",
headers=_headers(),
timeout=10,
)
# Either empty list or error is acceptable
if r.status_code == 200:
data = r.json()
assert isinstance(data, (dict, list))
if isinstance(data, dict):
assert "requests" in data
assert isinstance(data["requests"], list)
assert data["requests"] == []
else:
assert data == []
else:
# 404 or similar is fine for non-existent channel
assert r.status_code in (404, 400)
async def test_approve_invalid_code_rejected(ironclaw_server):
"""POST /api/pairing/{channel}/approve with bad code returns error."""
async with httpx.AsyncClient() as client:
r = await client.post(
f"{ironclaw_server}/api/pairing/test-channel/approve",
json={"code": "INVALID0"},
headers=_headers(),
timeout=10,
)
# Should fail — no pending request with this code
if r.status_code == 200:
data = r.json()
assert data.get("success") is False or data.get("ok") is False or "error" in str(data).lower()
else:
assert r.status_code >= 400
async def test_approve_empty_code_rejected(ironclaw_server):
"""POST /api/pairing/{channel}/approve with empty code returns error."""
async with httpx.AsyncClient() as client:
r = await client.post(
f"{ironclaw_server}/api/pairing/test-channel/approve",
json={"code": ""},
headers=_headers(),
timeout=10,
)
if r.status_code == 200:
data = r.json()
assert data.get("success") is False or data.get("ok") is False
else:
assert r.status_code >= 400
async def test_pairing_approve_requires_auth(ironclaw_server):
"""POST /api/pairing/{channel}/approve without auth token is rejected."""
async with httpx.AsyncClient() as client:
r = await client.post(
f"{ironclaw_server}/api/pairing/test-channel/approve",
json={"code": "ABCD1234"},
timeout=10,
)
assert r.status_code == 401 or r.status_code == 403
@@ -0,0 +1,94 @@
"""Tool execution e2e tests.
Tests the agent loop: user message -> mock LLM returns tool_calls -> tool
executes -> result displayed in chat. Requires the enhanced mock_llm.py
with TOOL_CALL_PATTERNS support.
"""
from helpers import SEL
async def _send_and_get_response(
page,
message: str,
*,
expected_fragment: str,
timeout: int = 30000,
) -> str:
"""Send a message and return the text of the newest assistant response.
Counts existing assistant messages before sending, then waits for a new
one to appear and contain the expected final text fragment. This avoids
reading partial streamed content before the assistant response is complete.
"""
chat_input = page.locator(SEL["chat_input"])
await chat_input.wait_for(state="visible", timeout=5000)
# Count existing assistant messages before sending
assistant_sel = SEL["message_assistant"]
before_count = await page.locator(assistant_sel).count()
await chat_input.fill(message)
await chat_input.press("Enter")
# Wait for the final assistant message to exist and include the expected
# text fragment rather than returning on the first streamed chunk.
expected = before_count + 1
await page.wait_for_function(
"""({ assistantSelector, expectedCount, expectedFragment }) => {
const messages = document.querySelectorAll(assistantSelector);
if (messages.length < expectedCount) return false;
const text = (messages[messages.length - 1].innerText || '').trim().toLowerCase();
return text.includes(expectedFragment.toLowerCase());
}""",
arg={
"assistantSelector": assistant_sel,
"expectedCount": expected,
"expectedFragment": expected_fragment,
},
timeout=timeout,
)
return await page.locator(assistant_sel).last.inner_text()
async def test_builtin_echo_tool(page):
"""Send a message that triggers the echo tool via mock LLM function calling."""
text = await _send_and_get_response(
page,
"echo hello world",
expected_fragment="hello world",
)
# The mock LLM returns "The echo tool returned: <result>"
assert "echo" in text.lower() or "hello world" in text.lower(), (
f"Expected echo result in response, got: {text}"
)
async def test_builtin_time_tool(page):
"""Send a message that triggers the time tool via mock LLM function calling."""
text = await _send_and_get_response(
page,
"what time is it",
expected_fragment="time",
)
# The mock LLM returns "The time tool returned: <json with iso/unix>"
assert "time" in text.lower(), (
f"Expected time result in response, got: {text}"
)
async def test_non_tool_message_still_works(page):
"""Messages that don't match tool patterns still get text responses."""
text = await _send_and_get_response(
page,
"What is 2+2?",
expected_fragment="4",
timeout=15000,
)
assert "4" in text, (
f"Expected '4' in response, got: {text}"
)
+517
View File
@@ -0,0 +1,517 @@
"""Comprehensive WASM extension lifecycle e2e tests.
Tests the full extension pipeline: registry install fields configure
activate tools remove reinstall. Validates response fields, not just
status codes, to catch production bugs like missing capabilities, wrong
activation state, and stale registry flags.
Lifecycle stages are expressed as scoped fixtures so each test requests the
state it needs explicitly rather than relying on module-global flags.
"""
from pathlib import Path
import pytest
from helpers import SEL, api_get, api_post
async def _get_extension(base_url, name):
"""Get a specific extension from the extensions list, or None."""
r = await api_get(base_url, "/api/extensions")
for ext in r.json().get("extensions", []):
if ext["name"] == name:
return ext
return None
async def _ensure_removed(base_url, name):
"""Remove extension if already installed (idempotent cleanup)."""
ext = await _get_extension(base_url, name)
if ext:
await api_post(base_url, f"/api/extensions/{name}/remove", timeout=30)
async def _install_extension(base_url, name):
"""Install an extension and assert success."""
r = await api_post(
base_url,
"/api/extensions/install",
json={"name": name},
timeout=180,
)
assert r.status_code == 200, f"Install HTTP error: {r.status_code} {r.text[:300]}"
data = r.json()
assert data.get("success") is True, f"Install failed: {data.get('message', '')}"
return data
@pytest.fixture(scope="module", autouse=True)
async def extension_lifecycle_cleanup(ironclaw_server):
"""Start and end the module with a clean extension set."""
await _ensure_removed(ironclaw_server, "web-search")
await _ensure_removed(ironclaw_server, "gmail")
yield
await _ensure_removed(ironclaw_server, "web-search")
await _ensure_removed(ironclaw_server, "gmail")
@pytest.fixture(scope="module")
async def web_search_installed(ironclaw_server, extension_lifecycle_cleanup):
"""Install web-search once for tests that require the pre-configure state."""
data = await _install_extension(ironclaw_server, "web-search")
return {"name": "web-search", "install": data}
@pytest.fixture(scope="module")
async def web_search_configured(ironclaw_server, web_search_installed):
"""Configure web-search once for tests that require the active state."""
r = await api_post(
ironclaw_server,
"/api/extensions/web-search/setup",
json={"secrets": {"brave_api_key": "test-key-123"}},
timeout=30,
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True, f"Configure failed: {data.get('message', '')}"
assert data.get("activated") is True, "Should auto-activate after configure"
return {"name": "web-search", "configure": data}
@pytest.fixture(scope="module")
async def gmail_installed(ironclaw_server, extension_lifecycle_cleanup):
"""Install gmail once for multi-extension and OAuth setup assertions."""
data = await _install_extension(ironclaw_server, "gmail")
return {"name": "gmail", "install": data}
@pytest.fixture(scope="module")
async def web_search_removed(ironclaw_server, web_search_configured):
"""Remove web-search once for post-uninstall assertions."""
r = await api_post(
ironclaw_server, "/api/extensions/web-search/remove", timeout=30
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True, f"Remove failed: {data.get('message', '')}"
return {"name": "web-search", "remove": data}
@pytest.fixture(scope="module")
async def web_search_reinstalled(ironclaw_server, web_search_removed):
"""Reinstall web-search after removal to verify saved-secret recovery."""
await _ensure_removed(ironclaw_server, "web-search")
data = await _install_extension(ironclaw_server, "web-search")
return {"name": "web-search", "install": data}
# ── Section A: Registry Validation ──────────────────────────────────────
async def test_registry_lists_extensions(ironclaw_server):
"""Registry endpoint returns entries from the embedded catalog."""
r = await api_get(ironclaw_server, "/api/extensions/registry")
assert r.status_code == 200
data = r.json()
assert "entries" in data
names = [e["name"] for e in data["entries"]]
assert "web-search" in names
assert "gmail" in names
async def test_registry_entry_fields(ironclaw_server):
"""Every registry entry has all required fields with correct types."""
r = await api_get(ironclaw_server, "/api/extensions/registry")
entries = r.json()["entries"]
assert len(entries) > 0, "Registry should have entries"
for entry in entries:
assert "name" in entry and isinstance(entry["name"], str) and entry["name"]
assert "display_name" in entry and isinstance(entry["display_name"], str)
assert "kind" in entry and isinstance(entry["kind"], str)
assert "description" in entry and isinstance(entry["description"], str)
assert "installed" in entry and isinstance(entry["installed"], bool)
assert "keywords" in entry and isinstance(entry["keywords"], list)
async def test_registry_installed_flag_false_initially(ironclaw_server):
"""Before any install, all registry entries have installed=False."""
# Clean up in case previous test run left extensions installed
await _ensure_removed(ironclaw_server, "web-search")
await _ensure_removed(ironclaw_server, "gmail")
r = await api_get(ironclaw_server, "/api/extensions/registry")
entries = r.json()["entries"]
for entry in entries:
if entry["name"] in ("web-search", "gmail"):
assert entry["installed"] is False, (
f"{entry['name']} should not be installed yet"
)
async def test_registry_search_filters(ironclaw_server):
"""Search query filters registry results."""
r = await api_get(
ironclaw_server, "/api/extensions/registry", params={"query": "search"}
)
assert r.status_code == 200
entries = r.json()["entries"]
names = [e["name"] for e in entries]
assert "web-search" in names
async def test_registry_search_no_match(ironclaw_server):
"""Nonsense query returns empty results."""
r = await api_get(
ironclaw_server,
"/api/extensions/registry",
params={"query": "xyznonexistent999"},
)
assert r.status_code == 200
assert len(r.json()["entries"]) == 0
# ── Section B: Install Lifecycle (web-search) ───────────────────────────
async def test_install_web_search(web_search_installed):
"""Install web-search from registry. Asserts success — failure here means
the registry/download/build pipeline is broken."""
assert "message" in web_search_installed["install"]
async def test_installed_extension_fields(ironclaw_server, web_search_installed):
"""After install, extension list shows correct fields."""
ext = await _get_extension(ironclaw_server, "web-search")
assert ext is not None, "web-search not in extensions list after install"
assert ext["kind"] == "wasm_tool"
assert ext["needs_setup"] is True, "Should need setup (has brave_api_key secret)"
assert ext["authenticated"] is False, "Should not be authenticated before configure"
async def test_installed_in_registry(ironclaw_server, web_search_installed):
"""Registry marks installed extension with installed=True."""
r = await api_get(ironclaw_server, "/api/extensions/registry")
entries = r.json()["entries"]
ws_entry = next((e for e in entries if e["name"] == "web-search"), None)
assert ws_entry is not None
assert ws_entry["installed"] is True, "Registry should show installed=True"
async def test_setup_schema_has_secrets(ironclaw_server, web_search_installed):
"""Setup schema returns brave_api_key with correct field info."""
r = await api_get(ironclaw_server, "/api/extensions/web-search/setup")
assert r.status_code == 200
data = r.json()
assert "secrets" in data
secrets = {s["name"]: s for s in data["secrets"]}
assert "brave_api_key" in secrets, (
f"brave_api_key not in setup schema secrets: {list(secrets.keys())}"
)
key_info = secrets["brave_api_key"]
assert key_info["provided"] is False, "Should not be provided yet"
async def test_extension_not_authenticated_before_configure(
ironclaw_server, web_search_installed
):
"""Installed but not configured extension is not authenticated."""
ext = await _get_extension(ironclaw_server, "web-search")
assert ext is not None
# Before configuring secrets, extension shouldn't be fully authenticated
assert ext["needs_setup"] is True, "Should still need setup before configure"
async def test_activate_before_configure_rejected(ironclaw_server, web_search_installed):
"""Activating a tool that needs setup secrets is rejected."""
r = await api_post(
ironclaw_server, "/api/extensions/web-search/activate", timeout=30
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is False, (
f"Activate should fail before configure: {data}"
)
msg = data.get("message", "").lower()
assert "requires configuration" in msg or "setup" in msg, (
f"Error should mention configuration: {data.get('message')}"
)
# ── Section C: Configure + Activate (web-search) ────────────────────────
async def test_configure_rejects_unknown_secret(ironclaw_server, web_search_installed):
"""Submitting an unknown secret name is rejected."""
r = await api_post(
ironclaw_server,
"/api/extensions/web-search/setup",
json={"secrets": {"fake_unknown_key": "value"}},
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is False, f"Should reject unknown secret: {data}"
assert "unknown" in data.get("message", "").lower() or "not found" in data.get(
"message", ""
).lower(), f"Error should mention unknown secret: {data.get('message')}"
async def test_configure_with_valid_secret(web_search_configured):
"""Configure with valid brave_api_key succeeds and auto-activates."""
assert web_search_configured["configure"].get("activated") is True
async def test_extension_active_after_configure(ironclaw_server, web_search_configured):
"""After configure, extension shows authenticated=True and active=True."""
ext = await _get_extension(ironclaw_server, "web-search")
assert ext is not None
assert ext["authenticated"] is True, "Should be authenticated after configure"
assert ext["active"] is True, "Should be active after auto-activation"
assert len(ext.get("tools", [])) > 0, "Should have tools registered"
async def test_setup_shows_provided(ironclaw_server, web_search_configured):
"""After configure, setup schema shows secret as provided."""
r = await api_get(ironclaw_server, "/api/extensions/web-search/setup")
assert r.status_code == 200
secrets = {s["name"]: s for s in r.json()["secrets"]}
assert "brave_api_key" in secrets
assert secrets["brave_api_key"]["provided"] is True
async def test_tools_registered_after_activate(
ironclaw_server, web_search_configured
):
"""After activation, extension tools appear in the tools endpoint."""
r = await api_get(ironclaw_server, "/api/extensions/tools")
assert r.status_code == 200
tool_names = [t["name"] for t in r.json()["tools"]]
assert "web-search" in tool_names, (
f"web-search tool not found in tools list: {tool_names}"
)
async def test_activate_already_active_idempotent(
ironclaw_server, web_search_configured
):
"""Activating an already-active extension succeeds (idempotent)."""
r = await api_post(
ironclaw_server, "/api/extensions/web-search/activate", timeout=30
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True, (
f"Re-activation should succeed: {data.get('message', '')}"
)
async def test_configure_empty_secret_skipped(ironclaw_server, web_search_configured):
"""Submitting an empty string for a secret skips it (doesn't overwrite)."""
r = await api_post(
ironclaw_server,
"/api/extensions/web-search/setup",
json={"secrets": {"brave_api_key": ""}},
timeout=30,
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is True
# Verify the secret is still provided (not cleared)
r2 = await api_get(ironclaw_server, "/api/extensions/web-search/setup")
secrets = {s["name"]: s for s in r2.json()["secrets"]}
assert secrets["brave_api_key"]["provided"] is True, (
"Empty value should not clear existing secret"
)
# ── Section D: Install gmail (multi-extension) ──────────────────────────
async def test_install_gmail(gmail_installed):
"""Install gmail from registry (second extension, tests isolation)."""
assert "message" in gmail_installed["install"]
async def test_gmail_fields(ironclaw_server, gmail_installed):
"""Gmail extension has correct field values (OAuth-based auth)."""
ext = await _get_extension(ironclaw_server, "gmail")
assert ext is not None, "gmail not in extensions list"
assert ext["kind"] == "wasm_tool"
assert ext["has_auth"] is True, "Gmail should have OAuth auth"
async def test_both_extensions_listed(
ironclaw_server, web_search_configured, gmail_installed
):
"""Both web-search and gmail appear in extensions list (no clobbering)."""
r = await api_get(ironclaw_server, "/api/extensions")
names = [e["name"] for e in r.json()["extensions"]]
assert "web-search" in names, f"web-search missing from: {names}"
assert "gmail" in names, f"gmail missing from: {names}"
async def test_gmail_setup_schema_auto_resolves(ironclaw_server, gmail_installed):
"""Gmail setup schema returns empty secrets (builtin creds auto-resolve)."""
r = await api_get(ironclaw_server, "/api/extensions/gmail/setup")
assert r.status_code == 200
data = r.json()
secrets = data.get("secrets", [])
# Builtin Google credentials auto-resolve client_id/client_secret via
# is_auto_resolved_oauth_field(), so the setup schema should have no
# user-facing secrets (or only auto-generated ones).
user_facing = [s for s in secrets if not s.get("auto_generate", False)]
assert len(user_facing) == 0, (
f"Gmail should have no user-facing secrets (auto-resolved), got: "
f"{[s['name'] for s in user_facing]}"
)
# ── Section E: Remove + Cleanup ─────────────────────────────────────────
async def test_remove_web_search(web_search_removed):
"""Remove web-search succeeds."""
assert web_search_removed["remove"].get("success") is True
async def test_removed_not_in_extensions(ironclaw_server, web_search_removed):
"""Removed extension no longer appears in extensions list."""
ext = await _get_extension(ironclaw_server, "web-search")
assert ext is None, "web-search should not be in extensions list after removal"
async def test_removed_extension_not_listed(ironclaw_server, web_search_removed):
"""Removed extension should not appear in the extension tools list."""
r = await api_get(ironclaw_server, "/api/extensions/tools")
assert r.status_code == 200
tool_names = [t["name"] for t in r.json()["tools"]]
assert "web-search" not in tool_names, (
f"Removed web-search tool should not remain registered: {tool_names}"
)
async def test_removed_not_in_registry_installed(ironclaw_server, web_search_removed):
"""Registry shows removed extension as installed=False."""
r = await api_get(ironclaw_server, "/api/extensions/registry")
ws_entry = next(
(e for e in r.json()["entries"] if e["name"] == "web-search"), None
)
assert ws_entry is not None
assert ws_entry["installed"] is False, "Registry should show installed=False"
async def test_activate_after_remove_uses_replacement_bytes_not_cached_module(
ironclaw_server, wasm_tools_dir, web_search_removed
):
"""After removal, activation must use the replacement bytes rather than a stale cache."""
wasm_path = Path(wasm_tools_dir) / "web-search.wasm"
wasm_path.write_bytes(b"not-a-valid-wasm-component")
r = await api_post(
ironclaw_server, "/api/extensions/web-search/activate", timeout=30
)
assert r.status_code == 200
data = r.json()
assert data.get("success") is False, (
f"Activation should fail against replacement bytes, got: {data}"
)
async def test_reinstall_after_remove(ironclaw_server, web_search_reinstalled):
"""Extension can be reinstalled after removal without stale activation errors."""
ext = await _get_extension(ironclaw_server, "web-search")
assert ext is not None, "web-search not found after reinstall"
assert ext["active"] is True, "Reinstalled tool should auto-activate via saved secrets"
assert ext["authenticated"] is True, "Saved secret should still authenticate on reinstall"
# Verify no stale activation error from previous install
assert ext.get("activation_error") is None or ext.get("activation_error") == "", (
f"Reinstalled extension should have no stale activation error: {ext}"
)
# ── Section F: Error Paths ──────────────────────────────────────────────
async def test_install_nonexistent(ironclaw_server):
"""Installing a nonexistent extension returns an error."""
r = await api_post(
ironclaw_server,
"/api/extensions/install",
json={"name": "nonexistent-tool-xyz-999"},
timeout=30,
)
if r.status_code == 200:
assert r.json().get("success") is False
else:
assert r.status_code >= 400
async def test_install_empty_name(ironclaw_server):
"""Installing with empty name returns an error."""
r = await api_post(
ironclaw_server,
"/api/extensions/install",
json={"name": ""},
timeout=10,
)
if r.status_code == 200:
assert r.json().get("success") is False
else:
assert r.status_code >= 400
async def test_remove_noninstalled(ironclaw_server):
"""Removing a non-installed extension returns an error."""
r = await api_post(
ironclaw_server, "/api/extensions/nonexistent-xyz/remove", timeout=10
)
if r.status_code == 200:
assert r.json().get("success") is False
else:
assert r.status_code >= 400
async def test_activate_noninstalled(ironclaw_server):
"""Activating a non-installed extension returns an error."""
r = await api_post(
ironclaw_server, "/api/extensions/nonexistent-xyz/activate", timeout=10
)
if r.status_code == 200:
assert r.json().get("success") is False
else:
assert r.status_code >= 400
async def test_setup_noninstalled(ironclaw_server):
"""Setup for non-installed extension returns an error."""
r = await api_get(ironclaw_server, "/api/extensions/nonexistent-xyz/setup")
# May return 500 or a JSON error
assert r.status_code >= 400 or r.json().get("success") is False
async def test_configure_noninstalled(ironclaw_server):
"""Configure for non-installed extension returns an error."""
r = await api_post(
ironclaw_server,
"/api/extensions/nonexistent-xyz/setup",
json={"secrets": {}},
timeout=10,
)
if r.status_code == 200:
assert r.json().get("success") is False
else:
assert r.status_code >= 400
# ── Section G: Browser UI ──────────────────────────────────────────────
async def test_extensions_tab_shows_registry(page):
"""Extensions tab loads and shows available extensions from registry."""
tab_btn = page.locator(SEL["tab_button"].format(tab="extensions"))
await tab_btn.click()
panel = page.locator(SEL["tab_panel"].format(tab="extensions"))
await panel.wait_for(state="visible", timeout=5000)
available_section = page.locator(SEL["available_wasm_list"])
await available_section.wait_for(state="visible", timeout=10000)
+138
View File
@@ -9,6 +9,10 @@ mod support;
mod advanced {
use std::time::Duration;
use ironclaw::agent::routine::Trigger;
use ironclaw::channels::IncomingMessage;
use ironclaw::db::Database;
use crate::support::cleanup::CleanupGuard;
use crate::support::test_rig::TestRigBuilder;
use crate::support::trace_llm::LlmTrace;
@@ -19,6 +23,28 @@ mod advanced {
);
const TIMEOUT: Duration = Duration::from_secs(30);
async fn wait_for_routine_run(
db: &std::sync::Arc<dyn Database>,
routine_id: uuid::Uuid,
timeout: Duration,
) -> Vec<ironclaw::agent::routine::RoutineRun> {
let deadline = tokio::time::Instant::now() + timeout;
loop {
let runs = db
.list_routine_runs(routine_id, 10)
.await
.expect("list_routine_runs");
if !runs.is_empty() {
return runs;
}
assert!(
tokio::time::Instant::now() < deadline,
"timed out waiting for routine run"
);
tokio::time::sleep(Duration::from_millis(100)).await;
}
}
// -----------------------------------------------------------------------
// 1. Multi-turn memory coherence
// -----------------------------------------------------------------------
@@ -380,6 +406,118 @@ mod advanced {
rig.shutdown();
}
// -----------------------------------------------------------------------
// 6b. Event routine: Telegram-scoped trigger fires on matching message
// -----------------------------------------------------------------------
#[tokio::test]
async fn routine_event_trigger_telegram_channel_fires() {
let trace = LlmTrace::from_file(format!("{FIXTURES}/routine_event_telegram.json")).unwrap();
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_routines()
.with_auto_approve_tools(true)
.build()
.await;
rig.send_message(
"Create a routine that watches Telegram messages starting with 'bug:' and alerts me.",
)
.await;
let create_responses = rig.wait_for_responses(1, TIMEOUT).await;
rig.verify_trace_expects(&trace, &create_responses);
let routine = rig
.database()
.get_routine_by_name("test-user", "telegram-bug-watcher")
.await
.expect("get_routine_by_name")
.expect("telegram-bug-watcher should exist");
match &routine.trigger {
Trigger::Event { channel, pattern } => {
assert_eq!(channel.as_deref(), Some("telegram"));
assert_eq!(pattern, "^bug\\b");
}
other => panic!("expected event trigger, got {other:?}"),
}
rig.send_incoming(IncomingMessage::new(
"telegram",
"test-user",
"bug: home button broken",
))
.await;
let runs = wait_for_routine_run(rig.database(), routine.id, TIMEOUT).await;
assert_eq!(runs[0].trigger_type, "event");
let responses = rig.wait_for_responses(3, TIMEOUT).await;
assert!(
responses.iter().any(|response| {
response
.metadata
.get("source")
.and_then(|value| value.as_str())
== Some("routine")
&& response.content.contains("telegram-bug-watcher")
&& response.content.contains("Bug report detected")
}),
"expected routine notification in responses: {responses:?}"
);
rig.shutdown();
}
// -----------------------------------------------------------------------
// 6c. Event routine without channel filter still fires on Telegram
// -----------------------------------------------------------------------
#[tokio::test]
async fn routine_event_trigger_without_channel_filter_still_fires() {
let trace =
LlmTrace::from_file(format!("{FIXTURES}/routine_event_any_channel.json")).unwrap();
let rig = TestRigBuilder::new()
.with_trace(trace)
.with_routines()
.with_auto_approve_tools(true)
.build()
.await;
rig.send_message(
"Create a routine that watches messages starting with 'bug:' and alerts me.",
)
.await;
let _ = rig.wait_for_responses(1, TIMEOUT).await;
let routine = rig
.database()
.get_routine_by_name("test-user", "any-channel-bug-watcher")
.await
.expect("get_routine_by_name")
.expect("any-channel-bug-watcher should exist");
match &routine.trigger {
Trigger::Event { channel, pattern } => {
assert_eq!(channel, &None);
assert_eq!(pattern, "^bug\\b");
}
other => panic!("expected event trigger, got {other:?}"),
}
rig.send_incoming(IncomingMessage::new(
"telegram",
"test-user",
"bug: login button broken",
))
.await;
let runs = wait_for_routine_run(rig.database(), routine.id, TIMEOUT).await;
assert_eq!(runs[0].trigger_type, "event");
rig.shutdown();
}
// -----------------------------------------------------------------------
// 7. Prompt injection resilience
// -----------------------------------------------------------------------
+201 -3
View File
@@ -10,6 +10,8 @@ mod support;
mod tests {
use std::time::Duration;
use ironclaw::agent::routine::{RoutineAction, Trigger};
use crate::support::test_rig::TestRigBuilder;
use crate::support::trace_llm::LlmTrace;
@@ -123,6 +125,39 @@ mod tests {
"routine_list should succeed: {completed:?}"
);
let routine = rig
.database()
.get_routine_by_name("test-user", "daily-check")
.await
.expect("get_routine_by_name")
.expect("daily-check should exist");
match &routine.trigger {
Trigger::Cron { schedule, timezone } => {
assert_eq!(schedule, "0 0 9 * * *");
assert_eq!(timezone.as_deref(), Some("America/New_York"));
}
other => panic!("expected cron trigger, got {other:?}"),
}
match &routine.action {
RoutineAction::Lightweight {
context_paths,
use_tools,
max_tool_rounds,
..
} => {
assert_eq!(context_paths, &vec!["context/priorities.md".to_string()]);
assert!(*use_tools, "lightweight routine should keep use_tools=true");
assert_eq!(*max_tool_rounds, 2);
}
other => panic!("expected lightweight action, got {other:?}"),
}
assert_eq!(routine.notify.channel.as_deref(), Some("telegram"));
assert_eq!(routine.notify.user, "ops-team");
assert_eq!(routine.guardrails.cooldown.as_secs(), 600);
rig.shutdown();
}
@@ -168,7 +203,48 @@ mod tests {
}
// -----------------------------------------------------------------------
// Test 5: routine_history
// Test 5: routine_manual_create
// -----------------------------------------------------------------------
#[tokio::test]
async fn routine_manual_create() {
let trace = LlmTrace::from_file(concat!(
env!("CARGO_MANIFEST_DIR"),
"/tests/fixtures/llm_traces/tools/routine_manual_create.json"
))
.expect("failed to load routine_manual_create.json");
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_auto_approve_tools(true)
.build()
.await;
rig.send_message("Create a manual routine for bug triage")
.await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
let routine = rig
.database()
.get_routine_by_name("test-user", "manual-triage")
.await
.expect("get_routine_by_name")
.expect("manual-triage should exist");
assert!(matches!(routine.trigger, Trigger::Manual));
assert!(
matches!(&routine.action, RoutineAction::Lightweight { use_tools, .. } if !*use_tools),
"manual routine should default to lightweight without tools: {:?}",
routine.action
);
rig.shutdown();
}
// -----------------------------------------------------------------------
// Test 6: routine_history
// -----------------------------------------------------------------------
#[tokio::test]
@@ -205,7 +281,7 @@ mod tests {
}
// -----------------------------------------------------------------------
// Test 6: routine_system_event_emit
// Test 7: routine_system_event_emit
// -----------------------------------------------------------------------
#[tokio::test]
@@ -253,11 +329,47 @@ mod tests {
emit_result.1
);
let routine = rig
.database()
.get_routine_by_name("test-user", "gh-issue-emit-test")
.await
.expect("get_routine_by_name")
.expect("gh-issue-emit-test should exist");
match &routine.trigger {
Trigger::SystemEvent {
source,
event_type,
filters,
} => {
assert_eq!(source, "github");
assert_eq!(event_type, "issue.opened");
assert_eq!(
filters.get("repository").map(String::as_str),
Some("nearai/ironclaw")
);
assert_eq!(filters.get("priority").map(String::as_str), Some("p1"));
}
other => panic!("expected system_event trigger, got {other:?}"),
}
match &routine.action {
RoutineAction::FullJob {
description,
tool_permissions,
..
} => {
assert!(description.contains("Summarize the new issue"));
assert_eq!(tool_permissions, &vec!["shell".to_string()]);
}
other => panic!("expected full_job action, got {other:?}"),
}
rig.shutdown();
}
// -----------------------------------------------------------------------
// Test 7: skill_install_routine_webhook_sim
// Test 8: skill_install_routine_webhook_sim
// -----------------------------------------------------------------------
#[tokio::test]
@@ -457,4 +569,90 @@ mod tests {
rig.shutdown();
}
// -----------------------------------------------------------------------
// Test: tool_info_discovery (two-level detail)
// -----------------------------------------------------------------------
// Verifies the tool_info built-in returns:
// - Default (no include_schema): name, description, parameter names array
// - With include_schema: true: adds full typed JSON Schema
#[tokio::test]
async fn tool_info_discovery() {
let trace = LlmTrace::from_file(concat!(
env!("CARGO_MANIFEST_DIR"),
"/tests/fixtures/llm_traces/tools/tool_info_discovery.json"
))
.expect("failed to load tool_info_discovery.json");
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_auto_approve_tools(true)
.build()
.await;
rig.send_message("What is the schema for the echo and time tools?")
.await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
// tool_info should have been called twice (echo + time), both succeeding.
let completed = rig.tool_calls_completed();
let tool_info_calls: Vec<_> = completed.iter().filter(|(n, _)| n == "tool_info").collect();
assert_eq!(
tool_info_calls.len(),
2,
"Expected 2 tool_info calls, got {tool_info_calls:?}"
);
assert!(
tool_info_calls.iter().all(|(_, ok)| *ok),
"All tool_info calls should succeed: {tool_info_calls:?}"
);
// Verify the results contain expected fields.
let results = rig.tool_results();
let info_results: Vec<_> = results.iter().filter(|(n, _)| n == "tool_info").collect();
// First call was for "echo" (default, no include_schema) — result should
// contain "echo" and "parameters" as an array of names (not full schema).
let echo_result = info_results
.iter()
.find(|(_, preview)| preview.contains("echo"))
.expect("tool_info result should contain 'echo'");
assert!(
echo_result.1.contains("message"),
"echo default result should list 'message' parameter name: {:?}",
echo_result.1
);
// Default mode should NOT include the full "schema" key
let echo_json: serde_json::Value = serde_json::from_str(&echo_result.1)
.expect("echo tool_info result should be valid JSON");
assert!(
echo_json.get("schema").is_none(),
"Default tool_info should not include schema field: {:?}",
echo_result.1
);
// Second call was for "time" with include_schema: true — result should
// contain "time", "schema" field with full object.
let time_result = info_results
.iter()
.find(|(_, preview)| preview.contains("time"))
.expect("tool_info result should contain 'time'");
let time_json: serde_json::Value = serde_json::from_str(&time_result.1)
.expect("time tool_info result should be valid JSON");
assert!(
time_json.get("schema").is_some(),
"include_schema: true should include schema field: {:?}",
time_result.1
);
assert!(
time_json["schema"]["properties"].is_object(),
"schema should have properties: {:?}",
time_result.1
);
rig.shutdown();
}
}
@@ -0,0 +1,54 @@
{
"model_name": "advanced-routine-event-any-channel",
"expects": {
"tools_used": ["routine_create"],
"all_tools_succeeded": true,
"min_responses": 1
},
"steps": [
{
"response": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_routine_create_event_any_channel",
"name": "routine_create",
"arguments": {
"name": "any-channel-bug-watcher",
"description": "Watch bug reports from any incoming channel.",
"trigger_type": "event",
"event_pattern": "^bug\\b",
"prompt": "Summarize the bug report in one line."
}
}
],
"input_tokens": 130,
"output_tokens": 38
}
},
{
"response": {
"type": "text",
"content": "Created the any-channel-bug-watcher routine for bug messages.",
"input_tokens": 170,
"output_tokens": 18
}
},
{
"response": {
"type": "text",
"content": "I saw the Telegram message.",
"input_tokens": 90,
"output_tokens": 12
}
},
{
"response": {
"type": "text",
"content": "Bug report detected: login button broken.",
"input_tokens": 120,
"output_tokens": 14
}
}
]
}
@@ -0,0 +1,55 @@
{
"model_name": "advanced-routine-event-telegram",
"expects": {
"tools_used": ["routine_create"],
"all_tools_succeeded": true,
"min_responses": 1
},
"steps": [
{
"response": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_routine_create_event_telegram",
"name": "routine_create",
"arguments": {
"name": "telegram-bug-watcher",
"description": "Watch Telegram bug reports and alert on them.",
"trigger_type": "event",
"event_channel": "telegram",
"event_pattern": "^bug\\b",
"prompt": "Summarize the bug report in one line."
}
}
],
"input_tokens": 140,
"output_tokens": 40
}
},
{
"response": {
"type": "text",
"content": "Created the telegram-bug-watcher routine for Telegram bug messages.",
"input_tokens": 180,
"output_tokens": 20
}
},
{
"response": {
"type": "text",
"content": "I saw the Telegram message.",
"input_tokens": 90,
"output_tokens": 12
}
},
{
"response": {
"type": "text",
"content": "Bug report detected: home button broken.",
"input_tokens": 120,
"output_tokens": 14
}
}
]
}
+9 -1
View File
@@ -18,8 +18,16 @@
"name": "daily-check",
"trigger_type": "cron",
"schedule": "0 0 9 * * *",
"timezone": "America/New_York",
"prompt": "Check system status and report any issues.",
"description": "Daily system health check"
"description": "Daily system health check",
"context_paths": ["context/priorities.md"],
"action_type": "lightweight",
"use_tools": true,
"max_tool_rounds": 2,
"cooldown_secs": 600,
"notify_channel": "telegram",
"notify_user": "ops-team"
}
}
],
@@ -0,0 +1,36 @@
{
"model_name": "test-routine-manual-create",
"expects": {
"tools_used": ["routine_create"],
"all_tools_succeeded": true,
"min_responses": 1
},
"steps": [
{
"response": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_rc_manual_1",
"name": "routine_create",
"arguments": {
"name": "manual-triage",
"trigger_type": "manual",
"prompt": "Summarize the latest bug reports when this routine is fired."
}
}
],
"input_tokens": 90,
"output_tokens": 22
}
},
{
"response": {
"type": "text",
"content": "Created the manual-triage routine. It will only run when explicitly fired.",
"input_tokens": 140,
"output_tokens": 18
}
}
]
}
@@ -21,7 +21,12 @@
"trigger_type": "system_event",
"event_source": "github",
"event_type": "issue.opened",
"event_filters": {
"repository": "nearai/ironclaw",
"priority": "p1"
},
"action_type": "full_job",
"tool_permissions": ["shell"],
"prompt": "Summarize the new issue and propose next steps."
}
}
@@ -42,6 +47,7 @@
"event_type": "issue.opened",
"payload": {
"repository": "nearai/ironclaw",
"priority": "p1",
"issue_number": 123,
"title": "Support event-driven project workflow"
}
@@ -0,0 +1,50 @@
{
"model_name": "test-tool-info-discovery",
"expects": {
"tools_used": ["tool_info"],
"all_tools_succeeded": true,
"min_responses": 1,
"tool_results_contain": {
"tool_info": "echo"
}
},
"steps": [
{
"request_hint": { "last_user_message_contains": "schema" },
"response": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_tool_info_echo",
"name": "tool_info",
"arguments": { "name": "echo" }
}
],
"input_tokens": 100,
"output_tokens": 20
}
},
{
"response": {
"type": "tool_calls",
"tool_calls": [
{
"id": "call_tool_info_time",
"name": "tool_info",
"arguments": { "name": "time", "include_schema": true }
}
],
"input_tokens": 200,
"output_tokens": 20
}
},
{
"response": {
"type": "text",
"content": "I found the info for both tools. The echo tool has a 'message' parameter. The time tool accepts an 'operation' parameter with options like 'now', 'parse', and 'diff'.",
"input_tokens": 400,
"output_tokens": 40
}
}
]
}
@@ -1,6 +1,41 @@
{
"version": "0.2.0",
"wit_version": "0.3.0",
"description": "Search the web using Brave Search. Returns titles, URLs, descriptions, and publication dates for matching web pages. Supports filtering by country, language, and freshness. Authentication is handled via the 'brave_api_key' secret injected by the host.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The search query to look up on the web"
},
"count": {
"type": "integer",
"description": "Number of results to return (1-20, default 5)",
"minimum": 1,
"maximum": 20,
"default": 5
},
"country": {
"type": "string",
"description": "2-letter uppercase country code to bias results (e.g. 'US', 'DE', 'JP')"
},
"search_lang": {
"type": "string",
"description": "2-letter lowercase language code for search results (e.g. 'en', 'de', 'fr')"
},
"ui_lang": {
"type": "string",
"description": "Locale in language-region format (e.g. 'en-US', 'de-DE')"
},
"freshness": {
"type": "string",
"description": "Filter by discovery time: 'pd' (past day), 'pw' (past week), 'pm' (past month), 'py' (past year), or date range 'YYYY-MM-DDtoYYYY-MM-DD'"
}
},
"required": ["query"],
"additionalProperties": false
},
"capabilities": {
"http": {
"allowlist": [