Compare commits

...
Author SHA1 Message Date
Henry ParkandGitHub 3c55a7e54f Merge branch 'main' into release-plz-2026-03-05T01-48-51Z 2026-03-05 19:39:20 -08:00
Nick PismenkovandGitHub 14de4c1b57 feat: Add HMAC-SHA256 webhook signature validation for Slack (#588)
* feat: Add HMAC-SHA256 webhook signature validation for Slack

* review fixes
2026-03-05 19:27:10 -08:00
2d332f12f0 feat(tools): add Google Discovery API URLs to WASM tool descriptions (#585)
Add Google Discovery Service URLs to all 6 Google WASM tool
descriptions so the LLM can fetch full API documentation on demand
using its built-in HTTP tool. Discovery API is public and requires
no authentication.

URLs added:
- Gmail: googleapis.com/discovery/v1/apis/gmail/v1/rest
- Calendar: calendar-json.googleapis.com/$discovery/rest?version=v3
- Drive: googleapis.com/discovery/v1/apis/drive/v3/rest
- Docs: googleapis.com/discovery/v1/apis/docs/v1/rest
- Sheets: googleapis.com/discovery/v1/apis/sheets/v4/rest
- Slides: googleapis.com/discovery/v1/apis/slides/v1/rest

[skip-regression-check]

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-05 19:20:29 -08:00
46218ec794 test: add WIT compatibility tests for WASM extensions (#586)
* test: add WIT compatibility tests for all WASM tools and channels

Adds CI and integration tests to catch WIT interface breakage across
all 14 WASM extensions (10 tools + 4 channels). Previously, changing
wit/tool.wit or wit/channel.wit could silently break guest-side tools
that weren't rebuilt until release time.

Three new pieces:

1. scripts/build-wasm-extensions.sh — builds all WASM extensions from
   source by reading registry manifests. Used by CI and locally.

2. tests/wit_compat.rs — integration tests that compile and instantiate
   each .wasm binary against the current wasmtime host linker with
   stubbed host functions. Catches added/removed/renamed WIT functions,
   signature mismatches, and missing exports. Skips gracefully when
   artifacts aren't built so `cargo test` still passes standalone.

3. .github/workflows/test.yml — new wasm-wit-compat CI job that builds
   all extensions then runs instantiation tests on every PR. Added to
   the branch protection roll-up.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting in wit_compat tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on WIT compat tests

- Switch build script from python3 to jq for JSON parsing, consistent
  with release.yml and avoids python3 dependency (#1, #7)
- Use dirs::home_dir() instead of HOME env var for portability (#2)
- Filter extensions by manifest "kind" field instead of path (#3)
- Replace .flatten() with explicit error handling in dir iteration (#4, #5)
- Split stub_tool_host_functions into stub_shared_host_functions +
  tool-only tool-invoke stub, since tool-invoke is not in channel WIT (#6)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 02:36:59 +00:00
6a2a6cd050 fix(security): use OsRng for all security-critical key and token generation (#519)
* fix(security): use OsRng for all security-critical key and token generation

Replace rand::thread_rng() with rand::rngs::OsRng in all security-critical
code paths that generate cryptographic key material, bearer tokens, PKCE
verifiers, CSRF state parameters, and webhook secrets. thread_rng() uses a
userspace CSPRNG (ChaCha) seeded from OS entropy, which is fine for
non-security contexts but adds an unnecessary intermediate layer for
key material where direct OS entropy (OsRng) is the correct choice.

Files changed:
- src/secrets/keychain.rs: master encryption key generation
- src/secrets/crypto.rs: per-secret HKDF salt generation
- src/orchestrator/auth.rs: per-job bearer token generation
- src/channels/web/mod.rs: gateway auth token fallback
- src/cli/oauth_defaults.rs: OAuth PKCE verifier and CSRF state
- src/tools/mcp/auth.rs: MCP OAuth PKCE verifier
- src/extensions/manager.rs: auto-generated extension secrets
- src/setup/channels.rs: webhook secret generation

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(security): address PR review feedback for OsRng migration

- Remove shadowing inner `use rand::rngs::OsRng` in `generate_salt()`;
  use module-level `aes_gcm::aead::OsRng` import instead (same type,
  avoids divergence risk if rand_core versions drift)
- Fix missed callsites in `pairing/store.rs`: `random_code()` and
  `generate_unique_code()` now use `OsRng` for pairing auth codes
- Add regression tests for `generate_salt()`: correct length,
  non-zero output, uniqueness across calls

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-06 02:36:38 +00:00
df49b17d0f fix: prevent concurrent memory hygiene passes and Windows file lock errors (#535)
* fix: prevent concurrent memory hygiene passes and Windows file lock errors (#495)

The heartbeat system spawns hygiene passes via tokio::spawn on every
tick, creating a TOCTOU race where multiple tasks read the state file
before any saves, causing all to execute concurrently. On Windows this
also triggers OS error 1224 (file locked by memory-mapped section)
when multiple tasks call std::fs::write on the same file.

Three fixes:
- AtomicBool guard (RUNNING + RunningGuard RAII) ensures only one
  hygiene pass runs at a time
- State file is saved before cleanup (not after) to claim the cadence
  window early and close the TOCTOU race
- Atomic file write (write to .tmp then rename) avoids Windows
  file-locking errors from concurrent writers

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Mutex to serialize tests touching global RUNNING AtomicBool

Address PR review feedback: the running_guard_prevents_reentry test
manipulates a global static AtomicBool, which could cause flaky
failures if future tests also touch it and run in parallel. A test-only
Mutex ensures serialization.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 02:23:11 +00:00
c87525d81f fix: sort tool_definitions() for deterministic LLM tool ordering (#582)
* fix: sort tool_definitions() for deterministic LLM tool ordering

HashMap iteration order is non-deterministic, causing the LLM to receive
tools in different orders across calls. Sort alphabetically by name to
eliminate position bias in tool selection.

Closes #566

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: use sort_unstable_by for tool definitions ordering

Stable sort is unnecessary since tool names are unique. Unstable sort
avoids the overhead of preserving equal-element order.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: repair bad merge in registry.rs (missing closing brace and test attribute)

The merge of main into fix/sort-tool-definitions dropped the closing `}`
of test_tool_definitions_sorted_alphabetically and the `#[tokio::test]`
attribute on test_retain_only_filters_tools, causing an unclosed delimiter
parse error that failed all CI jobs.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-06 02:20:56 +00:00
Nick PismenkovandGitHub 9ae04f14e3 feat: restart (#531)
* feat: restart

* review fixes

* add IRONCLAW_IN_DOCKER env variable

* review fixes

* fix tests

* set default value as false
2026-03-05 17:12:49 -08:00
470de5bd2d feat: merge http/web_fetch tools, add tool output stash for large responses (#578)
* feat: merge http/web_fetch tools, add tool output stash for large responses

Merge `web_fetch` into `http` tool with smart approval: plain GETs (no
headers, no body) run without approval and follow redirects with SSRF
re-validation per hop; all other requests require approval as before.

Add `tool_output_stash` on JobContext so full tool outputs are preserved
before safety-layer truncation. The `json` tool gains a
`source_tool_call_id` parameter to reference stashed outputs, enabling
reliable parsing of large API responses that exceed the 100KB context
limit.

Other improvements:
- Descriptive User-Agent header using CARGO_PKG_VERSION
- Truncation now keeps partial data + hint about source_tool_call_id
- System prompt reinforces tool_calls over narration
- json tool query/stringify handle pre-parsed (non-string) data

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: delete dead web_fetch.rs (merged into http tool)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: rename shadowed data binding for clarity in json tool

Address PR review: rename owned `data` to `data_value` before
re-binding as `let data = &data_value` to make ownership explicit.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): mark network-dependent trace tests as #[ignore]

The weather_sf and baseball_stats tests hit live external APIs (wttr.in,
ESPN) which are unreliable in CI. Mark them #[ignore] so they don't
block the pipeline. Run locally with `--ignored` to include them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replay recorded HTTP exchanges in trace tests instead of hitting live APIs

Wire ReplayingHttpInterceptor into TestRig when the trace fixture
contains http_exchanges. This replays recorded responses instead of
making live network calls, making tests deterministic and CI-stable.

Add captured HTTP responses to weather_sf.json (wttr.in) and
baseball_stats.json (ESPN API) fixtures.

Revert #[ignore] on both tests — they now run offline.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: recover inline bracket-format tool calls from LLM text responses

When flatten_tool_messages converts tool calls to text like
`[Called tool `http` with arguments: {...}]` for NEAR AI compatibility,
the LLM sometimes echoes this format back in its text responses instead
of using proper tool_calls. Add recovery for this bracket format in
recover_tool_calls_from_content and strip it in clean_response so
users don't see raw tool call syntax.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-06 00:49:10 +00:00
69cddb10fd feat: integrate 13-dimension complexity scorer into smart routing (#529)
* feat(llm): add smart model routing based on request complexity

Automatically selects optimal model tier (flash/standard/pro/frontier) for each
request based on 13-dimension complexity scoring:

- Reasoning words, multi-step signals, code indicators
- Domain-specific terms, creativity, precision
- Safety sensitivity, tool likelihood, question complexity
- Token estimate, context dependency, sentence complexity

Features:
- Pattern overrides for fast-path routing (greetings → flash, security audits → frontier)
- Configurable tier-to-model mappings (defaults to -latest aliases)
- Thinking mode per tier (pro: low, frontier: medium)
- User-configurable pattern overrides
- Zero-config for default benefits, full control for power users

Expected cost savings: 50-70% vs always-using-frontier baseline.

Refs: smart-routing-spec.md

* fix(routing): address Gemini Code Assist review feedback

- Add tracing warnings for invalid tier/regex in user overrides (router.rs)
- Use unreachable!() for tier hint match since regex enforces valid tiers (scorer.rs)
- Refactor weighted total to array iteration for maintainability (scorer.rs)
- Add TODO for making domain keywords configurable (scorer.rs)

Refs: PR #208

* feat(routing): make domain keywords configurable

- Add ScorerConfig with optional domain_keywords field
- Add DEFAULT_DOMAIN_KEYWORDS constant (exported for reference)
- Add domain_keywords to RouterConfig for top-level configuration
- Build domain regex at runtime from config, fallback to defaults
- Add score_complexity_with_config() function
- Add test for custom domain keywords

Users can now provide project-specific keywords:

  RouterConfig {
      domain_keywords: Some(vec!["mycompany".into(), "myproduct".into()]),
      ..Default::default()
  }

Addresses Gemini Code Assist review feedback on PR #208.

Tests: 20/20 passing

* docs: add domain_keywords to routing config example

* feat: integrate 13-dimension complexity scorer into smart routing (takeover #208)

Folds the 13-dimension complexity scorer and pattern overrides from PR #208
into the existing SmartRoutingProvider, replacing the simpler keyword-based
classifier. Adds 4-tier system (Flash/Standard/Pro/Frontier), configurable
scorer weights, domain keywords, regex pattern overrides, tier hints, and
multi-dimensional boost. Removes separate routing/ directory and lazy_static
dependency in favor of std::sync::LazyLock. Includes 44 tests covering all
scoring dimensions, tier boundaries, pattern overrides, and provider routing.

Co-Authored-By: onlyamicrowave <[email protected]>
Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback on smart routing PR (#529)

- Cache compiled domain regex in SmartRoutingProvider (built once at
  construction, not per-request) and add score_complexity_with_regex() API
- Check explicit tier hints before pattern overrides so user intent wins
  (e.g. "[tier:flash] security audit" routes as Flash, not Frontier)
- Trim input before matching/scoring so trailing whitespace doesn't break
  anchored override regexes or skew token-length scoring
- Fix token estimate comment (>=520 chars = 100, not >500)
- Update spec: check implementation plan boxes, fix file paths, add note
  that llm.routing YAML schema is target design (current config uses env vars)
- Add regression tests for tier hint precedence and trimmed greeting matching

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: restore Cargo.lock from main to fix html_to_markdown test

The lockfile was fully regenerated during the PR #208 merge conflict
resolution, which bumped html-to-markdown-rs from 2.25.1 to 2.27.2.
The new version produces different output that breaks the golden-file
snapshot test. Restore the original lockfile from main — lazy_static
was never in main's lockfile, so no further changes needed.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address second round of review feedback (#529)

- Tighten quick-lookup override regex with end anchor to prevent matching
  complex questions like "What time complexity is merge sort?"
- Handle empty domain keywords list by falling back to defaults instead of
  producing a broken regex that matches empty strings everywhere
- Clarify spec architecture diagram: current impl uses 2-provider split
  (cheap/primary), per-tier model mapping is target design
- Add regression tests for both fixes

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Microwave <[email protected]>
Co-authored-by: Joe <[email protected]>
Co-authored-by: onlyamicrowave <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-05 09:14:07 +00:00
b4b19738a8 Trajectory benchmarks and e2e trace test rig (#553)
* refactor: extract shared assertion helpers to support/assertions.rs

Move 5 assertion helpers from e2e_spot_checks.rs to a shared module.
Add assert_all_tools_succeeded and assert_tool_succeeded for eliminating
false positives in E2E tests.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add tool output capture via tool_results() accessor

Extract (name, preview) from ToolResult status events in TestChannel
and TestRig, enabling content assertions on tool outputs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: correct tool parameters in 3 broken trace fixtures

- tool_time.json: add missing "operation": "now" for time tool
- robust_correct_tool.json: same fix
- memory_full_cycle.json: change "path" to "target" for memory_write

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add tool success and output assertions to eliminate false positives

Every E2E test that exercises tools now calls assert_all_tools_succeeded.
Added tool output content assertions where tool results are predictable
(time year, read_file content, memory_read content).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: capture per-tool timing from ToolStarted/ToolCompleted events

Record Instant on ToolStarted and compute elapsed duration on
ToolCompleted, wiring real timing data into collect_metrics() instead
of hardcoded zeros.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: add RAII CleanupGuard for temp file/dir cleanup in tests

Replace manual cleanup_test_dir() calls and inline remove_file() with
Drop-based CleanupGuard that ensures cleanup even if a test panics.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add Drop impl and graceful shutdown for TestRig

Wrap agent_handle in Option so Drop can abort leaked tasks. Signal
the channel shutdown before aborting for future cooperative shutdown.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace agent startup sleep with oneshot ready signal

Use a oneshot channel fired in Channel::start() instead of a fixed
100ms sleep, eliminating the race condition on slow systems.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace fragile string-matching iteration limit with count-based detection

Use tool completion count vs max_tool_iterations instead of scanning
status messages for "iteration"/"limit" substrings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use assert_all_tools_succeeded for memory_full_cycle test

Remove incorrect comment about memory_tree failing with empty path
(it actually succeeds). Omit empty path from fixture and use the
standard assert_all_tools_succeeded instead of per-tool assertions.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: promote benchmark metrics types to library code

Move TraceMetrics, ScenarioResult, RunResult, MetricDelta, and
compare_runs() from tests/support/metrics.rs to src/benchmark/metrics.rs.
Existing tests use re-export for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add Scenario and Criterion types for agent benchmarking

Scenario defines a task with input, success criteria, and resource
limits. Criterion is an enum of programmatic checks (tool_used,
response_contains, etc.) evaluated without LLM judgment.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add initial benchmark scenario suite (12 scenarios across 5 categories)

Scenarios cover tool_selection, tool_chaining, error_recovery,
efficiency, and memory_operations. All loaded from JSON with
deserialization validation test.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add benchmark runner with BenchChannel and InstrumentedLlm

BenchChannel is a minimal Channel implementation for benchmarks.
InstrumentedLlm wraps any LlmProvider to capture per-call metrics.
Runner creates a fresh agent per scenario, evaluates success criteria,
and produces RunResult with timing, token, and cost metrics.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add baseline management, reports, and benchmark entry point

- baseline.rs: load/save/promote benchmark results
- report.rs: format comparison reports with regression detection
- benchmark_runner.rs: integration test with real LLM (feature-gated)
- Add benchmark feature flag to Cargo.toml

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt to benchmark module

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add multi-turn scenario types with setup, judge, ResponseNotContains

Add BenchScenario, Turn, TurnAssertions, JudgeConfig, ScenarioSetup,
WorkspaceSetup, SeedDocument types for multi-turn benchmark scenarios.
Add ResponseNotContains criterion variant. Add TurnAssertions::to_criteria()
converter for backward compat with existing evaluation engine.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add JSON scenario loader with recursive discovery and tag filter

Add load_bench_scenarios() for the new BenchScenario format with recursive
directory traversal and tag-based filtering. Create 4 initial trajectory
scenarios across tool-selection, multi-turn, and efficiency categories.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): multi-turn runner with workspace seeding and per-turn metrics

Add run_bench_scenario() that loops over BenchScenario turns, seeds workspace
documents, collects per-turn metrics (tokens, tool calls, wall time), and
evaluates per-turn assertions. Add TurnMetrics to metrics.rs and
clear_for_next_turn() to BenchChannel.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add LLM-as-judge scoring with prompt formatting and score parsing

Create judge.rs with format_judge_prompt, parse_judge_score, and judge_turn.
Wire into run_bench_scenario for turns with judge config -- scores below
min_score fail the turn.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add CLI subcommand (ironclaw benchmark)

Add BenchmarkCommand with --tags, --scenario, --no-judge, --timeout,
--update-baseline flags. Wire into Command enum and main.rs dispatch.
Feature-gated behind benchmark flag.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): per-scenario JSON output with full trajectory

Add save_scenario_results() that writes per-scenario JSON files alongside
the run summary. Each scenario gets its own file with turn_metrics trajectory.
Update CLI to use new output format.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add ToolRegistry::retain_only and wire tool filtering in scenarios

Add a retain_only() method to ToolRegistry that filters tools down to a
given allowlist. Wire this into run_bench_scenario() so that when a
scenario specifies a tools list in its setup, only those tools are
available during the benchmark run. Includes two tests for the new
method: one verifying filtering works and one verifying empty input
is a no-op.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): wire identity overrides into workspace before agent start

Add seed_identity() helper that writes identity files (IDENTITY.md,
USER.md, etc.) into the workspace before the agent starts, so that
workspace.system_prompt() picks them up. Wire it into
run_bench_scenario() after workspace seeding. Include a test that
verifies identity files are written and readable.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add --parallel and --max-cost CLI flags

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(benchmark): use feature-conditional snapshot names for CLI help tests

Prevents snapshot conflicts between default (no benchmark) and
all-features (with benchmark) builds by using separate snapshot names
per feature set.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): parallel execution with JoinSet and budget cap enforcement

Replace sequential loop in run_all_bench() with parallel execution using
JoinSet + semaphore when config.parallel > 1. Add budget cap enforcement
that skips remaining scenarios when max_total_cost_usd is exceeded.
Track skipped count in RunResult.skipped_scenarios and display it in
format_report().

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add tool restriction and identity override test scenarios

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: fix formatting for Phase 3

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add SkillRegistry::retain_only and wire skill filtering in scenarios

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(benchmark): add --json flag for machine-readable output

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: add GitHub Actions benchmark workflow (manual trigger)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(benchmark): remove in-tree benchmark harness, keep retain_only utilities

Move benchmark-specific code out of ironclaw in preparation for the
nearai/benchmarks trajectory adapter. This removes:

- src/benchmark/ (runner, scenarios, metrics, judge, report, etc.)
- src/cli/benchmark.rs and the Benchmark CLI subcommand
- benchmarks/ data directory (scenarios + trajectories)
- .github/workflows/benchmark.yml
- The "benchmark" Cargo feature flag

What remains:
- ToolRegistry::retain_only() and SkillRegistry::retain_only()
- Test support types (TraceMetrics, InstrumentedLlm) inlined into
  tests/support/ instead of re-exporting from the deleted module

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: add README for LLM trace fixture format

Documents the trajectory JSON format, response types, request hints,
directory structure, and how to write new traces.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(test): unify trace format around turns, add multi-turn support

Introduce TraceTurn type that groups user_input with LLM response steps,
making traces self-contained conversation trajectories. Add run_trace()
to TestRig for automatic multi-turn replay. Backward-compatible: flat
"steps" JSON is deserialized as a single turn transparently.

Includes all trace fixtures (spot, coverage, advanced), plan docs, and
new e2e tests for steering, error recovery, long chains, memory, and
prompt injection resilience.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures after merging main

- Fix tool_json fixture: use "data" parameter (not "input") to match
  JsonTool schema
- Fix status_events test: remove assertion for "time" tool that isn't
  in the fixture (only "echo" calls are used)
- Allow dead_code in test support metrics/instrumented_llm modules
  (utilities for future benchmark tests)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Working on recording traces and testing them

* feat(test): add declarative expects to trace fixtures, split infra tests

Add TraceExpects struct with 9 optional assertion fields (response_contains,
tools_used, all_tools_succeeded, etc.) that can be declared in fixture JSON
instead of hand-written Rust. Add verify_expects() and run_recorded_trace()
so recorded trace tests become one-liners.

Split trace infra tests (deserialization, backward compat) into
tests/trace_format.rs which doesn't require the libsql feature gate.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): add expects to all trace fixtures, simplify e2e tests

Add declarative expects blocks to all 19 trace fixture JSONs across
spot/, coverage/, advanced/, and root directories. Update all 8 e2e
test files to use verify_trace_expects() / run_and_verify_trace(),
replacing ~270 lines of hand-written assertions with fixture-driven
verification.

Tests that check things beyond expects (file content on disk, metrics,
event ordering) keep those extra assertions alongside the declarative
ones.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): adapt tests to AppBuilder refactor, fix formatting

Update test files to work with refactored TestRigBuilder that uses
AppBuilder::build_all() (removing with_tools/with_workspace methods).
Update telegram_check fixture to use tool_list instead of echo.
Fix cargo fmt issues in src/llm/mod.rs and src/llm/recording.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): deduplicate support unit tests into single binary

Support modules (assertions, cleanup, test_channel, test_rig, trace_llm)
had #[cfg(test)] mod tests blocks that were compiled and run 12 times —
once per e2e test binary that declares `mod support;`. Extracted all 29
support unit tests into a dedicated `tests/support_unit_tests.rs` so they
run exactly once.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix trailing newlines in support files

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(test): unify trace types and fix recorded multi-turn replay

Import shared types (TraceStep, TraceResponse, TraceToolCall, RequestHint,
ExpectedToolResult, MemorySnapshotEntry, HttpExchange*) from
ironclaw::llm::recording instead of redefining them in trace_llm.rs.

Fix the flat-steps deserializer to split at UserInput boundaries into
multiple turns, instead of filtering them out and wrapping everything
into a single turn. This enables recorded multi-turn traces to be
replayed as proper multi-turn conversations via run_trace().

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures - unused imports and missing struct fields

- Add #[allow(unused_imports)] on pub use re-exports in trace_llm.rs
  (types are re-exported for downstream test files, not used locally)
- Add `..` to ToolCompleted pattern in test_channel.rs to match new
  `error` and `parameters` fields

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): fix CI failures after merging main

- Add missing `error` and `parameters` fields to ToolCompleted
  constructors in support_unit_tests.rs
- Add `..` to ToolCompleted pattern match in support_unit_tests.rs
- Add #[allow(dead_code)] to CleanupGuard, LlmTrace impl, and
  TraceLlm impl (only used behind #[cfg(feature = "libsql")])

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Adding coverage running script

* fix(test): address review feedback on E2E test infrastructure

- Increase wait_for_responses polling to exponential backoff (50ms-500ms)
  and raise default timeout from 15s to 30s to reduce CI flakiness (#1)
- Strengthen prompt_injection_resilience test with positive safety layer
  assertion via has_safety_warnings(), enable injection_check (#2)
- Add assert_tool_order() helper and tools_order field in TraceExpects
  for verifying tool execution ordering in multi-step traces (#3)
- Document TraceLlm sequential-call assumption for concurrency (#6)
- Clean up CleanupGuard with PathKind enum instead of shotgun
  remove_file + remove_dir_all on every path (#8)
- Fix coverage.sh: default to --lib only, fix multi-filter syntax,
  add COV_ALL_TARGETS option
- Add coverage/ to .gitignore
- Remove planning docs from PR

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review - use HashSet in retain_only, improve skill test

- Use HashSet for O(N+M) lookup in SkillRegistry::retain_only and
  ToolRegistry::retain_only instead of linear scan
- Strengthen test_retain_only_empty_is_noop in SkillRegistry to
  pre-populate with a skill before asserting the no-op behavior

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): revert incorrect safety layer assertion in injection test

The safety layer sanitizes tool output, not user input. The injection
test sends a malicious user message with no tools called, so the safety
layer never fires. Reverted to the original test which correctly
validates the LLM refuses via trace expects. Also fixed case-sensitive
request hint ("ignore" -> "Ignore") to suppress noisy warning.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: clean stale profdata before coverage run

Adds `cargo llvm-cov clean` before each run to prevent
"mismatched data" warnings from stale instrumentation profiles.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in retain_only test

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-05 09:13:09 +00:00
github-actions[bot]andGitHub dd64060b8b chore: release v0.15.1 2026-03-05 01:48:52 +00:00
a1f0208956 fix(ci): persist all cargo-llvm-cov env vars for E2E coverage (#559)
* fix(ci): persist all cargo-llvm-cov env vars for E2E coverage

Newer cargo-llvm-cov versions output CARGO_ENCODED_RUSTFLAGS instead of
RUSTFLAGS from show-env. The workflow was cherry-picking specific vars
(RUSTFLAGS, LLVM_PROFILE_FILE, etc.) to persist to $GITHUB_ENV, so
CARGO_ENCODED_RUSTFLAGS was never set during the build step, producing a
non-instrumented binary and zero .profraw files.

Replace the manual echo lines with `cargo llvm-cov show-env >> $GITHUB_ENV`
to forward all vars (including CARGO_ENCODED_RUSTFLAGS, CARGO_INCREMENTAL,
etc.) regardless of cargo-llvm-cov version.

Also forward CARGO_ENCODED_RUSTFLAGS in the E2E conftest subprocess env.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): address PR review — prefix-based env forwarding, split clean step

- conftest.py: replace explicit env var list with prefix-based matching
  (CARGO_LLVM_COV*, LLVM_*) plus specific vars (CARGO_ENCODED_RUSTFLAGS,
  CARGO_INCREMENTAL) to stay resilient to cargo-llvm-cov changes.
- coverage.yml: move `cargo llvm-cov clean` to its own step so the env
  vars from show-env (persisted via $GITHUB_ENV) are active when clean runs.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-05 01:44:03 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
3615967f92 chore: release v0.15.0 (#526)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 16:37:10 -08:00
704d63f16a feat(oauth): route callbacks through web gateway for hosted instances (#555)
* feat: route OAuth callbacks through web gateway for hosted instances

On hosted instances (e.g., NEAR AI), OAuth callbacks can't reach the
local TCP listener on port 9876. This adds a gateway-routed OAuth flow
that works behind reverse proxies and load balancers.

Backend changes:
- Add /oauth/callback as a public route on the web gateway
- PendingOAuthFlow registry shared between ExtensionManager and handler
- Gateway mode auto-detected via IRONCLAW_OAUTH_CALLBACK_URL env var
- Platform state format (instance:nonce) for nginx routing
- Token exchange proxy support via IRONCLAW_OAUTH_EXCHANGE_URL
- Local TCP listener mode preserved as backward-compatible fallback

UX improvements:
- Hide Configure button for tools with auto-resolved OAuth credentials
  (builtin defaults or platform-injected env vars)
- Skip client_id/client_secret fields in setup schema when auto-resolved
- Show Reconfigure only after successful authentication

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(oauth): harden gateway callback and refactor AuthResult

- Add 60s timeout to exchange_via_proxy HTTP client (matching exchange_oauth_code)
- Read GATEWAY_AUTH_TOKEN once at ExtensionManager construction instead of
  per-flow from env (prevents coupling and clarifies token provenance)
- Extract oauth_error_page() helper to deduplicate error landing pages
- Remove IRONCLAW_FORCE_GATEWAY_CALLBACK env var (auto-detection suffices)
- Refactor AuthResult into typed AuthStatus enum with constructors,
  eliminating stringly-typed status and Option fields that were always None
- Adapt all handlers (chat, extensions, ws) to new AuthResult/AuthStatus API
- Use setup_url (not validation_endpoint) for awaiting_token responses

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(oauth): address review feedback — empty token guard, test flakiness, doc typos

- Fail early in exchange_via_proxy() when gateway_token is empty instead
  of sending an unauthenticated request to the exchange proxy
- Fix test_oauth_callback_strips_instance_prefix to use an expired flow
  so it never attempts a real HTTP token exchange (prevents CI flakiness)
- Fix doc comments: /auth/callback → /oauth/callback in PendingOAuthFlow
  and ExtensionManager pending_oauth_flows docs

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix: clarify strip_instance_prefix safety, wrapper credential fix, test assertion

- Add comment to strip_instance_prefix noting nonces are base64url (no colons)
- Expand wrapper.rs comment explaining the credential_user_id bug fix
- Fix test_oauth_callback_strips_instance_prefix assertion: landing_html
  does not include provider_name on error pages

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 23:47:45 +00:00
902492bcdb feat(web): show error details for failed tool calls (#490)
* feat(web): show error details and input params for failed tool calls

Failed tool calls in the gateway UI previously showed only a red X icon
with an empty expandable body. This change:

- Adds optional `error` and `parameters` fields to `ToolCompleted` SSE
  events so the browser receives failure details in real-time
- Auto-expands failed tool cards to make errors immediately visible
- Adds `StatusUpdate::tool_completed()` constructor that centralizes
  the 5 duplicated construction sites and applies `redact_params()` to
  prevent sensitive values (e.g. secret_save's "value" param) from
  leaking through SSE broadcasts
- Adds `sensitive_params()` trait method to `Tool` for declaring which
  parameters must be redacted before logging, hooks, and UI display
- Adds `redact_params()` utility and wires it through hooks, approvals,
  ActionRecord storage, and debug logs in dispatcher/worker
- Adds `SecretListTool` and `SecretDeleteTool` for LLM-driven secret
  management (values never returned, only names/metadata)
- Fixes auth flow: setup-only extensions show configure modal instead
  of OAuth card; auth_completed SSE dismisses both UI paths
- CI: release workflow creates PR instead of pushing directly to main
- Registry: MissingChecksum error enables source fallback for
  bootstrapping when checksums haven't been populated yet

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: keep original params in PendingApproval for execution, redact only for display

Address two PR review comments:

1. execute_chat_tool_standalone now redacts sensitive params before logging,
   matching the pattern already used in worker.rs.

2. PendingApproval previously stored redacted parameters, which meant
   approved tool calls received "[REDACTED]" instead of the actual values.
   Add a display_parameters field for UI/logs and keep parameters as the
   original values used for execution.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- worker.rs: redact sensitive params before BeforeToolCall hook, matching
  dispatcher.rs — hooks in the autonomous job path now receive redacted
  params instead of raw values
- registry.rs: fix docstring for register_secrets_tools (list, delete,
  not save/list/delete — no SecretSaveTool is registered)
- app.js: fix double toast/loadExtensions in submitConfigureModal —
  for non-OAuth success the auth_completed SSE already handles both,
  so skip them in the HTTP response handler to avoid duplicates

[skip-regression-check]

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 15:38:26 -08:00
13697976db feat(extensions): improve auth UX and add load-time validation (#536)
* feat(extensions): add load-time validation for auth capabilities

Catch common misconfigurations (missing auth section, missing setup_url,
short prompts) at startup via tracing::warn instead of silently failing
at auth time.

* feat(extensions): improve auth prompts, setup_url, and showAuthCard

Add setup_url and descriptive prompts to channel and tool capabilities
files. Fix showAuthCard in web gateway and improve extension manager
auth flow messaging.

* refactor(extensions): extract MIN_PROMPT_LENGTH constant in validate()

Address review feedback: replace magic number 30 with a named constant
for readability and maintainability.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 13:57:10 -08:00
cbcd5adcc0 fix(security): restrict query-token auth to SSE endpoints only (#528)
* fix(security): restrict query-token auth to SSE endpoints only

Query-string `?token=xxx` auth was accepted on all endpoints, exposing
the main auth token in server logs, Referer headers, and browser history
for state-changing routes. Now only GET /api/chat/events and
GET /api/logs/events accept query tokens; all other endpoints require
the Authorization header.

Supersedes #364.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add WebSocket endpoint to query-token allowlist, add URL-encoding tests

The WS upgrade at /api/chat/ws also can't set custom headers, so it
needs query-token auth like the SSE endpoints. Also adds tests for
URL-encoded token values to cover the form_urlencoded parser.

Addresses review feedback from Gemini (partially, /api/jobs/{id}/events
is a JSON endpoint not SSE, so it correctly stays excluded) and Copilot
(URL-encoded token test).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:06:51 +00:00
e24c33ff90 fix(ci): flush profraw coverage data in E2E teardown (#550)
The ironclaw binary only handles SIGINT (via tokio::signal::ctrl_c),
not SIGTERM. When conftest.py sent SIGTERM during teardown, the OS
killed the process immediately without running atexit handlers, so
LLVM never flushed .profraw files. cargo llvm-cov report then found
zero profraw files and failed.

- Send SIGINT instead of SIGTERM so the existing ctrl_c handler
  triggers graceful shutdown → main() returns → atexit runs → profraw
  flushed
- Increase shutdown wait from 5s to 10s for graceful cleanup
- Add a diagnostic step to verify profraw files exist before the
  report step, making future issues visible in CI logs

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 20:05:46 +00:00
f99991d27b fix(wasm): coerce string parameters to schema-declared types (#498)
* fix(wasm): coerce string parameters to schema-declared types

LLMs frequently pass numeric values as JSON strings ("5" instead of 5)
or booleans as strings ("true" instead of true). The WASM module's
serde deserializer rejects these type mismatches. This adds a
coerce_params_to_schema() helper that walks the params JSON object
and converts string values to their schema-declared types (number,
integer, boolean) before passing to the WASM module.

Adds 5 unit tests covering number, integer, boolean coercion,
already-correct types, and unparseable strings.

Closes #486

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use in-place mutation and case-insensitive boolean coercion

Address review feedback:
- Use get_mut instead of clone+insert to avoid allocations
- Make boolean coercion case-insensitive (handles "True", "FALSE", etc.)
- Expand boolean test to cover false and mixed-case values

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: collapse nested if-let to satisfy clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 18:19:01 +00:00
89600e2b5c fix(agent): strip leaked [Called tool ...] text from responses (#497)
* fix(agent): strip leaked [Called tool ...] text from agent responses

When the NEAR AI provider flattens tool_call messages to plain text,
markers like [Called tool ...] and [Tool ... returned: ...] can leak
into the user-visible response if the LLM echoes them back. This adds
a sanitization step in the agentic loop's text response path that
strips these internal markers before returning. If stripping leaves
the response empty, a generic fallback message is returned instead.

Closes #487

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use fold instead of collect+join to avoid heap allocation

Address review feedback: replace Vec collect + join with fold to build
the filtered string directly, avoiding an intermediate heap allocation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Pierre LE GUEN <[email protected]>
2026-03-04 18:16:26 +00:00
e4e78d8a87 fix(web): reset job list UI on restart failure (#499)
* fix(web): reset job list UI on restart failure

The restartJob() catch handler was missing a loadJobs() call, so the
job row stayed in a stale highlighted state after a failed restart
attempt. Add loadJobs() to match the success path behavior.

Closes #485

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use .finally() for loadJobs() instead of duplicating

Move loadJobs() to a .finally() block so it runs on both success and
failure without duplication.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 10:05:42 -08:00
b9446712e9 fix(telegram): add missing webhook section to capabilities.json (#381)
The Telegram channel capabilities file was missing the `webhook`
block inside `capabilities.channel`, causing the router to fall back
to the default `X-Webhook-Secret` header instead of the Telegram-
specific `X-Telegram-Bot-Api-Secret-Token`.

When a webhook secret is configured (via `telegram_webhook_secret`),
incoming updates are rejected with 401 because Telegram sends the
token in `X-Telegram-Bot-Api-Secret-Token` but the router looks for
`X-Webhook-Secret`.

The existing test in `schema.rs` already expects the correct header
name, confirming this is an oversight in the shipped capabilities
file.

Co-authored-by: SMKRV <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
2026-03-04 14:40:28 +00:00
LawyeredandGitHub 31a4330f24 Fix UTF-8 unsafe truncation in sandbox log capture (#359) 2026-03-04 15:25:26 +01:00
9b47dbbaed fix(security): replace .unwrap() panics in pairing store with proper error handling (#515)
The pairing store called .unwrap() on path.parent() in three locations
(upsert_request, record_failed_approve, add_allow_from). If a path has
no parent (root path or empty), this panics — a potential denial-of-service
vector if an attacker can influence the path.

Added InvalidPath variant to PairingStoreError and replaced all three
.unwrap() calls with ok_or_else error propagation. This follows the
project's no-panics-in-production policy.

Locations fixed:
- upsert_request (line ~227)
- record_failed_approve (line ~322)
- add_allow_from (line ~465)

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-04 08:21:19 +00:00
ac3c928853 ci: enhance coverage with feature matrix, postgres, and E2E (#523)
* ci: enhance coverage workflow with feature matrix, postgres, and E2E

Replace single-config coverage job with a multi-job pipeline:

- Mirror test.yml's 3-config feature matrix (all-features, default, libsql-only)
- Add PostgreSQL service (pgvector/pgvector:pg16) with migrations for
  postgres configs so integration tests actually run instead of skipping
- Add E2E coverage job using cargo-llvm-cov instrumented binary with
  Playwright browser tests
- Add coverage-gate roll-up job for branch protection
- Upload per-config flags to Codecov (all-features, default, libsql-only, e2e)
- Forward LLVM coverage env vars in E2E conftest.py so profraw data
  lands where cargo-llvm-cov report expects it

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on coverage workflow

- Avoid setting DATABASE_URL to empty string for libsql-only config;
  use $GITHUB_ENV conditional step so the var is unset entirely
- Add set -euo pipefail and psql -v ON_ERROR_STOP=1 to migrations
  so SQL errors fail the job immediately

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 06:23:53 +00:00
Pierre LE GUENandGitHub bf2a08be94 feat: add local-test skill and Dockerfile.test for web gateway testing (#524)
Add Dockerfile.test as reusable infrastructure for spinning up local
test instances with libsql (no PostgreSQL dependency). Defaults to
port 3003 to avoid conflict with dev server.

Add local-test workspace skill that teaches the agent how to build,
run, and test against local Docker containers using Chrome MCP browser
automation tools. Covers LLM backend configuration, multi-instance
testing, cleanup, and troubleshooting.
2026-03-04 05:53:35 +00:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
308758c27c chore: release v0.14.0 (#480)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 05:02:33 +00:00
f60c91e9a7 ci: enforce regression tests for fix commits (#517)
* ci: enforce regression tests for fix commits

Add a commit-msg hook and CI workflow that require test changes
alongside bug fix commits, ensuring every fix includes a regression
test that would have caught the bug.

- scripts/commit-msg-regression.sh: local git hook (blocks fix commits
  without test changes; exempts static/docs-only; bypass via
  [skip-regression-check] marker)
- .github/workflows/regression-test-check.yml: CI mirror on PRs
  (checks title + commit messages; skip via label)
- scripts/dev-setup.sh: install hook in step 6
- .github/scripts/create-labels.sh: add skip-regression-check label
- CLAUDE.md: document regression test policy

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on regression test enforcement

- Use here-strings instead of echo|grep to avoid misinterpreting
  special characters in variables
- Use git diff -W (whole-function context) to detect edits inside
  existing test functions, not just new #[test] attributes
- Honor [skip-regression-check] in commit messages in CI (not just
  the PR label)
- Use git rev-parse --git-path hooks for worktree-safe hook install

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Update .github/workflows/regression-test-check.yml

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
2026-03-04 04:35:54 +00:00
a22d44f2b2 ci: add code coverage with cargo-llvm-cov and Codecov (#511)
* ci: add code coverage with cargo-llvm-cov and Codecov

Add a Coverage workflow that runs on PRs and pushes to main using
cargo-llvm-cov with --all-features, uploading LCOV results to Codecov.
Include codecov.yml config with project/patch targets and ignore rules
for stub files.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: switch Codecov upload to OIDC (tokenless)

Use GitHub OIDC tokens instead of CODECOV_TOKEN secret so coverage
uploads work for fork PRs where secrets are not available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: fail coverage upload strictly on push, leniently on PRs

Use a conditional so pushes to main fail if Codecov upload breaks
(preventing silent reporting gaps) while PRs stay lenient to avoid
blocking fork PRs where OIDC may not be available.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: disable Codecov auto-detection to suppress warnings

We provide lcov.info explicitly, so disable auto-search for gcov,
coverage.py, and Xcode formats that produce noisy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: include channels-src and tools-src in coverage reporting

These WASM source directories should be tracked for test coverage
rather than ignored.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: remove stale ignore entries from codecov.yml

The marketplace, ecommerce, taskrabbit, and restaurant stub files
no longer exist in the codebase.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* ci: run coverage on push to main only

Avoids running tests twice on PRs (once in test.yml, once for coverage).
Coverage runs on merge to main instead. Simplify fail_ci_if_error to
always true since it only runs on push now.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 02:14:15 +00:00
a181c8b384 fix(web): mobile browser bar obscures chat input (#508)
* fix(web): use dvh units to prevent mobile browser bar from obscuring chat input

On mobile browsers (Brave/Android, Safari/iOS), the bottom navigation bar
covers the chat input because 100vh includes space behind browser chrome.
Switch to 100dvh (dynamic viewport height) with vh fallback for older
browsers, and add safe-area-inset padding for notched devices.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Fix padding declaration in chat input style

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-03-04 01:55:52 +00:00
35a79caf87 fix(web): assign unique thread_id to manual routine triggers (#500)
* fix(web): assign unique thread_id to manual routine triggers

Manual routine triggers via the web API created an IncomingMessage
without a thread_id, causing session_manager.resolve_thread() to
route the output to whatever thread was last associated with the
(user, "gateway", None) key. This sets a unique thread_id of the
form "routine-{id}-{timestamp}" so each manual trigger gets its own
dedicated thread.

Closes #484

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add ownership check to routine trigger handler (IDOR)

Address review feedback: verify routine.user_id matches the
authenticated user before allowing the trigger, preventing
unauthorized cross-user routine execution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:41:41 +00:00
85999b25a8 fix(web): refresh routine UI after Run Now trigger (#501)
* fix(web): refresh routine UI after "Run Now" trigger

triggerRoutine() only showed a toast but did not refresh the routine
data after triggering. This adds openRoutineDetail() / loadRoutines()
calls after the toast, matching the pattern used by toggleRoutine().

Closes #483

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: only refresh detail view if triggered routine matches current view

Check currentRoutineId === id before refreshing the detail panel to
avoid refreshing the wrong routine's view.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-04 00:38:48 +00:00
b60e5e907a fix(skills): use slug for skill download URL from ClawHub (#502)
* fix(web): use slug for skill download URL from ClawHub

The skill install handler was using req.name (display name like
"Markdown Converter") instead of the slug (like "owner/markdown-converter")
when constructing the download URL. The registry endpoint expects a slug,
so display names caused 502 errors.

- Add optional `slug` field to SkillInstallRequest
- Prefer slug over name when building the download URL
- JS installSkill() now sends slug from search results

Closes #482

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: guard against empty slug string in skill download URL

Filter out empty slug strings so we fall back to name instead of
constructing an invalid download URL.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:53:16 +00:00
d562dc8d90 fix(workspace): thread document path through search results (#503)
* fix(workspace): thread document path through search results

Memory search results were showing chunk UUIDs instead of source file
paths. Thread document_path through RankedResult, SearchResult, and the
RRF fusion pipeline so handlers can display the actual file path.

Fixes #481

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: use into_iter to move values instead of cloning

Address review feedback: consume results with into_iter() to move
String fields directly instead of cloning them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 23:31:30 +00:00
Nick PismenkovandGitHub c239a4fc2a feat: remove the okta tool (#506) 2026-03-03 21:30:54 +00:00
944968bf76 fix(workspace): import custom templates before seeding defaults (#505)
Swap the order of import_from_directory() and seed_if_empty() so that
custom workspace templates from WORKSPACE_IMPORT_DIR take priority
over generic seeds. Previously, seed_if_empty() ran first and created
all default files, causing import_from_directory() to skip everything
since the files already existed in the DB.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 11:14:02 -08:00
18b59ae9a7 feat: add OAuth support for WASM tools in web gateway (#489)
* feat: add OAuth support for WASM tools in web gateway

Extract reusable OAuth functions (build_oauth_url, exchange_oauth_code,
store_oauth_tokens, validate_oauth_token) from CLI into shared
oauth_defaults module, then wire them into the web gateway's
ExtensionManager.

Key changes:
- Install auto-activates WASM tools (no separate Activate button)
- Configure button triggers OAuth flow via save_setup_secrets
- Scope merging: installing a second Google tool triggers re-auth with
  merged scopes from all tools sharing the same secret_name
- Cancel-and-retry: aborting stale OAuth listeners prevents port conflicts
- Post-auth validation: wrong account detected via validation_endpoint
- Reconfigure always re-auths (deletes old token before starting fresh)
- UI shows error toast on OAuth failure, refreshes extension list

Flow: Install → Active → Configure (enter client_id/secret) → Save →
OAuth popup → authorize → done. Second Google tool install auto-triggers
scope expansion OAuth.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Add custom headers support to ValidationEndpointSchema (fixes
  missing Notion-Version header regression)
- Guard activate handler auth check with status == "awaiting_authorization"
  to prevent unexpected OAuth popups
- Add window dimensions to OAuth popup in activateExtension()
- Simplify UTF-8 truncation boundary check

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address Copilot PR review comments (security, UX, bugs)

- Add CSRF state parameter to OAuth flow (random state in auth URL, validated in callback)
- Restore MCP server Activate button in web UI (was hidden for all non-channel extensions)
- Abort JoinHandle in cleanup_expired_auths to prevent port 9876 conflicts
- Fix Google-specific error message for non-Google OAuth providers
- Add has_auth field to ExtensionInfo API response (fixes Configure button visibility)
- Use oauth_defaults::callback_url() instead of hardcoded redirect_uri (both CLI and manager)
- Update auth check comment to match actual behavior (scope expansion + first-time auth)
- Add unit tests for build_oauth_url (basic, PKCE, extra params, state uniqueness)
- Check all required setup secrets (client_id + client_secret) before starting OAuth

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-04 01:08:40 +08:00
f4855962fc fix: use std::sync::RwLock in MessageTool to avoid runtime panic (#411)
* fix: use std::sync::RwLock in MessageTool to avoid runtime panic

The `requires_approval` method is synchronous but was using
`tokio::sync::RwLock` with `.await` which requires blocking the
runtime. This caused a panic:
"Cannot block the current thread from within a runtime"

Changes:
- Replace `tokio::sync::RwLock` with `std::sync::RwLock` for
  `default_channel` and `default_target` fields
- Use `unwrap_or_else(|e| e.into_inner())` to gracefully handle
  poisoned locks (recovers instead of panicking)
- Update all usages from `.read().await` to `.read().unwrap_or_else()`

The locks are short-held (just cloning strings), making std::sync::RwLock
appropriate for sync methods called from async contexts.

Fixes: "Cannot block the current thread from within a runtime" panic
when the LLM tries to send a message via the message tool.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address code review feedback for MessageTool RwLock fix

- Fix formatting (long lines broken up per rustfmt)
- Add regression test that demonstrates the panic with tokio::sync::RwLock
  and passes with std::sync::RwLock when calling requires_approval()
  (sync method) from async context

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-03 16:16:02 +00:00
f18fb5173b feat(web): fix jobs UI parity for non-sandbox mode (#491)
* feat(web): fix jobs UI parity for non-sandbox mode

The web gateway Jobs UI was built primarily for sandbox (Docker) jobs.
When running without sandbox (common for NEAR AI hosted envs), multiple
features were broken. This change fixes all of them:

- Agent jobs now broadcast live SSE events to the web UI (Activity tab)
- Agent job restart via scheduler.dispatch_job (not chat message)
- Follow-up prompts for agent jobs via WorkerMessage injection
- Capability flags (can_restart, can_prompt, job_kind) in job detail API
- Rate-limit retry with cap (10 consecutive) and Retry-After header parsing
- Plan interruption on user message (breaks out of plan, re-evaluates)
- Correct SSE status field in mark_completed/mark_failed/mark_stuck
- SseManager preserved across rebuild_state calls

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: fix formatting in db/mod.rs and nearai_chat.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:23:30 +08:00
78878ad7ef Remove restart infrastructure, generalize WASM channel setup (#493)
* refactor: remove restart infrastructure and generalize Telegram-specific code

Remove the gateway restart mechanism (hot-activation works, restart won't
fix activation failures) and generalize Telegram-specific hardcoded checks
so all WASM channels get equal treatment.

Part 1 - Remove restart infrastructure:
- Remove needs_restart from ActionResponse, restart_requested from GatewayState
- Remove gateway_restart_handler, /api/gateway/restart route, exit code 75
- Remove restart overlay JS/CSS (dead code - restartGateway() never called)
- Surface actual activation errors instead of suggesting restart

Part 2 - Generalize Telegram-specific code:
- Replace telegram_owner_id: Option<i64> with generic
  wasm_channel_owner_ids: HashMap<String, i64> (backwards-compatible
  via TELEGRAM_OWNER_ID env var)
- Pairing status check now applies to all active WASM channels
- All channels get 3-step stepper in web UI, remove "coming soon" note
- Remove dead setup_telegram() code (~700 lines) - Telegram's
  capabilities.json declares required_secrets, so the generic
  setup_wasm_channel() path handles it

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add Settings::set() test for wasm_channel_owner_ids

Addresses review feedback: verify that setting per-channel owner IDs
via the dotted-path Settings::set() API works correctly with the new
HashMap<String, i64> type.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): refresh extension stepper after pairing approval

loadPairingRequests only refreshed the pairing section, not the
stepper status. Call loadExtensions() instead so the stepper updates
from "Awaiting Pairing" to "Active" immediately after approval.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-03 22:10:32 +08:00
5f841554d5 feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import (#477)
* feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import

Add two new OpenClaw-compatible workspace markdown files:

- TOOLS.md: Environment-specific tool notes (SSH hosts, device names,
  etc.) injected into the system prompt under "## Tool Notes". Seeded
  as comment-only (like HEARTBEAT.md) so it's effectively empty until
  the user adds real content. Not write-protected — the agent can
  update it as it learns the environment.

- BOOTSTRAP.md: First-run onboarding ritual. Injected FIRST in the
  system prompt when present. Guides the agent through introducing
  itself, learning about the user, and updating workspace files.
  Only seeded on truly fresh workspaces (no existing identity files)
  to avoid triggering the ritual on existing deployments. Agent clears
  it via `memory_write(target="bootstrap")` when done.

Add `Workspace::import_from_directory()` for disk-to-DB import:

- Scans a directory for *.md files and imports any that don't already
  exist in the database (never overwrites user edits)
- Controlled by WORKSPACE_IMPORT_DIR env var, runs after seed_if_empty()
- Enables Docker images / deployment scripts to ship customized
  workspace templates that override generic seeds
- Backwards compatible: no-op when env var is unset

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments

- Use stable `path.extension() != Some(OsStr::new("md"))` instead of
  unstable `is_none_or` (nightly-only)
- Use `tokio::join!` for concurrent DB reads in fresh-workspace check
- Skip unreadable directory entries instead of failing the entire import
- Skip unreadable files instead of failing the entire import

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 19:00:21 -08:00
6adf95b6d1 fix: wire secrets store into all WASM runtime activation paths (#479)
WASM tools and channels activated at runtime (via web UI or CLI) were
missing secrets store wiring, causing credential injection to silently
fail. Tools like web-search would get 401s from APIs even though the
user had configured their API key.

Four bugs fixed:
- activate_wasm_tool(): WasmToolLoader created without .with_secrets_store()
- register_wasm_from_storage(): hardcoded secrets_store: None
- WasmChannelLoader: no secrets_store field at all (added field + builder)
- activate_wasm_channel() and startup path: both missed wiring secrets

The startup path in app.rs was correct; all runtime paths now match it.

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
2026-03-02 16:56:24 -08:00
github-actions[bot]GitHubgithub-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8530f44630 chore: release v0.13.1 (#453)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-02 21:49:13 +00:00
5257fecca1 feat: add Brave Web Search WASM tool (#474)
* feat: add Brave Web Search WASM tool

Add a new WASM tool for searching the web via the Brave Search API.
Follows the same architecture as the GitHub WASM tool with zero-exposure
credential injection (X-Subscription-Token header).

Features:
- Full Brave Search API support (query, count, country, search_lang,
  ui_lang, freshness)
- Input validation on all parameters
- Retry logic for 429/5xx transient errors
- RFC 3986 percent-encoding
- Registry manifest for Extensions tab discovery

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: avoid Vec allocation in is_valid_ui_lang

Use iterator-based destructuring instead of collecting into a Vec,
avoiding a heap allocation in the WASM sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:07:18 -08:00
20073ccf57 fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown
- Make Enter complete the first matching command when autocomplete is
  visible, instead of requiring explicit arrow-key navigation first

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 13:06:55 -08:00
1a26b1e57f fix: correct download URLs for telegram-mtproto and slack-tool extensions (#470)
The tool manifests pointed to channel bundle URLs (telegram-wasm32-wasip2.tar.gz,
slack-wasm32-wasip2.tar.gz) instead of the tool bundles (telegram-mtproto-...,
slack-tool-...). This caused install to fail because the archive contents
didn't match the expected .wasm filename.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-02 12:37:50 -08:00
188 changed files with 18569 additions and 2771 deletions
+7
View File
@@ -115,5 +115,12 @@ HEARTBEAT_NOTIFY_USER=default
SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
# Restart Feature (Docker containers only)
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Without this, the restart tool and /restart command will be disabled.
# IRONCLAW_IN_DOCKER=false
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Logging
RUST_LOG=ironclaw=debug,tower_http=debug
+3
View File
@@ -62,6 +62,9 @@ create "scope: ci" "546E7A" "CI/CD workflows"
create "scope: docs" "78909C" "Documentation"
create "scope: dependencies" "90A4AE" "Dependency updates"
echo "==> Creating workflow labels..."
create "skip-regression-check" "9E9E9E" "Acknowledged: fix without regression test"
echo "==> Creating contributor labels..."
create "contributor: new" "FFF9C4" "First-time contributor"
create "contributor: regular" "FFE082" "2-5 merged PRs"
+177
View File
@@ -0,0 +1,177 @@
name: Code Coverage
on:
push:
branches: [main]
permissions:
id-token: write
contents: read
jobs:
coverage:
name: Coverage (${{ matrix.name }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- name: all-features
flags: "--all-features"
has_postgres: true
- name: default
flags: ""
has_postgres: true
- name: libsql-only
flags: "--no-default-features --features libsql"
has_postgres: false
services:
postgres:
image: pgvector/pgvector:pg16
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ironclaw_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
key: coverage-${{ matrix.name }}
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Run database migrations
if: matrix.has_postgres
run: |
set -euo pipefail
for f in migrations/V*.sql; do
echo "Applying $f..."
psql -v ON_ERROR_STOP=1 -f "$f"
done
env:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: ironclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
- name: Upload to Codecov
uses: codecov/codecov-action@v5
with:
files: lcov.info
flags: ${{ matrix.name }}
disable_search: true
use_oidc: true
fail_ci_if_error: true
e2e-coverage:
name: E2E Coverage
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
key: e2e-coverage
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Set up coverage instrumentation
run: |
# Append ALL env vars from show-env (including CARGO_ENCODED_RUSTFLAGS,
# CARGO_INCREMENTAL, LLVM_PROFILE_FILE, etc.) so the build step
# compiles an instrumented binary regardless of cargo-llvm-cov version.
cargo llvm-cov show-env >> "$GITHUB_ENV"
- name: Clean coverage workspace
run: cargo llvm-cov clean --workspace
- name: Build instrumented binary
run: cargo build --no-default-features --features libsql
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install E2E dependencies
run: |
cd tests/e2e
pip install -e .
playwright install --with-deps chromium
- name: Run E2E tests
run: |
pytest tests/e2e/ -v -x --timeout=120
env:
RUST_LOG: ironclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
if: always()
run: |
echo "LLVM_PROFILE_FILE=${LLVM_PROFILE_FILE}"
echo "CARGO_LLVM_COV_TARGET_DIR=${CARGO_LLVM_COV_TARGET_DIR}"
profraw_count=$(find target/ -name '*.profraw' 2>/dev/null | wc -l)
echo "Found ${profraw_count} .profraw files under target/"
find target/ -name '*.profraw' 2>/dev/null || true
if [ "$profraw_count" -eq 0 ]; then
echo "::warning::No .profraw files found — coverage report will fail"
fi
- name: Generate coverage report
if: always()
run: cargo llvm-cov report --lcov --output-path e2e-coverage.info
- name: Upload to Codecov
if: always()
uses: codecov/codecov-action@v5
with:
files: e2e-coverage.info
flags: e2e
disable_search: true
use_oidc: true
fail_ci_if_error: true
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: e2e-screenshots
path: tests/e2e/screenshots/
if-no-files-found: ignore
coverage-gate:
name: Coverage
runs-on: ubuntu-latest
if: always()
needs: [coverage, e2e-coverage]
steps:
- run: |
if [[ "${{ needs.coverage.result }}" != "success" || "${{ needs.e2e-coverage.result }}" != "success" ]]; then
echo "One or more coverage jobs failed"
exit 1
fi
+107
View File
@@ -0,0 +1,107 @@
name: Regression Test Check
on:
pull_request:
jobs:
regression-test:
name: Regression test enforcement
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check for regression tests
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_LABELS: ${{ join(github.event.pull_request.labels.*.name, ',') }}
run: |
set -euo pipefail
BASE_REF="origin/${{ github.event.pull_request.base.ref }}"
# --- 1. Is this a fix PR? Check title first, then commit messages ---
IS_FIX=false
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$PR_TITLE"; then
IS_FIX=true
fi
if [ "$IS_FIX" = false ]; then
COMMITS=$(git log --format='%s' "${BASE_REF}..HEAD")
if grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$COMMITS"; then
IS_FIX=true
fi
fi
if [ "$IS_FIX" = false ]; then
echo "Not a fix PR — skipping regression test check."
exit 0
fi
echo "Fix PR detected."
# --- 2. Skip label or commit message marker ---
if grep -qF ',skip-regression-check,' <<< ",$PR_LABELS,"; then
echo "skip-regression-check label present — skipping."
exit 0
fi
COMMIT_BODIES=$(git log --format='%B' "${BASE_REF}..HEAD")
if grep -qF '[skip-regression-check]' <<< "$COMMIT_BODIES"; then
echo "[skip-regression-check] found in commit message — skipping."
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
CHANGED_FILES=$(git diff --name-only "${BASE_REF}...HEAD")
if [ -z "$CHANGED_FILES" ]; then
echo "No changed files — skipping."
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$CHANGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
echo "All changes are static assets or docs — skipping."
exit 0
fi
# --- 4. Look for test changes ---
# Fast path: new test attributes or test modules in added lines.
if git diff "${BASE_REF}...HEAD" -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
echo "Test changes found in .rs files."
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
if git diff "${BASE_REF}...HEAD" -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
echo "Test changes found in existing test functions."
exit 0
fi
if grep -qE '^tests/' <<< "$CHANGED_FILES"; then
echo "Test file changes found under tests/."
exit 0
fi
# --- 5. No tests found ---
echo "::warning::This PR looks like a bug fix but contains no test changes. Every fix should include a regression test. Add a #[test] or #[tokio::test], or apply the 'skip-regression-check' label if not feasible."
exit 1
+12 -2
View File
@@ -413,6 +413,9 @@ jobs:
- build-wasm-extensions
if: ${{ always() && needs.host.result == 'success' && needs.build-wasm-extensions.result == 'success' }}
runs-on: "ubuntu-22.04"
permissions:
contents: write
pull-requests: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
@@ -445,7 +448,7 @@ jobs:
fi
done
done < "$CHECKSUMS"
- name: Commit updated manifests
- name: Create PR with updated manifests
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
@@ -453,8 +456,15 @@ jobs:
if git diff --cached --quiet; then
echo "No manifest changes to commit"
else
BRANCH="chore/update-checksums-$(date +%s)"
git checkout -b "$BRANCH"
git commit -m "chore: update WASM artifact SHA256 checksums [skip ci]"
git push
git push origin "$BRANCH"
gh pr create \
--title "chore: update WASM artifact SHA256 checksums" \
--body "Auto-generated by release CI. Updates SHA256 checksums in registry manifests to match the released WASM artifacts." \
--base main \
--head "$BRANCH"
fi
announce:
+23 -2
View File
@@ -46,6 +46,27 @@ jobs:
- name: Run Telegram Channel Tests
run: cargo test --manifest-path channels-src/telegram/Cargo.toml -- --nocapture
wasm-wit-compat:
name: WASM WIT Compatibility
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
with:
profile: minimal
targets: wasm32-wasip2
- uses: Swatinem/rust-cache@v2
with:
key: wasm-extensions
- name: Install cargo-component
run: cargo install cargo-component --locked || true
- name: Build all WASM extensions against current WIT
run: ./scripts/build-wasm-extensions.sh
- name: Instantiation test (host linker compatibility)
run: cargo test --all-features wit_compat -- --nocapture
docker-build:
name: Docker Build
runs-on: ubuntu-latest
@@ -60,10 +81,10 @@ jobs:
name: Run Tests
runs-on: ubuntu-latest
if: always()
needs: [tests, telegram-tests, docker-build]
needs: [tests, telegram-tests, wasm-wit-compat, docker-build]
steps:
- run: |
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
if [[ "${{ needs.tests.result }}" != "success" || "${{ needs.telegram-tests.result }}" != "success" || "${{ needs.wasm-wit-compat.result }}" != "success" || "${{ needs.docker-build.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
+3
View File
@@ -16,6 +16,9 @@ target/
# Benchmark results (local runs, not committed)
bench-results/
# Coverage reports (local runs, not committed)
coverage/
# WASM build artifacts (loaded from disk, not bundled)
*.wasm
+66
View File
@@ -7,6 +7,72 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.15.1](https://github.com/nearai/ironclaw/compare/v0.15.0...v0.15.1) - 2026-03-05
### Fixed
- *(ci)* persist all cargo-llvm-cov env vars for E2E coverage ([#559](https://github.com/nearai/ironclaw/pull/559))
## [0.15.0](https://github.com/nearai/ironclaw/compare/v0.14.0...v0.15.0) - 2026-03-04
### Added
- *(oauth)* route callbacks through web gateway for hosted instances ([#555](https://github.com/nearai/ironclaw/pull/555))
- *(web)* show error details for failed tool calls ([#490](https://github.com/nearai/ironclaw/pull/490))
- *(extensions)* improve auth UX and add load-time validation ([#536](https://github.com/nearai/ironclaw/pull/536))
- add local-test skill and Dockerfile.test for web gateway testing ([#524](https://github.com/nearai/ironclaw/pull/524))
### Fixed
- *(security)* restrict query-token auth to SSE endpoints only ([#528](https://github.com/nearai/ironclaw/pull/528))
- *(ci)* flush profraw coverage data in E2E teardown ([#550](https://github.com/nearai/ironclaw/pull/550))
- *(wasm)* coerce string parameters to schema-declared types ([#498](https://github.com/nearai/ironclaw/pull/498))
- *(agent)* strip leaked [Called tool ...] text from responses ([#497](https://github.com/nearai/ironclaw/pull/497))
- *(web)* reset job list UI on restart failure ([#499](https://github.com/nearai/ironclaw/pull/499))
- *(security)* replace .unwrap() panics in pairing store with proper error handling ([#515](https://github.com/nearai/ironclaw/pull/515))
### Other
- Fix UTF-8 unsafe truncation in sandbox log capture ([#359](https://github.com/nearai/ironclaw/pull/359))
- enhance coverage with feature matrix, postgres, and E2E ([#523](https://github.com/nearai/ironclaw/pull/523))
## [0.14.0](https://github.com/nearai/ironclaw/compare/v0.13.1...v0.14.0) - 2026-03-04
### Added
- remove the okta tool ([#506](https://github.com/nearai/ironclaw/pull/506))
- add OAuth support for WASM tools in web gateway ([#489](https://github.com/nearai/ironclaw/pull/489))
- *(web)* fix jobs UI parity for non-sandbox mode ([#491](https://github.com/nearai/ironclaw/pull/491))
- *(workspace)* add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import ([#477](https://github.com/nearai/ironclaw/pull/477))
### Fixed
- *(web)* mobile browser bar obscures chat input ([#508](https://github.com/nearai/ironclaw/pull/508))
- *(web)* assign unique thread_id to manual routine triggers ([#500](https://github.com/nearai/ironclaw/pull/500))
- *(web)* refresh routine UI after Run Now trigger ([#501](https://github.com/nearai/ironclaw/pull/501))
- *(skills)* use slug for skill download URL from ClawHub ([#502](https://github.com/nearai/ironclaw/pull/502))
- *(workspace)* thread document path through search results ([#503](https://github.com/nearai/ironclaw/pull/503))
- *(workspace)* import custom templates before seeding defaults ([#505](https://github.com/nearai/ironclaw/pull/505))
- use std::sync::RwLock in MessageTool to avoid runtime panic ([#411](https://github.com/nearai/ironclaw/pull/411))
- wire secrets store into all WASM runtime activation paths ([#479](https://github.com/nearai/ironclaw/pull/479))
### Other
- enforce regression tests for fix commits ([#517](https://github.com/nearai/ironclaw/pull/517))
- add code coverage with cargo-llvm-cov and Codecov ([#511](https://github.com/nearai/ironclaw/pull/511))
- Remove restart infrastructure, generalize WASM channel setup ([#493](https://github.com/nearai/ironclaw/pull/493))
## [0.13.1](https://github.com/nearai/ironclaw/compare/v0.13.0...v0.13.1) - 2026-03-02
### Added
- add Brave Web Search WASM tool ([#474](https://github.com/nearai/ironclaw/pull/474))
### Fixed
- *(web)* auto-scroll and Enter key completion for slash command autocomplete ([#475](https://github.com/nearai/ironclaw/pull/475))
- correct download URLs for telegram-mtproto and slack-tool extensions ([#470](https://github.com/nearai/ironclaw/pull/470))
## [0.13.0](https://github.com/nearai/ironclaw/compare/v0.12.0...v0.13.0) - 2026-03-02
### Added
+3
View File
@@ -321,6 +321,8 @@ cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
**Regression test with every fix:** Every bug fix must include a test that would have caught the bug. Add a `#[test]` or `#[tokio::test]` that reproduces the original failure. Exempt: changes limited to `src/channels/web/static/` or `.md` files. Use `[skip-regression-check]` in commit message or PR label if genuinely not feasible. The `commit-msg` hook and CI workflow enforce this automatically.
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Mechanical verification before committing:** Run these checks on changed files before committing:
@@ -328,6 +330,7 @@ Dead code behind the wrong `#[cfg]` gate will only show up when building with a
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
## Configuration
Generated
+2 -1
View File
@@ -2828,7 +2828,7 @@ dependencies = [
[[package]]
name = "ironclaw"
version = "0.13.0"
version = "0.15.1"
dependencies = [
"aes-gcm",
"aho-corasick",
@@ -2853,6 +2853,7 @@ dependencies = [
"futures",
"hex",
"hkdf",
"hmac",
"html-to-markdown-rs",
"http-body-util",
"hyper 1.8.1",
+2 -2
View File
@@ -12,14 +12,13 @@ exclude = [
"tools-src/google-drive",
"tools-src/google-sheets",
"tools-src/google-slides",
"tools-src/okta",
"tools-src/slack",
"tools-src/telegram",
]
[package]
name = "ironclaw"
version = "0.13.0"
version = "0.15.1"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
@@ -129,6 +128,7 @@ wasmparser = "0.220" # WASM binary parsing for validation
# Cryptography for secrets management
aes-gcm = "0.10"
hkdf = "0.12"
hmac = "0.12"
sha2 = "0.10"
blake3 = "1"
rand = "0.8"
+57
View File
@@ -0,0 +1,57 @@
# Lightweight test Dockerfile for IronClaw web gateway testing.
#
# Build:
# docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
#
# Run (each on a different port):
# docker run --rm -p 3003:3003 ironclaw-test
# docker run --rm -p 3004:3003 ironclaw-test
# docker run --rm -p 3005:3003 ironclaw-test
# Stage 1: Build (libsql only — no PostgreSQL dependency)
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
RUN cargo build --release --no-default-features --features libsql --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
WORKDIR /home/ironclaw
EXPOSE 3003
ENV RUST_LOG=ironclaw=info \
GATEWAY_ENABLED=true \
GATEWAY_HOST=0.0.0.0 \
GATEWAY_PORT=3003 \
GATEWAY_AUTH_TOKEN=test \
DATABASE_BACKEND=libsql \
LIBSQL_PATH=/home/ironclaw/test.db \
SANDBOX_ENABLED=false
ENTRYPOINT ["ironclaw", "--no-onboard"]
@@ -6,15 +6,16 @@
"required_secrets": [
{
"name": "discord_bot_token",
"prompt": "Enter your Discord Bot Token (from Developer Portal)",
"prompt": "Enter your Discord Bot Token. Find it under Bot > Token in your Discord Application settings.",
"optional": false
},
{
"name": "discord_public_key",
"prompt": "Enter your Discord Application Public Key (from Developer Portal > General Information)",
"prompt": "Enter your Discord Application Public Key (found under General Information in your Discord Application settings).",
"optional": false
}
]
],
"setup_url": "https://discord.com/developers/applications"
},
"capabilities": {
"http": {
+7 -3
View File
@@ -6,15 +6,16 @@
"required_secrets": [
{
"name": "slack_bot_token",
"prompt": "Enter your Slack Bot OAuth Token (xoxb-...)",
"prompt": "Enter your Slack Bot User OAuth Token (starts with xoxb-). Find it under OAuth & Permissions in your Slack App settings.",
"optional": false
},
{
"name": "slack_signing_secret",
"prompt": "Enter your Slack Signing Secret (from App Credentials)",
"prompt": "Enter your Slack App Signing Secret (found under Basic Information > App Credentials in your Slack App settings).",
"optional": false
}
]
],
"setup_url": "https://api.slack.com/apps"
},
"capabilities": {
"http": {
@@ -43,6 +44,9 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"hmac_secret_name": "slack_signing_secret"
}
}
},
@@ -9,7 +9,8 @@
"prompt": "Enter your Telegram Bot API token (from @BotFather)",
"optional": false
}
]
],
"setup_url": "https://t.me/BotFather"
},
"capabilities": {
"http": {
@@ -39,6 +40,10 @@
"emit_rate_limit": {
"messages_per_minute": 100,
"messages_per_hour": 5000
},
"webhook": {
"secret_header": "X-Telegram-Bot-Api-Secret-Token",
"secret_name": "telegram_webhook_secret"
}
}
},
@@ -6,7 +6,7 @@
"required_secrets": [
{
"name": "whatsapp_access_token",
"prompt": "Enter your WhatsApp Cloud API access token (from Meta Developer Portal)",
"prompt": "Enter your WhatsApp Cloud API permanent access token (from the Meta Developer Portal under your app's WhatsApp > API Setup).",
"validation": "^[A-Za-z0-9_-]+$"
},
{
@@ -16,7 +16,8 @@
"auto_generate": { "length": 32 }
}
],
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}"
"validation_endpoint": "https://graph.facebook.com/v18.0/me?access_token={whatsapp_access_token}",
"setup_url": "https://developers.facebook.com/apps"
},
"capabilities": {
"http": {
+10
View File
@@ -0,0 +1,10 @@
coverage:
status:
project:
default:
target: auto
threshold: 1%
patch:
default:
target: 80%
threshold: 5%
+9
View File
@@ -24,6 +24,15 @@ GATEWAY_HOST=0.0.0.0
GATEWAY_PORT=3000
GATEWAY_AUTH_TOKEN=CHANGE_ME
# Restart Feature (Docker containers only)
# IMPORTANT: Set this in the container entrypoint or docker-compose to enable restart.
# The Docker entrypoint loop monitors exit codes:
# - Exit code 0 = clean restart: reset failure counter, wait IRONCLAW_RESTART_DELAY, restart
# - Exit code ≠ 0 = failure: increment counter, exit after IRONCLAW_MAX_FAILURES
IRONCLAW_IN_DOCKER=false
IRONCLAW_RESTART_DELAY=5 # seconds to wait before restarting (range: 1-30)
IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Disabled for initial deploy
SANDBOX_ENABLED=false
HEARTBEAT_ENABLED=false
+195
View File
@@ -0,0 +1,195 @@
# Smart Model Routing for IronClaw
**Status:** Implemented
**Author:** Microwave
**Date:** 2026-02-19
## What
Automatic model selection based on request complexity. The router analyzes each user message and selects an appropriate model tier (flash/standard/pro/frontier), then maps that tier to a configured model.
## Why
1. **Cost optimization** — Simple requests ("hi", "what time is it") don't need expensive models
2. **User experience** — Simple requests return faster with lightweight models
3. **NEAR AI native** — Default backend uses NEAR AI inference where costs vary by model
4. **Zero-config value** — Users benefit immediately without configuration
5. **Not just power users** — Everyone gets smart defaults, power users can override
## How
### Architecture
```
User Message
┌──────────────────┐
│ Pattern Overrides │ ← Fast-path for obvious cases (greetings, security audits)
└────────┬─────────┘
│ no match
┌──────────────────┐
│ Complexity Scorer │ ← 13-dimension analysis
└────────┬─────────┘
│ score 0-100
┌──────────────────┐
│ Tier Mapping │ ← 0-15: flash, 16-40: standard, 41-65: pro, 66+: frontier
└────────┬─────────┘
│ tier
┌──────────────────┐
│ Model Selection │ ← Currently: cheap provider (Flash/Standard/Pro) vs primary (Frontier)
└────────┬─────────┘ Target: per-tier model mapping via config
LLM Provider
```
### Complexity Scorer (13 Dimensions)
Each dimension produces a 0-100 score. Weighted sum determines total.
| Dimension | Weight | Signals |
|-----------|--------|---------|
| Reasoning Words | 14% | "why", "explain", "compare", "trade-offs" |
| Token Estimate | 12% | Prompt length |
| Code Indicators | 10% | Backticks, syntax, "implement", "PR" |
| Multi-Step | 10% | "first", "then", "after", "steps" |
| Domain Specific | 10% | Technical terms (configurable) |
| Creativity | 7% | "write", "summarize", "tweet", "blog" |
| Question Complexity | 7% | Multiple questions, open-ended starters |
| Precision | 6% | Numbers, "exactly", "calculate" |
| Ambiguity | 5% | Vague references |
| Context Dependency | 5% | "previous", "you said" |
| Sentence Complexity | 5% | Commas, conjunctions, clause depth |
| Tool Likelihood | 5% | "read", "deploy", "install" |
| Safety Sensitivity | 4% | "password", "auth", "vulnerability" |
**Multi-dimensional boost:** +30% when 3+ dimensions score above threshold.
### Tier Boundaries
| Score | Tier | Typical Use Case |
|-------|------|------------------|
| 0-15 | flash | Greetings, acknowledgments, quick lookups |
| 16-40 | standard | Writing, comparisons, defined tasks |
| 41-65 | pro | Multi-step analysis, code review |
| 66+ | frontier | Critical decisions, security audits |
### Pattern Overrides
Fast-path rules that bypass scoring for obvious cases:
```yaml
# Force flash tier
- "^(hi|hello|hey|thanks|ok|sure|yes|no)$"
- "^what.*(time|date|day)"
# Force frontier tier
- "security.*(audit|review|scan)"
- "vulnerabilit(y|ies).*(review|scan|check|audit)"
# Force pro tier
- "deploy.*(mainnet|production)"
```
### Configuration
> **Note:** The current implementation supports smart routing via
> `NEARAI_CHEAP_MODEL` and `SMART_ROUTING_CASCADE` env vars, plus
> `domain_keywords` on `SmartRoutingConfig`. The full `llm.routing` YAML
> schema below is the target design — not all knobs are wired yet.
**Default (zero-config):**
```yaml
llm:
routing:
enabled: true # default
```
**Power user overrides (target schema):**
```yaml
llm:
routing:
enabled: true
tiers:
flash: "claude-3-5-haiku-latest"
standard: "claude-sonnet-4-5-latest"
pro: "claude-sonnet-4-5-latest"
frontier: "claude-opus-4-5-latest"
thinking:
pro: "low"
frontier: "medium"
overrides:
- pattern: "my-custom-pattern"
tier: "pro"
domain_keywords: # Custom keywords for your domain
- "mycompany"
- "myproduct"
- "internal-tool"
```
If `domain_keywords` is not set, uses `DEFAULT_DOMAIN_KEYWORDS` which covers common web3/infra terms.
**Disable routing (pin model):**
```yaml
llm:
routing:
enabled: false
model: "claude-opus-4-5"
```
**Bring your own keys:**
```yaml
llm:
backend: anthropic
api_key: "sk-..."
routing:
enabled: true # still works with external providers
```
### Integration Points
1. **RoutingProvider** — New wrapper implementing `LlmProvider` trait (like `FailoverProvider`)
2. **Scorer** — Pure function, no I/O, fast (~1ms)
3. **Config schema** — Extend `LlmConfig` with `routing` section
4. **Telemetry** — Log routing decisions for observability
### Model Agnosticism
**Critical:** No hardcoded model names in the router logic itself.
- Tier→model mappings come from config
- Default mappings use `-latest` patterns where supported
- NEAR AI backend handles actual model resolution
- Router only knows about tiers
### Layers of Control
| Layer | User Type | Config |
|-------|-----------|--------|
| 1. Zero-config | Everyone | `routing.enabled: true` (default) |
| 2. Tier tuning | Power users | Custom `routing.tiers` mapping |
| 3. Pattern overrides | Power users | Custom `routing.overrides` |
| 4. Model pinning | Power users | `routing.enabled: false` + `model: X` |
| 5. Own API keys | Power users | `backend: anthropic` + `api_key` |
## Implementation Plan
1. [x] Port scorer to Rust (`src/llm/smart_routing.rs`)
2. [x] Implement router wrapper (`src/llm/smart_routing.rs`)
3. [x] Extend config schema (`src/config.rs`)
4. [x] Wire into provider creation (`src/llm/mod.rs`)
5. [x] Add telemetry/logging
6. [x] Tests with real conversation samples
7. [x] Codex + Gemini security review
8. [x] Documentation updated (this spec)
## Expected Outcomes
- **50-70% cost reduction** for typical usage patterns
- **Faster responses** for simple requests
- **Zero config required** for default benefits
- **Full control** for power users who want it
-31
View File
@@ -1,31 +0,0 @@
{
"name": "okta",
"display_name": "Okta",
"kind": "tool",
"version": "0.1.0",
"description": "Okta SSO for user profile, app catalog, and SSO launch links",
"keywords": ["sso", "identity", "authentication", "okta"],
"source": {
"dir": "tools-src/okta",
"capabilities": "okta-tool.capabilities.json",
"crate_name": "okta-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/okta-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "oauth",
"provider": "Okta",
"secrets": ["okta_oauth_token"],
"shared_auth": null,
"setup_url": "https://developer.okta.com/docs/guides/implement-oauth-for-okta/main/"
},
"tags": ["identity"]
}
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+1 -1
View File
@@ -14,7 +14,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": null
}
},
+31
View File
@@ -0,0 +1,31 @@
{
"name": "web-search",
"display_name": "Web Search",
"kind": "tool",
"version": "0.1.0",
"description": "Search the web using Brave Search API",
"keywords": ["search", "web", "brave", "internet"],
"source": {
"dir": "tools-src/web-search",
"capabilities": "web-search-tool.capabilities.json",
"crate_name": "web-search-tool"
},
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": null
}
},
"auth_summary": {
"method": "manual",
"provider": "Brave",
"secrets": ["brave_api_key"],
"shared_auth": null,
"setup_url": "https://brave.com/search/api/"
},
"tags": ["default", "search"]
}
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# Build all WASM tools and channels from source.
#
# Verifies that every tool/channel in the registry compiles against the
# current WIT definitions. Used by CI and can be run locally.
#
# Prerequisites:
# rustup target add wasm32-wasip2
# cargo install cargo-component --locked
#
# Usage:
# ./scripts/build-wasm-extensions.sh # build all
# ./scripts/build-wasm-extensions.sh --tools # tools only
# ./scripts/build-wasm-extensions.sh --channels # channels only
set -euo pipefail
cd "$(dirname "$0")/.."
BUILD_TOOLS=true
BUILD_CHANNELS=true
FAILED=()
if [[ "${1:-}" == "--tools" ]]; then
BUILD_CHANNELS=false
elif [[ "${1:-}" == "--channels" ]]; then
BUILD_TOOLS=false
fi
build_extension() {
local manifest_path="$1"
local source_dir
local crate_name
source_dir=$(jq -r '.source.dir' "$manifest_path")
crate_name=$(jq -r '.source.crate_name' "$manifest_path")
local name
name=$(basename "$manifest_path" .json)
if [ ! -d "$source_dir" ]; then
echo " SKIP $name (source dir $source_dir not found)"
return 0
fi
echo " BUILD $name ($crate_name) from $source_dir"
if ! cargo component build --release --manifest-path "$source_dir/Cargo.toml" 2>&1; then
echo " FAIL $name"
FAILED+=("$name")
return 1
fi
echo " OK $name"
}
if $BUILD_TOOLS; then
echo "Building WASM tools..."
for manifest in registry/tools/*.json; do
build_extension "$manifest" || true
done
fi
if $BUILD_CHANNELS; then
echo "Building WASM channels..."
for manifest in registry/channels/*.json; do
build_extension "$manifest" || true
done
fi
echo ""
if [ ${#FAILED[@]} -gt 0 ]; then
echo "FAILED: ${FAILED[*]}"
exit 1
else
echo "All WASM extensions built successfully."
fi
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env bash
# commit-msg hook: require regression tests for fix commits.
#
# Installed by scripts/dev-setup.sh as .git/hooks/commit-msg.
# Bypass with [skip-regression-check] in the commit message.
set -euo pipefail
MSG_FILE="$1"
FIRST_LINE=$(head -1 "$MSG_FILE")
# --- 1. Is this a fix commit? ---
if ! grep -qiE '^(fix(\(.*\))?|hotfix|bugfix):' <<< "$FIRST_LINE"; then
exit 0
fi
# --- 2. Skip marker ---
if grep -qF '[skip-regression-check]' "$MSG_FILE"; then
exit 0
fi
# --- 3. Exempt static-only / docs-only changes ---
# Get staged files (commit-msg runs after staging is finalized).
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACMR)
if [ -z "$STAGED_FILES" ]; then
exit 0
fi
ALL_EXEMPT=true
while IFS= read -r file; do
case "$file" in
src/channels/web/static/*) ;;
*.md) ;;
*) ALL_EXEMPT=false; break ;;
esac
done <<< "$STAGED_FILES"
if [ "$ALL_EXEMPT" = true ]; then
exit 0
fi
# --- 4. Look for test changes in staged .rs files ---
# Fast path: new test attributes or test modules in added lines.
if git diff --cached -U0 -- '*.rs' | grep -qE '^\+.*(#\[test\]|#\[tokio::test\]|#\[cfg\(test\)\]|mod tests)'; then
exit 0
fi
# Whole-function context: detect edits inside existing test functions.
# -W shows the full enclosing function, so #[test] appears in context
# lines when changes are inside a test function.
if git diff --cached -W -- '*.rs' | awk '
/^@@/ { if (has_test && has_add) { found=1; exit } has_test=0; has_add=0 }
/^ .*#\[test\]/ || /^ .*#\[tokio::test\]/ || /^ .*#\[cfg\(test\)\]/ || /^ .*mod tests/ { has_test=1 }
/^\+.*#\[test\]/ || /^\+.*#\[tokio::test\]/ || /^\+.*#\[cfg\(test\)\]/ || /^\+.*mod tests/ { has_test=1 }
/^\+[^+]/ { has_add=1 }
END { if (has_test && has_add) found=1; exit !found }
'; then
exit 0
fi
# Also check for new/modified files under tests/
if grep -qE '^tests/' <<< "$STAGED_FILES"; then
exit 0
fi
# --- 5. No test found — block the commit ---
echo ""
echo "╔══════════════════════════════════════════════════════════════╗"
echo "║ REGRESSION TEST REQUIRED ║"
echo "║ ║"
echo "║ This commit looks like a bug fix but has no test changes. ║"
echo "║ Every fix should include a test that reproduces the bug. ║"
echo "║ ║"
echo "║ Options: ║"
echo "║ • Add a #[test] or #[tokio::test] that catches the bug ║"
echo "║ • Add [skip-regression-check] to your commit message ║"
echo "╚══════════════════════════════════════════════════════════════╝"
echo ""
exit 1
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
# Generate an HTML coverage report for a given set of tests.
#
# Usage:
# ./scripts/coverage.sh # all tests (lib only)
# ./scripts/coverage.sh safety # tests matching "safety"
# ./scripts/coverage.sh safety::sanitizer # specific module tests
# ./scripts/coverage.sh test_a test_b test_c # multiple test filters
#
# Options (env vars):
# COV_OPEN=1 Auto-open the report in a browser (default: 1)
# COV_FORMAT=html Output format: html, text, json, lcov (default: html)
# COV_OUT=coverage Output directory (default: coverage/)
# COV_FEATURES="" Extra --features to pass (default: none)
# COV_ALL_TARGETS=0 Set to 1 to include integration tests (default: lib only)
#
# Requires: cargo-llvm-cov (install: cargo install cargo-llvm-cov)
set -euo pipefail
COV_OPEN="${COV_OPEN:-1}"
COV_FORMAT="${COV_FORMAT:-html}"
COV_OUT="${COV_OUT:-coverage}"
COV_FEATURES="${COV_FEATURES:-}"
COV_ALL_TARGETS="${COV_ALL_TARGETS:-0}"
cd "$(git rev-parse --show-toplevel)"
if ! command -v cargo-llvm-cov &>/dev/null; then
echo "ERROR: cargo-llvm-cov not found. Install with: cargo install cargo-llvm-cov"
exit 1
fi
# Clean stale profiling data to avoid "mismatched data" warnings.
cargo llvm-cov clean --workspace 2>/dev/null || true
# Build the cargo llvm-cov command
cmd=(cargo llvm-cov)
# Features
if [[ -n "$COV_FEATURES" ]]; then
cmd+=(--features "$COV_FEATURES")
else
cmd+=(--all-features)
fi
# By default, only run the lib unit tests (fast, no integration test compilation).
# Set COV_ALL_TARGETS=1 to include integration tests.
if [[ "$COV_ALL_TARGETS" != "1" ]]; then
cmd+=(--lib)
fi
# Output format
case "$COV_FORMAT" in
html)
cmd+=(--html --output-dir "$COV_OUT")
;;
text)
cmd+=(--text)
;;
json)
cmd+=(--json --output-path "$COV_OUT/coverage.json")
;;
lcov)
cmd+=(--lcov --output-path "$COV_OUT/lcov.info")
;;
*)
echo "ERROR: Unknown format '$COV_FORMAT'. Use: html, text, json, lcov"
exit 1
;;
esac
# Test name filters (passed after -- to cargo test)
if [[ $# -gt 0 ]]; then
if [[ $# -eq 1 ]]; then
cmd+=(-- "$1")
else
# Join filters with | for regex matching
filter=$(IFS='|'; echo "$*")
cmd+=(-- "$filter")
fi
fi
echo "Running: ${cmd[*]}"
echo ""
"${cmd[@]}"
# Open report
if [[ "$COV_FORMAT" == "html" && "$COV_OPEN" == "1" ]]; then
index="$COV_OUT/html/index.html"
if [[ -f "$index" ]]; then
echo ""
echo "Report: $index"
if command -v open &>/dev/null; then
open "$index"
elif command -v xdg-open &>/dev/null; then
xdg-open "$index"
fi
fi
fi
+17 -5
View File
@@ -24,14 +24,14 @@ if ! command -v rustup &>/dev/null; then
echo "ERROR: rustup not found. Install from https://rustup.rs"
exit 1
fi
echo "[1/5] rustup found: $(rustup --version 2>/dev/null | head -1)"
echo "[1/6] rustup found: $(rustup --version 2>/dev/null | head -1)"
# 2. Add WASM target (required by build.rs for channel compilation)
echo "[2/5] Adding wasm32-wasip2 target..."
echo "[2/6] Adding wasm32-wasip2 target..."
rustup target add wasm32-wasip2
# 3. Install wasm-tools (required by build.rs for WASM component model)
echo "[3/5] Installing wasm-tools..."
echo "[3/6] Installing wasm-tools..."
if command -v wasm-tools &>/dev/null; then
echo " wasm-tools already installed: $(wasm-tools --version)"
else
@@ -39,13 +39,25 @@ else
fi
# 4. Verify the project compiles
echo "[4/5] Running cargo check..."
echo "[4/6] Running cargo check..."
cargo check
# 5. Run tests using libsql temp DB (no Docker/external DB needed)
echo "[5/5] Running tests (no external DB required)..."
echo "[5/6] Running tests (no external DB required)..."
cargo test
# 6. Install git hooks
echo "[6/6] Installing git hooks..."
HOOKS_DIR=$(git rev-parse --git-path hooks 2>/dev/null) || true
if [ -n "$HOOKS_DIR" ]; then
mkdir -p "$HOOKS_DIR"
SCRIPT_ABS="$(cd "$(dirname "$0")" && pwd)/commit-msg-regression.sh"
ln -sf "$SCRIPT_ABS" "$HOOKS_DIR/commit-msg"
echo " commit-msg hook installed (regression test enforcement)"
else
echo " Skipped: not a git repository"
fi
echo ""
echo "=== Setup complete ==="
echo ""
+225
View File
@@ -0,0 +1,225 @@
---
name: local-test
version: 0.1.0
description: Build, run, and test IronClaw locally using Docker containers and Chrome MCP browser automation.
activation:
keywords:
- test locally
- local test
- docker test
- test my changes
- test in docker
- test web gateway
- spin up test
- test container
patterns:
- "test.*local"
- "docker.*test"
- "spin.*up.*test"
- "test.*changes.*docker"
max_context_tokens: 3000
---
# Local Testing with Docker + Chrome MCP
Use this skill to build, run, and test IronClaw web gateway changes locally using `Dockerfile.test` and Chrome MCP browser automation tools.
## Quick Start
```bash
# Build the test image (libsql-only, no PostgreSQL needed)
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
# Run on port 3003 (default)
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<key> \
ironclaw-test
# Open in browser
# http://localhost:3003/?token=test
```
## Building the Image
The test Dockerfile uses a two-stage build: Rust compilation with `--features libsql` (no PostgreSQL dependency), then a minimal Debian runtime image.
```bash
docker build --platform linux/amd64 -f Dockerfile.test -t ironclaw-test .
```
Build takes ~5-10 minutes on first run (cached subsequent builds are faster). The `--platform linux/amd64` flag avoids QEMU warnings on Apple Silicon but can be omitted if targeting native architecture.
## Running Containers
### Required Environment Variables
| Variable | Purpose | Default in Dockerfile |
|----------|---------|----------------------|
| `ONBOARD_COMPLETED=true` | Skip onboarding wizard (exits immediately otherwise) | not set |
| `CLI_ENABLED=false` | Disable TUI/REPL (causes EOF shutdown otherwise) | not set |
### LLM Backend Configuration
Pick ONE of these configurations:
**NEAR AI (API key mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=<your-key> \
ironclaw-test
```
**NEAR AI (session token mode):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_SESSION_TOKEN=<sess_xxx> \
-e NEARAI_BASE_URL=https://private.near.ai \
ironclaw-test
```
**OpenAI:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=openai \
-e OPENAI_API_KEY=<your-key> \
ironclaw-test
```
**Anthropic:**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e LLM_BACKEND=anthropic \
-e ANTHROPIC_API_KEY=<your-key> \
ironclaw-test
```
**Dummy run (no LLM, just test the UI loads):**
```bash
docker run --rm -p 3003:3003 \
-e ONBOARD_COMPLETED=true \
-e CLI_ENABLED=false \
-e NEARAI_API_KEY=dummy \
ironclaw-test
```
### Common Overrides
| Variable | Purpose | Example |
|----------|---------|---------|
| `GATEWAY_PORT` | Change the listen port | `3003` (default) |
| `GATEWAY_AUTH_TOKEN` | Auth token for API | `test` (default) |
| `NEARAI_MODEL` | Override LLM model | `claude-3-5-sonnet-20241022` |
| `RUST_LOG` | Logging verbosity | `ironclaw=debug` |
| `ROUTINES_ENABLED` | Enable routines | `true`/`false` |
| `SKILLS_ENABLED` | Enable skills system | `true` (default) |
### Multi-Instance Testing
Run multiple containers on different host ports:
```bash
docker run --rm -d --name ic-test-a -p 3003:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
docker run --rm -d --name ic-test-b -p 3004:3003 -e ONBOARD_COMPLETED=true -e CLI_ENABLED=false -e NEARAI_API_KEY=dummy ironclaw-test
```
## Chrome MCP Testing Workflow
Use the Claude for Chrome browser automation tools to test the web UI.
### Step 1: Get Browser Context
```
mcp__claude-in-chrome__tabs_context_mcp
```
Always start here to see current tabs and get fresh tab IDs.
### Step 2: Open the Gateway
```
mcp__claude-in-chrome__tabs_create_mcp url=http://localhost:3003/?token=test
```
### Step 3: Verify the Page
```
mcp__claude-in-chrome__read_page
```
Check for:
- "Connected" indicator in top-right
- All tabs visible: Chat, Memory, Jobs, Routines, Extensions, Skills
### Step 4: Take Screenshots
```
mcp__claude-in-chrome__computer action=screenshot
```
### Step 5: Test Mobile Viewport
```
mcp__claude-in-chrome__resize_window width=375 height=812
mcp__claude-in-chrome__computer action=screenshot
```
Reset to desktop:
```
mcp__claude-in-chrome__resize_window width=1280 height=800
```
### Step 6: Run JavaScript Checks
```
mcp__claude-in-chrome__javascript_tool script="document.querySelector('.connection-status')?.textContent"
```
### Step 7: Test Interactions
Click tabs, send messages, search skills — use `computer` tool with `action=click` and coordinate-based clicks, or use `find` + `form_input` for text entry.
## Cleanup
```bash
# Stop a specific container
docker stop ic-test-a
# Stop all test containers
docker ps --filter ancestor=ironclaw-test -q | xargs -r docker stop
# Remove the test image
docker rmi ironclaw-test
```
## Troubleshooting
### Container exits immediately
- **Missing `ONBOARD_COMPLETED=true`**: The onboarding wizard tries to read stdin, gets EOF, and exits.
- **Missing `CLI_ENABLED=false`**: The REPL channel reads stdin, gets EOF, and shuts down the agent.
### "Model not found" or LLM errors
- Check that your API key/token is valid and the model name is correct.
- For NEAR AI session token mode, you also need `NEARAI_BASE_URL=https://private.near.ai`.
### Platform mismatch warnings on Apple Silicon
- The `--platform linux/amd64` flag causes QEMU emulation warnings — these are harmless.
- Alternatively, omit the flag and build natively if your dependencies support ARM64.
### Port already in use
- The dev server defaults to port 3001; the test Dockerfile defaults to 3003 to avoid conflicts.
- Use a different host port: `-p 3005:3003`.
### Cannot connect from browser
- Verify `GATEWAY_HOST=0.0.0.0` (set by default in Dockerfile).
- Check the container logs: `docker logs <container-id>`.
- Make sure you include the token query param: `?token=test`.
+22 -3
View File
@@ -73,6 +73,10 @@ pub struct AgentDeps {
pub hooks: Arc<HookRegistry>,
/// Cost enforcement guardrails (daily budget, hourly rate limits).
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>,
/// HTTP interceptor for trace recording/replay.
pub http_interceptor: Option<Arc<dyn crate::llm::recording::HttpInterceptor>>,
}
/// The main agent that coordinates all components.
@@ -111,7 +115,7 @@ impl Agent {
let session_manager = session_manager.unwrap_or_else(|| Arc::new(SessionManager::new()));
let scheduler = Arc::new(Scheduler::new(
let mut scheduler = Scheduler::new(
config.clone(),
context_manager.clone(),
deps.llm.clone(),
@@ -119,7 +123,11 @@ impl Agent {
deps.tools.clone(),
deps.store.clone(),
deps.hooks.clone(),
));
);
if let Some(ref tx) = deps.sse_tx {
scheduler.set_sse_sender(tx.clone());
}
let scheduler = Arc::new(scheduler);
Self {
config,
@@ -627,6 +635,10 @@ impl Agent {
// Parse submission type first
let mut submission = SubmissionParser::parse(&message.content);
tracing::debug!(
"[agent_loop] Parsed submission: {:?}",
std::any::type_name_of_val(&submission)
);
// Hook: BeforeInbound — allow hooks to modify or reject user input
if let Submission::UserInput { ref content } = submission {
@@ -711,7 +723,14 @@ impl Agent {
.await
}
Submission::SystemCommand { command, args } => {
self.handle_system_command(&command, &args).await
tracing::debug!(
"[agent_loop] SystemCommand: command={}, channel={}",
command,
message.channel
);
// Authorization checks (including restart channel check) are enforced in handle_system_command
self.handle_system_command(&command, &args, &message.channel)
.await
}
Submission::Undo => self.process_undo(session, thread_id).await,
Submission::Redo => self.process_redo(session, thread_id).await,
+70 -2
View File
@@ -68,7 +68,10 @@ impl Agent {
self.handle_help_job(&message.user_id, &job_id).await?
}
MessageIntent::Command { command, args } => {
match self.handle_command(&command, &args).await? {
match self
.handle_command(&command, &args, &message.channel)
.await?
{
Some(s) => s,
None => return Ok(SubmissionResult::Ok { message: None }), // Shutdown signal
}
@@ -466,6 +469,7 @@ impl Agent {
&self,
command: &str,
args: &[String],
channel: &str,
) -> Result<SubmissionResult, Error> {
match command {
"help" => Ok(SubmissionResult::response(concat!(
@@ -501,12 +505,75 @@ impl Agent {
" /heartbeat Run heartbeat check\n",
" /summarize Summarize current thread\n",
" /suggest Suggest next steps\n",
" /restart Gracefully restart the process\n",
"\n",
" /quit Exit",
))),
"ping" => Ok(SubmissionResult::response("pong!")),
"restart" => {
tracing::info!("[commands::restart] Restart command received");
// Channel authorization check: restart is only available via web interface
if channel != "gateway" {
tracing::warn!(
"[commands::restart] Restart rejected: not from gateway channel (from: {})",
channel
);
return Ok(SubmissionResult::error(
"Restart is only available through the web interface with explicit user confirmation. \
Use the Restart button in the UI."
.to_string(),
));
}
// Environment check: restart is only available in Docker containers
let in_docker = std::env::var("IRONCLAW_IN_DOCKER")
.map(|v| v.to_lowercase() == "true")
.unwrap_or(false);
tracing::debug!("[commands::restart] IRONCLAW_IN_DOCKER={}", in_docker);
if !in_docker {
tracing::warn!(
"[commands::restart] Restart rejected: not in Docker environment"
);
return Ok(SubmissionResult::error(
"Restart is not available in this environment. \
The IRONCLAW_IN_DOCKER environment variable must be set to 'true' for Docker deployments."
.to_string(),
));
}
// Execute restart tool directly (don't dispatch as a job for LLM planning)
// This ensures the tool runs immediately without LLM involvement
use crate::tools::Tool;
let tool = crate::tools::builtin::RestartTool;
let params = serde_json::json!({});
// Create a minimal JobContext for the tool
let dummy_ctx =
crate::context::JobContext::with_user("system", "Restart", "Graceful restart");
match tool.execute(params, &dummy_ctx).await {
Ok(output) => {
tracing::info!("[commands::restart] RestartTool executed successfully");
// Extract text from the ToolOutput result
let response = match output.result {
serde_json::Value::String(s) => s,
_ => output.result.to_string(),
};
Ok(SubmissionResult::response(response))
}
Err(e) => {
tracing::error!(
"[commands::restart] RestartTool execution failed: {:?}",
e
);
Ok(SubmissionResult::error(format!("Restart failed: {}", e)))
}
}
}
"version" => Ok(SubmissionResult::response(format!(
"{} v{}",
env!("CARGO_PKG_NAME"),
@@ -744,10 +811,11 @@ impl Agent {
&self,
command: &str,
args: &[String],
channel: &str,
) -> Result<Option<String>, Error> {
// System commands are now handled directly via Submission::SystemCommand,
// but the router may still send us unknown /commands.
match self.handle_system_command(command, args).await? {
match self.handle_system_command(command, args, channel).await? {
SubmissionResult::Response { content } => Ok(Some(content)),
SubmissionResult::Ok { message } => Ok(message),
SubmissionResult::Error { message } => Ok(Some(format!("Error: {}", message))),
+148 -19
View File
@@ -15,6 +15,7 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::{ChatMessage, Reasoning, ReasoningContext, RespondResult};
use crate::tools::redact_params;
/// Result of the agentic loop execution.
pub(super) enum AgenticLoopResult {
@@ -126,7 +127,9 @@ impl Agent {
let mut context_messages = initial_messages;
// Create a JobContext for tool execution (chat doesn't have a real job)
let job_ctx = JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
let max_tool_iterations = self.config.max_tool_iterations;
// Force a text-only response on the last iteration to guarantee termination
@@ -291,7 +294,11 @@ impl Agent {
match output.result {
RespondResult::Text(text) => {
return Ok(AgenticLoopResult::Response(text));
// Strip internal "[Called tool ...]" text that can leak when
// provider flattening (e.g. NEAR AI) converts tool_calls to
// plain text and the LLM echoes it back.
let sanitized = strip_internal_tool_call_text(&text);
return Ok(AgenticLoopResult::Response(sanitized));
}
RespondResult::ToolCalls {
tool_calls,
@@ -317,14 +324,25 @@ impl Agent {
)
.await;
// Record tool calls in the thread
// Record tool calls in the thread with sensitive params redacted.
// Look up each tool's sensitive_params before acquiring the session lock.
{
let mut redacted_args: Vec<serde_json::Value> =
Vec::with_capacity(tool_calls.len());
for tc in &tool_calls {
let safe = if let Some(tool) = self.tools().get(&tc.name).await {
redact_params(&tc.arguments, tool.sensitive_params())
} else {
tc.arguments.clone()
};
redacted_args.push(safe);
}
let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id)
&& let Some(turn) = thread.last_turn_mut()
{
for tc in &tool_calls {
turn.record_tool_call(&tc.name, tc.arguments.clone());
for (tc, safe_args) in tool_calls.iter().zip(redacted_args) {
turn.record_tool_call(&tc.name, safe_args);
}
}
}
@@ -353,11 +371,22 @@ impl Agent {
for (idx, original_tc) in tool_calls.iter().enumerate() {
let mut tc = original_tc.clone();
// Fetch the tool upfront so we can redact sensitive params
// before they touch hooks or approval display.
let tool_opt = self.tools().get(&tc.name).await;
let sensitive = tool_opt
.as_ref()
.map(|t| t.sensitive_params())
.unwrap_or(&[]);
// Hook: BeforeToolCall (runs before approval so hooks can
// modify parameters — approval is checked on final params)
// modify parameters — approval is checked on final params).
// Hooks receive redacted params so sensitive values are not
// exposed to hook handlers or their logs.
let hook_params = redact_params(&tc.arguments, sensitive);
let event = crate::hooks::HookEvent::ToolCall {
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
parameters: hook_params,
user_id: message.user_id.clone(),
context: "chat".to_string(),
};
@@ -384,8 +413,20 @@ impl Agent {
}
Ok(crate::hooks::HookOutcome::Continue {
modified: Some(new_params),
}) => match serde_json::from_str(&new_params) {
Ok(parsed) => tc.arguments = parsed,
}) => match serde_json::from_str::<serde_json::Value>(&new_params) {
Ok(mut parsed) => {
// Restore original sensitive param values so a hook
// cannot overwrite them (they were sent as [REDACTED]).
if let Some(obj) = parsed.as_object_mut() {
for key in sensitive {
if let Some(orig_val) = original_tc.arguments.get(*key)
{
obj.insert((*key).to_string(), orig_val.clone());
}
}
}
tc.arguments = parsed;
}
Err(e) => {
tracing::warn!(
tool = %tc.name,
@@ -400,7 +441,7 @@ impl Agent {
// Check if tool requires approval on the final (post-hook)
// parameters. Skipped when auto_approve_tools is set.
if !self.config.auto_approve_tools
&& let Some(tool) = self.tools().get(&tc.name).await
&& let Some(tool) = tool_opt
{
use crate::tools::ApprovalRequirement;
let needs_approval = match tool.requires_approval(&tc.arguments) {
@@ -447,14 +488,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let disp_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
disp_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -495,13 +539,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -641,6 +688,15 @@ impl Agent {
deferred_auth = Some(instructions);
}
// Stash full output so subsequent tools can reference it
if let Ok(ref output) = tool_result {
job_ctx
.tool_output_stash
.write()
.await
.insert(tc.id.clone(), output.clone());
}
// Sanitize and add tool result to context
let result_content = match tool_result {
Ok(output) => {
@@ -671,10 +727,15 @@ impl Agent {
// Handle approval if a tool needed it
if let Some((approval_idx, tc, tool)) = approval_needed {
// Show redacted params in the approval UI — the user already knows
// the sensitive value (they provided it); showing it again is
// unnecessary and creates a leakage path through channel logs.
let display_params = redact_params(&tc.arguments, tool.sensitive_params());
let pending = PendingApproval {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: display_params,
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -734,9 +795,10 @@ pub(super) async fn execute_chat_tool_standalone(
.into());
}
let safe_params = redact_params(params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
"Tool call started"
);
@@ -900,6 +962,38 @@ fn compact_messages_for_retry(messages: &[ChatMessage]) -> Vec<ChatMessage> {
compacted
}
/// Strip internal `[Called tool ...]` and `[Tool ... returned: ...]` markers
/// from a response string. These markers are inserted by provider-level message
/// flattening (e.g. NEAR AI) and can leak into the user-visible response when
/// the LLM echoes them back.
fn strip_internal_tool_call_text(text: &str) -> String {
// Remove lines that are purely internal tool-call markers.
// Pattern: lines matching `[Called tool <name>(...)]` or `[Tool <name> returned: ...]`
let result = text
.lines()
.filter(|line| {
let trimmed = line.trim();
!((trimmed.starts_with("[Called tool ") && trimmed.ends_with(']'))
|| (trimmed.starts_with("[Tool ")
&& trimmed.contains(" returned:")
&& trimmed.ends_with(']')))
})
.fold(String::new(), |mut acc, s| {
if !acc.is_empty() {
acc.push('\n');
}
acc.push_str(s);
acc
});
let result = result.trim();
if result.is_empty() {
"I wasn't able to complete that request. Could you try rephrasing or providing more details?".to_string()
} else {
result.to_string()
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
@@ -982,6 +1076,8 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
@@ -1085,6 +1181,7 @@ mod tests {
request_id: uuid::Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "echo hi"}),
display_parameters: serde_json::json!({"command": "echo hi"}),
description: "Run shell command".to_string(),
tool_call_id: "call_1".to_string(),
context_messages: vec![],
@@ -1719,6 +1816,8 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
@@ -1830,6 +1929,8 @@ mod tests {
skills_config: SkillsConfig::default(),
hooks: Arc::new(HookRegistry::new()),
cost_guard: Arc::new(CostGuard::new(CostGuardConfig::default())),
sse_tx: None,
http_interceptor: None,
};
Agent::new(
@@ -1899,4 +2000,32 @@ mod tests {
}
}
}
#[test]
fn test_strip_internal_tool_call_text_removes_markers() {
let input = "[Called tool search({\"query\": \"test\"})]\nHere is the answer.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Here is the answer.");
}
#[test]
fn test_strip_internal_tool_call_text_removes_returned_markers() {
let input = "[Tool search returned: some result]\nSummary of findings.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, "Summary of findings.");
}
#[test]
fn test_strip_internal_tool_call_text_all_markers_yields_fallback() {
let input = "[Called tool search({\"query\": \"test\"})]\n[Tool search returned: error]";
let result = super::strip_internal_tool_call_text(input);
assert!(result.contains("wasn't able to complete"));
}
#[test]
fn test_strip_internal_tool_call_text_preserves_normal_text() {
let input = "This is a normal response with [brackets] inside.";
let result = super::strip_internal_tool_call_text(input);
assert_eq!(result, input);
}
}
+32
View File
@@ -10,6 +10,7 @@ use uuid::Uuid;
use crate::agent::task::{Task, TaskContext, TaskOutput};
use crate::agent::worker::{Worker, WorkerDeps};
use crate::channels::web::types::SseEvent;
use crate::config::AgentConfig;
use crate::context::{ContextManager, JobContext, JobState};
use crate::db::Database;
@@ -28,6 +29,8 @@ pub enum WorkerMessage {
Stop,
/// Check health.
Ping,
/// Inject a follow-up user message into the worker's reasoning context.
UserMessage(String),
}
/// Status of a scheduled job.
@@ -51,6 +54,8 @@ pub struct Scheduler {
tools: Arc<ToolRegistry>,
store: Option<Arc<dyn Database>>,
hooks: Arc<HookRegistry>,
/// SSE broadcast sender for live job event streaming.
sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
/// Running jobs (main LLM-driven jobs).
jobs: Arc<RwLock<HashMap<Uuid, ScheduledJob>>>,
/// Running sub-tasks (tool executions, background tasks).
@@ -76,11 +81,17 @@ impl Scheduler {
tools,
store,
hooks,
sse_tx: None,
jobs: Arc::new(RwLock::new(HashMap::new())),
subtasks: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Set the SSE broadcast sender for live job event streaming.
pub fn set_sse_sender(&mut self, tx: tokio::sync::broadcast::Sender<SseEvent>) {
self.sse_tx = Some(tx);
}
/// Create, persist, and schedule a job in one shot.
///
/// This is the preferred entry point for dispatching new jobs. It:
@@ -169,6 +180,7 @@ impl Scheduler {
hooks: self.hooks.clone(),
timeout: self.config.job_timeout,
use_planning: self.config.use_planning,
sse_tx: self.sse_tx.clone(),
};
let worker = Worker::new(job_id, deps);
@@ -500,6 +512,26 @@ impl Scheduler {
Ok(())
}
/// Send a follow-up user message to a running job.
///
/// Returns `Ok(())` if the message was queued, `Err` if the job is not running.
pub async fn send_message(&self, job_id: Uuid, content: String) -> Result<(), JobError> {
// Clone the sender while holding the lock, then release before the
// async send to avoid blocking scheduler writes during backpressure.
let tx = {
let jobs = self.jobs.read().await;
let scheduled = jobs.get(&job_id).ok_or(JobError::NotFound { id: job_id })?;
scheduled.tx.clone()
};
tx.send(WorkerMessage::UserMessage(content))
.await
.map_err(|_| JobError::Failed {
id: job_id,
reason: "Worker channel closed".to_string(),
})?;
Ok(())
}
/// Check if a job is running.
pub async fn is_running(&self, job_id: Uuid) -> bool {
self.jobs.read().await.contains_key(&job_id)
+7 -1
View File
@@ -148,8 +148,12 @@ pub struct PendingApproval {
pub request_id: Uuid,
/// Tool name requiring approval.
pub tool_name: String,
/// Tool parameters.
/// Tool parameters (original values, used for execution).
pub parameters: serde_json::Value,
/// Redacted tool parameters (sensitive values replaced with `[REDACTED]`).
/// Used for display in approval UI, logs, and SSE broadcasts.
#[serde(default)]
pub display_parameters: serde_json::Value,
/// Description of what the tool will do.
pub description: String,
/// Tool call ID from LLM (for proper context continuation).
@@ -950,6 +954,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "shell".to_string(),
parameters: serde_json::json!({"command": "rm -rf /"}),
display_parameters: serde_json::json!({"command": "rm -rf /"}),
description: "dangerous command".to_string(),
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
@@ -974,6 +979,7 @@ mod tests {
request_id: Uuid::new_v4(),
tool_name: "http".to_string(),
parameters: serde_json::json!({}),
display_parameters: serde_json::json!({}),
description: "test".to_string(),
tool_call_id: "call_456".to_string(),
context_messages: vec![],
+8
View File
@@ -14,6 +14,7 @@ impl SubmissionParser {
pub fn parse(content: &str) -> Submission {
let trimmed = content.trim();
let lower = trimmed.to_lowercase();
tracing::debug!("[SubmissionParser::parse] Parsing input: {:?}", trimmed);
// Control commands (exact match or prefix)
if lower == "/undo" {
@@ -91,6 +92,13 @@ impl SubmissionParser {
args: vec![],
};
}
if lower == "/restart" {
tracing::debug!("[SubmissionParser::parse] Recognized /restart command");
return Submission::SystemCommand {
command: "restart".to_string(),
args: vec![],
};
}
if lower.starts_with("/model") {
let args: Vec<String> = trimmed
.split_whitespace()
+33 -21
View File
@@ -21,6 +21,7 @@ use crate::channels::{IncomingMessage, StatusUpdate};
use crate::context::JobContext;
use crate::error::Error;
use crate::llm::ChatMessage;
use crate::tools::redact_params;
impl Agent {
/// Hydrate a historical thread from DB into memory if not already present.
@@ -357,7 +358,7 @@ impl Agent {
let request_id = pending.request_id;
let tool_name = pending.tool_name.clone();
let description = pending.description.clone();
let parameters = pending.parameters.clone();
let parameters = pending.display_parameters.clone();
thread.await_approval(pending);
let _ = self
.channels
@@ -733,8 +734,9 @@ impl Agent {
}
// Execute the approved tool and continue the loop
let job_ctx =
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
let _ = self
.channels
@@ -751,14 +753,17 @@ impl Agent {
.execute_chat_tool(&pending.tool_name, &pending.parameters, &job_ctx)
.await;
let tool_ref = self.tools().get(&pending.tool_name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: pending.tool_name.clone(),
success: tool_result.is_ok(),
},
StatusUpdate::tool_completed(
pending.tool_name.clone(),
&tool_result,
&pending.display_parameters,
tool_ref.as_deref(),
),
&message.metadata,
)
.await;
@@ -908,14 +913,17 @@ impl Agent {
.execute_chat_tool(&tc.name, &tc.arguments, &job_ctx)
.await;
let deferred_tool = self.tools().get(&tc.name).await;
let _ = self
.channels
.send_status(
&message.channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
deferred_tool.as_deref(),
),
&message.metadata,
)
.await;
@@ -957,13 +965,16 @@ impl Agent {
)
.await;
let par_tool = tools.get(&tc.name).await;
let _ = channels
.send_status(
&channel,
StatusUpdate::ToolCompleted {
name: tc.name.clone(),
success: result.is_ok(),
},
StatusUpdate::tool_completed(
tc.name.clone(),
&result,
&tc.arguments,
par_tool.as_deref(),
),
&metadata,
)
.await;
@@ -1086,6 +1097,7 @@ impl Agent {
request_id: Uuid::new_v4(),
tool_name: tc.name.clone(),
parameters: tc.arguments.clone(),
display_parameters: redact_params(&tc.arguments, tool.sensitive_params()),
description: tool.description().to_string(),
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
@@ -1095,7 +1107,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
{
let mut sess = session.lock().await;
@@ -1162,7 +1174,7 @@ impl Agent {
let request_id = new_pending.request_id;
let tool_name = new_pending.tool_name.clone();
let description = new_pending.description.clone();
let parameters = new_pending.parameters.clone();
let parameters = new_pending.display_parameters.clone();
thread.await_approval(new_pending);
let _ = self
.channels
@@ -1284,7 +1296,7 @@ impl Agent {
};
match ext_mgr.auth(&pending.extension_name, Some(token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
tracing::info!(
"Extension '{}' authenticated via auth mode",
pending.extension_name
@@ -1353,8 +1365,8 @@ impl Agent {
}
}
let msg = result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token. Please try again.".to_string());
// Re-emit AuthRequired so web UI re-shows the card
let _ = self
@@ -1364,8 +1376,8 @@ impl Agent {
StatusUpdate::AuthRequired {
extension_name: pending.extension_name.clone(),
instructions: Some(msg.clone()),
auth_url: result.auth_url,
setup_url: result.setup_url,
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
},
&message.metadata,
)
+220 -25
View File
@@ -9,6 +9,7 @@ use uuid::Uuid;
use crate::agent::scheduler::WorkerMessage;
use crate::agent::task::TaskOutput;
use crate::channels::web::types::SseEvent;
use crate::context::{ContextManager, JobState};
use crate::db::Database;
use crate::error::Error;
@@ -17,8 +18,8 @@ use crate::llm::{
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolSelection,
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::rate_limiter::RateLimitResult;
use crate::tools::{ToolRegistry, redact_params};
/// Shared dependencies for worker execution.
///
@@ -34,6 +35,8 @@ pub struct WorkerDeps {
pub hooks: Arc<HookRegistry>,
pub timeout: Duration,
pub use_planning: bool,
/// SSE broadcast sender for live job event streaming to the web gateway.
pub sse_tx: Option<tokio::sync::broadcast::Sender<SseEvent>>,
}
/// Worker that executes a single job.
@@ -98,18 +101,90 @@ impl Worker {
}
}
/// Fire-and-forget persistence of a job event.
/// Fire-and-forget persistence of a job event and SSE broadcast.
fn log_event(&self, event_type: &str, data: serde_json::Value) {
let job_id = self.job_id;
// Persist to DB
if let Some(store) = self.store() {
let store = store.clone();
let job_id = self.job_id;
let event_type = event_type.to_string();
let et = event_type.to_string();
let d = data.clone();
tokio::spawn(async move {
if let Err(e) = store.save_job_event(job_id, &event_type, &data).await {
if let Err(e) = store.save_job_event(job_id, &et, &d).await {
tracing::warn!("Failed to persist event for job {}: {}", job_id, e);
}
});
}
// Broadcast SSE for live web UI updates
if let Some(ref tx) = self.deps.sse_tx {
let job_id_str = job_id.to_string();
let event = match event_type {
"message" => Some(SseEvent::JobMessage {
job_id: job_id_str,
role: data
.get("role")
.and_then(|v| v.as_str())
.unwrap_or("assistant")
.to_string(),
content: data
.get("content")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"tool_use" => Some(SseEvent::JobToolUse {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
input: data
.get("input")
.cloned()
.unwrap_or(serde_json::Value::Null),
}),
"tool_result" => Some(SseEvent::JobToolResult {
job_id: job_id_str,
tool_name: data
.get("tool_name")
.and_then(|v| v.as_str())
.unwrap_or("unknown")
.to_string(),
output: data
.get("output")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"status" => Some(SseEvent::JobStatus {
job_id: job_id_str,
message: data
.get("message")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
}),
"result" => Some(SseEvent::JobResult {
job_id: job_id_str,
status: data
.get("status")
.and_then(|v| v.as_str())
.unwrap_or("completed")
.to_string(),
session_id: data
.get("session_id")
.and_then(|v| v.as_str())
.map(|s| s.to_string()),
}),
_ => None,
};
if let Some(event) = event {
let _ = tx.send(event);
}
}
}
/// Run the worker until the job is complete or stopped.
@@ -123,7 +198,7 @@ impl Worker {
tracing::debug!("Worker for job {} stopped before starting", self.job_id);
return Ok(());
}
Some(WorkerMessage::Ping) => {}
Some(WorkerMessage::Ping) | Some(WorkerMessage::UserMessage(_)) => {}
}
// Get job context
@@ -219,6 +294,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.unwrap_or(50) as usize;
let max_iterations = max_iterations.min(MAX_WORKER_ITERATIONS);
let mut iteration = 0;
const MAX_CONSECUTIVE_RATE_LIMITS: usize = 10;
let mut consecutive_rate_limits = 0usize;
// Initial tool definitions for planning (will be refreshed in loop)
reason_ctx.available_tools = self.tools().tool_definitions().await;
@@ -269,15 +346,27 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
None
};
// If we have a plan, execute it
// If we have a plan, execute it. Two exit paths:
// 1. Plan ran to completion → job is Completed or needs continuation
// (check state and only fall through if not terminal)
// 2. Plan was interrupted by UserMessage → fall through to direct loop
if let Some(ref plan) = plan {
return self.execute_plan(rx, reasoning, reason_ctx, plan).await;
self.execute_plan(rx, reasoning, reason_ctx, plan).await?;
// If the plan marked the job terminal, we're done. Only fall
// through to the direct selection loop if the plan was
// interrupted or explicitly left the job in-progress.
if let Ok(ctx) = self.context_manager().get_context(self.job_id).await
&& (ctx.state.is_terminal() || ctx.state == JobState::Stuck)
{
return Ok(());
}
}
// Otherwise, use direct tool selection loop
// Direct tool selection loop (also used as fallback after plan interruption)
loop {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!("Worker for job {} received stop signal", self.job_id);
@@ -287,6 +376,20 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"Worker received follow-up user message"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
}
}
}
@@ -307,12 +410,64 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Refresh tool definitions so newly built tools become visible
reason_ctx.available_tools = self.tools().tool_definitions().await;
// Select next tool(s) to use
let selections = reasoning.select_tools(reason_ctx).await?;
// Select next tool(s) to use, with rate-limit retry.
let selections = match reasoning.select_tools(reason_ctx).await {
Ok(s) => s,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during tool selection, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
if selections.is_empty() {
// No tools from select_tools, ask LLM directly (may still return tool calls)
let respond_output = reasoning.respond_with_tools(reason_ctx).await?;
let respond_output = match reasoning.respond_with_tools(reason_ctx).await {
Ok(o) => o,
Err(crate::error::LlmError::RateLimited { retry_after, .. }) => {
consecutive_rate_limits += 1;
let wait = retry_after.unwrap_or(Duration::from_secs(5));
tracing::warn!(
job_id = %self.job_id,
wait_secs = wait.as_secs(),
attempt = consecutive_rate_limits,
"LLM rate limited during respond_with_tools, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
return Ok(());
}
self.log_event(
"status",
serde_json::json!({
"message": format!("Rate limited, retrying in {}s ({}/{})...",
wait.as_secs(), consecutive_rate_limits, MAX_CONSECUTIVE_RATE_LIMITS),
}),
);
tokio::time::sleep(wait).await;
continue;
}
Err(e) => return Err(e.into()),
};
match respond_output.result {
RespondResult::Text(response) => {
@@ -424,6 +579,11 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
// Reset rate-limit counter after a successful iteration (all LLM
// calls succeeded). Placed here so alternating success/fail between
// select_tools and respond_with_tools cannot bypass the cap.
consecutive_rate_limits = 0;
// Small delay between iterations
tokio::time::sleep(Duration::from_millis(100)).await;
}
@@ -540,9 +700,10 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
// Run BeforeToolCall hook
let params = {
use crate::hooks::{HookError, HookEvent, HookOutcome};
let hook_params = redact_params(params, tool.sensitive_params());
let event = HookEvent::ToolCall {
tool_name: tool_name.to_string(),
parameters: params.clone(),
parameters: hook_params,
user_id: job_ctx.user_id.clone(),
context: format!("job:{}", job_id),
};
@@ -598,9 +759,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.into());
}
// Redact sensitive parameter values (e.g. secret_save's "value") before
// they touch any observability or audit path.
let safe_params = redact_params(&params, tool.sensitive_params());
tracing::debug!(
tool = %tool_name,
params = %params,
params = %safe_params,
job = %job_id,
"Tool call started"
);
@@ -652,7 +816,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
match deps
.context_manager
.update_memory(job_id, |mem| {
let rec = mem.create_action(tool_name, params.clone()).succeed(
let rec = mem.create_action(tool_name, safe_params.clone()).succeed(
output_str.clone(),
output.result.clone(),
elapsed,
@@ -674,7 +838,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail(e.to_string(), elapsed);
mem.record_action(rec.clone());
rec
@@ -693,7 +857,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.context_manager
.update_memory(job_id, |mem| {
let rec = mem
.create_action(tool_name, params.clone())
.create_action(tool_name, safe_params.clone())
.fail("Execution timeout", elapsed);
mem.record_action(rec.clone());
rec
@@ -836,8 +1000,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
plan: &ActionPlan,
) -> Result<(), Error> {
for (i, action) in plan.actions.iter().enumerate() {
// Check for stop signal
if let Ok(msg) = rx.try_recv() {
// Check for stop signal and injected user messages
while let Ok(msg) = rx.try_recv() {
match msg {
WorkerMessage::Stop => {
tracing::debug!(
@@ -850,6 +1014,29 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tracing::trace!("Worker for job {} received ping", self.job_id);
}
WorkerMessage::Start => {}
WorkerMessage::UserMessage(content) => {
tracing::info!(
job_id = %self.job_id,
"User message received during plan execution, abandoning plan"
);
reason_ctx.messages.push(ChatMessage::user(&content));
self.log_event(
"message",
serde_json::json!({
"role": "user",
"content": content,
}),
);
self.log_event(
"status",
serde_json::json!({
"message": "Plan interrupted by user message, re-evaluating...",
}),
);
// Return Ok to break out of plan; caller falls through to
// the direct selection loop for LLM re-evaluation.
return Ok(());
}
}
}
@@ -902,14 +1089,18 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
if crate::util::llm_signals_completion(&response) {
self.mark_completed().await?;
} else {
// Job not complete, could re-plan or fall back to direct selection
// Job not complete — return Ok without marking terminal so the
// caller falls through to the direct selection loop for continuation.
tracing::info!(
"Job {} plan completed but work remains, falling back to direct selection",
self.job_id
);
// Continue with standard execution loop by returning (will be picked up by main loop)
self.mark_stuck("Plan completed but job incomplete - needs re-planning")
.await?;
self.log_event(
"status",
serde_json::json!({
"message": "Plan completed but job needs more work, continuing...",
}),
);
}
Ok(())
@@ -940,6 +1131,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "completed",
"success": true,
"message": "Job completed successfully",
}),
@@ -965,6 +1157,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "failed",
"success": false,
"message": format!("Execution failed: {}", reason),
}),
@@ -985,6 +1178,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
self.log_event(
"result",
serde_json::json!({
"status": "stuck",
"success": false,
"message": format!("Job stuck: {}", reason),
}),
@@ -1103,6 +1297,7 @@ mod tests {
hooks: Arc::new(crate::hooks::HookRegistry::new()),
timeout: Duration::from_secs(30),
use_planning: false,
sse_tx: None,
};
Worker::new(job_id, deps)
+66 -5
View File
@@ -15,7 +15,7 @@ use crate::context::ContextManager;
use crate::db::Database;
use crate::extensions::ExtensionManager;
use crate::hooks::HookRegistry;
use crate::llm::{LlmProvider, SessionManager};
use crate::llm::{LlmProvider, RecordingLlm, SessionManager};
use crate::safety::SafetyLayer;
use crate::secrets::SecretsStore;
use crate::skills::SkillRegistry;
@@ -48,6 +48,7 @@ pub struct AppComponents {
pub skill_registry: Option<Arc<std::sync::RwLock<SkillRegistry>>>,
pub skill_catalog: Option<Arc<SkillCatalog>>,
pub cost_guard: Arc<crate::agent::cost_guard::CostGuard>,
pub recording_handle: Option<Arc<RecordingLlm>>,
pub session: Arc<SessionManager>,
pub catalog_entries: Vec<crate::extensions::RegistryEntry>,
pub dev_loaded_tool_names: Vec<String>,
@@ -71,6 +72,9 @@ pub struct AppBuilder {
db: Option<Arc<dyn Database>>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
// Test overrides
llm_override: Option<Arc<dyn LlmProvider>>,
// Backend-specific handles needed by secrets store
#[cfg(feature = "postgres")]
pg_pool: Option<deadpool_postgres::Pool>,
@@ -99,6 +103,7 @@ impl AppBuilder {
log_broadcaster,
db: None,
secrets_store: None,
llm_override: None,
#[cfg(feature = "postgres")]
pg_pool: None,
#[cfg(feature = "libsql")]
@@ -106,11 +111,26 @@ impl AppBuilder {
}
}
/// Inject a pre-created database, skipping `init_database()`.
pub fn with_database(&mut self, db: Arc<dyn Database>) {
self.db = Some(db);
}
/// Inject a pre-created LLM provider, skipping `init_llm()`.
pub fn with_llm(&mut self, llm: Arc<dyn LlmProvider>) {
self.llm_override = Some(llm);
}
/// Phase 1: Initialize database backend.
///
/// Creates the database connection, runs migrations, reloads config
/// from DB, attaches DB to session manager, and cleans up stale jobs.
pub async fn init_database(&mut self) -> Result<(), anyhow::Error> {
if self.db.is_some() {
tracing::debug!("Database already provided, skipping init_database()");
return Ok(());
}
if self.flags.no_db {
tracing::warn!("Running without database connection");
return Ok(());
@@ -297,10 +317,17 @@ impl AppBuilder {
#[allow(clippy::type_complexity)]
pub fn init_llm(
&self,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), anyhow::Error> {
let (llm, cheap_llm) =
) -> Result<
(
Arc<dyn LlmProvider>,
Option<Arc<dyn LlmProvider>>,
Option<Arc<RecordingLlm>>,
),
anyhow::Error,
> {
let (llm, cheap_llm, recording_handle) =
crate::llm::build_provider_chain(&self.config.llm, self.session.clone())?;
Ok((llm, cheap_llm))
Ok((llm, cheap_llm, recording_handle))
}
/// Phase 4: Initialize safety, tools, embeddings, and workspace.
@@ -331,6 +358,10 @@ impl AppBuilder {
};
tools.register_builtin_tools();
if let Some(ref ss) = self.secrets_store {
tools.register_secrets_tools(Arc::clone(ss));
}
// Create embeddings provider using the unified method
let embeddings = self
.config
@@ -649,7 +680,11 @@ impl AppBuilder {
self.init_database().await?;
self.init_secrets().await?;
let (llm, cheap_llm) = self.init_llm()?;
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
(llm, None, None)
} else {
self.init_llm()?
};
let (safety, tools, embeddings, workspace) = self.init_tools(&llm).await?;
// Create hook registry early so runtime extension activation can register hooks.
@@ -665,6 +700,31 @@ impl AppBuilder {
// Seed workspace and backfill embeddings
if let Some(ref ws) = workspace {
// Import workspace files from disk FIRST if WORKSPACE_IMPORT_DIR is set.
// This lets Docker images / deployment scripts ship customized
// workspace templates (e.g., AGENTS.md, TOOLS.md) that override
// the generic seeds. Only imports files that don't already exist
// in the database — never overwrites user edits.
//
// Runs before seed_if_empty() so that custom templates take priority
// over generic seeds. seed_if_empty() then fills any remaining gaps.
if let Ok(import_dir) = std::env::var("WORKSPACE_IMPORT_DIR") {
let import_path = std::path::Path::new(&import_dir);
match ws.import_from_directory(import_path).await {
Ok(count) if count > 0 => {
tracing::info!("Imported {} workspace file(s) from {}", count, import_dir);
}
Ok(_) => {}
Err(e) => {
tracing::warn!(
"Failed to import workspace files from {}: {}",
import_dir,
e
);
}
}
}
match ws.seed_if_empty().await {
Ok(_) => {}
Err(e) => {
@@ -736,6 +796,7 @@ impl AppBuilder {
skill_registry,
skill_catalog,
cost_guard,
recording_handle,
session: self.session,
catalog_entries,
dev_loaded_tool_names,
+174 -1
View File
@@ -117,7 +117,20 @@ pub enum StatusUpdate {
/// Tool execution started.
ToolStarted { name: String },
/// Tool execution completed.
ToolCompleted { name: String, success: bool },
///
/// Use [`StatusUpdate::tool_completed`] to construct this variant — it
/// handles redaction of sensitive parameters and keeps the 9-line pattern
/// in one place.
ToolCompleted {
name: String,
success: bool,
/// Error message when success is false.
error: Option<String>,
/// Tool input parameters (JSON string) for display on failure.
/// Only populated when `success` is `false`. Values listed in the
/// tool's `sensitive_params()` are replaced with `"[REDACTED]"`.
parameters: Option<String>,
},
/// Brief preview of tool execution output.
ToolResult { name: String, preview: String },
/// Streaming text chunk.
@@ -152,6 +165,38 @@ pub enum StatusUpdate {
},
}
impl StatusUpdate {
/// Build a `ToolCompleted` status with redacted parameters.
///
/// On failure, serializes the tool's input parameters as pretty JSON after
/// replacing any keys listed in the tool's `sensitive_params()` with
/// `"[REDACTED]"`. On success, no parameters or error are included.
///
/// Pass the resolved `Tool` reference (if available) so this method can
/// query `sensitive_params()` directly — callers don't need to manage the
/// borrow lifetime of the sensitive slice.
pub fn tool_completed(
name: String,
result: &Result<String, crate::error::Error>,
params: &serde_json::Value,
tool: Option<&dyn crate::tools::Tool>,
) -> Self {
let success = result.is_ok();
let sensitive = tool.map(|t| t.sensitive_params()).unwrap_or(&[]);
Self::ToolCompleted {
name,
success,
error: result.as_ref().err().map(|e| e.to_string()),
parameters: if !success {
let safe = crate::tools::redact_params(params, sensitive);
Some(serde_json::to_string_pretty(&safe).unwrap_or_else(|_| safe.to_string()))
} else {
None
},
}
}
}
/// Trait for message channels.
///
/// Channels receive messages from external sources and convert them to
@@ -223,3 +268,131 @@ pub trait Channel: Send + Sync {
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Stub tool that marks `"value"` as sensitive.
struct SecretTool;
#[async_trait]
impl crate::tools::Tool for SecretTool {
fn name(&self) -> &str {
"secret_save"
}
fn description(&self) -> &str {
"stub"
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({"type": "object", "properties": {}})
}
async fn execute(
&self,
_params: serde_json::Value,
_ctx: &crate::context::JobContext,
) -> Result<crate::tools::ToolOutput, crate::tools::ToolError> {
unreachable!()
}
fn sensitive_params(&self) -> &[&str] {
&["value"]
}
}
#[test]
fn tool_completed_redacts_sensitive_params_on_failure() {
let params = serde_json::json!({"name": "api_key", "value": "sk-secret-123"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "secret_save".into(),
reason: "db error".into(),
}
.into());
let tool = SecretTool;
let status = StatusUpdate::tool_completed(
"secret_save".into(),
&err,
&params,
Some(&tool as &dyn crate::tools::Tool),
);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(!success);
let err_msg = error.as_deref().expect("should have error");
assert!(err_msg.contains("db error"), "error: {}", err_msg);
let param_str = parameters
.as_ref()
.expect("should have parameters on failure");
assert!(
param_str.contains("[REDACTED]"),
"sensitive value should be redacted: {}",
param_str
);
assert!(
!param_str.contains("sk-secret-123"),
"raw secret should not appear: {}",
param_str
);
assert!(
param_str.contains("api_key"),
"non-sensitive params should be preserved: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_params_on_success() {
let params = serde_json::json!({"name": "key", "value": "secret"});
let ok: Result<String, crate::error::Error> = Ok("done".into());
let status = StatusUpdate::tool_completed("secret_save".into(), &ok, &params, None);
if let StatusUpdate::ToolCompleted {
success,
error,
parameters,
..
} = &status
{
assert!(success);
assert!(error.is_none());
assert!(parameters.is_none(), "no params should be sent on success");
} else {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn tool_completed_no_tool_passes_params_unredacted() {
let params = serde_json::json!({"cmd": "ls -la"});
let err: Result<String, crate::error::Error> =
Err(crate::error::ToolError::ExecutionFailed {
name: "shell".into(),
reason: "timeout".into(),
}
.into());
let status = StatusUpdate::tool_completed("shell".into(), &err, &params, None);
if let StatusUpdate::ToolCompleted { parameters, .. } = &status {
let param_str = parameters.as_ref().expect("should have parameters");
assert!(
param_str.contains("ls -la"),
"non-sensitive params should pass through: {}",
param_str
);
} else {
panic!("expected ToolCompleted variant");
}
}
}
+1 -1
View File
@@ -466,7 +466,7 @@ impl Channel for ReplChannel {
StatusUpdate::ToolStarted { name } => {
eprintln!(" \x1b[33m\u{25CB} {name}\x1b[0m");
}
StatusUpdate::ToolCompleted { name, success } => {
StatusUpdate::ToolCompleted { name, success, .. } => {
if success {
eprintln!(" \x1b[32m\u{25CF} {name}\x1b[0m");
} else {
+1 -1
View File
@@ -974,7 +974,7 @@ impl Channel for SignalChannel {
// Send tool completed notification (debug mode only)
if self.is_debug()
&& let StatusUpdate::ToolCompleted { name, success } = &status
&& let StatusUpdate::ToolCompleted { name, success, .. } = &status
&& let Some(target_str) = metadata.get("signal_target").and_then(|v| v.as_str())
{
let (icon, color) = if *success {
+21 -1
View File
@@ -19,12 +19,14 @@ use crate::channels::wasm::schema::ChannelCapabilitiesFile;
use crate::channels::wasm::wrapper::WasmChannel;
use crate::db::SettingsStore;
use crate::pairing::PairingStore;
use crate::secrets::SecretsStore;
/// Loads WASM channels from the filesystem.
pub struct WasmChannelLoader {
runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
settings_store: Option<Arc<dyn SettingsStore>>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
}
impl WasmChannelLoader {
@@ -38,9 +40,16 @@ impl WasmChannelLoader {
runtime,
pairing_store,
settings_store,
secrets_store: None,
}
}
/// Set the secrets store for host-based credential injection in WASM channels.
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
}
/// Load a single WASM channel from a file pair.
///
/// Expects:
@@ -72,6 +81,7 @@ impl WasmChannelLoader {
let cap_bytes = fs::read(cap_path).await?;
let cap_file = ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| WasmChannelError::InvalidCapabilities(e.to_string()))?;
cap_file.validate();
// Debug: log raw capabilities
tracing::debug!(
@@ -127,7 +137,7 @@ impl WasmChannelLoader {
.await?;
// Create the channel
let channel = WasmChannel::new(
let mut channel = WasmChannel::new(
self.runtime.clone(),
prepared,
capabilities,
@@ -135,6 +145,9 @@ impl WasmChannelLoader {
self.pairing_store.clone(),
self.settings_store.clone(),
);
if let Some(ref secrets) = self.secrets_store {
channel = channel.with_secrets_store(Arc::clone(secrets));
}
tracing::info!(
name = name,
@@ -264,6 +277,13 @@ impl LoadedChannel {
.and_then(|f| f.signature_key_secret_name().map(|s| s.to_string()))
}
/// Get the HMAC-SHA256 signing secret name from capabilities.
pub fn hmac_secret_name(&self) -> Option<String> {
self.capabilities_file
.as_ref()
.and_then(|f| f.hmac_secret_name().map(|s| s.to_string()))
}
/// Get the webhook secret name from capabilities.
pub fn webhook_secret_name(&self) -> String {
self.capabilities_file
+337 -1
View File
@@ -44,6 +44,8 @@ pub struct WasmChannelRouter {
secret_headers: RwLock<HashMap<String, String>>,
/// Ed25519 public keys for signature verification by channel name (hex-encoded).
signature_keys: RwLock<HashMap<String, String>>,
/// HMAC-SHA256 signing secrets for signature verification by channel name (Slack-style).
hmac_secrets: RwLock<HashMap<String, String>>,
}
impl WasmChannelRouter {
@@ -55,6 +57,7 @@ impl WasmChannelRouter {
secrets: RwLock::new(HashMap::new()),
secret_headers: RwLock::new(HashMap::new()),
signature_keys: RwLock::new(HashMap::new()),
hmac_secrets: RwLock::new(HashMap::new()),
}
}
@@ -134,6 +137,7 @@ impl WasmChannelRouter {
self.secrets.write().await.remove(channel_name);
self.secret_headers.write().await.remove(channel_name);
self.signature_keys.write().await.remove(channel_name);
self.hmac_secrets.write().await.remove(channel_name);
// Remove all paths for this channel
self.path_to_channel
@@ -208,6 +212,24 @@ impl WasmChannelRouter {
pub async fn get_signature_key(&self, channel_name: &str) -> Option<String> {
self.signature_keys.read().await.get(channel_name).cloned()
}
/// Register an HMAC-SHA256 signing secret for signature verification.
///
/// Channels with a registered secret will have Slack-style HMAC-SHA256
/// signature validation performed before forwarding to WASM.
pub async fn register_hmac_secret(&self, channel_name: &str, secret: &str) {
self.hmac_secrets
.write()
.await
.insert(channel_name.to_string(), secret.to_string());
}
/// Get the HMAC signing secret for a channel.
///
/// Returns `None` if no secret is registered (no HMAC check needed).
pub async fn get_hmac_secret(&self, channel_name: &str) -> Option<String> {
self.hmac_secrets.read().await.get(channel_name).cloned()
}
}
impl Default for WasmChannelRouter {
@@ -427,6 +449,57 @@ async fn webhook_handler(
}
}
// HMAC-SHA256 signature verification (Slack-style)
if let Some(hmac_secret) = state.router.get_hmac_secret(channel_name).await {
let timestamp = headers
.get("x-slack-request-timestamp")
.and_then(|v| v.to_str().ok());
let sig_header = headers
.get("x-slack-signature")
.and_then(|v| v.to_str().ok());
match (timestamp, sig_header) {
(Some(ts), Some(sig)) => {
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_secs() as i64;
if !crate::channels::wasm::signature::verify_slack_signature(
&hmac_secret,
ts,
&body,
sig,
now_secs,
) {
tracing::warn!(
channel = %channel_name,
"HMAC-SHA256 signature verification failed"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Invalid Slack signature"
})),
);
}
tracing::debug!(channel = %channel_name, "HMAC-SHA256 signature verified");
}
_ => {
tracing::warn!(
channel = %channel_name,
"Slack signature headers missing but secret is registered"
);
return (
StatusCode::UNAUTHORIZED,
Json(serde_json::json!({
"error": "Missing Slack signature headers"
})),
);
}
}
}
// Convert headers to HashMap
let headers_map: HashMap<String, String> = headers
.iter()
@@ -731,7 +804,59 @@ mod tests {
assert_eq!(router.get_secret_header("slack").await, "X-Webhook-Secret");
}
// ── Category 3: Router Signature Key Management ─────────────────────
// ── Category 3: Router HMAC Secret Management ───────────────────────
#[tokio::test]
async fn test_register_and_get_hmac_secret() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
let hmac_secret = "my-slack-signing-secret";
router.register_hmac_secret("slack", hmac_secret).await;
let retrieved = router.get_hmac_secret("slack").await;
assert_eq!(retrieved, Some(hmac_secret.to_string()));
}
#[tokio::test]
async fn test_no_hmac_secret_returns_none() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
router.register(channel, vec![], None, None).await;
// Slack has no HMAC secret registered
let secret = router.get_hmac_secret("slack").await;
assert!(secret.is_none());
}
#[tokio::test]
async fn test_unregister_removes_hmac_secret() {
let router = WasmChannelRouter::new();
let channel = create_test_channel("slack");
let endpoints = vec![RegisteredEndpoint {
channel_name: "slack".to_string(),
path: "/webhook/slack".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
router.register(channel, endpoints, None, None).await;
router.register_hmac_secret("slack", "signing-secret").await;
// Secret should exist
assert!(router.get_hmac_secret("slack").await.is_some());
// Unregister
router.unregister("slack").await;
// Secret should be gone
assert!(router.get_hmac_secret("slack").await.is_none());
}
// ── Category 4: Router Signature Key Management ─────────────────────
#[tokio::test]
async fn test_register_and_get_signature_key() {
@@ -1163,4 +1288,215 @@ mod tests {
"Valid secret + valid signature should not return 401"
);
}
// ── HMAC-SHA256 Webhook Signature Tests ────────────────────────────
/// Helper to create a router with a registered channel at /webhook/slack.
async fn setup_slack_router() -> (Arc<WasmChannelRouter>, AxumRouter) {
let wasm_router = Arc::new(WasmChannelRouter::new());
let channel = create_test_channel("slack");
let endpoints = vec![RegisteredEndpoint {
channel_name: "slack".to_string(),
path: "/webhook/slack".to_string(),
methods: vec!["POST".to_string()],
require_secret: false,
}];
wasm_router.register(channel, endpoints, None, None).await;
let app = create_wasm_channel_router(wasm_router.clone(), None);
(wasm_router, app)
}
/// Helper: compute expected Slack signature for testing.
fn slack_signature(signing_secret: &str, timestamp: &str, body: &[u8]) -> String {
use hmac::{Hmac, Mac};
use sha2::Sha256;
let mut basestring = Vec::new();
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
let mut mac = Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()).unwrap();
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
format!("v0={}", hex::encode(computed))
}
#[tokio::test]
async fn test_webhook_hmac_rejects_missing_sig_headers() {
let (wasm_router, app) = setup_slack_router().await;
wasm_router
.register_hmac_secret("slack", "my-signing-secret")
.await;
// Send request without HMAC signature headers
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Missing HMAC signature headers should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_rejects_invalid_signature() {
let (wasm_router, app) = setup_slack_router().await;
wasm_router
.register_hmac_secret("slack", "my-signing-secret")
.await;
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", "1234567890")
.header("x-slack-signature", "v0=deadbeefdeadbeef")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Invalid HMAC signature should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_accepts_valid_signature() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let now_secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_secs();
let timestamp = now_secs.to_string();
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = slack_signature(signing_secret, &timestamp, body);
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", &timestamp)
.header("x-slack-signature", &signature)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 — signature is valid (may be 500 since no WASM module)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Valid HMAC signature should not return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_skips_check_for_no_secret() {
let (_wasm_router, app) = setup_slack_router().await;
// No HMAC secret registered — should not require signature
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.body(Body::from("token=xyzz0WbapA4vBCDEFasx0q6G"))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
// Should NOT be 401 (may be 500 since no WASM module, but not auth failure)
assert_ne!(
resp.status(),
StatusCode::UNAUTHORIZED,
"No HMAC secret registered — should skip check"
);
}
#[tokio::test]
async fn test_webhook_hmac_uses_correct_body() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let timestamp = "1234567890";
let body_a = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let body_b = b"token=MODIFIED";
// Sign body A
let signature = slack_signature(signing_secret, timestamp, body_a);
// But send body B
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", timestamp)
.header("x-slack-signature", &signature)
.body(Body::from(&body_b[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature for different body should return 401"
);
}
#[tokio::test]
async fn test_webhook_hmac_uses_correct_timestamp() {
let (wasm_router, app) = setup_slack_router().await;
let signing_secret = "my-signing-secret";
wasm_router
.register_hmac_secret("slack", signing_secret)
.await;
let timestamp_a = "1234567890";
let timestamp_b = "9999999999";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
// Sign with timestamp A
let signature = slack_signature(signing_secret, timestamp_a, body);
// But send timestamp B in the header
let req = Request::builder()
.method("POST")
.uri("/webhook/slack")
.header("content-type", "application/json")
.header("x-slack-request-timestamp", timestamp_b)
.header("x-slack-signature", &signature)
.body(Body::from(&body[..]))
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(
resp.status(),
StatusCode::UNAUTHORIZED,
"Signature with mismatched timestamp should return 401"
);
}
}
+110
View File
@@ -90,6 +90,37 @@ impl ChannelCapabilitiesFile {
serde_json::from_slice(bytes)
}
/// Validate the capabilities file and emit warnings for common misconfigurations.
///
/// Called once at load time to catch issues early. Warnings are emitted via
/// `tracing::warn` so they show up in startup logs without blocking loading.
pub fn validate(&self) {
const MIN_PROMPT_LENGTH: usize = 30;
// Check for short prompts in required_secrets
for secret in &self.setup.required_secrets {
if secret.prompt.len() < MIN_PROMPT_LENGTH {
tracing::warn!(
channel = self.name,
secret = secret.name,
prompt = secret.prompt,
"setup.required_secrets prompt is shorter than {} chars — \
consider a more descriptive prompt that tells the user where to find this value",
MIN_PROMPT_LENGTH
);
}
}
// Has required_secrets but no setup_url
if !self.setup.required_secrets.is_empty() && self.setup.setup_url.is_none() {
tracing::warn!(
channel = self.name,
"setup.required_secrets defined but no setup.setup_url — \
user has no link to obtain credentials"
);
}
}
/// Convert to runtime ChannelCapabilities.
pub fn to_capabilities(&self) -> ChannelCapabilities {
self.capabilities.to_channel_capabilities(&self.name)
@@ -123,6 +154,18 @@ impl ChannelCapabilitiesFile {
.and_then(|w| w.signature_key_secret_name.as_deref())
}
/// Get the HMAC-SHA256 signing secret name for this channel.
///
/// Returns the secret name declared in `webhook.hmac_secret_name`,
/// used to look up the HMAC signing secret in the secrets store (Slack-style).
pub fn hmac_secret_name(&self) -> Option<&str> {
self.capabilities
.channel
.as_ref()
.and_then(|c| c.webhook.as_ref())
.and_then(|w| w.hmac_secret_name.as_deref())
}
/// Get the webhook secret name for this channel.
///
/// Returns the configured secret name or defaults to "{channel_name}_webhook_secret".
@@ -247,6 +290,10 @@ pub struct WebhookSchema {
/// for signature verification (e.g., Discord interaction verification).
#[serde(default)]
pub signature_key_secret_name: Option<String>,
/// Secret name in secrets store for HMAC-SHA256 signing (Slack-style).
#[serde(default)]
pub hmac_secret_name: Option<String>,
}
/// Setup configuration schema.
@@ -262,6 +309,10 @@ pub struct SetupSchema {
/// Placeholders like {secret_name} are replaced with actual values.
#[serde(default)]
pub validation_endpoint: Option<String>,
/// User-facing URL where they can create/manage credentials.
#[serde(default)]
pub setup_url: Option<String>,
}
/// Configuration for a secret required during setup.
@@ -605,6 +656,65 @@ mod tests {
// ── Category 5: Discord Capabilities Setup & Configuration ──────────
#[test]
fn test_validate_channel_short_prompt() {
// prompt < 30 chars — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{ "name": "bot_token", "prompt": "Bot token" }
],
"setup_url": "https://example.com"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for short prompt
file.validate();
}
#[test]
fn test_validate_channel_missing_setup_url() {
// required_secrets without setup_url — should not panic
let json = r#"{
"name": "test-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from the developer portal settings"
}
]
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic; warning emitted for missing setup_url
file.validate();
}
#[test]
fn test_validate_clean_channel() {
// Well-configured channel — should not panic or warn
let json = r#"{
"name": "good-channel",
"setup": {
"required_secrets": [
{
"name": "bot_token",
"prompt": "Enter your bot token from https://example.com/bot-settings"
}
],
"setup_url": "https://example.com/bot-settings"
}
}"#;
let file = ChannelCapabilitiesFile::from_json(json).unwrap();
// Should not panic and emits no warnings
file.validate();
}
#[test]
fn test_discord_capabilities_has_public_key_secret() {
let json = include_str!("../../../channels-src/discord/discord.capabilities.json");
+319 -3
View File
@@ -1,9 +1,11 @@
//! Discord Ed25519 signature verification.
//! Webhook signature verification (Discord Ed25519 and Slack HMAC-SHA256).
//!
//! Validates `X-Signature-Ed25519` and `X-Signature-Timestamp` headers
//! on incoming Discord interaction webhooks, per Discord's security requirements.
//! Validates request signatures for incoming webhooks:
//! - Discord: `X-Signature-Ed25519` and `X-Signature-Timestamp` headers
//! - Slack: `X-Slack-Signature` and `X-Slack-Request-Timestamp` headers
//!
//! See: <https://discord.com/developers/docs/interactions/overview#validating-security-request-headers>
//! See: <https://api.slack.com/authentication/verifying-requests-from-slack>
/// Verify a Discord interaction signature.
///
@@ -50,6 +52,60 @@ pub fn verify_discord_signature(
verifying_key.verify_strict(&message, &signature).is_ok()
}
/// Verify a Slack webhook signature using HMAC-SHA256.
///
/// Slack signs each webhook request with HMAC-SHA256 using:
/// - basestring = `"v0:" + timestamp + ":" + body`
/// - signature = hex-encoded HMAC-SHA256(signing_secret, basestring)
/// - header = `"v0=" + signature` (in `X-Slack-Signature` header)
///
/// Includes staleness check: rejects requests with timestamps older than 5 minutes.
/// Returns `true` if the signature is valid, `false` on any error
/// (bad timing, mismatched signature, invalid format, etc.).
pub fn verify_slack_signature(
signing_secret: &str,
timestamp: &str,
body: &[u8],
signature_header: &str,
now_secs: i64,
) -> bool {
use hmac::{Hmac, Mac};
use sha2::Sha256;
// 1. Parse and check staleness (5-minute window)
let ts: i64 = match timestamp.parse() {
Ok(v) => v,
Err(_) => return false,
};
if (now_secs - ts).abs() > 300 {
return false;
}
// 2. Build the basestring: "v0:{timestamp}:{body}"
let mut basestring = Vec::with_capacity(3 + timestamp.len() + 1 + body.len());
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
// 3. Compute HMAC-SHA256
let mut mac = match Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()) {
Ok(m) => m,
Err(_) => return false,
};
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
let computed_hex = hex::encode(computed);
let expected = format!("v0={}", computed_hex);
// 4. Constant-time compare (avoids timing side-channels)
use subtle::ConstantTimeEq;
expected
.as_bytes()
.ct_eq(signature_header.as_bytes())
.into()
}
#[cfg(test)]
mod tests {
use super::*;
@@ -338,4 +394,264 @@ mod tests {
"Negative timestamp should be rejected"
);
}
// ── Category: HMAC-SHA256 Signature Verification (Slack) ────────────
/// Helper: compute expected Slack signature for a given secret, timestamp, and body.
fn sign_slack_message(signing_secret: &str, timestamp: &str, body: &[u8]) -> String {
use hmac::{Hmac, Mac};
use sha2::Sha256;
let mut basestring = Vec::new();
basestring.extend_from_slice(b"v0:");
basestring.extend_from_slice(timestamp.as_bytes());
basestring.push(b':');
basestring.extend_from_slice(body);
let mut mac = Hmac::<Sha256>::new_from_slice(signing_secret.as_bytes()).unwrap();
mac.update(&basestring);
let computed = mac.finalize().into_bytes();
format!("v0={}", hex::encode(computed))
}
const SLACK_TEST_TS: i64 = 1234567890;
#[test]
fn test_slack_valid_signature_succeeds() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS
));
}
#[test]
fn test_slack_tampered_body_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let original_body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let tampered_body = b"token=MODIFIED&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, original_body);
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
tampered_body,
&signature,
SLACK_TEST_TS
),
"Signature for different body should fail"
);
}
#[test]
fn test_slack_tampered_timestamp_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(
!verify_slack_signature(
signing_secret,
"9999999999", // Different timestamp in signature
body,
&signature,
SLACK_TEST_TS
),
"Signature with wrong timestamp should fail"
);
}
#[test]
fn test_slack_tampered_signature_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G&team_id=T1DC2JH3J";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Flip a byte in the signature hex (change first char after "v0=")
let chars: Vec<char> = signature.chars().collect();
let mut new_chars = chars.clone();
if chars.len() > 3 {
new_chars[3] = if chars[3] == 'a' { 'b' } else { 'a' };
}
let modified_sig: String = new_chars.iter().collect();
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&modified_sig,
SLACK_TEST_TS
),
"Tampered signature should fail"
);
}
#[test]
fn test_slack_stale_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// now_secs is 400 seconds after timestamp — too stale
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 400
),
"Stale timestamp (400s old) should be rejected"
);
}
#[test]
fn test_slack_future_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// now_secs is 400 seconds before timestamp — future
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS - 400
),
"Future timestamp (400s ahead) should be rejected"
);
}
#[test]
fn test_slack_boundary_300s_accepted() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Exactly 300 seconds difference — should be accepted
assert!(
verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 300
),
"Timestamp exactly 300s old should be accepted"
);
}
#[test]
fn test_slack_boundary_301s_rejected() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// 301 seconds difference — should be rejected
assert!(
!verify_slack_signature(
signing_secret,
timestamp,
body,
&signature,
SLACK_TEST_TS + 301
),
"Timestamp 301s old should be rejected"
);
}
#[test]
fn test_slack_non_numeric_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "not-a-number", body, "v0=abc123", 0),
"Non-numeric timestamp should be rejected"
);
}
#[test]
fn test_slack_missing_v0_prefix_fails() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(signing_secret, timestamp, body);
// Remove the "v0=" prefix
let bad_sig = signature.strip_prefix("v0=").unwrap_or(&signature);
assert!(
!verify_slack_signature(signing_secret, timestamp, body, bad_sig, SLACK_TEST_TS),
"Missing v0= prefix should fail"
);
}
#[test]
fn test_slack_wrong_signing_secret_fails() {
let secret_a = "secret-a";
let secret_b = "secret-b";
let timestamp = "1234567890";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
let signature = sign_slack_message(secret_a, timestamp, body);
// Try to verify with a different secret
assert!(
!verify_slack_signature(secret_b, timestamp, body, &signature, SLACK_TEST_TS),
"Signature from different secret should fail"
);
}
#[test]
fn test_slack_empty_body_valid() {
let signing_secret = "my-signing-secret";
let timestamp = "1234567890";
let body = b"";
let signature = sign_slack_message(signing_secret, timestamp, body);
assert!(
verify_slack_signature(signing_secret, timestamp, body, &signature, SLACK_TEST_TS),
"Empty body with valid signature should succeed"
);
}
#[test]
fn test_slack_negative_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "-1", body, "v0=abc123", 0),
"Negative timestamp should be rejected"
);
}
#[test]
fn test_slack_empty_timestamp_rejected() {
let signing_secret = "my-signing-secret";
let body = b"token=xyzz0WbapA4vBCDEFasx0q6G";
assert!(
!verify_slack_signature(signing_secret, "", body, "v0=abc123", 0),
"Empty timestamp should be rejected"
);
}
}
+5 -1
View File
@@ -2479,7 +2479,7 @@ fn status_to_wit(status: &StatusUpdate, metadata: &serde_json::Value) -> wit_cha
message: format!("Tool started: {}", name),
metadata_json,
},
StatusUpdate::ToolCompleted { name, success } => wit_channel::StatusUpdate {
StatusUpdate::ToolCompleted { name, success, .. } => wit_channel::StatusUpdate {
status: wit_channel::StatusType::ToolCompleted,
message: format!(
"Tool completed: {} ({})",
@@ -3387,6 +3387,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: true,
error: None,
parameters: None,
},
&metadata,
);
@@ -3407,6 +3409,8 @@ mod tests {
&crate::channels::StatusUpdate::ToolCompleted {
name: "http_request".to_string(),
success: false,
error: Some("connection refused".to_string()),
parameters: None,
},
&metadata,
);
+148 -30
View File
@@ -2,7 +2,7 @@
use axum::{
extract::{Request, State},
http::{HeaderMap, StatusCode},
http::{HeaderMap, Method, StatusCode},
middleware::Next,
response::{IntoResponse, Response},
};
@@ -14,10 +14,44 @@ pub struct AuthState {
pub token: String,
}
/// Whether query-string token auth is allowed for this request.
///
/// Only GET requests to streaming endpoints may use `?token=xxx`. This
/// minimizes token-in-URL exposure on state-changing routes, where the token
/// would leak via server logs, Referer headers, and browser history.
///
/// Allowed endpoints:
/// - SSE: `/api/chat/events`, `/api/logs/events` (EventSource can't set headers)
/// - WebSocket: `/api/chat/ws` (WS upgrade can't set custom headers)
///
/// If you add a new SSE or WebSocket endpoint, add its path here.
fn allows_query_token_auth(request: &Request) -> bool {
if request.method() != Method::GET {
return false;
}
matches!(
request.uri().path(),
"/api/chat/events" | "/api/logs/events" | "/api/chat/ws"
)
}
/// Extract the `token` query parameter value, URL-decoded.
fn query_token(request: &Request) -> Option<String> {
let query = request.uri().query()?;
url::form_urlencoded::parse(query.as_bytes()).find_map(|(k, v)| {
if k == "token" {
Some(v.into_owned())
} else {
None
}
})
}
/// Auth middleware that validates bearer token from header or query param.
///
/// SSE connections can't set headers from `EventSource`, so we also accept
/// `?token=xxx` as a query parameter.
/// `?token=xxx` as a query parameter, but only on SSE endpoints.
pub async fn auth_middleware(
State(auth): State<AuthState>,
headers: HeaderMap,
@@ -35,15 +69,12 @@ pub async fn auth_middleware(
return next.run(request).await;
}
// Fall back to query parameter for SSE EventSource (constant-time comparison)
if let Some(query) = request.uri().query() {
for pair in query.split('&') {
if let Some(token) = pair.strip_prefix("token=")
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
}
// Fall back to query parameter, but only for SSE endpoints (constant-time comparison).
if allows_query_token_auth(&request)
&& let Some(token) = query_token(&request)
&& bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await;
}
(StatusCode::UNAUTHORIZED, "Invalid or missing auth token").into_response()
@@ -62,24 +93,28 @@ mod tests {
assert_eq!(cloned.token, "test-token");
}
// === QA Plan - Web gateway auth tests ===
use axum::Router;
use axum::body::Body;
use axum::middleware;
use axum::routing::get;
use axum::routing::{get, post};
use tower::ServiceExt;
async fn dummy_handler() -> &'static str {
"ok"
}
/// Router with streaming endpoints (query auth allowed) and regular
/// endpoints (query auth rejected).
fn test_app(token: &str) -> Router {
let state = AuthState {
token: token.to_string(),
};
Router::new()
.route("/test", get(dummy_handler))
.route("/api/chat/events", get(dummy_handler))
.route("/api/logs/events", get(dummy_handler))
.route("/api/chat/ws", get(dummy_handler))
.route("/api/chat/history", get(dummy_handler))
.route("/api/chat/send", post(dummy_handler))
.layer(middleware::from_fn_with_state(state, auth_middleware))
}
@@ -87,7 +122,7 @@ mod tests {
async fn test_valid_bearer_token_passes() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
@@ -99,7 +134,7 @@ mod tests {
async fn test_invalid_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer wrong-token")
.body(Body::empty())
.unwrap();
@@ -108,10 +143,10 @@ mod tests {
}
#[tokio::test]
async fn test_missing_auth_header_falls_through_to_query() {
async fn test_query_token_allowed_for_chat_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=secret-token")
.uri("/api/chat/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
@@ -119,10 +154,80 @@ mod tests {
}
#[tokio::test]
async fn test_query_param_invalid_token_rejected() {
async fn test_query_token_allowed_for_logs_events() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test?token=wrong-token")
.uri("/api/logs/events?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_allowed_for_ws_upgrade() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/ws?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded() {
// Token with characters that get percent-encoded in URLs.
let raw_token = "tok+en/with spaces";
let app = test_app(raw_token);
let req = Request::builder()
.uri("/api/chat/events?token=tok%2Ben%2Fwith%20spaces")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_query_token_url_encoded_mismatch() {
let app = test_app("real-token");
// Encoded value decodes to "wrong-token", not "real-token".
let req = Request::builder()
.uri("/api/chat/events?token=wrong%2Dtoken")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_non_sse_get() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/history?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_rejected_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.method(Method::POST)
.uri("/api/chat/send?token=secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_query_token_invalid_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events?token=wrong-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
@@ -132,17 +237,32 @@ mod tests {
#[tokio::test]
async fn test_no_auth_at_all_rejected() {
let app = test_app("secret-token");
let req = Request::builder().uri("/test").body(Body::empty()).unwrap();
let req = Request::builder()
.uri("/api/chat/events")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
// RFC 6750 Section 2.1: auth-scheme comparison must be case-insensitive.
async fn test_bearer_header_works_for_post() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.method(Method::POST)
.uri("/api/chat/send")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
let resp = app.oneshot(req).await.unwrap();
assert_eq!(resp.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_bearer_prefix_case_insensitive() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/api/chat/events")
.header("Authorization", "bearer secret-token")
.body(Body::empty())
.unwrap();
@@ -154,7 +274,7 @@ mod tests {
async fn test_bearer_prefix_mixed_case() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "BEARER secret-token")
.body(Body::empty())
.unwrap();
@@ -166,7 +286,7 @@ mod tests {
async fn test_empty_bearer_token_rejected() {
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer ")
.body(Body::empty())
.unwrap();
@@ -176,11 +296,9 @@ mod tests {
#[tokio::test]
async fn test_token_with_whitespace_rejected() {
// Extra space after "Bearer " means the token value starts with a space,
// which should not match the expected token.
let app = test_app("secret-token");
let req = Request::builder()
.uri("/test")
.uri("/api/chat/events")
.header("Authorization", "Bearer secret-token")
.body(Body::empty())
.unwrap();
+6 -5
View File
@@ -142,7 +142,7 @@ pub async fn chat_auth_token_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if result.status == "authenticated" {
if result.is_authenticated() {
// Auto-activate so tools are available immediately
let msg = match ext_mgr.activate(&req.extension_name).await {
Ok(r) => format!(
@@ -170,13 +170,14 @@ pub async fn chat_auth_token_handler(
// Re-emit auth_required for retry
state.sse.broadcast(SseEvent::AuthRequired {
extension_name: req.extension_name.clone(),
instructions: result.instructions.clone(),
auth_url: result.auth_url.clone(),
setup_url: result.setup_url.clone(),
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
Ok(Json(ActionResponse::fail(
result
.instructions
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token".to_string()),
)))
}
+13 -8
View File
@@ -33,7 +33,7 @@ pub async fn extensions_list_handler(
"failed".to_string()
} else if !ext.authenticated {
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
} else if ext.active {
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
@@ -59,6 +59,7 @@ pub async fn extensions_list_handler(
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
has_auth: ext.has_auth,
activation_status,
activation_error: ext.activation_error,
}
@@ -123,7 +124,11 @@ pub async fn extensions_activate_handler(
))?;
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
// Activation just loads the WASM module. Auth (OAuth/manual) is
// triggered separately via save_setup_secrets or the auth endpoint.
Ok(Json(ActionResponse::ok(result.message)))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
@@ -136,7 +141,7 @@ pub async fn extensions_activate_handler(
// Activation failed due to auth; try authenticating first.
match ext_mgr.auth(&name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded, retry activation.
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
@@ -147,13 +152,13 @@ pub async fn extensions_activate_handler(
// Auth in progress (OAuth URL or awaiting manual token).
let mut resp = ActionResponse::fail(
auth_result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| format!("'{}' requires authentication.", name)),
);
resp.auth_url = auth_result.auth_url;
resp.awaiting_token = Some(auth_result.awaiting_token);
resp.instructions = auth_result.instructions;
resp.auth_url = auth_result.auth_url().map(String::from);
resp.awaiting_token = Some(auth_result.is_awaiting_token());
resp.instructions = auth_result.instructions().map(String::from);
Ok(Json(resp))
}
Err(auth_err) => Ok(Json(ActionResponse::fail(format!(
+202 -94
View File
@@ -181,6 +181,9 @@ pub async fn jobs_detail_handler(
});
}
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
let is_claude_code = mode.as_deref() == Some("claude_code");
return Ok(Json(JobDetailResponse {
id: job.id,
title: job.task.clone(),
@@ -193,11 +196,11 @@ pub async fn jobs_detail_handler(
elapsed_secs,
project_dir: Some(job.project_dir.clone()),
browse_url: Some(format!("/projects/{}/", browse_id)),
job_mode: {
let mode = store.get_sandbox_job_mode(job.id).await.ok().flatten();
mode.filter(|m| m != "worker")
},
job_mode: mode.filter(|m| m != "worker"),
transitions,
can_restart: state.job_manager.is_some(),
can_prompt: is_claude_code && state.prompt_queue.is_some(),
job_kind: Some("sandbox".to_string()),
}));
}
@@ -208,6 +211,12 @@ pub async fn jobs_detail_handler(
(end - start).num_seconds().max(0) as u64
});
// Only show prompt bar for jobs that have a running worker (Pending/InProgress).
// Stuck jobs have no active worker loop, so messages would be silently dropped.
let is_promptable = matches!(
ctx.state,
crate::context::JobState::Pending | crate::context::JobState::InProgress
);
return Ok(Json(JobDetailResponse {
id: ctx.job_id,
title: ctx.title.clone(),
@@ -222,6 +231,9 @@ pub async fn jobs_detail_handler(
browse_url: None,
job_mode: None,
transitions: Vec::new(),
can_restart: state.scheduler.is_some(),
can_prompt: is_promptable && state.scheduler.is_some(),
job_kind: Some("agent".to_string()),
}));
}
@@ -295,108 +307,164 @@ pub async fn jobs_restart_handler(
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
let old_job_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
let old_job = store
.get_sandbox_job(old_job_id)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Job not found".to_string()))?;
// Try sandbox job restart first.
if let Ok(Some(old_job)) = store.get_sandbox_job(old_job_id).await {
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
}
if old_job.status != "interrupted" && old_job.status != "failed" {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.status),
));
let jm = state.job_manager.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Sandbox not enabled".to_string(),
))?;
// Enrich the task with failure context.
let task = if let Some(ref reason) = old_job.failure_reason {
format!(
"Previous attempt failed: {}. Retry: {}",
reason, old_job.task
)
} else {
old_job.task.clone()
};
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => {
crate::orchestrator::job_manager::JobMode::ClaudeCode
}
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
)
})?;
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
// Create a new job with the same task and project_dir.
let new_job_id = Uuid::new_v4();
let now = chrono::Utc::now();
// Try agent job restart: dispatch a new job via the scheduler.
if let Ok(Some(old_job)) = store.get_job(old_job_id).await {
if old_job.state.is_active() {
return Err((
StatusCode::CONFLICT,
format!("Cannot restart job in state '{}'", old_job.state),
));
}
let record = crate::history::SandboxJobRecord {
id: new_job_id,
task: old_job.task.clone(),
status: "creating".to_string(),
user_id: old_job.user_id.clone(),
project_dir: old_job.project_dir.clone(),
success: None,
failure_reason: None,
created_at: now,
started_at: None,
completed_at: None,
credential_grants_json: old_job.credential_grants_json.clone(),
};
store
.save_sandbox_job(&record)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let slot = state.scheduler.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Scheduler not available".to_string(),
))?;
let scheduler_guard = slot.read().await;
let scheduler = scheduler_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Agent not started yet".to_string(),
))?;
// Look up the original job's mode so the restart uses the same mode.
let mode = match store.get_sandbox_job_mode(old_job_id).await {
Ok(Some(m)) if m == "claude_code" => crate::orchestrator::job_manager::JobMode::ClaudeCode,
_ => crate::orchestrator::job_manager::JobMode::Worker,
};
// Look up failure reason (O(1) point lookup).
let failure_reason = store
.get_agent_job_failure_reason(old_job_id)
.await
.ok()
.flatten()
.unwrap_or_default();
// Restore credential grants from the original job so the restarted container
// has access to the same secrets.
let credential_grants: Vec<crate::orchestrator::auth::CredentialGrant> =
serde_json::from_str(&old_job.credential_grants_json).unwrap_or_else(|e| {
tracing::warn!(
job_id = %old_job.id,
"Failed to deserialize credential grants from stored job: {}. \
Restarted job will have no credentials.",
e
);
vec![]
});
let project_dir = std::path::PathBuf::from(&old_job.project_dir);
let _token = jm
.create_job(
new_job_id,
&old_job.task,
Some(project_dir),
mode,
credential_grants,
)
.await
.map_err(|e| {
(
StatusCode::INTERNAL_SERVER_ERROR,
format!("Failed to create container: {}", e),
let title = if !failure_reason.is_empty() {
format!(
"Previous attempt failed: {}. Retry: {}",
failure_reason, old_job.title
)
})?;
} else {
old_job.title.clone()
};
store
.update_sandbox_job_status(new_job_id, "running", None, None, Some(now), None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let new_job_id = scheduler
.dispatch_job(&old_job.user_id, &title, &old_job.description, None)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})))
return Ok(Json(serde_json::json!({
"status": "restarted",
"old_job_id": old_job_id,
"new_job_id": new_job_id,
})));
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
}
/// Submit a follow-up prompt to a running Claude Code sandbox job.
/// Submit a follow-up prompt to a running job.
///
/// Routes to the appropriate backend:
/// - Claude Code sandbox jobs → prompt queue (polled by the bridge)
/// - Agent (non-sandbox) jobs → WorkerMessage injection via scheduler
/// - Worker-mode sandbox jobs → not supported (no mechanism to inject)
pub async fn jobs_prompt_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
Json(body): Json<serde_json::Value>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let job_id: uuid::Uuid = id
.parse()
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
@@ -412,17 +480,57 @@ pub async fn jobs_prompt_handler(
let done = body.get("done").and_then(|v| v.as_bool()).unwrap_or(false);
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
// Try sandbox job path: check if we have a sandbox record for this ID.
if let Some(ref s) = state.store
&& let Ok(Some(_)) = s.get_sandbox_job(job_id).await
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
// It's a sandbox job. Check if Claude Code mode.
let mode = s.get_sandbox_job_mode(job_id).await.ok().flatten();
if mode.as_deref() == Some("claude_code") {
let prompt_queue = state.prompt_queue.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Claude Code not configured".to_string(),
))?;
let prompt = crate::orchestrator::api::PendingPrompt { content, done };
{
let mut queue = prompt_queue.lock().await;
queue.entry(job_id).or_default().push_back(prompt);
}
return Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})));
} else {
return Err((
StatusCode::NOT_IMPLEMENTED,
"Follow-up prompts are not supported for worker-mode sandbox jobs".to_string(),
));
}
}
Ok(Json(serde_json::json!({
"status": "queued",
"job_id": job_id.to_string(),
})))
// Try agent job path: send via scheduler.
let slot = state.scheduler.as_ref().ok_or((
StatusCode::NOT_IMPLEMENTED,
"Agent job prompts require the scheduler to be configured".to_string(),
))?;
let scheduler_guard = slot.read().await;
if let Some(ref scheduler) = *scheduler_guard
&& scheduler.is_running(job_id).await
{
scheduler
.send_message(job_id, content)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
return Ok(Json(serde_json::json!({
"status": "sent",
"job_id": job_id.to_string(),
})));
}
Err((
StatusCode::NOT_FOUND,
"Job not found or not running".to_string(),
))
}
/// Load persisted job events for a job (for history replay on page open).
+3 -3
View File
@@ -159,10 +159,10 @@ pub async fn memory_search_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
let hits: Vec<SearchHit> = results
.iter()
.into_iter()
.map(|r| SearchHit {
path: r.document_id.to_string(),
content: r.content.clone(),
path: r.document_path,
content: r.content,
score: r.score as f64,
})
.collect();
+10 -1
View File
@@ -147,6 +147,10 @@ pub async fn routines_trigger_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
if routine.user_id != state.user_id {
return Err((StatusCode::FORBIDDEN, "Access denied".to_string()));
}
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
@@ -156,7 +160,12 @@ pub async fn routines_trigger_handler(
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let msg = IncomingMessage::new("gateway", &state.user_id, content);
let thread_id = format!(
"routine-{}-{}",
routine_id,
chrono::Utc::now().timestamp_millis()
);
let msg = IncomingMessage::new("gateway", &state.user_id, content).with_thread(thread_id);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
+8 -1
View File
@@ -148,7 +148,14 @@ pub async fn skills_install_handler(
.await
.map_err(|e| (StatusCode::BAD_REQUEST, e.to_string()))?
} else if let Some(ref catalog) = state.skill_catalog {
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), &req.name);
// Prefer slug (e.g. "owner/skill-name") over display name for the
// download URL, since the registry endpoint expects a slug.
let download_key = req
.slug
.as_deref()
.filter(|s| !s.is_empty())
.unwrap_or(&req.name);
let url = crate::skills::catalog::skill_download_url(catalog.registry_url(), download_key);
crate::tools::builtin::skill_tools::fetch_skill_content(&url)
.await
.map_err(|e| (StatusCode::BAD_GATEWAY, e.to_string()))?
+23 -11
View File
@@ -63,13 +63,11 @@ impl GatewayChannel {
/// If no auth token is configured, generates a random one and prints it.
pub fn new(config: GatewayConfig) -> Self {
let auth_token = config.auth_token.clone().unwrap_or_else(|| {
use rand::Rng;
let token: String = rand::thread_rng()
.sample_iter(&rand::distributions::Alphanumeric)
.take(32)
.map(char::from)
.collect();
token
use rand::RngCore;
use rand::rngs::OsRng;
let mut bytes = [0u8; 32];
OsRng.fill_bytes(&mut bytes);
bytes.iter().map(|b| format!("{b:02x}")).collect()
});
let state = Arc::new(GatewayState {
@@ -84,6 +82,7 @@ impl GatewayChannel {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: config.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(ws::WsConnectionTracker::new())),
@@ -94,7 +93,6 @@ impl GatewayChannel {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
});
Self {
@@ -108,7 +106,8 @@ impl GatewayChannel {
fn rebuild_state(&mut self, mutate: impl FnOnce(&mut GatewayState)) {
let mut new_state = GatewayState {
msg_tx: tokio::sync::RwLock::new(None),
sse: SseManager::new(),
// Preserve the existing broadcast channel so sender handles remain valid.
sse: SseManager::from_sender(self.state.sse.sender()),
workspace: self.state.workspace.clone(),
session_manager: self.state.session_manager.clone(),
log_broadcaster: self.state.log_broadcaster.clone(),
@@ -118,6 +117,7 @@ impl GatewayChannel {
store: self.state.store.clone(),
job_manager: self.state.job_manager.clone(),
prompt_queue: self.state.prompt_queue.clone(),
scheduler: self.state.scheduler.clone(),
user_id: self.state.user_id.clone(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: self.state.ws_tracker.clone(),
@@ -128,7 +128,6 @@ impl GatewayChannel {
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
startup_time: self.state.startup_time,
restart_requested: std::sync::atomic::AtomicBool::new(false),
};
mutate(&mut new_state);
self.state = Arc::new(new_state);
@@ -198,6 +197,12 @@ impl GatewayChannel {
self
}
/// Inject the scheduler for sending follow-up messages to agent jobs.
pub fn with_scheduler(mut self, slot: crate::tools::builtin::SchedulerSlot) -> Self {
self.rebuild_state(|s| s.scheduler = Some(slot));
self
}
/// Inject the skill registry for skill management API.
pub fn with_skill_registry(mut self, sr: Arc<std::sync::RwLock<SkillRegistry>>) -> Self {
self.rebuild_state(|s| s.skill_registry = Some(sr));
@@ -297,9 +302,16 @@ impl Channel for GatewayChannel {
name,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolCompleted { name, success } => SseEvent::ToolCompleted {
StatusUpdate::ToolCompleted {
name,
success,
error,
parameters,
} => SseEvent::ToolCompleted {
name,
success,
error,
parameters,
thread_id: thread_id.clone(),
},
StatusUpdate::ToolResult { name, preview } => SseEvent::ToolResult {
+603 -50
View File
@@ -156,6 +156,8 @@ pub struct GatewayState {
pub skill_registry: Option<Arc<std::sync::RwLock<crate::skills::SkillRegistry>>>,
/// Skill catalog for searching the ClawHub registry.
pub skill_catalog: Option<Arc<crate::skills::catalog::SkillCatalog>>,
/// Scheduler for sending follow-up messages to running agent jobs.
pub scheduler: Option<crate::tools::builtin::SchedulerSlot>,
/// Rate limiter for chat endpoints (30 messages per 60 seconds).
pub chat_rate_limiter: RateLimiter,
/// Registry catalog entries for the available extensions API.
@@ -165,8 +167,6 @@ pub struct GatewayState {
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
/// Flag set when a restart has been requested via the API.
pub restart_requested: std::sync::atomic::AtomicBool,
}
/// Start the gateway HTTP server.
@@ -192,7 +192,9 @@ pub async fn start_server(
})?;
// Public routes (no auth)
let public = Router::new().route("/api/health", get(health_handler));
let public = Router::new()
.route("/api/health", get(health_handler))
.route("/oauth/callback", get(oauth_callback_handler));
// Protected routes (require auth)
let auth_state = AuthState { token: auth_token };
@@ -247,8 +249,6 @@ pub async fn start_server(
"/api/extensions/{name}/setup",
get(extensions_setup_handler).post(extensions_setup_submit_handler),
)
// Gateway management
.route("/api/gateway/restart", post(gateway_restart_handler))
// Pairing
.route("/api/pairing/{channel}", get(pairing_list_handler))
.route(
@@ -426,12 +426,192 @@ async fn health_handler() -> Json<HealthResponse> {
})
}
/// Return an OAuth error landing page response.
fn oauth_error_page(label: &str) -> axum::response::Response {
let html = crate::cli::oauth_defaults::landing_html(label, false);
axum::response::Html(html).into_response()
}
/// OAuth callback handler for the web gateway.
///
/// This is a PUBLIC route (no Bearer token required) because OAuth providers
/// redirect the user's browser here. The `state` query parameter correlates
/// the callback with a pending OAuth flow registered by `start_wasm_oauth()`.
///
/// Used on hosted instances where `IRONCLAW_OAUTH_CALLBACK_URL` points to
/// the gateway (e.g., `https://kind-deer.agent1.near.ai/oauth/callback`).
/// Local/desktop mode continues to use the TCP listener on port 9876.
async fn oauth_callback_handler(
State(state): State<Arc<GatewayState>>,
Query(params): Query<std::collections::HashMap<String, String>>,
) -> impl IntoResponse {
use crate::cli::oauth_defaults;
// Check for error from OAuth provider (e.g., user denied consent)
if let Some(error) = params.get("error") {
let description = params
.get("error_description")
.cloned()
.unwrap_or_else(|| error.clone());
return oauth_error_page(&description);
}
let state_param = match params.get("state") {
Some(s) if !s.is_empty() => s.clone(),
_ => return oauth_error_page("IronClaw"),
};
let code = match params.get("code") {
Some(c) if !c.is_empty() => c.clone(),
_ => return oauth_error_page("IronClaw"),
};
// Look up the pending flow by CSRF state (atomic remove prevents replay)
let ext_mgr = match state.extension_manager.as_ref() {
Some(mgr) => mgr,
None => return oauth_error_page("IronClaw"),
};
// Strip instance prefix from state for registry lookup.
// Platform nginx sends `state=instance:nonce` but flows are keyed by nonce only.
let lookup_key = oauth_defaults::strip_instance_prefix(&state_param);
let flow = ext_mgr
.pending_oauth_flows()
.write()
.await
.remove(lookup_key);
let flow = match flow {
Some(f) => f,
None => {
tracing::warn!(
state = %state_param,
lookup_key = %lookup_key,
"OAuth callback received with unknown or expired state"
);
return oauth_error_page("IronClaw");
}
};
// Check flow expiry (5 minutes, matching TCP listener timeout)
if flow.created_at.elapsed() > oauth_defaults::OAUTH_FLOW_EXPIRY {
tracing::warn!(
extension = %flow.extension_name,
"OAuth flow expired"
);
return oauth_error_page(&flow.display_name);
}
// Exchange the authorization code for tokens.
// Use the platform exchange proxy when configured (keeps client_secret off container),
// otherwise call the provider's token URL directly.
let exchange_proxy_url = std::env::var("IRONCLAW_OAUTH_EXCHANGE_URL").ok();
let result: Result<(), String> = async {
let token_response = if let Some(ref proxy_url) = exchange_proxy_url {
let gateway_token = flow.gateway_token.as_deref().unwrap_or_default();
oauth_defaults::exchange_via_proxy(
proxy_url,
gateway_token,
&code,
&flow.redirect_uri,
flow.code_verifier.as_deref(),
&flow.access_token_field,
)
.await
.map_err(|e| e.to_string())?
} else {
oauth_defaults::exchange_oauth_code(
&flow.token_url,
&flow.client_id,
flow.client_secret.as_deref(),
&code,
&flow.redirect_uri,
flow.code_verifier.as_deref(),
&flow.access_token_field,
)
.await
.map_err(|e| e.to_string())?
};
// Validate the token before storing (catches wrong account, etc.)
if let Some(ref validation) = flow.validation_endpoint {
oauth_defaults::validate_oauth_token(&token_response.access_token, validation)
.await
.map_err(|e| e.to_string())?;
}
// Store tokens encrypted in the secrets store
oauth_defaults::store_oauth_tokens(
flow.secrets.as_ref(),
&flow.user_id,
&flow.secret_name,
flow.provider.as_deref(),
&token_response.access_token,
token_response.refresh_token.as_deref(),
token_response.expires_in,
&flow.scopes,
)
.await
.map_err(|e| e.to_string())?;
Ok(())
}
.await;
let (success, message) = match &result {
Ok(()) => (
true,
format!("{} authenticated successfully", flow.display_name),
),
Err(e) => (
false,
format!("{} authentication failed: {}", flow.display_name, e),
),
};
match &result {
Ok(()) => {
tracing::info!(
extension = %flow.extension_name,
"OAuth completed successfully via gateway callback"
);
}
Err(e) => {
tracing::warn!(
extension = %flow.extension_name,
error = %e,
"OAuth failed via gateway callback"
);
}
}
// Broadcast SSE event to notify the web UI
if let Some(ref sender) = flow.sse_sender {
let _ = sender.send(SseEvent::AuthCompleted {
extension_name: flow.extension_name,
success,
message,
});
}
let html = oauth_defaults::landing_html(&flow.display_name, success);
axum::response::Html(html).into_response()
}
// --- Chat handlers ---
async fn chat_send_handler(
State(state): State<Arc<GatewayState>>,
Json(req): Json<SendMessageRequest>,
) -> Result<(StatusCode, Json<SendMessageResponse>), (StatusCode, String)> {
tracing::debug!(
"[chat_send_handler] Received message: content={:?}, thread_id={:?}",
req.content,
req.thread_id
);
if !state.chat_rate_limiter.check() {
return Err((
StatusCode::TOO_MANY_REQUESTS,
@@ -447,6 +627,11 @@ async fn chat_send_handler(
}
let msg_id = msg.id;
tracing::debug!(
"[chat_send_handler] Created message id={}, content={:?}",
msg_id,
req.content
);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
@@ -454,6 +639,7 @@ async fn chat_send_handler(
"Channel not started".to_string(),
))?;
tracing::debug!("[chat_send_handler] Sending message through channel");
tx.send(msg).await.map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
@@ -461,6 +647,8 @@ async fn chat_send_handler(
)
})?;
tracing::debug!("[chat_send_handler] Message sent successfully, returning 202 ACCEPTED");
Ok((
StatusCode::ACCEPTED,
Json(SendMessageResponse {
@@ -554,7 +742,7 @@ async fn chat_auth_token_handler(
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if result.status == "authenticated" {
if result.is_authenticated() {
// Auto-activate so tools are available immediately
let msg = match ext_mgr.activate(&req.extension_name).await {
Ok(r) => format!(
@@ -582,13 +770,14 @@ async fn chat_auth_token_handler(
// Re-emit auth_required for retry
state.sse.broadcast(SseEvent::AuthRequired {
extension_name: req.extension_name.clone(),
instructions: result.instructions.clone(),
auth_url: result.auth_url.clone(),
setup_url: result.setup_url.clone(),
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
Ok(Json(ActionResponse::fail(
result
.instructions
.instructions()
.map(String::from)
.unwrap_or_else(|| "Invalid token".to_string()),
)))
}
@@ -1218,8 +1407,8 @@ async fn extensions_list_handler(
} else if !ext.authenticated {
// No credentials configured yet.
"installed".to_string()
} else if ext.active && ext.name == "telegram" {
// Telegram: check pairing status (end-to-end setup via web UI).
} else if ext.active {
// Check pairing status for active channels.
let has_paired = pairing_store
.read_allow_from(&ext.name)
.map(|list| !list.is_empty())
@@ -1230,7 +1419,7 @@ async fn extensions_list_handler(
"pairing".to_string()
}
} else {
// Authenticated but not fully active (or non-Telegram).
// Authenticated but not yet active.
"configured".to_string()
})
} else {
@@ -1246,6 +1435,7 @@ async fn extensions_list_handler(
active: ext.active,
tools: ext.tools,
needs_setup: ext.needs_setup,
has_auth: ext.has_auth,
activation_status,
activation_error: ext.activation_error,
}
@@ -1315,7 +1505,34 @@ async fn extensions_install_handler(
.install(&req.name, req.url.as_deref(), kind_hint)
.await
{
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
let mut resp = ActionResponse::ok(result.message);
// Auto-activate WASM tools after install (install = active).
if result.kind == crate::extensions::ExtensionKind::WasmTool {
if let Err(e) = ext_mgr.activate(&req.name).await {
tracing::debug!(
extension = %req.name,
error = %e,
"Auto-activation after install failed"
);
}
// Check auth after activation. This may initiate OAuth both for scope
// expansion and for first-time auth when credentials are already
// configured (e.g., built-in providers). We only surface an auth_url
// when the extension reports it is awaiting authorization.
match ext_mgr.auth(&req.name, None).await {
Ok(auth_result) if auth_result.auth_url().is_some() => {
// Scope expansion or initial OAuth: user needs to authorize
resp.auth_url = auth_result.auth_url().map(String::from);
}
_ => {}
}
}
Ok(Json(resp))
}
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
@@ -1330,7 +1547,19 @@ async fn extensions_activate_handler(
))?;
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
Ok(result) => {
// Activation loaded the WASM module. Check if the tool needs
// OAuth scope expansion (e.g., adding google-docs when gmail
// already has a token but missing the documents scope).
// Initial OAuth setup is triggered via save_setup_secrets.
let mut resp = ActionResponse::ok(result.message);
if let Ok(auth_result) = ext_mgr.auth(&name, None).await
&& auth_result.auth_url().is_some()
{
resp.auth_url = auth_result.auth_url().map(String::from);
}
Ok(Json(resp))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
@@ -1343,7 +1572,7 @@ async fn extensions_activate_handler(
// Activation failed due to auth; try authenticating first.
match ext_mgr.auth(&name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded, retry activation.
match ext_mgr.activate(&name).await {
Ok(result) => Ok(Json(ActionResponse::ok(result.message))),
@@ -1354,13 +1583,13 @@ async fn extensions_activate_handler(
// Auth in progress (OAuth URL or awaiting manual token).
let mut resp = ActionResponse::fail(
auth_result
.instructions
.clone()
.instructions()
.map(String::from)
.unwrap_or_else(|| format!("'{}' requires authentication.", name)),
);
resp.auth_url = auth_result.auth_url;
resp.awaiting_token = Some(auth_result.awaiting_token);
resp.instructions = auth_result.instructions;
resp.auth_url = auth_result.auth_url().map(String::from);
resp.awaiting_token = Some(auth_result.is_awaiting_token());
resp.instructions = auth_result.instructions().map(String::from);
Ok(Json(resp))
}
Err(auth_err) => Ok(Json(ActionResponse::fail(format!(
@@ -1550,43 +1779,22 @@ async fn extensions_setup_submit_handler(
match ext_mgr.save_setup_secrets(&name, &req.secrets).await {
Ok(result) => {
// Broadcast auth_completed so the chat UI can dismiss any in-progress
// auth card or setup modal that was triggered by tool_auth/tool_activate.
state.sse.broadcast(SseEvent::AuthCompleted {
extension_name: name.clone(),
success: true,
message: result.message.clone(),
});
let mut resp = ActionResponse::ok(result.message);
resp.activated = Some(result.activated);
if !result.activated {
resp.needs_restart = Some(true);
}
resp.auth_url = result.auth_url;
Ok(Json(resp))
}
Err(e) => Ok(Json(ActionResponse::fail(e.to_string()))),
}
}
// --- Gateway management handlers ---
async fn gateway_restart_handler(State(state): State<Arc<GatewayState>>) -> Json<ActionResponse> {
// Idempotency guard: only allow one restart at a time.
if state
.restart_requested
.compare_exchange(
false,
true,
std::sync::atomic::Ordering::SeqCst,
std::sync::atomic::Ordering::SeqCst,
)
.is_err()
{
return Json(ActionResponse::ok("Restart already in progress"));
}
// Take the shutdown sender and trigger graceful shutdown.
if let Some(tx) = state.shutdown_tx.write().await.take() {
let _ = tx.send(());
tracing::info!("Gateway restart requested via API");
}
Json(ActionResponse::ok("Restarting..."))
}
// --- Pairing handlers ---
async fn pairing_list_handler(
@@ -2106,11 +2314,16 @@ async fn gateway_status_handler(
(None, None, None)
};
let restart_enabled = std::env::var("IRONCLAW_IN_DOCKER")
.map(|v| v.to_lowercase() == "true")
.unwrap_or(false);
Json(GatewayStatusResponse {
sse_connections,
ws_connections,
total_connections: sse_connections + ws_connections,
uptime_secs,
restart_enabled,
daily_cost,
actions_this_hour,
model_usage,
@@ -2131,6 +2344,7 @@ struct GatewayStatusResponse {
ws_connections: u64,
total_connections: u64,
uptime_secs: u64,
restart_enabled: bool,
#[serde(skip_serializing_if = "Option::is_none")]
daily_cost: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -2218,4 +2432,343 @@ mod tests {
let turns = build_turns_from_db_messages(&[]);
assert!(turns.is_empty());
}
// --- OAuth callback handler tests ---
/// Build a minimal `GatewayState` for testing the OAuth callback handler.
fn test_gateway_state(ext_mgr: Option<Arc<ExtensionManager>>) -> Arc<GatewayState> {
Arc::new(GatewayState {
msg_tx: tokio::sync::RwLock::new(None),
sse: SseManager::new(),
workspace: None,
session_manager: None,
log_broadcaster: None,
log_level_handle: None,
extension_manager: ext_mgr,
tool_registry: None,
store: None,
job_manager: None,
prompt_queue: None,
user_id: "test".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: None,
llm_provider: None,
skill_registry: None,
skill_catalog: None,
scheduler: None,
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: vec![],
cost_guard: None,
startup_time: std::time::Instant::now(),
})
}
/// Build a test router with just the OAuth callback route.
fn test_oauth_router(state: Arc<GatewayState>) -> Router {
Router::new()
.route("/oauth/callback", get(oauth_callback_handler))
.with_state(state)
}
#[tokio::test]
async fn test_oauth_callback_missing_params() {
use axum::body::Body;
use tower::ServiceExt;
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_error_from_provider() {
use axum::body::Body;
use tower::ServiceExt;
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?error=access_denied&error_description=access_denied")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_unknown_state() {
use axum::body::Body;
use tower::ServiceExt;
// Build an ExtensionManager so the handler can look up flows
let secrets = Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets,
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=unknown_state_value")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_expired_flow() {
use axum::body::Body;
use tower::ServiceExt;
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
// Insert an expired flow (created 10 minutes ago)
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "test_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: None,
gateway_token: None,
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
.pending_oauth_flows()
.write()
.await
.insert("expired_state".to_string(), flow);
let state = test_gateway_state(Some(ext_mgr));
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=expired_state")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
// Expired flow → error landing page
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_no_extension_manager() {
use axum::body::Body;
use tower::ServiceExt;
// No extension manager set → graceful error
let state = test_gateway_state(None);
let app = test_oauth_router(state);
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=test_code&state=some_state")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
assert!(html.contains("Authorization Failed"));
}
#[tokio::test]
async fn test_oauth_callback_strips_instance_prefix() {
use axum::body::Body;
use tower::ServiceExt;
let secrets: Arc<dyn crate::secrets::SecretsStore + Send + Sync> =
Arc::new(crate::secrets::InMemorySecretsStore::new(Arc::new(
crate::secrets::SecretsCrypto::new(secrecy::SecretString::from(
"test-key-at-least-32-chars-long!!".to_string(),
))
.expect("crypto"),
)));
let tool_registry = Arc::new(ToolRegistry::new());
let mcp_sm = Arc::new(crate::tools::mcp::session::McpSessionManager::new());
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
secrets.clone(),
tool_registry,
None,
None,
std::path::PathBuf::from("/tmp/wasm_tools"),
std::path::PathBuf::from("/tmp/wasm_channels"),
None,
"test".to_string(),
None,
vec![],
));
// Insert a flow keyed by raw nonce "test_nonce" (without instance prefix).
// Use an expired flow so the handler exits before attempting a real HTTP
// token exchange — we only need to verify that the instance prefix was
// stripped and the flow was found by the raw nonce.
let flow = crate::cli::oauth_defaults::PendingOAuthFlow {
extension_name: "test_tool".to_string(),
display_name: "Test Tool".to_string(),
token_url: "https://example.com/token".to_string(),
client_id: "client123".to_string(),
client_secret: None,
redirect_uri: "https://example.com/oauth/callback".to_string(),
code_verifier: None,
access_token_field: "access_token".to_string(),
secret_name: "test_token".to_string(),
provider: None,
validation_endpoint: None,
scopes: vec![],
user_id: "test".to_string(),
secrets,
sse_sender: None,
gateway_token: None,
// Expired — handler will reject after lookup (no network I/O)
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
};
ext_mgr
.pending_oauth_flows()
.write()
.await
.insert("test_nonce".to_string(), flow);
let state = test_gateway_state(Some(ext_mgr.clone()));
let app = test_oauth_router(state);
// Send callback with instance prefix: "myinstance:test_nonce"
// The handler should strip "myinstance:" and find the flow keyed by "test_nonce"
let req = axum::http::Request::builder()
.uri("/oauth/callback?code=fake_code&state=myinstance:test_nonce")
.body(Body::empty())
.expect("request");
let resp = ServiceExt::<axum::http::Request<Body>>::oneshot(app, req)
.await
.expect("response");
assert_eq!(resp.status(), StatusCode::OK);
let body = axum::body::to_bytes(resp.into_body(), 1024 * 64)
.await
.expect("body");
let html = String::from_utf8_lossy(&body);
// The flow was found (stripped prefix matched) but is expired, so the
// handler returns an error landing page. The flow being consumed from
// the registry (checked below) proves the prefix was stripped correctly.
assert!(
html.contains("Authorization Failed"),
"Expected error page, html was: {}",
&html[..html.len().min(500)]
);
// Verify the flow was consumed (removed from registry)
assert!(
ext_mgr
.pending_oauth_flows()
.read()
.await
.get("test_nonce")
.is_none()
);
}
}
+17
View File
@@ -36,6 +36,23 @@ impl SseManager {
}
}
/// Create an SSE manager that reuses an existing broadcast sender.
///
/// This preserves the broadcast channel across `rebuild_state` calls so
/// that sender handles captured by other components remain valid.
///
/// **Important:** The connection counter is reset to zero. This method must
/// only be called before the server starts accepting connections (i.e.,
/// during startup wiring). Calling it after connections are established
/// will break connection tracking and allow exceeding `MAX_CONNECTIONS`.
pub fn from_sender(tx: broadcast::Sender<SseEvent>) -> Self {
Self {
tx,
connection_count: Arc::new(AtomicU64::new(0)),
max_connections: MAX_CONNECTIONS,
}
}
/// Broadcast an event to all connected clients.
pub fn broadcast(&self, event: SseEvent) {
// Ignore send errors (no receivers is fine)
+231 -104
View File
@@ -133,6 +133,110 @@ function apiFetch(path, options) {
});
}
// --- Restart Feature ---
let isRestarting = false; // Track if we're currently restarting
let restartEnabled = false; // Track if restart is available in this deployment
function triggerRestart() {
if (!currentThreadId) {
alert('Please start a conversation first');
return;
}
// Show the confirmation modal
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'flex';
}
function confirmRestart() {
if (!currentThreadId) {
alert('Please start a conversation first');
return;
}
// Hide confirmation modal
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'none';
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
// Mark as restarting
isRestarting = true;
restartBtn.disabled = true;
if (restartIcon) restartIcon.classList.add('spinning');
// Show progress modal
const loaderEl = document.getElementById('restart-loader');
loaderEl.style.display = 'flex';
// Send restart command via chat
console.log('[confirmRestart] Sending /restart command to server');
apiFetch('/api/chat/send', {
method: 'POST',
body: {
content: '/restart',
thread_id: currentThreadId,
},
})
.then((response) => {
console.log('[confirmRestart] API call succeeded, response:', response);
})
.catch((err) => {
console.error('[confirmRestart] Restart request failed:', err);
addMessage('system', 'Restart failed: ' + err.message);
isRestarting = false;
restartBtn.disabled = false;
if (restartIcon) restartIcon.classList.remove('spinning');
loaderEl.style.display = 'none';
});
}
function cancelRestart() {
const confirmModal = document.getElementById('restart-confirm-modal');
confirmModal.style.display = 'none';
}
function tryShowRestartModal() {
// Defensive callback for when restart is detected in messages.
if (!isRestarting) {
isRestarting = true;
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
restartBtn.disabled = true;
if (restartIcon) restartIcon.classList.add('spinning');
// Show progress modal
const loaderEl = document.getElementById('restart-loader');
loaderEl.style.display = 'flex';
}
}
function updateRestartButtonVisibility() {
const restartBtn = document.getElementById('restart-btn');
if (restartBtn) {
restartBtn.style.display = restartEnabled ? 'block' : 'none';
}
}
function startGatewayStatusPolling() {
fetchGatewayStatus();
// Poll every 5 seconds
setInterval(fetchGatewayStatus, 5000);
}
function fetchGatewayStatus() {
apiFetch('/api/gateway/status')
.then((data) => {
restartEnabled = data.restart_enabled || false;
updateRestartButtonVisibility();
})
.catch((err) => {
console.warn('[gateway status] Failed to fetch:', err);
});
}
// --- SSE ---
function connectSSE() {
@@ -143,6 +247,18 @@ function connectSSE() {
eventSource.onopen = () => {
document.getElementById('sse-dot').classList.remove('disconnected');
document.getElementById('sse-status').textContent = 'Connected';
// If we were restarting, close the modal and reset button now that server is back
if (isRestarting) {
const loaderEl = document.getElementById('restart-loader');
if (loaderEl) loaderEl.style.display = 'none';
const restartBtn = document.getElementById('restart-btn');
const restartIcon = document.getElementById('restart-icon');
if (restartBtn) restartBtn.disabled = false;
if (restartIcon) restartIcon.classList.remove('spinning');
isRestarting = false;
}
if (sseHasConnectedBefore && currentThreadId) {
finalizeActivityGroup();
loadHistory();
@@ -163,6 +279,11 @@ function connectSSE() {
enableChatInput();
// Refresh thread list so new titles appear after first message
loadThreads();
// Show restart modal if the response indicates restart was initiated
if (data.content && data.content.toLowerCase().includes('restart initiated')) {
setTimeout(() => tryShowRestartModal(), 500);
}
});
eventSource.addEventListener('thinking', (e) => {
@@ -180,7 +301,12 @@ function connectSSE() {
eventSource.addEventListener('tool_completed', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
completeToolCard(data.name, data.success);
completeToolCard(data.name, data.success, data.error, data.parameters);
// Show restart modal only when the restart tool succeeds
if (data.name.toLowerCase() === 'restart' && data.success) {
setTimeout(() => tryShowRestartModal(), 500);
}
});
eventSource.addEventListener('tool_result', (e) => {
@@ -222,13 +348,24 @@ function connectSSE() {
eventSource.addEventListener('auth_required', (e) => {
const data = JSON.parse(e.data);
showAuthCard(data);
if (data.auth_url) {
// OAuth flow: show the auth card with an OAuth button + optional token paste field.
showAuthCard(data);
} else {
// Setup flow: fetch the extension's credential schema and show the multi-field
// configure modal (the same UI used by the Extensions tab "Setup" button).
showConfigureModal(data.extension_name);
}
});
eventSource.addEventListener('auth_completed', (e) => {
const data = JSON.parse(e.data);
// Dismiss whichever UI path was active: auth card (OAuth) or configure modal (setup).
removeAuthCard(data.extension_name);
showToast(data.message, 'success');
closeConfigureModal();
showToast(data.message, data.success ? 'success' : 'error');
// Refresh extensions list so status indicators update
if (currentTab === 'extensions') loadExtensions();
enableChatInput();
});
@@ -359,6 +496,9 @@ function selectSlashItem(cmd) {
function updateSlashHighlight() {
const items = document.querySelectorAll('#slash-autocomplete .slash-ac-item');
items.forEach((el, i) => el.classList.toggle('selected', i === _slashSelected));
if (_slashSelected >= 0 && items[_slashSelected]) {
items[_slashSelected].scrollIntoView({ block: 'nearest' });
}
}
function filterSlashCommands(value) {
@@ -581,7 +721,7 @@ function addToolCard(name) {
container.scrollTop = container.scrollHeight;
}
function completeToolCard(name, success) {
function completeToolCard(name, success, error, parameters) {
const entries = _activeToolCards[name];
if (!entries || entries.length === 0) return;
// Find first running card
@@ -602,6 +742,27 @@ function completeToolCard(name, success) {
? '<span class="activity-icon-success">&#10003;</span>'
: '<span class="activity-icon-fail">&#10007;</span>';
entry.card.setAttribute('data-status', success ? 'success' : 'fail');
// For failed tools, populate the body with error details and auto-expand
if (!success && (error || parameters)) {
const output = entry.card.querySelector('.activity-tool-output');
if (output) {
let detail = '';
if (parameters) {
detail += 'Input:\n' + parameters + '\n\n';
}
if (error) {
detail += 'Error:\n' + error;
}
output.textContent = detail;
// Auto-expand so the error is immediately visible
const body = entry.card.querySelector('.activity-tool-body');
const chevron = entry.card.querySelector('.activity-tool-chevron');
if (body) body.style.display = 'block';
if (chevron) chevron.classList.add('expanded');
}
}
}
function setToolCardOutput(name, preview) {
@@ -865,7 +1026,7 @@ function showAuthCard(data) {
const tokenInput = document.createElement('input');
tokenInput.type = 'password';
tokenInput.placeholder = 'Paste your API key or token';
tokenInput.placeholder = data.instructions || 'Paste your API key or token';
tokenInput.addEventListener('keydown', (e) => {
if (e.key === 'Enter') submitAuthToken(data.extension_name, tokenInput.value);
});
@@ -1196,7 +1357,7 @@ chatInput.addEventListener('keydown', (e) => {
updateSlashHighlight();
return;
}
if (e.key === 'Tab' || (e.key === 'Enter' && _slashSelected >= 0)) {
if (e.key === 'Tab' || e.key === 'Enter') {
e.preventDefault();
const pick = _slashSelected >= 0 ? _slashMatches[_slashSelected] : _slashMatches[0];
if (pick) selectSlashItem(pick.cmd);
@@ -1757,6 +1918,11 @@ function renderAvailableExtensionCard(entry) {
}).then(function(res) {
if (res.success) {
showToast('Installed ' + entry.display_name, 'success');
// OAuth popup if auth started during install (builtin creds)
if (res.auth_url) {
showToast('Opening authentication for ' + entry.display_name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
}
loadExtensions();
// Auto-open configure for WASM channels
if (entry.kind === 'wasm_channel') {
@@ -1928,14 +2094,6 @@ function renderExtensionCard(ext) {
card.appendChild(errorDiv);
}
// Show "coming soon" note for non-Telegram channels that are configured but not fully supported yet
if (ext.kind === 'wasm_channel' && ext.name !== 'telegram'
&& (ext.activation_status === 'configured' || ext.active)) {
const noteDiv = document.createElement('div');
noteDiv.className = 'ext-note';
noteDiv.textContent = 'Full integration coming soon. Use the CLI to complete setup.';
card.appendChild(noteDiv);
}
const actions = document.createElement('div');
actions.className = 'ext-actions';
@@ -1966,24 +2124,29 @@ function renderExtensionCard(ext) {
actions.appendChild(setupBtn);
}
} else {
// Non-WASM-channel extensions: original behavior
if (!ext.active) {
// WASM tools / MCP servers
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = ext.active ? 'Active' : 'Installed';
actions.appendChild(activeLabel);
// MCP servers may be installed but inactive — show Activate button
if (ext.kind === 'mcp_server' && !ext.active) {
const activateBtn = document.createElement('button');
activateBtn.className = 'btn-ext activate';
activateBtn.textContent = 'Activate';
activateBtn.addEventListener('click', () => activateExtension(ext.name));
actions.appendChild(activateBtn);
} else {
const activeLabel = document.createElement('span');
activeLabel.className = 'ext-active-label';
activeLabel.textContent = 'Active';
actions.appendChild(activeLabel);
}
if (ext.needs_setup) {
// Show Configure/Reconfigure button when there are secrets to enter.
// Skip when has_auth is true but needs_setup is false and not yet authenticated —
// this means OAuth credentials resolve automatically (builtin/env) and the user
// just needs to complete the OAuth flow, not fill in a config form.
if (ext.needs_setup || (ext.has_auth && ext.authenticated)) {
const configBtn = document.createElement('button');
configBtn.className = 'btn-ext configure';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Setup';
configBtn.textContent = ext.authenticated ? 'Reconfigure' : 'Configure';
configBtn.addEventListener('click', () => showConfigureModal(ext.name));
actions.appendChild(configBtn);
}
@@ -2013,6 +2176,11 @@ function activateExtension(name) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/activate', { method: 'POST' })
.then((res) => {
if (res.success) {
// Even on success, the tool may need OAuth (e.g., WASM loaded but no token yet)
if (res.auth_url) {
showToast('Opening authentication for ' + name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
}
loadExtensions();
return;
}
@@ -2163,17 +2331,19 @@ function submitConfigureModal(name, fields) {
.then((res) => {
closeConfigureModal();
if (res.success) {
if (res.activated) {
showToast('Configured and activated ' + name, 'success');
} else if (res.needs_restart) {
showToast('Configured ' + name + '. Use Reconfigure to re-enter credentials and activate.', 'info');
} else {
showToast(res.message, 'success');
if (res.auth_url) {
// OAuth flow started — open consent popup. The auth_completed SSE will
// not arrive immediately (it fires after OAuth callback), so show a toast now.
showToast('Opening OAuth authorization for ' + name, 'info');
window.open(res.auth_url, '_blank', 'width=600,height=700');
loadExtensions();
}
// For non-OAuth success: the server always broadcasts auth_completed SSE,
// which will show the toast and refresh extensions — no need to do it here too.
} else {
showToast(res.message || 'Configuration failed', 'error');
loadExtensions();
}
loadExtensions();
})
.catch((err) => {
btns.forEach(function(b) { b.disabled = false; });
@@ -2232,7 +2402,7 @@ function approvePairing(channel, code, container) {
}).then(res => {
if (res.success) {
showToast('Pairing approved', 'success');
loadPairingRequests(channel, container);
loadExtensions();
} else {
showToast(res.message || 'Approve failed', 'error');
}
@@ -2255,53 +2425,6 @@ function stopPairingPoll() {
}
}
// --- Gateway restart ---
function restartGateway() {
if (!confirm('Restart IronClaw gateway? Active connections will be dropped.')) return;
apiFetch('/api/gateway/restart', { method: 'POST' })
.then(function() {
showRestartOverlay();
})
.catch(function() {
showRestartOverlay();
});
}
function showRestartOverlay() {
var overlay = document.createElement('div');
overlay.className = 'restart-overlay';
overlay.innerHTML = '<div class="restart-message">'
+ '<div class="restart-spinner"></div>'
+ '<h2>Restarting IronClaw...</h2>'
+ '<p>Waiting for server to come back online</p>'
+ '</div>';
document.body.appendChild(overlay);
var pollCount = 0;
var pollTimer = setInterval(function() {
pollCount++;
if (pollCount > 30) { // 60 seconds
clearInterval(pollTimer);
overlay.querySelector('h2').textContent = 'Restart timed out';
overlay.querySelector('p').textContent = 'Server did not come back within 60 seconds. Check logs.';
overlay.querySelector('.restart-spinner').style.display = 'none';
return;
}
fetch('/api/gateway/status', {
headers: { 'Authorization': 'Bearer ' + token },
})
.then(function(r) {
if (r.ok) {
clearInterval(pollTimer);
window.location.reload();
}
})
.catch(function() { /* still restarting */ });
}, 2000);
}
// --- WASM channel stepper ---
function renderWasmChannelStepper(ext) {
@@ -2309,23 +2432,17 @@ function renderWasmChannelStepper(ext) {
stepper.className = 'ext-stepper';
var status = ext.activation_status || 'installed';
var isTelegram = ext.name === 'telegram';
// Telegram gets a 3-step stepper (Installed → Configured → Active/Pairing).
// Other channels only get 2 steps (Installed → Configured) since full
// integration isn't available in the web UI yet.
var steps = [
{ label: 'Installed', key: 'installed' },
{ label: 'Configured', key: 'configured' },
{ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' },
];
if (isTelegram) {
steps.push({ label: status === 'pairing' ? 'Awaiting Pairing' : 'Active', key: 'active' });
}
var reachedIdx;
if (status === 'active') reachedIdx = isTelegram ? 2 : 1;
if (status === 'active') reachedIdx = 2;
else if (status === 'pairing') reachedIdx = 2;
else if (status === 'failed') reachedIdx = isTelegram ? 2 : 1;
else if (status === 'failed') reachedIdx = 2;
else if (status === 'configured') reachedIdx = 1;
else reachedIdx = 0;
@@ -2436,9 +2553,8 @@ function renderJobsList(jobs) {
let actionBtns = '';
if (job.state === 'pending' || job.state === 'in_progress') {
actionBtns = '<button class="btn-cancel" onclick="event.stopPropagation(); cancelJob(\'' + job.id + '\')">Cancel</button>';
} else if (job.state === 'failed' || job.state === 'interrupted') {
actionBtns = '<button class="btn-restart" onclick="event.stopPropagation(); restartJob(\'' + job.id + '\')">Restart</button>';
}
// Retry is only shown in the detail view where can_restart is available.
return '<tr class="job-row" onclick="openJobDetail(\'' + job.id + '\')">'
+ '<td title="' + escapeHtml(job.id) + '">' + shortId + '</td>'
@@ -2467,10 +2583,12 @@ function restartJob(jobId) {
apiFetch('/api/jobs/' + jobId + '/restart', { method: 'POST' })
.then((res) => {
showToast('Job restarted as ' + (res.new_job_id || '').substring(0, 8), 'success');
loadJobs();
})
.catch((err) => {
showToast('Failed to restart job: ' + err.message, 'error');
})
.finally(() => {
loadJobs();
});
}
@@ -2505,8 +2623,8 @@ function renderJobDetail(job) {
+ '<h2>' + escapeHtml(job.title) + '</h2>'
+ '<span class="badge ' + stateClass + '">' + escapeHtml(job.state) + '</span>';
if (job.state === 'failed' || job.state === 'interrupted') {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Restart</button>';
if ((job.state === 'failed' || job.state === 'interrupted') && job.can_restart === true) {
headerHtml += '<button class="btn-restart" onclick="restartJob(\'' + job.id + '\')">Retry</button>';
}
if (job.browse_url) {
headerHtml += '<a class="btn-browse" href="' + escapeHtml(job.browse_url) + '" target="_blank">Browse Files</a>';
@@ -2753,7 +2871,7 @@ function renderJobActivity(container, job) {
activityCurrentJobId = job ? job.id : null;
activityRenderedLiveIndex = 0;
container.innerHTML = '<div class="activity-toolbar">'
let html = '<div class="activity-toolbar">'
+ '<select id="activity-type-filter">'
+ '<option value="all">All Events</option>'
+ '<option value="message">Messages</option>'
@@ -2762,12 +2880,17 @@ function renderJobActivity(container, job) {
+ '</select>'
+ '<label class="logs-checkbox"><input type="checkbox" id="activity-autoscroll" checked> Auto-scroll</label>'
+ '</div>'
+ '<div class="activity-terminal" id="activity-terminal"></div>'
+ '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
+ '<div class="activity-terminal" id="activity-terminal"></div>';
if (job && job.can_prompt === true) {
html += '<div class="activity-input-bar" id="activity-input-bar">'
+ '<input type="text" id="activity-prompt-input" placeholder="Send follow-up prompt..." />'
+ '<button id="activity-send-btn">Send</button>'
+ '<button id="activity-done-btn" title="Signal done">Done</button>'
+ '</div>';
}
container.innerHTML = html;
document.getElementById('activity-type-filter').addEventListener('change', applyActivityFilter);
@@ -2776,9 +2899,9 @@ function renderJobActivity(container, job) {
const sendBtn = document.getElementById('activity-send-btn');
const doneBtn = document.getElementById('activity-done-btn');
sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
input.addEventListener('keydown', (e) => {
if (sendBtn) sendBtn.addEventListener('click', () => sendJobPrompt(job.id, false));
if (doneBtn) doneBtn.addEventListener('click', () => sendJobPrompt(job.id, true));
if (input) input.addEventListener('keydown', (e) => {
if (e.key === 'Enter') sendJobPrompt(job.id, false);
});
@@ -3065,7 +3188,11 @@ function renderRoutineDetail(routine) {
function triggerRoutine(id) {
apiFetch('/api/routines/' + id + '/trigger', { method: 'POST' })
.then(() => showToast('Routine triggered', 'success'))
.then(() => {
showToast('Routine triggered', 'success');
if (currentRoutineId === id) openRoutineDetail(id);
else loadRoutines();
})
.catch((err) => showToast('Trigger failed: ' + err.message, 'error'));
}
@@ -3615,7 +3742,7 @@ function formatTimeAgo(epochMs) {
}
function installSkill(nameOrSlug, url, btn) {
var body = { name: nameOrSlug };
var body = { name: nameOrSlug, slug: nameOrSlug };
if (url) body.url = url;
apiFetch('/api/skills/install', {
+51 -1
View File
@@ -2,7 +2,7 @@
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<title>IronClaw</title>
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<link rel="preconnect" href="https://fonts.googleapis.com">
@@ -33,6 +33,48 @@
</div>
</div>
<!-- Restart Confirmation Modal -->
<div id="restart-confirm-modal" class="restart-modal" style="display: none;">
<div class="restart-modal-overlay" onclick="cancelRestart()"></div>
<div class="restart-modal-content">
<div class="restart-modal-header">
<h2>Restart IronClaw Instance</h2>
<button class="restart-modal-close" onclick="cancelRestart()" title="Close">×</button>
</div>
<div class="restart-modal-body">
<p class="restart-modal-description">
Are you sure you want to restart the IronClaw instance? This will gracefully restart the process.
</p>
<div class="restart-modal-warning">
<span class="restart-modal-warning-icon">⚠️</span>
<p>Any in-progress jobs may be interrupted. The restart will complete within a few seconds.</p>
</div>
</div>
<div class="restart-modal-footer">
<button class="restart-modal-btn cancel" onclick="cancelRestart()">Cancel</button>
<button class="restart-modal-btn confirm" onclick="confirmRestart()">Confirm Restart</button>
</div>
</div>
</div>
<!-- Restart Progress Modal -->
<div id="restart-loader" class="restart-loader" style="display: none;">
<div class="restart-loader-overlay"></div>
<div class="restart-loader-content">
<div class="restart-spinner"></div>
<div class="restart-loader-text">
<p class="restart-title">Restarting IronClaw</p>
<p class="restart-subtitle">Please wait while the process restarts...</p>
</div>
<div class="restart-progress-bar">
<div class="restart-progress-fill"></div>
</div>
<p class="restart-modal-info">
Check the Logs tab for details after the restart completes.
</p>
</div>
</div>
<!-- Main App (hidden until authenticated) -->
<div id="app">
<!-- Tab Bar -->
@@ -57,6 +99,14 @@
<span id="sse-status">Connected</span>
<div class="gateway-popover" id="gateway-popover"></div>
</div>
<button class="restart-btn" id="restart-btn" onclick="triggerRestart()" title="Gracefully restart the process">
<svg id="restart-icon" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M23 4v6h-6"></path>
<path d="M1 20v-6h6"></path>
<path d="M3.51 9a9 9 0 0114.85-3.36M20.49 15a9 9 0 01-14.85 3.36"></path>
</svg>
<span>Restart</span>
</button>
</div>
<!-- Chat Tab -->
+287 -38
View File
@@ -30,6 +30,7 @@ body {
background: var(--bg);
color: var(--text);
height: 100vh;
height: 100dvh;
display: flex;
flex-direction: column;
overflow: hidden;
@@ -41,6 +42,7 @@ body {
align-items: center;
justify-content: center;
height: 100vh;
height: 100dvh;
}
.auth-card-login {
@@ -141,6 +143,7 @@ body {
display: none;
flex-direction: column;
height: 100vh;
height: 100dvh;
}
/* Tab Bar */
@@ -256,6 +259,284 @@ body {
white-space: nowrap;
}
/* Restart Button */
.restart-btn {
display: flex;
align-items: center;
gap: 0.375rem;
padding: 0.25rem 0.75rem;
border-radius: 0.5rem;
font-size: 0.8rem;
border: 1px solid;
border-color: #00d894;
color: #00d894;
background-color: transparent;
cursor: pointer;
transition: color 150ms, background-color 150ms, border-color 150ms;
}
.restart-btn:hover:not(:disabled) {
background-color: rgba(0, 216, 148, 0.1);
}
.restart-btn:disabled {
border-color: #333;
color: #666;
cursor: not-allowed;
}
.restart-btn:disabled:hover {
background-color: transparent;
}
.restart-btn svg {
flex-shrink: 0;
width: 13px;
height: 13px;
}
.restart-btn svg.spinning {
animation: spin-icon 1s linear infinite;
}
@keyframes spin-icon {
from { transform: rotate(0deg); }
to { transform: rotate(360deg); }
}
/* Restart Loader Overlay */
.restart-loader {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
z-index: 9999;
display: flex;
align-items: center;
justify-content: center;
}
.restart-loader-overlay {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
backdrop-filter: blur(4px);
z-index: -1;
}
.restart-loader-content {
position: relative;
z-index: 10000;
background-color: #1a1a1a;
border: 1px solid #333;
border-radius: 0.75rem;
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);
width: 100%;
max-width: 28rem;
margin: 0 1rem;
overflow: hidden;
padding: 1.25rem;
}
.restart-spinner {
display: none;
}
.restart-loader-text {
padding: 0;
}
.restart-title {
color: #e0e0e0;
font-size: 0.85rem;
margin-bottom: 1rem;
margin-top: 0;
}
.restart-subtitle {
display: none;
}
/* Restart Modal (Confirmation) */
.restart-modal {
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
z-index: 9999;
display: flex;
align-items: center;
justify-content: center;
}
.restart-modal-overlay {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
backdrop-filter: blur(4px);
}
.restart-modal-content {
position: relative;
z-index: 10000;
background-color: #1a1a1a;
border: 1px solid #333;
border-radius: 0.75rem;
box-shadow: 0 25px 50px -12px rgba(0, 0, 0, 0.25);
width: 100%;
max-width: 28rem;
margin: 0 1rem;
overflow: hidden;
}
.restart-modal-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 1rem 1.25rem;
border-bottom: 1px solid #2a2a2a;
}
.restart-modal-header h2 {
color: #e0e0e0;
font-size: 0.95rem;
margin: 0;
}
.restart-modal-close {
color: #888;
padding: 0.25rem;
border-radius: 0.25rem;
background-color: transparent;
border: none;
cursor: pointer;
transition: color 150ms, background-color 150ms;
display: flex;
align-items: center;
justify-content: center;
}
.restart-modal-close:hover {
color: #ccc;
background-color: #2a2a2a;
}
.restart-modal-body {
padding: 1.25rem;
}
.restart-modal-description {
color: #aaa;
font-size: 0.85rem;
margin: 0;
}
.restart-modal-warning {
margin-top: 1rem;
background-color: #1e1400;
border: 1px solid #3a2a00;
border-radius: 0.5rem;
padding: 0.75rem 1rem;
}
.restart-modal-warning p {
color: #facc15;
font-size: 0.8rem;
margin: 0;
}
.restart-modal-footer {
display: flex;
align-items: center;
justify-content: flex-end;
gap: 0.75rem;
padding: 1rem 1.25rem;
border-top: 1px solid #2a2a2a;
}
.restart-modal-btn {
padding: 0.5rem 1rem;
border-radius: 0.5rem;
font-size: 0.85rem;
border: none;
cursor: pointer;
transition: background-color 150ms;
}
.restart-modal-btn.cancel {
color: #ccc;
background-color: transparent;
}
.restart-modal-btn.cancel:hover {
background-color: #2a2a2a;
}
.restart-modal-btn.confirm {
background-color: #00D894;
color: #111;
}
.restart-modal-btn.confirm:hover {
background-color: #00be82;
}
/* Progress Bar for Restart */
.restart-progress-bar {
width: 100%;
height: 0.375rem;
background-color: #2a2a2a;
border-radius: 9999px;
overflow: hidden;
}
.restart-progress-fill {
height: 100%;
border-radius: 9999px;
background-color: #00D894;
width: 40%;
animation: indeterminate 1.5s ease-in-out infinite;
}
@keyframes indeterminate {
0% {
margin-left: 0;
width: 40%;
}
50% {
margin-left: 60%;
width: 40%;
}
100% {
margin-left: 0;
width: 40%;
}
}
.restart-modal-info {
color: #666;
font-size: 0.8rem;
margin-top: 1.25rem;
margin-bottom: 0;
}
.restart-modal-info a {
color: #00D894;
text-decoration: none;
}
.restart-modal-info a:hover {
text-decoration: underline;
}
.tee-popover {
display: none;
position: absolute;
@@ -550,6 +831,10 @@ body {
border-color: rgba(230, 76, 76, 0.3);
}
.activity-tool-card[data-status="fail"] .activity-tool-name {
color: var(--danger);
}
.activity-tool-header {
display: flex;
align-items: center;
@@ -987,7 +1272,7 @@ body {
/* Chat input */
.chat-input {
display: flex;
padding: 12px 16px;
padding: 12px 16px max(12px, env(safe-area-inset-bottom)) 16px;
gap: 8px;
background: var(--bg-secondary);
border-top: 1px solid var(--border);
@@ -1808,6 +2093,7 @@ body {
.job-files {
display: flex;
height: calc(100vh - 280px);
height: calc(100dvh - 280px);
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
@@ -2312,43 +2598,6 @@ body {
margin-top: 6px;
}
/* Restart overlay */
.restart-overlay {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
background: rgba(0, 0, 0, 0.8);
z-index: 2000;
display: flex;
align-items: center;
justify-content: center;
}
.restart-message {
text-align: center;
color: var(--text);
}
.restart-message h2 {
margin: 16px 0 8px;
}
.restart-message p {
color: var(--text-secondary);
}
.restart-spinner {
width: 40px;
height: 40px;
border: 3px solid var(--border);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.8s linear infinite;
margin: 0 auto;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
+19 -5
View File
@@ -123,6 +123,10 @@ pub enum SseEvent {
name: String,
success: bool,
#[serde(skip_serializing_if = "Option::is_none")]
error: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
parameters: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
thread_id: Option<String>,
},
#[serde(rename = "tool_result")]
@@ -332,6 +336,15 @@ pub struct JobDetailResponse {
#[serde(skip_serializing_if = "Option::is_none")]
pub job_mode: Option<String>,
pub transitions: Vec<TransitionInfo>,
/// Whether this job can be restarted from the UI.
#[serde(default)]
pub can_restart: bool,
/// Whether follow-up prompts can be sent to this job.
#[serde(default)]
pub can_prompt: bool,
/// The kind of job: "sandbox" or "agent".
#[serde(skip_serializing_if = "Option::is_none")]
pub job_kind: Option<String>,
}
// --- Project Files ---
@@ -379,6 +392,9 @@ pub struct ExtensionInfo {
/// Whether this extension has configurable secrets (setup schema).
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
#[serde(default)]
pub has_auth: bool,
/// WASM channel activation status: "installed", "configured", "active", "failed".
#[serde(skip_serializing_if = "Option::is_none")]
pub activation_status: Option<String>,
@@ -451,9 +467,6 @@ pub struct ActionResponse {
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a gateway restart is needed (activation failed).
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
}
impl ActionResponse {
@@ -465,7 +478,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
@@ -477,7 +489,6 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
}
}
}
@@ -562,6 +573,9 @@ pub struct SkillSearchResponse {
#[derive(Debug, Deserialize)]
pub struct SkillInstallRequest {
pub name: String,
/// Registry slug (e.g. "owner/skill-name"). Preferred over `name` for
/// constructing the download URL when fetching from ClawHub.
pub slug: Option<String>,
pub url: Option<String>,
pub content: Option<String>,
}
+5 -5
View File
@@ -242,7 +242,7 @@ async fn handle_client_message(
} => {
if let Some(ref ext_mgr) = state.extension_manager {
match ext_mgr.auth(&extension_name, Some(&token)).await {
Ok(result) if result.status == "authenticated" => {
Ok(result) if result.is_authenticated() => {
let msg = match ext_mgr.activate(&extension_name).await {
Ok(r) => format!(
"{} authenticated ({} tools loaded)",
@@ -268,9 +268,9 @@ async fn handle_client_message(
.sse
.broadcast(crate::channels::web::types::SseEvent::AuthRequired {
extension_name,
instructions: result.instructions,
auth_url: result.auth_url,
setup_url: result.setup_url,
instructions: result.instructions().map(String::from),
auth_url: result.auth_url().map(String::from),
setup_url: result.setup_url().map(String::from),
});
}
Err(e) => {
@@ -483,6 +483,7 @@ mod tests {
store: None,
job_manager: None,
prompt_queue: None,
scheduler: None,
user_id: "test".to_string(),
shutdown_tx: tokio::sync::RwLock::new(None),
ws_tracker: Some(Arc::new(WsConnectionTracker::new())),
@@ -493,7 +494,6 @@ mod tests {
registry_entries: Vec::new(),
cost_guard: None,
startup_time: std::time::Instant::now(),
restart_requested: std::sync::atomic::AtomicBool::new(false),
}
}
}
+812 -13
View File
@@ -17,10 +17,18 @@
//! - **Runtime**: Users can set GOOGLE_OAUTH_CLIENT_ID / GOOGLE_OAUTH_CLIENT_SECRET
//! env vars, which take priority over built-in defaults.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore;
use sha2::{Digest, Sha256};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener;
use tokio::sync::RwLock;
use crate::secrets::{CreateSecretParams, SecretsStore};
// ── Built-in credentials ────────────────────────────────────────────────
@@ -121,6 +129,9 @@ pub enum OAuthCallbackError {
#[error("Timed out waiting for authorization")]
Timeout,
#[error("CSRF state mismatch: expected {expected}, got {actual}")]
StateMismatch { expected: String, actual: String },
#[error("IO error: {0}")]
Io(String),
}
@@ -177,16 +188,22 @@ pub async fn bind_callback_listener() -> Result<TcpListener, OAuthCallbackError>
/// extracts the value of `param_name` (e.g., "code" or "token"), and shows a branded
/// landing page using `display_name` (e.g., "Google", "Notion", "NEAR AI").
///
/// When `expected_state` is `Some`, the callback's `state` query parameter is validated
/// against it to prevent CSRF attacks. If the state doesn't match, the callback is
/// rejected with an error page.
///
/// Times out after 5 minutes.
pub async fn wait_for_callback(
listener: TcpListener,
path_prefix: &str,
param_name: &str,
display_name: &str,
expected_state: Option<&str>,
) -> Result<String, OAuthCallbackError> {
let path_prefix = path_prefix.to_string();
let param_name = param_name.to_string();
let display_name = display_name.to_string();
let expected_state = expected_state.map(String::from);
tokio::time::timeout(Duration::from_secs(300), async move {
loop {
@@ -221,17 +238,29 @@ pub async fn wait_for_callback(
return Err(OAuthCallbackError::Denied);
}
// Look for the target parameter
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == param_name {
let value = urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned();
// Parse all query params into a map for validation
let params: HashMap<&str, String> = query
.split('&')
.filter_map(|p| {
let mut parts = p.splitn(2, '=');
let key = parts.next()?;
let val = parts.next().unwrap_or("");
Some((
key,
urlencoding::decode(val)
.unwrap_or_else(|_| val.into())
.into_owned(),
))
})
.collect();
let html = landing_html(&display_name, true);
// Validate CSRF state parameter
if let Some(ref expected) = expected_state {
let actual = params.get("state").cloned().unwrap_or_default();
if actual != *expected {
let html = landing_html(&display_name, false);
let response = format!(
"HTTP/1.1 200 OK\r\n\
"HTTP/1.1 403 Forbidden\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
@@ -239,11 +268,29 @@ pub async fn wait_for_callback(
html
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(value);
return Err(OAuthCallbackError::StateMismatch {
expected: expected.clone(),
actual,
});
}
}
// Look for the target parameter
if let Some(value) = params.get(param_name.as_str()) {
let html = landing_html(&display_name, true);
let response = format!(
"HTTP/1.1 200 OK\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
{}",
html
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(value.clone());
}
}
// Not the callback we're looking for
@@ -271,7 +318,288 @@ fn html_escape(s: &str) -> String {
out
}
/// HTML landing page shown in the browser after an OAuth redirect.
// ── Shared OAuth flow steps ─────────────────────────────────────────
/// Response from the OAuth token exchange.
pub struct OAuthTokenResponse {
pub access_token: String,
pub refresh_token: Option<String>,
pub expires_in: Option<u64>,
}
/// Result of building an OAuth 2.0 authorization URL.
pub struct OAuthUrlResult {
/// The full authorization URL to redirect the user to.
pub url: String,
/// PKCE code verifier (must be sent with the token exchange request).
pub code_verifier: Option<String>,
/// Random state parameter for CSRF protection (must be validated in callback).
pub state: String,
}
/// Build an OAuth 2.0 authorization URL with optional PKCE and CSRF state.
///
/// Returns an `OAuthUrlResult` containing the authorization URL, optional PKCE
/// code verifier, and a random `state` parameter for CSRF protection. The caller
/// must validate the `state` value in the callback before exchanging the code.
pub fn build_oauth_url(
authorization_url: &str,
client_id: &str,
redirect_uri: &str,
scopes: &[String],
use_pkce: bool,
extra_params: &HashMap<String, String>,
) -> OAuthUrlResult {
// Generate PKCE verifier and challenge
let (code_verifier, code_challenge) = if use_pkce {
let mut verifier_bytes = [0u8; 32];
rand::rngs::OsRng.fill_bytes(&mut verifier_bytes);
let verifier = URL_SAFE_NO_PAD.encode(verifier_bytes);
let mut hasher = Sha256::new();
hasher.update(verifier.as_bytes());
let challenge = URL_SAFE_NO_PAD.encode(hasher.finalize());
(Some(verifier), Some(challenge))
} else {
(None, None)
};
// Generate random state for CSRF protection
let mut state_bytes = [0u8; 32];
rand::rngs::OsRng.fill_bytes(&mut state_bytes);
let state = URL_SAFE_NO_PAD.encode(state_bytes);
// Build authorization URL
let mut auth_url = format!(
"{}?client_id={}&response_type=code&redirect_uri={}&state={}",
authorization_url,
urlencoding::encode(client_id),
urlencoding::encode(redirect_uri),
urlencoding::encode(&state),
);
if !scopes.is_empty() {
auth_url.push_str(&format!(
"&scope={}",
urlencoding::encode(&scopes.join(" "))
));
}
if let Some(ref challenge) = code_challenge {
auth_url.push_str(&format!(
"&code_challenge={}&code_challenge_method=S256",
challenge
));
}
for (key, value) in extra_params {
auth_url.push_str(&format!(
"&{}={}",
urlencoding::encode(key),
urlencoding::encode(value)
));
}
OAuthUrlResult {
url: auth_url,
code_verifier,
state,
}
}
/// Exchange an OAuth authorization code for tokens.
///
/// POSTs to `token_url` with the authorization code and optional PKCE verifier.
/// If `client_secret` is provided, uses HTTP Basic auth; otherwise includes
/// `client_id` in the form body (for public clients).
pub async fn exchange_oauth_code(
token_url: &str,
client_id: &str,
client_secret: Option<&str>,
code: &str,
redirect_uri: &str,
code_verifier: Option<&str>,
access_token_field: &str,
) -> Result<OAuthTokenResponse, OAuthCallbackError> {
let client = reqwest::Client::new();
let mut token_params = vec![
("grant_type", "authorization_code".to_string()),
("code", code.to_string()),
("redirect_uri", redirect_uri.to_string()),
];
if let Some(verifier) = code_verifier {
token_params.push(("code_verifier", verifier.to_string()));
}
let mut request = client.post(token_url);
if let Some(secret) = client_secret {
request = request.basic_auth(client_id, Some(secret));
} else {
token_params.push(("client_id", client_id.to_string()));
}
let token_response = request
.form(&token_params)
.send()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Token exchange request failed: {}", e)))?;
if !token_response.status().is_success() {
let status = token_response.status();
let body = token_response.text().await.unwrap_or_default();
return Err(OAuthCallbackError::Io(format!(
"Token exchange failed: {} - {}",
status, body
)));
}
let token_data: serde_json::Value = token_response
.json()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to parse token response: {}", e)))?;
let access_token = token_data
.get(access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
// Log only the field names present, not values (which may contain tokens)
let fields: Vec<&str> = token_data
.as_object()
.map(|o| o.keys().map(|k| k.as_str()).collect())
.unwrap_or_default();
OAuthCallbackError::Io(format!(
"No '{}' field in token response (fields present: {:?})",
access_token_field, fields
))
})?
.to_string();
let refresh_token = token_data
.get("refresh_token")
.and_then(|v| v.as_str())
.map(String::from);
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
Ok(OAuthTokenResponse {
access_token,
refresh_token,
expires_in,
})
}
/// Store OAuth tokens (access + refresh) in the secrets store.
///
/// Also stores the granted scopes as `{secret_name}_scopes` so that scope
/// expansion can be detected on subsequent activations.
#[allow(clippy::too_many_arguments)]
pub async fn store_oauth_tokens(
store: &(dyn SecretsStore + Send + Sync),
user_id: &str,
secret_name: &str,
provider: Option<&str>,
access_token: &str,
refresh_token: Option<&str>,
expires_in: Option<u64>,
scopes: &[String],
) -> Result<(), OAuthCallbackError> {
let mut params = CreateSecretParams::new(secret_name, access_token);
if let Some(prov) = provider {
params = params.with_provider(prov);
}
if let Some(secs) = expires_in {
let expires_at = chrono::Utc::now() + chrono::Duration::seconds(secs as i64);
params = params.with_expiry(expires_at);
}
store
.create(user_id, params)
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to save token: {}", e)))?;
// Store refresh token separately (no expiry, it's long-lived)
if let Some(rt) = refresh_token {
let refresh_name = format!("{}_refresh_token", secret_name);
let mut refresh_params = CreateSecretParams::new(&refresh_name, rt);
if let Some(prov) = provider {
refresh_params = refresh_params.with_provider(prov);
}
store
.create(user_id, refresh_params)
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to save refresh token: {}", e)))?;
}
// Store granted scopes for scope expansion detection
if !scopes.is_empty() {
let scopes_name = format!("{}_scopes", secret_name);
let scopes_value = scopes.join(" ");
let scopes_params = CreateSecretParams::new(&scopes_name, &scopes_value);
// Best-effort: scope tracking failure shouldn't block auth
let _ = store.create(user_id, scopes_params).await;
}
Ok(())
}
/// Validate an OAuth token against a tool's validation endpoint.
///
/// Sends a request to the configured endpoint with the token as a Bearer header.
/// Returns `Ok(())` if the response status matches the expected success status,
/// or an error with details if validation fails (wrong account, expired token, etc.).
pub async fn validate_oauth_token(
token: &str,
validation: &crate::tools::wasm::ValidationEndpointSchema,
) -> Result<(), OAuthCallbackError> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(10))
.build()
.map_err(|e| OAuthCallbackError::Io(format!("Failed to build HTTP client: {}", e)))?;
let request = match validation.method.to_uppercase().as_str() {
"POST" => client.post(&validation.url),
_ => client.get(&validation.url),
};
let mut request = request.header("Authorization", format!("Bearer {}", token));
// Add custom headers from the validation schema (e.g., Notion-Version)
for (key, value) in &validation.headers {
request = request.header(key, value);
}
let response = request
.send()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Validation request failed: {}", e)))?;
if response.status().as_u16() == validation.success_status {
Ok(())
} else {
let status = response.status();
let body = response.text().await.unwrap_or_default();
let truncated: String = if body.len() > 200 {
let mut end = 200;
while end > 0 && !body.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &body[..end])
} else {
body
};
Err(OAuthCallbackError::Io(format!(
"Token validation failed: HTTP {} (expected {}): {}",
status, validation.success_status, truncated
)))
}
}
// ── Landing pages ───────────────────────────────────────────────────
pub fn landing_html(provider_name: &str, success: bool) -> String {
let safe_name = html_escape(provider_name);
let (icon, heading, subtitle, accent) = if success {
@@ -357,6 +685,219 @@ pub fn landing_html(provider_name: &str, success: bool) -> String {
)
}
// ── Gateway callback support ─────────────────────────────────────────
/// State for an in-progress OAuth flow, keyed by CSRF `state` parameter.
///
/// Created by `start_wasm_oauth()` and consumed by the web gateway's
/// `/oauth/callback` handler when running in hosted mode.
pub struct PendingOAuthFlow {
/// Extension name (e.g., "google_calendar").
pub extension_name: String,
/// Human-readable display name (e.g., "Google Calendar").
pub display_name: String,
/// OAuth token exchange URL.
pub token_url: String,
/// OAuth client ID.
pub client_id: String,
/// OAuth client secret (optional for PKCE-only flows).
pub client_secret: Option<String>,
/// The redirect_uri used in the authorization request.
pub redirect_uri: String,
/// PKCE code verifier (must match the code_challenge sent in the auth URL).
pub code_verifier: Option<String>,
/// Field name in token response containing the access token.
pub access_token_field: String,
/// Secret name for storage (e.g., "google_oauth_token").
pub secret_name: String,
/// Provider hint (e.g., "google").
pub provider: Option<String>,
/// Token validation endpoint (optional).
pub validation_endpoint: Option<crate::tools::wasm::ValidationEndpointSchema>,
/// Scopes that were requested.
pub scopes: Vec<String>,
/// User ID for secret storage.
pub user_id: String,
/// Secrets store reference for token persistence.
pub secrets: Arc<dyn SecretsStore + Send + Sync>,
/// SSE broadcast sender for notifying the web UI.
pub sse_sender: Option<tokio::sync::broadcast::Sender<crate::channels::web::types::SseEvent>>,
/// Gateway auth token for authenticating with the platform token exchange proxy.
pub gateway_token: Option<String>,
/// When this flow was created (for expiry).
pub created_at: std::time::Instant,
}
impl std::fmt::Debug for PendingOAuthFlow {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("PendingOAuthFlow")
.field("extension_name", &self.extension_name)
.field("display_name", &self.display_name)
.field("secret_name", &self.secret_name)
.field("created_at", &self.created_at)
.finish_non_exhaustive()
}
}
/// Thread-safe registry of pending OAuth flows, keyed by CSRF `state` parameter.
pub type PendingOAuthRegistry = Arc<RwLock<HashMap<String, PendingOAuthFlow>>>;
/// Create a new empty pending OAuth flow registry.
pub fn new_pending_oauth_registry() -> PendingOAuthRegistry {
Arc::new(RwLock::new(HashMap::new()))
}
/// Returns `true` if OAuth callbacks should be routed through the web gateway
/// instead of the local TCP listener.
///
/// This is the case when `IRONCLAW_OAUTH_CALLBACK_URL` is set to a non-loopback
/// URL, meaning the user's browser will redirect to a hosted gateway rather than
/// localhost.
pub fn use_gateway_callback() -> bool {
std::env::var("IRONCLAW_OAUTH_CALLBACK_URL")
.ok()
.filter(|v| !v.is_empty())
.map(|raw| {
url::Url::parse(&raw)
.ok()
.and_then(|u| u.host_str().map(String::from))
.map(|host| !is_loopback_host(&host))
.unwrap_or(false)
})
.unwrap_or(false)
}
/// Maximum age for pending OAuth flows (5 minutes, matching TCP listener timeout).
pub const OAUTH_FLOW_EXPIRY: Duration = Duration::from_secs(300);
/// Remove expired flows from the registry.
///
/// Called when inserting new flows to prevent accumulation from abandoned
/// OAuth attempts.
pub async fn sweep_expired_flows(registry: &PendingOAuthRegistry) {
let mut flows = registry.write().await;
flows.retain(|_, flow| flow.created_at.elapsed() < OAUTH_FLOW_EXPIRY);
}
// ── Platform routing helpers ────────────────────────────────────────
/// Prepend instance name to CSRF state for platform routing.
///
/// The NEAR AI platform nginx proxy at `auth.DOMAIN` parses the instance name
/// from the `state` query parameter (format: `instance:nonce`) to route the
/// OAuth callback to the correct container.
///
/// Returns the nonce unchanged when `IRONCLAW_INSTANCE_NAME` is not set
/// (local/non-platform mode).
pub fn build_platform_state(nonce: &str) -> String {
let instance = std::env::var("IRONCLAW_INSTANCE_NAME")
.or_else(|_| std::env::var("OPENCLAW_INSTANCE_NAME"))
.ok()
.filter(|v| !v.is_empty());
match instance {
Some(name) => format!("{}:{}", name, nonce),
None => nonce.to_string(),
}
}
/// Strip the instance prefix from a state parameter to recover the lookup nonce.
///
/// `"myinstance:abc123"` → `"abc123"`, `"abc123"` → `"abc123"` (no prefix).
///
/// Safe because nonces are base64url-encoded (`[A-Za-z0-9_-]`, no colons).
pub fn strip_instance_prefix(state: &str) -> &str {
state
.split_once(':')
.map(|(_, nonce)| nonce)
.unwrap_or(state)
}
/// Exchange an OAuth authorization code via the platform's token exchange proxy.
///
/// The proxy holds `client_secret` server-side so the container never sees it.
/// Authenticated via the gateway auth token (Bearer header).
///
/// The proxy expects form params `{code, redirect_uri, code_verifier}` and
/// returns a standard Google token response `{access_token, refresh_token, expires_in}`.
pub async fn exchange_via_proxy(
proxy_url: &str,
gateway_token: &str,
code: &str,
redirect_uri: &str,
code_verifier: Option<&str>,
access_token_field: &str,
) -> Result<OAuthTokenResponse, OAuthCallbackError> {
if gateway_token.is_empty() {
return Err(OAuthCallbackError::Io(
"Gateway auth token is required for proxy token exchange".to_string(),
));
}
let exchange_url = format!("{}/oauth/exchange", proxy_url.trim_end_matches('/'));
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(60))
.build()
.map_err(|e| OAuthCallbackError::Io(format!("Failed to build HTTP client: {}", e)))?;
let mut params = vec![
("code", code.to_string()),
("redirect_uri", redirect_uri.to_string()),
];
if let Some(verifier) = code_verifier {
params.push(("code_verifier", verifier.to_string()));
}
let response = client
.post(&exchange_url)
.bearer_auth(gateway_token)
.form(&params)
.send()
.await
.map_err(|e| {
OAuthCallbackError::Io(format!("Token exchange proxy request failed: {}", e))
})?;
if !response.status().is_success() {
let status = response.status();
let body = response.text().await.unwrap_or_default();
return Err(OAuthCallbackError::Io(format!(
"Token exchange proxy failed: {} - {}",
status, body
)));
}
let token_data: serde_json::Value = response
.json()
.await
.map_err(|e| OAuthCallbackError::Io(format!("Failed to parse proxy response: {}", e)))?;
let access_token = token_data
.get(access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
let fields: Vec<&str> = token_data
.as_object()
.map(|o| o.keys().map(|k| k.as_str()).collect())
.unwrap_or_default();
OAuthCallbackError::Io(format!(
"No '{}' field in proxy response (fields present: {:?})",
access_token_field, fields
))
})?
.to_string();
let refresh_token = token_data
.get("refresh_token")
.and_then(|v| v.as_str())
.map(String::from);
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
Ok(OAuthTokenResponse {
access_token,
refresh_token,
expires_in,
})
}
#[cfg(test)]
mod tests {
use std::sync::Mutex;
@@ -512,4 +1053,262 @@ mod tests {
assert!(html.contains("#ef4444")); // red accent
assert!(!html.contains("Connected"));
}
#[test]
fn test_build_oauth_url_basic() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result = build_oauth_url(
"https://accounts.google.com/o/oauth2/auth",
"my-client-id",
"http://localhost:9876/callback",
&["openid".to_string(), "email".to_string()],
false,
&HashMap::new(),
);
assert!(
result
.url
.starts_with("https://accounts.google.com/o/oauth2/auth?")
);
assert!(result.url.contains("client_id=my-client-id"));
assert!(result.url.contains("response_type=code"));
assert!(result.url.contains("redirect_uri="));
assert!(result.url.contains("scope=openid%20email"));
assert!(result.url.contains("state="));
assert!(result.code_verifier.is_none());
assert!(!result.state.is_empty());
}
#[test]
fn test_build_oauth_url_with_pkce() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result = build_oauth_url(
"https://auth.example.com/authorize",
"client-123",
"http://localhost:9876/callback",
&[],
true,
&HashMap::new(),
);
assert!(result.url.contains("code_challenge="));
assert!(result.url.contains("code_challenge_method=S256"));
assert!(result.code_verifier.is_some());
let verifier = result.code_verifier.unwrap();
assert!(!verifier.is_empty());
}
#[test]
fn test_build_oauth_url_with_extra_params() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let mut extra = HashMap::new();
extra.insert("access_type".to_string(), "offline".to_string());
extra.insert("prompt".to_string(), "consent".to_string());
let result = build_oauth_url(
"https://auth.example.com/authorize",
"client-123",
"http://localhost:9876/callback",
&["read".to_string()],
false,
&extra,
);
assert!(result.url.contains("access_type=offline"));
assert!(result.url.contains("prompt=consent"));
}
#[test]
fn test_build_oauth_url_state_is_unique() {
use std::collections::HashMap;
use crate::cli::oauth_defaults::build_oauth_url;
let result1 = build_oauth_url(
"https://auth.example.com/authorize",
"client",
"http://localhost:9876/callback",
&[],
false,
&HashMap::new(),
);
let result2 = build_oauth_url(
"https://auth.example.com/authorize",
"client",
"http://localhost:9876/callback",
&[],
false,
&HashMap::new(),
);
// State should be different each time (random)
assert_ne!(result1.state, result2.state);
}
#[test]
fn test_use_gateway_callback_false_by_default() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
}
}
}
#[test]
fn test_use_gateway_callback_true_for_hosted() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var(
"IRONCLAW_OAUTH_CALLBACK_URL",
"https://kind-deer.agent1.near.ai",
);
}
assert!(crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_use_gateway_callback_false_for_localhost() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", "http://127.0.0.1:3001");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_use_gateway_callback_false_for_empty() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_OAUTH_CALLBACK_URL").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", "");
}
assert!(!crate::cli::oauth_defaults::use_gateway_callback());
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_OAUTH_CALLBACK_URL", val);
} else {
std::env::remove_var("IRONCLAW_OAUTH_CALLBACK_URL");
}
}
}
#[test]
fn test_build_platform_state_with_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::set_var("IRONCLAW_INSTANCE_NAME", "kind-deer");
}
assert_eq!(build_platform_state("abc123"), "kind-deer:abc123");
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
} else {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
}
}
}
#[test]
fn test_build_platform_state_without_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
let original_oc = std::env::var("OPENCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
std::env::remove_var("OPENCLAW_INSTANCE_NAME");
}
assert_eq!(build_platform_state("abc123"), "abc123");
unsafe {
if let Some(val) = original {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
}
if let Some(val) = original_oc {
std::env::set_var("OPENCLAW_INSTANCE_NAME", val);
}
}
}
#[test]
fn test_build_platform_state_with_openclaw_instance() {
use crate::cli::oauth_defaults::build_platform_state;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
let original_ic = std::env::var("IRONCLAW_INSTANCE_NAME").ok();
let original_oc = std::env::var("OPENCLAW_INSTANCE_NAME").ok();
// SAFETY: Under ENV_MUTEX, no concurrent env access.
unsafe {
std::env::remove_var("IRONCLAW_INSTANCE_NAME");
std::env::set_var("OPENCLAW_INSTANCE_NAME", "quiet-lion");
}
assert_eq!(build_platform_state("xyz789"), "quiet-lion:xyz789");
unsafe {
if let Some(val) = original_ic {
std::env::set_var("IRONCLAW_INSTANCE_NAME", val);
}
if let Some(val) = original_oc {
std::env::set_var("OPENCLAW_INSTANCE_NAME", val);
} else {
std::env::remove_var("OPENCLAW_INSTANCE_NAME");
}
}
}
#[test]
fn test_strip_instance_prefix_with_colon() {
use crate::cli::oauth_defaults::strip_instance_prefix;
assert_eq!(strip_instance_prefix("kind-deer:abc123"), "abc123");
assert_eq!(strip_instance_prefix("my-instance:xyz"), "xyz");
}
#[test]
fn test_strip_instance_prefix_without_colon() {
use crate::cli::oauth_defaults::strip_instance_prefix;
assert_eq!(strip_instance_prefix("abc123"), "abc123");
assert_eq!(strip_instance_prefix(""), "");
}
}
+58 -184
View File
@@ -782,11 +782,7 @@ async fn auth_tool_oauth(
auth: &crate::tools::wasm::AuthCapabilitySchema,
oauth: &crate::tools::wasm::OAuthConfigSchema,
) -> anyhow::Result<()> {
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore;
use sha2::{Digest, Sha256};
use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
use crate::cli::oauth_defaults;
let display_name = auth.display_name.as_deref().unwrap_or(&auth.secret_name);
@@ -827,142 +823,69 @@ async fn auth_tool_oauth(
println!();
let listener = oauth_defaults::bind_callback_listener().await?;
let redirect_uri = format!("http://localhost:{}/callback", OAUTH_CALLBACK_PORT);
let redirect_uri = format!("{}/callback", oauth_defaults::callback_url());
// Generate PKCE verifier and challenge
let (code_verifier, code_challenge) = if oauth.use_pkce {
let mut verifier_bytes = [0u8; 32];
rand::thread_rng().fill_bytes(&mut verifier_bytes);
let verifier = URL_SAFE_NO_PAD.encode(verifier_bytes);
let mut hasher = Sha256::new();
hasher.update(verifier.as_bytes());
let challenge = URL_SAFE_NO_PAD.encode(hasher.finalize());
(Some(verifier), Some(challenge))
} else {
(None, None)
};
// Build authorization URL
let mut auth_url = format!(
"{}?client_id={}&response_type=code&redirect_uri={}",
oauth.authorization_url,
urlencoding::encode(&client_id),
urlencoding::encode(&redirect_uri)
// Build authorization URL with PKCE and CSRF state
let oauth_result = oauth_defaults::build_oauth_url(
&oauth.authorization_url,
&client_id,
&redirect_uri,
&oauth.scopes,
oauth.use_pkce,
&oauth.extra_params,
);
if !oauth.scopes.is_empty() {
auth_url.push_str(&format!(
"&scope={}",
urlencoding::encode(&oauth.scopes.join(" "))
));
}
if let Some(ref challenge) = code_challenge {
auth_url.push_str(&format!(
"&code_challenge={}&code_challenge_method=S256",
challenge
));
}
// Add extra params
for (key, value) in &oauth.extra_params {
auth_url.push_str(&format!(
"&{}={}",
urlencoding::encode(key),
urlencoding::encode(value)
));
}
let code_verifier = oauth_result.code_verifier;
println!(" Opening browser for {} login...", display_name);
println!();
if let Err(e) = open::that(&auth_url) {
if let Err(e) = open::that(&oauth_result.url) {
println!(" Could not open browser: {}", e);
println!(" Please open this URL manually:");
println!(" {}", auth_url);
println!(" {}", oauth_result.url);
}
println!(" Waiting for authorization...");
let code =
oauth_defaults::wait_for_callback(listener, "/callback", "code", display_name).await?;
let code = oauth_defaults::wait_for_callback(
listener,
"/callback",
"code",
display_name,
Some(&oauth_result.state),
)
.await?;
println!();
println!(" Exchanging code for token...");
// Exchange code for token
let client = reqwest::Client::new();
let mut token_params = vec![
("grant_type", "authorization_code".to_string()),
("code", code),
("redirect_uri", redirect_uri),
];
if let Some(ref verifier) = code_verifier {
token_params.push(("code_verifier", verifier.to_string()));
}
// Build token request
let mut request = client.post(&oauth.token_url);
// Use Basic auth if client_secret is provided, otherwise include client_id in body
if let Some(ref secret) = client_secret {
request = request.basic_auth(&client_id, Some(secret));
} else {
token_params.push(("client_id", client_id));
}
let token_response = request.form(&token_params).send().await?;
if !token_response.status().is_success() {
let status = token_response.status();
let body = token_response.text().await.unwrap_or_default();
return Err(anyhow::anyhow!(
"Token exchange failed: {} - {}",
status,
body
));
}
let token_data: serde_json::Value = token_response.json().await?;
let access_token = token_data
.get(&oauth.access_token_field)
.and_then(|v| v.as_str())
.ok_or_else(|| {
anyhow::anyhow!(
"No {} in token response: {:?}",
oauth.access_token_field,
token_data
)
})?;
let refresh_token = token_data.get("refresh_token").and_then(|v| v.as_str());
let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
// Save the token (with refresh token and expiry if provided)
save_token(
store,
user_id,
auth,
access_token,
refresh_token,
expires_in,
let token_response = oauth_defaults::exchange_oauth_code(
&oauth.token_url,
&client_id,
client_secret.as_deref(),
&code,
&redirect_uri,
code_verifier.as_deref(),
&oauth.access_token_field,
)
.await?;
// Extract any additional info for display
let workspace_name = token_data
.get("workspace_name")
.and_then(|v| v.as_str())
.or_else(|| token_data.get("team_name").and_then(|v| v.as_str()));
// Save tokens (access + refresh + scopes)
oauth_defaults::store_oauth_tokens(
store,
user_id,
&auth.secret_name,
auth.provider.as_deref(),
&token_response.access_token,
token_response.refresh_token.as_deref(),
token_response.expires_in,
&oauth.scopes,
)
.await?;
println!();
println!("{} connected!", display_name);
if let Some(workspace) = workspace_name {
println!(" Workspace: {}", workspace);
}
println!();
println!(" The tool can now access the API.");
println!();
@@ -1107,46 +1030,15 @@ async fn validate_token(
validation: &crate::tools::wasm::ValidationEndpointSchema,
_secret_name: &str,
) -> anyhow::Result<()> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()?;
// Build request based on method
let request = match validation.method.to_uppercase().as_str() {
"GET" => client.get(&validation.url),
"POST" => client.post(&validation.url),
_ => client.get(&validation.url),
};
// Add authorization header (assume Bearer for now, could be extended)
let response = request
.header("Authorization", format!("Bearer {}", token))
.header("Notion-Version", "2022-06-28") // Notion-specific, but harmless for others
.send()
.await?;
if response.status().as_u16() == validation.success_status {
Ok(())
} else {
let status = response.status();
let body = response.text().await.unwrap_or_default();
Err(anyhow::anyhow!(
"HTTP {} (expected {}): {}",
status,
validation.success_status,
if body.len() > 100 {
format!("{}...", &body[..100])
} else {
body
}
))
}
crate::cli::oauth_defaults::validate_oauth_token(token, validation)
.await
.map_err(|e| anyhow::anyhow!("{}", e))
}
/// Save token to secrets store.
///
/// Optionally stores a refresh token (as `{secret_name}_refresh_token`) and
/// sets `expires_at` on the access token so the runtime can auto-refresh.
/// Delegates to the shared `store_oauth_tokens` for OAuth tokens, or stores
/// directly for manual/env-var tokens (no scopes or refresh token).
async fn save_token(
store: &(dyn SecretsStore + Send + Sync),
user_id: &str,
@@ -1155,36 +1047,18 @@ async fn save_token(
refresh_token: Option<&str>,
expires_in: Option<u64>,
) -> anyhow::Result<()> {
let mut params = CreateSecretParams::new(&auth.secret_name, token);
if let Some(ref provider) = auth.provider {
params = params.with_provider(provider);
}
if let Some(secs) = expires_in {
let expires_at = chrono::Utc::now() + chrono::Duration::seconds(secs as i64);
params = params.with_expiry(expires_at);
}
store
.create(user_id, params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save token: {}", e))?;
// Store refresh token separately (no expiry, it's long-lived)
if let Some(rt) = refresh_token {
let refresh_name = format!("{}_refresh_token", auth.secret_name);
let mut refresh_params = CreateSecretParams::new(&refresh_name, rt);
if let Some(ref provider) = auth.provider {
refresh_params = refresh_params.with_provider(provider);
}
store
.create(user_id, refresh_params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save refresh token: {}", e))?;
}
Ok(())
crate::cli::oauth_defaults::store_oauth_tokens(
store,
user_id,
&auth.secret_name,
auth.provider.as_deref(),
token,
refresh_token,
expires_in,
&[], // No scopes for manual/env-var tokens
)
.await
.map_err(|e| anyhow::anyhow!("{}", e))
}
/// Print success message.
+20
View File
@@ -30,6 +30,26 @@ pub struct AgentConfig {
}
impl AgentConfig {
/// Create a test-friendly config without reading env vars.
#[cfg(feature = "libsql")]
pub fn for_testing() -> Self {
Self {
name: "test-rig".to_string(),
max_parallel_jobs: 1,
job_timeout: Duration::from_secs(30),
stuck_threshold: Duration::from_secs(300),
repair_check_interval: Duration::from_secs(3600),
max_repair_attempts: 0,
use_planning: false,
session_idle_timeout: Duration::from_secs(3600),
allow_local_tools: true,
max_cost_per_day_cents: None,
max_actions_per_hour: None,
max_tool_iterations: 10,
auto_approve_tools: true,
}
}
pub(crate) fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
name: parse_optional_env("AGENT_NAME", settings.agent.name.clone())?,
+18 -10
View File
@@ -1,3 +1,4 @@
use std::collections::HashMap;
use std::path::PathBuf;
use secrecy::SecretString;
@@ -18,8 +19,9 @@ pub struct ChannelsConfig {
pub wasm_channels_dir: std::path::PathBuf,
/// Whether WASM channels are enabled.
pub wasm_channels_enabled: bool,
/// Telegram owner user ID. When set, the bot only responds to this user.
pub telegram_owner_id: Option<i64>,
/// Per-channel owner user IDs. When set, the channel only responds to this user.
/// Key: channel name (e.g., "telegram"), Value: owner user ID.
pub wasm_channel_owner_ids: HashMap<String, i64>,
}
#[derive(Debug, Clone)]
@@ -180,14 +182,20 @@ impl ChannelsConfig {
.map(PathBuf::from)
.unwrap_or_else(default_channels_dir),
wasm_channels_enabled: parse_bool_env("WASM_CHANNELS_ENABLED", true)?,
telegram_owner_id: optional_env("TELEGRAM_OWNER_ID")?
.map(|s| s.parse())
.transpose()
.map_err(|e: std::num::ParseIntError| ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
})?
.or(settings.channels.telegram_owner_id),
wasm_channel_owner_ids: {
let mut ids = settings.channels.wasm_channel_owner_ids.clone();
// Backwards compat: TELEGRAM_OWNER_ID env var
if let Some(id_str) = optional_env("TELEGRAM_OWNER_ID")? {
let id: i64 = id_str.parse().map_err(|e: std::num::ParseIntError| {
ConfigError::InvalidValue {
key: "TELEGRAM_OWNER_ID".to_string(),
message: format!("must be an integer: {e}"),
}
})?;
ids.insert("telegram".to_string(), id);
}
ids
},
})
}
}
+34
View File
@@ -195,6 +195,40 @@ pub struct NearAiConfig {
}
impl LlmConfig {
/// Create a test-friendly config without reading env vars.
///
/// Uses NearAi backend with dummy values. The LLM provider is replaced
/// by `TraceLlm` via `AppBuilder::with_llm()`, so these values are unused.
#[cfg(feature = "libsql")]
pub fn for_testing() -> Self {
Self {
backend: LlmBackend::NearAi,
nearai: NearAiConfig {
model: "test-model".to_string(),
cheap_model: None,
base_url: "http://localhost:0".to_string(),
auth_base_url: "http://localhost:0".to_string(),
session_path: PathBuf::from("/tmp/ironclaw-test-session.json"),
api_key: None,
fallback_model: None,
max_retries: 0,
circuit_breaker_threshold: None,
circuit_breaker_recovery_secs: 30,
response_cache_enabled: false,
response_cache_ttl_secs: 3600,
response_cache_max_entries: 100,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
smart_routing_cascade: false,
},
openai: None,
anthropic: None,
ollama: None,
openai_compatible: None,
tinfoil: None,
}
}
/// Resolve a model name from env var → settings.selected_model → hardcoded default.
fn resolve_model(
env_var: &str,
+71
View File
@@ -78,6 +78,77 @@ pub struct Config {
}
impl Config {
/// Create a full Config for integration tests without reading env vars.
///
/// Requires the `libsql` feature. Sets up:
/// - libSQL database at the given path
/// - WASM and embeddings disabled
/// - Skills enabled with the given directories
/// - Heartbeat, routines, sandbox, builder all disabled
/// - Safety with injection check off, 100k output limit
#[cfg(feature = "libsql")]
pub fn for_testing(
libsql_path: std::path::PathBuf,
skills_dir: std::path::PathBuf,
installed_skills_dir: std::path::PathBuf,
) -> Self {
Self {
database: DatabaseConfig {
backend: DatabaseBackend::LibSql,
url: secrecy::SecretString::from("unused://test".to_string()),
pool_size: 1,
ssl_mode: SslMode::Disable,
libsql_path: Some(libsql_path),
libsql_url: None,
libsql_auth_token: None,
},
llm: LlmConfig::for_testing(),
embeddings: EmbeddingsConfig::default(),
tunnel: TunnelConfig::default(),
channels: ChannelsConfig {
cli: CliConfig { enabled: false },
http: None,
gateway: None,
signal: None,
wasm_channels_dir: std::path::PathBuf::from("/tmp/ironclaw-test-channels"),
wasm_channels_enabled: false,
wasm_channel_owner_ids: HashMap::new(),
},
agent: AgentConfig::for_testing(),
safety: SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
},
wasm: WasmConfig {
enabled: false,
..WasmConfig::default()
},
secrets: SecretsConfig::default(),
builder: BuilderModeConfig {
enabled: false,
..BuilderModeConfig::default()
},
heartbeat: HeartbeatConfig::default(),
hygiene: HygieneConfig::default(),
routines: RoutineConfig {
enabled: false,
..RoutineConfig::default()
},
sandbox: SandboxModeConfig {
enabled: false,
..SandboxModeConfig::default()
},
claude_code: ClaudeCodeConfig::default(),
skills: SkillsConfig {
enabled: true,
local_dir: skills_dir,
installed_dir: installed_skills_dir,
..SkillsConfig::default()
},
observability: crate::observability::ObservabilityConfig::default(),
}
}
/// Load configuration from environment variables and the database.
///
/// Priority: env var > TOML config file > DB settings > default.
+20
View File
@@ -9,6 +9,8 @@ use rust_decimal::Decimal;
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::llm::recording::HttpInterceptor;
/// State of a job.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
@@ -146,6 +148,22 @@ pub struct JobContext {
/// Wrapped in `Arc` for cheap cloning on every tool invocation.
#[serde(skip)]
pub extra_env: Arc<HashMap<String, String>>,
/// Optional HTTP interceptor for trace recording/replay.
///
/// When set, tools that make outgoing HTTP requests should check this
/// interceptor before sending real requests. During recording, the
/// interceptor captures request/response pairs. During replay, it
/// returns pre-recorded responses.
#[serde(skip)]
pub http_interceptor: Option<Arc<dyn HttpInterceptor>>,
/// Stash of full tool outputs keyed by tool_call_id.
///
/// Tool outputs may be truncated before reaching the LLM context window,
/// but subsequent tools (e.g., `json`) may need the full output. This
/// stash stores the complete, unsanitized output so tools can reference
/// previous results by ID via `$tool_call_id` parameter syntax.
#[serde(skip)]
pub tool_output_stash: Arc<tokio::sync::RwLock<HashMap<String, String>>>,
}
impl JobContext {
@@ -182,7 +200,9 @@ impl JobContext {
repair_attempts: 0,
transitions: Vec::new(),
extra_env: Arc::new(HashMap::new()),
http_interceptor: None,
metadata: serde_json::Value::Null,
tool_output_stash: Arc::new(tokio::sync::RwLock::new(HashMap::new())),
}
}
+28
View File
@@ -117,6 +117,10 @@ impl JobStore for LibSqlBackend {
transitions: Vec::new(),
metadata: serde_json::Value::Null,
extra_env: std::sync::Arc::new(std::collections::HashMap::new()),
http_interceptor: None,
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
}))
}
None => Ok(None),
@@ -213,6 +217,30 @@ impl JobStore for LibSqlBackend {
Ok(jobs)
}
async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
"SELECT failure_reason FROM agent_jobs WHERE id = ?1",
[id.to_string()],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
Ok(get_opt_text(&row, 0))
} else {
Ok(None)
}
}
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
+6 -4
View File
@@ -515,7 +515,7 @@ impl WorkspaceStore for LibSqlBackend {
let mut rows = conn
.query(
r#"
SELECT c.id, c.document_id, c.content
SELECT c.id, c.document_id, d.path, c.content
FROM memory_chunks_fts fts
JOIN memory_chunks c ON c._rowid = fts.rowid
JOIN memory_documents d ON d.id = c.document_id
@@ -542,7 +542,8 @@ impl WorkspaceStore for LibSqlBackend {
results.push(RankedResult {
chunk_id: get_text(&row, 0).parse().unwrap_or_default(),
document_id: get_text(&row, 1).parse().unwrap_or_default(),
content: get_text(&row, 2),
document_path: get_text(&row, 2),
content: get_text(&row, 3),
rank: results.len() as u32 + 1,
});
}
@@ -563,7 +564,7 @@ impl WorkspaceStore for LibSqlBackend {
let mut rows = conn
.query(
r#"
SELECT c.id, c.document_id, c.content
SELECT c.id, c.document_id, d.path, c.content
FROM vector_top_k('idx_memory_chunks_embedding', vector(?1), ?2) AS top_k
JOIN memory_chunks c ON c._rowid = top_k.id
JOIN memory_documents d ON d.id = c.document_id
@@ -587,7 +588,8 @@ impl WorkspaceStore for LibSqlBackend {
results.push(RankedResult {
chunk_id: get_text(&row, 0).parse().unwrap_or_default(),
document_id: get_text(&row, 1).parse().unwrap_or_default(),
content: get_text(&row, 2),
document_path: get_text(&row, 2),
content: get_text(&row, 3),
rank: results.len() as u32 + 1,
});
}
+3
View File
@@ -177,6 +177,9 @@ pub trait JobStore: Send + Sync {
async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>;
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError>;
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError>;
/// Get the failure reason for a single agent job (O(1) lookup).
async fn get_agent_job_failure_reason(&self, id: Uuid)
-> Result<Option<String>, DatabaseError>;
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>;
async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>;
async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>;
+7
View File
@@ -223,6 +223,13 @@ impl JobStore for PgBackend {
self.store.agent_job_summary().await
}
async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
self.store.get_agent_job_failure_reason(id).await
}
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError> {
self.store.save_action(job_id, action).await
}
+3
View File
@@ -331,6 +331,9 @@ pub enum WorkspaceError {
#[error("Heartbeat error: {reason}")]
HeartbeatError { reason: String },
#[error("I/O error: {reason}")]
IoError { reason: String },
}
/// Orchestrator errors (internal API, container management).
+848 -290
View File
File diff suppressed because it is too large Load Diff
+382 -18
View File
@@ -24,6 +24,7 @@ pub use discovery::OnlineDiscovery;
pub use manager::ExtensionManager;
pub use registry::ExtensionRegistry;
use serde::ser::SerializeMap;
use serde::{Deserialize, Serialize};
/// The kind of extension, determining how it's installed, authenticated, and activated.
@@ -145,28 +146,267 @@ pub struct InstallResult {
pub message: String,
}
/// Auth readiness state for the extensions list UI.
///
/// Used by `check_tool_auth_status` and `check_channel_auth_status` to
/// communicate a tool's credential state to the list handler without
/// ambiguous `(bool, bool)` tuples.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ToolAuthState {
/// Token/credentials are present — ready to use.
Ready,
/// Auth section exists but the access token is missing (OAuth not completed).
NeedsAuth,
/// Setup credentials (client_id/secret) must be configured before OAuth can start.
NeedsSetup,
/// No auth configuration at all (no capabilities or auth section).
NoAuth,
}
/// The typed auth status, carrying only the data relevant to each state.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AuthStatus {
/// Authentication is complete; no further action needed.
Authenticated,
/// No authentication is required for this extension.
NoAuthRequired,
/// OAuth flow started — user must open `auth_url` in their browser.
AwaitingAuthorization {
auth_url: String,
callback_type: String,
},
/// Waiting for user to provide a token/key manually.
AwaitingToken {
instructions: String,
setup_url: Option<String>,
},
/// OAuth client credentials need to be configured before auth can proceed.
NeedsSetup {
instructions: String,
setup_url: Option<String>,
},
}
impl AuthStatus {
/// The wire-format status string (backward-compatible with JS consumers).
pub fn as_str(&self) -> &'static str {
match self {
AuthStatus::Authenticated => "authenticated",
AuthStatus::NoAuthRequired => "no_auth_required",
AuthStatus::AwaitingAuthorization { .. } => "awaiting_authorization",
AuthStatus::AwaitingToken { .. } => "awaiting_token",
AuthStatus::NeedsSetup { .. } => "needs_setup",
}
}
}
/// Result of authenticating an extension.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[derive(Debug, Clone)]
pub struct AuthResult {
pub name: String,
pub kind: ExtensionKind,
/// OAuth URL to open (for OAuth flows).
#[serde(skip_serializing_if = "Option::is_none")]
pub auth_url: Option<String>,
/// Whether using local or remote callback.
#[serde(skip_serializing_if = "Option::is_none")]
pub callback_type: Option<String>,
/// Instructions for manual token entry (for WASM tools).
#[serde(skip_serializing_if = "Option::is_none")]
pub instructions: Option<String>,
/// URL for manual token setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub setup_url: Option<String>,
/// Whether the tool is waiting for a token from the user.
#[serde(default)]
pub awaiting_token: bool,
/// Current auth status.
pub status: String,
pub status: AuthStatus,
}
impl AuthResult {
// ── Constructors ──────────────────────────────────────────────────
pub fn authenticated(name: impl Into<String>, kind: ExtensionKind) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::Authenticated,
}
}
pub fn no_auth_required(name: impl Into<String>, kind: ExtensionKind) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::NoAuthRequired,
}
}
pub fn awaiting_authorization(
name: impl Into<String>,
kind: ExtensionKind,
auth_url: String,
callback_type: String,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::AwaitingAuthorization {
auth_url,
callback_type,
},
}
}
pub fn awaiting_token(
name: impl Into<String>,
kind: ExtensionKind,
instructions: String,
setup_url: Option<String>,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::AwaitingToken {
instructions,
setup_url,
},
}
}
pub fn needs_setup(
name: impl Into<String>,
kind: ExtensionKind,
instructions: String,
setup_url: Option<String>,
) -> Self {
Self {
name: name.into(),
kind,
status: AuthStatus::NeedsSetup {
instructions,
setup_url,
},
}
}
// ── Accessors ─────────────────────────────────────────────────────
pub fn is_authenticated(&self) -> bool {
matches!(self.status, AuthStatus::Authenticated)
}
pub fn auth_url(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingAuthorization { auth_url, .. } => Some(auth_url),
_ => None,
}
}
pub fn callback_type(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingAuthorization { callback_type, .. } => Some(callback_type),
_ => None,
}
}
pub fn instructions(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingToken { instructions, .. }
| AuthStatus::NeedsSetup { instructions, .. } => Some(instructions),
_ => None,
}
}
pub fn setup_url(&self) -> Option<&str> {
match &self.status {
AuthStatus::AwaitingToken { setup_url, .. }
| AuthStatus::NeedsSetup { setup_url, .. } => setup_url.as_deref(),
_ => None,
}
}
pub fn is_awaiting_token(&self) -> bool {
matches!(self.status, AuthStatus::AwaitingToken { .. })
}
pub fn status_str(&self) -> &'static str {
self.status.as_str()
}
}
/// Serialize `AuthResult` to the same flat JSON shape the JS frontend expects.
impl Serialize for AuthResult {
fn serialize<S: serde::Serializer>(&self, serializer: S) -> Result<S::Ok, S::Error> {
// Count fields: name + kind + status + optional fields
let optional_count = self.auth_url().is_some() as usize
+ self.callback_type().is_some() as usize
+ self.instructions().is_some() as usize
+ self.setup_url().is_some() as usize;
let mut map = serializer.serialize_map(Some(4 + optional_count))?;
map.serialize_entry("name", &self.name)?;
map.serialize_entry("kind", &self.kind)?;
if let Some(url) = self.auth_url() {
map.serialize_entry("auth_url", url)?;
}
if let Some(cb) = self.callback_type() {
map.serialize_entry("callback_type", cb)?;
}
if let Some(inst) = self.instructions() {
map.serialize_entry("instructions", inst)?;
}
if let Some(url) = self.setup_url() {
map.serialize_entry("setup_url", url)?;
}
map.serialize_entry("awaiting_token", &self.is_awaiting_token())?;
map.serialize_entry("status", self.status_str())?;
map.end()
}
}
/// Deserialize from the flat JSON shape back into the typed enum.
impl<'de> Deserialize<'de> for AuthResult {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
/// Flat helper matching the old JSON shape.
#[derive(Deserialize)]
#[allow(dead_code)]
struct Raw {
name: String,
kind: ExtensionKind,
#[serde(default)]
auth_url: Option<String>,
#[serde(default)]
callback_type: Option<String>,
#[serde(default)]
instructions: Option<String>,
#[serde(default)]
setup_url: Option<String>,
#[serde(default)]
awaiting_token: bool,
status: String,
}
let raw = Raw::deserialize(deserializer)?;
let status = match raw.status.as_str() {
"authenticated" => AuthStatus::Authenticated,
"no_auth_required" => AuthStatus::NoAuthRequired,
"awaiting_authorization" => AuthStatus::AwaitingAuthorization {
auth_url: raw.auth_url.unwrap_or_default(),
callback_type: raw.callback_type.unwrap_or_default(),
},
"awaiting_token" => AuthStatus::AwaitingToken {
instructions: raw.instructions.unwrap_or_default(),
setup_url: raw.setup_url,
},
"needs_setup" => AuthStatus::NeedsSetup {
instructions: raw.instructions.unwrap_or_default(),
setup_url: raw.setup_url,
},
other => {
return Err(serde::de::Error::unknown_variant(
other,
&[
"authenticated",
"no_auth_required",
"awaiting_authorization",
"awaiting_token",
"needs_setup",
],
));
}
};
Ok(AuthResult {
name: raw.name,
kind: raw.kind,
status,
})
}
}
/// Result of activating an extension.
@@ -204,6 +444,9 @@ pub struct InstalledExtension {
/// Whether this extension has a setup schema (required_secrets) that can be configured.
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
#[serde(default)]
pub has_auth: bool,
/// Whether this extension is installed locally (false = available in registry but not installed).
#[serde(default = "default_true")]
pub installed: bool,
@@ -254,3 +497,124 @@ pub enum ExtensionError {
#[error("{0}")]
Other(String),
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn auth_result_authenticated_round_trip() {
let result = AuthResult::authenticated("gmail", ExtensionKind::WasmTool);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "authenticated");
assert_eq!(json["name"], "gmail");
assert_eq!(json["kind"], "wasm_tool");
assert_eq!(json["awaiting_token"], false);
assert!(json.get("auth_url").is_none());
assert!(json.get("instructions").is_none());
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(back.is_authenticated());
assert!(back.auth_url().is_none());
}
#[test]
fn auth_result_awaiting_authorization_round_trip() {
let result = AuthResult::awaiting_authorization(
"google-drive",
ExtensionKind::WasmTool,
"https://accounts.google.com/o/oauth2/v2/auth?state=abc".to_string(),
"local".to_string(),
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "awaiting_authorization");
assert_eq!(
json["auth_url"],
"https://accounts.google.com/o/oauth2/v2/auth?state=abc"
);
assert_eq!(json["callback_type"], "local");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert_eq!(
back.auth_url(),
Some("https://accounts.google.com/o/oauth2/v2/auth?state=abc")
);
assert_eq!(back.callback_type(), Some("local"));
assert!(!back.is_authenticated());
}
#[test]
fn auth_result_awaiting_token_round_trip() {
let result = AuthResult::awaiting_token(
"telegram",
ExtensionKind::WasmChannel,
"Enter your bot token".to_string(),
None,
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "awaiting_token");
assert_eq!(json["instructions"], "Enter your bot token");
assert_eq!(json["awaiting_token"], true);
assert!(json.get("auth_url").is_none());
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(back.is_awaiting_token());
assert_eq!(back.instructions(), Some("Enter your bot token"));
}
#[test]
fn auth_result_needs_setup_round_trip() {
let result = AuthResult::needs_setup(
"custom-tool",
ExtensionKind::WasmTool,
"Configure OAuth credentials in the Setup tab.".to_string(),
Some("https://console.cloud.google.com".to_string()),
);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "needs_setup");
assert_eq!(json["setup_url"], "https://console.cloud.google.com");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(!back.is_authenticated());
assert!(!back.is_awaiting_token());
assert_eq!(back.setup_url(), Some("https://console.cloud.google.com"));
}
#[test]
fn auth_result_no_auth_required_round_trip() {
let result = AuthResult::no_auth_required("echo", ExtensionKind::WasmTool);
let json = serde_json::to_value(&result).unwrap();
assert_eq!(json["status"], "no_auth_required");
assert_eq!(json["awaiting_token"], false);
let back: AuthResult = serde_json::from_value(json).unwrap();
assert!(!back.is_authenticated());
assert_eq!(back.status, AuthStatus::NoAuthRequired);
}
#[test]
fn auth_status_type_safety() {
// AwaitingAuthorization always has auth_url
let result = AuthResult::awaiting_authorization(
"test",
ExtensionKind::WasmTool,
"https://example.com".to_string(),
"local".to_string(),
);
assert!(result.auth_url().is_some());
assert!(!result.is_awaiting_token());
// Authenticated never has auth_url
let result = AuthResult::authenticated("test", ExtensionKind::WasmTool);
assert!(result.auth_url().is_none());
assert!(result.instructions().is_none());
assert!(result.setup_url().is_none());
}
}
+19
View File
@@ -237,6 +237,10 @@ impl Store {
total_tokens_used: 0,
max_tokens: 0,
extra_env: std::sync::Arc::new(std::collections::HashMap::new()),
http_interceptor: None,
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
}))
}
None => Ok(None),
@@ -821,6 +825,21 @@ impl Store {
.collect())
}
/// Get the failure reason for a single agent job.
pub async fn get_agent_job_failure_reason(
&self,
id: Uuid,
) -> Result<Option<String>, DatabaseError> {
let conn = self.conn().await?;
let row = conn
.query_opt(
"SELECT failure_reason FROM agent_jobs WHERE id = $1",
&[&id],
)
.await?;
Ok(row.and_then(|r| r.get::<_, Option<String>>("failure_reason")))
}
/// Summary counts for agent (non-sandbox) jobs.
pub async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError> {
let conn = self.conn().await?;
+19 -2
View File
@@ -13,6 +13,7 @@ pub mod failover;
mod nearai_chat;
mod provider;
mod reasoning;
pub mod recording;
pub mod response_cache;
pub mod retry;
mod rig_adapter;
@@ -30,6 +31,7 @@ pub use reasoning::{
ActionPlan, Reasoning, ReasoningContext, RespondOutput, RespondResult, SILENT_REPLY_TOKEN,
TokenUsage, ToolSelection, is_silent_reply,
};
pub use recording::RecordingLlm;
pub use response_cache::{CachedProvider, ResponseCacheConfig};
pub use retry::{RetryConfig, RetryProvider};
pub use rig_adapter::RigAdapter;
@@ -314,7 +316,14 @@ pub fn create_cheap_llm_provider(
pub fn build_provider_chain(
config: &LlmConfig,
session: Arc<SessionManager>,
) -> Result<(Arc<dyn LlmProvider>, Option<Arc<dyn LlmProvider>>), LlmError> {
) -> Result<
(
Arc<dyn LlmProvider>,
Option<Arc<dyn LlmProvider>>,
Option<Arc<RecordingLlm>>,
),
LlmError,
> {
let llm = create_llm_provider(config, session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
@@ -427,13 +436,21 @@ pub fn build_provider_chain(
llm
};
// 6. Recording (trace capture for replay testing)
let recording_handle = RecordingLlm::from_env(llm.clone());
let llm: Arc<dyn LlmProvider> = if let Some(ref recorder) = recording_handle {
Arc::clone(recorder) as Arc<dyn LlmProvider>
} else {
llm
};
// Standalone cheap LLM for heartbeat/evaluation (not part of the chain)
let cheap_llm = create_cheap_llm_provider(config, session)?;
if let Some(ref cheap) = cheap_llm {
tracing::info!("Cheap LLM provider initialized: {}", cheap.model_name());
}
Ok((llm, cheap_llm))
Ok((llm, cheap_llm, recording_handle))
}
#[cfg(test)]
+24 -1
View File
@@ -199,6 +199,29 @@ impl NearAiChatProvider {
})?;
let status = response.status();
// Extract Retry-After header before consuming the response body.
// Supports both delay-seconds (RFC 7231 §7.1.3) and HTTP-date formats.
let retry_after_header = response
.headers()
.get("retry-after")
.and_then(|v| v.to_str().ok())
.and_then(|v| {
// Try delay-seconds first (most common from API providers)
if let Ok(secs) = v.trim().parse::<u64>() {
return Some(std::time::Duration::from_secs(secs));
}
// Try HTTP-date (e.g. "Mon, 02 Mar 2026 18:00:00 GMT")
if let Ok(dt) = chrono::DateTime::parse_from_rfc2822(v.trim()) {
let now = chrono::Utc::now();
let delta = dt.signed_duration_since(now);
// Use max(0) so past/present dates yield Duration::ZERO
// rather than None (which would cause an immediate retry).
return Some(std::time::Duration::from_secs(
delta.num_seconds().max(0) as u64
));
}
None
});
let response_text = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: format!("Failed to read response body: {}", e),
@@ -230,7 +253,7 @@ impl NearAiChatProvider {
if status_code == 429 {
return Err(LlmError::RateLimited {
provider: "nearai_chat".to_string(),
retry_after: None,
retry_after: retry_after_header,
});
}
+104
View File
@@ -689,6 +689,8 @@ Example:
- If tools return empty or irrelevant results, answer with what you already know rather than retrying
## Tool Call Style
- ALWAYS call tools via tool_calls never just describe what you would do
- If you say "let me fetch/check/look up X", you MUST include the actual tool call in the same response
- Do not narrate routine, low-risk tool calls; just call the tool
- Narrate only when it helps: multi-step work, sensitive actions, or when the user asks
- For multi-step tasks, call independent tools in parallel when possible
@@ -1131,6 +1133,51 @@ fn recover_tool_calls_from_content(
}
}
// Bracket format from flatten_tool_messages:
// [Called tool `name` with arguments: {...}]
{
let mut remaining = content;
while let Some(start) = remaining.find("[Called tool `") {
let after_prefix = &remaining[start + "[Called tool `".len()..];
let Some(backtick_end) = after_prefix.find('`') else {
break;
};
let name = &after_prefix[..backtick_end];
let after_name = &after_prefix[backtick_end + 1..];
if !tool_names.contains(name) {
remaining = after_name;
continue;
}
// Look for " with arguments: " followed by JSON until "]"
if let Some(args_start) = after_name.strip_prefix(" with arguments: ") {
// Find the closing "]" — but the JSON itself may contain "]",
// so find the last "]" on this logical line.
if let Some(bracket_end) = args_start.rfind(']') {
let args_str = &args_start[..bracket_end];
let arguments = serde_json::from_str::<serde_json::Value>(args_str)
.unwrap_or(serde_json::Value::Object(Default::default()));
calls.push(ToolCall {
id: format!("recovered_{}", calls.len()),
name: name.to_string(),
arguments,
});
remaining = &args_start[bracket_end + 1..];
continue;
}
}
// No arguments or malformed — call with empty args
calls.push(ToolCall {
id: format!("recovered_{}", calls.len()),
name: name.to_string(),
arguments: serde_json::Value::Object(Default::default()),
});
remaining = after_name;
}
}
calls
}
@@ -1174,10 +1221,39 @@ fn clean_response(text: &str) -> String {
result = strip_pipe_tag(&result, tag);
}
// 6b. Strip bracket-format inline tool calls: [Called tool `name` with arguments: {...}]
result = strip_bracket_tool_calls(&result);
// 7. Collapse triple+ newlines, trim
collapse_newlines(&result)
}
/// Strip bracket-format inline tool calls produced by `flatten_tool_messages`.
///
/// Removes patterns like `[Called tool `name` with arguments: {...}]` from text
/// so the user doesn't see raw tool call syntax when the model echoes it back.
fn strip_bracket_tool_calls(text: &str) -> String {
let mut result = String::with_capacity(text.len());
let mut remaining = text;
while let Some(start) = remaining.find("[Called tool `") {
result.push_str(&remaining[..start]);
let after = &remaining[start..];
// Find the closing "]" for this bracket expression
if let Some(end) = after.find("]\n").map(|i| i + 2).or_else(|| {
// If it's at the end of the string, just find "]"
after.rfind(']').map(|i| i + 1)
}) {
remaining = &after[end..];
} else {
// Malformed — keep the rest
result.push_str(after);
return result;
}
}
result.push_str(remaining);
result
}
/// Tool-related tags stripped with simple string matching (no code-awareness needed).
const TOOL_TAGS: &[&str] = &["tool_call", "function_call", "tool_calls"];
@@ -1841,4 +1917,32 @@ That's my plan."#;
assert_eq!(calls.len(), 1);
assert_eq!(calls[0].name, "tool_list");
}
#[test]
fn test_recover_bracket_format_tool_call() {
let tools = make_tools(&["http"]);
let content = "Let me try that. [Called tool `http` with arguments: {\"method\":\"GET\",\"url\":\"https://example.com\"}]";
let calls = recover_tool_calls_from_content(content, &tools);
assert_eq!(calls.len(), 1);
assert_eq!(calls[0].name, "http");
assert_eq!(calls[0].arguments["method"], "GET");
assert_eq!(calls[0].arguments["url"], "https://example.com");
}
#[test]
fn test_recover_bracket_format_unknown_tool_ignored() {
let tools = make_tools(&["http"]);
let content = "[Called tool `unknown_tool` with arguments: {}]";
let calls = recover_tool_calls_from_content(content, &tools);
assert!(calls.is_empty());
}
#[test]
fn test_clean_response_strips_bracket_tool_calls() {
let input = "Let me fetch that.\n[Called tool `http` with arguments: {\"method\":\"GET\",\"url\":\"https://example.com\"}]\nHere are the results.";
let cleaned = clean_response(input);
assert!(!cleaned.contains("[Called tool"));
assert!(cleaned.contains("Let me fetch that."));
assert!(cleaned.contains("Here are the results."));
}
}
+917
View File
@@ -0,0 +1,917 @@
//! Live trace recording mode.
//!
//! Wraps any [`LlmProvider`] and captures every LLM interaction into
//! the trace fixture format used by `TraceLlm` for deterministic E2E
//! testing. Recorded traces can be replayed later via `TraceLlm`.
//!
//! The trace includes:
//! - **Memory snapshot**: workspace documents captured before the first LLM call
//! - **HTTP exchanges**: all outgoing HTTP request/response pairs from tools
//! - **Steps**: user inputs, LLM responses (text/tool_calls), and expected tool
//! results for verifying tool output during replay
//!
//! Enable by setting `IRONCLAW_RECORD_TRACE=1` at runtime.
use std::collections::VecDeque;
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use rust_decimal::Decimal;
use serde::{Deserialize, Serialize};
use tokio::sync::Mutex;
use crate::error::LlmError;
use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, LlmProvider, ModelMetadata, Role,
ToolCompletionRequest, ToolCompletionResponse,
};
// ── Trace format types ─────────────────────────────────────────────
/// Top-level trace file — extended format with memory snapshot and HTTP exchanges.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TraceFile {
pub model_name: String,
/// Workspace memory documents captured before the recording session.
/// Replay should restore these before running the trace.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub memory_snapshot: Vec<MemorySnapshotEntry>,
/// HTTP exchanges recorded during the session, in order.
/// Replay should return these instead of making real HTTP requests.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub http_exchanges: Vec<HttpExchange>,
pub steps: Vec<TraceStep>,
}
/// A memory document captured at recording start.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MemorySnapshotEntry {
pub path: String,
pub content: String,
}
/// A recorded HTTP request/response pair.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HttpExchange {
pub request: HttpExchangeRequest,
pub response: HttpExchangeResponse,
}
/// The request side of an HTTP exchange.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HttpExchangeRequest {
pub method: String,
pub url: String,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub headers: Vec<(String, String)>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub body: Option<String>,
}
/// The response side of an HTTP exchange.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct HttpExchangeResponse {
pub status: u16,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub headers: Vec<(String, String)>,
pub body: String,
}
/// A single step in the trace.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TraceStep {
#[serde(skip_serializing_if = "Option::is_none")]
pub request_hint: Option<RequestHint>,
pub response: TraceResponse,
/// Tool results that appeared in the message context since the previous step.
/// During replay, the test harness can compare actual tool results against
/// these to verify tool output hasn't changed (regression detection).
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub expected_tool_results: Vec<ExpectedToolResult>,
}
/// Soft validation hints for matching a step to a request.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RequestHint {
#[serde(skip_serializing_if = "Option::is_none")]
pub last_user_message_contains: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub min_message_count: Option<usize>,
}
/// Tagged response enum — text, tool_calls, or user_input.
///
/// `user_input` steps are metadata markers — they record what the user said
/// but do **not** correspond to an LLM call. During replay, `TraceLlm` must
/// skip `user_input` steps and only consume `text`/`tool_calls` steps.
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum TraceResponse {
Text {
content: String,
input_tokens: u32,
output_tokens: u32,
},
ToolCalls {
tool_calls: Vec<TraceToolCall>,
input_tokens: u32,
output_tokens: u32,
},
/// Marker for a user message that triggered subsequent LLM calls.
/// Not an LLM response — replay providers must skip these.
UserInput { content: String },
}
/// A tool call in a trace step.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TraceToolCall {
pub id: String,
pub name: String,
pub arguments: serde_json::Value,
}
/// Recorded tool result for regression checking during replay.
///
/// During replay, after tools execute and before returning the canned LLM
/// response, the test harness should compare actual `Role::Tool` messages
/// against these entries. A content mismatch indicates a tool behavior change.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ExpectedToolResult {
pub tool_call_id: String,
pub name: String,
/// The full tool result content as it appeared in the message context.
pub content: String,
}
// ── HTTP interceptor ───────────────────────────────────────────────
/// Trait for intercepting HTTP requests from tools.
///
/// During recording, the interceptor captures exchanges after the real
/// request completes. During replay, it short-circuits with a recorded response.
#[async_trait]
pub trait HttpInterceptor: Send + Sync + std::fmt::Debug {
/// Called before making an HTTP request.
///
/// Return `Some(response)` to short-circuit (replay mode).
/// Return `None` to let the real request proceed (recording mode).
async fn before_request(&self, request: &HttpExchangeRequest) -> Option<HttpExchangeResponse>;
/// Called after a real HTTP request completes (recording mode only).
async fn after_response(&self, request: &HttpExchangeRequest, response: &HttpExchangeResponse);
}
/// Records HTTP exchanges during a live session.
#[derive(Debug)]
pub struct RecordingHttpInterceptor {
exchanges: Mutex<Vec<HttpExchange>>,
}
impl Default for RecordingHttpInterceptor {
fn default() -> Self {
Self::new()
}
}
impl RecordingHttpInterceptor {
pub fn new() -> Self {
Self {
exchanges: Mutex::new(Vec::new()),
}
}
/// Return all recorded exchanges.
pub async fn take_exchanges(&self) -> Vec<HttpExchange> {
self.exchanges.lock().await.clone()
}
}
#[async_trait]
impl HttpInterceptor for RecordingHttpInterceptor {
async fn before_request(&self, _request: &HttpExchangeRequest) -> Option<HttpExchangeResponse> {
// Recording mode: let the real request proceed
None
}
async fn after_response(&self, request: &HttpExchangeRequest, response: &HttpExchangeResponse) {
self.exchanges.lock().await.push(HttpExchange {
request: request.clone(),
response: response.clone(),
});
}
}
/// Replays recorded HTTP exchanges during test runs.
///
/// Returns responses in order. If more requests arrive than recorded
/// exchanges, returns a 599 error response.
#[derive(Debug)]
pub struct ReplayingHttpInterceptor {
exchanges: Mutex<VecDeque<HttpExchange>>,
}
impl ReplayingHttpInterceptor {
pub fn new(exchanges: Vec<HttpExchange>) -> Self {
Self {
exchanges: Mutex::new(VecDeque::from(exchanges)),
}
}
}
#[async_trait]
impl HttpInterceptor for ReplayingHttpInterceptor {
async fn before_request(&self, request: &HttpExchangeRequest) -> Option<HttpExchangeResponse> {
let mut queue = self.exchanges.lock().await;
if let Some(exchange) = queue.pop_front() {
// Soft-check: warn if the request doesn't match
if exchange.request.url != request.url || exchange.request.method != request.method {
tracing::warn!(
expected_url = %exchange.request.url,
actual_url = %request.url,
expected_method = %exchange.request.method,
actual_method = %request.method,
"HTTP replay: request mismatch (returning recorded response anyway)"
);
}
Some(exchange.response)
} else {
tracing::error!(
url = %request.url,
method = %request.method,
"HTTP replay: no more recorded exchanges, returning error"
);
Some(HttpExchangeResponse {
status: 599,
headers: Vec::new(),
body: "trace replay: no more recorded HTTP exchanges".to_string(),
})
}
}
async fn after_response(
&self,
_request: &HttpExchangeRequest,
_response: &HttpExchangeResponse,
) {
// Replay mode: nothing to record
}
}
// ── RecordingLlm ───────────────────────────────────────────────────
/// LLM provider decorator that records interactions into a trace file.
pub struct RecordingLlm {
inner: Arc<dyn LlmProvider>,
steps: Mutex<Vec<TraceStep>>,
prev_message_count: Mutex<usize>,
output_path: PathBuf,
model_name: String,
memory_snapshot: Mutex<Vec<MemorySnapshotEntry>>,
http_interceptor: Arc<RecordingHttpInterceptor>,
}
impl RecordingLlm {
/// Wrap a provider for recording.
pub fn new(inner: Arc<dyn LlmProvider>, output_path: PathBuf, model_name: String) -> Self {
Self {
inner,
steps: Mutex::new(Vec::new()),
prev_message_count: Mutex::new(0),
output_path,
model_name,
memory_snapshot: Mutex::new(Vec::new()),
http_interceptor: Arc::new(RecordingHttpInterceptor::new()),
}
}
/// Create from environment variables if recording is enabled.
///
/// - `IRONCLAW_RECORD_TRACE` — any non-empty value enables recording
/// - `IRONCLAW_TRACE_OUTPUT` — file path (default: `./trace_{timestamp}.json`)
/// - `IRONCLAW_TRACE_MODEL_NAME` — model_name field (default: `recorded-{inner.model_name()}`)
pub fn from_env(inner: Arc<dyn LlmProvider>) -> Option<Arc<Self>> {
let enabled = std::env::var("IRONCLAW_RECORD_TRACE")
.ok()
.filter(|v| !v.is_empty());
enabled?;
let output_path = std::env::var("IRONCLAW_TRACE_OUTPUT")
.ok()
.filter(|v| !v.is_empty())
.map(PathBuf::from)
.unwrap_or_else(|| {
let ts = chrono::Local::now().format("%Y%m%dT%H%M%S");
PathBuf::from(format!("trace_{ts}.json"))
});
let model_name = std::env::var("IRONCLAW_TRACE_MODEL_NAME")
.ok()
.filter(|v| !v.is_empty())
.unwrap_or_else(|| format!("recorded-{}", inner.model_name()));
tracing::info!(
output = %output_path.display(),
model = %model_name,
"LLM trace recording enabled"
);
Some(Arc::new(Self::new(inner, output_path, model_name)))
}
/// Get the HTTP interceptor for wiring into tools.
///
/// Pass this to `JobContext` or `HttpTool` so outgoing HTTP requests
/// are recorded into the trace.
pub fn http_interceptor(&self) -> Arc<dyn HttpInterceptor> {
Arc::clone(&self.http_interceptor) as Arc<dyn HttpInterceptor>
}
/// Snapshot all memory documents from a workspace.
///
/// Call this once after creation, before the agent starts processing.
pub async fn snapshot_memory(&self, workspace: &crate::workspace::Workspace) {
match workspace.list_all().await {
Ok(paths) => {
let mut snapshot = self.memory_snapshot.lock().await;
for path in paths {
match workspace.read(&path).await {
Ok(doc) => {
snapshot.push(MemorySnapshotEntry {
path: doc.path,
content: doc.content,
});
}
Err(e) => {
tracing::debug!(path = %path, error = %e, "Skipped memory doc in snapshot");
}
}
}
tracing::info!(
documents = snapshot.len(),
"Captured memory snapshot for trace recording"
);
}
Err(e) => {
tracing::warn!("Failed to snapshot memory for trace recording: {}", e);
}
}
}
/// Flush accumulated steps, memory snapshot, and HTTP exchanges to the output file.
pub async fn flush(&self) -> Result<(), std::io::Error> {
let steps = self.steps.lock().await;
let memory_snapshot = self.memory_snapshot.lock().await;
let http_exchanges = self.http_interceptor.take_exchanges().await;
let trace = TraceFile {
model_name: self.model_name.clone(),
memory_snapshot: memory_snapshot.clone(),
http_exchanges,
steps: steps.clone(),
};
let json = serde_json::to_string_pretty(&trace).map_err(std::io::Error::other)?;
tokio::fs::write(&self.output_path, json).await?;
tracing::info!(
steps = steps.len(),
memory_docs = memory_snapshot.len(),
path = %self.output_path.display(),
"Flushed LLM trace recording"
);
Ok(())
}
/// Extract new user messages, tool results, and build request hint.
///
/// Returns `(hint, tool_results)` where tool_results are new `Role::Tool`
/// messages since the last call — these become `expected_tool_results` on
/// the next step for replay verification.
async fn capture_new_messages(
&self,
messages: &[ChatMessage],
) -> (Option<RequestHint>, Vec<ExpectedToolResult>) {
let mut prev_count = self.prev_message_count.lock().await;
let current_count = messages.len();
// After context compaction, the message list may shrink below
// prev_count. Clamp to avoid an out-of-bounds slice.
let start = (*prev_count).min(current_count);
let new_messages = &messages[start..];
// Emit UserInput steps for new user messages
let new_user_messages: Vec<&ChatMessage> = new_messages
.iter()
.filter(|m| m.role == Role::User)
.collect();
if !new_user_messages.is_empty() {
let mut steps = self.steps.lock().await;
for msg in &new_user_messages {
steps.push(TraceStep {
request_hint: None,
response: TraceResponse::UserInput {
content: msg.content.clone(),
},
expected_tool_results: Vec::new(),
});
}
}
// Capture new tool result messages for expected_tool_results
let tool_results: Vec<ExpectedToolResult> = new_messages
.iter()
.filter(|m| m.role == Role::Tool)
.map(|m| ExpectedToolResult {
tool_call_id: m.tool_call_id.clone().unwrap_or_default(),
name: m.name.clone().unwrap_or_default(),
content: m.content.clone(),
})
.collect();
*prev_count = current_count;
// Build request hint from last user message
let hint = messages
.iter()
.rev()
.find(|m| m.role == Role::User)
.map(|msg| {
let hint_text = if msg.content.len() > 80 {
msg.content[..80].to_string()
} else {
msg.content.clone()
};
RequestHint {
last_user_message_contains: Some(hint_text),
min_message_count: Some(current_count),
}
});
(hint, tool_results)
}
}
#[async_trait]
impl LlmProvider for RecordingLlm {
fn model_name(&self) -> &str {
self.inner.model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.inner.cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let (hint, tool_results) = self.capture_new_messages(&request.messages).await;
let response = self.inner.complete(request).await?;
self.steps.lock().await.push(TraceStep {
request_hint: hint,
response: TraceResponse::Text {
content: response.content.clone(),
input_tokens: response.input_tokens,
output_tokens: response.output_tokens,
},
expected_tool_results: tool_results,
});
Ok(response)
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let (hint, tool_results) = self.capture_new_messages(&request.messages).await;
let response = self.inner.complete_with_tools(request).await?;
let step = if response.tool_calls.is_empty() {
TraceStep {
request_hint: hint,
response: TraceResponse::Text {
content: response.content.clone().unwrap_or_default(),
input_tokens: response.input_tokens,
output_tokens: response.output_tokens,
},
expected_tool_results: tool_results,
}
} else {
TraceStep {
request_hint: hint,
response: TraceResponse::ToolCalls {
tool_calls: response
.tool_calls
.iter()
.map(|tc| TraceToolCall {
id: tc.id.clone(),
name: tc.name.clone(),
arguments: tc.arguments.clone(),
})
.collect(),
input_tokens: response.input_tokens,
output_tokens: response.output_tokens,
},
expected_tool_results: tool_results,
}
};
self.steps.lock().await.push(step);
Ok(response)
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
self.inner.list_models().await
}
async fn model_metadata(&self) -> Result<ModelMetadata, LlmError> {
self.inner.model_metadata().await
}
fn effective_model_name(&self, requested_model: Option<&str>) -> String {
self.inner.effective_model_name(requested_model)
}
fn active_model_name(&self) -> String {
self.inner.active_model_name()
}
fn set_model(&self, model: &str) -> Result<(), LlmError> {
self.inner.set_model(model)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::testing::StubLlm;
fn make_recorder(stub: Arc<StubLlm>) -> RecordingLlm {
RecordingLlm::new(
stub,
PathBuf::from("/tmp/test_recording.json"),
"test-recording".to_string(),
)
}
#[tokio::test]
async fn captures_user_input_before_first_response() {
let stub = Arc::new(StubLlm::new("hello back"));
let recorder = make_recorder(stub);
let request = CompletionRequest::new(vec![
ChatMessage::system("You are helpful."),
ChatMessage::user("Hello!"),
]);
recorder.complete(request).await.unwrap();
let steps = recorder.steps.lock().await;
assert_eq!(steps.len(), 2);
// First step: user_input
assert!(
matches!(&steps[0].response, TraceResponse::UserInput { content } if content == "Hello!")
);
// Second step: text response
assert!(
matches!(&steps[1].response, TraceResponse::Text { content, .. } if content == "hello back")
);
}
#[tokio::test]
async fn captures_text_response_correctly() {
let stub = Arc::new(StubLlm::new("test response"));
let recorder = make_recorder(stub);
let request = CompletionRequest::new(vec![ChatMessage::user("question")]);
recorder.complete(request).await.unwrap();
let steps = recorder.steps.lock().await;
// user_input + text
assert_eq!(steps.len(), 2);
match &steps[1].response {
TraceResponse::Text {
content,
input_tokens,
output_tokens,
} => {
assert_eq!(content, "test response");
// StubLlm returns 0s for tokens, which is fine
let _ = (*input_tokens, *output_tokens);
}
_ => panic!("Expected Text response"),
}
}
#[tokio::test]
async fn captures_tool_calls_response() {
let stub = Arc::new(StubLlm::new("tool result"));
let recorder = make_recorder(stub);
// complete_with_tools on StubLlm returns text, not tool_calls.
// But we can still verify the recording captures it as text.
let request = ToolCompletionRequest::new(vec![ChatMessage::user("use a tool")], vec![]);
recorder.complete_with_tools(request).await.unwrap();
let steps = recorder.steps.lock().await;
assert_eq!(steps.len(), 2); // user_input + text (StubLlm doesn't return tool_calls)
}
#[tokio::test]
async fn no_spurious_user_input_for_tool_iterations() {
let stub = Arc::new(StubLlm::new("response"));
let recorder = make_recorder(stub);
// First call with user message
let request = CompletionRequest::new(vec![
ChatMessage::system("sys"),
ChatMessage::user("Do something"),
]);
recorder.complete(request).await.unwrap();
// Second call: same messages plus tool result (no new user message)
let request = CompletionRequest::new(vec![
ChatMessage::system("sys"),
ChatMessage::user("Do something"),
ChatMessage::assistant("I'll use a tool"),
ChatMessage::tool_result("call_1", "echo", "result"),
]);
recorder.complete(request).await.unwrap();
let steps = recorder.steps.lock().await;
// Step 0: user_input "Do something"
// Step 1: text response
// Step 2: text response (no new user_input since no new user messages)
assert_eq!(steps.len(), 3);
assert!(matches!(
&steps[0].response,
TraceResponse::UserInput { .. }
));
assert!(matches!(&steps[1].response, TraceResponse::Text { .. }));
assert!(matches!(&steps[2].response, TraceResponse::Text { .. }));
}
#[tokio::test]
async fn captures_tool_results_for_verification() {
let stub = Arc::new(StubLlm::new("response"));
let recorder = make_recorder(stub);
// First call: user asks something
let request = CompletionRequest::new(vec![
ChatMessage::system("sys"),
ChatMessage::user("Do something"),
]);
recorder.complete(request).await.unwrap();
// Second call: includes tool results from previous tool_calls
let request = CompletionRequest::new(vec![
ChatMessage::system("sys"),
ChatMessage::user("Do something"),
ChatMessage::assistant("I'll use a tool"),
ChatMessage::tool_result("call_1", "echo", "echoed: hello"),
ChatMessage::tool_result("call_2", "time", "2026-03-04T14:00:00Z"),
]);
recorder.complete(request).await.unwrap();
let steps = recorder.steps.lock().await;
// Step 2 (the second LLM response) should have expected_tool_results
let step = &steps[2];
assert_eq!(step.expected_tool_results.len(), 2);
assert_eq!(step.expected_tool_results[0].name, "echo");
assert_eq!(step.expected_tool_results[0].content, "echoed: hello");
assert_eq!(step.expected_tool_results[1].name, "time");
}
#[tokio::test]
async fn request_hint_extraction() {
let stub = Arc::new(StubLlm::new("response"));
let recorder = make_recorder(stub);
let request = CompletionRequest::new(vec![
ChatMessage::system("sys"),
ChatMessage::user("What time is it?"),
]);
recorder.complete(request).await.unwrap();
let steps = recorder.steps.lock().await;
let text_step = &steps[1];
let hint = text_step.request_hint.as_ref().unwrap();
assert_eq!(
hint.last_user_message_contains.as_deref(),
Some("What time is it?")
);
assert_eq!(hint.min_message_count, Some(2));
}
#[tokio::test]
async fn flush_writes_valid_json_with_all_fields() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("trace.json");
let stub = Arc::new(StubLlm::new("response"));
let recorder = RecordingLlm::new(stub, path.clone(), "flush-test".to_string());
// Simulate a memory snapshot
recorder
.memory_snapshot
.lock()
.await
.push(MemorySnapshotEntry {
path: "context/test.md".to_string(),
content: "test content".to_string(),
});
// Simulate an HTTP exchange
recorder
.http_interceptor
.after_response(
&HttpExchangeRequest {
method: "GET".to_string(),
url: "https://api.example.com/data".to_string(),
headers: Vec::new(),
body: None,
},
&HttpExchangeResponse {
status: 200,
headers: Vec::new(),
body: r#"{"ok": true}"#.to_string(),
},
)
.await;
let request = CompletionRequest::new(vec![ChatMessage::user("hello")]);
recorder.complete(request).await.unwrap();
recorder.flush().await.unwrap();
let content = tokio::fs::read_to_string(&path).await.unwrap();
let trace: TraceFile = serde_json::from_str(&content).unwrap();
assert_eq!(trace.model_name, "flush-test");
assert_eq!(trace.memory_snapshot.len(), 1);
assert_eq!(trace.memory_snapshot[0].path, "context/test.md");
assert_eq!(trace.http_exchanges.len(), 1);
assert_eq!(trace.http_exchanges[0].response.status, 200);
assert_eq!(trace.steps.len(), 2);
}
#[test]
fn from_env_returns_none_when_unset() {
// SAFETY: This test is single-threaded and no other thread reads this var.
unsafe { std::env::remove_var("IRONCLAW_RECORD_TRACE") };
let stub = Arc::new(StubLlm::new("response"));
let result = RecordingLlm::from_env(stub);
assert!(result.is_none());
}
#[tokio::test]
async fn recording_http_interceptor_passes_through_and_records() {
let interceptor = RecordingHttpInterceptor::new();
let req = HttpExchangeRequest {
method: "GET".to_string(),
url: "https://example.com".to_string(),
headers: Vec::new(),
body: None,
};
// before_request should return None (pass through)
assert!(interceptor.before_request(&req).await.is_none());
// after_response records the exchange
let resp = HttpExchangeResponse {
status: 200,
headers: Vec::new(),
body: "ok".to_string(),
};
interceptor.after_response(&req, &resp).await;
let exchanges = interceptor.take_exchanges().await;
assert_eq!(exchanges.len(), 1);
assert_eq!(exchanges[0].request.url, "https://example.com");
}
#[tokio::test]
async fn replaying_http_interceptor_returns_recorded_responses() {
let exchanges = vec![HttpExchange {
request: HttpExchangeRequest {
method: "GET".to_string(),
url: "https://api.example.com/data".to_string(),
headers: Vec::new(),
body: None,
},
response: HttpExchangeResponse {
status: 200,
headers: Vec::new(),
body: r#"{"items": []}"#.to_string(),
},
}];
let interceptor = ReplayingHttpInterceptor::new(exchanges);
// First request: returns recorded response
let req = HttpExchangeRequest {
method: "GET".to_string(),
url: "https://api.example.com/data".to_string(),
headers: Vec::new(),
body: None,
};
let resp = interceptor.before_request(&req).await.unwrap();
assert_eq!(resp.status, 200);
assert_eq!(resp.body, r#"{"items": []}"#);
// Second request: no more exchanges → 599
let resp = interceptor.before_request(&req).await.unwrap();
assert_eq!(resp.status, 599);
}
#[test]
fn serde_roundtrip_extended_format() {
let trace = TraceFile {
model_name: "test".to_string(),
memory_snapshot: vec![MemorySnapshotEntry {
path: "context/vision.md".to_string(),
content: "Be helpful.".to_string(),
}],
http_exchanges: vec![HttpExchange {
request: HttpExchangeRequest {
method: "GET".to_string(),
url: "https://api.example.com".to_string(),
headers: vec![("Accept".to_string(), "application/json".to_string())],
body: None,
},
response: HttpExchangeResponse {
status: 200,
headers: Vec::new(),
body: "{}".to_string(),
},
}],
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::UserInput {
content: "hello".to_string(),
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: Some(RequestHint {
last_user_message_contains: Some("hello".to_string()),
min_message_count: Some(2),
}),
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_1".to_string(),
name: "echo".to_string(),
arguments: serde_json::json!({"message": "hi"}),
}],
input_tokens: 50,
output_tokens: 20,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "done".to_string(),
input_tokens: 80,
output_tokens: 10,
},
expected_tool_results: vec![ExpectedToolResult {
tool_call_id: "call_1".to_string(),
name: "echo".to_string(),
content: "hi".to_string(),
}],
},
],
};
let json = serde_json::to_string_pretty(&trace).unwrap();
let parsed: TraceFile = serde_json::from_str(&json).unwrap();
assert_eq!(parsed.model_name, "test");
assert_eq!(parsed.memory_snapshot.len(), 1);
assert_eq!(parsed.http_exchanges.len(), 1);
assert_eq!(parsed.steps.len(), 3);
assert_eq!(parsed.steps[2].expected_tool_results.len(), 1);
}
#[test]
fn backward_compatible_with_old_format() {
// Old format without memory_snapshot, http_exchanges, expected_tool_results
let json = r#"{
"model_name": "old-trace",
"steps": [
{
"response": {
"type": "text",
"content": "hello",
"input_tokens": 10,
"output_tokens": 5
}
}
]
}"#;
let trace: TraceFile = serde_json::from_str(json).unwrap();
assert_eq!(trace.model_name, "old-trace");
assert!(trace.memory_snapshot.is_empty());
assert!(trace.http_exchanges.is_empty());
assert!(trace.steps[0].expected_tool_results.is_empty());
}
}
+1 -1
View File
@@ -347,7 +347,7 @@ impl SessionManager {
// The NEAR AI API redirects to: {frontend_callback}/auth/callback?token=X&...
let session_token =
oauth_defaults::wait_for_callback(listener, "/auth/callback", "token", "NEAR AI")
oauth_defaults::wait_for_callback(listener, "/auth/callback", "token", "NEAR AI", None)
.await
.map_err(|e| LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
+1225 -196
View File
File diff suppressed because it is too large Load Diff
+45 -20
View File
@@ -484,8 +484,6 @@ async fn async_main() -> anyhow::Result<()> {
let mut sse_sender: Option<
tokio::sync::broadcast::Sender<ironclaw::channels::web::types::SseEvent>,
> = None;
let mut gateway_state: Option<std::sync::Arc<ironclaw::channels::web::server::GatewayState>> =
None;
if let Some(ref gw_config) = config.channels.gateway {
let mut gw =
GatewayChannel::new(gw_config.clone()).with_llm_provider(Arc::clone(&components.llm));
@@ -508,6 +506,7 @@ async fn async_main() -> anyhow::Result<()> {
if let Some(ref jm) = container_job_manager {
gw = gw.with_job_manager(Arc::clone(jm));
}
gw = gw.with_scheduler(scheduler_slot.clone());
if let Some(ref sr) = components.skill_registry {
gw = gw.with_skill_registry(Arc::clone(sr));
}
@@ -542,7 +541,6 @@ async fn async_main() -> anyhow::Result<()> {
// IMPORTANT: This must come after all `with_*` calls since `rebuild_state`
// creates a new SseManager, which would orphan this sender.
sse_sender = Some(gw.state().sse.sender());
gateway_state = Some(Arc::clone(gw.state()));
channel_names.push("gateway".to_string());
channels.add(Box::new(gw)).await;
@@ -618,7 +616,7 @@ async fn async_main() -> anyhow::Result<()> {
rt,
ps,
router,
config.channels.telegram_owner_id,
config.channels.wasm_channel_owner_ids.clone(),
)
.await;
tracing::info!("Channel runtime wired into extension manager for hot-activation");
@@ -649,11 +647,22 @@ async fn async_main() -> anyhow::Result<()> {
// Wire SSE sender into extension manager for broadcasting status events.
if let Some(ref ext_mgr) = components.extension_manager
&& let Some(sender) = sse_sender
&& let Some(ref sender) = sse_sender
{
ext_mgr.set_sse_sender(sender).await;
ext_mgr.set_sse_sender(sender.clone()).await;
}
// Snapshot memory for trace recording before the agent starts
if let Some(ref recorder) = components.recording_handle
&& let Some(ref ws) = components.workspace
{
recorder.snapshot_memory(ws).await;
}
let http_interceptor = components
.recording_handle
.as_ref()
.map(|r| r.http_interceptor());
let deps = AgentDeps {
store: components.db,
llm: components.llm,
@@ -667,6 +676,8 @@ async fn async_main() -> anyhow::Result<()> {
skills_config: config.skills.clone(),
hooks: components.hooks,
cost_guard: components.cost_guard,
sse_tx: sse_sender,
http_interceptor,
};
let agent = Agent::new(
@@ -687,6 +698,13 @@ async fn async_main() -> anyhow::Result<()> {
// ── Shutdown ────────────────────────────────────────────────────────
// Flush LLM trace recording if enabled
if let Some(ref recorder) = components.recording_handle
&& let Err(e) = recorder.flush().await
{
tracing::warn!("Failed to write LLM trace: {}", e);
}
if let Some(ref mut server) = webhook_server {
server.shutdown().await;
}
@@ -700,16 +718,6 @@ async fn async_main() -> anyhow::Result<()> {
tracing::info!("Agent shutdown complete");
// Check if a restart was requested via the gateway API.
if let Some(ref gw_state) = gateway_state
&& gw_state
.restart_requested
.load(std::sync::atomic::Ordering::Relaxed)
{
eprintln!("Restarting IronClaw (exit code 75)...");
std::process::exit(75);
}
Ok(())
}
@@ -911,11 +919,14 @@ async fn setup_wasm_channels(
let pairing_store = Arc::new(PairingStore::new());
let settings_store: Option<Arc<dyn ironclaw::db::SettingsStore>> =
database.map(|db| Arc::clone(db) as Arc<dyn ironclaw::db::SettingsStore>);
let loader = WasmChannelLoader::new(
let mut loader = WasmChannelLoader::new(
Arc::clone(&runtime),
Arc::clone(&pairing_store),
settings_store,
);
if let Some(secrets) = secrets_store {
loader = loader.with_secrets_store(Arc::clone(secrets));
}
let results = match loader
.load_from_dir(&config.channels.wasm_channels_dir)
@@ -939,6 +950,7 @@ async fn setup_wasm_channels(
let secret_name = loaded.webhook_secret_name();
let sig_key_secret_name = loaded.signature_key_secret_name();
let hmac_secret_name = loaded.hmac_secret_name();
let webhook_secret = if let Some(secrets) = secrets_store {
secrets
@@ -979,9 +991,11 @@ async fn setup_wasm_channels(
);
}
// Inject owner_id for Telegram so the bot only responds to the bound user.
if channel_name == "telegram"
&& let Some(owner_id) = config.channels.telegram_owner_id
// Inject owner_id if configured for this channel.
if let Some(&owner_id) = config
.channels
.wasm_channel_owner_ids
.get(channel_name.as_str())
{
config_updates.insert("owner_id".to_string(), serde_json::json!(owner_id));
}
@@ -1031,6 +1045,17 @@ async fn setup_wasm_channels(
}
}
// Register HMAC signing secret if declared in capabilities
if let Some(ref hmac_secret_name) = hmac_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(secret) = secrets.get_decrypted("default", hmac_secret_name).await
{
wasm_router
.register_hmac_secret(&channel_name, secret.expose())
.await;
tracing::info!(channel = %channel_name, "Registered HMAC signing secret");
}
if let Some(secrets) = secrets_store {
match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await {
Ok(count) => {
+3 -2
View File
@@ -14,7 +14,6 @@ use axum::extract::{Request, State};
use axum::http::StatusCode;
use axum::middleware::Next;
use axum::response::Response;
use rand::Rng;
use serde::{Deserialize, Serialize};
use subtle::ConstantTimeEq;
use tokio::sync::RwLock;
@@ -98,8 +97,10 @@ impl Default for TokenStore {
/// Generate a cryptographically random token (32 bytes, hex-encoded = 64 chars).
fn generate_token() -> String {
use rand::RngCore;
use rand::rngs::OsRng;
let mut bytes = [0u8; 32];
rand::thread_rng().fill(&mut bytes);
OsRng.fill_bytes(&mut bytes);
// Hex-encode without pulling in a crate: fixed-size array, no allocation concern.
bytes.iter().fold(String::with_capacity(64), |mut s, b| {
use std::fmt::Write;
+18 -5
View File
@@ -10,6 +10,7 @@ use std::time::{SystemTime, UNIX_EPOCH};
use fs4::FileExt;
use rand::Rng;
use rand::rngs::OsRng;
use serde::{Deserialize, Serialize};
use crate::bootstrap::ironclaw_base_dir;
@@ -30,6 +31,9 @@ pub enum PairingStoreError {
#[error("Invalid channel: {0}")]
InvalidChannel(String),
#[error("Invalid path: {0}")]
InvalidPath(String),
#[error("IO error: {0}")]
Io(#[from] std::io::Error),
@@ -144,7 +148,7 @@ fn is_expired(req: &PairingRequest, now_secs: u64) -> bool {
}
fn random_code() -> String {
let mut rng = rand::thread_rng();
let mut rng = OsRng;
(0..PAIRING_CODE_LENGTH)
.map(|_| {
let idx = rng.gen_range(0..PAIRING_ALPHABET.len());
@@ -154,7 +158,7 @@ fn random_code() -> String {
}
fn generate_unique_code(existing: &HashSet<String>) -> String {
let mut rng = rand::thread_rng();
let mut rng = OsRng;
for _ in 0..500 {
let code = random_code();
if !existing.contains(&code) {
@@ -224,7 +228,10 @@ impl PairingStore {
meta: Option<serde_json::Value>,
) -> Result<UpsertResult, PairingStoreError> {
let path = pairing_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
let mut file = fs::OpenOptions::new()
.read(true)
@@ -319,7 +326,10 @@ impl PairingStore {
fn record_failed_approve(&self, channel: &str) -> Result<(), PairingStoreError> {
let path = approve_attempts_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
// Open (or create) and lock before reading so concurrent callers
// don't clobber each other's writes.
@@ -462,7 +472,10 @@ impl PairingStore {
}
let path = allow_from_path(&self.base_dir, channel)?;
fs::create_dir_all(path.parent().unwrap())?;
let parent = path.parent().ok_or_else(|| {
PairingStoreError::InvalidPath(format!("path has no parent: {}", path.display()))
})?;
fs::create_dir_all(parent)?;
let file = fs::OpenOptions::new()
.read(true)
+3
View File
@@ -47,6 +47,9 @@ pub enum RegistryError {
actual_sha256: String,
},
#[error("Missing SHA256 checksum for '{name}' artifact. Use --build to build from source.")]
MissingChecksum { name: String },
#[error(
"Source fallback unavailable for '{name}' after artifact install failed. Retry artifact download or run from a repository checkout."
)]
+17 -8
View File
@@ -20,6 +20,10 @@ const ALLOWED_ARTIFACT_HOSTS: &[&str] = &[
];
fn should_attempt_source_fallback(err: &RegistryError) -> bool {
// MissingChecksum is intentionally allowed here — it's a bootstrapping issue
// (no release has populated checksums yet), not a security concern. Source
// builds use local trusted code. ChecksumMismatch (tampered artifact) and
// InvalidManifest (structural problem) remain blocked.
!matches!(
err,
RegistryError::AlreadyInstalled { .. }
@@ -367,15 +371,15 @@ impl RegistryInstaller {
// Require SHA256 — refuse to install unverified binaries. Check before
// downloading to avoid wasting bandwidth on manifests that are missing
// checksums.
// checksums. Uses MissingChecksum (not InvalidManifest) so that
// install_with_source_fallback can fall back to building from source
// when checksums haven't been populated yet (bootstrapping).
let expected_sha =
artifact
.sha256
.as_ref()
.ok_or_else(|| RegistryError::InvalidManifest {
.ok_or_else(|| RegistryError::MissingChecksum {
name: manifest.name.clone(),
field: "artifacts.wasm32-wasip2.sha256",
reason: "sha256 is required for artifact downloads".to_string(),
})?;
let target_dir = match manifest.kind {
@@ -500,7 +504,7 @@ impl RegistryInstaller {
if prefer_build || !has_artifact {
self.install_from_source(manifest, force).await
} else {
self.install_from_artifact(manifest, force).await
self.install_with_source_fallback(manifest, force).await
}
}
@@ -905,9 +909,8 @@ mod tests {
let result = installer.install_from_artifact(&manifest, false).await;
match result {
Err(RegistryError::InvalidManifest { field, reason, .. }) => {
assert_eq!(field, "artifacts.wasm32-wasip2.sha256");
assert!(reason.contains("required"), "reason: {}", reason);
Err(RegistryError::MissingChecksum { name }) => {
assert_eq!(name, "demo");
}
other => panic!("unexpected result: {:?}", other),
}
@@ -942,6 +945,12 @@ mod tests {
reason: "host not allowed".to_string(),
};
assert!(!should_attempt_source_fallback(&invalid));
// MissingChecksum SHOULD allow source fallback (bootstrapping)
let missing = RegistryError::MissingChecksum {
name: "demo".to_string(),
};
assert!(should_attempt_source_fallback(&missing));
}
#[test]
+14 -6
View File
@@ -47,14 +47,22 @@ impl SafetyLayer {
/// Sanitize tool output before it reaches the LLM.
pub fn sanitize_tool_output(&self, tool_name: &str, output: &str) -> SanitizedOutput {
// Check length limits first
// Check length limits — keep the beginning so the LLM has partial data
if output.len() > self.config.max_output_length {
// Find a safe truncation point on a char boundary
let mut cut = self.config.max_output_length;
while cut > 0 && !output.is_char_boundary(cut) {
cut -= 1;
}
let truncated = &output[..cut];
let notice = format!(
"\n\n[... truncated: showing {}/{} bytes. Use the json tool with \
source_tool_call_id to query the full output.]",
cut,
output.len()
);
return SanitizedOutput {
content: format!(
"[Output truncated: {} bytes exceeded maximum of {} bytes]",
output.len(),
self.config.max_output_length
),
content: format!("{}{}", truncated, notice),
warnings: vec![InjectionWarning {
pattern: "output_too_large".to_string(),
severity: Severity::Low,
+51 -28
View File
@@ -63,6 +63,29 @@ pub struct ContainerRunner {
proxy_port: u16,
}
/// Append `text` into `buffer` up to `limit` bytes without breaking UTF-8.
///
/// Returns `true` when truncation occurred.
fn append_with_limit(buffer: &mut String, text: &str, limit: usize) -> bool {
if text.is_empty() {
return false;
}
if buffer.len() >= limit {
return true;
}
let remaining = limit - buffer.len();
if text.len() <= remaining {
buffer.push_str(text);
return false;
}
let end = crate::util::floor_char_boundary(text, remaining);
buffer.push_str(&text[..end]);
true
}
impl ContainerRunner {
/// Create a new container runner.
pub fn new(docker: Docker, image: String, proxy_port: u16) -> Self {
@@ -393,23 +416,11 @@ impl ContainerRunner {
match result {
Ok(LogOutput::StdOut { message }) => {
let text = String::from_utf8_lossy(&message);
if stdout.len() + text.len() > half_max {
truncated = true;
let remaining = half_max.saturating_sub(stdout.len());
stdout.push_str(&text[..remaining.min(text.len())]);
} else {
stdout.push_str(&text);
}
truncated |= append_with_limit(&mut stdout, &text, half_max);
}
Ok(LogOutput::StdErr { message }) => {
let text = String::from_utf8_lossy(&message);
if stderr.len() + text.len() > half_max {
truncated = true;
let remaining = half_max.saturating_sub(stderr.len());
stderr.push_str(&text[..remaining.min(text.len())]);
} else {
stderr.push_str(&text);
}
truncated |= append_with_limit(&mut stderr, &text, half_max);
}
Ok(_) => {}
Err(e) => {
@@ -439,23 +450,11 @@ impl ContainerRunner {
match result {
Ok(LogOutput::StdOut { message }) => {
let text = String::from_utf8_lossy(&message);
if stdout.len() < half_max {
let remaining = half_max.saturating_sub(stdout.len());
stdout.push_str(&text[..remaining.min(text.len())]);
if text.len() > remaining {
truncated = true;
}
}
truncated |= append_with_limit(&mut stdout, &text, half_max);
}
Ok(LogOutput::StdErr { message }) => {
let text = String::from_utf8_lossy(&message);
if stderr.len() < half_max {
let remaining = half_max.saturating_sub(stderr.len());
stderr.push_str(&text[..remaining.min(text.len())]);
if text.len() > remaining {
truncated = true;
}
}
truncated |= append_with_limit(&mut stderr, &text, half_max);
}
Ok(_) => {}
Err(e) => {
@@ -577,6 +576,30 @@ fn unix_socket_candidates_from_env(
mod tests {
use super::*;
#[test]
fn append_with_limit_truncates_on_utf8_boundary() {
let mut out = String::new();
let truncated = append_with_limit(&mut out, "ab🙂cd", 5);
assert!(truncated);
assert_eq!(out, "ab");
}
#[test]
fn append_with_limit_marks_truncated_when_full() {
let mut out = "abc".to_string();
let truncated = append_with_limit(&mut out, "z", 3);
assert!(truncated);
assert_eq!(out, "abc");
}
#[test]
fn append_with_limit_appends_without_truncation() {
let mut out = String::new();
let truncated = append_with_limit(&mut out, "hello", 10);
assert!(!truncated);
assert_eq!(out, "hello");
}
#[cfg(unix)]
#[test]
fn test_unix_socket_candidates_include_rootless_paths() {
+20 -1
View File
@@ -59,7 +59,7 @@ impl SecretsCrypto {
/// Generate a random salt for a new secret.
pub fn generate_salt() -> Vec<u8> {
let mut salt = vec![0u8; SALT_SIZE];
rand::RngCore::fill_bytes(&mut rand::thread_rng(), &mut salt);
rand::RngCore::fill_bytes(&mut OsRng, &mut salt);
salt
}
@@ -247,4 +247,23 @@ mod tests {
let decrypted = crypto.decrypt(&encrypted, &salt).unwrap();
assert_eq!(decrypted.expose().as_bytes(), plaintext.as_slice());
}
#[test]
fn test_generate_salt_correct_length() {
let salt = SecretsCrypto::generate_salt();
assert_eq!(salt.len(), super::SALT_SIZE);
}
#[test]
fn test_generate_salt_nonzero() {
let salt = SecretsCrypto::generate_salt();
assert!(salt.iter().any(|&b| b != 0), "salt should not be all zeros");
}
#[test]
fn test_generate_salt_unique() {
let s1 = SecretsCrypto::generate_salt();
let s2 = SecretsCrypto::generate_salt();
assert_ne!(s1, s2, "two generated salts should not be identical");
}
}
+2 -1
View File
@@ -28,8 +28,9 @@ const MASTER_KEY_ACCOUNT: &str = "master_key";
/// Generate a random 32-byte master key.
pub fn generate_master_key() -> Vec<u8> {
use rand::RngCore;
use rand::rngs::OsRng;
let mut key = vec![0u8; 32];
rand::thread_rng().fill_bytes(&mut key);
OsRng.fill_bytes(&mut key);
key
}
+28 -15
View File
@@ -249,10 +249,10 @@ pub struct ChannelSettings {
#[serde(default)]
pub signal_group_allow_from: Option<String>,
/// Telegram owner user ID. When set, the bot only responds to this user.
/// Captured during setup by having the user message the bot.
/// Per-channel owner user IDs. When set, the channel only responds to this user.
/// Key: channel name (e.g., "telegram"), Value: owner user ID.
#[serde(default)]
pub telegram_owner_id: Option<i64>,
pub wasm_channel_owner_ids: std::collections::HashMap<String, i64>,
/// Enabled WASM channels by name.
/// Channels not in this list but present in the channels directory will still load.
@@ -1049,28 +1049,37 @@ mod tests {
}
#[test]
fn test_telegram_owner_id_db_round_trip() {
fn test_wasm_channel_owner_ids_db_round_trip() {
let mut settings = Settings::default();
settings.channels.telegram_owner_id = Some(123456789);
settings
.channels
.wasm_channel_owner_ids
.insert("telegram".to_string(), 123456789);
let map = settings.to_db_map();
let restored = Settings::from_db_map(&map);
assert_eq!(restored.channels.telegram_owner_id, Some(123456789));
assert_eq!(
restored.channels.wasm_channel_owner_ids.get("telegram"),
Some(&123456789)
);
}
#[test]
fn test_telegram_owner_id_default_none() {
fn test_wasm_channel_owner_ids_default_empty() {
let settings = Settings::default();
assert_eq!(settings.channels.telegram_owner_id, None);
assert!(settings.channels.wasm_channel_owner_ids.is_empty());
}
#[test]
fn test_telegram_owner_id_via_set() {
fn test_wasm_channel_owner_ids_via_set() {
let mut settings = Settings::default();
settings
.set("channels.telegram_owner_id", "987654321")
.set("channels.wasm_channel_owner_ids.telegram", "987654321")
.unwrap();
assert_eq!(settings.channels.telegram_owner_id, Some(987654321));
assert_eq!(
settings.channels.wasm_channel_owner_ids.get("telegram"),
Some(&987654321)
);
}
#[test]
@@ -1406,7 +1415,11 @@ mod tests {
channels: ChannelSettings {
http_enabled: true,
http_port: Some(9090),
telegram_owner_id: Some(12345),
wasm_channel_owner_ids: {
let mut m = std::collections::HashMap::new();
m.insert("telegram".to_string(), 12345);
m
},
..Default::default()
},
heartbeat: HeartbeatSettings {
@@ -1473,9 +1486,9 @@ mod tests {
assert!(restored.channels.http_enabled, "http_enabled lost");
assert_eq!(restored.channels.http_port, Some(9090), "http_port lost");
assert_eq!(
restored.channels.telegram_owner_id,
Some(12345),
"telegram_owner_id lost"
restored.channels.wasm_channel_owner_ids.get("telegram"),
Some(&12345),
"wasm_channel_owner_ids lost"
);
assert!(restored.heartbeat.enabled, "heartbeat.enabled lost");
assert_eq!(
+4 -339
View File
@@ -1,6 +1,6 @@
//! Channel-specific setup flows.
//! Channel setup flows.
//!
//! Each channel (Telegram, HTTP, etc.) has its own setup function that:
//! Each channel (HTTP, Signal, WASM, etc.) has its own setup function that:
//! 1. Displays setup instructions
//! 2. Collects configuration (tokens, ports, etc.)
//! 3. Validates the configuration
@@ -9,9 +9,7 @@
use std::sync::Arc;
use base64::Engine;
use reqwest::Client;
use secrecy::{ExposeSecret, SecretString};
use serde::Deserialize;
use url::Url;
use uuid::Uuid;
@@ -105,261 +103,6 @@ impl SecretsContext {
}
}
/// Result of Telegram setup.
#[derive(Debug, Clone)]
pub struct TelegramSetupResult {
pub enabled: bool,
pub bot_username: Option<String>,
pub webhook_secret: Option<String>,
pub owner_id: Option<i64>,
}
/// Telegram Bot API response for getMe.
#[derive(Debug, Deserialize)]
struct TelegramGetMeResponse {
ok: bool,
result: Option<TelegramUser>,
}
#[derive(Debug, Deserialize)]
struct TelegramUser {
username: Option<String>,
#[allow(dead_code)]
first_name: String,
}
/// Telegram Bot API response for getUpdates.
#[derive(Debug, Deserialize)]
struct TelegramGetUpdatesResponse {
ok: bool,
result: Vec<TelegramUpdate>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdate {
update_id: i64,
message: Option<TelegramUpdateMessage>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdateMessage {
from: Option<TelegramUpdateUser>,
}
#[derive(Debug, Deserialize)]
struct TelegramUpdateUser {
id: i64,
first_name: String,
username: Option<String>,
}
/// Set up Telegram bot channel.
///
/// Guides the user through:
/// 1. Creating a bot with @BotFather
/// 2. Entering the bot token
/// 3. Validating the token
/// 4. Saving the token to the database
pub async fn setup_telegram(
secrets: &SecretsContext,
settings: &Settings,
) -> Result<TelegramSetupResult, ChannelSetupError> {
println!("Telegram Setup:");
println!();
print_info("To create a Telegram bot:");
print_info("1. Open Telegram and message @BotFather");
print_info("2. Send /newbot and follow the prompts");
print_info("3. Copy the bot token (looks like 123456:ABC-DEF...)");
println!();
// Check if token already exists
if secrets.secret_exists("telegram_bot_token").await {
print_info("Existing Telegram token found in database.");
if !confirm("Replace existing token?", false)? {
// Still offer to configure webhook secret and owner binding
let webhook_secret = setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
let owner_id = bind_telegram_owner_flow(secrets, settings).await?;
return Ok(TelegramSetupResult {
enabled: true,
bot_username: None,
webhook_secret,
owner_id,
});
}
}
loop {
let token = secret_input("Bot token (from @BotFather)")?;
// Validate the token
print_info("Validating bot token...");
match validate_telegram_token(&token).await {
Ok(username) => {
print_success(&format!(
"Bot validated: @{}",
username.as_deref().unwrap_or("unknown")
));
// Save to database
secrets.save_secret("telegram_bot_token", &token).await?;
print_success("Token saved to database");
// Bind bot to owner's Telegram account
let owner_id = bind_telegram_owner(&token).await?;
// Offer webhook secret configuration
let webhook_secret =
setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
return Ok(TelegramSetupResult {
enabled: true,
bot_username: username,
webhook_secret,
owner_id,
});
}
Err(e) => {
print_error(&format!("Token validation failed: {}", e));
if !confirm("Try again?", true)? {
return Ok(TelegramSetupResult {
enabled: false,
bot_username: None,
webhook_secret: None,
owner_id: None,
});
}
}
}
}
}
/// Bind the bot to the owner's Telegram account by having them send a message.
///
/// Polls `getUpdates` until a message arrives, then captures the sender's user ID.
/// Returns `None` if the user declines or the flow times out.
async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, ChannelSetupError> {
println!();
print_info("Account Binding (recommended):");
print_info("Binding restricts the bot so only YOU can use it.");
print_info("Without this, anyone who finds your bot can send it messages.");
println!();
if !confirm("Bind bot to your Telegram account?", true)? {
print_info("Skipping account binding. Bot will accept messages from all users.");
return Ok(None);
}
print_info("Send any message (e.g. /start) to your bot in Telegram.");
print_info("Waiting for your message (up to 120 seconds)...");
let client = Client::builder()
.timeout(std::time::Duration::from_secs(35))
.build()
.map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
// Clear any existing webhook so getUpdates works
let delete_url = format!(
"https://api.telegram.org/bot{}/deleteWebhook",
token.expose_secret()
);
if let Err(e) = client.post(&delete_url).send().await {
tracing::warn!("Failed to delete webhook (getUpdates may not work): {e}");
}
let updates_url = format!(
"https://api.telegram.org/bot{}/getUpdates",
token.expose_secret()
);
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(120);
while std::time::Instant::now() < deadline {
let response = client
.get(&updates_url)
.query(&[("timeout", "30"), ("allowed_updates", "[\"message\"]")])
.send()
.await
.map_err(|e| ChannelSetupError::Network(format!("getUpdates request failed: {}", e)))?;
if !response.status().is_success() {
return Err(ChannelSetupError::Network(format!(
"getUpdates returned status {}",
response.status()
)));
}
let body: TelegramGetUpdatesResponse = response.json().await.map_err(|e| {
ChannelSetupError::Network(format!("Failed to parse getUpdates response: {}", e))
})?;
if !body.ok {
return Err(ChannelSetupError::Network(
"Telegram API returned error for getUpdates".to_string(),
));
}
// Find the first message with a sender
for update in &body.result {
if let Some(ref msg) = update.message
&& let Some(ref from) = msg.from
{
let display_name = from
.username
.as_ref()
.map(|u| format!("@{}", u))
.unwrap_or_else(|| from.first_name.clone());
print_success(&format!(
"Received message from {} (ID: {})",
display_name, from.id
));
// Acknowledge the update so it doesn't pile up
let ack_url = format!(
"https://api.telegram.org/bot{}/getUpdates",
token.expose_secret()
);
if let Err(e) = client
.get(&ack_url)
.query(&[("offset", &(update.update_id + 1).to_string())])
.send()
.await
{
tracing::warn!("Failed to acknowledge Telegram update: {e}");
}
return Ok(Some(from.id));
}
}
}
print_error("Timed out waiting for a message. You can re-run setup to try again.");
print_info("Bot will accept messages from all users until owner is bound.");
Ok(None)
}
/// Bind flow when the token already exists (reads from secrets store).
///
/// Retrieves the saved bot token and delegates to `bind_telegram_owner`.
async fn bind_telegram_owner_flow(
secrets: &SecretsContext,
settings: &Settings,
) -> Result<Option<i64>, ChannelSetupError> {
if settings.channels.telegram_owner_id.is_some() {
print_info("Bot is already bound to a Telegram account.");
if !confirm("Re-bind to a different account?", false)? {
return Ok(settings.channels.telegram_owner_id);
}
}
// We need the token to poll getUpdates
let token = secrets.get_secret("telegram_bot_token").await?;
bind_telegram_owner(&token).await
}
/// Set up a tunnel for exposing the agent to the internet.
///
/// This is shared across all channels that need webhook endpoints.
@@ -725,84 +468,6 @@ fn setup_tunnel_static() -> Result<TunnelSettings, ChannelSetupError> {
})
}
/// Set up Telegram webhook secret for signature validation.
///
/// Returns the webhook secret if configured.
async fn setup_telegram_webhook_secret(
secrets: &SecretsContext,
tunnel: &TunnelSettings,
) -> Result<Option<String>, ChannelSetupError> {
if tunnel.public_url.is_none() {
print_info("");
print_info("No tunnel configured. Telegram will use polling mode (30s+ delay).");
print_info("Run setup again to configure a tunnel for instant delivery.");
return Ok(None);
}
println!();
print_info("Telegram Webhook Security:");
print_info("A webhook secret adds an extra layer of security by validating");
print_info("that requests actually come from Telegram's servers.");
if !confirm("Generate a webhook secret?", true)? {
return Ok(None);
}
let secret = generate_webhook_secret();
secrets
.save_secret(
"telegram_webhook_secret",
&SecretString::from(secret.clone()),
)
.await?;
print_success("Webhook secret generated and saved");
Ok(Some(secret))
}
/// Validate a Telegram bot token by calling the getMe API.
///
/// Returns the bot's username if valid.
pub async fn validate_telegram_token(
token: &SecretString,
) -> Result<Option<String>, ChannelSetupError> {
let client = Client::builder()
.timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
let url = format!(
"https://api.telegram.org/bot{}/getMe",
token.expose_secret()
);
let response = client
.get(&url)
.send()
.await
.map_err(|e| ChannelSetupError::Network(format!("Request failed: {}", e)))?;
if !response.status().is_success() {
return Err(ChannelSetupError::Network(format!(
"API returned status {}",
response.status()
)));
}
let body: TelegramGetMeResponse = response
.json()
.await
.map_err(|e| ChannelSetupError::Network(format!("Failed to parse response: {}", e)))?;
if body.ok {
Ok(body.result.and_then(|u| u.username))
} else {
Err(ChannelSetupError::Network(
"Telegram API returned error".to_string(),
))
}
}
/// Result of HTTP webhook setup.
#[derive(Debug, Clone)]
pub struct HttpSetupResult {
@@ -1236,9 +901,9 @@ fn validate_cloudflare_token_format(token: &str) -> bool {
/// Generate a random secret of specified length (in bytes).
fn generate_secret_with_length(length: usize) -> String {
use rand::RngCore;
let mut rng = rand::thread_rng();
use rand::rngs::OsRng;
let mut bytes = vec![0u8; length];
rng.fill_bytes(&mut bytes);
OsRng.fill_bytes(&mut bytes);
bytes.iter().map(|b| format!("{:02x}", b)).collect()
}
+1 -4
View File
@@ -24,10 +24,7 @@ mod prompts;
#[cfg(any(feature = "postgres", feature = "libsql"))]
mod wizard;
pub use channels::{
ChannelSetupError, SecretsContext, setup_http, setup_telegram, setup_tunnel,
validate_telegram_token,
};
pub use channels::{ChannelSetupError, SecretsContext, setup_http, setup_tunnel};
pub use prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
print_success, secret_input, select_many, select_one,
+1 -10
View File
@@ -26,7 +26,7 @@ use crate::llm::{SessionConfig, SessionManager};
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::settings::{KeySource, Settings};
use crate::setup::channels::{
SecretsContext, setup_http, setup_signal, setup_telegram, setup_tunnel, setup_wasm_channel,
SecretsContext, setup_http, setup_signal, setup_tunnel, setup_wasm_channel,
};
use crate::setup::prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
@@ -1670,15 +1670,6 @@ impl SetupWizard {
let result = if let Some(cap_file) = discovered_by_name.get(&channel_name) {
if !cap_file.setup.required_secrets.is_empty() {
setup_wasm_channel(ctx, &channel_name, &cap_file.setup).await?
} else if channel_name == "telegram" {
let telegram_result = setup_telegram(ctx, &self.settings).await?;
if let Some(owner_id) = telegram_result.owner_id {
self.settings.channels.telegram_owner_id = Some(owner_id);
}
crate::setup::channels::WasmChannelSetupResult {
enabled: telegram_result.enabled,
channel_name: "telegram".to_string(),
}
} else {
print_info(&format!(
"No setup configuration found for {}",
+33
View File
@@ -288,6 +288,18 @@ impl SkillRegistry {
self.skills.len()
}
/// Retain only skills whose names are in the given allowlist.
///
/// If `names` is empty, this is a no-op (all skills are kept).
pub fn retain_only(&mut self, names: &[&str]) {
if names.is_empty() {
return;
}
let names_set: HashSet<&str> = names.iter().copied().collect();
self.skills
.retain(|s| names_set.contains(s.manifest.name.as_str()));
}
/// Check if a skill with the given name is loaded.
pub fn has(&self, name: &str) -> bool {
self.skills.iter().any(|s| s.manifest.name == name)
@@ -982,6 +994,27 @@ mod tests {
assert_eq!(skill.lowercased_tags, vec!["email", "prose"]);
}
#[tokio::test]
async fn test_retain_only_empty_is_noop() {
let dir = tempfile::tempdir().unwrap();
fs::write(
dir.path().join("SKILL.md"),
"---\nname: keep-me\ndescription: test\nactivation:\n keywords: [\"test\"]\n---\n\nKeep this skill.\n",
)
.unwrap();
let mut registry = SkillRegistry::new(dir.path().to_path_buf());
registry.discover_all().await;
assert_eq!(registry.count(), 1);
registry.retain_only(&[]);
assert_eq!(
registry.count(),
1,
"empty retain_only should keep all skills"
);
}
#[test]
fn test_compute_hash_deterministic() {
let h1 = compute_hash("hello world");
+2
View File
@@ -293,6 +293,8 @@ impl TestHarnessBuilder {
skills_config: SkillsConfig::default(),
hooks,
cost_guard,
sse_tx: None,
http_interceptor: None,
};
TestHarness {
+2 -2
View File
@@ -218,7 +218,7 @@ impl Tool for ToolAuthTool {
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
// Auto-activate after successful auth so tools are available immediately
if result.status == "authenticated" {
if result.is_authenticated() {
match self.manager.activate(name).await {
Ok(activate_result) => {
let output = serde_json::json!({
@@ -324,7 +324,7 @@ impl Tool for ToolActivateTool {
// Activation failed due to missing auth; initiate auth flow
// so the agent loop can show the auth card.
match self.manager.auth(name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
Ok(auth_result) if auth_result.is_authenticated() => {
// Auth succeeded (e.g. env var was set); retry activation.
let result = self
.manager
+226 -31
View File
@@ -1,4 +1,12 @@
//! HTTP request tool.
//!
//! Unified HTTP tool that handles both simple page/API fetches (GET, no auth)
//! and full API calls (any method, custom headers, credential injection).
//!
//! - Plain GET without auth headers/body → no approval needed, follows redirects
//! - Everything else → requires approval
//!
//! Replaces the former `web_fetch` tool which was a separate GET-only tool.
use std::collections::HashMap;
use std::net::{IpAddr, ToSocketAddrs};
@@ -25,6 +33,16 @@ use crate::tools::builtin::convert_html_to_markdown;
/// HTTP wrapper uses the same limit for consistency.
const MAX_RESPONSE_SIZE: usize = 5 * 1024 * 1024;
/// Maximum number of redirects to follow for simple GET requests.
const MAX_REDIRECTS: usize = 3;
/// Descriptive User-Agent so public APIs don't reject bare requests.
const USER_AGENT: &str = concat!(
"IronClaw-Agent/",
env!("CARGO_PKG_VERSION"),
" (https://github.com/nearai/ironclaw)"
);
/// Tool for making HTTP requests.
pub struct HttpTool {
client: Client,
@@ -38,6 +56,7 @@ impl HttpTool {
let client = Client::builder()
.timeout(Duration::from_secs(30))
.redirect(reqwest::redirect::Policy::none())
.user_agent(USER_AGENT)
.build()
.expect("Failed to create HTTP client");
@@ -201,7 +220,10 @@ impl Tool for HttpTool {
}
fn description(&self) -> &str {
"Make HTTP requests to external APIs. Supports GET, POST, PUT, DELETE methods."
"Make HTTP requests. Simple GET requests (no auth, no custom headers) run without \
approval and follow redirects use for fetching weather, public JSON APIs, web pages, \
and documentation. Requests with authentication, custom headers, or non-GET methods \
(POST, PUT, DELETE, PATCH) require user approval."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -245,7 +267,7 @@ impl Tool for HttpTool {
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
@@ -311,7 +333,7 @@ impl Tool for HttpTool {
let matched: Vec<crate::secrets::CredentialMapping> = registry.find_for_host(host);
for mapping in &matched {
match store
.get_decrypted(&_ctx.user_id, &mapping.secret_name)
.get_decrypted(&ctx.user_id, &mapping.secret_name)
.await
{
Ok(secret) => {
@@ -343,25 +365,133 @@ impl Tool for HttpTool {
.scan_http_request(parsed_url.as_str(), &headers_vec, body_bytes.as_deref())
.map_err(|e| ToolError::NotAuthorized(format!("{}", e)))?;
// Execute request
let response = request.send().await.map_err(|e| {
if e.is_timeout() {
ToolError::Timeout(Duration::from_secs(30))
} else {
ToolError::ExternalService(e.to_string())
// Build the interceptor request descriptor for recording/replay
let intercept_req = crate::llm::recording::HttpExchangeRequest {
method: method.to_uppercase(),
url: parsed_url.to_string(),
headers: headers_vec.clone(),
body: body_bytes
.as_ref()
.map(|b| String::from_utf8_lossy(b).into_owned()),
};
// Check HTTP interceptor (replay mode returns pre-recorded response)
if let Some(ref interceptor) = ctx.http_interceptor
&& let Some(recorded) = interceptor.before_request(&intercept_req).await
{
let headers: HashMap<String, String> = recorded.headers.iter().cloned().collect();
let body: serde_json::Value = serde_json::from_str(&recorded.body)
.unwrap_or_else(|_| serde_json::Value::String(recorded.body.clone()));
let result = serde_json::json!({
"status": recorded.status,
"headers": headers,
"body": body
});
return Ok(ToolOutput::success(result, start.elapsed()).with_raw(recorded.body));
}
// Determine if this is a simple GET (eligible for redirect following).
let is_simple_get =
method.eq_ignore_ascii_case("GET") && headers_vec.is_empty() && body_bytes.is_none();
// Execute request, optionally following redirects for simple GETs.
let response = if is_simple_get {
let mut redirects_remaining = MAX_REDIRECTS;
loop {
let resp = self
.client
.get(parsed_url.clone())
.header(
reqwest::header::ACCEPT,
"text/markdown, text/html;q=0.9, application/json;q=0.9, */*;q=0.8",
)
.send()
.await
.map_err(|e| {
if e.is_timeout() {
ToolError::Timeout(Duration::from_secs(30))
} else {
ToolError::ExternalService(e.to_string())
}
})?;
let status = resp.status().as_u16();
if (300..400).contains(&status) {
if redirects_remaining == 0 {
return Err(ToolError::ExecutionFailed(format!(
"too many redirects (max {})",
MAX_REDIRECTS
)));
}
let location = resp
.headers()
.get(reqwest::header::LOCATION)
.and_then(|v| v.to_str().ok())
.ok_or_else(|| {
ToolError::ExecutionFailed(format!(
"redirect (HTTP {}) has no Location header",
status
))
})?;
let next_url_str =
if location.starts_with("http://") || location.starts_with("https://") {
location.to_string()
} else {
parsed_url
.join(location)
.map(|u| u.to_string())
.map_err(|e| {
ToolError::ExecutionFailed(format!(
"could not resolve relative redirect '{}': {}",
location, e
))
})?
};
// SSRF re-validation on every hop.
parsed_url = validate_url(&next_url_str)?;
let detector = LeakDetector::new();
detector
.scan_http_request(parsed_url.as_str(), &[], None)
.map_err(|e| ToolError::NotAuthorized(e.to_string()))?;
redirects_remaining -= 1;
tracing::debug!(
to = %parsed_url,
hops_left = redirects_remaining,
"http tool following redirect"
);
continue;
}
break resp;
}
})?;
} else {
let resp = request.send().await.map_err(|e| {
if e.is_timeout() {
ToolError::Timeout(Duration::from_secs(30))
} else {
ToolError::ExternalService(e.to_string())
}
})?;
let status = resp.status().as_u16();
// Block redirects for non-simple requests (potential SSRF)
if (300..400).contains(&status) {
return Err(ToolError::NotAuthorized(format!(
"request returned redirect (HTTP {}), which is blocked to prevent SSRF",
status
)));
}
resp
};
let status = response.status().as_u16();
// Block redirects: the server tried to send us elsewhere (potential SSRF)
if (300..400).contains(&status) {
return Err(ToolError::NotAuthorized(format!(
"request returned redirect (HTTP {}), which is blocked to prevent SSRF",
status
)));
}
let headers: HashMap<String, String> = response
.headers()
.iter()
@@ -407,6 +537,24 @@ impl Tool for HttpTool {
let body_text = String::from_utf8_lossy(&body_bytes).into_owned();
// Record the HTTP exchange if interceptor is present (recording mode)
if let Some(ref interceptor) = ctx.http_interceptor {
let resp_headers: Vec<(String, String)> = headers
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
interceptor
.after_response(
&intercept_req,
&crate::llm::recording::HttpExchangeResponse {
status,
headers: resp_headers,
body: body_text.clone(),
},
)
.await;
}
#[cfg(feature = "html-to-markdown")]
let body_text = if is_html_response(&headers) {
match convert_html_to_markdown(&body_text, parsed_url.as_str()) {
@@ -453,6 +601,25 @@ impl Tool for HttpTool {
{
return ApprovalRequirement::Always;
}
// 3. Plain GET without headers or body → no approval needed
let method = params
.get("method")
.and_then(|v| v.as_str())
.unwrap_or("GET");
let has_headers = params
.get("headers")
.map(|h| match h {
serde_json::Value::Array(a) => !a.is_empty(),
serde_json::Value::Object(o) => !o.is_empty(),
_ => false,
})
.unwrap_or(false);
let has_body = params.get("body").is_some();
if method.eq_ignore_ascii_case("GET") && !has_headers && !has_body {
return ApprovalRequirement::Never;
}
// Default: outbound HTTP still needs approval unless auto-approved
ApprovalRequirement::UnlessAutoApproved
}
@@ -579,12 +746,37 @@ mod tests {
// ── Approval requirement tests ──────────────────────────────────────
#[test]
fn test_no_auth_headers_returns_unless_auto_approved() {
fn test_plain_get_returns_never() {
let tool = HttpTool::new();
let params = serde_json::json!({
"method": "GET",
"url": "https://api.example.com/data"
});
assert_eq!(tool.requires_approval(&params), ApprovalRequirement::Never);
}
#[test]
fn test_post_returns_unless_auto_approved() {
let tool = HttpTool::new();
let params = serde_json::json!({
"method": "POST",
"url": "https://api.example.com/data",
"body": {"key": "value"}
});
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::UnlessAutoApproved
);
}
#[test]
fn test_get_with_headers_returns_unless_auto_approved() {
let tool = HttpTool::new();
let params = serde_json::json!({
"method": "GET",
"url": "https://api.example.com/data",
"headers": [{"name": "X-Custom", "value": "test"}]
});
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::UnlessAutoApproved
@@ -682,30 +874,24 @@ mod tests {
}
#[test]
fn test_empty_headers_return_unless_auto_approved() {
fn test_empty_headers_get_returns_never() {
let tool = HttpTool::new();
// Empty object
// Empty object — still a plain GET
let params = serde_json::json!({
"method": "GET",
"url": "https://example.com",
"headers": {}
});
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::UnlessAutoApproved
);
assert_eq!(tool.requires_approval(&params), ApprovalRequirement::Never);
// Empty array
// Empty array — still a plain GET
let params = serde_json::json!({
"method": "GET",
"url": "https://example.com",
"headers": []
});
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::UnlessAutoApproved
);
assert_eq!(tool.requires_approval(&params), ApprovalRequirement::Never);
}
// ── Credential registry approval tests ─────────────────────────────
@@ -740,7 +926,7 @@ mod tests {
}
#[test]
fn test_host_without_credential_mapping_returns_unless_auto_approved() {
fn test_host_without_credential_mapping_get_returns_never() {
use crate::tools::wasm::SharedCredentialRegistry;
let registry = Arc::new(SharedCredentialRegistry::new());
@@ -756,10 +942,19 @@ mod tests {
))),
);
// Plain GET with no credentials → Never
let params = serde_json::json!({
"method": "GET",
"url": "https://api.example.com/data"
});
assert_eq!(tool.requires_approval(&params), ApprovalRequirement::Never);
// POST with no credentials → UnlessAutoApproved
let params = serde_json::json!({
"method": "POST",
"url": "https://api.example.com/data",
"body": {"key": "value"}
});
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::UnlessAutoApproved
+86 -7
View File
@@ -15,7 +15,9 @@ impl Tool for JsonTool {
}
fn description(&self) -> &str {
"Parse, query, and transform JSON data. Supports JSONPath-like queries."
"Parse, query, and transform JSON data. Supports JSONPath-like queries. \
Use `source_tool_call_id` to reference the full output of a previous tool call \
(avoids truncation issues with large responses)."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -28,27 +30,48 @@ impl Tool for JsonTool {
"description": "The JSON operation to perform"
},
"data": {
"description": "JSON input data. Pass a string for parse, or any JSON value (object, array, string, number, boolean, null) otherwise."
"description": "JSON input data. Pass a string for parse, or any JSON value otherwise. Not required when source_tool_call_id is provided."
},
"source_tool_call_id": {
"type": "string",
"description": "Reference a previous tool call's full output by its ID (e.g., 'call_abc123'). Use this instead of data when the previous tool output was large and may have been truncated."
},
"path": {
"type": "string",
"description": "JSONPath-like path for query operation (e.g., 'foo.bar[0].baz')"
}
},
"required": ["operation", "data"]
"required": ["operation"]
})
}
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let operation = require_str(&params, "operation")?;
let data = require_param(&params, "data")?;
// Resolve data: from stash (via source_tool_call_id) or from params
let data_value =
if let Some(ref_id) = params.get("source_tool_call_id").and_then(|v| v.as_str()) {
let stash = ctx.tool_output_stash.read().await;
let full_output = stash.get(ref_id).ok_or_else(|| {
ToolError::InvalidParameters(format!(
"no tool output found for call ID '{}'. Available IDs: {:?}",
ref_id,
stash.keys().collect::<Vec<_>>()
))
})?;
// Parse the stashed output as JSON, or wrap as string
serde_json::from_str::<serde_json::Value>(full_output)
.unwrap_or_else(|_| serde_json::Value::String(full_output.clone()))
} else {
require_param(&params, "data")?.clone()
};
let data = &data_value;
let result = match operation {
"parse" => {
@@ -64,7 +87,11 @@ impl Tool for JsonTool {
parsed
}
"stringify" => {
let value = parse_json_input(data)?;
let value = if data.is_string() {
parse_json_input(data)?
} else {
data.clone()
};
let json_str = serde_json::to_string_pretty(&value).map_err(|e| {
ToolError::ExecutionFailed(format!("failed to stringify: {}", e))
})?;
@@ -76,7 +103,11 @@ impl Tool for JsonTool {
ToolError::InvalidParameters("missing 'path' parameter for query".to_string())
})?;
let value = parse_json_input(data)?;
let value = if data.is_string() {
parse_json_input(data)?
} else {
data.clone()
};
query_json(&value, path)?
}
"validate" => {
@@ -190,6 +221,54 @@ mod tests {
assert!(err.to_string().contains("invalid JSON input"));
}
#[tokio::test]
async fn test_query_with_object_data_from_stash() {
use crate::context::JobContext;
let ctx = JobContext::with_user("test", "chat", "test-session");
// Simulate stashed output: the http tool stores serialized JSON
// containing {"status": 200, "body": {"leagues": [{"name": "MLB"}]}}
let stashed = r#"{"status": 200, "body": {"leagues": [{"name": "MLB"}]}}"#;
ctx.tool_output_stash
.write()
.await
.insert("call_http_01".to_string(), stashed.to_string());
let tool = JsonTool;
let params = serde_json::json!({
"operation": "query",
"source_tool_call_id": "call_http_01",
"path": "body.leagues[0].name"
});
let result = tool.execute(params, &ctx).await.unwrap();
assert_eq!(result.result, serde_json::json!("MLB"));
}
#[tokio::test]
async fn test_stringify_with_object_data_from_stash() {
use crate::context::JobContext;
let ctx = JobContext::with_user("test", "chat", "test-session");
let stashed = r#"{"key": "value"}"#;
ctx.tool_output_stash
.write()
.await
.insert("call_01".to_string(), stashed.to_string());
let tool = JsonTool;
let params = serde_json::json!({
"operation": "stringify",
"source_tool_call_id": "call_01"
});
let result = tool.execute(params, &ctx).await.unwrap();
let stringified = result.result.as_str().unwrap();
assert!(stringified.contains("\"key\": \"value\""));
}
#[test]
fn test_json_tool_schema_data_is_freeform() {
let schema = JsonTool.parameters_schema();
+31 -9
View File
@@ -95,15 +95,17 @@ impl Tool for MemorySearchTool {
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Search failed: {}", e)))?;
let result_count = results.len();
let output = serde_json::json!({
"query": query,
"results": results.iter().map(|r| serde_json::json!({
"results": results.into_iter().map(|r| serde_json::json!({
"content": r.content,
"score": r.score,
"path": r.document_path,
"document_id": r.document_id.to_string(),
"is_hybrid_match": r.is_hybrid(),
})).collect::<Vec<_>>(),
"result_count": results.len(),
"result_count": result_count,
});
Ok(ToolOutput::success(output, start.elapsed()))
@@ -140,7 +142,8 @@ impl Tool for MemoryWriteTool {
Use for important facts, decisions, preferences, or lessons learned that should \
be remembered across sessions. Targets: 'memory' for curated long-term facts, \
'daily_log' for timestamped session notes, 'heartbeat' for the periodic \
checklist (HEARTBEAT.md), or provide a custom path for arbitrary file creation."
checklist (HEARTBEAT.md), 'bootstrap' to clear the first-run ritual file, \
or provide a custom path for arbitrary file creation."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -153,7 +156,7 @@ impl Tool for MemoryWriteTool {
},
"target": {
"type": "string",
"description": "Where to write: 'memory' for MEMORY.md, 'daily_log' for today's log, 'heartbeat' for HEARTBEAT.md checklist, or a path like 'projects/alpha/notes.md'",
"description": "Where to write: 'memory' for MEMORY.md, 'daily_log' for today's log, 'heartbeat' for HEARTBEAT.md checklist, 'bootstrap' to clear BOOTSTRAP.md (content is ignored; the file is always cleared), or a path like 'projects/alpha/notes.md'",
"default": "daily_log"
},
"append": {
@@ -175,17 +178,36 @@ impl Tool for MemoryWriteTool {
let content = require_str(&params, "content")?;
let target = params
.get("target")
.and_then(|v| v.as_str())
.unwrap_or("daily_log");
// Bootstrap target: clear BOOTSTRAP.md to mark first-run ritual complete.
// Handled early because it accepts empty content (unlike other targets).
if target == "bootstrap" {
// Write empty content to effectively disable the bootstrap injection.
// system_prompt_for_context() skips empty files.
self.workspace
.write(paths::BOOTSTRAP, "")
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Write failed: {}", e)))?;
let output = serde_json::json!({
"status": "cleared",
"path": paths::BOOTSTRAP,
"message": "BOOTSTRAP.md cleared. First-run ritual will not repeat.",
});
return Ok(ToolOutput::success(output, start.elapsed()));
}
if content.trim().is_empty() {
return Err(ToolError::InvalidParameters(
"content cannot be empty".to_string(),
));
}
let target = params
.get("target")
.and_then(|v| v.as_str())
.unwrap_or("daily_log");
// Reject writes to identity files that are loaded into the system prompt.
// An attacker could use prompt injection to trick the agent into overwriting
// these, poisoning future conversations.

Some files were not shown because too many files have changed in this diff Show More