Compare commits

...
Author SHA1 Message Date
[email protected]andClaude Opus 4.6 97cbe38949 fix(fuzz): address PR review — LazyLock for expensive constructors, fix assertions and docs
- Use std::sync::LazyLock to construct Sanitizer, Validator, and LeakDetector
  once instead of on every fuzz iteration (they compile regex/Aho-Corasick)
- Remove fuzz_config_env assertion that panics on null-byte-only input
- Remove no-op length check with misleading comment in fuzz_config_env
- Update fuzz_config_env description in README to match actual behavior

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-10 11:15:56 -07:00
[email protected]andClaude Opus 4.6 8d1d92937b fix: rewrite fuzz_config_env to exercise IronClaw safety code directly
Replace SafetyLayer wrapper usage with direct Sanitizer, Validator, and
LeakDetector instantiation and invocation. Adds meaningful consistency
assertions (non-empty output, valid-means-no-errors, scan/clean agreement).
Removes the config construction that was only exercising struct instantiation.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-10 00:36:03 -07:00
[email protected]andClaude Opus 4.6 4bd19a7ece fix: replace redundant detect() call with meaningful invariant assertion
Replace the double sanitize()+detect() call with an assertion that
critical severity warnings always trigger content modification.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-09 23:14:10 -07:00
[email protected]andClaude Opus 4.6 3c6f4a97dc fix: improve fuzz targets to exercise real IronClaw code paths
- fuzz_config_env: exercise SafetyLayer end-to-end (sanitize, validate,
  policy check) instead of generic TOML/JSON parsing
- fuzz_tool_params: add validate_tool_schema coverage alongside
  validate_tool_params
- Add "fuzz" to workspace exclude in root Cargo.toml
- Update README descriptions to match actual target behavior

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-09 23:09:25 -07:00
[email protected]andClaude Opus 4.6 e41eb8ae33 feat: add fuzzing targets for untrusted input parsers
Add cargo-fuzz infrastructure with 5 fuzz targets exercising
security-critical code paths:

- fuzz_safety_sanitizer: Aho-Corasick + regex injection detection
- fuzz_safety_validator: Input validation (length, encoding, patterns)
- fuzz_leak_detector: Secret leak scanning (API keys, tokens)
- fuzz_tool_params: Tool parameter JSON validation
- fuzz_config_env: TOML/JSON config parsing

Each target exercises real IronClaw business logic with invariant
assertions. Includes corpus directories and setup documentation.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-09 23:05:58 -07:00
3a2989d009 feat(setup): quick onboarding, preserve .env vars, clean up boot logging (#821)
* feat(setup): quick onboarding, preserve .env vars, clean up boot logging (#751, #674)

- Add upsert_bootstrap_vars() to preserve user-added .env vars on re-onboarding
- Add --quick mode: auto-defaults DB + security, asks only LLM provider (2 steps)
- Auto-triggered onboarding uses quick mode for near-instant first run
- Fix NEAR AI model fetch to use cloud-api.near.ai when API key is set
- Handle missing WASM tools/channels directories gracefully
- Downgrade all boot/shutdown tracing::info! to debug (boot screen shows user output)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(setup): gate env_backend variable behind postgres feature flag [skip-regression-check]

Clippy lint fix — not a behavioral change, just moving a variable declaration
inside the cfg(feature = "postgres") block where it's used.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(review): address PR review comments

- WASM loaders: use tokio::fs::metadata, only treat NotFound as empty,
  propagate other IO errors, handle TOCTOU in read_dir
- bootstrap: only ignore NotFound in read_to_string, propagate other errors
- wizard: restore print_info/print_success for migrations in interactive
  mode (gated by !config.quick), keep tracing::debug for diagnostics
- tests: use shared crate::config::helpers::ENV_MUTEX instead of separate
  NEARAI_ENV_MUTEX to prevent cross-test env var races
- README: fix quick mode description to mention model selection, clarify
  auto_setup_database may prompt when DATABASE_URL is set

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat(setup): skip prompts for DATABASE_URL in quick mode [skip-regression-check]

auto_setup_database() now uses DATABASE_URL directly without calling
step_database_postgres() (which prompts for confirmation). Quick mode
should be fully non-interactive when env vars are already set.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(cli): update --quick help text to mention model selection [skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-10 05:02:33 +00:00
94d101924e refactor: encapsulate leaked abstractions into owning modules (#778)
* refactor: encapsulate leaked abstractions from main.rs and app.rs into owning modules

Move module-specific initialization logic out of main.rs (1222→665 lines, -46%) and
app.rs (944→780 lines, -17%) into their respective owning modules as public factory
functions. This enforces separation of concerns so that adding a new DB backend, MCP
transport, or channel doesn't require editing main.rs/app.rs.

Key changes:
- Tracing init functions → src/tracing_fmt.rs
- DB connection factory (connect_with_handles + DatabaseHandles) → src/db/mod.rs
- Secrets store factory (create_secrets_store) → src/secrets/mod.rs
- MCP transport dispatch factory (create_client_from_config) → src/tools/mcp/factory.rs
- Orchestrator setup (setup_orchestrator + OrchestratorSetup) → src/orchestrator/mod.rs
- WASM channel setup (setup_wasm_channels) → src/channels/wasm/setup.rs
- Worker entry points (run_worker, run_claude_bridge) → src/worker/mod.rs
- Shared CLI secrets init (init_secrets_store) → src/cli/mod.rs
- Tunnel startup (start_managed_tunnel) → src/tunnel/mod.rs
- Onboard check (check_onboard_needed) → src/setup/mod.rs
- ExtensionManager unified MCP: uses create_client_from_config via McpProcessManager,
  enabling stdio/Unix transports for hot-activated MCP servers
- Deduplicated ~130 lines of secrets store init across cli/mcp.rs and cli/tool.rs
- CLAUDE.md updated with module-owned initialization guideline

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: address review feedback — deduplicate db factory, extract channel helper

- connect_from_config() now delegates to connect_with_handles() to eliminate
  duplicated backend-matching logic (Copilot review feedback)
- Extract register_channel() helper from setup_wasm_channels() loop body
  to improve readability (Gemini review feedback)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt line wrapping in setup_wasm_channels

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add integration test for module-owned initialization factories

Exercises the full factory chain end-to-end to verify nothing was lost
when initialization logic was moved from main.rs/app.rs into owning modules:

- connect_with_handles returns Database + populated backend handles
- connect_from_config delegates correctly (produces working Database)
- secrets::create_secrets_store builds working store from DatabaseHandles
- db::create_secrets_store standalone factory round-trips secrets
- Both secrets factories produce compatible stores (cross-read works)
- ExtensionManager constructs with McpProcessManager and is functional
- DatabaseHandles default is empty

All tests run without external services using libsql in-memory/tempfile.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wire cli/mcp.rs and cli/tool.rs to shared init_secrets_store()

Both files had inline implementations identical to cli::init_secrets_store().
Replace with delegation to complete the claimed deduplication.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt line wrapping in integration test

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(review): remove unused Config import and deduplicate Error Handling section

- Remove `#[allow(unused_imports)]` and unused `use crate::config::Config`
  from cli/tool.rs (no longer needed after delegating to shared
  `cli::init_secrets_store()`)
- Remove duplicate Error Handling subsection from CLAUDE.md Key Patterns
  (all four bullets already exist in Code Style section and
  review-discipline.md)

Addresses Copilot review comments.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(review): address remaining Copilot review comments

- secrets/mod.rs: clarify docstring that None is a normal no-db condition
- app.rs: add comment explaining the empty_handles fallback path
- orchestrator/mod.rs: combine duplicated sandbox condition into single block
- setup/mod.rs: document env var reads and thread-safety caveat

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Henry Park <[email protected]>
2026-03-10 04:39:51 +00:00
a868b14221 Fix/lightweight action tool (#785)
* feat: add tool execution support to lightweight routines

Lightweight routines now execute tools instead of outputting raw tool-call XML.

**Problem:** Lightweight routines had no tool execution loop, causing the LLM to generate
tool-call XML as text output (visible to users as garbage on Telegram). All 4 scheduled
routines were disabled and Emil saw the same issue in health-ping routine.

**Solution:** Implement a simplified agentic loop for lightweight routines that:
- Supports up to 3-5 tool iterations (configurable, capped at 5)
- Executes tools sequentially (not parallel, keeps overhead low)
- Auto-approves non-Always tools (lightweight routines are autonomous)
- Sanitizes and wraps tool outputs via SafetyLayer (same as dispatcher)
- Forces text-only response at iteration limit (guarantees termination)
- Maintains backward compatibility (disabled by default, toggled by config)

**Changes:**
1. **src/config/routines.rs:**
   - Added lightweight_tools_enabled (default: true)
   - Added lightweight_max_iterations (default: 3, capped at 5)
   - Added env var support: ROUTINES_LIGHTWEIGHT_TOOLS, ROUTINES_LIGHTWEIGHT_MAX_ITERATIONS

2. **src/agent/routine_engine.rs:**
   - Extended EngineContext with tools and safety fields
   - Split execute_lightweight into three functions:
     - execute_lightweight: router that dispatches to tool or no-tool version
     - execute_lightweight_no_tools: original single-call behavior
     - execute_lightweight_with_tools: new agentic loop with tool support
   - Added execute_routine_tool: isolated tool execution with validation and timeout
   - Uses ToolCompletionRequest/ToolCompletionResponse for tool-aware LLM calls
   - Integrates SafetyLayer for tool output sanitization

3. **src/agent/agent_loop.rs:**
   - Updated RoutineEngine::new call to pass tools and safety

**Tool Execution Loop:**
1. Build initial messages (system + user prompt)
2. Get tool definitions (empty at iteration limit)
3. Call LLM with ToolCompletionRequest
4. If text response: check for ROUTINE_OK sentinel, return result
5. If tool calls: execute sequentially, sanitize, wrap, add to context, loop
6. Safety ceiling at 5 iterations prevents runaway execution

**Approval Handling:** Auto-approves UnlessAutoApproved and Never tools;
blocks Always tools with error message (routines are autonomous by design).

**Testing:** All 2756 tests pass. Zero clippy warnings.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* test: add comprehensive unit tests for lightweight routine tool execution

Added 9 new unit tests covering:
- Configuration defaults (lightweight_tools_enabled, lightweight_max_iterations)
- Max iterations capped at 5 (safety ceiling)
- Routine name sanitization (special chars, alphanumeric preservation)
- Sentinel detection for ROUTINE_OK (exact match, contains, whitespace handling)
- Iteration limit safety ceiling enforcement
- Approval requirement pattern matching (Never, UnlessAutoApproved, Always)
- Empty response handling (finish_reason detection)

All 2765 tests pass (11 routine_engine tests, +9 new).

The tests cover the core logic paths of:
- Configuration validation
- Response parsing and sentinel detection
- Name sanitization for workspace paths
- Approval requirement logic
- Iteration limits and safety ceilings

Note: These are unit tests for core logic. Full integration tests with mock LLM
and tool registry would require more complex test infrastructure and are a future enhancement.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* style: format routine_engine.rs per cargo fmt

Apply consistent formatting to match Rust style guidelines:
- Break long import lines
- Reformat method chains for readability
- Format multi-line return tuples

No functional changes.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* fix: address security and code quality issues in lightweight routine tool execution

**Security Fixes:**

1. Sanitize tool error messages (medium severity)
   - Tool error messages were sent directly to LLM without sanitization
   - Now wrapped through SafetyLayer like successful outputs
   - Prevents leakage of API keys, internal paths, or PII from errors

2. Use unique job_id for each routine run (medium severity)
   - Previously reused routine.id across all executions
   - Caused state collisions and race conditions
   - Now generates unique run_id (Uuid::new_v4()) for each execution
   - Matches behavior of full_job routines

**Code Quality Fixes:**

3. Remove unreachable code
   - Deleted dead if iteration > 5 check
   - max_iterations is capped at 5 via .min(5), so check was impossible
   - Improves code clarity

4. Extract duplicated response handling logic
   - Created handle_text_response() helper function
   - Eliminated 20+ lines of duplicated ROUTINE_OK sentinel detection
   - Reduces maintenance burden and risk of inconsistencies

5. Fix test duplication
   - Tests now call actual super::sanitize_routine_name()
   - Removes duplicate implementation in tests
   - Ensures tests detect changes to original function

**Testing:**
- All 2765 tests pass (no regressions)
- Zero clippy warnings
- Test coverage maintained

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* fix: address security issue and improve code quality in lightweight routine tool execution

**SECURITY FIX (High Severity):**

1. Block UnlessAutoApproved tools in lightweight routines
   - Previously auto-approved UnlessAutoApproved tools, creating prompt injection vulnerability
   - Lightweight routines can be triggered by external events (channel messages, webhooks)
   - If susceptible to prompt injection, attacker could trick LLM into calling sensitive tools
   - Now blocks both UnlessAutoApproved and Always tools (only Never tools allowed)
   - Only safe approach without requiring tool_permissions allowlist in routine data model
   - Prevents unauthorized file access, network requests, and other sensitive operations

**Code Quality Improvements:**

2. Use ToolError::Timeout for consistent error handling (medium)
   - Changed from std::io::Error to proper ToolError::Timeout variant
   - More idiomatic and consistent with tool execution error handling
   - Makes errors easier to debug and handle uniformly

3. Fix misleading test names and remove tautological tests (medium)
   - Renamed test_routine_config_lightweight_max_iterations_capped_at_five to
     test_routine_config_can_hold_uncapped_max_iterations
   - Clarified comments to explain where capping actually occurs
   - Removed test_iteration_limit_safety_ceiling (tautological: asserts x.min(5) <= 5)
   - Improves test clarity and prevents false sense of coverage

**Testing:**
- 2764 tests passing (1 test removed, no regressions)
- Zero clippy warnings
- Security vulnerability eliminated

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* style: format routine_engine.rs per cargo fmt

Apply consistent formatting:
- Fix method chain indentation for LLM completion calls
- Reformat error handling closures for readability
- Break long method calls (wrap_for_llm) across multiple lines

No functional changes.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* style: apply cargo fmt formatting fixes to routine_engine.rs

Align formatting with project standards:
- Break long method chains across multiple lines for readability
- Reformat error return statements for consistency
- Split long assert/assert_eq statements across multiple lines

No logic changes; purely cosmetic formatting.

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

* test: update routine engine tests for tool/safety layer parameters

Update test code to pass newly required ToolRegistry and SafetyLayer
parameters to RoutineEngine::new(). Also add missing lightweight_tools_enabled
and lightweight_max_iterations fields to RoutineConfig initializers in tests.

Tests affected:
- tests/support/test_rig.rs: Added tools and safety layer to RoutineEngine::new()
- tests/e2e_routine_heartbeat.rs: Added three instances of tools and safety layer construction

All tests pass (2764 tests).

Co-Authored-By: Claude Haiku 4.5 <[email protected]>

---------

Co-authored-by: Claude Haiku 4.5 <[email protected]>
Co-authored-by: Henry Park <[email protected]>
2026-03-09 20:22:10 -07:00
Illia PolosukhinandGitHub a95f5ebb05 Updating feature parity 03/09 (#808) 2026-03-10 02:59:20 +00:00
83950d11a4 fix: job token budget, iteration cap → Failed, web cancel stops worker (#788)
* feat: persist user_id in save_job and expose job_id on routine runs (#709)

* feat: persist worker events to DB and fix activity tab rendering

In-process Worker (used by Scheduler::dispatch_job) now persists events
via save_job_event at key execution points: plan creation, LLM
responses, tool_use, tool_result, and job completion/failure/stuck.
Event data shapes match the container worker format so the gateway
activity tab renders them correctly.

Frontend: tool_result errors now show a red X icon with danger styling
instead of a silent empty output. The result event falls back to the
error field when message is absent.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: wire RoutineEngine into gateway for direct manual trigger firing

Replace the message-channel hack in routines_trigger_handler with a
direct call to RoutineEngine::fire_manual(), ensuring FullJob routines
dispatch correctly when triggered from the web UI. Inject the engine
into GatewayState from Agent::run after construction.

Also persists user_id in save_job for both PG and libSQL backends,
removes the source='sandbox' filter so all jobs are visible, and
exposes job_id on RoutineRunInfo for the frontend job link.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: remove stale gateway_state argument from Agent::new test call sites

The gateway_state parameter was removed from Agent::new during rebase
(replaced by post-construction set_routine_engine_slot), but three test
call sites still passed the extra None argument.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — restore sandbox source filter, remove blank lines

- Revert removal of `source = 'sandbox'` filter in all SandboxStore
  queries (8 sites across PG and libSQL). Sandbox-specific APIs should
  stay scoped to sandbox jobs; unified job listing for the Jobs tab
  should use a separate query path.
- Remove extra blank lines in agent_loop.rs and worker.rs that caused
  formatting CI failure.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review — regenerate Cargo.lock, add user_id regression test

- Regenerate Cargo.lock from main's lockfile to eliminate dependency
  version downgrades (anyhow, syn, etc.) that were churn from rebase.
- Add regression test verifying user_id round-trips through save_job
  and get_job in the libSQL backend.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: remove trailing blank line in libsql jobs.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add Postgres-side regression test for user_id persistence in save_job

Mirrors the existing libSQL test (test_save_job_persists_user_id) for the
Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore]
since it requires a running PostgreSQL instance (integration tier).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

* fix: add job token budget, change iteration cap to Failed, fix web cancel (#698)

Jobs could enter infinite retry loops because: (1) no token budget was
enforced, (2) iteration cap marked jobs as Stuck (allowing self-repair to
restart them), and (3) the web UI cancel button only updated the DB without
stopping the running worker.

- Add `max_tokens_per_job` config (settings.json + AGENT_MAX_TOKENS_PER_JOB
  env var, default 0 = unlimited) with per-job metadata override
- Track token usage after respond_with_tools() and fail the job on budget
  exceeded
- Change iteration cap and persistent rate limiting from mark_stuck to
  mark_failed, preventing self-repair restart loops
- Fix web cancel handler to call scheduler.stop() which updates in-memory
  state AND aborts the worker task, falling back to DB-only update

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — always persist cancel to DB, simplify token check

- Cancel handler now always persists Cancelled to DB regardless of whether
  scheduler.stop() ran, fixing the edge case where stop() returns Ok(())
  for jobs not in the scheduler map
- Collapse nested ifs per clippy (let-chains)
- Add NOTE comment about select_tools() not exposing TokenUsage

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: rustfmt formatting in wizard.rs (pre-existing)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-10 02:19:56 +00:00
Nick PismenkovandGitHub 764be8547f fix: fmt (#805) 2026-03-09 19:14:36 -07:00
7de639e782 fix(ci): cherry-pick fmt + clippy for staging PRs [skip-regression-check] (#803)
Cherry-pick of #802: run fmt + clippy on staging PRs, skip Windows clippy,
simplify claude-review trigger to labeled-only.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-09 18:44:57 -07:00
a5f88b32fd fix(setup): pass NEARAI_API_KEY to provider in model selection step (#799)
When users authenticate via NEAR AI Cloud API key (option 4) during
onboarding, the key is stored as an env var but fetch_nearai_models()
was hardcoding api_key: None. This caused resolve_bearer_token() to
re-trigger the interactive auth prompt at step 4 (model selection).

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-10 01:26:12 +00:00
Nick PismenkovandGitHub 7d8576a464 fix: destructive actions from ambiguous user prompts (#782)
* fix: destructive actions from ambiguous user prompts

* review fixes

* review fixes
2026-03-09 18:03:39 -07:00
f4b7309523 fix(ci): cherry-pick CI cleanup onto staging [skip-regression-check] (#798)
Cherry-pick of #794: remove continue-on-error hack, skip redundant checks
on staging PRs, allow ironclaw-ci[bot] in Claude Code review.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-09 17:59:59 -07:00
Henry ParkandClaude Sonnet 4.6 577e26eff4 fix(ci): secrets can't be used in step if conditions [skip-regression-check]
GitHub Actions step-level `if:` doesn't have access to `secrets` context.
Replace `if: secrets.X != ''` with `continue-on-error: true` and let
the Set token step handle the fallback.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-03-09 16:41:43 -07:00
70 changed files with 2990 additions and 1020 deletions
+2
View File
@@ -115,6 +115,8 @@ AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
AGENT_JOB_TIMEOUT_SECS=3600
AGENT_STUCK_THRESHOLD_SECS=300
# Maximum tokens per job (0 = unlimited, also settable via settings.json agent.max_tokens_per_job)
# AGENT_MAX_TOKENS_PER_JOB=0
# Enable planning phase before tool execution (default: true)
AGENT_USE_PLANNING=true
+2 -1
View File
@@ -2,7 +2,7 @@ name: Claude Code Review
on:
pull_request:
types: [opened, labeled]
types: [labeled]
permissions:
contents: read
@@ -28,6 +28,7 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
allowed_bots: "ironclaw-ci[bot]"
claude_args: "--max-turns 50 --model claude-haiku-4-5-20251001 --allowedTools 'Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*),Bash(git blame:*),Bash(git log:*),Bash(git diff:*)'"
prompt: |
Code review this pull request. Follow these steps precisely:
+7 -1
View File
@@ -44,6 +44,7 @@ jobs:
clippy-windows:
name: Clippy Windows (${{ matrix.name }})
if: github.base_ref == 'main'
runs-on: windows-latest
strategy:
fail-fast: false
@@ -76,7 +77,12 @@ jobs:
needs: [format, clippy, clippy-windows]
steps:
- run: |
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" || "${{ needs.clippy-windows.result }}" != "success" ]]; then
if [[ "${{ needs.format.result }}" != "success" || "${{ needs.clippy.result }}" != "success" ]]; then
echo "One or more jobs failed"
exit 1
fi
# clippy-windows only runs on main PRs, so skip/success are both acceptable
if [[ "${{ needs.clippy-windows.result }}" == "failure" ]]; then
echo "Windows clippy failed"
exit 1
fi
-2
View File
@@ -115,7 +115,6 @@ jobs:
- name: Generate GitHub App token
id: app-token
if: ${{ secrets.GH_RELEASES_MANAGER_APP_ID != '' }}
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
@@ -230,7 +229,6 @@ jobs:
- name: Generate GitHub App token
id: app-token
if: ${{ secrets.GH_RELEASES_MANAGER_APP_ID != '' }}
uses: actions/create-github-app-token@v2
with:
app-id: ${{ secrets.GH_RELEASES_MANAGER_APP_ID }}
+2
View File
@@ -2,6 +2,8 @@ name: Run Tests
on:
workflow_call:
pull_request:
branches:
- main
push:
branches:
- main
+36 -3
View File
@@ -64,6 +64,13 @@ src/
│ ├── repl.rs # Simple REPL (for testing)
│ ├── web/ # Web gateway (browser UI) — see src/channels/web/CLAUDE.md
│ └── wasm/ # WASM channel runtime
│ ├── mod.rs
│ ├── bundled.rs # Bundled channel discovery
│ ├── capabilities.rs # Channel-specific capabilities (HTTP endpoint, emit rate)
│ ├── error.rs # WASM channel error types
│ ├── runtime.rs # WASM channel execution runtime
│ ├── setup.rs # WasmChannelSetup, setup_wasm_channels(), inject_channel_credentials()
│ └── wrapper.rs # Channel trait wrapper for WASM modules
├── cli/ # CLI subcommands (clap)
│ ├── mod.rs # Cli struct, Command enum (run/onboard/config/tool/registry/mcp/memory/pairing/service/doctor/status/completion)
@@ -76,7 +83,13 @@ src/
├── hooks/ # Lifecycle hooks (6 points: BeforeInbound, BeforeToolCall, BeforeOutbound, OnSessionStart, OnSessionEnd, TransformResponse)
├── tunnel/ # Tunnel abstraction (cloudflare, ngrok, tailscale, custom, none)
├── tunnel/ # Tunnel abstraction for public internet exposure
│ ├── mod.rs # Tunnel trait, TunnelProviderConfig, create_tunnel(), start_managed_tunnel()
│ ├── cloudflare.rs # CloudflareTunnel (cloudflared binary)
│ ├── ngrok.rs # NgrokTunnel
│ ├── tailscale.rs # TailscaleTunnel (serve/funnel modes)
│ ├── custom.rs # CustomTunnel (arbitrary command with {host}/{port})
│ └── none.rs # NoneTunnel (local-only, no exposure)
├── observability/ # Pluggable event/metric recording (noop, log, multi)
@@ -105,8 +118,26 @@ src/
│ ├── rate_limiter.rs # Shared sliding-window rate limiter
│ ├── builtin/ # Built-in tools (echo, time, json, http, web_fetch, file, shell, memory, message, job, routine, extension_tools, skill_tools, secrets_tools)
│ ├── builder/ # Dynamic tool building
│ ├── mcp/ # Model Context Protocol client
└── wasm/ # Full WASM sandbox (wasmtime) — runtime, host functions, fuel metering, allowlist, credential injection
│ ├── core.rs # BuildRequirement, SoftwareType, Language
│ ├── templates.rs # Project scaffolding
│ │ ├── testing.rs # Test harness integration
│ │ └── validation.rs # WASM validation
│ ├── mcp/ # Model Context Protocol
│ │ ├── client.rs # MCP client over HTTP
│ │ ├── factory.rs # create_client_from_config() — transport dispatch factory
│ │ ├── protocol.rs # JSON-RPC types
│ │ └── session.rs # MCP session management (Mcp-Session-Id header, per-server state)
│ └── wasm/ # Full WASM sandbox (wasmtime)
│ ├── runtime.rs # Module compilation and caching
│ ├── wrapper.rs # Tool trait wrapper for WASM modules
│ ├── host.rs # Host functions (logging, time, workspace)
│ ├── limits.rs # Fuel metering and memory limiting
│ ├── allowlist.rs # Network endpoint allowlisting
│ ├── credential_injector.rs # Safe credential injection
│ ├── loader.rs # WASM tool discovery from filesystem
│ ├── rate_limiter.rs # Per-tool rate limiting
│ ├── error.rs # WASM-specific error types
│ └── storage.rs # Linear memory persistence
├── db/ # Dual-backend persistence (PostgreSQL + libSQL) — see src/db/CLAUDE.md
@@ -144,6 +175,8 @@ Dual-backend: PostgreSQL + libSQL/Turso. **All new persistence features must sup
When modifying a module with a spec, read the spec first. Code follows spec; spec is the tiebreaker.
**Module-owned initialization:** Module-specific initialization logic (database connection, transport creation, channel setup) must live in the owning module as a public factory function — not in `main.rs` or `app.rs`. These entry-point files orchestrate calls to module factories. Feature-flag branching (`#[cfg(feature = ...)]`) must be confined to the module that owns the abstraction.
| Module | Spec |
|--------|------|
| `src/agent/` | `src/agent/CLAUDE.md` |
+1
View File
@@ -14,6 +14,7 @@ exclude = [
"tools-src/google-slides",
"tools-src/slack",
"tools-src/telegram",
"fuzz",
]
[package]
+51 -44
View File
@@ -10,6 +10,8 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- 🚫 Out of scope (intentionally skipped)
- N/A (not applicable to Rust implementation)
**Last reviewed against OpenClaw PRs:** 2026-03-10 (merged 2026-02-24 through 2026-03-10)
---
## 1. Architecture
@@ -39,11 +41,11 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Network modes (loopback/LAN/remote) | ✅ | 🚧 | HTTP only |
| OpenAI-compatible HTTP API | ✅ | ✅ | /v1/chat/completions, per-request `model` override |
| Canvas hosting | ✅ | ❌ | Agent-driven UI |
| Gateway lock (PID-based) | ✅ | | `fs4` flock-based, acquired in `main.rs` before agent startup |
| Gateway lock (PID-based) | ✅ | | |
| launchd/systemd integration | ✅ | ❌ | |
| Bonjour/mDNS discovery | ✅ | ❌ | |
| Tailscale integration | ✅ | ❌ | |
| Health check endpoints | ✅ | ✅ | /api/health + /api/gateway/status |
| Health check endpoints | ✅ | ✅ | /api/health + /api/gateway/status + /healthz + /readyz, with channel-backed readiness probes |
| `doctor` diagnostics | ✅ | ❌ | |
| Agent event broadcast | ✅ | 🚧 | SSE broadcast manager exists (SseManager) but tool/job-state events not fully wired |
| Channel health monitor | ✅ | ❌ | Auto-restart with configurable interval |
@@ -66,17 +68,17 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| REPL (simple) | ✅ | ✅ | - | For testing |
| WASM channels | ❌ | ✅ | - | IronClaw innovation |
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username, DM topics |
| Discord | ✅ | ❌ | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | ✅ | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
| Linq | ✅ | ❌ | P3 | Real iMessage via API, no Mac required |
| Feishu/Lark | ✅ | ❌ | P3 | Bitable create app/field tools |
| Feishu/Lark | ✅ | ❌ | P3 | Bitable create app/field tools, Docx table/image/file actions, rich-text media extraction |
| LINE | ✅ | ❌ | P3 | |
| WebChat | ✅ | ✅ | - | Web gateway chat |
| Matrix | ✅ | ❌ | P3 | E2EE support |
| Mattermost | ✅ | ❌ | P3 | Emoji reactions |
| Mattermost | ✅ | ❌ | P3 | Emoji reactions, interactive buttons, model picker |
| Google Chat | ✅ | ❌ | P3 | |
| MS Teams | ✅ | ❌ | P3 | |
| Twitch | ✅ | ❌ | P3 | |
@@ -92,6 +94,8 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| User message reactions | ✅ | ❌ | Surface inbound reactions |
| sendPoll | ✅ | ❌ | Poll creation via agent |
| Cron/heartbeat topic targeting | ✅ | ❌ | Messages land in correct topic |
| DM topics support | ✅ | ❌ | Agent/topic bindings in DMs and agent-scoped SessionKeys |
| Persistent ACP topic binding | ✅ | ❌ | ACP harness sessions can pin to Telegram forum or DM topics |
### Discord-Specific Features (since Feb 2025)
@@ -107,21 +111,36 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
|---------|----------|----------|-------|
| Streaming draft replies | ✅ | ❌ | Partial replies via draft message updates |
| Configurable stream modes | ✅ | ❌ | Per-channel stream behavior |
| Thread ownership | ✅ | ❌ | Thread-level ownership tracking |
| Thread ownership | ✅ | ❌ | Thread-level ownership tracking plus reply participation memory |
| Download-file action | ✅ | ❌ | On-demand attachment downloads via message actions |
### Mattermost-Specific Features (since Mar 2026)
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Interactive buttons | ✅ | ❌ | Clickable message buttons with signed callback flow |
| Interactive model picker | ✅ | ❌ | In-channel provider/model chooser |
### Feishu/Lark-Specific Features (since Mar 2026)
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Doc/table actions | ✅ | ❌ | `feishu_doc` supports tables, positional insert, color_text, image upload, and file upload |
| Rich-text embedded media extraction | ✅ | ❌ | Pull video/media attachments from post messages |
### Channel Features
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| DM pairing codes | ✅ | ✅ | `ironclaw pairing list/approve`, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| Allowlist/blocklist | ✅ | 🚧 | `allow_from` + pairing store + hardened command/group allowlists |
| Self-message bypass | ✅ | ❌ | Own messages skip pairing |
| Mention-based activation | ✅ | ✅ | bot_username + respond_to_all_group_messages |
| Per-group tool policies | ✅ | ❌ | Allow/deny specific tools |
| Thread isolation | ✅ | ✅ | Separate sessions per thread |
| Per-channel media limits | ✅ | | Attachment type in WIT; max 10 per msg, 20MB total, MIME allowlist |
| Typing indicators | ✅ | 🚧 | TUI + Telegram typing/actionable status prompts; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions |
| Thread isolation | ✅ | ✅ | Separate sessions per thread/topic |
| Per-channel media limits | ✅ | 🚧 | Caption support plus `mediaMaxMb` enforcement for WhatsApp, Telegram, and Discord |
| Typing indicators | ✅ | 🚧 | TUI + channel typing, with configurable silence timeout; richer parity pending |
| Per-channel ackReaction config | ✅ | ❌ | Customizable acknowledgement reactions/scopes |
| Group session priming | ✅ | ❌ | Member roster injected for context |
| Sender_id in trusted metadata | ✅ | ❌ | Exposed in system metadata |
@@ -138,7 +157,8 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| `gateway start/stop` | ✅ | ❌ | P2 | |
| `onboard` (wizard) | ✅ | ✅ | - | Interactive setup |
| `tui` | ✅ | ✅ | - | Ratatui TUI |
| `config` | ✅ | ✅ | - | Read/write config |
| `config` | ✅ | ✅ | - | Read/write config plus validate/path helpers |
| `backup` | ✅ | ❌ | P3 | Create/verify local backup archives |
| `channels` | ✅ | ❌ | P2 | Channel management |
| `models` | ✅ | 🚧 | - | Model selector in TUI |
| `status` | ✅ | ✅ | - | System status (enriched session details) |
@@ -177,14 +197,15 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Global sessions | ✅ | ❌ | Optional shared context |
| Session pruning | ✅ | ❌ | Auto cleanup old sessions |
| Context compaction | ✅ | ✅ | Auto summarization |
| Compaction model override | ✅ | ❌ | Use a dedicated provider/model for summarization only |
| Post-compaction read audit | ✅ | ❌ | Layer 3: workspace rules appended to summaries |
| Post-compaction context injection | ✅ | ❌ | Workspace context as system event |
| Custom system prompts | ✅ | ✅ | Template variables, safety guardrails |
| Skills (modular capabilities) | ✅ | ✅ | Prompt-based skills with trust gating, attenuation, activation criteria, catalog, selector |
| Skill routing blocks | ✅ | 🚧 | ActivationCriteria (keywords, patterns, tags) but no "Use when / Don't use when" blocks |
| Skill path compaction | ✅ | ❌ | ~ prefix to reduce prompt tokens |
| Thinking modes (low/med/high) | ✅ | ❌ | Configurable reasoning depth |
| Per-model thinkingDefault override | ✅ | ❌ | Override thinking level per model |
| Thinking modes (off/minimal/low/medium/high/xhigh/adaptive) | ✅ | ❌ | Configurable reasoning depth |
| Per-model thinkingDefault override | ✅ | ❌ | Override thinking level per model; Anthropic Claude 4.6 defaults to adaptive |
| Block-level streaming | ✅ | ❌ | |
| Tool-level streaming | ✅ | ❌ | |
| Z.AI tool_stream | ✅ | ❌ | Real-time tool call streaming |
@@ -213,15 +234,11 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Provider | OpenClaw | IronClaw | Priority | Notes |
|----------|----------|----------|----------|-------|
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | ✅ | - | Native Converse API via aws-sdk-bedrockruntime (requires `--features bedrock`) |
| Google Gemini | ✅ | | P3 | Via `gemini` adapter |
| io.net | ✅ | | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6, adaptive thinking default |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy; GPT-5.4 + Codex OAuth |
| AWS Bedrock | ✅ | ❌ | P3 | |
| Google Gemini | ✅ | | P3 | |
| NVIDIA API | ✅ | | P3 | New provider |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
@@ -242,7 +259,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Per-session model override | ✅ | ✅ | Model selector in TUI |
| Model selection UI | ✅ | ✅ | TUI keyboard shortcut |
| Per-model thinkingDefault | ✅ | ❌ | Override thinking level per model in config |
| 1M context beta header | ✅ | ❌ | Anthropic extended context support |
| 1M context support | ✅ | ❌ | Anthropic extended context beta + OpenAI Codex GPT-5.4 1M context |
### Owner: _Unassigned_
@@ -252,32 +269,20 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| WIT inbound-attachment type | N/A | ✅ | P1 | `inbound-attachment` record in channel-host (id, mime_type, filename, size_bytes, source_url, storage_key, extracted_text) |
| WIT outbound attachment type | N/A | ✅ | P1 | `attachment` record in channel (filename, mime_type, data) on `agent-response` |
| WIT on-broadcast export | N/A | ✅ | P1 | Proactive message sending without prior incoming message |
| IncomingMessage attachments | N/A | ✅ | P1 | `IncomingAttachment` struct on `IncomingMessage`, populated from WASM channels |
| OutgoingResponse attachments | N/A | ✅ | P1 | File paths on `OutgoingResponse`, read from disk and sent as WIT attachments |
| Attachment security (size/MIME) | N/A | ✅ | P1 | Inbound: max 10, 20MB total, MIME allowlist. Outbound: 50MB total |
| Telegram media parsing | ✅ | ✅ | P1 | Photo, document, audio, video, voice, sticker parsed and emitted as attachments |
| Telegram media sending | ✅ | ✅ | P1 | sendPhoto/sendDocument multipart upload, auto photo→document fallback >10MB |
| Slack file parsing | ✅ | ✅ | P1 | `files` array from Events API parsed into attachments |
| WhatsApp media parsing | ✅ | ✅ | P1 | Image, audio, video, document parsed with caption as extracted_text |
| Discord attachment parsing | ✅ | ❌ | P2 | Discord interaction payloads don't include file attachments (needs message events) |
| HTTP tool save_to | N/A | ✅ | P1 | Download binary files to /tmp/ for attachment sending (50MB limit, path traversal protection) |
| Credential env var fallback | N/A | ✅ | P2 | Channels can use env vars (e.g., TELEGRAM_BOT_TOKEN) when secrets store not configured |
| Image processing (Sharp) | ✅ | ❌ | P2 | Resize, format convert |
| Configurable image resize dims | ✅ | ❌ | P2 | Per-agent dimension config |
| Multiple images per tool call | ✅ | ❌ | P2 | Single tool invocation, multiple images |
| Audio transcription | ✅ | ❌ | P2 | |
| Video support | ✅ | ❌ | P3 | |
| PDF parsing | ✅ | ❌ | P2 | pdfjs-dist |
| MIME detection | ✅ | | P2 | MIME allowlist in host validates attachment types |
| PDF analysis tool | ✅ | ❌ | P2 | Native Anthropic/Gemini path with text/image extraction fallback |
| PDF parsing | ✅ | | P2 | `pdfjs-dist` fallback path |
| MIME detection | ✅ | ❌ | P2 | |
| Media caching | ✅ | ❌ | P3 | |
| Vision model integration | ✅ | ❌ | P2 | Image understanding |
| TTS (Edge TTS) | ✅ | ❌ | P3 | Text-to-speech |
| TTS (OpenAI) | ✅ | ❌ | P3 | |
| Incremental TTS playback | ✅ | ❌ | P3 | iOS progressive playback |
| Sticker-to-image | ✅ | | P3 | Telegram stickers emitted as image/webp attachments |
| Sticker-to-image | ✅ | | P3 | Telegram stickers |
### Owner: _Unassigned_
@@ -293,7 +298,8 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Workspace-relative install | ✅ | ✅ | ~/.ironclaw/tools/ |
| Channel plugins | ✅ | ✅ | WASM channels |
| Auth plugins | ✅ | ❌ | |
| Memory plugins | ✅ | ❌ | Custom backends |
| Memory plugins | ✅ | ❌ | Custom backends + selectable memory slot |
| Context-engine plugins | ✅ | ❌ | Custom context management + subagent/context hooks |
| Tool plugins | ✅ | ✅ | WASM tools |
| Hook plugins | ✅ | ✅ | Declarative hooks from extension capabilities |
| Provider plugins | ✅ | ❌ | |
@@ -315,7 +321,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| JSON5 support | ✅ | ❌ | Comments, trailing commas |
| YAML alternative | ✅ | ❌ | |
| Environment variable interpolation | ✅ | ✅ | `${VAR}` |
| Config validation/schema | ✅ | ✅ | Type-safe Config struct |
| Config validation/schema | ✅ | ✅ | Type-safe Config struct + `openclaw config validate` |
| Hot-reload | ✅ | ❌ | |
| Legacy migration | ✅ | | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.ironclaw/` | |
@@ -422,6 +428,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Cron jobs | ✅ | ✅ | - | Routines with cron trigger |
| Per-job model fallback override | ✅ | ❌ | P2 | `payload.fallbacks` overrides agent-level fallbacks |
| Cron stagger controls | ✅ | ❌ | P3 | Default stagger for scheduled jobs |
| Cron finished-run webhook | ✅ | ❌ | P3 | Webhook on job completion |
| Timezone support | ✅ | ✅ | - | Via cron expressions |
@@ -475,10 +482,10 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Elevated mode | ✅ | ❌ | |
| Safe bins allowlist | ✅ | ❌ | Hardened path trust |
| LD*/DYLD* validation | ✅ | ❌ | |
| Path traversal prevention | ✅ | ✅ | Including config includes (OC-06) |
| Path traversal prevention | ✅ | ✅ | Including config includes (OC-06) + workspace-only tool mounts |
| Credential theft via env injection | ✅ | 🚧 | Shell env scrubbing + command injection detection; no full OC-09 defense |
| Session file permissions (0o600) | ✅ | ✅ | Session token file set to 0o600 in llm/session.rs |
| Skill download path restriction | ✅ | ❌ | Prevent arbitrary write targets |
| Skill download path restriction | ✅ | ❌ | Validated download roots prevent arbitrary write targets |
| Webhook signature verification | ✅ | ✅ | |
| Media URL validation | ✅ | ❌ | |
| Prompt injection defense | ✅ | ✅ | Pattern detection, sanitization |
+40
View File
@@ -0,0 +1,40 @@
[package]
name = "ironclaw-fuzz"
version = "0.0.0"
publish = false
edition = "2021"
[package.metadata]
cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4"
serde_json = "1"
[dependencies.ironclaw]
path = ".."
[[bin]]
name = "fuzz_safety_sanitizer"
path = "fuzz_targets/fuzz_safety_sanitizer.rs"
doc = false
[[bin]]
name = "fuzz_safety_validator"
path = "fuzz_targets/fuzz_safety_validator.rs"
doc = false
[[bin]]
name = "fuzz_leak_detector"
path = "fuzz_targets/fuzz_leak_detector.rs"
doc = false
[[bin]]
name = "fuzz_tool_params"
path = "fuzz_targets/fuzz_tool_params.rs"
doc = false
[[bin]]
name = "fuzz_config_env"
path = "fuzz_targets/fuzz_config_env.rs"
doc = false
+43
View File
@@ -0,0 +1,43 @@
# IronClaw Fuzz Targets
Fuzz testing for security-critical input parsing paths using [cargo-fuzz](https://github.com/rust-fuzz/cargo-fuzz) (libFuzzer).
## Targets
| Target | What it exercises |
|--------|-------------------|
| `fuzz_safety_sanitizer` | Prompt injection pattern detection (Aho-Corasick + regex) |
| `fuzz_safety_validator` | Input validation (length, encoding, forbidden patterns) |
| `fuzz_leak_detector` | Secret leak detection (API keys, tokens, credentials) |
| `fuzz_tool_params` | Tool parameter and schema JSON validation |
| `fuzz_config_env` | Combined safety primitives (sanitize, validate, leak detect) |
## Setup
```bash
cargo install cargo-fuzz
rustup install nightly
```
## Running
```bash
# Run a specific target (runs until stopped or crash found)
cargo +nightly fuzz run fuzz_safety_sanitizer
# Run with a time limit (5 minutes)
cargo +nightly fuzz run fuzz_leak_detector -- -max_total_time=300
# Run all targets for 60 seconds each
for target in fuzz_safety_sanitizer fuzz_safety_validator fuzz_leak_detector fuzz_tool_params fuzz_config_env; do
echo "==> $target"
cargo +nightly fuzz run "$target" -- -max_total_time=60
done
```
## Adding New Targets
1. Create `fuzz/fuzz_targets/fuzz_<name>.rs` following the existing pattern
2. Add a `[[bin]]` entry in `fuzz/Cargo.toml`
3. Create `fuzz/corpus/fuzz_<name>/` for seed inputs
4. Exercise real IronClaw code paths, not just generic serde
+44
View File
@@ -0,0 +1,44 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use std::sync::LazyLock;
use ironclaw::safety::{LeakDetector, Sanitizer, Validator};
static SANITIZER: LazyLock<Sanitizer> = LazyLock::new(Sanitizer::new);
static VALIDATOR: LazyLock<Validator> = LazyLock::new(Validator::new);
static LEAK_DETECTOR: LazyLock<LeakDetector> = LazyLock::new(LeakDetector::new);
fuzz_target!(|data: &[u8]| {
if let Ok(input) = std::str::from_utf8(data) {
// Exercise Sanitizer: detect and neutralize prompt injection attempts.
let sanitized = SANITIZER.sanitize(input);
// If no modification occurred, content must equal input.
if !sanitized.was_modified {
assert_eq!(sanitized.content, input);
}
// Exercise Validator: input validation (length, encoding, patterns).
let result = VALIDATOR.validate(input);
// ValidationResult must always be well-formed: if valid, no errors.
if result.is_valid {
assert!(
result.errors.is_empty(),
"valid result should have no errors"
);
}
// Exercise LeakDetector: secret detection (API keys, tokens, etc.).
let scan = LEAK_DETECTOR.scan(input);
// scan_and_clean must not panic and must return valid UTF-8.
let cleaned = LEAK_DETECTOR.scan_and_clean(input);
// If scan found no matches, scan_and_clean should return the input unchanged.
if scan.matches.is_empty() {
if let Ok(ref clean_str) = cleaned {
assert_eq!(
clean_str, input,
"scan_and_clean changed content despite no matches"
);
}
}
}
});
+25
View File
@@ -0,0 +1,25 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use std::sync::LazyLock;
use ironclaw::safety::LeakDetector;
static DETECTOR: LazyLock<LeakDetector> = LazyLock::new(LeakDetector::new);
fuzz_target!(|data: &[u8]| {
if let Ok(s) = std::str::from_utf8(data) {
// Exercise scan path
let result = DETECTOR.scan(s);
// Invariant: if should_block, there must be matches
if result.should_block {
assert!(!result.matches.is_empty());
}
// Invariant: match locations must be valid
for m in &result.matches {
assert!(m.location.end <= s.len());
}
// Exercise scan_and_clean path
let _ = DETECTOR.scan_and_clean(s);
}
});
@@ -0,0 +1,25 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use std::sync::LazyLock;
use ironclaw::safety::Sanitizer;
static SANITIZER: LazyLock<Sanitizer> = LazyLock::new(Sanitizer::new);
fuzz_target!(|data: &[u8]| {
if let Ok(s) = std::str::from_utf8(data) {
// Exercise the main sanitization path
let result = SANITIZER.sanitize(s);
// Verify invariant: warnings should have valid ranges
for w in &result.warnings {
assert!(w.location.end <= s.len());
}
// Verify invariant: critical severity triggers modification
let has_critical = result.warnings.iter().any(|w| {
w.severity == ironclaw::safety::Severity::Critical
});
if has_critical {
assert!(result.was_modified);
}
}
});
@@ -0,0 +1,23 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use std::sync::LazyLock;
use ironclaw::safety::Validator;
static VALIDATOR: LazyLock<Validator> = LazyLock::new(Validator::new);
fuzz_target!(|data: &[u8]| {
if let Ok(s) = std::str::from_utf8(data) {
// Exercise input validation
let result = VALIDATOR.validate(s);
// Invariant: empty input is always invalid
if s.is_empty() {
assert!(!result.is_valid);
}
// Exercise tool parameter validation with arbitrary JSON
if let Ok(value) = serde_json::from_str::<serde_json::Value>(s) {
let _ = VALIDATOR.validate_tool_params(&value);
}
}
});
+25
View File
@@ -0,0 +1,25 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use std::sync::LazyLock;
use ironclaw::safety::Validator;
use ironclaw::tools::validate_tool_schema;
static VALIDATOR: LazyLock<Validator> = LazyLock::new(Validator::new);
fuzz_target!(|data: &[u8]| {
if let Ok(s) = std::str::from_utf8(data) {
// Try parsing as JSON and validating as tool parameters
if let Ok(value) = serde_json::from_str::<serde_json::Value>(s) {
// Exercise Validator::validate_tool_params with arbitrary JSON
let result = VALIDATOR.validate_tool_params(&value);
// Invariant: result should always be well-formed
if !result.is_valid {
assert!(!result.errors.is_empty());
}
// Exercise validate_tool_schema with arbitrary JSON as a schema
let _ = validate_tool_schema(&value, "fuzz");
}
}
});
+8 -6
View File
@@ -446,6 +446,8 @@ impl Agent {
Arc::clone(workspace),
notify_tx,
Some(self.scheduler.clone()),
self.tools().clone(),
self.safety().clone(),
));
// Register routine tools
@@ -514,7 +516,7 @@ impl Agent {
*slot.write().await = Some(Arc::clone(&engine));
}
tracing::info!(
tracing::debug!(
"Routines enabled: cron ticker every {}s, max {} concurrent",
rt_config.cron_check_interval_secs,
rt_config.max_concurrent_routines
@@ -536,20 +538,20 @@ impl Agent {
let routine_engine_for_loop = routine_handle.as_ref().map(|(_, e)| Arc::clone(e));
// Main message loop
tracing::info!("Agent {} ready and listening", self.config.name);
tracing::debug!("Agent {} ready and listening", self.config.name);
loop {
let message = tokio::select! {
biased;
_ = tokio::signal::ctrl_c() => {
tracing::info!("Ctrl+C received, shutting down...");
tracing::debug!("Ctrl+C received, shutting down...");
break;
}
msg = message_stream.next() => {
match msg {
Some(m) => m,
None => {
tracing::info!("All channel streams ended, shutting down...");
tracing::debug!("All channel streams ended, shutting down...");
break;
}
}
@@ -624,7 +626,7 @@ impl Agent {
}
Ok(None) => {
// Shutdown signal received (/quit, /exit, /shutdown)
tracing::info!("Shutdown command received, exiting...");
tracing::debug!("Shutdown command received, exiting...");
break;
}
Err(e) => {
@@ -653,7 +655,7 @@ impl Agent {
}
// Cleanup
tracing::info!("Agent shutting down...");
tracing::debug!("Agent shutting down...");
repair_handle.abort();
pruning_handle.abort();
if let Some(handle) = heartbeat_handle {
+93
View File
@@ -1205,6 +1205,7 @@ mod tests {
max_tool_iterations: 50,
auto_approve_tools: false,
default_timezone: "UTC".to_string(),
max_tokens_per_job: 0,
},
deps,
Arc::new(ChannelManager::new()),
@@ -1263,6 +1264,96 @@ mod tests {
}
}
#[test]
fn test_always_approval_requirement_bypasses_session_auto_approve() {
// Regression test: even if tool is auto-approved in session,
// ApprovalRequirement::Always must still trigger approval.
use crate::tools::ApprovalRequirement;
let mut session = Session::new("user-1");
let tool_name = "tool_remove";
// Manually auto-approve tool_remove in this session
session.auto_approve_tool(tool_name);
assert!(
session.is_tool_auto_approved(tool_name),
"tool should be auto-approved"
);
// However, ApprovalRequirement::Always should always require approval
// This is verified by the dispatcher logic: Always => true (ignores session state)
let always_req = ApprovalRequirement::Always;
let requires_approval = match always_req {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => !session.is_tool_auto_approved(tool_name),
ApprovalRequirement::Always => true,
};
assert!(
requires_approval,
"ApprovalRequirement::Always must require approval even when tool is auto-approved"
);
}
#[test]
fn test_always_approval_requirement_vs_unless_auto_approved() {
// Verify the two requirements behave differently
use crate::tools::ApprovalRequirement;
let mut session = Session::new("user-2");
let tool_name = "http";
// Scenario 1: Tool is auto-approved
session.auto_approve_tool(tool_name);
// UnlessAutoApproved → doesn't require approval if auto-approved
let unless_req = ApprovalRequirement::UnlessAutoApproved;
let unless_needs = match unless_req {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => !session.is_tool_auto_approved(tool_name),
ApprovalRequirement::Always => true,
};
assert!(
!unless_needs,
"UnlessAutoApproved should not need approval when auto-approved"
);
// Always → always requires approval
let always_req = ApprovalRequirement::Always;
let always_needs = match always_req {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => !session.is_tool_auto_approved(tool_name),
ApprovalRequirement::Always => true,
};
assert!(
always_needs,
"Always must always require approval, even when auto-approved"
);
// Scenario 2: Tool is NOT auto-approved
let new_tool = "new_tool";
assert!(!session.is_tool_auto_approved(new_tool));
// UnlessAutoApproved → requires approval
let unless_needs = match unless_req {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => !session.is_tool_auto_approved(new_tool),
ApprovalRequirement::Always => true,
};
assert!(
unless_needs,
"UnlessAutoApproved should need approval when not auto-approved"
);
// Always → always requires approval
let always_needs = match always_req {
ApprovalRequirement::Never => false,
ApprovalRequirement::UnlessAutoApproved => !session.is_tool_auto_approved(new_tool),
ApprovalRequirement::Always => true,
};
assert!(always_needs, "Always must always require approval");
}
#[test]
fn test_pending_approval_serialization_backcompat_without_deferred_calls() {
// PendingApproval from before the deferred_tool_calls field was added
@@ -1953,6 +2044,7 @@ mod tests {
max_tool_iterations,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
max_tokens_per_job: 0,
},
deps,
Arc::new(ChannelManager::new()),
@@ -2069,6 +2161,7 @@ mod tests {
max_tool_iterations: max_iter,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
max_tokens_per_job: 0,
},
deps,
Arc::new(ChannelManager::new()),
+459 -14
View File
@@ -25,10 +25,14 @@ use crate::agent::routine::{
};
use crate::channels::{IncomingMessage, OutgoingResponse};
use crate::config::RoutineConfig;
use crate::context::JobContext;
use crate::db::Database;
use crate::error::RoutineError;
use crate::llm::{ChatMessage, CompletionRequest, FinishReason, LlmProvider};
use crate::tools::ApprovalContext;
use crate::llm::{
ChatMessage, CompletionRequest, FinishReason, LlmProvider, ToolCall, ToolCompletionRequest,
};
use crate::safety::SafetyLayer;
use crate::tools::{ApprovalContext, ApprovalRequirement, ToolError, ToolRegistry, redact_params};
use crate::workspace::Workspace;
/// The routine execution engine.
@@ -45,9 +49,14 @@ pub struct RoutineEngine {
event_cache: Arc<RwLock<Vec<(Uuid, Routine, Regex)>>>,
/// Scheduler for dispatching jobs (FullJob mode).
scheduler: Option<Arc<Scheduler>>,
/// Tool registry for lightweight routine tool execution.
tools: Arc<ToolRegistry>,
/// Safety layer for tool output sanitization.
safety: Arc<SafetyLayer>,
}
impl RoutineEngine {
#[allow(clippy::too_many_arguments)]
pub fn new(
config: RoutineConfig,
store: Arc<dyn Database>,
@@ -55,6 +64,8 @@ impl RoutineEngine {
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
scheduler: Option<Arc<Scheduler>>,
tools: Arc<ToolRegistry>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
config,
@@ -65,6 +76,8 @@ impl RoutineEngine {
running_count: Arc::new(AtomicUsize::new(0)),
event_cache: Arc::new(RwLock::new(Vec::new())),
scheduler,
tools,
safety,
}
}
@@ -240,12 +253,15 @@ impl RoutineEngine {
// Execute inline for manual triggers (caller wants to wait)
let engine = EngineContext {
config: self.config.clone(),
store: self.store.clone(),
llm: self.llm.clone(),
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
scheduler: self.scheduler.clone(),
tools: self.tools.clone(),
safety: self.safety.clone(),
};
tokio::spawn(async move {
@@ -272,12 +288,15 @@ impl RoutineEngine {
};
let engine = EngineContext {
config: self.config.clone(),
store: self.store.clone(),
llm: self.llm.clone(),
workspace: self.workspace.clone(),
notify_tx: self.notify_tx.clone(),
running_count: self.running_count.clone(),
scheduler: self.scheduler.clone(),
tools: self.tools.clone(),
safety: self.safety.clone(),
};
// Record the run in DB, then spawn execution
@@ -319,12 +338,15 @@ impl RoutineEngine {
/// Shared context passed to the execution function.
struct EngineContext {
config: RoutineConfig,
store: Arc<dyn Database>,
llm: Arc<dyn LlmProvider>,
workspace: Arc<Workspace>,
notify_tx: mpsc::Sender<OutgoingResponse>,
running_count: Arc<AtomicUsize>,
scheduler: Option<Arc<Scheduler>>,
tools: Arc<ToolRegistry>,
safety: Arc<SafetyLayer>,
}
/// Execute a routine run. Handles both lightweight and full_job modes.
@@ -538,7 +560,10 @@ async fn execute_full_job(
Ok((RunStatus::Ok, Some(summary), None))
}
/// Execute a lightweight routine (single LLM call).
/// Execute a lightweight routine with optional tool support.
///
/// If tools are enabled, this runs a simplified agentic loop (max 3-5 iterations).
/// If tools are disabled, this does a single LLM call (original behavior).
async fn execute_lightweight(
ctx: &EngineContext,
routine: &Routine,
@@ -570,7 +595,7 @@ async fn execute_lightweight(
Err(_) => None,
};
// Build the prompt
// Build the user-facing prompt
let mut full_prompt = String::new();
full_prompt.push_str(prompt);
@@ -598,15 +623,6 @@ async fn execute_lightweight(
}
};
let messages = if system_prompt.is_empty() {
vec![ChatMessage::user(&full_prompt)]
} else {
vec![
ChatMessage::system(&system_prompt),
ChatMessage::user(&full_prompt),
]
};
// Determine max_tokens from model metadata with fallback
let effective_max_tokens = match ctx.llm.model_metadata().await {
Ok(meta) => {
@@ -616,6 +632,45 @@ async fn execute_lightweight(
Err(_) => max_tokens,
};
// If tools are enabled, use the tool execution loop; otherwise, single LLM call
if ctx.config.lightweight_tools_enabled {
execute_lightweight_with_tools(
ctx,
routine,
&system_prompt,
&full_prompt,
effective_max_tokens,
)
.await
} else {
execute_lightweight_no_tools(
ctx,
routine,
&system_prompt,
&full_prompt,
effective_max_tokens,
)
.await
}
}
/// Execute a lightweight routine without tool support (original single-call behavior).
async fn execute_lightweight_no_tools(
ctx: &EngineContext,
_routine: &Routine,
system_prompt: &str,
full_prompt: &str,
effective_max_tokens: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let messages = if system_prompt.is_empty() {
vec![ChatMessage::user(full_prompt)]
} else {
vec![
ChatMessage::system(system_prompt),
ChatMessage::user(full_prompt),
]
};
let request = CompletionRequest::new(messages)
.with_max_tokens(effective_max_tokens)
.with_temperature(0.3);
@@ -631,7 +686,7 @@ async fn execute_lightweight(
let content = response.content.trim();
let tokens_used = Some((response.input_tokens + response.output_tokens) as i32);
// Empty content guard (same as heartbeat)
// Empty content guard
if content.is_empty() {
return if response.finish_reason == FinishReason::Length {
Err(RoutineError::TruncatedResponse)
@@ -648,6 +703,269 @@ async fn execute_lightweight(
Ok((RunStatus::Attention, Some(content.to_string()), tokens_used))
}
/// Handle a text-only LLM response in lightweight routine execution.
///
/// Checks for the ROUTINE_OK sentinel, validates content, and returns appropriate status.
fn handle_text_response(
content: &str,
finish_reason: FinishReason,
total_input_tokens: u32,
total_output_tokens: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let content = content.trim();
// Empty content guard
if content.is_empty() {
return if finish_reason == FinishReason::Length {
Err(RoutineError::TruncatedResponse)
} else {
Err(RoutineError::EmptyResponse)
};
}
// Check for the "nothing to do" sentinel
if content == "ROUTINE_OK" || content.contains("ROUTINE_OK") {
let total_tokens = Some((total_input_tokens + total_output_tokens) as i32);
return Ok((RunStatus::Ok, None, total_tokens));
}
let total_tokens = Some((total_input_tokens + total_output_tokens) as i32);
Ok((
RunStatus::Attention,
Some(content.to_string()),
total_tokens,
))
}
/// Execute a lightweight routine with tool execution support (agentic loop).
///
/// This is a simplified version of the full dispatcher loop:
/// - Max 3-5 iterations (configurable)
/// - Sequential tool execution (not parallel)
/// - Auto-approval of non-Always tools
/// - No hooks or approval dialogs
async fn execute_lightweight_with_tools(
ctx: &EngineContext,
routine: &Routine,
system_prompt: &str,
full_prompt: &str,
effective_max_tokens: u32,
) -> Result<(RunStatus, Option<String>, Option<i32>), RoutineError> {
let mut messages = if system_prompt.is_empty() {
vec![ChatMessage::user(full_prompt)]
} else {
vec![
ChatMessage::system(system_prompt),
ChatMessage::user(full_prompt),
]
};
let max_iterations = ctx.config.lightweight_max_iterations.min(5);
let mut iteration = 0;
let mut total_input_tokens = 0;
let mut total_output_tokens = 0;
// Create a minimal job context for tool execution with unique run ID
let run_id = Uuid::new_v4();
let job_ctx = JobContext {
job_id: run_id,
user_id: routine.user_id.clone(),
title: "Lightweight Routine".to_string(),
description: routine.name.clone(),
..Default::default()
};
loop {
iteration += 1;
// Force text-only response at iteration limit
let force_text = iteration >= max_iterations;
if force_text {
// Final iteration: no tools, just get text response
let request = CompletionRequest::new(messages)
.with_max_tokens(effective_max_tokens)
.with_temperature(0.3);
let response =
ctx.llm
.complete(request)
.await
.map_err(|e| RoutineError::LlmFailed {
reason: e.to_string(),
})?;
total_input_tokens += response.input_tokens;
total_output_tokens += response.output_tokens;
return handle_text_response(
&response.content,
response.finish_reason,
total_input_tokens,
total_output_tokens,
);
} else {
// Tool-enabled iteration
let tool_defs = ctx.tools.tool_definitions().await;
let request = ToolCompletionRequest::new(messages.clone(), tool_defs)
.with_max_tokens(effective_max_tokens)
.with_temperature(0.3);
let response = ctx.llm.complete_with_tools(request).await.map_err(|e| {
RoutineError::LlmFailed {
reason: e.to_string(),
}
})?;
total_input_tokens += response.input_tokens;
total_output_tokens += response.output_tokens;
// Check if LLM returned text (no tool calls)
if response.tool_calls.is_empty() {
let content = response.content.unwrap_or_default();
return handle_text_response(
&content,
response.finish_reason,
total_input_tokens,
total_output_tokens,
);
}
// LLM returned tool calls: add assistant message and execute tools
messages.push(ChatMessage::assistant_with_tool_calls(
response.content.clone(),
response.tool_calls.clone(),
));
// Execute tools sequentially
for tc in response.tool_calls {
let result = execute_routine_tool(ctx, &job_ctx, &tc).await;
// Sanitize and wrap result (including errors)
let result_content = match result {
Ok(output) => {
let sanitized = ctx.safety.sanitize_tool_output(&tc.name, &output);
ctx.safety.wrap_for_llm(
&tc.name,
&sanitized.content,
sanitized.was_modified,
)
}
Err(e) => {
let error_msg = format!("Tool '{}' failed: {}", tc.name, e);
let sanitized = ctx.safety.sanitize_tool_output(&tc.name, &error_msg);
ctx.safety.wrap_for_llm(
&tc.name,
&sanitized.content,
sanitized.was_modified,
)
}
};
// Add tool result to context
messages.push(ChatMessage::tool_result(&tc.id, &tc.name, &result_content));
}
// Continue loop to next LLM call
}
}
}
/// Execute a single tool for a lightweight routine.
async fn execute_routine_tool(
ctx: &EngineContext,
job_ctx: &JobContext,
tc: &ToolCall,
) -> Result<String, Box<dyn std::error::Error + Send + Sync>> {
// Check if tool exists
let tool = ctx
.tools
.get(&tc.name)
.await
.ok_or_else(|| format!("Tool '{}' not found", tc.name))?;
// Check approval requirement: only allow Never tools in lightweight routines.
// UnlessAutoApproved and Always tools are blocked to prevent prompt injection attacks.
// Lightweight routines can be triggered by external events and may process untrusted data,
// making them vulnerable to prompt injection that could trick the LLM into calling
// sensitive tools. Blocking these tools entirely is the safest approach.
match tool.requires_approval(&tc.arguments) {
ApprovalRequirement::Never => {}
ApprovalRequirement::UnlessAutoApproved | ApprovalRequirement::Always => {
return Err(format!(
"Tool '{}' requires manual approval and cannot be used in lightweight routines",
tc.name
)
.into());
}
}
// Validate tool parameters
let validation = ctx.safety.validator().validate_tool_params(&tc.arguments);
if !validation.is_valid {
let details = validation
.errors
.iter()
.map(|e| format!("{}: {}", e.field, e.message))
.collect::<Vec<_>>()
.join("; ");
return Err(format!("Invalid tool parameters: {}", details).into());
}
let safe_params = redact_params(&tc.arguments, tool.sensitive_params());
tracing::debug!(
tool = %tc.name,
params = %safe_params,
"Lightweight routine tool call started"
);
// Execute with per-tool timeout
let timeout = tool.execution_timeout();
let start = std::time::Instant::now();
let result = tokio::time::timeout(timeout, async {
tool.execute(tc.arguments.clone(), job_ctx).await
})
.await;
let elapsed = start.elapsed();
match &result {
Ok(Ok(_)) => {
tracing::debug!(
tool = %tc.name,
elapsed_ms = elapsed.as_millis() as u64,
"Lightweight routine tool call succeeded"
);
}
Ok(Err(e)) => {
tracing::debug!(
tool = %tc.name,
elapsed_ms = elapsed.as_millis() as u64,
error = %e,
"Lightweight routine tool call failed"
);
}
Err(_) => {
tracing::debug!(
tool = %tc.name,
elapsed_ms = elapsed.as_millis() as u64,
timeout_secs = timeout.as_secs(),
"Lightweight routine tool call timed out"
);
}
}
let result = result
.map_err(|_| ToolError::Timeout(timeout))
.map_err(|e| Box::new(e) as Box<dyn std::error::Error + Send + Sync>)?
.map_err(|e| Box::new(e) as Box<dyn std::error::Error + Send + Sync>)?;
// Serialize result to JSON string
let result_str =
serde_json::to_string(&result.result).unwrap_or_else(|_| "<serialize error>".to_string());
Ok(result_str)
}
/// Send a notification based on the routine's notify config and run status.
async fn send_notification(
tx: &mpsc::Sender<OutgoingResponse>,
@@ -727,6 +1045,7 @@ fn truncate(s: &str, max: usize) -> String {
#[cfg(test)]
mod tests {
use crate::agent::routine::{NotifyConfig, RunStatus};
use crate::config::RoutineConfig;
#[test]
fn test_notification_gating() {
@@ -755,4 +1074,130 @@ mod tests {
let _ = status.to_string();
}
}
#[test]
fn test_routine_config_lightweight_tools_enabled_default() {
let config = RoutineConfig::default();
assert!(
config.lightweight_tools_enabled,
"Tools should be enabled by default"
);
}
#[test]
fn test_routine_config_lightweight_max_iterations_default() {
let config = RoutineConfig::default();
assert_eq!(
config.lightweight_max_iterations, 3,
"Default should be 3 iterations"
);
}
#[test]
fn test_routine_config_can_hold_uncapped_max_iterations() {
// The `RoutineConfig` struct can hold a value greater than the safety cap.
let config = RoutineConfig {
lightweight_max_iterations: 10, // Set a value higher than the cap.
..RoutineConfig::default()
};
// The actual capping to a maximum of 5 is handled at runtime in
// `execute_lightweight_with_tools` and during config resolution from env vars.
assert_eq!(
config.lightweight_max_iterations, 10,
"Config struct should store the provided value"
);
}
#[test]
fn test_sanitize_routine_name_replaces_special_chars() {
let test_cases = vec![
("valid-routine", "valid-routine"),
("routine_with_underscore", "routine_with_underscore"),
("Routine With Spaces", "Routine_With_Spaces"),
("routine/with/slashes", "routine_with_slashes"),
("routine@with#symbols", "routine_with_symbols"),
];
for (input, expected) in test_cases {
let result = super::sanitize_routine_name(input);
assert_eq!(
result, expected,
"sanitize_routine_name({}) should be {}",
input, expected
);
}
}
#[test]
fn test_sanitize_routine_name_preserves_alphanumeric_dash_underscore() {
let names = vec!["routine123", "routine-name", "routine_name", "ROUTINE"];
for name in names {
let result = super::sanitize_routine_name(name);
assert_eq!(result, name, "Should preserve {}", name);
}
}
#[test]
fn test_routine_sentinel_detection_exact_match() {
// The execute_lightweight_no_tools checks: content == "ROUTINE_OK" || content.contains("ROUTINE_OK")
// After trim(), whitespace is removed
let test_cases = vec![
("ROUTINE_OK", true),
(" ROUTINE_OK ", true), // After trim, whitespace is removed so matches
("something ROUTINE_OK something", true),
("ROUTINE_OK is done", true),
("done ROUTINE_OK", true),
("no sentinel here", false),
];
for (content, should_match) in test_cases {
let trimmed = content.trim();
let matches = trimmed == "ROUTINE_OK" || trimmed.contains("ROUTINE_OK");
assert_eq!(
matches, should_match,
"Content '{}' sentinel detection should be {}, got {}",
content, should_match, matches
);
}
}
#[test]
fn test_approval_requirement_pattern_matching() {
// Test the approval requirement logic (Never, UnlessAutoApproved, Always)
use crate::tools::ApprovalRequirement;
let requirements = vec![
(ApprovalRequirement::Never, "auto-approved"),
(ApprovalRequirement::UnlessAutoApproved, "auto-approved"),
(ApprovalRequirement::Always, "blocks"),
];
for (req, expected) in requirements {
let can_auto_approve = matches!(
req,
ApprovalRequirement::Never | ApprovalRequirement::UnlessAutoApproved
);
let label = if can_auto_approve {
"auto-approved"
} else {
"blocks"
};
assert_eq!(label, expected, "Approval pattern should match");
}
}
#[test]
fn test_empty_response_handling() {
// Simulate the empty content guard logic
let empty_content = "";
let finish_reason_length = crate::llm::FinishReason::Length;
let finish_reason_stop = crate::llm::FinishReason::Stop;
assert!(
empty_content.trim().is_empty(),
"Should detect empty content"
);
assert_eq!(finish_reason_length, crate::llm::FinishReason::Length);
assert_eq!(finish_reason_stop, crate::llm::FinishReason::Stop);
}
}
+16
View File
@@ -160,6 +160,13 @@ impl Scheduler {
.create_job_for_user(user_id, title, description)
.await?;
// Apply token budget from config, allowing per-job metadata override.
let max_tokens = metadata
.as_ref()
.and_then(|m| m.get("max_tokens"))
.and_then(|v| v.as_u64())
.unwrap_or(self.config.max_tokens_per_job);
// Apply metadata if provided
if let Some(meta) = metadata {
self.context_manager
@@ -169,6 +176,15 @@ impl Scheduler {
.await?;
}
// Set token budget (separate update to avoid overwriting metadata)
if max_tokens > 0 {
self.context_manager
.update_context(job_id, |ctx| {
ctx.max_tokens = max_tokens;
})
.await?;
}
// Persist to DB before scheduling so the worker's FK references are valid
if let Some(ref store) = self.store {
let ctx = self.context_manager.get_context(job_id).await?;
+100 -3
View File
@@ -417,7 +417,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
iteration += 1;
if iteration > max_iterations {
self.mark_stuck("Maximum iterations exceeded").await?;
self.mark_failed("Maximum iterations exceeded: job hit the iteration cap")
.await?;
return Ok(());
}
@@ -437,7 +438,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
"LLM rate limited during tool selection, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
self.mark_failed("Persistent rate limiting: exceeded retry limit")
.await?;
return Ok(());
}
self.log_event(
@@ -467,7 +469,8 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
"LLM rate limited during respond_with_tools, backing off"
);
if consecutive_rate_limits >= MAX_CONSECUTIVE_RATE_LIMITS {
self.mark_stuck("Persistent rate limiting").await?;
self.mark_failed("Persistent rate limiting: exceeded retry limit")
.await?;
return Ok(());
}
self.log_event(
@@ -483,6 +486,20 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
Err(e) => return Err(e.into()),
};
// Track token usage from LLM call against the job budget.
// NOTE: select_tools() also makes LLM calls but doesn't expose
// TokenUsage; only respond_with_tools() usage is tracked here.
let total_tokens = respond_output.usage.total() as u64;
if total_tokens > 0
&& let Err(msg) = self
.context_manager()
.update_context(self.job_id, |ctx| ctx.add_tokens(total_tokens))
.await?
{
self.mark_failed(&msg).await?;
return Ok(());
}
match respond_output.result {
RespondResult::Text(response) => {
// Check for explicit completion phrases. Use word-boundary
@@ -1762,4 +1779,84 @@ mod tests {
"Always tool should be allowed with permission"
);
}
#[tokio::test]
async fn test_token_budget_exceeded_fails_job() {
let worker = make_worker(vec![]).await;
// Transition to InProgress (required for mark_failed)
worker
.context_manager()
.update_context(worker.job_id, |ctx| {
ctx.transition_to(JobState::InProgress, None)
})
.await
.unwrap()
.unwrap();
// Set a token budget
worker
.context_manager()
.update_context(worker.job_id, |ctx| {
ctx.max_tokens = 100;
})
.await
.unwrap();
// Simulate adding tokens that exceed the budget
let budget_result = worker
.context_manager()
.update_context(worker.job_id, |ctx| ctx.add_tokens(200))
.await
.unwrap();
assert!(
budget_result.is_err(),
"Should return error when token budget exceeded"
);
// Verify that mark_failed transitions job to Failed
worker
.mark_failed(&budget_result.unwrap_err())
.await
.unwrap();
let ctx = worker
.context_manager()
.get_context(worker.job_id)
.await
.unwrap();
assert_eq!(ctx.state, JobState::Failed);
}
#[tokio::test]
async fn test_iteration_cap_marks_failed_not_stuck() {
let worker = make_worker(vec![]).await;
// Transition to InProgress (required for mark_failed)
worker
.context_manager()
.update_context(worker.job_id, |ctx| {
ctx.transition_to(JobState::InProgress, None)
})
.await
.unwrap()
.unwrap();
// Simulate what the execution loop does when max_iterations is exceeded
worker
.mark_failed("Maximum iterations exceeded: job hit the iteration cap")
.await
.unwrap();
let ctx = worker
.context_manager()
.get_context(worker.job_id)
.await
.unwrap();
assert_eq!(
ctx.state,
JobState::Failed,
"Iteration cap should transition to Failed, not Stuck"
);
}
}
+45 -203
View File
@@ -77,10 +77,7 @@ pub struct AppBuilder {
llm_override: Option<Arc<dyn LlmProvider>>,
// Backend-specific handles needed by secrets store
#[cfg(feature = "postgres")]
pg_pool: Option<deadpool_postgres::Pool>,
#[cfg(feature = "libsql")]
libsql_db: Option<Arc<libsql::Database>>,
handles: Option<crate::db::DatabaseHandles>,
}
impl AppBuilder {
@@ -105,10 +102,7 @@ impl AppBuilder {
db: None,
secrets_store: None,
llm_override: None,
#[cfg(feature = "postgres")]
pg_pool: None,
#[cfg(feature = "libsql")]
libsql_db: None,
handles: None,
}
}
@@ -137,71 +131,10 @@ impl AppBuilder {
return Ok(());
}
let db: Arc<dyn Database> = match self.config.database.backend {
#[cfg(feature = "libsql")]
crate::config::DatabaseBackend::LibSql => {
use crate::db::Database as _;
use crate::db::libsql::LibSqlBackend;
use secrecy::ExposeSecret as _;
let default_path = crate::config::default_libsql_path();
let db_path = self
.config
.database
.libsql_path
.as_deref()
.unwrap_or(&default_path);
let backend = if let Some(ref url) = self.config.database.libsql_url {
let token =
self.config
.database
.libsql_auth_token
.as_ref()
.ok_or_else(|| {
anyhow::anyhow!(
"LIBSQL_AUTH_TOKEN is required when LIBSQL_URL is set"
)
})?;
LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret()).await?
} else {
LibSqlBackend::new_local(db_path).await?
};
backend.run_migrations().await?;
tracing::info!("libSQL database connected and migrations applied");
#[cfg(feature = "libsql")]
{
self.libsql_db = Some(backend.shared_db());
}
Arc::new(backend) as Arc<dyn Database>
}
#[cfg(feature = "postgres")]
_ => {
use crate::db::Database as _;
let pg = crate::db::postgres::PgBackend::new(&self.config.database)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
pg.run_migrations()
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
tracing::info!("PostgreSQL database connected and migrations applied");
#[cfg(feature = "postgres")]
{
self.pg_pool = Some(pg.pool());
}
Arc::new(pg) as Arc<dyn Database>
}
#[cfg(not(feature = "postgres"))]
_ => {
anyhow::bail!(
"No database backend available. Enable 'postgres' or 'libsql' feature."
);
}
};
let (db, handles) = crate::db::connect_with_handles(&self.config.database)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
self.handles = Some(handles);
// Post-init: migrate disk config, reload config from DB, attach session, cleanup
if let Err(e) = crate::bootstrap::migrate_disk_to_db(db.as_ref(), "default").await {
@@ -212,7 +145,7 @@ impl AppBuilder {
match Config::from_db_with_toml(db.as_ref(), "default", toml_path).await {
Ok(db_config) => {
self.config = db_config;
tracing::info!("Configuration reloaded from database");
tracing::debug!("Configuration reloaded from database");
}
Err(e) => {
tracing::warn!(
@@ -251,10 +184,7 @@ impl AppBuilder {
crate::config::inject_os_credentials();
// Consume unused handles
#[cfg(feature = "libsql")]
{
self.libsql_db.take();
}
self.handles.take();
// Re-resolve only the LLM config with OS credentials.
let store: Option<&(dyn crate::db::SettingsStore + Sync)> =
@@ -278,35 +208,16 @@ impl AppBuilder {
Ok(c) => Arc::new(c),
Err(e) => {
tracing::warn!("Failed to initialize secrets crypto: {}", e);
#[cfg(feature = "libsql")]
{
self.libsql_db.take();
}
self.handles.take();
return Ok(());
}
};
let store: Option<Arc<dyn SecretsStore + Send + Sync>> = None;
#[cfg(feature = "libsql")]
let store = store.or_else(|| {
self.libsql_db.take().map(|db| {
Arc::new(crate::secrets::LibSqlSecretsStore::new(
db,
Arc::clone(&crypto),
)) as Arc<dyn SecretsStore + Send + Sync>
})
});
#[cfg(feature = "postgres")]
let store = store.or_else(|| {
self.pg_pool.as_ref().map(|pool| {
Arc::new(crate::secrets::PostgresSecretsStore::new(
pool.clone(),
Arc::clone(&crypto),
)) as Arc<dyn SecretsStore + Send + Sync>
})
});
// Fallback covers the no-database path where `init_database` returned
// early before populating `self.handles`.
let empty_handles = crate::db::DatabaseHandles::default();
let handles = self.handles.as_ref().unwrap_or(&empty_handles);
let store = crate::secrets::create_secrets_store(crypto, handles);
if let Some(ref secrets) = store {
// Inject LLM API keys from encrypted storage
@@ -363,7 +274,7 @@ impl AppBuilder {
anyhow::Error,
> {
let safety = Arc::new(SafetyLayer::new(&self.config.safety));
tracing::info!("Safety layer initialized");
tracing::debug!("Safety layer initialized");
// Initialize tool registry with credential injection support
let credential_registry = Arc::new(SharedCredentialRegistry::new());
@@ -450,7 +361,7 @@ impl AppBuilder {
tools
.register_builder_tool(llm.clone(), Some(self.config.builder.to_builder_config()))
.await;
tracing::info!("Builder mode enabled");
tracing::debug!("Builder mode enabled");
}
Ok((safety, tools, embeddings, workspace))
@@ -472,9 +383,7 @@ impl AppBuilder {
),
anyhow::Error,
> {
use crate::tools::mcp::{
McpClient, McpTransport, config::load_mcp_servers_from_db, is_authenticated,
};
use crate::tools::mcp::config::load_mcp_servers_from_db;
use crate::tools::wasm::{WasmToolLoader, load_dev_tools};
let mcp_session_manager = Arc::new(McpSessionManager::new());
@@ -510,7 +419,7 @@ impl AppBuilder {
match loader.load_from_dir(&wasm_config.tools_dir).await {
Ok(results) => {
if !results.loaded.is_empty() {
tracing::info!(
tracing::debug!(
"Loaded {} WASM tools from {}",
results.loaded.len(),
wasm_config.tools_dir.display()
@@ -533,7 +442,7 @@ impl AppBuilder {
Ok(results) => {
dev_loaded_tool_names.extend(results.loaded.iter().cloned());
if !dev_loaded_tool_names.is_empty() {
tracing::info!(
tracing::debug!(
"Loaded {} dev WASM tools from build artifacts",
dev_loaded_tool_names.len()
);
@@ -565,7 +474,10 @@ impl AppBuilder {
Ok(servers) => {
let enabled: Vec<_> = servers.enabled_servers().cloned().collect();
if !enabled.is_empty() {
tracing::info!("Loading {} configured MCP server(s)...", enabled.len());
tracing::debug!(
"Loading {} configured MCP server(s)...",
enabled.len()
);
}
let mut join_set = tokio::task::JoinSet::new();
@@ -578,95 +490,24 @@ impl AppBuilder {
join_set.spawn(async move {
let server_name = server.name.clone();
let client: McpClient = match server.effective_transport() {
crate::tools::mcp::config::EffectiveTransport::Stdio {
command,
args,
env,
} => {
match pm
.spawn_stdio(
&server_name,
command,
args.to_vec(),
env.clone(),
)
.await
{
Ok(transport) => McpClient::new_with_transport(
&server_name,
transport as Arc<dyn McpTransport>,
None,
secrets,
"default",
Some(server),
),
Err(e) => {
tracing::warn!(
"Failed to spawn stdio MCP server '{}': {}",
server_name,
e
);
return;
}
}
}
#[cfg(unix)]
crate::tools::mcp::config::EffectiveTransport::Unix {
socket_path,
} => {
match crate::tools::mcp::unix_transport::UnixMcpTransport::connect(
&server_name,
socket_path,
)
.await
{
Ok(transport) => McpClient::new_with_transport(
&server_name,
Arc::new(transport) as Arc<dyn McpTransport>,
None,
secrets,
"default",
Some(server),
),
Err(e) => {
tracing::warn!(
"Failed to connect to Unix MCP server '{}': {}",
server_name,
e
);
return;
}
}
}
#[cfg(not(unix))]
crate::tools::mcp::config::EffectiveTransport::Unix { .. } => {
let client = match crate::tools::mcp::create_client_from_config(
server,
&mcp_sm,
&pm,
secrets,
"default",
)
.await
{
Ok(c) => c,
Err(e) => {
tracing::warn!(
"Unix socket transport is not supported on this platform (server '{}')",
server_name
"Failed to create MCP client for '{}': {}",
server_name,
e
);
return;
}
crate::tools::mcp::config::EffectiveTransport::Http => {
if let Some(ref secrets) = secrets {
let has_tokens =
is_authenticated(&server, secrets, "default")
.await;
if has_tokens || server.requires_auth() {
McpClient::new_authenticated(
server,
Arc::clone(&mcp_sm),
Arc::clone(secrets),
"default",
)
} else {
McpClient::new_with_config(server)
}
} else {
McpClient::new_with_config(server)
}
}
};
match client.list_tools().await {
@@ -677,7 +518,7 @@ impl AppBuilder {
for tool in tool_impls {
tools.register(tool).await;
}
tracing::info!(
tracing::debug!(
"Loaded {} tools from MCP server '{}'",
tool_count,
server_name
@@ -738,7 +579,7 @@ impl AppBuilder {
.iter()
.map(|m| m.to_registry_entry())
.collect();
tracing::info!(
tracing::debug!(
count = entries.len(),
"Loaded registry catalog entries for extension discovery"
);
@@ -767,6 +608,7 @@ impl AppBuilder {
let extension_manager = {
let manager = Arc::new(ExtensionManager::new(
Arc::clone(&mcp_session_manager),
Arc::clone(&mcp_process_manager),
ext_secrets,
Arc::clone(tools),
Some(Arc::clone(hooks)),
@@ -779,7 +621,7 @@ impl AppBuilder {
catalog_entries.clone(),
));
tools.register_extension_tools(Arc::clone(&manager));
tracing::info!("Extension manager initialized with in-chat discovery tools");
tracing::debug!("Extension manager initialized with in-chat discovery tools");
Some(manager)
};
@@ -850,7 +692,7 @@ impl AppBuilder {
let import_path = std::path::Path::new(&import_dir);
match ws.import_from_directory(import_path).await {
Ok(count) if count > 0 => {
tracing::info!("Imported {} workspace file(s) from {}", count, import_dir);
tracing::debug!("Imported {} workspace file(s) from {}", count, import_dir);
}
Ok(_) => {}
Err(e) => {
@@ -875,7 +717,7 @@ impl AppBuilder {
tokio::spawn(async move {
match ws_bg.backfill_embeddings().await {
Ok(count) if count > 0 => {
tracing::info!("Backfilled embeddings for {} chunks", count);
tracing::debug!("Backfilled embeddings for {} chunks", count);
}
Ok(_) => {}
Err(e) => {
@@ -892,7 +734,7 @@ impl AppBuilder {
.with_installed_dir(self.config.skills.installed_dir.clone());
let loaded = registry.discover_all().await;
if !loaded.is_empty() {
tracing::info!("Loaded {} skill(s): {}", loaded.len(), loaded.join(", "));
tracing::debug!("Loaded {} skill(s): {}", loaded.len(), loaded.join(", "));
}
let registry = Arc::new(std::sync::RwLock::new(registry));
let catalog = crate::skills::catalog::shared_catalog();
@@ -910,7 +752,7 @@ impl AppBuilder {
},
));
tracing::info!(
tracing::debug!(
"Tool registry initialized with {} total tools",
tools.count()
);
+156
View File
@@ -198,6 +198,58 @@ pub fn save_bootstrap_env_to(path: &std::path::Path, vars: &[(&str, &str)]) -> s
Ok(())
}
/// Update or add multiple variables in `~/.ironclaw/.env`, preserving existing content.
///
/// Like `upsert_bootstrap_var` but batched — replaces lines for any key in `vars`
/// and preserves all other existing lines. Use this instead of `save_bootstrap_env`
/// when you want to update specific keys without destroying user-added variables.
pub fn upsert_bootstrap_vars(vars: &[(&str, &str)]) -> std::io::Result<()> {
upsert_bootstrap_vars_to(&ironclaw_env_path(), vars)
}
/// Update or add multiple variables at an arbitrary path (testable variant).
pub fn upsert_bootstrap_vars_to(
path: &std::path::Path,
vars: &[(&str, &str)],
) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let keys_being_written: std::collections::HashSet<&str> =
vars.iter().map(|(k, _)| *k).collect();
let existing = match std::fs::read_to_string(path) {
Ok(contents) => contents,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => String::new(),
Err(e) => return Err(e),
};
let mut result = String::new();
for line in existing.lines() {
// Extract key from lines matching `KEY=...`
let is_overwritten = line
.split_once('=')
.map(|(k, _)| keys_being_written.contains(k.trim()))
.unwrap_or(false);
if !is_overwritten {
result.push_str(line);
result.push('\n');
}
}
// Append all new key=value pairs
for (key, value) in vars {
let escaped = value.replace('\\', "\\\\").replace('"', "\\\"");
result.push_str(&format!("{}=\"{}\"\n", key, escaped));
}
std::fs::write(path, &result)?;
restrict_file_permissions(path)?;
Ok(())
}
/// Update or add a single variable in `~/.ironclaw/.env`, preserving existing content.
///
/// Unlike `save_bootstrap_env` (which overwrites the entire file), this
@@ -1237,4 +1289,108 @@ INJECTED="pwned"#;
let lock = PidLock::acquire_at(pid_path).unwrap();
drop(lock);
}
#[test]
fn upsert_bootstrap_vars_preserves_unknown_keys() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Simulate a user-edited .env with custom vars
let initial =
"HTTP_HOST=\"0.0.0.0\"\nDATABASE_BACKEND=\"postgres\"\nCUSTOM_VAR=\"keep_me\"\n";
std::fs::write(&env_path, initial).unwrap();
// Upsert wizard vars — should preserve HTTP_HOST and CUSTOM_VAR
let vars = [("DATABASE_BACKEND", "libsql"), ("LLM_BACKEND", "openai")];
upsert_bootstrap_vars_to(&env_path, &vars).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(
parsed.len(),
4,
"should have 4 vars (2 preserved + 2 upserted)"
);
// User-added vars must be preserved
assert!(
parsed
.iter()
.any(|(k, v)| k == "HTTP_HOST" && v == "0.0.0.0"),
"HTTP_HOST must be preserved"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "CUSTOM_VAR" && v == "keep_me"),
"CUSTOM_VAR must be preserved"
);
// Wizard vars must be updated/added
assert!(
parsed
.iter()
.any(|(k, v)| k == "DATABASE_BACKEND" && v == "libsql"),
"DATABASE_BACKEND must be updated to libsql"
);
assert!(
parsed
.iter()
.any(|(k, v)| k == "LLM_BACKEND" && v == "openai"),
"LLM_BACKEND must be added"
);
// Now update LLM_BACKEND and verify HTTP_HOST still preserved
let vars2 = [("LLM_BACKEND", "anthropic")];
upsert_bootstrap_vars_to(&env_path, &vars2).unwrap();
let parsed2: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(
parsed2.len(),
4,
"should still have 4 vars after second upsert"
);
assert!(
parsed2
.iter()
.any(|(k, v)| k == "HTTP_HOST" && v == "0.0.0.0"),
"HTTP_HOST must still be preserved after second upsert"
);
assert!(
parsed2
.iter()
.any(|(k, v)| k == "LLM_BACKEND" && v == "anthropic"),
"LLM_BACKEND must be updated to anthropic"
);
}
#[test]
fn upsert_bootstrap_vars_creates_file_if_missing() {
let dir = tempdir().unwrap();
let env_path = dir.path().join("subdir").join(".env");
// File doesn't exist yet
assert!(!env_path.exists());
let vars = [("DATABASE_BACKEND", "libsql")];
upsert_bootstrap_vars_to(&env_path, &vars).unwrap();
assert!(env_path.exists());
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(
parsed[0],
("DATABASE_BACKEND".to_string(), "libsql".to_string())
);
}
}
+2 -2
View File
@@ -75,7 +75,7 @@ impl ChannelManager {
break;
}
}
tracing::info!(channel = %name, "Hot-added channel stream ended");
tracing::debug!(channel = %name, "Hot-added channel stream ended");
});
Ok(())
@@ -92,7 +92,7 @@ impl ChannelManager {
for (name, channel) in channels.iter() {
match channel.start().await {
Ok(stream) => {
tracing::info!("Started channel: {}", name);
tracing::debug!("Started channel: {}", name);
streams.push(stream);
}
Err(e) => {
+37 -6
View File
@@ -184,18 +184,32 @@ impl WasmChannelLoader {
/// └── telegram.capabilities.json
/// ```
pub async fn load_from_dir(&self, dir: &Path) -> Result<LoadResults, WasmChannelError> {
if !dir.is_dir() {
return Err(WasmChannelError::Io(std::io::Error::new(
std::io::ErrorKind::NotADirectory,
format!("{} is not a directory", dir.display()),
)));
match fs::metadata(dir).await {
Ok(meta) if meta.is_dir() => {}
Ok(_) => {
return Err(WasmChannelError::Io(std::io::Error::new(
std::io::ErrorKind::NotADirectory,
format!("{} is not a directory", dir.display()),
)));
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Ok(LoadResults::default());
}
Err(e) => return Err(WasmChannelError::Io(e)),
}
let mut results = LoadResults::default();
// Collect all .wasm entries first, then load in parallel
let mut channel_entries = Vec::new();
let mut entries = fs::read_dir(dir).await?;
// Handle TOCTOU: if read_dir fails with NotFound, treat as empty
let mut entries = match fs::read_dir(dir).await {
Ok(entries) => entries,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Ok(LoadResults::default());
}
Err(e) => return Err(WasmChannelError::Io(e)),
};
while let Some(entry) = entries.next_entry().await? {
let path = entry.path();
@@ -486,4 +500,21 @@ mod tests {
let result = loader.load_from_files("", &wasm_path, None).await;
assert!(result.is_err());
}
#[tokio::test]
async fn load_from_dir_returns_empty_when_dir_missing() {
let config = WasmChannelRuntimeConfig::for_testing();
let runtime = Arc::new(WasmChannelRuntime::new(config).unwrap());
let loader = WasmChannelLoader::new(runtime, Arc::new(PairingStore::new()), None);
let dir = TempDir::new().unwrap();
let missing = dir.path().join("nonexistent_channels_dir");
let results = loader.load_from_dir(&missing).await;
// Must succeed with empty results, not error
let results = results.expect("missing dir should return Ok, not Err");
assert!(results.loaded.is_empty());
assert!(results.errors.is_empty());
}
}
+2
View File
@@ -86,6 +86,7 @@ mod loader;
mod router;
mod runtime;
mod schema;
pub mod setup;
pub(crate) mod signature;
#[allow(dead_code)]
pub(crate) mod storage;
@@ -105,4 +106,5 @@ pub use runtime::{PreparedChannelModule, WasmChannelRuntime, WasmChannelRuntimeC
pub use schema::{
ChannelCapabilitiesFile, ChannelConfig, SecretSetupSchema, SetupSchema, WebhookSchema,
};
pub use setup::{WasmChannelSetup, inject_channel_credentials, setup_wasm_channels};
pub use wrapper::{HttpResponse, SharedWasmChannel, WasmChannel};
+324
View File
@@ -0,0 +1,324 @@
//! WASM channel setup and credential injection.
//!
//! Encapsulates the logic for loading WASM channels, registering their
//! webhook routes, and injecting credentials from the secrets store.
use std::collections::HashSet;
use std::sync::Arc;
use crate::channels::wasm::{
LoadedChannel, RegisteredEndpoint, SharedWasmChannel, WasmChannel, WasmChannelLoader,
WasmChannelRouter, WasmChannelRuntime, WasmChannelRuntimeConfig, create_wasm_channel_router,
};
use crate::config::Config;
use crate::db::Database;
use crate::extensions::ExtensionManager;
use crate::pairing::PairingStore;
use crate::secrets::SecretsStore;
/// Result of WASM channel setup.
pub struct WasmChannelSetup {
pub channels: Vec<(String, Box<dyn crate::channels::Channel>)>,
pub channel_names: Vec<String>,
pub webhook_routes: Option<axum::Router>,
/// Runtime objects needed for hot-activation via ExtensionManager.
pub wasm_channel_runtime: Arc<WasmChannelRuntime>,
pub pairing_store: Arc<PairingStore>,
pub wasm_channel_router: Arc<WasmChannelRouter>,
}
/// Load WASM channels and register their webhook routes.
pub async fn setup_wasm_channels(
config: &Config,
secrets_store: &Option<Arc<dyn SecretsStore + Send + Sync>>,
extension_manager: Option<&Arc<ExtensionManager>>,
database: Option<&Arc<dyn Database>>,
) -> Option<WasmChannelSetup> {
let runtime = match WasmChannelRuntime::new(WasmChannelRuntimeConfig::default()) {
Ok(r) => Arc::new(r),
Err(e) => {
tracing::warn!("Failed to initialize WASM channel runtime: {}", e);
return None;
}
};
let pairing_store = Arc::new(PairingStore::new());
let settings_store: Option<Arc<dyn crate::db::SettingsStore>> =
database.map(|db| Arc::clone(db) as Arc<dyn crate::db::SettingsStore>);
let mut loader = WasmChannelLoader::new(
Arc::clone(&runtime),
Arc::clone(&pairing_store),
settings_store,
);
if let Some(secrets) = secrets_store {
loader = loader.with_secrets_store(Arc::clone(secrets));
}
let results = match loader
.load_from_dir(&config.channels.wasm_channels_dir)
.await
{
Ok(r) => r,
Err(e) => {
tracing::warn!("Failed to scan WASM channels directory: {}", e);
return None;
}
};
let wasm_router = Arc::new(WasmChannelRouter::new());
let mut channels: Vec<(String, Box<dyn crate::channels::Channel>)> = Vec::new();
let mut channel_names: Vec<String> = Vec::new();
for loaded in results.loaded {
let (name, channel) = register_channel(loaded, config, secrets_store, &wasm_router).await;
channel_names.push(name.clone());
channels.push((name, channel));
}
for (path, err) in &results.errors {
tracing::warn!("Failed to load WASM channel {}: {}", path.display(), err);
}
// Always create webhook routes (even with no channels loaded) so that
// channels hot-added at runtime can receive webhooks without a restart.
let webhook_routes = {
Some(create_wasm_channel_router(
Arc::clone(&wasm_router),
extension_manager.map(Arc::clone),
))
};
Some(WasmChannelSetup {
channels,
channel_names,
webhook_routes,
wasm_channel_runtime: runtime,
pairing_store,
wasm_channel_router: wasm_router,
})
}
/// Process a single loaded WASM channel: retrieve secrets, inject config,
/// register with the router, and set up signing keys and credentials.
async fn register_channel(
loaded: LoadedChannel,
config: &Config,
secrets_store: &Option<Arc<dyn SecretsStore + Send + Sync>>,
wasm_router: &Arc<WasmChannelRouter>,
) -> (String, Box<dyn crate::channels::Channel>) {
let channel_name = loaded.name().to_string();
tracing::info!("Loaded WASM channel: {}", channel_name);
let secret_name = loaded.webhook_secret_name();
let sig_key_secret_name = loaded.signature_key_secret_name();
let hmac_secret_name = loaded.hmac_secret_name();
let webhook_secret = if let Some(secrets) = secrets_store {
secrets
.get_decrypted("default", &secret_name)
.await
.ok()
.map(|s| s.expose().to_string())
} else {
None
};
let secret_header = loaded.webhook_secret_header().map(|s| s.to_string());
let webhook_path = format!("/webhook/{}", channel_name);
let endpoints = vec![RegisteredEndpoint {
channel_name: channel_name.clone(),
path: webhook_path,
methods: vec!["POST".to_string()],
require_secret: webhook_secret.is_some(),
}];
let channel_arc = Arc::new(loaded.channel);
// Inject runtime config (tunnel URL, webhook secret, owner_id).
{
let mut config_updates = std::collections::HashMap::new();
if let Some(ref tunnel_url) = config.tunnel.public_url {
config_updates.insert(
"tunnel_url".to_string(),
serde_json::Value::String(tunnel_url.clone()),
);
}
if let Some(ref secret) = webhook_secret {
config_updates.insert(
"webhook_secret".to_string(),
serde_json::Value::String(secret.clone()),
);
}
if let Some(&owner_id) = config
.channels
.wasm_channel_owner_ids
.get(channel_name.as_str())
{
config_updates.insert("owner_id".to_string(), serde_json::json!(owner_id));
}
if !config_updates.is_empty() {
channel_arc.update_config(config_updates).await;
tracing::info!(
channel = %channel_name,
has_tunnel = config.tunnel.public_url.is_some(),
has_webhook_secret = webhook_secret.is_some(),
"Injected runtime config into channel"
);
}
}
tracing::info!(
channel = %channel_name,
has_webhook_secret = webhook_secret.is_some(),
secret_header = ?secret_header,
"Registering channel with router"
);
wasm_router
.register(
Arc::clone(&channel_arc),
endpoints,
webhook_secret.clone(),
secret_header,
)
.await;
// Register Ed25519 signature key if declared in capabilities.
if let Some(ref sig_key_name) = sig_key_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(key_secret) = secrets.get_decrypted("default", sig_key_name).await
{
match wasm_router
.register_signature_key(&channel_name, key_secret.expose())
.await
{
Ok(()) => {
tracing::info!(channel = %channel_name, "Registered Ed25519 signature key")
}
Err(e) => {
tracing::error!(channel = %channel_name, error = %e, "Invalid signature key in secrets store")
}
}
}
// Register HMAC signing secret if declared in capabilities.
if let Some(ref hmac_secret_name) = hmac_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(secret) = secrets.get_decrypted("default", hmac_secret_name).await
{
wasm_router
.register_hmac_secret(&channel_name, secret.expose())
.await;
tracing::info!(channel = %channel_name, "Registered HMAC signing secret");
}
// Inject credentials from secrets store / environment.
if let Some(secrets) = secrets_store {
match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await {
Ok(count) => {
if count > 0 {
tracing::info!(
channel = %channel_name,
credentials_injected = count,
"Channel credentials injected"
);
}
}
Err(e) => {
tracing::error!(
channel = %channel_name,
error = %e,
"Failed to inject channel credentials"
);
}
}
}
(channel_name, Box::new(SharedWasmChannel::new(channel_arc)))
}
/// Inject credentials for a channel based on naming convention.
///
/// Looks for secrets matching the pattern `{channel_name}_*` and injects them
/// as credential placeholders (e.g., `telegram_bot_token` -> `{TELEGRAM_BOT_TOKEN}`).
///
/// Falls back to environment variables with the uppercase name if not found
/// in the secrets store (e.g., `TELEGRAM_BOT_TOKEN`).
pub async fn inject_channel_credentials(
channel: &Arc<WasmChannel>,
secrets: &dyn SecretsStore,
channel_name: &str,
) -> anyhow::Result<usize> {
let all_secrets = secrets
.list("default")
.await
.map_err(|e| anyhow::anyhow!("Failed to list secrets: {}", e))?;
let prefix = format!("{}_", channel_name);
let mut count = 0;
let mut injected_placeholders = HashSet::new();
for secret_meta in all_secrets {
if !secret_meta.name.starts_with(&prefix) {
continue;
}
let decrypted = match secrets.get_decrypted("default", &secret_meta.name).await {
Ok(d) => d,
Err(e) => {
tracing::warn!(
secret = %secret_meta.name,
error = %e,
"Failed to decrypt secret for channel credential injection"
);
continue;
}
};
let placeholder = secret_meta.name.to_uppercase();
tracing::debug!(
channel = %channel_name,
secret = %secret_meta.name,
placeholder = %placeholder,
"Injecting credential"
);
channel
.set_credential(&placeholder, decrypted.expose().to_string())
.await;
injected_placeholders.insert(placeholder);
count += 1;
}
// Fall back to environment variables for required secrets not found in the store.
// This allows channels to work when configured via env vars (e.g., TELEGRAM_BOT_TOKEN)
// without requiring the setup wizard to have run.
let caps = channel.capabilities();
if let Some(ref http_cap) = caps.tool_capabilities.http {
for cred_mapping in http_cap.credentials.values() {
let placeholder = cred_mapping.secret_name.to_uppercase();
if injected_placeholders.contains(&placeholder) {
continue;
}
if let Ok(env_value) = std::env::var(&placeholder)
&& !env_value.is_empty()
{
tracing::debug!(
channel = %channel_name,
placeholder = %placeholder,
"Injecting credential from environment variable"
);
channel.set_credential(&placeholder, env_value).await;
count += 1;
}
}
}
Ok(count)
}
+15 -1
View File
@@ -276,11 +276,25 @@ pub async fn jobs_cancel_handler(
})));
}
// Fall back to agent job cancellation via DB status update.
// Fall back to agent job cancellation: stop the worker via the scheduler
// (which updates the in-memory ContextManager AND aborts the task handle),
// then persist the status to the DB as a fallback.
if let Some(ref store) = state.store
&& let Ok(Some(job)) = store.get_job(job_id).await
{
if job.state.is_active() {
// Try to stop via scheduler (aborts the worker task + updates
// in-memory ContextManager). This is best-effort — the job may
// not be in the scheduler map if it already finished.
if let Some(ref slot) = state.scheduler
&& let Some(ref scheduler) = *slot.read().await
{
let _ = scheduler.stop(job_id).await;
}
// Always persist cancellation to the DB so the state is
// consistent even if the scheduler wasn't available or the
// job wasn't in its in-memory map.
store
.update_job_status(
job_id,
+2
View File
@@ -108,6 +108,7 @@ pub async fn routines_detail_handler(
status: format!("{:?}", run.status),
result_summary: run.result_summary.clone(),
tokens_used: run.tokens_used,
job_id: run.job_id,
})
.collect();
@@ -252,6 +253,7 @@ pub async fn routines_runs_handler(
status: format!("{:?}", run.status),
result_summary: run.result_summary.clone(),
tokens_used: run.tokens_used,
job_id: run.job_id,
})
.collect();
+6 -1
View File
@@ -370,7 +370,7 @@ pub async fn start_server(
if let Err(e) = axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = shutdown_rx.await;
tracing::info!("Web gateway shutting down");
tracing::debug!("Web gateway shutting down");
})
.await
{
@@ -2017,6 +2017,7 @@ async fn routines_detail_handler(
status: format!("{:?}", run.status),
result_summary: run.result_summary.clone(),
tokens_used: run.tokens_used,
job_id: run.job_id,
})
.collect();
@@ -2169,6 +2170,7 @@ async fn routines_runs_handler(
status: format!("{:?}", run.status),
result_summary: run.result_summary.clone(),
tokens_used: run.tokens_used,
job_id: run.job_id,
})
.collect();
@@ -2607,6 +2609,7 @@ mod tests {
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets,
tool_registry,
None,
@@ -2656,6 +2659,7 @@ mod tests {
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets.clone(),
tool_registry,
None,
@@ -2761,6 +2765,7 @@ mod tests {
let ext_mgr = Arc::new(ExtensionManager::new(
mcp_sm,
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
secrets.clone(),
tool_registry,
None,
+1
View File
@@ -776,6 +776,7 @@ pub struct RoutineRunInfo {
pub status: String,
pub result_summary: Option<String>,
pub tokens_used: Option<i32>,
pub job_id: Option<Uuid>,
}
// --- Settings ---
+1 -1
View File
@@ -68,7 +68,7 @@ impl WebhookServer {
if let Err(e) = axum::serve(listener, app)
.with_graceful_shutdown(async {
let _ = shutdown_rx.await;
tracing::info!("Webhook server shutting down");
tracing::debug!("Webhook server shutting down");
})
.await
{
+2 -12
View File
@@ -10,7 +10,7 @@ use clap::{Args, Subcommand};
use crate::config::Config;
use crate::db::Database;
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::secrets::SecretsStore;
use crate::tools::mcp::{
McpClient, McpServerConfig, McpSessionManager, OAuthConfig,
auth::{authorize_mcp_server, is_authenticated},
@@ -628,17 +628,7 @@ async fn save_servers(
/// Initialize and return the secrets store.
async fn get_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Sync>> {
let config = Config::from_env().await?;
let master_key = config.secrets.master_key().ok_or_else(|| {
anyhow::anyhow!(
"SECRETS_MASTER_KEY not set. Run 'ironclaw onboard' first or set it in .env"
)
})?;
let crypto = Arc::new(SecretsCrypto::new(master_key.clone())?);
Ok(crate::db::create_secrets_store(&config.database, crypto).await?)
crate::cli::init_secrets_store().await
}
#[cfg(test)]
+45 -4
View File
@@ -28,8 +28,6 @@ pub use config::{ConfigCommand, run_config_command};
pub use doctor::run_doctor_command;
pub use mcp::{McpCommand, run_mcp_command};
pub use memory::MemoryCommand;
#[cfg(feature = "postgres")]
pub use memory::run_memory_command;
pub use memory::run_memory_command_with_db;
pub use pairing::{PairingCommand, run_pairing_command, run_pairing_command_with_store};
pub use registry::{RegistryCommand, run_registry_command};
@@ -37,6 +35,8 @@ pub use service::{ServiceCommand, run_service_command};
pub use status::run_status_command;
pub use tool::{ToolCommand, run_tool_command};
use std::sync::Arc;
use clap::{ColorChoice, Parser, Subcommand};
#[derive(Parser, Debug)]
@@ -94,12 +94,16 @@ pub enum Command {
skip_auth: bool,
/// Reconfigure channels only
#[arg(long, conflicts_with = "provider_only")]
#[arg(long, conflicts_with_all = ["provider_only", "quick"])]
channels_only: bool,
/// Reconfigure LLM provider and model only
#[arg(long, conflicts_with = "channels_only")]
#[arg(long, conflicts_with_all = ["channels_only", "quick"])]
provider_only: bool,
/// Quick setup: auto-defaults everything except LLM provider and model
#[arg(long, conflicts_with_all = ["channels_only", "provider_only"])]
quick: bool,
},
/// Manage configuration settings
@@ -225,6 +229,43 @@ impl Cli {
}
}
/// Initialize a secrets store from environment config.
///
/// Shared helper for CLI subcommands (`mcp auth`, `tool auth`, etc.) that need
/// access to encrypted secrets without spinning up the full AppBuilder.
pub async fn init_secrets_store()
-> anyhow::Result<Arc<dyn crate::secrets::SecretsStore + Send + Sync>> {
let config = crate::config::Config::from_env().await?;
let master_key = config.secrets.master_key().ok_or_else(|| {
anyhow::anyhow!(
"SECRETS_MASTER_KEY not set. Run 'ironclaw onboard' first or set it in .env"
)
})?;
let crypto = Arc::new(crate::secrets::SecretsCrypto::new(master_key.clone())?);
Ok(crate::db::create_secrets_store(&config.database, crypto).await?)
}
/// Run the Memory CLI subcommand.
pub async fn run_memory_command(mem_cmd: &MemoryCommand) -> anyhow::Result<()> {
let config = crate::config::Config::from_env()
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
let session = crate::llm::create_session_manager(config.llm.session.clone()).await;
let embeddings = config
.embeddings
.create_provider(&config.llm.nearai.base_url, session);
let db: Arc<dyn crate::db::Database> = crate::db::connect_from_config(&config.database)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
run_memory_command_with_db(mem_cmd.clone(), db, embeddings).await
}
#[cfg(test)]
mod tests {
use super::*;
+2 -12
View File
@@ -10,8 +10,7 @@ use clap::Subcommand;
use tokio::fs;
use crate::bootstrap::ironclaw_base_dir;
use crate::config::Config;
use crate::secrets::{CreateSecretParams, SecretsCrypto, SecretsStore};
use crate::secrets::{CreateSecretParams, SecretsStore};
use crate::tools::wasm::{CapabilitiesFile, compute_binary_hash};
/// Default tools directory.
@@ -552,16 +551,7 @@ fn validate_tool_name(name: &str) -> anyhow::Result<()> {
/// Initialize the secrets store from environment config.
async fn init_secrets_store() -> anyhow::Result<Arc<dyn SecretsStore + Send + Sync>> {
let config = Config::from_env().await?;
let master_key = config.secrets.master_key().ok_or_else(|| {
anyhow::anyhow!(
"SECRETS_MASTER_KEY not set. Run 'ironclaw onboard' first or set it in .env"
)
})?;
let crypto = Arc::new(SecretsCrypto::new(master_key.clone())?);
Ok(crate::db::create_secrets_store(&config.database, crypto).await?)
crate::cli::init_secrets_store().await
}
/// Configure authentication for a tool.
+7
View File
@@ -29,6 +29,8 @@ pub struct AgentConfig {
pub auto_approve_tools: bool,
/// Default timezone for new sessions (IANA name, e.g. "America/New_York").
pub default_timezone: String,
/// Maximum tokens per job (0 = unlimited).
pub max_tokens_per_job: u64,
}
impl AgentConfig {
@@ -50,6 +52,7 @@ impl AgentConfig {
max_tool_iterations: 10,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
max_tokens_per_job: 0,
}
}
@@ -105,6 +108,10 @@ impl AgentConfig {
}
tz
},
max_tokens_per_job: parse_optional_env(
"AGENT_MAX_TOKENS_PER_JOB",
settings.agent.max_tokens_per_job,
)?,
})
}
}
+4 -4
View File
@@ -100,13 +100,13 @@ impl EmbeddingsConfig {
session: Arc<SessionManager>,
) -> Option<Arc<dyn EmbeddingProvider>> {
if !self.enabled {
tracing::info!("Embeddings disabled (set EMBEDDING_ENABLED=true to enable)");
tracing::debug!("Embeddings disabled (set EMBEDDING_ENABLED=true to enable)");
return None;
}
match self.provider.as_str() {
"nearai" => {
tracing::info!(
tracing::debug!(
"Embeddings enabled via NEAR AI (model: {}, dim: {})",
self.model,
self.dimension,
@@ -117,7 +117,7 @@ impl EmbeddingsConfig {
))
}
"ollama" => {
tracing::info!(
tracing::debug!(
"Embeddings enabled via Ollama (model: {}, url: {}, dim: {})",
self.model,
self.ollama_base_url,
@@ -130,7 +130,7 @@ impl EmbeddingsConfig {
}
_ => {
if let Some(api_key) = self.openai_api_key() {
tracing::info!(
tracing::debug!(
"Embeddings enabled via OpenAI (model: {}, dim: {})",
self.model,
self.dimension,
+9
View File
@@ -14,6 +14,10 @@ pub struct RoutineConfig {
pub default_cooldown_secs: u64,
/// Max output tokens for lightweight routine LLM calls.
pub max_lightweight_tokens: u32,
/// Enable tool execution in lightweight routines (default: true).
pub lightweight_tools_enabled: bool,
/// Max tool iterations for lightweight routines (default: 3, max: 5).
pub lightweight_max_iterations: u32,
}
impl Default for RoutineConfig {
@@ -24,18 +28,23 @@ impl Default for RoutineConfig {
max_concurrent_routines: 10,
default_cooldown_secs: 300,
max_lightweight_tokens: 4096,
lightweight_tools_enabled: true,
lightweight_max_iterations: 3,
}
}
}
impl RoutineConfig {
pub(crate) fn resolve() -> Result<Self, ConfigError> {
let max_iterations: u32 = parse_optional_env("ROUTINES_LIGHTWEIGHT_MAX_ITERATIONS", 3)?;
Ok(Self {
enabled: parse_bool_env("ROUTINES_ENABLED", true)?,
cron_check_interval_secs: parse_optional_env("ROUTINES_CRON_INTERVAL", 15)?,
max_concurrent_routines: parse_optional_env("ROUTINES_MAX_CONCURRENT", 10)?,
default_cooldown_secs: parse_optional_env("ROUTINES_DEFAULT_COOLDOWN", 300)?,
max_lightweight_tokens: parse_optional_env("ROUTINES_MAX_TOKENS", 4096)?,
lightweight_tools_enabled: parse_bool_env("ROUTINES_LIGHTWEIGHT_TOOLS", true)?,
lightweight_max_iterations: max_iterations.min(5), // cap at 5
})
}
}
+4 -1
View File
@@ -28,14 +28,16 @@ impl JobStore for LibSqlBackend {
r#"
INSERT INTO agent_jobs (
id, conversation_id, title, description, category, status, source,
user_id,
budget_amount, budget_token, bid_amount, estimated_cost, estimated_time_secs,
actual_cost, repair_attempts, created_at, started_at, completed_at
) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17)
) VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18)
ON CONFLICT (id) DO UPDATE SET
title = excluded.title,
description = excluded.description,
category = excluded.category,
status = excluded.status,
user_id = excluded.user_id,
estimated_cost = excluded.estimated_cost,
estimated_time_secs = excluded.estimated_time_secs,
actual_cost = excluded.actual_cost,
@@ -51,6 +53,7 @@ impl JobStore for LibSqlBackend {
opt_text(ctx.category.as_deref()),
status,
"direct",
ctx.user_id.as_str(),
opt_text_owned(ctx.budget.map(|d| d.to_string())),
opt_text(ctx.budget_token.as_deref()),
opt_text_owned(ctx.bid_amount.map(|d| d.to_string())),
+18
View File
@@ -482,6 +482,24 @@ mod tests {
assert_eq!(timeout, 5000);
}
/// Regression test: save_job must persist user_id and get_job must return it.
#[tokio::test]
async fn test_save_job_persists_user_id() {
use crate::context::JobContext;
use crate::db::JobStore;
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_user_id.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let ctx = JobContext::with_user("test-user-42", "Test Job", "A test job");
backend.save_job(&ctx).await.unwrap();
let loaded = backend.get_job(ctx.job_id).await.unwrap().unwrap();
assert_eq!(loaded.user_id, "test-user-42");
}
#[tokio::test]
async fn test_concurrent_writes_succeed() {
// Use a temp file so connections share state (in-memory DBs are connection-local)
+33 -2
View File
@@ -51,6 +51,29 @@ use crate::workspace::{SearchConfig, SearchResult};
pub async fn connect_from_config(
config: &crate::config::DatabaseConfig,
) -> Result<Arc<dyn Database>, DatabaseError> {
let (db, _handles) = connect_with_handles(config).await?;
Ok(db)
}
/// Backend-specific handles retained after database connection.
///
/// These are needed by satellite stores (e.g., `SecretsStore`) that require
/// a backend-specific handle rather than the generic `Arc<dyn Database>`.
#[derive(Default)]
pub struct DatabaseHandles {
#[cfg(feature = "postgres")]
pub pg_pool: Option<deadpool_postgres::Pool>,
#[cfg(feature = "libsql")]
pub libsql_db: Option<Arc<::libsql::Database>>,
}
/// Connect to the database, run migrations, and return both the generic
/// `Database` trait object and the backend-specific handles.
pub async fn connect_with_handles(
config: &crate::config::DatabaseConfig,
) -> Result<(Arc<dyn Database>, DatabaseHandles), DatabaseError> {
let mut handles = DatabaseHandles::default();
match config.backend {
#[cfg(feature = "libsql")]
crate::config::DatabaseBackend::LibSql => {
@@ -74,7 +97,11 @@ pub async fn connect_from_config(
.map_err(|e| DatabaseError::Pool(e.to_string()))?
};
backend.run_migrations().await?;
Ok(Arc::new(backend))
tracing::info!("libSQL database connected and migrations applied");
handles.libsql_db = Some(backend.shared_db());
Ok((Arc::new(backend) as Arc<dyn Database>, handles))
}
#[cfg(feature = "postgres")]
_ => {
@@ -82,7 +109,11 @@ pub async fn connect_from_config(
.await
.map_err(|e| DatabaseError::Pool(e.to_string()))?;
pg.run_migrations().await?;
Ok(Arc::new(pg))
tracing::info!("PostgreSQL database connected and migrations applied");
handles.pg_pool = Some(pg.pool());
Ok((Arc::new(pg) as Arc<dyn Database>, handles))
}
#[cfg(not(feature = "postgres"))]
_ => Err(DatabaseError::Pool(
+16 -12
View File
@@ -73,6 +73,7 @@ pub struct ExtensionManager {
// MCP infrastructure
mcp_session_manager: Arc<McpSessionManager>,
mcp_process_manager: Arc<crate::tools::mcp::process::McpProcessManager>,
/// Active MCP clients keyed by server name.
mcp_clients: RwLock<HashMap<String, Arc<McpClient>>>,
@@ -116,6 +117,7 @@ impl ExtensionManager {
#[allow(clippy::too_many_arguments)]
pub fn new(
mcp_session_manager: Arc<McpSessionManager>,
mcp_process_manager: Arc<crate::tools::mcp::process::McpProcessManager>,
secrets: Arc<dyn SecretsStore + Send + Sync>,
tool_registry: Arc<ToolRegistry>,
hooks: Option<Arc<HookRegistry>>,
@@ -136,6 +138,7 @@ impl ExtensionManager {
registry,
discovery: OnlineDiscovery::new(),
mcp_session_manager,
mcp_process_manager,
mcp_clients: RwLock::new(HashMap::new()),
wasm_tool_runtime,
wasm_tools_dir,
@@ -2467,18 +2470,15 @@ impl ExtensionManager {
.await
.map_err(|e| ExtensionError::NotInstalled(e.to_string()))?;
let has_tokens = is_authenticated(&server, &self.secrets, &self.user_id).await;
let client = if has_tokens || server.requires_auth() {
McpClient::new_authenticated(
server.clone(),
Arc::clone(&self.mcp_session_manager),
Arc::clone(&self.secrets),
&self.user_id,
)
} else {
McpClient::new_with_config(server.clone())
};
let client = crate::tools::mcp::create_client_from_config(
server.clone(),
&self.mcp_session_manager,
&self.mcp_process_manager,
Some(Arc::clone(&self.secrets)),
&self.user_id,
)
.await
.map_err(|e| ExtensionError::ActivationFailed(e.to_string()))?;
// Try to list and create tools
let mcp_tools = client
@@ -3736,6 +3736,7 @@ mod tests {
tools_dir: std::path::PathBuf,
) -> crate::extensions::manager::ExtensionManager {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
use crate::tools::mcp::process::McpProcessManager;
use crate::tools::mcp::session::McpSessionManager;
let key = secrecy::SecretString::from(crate::secrets::keychain::generate_master_key_hex());
@@ -3747,6 +3748,7 @@ mod tests {
crate::extensions::manager::ExtensionManager::new(
mcp,
Arc::new(McpProcessManager::new()),
secrets,
tools,
None, // hooks
@@ -3906,6 +3908,7 @@ mod tests {
) -> ExtensionManager {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
use crate::tools::ToolRegistry;
use crate::tools::mcp::process::McpProcessManager;
use crate::tools::mcp::session::McpSessionManager;
std::fs::create_dir_all(&tools_dir).ok();
@@ -3917,6 +3920,7 @@ mod tests {
ExtensionManager::new(
Arc::new(McpSessionManager::new()),
Arc::new(McpProcessManager::new()),
Arc::new(InMemorySecretsStore::new(crypto)),
Arc::new(ToolRegistry::new()),
None,
+36 -1
View File
@@ -149,14 +149,16 @@ impl Store {
r#"
INSERT INTO agent_jobs (
id, conversation_id, title, description, category, status, source,
user_id,
budget_amount, budget_token, bid_amount, estimated_cost, estimated_time_secs,
actual_cost, repair_attempts, created_at, started_at, completed_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17)
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18)
ON CONFLICT (id) DO UPDATE SET
title = EXCLUDED.title,
description = EXCLUDED.description,
category = EXCLUDED.category,
status = EXCLUDED.status,
user_id = EXCLUDED.user_id,
estimated_cost = EXCLUDED.estimated_cost,
estimated_time_secs = EXCLUDED.estimated_time_secs,
actual_cost = EXCLUDED.actual_cost,
@@ -172,6 +174,7 @@ impl Store {
&ctx.category,
&status,
&"direct", // source
&ctx.user_id,
&ctx.budget,
&ctx.budget_token,
&ctx.bid_amount,
@@ -2133,4 +2136,36 @@ mod tests {
assert_eq!(summary.channel, ch);
}
}
/// Regression test: save_job must persist user_id and get_job must return it.
/// Requires a running PostgreSQL instance (integration tier).
#[cfg(feature = "postgres")]
#[tokio::test]
#[ignore]
async fn test_save_job_persists_user_id() {
use crate::config::Config;
use crate::context::JobContext;
let _ = dotenvy::dotenv();
let config = Config::from_env().await.expect("Failed to load config");
let store = Store::new(&config.database)
.await
.expect("Failed to connect to database");
store
.run_migrations()
.await
.expect("Failed to run migrations");
let ctx = JobContext::with_user("test-user-42", "PG user_id test", "regression test");
store.save_job(&ctx).await.unwrap();
let loaded = store.get_job(ctx.job_id).await.unwrap().unwrap();
assert_eq!(loaded.user_id, "test-user-42");
// Clean up
let conn = store.conn().await.unwrap();
conn.execute("DELETE FROM agent_jobs WHERE id = $1", &[&ctx.job_id])
.await
.unwrap();
}
}
+14 -14
View File
@@ -117,7 +117,7 @@ pub fn create_llm_provider_with_config(
} else {
"session token"
};
tracing::info!(
tracing::debug!(
model = %config.model,
base_url = %config.base_url,
auth = auth_mode,
@@ -156,7 +156,7 @@ async fn create_bedrock_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvid
})?;
let provider = bedrock::BedrockProvider::new(br).await?;
tracing::info!(
tracing::debug!(
"Using AWS Bedrock (Converse API, region: {}, model: {})",
br.region,
provider.active_model_name(),
@@ -221,7 +221,7 @@ fn create_openai_compat_from_registry(
let client = client.completions_api();
let model = client.completion_model(&config.model);
tracing::info!(
tracing::debug!(
provider = %config.provider_id,
model = %config.model,
base_url = %config.base_url,
@@ -242,7 +242,7 @@ fn create_anthropic_from_registry(
.as_ref()
.is_some_and(|k| k.expose_secret() == crate::llm::config::OAUTH_PLACEHOLDER);
if config.oauth_token.is_some() && (config.api_key.is_none() || api_key_is_placeholder) {
tracing::info!(
tracing::debug!(
provider = %config.provider_id,
model = %config.model,
base_url = if config.base_url.is_empty() { "default" } else { &config.base_url },
@@ -281,14 +281,14 @@ fn create_anthropic_from_registry(
let model = client.completion_model(&config.model);
if cache_retention != CacheRetention::None {
tracing::info!(
tracing::debug!(
model = %config.model,
retention = %cache_retention,
"Anthropic automatic prompt caching enabled"
);
}
tracing::info!(
tracing::debug!(
provider = %config.provider_id,
model = %config.model,
base_url = if config.base_url.is_empty() { "default" } else { &config.base_url },
@@ -317,7 +317,7 @@ fn create_ollama_from_registry(
let model = client.completion_model(&config.model);
tracing::info!(
tracing::debug!(
provider = %config.provider_id,
model = %config.model,
base_url = %config.base_url,
@@ -385,14 +385,14 @@ pub async fn build_provider_chain(
LlmError,
> {
let llm = create_llm_provider(config, session.clone()).await?;
tracing::info!("LLM provider initialized: {}", llm.model_name());
tracing::debug!("LLM provider initialized: {}", llm.model_name());
// 1. Retry
let retry_config = RetryConfig {
max_retries: config.nearai.max_retries,
};
let llm: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
tracing::info!(
tracing::debug!(
max_retries = retry_config.max_retries,
"LLM retry wrapper enabled"
);
@@ -415,7 +415,7 @@ pub async fn build_provider_chain(
} else {
cheap
};
tracing::info!(
tracing::debug!(
primary = %llm.model_name(),
cheap = %cheap.model_name(),
"Smart routing enabled"
@@ -446,7 +446,7 @@ pub async fn build_provider_chain(
session.clone(),
config.request_timeout_secs,
)?;
tracing::info!(
tracing::debug!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
"LLM failover enabled"
@@ -478,7 +478,7 @@ pub async fn build_provider_chain(
),
..CircuitBreakerConfig::default()
};
tracing::info!(
tracing::debug!(
threshold,
recovery_secs = config.nearai.circuit_breaker_recovery_secs,
"LLM circuit breaker enabled"
@@ -494,7 +494,7 @@ pub async fn build_provider_chain(
ttl: std::time::Duration::from_secs(config.nearai.response_cache_ttl_secs),
max_entries: config.nearai.response_cache_max_entries,
};
tracing::info!(
tracing::debug!(
ttl_secs = config.nearai.response_cache_ttl_secs,
max_entries = config.nearai.response_cache_max_entries,
"LLM response cache enabled"
@@ -515,7 +515,7 @@ pub async fn build_provider_chain(
// Standalone cheap LLM for heartbeat/evaluation (not part of the chain)
let cheap_llm = create_cheap_llm_provider(config, session)?;
if let Some(ref cheap) = cheap_llm {
tracing::info!("Cheap LLM provider initialized: {}", cheap.model_name());
tracing::debug!("Cheap LLM provider initialized: {}", cheap.model_name());
}
Ok((llm, cheap_llm, recording_handle))
+1 -1
View File
@@ -110,7 +110,7 @@ impl NearAiChatProvider {
handle.spawn(async move {
match fetch_pricing(&client, &base_url, api_key.as_ref(), &session).await {
Ok(map) if !map.is_empty() => {
tracing::info!("Loaded NEAR AI pricing for {} model(s)", map.len());
tracing::debug!("Loaded NEAR AI pricing for {} model(s)", map.len());
match pricing.write() {
Ok(mut guard) => *guard = map,
Err(poisoned) => *poisoned.into_inner() = map,
+46 -597
View File
@@ -4,7 +4,6 @@ use std::sync::Arc;
use std::time::Duration;
use clap::Parser;
use tracing_subscriber::EnvFilter;
use ironclaw::{
agent::{Agent, AgentDeps},
@@ -12,10 +11,7 @@ use ironclaw::{
channels::{
ChannelManager, GatewayChannel, HttpChannel, ReplChannel, SignalChannel, WebhookServer,
WebhookServerConfig,
wasm::{
RegisteredEndpoint, SharedWasmChannel, WasmChannelLoader, WasmChannelRouter,
WasmChannelRuntime, WasmChannelRuntimeConfig, create_wasm_channel_router,
},
wasm::{WasmChannelRouter, WasmChannelRuntime},
web::log_layer::LogBroadcaster,
},
cli::{
@@ -25,26 +21,14 @@ use ironclaw::{
config::Config,
hooks::bootstrap_hooks,
llm::create_session_manager,
orchestrator::{
ContainerJobConfig, ContainerJobManager, OrchestratorApi, ReaperConfig, SandboxReaper,
TokenStore, api::OrchestratorState,
},
orchestrator::{ReaperConfig, SandboxReaper},
pairing::PairingStore,
secrets::SecretsStore,
tracing_fmt::{init_cli_tracing, init_worker_tracing},
};
#[cfg(any(feature = "postgres", feature = "libsql"))]
use ironclaw::setup::{SetupConfig, SetupWizard};
/// Initialize tracing for simple CLI commands (warn level, no fancy layers).
fn init_cli_tracing() {
tracing_subscriber::fmt()
.with_env_filter(
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("warn")),
)
.init();
}
/// Synchronous entry point. Loads `.env` files before the Tokio runtime
/// starts so that `std::env::set_var` is safe (no worker threads yet).
fn main() -> anyhow::Result<()> {
@@ -80,7 +64,7 @@ async fn async_main() -> anyhow::Result<()> {
}
Some(Command::Memory(mem_cmd)) => {
init_cli_tracing();
return run_memory_command(mem_cmd).await;
return ironclaw::cli::run_memory_command(mem_cmd).await;
}
Some(Command::Pairing(pairing_cmd)) => {
init_cli_tracing();
@@ -108,7 +92,7 @@ async fn async_main() -> anyhow::Result<()> {
max_iterations,
}) => {
init_worker_tracing();
return run_worker(*job_id, orchestrator_url, *max_iterations).await;
return ironclaw::worker::run_worker(*job_id, orchestrator_url, *max_iterations).await;
}
Some(Command::ClaudeBridge {
job_id,
@@ -117,12 +101,19 @@ async fn async_main() -> anyhow::Result<()> {
model,
}) => {
init_worker_tracing();
return run_claude_bridge(*job_id, orchestrator_url, *max_turns, model).await;
return ironclaw::worker::run_claude_bridge(
*job_id,
orchestrator_url,
*max_turns,
model,
)
.await;
}
Some(Command::Onboard {
skip_auth,
channels_only,
provider_only,
quick,
}) => {
#[cfg(any(feature = "postgres", feature = "libsql"))]
{
@@ -130,13 +121,14 @@ async fn async_main() -> anyhow::Result<()> {
skip_auth: *skip_auth,
channels_only: *channels_only,
provider_only: *provider_only,
quick: *quick,
};
let mut wizard = SetupWizard::with_config(config);
wizard.run().await?;
}
#[cfg(not(any(feature = "postgres", feature = "libsql")))]
{
let _ = (skip_auth, channels_only, provider_only);
let _ = (skip_auth, channels_only, provider_only, quick);
eprintln!("Onboarding wizard requires the 'postgres' or 'libsql' feature.");
}
return Ok(());
@@ -169,11 +161,14 @@ async fn async_main() -> anyhow::Result<()> {
// Enhanced first-run detection
#[cfg(any(feature = "postgres", feature = "libsql"))]
if !cli.no_onboard
&& let Some(reason) = check_onboard_needed()
&& let Some(reason) = ironclaw::setup::check_onboard_needed()
{
println!("Onboarding needed: {}", reason);
println!();
let mut wizard = SetupWizard::new();
let mut wizard = SetupWizard::with_config(SetupConfig {
quick: true,
..Default::default()
});
wizard.run().await?;
}
@@ -206,9 +201,9 @@ async fn async_main() -> anyhow::Result<()> {
let log_level_handle =
ironclaw::channels::web::log_layer::init_tracing(Arc::clone(&log_broadcaster));
tracing::info!("Starting IronClaw...");
tracing::info!("Loaded configuration for agent: {}", config.agent.name);
tracing::info!("LLM backend: {}", config.llm.backend);
tracing::debug!("Starting IronClaw...");
tracing::debug!("Loaded configuration for agent: {}", config.agent.name);
tracing::debug!("LLM backend: {}", config.llm.backend);
// ── Phase 1-5: Build all core components via AppBuilder ────────────
@@ -227,95 +222,21 @@ async fn async_main() -> anyhow::Result<()> {
// ── Tunnel setup ───────────────────────────────────────────────────
let (config, active_tunnel) = start_tunnel(config).await;
let (config, active_tunnel) = ironclaw::tunnel::start_managed_tunnel(config).await;
// ── Orchestrator / container job manager ────────────────────────────
// Proactive Docker detection
let docker_status = if config.sandbox.enabled {
let detection = ironclaw::sandbox::check_docker().await;
match detection.status {
ironclaw::sandbox::DockerStatus::Available => {
tracing::info!("Docker is available");
}
ironclaw::sandbox::DockerStatus::NotInstalled => {
tracing::warn!(
"Docker is not installed -- sandbox disabled for this session. {}",
detection.platform.install_hint()
);
}
ironclaw::sandbox::DockerStatus::NotRunning => {
tracing::warn!(
"Docker is installed but not running -- sandbox disabled for this session. {}",
detection.platform.start_hint()
);
}
ironclaw::sandbox::DockerStatus::Disabled => {}
}
detection.status
} else {
ironclaw::sandbox::DockerStatus::Disabled
};
let job_event_tx: Option<
tokio::sync::broadcast::Sender<(uuid::Uuid, ironclaw::channels::web::types::SseEvent)>,
> = if config.sandbox.enabled && docker_status.is_ok() {
let (tx, _) = tokio::sync::broadcast::channel(256);
Some(tx)
} else {
None
};
let prompt_queue = Arc::new(tokio::sync::Mutex::new(std::collections::HashMap::<
uuid::Uuid,
std::collections::VecDeque<ironclaw::orchestrator::api::PendingPrompt>,
>::new()));
let container_job_manager: Option<Arc<ContainerJobManager>> =
if config.sandbox.enabled && docker_status.is_ok() {
let token_store = TokenStore::new();
let job_config = ContainerJobConfig {
image: config.sandbox.image.clone(),
memory_limit_mb: config.sandbox.memory_limit_mb,
cpu_shares: config.sandbox.cpu_shares,
orchestrator_port: 50051,
claude_code_api_key: std::env::var("ANTHROPIC_API_KEY").ok(),
claude_code_oauth_token: ironclaw::config::ClaudeCodeConfig::extract_oauth_token(),
claude_code_model: config.claude_code.model.clone(),
claude_code_max_turns: config.claude_code.max_turns,
claude_code_memory_limit_mb: config.claude_code.memory_limit_mb,
claude_code_allowed_tools: config.claude_code.allowed_tools.clone(),
};
let jm = Arc::new(ContainerJobManager::new(job_config, token_store.clone()));
// Start the orchestrator internal API in the background
let orchestrator_state = OrchestratorState {
llm: components.llm.clone(),
job_manager: Arc::clone(&jm),
token_store,
job_event_tx: job_event_tx.clone(),
prompt_queue: Arc::clone(&prompt_queue),
store: components.db.clone(),
secrets_store: components.secrets_store.clone(),
user_id: "default".to_string(),
};
tokio::spawn(async move {
if let Err(e) = OrchestratorApi::start(orchestrator_state, 50051).await {
tracing::error!("Orchestrator API failed: {}", e);
}
});
if config.claude_code.enabled {
tracing::info!(
"Claude Code sandbox mode available (model: {}, max_turns: {})",
config.claude_code.model,
config.claude_code.max_turns
);
}
Some(jm)
} else {
None
};
let orch = ironclaw::orchestrator::setup_orchestrator(
&config,
&components.llm,
components.db.as_ref(),
components.secrets_store.as_ref(),
)
.await;
let container_job_manager = orch.container_job_manager;
let job_event_tx = orch.job_event_tx;
let prompt_queue = orch.prompt_queue;
let docker_status = orch.docker_status;
// ── Channel setup ──────────────────────────────────────────────────
@@ -343,10 +264,10 @@ async fn async_main() -> anyhow::Result<()> {
if let Some(repl) = repl_channel {
channels.add(Box::new(repl)).await;
if cli.message.is_some() {
tracing::info!("Single message mode");
tracing::debug!("Single message mode");
} else {
channel_names.push("repl".to_string());
tracing::info!("REPL mode enabled");
tracing::debug!("REPL mode enabled");
}
}
@@ -355,7 +276,7 @@ async fn async_main() -> anyhow::Result<()> {
// Load WASM channels and register their webhook routes.
if config.channels.wasm_channels_enabled && config.channels.wasm_channels_dir.exists() {
let wasm_result = setup_wasm_channels(
let wasm_result = ironclaw::channels::wasm::setup_wasm_channels(
&config,
&components.secrets_store,
components.extension_manager.as_ref(),
@@ -388,7 +309,7 @@ async fn async_main() -> anyhow::Result<()> {
channel_names.push("signal".to_string());
channels.add(Box::new(signal_channel)).await;
let safe_url = SignalChannel::redact_url(&signal_config.http_url);
tracing::info!(
tracing::debug!(
url = %safe_url,
"Signal channel enabled"
);
@@ -414,7 +335,7 @@ async fn async_main() -> anyhow::Result<()> {
);
channel_names.push("http".to_string());
channels.add(Box::new(http_channel)).await;
tracing::info!(
tracing::debug!(
"HTTP channel enabled on {}:{}",
http_config.host,
http_config.port
@@ -455,7 +376,7 @@ async fn async_main() -> anyhow::Result<()> {
&components.dev_loaded_tool_names,
)
.await;
tracing::info!(
tracing::debug!(
bundled = hook_bootstrap.bundled_hooks,
plugin = hook_bootstrap.plugin_hooks,
workspace = hook_bootstrap.workspace_hooks,
@@ -548,7 +469,7 @@ async fn async_main() -> anyhow::Result<()> {
gw.auth_token()
));
tracing::info!("Web UI: http://{}:{}/", gw_config.host, gw_config.port);
tracing::debug!("Web UI: http://{}:{}/", gw_config.host, gw_config.port);
// Capture SSE sender and routine engine slot before moving gw into channels.
// IMPORTANT: This must come after all `with_*` calls since `rebuild_state`
@@ -633,7 +554,7 @@ async fn async_main() -> anyhow::Result<()> {
config.channels.wasm_channel_owner_ids.clone(),
)
.await;
tracing::info!("Channel runtime wired into extension manager for hot-activation");
tracing::debug!("Channel runtime wired into extension manager for hot-activation");
// Auto-activate channels that were active in a previous session.
let persisted = ext_mgr.load_persisted_active_channels().await;
@@ -641,7 +562,7 @@ async fn async_main() -> anyhow::Result<()> {
if !active_at_startup.contains(name) {
match ext_mgr.activate(name).await {
Ok(result) => {
tracing::info!(
tracing::debug!(
channel = %name,
message = %result.message,
"Auto-activated persisted channel"
@@ -759,485 +680,13 @@ async fn async_main() -> anyhow::Result<()> {
}
if let Some(tunnel) = active_tunnel {
tracing::info!("Stopping {} tunnel...", tunnel.name());
tracing::debug!("Stopping {} tunnel...", tunnel.name());
if let Err(e) = tunnel.stop().await {
tracing::warn!("Failed to stop tunnel cleanly: {}", e);
}
}
tracing::info!("Agent shutdown complete");
tracing::debug!("Agent shutdown complete");
Ok(())
}
// ── Helper functions ────────────────────────────────────────────────────
/// Initialize tracing for worker/bridge processes (info level).
fn init_worker_tracing() {
tracing_subscriber::fmt()
.with_env_filter(
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("ironclaw=info")),
)
.init();
}
/// Run the Memory CLI subcommand.
async fn run_memory_command(mem_cmd: &ironclaw::cli::MemoryCommand) -> anyhow::Result<()> {
let config = Config::from_env()
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
let session = create_session_manager(config.llm.session.clone()).await;
let embeddings = config
.embeddings
.create_provider(&config.llm.nearai.base_url, session);
let db: Arc<dyn ironclaw::db::Database> = ironclaw::db::connect_from_config(&config.database)
.await
.map_err(|e| anyhow::anyhow!("{}", e))?;
ironclaw::cli::run_memory_command_with_db(mem_cmd.clone(), db, embeddings).await
}
/// Run the Worker subcommand (inside Docker containers).
async fn run_worker(
job_id: uuid::Uuid,
orchestrator_url: &str,
max_iterations: u32,
) -> anyhow::Result<()> {
tracing::info!(
"Starting worker for job {} (orchestrator: {})",
job_id,
orchestrator_url
);
let config = ironclaw::worker::runtime::WorkerConfig {
job_id,
orchestrator_url: orchestrator_url.to_string(),
max_iterations,
timeout: std::time::Duration::from_secs(600),
};
let runtime = ironclaw::worker::WorkerRuntime::new(config)
.map_err(|e| anyhow::anyhow!("Worker init failed: {}", e))?;
runtime
.run()
.await
.map_err(|e| anyhow::anyhow!("Worker failed: {}", e))
}
/// Run the Claude Code bridge subcommand (inside Docker containers).
async fn run_claude_bridge(
job_id: uuid::Uuid,
orchestrator_url: &str,
max_turns: u32,
model: &str,
) -> anyhow::Result<()> {
tracing::info!(
"Starting Claude Code bridge for job {} (orchestrator: {}, model: {})",
job_id,
orchestrator_url,
model
);
let config = ironclaw::worker::claude_bridge::ClaudeBridgeConfig {
job_id,
orchestrator_url: orchestrator_url.to_string(),
max_turns,
model: model.to_string(),
timeout: std::time::Duration::from_secs(1800),
allowed_tools: ironclaw::config::ClaudeCodeConfig::from_env().allowed_tools,
};
let runtime = ironclaw::worker::ClaudeBridgeRuntime::new(config)
.map_err(|e| anyhow::anyhow!("Claude bridge init failed: {}", e))?;
runtime
.run()
.await
.map_err(|e| anyhow::anyhow!("Claude bridge failed: {}", e))
}
/// Start managed tunnel if configured and no static URL is already set.
async fn start_tunnel(
mut config: ironclaw::config::Config,
) -> (
ironclaw::config::Config,
Option<Box<dyn ironclaw::tunnel::Tunnel>>,
) {
if config.tunnel.public_url.is_some() {
tracing::info!(
"Static tunnel URL in use: {}",
config.tunnel.public_url.as_deref().unwrap_or("?")
);
return (config, None);
}
let Some(ref provider_config) = config.tunnel.provider else {
return (config, None);
};
let gateway_port = config
.channels
.gateway
.as_ref()
.map(|g| g.port)
.unwrap_or(3000);
let gateway_host = config
.channels
.gateway
.as_ref()
.map(|g| g.host.as_str())
.unwrap_or("127.0.0.1");
match ironclaw::tunnel::create_tunnel(provider_config) {
Ok(Some(tunnel)) => {
tracing::info!(
"Starting {} tunnel on {}:{}...",
tunnel.name(),
gateway_host,
gateway_port
);
match tunnel.start(gateway_host, gateway_port).await {
Ok(url) => {
tracing::info!("Tunnel started: {}", url);
config.tunnel.public_url = Some(url);
(config, Some(tunnel))
}
Err(e) => {
tracing::error!("Failed to start tunnel: {}", e);
(config, None)
}
}
}
Ok(None) => (config, None),
Err(e) => {
tracing::error!("Failed to create tunnel: {}", e);
(config, None)
}
}
}
/// Result of WASM channel setup.
struct WasmChannelSetup {
channels: Vec<(String, Box<dyn ironclaw::channels::Channel>)>,
channel_names: Vec<String>,
webhook_routes: Option<axum::Router>,
/// Runtime objects needed for hot-activation via ExtensionManager.
wasm_channel_runtime: Arc<WasmChannelRuntime>,
pairing_store: Arc<PairingStore>,
wasm_channel_router: Arc<WasmChannelRouter>,
}
/// Load WASM channels and register their webhook routes.
async fn setup_wasm_channels(
config: &ironclaw::config::Config,
secrets_store: &Option<Arc<dyn SecretsStore + Send + Sync>>,
extension_manager: Option<&Arc<ironclaw::extensions::ExtensionManager>>,
database: Option<&Arc<dyn ironclaw::db::Database>>,
) -> Option<WasmChannelSetup> {
let runtime = match WasmChannelRuntime::new(WasmChannelRuntimeConfig::default()) {
Ok(r) => Arc::new(r),
Err(e) => {
tracing::warn!("Failed to initialize WASM channel runtime: {}", e);
return None;
}
};
let pairing_store = Arc::new(PairingStore::new());
let settings_store: Option<Arc<dyn ironclaw::db::SettingsStore>> =
database.map(|db| Arc::clone(db) as Arc<dyn ironclaw::db::SettingsStore>);
let mut loader = WasmChannelLoader::new(
Arc::clone(&runtime),
Arc::clone(&pairing_store),
settings_store,
);
if let Some(secrets) = secrets_store {
loader = loader.with_secrets_store(Arc::clone(secrets));
}
let results = match loader
.load_from_dir(&config.channels.wasm_channels_dir)
.await
{
Ok(r) => r,
Err(e) => {
tracing::warn!("Failed to scan WASM channels directory: {}", e);
return None;
}
};
let wasm_router = Arc::new(WasmChannelRouter::new());
let mut channels: Vec<(String, Box<dyn ironclaw::channels::Channel>)> = Vec::new();
let mut channel_names: Vec<String> = Vec::new();
for loaded in results.loaded {
let channel_name = loaded.name().to_string();
channel_names.push(channel_name.clone());
tracing::info!("Loaded WASM channel: {}", channel_name);
let secret_name = loaded.webhook_secret_name();
let sig_key_secret_name = loaded.signature_key_secret_name();
let hmac_secret_name = loaded.hmac_secret_name();
let webhook_secret = if let Some(secrets) = secrets_store {
secrets
.get_decrypted("default", &secret_name)
.await
.ok()
.map(|s| s.expose().to_string())
} else {
None
};
let secret_header = loaded.webhook_secret_header().map(|s| s.to_string());
let webhook_path = format!("/webhook/{}", channel_name);
let endpoints = vec![RegisteredEndpoint {
channel_name: channel_name.clone(),
path: webhook_path,
methods: vec!["POST".to_string()],
require_secret: webhook_secret.is_some(),
}];
let channel_arc = Arc::new(loaded.channel);
{
let mut config_updates = std::collections::HashMap::new();
if let Some(ref tunnel_url) = config.tunnel.public_url {
config_updates.insert(
"tunnel_url".to_string(),
serde_json::Value::String(tunnel_url.clone()),
);
}
if let Some(ref secret) = webhook_secret {
config_updates.insert(
"webhook_secret".to_string(),
serde_json::Value::String(secret.clone()),
);
}
// Inject owner_id if configured for this channel.
if let Some(&owner_id) = config
.channels
.wasm_channel_owner_ids
.get(channel_name.as_str())
{
config_updates.insert("owner_id".to_string(), serde_json::json!(owner_id));
}
if !config_updates.is_empty() {
channel_arc.update_config(config_updates).await;
tracing::info!(
channel = %channel_name,
has_tunnel = config.tunnel.public_url.is_some(),
has_webhook_secret = webhook_secret.is_some(),
"Injected runtime config into channel"
);
}
}
tracing::info!(
channel = %channel_name,
has_webhook_secret = webhook_secret.is_some(),
secret_header = ?secret_header,
"Registering channel with router"
);
wasm_router
.register(
Arc::clone(&channel_arc),
endpoints,
webhook_secret.clone(),
secret_header,
)
.await;
// Register Ed25519 signature key if declared in capabilities
if let Some(ref sig_key_name) = sig_key_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(key_secret) = secrets.get_decrypted("default", sig_key_name).await
{
match wasm_router
.register_signature_key(&channel_name, key_secret.expose())
.await
{
Ok(()) => {
tracing::info!(channel = %channel_name, "Registered Ed25519 signature key")
}
Err(e) => {
tracing::error!(channel = %channel_name, error = %e, "Invalid signature key in secrets store")
}
}
}
// Register HMAC signing secret if declared in capabilities
if let Some(ref hmac_secret_name) = hmac_secret_name
&& let Some(secrets) = secrets_store
&& let Ok(secret) = secrets.get_decrypted("default", hmac_secret_name).await
{
wasm_router
.register_hmac_secret(&channel_name, secret.expose())
.await;
tracing::info!(channel = %channel_name, "Registered HMAC signing secret");
}
if let Some(secrets) = secrets_store {
match inject_channel_credentials(&channel_arc, secrets.as_ref(), &channel_name).await {
Ok(count) => {
if count > 0 {
tracing::info!(
channel = %channel_name,
credentials_injected = count,
"Channel credentials injected"
);
}
}
Err(e) => {
tracing::error!(
channel = %channel_name,
error = %e,
"Failed to inject channel credentials"
);
}
}
}
channels.push((channel_name, Box::new(SharedWasmChannel::new(channel_arc))));
}
for (path, err) in &results.errors {
tracing::warn!("Failed to load WASM channel {}: {}", path.display(), err);
}
// Always create webhook routes (even with no channels loaded) so that
// channels hot-added at runtime can receive webhooks without a restart.
let webhook_routes = {
Some(create_wasm_channel_router(
Arc::clone(&wasm_router),
extension_manager.map(Arc::clone),
))
};
Some(WasmChannelSetup {
channels,
channel_names,
webhook_routes,
wasm_channel_runtime: runtime,
pairing_store,
wasm_channel_router: wasm_router,
})
}
/// Check if onboarding is needed and return the reason.
#[cfg(any(feature = "postgres", feature = "libsql"))]
fn check_onboard_needed() -> Option<&'static str> {
let has_db = std::env::var("DATABASE_URL").is_ok()
|| std::env::var("LIBSQL_PATH").is_ok()
|| ironclaw::config::default_libsql_path().exists();
if !has_db {
return Some("Database not configured");
}
if std::env::var("ONBOARD_COMPLETED")
.map(|v| v == "true")
.unwrap_or(false)
{
return None;
}
if std::env::var("NEARAI_API_KEY").is_err() {
let session_path = ironclaw::config::default_session_path();
if !session_path.exists() {
return Some("First run");
}
}
None
}
/// Inject credentials for a channel based on naming convention.
///
/// Looks for secrets matching the pattern `{channel_name}_*` and injects them
/// as credential placeholders (e.g., `telegram_bot_token` -> `{TELEGRAM_BOT_TOKEN}`).
///
/// Falls back to environment variables with the uppercase name if not found
/// in the secrets store (e.g., `TELEGRAM_BOT_TOKEN`).
async fn inject_channel_credentials(
channel: &Arc<ironclaw::channels::wasm::WasmChannel>,
secrets: &dyn SecretsStore,
channel_name: &str,
) -> anyhow::Result<usize> {
let all_secrets = secrets
.list("default")
.await
.map_err(|e| anyhow::anyhow!("Failed to list secrets: {}", e))?;
let prefix = format!("{}_", channel_name);
let mut count = 0;
let mut injected_placeholders = std::collections::HashSet::new();
for secret_meta in all_secrets {
if !secret_meta.name.starts_with(&prefix) {
continue;
}
let decrypted = match secrets.get_decrypted("default", &secret_meta.name).await {
Ok(d) => d,
Err(e) => {
tracing::warn!(
secret = %secret_meta.name,
error = %e,
"Failed to decrypt secret for channel credential injection"
);
continue;
}
};
let placeholder = secret_meta.name.to_uppercase();
tracing::debug!(
channel = %channel_name,
secret = %secret_meta.name,
placeholder = %placeholder,
"Injecting credential"
);
channel
.set_credential(&placeholder, decrypted.expose().to_string())
.await;
injected_placeholders.insert(placeholder);
count += 1;
}
// Fall back to environment variables for required secrets not found in the store.
// This allows channels to work when configured via env vars (e.g., TELEGRAM_BOT_TOKEN)
// without requiring the setup wizard to have run.
let caps = channel.capabilities();
if let Some(ref http_cap) = caps.tool_capabilities.http {
for cred_mapping in http_cap.credentials.values() {
let placeholder = cred_mapping.secret_name.to_uppercase();
if injected_placeholders.contains(&placeholder) {
continue;
}
if let Ok(env_value) = std::env::var(&placeholder)
&& !env_value.is_empty()
{
tracing::debug!(
channel = %channel_name,
placeholder = %placeholder,
"Injecting credential from environment variable"
);
channel.set_credential(&placeholder, env_value).await;
count += 1;
}
}
}
Ok(count)
}
+112
View File
@@ -39,3 +39,115 @@ pub use job_manager::{
CompletionResult, ContainerHandle, ContainerJobConfig, ContainerJobManager, JobMode,
};
pub use reaper::{ReaperConfig, SandboxReaper};
use std::collections::{HashMap, VecDeque};
use std::sync::Arc;
use tokio::sync::{Mutex, broadcast};
use uuid::Uuid;
use crate::channels::web::types::SseEvent;
use crate::db::Database;
use crate::llm::LlmProvider;
use crate::secrets::SecretsStore;
/// Result of orchestrator setup, containing all handles needed by the agent.
pub struct OrchestratorSetup {
pub container_job_manager: Option<Arc<ContainerJobManager>>,
pub job_event_tx: Option<broadcast::Sender<(Uuid, SseEvent)>>,
pub prompt_queue: Arc<Mutex<HashMap<Uuid, VecDeque<api::PendingPrompt>>>>,
pub docker_status: crate::sandbox::DockerStatus,
}
/// Detect Docker availability, create the container job manager, and start
/// the orchestrator internal API in the background.
pub async fn setup_orchestrator(
config: &crate::config::Config,
llm: &Arc<dyn LlmProvider>,
db: Option<&Arc<dyn Database>>,
secrets_store: Option<&Arc<dyn SecretsStore + Send + Sync>>,
) -> OrchestratorSetup {
let prompt_queue = Arc::new(Mutex::new(
HashMap::<Uuid, VecDeque<api::PendingPrompt>>::new(),
));
let docker_status = if config.sandbox.enabled {
let detection = crate::sandbox::check_docker().await;
match detection.status {
crate::sandbox::DockerStatus::Available => {
tracing::info!("Docker is available");
}
crate::sandbox::DockerStatus::NotInstalled => {
tracing::warn!(
"Docker is not installed -- sandbox disabled for this session. {}",
detection.platform.install_hint()
);
}
crate::sandbox::DockerStatus::NotRunning => {
tracing::warn!(
"Docker is installed but not running -- sandbox disabled for this session. {}",
detection.platform.start_hint()
);
}
crate::sandbox::DockerStatus::Disabled => {}
}
detection.status
} else {
crate::sandbox::DockerStatus::Disabled
};
let (job_event_tx, container_job_manager) = if config.sandbox.enabled && docker_status.is_ok() {
let (tx, _) = broadcast::channel(256);
let job_event_tx = Some(tx);
let token_store = TokenStore::new();
let job_config = ContainerJobConfig {
image: config.sandbox.image.clone(),
memory_limit_mb: config.sandbox.memory_limit_mb,
cpu_shares: config.sandbox.cpu_shares,
orchestrator_port: 50051,
claude_code_api_key: std::env::var("ANTHROPIC_API_KEY").ok(),
claude_code_oauth_token: crate::config::ClaudeCodeConfig::extract_oauth_token(),
claude_code_model: config.claude_code.model.clone(),
claude_code_max_turns: config.claude_code.max_turns,
claude_code_memory_limit_mb: config.claude_code.memory_limit_mb,
claude_code_allowed_tools: config.claude_code.allowed_tools.clone(),
};
let jm = Arc::new(ContainerJobManager::new(job_config, token_store.clone()));
let orchestrator_state = api::OrchestratorState {
llm: Arc::clone(llm),
job_manager: Arc::clone(&jm),
token_store,
job_event_tx: job_event_tx.clone(),
prompt_queue: Arc::clone(&prompt_queue),
store: db.cloned(),
secrets_store: secrets_store.cloned(),
user_id: "default".to_string(),
};
tokio::spawn(async move {
if let Err(e) = OrchestratorApi::start(orchestrator_state, 50051).await {
tracing::error!("Orchestrator API failed: {}", e);
}
});
if config.claude_code.enabled {
tracing::info!(
"Claude Code sandbox mode available (model: {}, max_turns: {})",
config.claude_code.model,
config.claude_code.max_turns
);
}
(job_event_tx, Some(jm))
} else {
(None, None)
};
OrchestratorSetup {
container_job_manager,
job_event_tx,
prompt_queue,
docker_status,
}
}
+1 -1
View File
@@ -185,7 +185,7 @@ impl SandboxManager {
self.initialized
.store(false, std::sync::atomic::Ordering::SeqCst);
tracing::info!("Sandbox shut down");
tracing::debug!("Sandbox shut down");
}
/// Execute a command in the sandbox.
+1 -1
View File
@@ -154,7 +154,7 @@ impl HttpProxy {
}
}
_ = &mut shutdown_rx => {
tracing::info!("Sandbox proxy shutting down");
tracing::debug!("Sandbox proxy shutting down");
break;
}
}
+34
View File
@@ -75,3 +75,37 @@ pub use types::{
};
pub use store::in_memory::InMemorySecretsStore;
/// Create a secrets store from a master key and database handles.
///
/// Returns `None` if no matching backend handle is available (e.g. when
/// running without a database). This is a normal condition in no-db mode,
/// not an error — callers should treat `None` as "secrets unavailable".
pub fn create_secrets_store(
crypto: std::sync::Arc<SecretsCrypto>,
handles: &crate::db::DatabaseHandles,
) -> Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> {
let store: Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> = None;
#[cfg(feature = "libsql")]
let store = store.or_else(|| {
handles.libsql_db.as_ref().map(|db| {
std::sync::Arc::new(LibSqlSecretsStore::new(
std::sync::Arc::clone(db),
std::sync::Arc::clone(&crypto),
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
})
});
#[cfg(feature = "postgres")]
let store = store.or_else(|| {
handles.pg_pool.as_ref().map(|pool| {
std::sync::Arc::new(PostgresSecretsStore::new(
pool.clone(),
std::sync::Arc::clone(&crypto),
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
})
});
store
}
+5
View File
@@ -386,6 +386,10 @@ pub struct AgentSettings {
/// Default timezone for new sessions (IANA name, e.g. "America/New_York").
#[serde(default = "default_timezone")]
pub default_timezone: String,
/// Maximum tokens per job (0 = unlimited).
#[serde(default)]
pub max_tokens_per_job: u64,
}
fn default_agent_name() -> String {
@@ -442,6 +446,7 @@ impl Default for AgentSettings {
max_tool_iterations: default_max_tool_iterations(),
auto_approve_tools: false,
default_timezone: default_timezone(),
max_tokens_per_job: 0,
}
}
}
+40 -3
View File
@@ -10,7 +10,7 @@ file first, then adjust the code to match.
## Entry Points
```
ironclaw onboard [--skip-auth] [--channels-only]
ironclaw onboard [--skip-auth] [--channels-only] [--provider-only] [--quick]
```
Explicit invocation. Loads `.env` files, runs the wizard, exits.
@@ -26,6 +26,8 @@ the wizard). Otherwise triggers when no database is configured:
- `LIBSQL_PATH` env var is set
- `~/.ironclaw/ironclaw.db` exists on disk
Auto-triggered onboarding uses **quick mode** by default.
The `--no-onboard` CLI flag suppresses auto-detection.
---
@@ -50,7 +52,41 @@ The `--no-onboard` CLI flag suppresses auto-detection.
---
## The 8-Step Wizard
## Quick Mode
Quick mode (`--quick` flag, or auto-triggered on first run) provides a
near-instant onboarding experience by auto-defaulting everything except
the LLM provider and model selection.
```
auto_setup_database() → libsql at ~/.ironclaw/ironclaw.db (zero prompts)
auto_setup_security() → keychain or env var (zero prompts)
Step 1/2: Inference Provider ← only interactive step
Step 2/2: Model Selection ← only interactive step
save_and_summarize() → includes tip to run `ironclaw onboard`
```
**`auto_setup_database()`:** Uses existing env vars if set (`DATABASE_URL`
for postgres, `LIBSQL_PATH` for libsql) without prompting. Otherwise
defaults to libsql at `~/.ironclaw/ironclaw.db`, creates the database,
and runs migrations silently. Falls back to interactive mode only when
just the postgres feature is compiled and no `DATABASE_URL` is set.
**`auto_setup_security()`:** Checks for existing `SECRETS_MASTER_KEY`
env var or OS keychain key. If neither exists, generates a new key and
stores it in the keychain (macOS) or env var (Linux/other). Zero prompts
except unavoidable macOS keychain dialogs.
**`.env` preservation (fix for #751):** `write_bootstrap_env()` now uses
`upsert_bootstrap_vars()` instead of `save_bootstrap_env()`, preserving
user-added variables like `HTTP_HOST` across re-onboarding.
The full 9-step wizard remains available via `ironclaw onboard`.
---
## The 9-Step Wizard
### Overview
@@ -62,7 +98,8 @@ Step 4: Model Selection
Step 5: Embeddings
Step 6: Channel Configuration
Step 7: Extensions (tools)
Step 8: Background Tasks (heartbeat)
Step 8: Docker Sandbox
Step 9: Background Tasks (heartbeat)
save_and_summarize()
```
+32
View File
@@ -31,3 +31,35 @@ pub use prompts::{
};
#[cfg(any(feature = "postgres", feature = "libsql"))]
pub use wizard::{SetupConfig, SetupWizard};
/// Check if onboarding is needed and return the reason.
///
/// Reads environment variables (`DATABASE_URL`, `LIBSQL_PATH`,
/// `ONBOARD_COMPLETED`, `NEARAI_API_KEY`) and checks for the default
/// session file on disk. Not safe to call concurrently with `env::set_var`.
#[cfg(any(feature = "postgres", feature = "libsql"))]
pub fn check_onboard_needed() -> Option<&'static str> {
let has_db = std::env::var("DATABASE_URL").is_ok()
|| std::env::var("LIBSQL_PATH").is_ok()
|| crate::config::default_libsql_path().exists();
if !has_db {
return Some("Database not configured");
}
if std::env::var("ONBOARD_COMPLETED")
.map(|v| v == "true")
.unwrap_or(false)
{
return None;
}
if std::env::var("NEARAI_API_KEY").is_err() {
let session_path = crate::config::default_session_path();
if !session_path.exists() {
return Some("First run");
}
}
None
}
+306 -37
View File
@@ -76,6 +76,8 @@ pub struct SetupConfig {
pub channels_only: bool,
/// Only reconfigure LLM provider and model selection.
pub provider_only: bool,
/// Quick setup: auto-defaults everything except LLM provider and model.
pub quick: bool,
}
/// Interactive setup wizard for IronClaw.
@@ -154,6 +156,26 @@ impl SetupWizard {
print_step(1, 2, "Inference Provider");
self.step_inference_provider().await?;
self.persist_after_step().await;
print_step(2, 2, "Model Selection");
self.step_model_selection().await?;
self.persist_after_step().await;
} else if self.config.quick {
// Quick mode: auto-default database + security, only ask for
// LLM provider + model. Designed for first-run experience.
self.auto_setup_database().await?;
// Load existing settings from DB (if any prior partial run)
let step1_settings = self.settings.clone();
self.try_load_existing_settings().await;
self.settings.merge_from(&step1_settings);
self.auto_setup_security().await?;
self.persist_after_step().await;
print_step(1, 2, "Inference Provider");
self.step_inference_provider().await?;
self.persist_after_step().await;
print_step(2, 2, "Model Selection");
self.step_model_selection().await?;
self.persist_after_step().await;
@@ -659,7 +681,10 @@ impl SetupWizard {
use refinery::embed_migrations;
embed_migrations!("migrations");
print_info("Running migrations...");
if !self.config.quick {
print_info("Running migrations...");
}
tracing::debug!("Running PostgreSQL migrations...");
let mut client = pool
.get()
@@ -671,7 +696,10 @@ impl SetupWizard {
.await
.map_err(|e| SetupError::Database(format!("Migration failed: {}", e)))?;
print_success("Migrations applied");
if !self.config.quick {
print_success("Migrations applied");
}
tracing::debug!("PostgreSQL migrations applied");
}
Ok(())
}
@@ -682,14 +710,20 @@ impl SetupWizard {
if let Some(ref backend) = self.db_backend {
use crate::db::Database;
print_info("Running migrations...");
if !self.config.quick {
print_info("Running migrations...");
}
tracing::debug!("Running libSQL migrations...");
backend
.run_migrations()
.await
.map_err(|e| SetupError::Database(format!("Migration failed: {}", e)))?;
print_success("Migrations applied");
if !self.config.quick {
print_success("Migrations applied");
}
tracing::debug!("libSQL migrations applied");
}
Ok(())
}
@@ -804,6 +838,140 @@ impl SetupWizard {
Ok(())
}
/// Auto-setup database with zero prompts (quick mode).
///
/// Uses existing env vars if present, otherwise defaults to libsql at the
/// standard path. Falls back to the interactive `step_database()` only when
/// just the postgres feature is compiled (can't auto-default postgres).
async fn auto_setup_database(&mut self) -> Result<(), SetupError> {
// If DATABASE_URL or LIBSQL_PATH already set, respect existing config
#[cfg(feature = "postgres")]
let env_backend = std::env::var("DATABASE_BACKEND").ok();
#[cfg(feature = "postgres")]
if let Some(ref backend) = env_backend
&& (backend == "postgres" || backend == "postgresql")
{
if let Ok(url) = std::env::var("DATABASE_URL") {
print_info("Using existing PostgreSQL configuration");
self.settings.database_backend = Some("postgres".to_string());
self.settings.database_url = Some(url);
return Ok(());
}
// Postgres configured but no URL — fall through to interactive
return self.step_database().await;
}
#[cfg(feature = "postgres")]
if let Ok(url) = std::env::var("DATABASE_URL") {
print_info("Using existing PostgreSQL configuration");
self.settings.database_backend = Some("postgres".to_string());
self.settings.database_url = Some(url);
return Ok(());
}
// Auto-default to libsql if the feature is compiled
#[cfg(feature = "libsql")]
{
self.settings.database_backend = Some("libsql".to_string());
let existing_path = std::env::var("LIBSQL_PATH")
.ok()
.or_else(|| self.settings.libsql_path.clone());
let db_path = existing_path.unwrap_or_else(|| {
crate::config::default_libsql_path()
.to_string_lossy()
.to_string()
});
let turso_url = std::env::var("LIBSQL_URL").ok();
let turso_token = std::env::var("LIBSQL_AUTH_TOKEN").ok();
self.test_database_connection_libsql(
&db_path,
turso_url.as_deref(),
turso_token.as_deref(),
)
.await?;
self.run_migrations_libsql().await?;
self.settings.libsql_path = Some(db_path.clone());
if let Some(url) = turso_url {
self.settings.libsql_url = Some(url);
}
print_success(&format!("Using embedded database at {}", db_path));
return Ok(());
}
// Only postgres feature compiled — can't auto-default, use interactive
#[allow(unreachable_code)]
{
self.step_database().await
}
}
/// Auto-setup security with zero prompts (quick mode).
///
/// Silently configures the master key: uses existing env var or keychain
/// key if available, otherwise generates and stores one automatically
/// (keychain on macOS, env var fallback).
async fn auto_setup_security(&mut self) -> Result<(), SetupError> {
// Check env var first
if std::env::var("SECRETS_MASTER_KEY").is_ok() {
self.settings.secrets_master_key_source = KeySource::Env;
print_success("Security configured (env var)");
return Ok(());
}
// Try existing keychain key (no prompts — get_master_key may show
// OS dialogs on macOS, but that's unavoidable for keychain access)
if let Ok(keychain_key_bytes) = crate::secrets::keychain::get_master_key().await {
let key_hex: String = keychain_key_bytes
.iter()
.map(|b| format!("{:02x}", b))
.collect();
self.secrets_crypto = Some(Arc::new(
SecretsCrypto::new(SecretString::from(key_hex))
.map_err(|e| SetupError::Config(e.to_string()))?,
));
self.settings.secrets_master_key_source = KeySource::Keychain;
print_success("Security configured (keychain)");
return Ok(());
}
// No existing key — generate one
// Try keychain first (preferred on macOS)
let key = crate::secrets::keychain::generate_master_key();
if crate::secrets::keychain::store_master_key(&key)
.await
.is_ok()
{
let key_hex: String = key.iter().map(|b| format!("{:02x}", b)).collect();
self.secrets_crypto = Some(Arc::new(
SecretsCrypto::new(SecretString::from(key_hex))
.map_err(|e| SetupError::Config(e.to_string()))?,
));
self.settings.secrets_master_key_source = KeySource::Keychain;
print_success("Master key stored in OS keychain");
return Ok(());
}
// Keychain unavailable — fall back to env var mode
let key_hex = crate::secrets::keychain::generate_master_key_hex();
self.secrets_crypto = Some(Arc::new(
SecretsCrypto::new(SecretString::from(key_hex.clone()))
.map_err(|e| SetupError::Config(e.to_string()))?,
));
crate::config::inject_single_var("SECRETS_MASTER_KEY", &key_hex);
self.settings.secrets_master_key_hex = Some(key_hex);
self.settings.secrets_master_key_source = KeySource::Env;
print_success("Master key stored in ~/.ironclaw/.env");
Ok(())
}
/// Step 3: Inference provider selection.
///
/// Uses the provider registry to dynamically build the selection menu.
@@ -1573,46 +1741,18 @@ impl SetupWizard {
}
/// Fetch available models from the NEAR AI API.
///
/// Uses [`build_nearai_model_fetch_config`] to construct the provider config,
/// which reads `NEARAI_API_KEY` from the environment when present.
async fn fetch_nearai_models(&self) -> Vec<String> {
let session = match self.session_manager {
Some(ref s) => Arc::clone(s),
None => return vec![],
};
use crate::config::LlmConfig;
use crate::llm::create_llm_provider;
let base_url = std::env::var("NEARAI_BASE_URL")
.unwrap_or_else(|_| "https://private.near.ai".to_string());
let auth_base_url = std::env::var("NEARAI_AUTH_URL")
.unwrap_or_else(|_| "https://private.near.ai".to_string());
let config = LlmConfig {
backend: "nearai".to_string(),
session: crate::llm::session::SessionConfig {
auth_base_url,
session_path: crate::config::llm::default_session_path(),
},
nearai: crate::config::NearAiConfig {
model: "dummy".to_string(),
cheap_model: None,
base_url,
api_key: None,
fallback_model: None,
max_retries: 3,
circuit_breaker_threshold: None,
circuit_breaker_recovery_secs: 30,
response_cache_enabled: false,
response_cache_ttl_secs: 3600,
response_cache_max_entries: 1000,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
smart_routing_cascade: true,
},
provider: None,
bedrock: None,
request_timeout_secs: 120,
};
let config = build_nearai_model_fetch_config();
match create_llm_provider(&config, session).await {
Ok(provider) => match provider.list_models().await {
@@ -2534,7 +2674,7 @@ impl SetupWizard {
.iter()
.map(|(k, v)| (k.as_str(), v.as_str()))
.collect();
crate::bootstrap::save_bootstrap_env(&pairs).map_err(|e| {
crate::bootstrap::upsert_bootstrap_vars(&pairs).map_err(|e| {
SetupError::Io(std::io::Error::other(format!(
"Failed to save bootstrap env to .env: {}",
e
@@ -2806,6 +2946,13 @@ impl SetupWizard {
println!(" ironclaw onboard");
println!();
if self.config.quick {
print_info(
"Tip: Run `ironclaw onboard` to configure channels, extensions, embeddings, and more.",
);
println!();
}
Ok(())
}
}
@@ -3240,6 +3387,58 @@ async fn discover_wasm_channels(dir: &std::path::Path) -> Vec<(String, ChannelCa
/// Mask an API key for display: show first 6 + last 4 chars.
///
/// Uses char-based indexing to avoid panicking on multi-byte UTF-8.
/// Build the `LlmConfig` used by `fetch_nearai_models` to list available models.
///
/// Reads `NEARAI_API_KEY` from the environment so that users who authenticated
/// via Cloud API key (option 4) don't get re-prompted during model selection.
fn build_nearai_model_fetch_config() -> crate::config::LlmConfig {
// If the user authenticated via API key (option 4), the key is stored
// as an env var. Pass it through so `resolve_bearer_token()` doesn't
// re-trigger the interactive auth prompt.
let api_key = std::env::var("NEARAI_API_KEY")
.ok()
.filter(|k| !k.is_empty())
.map(secrecy::SecretString::from);
// Match the same base_url logic as LlmConfig::resolve(): use cloud-api
// when an API key is present, private.near.ai for session-token auth.
let default_base = if api_key.is_some() {
"https://cloud-api.near.ai"
} else {
"https://private.near.ai"
};
let base_url = std::env::var("NEARAI_BASE_URL").unwrap_or_else(|_| default_base.to_string());
let auth_base_url =
std::env::var("NEARAI_AUTH_URL").unwrap_or_else(|_| "https://private.near.ai".to_string());
crate::config::LlmConfig {
backend: "nearai".to_string(),
session: crate::llm::session::SessionConfig {
auth_base_url,
session_path: crate::config::llm::default_session_path(),
},
nearai: crate::config::NearAiConfig {
model: "dummy".to_string(),
cheap_model: None,
base_url,
api_key,
fallback_model: None,
max_retries: 3,
circuit_breaker_threshold: None,
circuit_breaker_recovery_secs: 30,
response_cache_enabled: false,
response_cache_ttl_secs: 3600,
response_cache_max_entries: 1000,
failover_cooldown_secs: 300,
failover_cooldown_threshold: 3,
smart_routing_cascade: true,
},
provider: None,
bedrock: None,
request_timeout_secs: 120,
}
}
fn mask_api_key(key: &str) -> String {
let chars: Vec<char> = key.chars().collect();
if chars.len() < 12 {
@@ -3448,6 +3647,7 @@ mod tests {
use tempfile::tempdir;
use super::*;
use crate::config::helpers::ENV_MUTEX;
#[test]
fn test_wizard_creation() {
@@ -3462,6 +3662,7 @@ mod tests {
skip_auth: true,
channels_only: false,
provider_only: false,
quick: false,
};
let wizard = SetupWizard::with_config(config);
assert!(wizard.config.skip_auth);
@@ -3640,6 +3841,14 @@ mod tests {
}
impl EnvGuard {
fn set(key: &'static str, value: &str) -> Self {
let original = std::env::var(key).ok();
unsafe {
std::env::set_var(key, value);
}
Self { key, original }
}
fn clear(key: &'static str) -> Self {
let original = std::env::var(key).ok();
unsafe {
@@ -3826,4 +4035,64 @@ mod tests {
};
assert!(settings.secrets_master_key_hex.is_some());
}
/// Regression test for #799: `fetch_nearai_models` hardcoded `api_key: None`,
/// causing the auth prompt to re-appear during model selection when the user
/// had authenticated via NEAR AI Cloud API key (option 4).
#[test]
fn test_build_nearai_model_fetch_config_picks_up_api_key_env() {
use secrecy::ExposeSecret;
let _lock = ENV_MUTEX.lock().unwrap();
let _guard = EnvGuard::set("NEARAI_API_KEY", "test-cloud-api-key-12345");
let _guard2 = EnvGuard::clear("NEARAI_BASE_URL");
let config = build_nearai_model_fetch_config();
assert!(
config.nearai.api_key.is_some(),
"config should include NEARAI_API_KEY from env"
);
assert_eq!(
config.nearai.api_key.as_ref().unwrap().expose_secret(),
"test-cloud-api-key-12345"
);
// With API key, base_url must point to cloud-api (not private.near.ai)
assert_eq!(
config.nearai.base_url, "https://cloud-api.near.ai",
"API key auth must use cloud-api base URL for model fetching"
);
}
/// Regression test for #799: when NEARAI_API_KEY is absent or empty,
/// the config should have `api_key: None` (session token path).
#[test]
fn test_build_nearai_model_fetch_config_none_when_no_api_key() {
let _lock = ENV_MUTEX.lock().unwrap();
let _guard = EnvGuard::clear("NEARAI_API_KEY");
let _guard2 = EnvGuard::clear("NEARAI_BASE_URL");
let config = build_nearai_model_fetch_config();
assert!(
config.nearai.api_key.is_none(),
"config should have no api_key when env var is absent"
);
// Without API key, base_url must point to private.near.ai (session token)
assert_eq!(
config.nearai.base_url, "https://private.near.ai",
"session-token auth must use private.near.ai base URL"
);
}
/// Regression test for #799: empty NEARAI_API_KEY should be treated as absent.
#[test]
fn test_build_nearai_model_fetch_config_none_when_empty_api_key() {
let _lock = ENV_MUTEX.lock().unwrap();
let _guard = EnvGuard::set("NEARAI_API_KEY", "");
let config = build_nearai_model_fetch_config();
assert!(
config.nearai.api_key.is_none(),
"config should have no api_key when env var is empty"
);
}
}
+33 -4
View File
@@ -451,8 +451,8 @@ impl Tool for ToolRemoveTool {
}
fn description(&self) -> &str {
"Remove an installed extension (channel, tool, or MCP server). \
Unregisters tools and deletes configuration."
"Permanently remove an installed extension (channel, tool, or MCP server) from disk. \
This action cannot be undone the WASM binary and configuration files will be deleted."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -492,7 +492,7 @@ impl Tool for ToolRemoveTool {
}
fn requires_approval(&self, _params: &serde_json::Value) -> ApprovalRequirement {
ApprovalRequirement::UnlessAutoApproved
ApprovalRequirement::Always
}
}
@@ -701,10 +701,38 @@ mod tests {
assert_eq!(tool.name(), "tool_remove");
assert_eq!(
tool.requires_approval(&serde_json::json!({})),
ApprovalRequirement::UnlessAutoApproved
ApprovalRequirement::Always
);
}
#[test]
fn tool_remove_always_requires_approval_regardless_of_params() {
use crate::tools::tool::ApprovalRequirement;
let tool = ToolRemoveTool {
manager: test_manager_stub(),
};
let test_cases = vec![
("no params", serde_json::json!({})),
("empty name", serde_json::json!({"name": ""})),
("slack", serde_json::json!({"name": "slack"})),
("github-cli", serde_json::json!({"name": "github-cli"})),
(
"with extra fields",
serde_json::json!({"name": "tool", "extra": "field"}),
),
];
for (case_name, params) in test_cases {
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::Always,
"tool_remove must always require approval for case: {}",
case_name
);
}
}
#[test]
fn test_tool_upgrade_schema() {
use crate::tools::tool::ApprovalRequirement;
@@ -749,6 +777,7 @@ mod tests {
Arc::new(ExtensionManager::new(
Arc::new(McpSessionManager::new()),
Arc::new(crate::tools::mcp::process::McpProcessManager::new()),
Arc::new(InMemorySecretsStore::new(crypto)),
Arc::new(ToolRegistry::new()),
None,
+33 -3
View File
@@ -709,7 +709,8 @@ impl Tool for SkillRemoveTool {
}
fn description(&self) -> &str {
"Remove an installed skill by name. Only user-installed skills can be removed."
"Permanently remove an installed skill from disk. This action cannot be undone — \
the skill files will be deleted."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -770,7 +771,7 @@ impl Tool for SkillRemoveTool {
}
fn requires_approval(&self, _params: &serde_json::Value) -> ApprovalRequirement {
ApprovalRequirement::UnlessAutoApproved
ApprovalRequirement::Always
}
}
@@ -837,12 +838,41 @@ mod tests {
assert_eq!(tool.name(), "skill_remove");
assert_eq!(
tool.requires_approval(&serde_json::json!({})),
ApprovalRequirement::UnlessAutoApproved
ApprovalRequirement::Always
);
let schema = tool.parameters_schema();
assert!(schema["properties"].get("name").is_some());
}
#[test]
fn skill_remove_always_requires_approval_regardless_of_params() {
use crate::tools::tool::ApprovalRequirement;
let tool = SkillRemoveTool::new(test_registry());
let test_cases = vec![
("no params", serde_json::json!({})),
("empty name", serde_json::json!({"name": ""})),
(
"deployment skill",
serde_json::json!({"name": "deployment"}),
),
("custom skill", serde_json::json!({"name": "custom-skill"})),
(
"with extra fields",
serde_json::json!({"name": "skill", "extra": "field"}),
),
];
for (case_name, params) in test_cases {
assert_eq!(
tool.requires_approval(&params),
ApprovalRequirement::Always,
"skill_remove must always require approval for case: {}",
case_name
);
}
}
#[test]
fn test_validate_fetch_url_allows_https() {
assert!(super::validate_fetch_url("https://clawhub.ai/api/v1/download?slug=foo").is_ok());
+98
View File
@@ -0,0 +1,98 @@
//! Factory for creating MCP clients from server configuration.
//!
//! Encapsulates the transport dispatch logic (stdio, Unix socket, HTTP)
//! so that callers don't need to match on `EffectiveTransport` themselves.
use std::sync::Arc;
use crate::secrets::SecretsStore;
use crate::tools::mcp::config::{EffectiveTransport, McpServerConfig};
use crate::tools::mcp::{McpClient, McpProcessManager, McpSessionManager, McpTransport};
/// Error returned when MCP client creation fails.
#[derive(Debug, thiserror::Error)]
pub enum McpFactoryError {
#[error("Failed to spawn stdio MCP server '{name}': {reason}")]
StdioSpawn { name: String, reason: String },
#[error("Failed to connect to Unix MCP server '{name}': {reason}")]
UnixConnect { name: String, reason: String },
#[error("Unix socket transport is not supported on this platform (server '{name}')")]
UnixNotSupported { name: String },
}
/// Create an `McpClient` from a server configuration, dispatching on the
/// effective transport type.
pub async fn create_client_from_config(
server: McpServerConfig,
session_manager: &Arc<McpSessionManager>,
process_manager: &Arc<McpProcessManager>,
secrets: Option<Arc<dyn SecretsStore + Send + Sync>>,
user_id: &str,
) -> Result<McpClient, McpFactoryError> {
let server_name = server.name.clone();
match server.effective_transport() {
EffectiveTransport::Stdio { command, args, env } => {
let transport = process_manager
.spawn_stdio(&server_name, command, args.to_vec(), env.clone())
.await
.map_err(|e| McpFactoryError::StdioSpawn {
name: server_name.clone(),
reason: e.to_string(),
})?;
Ok(McpClient::new_with_transport(
&server_name,
transport as Arc<dyn McpTransport>,
None,
secrets,
user_id,
Some(server),
))
}
#[cfg(unix)]
EffectiveTransport::Unix { socket_path } => {
let transport = crate::tools::mcp::unix_transport::UnixMcpTransport::connect(
&server_name,
socket_path,
)
.await
.map_err(|e| McpFactoryError::UnixConnect {
name: server_name.clone(),
reason: e.to_string(),
})?;
Ok(McpClient::new_with_transport(
&server_name,
Arc::new(transport) as Arc<dyn McpTransport>,
None,
secrets,
user_id,
Some(server),
))
}
#[cfg(not(unix))]
EffectiveTransport::Unix { .. } => {
Err(McpFactoryError::UnixNotSupported { name: server_name })
}
EffectiveTransport::Http => {
if let Some(ref secrets) = secrets {
let has_tokens =
crate::tools::mcp::is_authenticated(&server, secrets, user_id).await;
if has_tokens || server.requires_auth() {
Ok(McpClient::new_authenticated(
server,
Arc::clone(session_manager),
Arc::clone(secrets),
user_id,
))
} else {
Ok(McpClient::new_with_config(server))
}
} else {
Ok(McpClient::new_with_config(server))
}
}
}
}
+2
View File
@@ -31,6 +31,7 @@
pub mod auth;
mod client;
pub mod config;
pub mod factory;
pub(crate) mod http_transport;
pub(crate) mod process;
mod protocol;
@@ -43,6 +44,7 @@ pub(crate) mod unix_transport;
pub use auth::{is_authenticated, refresh_access_token};
pub use client::McpClient;
pub use config::{McpServerConfig, McpServersFile, OAuthConfig};
pub use factory::{McpFactoryError, create_client_from_config};
pub use process::McpProcessManager;
pub use protocol::{InitializeResult, McpRequest, McpResponse, McpTool};
pub use session::McpSessionManager;
+14 -14
View File
@@ -241,7 +241,7 @@ impl ToolRegistry {
}
self.register_sync(Arc::new(http));
tracing::info!("Registered {} built-in tools", self.count());
tracing::debug!("Registered {} built-in tools", self.count());
}
/// Register only orchestrator-domain tools (safe for the main process).
@@ -289,7 +289,7 @@ impl ToolRegistry {
self.register_sync(Arc::new(ListDirTool::new()));
self.register_sync(Arc::new(ApplyPatchTool::new()));
tracing::info!("Registered 5 development tools");
tracing::debug!("Registered 5 development tools");
}
/// Register memory tools with a workspace.
@@ -302,7 +302,7 @@ impl ToolRegistry {
self.register_sync(Arc::new(MemoryReadTool::new(Arc::clone(&workspace))));
self.register_sync(Arc::new(MemoryTreeTool::new(workspace)));
tracing::info!("Registered 4 memory tools");
tracing::debug!("Registered 4 memory tools");
}
/// Register job management tools.
@@ -364,7 +364,7 @@ impl ToolRegistry {
job_tool_count += 1;
}
tracing::info!("Registered {} job management tools", job_tool_count);
tracing::debug!("Registered {} job management tools", job_tool_count);
}
/// Register secret management tools (list, delete).
@@ -378,7 +378,7 @@ impl ToolRegistry {
use crate::tools::builtin::{SecretDeleteTool, SecretListTool};
self.register_sync(Arc::new(SecretListTool::new(Arc::clone(&store))));
self.register_sync(Arc::new(SecretDeleteTool::new(store)));
tracing::info!("Registered 2 secret management tools (list, delete)");
tracing::debug!("Registered 2 secret management tools (list, delete)");
}
/// Register extension management tools (search, install, auth, activate, list, remove).
@@ -393,7 +393,7 @@ impl ToolRegistry {
self.register_sync(Arc::new(ToolRemoveTool::new(Arc::clone(&manager))));
self.register_sync(Arc::new(ToolUpgradeTool::new(Arc::clone(&manager))));
self.register_sync(Arc::new(ExtensionInfoTool::new(manager)));
tracing::info!("Registered 8 extension management tools");
tracing::debug!("Registered 8 extension management tools");
}
/// Register skill management tools (list, search, install, remove).
@@ -414,7 +414,7 @@ impl ToolRegistry {
Arc::clone(&catalog),
)));
self.register_sync(Arc::new(SkillRemoveTool::new(registry)));
tracing::info!("Registered 4 skill management tools");
tracing::debug!("Registered 4 skill management tools");
}
/// Register routine management tools.
@@ -448,7 +448,7 @@ impl ToolRegistry {
Arc::clone(&engine),
)));
self.register_sync(Arc::new(RoutineHistoryTool::new(store)));
tracing::info!("Registered 6 routine management tools");
tracing::debug!("Registered 6 routine management tools");
}
/// Register message tool for sending messages to channels.
@@ -467,7 +467,7 @@ impl ToolRegistry {
.write()
.await
.insert("message".to_string());
tracing::info!("Registered message tool");
tracing::debug!("Registered message tool");
}
/// Set the default channel and target for the message tool.
@@ -501,7 +501,7 @@ impl ToolRegistry {
gen_model,
base_dir,
)));
tracing::info!("Registered 2 image tools (generate, edit)");
tracing::debug!("Registered 2 image tools (generate, edit)");
}
/// Register vision/image analysis tools.
@@ -521,7 +521,7 @@ impl ToolRegistry {
vision_model,
base_dir,
)));
tracing::info!("Registered 1 vision tool (analyze)");
tracing::debug!("Registered 1 vision tool (analyze)");
}
/// Register the software builder tool.
@@ -549,7 +549,7 @@ impl ToolRegistry {
self.register(Arc::new(BuildSoftwareTool::new(builder)))
.await;
tracing::info!("Registered software builder tool");
tracing::debug!("Registered software builder tool");
}
/// Register a WASM tool from bytes.
@@ -619,7 +619,7 @@ impl ToolRegistry {
);
}
tracing::info!(name = reg.name, "Registered WASM tool");
tracing::debug!(name = reg.name, "Registered WASM tool");
Ok(())
}
@@ -676,7 +676,7 @@ impl ToolRegistry {
.await
.map_err(WasmRegistrationError::Wasm)?;
tracing::info!(
tracing::debug!(
name = tool_with_binary.tool.name,
user_id = user_id,
trust_level = %tool_with_binary.tool.trust_level,
+36 -8
View File
@@ -193,18 +193,31 @@ impl WasmToolLoader {
///
/// Tools without a capabilities file get no permissions (default deny).
pub async fn load_from_dir(&self, dir: &Path) -> Result<LoadResults, WasmLoadError> {
if !dir.is_dir() {
return Err(WasmLoadError::Io(std::io::Error::new(
std::io::ErrorKind::NotADirectory,
format!("{} is not a directory", dir.display()),
)));
match fs::metadata(dir).await {
Ok(meta) if meta.is_dir() => {}
Ok(_) => {
return Err(WasmLoadError::Io(std::io::Error::new(
std::io::ErrorKind::NotADirectory,
format!("{} is not a directory", dir.display()),
)));
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Ok(LoadResults::default());
}
Err(e) => return Err(WasmLoadError::Io(e)),
}
let mut results = LoadResults::default();
// Handle TOCTOU: if read_dir fails with NotFound, treat as empty
let mut entries = match fs::read_dir(dir).await {
Ok(entries) => entries,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Ok(LoadResults::default());
}
Err(e) => return Err(WasmLoadError::Io(e)),
};
// Collect all .wasm entries first, then load in parallel
let mut results = LoadResults::default();
let mut tool_entries = Vec::new();
let mut entries = fs::read_dir(dir).await?;
while let Some(entry) = entries.next_entry().await? {
let path = entry.path();
@@ -1077,4 +1090,19 @@ mod tests {
"nested.wasm inside subdir should NOT be discovered"
);
}
#[tokio::test]
async fn load_from_dir_returns_empty_when_dir_missing() {
let loader = make_loader();
let dir = TempDir::new().unwrap();
let missing = dir.path().join("nonexistent_tools_dir");
let results = loader.load_from_dir(&missing).await;
// Must succeed with empty results, not error
let results = results.expect("missing dir should return Ok, not Err");
assert!(results.loaded.is_empty());
assert!(results.errors.is_empty());
}
}
+19
View File
@@ -21,8 +21,27 @@
use std::io::{self, Write};
use tracing_subscriber::EnvFilter;
use tracing_subscriber::fmt::MakeWriter;
/// Initialize tracing for simple CLI commands (warn level, no fancy layers).
pub fn init_cli_tracing() {
tracing_subscriber::fmt()
.with_env_filter(
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("warn")),
)
.init();
}
/// Initialize tracing for worker/bridge processes (info level).
pub fn init_worker_tracing() {
tracing_subscriber::fmt()
.with_env_filter(
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("ironclaw=info")),
)
.init();
}
/// Maximum bytes per tracing event written to the terminal.
const TERMINAL_MAX_EVENT_BYTES: usize = 500;
+62
View File
@@ -180,6 +180,68 @@ pub fn create_tunnel(config: &TunnelProviderConfig) -> Result<Option<Box<dyn Tun
}
}
// ── Managed tunnel startup ───────────────────────────────────────
/// Start a managed tunnel if configured and no static URL is already set.
///
/// Returns the (potentially mutated) config with `tunnel.public_url` set,
/// plus the active tunnel handle (if one was started) for later shutdown.
pub async fn start_managed_tunnel(
mut config: crate::config::Config,
) -> (crate::config::Config, Option<Box<dyn Tunnel>>) {
if config.tunnel.public_url.is_some() {
tracing::info!(
"Static tunnel URL in use: {}",
config.tunnel.public_url.as_deref().unwrap_or("?")
);
return (config, None);
}
let Some(ref provider_config) = config.tunnel.provider else {
return (config, None);
};
let gateway_port = config
.channels
.gateway
.as_ref()
.map(|g| g.port)
.unwrap_or(3000);
let gateway_host = config
.channels
.gateway
.as_ref()
.map(|g| g.host.as_str())
.unwrap_or("127.0.0.1");
match create_tunnel(provider_config) {
Ok(Some(tunnel)) => {
tracing::info!(
"Starting {} tunnel on {}:{}...",
tunnel.name(),
gateway_host,
gateway_port
);
match tunnel.start(gateway_host, gateway_port).await {
Ok(url) => {
tracing::info!("Tunnel started: {}", url);
config.tunnel.public_url = Some(url);
(config, Some(tunnel))
}
Err(e) => {
tracing::error!("Failed to start tunnel: {}", e);
(config, None)
}
}
}
Ok(None) => (config, None),
Err(e) => {
tracing::error!("Failed to create tunnel: {}", e);
(config, None)
}
}
}
// ── Tests ────────────────────────────────────────────────────────
#[cfg(test)]
+58
View File
@@ -33,3 +33,61 @@ pub use api::WorkerHttpClient;
pub use claude_bridge::ClaudeBridgeRuntime;
pub use proxy_llm::ProxyLlmProvider;
pub use runtime::WorkerRuntime;
/// Run the Worker subcommand (inside Docker containers).
pub async fn run_worker(
job_id: uuid::Uuid,
orchestrator_url: &str,
max_iterations: u32,
) -> anyhow::Result<()> {
tracing::info!(
"Starting worker for job {} (orchestrator: {})",
job_id,
orchestrator_url
);
let config = runtime::WorkerConfig {
job_id,
orchestrator_url: orchestrator_url.to_string(),
max_iterations,
timeout: std::time::Duration::from_secs(600),
};
let rt =
WorkerRuntime::new(config).map_err(|e| anyhow::anyhow!("Worker init failed: {}", e))?;
rt.run()
.await
.map_err(|e| anyhow::anyhow!("Worker failed: {}", e))
}
/// Run the Claude Code bridge subcommand (inside Docker containers).
pub async fn run_claude_bridge(
job_id: uuid::Uuid,
orchestrator_url: &str,
max_turns: u32,
model: &str,
) -> anyhow::Result<()> {
tracing::info!(
"Starting Claude Code bridge for job {} (orchestrator: {}, model: {})",
job_id,
orchestrator_url,
model
);
let config = claude_bridge::ClaudeBridgeConfig {
job_id,
orchestrator_url: orchestrator_url.to_string(),
max_turns,
model: model.to_string(),
timeout: std::time::Duration::from_secs(1800),
allowed_tools: crate::config::ClaudeCodeConfig::from_env().allowed_tools,
};
let rt = ClaudeBridgeRuntime::new(config)
.map_err(|e| anyhow::anyhow!("Claude bridge init failed: {}", e))?;
rt.run()
.await
.map_err(|e| anyhow::anyhow!("Claude bridge failed: {}", e))
}
+33 -1
View File
@@ -20,8 +20,10 @@ mod tests {
use ironclaw::agent::routine_engine::RoutineEngine;
use ironclaw::agent::{HeartbeatConfig, HeartbeatRunner};
use ironclaw::channels::IncomingMessage;
use ironclaw::config::RoutineConfig;
use ironclaw::config::{RoutineConfig, SafetyConfig};
use ironclaw::db::Database;
use ironclaw::safety::SafetyLayer;
use ironclaw::tools::ToolRegistry;
use ironclaw::workspace::Workspace;
use ironclaw::workspace::hygiene::HygieneConfig;
@@ -103,6 +105,14 @@ mod tests {
let (notify_tx, mut notify_rx) = tokio::sync::mpsc::channel(16);
// Create minimal ToolRegistry and SafetyLayer for test.
let tools = Arc::new(ToolRegistry::new());
let safety_config = SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: true,
};
let safety = Arc::new(SafetyLayer::new(&safety_config));
let engine = Arc::new(RoutineEngine::new(
RoutineConfig::default(),
db.clone(),
@@ -110,6 +120,8 @@ mod tests {
ws,
notify_tx,
None,
tools,
safety,
));
// Insert a cron routine with next_fire_at in the past.
@@ -170,6 +182,14 @@ mod tests {
let llm = Arc::new(TraceLlm::from_trace(trace));
let (notify_tx, _notify_rx) = tokio::sync::mpsc::channel(16);
// Create minimal ToolRegistry and SafetyLayer for test.
let tools = Arc::new(ToolRegistry::new());
let safety_config = SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: true,
};
let safety = Arc::new(SafetyLayer::new(&safety_config));
let engine = Arc::new(RoutineEngine::new(
RoutineConfig::default(),
db.clone(),
@@ -177,6 +197,8 @@ mod tests {
ws,
notify_tx,
None,
tools,
safety,
));
// Insert an event routine matching "deploy.*production".
@@ -258,6 +280,14 @@ mod tests {
let llm = Arc::new(TraceLlm::from_trace(trace));
let (notify_tx, _notify_rx) = tokio::sync::mpsc::channel(16);
// Create minimal ToolRegistry and SafetyLayer for test.
let tools = Arc::new(ToolRegistry::new());
let safety_config = SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: true,
};
let safety = Arc::new(SafetyLayer::new(&safety_config));
let engine = Arc::new(RoutineEngine::new(
RoutineConfig::default(),
db.clone(),
@@ -265,6 +295,8 @@ mod tests {
ws,
notify_tx,
None,
tools,
safety,
));
// Insert an event routine with 1-hour cooldown.
+237
View File
@@ -0,0 +1,237 @@
//! Integration test for module-owned initialization factories.
//!
//! Verifies that the refactored factory functions in `db`, `secrets`,
//! `orchestrator`, and `extensions` modules wire up correctly end-to-end,
//! ensuring nothing was lost when initialization logic was moved out of
//! `main.rs` and `app.rs` into owning modules.
use std::sync::Arc;
use ironclaw::db::DatabaseHandles;
use ironclaw::secrets::{CreateSecretParams, SecretsCrypto, SecretsStore};
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
/// Build a libsql DatabaseConfig pointing at a temp file.
#[cfg(feature = "libsql")]
fn libsql_config(path: &std::path::Path) -> ironclaw::config::DatabaseConfig {
ironclaw::config::DatabaseConfig {
backend: ironclaw::config::DatabaseBackend::LibSql,
url: secrecy::SecretString::from(String::new()),
pool_size: 1,
ssl_mode: ironclaw::config::SslMode::Prefer,
libsql_path: Some(path.to_path_buf()),
libsql_url: None,
libsql_auth_token: None,
}
}
/// Build a master-key crypto instance for tests.
fn test_crypto() -> Arc<SecretsCrypto> {
let key = secrecy::SecretString::from(ironclaw::secrets::keychain::generate_master_key_hex());
Arc::new(SecretsCrypto::new(key).expect("test crypto"))
}
// ---------------------------------------------------------------------------
// connect_with_handles: returns Database + populated handles
// ---------------------------------------------------------------------------
#[cfg(feature = "libsql")]
#[tokio::test]
async fn connect_with_handles_returns_db_and_libsql_handle() {
let dir = tempfile::tempdir().expect("tempdir");
let db_path = dir.path().join("test.db");
let config = libsql_config(&db_path);
let (db, handles) = ironclaw::db::connect_with_handles(&config)
.await
.expect("connect_with_handles");
// Database trait object works — run a trivial operation.
db.run_migrations().await.expect("migrations");
// Handle is populated.
assert!(
handles.libsql_db.is_some(),
"libsql handle should be Some after connect_with_handles"
);
}
// ---------------------------------------------------------------------------
// connect_from_config delegates to connect_with_handles
// ---------------------------------------------------------------------------
#[cfg(feature = "libsql")]
#[tokio::test]
async fn connect_from_config_produces_working_db() {
let dir = tempfile::tempdir().expect("tempdir");
let db_path = dir.path().join("test.db");
let config = libsql_config(&db_path);
// connect_from_config delegates to connect_with_handles internally.
let db = ironclaw::db::connect_from_config(&config)
.await
.expect("connect_from_config");
// Verify usable — migrations should be idempotent.
db.run_migrations().await.expect("migrations");
}
// ---------------------------------------------------------------------------
// secrets::create_secrets_store from DatabaseHandles
// ---------------------------------------------------------------------------
#[cfg(feature = "libsql")]
#[tokio::test]
async fn secrets_store_from_handles_round_trips() {
let dir = tempfile::tempdir().expect("tempdir");
let db_path = dir.path().join("test.db");
let config = libsql_config(&db_path);
let (_db, handles) = ironclaw::db::connect_with_handles(&config)
.await
.expect("connect");
let crypto = test_crypto();
let store = ironclaw::secrets::create_secrets_store(crypto, &handles)
.expect("create_secrets_store should return Some for libsql");
// Round-trip a secret to prove the store works.
store
.create("test", CreateSecretParams::new("test_key", "test_value"))
.await
.expect("create secret");
let decrypted = store
.get_decrypted("test", "test_key")
.await
.expect("get_decrypted");
assert_eq!(decrypted.expose(), "test_value");
}
// ---------------------------------------------------------------------------
// db::create_secrets_store (standalone CLI factory)
// ---------------------------------------------------------------------------
#[cfg(feature = "libsql")]
#[tokio::test]
async fn db_create_secrets_store_standalone_round_trips() {
let dir = tempfile::tempdir().expect("tempdir");
let db_path = dir.path().join("test.db");
let config = libsql_config(&db_path);
let crypto = test_crypto();
let store = ironclaw::db::create_secrets_store(&config, crypto)
.await
.expect("db::create_secrets_store");
store
.create(
"test",
CreateSecretParams::new("standalone_key", "standalone_value"),
)
.await
.expect("create secret");
let decrypted = store
.get_decrypted("test", "standalone_key")
.await
.expect("get_decrypted");
assert_eq!(decrypted.expose(), "standalone_value");
}
// ---------------------------------------------------------------------------
// Both secrets factories produce equivalent stores
// ---------------------------------------------------------------------------
#[cfg(feature = "libsql")]
#[tokio::test]
async fn both_secrets_factories_produce_compatible_stores() {
let dir = tempfile::tempdir().expect("tempdir");
let db_path = dir.path().join("test.db");
let config = libsql_config(&db_path);
let crypto = test_crypto();
// Factory 1: connect_with_handles + secrets::create_secrets_store
let (_db, handles) = ironclaw::db::connect_with_handles(&config)
.await
.expect("connect");
let store_a = ironclaw::secrets::create_secrets_store(Arc::clone(&crypto), &handles)
.expect("store from handles");
// Factory 2: db::create_secrets_store (standalone)
let store_b = ironclaw::db::create_secrets_store(&config, crypto)
.await
.expect("standalone store");
// Write with factory 1, read with factory 2.
store_a
.create(
"test",
CreateSecretParams::new("cross_factory", "shared_secret"),
)
.await
.expect("create via store_a");
let decrypted = store_b
.get_decrypted("test", "cross_factory")
.await
.expect("read via store_b");
assert_eq!(decrypted.expose(), "shared_secret");
}
// ---------------------------------------------------------------------------
// ExtensionManager constructs with McpProcessManager
// ---------------------------------------------------------------------------
#[tokio::test]
async fn extension_manager_with_process_manager_constructs() {
use ironclaw::extensions::ExtensionManager;
use ironclaw::secrets::InMemorySecretsStore;
use ironclaw::tools::ToolRegistry;
use ironclaw::tools::mcp::McpProcessManager;
use ironclaw::tools::mcp::McpSessionManager;
let crypto = test_crypto();
let secrets: Arc<dyn SecretsStore + Send + Sync> = Arc::new(InMemorySecretsStore::new(crypto));
let tools = Arc::new(ToolRegistry::new());
let tools_dir = tempfile::tempdir().expect("tools_dir");
let channels_dir = tempfile::tempdir().expect("channels_dir");
let manager = ExtensionManager::new(
Arc::new(McpSessionManager::new()),
Arc::new(McpProcessManager::new()),
secrets,
tools,
None,
None,
tools_dir.path().to_path_buf(),
channels_dir.path().to_path_buf(),
None,
"test".to_string(),
None,
Vec::new(),
);
// Verify the manager is functional — list returns Ok.
let result = manager.list(None, false).await;
assert!(result.is_ok(), "list should succeed on empty manager");
assert!(result.unwrap().is_empty());
}
// ---------------------------------------------------------------------------
// DatabaseHandles: default is empty
// ---------------------------------------------------------------------------
#[test]
fn database_handles_default_is_empty() {
let handles = DatabaseHandles::default();
#[cfg(feature = "postgres")]
assert!(handles.pg_pool.is_none());
#[cfg(feature = "libsql")]
assert!(handles.libsql_db.is_none());
}
+4
View File
@@ -575,6 +575,8 @@ impl TestRigBuilder {
Arc::clone(ws),
notify_tx,
None,
components.tools.clone(),
components.safety.clone(),
));
components
.tools
@@ -644,6 +646,8 @@ impl TestRigBuilder {
max_concurrent_routines: 3,
default_cooldown_secs: 300,
max_lightweight_tokens: 4096,
lightweight_tools_enabled: true,
lightweight_max_iterations: 3,
})
} else {
None