gitops/optimclaw
1
0
Fork 0
mirror of https://github.com/outbackdingo/optimclaw.git synced 2026-08-27 08:00:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Sandbox jobs (#4)

Browse Source 
* Orchestrating jobs and running them in sandboxes

* Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback

- Query /v1/models API for context_length and set max_tokens to half
  (floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7
  need much larger budgets
- Guard against empty LLM content (reasoning models can burn all tokens
  on chain-of-thought and return content: null)
- Simplify notification routing: try configured channel first, fall back
  to broadcast_all so heartbeat alerts always reach someone
- Add ModelMetadata struct and model_metadata() to LlmProvider trait
- Refactor NearAiChatProvider::list_models into shared fetch_models()
- Add standalone test_heartbeat example for isolated debugging

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add job detail view with drill-down from jobs list

Click a job row to see full details across four sub-tabs:
Overview (metadata grid, description, state transitions timeline),
Actions (expandable tool call cards with input/output JSON),
Thinking (conversation messages styled by role), and
Files (embedded workspace tree browser).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400

Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the
content field instead of using the OpenAI tool_calls array. This XML leaks
through to channels as text, and Telegram's Markdown parser chokes on the
underscores, returning 400 "can't parse entities".

Two fixes:
- Generalize clean_response() to strip <tool_call>, <function_call>,
  <tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside
  the existing <thinking> tag stripping
- Add Telegram send_message helper with parse_mode fallback: try Markdown
  first, retry as plain text on "can't parse entities" 400 errors

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add SystemCommand submission type for thread-state-independent commands

System commands (/help, /model, /version, /tools, /ping, /debug) now
bypass thread-state checks and safety validation via a dedicated
Submission::SystemCommand variant. Previously these flowed through
process_user_input() which blocked them during Processing/AwaitingApproval
/Completed states.

- Add /model [name] for runtime model switching with provider validation
- Add active_model_name()/set_model() to LlmProvider trait with RwLock
  hot-swap in both NEAR AI providers
- Rewrite /help with aligned columns grouped by category
- Expand REPL tab-completion from 10 to 23 slash commands
- Remove REPL-local /help interception (now handled by agent)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files

The sandbox e2e pipeline (agent -> container -> built website -> browsable URL)
was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need
minutes, no auto-created project directory meant container output vanished, and
no HTTP route to browse the built files.

- Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four
  hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler,
  worker/runtime) with the per-tool value
- Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer)
- Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified,
  so every sandbox job gets a persistent bind mount
- Include `project_dir` and `browse_url` in sandbox tool output JSON
- Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes
  to the web gateway with path traversal protection and MIME type detection
- Add `mime_guess` dependency for content-type detection

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply cargo fmt to wizard.rs after merge

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Persist sandbox jobs in DB, fix web UI, unify job model

Sandbox container jobs were invisible to the web UI because they lived
only in ContainerJobManager's in-memory HashMap while the API queried
ContextManager. This persists them to the agent_jobs table and fixes
all six front-end bugs (empty job list, broken back button, empty
actions/thinking tabs, wrong files tab, stuck status, no persistence).

Key changes:
- V4 migration adds project_dir and user_id columns to agent_jobs
- Embedded migrations via refinery (no external CLI needed)
- SandboxJobRecord CRUD in Store with fire-and-forget DB writes
- Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager
- Web API queries DB for sandbox jobs, merges with ContextManager direct jobs
- New endpoints: restart, project file list/read with path traversal protection
- Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in
  chat stream, source badges, restart button for failed/interrupted jobs
- Gateway defaults to enabled, prints Web UI URL on startup
- Stale jobs marked "interrupted" on restart for visibility and restartability

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Secure in-chat auth: tokens never touch the LLM or chat history

Remove the token parameter from tool_auth so the LLM cannot pass raw
API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket
(auth_token) endpoints that route tokens directly to ext_mgr.auth(),
completely bypassing the message pipeline, turns, history, and compaction.

Web UI shows an auth card (password input + OAuth button) when the agent
enters auth mode, submitted via the dedicated endpoint. CLI auth mode
interception is unchanged (already secure).

New StatusUpdate::AuthRequired/AuthCompleted variants propagate through
all channels (SSE, WebSocket, REPL, WASM).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add Claude Code mode for sandbox jobs

Run Claude Code CLI inside Docker containers as an alternative to the
standard worker mode. The bridge spawns `claude -p` with stream-json
output, posts events to the orchestrator, and supports follow-up
prompts via `--resume`.

Key additions:
- `claude-bridge` CLI subcommand and ClaudeBridgeRuntime
- JobMode enum (Worker vs ClaudeCode) with per-mode container config
- Orchestrator endpoints for Claude events and prompt polling
- SSE event variants for real-time Claude Code streaming to frontend
- Claude Code sub-tab in web UI with terminal-style output and input bar
- Database migration for job_mode column and claude_code_events table
- ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.)
- Mode parameter on run_in_sandbox tool schema

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs

When sandbox mode is on, the LLM would call create_job (creating a
pending "direct" entry) then run_in_sandbox (creating a second "sandbox"
entry), producing two jobs in the list for a single user request.

Now register_job_tools() skips create_job when sandbox is enabled since
run_in_sandbox already creates tracked jobs. Also improved the
run_in_sandbox description to guide the LLM to use it directly and to
mention wait=false for async execution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Web gateway UI quality-of-life improvements

Phase 1: Send button disabled state to prevent double-sends, copy button
on code blocks, confirm() guards on destructive actions, SSE-driven job
list auto-refresh, log filters re-applied on tab switch, jobEvents memory
leak fix (cap at 500, cleanup after 60s).

Phase 2: Toast notification system replacing chat-based system messages,
memory search highlighting with centered snippets, keyboard shortcuts
(Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur),
activity tab toolbar with event type filter and auto-scroll toggle.

Phase 3: Thread sidebar with load/switch/create, thread_id passed with
messages, collapsible to hamburger. Memory inline editing with textarea,
Save/Cancel, POST to /api/memory/write.

Phase 4: Gateway status popover on hover (polls every 30s), extension
install form (name/URL/kind), markdown rendering in memory viewer for
.md files, mobile responsive layout at 768px breakpoint.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add routines system, remove non-sandbox job mode from web UI

Routines: scheduled & reactive job system with cron and event triggers,
lightweight (single LLM call) and full-job execution modes, guardrails
(cooldown, max concurrent, dedup), and LLM-facing tools for CRUD.

Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs
are now exclusively sandbox-backed (DB + container). Simplify job detail
response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo),
fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab
event rendering.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML

Three fixes:

1. Chat input stays disabled after agent finishes: the "Done" status
   SSE event now calls enableChatInput() as a safety net when the
   response event is empty or lost. Same for auth_completed and
   cancelAuth().

2. tool_activate never triggers auth: when activation fails due to
   missing authentication, it now auto-initiates the auth flow
   (same pattern as the web API handler). detect_auth_awaiting()
   also matches tool_activate results now.

3. Models like GLM-4.7 emit tool calls as XML tags in content
   (<tool_call>tool_list</tool_call>) instead of using the structured
   tool_calls array. recover_tool_calls_from_content() extracts and
   validates these before falling back to plain text.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add routines web UI tab, update docs for sandbox-jobs branch

Add full routines management to the web gateway (list, detail, trigger,
toggle, delete) with 7 new API endpoints, response types, and frontend
(HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new
subsystems, config, TODOs), and README.md (architecture diagram,
features, components, fix onboard command).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Bind Telegram bot to owner account during setup

Without owner binding, anyone who discovers the bot can send it messages.
The setup wizard now prompts the user to message their bot, captures their
Telegram user ID via getUpdates, and persists it as telegram_owner_id in
settings. On startup, the owner_id is injected into the WASM channel config
so the existing owner restriction logic drops messages from non-owners.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Move settings from disk to PostgreSQL database

Settings previously lived in three JSON files on disk (settings.json,
mcp-servers.json, session.json). This made them inaccessible from the
web UI and caused redundant disk reads (Settings::load() called 8+
times during startup).

Now all settings live in a `settings` table (user_id + key -> JSONB)
with only 4 bootstrap fields remaining on disk (database_url, pool
size, secrets key source, onboard_completed) since they're needed
before the DB connection exists.

- Add V8 migration for settings table
- Add BootstrapConfig (thin disk file) and Settings DB round-trip
- Add Store CRUD methods for settings (get/set/delete/list/bulk)
- Refactor Config to load from DB (env > DB > default cascade)
- Add SessionManager DB persistence for session tokens
- Add DB-backed MCP server config load/save functions
- Add 6 settings web API endpoints (list/get/set/delete/export/import)
- Add one-time disk-to-DB migration on first boot
- Make CLI config commands async with DB access (disk fallback)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth

- Add Workspace::seed_if_empty() to create core identity files (README,
  MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called
  on every boot without overwriting existing user edits
- Remove duplicate gateway log lines from web/mod.rs (main.rs has the
  useful clickable ?token= URL)
- Auto-authenticate from ?token= URL parameter in the web UI and strip
  the token from the address bar after successful auth

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Harden sandbox security (path traversal + orchestrator auth)

Two vulnerabilities fixed:

1. project_dir path traversal: The create_job tool let the LLM specify
   arbitrary host paths for Docker bind mounts. Removed project_dir from
   the tool schema entirely, and added canonicalization + prefix validation
   at both resolve_project_dir() and the job_manager bind mount point.

2. Orchestrator API auth bypass: worker_auth_middleware was defined but
   never applied. Each handler manually called validate_token(), so any
   new endpoint that forgot would be publicly accessible. Applied the
   middleware as route_layer on all /worker/ routes, removed manual auth
   from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps
   0.0.0.0 since containers reach host via docker bridge, not loopback).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining

Implements the 4-phase plan for overhauling the web gateway chat:

- Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below
- Phase 2: Cursor-based history pagination with infinite scroll
- Phase 3: NEAR AI previous_response_id chaining (delta-only messages),
  with fallback to full history on chain errors, and DB persistence of
  chain state across restarts
- Phase 4: SSE thread isolation (events filtered by thread_id)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Add per-request HTTP timeout to WASM host, redact credentials in errors

Three fixes for WASM channel reliability:

1. Per-request timeout: Add optional timeout-ms parameter to http-request
   in both channel and tool WIT interfaces. Telegram long-poll now specifies
   35s (outliving the 30s server-side hold), while regular API calls use
   the 30s default. Fixes the triple-30s timeout race that caused polling
   failures.

2. Credential redaction: reqwest::Error includes the full URL (with injected
   bot tokens) in its Display output. Scrub credential values from error
   messages before logging or returning to WASM.

3. Webhook route registration: Remove tunnel URL gate so webhook routes are
   always available when webhook channels exist, not only when TUNNEL_URL
   is configured.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: Fix clippy warnings in WASM tools and channels

- slack channel: allow dead_code on signing_secret_name (forward compat field)
- gmail tool: use div_ceil() instead of manual (n+2)/3
- google-calendar tool: extract CreateEventParams/UpdateEventParams structs
  to fix too-many-arguments warnings

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Fix approval flow

* fix: Rebuild bundled telegram.wasm with updated WIT interface

The bundled WASM binary must match the host's WIT definition.
Previous binary was compiled against the old 4-arg http-request;
this rebuild includes the new timeout-ms parameter.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: Load WASM channels from disk instead of bundling in binary

Remove include_bytes! embedding of telegram.wasm. Channels are now
loaded from their build output directories (channels-src/<name>/target/)
during onboarding, then from ~/.ironclaw/channels/ at runtime.

- bundled.rs: locate_channel_artifacts() finds WASM + capabilities from
  build output; IRONCLAW_CHANNELS_SRC env var overrides the default path
- available_channel_names(): only lists channels with build artifacts
- bundled_channel_names(): lists all known channels (manifest)
- Setup wizard uses available_channel_names() to offer installable channels
- Add *.wasm to .gitignore, remove tracked telegram.wasm

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Persist gateway auth token, fix thread hydration race, polish auth screen

Three web gateway UX fixes:

1. Token persistence: Store auth token in sessionStorage so refreshing
   the page doesn't force re-authentication. Hide the auth screen
   immediately when a saved token exists to prevent flash.

2. Thread hydration: Remove the !msgs.is_empty() bail-out in
   maybe_hydrate_thread so that even brand-new (empty) assistant threads
   get hydrated with their correct DB UUID. Previously resolve_thread
   would mint a fresh UUID, causing messages to land in the wrong
   conversation and duplicate threads to appear.

3. Auth screen: Redesign as a centered card with brand, tagline, labeled
   input, and hint text.

Also adds 34 new tests covering session/thread lifecycle, thread
resolution isolation (user, channel, external ID), hydration edge cases,
serialization round-trips, approval flows, and stale mapping recovery.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Use bindgen! for WASM tool wrapper, add dev tool loading

Three changes:

1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen!
   instead of manual linker.root().func_wrap(). This fixes the
   "component imports instance 'near:agent/host', but a matching
   implementation was not found in the linker" error. All 6 host functions
   (log, now-millis, workspace-read, http-request, secret-exists,
   tool-invoke) are now properly registered under the near:agent/host
   namespace. Also adds WASI support, credential injection, and leak
   detection for HTTP requests made by WASM tools.

2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the
   loader now also scans tools-src/*/target/wasm32-wasip2/release/ for
   build artifacts that are newer than installed copies. This means during
   development you just rebuild the WASM and restart the host; no manual
   copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir.

3. Wire up load_dev_tools() in main.rs alongside the existing
   load_from_dir() call.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Wire main startup and CLI to use DB-backed settings

main.rs now reloads Config from the database after connecting,
attaches the store to the session manager for dual-write tokens,
and loads MCP servers from DB instead of disk. ExtensionManager
and MCP CLI commands use DB when available with disk fallback.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
2026-02-11 08:31:25 +00:00
committed by GitHub
co-authored by Claude Opus 4.6
parent 202665a55c
commit ced83d5b4d
99 changed files with 17095 additions and 1162 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/tools/builder/core.rs
+12 -1
View File
@@ -689,9 +689,20 @@ Create alongside the .wasm file to grant capabilities:
.messages
.push(ChatMessage::user("Continue with the next step."));
}
RespondResult::ToolCalls(tool_calls) => {
RespondResult::ToolCalls {
tool_calls,
content,
} => {
tools_executed = true;
// Add assistant message with tool_calls (OpenAI protocol)
reason_ctx
.messages
.push(ChatMessage::assistant_with_tool_calls(
content,
tool_calls.clone(),
));
// Execute each tool call
for tc in tool_calls {
logs.push(BuildLog {
src/tools/builtin/extension_tools.rs
+55 -18
View File
@@ -185,8 +185,9 @@ impl Tool for ToolAuthTool {
}
fn description(&self) -> &str {
"Authenticate an installed extension. For MCP servers, starts OAuth flow. \
For WASM tools with manual auth, returns instructions; call again with token param to complete."
"Initiate authentication for an extension. For OAuth, returns a URL. \
For manual auth, returns instructions. The user provides their token \
through a secure channel, never through this tool."
}
fn parameters_schema(&self) -> serde_json::Value {
@@ -196,10 +197,6 @@ impl Tool for ToolAuthTool {
"name": {
"type": "string",
"description": "Extension name to authenticate"
},
"token": {
"type": "string",
"description": "API token/key for manual auth (WASM tools). Provide after user gives you the token."
}
},
"required": ["name"]
@@ -218,11 +215,9 @@ impl Tool for ToolAuthTool {
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("name is required".to_string()))?;
let token = params.get("token").and_then(|v| v.as_str());
let result = self
.manager
.auth(name, token)
.auth(name, None)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
@@ -316,16 +311,53 @@ impl Tool for ToolActivateTool {
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("name is required".to_string()))?;
let result = self
.manager
.activate(name)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
match self.manager.activate(name).await {
Ok(result) => {
let output = serde_json::to_value(&result)
.unwrap_or_else(|_| serde_json::json!({"error": "serialization failed"}));
Ok(ToolOutput::success(output, start.elapsed()))
}
Err(activate_err) => {
let err_str = activate_err.to_string();
let needs_auth = err_str.contains("authentication")
|| err_str.contains("401")
|| err_str.contains("Unauthorized")
|| err_str.contains("not authenticated");
let output = serde_json::to_value(&result)
.unwrap_or_else(|_| serde_json::json!({"error": "serialization failed"}));
if !needs_auth {
return Err(ToolError::ExecutionFailed(err_str));
}
Ok(ToolOutput::success(output, start.elapsed()))
// Activation failed due to missing auth; initiate auth flow
// so the agent loop can show the auth card.
match self.manager.auth(name, None).await {
Ok(auth_result) if auth_result.status == "authenticated" => {
// Auth succeeded (e.g. env var was set); retry activation.
let result = self
.manager
.activate(name)
.await
.map_err(|e| ToolError::ExecutionFailed(e.to_string()))?;
let output = serde_json::to_value(&result).unwrap_or_else(
|_| serde_json::json!({"error": "serialization failed"}),
);
Ok(ToolOutput::success(output, start.elapsed()))
}
Ok(auth_result) => {
// Auth needs user input (awaiting_token). Return the auth
// result so detect_auth_awaiting picks it up.
let output = serde_json::to_value(&auth_result).unwrap_or_else(
|_| serde_json::json!({"error": "serialization failed"}),
);
Ok(ToolOutput::success(output, start.elapsed()))
}
Err(auth_err) => Err(ToolError::ExecutionFailed(format!(
"Activation failed ({}), and authentication also failed: {}",
err_str, auth_err
))),
}
}
}
}
}
@@ -499,7 +531,11 @@ mod tests {
assert!(tool.requires_approval());
let schema = tool.parameters_schema();
assert!(schema["properties"].get("name").is_some());
assert!(schema["properties"].get("token").is_some());
// token param must NOT be in schema (security: tokens never go through LLM)
assert!(
schema["properties"].get("token").is_none(),
"tool_auth must not have a token parameter"
);
}
#[test]
@@ -550,6 +586,7 @@ mod tests {
std::path::PathBuf::from("/tmp/ironclaw-test-channels"),
None,
"test".to_string(),
None,
))
}
}
src/tools/builtin/file.rs
+17 -1
View File
@@ -11,7 +11,7 @@ use async_trait::async_trait;
use tokio::fs;
use crate::context::JobContext;
use crate::tools::tool::{Tool, ToolError, ToolOutput};
use crate::tools::tool::{Tool, ToolDomain, ToolError, ToolOutput};
use crate::workspace::paths as ws_paths;
/// Well-known workspace filenames that must go through memory_write, not write_file.
@@ -246,6 +246,10 @@ impl Tool for ReadFileTool {
fn requires_approval(&self) -> bool {
true // Reading local files should require approval
}
fn domain(&self) -> ToolDomain {
ToolDomain::Container
}
}
/// Write file contents tool.
@@ -359,6 +363,10 @@ impl Tool for WriteFileTool {
fn requires_sanitization(&self) -> bool {
false // We're writing, not reading external data
}
fn domain(&self) -> ToolDomain {
ToolDomain::Container
}
}
/// List directory contents tool.
@@ -467,6 +475,10 @@ impl Tool for ListDirTool {
fn requires_approval(&self) -> bool {
true // Directory listings can leak filesystem structure
}
fn domain(&self) -> ToolDomain {
ToolDomain::Container
}
}
/// Recursively list directory contents.
@@ -685,6 +697,10 @@ impl Tool for ApplyPatchTool {
fn requires_sanitization(&self) -> bool {
false // We're writing, not reading external data
}
fn domain(&self) -> ToolDomain {
ToolDomain::Container
}
}
#[cfg(test)]
src/tools/builtin/job.rs
+538 -42
View File
@@ -1,77 +1,108 @@
//! Job management tools.
//!
//! These tools allow the LLM to manage jobs:
//! - Create new jobs/tasks
//! - Create new jobs/tasks (with optional sandbox delegation)
//! - List existing jobs
//! - Check job status
//! - Cancel running jobs
use std::path::PathBuf;
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use crate::context::{ContextManager, JobContext, JobState};
use crate::history::{SandboxJobRecord, Store};
use crate::orchestrator::job_manager::{ContainerJobManager, JobMode};
use crate::tools::tool::{Tool, ToolError, ToolOutput};
/// Tool for creating a new job.
///
/// When sandbox deps are injected (via `with_sandbox`), the tool automatically
/// delegates execution to a Docker container. Otherwise it creates an in-memory
/// job via the ContextManager. The LLM never needs to know the difference.
pub struct CreateJobTool {
context_manager: Arc<ContextManager>,
job_manager: Option<Arc<ContainerJobManager>>,
store: Option<Arc<Store>>,
}
impl CreateJobTool {
pub fn new(context_manager: Arc<ContextManager>) -> Self {
Self { context_manager }
}
}
#[async_trait]
impl Tool for CreateJobTool {
fn name(&self) -> &str {
"create_job"
Self {
context_manager,
job_manager: None,
store: None,
}
}
fn description(&self) -> &str {
"Create a new job or task for the agent to work on. Use this when the user wants \
you to do something substantial that should be tracked as a separate job."
/// Inject sandbox dependencies so `create_job` delegates to Docker containers.
pub fn with_sandbox(
mut self,
job_manager: Arc<ContainerJobManager>,
store: Option<Arc<Store>>,
) -> Self {
self.job_manager = Some(job_manager);
self.store = store;
self
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"title": {
"type": "string",
"description": "A short title for the job (max 100 chars)"
},
"description": {
"type": "string",
"description": "Full description of what needs to be done"
fn sandbox_enabled(&self) -> bool {
self.job_manager.is_some()
}
/// Persist a sandbox job record (fire-and-forget).
fn persist_job(&self, record: SandboxJobRecord) {
if let Some(store) = self.store.clone() {
tokio::spawn(async move {
if let Err(e) = store.save_sandbox_job(&record).await {
tracing::warn!(job_id = %record.id, "Failed to persist sandbox job: {}", e);
}
},
"required": ["title", "description"]
})
});
}
}
async fn execute(
/// Update sandbox job status in DB (fire-and-forget).
fn update_status(
&self,
params: serde_json::Value,
job_id: Uuid,
status: &str,
success: Option<bool>,
message: Option<String>,
started_at: Option<chrono::DateTime<Utc>>,
completed_at: Option<chrono::DateTime<Utc>>,
) {
if let Some(store) = self.store.clone() {
let status = status.to_string();
tokio::spawn(async move {
if let Err(e) = store
.update_sandbox_job_status(
job_id,
&status,
success,
message.as_deref(),
started_at,
completed_at,
)
.await
{
tracing::warn!(job_id = %job_id, "Failed to update sandbox job status: {}", e);
}
});
}
}
/// Execute via in-memory ContextManager (no sandbox).
async fn execute_local(
&self,
title: &str,
description: &str,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let title = params
.get("title")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'title' parameter".into()))?;
let description = params
.get("description")
.and_then(|v| v.as_str())
.ok_or_else(|| {
ToolError::InvalidParameters("missing 'description' parameter".into())
})?;
match self
.context_manager
.create_job_for_user(&ctx.user_id, title, description)
@@ -95,6 +126,374 @@ impl Tool for CreateJobTool {
}
}
/// Execute via sandboxed Docker container.
async fn execute_sandbox(
&self,
task: &str,
explicit_dir: Option<PathBuf>,
wait: bool,
mode: JobMode,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let jm = self.job_manager.as_ref().expect("sandbox deps required");
let job_id = Uuid::new_v4();
let (project_dir, browse_id) = resolve_project_dir(explicit_dir, job_id)?;
let project_dir_str = project_dir.display().to_string();
// Persist the job to DB before creating the container.
self.persist_job(SandboxJobRecord {
id: job_id,
task: task.to_string(),
status: "creating".to_string(),
user_id: ctx.user_id.clone(),
project_dir: project_dir_str.clone(),
success: None,
failure_reason: None,
created_at: Utc::now(),
started_at: None,
completed_at: None,
});
// Persist the job mode to DB
if mode == JobMode::ClaudeCode {
if let Some(store) = self.store.clone() {
let job_id_copy = job_id;
tokio::spawn(async move {
if let Err(e) = store
.update_sandbox_job_mode(job_id_copy, "claude_code")
.await
{
tracing::warn!(job_id = %job_id_copy, "Failed to set job mode: {}", e);
}
});
}
}
// Create the container job with the pre-determined job_id.
let _token = jm
.create_job(job_id, task, Some(project_dir), mode)
.await
.map_err(|e| {
self.update_status(
job_id,
"failed",
Some(false),
Some(e.to_string()),
None,
Some(Utc::now()),
);
ToolError::ExecutionFailed(format!("failed to create container: {}", e))
})?;
// Container started successfully.
let now = Utc::now();
self.update_status(job_id, "running", None, None, Some(now), None);
if !wait {
let result = serde_json::json!({
"job_id": job_id.to_string(),
"status": "started",
"message": "Container started. Use job tools to check status.",
"project_dir": project_dir_str,
"browse_url": format!("/projects/{}", browse_id),
});
return Ok(ToolOutput::success(result, start.elapsed()));
}
// Wait for completion by polling the container state.
let timeout = Duration::from_secs(600);
let poll_interval = Duration::from_secs(2);
let deadline = tokio::time::Instant::now() + timeout;
loop {
if tokio::time::Instant::now() > deadline {
let _ = jm.stop_job(job_id).await;
jm.cleanup_job(job_id).await;
self.update_status(
job_id,
"failed",
Some(false),
Some("Timed out (10 minutes)".to_string()),
None,
Some(Utc::now()),
);
return Err(ToolError::ExecutionFailed(
"container execution timed out (10 minutes)".to_string(),
));
}
match jm.get_handle(job_id).await {
Some(handle) => match handle.state {
crate::orchestrator::job_manager::ContainerState::Running
| crate::orchestrator::job_manager::ContainerState::Creating => {
tokio::time::sleep(poll_interval).await;
}
crate::orchestrator::job_manager::ContainerState::Stopped => {
let message = handle
.completion_result
.as_ref()
.and_then(|r| r.message.clone())
.unwrap_or_else(|| "Container job completed".to_string());
let success = handle
.completion_result
.as_ref()
.map(|r| r.success)
.unwrap_or(true);
jm.cleanup_job(job_id).await;
let finished_at = Utc::now();
if success {
self.update_status(
job_id,
"completed",
Some(true),
None,
None,
Some(finished_at),
);
let result = serde_json::json!({
"job_id": job_id.to_string(),
"status": "completed",
"output": message,
"project_dir": project_dir_str,
"browse_url": format!("/projects/{}", browse_id),
});
return Ok(ToolOutput::success(result, start.elapsed()));
} else {
self.update_status(
job_id,
"failed",
Some(false),
Some(message.clone()),
None,
Some(finished_at),
);
return Err(ToolError::ExecutionFailed(format!(
"container job failed: {}",
message
)));
}
}
crate::orchestrator::job_manager::ContainerState::Failed => {
let message = handle
.completion_result
.as_ref()
.and_then(|r| r.message.clone())
.unwrap_or_else(|| "unknown failure".to_string());
jm.cleanup_job(job_id).await;
self.update_status(
job_id,
"failed",
Some(false),
Some(message.clone()),
None,
Some(Utc::now()),
);
return Err(ToolError::ExecutionFailed(format!(
"container job failed: {}",
message
)));
}
},
None => {
self.update_status(
job_id,
"completed",
Some(true),
None,
None,
Some(Utc::now()),
);
let result = serde_json::json!({
"job_id": job_id.to_string(),
"status": "completed",
"output": "Container job completed",
"project_dir": project_dir_str,
"browse_url": format!("/projects/{}", browse_id),
});
return Ok(ToolOutput::success(result, start.elapsed()));
}
}
}
}
}
/// The base directory where all project directories must live.
fn projects_base() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("projects")
}
/// Resolve the project directory, creating it if it doesn't exist.
///
/// Auto-creates `~/.ironclaw/projects/{project_id}/` so every sandbox job has a
/// persistent bind mount that survives container teardown.
///
/// When an explicit path is provided (e.g. job restarts reusing the old dir),
/// it is validated to fall within `~/.ironclaw/projects/` after canonicalization.
fn resolve_project_dir(
explicit: Option<PathBuf>,
project_id: Uuid,
) -> Result<(PathBuf, String), ToolError> {
let base = projects_base();
std::fs::create_dir_all(&base).map_err(|e| {
ToolError::ExecutionFailed(format!(
"failed to create projects base {}: {}",
base.display(),
e
))
})?;
let canonical_base = base.canonicalize().map_err(|e| {
ToolError::ExecutionFailed(format!("failed to canonicalize projects base: {}", e))
})?;
let dir = match explicit {
Some(d) => d,
None => canonical_base.join(project_id.to_string()),
};
std::fs::create_dir_all(&dir).map_err(|e| {
ToolError::ExecutionFailed(format!(
"failed to create project dir {}: {}",
dir.display(),
e
))
})?;
// Canonicalize resolves symlinks, `..`, etc. so we can do a reliable prefix check.
let canonical_dir = dir.canonicalize().map_err(|e| {
ToolError::ExecutionFailed(format!(
"failed to canonicalize project dir {}: {}",
dir.display(),
e
))
})?;
if !canonical_dir.starts_with(&canonical_base) {
return Err(ToolError::InvalidParameters(format!(
"project directory must be under {}",
canonical_base.display()
)));
}
let browse_id = canonical_dir
.file_name()
.map(|n| n.to_string_lossy().to_string())
.unwrap_or_else(|| project_id.to_string());
Ok((canonical_dir, browse_id))
}
#[async_trait]
impl Tool for CreateJobTool {
fn name(&self) -> &str {
"create_job"
}
fn description(&self) -> &str {
if self.sandbox_enabled() {
"Create and execute a job. The job runs in a sandboxed Docker container with its own \
sub-agent that has shell, file read/write, list_dir, and apply_patch tools. Use this \
whenever the user asks you to build, create, or work on something. The task \
description should be detailed enough for the sub-agent to work independently. \
Set wait=false to start immediately while continuing the conversation. Set mode \
to 'claude_code' for complex software engineering tasks."
} else {
"Create a new job or task for the agent to work on. Use this when the user wants \
you to do something substantial that should be tracked as a separate job."
}
}
fn parameters_schema(&self) -> serde_json::Value {
if self.sandbox_enabled() {
serde_json::json!({
"type": "object",
"properties": {
"title": {
"type": "string",
"description": "Clear description of what to accomplish"
},
"description": {
"type": "string",
"description": "Full description of what needs to be done"
},
"wait": {
"type": "boolean",
"description": "If true (default), wait for the container to complete and return results. \
If false, start the container and return the job_id immediately."
},
"mode": {
"type": "string",
"enum": ["worker", "claude_code"],
"description": "Execution mode. 'worker' (default) uses the IronClaw sub-agent. \
'claude_code' uses Claude Code CLI for full agentic software engineering."
}
},
"required": ["title", "description"]
})
} else {
serde_json::json!({
"type": "object",
"properties": {
"title": {
"type": "string",
"description": "A short title for the job (max 100 chars)"
},
"description": {
"type": "string",
"description": "Full description of what needs to be done"
}
},
"required": ["title", "description"]
})
}
}
fn execution_timeout(&self) -> Duration {
if self.sandbox_enabled() {
// Sandbox polls for up to 10 min internally; give an extra 60s buffer.
Duration::from_secs(660)
} else {
Duration::from_secs(30)
}
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let title = params
.get("title")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'title' parameter".into()))?;
let description = params
.get("description")
.and_then(|v| v.as_str())
.ok_or_else(|| {
ToolError::InvalidParameters("missing 'description' parameter".into())
})?;
if self.sandbox_enabled() {
let wait = params.get("wait").and_then(|v| v.as_bool()).unwrap_or(true);
let mode = match params.get("mode").and_then(|v| v.as_str()) {
Some("claude_code") => JobMode::ClaudeCode,
_ => JobMode::Worker,
};
// Combine title and description into the task prompt for the sub-agent.
let task = format!("{}\n\n{}", title, description);
self.execute_sandbox(&task, None, wait, mode, ctx).await
} else {
self.execute_local(title, description, ctx).await
}
}
fn requires_sanitization(&self) -> bool {
false
}
@@ -377,10 +776,13 @@ mod tests {
use super::*;
#[tokio::test]
async fn test_create_job_tool() {
async fn test_create_job_tool_local() {
let manager = Arc::new(ContextManager::new(5));
let tool = CreateJobTool::new(manager.clone());
// Without sandbox deps, it should use the local path
assert!(!tool.sandbox_enabled());
let params = serde_json::json!({
"title": "Test Job",
"description": "A test job description"
@@ -391,6 +793,37 @@ mod tests {
let job_id = result.result.get("job_id").unwrap().as_str().unwrap();
assert!(!job_id.is_empty());
assert_eq!(
result.result.get("status").unwrap().as_str().unwrap(),
"pending"
);
}
#[test]
fn test_schema_changes_with_sandbox() {
let manager = Arc::new(ContextManager::new(5));
// Without sandbox
let tool = CreateJobTool::new(Arc::clone(&manager));
let schema = tool.parameters_schema();
let props = schema.get("properties").unwrap().as_object().unwrap();
assert!(props.contains_key("title"));
assert!(props.contains_key("description"));
assert!(
!props.contains_key("project_dir"),
"project_dir must not be exposed to the LLM"
);
assert!(!props.contains_key("wait"));
assert!(!props.contains_key("mode"));
}
#[test]
fn test_execution_timeout_sandbox() {
let manager = Arc::new(ContextManager::new(5));
// Without sandbox: default timeout
let tool = CreateJobTool::new(Arc::clone(&manager));
assert_eq!(tool.execution_timeout(), Duration::from_secs(30));
}
#[tokio::test]
@@ -429,4 +862,67 @@ mod tests {
"Test Job"
);
}
#[test]
fn test_resolve_project_dir_auto() {
let project_id = Uuid::new_v4();
let (dir, browse_id) = resolve_project_dir(None, project_id).unwrap();
assert!(dir.exists());
assert!(dir.ends_with(project_id.to_string()));
assert_eq!(browse_id, project_id.to_string());
// Must be under the projects base
let base = projects_base().canonicalize().unwrap();
assert!(dir.starts_with(&base));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn test_resolve_project_dir_explicit_under_base() {
let base = projects_base();
std::fs::create_dir_all(&base).unwrap();
let explicit = base.join("test_explicit_project");
let project_id = Uuid::new_v4();
let (dir, browse_id) = resolve_project_dir(Some(explicit.clone()), project_id).unwrap();
assert!(dir.exists());
assert_eq!(browse_id, "test_explicit_project");
let canonical_base = base.canonicalize().unwrap();
assert!(dir.starts_with(&canonical_base));
let _ = std::fs::remove_dir_all(&explicit);
}
#[test]
fn test_resolve_project_dir_rejects_outside_base() {
let tmp = tempfile::tempdir().unwrap();
let escape_attempt = tmp.path().join("evil_project");
let result = resolve_project_dir(Some(escape_attempt), Uuid::new_v4());
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("must be under"),
"expected 'must be under' error, got: {}",
err
);
}
#[test]
fn test_resolve_project_dir_rejects_traversal() {
// Attempt to escape via `..` components
let base = projects_base();
let traversal = base.join("legit").join("..").join("..").join(".ssh");
let result = resolve_project_dir(Some(traversal), Uuid::new_v4());
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("must be under"),
"expected 'must be under' error, got: {}",
err
);
}
}
src/tools/builtin/mod.rs
+4
View File
@@ -10,6 +10,7 @@ mod json;
mod marketplace;
mod memory;
mod restaurant;
pub mod routine;
mod shell;
mod taskrabbit;
mod time;
@@ -26,6 +27,9 @@ pub use json::JsonTool;
pub use marketplace::MarketplaceTool;
pub use memory::{MemoryReadTool, MemorySearchTool, MemoryTreeTool, MemoryWriteTool};
pub use restaurant::RestaurantTool;
pub use routine::{
RoutineCreateTool, RoutineDeleteTool, RoutineHistoryTool, RoutineListTool, RoutineUpdateTool,
};
pub use shell::ShellTool;
pub use taskrabbit::TaskRabbitTool;
pub use time::TimeTool;
src/tools/builtin/routine.rs
+654
View File
@@ -0,0 +1,654 @@
//! LLM-facing tools for managing routines.
//!
//! Five tools let the agent manage routines conversationally:
//! - `routine_create` - Create a new routine
//! - `routine_list` - List all routines with status
//! - `routine_update` - Modify or toggle a routine
//! - `routine_delete` - Remove a routine
//! - `routine_history` - View past runs
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use chrono::Utc;
use uuid::Uuid;
use crate::agent::routine::{
NotifyConfig, Routine, RoutineAction, RoutineGuardrails, Trigger, next_cron_fire,
};
use crate::agent::routine_engine::RoutineEngine;
use crate::context::JobContext;
use crate::history::Store;
use crate::tools::tool::{Tool, ToolError, ToolOutput};
// ==================== routine_create ====================
pub struct RoutineCreateTool {
store: Arc<Store>,
engine: Arc<RoutineEngine>,
}
impl RoutineCreateTool {
pub fn new(store: Arc<Store>, engine: Arc<RoutineEngine>) -> Self {
Self { store, engine }
}
}
#[async_trait]
impl Tool for RoutineCreateTool {
fn name(&self) -> &str {
"routine_create"
}
fn description(&self) -> &str {
"Create a new routine (scheduled or event-driven task). \
Supports cron schedules, event pattern matching, webhooks, and manual triggers. \
Use this when the user wants something to happen periodically or reactively."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Unique name for the routine (e.g. 'daily-pr-review')"
},
"description": {
"type": "string",
"description": "What this routine does"
},
"trigger_type": {
"type": "string",
"enum": ["cron", "event", "webhook", "manual"],
"description": "When the routine fires"
},
"schedule": {
"type": "string",
"description": "Cron expression (for cron trigger). E.g. '0 9 * * MON-FRI' for weekdays at 9am. Uses 6-field cron (sec min hour day month weekday)."
},
"event_pattern": {
"type": "string",
"description": "Regex pattern to match messages (for event trigger)"
},
"event_channel": {
"type": "string",
"description": "Optional channel filter for event trigger (e.g. 'telegram')"
},
"prompt": {
"type": "string",
"description": "The prompt/instructions for the routine"
},
"context_paths": {
"type": "array",
"items": { "type": "string" },
"description": "Workspace paths to load as context (e.g. ['context/priorities.md'])"
},
"action_type": {
"type": "string",
"enum": ["lightweight", "full_job"],
"description": "Execution mode: 'lightweight' (single LLM call, default) or 'full_job' (multi-turn with tools)"
},
"cooldown_secs": {
"type": "integer",
"description": "Minimum seconds between fires (default: 300)"
}
},
"required": ["name", "trigger_type", "prompt"]
})
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = params
.get("name")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'name'".to_string()))?;
let description = params
.get("description")
.and_then(|v| v.as_str())
.unwrap_or("");
let trigger_type = params
.get("trigger_type")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'trigger_type'".to_string()))?;
let prompt = params
.get("prompt")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'prompt'".to_string()))?;
// Build trigger
let trigger = match trigger_type {
"cron" => {
let schedule =
params
.get("schedule")
.and_then(|v| v.as_str())
.ok_or_else(|| {
ToolError::InvalidParameters(
"cron trigger requires 'schedule'".to_string(),
)
})?;
// Validate cron expression
next_cron_fire(schedule).map_err(|e| {
ToolError::InvalidParameters(format!("invalid cron schedule: {e}"))
})?;
Trigger::Cron {
schedule: schedule.to_string(),
}
}
"event" => {
let pattern = params
.get("event_pattern")
.and_then(|v| v.as_str())
.ok_or_else(|| {
ToolError::InvalidParameters(
"event trigger requires 'event_pattern'".to_string(),
)
})?;
// Validate regex
regex::Regex::new(pattern)
.map_err(|e| ToolError::InvalidParameters(format!("invalid regex: {e}")))?;
let channel = params
.get("event_channel")
.and_then(|v| v.as_str())
.map(String::from);
Trigger::Event {
channel,
pattern: pattern.to_string(),
}
}
"webhook" => Trigger::Webhook {
path: None,
secret: None,
},
"manual" => Trigger::Manual,
other => {
return Err(ToolError::InvalidParameters(format!(
"unknown trigger_type: {other}"
)));
}
};
// Build action
let action_type = params
.get("action_type")
.and_then(|v| v.as_str())
.unwrap_or("lightweight");
let context_paths: Vec<String> = params
.get("context_paths")
.and_then(|v| v.as_array())
.map(|arr| {
arr.iter()
.filter_map(|v| v.as_str().map(String::from))
.collect()
})
.unwrap_or_default();
let action = match action_type {
"lightweight" => RoutineAction::Lightweight {
prompt: prompt.to_string(),
context_paths,
max_tokens: 4096,
},
"full_job" => RoutineAction::FullJob {
title: name.to_string(),
description: prompt.to_string(),
max_iterations: 10,
},
other => {
return Err(ToolError::InvalidParameters(format!(
"unknown action_type: {other}"
)));
}
};
let cooldown_secs = params
.get("cooldown_secs")
.and_then(|v| v.as_u64())
.unwrap_or(300);
// Compute next fire time for cron
let next_fire = if let Trigger::Cron { ref schedule } = trigger {
next_cron_fire(schedule).unwrap_or(None)
} else {
None
};
let routine = Routine {
id: Uuid::new_v4(),
name: name.to_string(),
description: description.to_string(),
user_id: ctx.user_id.clone(),
enabled: true,
trigger,
action,
guardrails: RoutineGuardrails {
cooldown: Duration::from_secs(cooldown_secs),
max_concurrent: 1,
dedup_window: None,
},
notify: NotifyConfig::default(),
last_run_at: None,
next_fire_at: next_fire,
run_count: 0,
consecutive_failures: 0,
state: serde_json::json!({}),
created_at: Utc::now(),
updated_at: Utc::now(),
};
self.store
.create_routine(&routine)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("failed to create routine: {e}")))?;
// Refresh event cache if this is an event trigger
if routine.trigger.type_tag() == "event" {
self.engine.refresh_event_cache().await;
}
let result = serde_json::json!({
"id": routine.id.to_string(),
"name": routine.name,
"trigger_type": routine.trigger.type_tag(),
"next_fire_at": routine.next_fire_at.map(|t| t.to_rfc3339()),
"status": "created",
});
Ok(ToolOutput::success(result, start.elapsed()))
}
fn requires_sanitization(&self) -> bool {
false
}
}
// ==================== routine_list ====================
pub struct RoutineListTool {
store: Arc<Store>,
}
impl RoutineListTool {
pub fn new(store: Arc<Store>) -> Self {
Self { store }
}
}
#[async_trait]
impl Tool for RoutineListTool {
fn name(&self) -> &str {
"routine_list"
}
fn description(&self) -> &str {
"List all routines with their status, trigger info, and next fire time."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {},
"required": []
})
}
async fn execute(
&self,
_params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let routines = self
.store
.list_routines(&ctx.user_id)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("failed to list routines: {e}")))?;
let list: Vec<serde_json::Value> = routines
.iter()
.map(|r| {
serde_json::json!({
"id": r.id.to_string(),
"name": r.name,
"description": r.description,
"enabled": r.enabled,
"trigger_type": r.trigger.type_tag(),
"action_type": r.action.type_tag(),
"last_run_at": r.last_run_at.map(|t| t.to_rfc3339()),
"next_fire_at": r.next_fire_at.map(|t| t.to_rfc3339()),
"run_count": r.run_count,
"consecutive_failures": r.consecutive_failures,
})
})
.collect();
let result = serde_json::json!({
"count": list.len(),
"routines": list,
});
Ok(ToolOutput::success(result, start.elapsed()))
}
fn requires_sanitization(&self) -> bool {
false
}
}
// ==================== routine_update ====================
pub struct RoutineUpdateTool {
store: Arc<Store>,
engine: Arc<RoutineEngine>,
}
impl RoutineUpdateTool {
pub fn new(store: Arc<Store>, engine: Arc<RoutineEngine>) -> Self {
Self { store, engine }
}
}
#[async_trait]
impl Tool for RoutineUpdateTool {
fn name(&self) -> &str {
"routine_update"
}
fn description(&self) -> &str {
"Update an existing routine. Can modify trigger, prompt, schedule, or toggle enabled state. \
Pass the routine name and only the fields you want to change."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the routine to update"
},
"enabled": {
"type": "boolean",
"description": "Enable or disable the routine"
},
"prompt": {
"type": "string",
"description": "New prompt/instructions"
},
"schedule": {
"type": "string",
"description": "New cron schedule (for cron triggers)"
},
"description": {
"type": "string",
"description": "New description"
}
},
"required": ["name"]
})
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = params
.get("name")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'name'".to_string()))?;
let mut routine = self
.store
.get_routine_by_name(&ctx.user_id, name)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("DB error: {e}")))?
.ok_or_else(|| ToolError::ExecutionFailed(format!("routine '{}' not found", name)))?;
// Apply updates
if let Some(enabled) = params.get("enabled").and_then(|v| v.as_bool()) {
routine.enabled = enabled;
}
if let Some(desc) = params.get("description").and_then(|v| v.as_str()) {
routine.description = desc.to_string();
}
if let Some(prompt) = params.get("prompt").and_then(|v| v.as_str()) {
match &mut routine.action {
RoutineAction::Lightweight { prompt: p, .. } => *p = prompt.to_string(),
RoutineAction::FullJob { description: d, .. } => *d = prompt.to_string(),
}
}
if let Some(schedule) = params.get("schedule").and_then(|v| v.as_str()) {
// Validate
next_cron_fire(schedule)
.map_err(|e| ToolError::InvalidParameters(format!("invalid cron schedule: {e}")))?;
routine.trigger = Trigger::Cron {
schedule: schedule.to_string(),
};
routine.next_fire_at = next_cron_fire(schedule).unwrap_or(None);
}
self.store
.update_routine(&routine)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("failed to update: {e}")))?;
// Refresh event cache in case trigger changed
self.engine.refresh_event_cache().await;
let result = serde_json::json!({
"name": routine.name,
"enabled": routine.enabled,
"trigger_type": routine.trigger.type_tag(),
"next_fire_at": routine.next_fire_at.map(|t| t.to_rfc3339()),
"status": "updated",
});
Ok(ToolOutput::success(result, start.elapsed()))
}
fn requires_sanitization(&self) -> bool {
false
}
}
// ==================== routine_delete ====================
pub struct RoutineDeleteTool {
store: Arc<Store>,
engine: Arc<RoutineEngine>,
}
impl RoutineDeleteTool {
pub fn new(store: Arc<Store>, engine: Arc<RoutineEngine>) -> Self {
Self { store, engine }
}
}
#[async_trait]
impl Tool for RoutineDeleteTool {
fn name(&self) -> &str {
"routine_delete"
}
fn description(&self) -> &str {
"Delete a routine permanently. This also removes all run history."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the routine to delete"
}
},
"required": ["name"]
})
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = params
.get("name")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'name'".to_string()))?;
let routine = self
.store
.get_routine_by_name(&ctx.user_id, name)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("DB error: {e}")))?
.ok_or_else(|| ToolError::ExecutionFailed(format!("routine '{}' not found", name)))?;
let deleted = self
.store
.delete_routine(routine.id)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("failed to delete: {e}")))?;
// Refresh event cache
self.engine.refresh_event_cache().await;
let result = serde_json::json!({
"name": name,
"deleted": deleted,
});
Ok(ToolOutput::success(result, start.elapsed()))
}
fn requires_sanitization(&self) -> bool {
false
}
}
// ==================== routine_history ====================
pub struct RoutineHistoryTool {
store: Arc<Store>,
}
impl RoutineHistoryTool {
pub fn new(store: Arc<Store>) -> Self {
Self { store }
}
}
#[async_trait]
impl Tool for RoutineHistoryTool {
fn name(&self) -> &str {
"routine_history"
}
fn description(&self) -> &str {
"View the execution history of a routine. Shows recent runs with status, duration, and results."
}
fn parameters_schema(&self) -> serde_json::Value {
serde_json::json!({
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the routine"
},
"limit": {
"type": "integer",
"description": "Max runs to return (default: 10)",
"default": 10
}
},
"required": ["name"]
})
}
async fn execute(
&self,
params: serde_json::Value,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let name = params
.get("name")
.and_then(|v| v.as_str())
.ok_or_else(|| ToolError::InvalidParameters("missing 'name'".to_string()))?;
let limit = params
.get("limit")
.and_then(|v| v.as_i64())
.unwrap_or(10)
.min(50);
let routine = self
.store
.get_routine_by_name(&ctx.user_id, name)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("DB error: {e}")))?
.ok_or_else(|| ToolError::ExecutionFailed(format!("routine '{}' not found", name)))?;
let runs = self
.store
.list_routine_runs(routine.id, limit)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("failed to list runs: {e}")))?;
let run_list: Vec<serde_json::Value> = runs
.iter()
.map(|r| {
let duration_secs = r
.completed_at
.map(|c| c.signed_duration_since(r.started_at).num_seconds());
serde_json::json!({
"id": r.id.to_string(),
"trigger_type": r.trigger_type,
"trigger_detail": r.trigger_detail,
"started_at": r.started_at.to_rfc3339(),
"completed_at": r.completed_at.map(|t| t.to_rfc3339()),
"duration_secs": duration_secs,
"status": r.status.to_string(),
"result_summary": r.result_summary,
"tokens_used": r.tokens_used,
})
})
.collect();
let result = serde_json::json!({
"routine": name,
"total_runs": routine.run_count,
"runs": run_list,
});
Ok(ToolOutput::success(result, start.elapsed()))
}
fn requires_sanitization(&self) -> bool {
false
}
}
src/tools/builtin/shell.rs
+5 -1
View File
@@ -30,7 +30,7 @@ use tokio::process::Command;
use crate::context::JobContext;
use crate::sandbox::{SandboxManager, SandboxPolicy};
use crate::tools::tool::{Tool, ToolError, ToolOutput};
use crate::tools::tool::{Tool, ToolDomain, ToolError, ToolOutput};
/// Maximum output size before truncation (64KB).
const MAX_OUTPUT_SIZE: usize = 64 * 1024;
@@ -386,6 +386,10 @@ impl Tool for ShellTool {
fn requires_sanitization(&self) -> bool {
true // Shell output could contain anything
}
fn domain(&self) -> ToolDomain {
ToolDomain::Container
}
}
/// Truncate output to fit within limits.
src/tools/mcp/config.rs
+80
View File
@@ -327,6 +327,86 @@ pub async fn get_mcp_server(name: &str) -> Result<McpServerConfig, ConfigError>
})
}
// ==================== Database-backed MCP server config ====================
/// Load MCP server configurations from the database settings table.
///
/// Falls back to the disk file if DB has no entry.
pub async fn load_mcp_servers_from_db(
store: &crate::history::Store,
user_id: &str,
) -> Result<McpServersFile, ConfigError> {
match store.get_setting(user_id, "mcp_servers").await {
Ok(Some(value)) => {
let config: McpServersFile = serde_json::from_value(value)?;
Ok(config)
}
Ok(None) => {
// No entry in DB, fall back to disk
load_mcp_servers().await
}
Err(e) => {
tracing::warn!(
"Failed to load MCP servers from DB: {}, falling back to disk",
e
);
load_mcp_servers().await
}
}
}
/// Save MCP server configurations to the database settings table.
pub async fn save_mcp_servers_to_db(
store: &crate::history::Store,
user_id: &str,
config: &McpServersFile,
) -> Result<(), ConfigError> {
let value = serde_json::to_value(config)?;
store
.set_setting(user_id, "mcp_servers", &value)
.await
.map_err(|e| {
ConfigError::Io(std::io::Error::new(
std::io::ErrorKind::Other,
e.to_string(),
))
})?;
Ok(())
}
/// Add a new MCP server configuration (DB-backed).
pub async fn add_mcp_server_db(
store: &crate::history::Store,
user_id: &str,
config: McpServerConfig,
) -> Result<(), ConfigError> {
config.validate()?;
let mut servers = load_mcp_servers_from_db(store, user_id).await?;
servers.upsert(config);
save_mcp_servers_to_db(store, user_id, &servers).await?;
Ok(())
}
/// Remove an MCP server by name (DB-backed).
pub async fn remove_mcp_server_db(
store: &crate::history::Store,
user_id: &str,
name: &str,
) -> Result<(), ConfigError> {
let mut servers = load_mcp_servers_from_db(store, user_id).await?;
if !servers.remove(name) {
return Err(ConfigError::ServerNotFound {
name: name.to_string(),
});
}
save_mcp_servers_to_db(store, user_id, &servers).await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
src/tools/mod.rs
+1 -1
View File
@@ -23,4 +23,4 @@ pub use builder::{
};
pub use registry::ToolRegistry;
pub use sandbox::ToolSandbox;
pub use tool::{Tool, ToolError, ToolOutput};
pub use tool::{Tool, ToolDomain, ToolError, ToolOutput};
src/tools/registry.rs
+79 -4
View File
@@ -7,7 +7,9 @@ use tokio::sync::RwLock;
use crate::context::ContextManager;
use crate::extensions::ExtensionManager;
use crate::history::Store;
use crate::llm::{LlmProvider, ToolDefinition};
use crate::orchestrator::job_manager::ContainerJobManager;
use crate::safety::SafetyLayer;
use crate::tools::builder::{BuildSoftwareTool, BuilderConfig, LlmSoftwareBuilder};
use crate::tools::builtin::{
@@ -16,7 +18,7 @@ use crate::tools::builtin::{
ReadFileTool, ShellTool, TimeTool, ToolActivateTool, ToolAuthTool, ToolInstallTool,
ToolListTool, ToolRemoveTool, ToolSearchTool, WriteFileTool,
};
use crate::tools::tool::Tool;
use crate::tools::tool::{Tool, ToolDomain};
use crate::tools::wasm::{
Capabilities, ResourceLimits, WasmError, WasmStorageError, WasmToolRuntime, WasmToolStore,
WasmToolWrapper,
@@ -120,6 +122,39 @@ impl ToolRegistry {
tracing::info!("Registered {} built-in tools", self.count());
}
/// Register only orchestrator-domain tools (safe for the main process).
///
/// This registers tools that don't touch the filesystem or run shell commands:
/// echo, time, json, http. Use this when `allow_local_tools = false` and
/// container-domain tools should only be available inside sandboxed containers.
pub fn register_orchestrator_tools(&self) {
self.register_builtin_tools();
// register_builtin_tools already only registers orchestrator-domain tools
}
/// Register container-domain tools (filesystem, shell, code).
///
/// These tools are intended to run inside sandboxed Docker containers.
/// Call this in the worker process, not the orchestrator (unless `allow_local_tools = true`).
pub fn register_container_tools(&self) {
self.register_dev_tools();
}
/// Get tool definitions filtered by domain.
pub async fn tool_definitions_for_domain(&self, domain: ToolDomain) -> Vec<ToolDefinition> {
self.tools
.read()
.await
.values()
.filter(|tool| tool.domain() == domain)
.map(|tool| ToolDefinition {
name: tool.name().to_string(),
description: tool.description().to_string(),
parameters: tool.parameters_schema(),
})
.collect()
}
/// Register development tools for building software.
///
/// These tools provide shell access, file operations, and code editing
@@ -151,9 +186,19 @@ impl ToolRegistry {
/// Register job management tools.
///
/// Job tools allow the LLM to create, list, check status, and cancel jobs.
/// These enable natural language job management without hardcoded intent parsing.
pub fn register_job_tools(&self, context_manager: Arc<ContextManager>) {
self.register_sync(Arc::new(CreateJobTool::new(Arc::clone(&context_manager))));
/// When sandbox deps are provided, `create_job` automatically delegates to
/// Docker containers. Otherwise it creates in-memory jobs via ContextManager.
pub fn register_job_tools(
&self,
context_manager: Arc<ContextManager>,
job_manager: Option<Arc<ContainerJobManager>>,
store: Option<Arc<Store>>,
) {
let mut create_tool = CreateJobTool::new(Arc::clone(&context_manager));
if let Some(jm) = job_manager {
create_tool = create_tool.with_sandbox(jm, store);
}
self.register_sync(Arc::new(create_tool));
self.register_sync(Arc::new(ListJobsTool::new(Arc::clone(&context_manager))));
self.register_sync(Arc::new(JobStatusTool::new(Arc::clone(&context_manager))));
self.register_sync(Arc::new(CancelJobTool::new(context_manager)));
@@ -174,6 +219,36 @@ impl ToolRegistry {
tracing::info!("Registered 6 extension management tools");
}
/// Register routine management tools.
///
/// These allow the LLM to create, list, update, delete, and view history
/// of routines (scheduled and event-driven tasks).
pub fn register_routine_tools(
&self,
store: Arc<Store>,
engine: Arc<crate::agent::routine_engine::RoutineEngine>,
) {
use crate::tools::builtin::{
RoutineCreateTool, RoutineDeleteTool, RoutineHistoryTool, RoutineListTool,
RoutineUpdateTool,
};
self.register_sync(Arc::new(RoutineCreateTool::new(
Arc::clone(&store),
Arc::clone(&engine),
)));
self.register_sync(Arc::new(RoutineListTool::new(Arc::clone(&store))));
self.register_sync(Arc::new(RoutineUpdateTool::new(
Arc::clone(&store),
Arc::clone(&engine),
)));
self.register_sync(Arc::new(RoutineDeleteTool::new(
Arc::clone(&store),
Arc::clone(&engine),
)));
self.register_sync(Arc::new(RoutineHistoryTool::new(store)));
tracing::info!("Registered 5 routine management tools");
}
/// Register the software builder tool.
///
/// The builder tool allows the agent to create new software including WASM tools,
src/tools/tool.rs
+35
View File
@@ -9,6 +9,18 @@ use thiserror::Error;
use crate::context::JobContext;
/// Where a tool should execute: orchestrator process or inside a container.
///
/// Orchestrator tools run in the main agent process (memory access, job mgmt, etc).
/// Container tools run inside Docker containers (shell, file ops, code mods).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum ToolDomain {
/// Safe to run in the orchestrator (pure functions, memory, job management).
Orchestrator,
/// Must run inside a sandboxed container (filesystem, shell, code).
Container,
}
/// Error type for tool execution.
#[derive(Debug, Error)]
pub enum ToolError {
@@ -160,6 +172,23 @@ pub trait Tool: Send + Sync {
false
}
/// Maximum time this tool is allowed to run before the caller kills it.
/// Override for long-running tools like sandbox execution.
/// Default: 60 seconds.
fn execution_timeout(&self) -> Duration {
Duration::from_secs(60)
}
/// Where this tool should execute.
///
/// `Orchestrator` tools run in the main agent process (safe, no FS access).
/// `Container` tools run inside Docker containers (shell, file ops).
///
/// Default: `Orchestrator` (safe for the main process).
fn domain(&self) -> ToolDomain {
ToolDomain::Orchestrator
}
/// Get the tool schema for LLM function calling.
fn schema(&self) -> ToolSchema {
ToolSchema {
@@ -242,4 +271,10 @@ mod tests {
assert_eq!(schema.name, "echo");
assert!(!schema.description.is_empty());
}
#[test]
fn test_execution_timeout_default() {
let tool = EchoTool;
assert_eq!(tool.execution_timeout(), Duration::from_secs(60));
}
}
src/tools/wasm/loader.rs
+188
View File
@@ -2,6 +2,7 @@
//!
//! This module provides a way to load WASM tools dynamically at runtime from:
//! - A directory containing `<name>.wasm` and `<name>.capabilities.json`
//! - Build artifacts in `tools-src/` (dev mode, auto-detected)
//! - Database storage (via [`WasmToolStore`])
//!
//! # Example: Loading from Directory
@@ -19,6 +20,13 @@
//! loader.load_from_dir(Path::new("~/.ironclaw/tools/")).await?;
//! ```
//!
//! # Dev Mode
//!
//! When `load_dev_tools()` is called, the loader scans `tools-src/*/` for build
//! artifacts. Tools found there are loaded directly from the build output,
//! skipping the install directory. This means during development you just
//! rebuild the WASM and restart the host, no manual copy step needed.
//!
//! # Security
//!
//! Tools loaded from files are assigned `TrustLevel::User` by default, meaning
@@ -312,6 +320,159 @@ impl LoadResults {
}
}
/// Compile-time project root, used to locate tools-src/ in dev builds.
const CARGO_MANIFEST_DIR: &str = env!("CARGO_MANIFEST_DIR");
/// Resolve the tools source directory.
///
/// Checks (in order):
/// 1. `IRONCLAW_TOOLS_SRC` env var
/// 2. `<CARGO_MANIFEST_DIR>/tools-src/` (dev builds)
fn tools_src_dir() -> PathBuf {
if let Ok(dir) = std::env::var("IRONCLAW_TOOLS_SRC") {
return PathBuf::from(dir);
}
PathBuf::from(CARGO_MANIFEST_DIR).join("tools-src")
}
/// Discover WASM tools available as build artifacts in `tools-src/`.
///
/// Scans each subdirectory for:
/// - `tools-src/<name>/target/wasm32-wasip2/release/<crate_name>_tool.wasm`
/// - `tools-src/<name>/<name>-tool.capabilities.json`
///
/// Returns a map of install-name (e.g. "gmail-tool") to paths.
pub async fn discover_dev_tools() -> Result<HashMap<String, DiscoveredTool>, std::io::Error> {
let src_dir = tools_src_dir();
let mut tools = HashMap::new();
if !src_dir.is_dir() {
return Ok(tools);
}
let mut entries = fs::read_dir(&src_dir).await?;
while let Some(entry) = entries.next_entry().await? {
let path = entry.path();
if !path.is_dir() {
continue;
}
let dir_name = match path.file_name().and_then(|n| n.to_str()) {
Some(n) => n.to_string(),
None => continue,
};
// Convention: crate name uses underscores, directory uses hyphens
let crate_name = dir_name.replace('-', "_");
let install_name = format!("{}-tool", dir_name);
let wasm_path = path
.join("target/wasm32-wasip2/release")
.join(format!("{}_tool.wasm", crate_name));
if !wasm_path.exists() {
continue;
}
let caps_path = path.join(format!("{}-tool.capabilities.json", dir_name));
tools.insert(
install_name,
DiscoveredTool {
wasm_path,
capabilities_path: if caps_path.exists() {
Some(caps_path)
} else {
None
},
},
);
}
Ok(tools)
}
/// Load WASM tools from build artifacts in `tools-src/`.
///
/// In dev mode, tools can be loaded directly from their build output without
/// needing to install them to `~/.ironclaw/tools/` first. Build artifacts
/// that are newer than installed copies take priority.
///
/// Set `IRONCLAW_TOOLS_SRC` env var to override the source directory.
pub async fn load_dev_tools(
loader: &WasmToolLoader,
install_dir: &Path,
) -> Result<LoadResults, WasmLoadError> {
let dev_tools = discover_dev_tools().await?;
let mut results = LoadResults::default();
if dev_tools.is_empty() {
return Ok(results);
}
for (name, discovered) in &dev_tools {
// Check if the build artifact is newer than the installed copy
let installed_path = install_dir.join(format!("{}.wasm", name));
let should_load = if installed_path.exists() {
// Compare modification times: prefer fresher build artifact
match (
fs::metadata(&discovered.wasm_path).await,
fs::metadata(&installed_path).await,
) {
(Ok(dev_meta), Ok(inst_meta)) => {
let dev_modified = dev_meta.modified().unwrap_or(std::time::UNIX_EPOCH);
let inst_modified = inst_meta.modified().unwrap_or(std::time::UNIX_EPOCH);
dev_modified > inst_modified
}
_ => true,
}
} else {
true
};
if !should_load {
continue;
}
tracing::info!(
name = name,
wasm_path = %discovered.wasm_path.display(),
"Loading dev tool from build artifacts (newer than installed)"
);
match loader
.load_from_files(
name,
&discovered.wasm_path,
discovered.capabilities_path.as_deref(),
)
.await
{
Ok(()) => {
results.loaded.push(name.clone());
}
Err(e) => {
tracing::error!(
name = name,
error = %e,
"Failed to load dev tool"
);
results.errors.push((discovered.wasm_path.clone(), e));
}
}
}
if !results.loaded.is_empty() {
tracing::info!(
count = results.loaded.len(),
tools = ?results.loaded,
"Loaded dev tools from build artifacts"
);
}
Ok(results)
}
/// Discover WASM tool files in a directory without loading them.
///
/// Returns a map of tool name -> (wasm_path, capabilities_path).
@@ -430,4 +591,31 @@ mod tests {
let err = WasmLoadError::WasmNotFound(std::path::PathBuf::from("/foo/bar.wasm"));
assert!(err.to_string().contains("/foo/bar.wasm"));
}
#[test]
fn test_tools_src_dir_default() {
let dir = super::tools_src_dir();
assert!(dir.ends_with("tools-src"));
}
#[tokio::test]
async fn test_discover_dev_tools_finds_build_artifacts() {
// This test relies on the actual tools-src/ directory in the repo.
// If build artifacts exist, they should be discovered.
let tools = super::discover_dev_tools().await.unwrap();
// If any tools have been built, they should appear with "-tool" suffix
for (name, discovered) in &tools {
assert!(
name.ends_with("-tool"),
"Dev tool name should end with -tool: {}",
name
);
assert!(
discovered.wasm_path.exists(),
"WASM should exist: {:?}",
discovered.wasm_path
);
}
}
}
src/tools/wasm/mod.rs
+4 -1
View File
@@ -115,7 +115,10 @@ pub use storage::{
};
// Loader
pub use loader::{DiscoveredTool, LoadResults, WasmLoadError, WasmToolLoader, discover_tools};
pub use loader::{
DiscoveredTool, LoadResults, WasmLoadError, WasmToolLoader, discover_dev_tools, discover_tools,
load_dev_tools,
};
// Capabilities schema (for parsing *.capabilities.json files)
pub use capabilities_schema::{
src/tools/wasm/wrapper.rs
+306 -154
View File
@@ -1,16 +1,23 @@
//! WASM tool wrapper implementing the Tool trait.
//!
//! Uses wasmtime::component::bindgen! to generate typed bindings from the WIT
//! interface, ensuring all host functions are properly registered under the
//! correct `near:agent/host` namespace.
//!
//! Each execution creates a fresh instance (NEAR pattern) to ensure
//! isolation and deterministic behavior.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
use async_trait::async_trait;
use wasmtime::Store;
use wasmtime::component::{Component, Linker, Val};
use wasmtime::component::{Component, Linker};
use wasmtime_wasi::{ResourceTable, WasiCtx, WasiCtxBuilder, WasiView};
use crate::context::JobContext;
use crate::safety::LeakDetector;
use crate::tools::tool::{Tool, ToolError, ToolOutput};
use crate::tools::wasm::capabilities::Capabilities;
use crate::tools::wasm::error::WasmError;
@@ -18,21 +25,259 @@ use crate::tools::wasm::host::{HostState, LogLevel};
use crate::tools::wasm::limits::{ResourceLimits, WasmResourceLimiter};
use crate::tools::wasm::runtime::{PreparedModule, WasmToolRuntime};
/// Store data for WASM execution.
// Generate component model bindings from the WIT file.
//
// This creates:
// - `near::agent::host::Host` trait + `add_to_linker()` for the import interface
// - `SandboxedTool` struct with `instantiate()` for the world
// - `exports::near::agent::tool::*` types for the export interface
wasmtime::component::bindgen!({
path: "wit/tool.wit",
world: "sandboxed-tool",
async: false,
with: {},
});
// Alias the export interface types for convenience.
use exports::near::agent::tool as wit_tool;
/// Store data for WASM tool execution.
///
/// Contains both the resource limiter and host state.
/// Contains the resource limiter, host state, WASI context, and injected
/// credentials. Fresh instance created per execution (NEAR pattern).
struct StoreData {
limiter: WasmResourceLimiter,
host_state: HostState,
wasi: WasiCtx,
table: ResourceTable,
/// Injected credentials for URL/header substitution.
/// Keys are placeholder names like "GOOGLE_ACCESS_TOKEN".
credentials: HashMap<String, String>,
}
impl StoreData {
fn new(memory_limit: u64, capabilities: Capabilities) -> Self {
fn new(
memory_limit: u64,
capabilities: Capabilities,
credentials: HashMap<String, String>,
) -> Self {
// Minimal WASI context: no filesystem, no env vars (security)
let wasi = WasiCtxBuilder::new().build();
Self {
limiter: WasmResourceLimiter::new(memory_limit),
host_state: HostState::new(capabilities),
wasi,
table: ResourceTable::new(),
credentials,
}
}
/// Inject credentials into a string by replacing placeholders.
///
/// Replaces patterns like `{GOOGLE_ACCESS_TOKEN}` with actual values.
/// WASM tools reference credentials by placeholder, never seeing real values.
fn inject_credentials(&self, input: &str, context: &str) -> String {
let mut result = input.to_string();
for (name, value) in &self.credentials {
let placeholder = format!("{{{}}}", name);
if result.contains(&placeholder) {
tracing::debug!(
placeholder = %placeholder,
context = %context,
"Replacing credential placeholder in tool request"
);
result = result.replace(&placeholder, value);
}
}
result
}
/// Replace injected credential values with `[REDACTED]` in text.
///
/// Prevents credentials from leaking through error messages or logs.
/// reqwest::Error includes the full URL in its Display output, so any
/// error from an injected-URL request will contain the raw credential
/// unless we scrub it.
fn redact_credentials(&self, text: &str) -> String {
let mut result = text.to_string();
for (name, value) in &self.credentials {
if !value.is_empty() {
result = result.replace(value, &format!("[REDACTED:{}]", name));
}
}
result
}
}
// Provide WASI context for the WASM component.
// Required because tools are compiled with wasm32-wasip2 target.
impl WasiView for StoreData {
fn ctx(&mut self) -> &mut WasiCtx {
&mut self.wasi
}
fn table(&mut self) -> &mut ResourceTable {
&mut self.table
}
}
// Implement the generated Host trait from bindgen.
//
// This registers all 6 host functions under the `near:agent/host` namespace:
// log, now-millis, workspace-read, http-request, secret-exists, tool-invoke
impl near::agent::host::Host for StoreData {
fn log(&mut self, level: near::agent::host::LogLevel, message: String) {
let log_level = match level {
near::agent::host::LogLevel::Trace => LogLevel::Trace,
near::agent::host::LogLevel::Debug => LogLevel::Debug,
near::agent::host::LogLevel::Info => LogLevel::Info,
near::agent::host::LogLevel::Warn => LogLevel::Warn,
near::agent::host::LogLevel::Error => LogLevel::Error,
};
let _ = self.host_state.log(log_level, message);
}
fn now_millis(&mut self) -> u64 {
self.host_state.now_millis()
}
fn workspace_read(&mut self, path: String) -> Option<String> {
self.host_state.workspace_read(&path).ok().flatten()
}
fn http_request(
&mut self,
method: String,
url: String,
headers_json: String,
body: Option<Vec<u8>>,
timeout_ms: Option<u32>,
) -> Result<near::agent::host::HttpResponse, String> {
// Inject credentials into URL (e.g., replace {TELEGRAM_BOT_TOKEN})
let injected_url = self.inject_credentials(&url, "url");
// Check HTTP allowlist
self.host_state
.check_http_allowed(&injected_url, &method)
.map_err(|e| format!("HTTP not allowed: {}", e))?;
// Record for rate limiting
self.host_state
.record_http_request()
.map_err(|e| format!("Rate limit exceeded: {}", e))?;
// Parse headers and inject credentials into header values
let raw_headers: HashMap<String, String> =
serde_json::from_str(&headers_json).unwrap_or_default();
let headers: HashMap<String, String> = raw_headers
.into_iter()
.map(|(k, v)| {
(
k.clone(),
self.inject_credentials(&v, &format!("header:{}", k)),
)
})
.collect();
let url = injected_url;
let leak_detector = LeakDetector::new();
let header_vec: Vec<(String, String)> = headers
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
leak_detector
.scan_http_request(&url, &header_vec, body.as_deref())
.map_err(|e| format!("Potential secret leak blocked: {}", e))?;
// Make HTTP request using blocking I/O.
// We're inside a spawn_blocking context, so use block_on.
let result = tokio::runtime::Handle::current().block_on(async {
let client = reqwest::Client::new();
let mut request = match method.to_uppercase().as_str() {
"GET" => client.get(&url),
"POST" => client.post(&url),
"PUT" => client.put(&url),
"DELETE" => client.delete(&url),
"PATCH" => client.patch(&url),
"HEAD" => client.head(&url),
_ => return Err(format!("Unsupported HTTP method: {}", method)),
};
for (key, value) in headers {
request = request.header(&key, &value);
}
if let Some(body_bytes) = body {
request = request.body(body_bytes);
}
// Caller-specified timeout (default 30s)
let timeout = Duration::from_millis(timeout_ms.unwrap_or(30_000) as u64);
let response = request.timeout(timeout).send().await.map_err(|e| {
// Walk the full error chain for the actual root cause
let mut chain = format!("HTTP request failed: {}", e);
let mut source = std::error::Error::source(&e);
while let Some(cause) = source {
chain.push_str(&format!(" -> {}", cause));
source = cause.source();
}
chain
})?;
let status = response.status().as_u16();
let response_headers: HashMap<String, String> = response
.headers()
.iter()
.filter_map(|(k, v)| {
v.to_str()
.ok()
.map(|v| (k.as_str().to_string(), v.to_string()))
})
.collect();
let headers_json = serde_json::to_string(&response_headers).unwrap_or_default();
let body = response
.bytes()
.await
.map_err(|e| format!("Failed to read response body: {}", e))?
.to_vec();
// Leak detection on response body
if let Ok(body_str) = std::str::from_utf8(&body) {
leak_detector
.scan_and_clean(body_str)
.map_err(|e| format!("Potential secret leak in response: {}", e))?;
}
Ok(near::agent::host::HttpResponse {
status,
headers_json,
body,
})
});
// Redact credentials from error messages before returning to WASM
result.map_err(|e| self.redact_credentials(&e))
}
fn tool_invoke(&mut self, alias: String, _params_json: String) -> Result<String, String> {
// Validate capability and resolve alias
let _real_name = self.host_state.check_tool_invoke_allowed(&alias)?;
self.host_state.record_tool_invoke()?;
// Tool invocation requires async context and access to the tool registry,
// which aren't available inside a synchronous WASM callback.
Err("Tool invocation from WASM tools is not yet supported".to_string())
}
fn secret_exists(&mut self, name: String) -> bool {
self.host_state.secret_exists(&name)
}
}
/// A Tool implementation backed by a WASM component.
@@ -49,6 +294,9 @@ pub struct WasmToolWrapper {
description: String,
/// Cached schema (from PreparedModule or override).
schema: serde_json::Value,
/// Injected credentials for HTTP requests (e.g., OAuth tokens).
/// Keys are placeholder names like "GOOGLE_ACCESS_TOKEN".
credentials: HashMap<String, String>,
}
impl WasmToolWrapper {
@@ -64,6 +312,7 @@ impl WasmToolWrapper {
runtime,
prepared,
capabilities,
credentials: HashMap::new(),
}
}
@@ -79,11 +328,34 @@ impl WasmToolWrapper {
self
}
/// Set credentials for HTTP request injection.
pub fn with_credentials(mut self, credentials: HashMap<String, String>) -> Self {
self.credentials = credentials;
self
}
/// Get the resource limits for this tool.
pub fn limits(&self) -> &ResourceLimits {
&self.prepared.limits
}
/// Add all host functions to the linker using generated bindings.
///
/// Uses the bindgen-generated `add_to_linker` function to properly register
/// all host functions with correct component model signatures under the
/// `near:agent/host` namespace.
fn add_host_functions(linker: &mut Linker<StoreData>) -> Result<(), WasmError> {
// Add WASI support (required by components built with wasm32-wasip2)
wasmtime_wasi::add_to_linker_sync(linker)
.map_err(|e| WasmError::ConfigError(format!("Failed to add WASI functions: {}", e)))?;
// Add our custom host interface using the generated add_to_linker
near::agent::host::add_to_linker(linker, |state| state)
.map_err(|e| WasmError::ConfigError(format!("Failed to add host functions: {}", e)))?;
Ok(())
}
/// Execute the WASM tool synchronously (called from spawn_blocking).
fn execute_sync(
&self,
@@ -94,7 +366,11 @@ impl WasmToolWrapper {
let limits = &self.prepared.limits;
// Create store with fresh state (NEAR pattern: fresh instance per call)
let store_data = StoreData::new(limits.memory_bytes, self.capabilities.clone());
let store_data = StoreData::new(
limits.memory_bytes,
self.capabilities.clone(),
self.credentials.clone(),
);
let mut store = Store::new(engine, store_data);
// Configure fuel if enabled
@@ -115,172 +391,46 @@ impl WasmToolWrapper {
let component = Component::new(engine, self.prepared.component_bytes())
.map_err(|e| WasmError::CompilationFailed(e.to_string()))?;
// Create linker and add host functions
// Create linker with all host functions properly namespaced
let mut linker = Linker::new(engine);
self.add_host_functions(&mut linker)?;
Self::add_host_functions(&mut linker)?;
// Instantiate the component
let instance = linker
.instantiate(&mut store, &component)
// Instantiate using the generated bindings
let instance = SandboxedTool::instantiate(&mut store, &component, &linker)
.map_err(|e| WasmError::InstantiationFailed(e.to_string()))?;
// Get the execute function
let execute_func = instance
.get_func(&mut store, "execute")
.ok_or_else(|| WasmError::MissingExport("execute".to_string()))?;
// Prepare request
// Prepare the request
let params_json = serde_json::to_string(&params)
.map_err(|e| WasmError::InvalidResponseJson(e.to_string()))?;
// Build request record
// Note: The exact calling convention depends on how WIT records are lowered.
// With component model, we'd use typed bindings from wit-bindgen.
// For now, we use the lower-level Val API.
let request_params = Val::String(params_json);
let request_context = match context_json {
Some(ctx) => Val::Option(Some(Box::new(Val::String(ctx)))),
None => Val::Option(None),
let request = wit_tool::Request {
params: params_json,
context: context_json,
};
// Create request record (params, context)
let request = Val::Record(vec![
("params".to_string(), request_params),
("context".to_string(), request_context),
]);
// Call the function
let mut results = vec![Val::Bool(false)]; // Placeholder for response
execute_func
.call(&mut store, &[request], &mut results)
.map_err(|e| {
// Check for specific trap types
let error_str = e.to_string();
if error_str.contains("out of fuel") {
WasmError::FuelExhausted { limit: limits.fuel }
} else if error_str.contains("unreachable") {
WasmError::Trapped("unreachable code executed".to_string())
} else {
WasmError::Trapped(error_str)
}
})?;
// Post-call completion (cleanup)
execute_func
.post_return(&mut store)
.map_err(|e| WasmError::Trapped(format!("post_return failed: {}", e)))?;
// Extract response
let response = &results[0];
let (result_str, error_str) = extract_response(response)?;
// Call execute using the generated typed interface
let tool_iface = instance.near_agent_tool();
let response = tool_iface.call_execute(&mut store, &request).map_err(|e| {
let error_str = e.to_string();
if error_str.contains("out of fuel") {
WasmError::FuelExhausted { limit: limits.fuel }
} else if error_str.contains("unreachable") {
WasmError::Trapped("unreachable code executed".to_string())
} else {
WasmError::Trapped(error_str)
}
})?;
// Get logs from host state
let logs = store.data_mut().host_state.take_logs();
// Check for tool-level error
if let Some(err) = error_str {
if let Some(err) = response.error {
return Err(WasmError::ToolReturnedError(err));
}
// Return result (or empty string if none)
Ok((result_str.unwrap_or_default(), logs))
}
/// Add host functions to the linker.
fn add_host_functions(&self, linker: &mut Linker<StoreData>) -> Result<(), WasmError> {
// Note: With WIT bindgen, these would be generated automatically.
// For now, we manually define the host functions.
//
// Component model func_wrap signature: F: Fn(StoreContextMut<T>, Params) -> Result<Return>
// where Params is a tuple of the function arguments.
// host.log(level: log-level, message: string)
linker
.root()
.func_wrap(
"log",
|mut ctx: wasmtime::StoreContextMut<'_, StoreData>,
(level, message): (i32, String)| {
let log_level = match level {
0 => LogLevel::Trace,
1 => LogLevel::Debug,
2 => LogLevel::Info,
3 => LogLevel::Warn,
4 => LogLevel::Error,
_ => LogLevel::Info,
};
// Ignore errors from logging (rate limiting)
let _ = ctx.data_mut().host_state.log(log_level, message);
Ok(())
},
)
.map_err(|e| WasmError::ConfigError(format!("Failed to add log function: {}", e)))?;
// host.now-millis() -> u64
linker
.root()
.func_wrap(
"now-millis",
|ctx: wasmtime::StoreContextMut<'_, StoreData>, (): ()| -> anyhow::Result<(u64,)> {
Ok((ctx.data().host_state.now_millis(),))
},
)
.map_err(|e| {
WasmError::ConfigError(format!("Failed to add now-millis function: {}", e))
})?;
// host.workspace-read(path: string) -> option<string>
linker
.root()
.func_wrap(
"workspace-read",
|ctx: wasmtime::StoreContextMut<'_, StoreData>,
(path,): (String,)|
-> anyhow::Result<(Option<String>,)> {
let result = ctx.data().host_state.workspace_read(&path).ok().flatten();
Ok((result,))
},
)
.map_err(|e| {
WasmError::ConfigError(format!("Failed to add workspace-read function: {}", e))
})?;
Ok(())
}
}
/// Extract result and error from a WIT response record.
fn extract_response(response: &Val) -> Result<(Option<String>, Option<String>), WasmError> {
match response {
Val::Record(fields) => {
let mut result = None;
let mut error = None;
for (name, val) in fields {
match name.as_str() {
"output" => {
if let Val::Option(Some(inner)) = val {
if let Val::String(s) = inner.as_ref() {
result = Some(s.to_string());
}
}
}
"error" => {
if let Val::Option(Some(inner)) = val {
if let Val::String(s) = inner.as_ref() {
error = Some(s.to_string());
}
}
}
_ => {}
}
}
Ok((result, error))
}
_ => Err(WasmError::InvalidResponseJson(
"Expected record response".to_string(),
)),
Ok((response.output.unwrap_or_default(), logs))
}
}
@@ -315,6 +465,7 @@ impl Tool for WasmToolWrapper {
let capabilities = self.capabilities.clone();
let description = self.description.clone();
let schema = self.schema.clone();
let credentials = self.credentials.clone();
// Execute in blocking task with timeout
let result = tokio::time::timeout(timeout, async move {
@@ -324,6 +475,7 @@ impl Tool for WasmToolWrapper {
capabilities,
description,
schema,
credentials,
};
tokio::task::spawn_blocking(move || wrapper.execute_sync(params, context_json))
@@ -359,7 +511,7 @@ impl Tool for WasmToolWrapper {
}
fn requires_sanitization(&self) -> bool {
// WASM tools always require sanitization - they're untrusted by definition
// WASM tools always require sanitization, they're untrusted by definition
true
}