Compare commits

..
Author SHA1 Message Date
gitops 3bece7ec47 updated 2026-03-30 19:20:47 +08:00
gitops 3c4095c7a1 mesh and xray 2026-03-30 07:55:53 +08:00
outbackdingoandGitHub 7e2ad874bd Merge branch 'nearai:staging' into staging 2026-03-29 13:24:47 +07:00
dingoandClaude Opus 4.6 26b1c5b493 docs: mesh cluster documentation, SVG diagrams, and ironclaw->optimclaw rename
- MESH_CLUSTER.md: full documentation for autonomous AI mesh network
- LAZY_TOOLS.md: lazy tool loading for smaller LLMs
- mesh-architecture.svg: colorful network topology diagram
- task-routing.svg: scoring algorithm visualization
- pq-handshake.svg: ML-KEM-768 key exchange sequence diagram
- HTML docs with dark theme styling
- Renamed ironclaw references to optimclaw in .env.example and README

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-29 13:16:20 +07:00
firat.sertgozandGitHub e0e530e646 docs: tighten contribution and PR guidance (#1704) 2026-03-29 09:11:58 +03:00
dingoandClaude Opus 4.6 93c27fa053 feat: autonomous AI mesh network with post-quantum crypto
Add peer-to-peer mesh cluster module (feature-gated behind `cluster`):

- UDP beacon discovery (port 9900) for zero-config LAN auto-discovery
- ML-KEM-768 (Kyber) post-quantum key exchange + AES-256-GCM encrypted
  WebSocket overlay mesh
- SWIM gossip protocol for membership and failure detection
- Intelligent task routing: scores nodes by load, VRAM, model match,
  hop distance
- Remote task execution via subprocess with streaming results
- REST API endpoints: /api/mesh/status, /api/mesh/nodes
- ed25519 signed beacons, persistent keypairs (~/.optimclaw/mesh_keys.json)
- Lazy tool loading (OPTIMCLAW_LAZY_TOOLS) to reduce system prompt size

Enable with: CLUSTER_ENABLED=1 optimclaw run

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-29 13:10:35 +07:00
a8e83210ff feat(discord): add gateway channel flow in wasm (#944)
* feat(discord): restore gateway channel flow in wasm

* chore(discord): bump channel version to 0.2.1

* fix(discord): address review feedback on gateway channel PR

- Add #[serde(default)] to DiscordMessageMetadata for backward compat
  with old Option<String> serialized metadata
- Restore mention polling alongside Gateway (on_poll processes gateway
  events first, then runs poll_for_mentions if configured)
- Update on_respond to handle source_message_id with message_reference
  for mention-poll reply threading
- Implement Gateway presence status: dnd before pairing, online after
- Implement Gateway resume (OP 6) with session_id tracking, falling
  back to fresh identify on Invalid Session (OP 9)
- Extract WebsocketSessionState and spawn_websocket_poll to reduce
  nesting in start_websocket_runtime
- Simplify should_apply_dm_pairing tautology
- Remove completed plan docs
- Fix clippy items_after_test_module in extensions handler

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(discord): address review findings in gateway channel PR

- Fix gateway presence always showing "online" by filtering empty
  owner_id strings from workspace store reads
- Fix interaction followup using POST instead of PATCH to
  /messages/@original, which left deferred "thinking" state unresolved
- Restore mention-poll pagination (up to 5 pages of 100 messages)
- Remove dead ed25519-dalek and hex dependencies from WASM crate
- Remove unused _channel_id parameter from remember_processed_id
- Clean up redundant let binding in send_pairing_reply

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(discord): address second-round review findings

- Log warning when gateway event queue JSON fails to deserialize
  instead of silently returning empty (zmanian review item 1)
- Defer presence update from OP 10 Hello to after OP 0 READY, per
  Discord gateway protocol which requires READY before non-Identify
  commands (zmanian review item 2)
- Add 0-25% random jitter to websocket reconnect backoff per Discord's
  reconnection recommendations (zmanian suggestion)
- Extract WebsocketPollContext struct to replace 19-parameter
  spawn_websocket_poll function (zmanian suggestion)
- Document intent bitmask 4609 = GUILDS + GUILD_MESSAGES +
  DIRECT_MESSAGES in capabilities JSON (zmanian suggestion)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: zhyaoyu <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 22:20:40 -07:00
OutBack Dingo 6d9dbbb3b9 fork: rename IronClaw → OptimClaw
Full rename of all identifiers, filenames, and references:
  ironclaw → optimclaw
  IronClaw → OptimClaw
  IRONCLAW → OPTIMCLAW
  ironclaw_common → optimclaw_common
  ironclaw_safety → optimclaw_safety

Upstream: nearai/ironclaw
2026-03-29 06:27:52 +07:00
8a320ae9db fix(routines): complete full_job execution reliability overhaul (#1650)
* fix(routines): persist full LLM transcript and remove sandbox gate for full_job

Routine execution output was invisible — routine_fire returned a one-liner,
routine_history had no actual output, and the conversation thread contained
only a summary. Full-job routines also hard-failed without Docker.

Three fixes:

1. **Full transcript persistence**: execute_lightweight now persists every
   message (prompt, LLM responses, tool calls with params, tool results) to
   the routine's conversation thread as it executes, not just a summary
   after the fact.

2. **Routine output visibility**: routine_history includes conversation_id
   and recent_output messages. routine_fire tells the user to check
   routine_history. Web detail page has a "View Execution Thread" button
   that navigates to the chat tab. ROUTINE_OK stores "No issues found"
   instead of None. Full-job summary pulls actual job output instead of
   generic "Job X finished".

3. **Remove SandboxReadiness gate**: full_job routines dispatch through the
   scheduler like regular /job commands — no Docker required. The
   SandboxReadiness enum is removed entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(worker): treat AutonomousUnavailable tool errors as recoverable

The job worker crashed the entire job when a tool was denied for
autonomous execution (e.g. secret_list). The error was already recorded
in reason_ctx for the LLM to see, but process_tool_result_job returned
Err which propagated through the agentic loop and terminated the job.

Now all tool errors (including AutonomousUnavailable) return Ok,
letting the LLM see the denial and try a different approach.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(llm): sanitize tool names for OpenAI Codex Responses API

The Codex API requires tool names to match `^[a-zA-Z0-9_-]+$` but
MCP/extension tools can have dots in their names (e.g. `mcp.server.tool`).
This caused HTTP 400 errors when the job worker sent tool calls back
to the LLM.

Sanitize tool names in both `convert_tool_definition` and
`convert_message` (function_call items) by replacing invalid characters
with underscores.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(routines): inject execution context into full_job description [skip-regression-check]

When a full_job routine dispatches a job, the LLM had no context that
it was already executing inside a routine. It wasted iterations on
infrastructure (discovering tools, creating routines, setting up auth)
instead of doing the actual work.

Prepend a clear directive to the job description telling the LLM that
tools and the routine are already configured, and to execute the task
directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(mcp): auto-refresh expired OAuth tokens on access [skip-regression-check]

When IronClaw restarts, MCP servers fail with "Secret has expired"
because get_access_token() checks token expiry locally and returns an
error before any HTTP request is made — so the existing 401-retry
refresh logic never triggers.

Now get_access_token() catches SecretError::Expired and automatically
calls refresh_access_token() using the stored refresh token. If the
refresh succeeds, the new token is returned transparently. If it fails,
the error message includes both the expiry and the refresh failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(mcp): align refresh token naming and set expiry on stored tokens

Two bugs prevented MCP OAuth token auto-refresh on restart:

1. Naming mismatch: the hosted OAuth flow stored the refresh token as
   `{token_secret_name}_refresh_token` (e.g. `mcp_notion_access_token_refresh_token`)
   but `McpServerConfig::refresh_token_secret_name()` returned
   `mcp_notion_refresh_token`. The refresh token was there but unfindable.

2. Missing expiry: `store_tokens` in auth.rs never called `with_expiry()`
   even though `AccessToken::expires_in` was available. Combined with the
   fix from the previous commit (auto-refresh on Expired), tokens stored
   via the MCP auth flow will now also trigger refresh correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(web): show activity and transitions for agent jobs in job detail [skip-regression-check]

The job events endpoint only checked sandbox jobs for ownership,
returning 404 for agent jobs dispatched from routines. The detail
handler also returned empty transitions for agent jobs.

- events handler: fall back to agent job ownership check
- detail handler: populate transitions from job's state history

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat(routines): expose max_iterations for full_job routines (default 25)

The max_iterations parameter was hardcoded to 10 and not configurable
via routine_create or routine_update, causing complex tasks to hit the
iteration cap.

- Add max_iterations to full_job execution schema (1-200, default 25)
- Thread it through parse → build → RoutineAction
- Support updating via routine_update
- Raise default from 10 to 25

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(routines): break self-dialogue loop after full_job plan execution

After plan execution, the completion-check Q&A ("Is the job complete?" /
"No, not complete...") was left in the message context, causing the
agentic loop to repeat the same analysis instead of calling tools.

Replace the stale dialogue with an action-oriented continuation prompt
that instructs the LLM to use tools for remaining work. Also strip
<suggestions> tags from all job output since they're only meaningful
for interactive chat sessions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(repl): prevent test hang in single-message mode

In single-message mode, start() stored a clone of the mpsc sender in
self.msg_tx for approval injection. After the thread sent /quit and
exited, the stored clone kept the stream alive, so stream.next()
blocked forever in the test assertion that the stream ends.

Skip storing the sender in single-message mode since interactive
approval is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(jobs): treat text responses as final answer in agentic loop

When the LLM produces a non-empty text response with no tool intent
(already filtered by the nudge mechanism), it is the job's final
answer. Previously, handle_text_response only exited the loop if the
text matched rigid completion phrases like "job is complete". Natural
summaries like "Weekly review completed and saved to Notion" were
added to context and the loop continued, causing the LLM to restate
the same summary until max_iterations was hit.

Now any non-empty text response marks the job complete and stops the
loop, matching the chat dispatcher behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* perf(tests): reduce skills catalog network failure test from 10s to 1s

The test_search_returns_error_on_network_failure test connects to an
unreachable RFC 5737 TEST-NET IP and waited for the full 10s production
REQUEST_TIMEOUT. Add with_url_and_timeout test helper and use a 1s
timeout instead. [skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(tools): accept 'message' as alias for 'content' in message tool

LLMs frequently call the message tool with {"message": "..."} instead
of {"content": "..."}. Fall back to the 'message' key when 'content'
is missing to avoid InvalidParameters errors during autonomous job
execution.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(tools): attach thread_id for gateway broadcast in message tool

When the message tool broadcasts to all channels (channel=null), it
sent an OutgoingResponse without a thread_id. The gateway silently
dropped these messages (returned Ok but never sent the SSE event),
so they appeared in repl but not in the web UI.

The thread_id was only populated when channel was explicitly "gateway".
Now it is always populated from notify_thread_id metadata, so
broadcast_all delivers to the gateway correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(gateway): return error instead of silently dropping messages

Gateway broadcast() and respond() previously returned Ok(()) when
thread_id was missing, silently swallowing the message. Callers
(message tool, agent loop) believed delivery succeeded when it didn't.

Now returns ChannelError::MissingRoutingTarget so callers can detect
and report the failure. Four regression tests verify the contract:
respond/broadcast with and without thread_id.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: resolve rebase conflicts with staging

Restore sandbox_readiness field removed by pre-rebase commits (staging
still uses it). Update repl test to match staging's single-message
behavior (no longer sends /quit). Add missing reasoning field to
ToolCall in codex test.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(tools): log error when routine conversation lookup fails

The routine_history tool silently swallowed errors from
get_or_create_routine_conversation, returning empty output without
any diagnostic logging. Add tracing::warn so failures are visible
in logs. [skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR #1650 review comments

- E2E test: accept submitted/accepted as success states in job assertion
- TimeTool: remove operation from required schema (defaults to "now")
- jobs handler: log DB errors server-side, return generic message to client
- routines handler: use read-only find_routine_conversation on GET
- codex provider: reverse-map sanitized tool names so MCP tools resolve

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address zmanian review feedback on PR #1650

- MCP refresh token: fall back to legacy secret name (mcp_{name}_refresh_token)
  so existing users don't need to re-authenticate after the naming fix
- Job worker: replace fragile messages.pop() with truncate-to-saved-count
  to avoid maintenance hazard if message flow changes
- Document cost implications of max_iterations 10->25 default bump
- Revert Cargo.toml dist profile change (thin LTO comment, codegen-units=16)
  as it's unrelated to this PR

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: resolve rebase conflicts and address new Copilot comments

- Fix no_silent_drop tests for updated GatewayConfig (user_id moved to
  GatewayChannel::new second arg, user_tokens removed)
- Fix handle_text_response param name (_reason_ctx -> reason_ctx)
- Fix missing has_text_response field in test JobDelegate
- Propagate row.get errors in find_routine_conversation instead of
  unwrap_or_default
- Only fall back to legacy refresh token name on NotFound/Expired,
  propagate real errors (DB, decryption)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 12:27:43 -07:00
fd41bdf4be fix(worker): treat empty LLM response after text output as completion (#1677)
* fix(worker): treat empty LLM response after text output as completion

When a job's LLM produces a substantive text response (e.g., formatted
results from a routine) and the next LLM call returns empty or errors,
the worker now treats this as successful completion instead of
continuing the loop until failure.

Previously, empty responses always triggered TextAction::Continue,
causing the loop to re-call the LLM. The LLM had nothing more to say,
so the provider returned "Response contained no message or tool call
(empty)". This made routine jobs that successfully produced results
report as "failed".

The fix adds a `has_text_response` flag to JobDelegate:
- After any non-empty text response: flag is set
- Empty text after flag is set: treated as completion
- LLM errors (select_tools/respond_with_tools) after flag: treated
  as completion instead of propagating
- Empty text before any output: still retries (rate-limit backoff)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(worker): restrict error swallowing to EmptyResponse variant only

- Add LlmError::EmptyResponse variant for when LLM returns no content
- Update nearai_chat and github_copilot providers to emit EmptyResponse
  instead of InvalidResponse for empty/no-choice responses
- try_complete_on_error now only swallows EmptyResponse (not AuthFailed,
  ContextLengthExceeded, Http, Io, etc.)
- Extract is_completion_eligible_error as testable pure function
- Log mark_completed errors at warn level instead of silently dropping
- Add EmptyResponse to retry and circuit breaker transient classifications
- Rewrite test to exercise real classification logic against all variants

Addresses review feedback from zmanian and gemini-code-assist.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(worker): extract mark_completed_or_warn helper to DRY completion logic

Extract shared mark-completed + warn-on-failure pattern into a single
helper method used by both try_complete_on_error and handle_text_response.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: j-bloggs <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 18:46:08 +01:00
de5a1c7b0d fix(worker): replace script -qfc with pty-process for injection-safe PTY (#1678)
- Add pty-process crate (MIT, tokio async support) for PTY allocation
- Spawn claude CLI with pty-process::Command::arg() chaining instead of
  building a shell string for script -qfc
- Eliminates all shell injection surfaces: prompt, model, session_id
  are passed via execve, never interpreted by a shell
- Keep stderr on separate pipe to prevent NDJSON parse breakage
  (pty-process attaches PTY to all fds by default)
- Gate PTY behind #[cfg(unix)] with direct-spawn fallback for Windows CI
- Read stdout from PTY master (implements tokio::io::AsyncRead)
- Add regression tests: arg vector construction + PTY allocation

Addresses review feedback from zmanian and gemini-code-assist.

Co-authored-by: j-bloggs <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 16:31:49 +01:00
9ce3a9fc53 feat(discord): implement on_broadcast via DM channel creation (#1693)
- Implement broadcast_dm() that creates a DM channel with the target
  user (POST /users/@me/channels, cached by Discord) and sends the
  message to it
- Extract DISCORD_API_BASE constant for all Discord REST API URLs
- Extract send_channel_message() shared helper to deduplicate message
  posting between on_respond and broadcast_dm
- Add snowflake validation on user_id before API calls
- Fix pre-existing clippy redundant_closure warning
- Use typed DmChannelResponse struct instead of serde_json::Value

Closes no specific issue — completes the previously stubbed on_broadcast.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 16:31:27 +01:00
AchieveandGitHub 9bb19a98f7 fix(web): redact database error details from API responses (#1711) 2026-03-28 15:13:28 +01:00
AchieveandGitHub 0b33ca9926 fix(oauth): tighten legacy state validation and fallback handling (#1701)
* fix(oauth): tighten legacy state validation and fallback handling

* style: fix formatting

* refactor: separate validation checks for clearer error messages
2026-03-28 15:10:39 +01:00
AchieveandGitHub 9ba10eac35 fix(db): add tracing warn for naive timestamp fallback and improve parse_timestamp tests (#1700)
* fix(db): add tracing warn for naive timestamp fallback and improve parse_timestamp tests

* style: fix formatting
2026-03-28 15:08:25 +01:00
AchieveandGitHub 27e8d6f8dd fix(wasm): use typed WASM schema as advertised schema when available (#1699) 2026-03-28 15:07:16 +01:00
Henry ParkandGitHub f49f368355 Clean up extension credentials on uninstall (#1718)
* Clean up extension credentials on uninstall

* Address PR review feedback

* Cover channel webhook secrets on uninstall

* Harden tool secret cleanup detection
2026-03-28 14:46:45 +01:00
8f8cb7f7b1 feat: DB-backed user management, admin secrets provisioning, and multi-tenant isolation (#1626)
* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling

Finishes the remaining isolation work from phases 2–4 of #59:

Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.

Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.

Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: use selected_model setting key to match /model command persistence

The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override

Three follow-up fixes for multi-tenant isolation:

1. Multi-user heartbeat now runs memory hygiene per user before each
   heartbeat check, matching single-user heartbeat behavior.

2. /model command in multi-tenant mode only persists to per-user
   settings (selected_model) without calling set_model() on the shared
   LlmProvider. The per-request model_override in the dispatcher reads
   from the same setting. Added multi_tenant flag to AgentConfig
   (auto-detected from GATEWAY_USER_TOKENS).

3. RigAdapter now supports per-request model overrides by injecting the
   model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
   API servers use last-key-wins for duplicate JSON keys, so the override
   takes effect via serde's flatten serialization order.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review — cost model attribution, heartbeat concurrency, pruning

Fixes from review comments on #1614:

- Cost tracking now uses the override model name (not active_model_name)
  when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
  instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
  max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
  unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: /status ownership, model persistence scoping, heartbeat robustness

Addresses second round of PR review on #1614:

- /status <job_id> DB path now validates job.user_id == requesting user
  before returning data (was missing ownership check, security fix).

- persist_selected_model takes user_id param instead of owner_id, and
  skips .env/TOML writes in multi-tenant mode (these are shared global
  files). handle_system_command now receives user_id from caller.

- JoinSet collection handles Err(JoinError) explicitly instead of
  silently dropping panicked tasks.

- Notification forwarder extracts owner_id from response metadata in
  multi-tenant mode for per-user routing instead of broadcasting to
  the agent owner.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap

Round 3 review fixes:

- Cost tracking passes None for cost_per_token when model override is
  active, letting CostGuard look up pricing by model name instead of
  using the default provider's rates (serrrfirat).

- fire_manual() now uses per-user workspace, matching spawn_fire()
  pattern (serrrfirat).

- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
  solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).

- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
  the LLM provider (serrrfirat + Copilot).

- Fixed inject_model_override doc comment accuracy (Copilot).

- Added comment explaining multi-tenant notification routing priority
  (Copilot).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: user-scoped webhook endpoint for multi-tenant isolation

Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.

The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.

Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
  when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
  from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
  services that can't send bearer tokens)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat(db): add UserStore trait with users, api_tokens, invitations tables

Foundation for DB-backed user management (#1605):

- UserRecord, ApiTokenRecord, InvitationRecord types in db/mod.rs
- UserStore sub-trait (17 methods) added to Database supertrait
- PostgreSQL migration V14__users.sql (users, api_tokens, invitations)
- libSQL schema + incremental migration V14
- Full implementations for both PgBackend (via Store delegation) and
  LibSqlBackend (direct SQL in libsql/users.rs)
- authenticate_token JOINs api_tokens+users with active/non-revoked
  checks; has_any_users for bootstrap detection

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat(web): DB-backed auth, user/token/invitation API handlers

Adds the web gateway layer for DB-backed user management (#1605):

Auth refactor:
- CombinedAuthState wraps env-var tokens (MultiAuthState) + optional
  DbAuthenticator for DB-backed token lookup with LRU cache (60s TTL,
  1024 max entries)
- auth_middleware tries env-var tokens first, then DB fallback
- From<MultiAuthState> impl for backward compatibility
- main.rs wires with_db_auth when database is available

API handlers (12 new endpoints):
- /api/admin/users — CRUD: create, list, detail, update, suspend, activate
- /api/tokens — create (returns plaintext once), list, revoke
- /api/invitations — create, list, accept (creates user + first token)

Token creation: 32 random bytes → hex plaintext, SHA-256 hash stored.
Invitation accept: validates hash + pending + not expired, creates
user record and first API token atomically.

All test files updated for CombinedAuthState type change.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: startup env-var user migration + UserStore integration tests

Completes the DB-backed user management feature (#1605):

- Startup migration: when GATEWAY_USER_TOKENS is set and the users
  table is empty, inserts env-var users + hashed tokens into DB.
  Logs deprecation notice when DB already has users.
- hash_token made pub for reuse in migration code.
- 10 integration tests for UserStore (libsql file-backed):
  - has_any_users bootstrap detection
  - create/get/get_by_email/list/update user lifecycle
  - token create → authenticate → revoke → reject cycle
  - suspended user tokens rejected
  - wrong-user token revoke returns false
  - invitation create → accept → user created
  - record_login and record_token_usage timestamps
- libSQL migration: removed FK constraints from V14 (incompatible
  with execute_batch inside transactions). Tables in both base SCHEMA
  and incremental migration for fresh and existing databases.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: remove GATEWAY_USER_TOKENS, fix review feedback

GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.

Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
  via db.has_any_users() in app.rs)

Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
  status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
  unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: add role-based access control (admin/member)

Adds a `role` field (admin|member) to user management:

Schema:
- `role TEXT NOT NULL DEFAULT 'member'` added to users table in both
  PostgreSQL V14 migration and libSQL schema/incremental migration
- UserRecord gains `role: String` field
- UserIdentity gains `role: String` field, populated from DB in
  DbAuthenticator and defaulting to "admin" for single-user mode

Access control:
- AdminUser extractor: returns 403 Forbidden if role != "admin"
- /api/admin/users/* handlers: require AdminUser (create, list,
  detail, update, suspend, activate)
- POST /api/invitations: requires AdminUser (only admins can invite)
- User creation accepts optional "role" param (defaults to "member")
- Invitation acceptance creates users with "member" role

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat(web): add Users admin tab to web UI

Adds a Users tab to the web gateway UI for managing users, tokens,
and roles without needing direct API calls.

Features:
- User list table with ID, name, email, role, status, created date
- Create user form with display name, email, role selector
- Suspend/activate actions per user
- Create API token for any user (shows plaintext once with copy button)
- Role badges (admin highlighted, member muted)
- Non-admin users see "Admin access required" message
- Keyboard shortcut: Cmd/Ctrl+5 switches to Users tab

CSS:
- Reuses routines-table styles for the user list
- Badge, token-display, btn-small, btn-danger, btn-primary components

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: move Users to Settings subtab, bootstrap admin user on first run

- Moved Users from top-level tab to Settings sidebar subtab (under
  Skills, before Theme toggle)
- On first startup with empty users table, automatically creates an
  admin user from GATEWAY_USER_ID config with a corresponding API
  token from GATEWAY_AUTH_TOKEN. This ensures the owner appears in
  the Users panel immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: user creation shows token, + Token works, no password save popup

Three UI/UX fixes:

1. Create user now generates an initial API token and shows it in a
   copy-able banner instead of triggering the browser's password save
   dialog. Uses autocomplete="off" and type="text" for email field.

2. "+ Token" button works: exposed createTokenForUser/suspendUser/
   activateUser on window for inline onclick handlers in dynamically
   generated table rows. Token creation uses showTokenBanner helper.

3. Admin token creation: POST /api/tokens now accepts optional
   "user_id" field when the requesting user is admin, allowing
   token creation for other users from the Users panel.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: use event delegation for user action buttons (CSP compliance)

Inline onclick handlers are blocked by the Content-Security-Policy
(script-src 'self' without 'unsafe-inline'). Switched to data-action
attributes with a delegated click listener on the users table.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: add i18n for Users subtab, show login link on user creation

- Added 'settings.users' i18n key for English and Chinese
- Token banner now shows a full login link (domain/?token=xxx)
  with a Copy Link button, plus the raw token below
- Login link works automatically via existing ?token= auto-auth

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: token hash mismatch — hash hex string, not raw bytes

Critical auth bug: token creation hashed the raw 32 bytes
(hasher.update(token_bytes)) but authentication hashed the hex-encoded
string (hash_token(candidate) where candidate is the hex string the
user sends). This meant newly created tokens could never authenticate.

Fixed all 4 token creation sites (users, tokens, invitations create,
invitations accept) to use hash_token(&plaintext_token) which hashes
the hex string consistently with the auth lookup path.

Removed now-unused sha2::Digest imports from handlers.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: remove invitation system

The invitation flow is redundant — admin create user already generates
a token and shows a login link. Invitations add complexity without
value until email integration exists.

Removed:
- InvitationRecord struct and 4 UserStore trait methods
- invitations table from V14 migration (postgres + both libsql schemas)
- PostgreSQL Store methods (create/get/accept/list invitations)
- libSQL UserStore invitation methods + row_to_invitation helper
- invitations.rs handler file (212 lines)
- /api/invitations routes (create, list, accept)
- test_invitation_lifecycle test

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: user deletion, self-service profile, per-user job limits, usage API

Four multi-tenancy improvements:

1. User deletion cascade (DELETE /api/admin/users/{id}):
   Deletes user and all data across 11 user-scoped tables (settings,
   secrets, routines, memory, jobs, conversations, etc.). Admin only.

2. Self-service profile (GET/PATCH /api/profile):
   Users can read and update their own display_name and metadata
   without admin privileges.

3. Per-user job concurrency (MAX_JOBS_PER_USER env var):
   Scheduler checks active_jobs_for(user_id) before dispatch.
   Prevents one user from exhausting all job slots.

4. Usage reporting (GET /api/admin/usage?user_id=X&period=day|week|month):
   Aggregates LLM costs from llm_calls via agent_jobs.user_id.
   Returns per-user, per-model breakdown of calls, tokens, and cost.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: add TenantCtx for compile-time tenant isolation

Implements zmanian's architectural proposal from #1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.

TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.

AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).

TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.

Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
  TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR #1626 review feedback — bounded LRU cache, admin auth, FK cleanup

- Replace HashMap with lru::LruCache in DbAuthenticator so the token
  cache is hard-bounded at 1024 entries (evicts LRU, not just expired)
- Gate admin user endpoints (list/detail/update/suspend/activate) with
  AdminUser extractor so members get 403 instead of full access
- Add api_tokens to libSQL delete_user cleanup list to prevent orphaned
  tokens (libSQL has no FK cascade)
- Add regression tests for all three fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: update CA certificates in runtime Docker image

Ensures the root certificate bundle is current so TLS handshakes
to services like Supabase succeed on Railway.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: resolve CI failures — formatting, no-panics check

- Run cargo fmt on test code
- Replace .expect() with const NonZeroUsize in DbAuthenticator
- Add // safety: comments for test-only code in multi_tenant.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: switch PostgreSQL TLS from rustls to native-tls

rustls with rustls-native-certs fails TLS handshake on Railway's
slim container (empty or stale root cert store). native-tls delegates
to OpenSSL on Linux which handles system certs more reliably.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* Adding user management api

* feat: admin secrets provisioning API + API documentation

- Add PUT/GET/DELETE /api/admin/users/{id}/secrets/{name} endpoints for
  application backends to provision per-user secrets (AES-256-GCM encrypted)
- Add secrets_store field to GatewayState with builder wiring
- Create docs/USER_MANAGEMENT_API.md with full API spec covering users,
  secrets, tokens, profile, and usage endpoints
- Update web gateway CLAUDE.md route table

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: add CatchPanicLayer to capture handler panics

Without this, panics in async handlers silently drop the connection
and the edge proxy returns a generic 503. Now panics are caught,
logged, and returned as 500 with the panic message.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address second-round review — transactional delete, overflow, error logging

- C1: Wrap PostgreSQL delete_user() in a transaction so partial cleanup
  can't leave users in a half-deleted state
- M2: Add job_events to delete cleanup (both backends) — FK to
  agent_jobs without CASCADE would cause FK violation
- H1/M4: Cap expires_in_days to 36500 before i64 cast (tokens + secrets)
- H2: Validate target user exists before creating admin token to prevent
  orphan tokens on libSQL
- H3: Log DB errors in DbAuthenticator::authenticate() instead of
  silently swallowing them as 401

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: revert to rustls with webpki-roots fallback for PostgreSQL TLS

native-tls/OpenSSL caused silent crashes (segfaults in C code) during
DB writes on Railway containers. Switch back to rustls but add
webpki-roots as a fallback when system certs are missing, which was
the original TLS handshake failure on slim container images.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* chore: update Cargo.lock for rustls + webpki-roots

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* debug: add /api/debug/db-write endpoint to diagnose user insert failure

Temporary diagnostic endpoint that tests DB INSERT to users table
with full error logging. No auth required. Will be removed after
debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* perf: use cargo-chef in Dockerfile for dependency caching

Splits the build into planner/deps/builder stages. Dependencies are
only recompiled when Cargo.toml or Cargo.lock change. Source-only
changes skip straight to the final build stage.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* debug: add tracing to users_create_handler

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: guard created_by FK in user creation handler

The auth identity user_id (from owner_id scope) may not match any
user row in the DB, causing a FK violation on the created_by column.
Check that the referenced user exists before setting created_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: collapse GATEWAY_USER_ID into IRONCLAW_OWNER_ID

Remove the separate GATEWAY_USER_ID config. The gateway now uses
IRONCLAW_OWNER_ID (config.owner_id) directly for auth identity,
bootstrap user creation, and workspace scoping.

Previously, with_owner_scope() rebinds the auth identity to owner_id
while keeping default_sender_id as the gateway user_id. This caused
a FK constraint violation when creating users because the auth
identity ("default") didn't match any user in the DB ("nearai").

Changes:
- Remove GATEWAY_USER_ID env var and gateway_user_id from settings
- Remove user_id field from GatewayConfig
- Add owner_id parameter to GatewayChannel::new()
- Remove with_owner_scope() method
- Remove default_sender_id from GatewayState
- Remove sender override logic in chat/approval handlers
- Remove debug endpoint and tracing from prior debugging
- Update all tests and E2E fixtures

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: hide Users tab for non-admins, remove auth hint text

- Fetch /api/profile after login and hide the Users settings tab
  when the user's role is not admin
- Remove the "Enter the GATEWAY_AUTH_TOKEN" hint from the login page
  since tokens are now managed via the admin panel, not .env files

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address review feedback (auth 503, token expiry, CORS PATCH)

- DB auth errors now return 503 instead of 401 so outages are
  distinguishable from invalid tokens (serrrfirat H3)
- Cap expires_in_days to 36500 before i64 cast to prevent negative
  duration from u64 overflow (serrrfirat H1)
- Add PATCH to CORS allowed methods for profile/user update
  endpoints (Copilot)
- Stop leaking panic details in CatchPanicLayer response body

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: harden multi-tenant isolation — review fixes from #1614

- Add conversation ownership checks in TenantScope: add_conversation_message,
  touch_conversation, list_conversation_messages (+ paginated),
  update_conversation_metadata_field, get_conversation_metadata now return
  NotFound for conversations not owned by the tenant (cross-tenant data leak)
- Fix multi-user heartbeat: clear notify_user_id per runner so notifications
  persist to the correct user, not the shared config target
- Move hygiene tasks into bounded JoinSet instead of unbounded tokio::spawn
- Revert send_notification to private visibility (only used within module)
- Use effective_model_name() for cost attribution in dispatcher so providers
  that ignore per-request model overrides report the actual model used
- Fix inject_model_override doc comment; add 3 unit tests
- Fix heartbeat doc comment ("routines" not "active routines")

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: add Jobs, Cost, Last Active columns to admin Users table

Add UserSummaryStats struct and user_summary_stats() batch query to the
UserStore trait (both PostgreSQL and libSQL backends). The admin users
list endpoint now fetches per-user aggregates (job count, total LLM
spend, most recent activity) in a single query and includes them inline
in the response. The frontend Users table displays three new columns.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address review comments and CI formatting failures

CI fixes:
- cargo fmt fixes in cli/mod.rs and db/tls.rs

Security/correctness (from Copilot + serrrfirat + pranavraja99 reviews):
- Token create: reject expires_in_days > 36500 with 400 instead of silent clamp
- Token create: return 404 when admin targets non-existent user
- User create: map duplicate email constraint violations to 409 Conflict
- User create: remove unnecessary DB roundtrip for created_by (use AdminUser directly)
- DB auth: log warn on DB lookup failures instead of silently swallowing errors
- libSQL: add FK constraints on users.created_by and api_tokens.user_id

Config fixes:
- agent.multi_tenant: resolve from AGENT_MULTI_TENANT env var instead of hardcoding false
- heartbeat.multi_tenant: fix doc comment to match actual env-var-based behavior

UI fix:
- showTokenBanner: pass correct title ("Token created!" vs "User created!")

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address remaining review comments (round 2)

- Secrets handlers: normalize name to lowercase before store operations,
  validate target user_id exists (returns 404 if not found)
- libSQL: propagate cost parsing errors instead of unwrap_or_default()
  in both user_usage_stats and user_summary_stats
- users_list_handler: propagate user_summary_stats DB errors (was
  silently swallowed with unwrap_or_default)
- loadUsers: distinguish 401/403 (admin required) from other errors
- Docs: fix users.id type (TEXT not UUID), remove "invitation flow"
  from V14 migration comment

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: i18n for Users tab, atomic user+token creation, transactional delete_user

i18n:
- Add 31 translation keys for all Users tab strings (en + zh-CN)
- Wire data-i18n attributes on HTML elements (headings, buttons, inputs,
  table headers, empty state)
- Replace all hard-coded strings in app.js with I18n.t() calls

Atomic user+token creation:
- Add create_user_with_token() to UserStore trait
- PostgreSQL: wraps both INSERTs in conn.transaction() with auto-rollback
- libSQL: wraps in explicit BEGIN/COMMIT with ROLLBACK on error
- Handler uses single atomic call instead of two separate operations

Transactional delete_user for libSQL:
- Wrap multi-table DELETE cascade in BEGIN/COMMIT transaction
- ROLLBACK on any error to prevent partial cleanup / inconsistent state
- Matches the PostgreSQL implementation which already used transactions

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: revert V14 migration to match deployed checksum [skip-regression-check]

Refinery checksums applied migrations — editing V14__users.sql after
it was already applied causes deployment failures. Revert the cosmetic
comment changes (added in df40b22f) to restore the original checksum.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: bootstrap onboarding flow for multi-tenant users

The bootstrap greeting and workspace seeding only ran for the owner
workspace at startup, so new users created via the admin API never
received the welcome message or identity files (BOOTSTRAP.md, SOUL.md,
AGENTS.md, USER.md, etc.).

Three fixes:
- tenant_ctx(): seed per-user workspace on first creation via
  seed_if_empty(), which writes identity files and sets
  bootstrap_pending when the workspace is truly fresh
- handle_message(): check take_bootstrap_pending() on the tenant
  workspace (not the owner workspace) and persist the greeting to
  the user's own assistant conversation + broadcast via SSE
- WorkspacePool: seed new per-user workspaces in the web gateway
  so memory tools also see identity files immediately

The existing single-user bootstrap in Agent::run() is preserved for
non-multi-tenant deployments.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address remaining PR review comments (round 3)

- Docs: fix metadata description from "merge patch" to "full replacement"
- Secrets: reject expires_in_days > 36500 with 400 (was silently clamped)
- libSQL: CAST(SUM(cost) AS TEXT) in user_usage_stats and user_summary_stats
  to prevent SQLite numeric coercion from crashing get_text() — this was
  the root cause of the Copilot "SUM returns numeric type" comments
- Add 3 regression tests: user_summary_stats (empty + with data) and
  user_usage_stats (multi-model aggregation)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: add role change support for users (admin/member toggle)

- Add update_user_role() to UserStore trait + both backends (PostgreSQL
  and libSQL)
- Extend PATCH /api/admin/users/{id} to accept optional "role" field
  with validation (must be "admin" or "member")
- Add "Make Admin" / "Make Member" toggle button in Users table actions
- Add i18n keys for role change (en + zh-CN)
- Update API docs to document the role field on PATCH
- Fix test helpers to use fmt_ts() for timestamps (was using SQLite
  datetime('now') which produces incompatible format for string comparison)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: show live LLM spend in Users table instead of only DB-recorded costs [skip-regression-check]

Chat turns record LLM cost in CostGuard (in-memory) but don't create
agent_jobs/llm_calls DB rows — those are only written for background
jobs. The Users table was querying only from DB, so it showed $0.00
for users who only chatted.

Now supplements DB stats with CostGuard.daily_spend_for_user() —
the same source displayed in the status bar token counter. Shows
whichever is larger (DB historical total vs live daily spend).

Also falls back to last_login_at for "Last Active" when no DB job
activity exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: persist chat LLM calls to DB and fix usage stats query

Two root causes for zero usage stats:

1. ChatDelegate only recorded LLM costs to CostGuard (in-memory) —
   never to the llm_calls DB table. Added DB persistence via
   TenantScope.record_llm_call() after each chat LLM call, with
   job_id=NULL and conversation_id=thread_id.

2. user_summary_stats query only joined agent_jobs→llm_calls, missing
   chat calls (which have job_id=NULL). Redesigned query to start from
   llm_calls and resolve user_id via COALESCE(agent_jobs.user_id,
   conversations.user_id) — covers both job and chat LLM calls.

Both PostgreSQL and libSQL queries updated. TenantScope gets
record_llm_call() method. Tests updated for new query semantics.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address review comments — input validation, cost semantics, panic safety [skip-regression-check]

- Validate display_name: trim whitespace, reject empty strings (create + update)
- Validate metadata: must be a JSON object, return 400 if not (admin + profile)
- secrets_list_handler: verify target user_id exists before listing
- Cost display: use DB total directly (chat calls now persist to DB),
  remove confusing max(db,live) CostGuard fallback
- CatchPanicLayer: truncate panic payload to 200 chars in log to limit
  potential sensitive data exposure

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address Copilot round 5 — docs, secrets consistency, token name, provider field [skip-regression-check]

- Docs: users.id note updated to "typically UUID v4 strings (bootstrap
  admin may use a custom ID)"
- secrets_list_handler: return 503 when DB store is None (was falling
  through to list secrets without user validation)
- tokens_create: trim + reject empty token name (matching display_name
  pattern)
- LlmCallRecord.provider: use llm_backend ("nearai","openai") instead
  of model_name() which returns the model identifier
- user_summary_stats zero-LLM users: acceptable — handler already falls
  back to 0 cost and last_login_at for missing entries

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: DB auth returns 503 on outage, scheduler counts only blocking jobs

From serrrfirat review:
- DB auth: return Err(()) on database errors so middleware returns 503
  instead of silently returning Ok(None) → 401 (auth miss)
- Scheduler: add parallel_blocking_count_for() that uses
  is_parallel_blocking() (Pending/InProgress/Stuck) instead of
  is_active() for per-user concurrency — Completed/Submitted jobs
  no longer count against MAX_JOBS_PER_USER

From Copilot:
- CLAUDE.md: fix secrets route paths from {id} to {user_id}
- token_hash: use .as_slice() instead of .to_vec() to avoid
  heap allocation on every token auth/creation call

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: immediate auth cache invalidation on security-critical actions (zmanian review #6)

Add DbAuthenticator::invalidate_user() that evicts all cached entries
for a user. Called after:
- Suspend user (immediate lockout, was 60s delay)
- Activate user (immediate access restoration)
- Role change (admin↔member takes effect immediately)
- Token revocation (revoked token can't be reused from cache)

The DbAuthenticator is shared (via Clone, which Arc-clones the cache)
between the auth middleware and GatewayState, so handlers can evict
entries from the same cache the middleware reads.

Also from zmanian's review:
- Items 1-5, 7-11 were already resolved in prior commits
- Item 12 (String→enum for status/role) is deferred as a broader refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: last-admin protection, usage stats for chat calls, UTF-8 safe panic truncation

Last-admin protection:
- Suspend, delete, and role-demotion of the last active admin now
  return 409 Conflict instead of succeeding and locking out the admin API
- Helper is_last_admin() checks active admin count before destructive ops

Usage stats:
- user_usage_stats() now includes chat LLM calls (job_id=NULL) by
  joining via conversations.user_id, matching user_summary_stats()
- Both PostgreSQL and libSQL queries updated

Panic handler:
- Use floor_char_boundary(200) instead of byte-index [..200] to
  prevent panic on multi-byte UTF-8 characters in panic messages

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: workspace seed race, bootstrap atomicity, email trim, secrets upsert response [skip-regression-check]

- WorkspacePool: await seed_if_empty() synchronously after inserting
  into cache (drop lock first to avoid blocking), so callers see
  identity files immediately instead of racing a background task
- Bootstrap admin: use create_user_with_token() for atomic user+token
  creation, matching the admin create endpoint
- Email: trim whitespace, treat empty as None to prevent " " being
  stored and breaking uniqueness
- Secrets PUT: report "updated" vs "created" based on prior existence
- Last token_hash.to_vec() → .as_slice() in authenticate_token

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: disable unscoped webhook endpoint in multi-tenant mode [skip-regression-check]

The original /api/webhooks/{path} endpoint looks up routines across all
users. In multi-tenant mode, anyone who knows the webhook path + secret
could trigger another user's routine. Now returns 410 Gone with a
message pointing to the scoped endpoint /api/webhooks/u/{user_id}/{path}.

Detection uses state.db_auth.is_some() — present only when DB-backed
auth is enabled (multi-tenant). Single-user deployments are unaffected.

From: standardtoaster review comment

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: webhook multi-tenant check, secrets error propagation, stale doc comment [skip-regression-check]

- Webhook: use workspace_pool.is_some() instead of db_auth.is_some()
  for multi-tenant detection — db_auth is set for any DB deployment,
  workspace_pool is only set when has_any_users() was true at startup
- Secrets: propagate exists() errors instead of unwrap_or(false) so
  backend outages surface as 500 rather than incorrect "created" status
- Config: fix stale workspace_read_scopes comment referencing user_id

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 00:19:17 -07:00
2f4eb08613 fix: sanitize tool error results before llm injection (#1639)
* fix: sanitize tool error results before llm injection

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <[email protected]>

* fix: wrap preflight tool rejection errors for llm safety

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <[email protected]>

* style: apply rustfmt to error-path regressions

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <[email protected]>

* fix: preserve wrapped tool errors in history replay

* fix: address review findings on PR #1639

- Simplify legacy error handling in rebuild_chat_messages_from_db:
  remove redundant "Error: " prefix since legacy errors already contain
  descriptive text (e.g. "Tool 'http' failed: timeout"). Both wrapped
  (new) and plain (legacy) errors now pass through as-is.
- Update existing test assertion to match simplified format.
- Restore error-path doc line on process_tool_result.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: satisfy clippy on builder tool safety helper

---------

Co-authored-by: Sisyphus <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 10:49:28 +03:00
30db07c58e fix: require Feishu webhook authentication (#1638)
* fix: require Feishu webhook authentication

* fix: handle Feishu v2 webhook token auth

* fix: skip empty verification token write, consistent with app_id/app_secret

Address zmanian review nit #4: only write verification_token to workspace
when present, matching the if-let pattern used for app_id and app_secret.
Functionally identical (the auth check filters empty strings), but
consistent.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 10:49:02 +03:00
7234700c78 fix(llm): prevent UTF-8 panic in line_bounds() (fixes #1669) (#1679)
* fix(llm): prevent UTF-8 panic in line_bounds() (fixes #1669)

`line_bounds()` used `text[..pos]` slicing which panics when `pos`
lands inside a multi-byte UTF-8 character. This happens when
`end.saturating_sub(1)` in `is_recoverable_tool_call_segment()` steps
back into a multi-byte char like emoji.

Fix: clamp `pos` to `text.len()` and walk backward to the nearest
char boundary before slicing. Add 5 regression tests covering
mid-char positions, emoji boundaries, and out-of-bounds pos.

Also fix pre-existing clippy `unnecessary_sort_by` warnings in
web gateway handlers.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>

* test: assert expected values in line_bounds UTF-8 tests

Address Gemini review: strengthen regression tests to verify correct
return values (not just absence of panic) when pos lands mid-char.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <[email protected]>
Co-Authored-By: Happy <[email protected]>

---------

Co-authored-by: willamhou <[email protected]>
Co-authored-by: Claude <[email protected]>
Co-authored-by: Happy <[email protected]>
2026-03-27 00:01:22 -07:00
9c5ba43ccd feat(gateway): add OpenAI Responses API endpoints (#1656)
* feat(gateway): add OpenAI Responses API endpoints

Add POST /v1/responses and GET /v1/responses/{id} to the web gateway,
implementing the OpenAI Responses API. Unlike the existing Chat
Completions proxy which passes through to the raw LLM, the Responses
API routes requests through the full agent loop — giving external
clients access to tools, memory, safety, and server-side conversation
state via a standard OpenAI-compatible interface.

Key design decisions:
- Response IDs encode thread UUIDs statelessly (resp_{uuid_simple})
- previous_response_id enables multi-turn conversations
- Streaming maps AppEvent variants to Responses API SSE events
- Tool approval returns response.failed (no interactive approval flow)
- GET endpoint reconstructs ResponseObject from conversation_messages

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(responses-api): address all review feedback on PR #1656

- Decouple response ID from thread ID: encode both a per-call
  response_uuid and the thread_uuid so each POST produces a unique ID
- Reject unsupported fields (instructions, tools, tool_choice,
  temperature, max_output_tokens, non-default model) with 400
- Add user_id to IncomingMessage metadata for user-scoped SSE events
- Add conversation_belongs_to_user() ownership check on GET endpoint
- Fix tool call parsing: handle both legacy array and object wrapper
  format; use call_id/tool_call_id/id key fallback chain
- Correlate tool role messages to preceding FunctionCall call_id
- Stabilize created_at (capture once in accumulator, reuse everywhere)
- Surface error_message via new ResponseObject.error field
- Handle streaming tool failures (emit FunctionCallOutput on error)
- Remove dead Incomplete status variant
- Fix formatting (cargo fmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 00:00:25 -07:00
45cd6682d3 fix: downgrade excessive debug logging in hot path (closes #1686) (#1694)
PR #1681 introduced 23 debug-level log statements across relay client,
web server handlers, and extension manager functions. Many of these fire
on every HTTP request or in loops (e.g. has_stored_team_id called per
extension in list_installed). Downgrade them to trace level to reduce
noise at the default debug log level while preserving warn/info logs
for actionable diagnostics.

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 23:54:38 -07:00
Henry ParkandGitHub 5b95d22218 Support direct hosted OAuth callbacks with proxy auth token (#1684)
* Support direct hosted OAuth callbacks with proxy auth token

* Make OAuth env tests panic-safe

* Preserve public OAuth field compatibility

* Fix OAuth proxy token whitespace fallback
2026-03-26 16:45:31 -07:00
dd0a0e10ab fix(routines): recover delete name after failed update fallback (#1108)
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 16:20:01 -07:00
1d5777824c fix(mcp): handle 202 Accepted and wire session manager for Streamable HTTP (#1437)
* fix(mcp): handle 202 Accepted for Streamable HTTP notifications

The MCP Streamable HTTP spec requires servers to respond with
202 Accepted (empty body) for JSON-RPC notifications like
`notifications/initialized`. The HTTP transport tried to parse
this empty body as JSON, which failed and broke the session
handshake — subsequent requests like `tools/list` were rejected
because the server considered the session uninitialized.

Add an early return for 202 responses that produces an empty
McpResponse without attempting body parsing.

Fixes #1436

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(mcp): wire session manager into transport for non-OAuth HTTP clients

The factory used McpClient::new_with_config().with_session_manager()
which only set the session manager on the client, not on the
HttpMcpTransport. The transport never captured Mcp-Session-Id from
responses, so subsequent requests lacked the header and the server
rejected them as uninitialized.

Fix by constructing the HttpMcpTransport with the session manager
before wrapping it in Arc, matching the pattern already used by
new_authenticated().

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(mcp): deduplicate factory HTTP path, gate dead-code methods as test-only

- Collapse the two identical non-OAuth HTTP branches in
  `create_client_from_config()` into one (early-return for the
  authenticated path, fall through for the common case).
- Gate `McpClient::new_with_config()` and `McpClient::with_session_manager()`
  as `#[cfg(test)]` — the factory was their only production caller and no
  longer uses them. Both methods silently skip wiring the session manager
  into the transport, which was the root cause of #1436.
- Add doc warnings on both methods explaining the footgun.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: [email protected] <[email protected]>
2026-03-26 14:47:31 -07:00
577 changed files with 21204 additions and 33170 deletions
+1 -1
View File
@@ -5,7 +5,7 @@ argument-hint: <event_name> [description]
model: opus
---
Add a new SSE event called `$ARGUMENTS` to the IronClaw web gateway. This involves changes across 5 files in a specific order. Follow each step exactly.
Add a new SSE event called `$ARGUMENTS` to the OptimClaw web gateway. This involves changes across 5 files in a specific order. Follow each step exactly.
## Step 1: Add `StatusUpdate` variant
+2 -2
View File
@@ -5,7 +5,7 @@ argument-hint: <tool_name> [description]
model: opus
---
Scaffold a new tool called `$ARGUMENTS` for the IronClaw agent. First, determine the tool type and then follow the appropriate path.
Scaffold a new tool called `$ARGUMENTS` for the OptimClaw agent. First, determine the tool type and then follow the appropriate path.
## Step 0: Determine tool type
@@ -43,7 +43,7 @@ Follow this exact pattern (adjust name and description):
name = "<name>-tool"
version = "0.1.0"
edition = "2021"
description = "<Description> tool for IronClaw (WASM component)"
description = "<Description> tool for OptimClaw (WASM component)"
license = "MIT OR Apache-2.0"
publish = false
+2 -2
View File
@@ -62,7 +62,7 @@ Enter planning mode to design the implementation. The plan MUST cover:
- Happy path (expected input produces expected output)
- Error paths (invalid input, missing data, permission denied)
- Edge cases (empty collections, boundary values, concurrent access)
5. **IronClaw-specific concerns**:
5. **OptimClaw-specific concerns**:
- If the change touches persistence, both database backends must be updated (`postgres.rs` and `libsql_backend.rs`)
- New `Database` trait methods need implementations in both backends
- No `.unwrap()` or `.expect()` in production code
@@ -80,7 +80,7 @@ After the plan is approved:
1. Implement each change from the plan.
2. Write all planned tests.
3. Run IronClaw's full quality gate:
3. Run OptimClaw's full quality gate:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features` (zero warnings)
- `cargo test --lib` (all tests pass)
+3 -3
View File
@@ -96,7 +96,7 @@ Wait for user confirmation (unless `--fix` flag set), then proceed to Phase 3.
Read EVERY changed file in full (not just diff hunks). For PRs touching >20 files, prioritize: service logic > handlers > types > tests > docs. Batch reads in parallel via Agent tool.
### IronClaw-specific checks (always)
### OptimClaw-specific checks (always)
- No `.unwrap()` or `.expect()` in production code
- Prefer `crate::` for cross-module imports (`super::` OK in tests/intra-module)
- Error types use `thiserror`
@@ -167,7 +167,7 @@ gh pr checkout {number}
1. All approved review comment fixes (from Phase 2a)
2. All approved review findings (from Phase 2b)
Follow IronClaw conventions:
Follow OptimClaw conventions:
- `thiserror` for errors
- `crate::` imports
- No `.unwrap()` in production
@@ -180,7 +180,7 @@ After all fixes implemented, proceed to Phase 4.
## Phase 4: Quality Gate
Run the full IronClaw shipping checklist:
Run the full OptimClaw shipping checklist:
```bash
cargo fmt
+2 -2
View File
@@ -60,7 +60,7 @@ Wait for user confirmation before proceeding to implementation.
After user confirms:
1. Implement each fix in the plan.
2. Run IronClaw's quality gate to verify nothing breaks:
2. Run OptimClaw's quality gate to verify nothing breaks:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features`
- `cargo test --lib`
@@ -77,5 +77,5 @@ For each comment addressed, reply on the PR with a short message stating what wa
- Group duplicate comments (same issue reported by multiple bots) and reply to all of them.
- Do not make changes beyond what the review comments ask for. Stay focused.
- If a comment suggests a change you disagree with, present your reasoning to the user during the planning phase rather than silently ignoring it.
- Follow IronClaw conventions: no `.unwrap()` in production code, use `crate::` imports, `thiserror` errors.
- Follow OptimClaw conventions: no `.unwrap()` in production code, use `crate::` imports, `thiserror` errors.
- If changes touch persistence, verify both database backends are updated.
+6 -6
View File
@@ -1,5 +1,5 @@
---
description: Deep audit of the IronClaw crate for vulnerabilities, bugs, unfinished work, inconsistencies, and oversights
description: Deep audit of the OptimClaw crate for vulnerabilities, bugs, unfinished work, inconsistencies, and oversights
disable-model-invocation: true
allowed-tools: Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*), Bash(cargo audit:*), Bash(git diff:*), Bash(git log:*), Bash(git show:*), Bash(wc:*), Read, Grep, Glob, Task
argument-hint: "[path/to/crate]"
@@ -93,7 +93,7 @@ Search for `.unwrap()`, `.expect(`, `panic!`, `unreachable!` in non-test code. F
- Is there a code path that reaches this with None/Err?
- Should it be replaced with proper error handling (`?`, `.ok()`, `.unwrap_or_default()`)?
IronClaw convention: `.unwrap()` and `.expect()` are banned in production code. Any occurrence outside `#[cfg(test)]` blocks is a **High severity** finding.
OptimClaw convention: `.unwrap()` and `.expect()` are banned in production code. Any occurrence outside `#[cfg(test)]` blocks is a **High severity** finding.
### 5c. SQL and injection vectors
@@ -102,7 +102,7 @@ Search for string formatting used in SQL queries, shell commands, or HTML:
- String interpolation in query construction vs parameterized queries
- User input flowing into file paths (`Path::new`, `std::fs::`)
IronClaw has two database backends (PostgreSQL and libSQL). Check both for injection vectors.
OptimClaw has two database backends (PostgreSQL and libSQL). Check both for injection vectors.
### 5d. Cryptographic issues
@@ -125,7 +125,7 @@ If the crate uses crypto:
- Are errors swallowed silently? (`let _ = ...`, `.ok()` discarding errors that matter)
- Do error types carry enough context to debug in production?
- Are there error type mismatches? (returning generic `anyhow::Error` where a typed error would prevent confusion)
- Is `thiserror` used consistently for error types (IronClaw convention)?
- Is `thiserror` used consistently for error types (OptimClaw convention)?
## Step 6: Check for inconsistencies
@@ -156,7 +156,7 @@ Look for:
### 6e. Import style
IronClaw convention: use `crate::` imports, not `super::`. Flag any `super::` imports in non-test code.
OptimClaw convention: use `crate::` imports, not `super::`. Flag any `super::` imports in non-test code.
## Step 7: Inspect for change oversights
@@ -172,7 +172,7 @@ IronClaw convention: use `crate::` imports, not `super::`. Flag any `super::` im
- Are there `impl` blocks that look incomplete?
- Are `Default` implementations sensible?
IronClaw key traits: `Database` (~60 methods), `Channel`, `Tool`, `LlmProvider`, `SuccessEvaluator`, `EmbeddingProvider`. If any new methods were added to `Database`, verify both `postgres.rs` and `libsql_backend.rs` implement them.
OptimClaw key traits: `Database` (~60 methods), `Channel`, `Tool`, `LlmProvider`, `SuccessEvaluator`, `EmbeddingProvider`. If any new methods were added to `Database`, verify both `postgres.rs` and `libsql_backend.rs` implement them.
### 7c. Test coverage gaps
+2 -2
View File
@@ -49,9 +49,9 @@ If the PR touches more than 20 files, still read all of them, but process in thi
Go through the changes with each of these lenses. For every finding, note the file, line range, severity, and a concrete description.
### IronClaw-specific checks
### OptimClaw-specific checks
In addition to the general lenses below, check IronClaw conventions (see CLAUDE.md):
In addition to the general lenses below, check OptimClaw conventions (see CLAUDE.md):
- No `.unwrap()` or `.expect()` in production code (tests are fine)
- Use `crate::` imports, not `super::`
- Error types use `thiserror` in `error.rs`
+1 -1
View File
@@ -3,7 +3,7 @@ description: Run the full Rust quality gate (fmt, clippy, tests) before shipping
allowed-tools: Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*)
---
Run the IronClaw shipping checklist. This is the mandatory quality gate before any change is considered done.
Run the OptimClaw shipping checklist. This is the mandatory quality gate before any change is considered done.
## Steps
+3 -3
View File
@@ -1,15 +1,15 @@
---
description: Trace a data flow or bug through the IronClaw codebase end-to-end
description: Trace a data flow or bug through the OptimClaw codebase end-to-end
allowed-tools: Read, Glob, Grep, Bash(cargo test:*)
argument-hint: <symptom or feature name>
model: sonnet
---
Trace the flow of `$ARGUMENTS` through the IronClaw codebase. Your job is to map every file and function involved, identify where data transforms or could break, and report the full chain.
Trace the flow of `$ARGUMENTS` through the OptimClaw codebase. Your job is to map every file and function involved, identify where data transforms or could break, and report the full chain.
## Architecture Reference
IronClaw has three main data flow paths. Identify which one(s) are relevant and trace through them:
OptimClaw has three main data flow paths. Identify which one(s) are relevant and trace through them:
### Message Flow (user input to LLM response)
```
+2 -2
View File
@@ -11,8 +11,8 @@ SKILL.md files extend the agent's prompt with domain-specific instructions. Each
| Trust Level | Source | Tool Access |
|-------------|--------|-------------|
| **Trusted** | User-placed in `~/.ironclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry (`~/.ironclaw/installed_skills/`) | Read-only tools only (no shell, file write, HTTP) |
| **Trusted** | User-placed in `~/.optimclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry (`~/.optimclaw/installed_skills/`) | Read-only tools only (no shell, file write, HTTP) |
## SKILL.md Format
+1 -1
View File
@@ -7,7 +7,7 @@ paths:
**Keep tool-specific logic out of the main agent codebase.** The main agent provides generic infrastructure; tools are self-contained units that declare requirements through `<name>.capabilities.json` sidecar files (in dev mode: `tools-src/<name>/<name>-tool.capabilities.json`).
Tools can be WASM (sandboxed, credential-injected, single binary) or MCP servers (ecosystem, any language, no sandbox). Both are first-class via `ironclaw tool install`.
Tools can be WASM (sandboxed, credential-injected, single binary) or MCP servers (ecosystem, any language, no sandbox). Both are first-class via `optimclaw tool install`.
See `src/tools/README.md` for full architecture, adding new tools, auth JSON examples, and WASM vs MCP decision guide.
+15 -15
View File
@@ -1,5 +1,5 @@
# Database Configuration
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_URL=postgres://localhost/optimclaw
DATABASE_POOL_SIZE=10
# LLM Provider
@@ -26,19 +26,19 @@ DATABASE_POOL_SIZE=10
# === GitHub Copilot ===
# Uses the OAuth token from your Copilot IDE sign-in (for example
# ~/.config/github-copilot/apps.json on Linux/macOS), or run `ironclaw onboard`
# ~/.config/github-copilot/apps.json on Linux/macOS), or run `optimclaw onboard`
# and choose the GitHub device login flow.
# LLM_BACKEND=github_copilot
# GITHUB_COPILOT_TOKEN=gho_...
# GITHUB_COPILOT_MODEL=gpt-4o
# IronClaw injects standard VS Code Copilot headers automatically.
# OptimClaw injects standard VS Code Copilot headers automatically.
# Optional advanced headers for custom overrides:
# GITHUB_COPILOT_EXTRA_HEADERS=Copilot-Integration-Id:vscode-chat
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
# Session token stored in ~/.ironclaw/session.json automatically.
# Session token stored in ~/.optimclaw/session.json automatically.
# Base URL defaults to https://private.near.ai
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
# Base URL defaults to https://cloud-api.near.ai
@@ -46,7 +46,7 @@ NEARAI_MODEL=Qwen/Qwen3.5-122B-A10B
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_SESSION_PATH=~/.optimclaw/session.json # optional, default shown
# NEARAI_API_KEY=... # API key from cloud.near.ai
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
@@ -63,7 +63,7 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_API_KEY=sk-... # optional for local servers
# Custom HTTP headers for OpenAI-compatible providers
# Format: comma-separated key:value pairs
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/optimclaw,X-Title:optimclaw
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
@@ -145,7 +145,7 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Webhook authentication uses HMAC-SHA256 signature verification.
# Callers must send an X-IronClaw-Signature header with format: sha256=<hex_digest>
# Callers must send an X-OptimClaw-Signature header with format: sha256=<hex_digest>
# where the digest is HMAC-SHA256(HTTP_WEBHOOK_SECRET, raw_request_body) in lowercase hex.
#
# Example (bash):
@@ -153,7 +153,7 @@ HTTP_WEBHOOK_SECRET=your-webhook-secret
# SIG=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "$HTTP_WEBHOOK_SECRET" | cut -d' ' -f2)
# curl -X POST http://localhost:8080/webhook \
# -H "Content-Type: application/json" \
# -H "X-IronClaw-Signature: sha256=$SIG" \
# -H "X-OptimClaw-Signature: sha256=$SIG" \
# -d "$BODY"
#
# DEPRECATED: Passing "secret" in the JSON body still works but will be removed in a future release.
@@ -170,7 +170,7 @@ HTTP_WEBHOOK_SECRET=your-webhook-secret
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=ironclaw
AGENT_NAME=optimclaw
AGENT_MAX_PARALLEL_JOBS=5
AGENT_JOB_TIMEOUT_SECS=3600
AGENT_STUCK_THRESHOLD_SECS=300
@@ -205,7 +205,7 @@ HEARTBEAT_NOTIFY_USER=default
# # commands directly on the host. Without this
# # set to "true", full_access is downgraded to
# # workspace_write.
# SANDBOX_IMAGE=ironclaw-worker:latest
# SANDBOX_IMAGE=optimclaw-worker:latest
# SANDBOX_TIMEOUT_SECS=120
# SANDBOX_MEMORY_LIMIT_MB=2048
@@ -214,11 +214,11 @@ SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
# Restart Feature (Docker containers only)
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Set OPTIMCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Without this, the restart tool and /restart command will be disabled.
# IRONCLAW_IN_DOCKER=false
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# OPTIMCLAW_IN_DOCKER=false
# OPTIMCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# OPTIMCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Logging
RUST_LOG=ironclaw=debug,tower_http=debug
RUST_LOG=optimclaw=debug,tower_http=debug
+12 -5
View File
@@ -6,7 +6,7 @@
## Change Type
<!-- Check one -->
<!-- Check all that apply. Refactor-only PRs are for core team or maintainer-requested work. -->
- [ ] Bug fix
- [ ] New feature
@@ -18,16 +18,19 @@
## Linked Issue
<!-- Closes #N, or "None" -->
<!-- Closes #N, Fixes #N, Related #N, or "None". New feature PRs must link an approved issue. -->
## Validation
<!-- How did you verify this works? -->
- [ ] `cargo fmt`
- [ ] `cargo clippy --all --benches --tests --examples --all-features`
- [ ] `cargo fmt --all -- --check`
- [ ] `cargo clippy --all --benches --tests --examples --all-features -- -D warnings`
- [ ] `cargo build`
- [ ] Relevant tests pass: <!-- list specific tests -->
- [ ] `cargo test --features integration` if database-backed or integration behavior changed
- [ ] Manual testing: <!-- describe what you tested -->
- [ ] If a coding agent was used and supports it, `review-pr` or `pr-shepherd --fix` was run before requesting review
## Security Impact
@@ -45,6 +48,10 @@
<!-- How to revert if this causes problems? For Track C changes, this is mandatory. -->
## Review Follow-Through
<!-- Review conversations are author-owned. Summarize any known follow-up or areas where reviewer judgment is still needed. -->
---
**Review track**: <!-- A (docs/tests/chore) | B (feature/refactor) | C (security/runtime/DB/CI) -->
**Review track**: <!-- A (docs/tests/chore) | B (feature/maintainer-requested refactor) | C (security/runtime/DB/CI) -->
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
allowed_bots: "ironclaw-ci[bot]"
allowed_bots: "optimclaw-ci[bot]"
claude_args: "--max-turns 50 --model claude-haiku-4-5-20251001 --allowedTools 'Read,Glob,Grep,Agent,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*),Bash(git blame:*),Bash(git log:*),Bash(git diff:*)'"
prompt: |
Code review this pull request. Follow these steps precisely:
+6 -6
View File
@@ -6,11 +6,11 @@
# What it does:
# - Runs unit and integration tests with coverage instrumentation
# - Runs E2E tests with coverage instrumentation
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/ironclaw)
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/optimclaw)
#
# Viewing coverage reports:
# - PRs automatically get coverage comments showing changes in coverage
# - Visit https://codecov.io/gh/nearai/ironclaw for detailed coverage reports
# - Visit https://codecov.io/gh/nearai/optimclaw for detailed coverage reports
# - Coverage reports are generated for three configurations:
# 1. all-features: Full feature set
# 2. default: Default features
@@ -58,7 +58,7 @@ jobs:
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: ironclaw_test
POSTGRES_DB: optimclaw_test
ports:
- 5432:5432
options: >-
@@ -103,11 +103,11 @@ jobs:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: ironclaw_test
PGDATABASE: optimclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/optimclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
@@ -176,7 +176,7 @@ jobs:
run: |
pytest tests/e2e/ -v --timeout=120
env:
RUST_LOG: ironclaw=info
RUST_LOG: optimclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
+5 -5
View File
@@ -14,7 +14,7 @@ on:
jobs:
# ── Step 1: compile once ──────────────────────────────────────────────────
build:
name: Build ironclaw (libsql)
name: Build optimclaw (libsql)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -35,8 +35,8 @@ jobs:
- name: Upload binary
uses: actions/upload-artifact@v4
with:
name: ironclaw-e2e-binary
path: target/debug/ironclaw
name: optimclaw-e2e-binary
path: target/debug/optimclaw
retention-days: 1
# ── Step 2: run test slices in parallel ───────────────────────────────────
@@ -63,11 +63,11 @@ jobs:
- name: Download binary
uses: actions/download-artifact@v4
with:
name: ironclaw-e2e-binary
name: optimclaw-e2e-binary
path: target/debug/
- name: Make binary executable
run: chmod +x target/debug/ironclaw
run: chmod +x target/debug/optimclaw
- uses: actions/setup-python@v5
with:
+1 -1
View File
@@ -56,7 +56,7 @@ jobs:
"src/agent/self_repair.rs"
"src/agent/agentic_loop.rs"
"src/tools/execute.rs"
"crates/ironclaw_safety/src/"
"crates/optimclaw_safety/src/"
)
for pattern in "${HIGH_RISK_PATTERNS[@]}"; do
+2 -2
View File
@@ -166,7 +166,7 @@ jobs:
continue
fi
name=$(echo "$filename" | sed "s/^${kind}-//" | sed 's/-[0-9].*-wasm32-wasip2\.tar\.gz$//')
url="https://github.com/nearai/ironclaw/releases/download/${RELEASE_TAG}/${filename}"
url="https://github.com/nearai/optimclaw/releases/download/${RELEASE_TAG}/${filename}"
manifest="registry/${kind}s/${name}.json"
if [ -f "$manifest" ]; then
@@ -496,7 +496,7 @@ jobs:
continue
fi
name=$(echo "$filename" | sed "s/^${kind}-//" | sed 's/-[0-9].*-wasm32-wasip2\.tar\.gz$//')
url="https://github.com/nearai/ironclaw/releases/download/${RELEASE_TAG}/${filename}"
url="https://github.com/nearai/optimclaw/releases/download/${RELEASE_TAG}/${filename}"
manifest="registry/${kind}s/${name}.json"
if [ -f "$manifest" ]; then
+1 -1
View File
@@ -165,7 +165,7 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t ironclaw-test:ci .
run: docker build -t optimclaw-test:ci .
version-check:
name: Version Bump Check
-1
View File
@@ -39,4 +39,3 @@ __pycache__/
*.pyc
*.pyo
*.pyd
engine_trace_*.json
+520 -520
View File
File diff suppressed because it is too large Load Diff
+12 -15
View File
@@ -1,6 +1,6 @@
# IronClaw Development Guide
# OptimClaw Development Guide
**IronClaw** is a secure personal AI assistant — user-first security, self-expanding tools, defense in depth, multi-channel access with proactive background execution.
**OptimClaw** is a secure personal AI assistant — user-first security, self-expanding tools, defense in depth, multi-channel access with proactive background execution.
## Build & Test
@@ -9,7 +9,7 @@ cargo fmt # format
cargo clippy --all --benches --tests --examples --all-features # lint (zero warnings)
cargo test # unit tests
cargo test --features integration # + PostgreSQL tests
RUST_LOG=ironclaw=debug cargo run # run with logging
RUST_LOG=optimclaw=debug cargo run # run with logging
```
E2E tests: see `tests/e2e/CLAUDE.md`.
@@ -24,8 +24,6 @@ E2E tests: see `tests/e2e/CLAUDE.md`.
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
- **Prompt templates live in files, not Rust code**: Multi-line prompt strings (mission goals, system prompts, CodeAct preambles) go in `crates/ironclaw_engine/prompts/*.md` and are loaded via `include_str!()`. Never inline large prompt templates as Rust string constants — they're hard to read, review, and iterate on. Single-line format strings are fine inline.
- **Logging levels matter for REPL/TUI**: `info!` and `warn!` output appears in the REPL and corrupts the terminal UI. Use `debug!` for internal diagnostics (trace analysis, reflection results, engine internals). Reserve `info!` for user-facing status that the REPL intentionally renders. Background tasks (reflection, trace analysis) must NEVER use `info!` — it breaks the interactive display.
## Architecture
@@ -37,20 +35,20 @@ All I/O is async with tokio. Use `Arc<T>` for shared state, `RwLock` for concurr
## Extracted Crates
Safety logic lives in `crates/ironclaw_safety/`, skills in `crates/ironclaw_skills/`. **Import directly from the extracted crate** (e.g. `use ironclaw_safety::SafetyLayer`, `use ironclaw_skills::SkillRegistry`). Do not use `crate::safety::` or `crate::skills::` for types that originate in extracted crates — `src/safety/mod.rs` and `src/skills/mod.rs` no longer glob-re-export. Local items defined in those modules (e.g. `crate::skills::attenuate_tools`) are fine.
Safety logic lives in `crates/optimclaw_safety/`. The `src/safety/mod.rs` shim re-exports everything for backward compatibility, but **new code should import from `optimclaw_safety` directly** (e.g. `use optimclaw_safety::SafetyLayer`). When touching a file that still uses `crate::safety::*`, migrate its imports to `optimclaw_safety::*`.
## Project Structure
```
crates/
└── ironclaw_safety/ # Extracted: prompt injection, validation, leak detection, policy
└── optimclaw_safety/ # Extracted: prompt injection, validation, leak detection, policy
src/
├── lib.rs # Library root, module declarations
├── main.rs # Entry point, CLI args, startup
├── app.rs # App startup orchestration (channel wiring, DB init)
├── bootstrap.rs # Base directory resolution (~/.ironclaw), early .env loading
├── settings.rs # User settings persistence (~/.ironclaw/settings.json)
├── bootstrap.rs # Base directory resolution (~/.optimclaw), early .env loading
├── settings.rs # User settings persistence (~/.optimclaw/settings.json)
├── service.rs # OS service management (launchd/systemd daemon install)
├── tracing_fmt.rs # Custom tracing formatter
├── util.rs # Shared utilities
@@ -113,7 +111,7 @@ src/
│ ├── claude_bridge.rs # Claude Code bridge (spawns claude CLI)
│ └── proxy_llm.rs # LlmProvider that proxies through orchestrator
├── safety/ # Re-export shim for crates/ironclaw_safety (see Extracted Crates)
├── safety/ # Re-export shim for crates/optimclaw_safety (see Extracted Crates)
├── llm/ # Multi-provider LLM integration — see src/llm/CLAUDE.md
@@ -193,7 +191,6 @@ When modifying a module with a spec, read the spec first. Code follows spec; spe
| `src/setup/` | `src/setup/README.md` |
| `src/tools/` | `src/tools/README.md` |
| `src/workspace/` | `src/workspace/README.md` |
| `crates/ironclaw_engine/` | `crates/ironclaw_engine/CLAUDE.md` |
| `tests/e2e/` | `tests/e2e/CLAUDE.md` |
## Job State Machine
@@ -209,7 +206,7 @@ Pending -> InProgress -> Completed -> Submitted -> Accepted
SKILL.md files extend the agent's prompt with domain-specific instructions. See `.claude/rules/skills.md` for full details.
- **Trust model**: Trusted (user-placed in `~/.ironclaw/skills/` or workspace `skills/`, full tool access) vs Installed (registry, read-only tools)
- **Trust model**: Trusted (user-placed in `~/.optimclaw/skills/` or workspace `skills/`, full tool access) vs Installed (registry, read-only tools)
- **Selection pipeline**: gating (check bin/env/config requirements) -> scoring (keywords/patterns/tags) -> budget (fit within `SKILLS_MAX_TOKENS`) -> attenuation (trust-based tool ceiling)
- **Skill tools**: `skill_list`, `skill_search`, `skill_install`, `skill_remove`
@@ -231,9 +228,9 @@ Persistent memory with hybrid search (FTS + vector via RRF). Four tools: `memory
## Debugging
```bash
RUST_LOG=ironclaw=trace cargo run # verbose
RUST_LOG=ironclaw::agent=debug cargo run # agent module only
RUST_LOG=ironclaw=debug,tower_http=debug cargo run # + HTTP request logging
RUST_LOG=optimclaw=trace cargo run # verbose
RUST_LOG=optimclaw::agent=debug cargo run # agent module only
RUST_LOG=optimclaw=debug,tower_http=debug cargo run # + HTTP request logging
```
## Current Limitations
+79 -4
View File
@@ -3,13 +3,49 @@
## Getting Started
```bash
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
./scripts/dev-setup.sh
```
This installs the Rust toolchain, WASM targets, git hooks, and runs initial checks.
## How to Contribute
- Bug fixes, docs improvements, and focused cleanup tied to a concrete problem are welcome.
- Search existing issues and PRs before opening a new one to avoid duplicates.
- Keep changes scoped. One bug, one feature, or one documentation improvement per PR.
### Creating Issues
Open an issue when you are reporting a bug, proposing a feature, or documenting a gap in behavior.
For bug reports, include:
- What you expected to happen
- What actually happened
- Clear reproduction steps
- Relevant logs, screenshots, or error output
- Environment details when they matter (OS, database backend, feature flags, commit/branch)
For feature requests:
- Open an issue first before writing code
- Explain the problem being solved, not just the implementation idea
- Wait for maintainer feedback before investing in a large PR
We require an issue for new features so maintainers can prioritize the work and confirm it fits the roadmap before anyone spends time implementing it.
### Fixing Bugs
- Small, targeted bug-fix PRs are welcome
- If there is already an issue, link it in your PR
- If the bug is non-trivial, security-sensitive, or changes behavior across subsystems, open or confirm an issue first so the approach can be aligned before implementation
### Refactor-Only PRs
Refactor-only PRs are not accepted from contributors outside the core team. If a refactor is necessary to land a bug fix or approved feature, keep it minimal and clearly tied to that change.
## Development Workflow
```bash
@@ -19,6 +55,45 @@ cargo test # unit tests
cargo test --features integration # + PostgreSQL tests
```
These commands are for day-to-day iteration while you are developing locally. The pre-submission checks below are intentionally stricter and use CI-style flags so you can catch formatting drift and clippy warnings before requesting review.
## Before You Open a PR
Run the local validation checks required before requesting a review. These are stricter than the commands for iterative development:
```bash
cargo fmt --all -- --check
cargo clippy --all --benches --tests --examples --all-features -- -D warnings
cargo build
cargo test
```
Also run this when your change touches database-backed or integration behavior:
```bash
cargo test --features integration
```
Before asking for review:
- Build and exercise the changed path locally, not just the narrowest unit test
- Keep the PR focused and avoid mixing unrelated concerns
- Fill out the PR template with a clear summary, validation notes, and impact assessment
- If your change affects tracked behavior, update `FEATURE_PARITY.md` in the same branch
- If onboarding or setup behavior changes, update the relevant setup docs in the same branch
- If you are using a coding agent and it supports them, run `review-pr` or `pr-shepherd --fix` before opening or updating the PR
- `codex review --base origin/main` is also encouraged before requesting review
## Review Follow-Through
Review conversations are author-owned.
- Address each review comment with a code change or a clear explanation
- Resolve conversations you have handled; leave them open only when reviewer judgment is still needed
- Do not leave review cleanup for maintainers when the follow-through belongs to the author
If a PR is stale for more than 48 hours after review feedback is posted, maintainers may take over the follow-up work and land the changes needed to accomplish the original PR or issue intent.
## Code Style
- Zero clippy warnings policy
@@ -46,14 +121,14 @@ All PRs follow a risk-based review process:
| Track | Scope | Requirements |
|-------|-------|-------------|
| **A** | Docs, tests, chore, dependency bumps | 1 approval + CI green |
| **B** | Features, refactors, new tools/channels | 1 approval + CI green + test evidence |
| **B** | Features, maintainer-requested refactors, new tools/channels | 1 approval + CI green + test evidence |
| **C** | Security (`src/safety/`, `src/secrets/`), runtime (`src/agent/`, `src/worker/`), database schema, CI workflows | 2 approvals + rollback plan documented |
Select the appropriate track in the PR template based on what your changes touch.
## Database Changes
IronClaw uses dual-backend persistence (PostgreSQL + libSQL). All new persistence features must support both backends. See `src/db/CLAUDE.md`.
OptimClaw uses dual-backend persistence (PostgreSQL + libSQL). All new persistence features must support both backends. See `src/db/CLAUDE.md`.
## Adding Dependencies
+2 -2
View File
@@ -1,6 +1,6 @@
# IronClaw Coverage Plan: 63.3% to 95%
# OptimClaw Coverage Plan: 63.3% to 95%
> Generated 2025-03-06 from [Codecov](https://app.codecov.io/gh/nearai/ironclaw/tree/main/src)
> Generated 2025-03-06 from [Codecov](https://app.codecov.io/gh/nearai/optimclaw/tree/main/src)
## Current State
Generated
+250 -622
View File
File diff suppressed because it is too large Load Diff
+25 -16
View File
@@ -1,5 +1,5 @@
[workspace]
members = [".", "crates/ironclaw_common", "crates/ironclaw_safety", "crates/ironclaw_skills", "crates/ironclaw_engine"]
members = [".", "crates/optimclaw_common", "crates/optimclaw_safety"]
exclude = [
"channels-src/discord",
"channels-src/telegram",
@@ -15,19 +15,19 @@ exclude = [
"tools-src/slack",
"tools-src/telegram",
"fuzz",
"crates/ironclaw_safety/fuzz",
"crates/optimclaw_safety/fuzz",
]
[package]
name = "ironclaw"
name = "optimclaw"
version = "0.22.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
homepage = "https://github.com/nearai/optimclaw"
repository = "https://github.com/nearai/optimclaw"
[package.metadata.wix]
upgrade-guid = "D0156E61-BA37-451E-8AB9-1A2ECCCFA48F"
@@ -40,6 +40,7 @@ eula = false
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
tokio-tungstenite = { version = "0.26", features = ["rustls-tls-native-roots"] }
eventsource-stream = "0.2"
# HTTP client
@@ -57,6 +58,7 @@ refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
webpki-roots = { version = "0.26", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
@@ -95,18 +97,16 @@ termimad = "0.34"
# Channel integrations
axum = { version = "0.8", features = ["ws"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["trace", "cors", "set-header"] }
tower-http = { version = "0.6", features = ["trace", "cors", "set-header", "catch-panic"] }
# Cron scheduling for routines
cron = "0.13"
# Shared types
ironclaw_common = { path = "crates/ironclaw_common", version = "0.1.0" }
optimclaw_common = { path = "crates/optimclaw_common", version = "0.1.0" }
# Safety/sanitization
ironclaw_engine = { path = "crates/ironclaw_engine" }
ironclaw_safety = { path = "crates/ironclaw_safety", version = "0.2.0" }
ironclaw_skills = { path = "crates/ironclaw_skills", version = "0.1.0" }
optimclaw_safety = { path = "crates/optimclaw_safety", version = "0.2.0" }
regex = "1"
aho-corasick = "1"
@@ -186,22 +186,30 @@ hex = "0.4.3"
# OpenClaw import (feature gated)
json5 = { version = "0.4", optional = true }
# Mesh cluster (feature gated)
pqcrypto-kyber = { version = "0.8", optional = true }
pqcrypto-traits = { version = "0.3", optional = true }
sys-info = { version = "0.9", optional = true }
hostname = { version = "0.4", optional = true }
quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring"], optional = true }
rcgen = { version = "0.13", optional = true }
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
security-framework = "3"
# PTY allocation for Claude CLI stdout buffering fix (Unix only)
[target.'cfg(unix)'.dependencies]
pty-process = { version = "0.5", features = ["async"] }
# Linux secret-service (GNOME Keyring, KWallet)
[target.'cfg(target_os = "linux")'.dependencies]
secret-service = { version = "4", features = ["rt-tokio-crypto-rust"] }
zbus = "4"
[build-dependencies]
serde_json = "1"
[dev-dependencies]
tokio-test = "0.4"
tracing-test = "0.2"
tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
@@ -224,6 +232,7 @@ postgres = [
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:webpki-roots",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
@@ -236,6 +245,7 @@ integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
import = ["dep:json5", "libsql"]
cluster = ["dep:pqcrypto-kyber", "dep:pqcrypto-traits", "dep:sys-info", "dep:hostname", "dep:quinn", "dep:rcgen", "dep:rustls"]
[[test]]
name = "e2e_thread_scheduling"
@@ -251,8 +261,7 @@ strip = true # Remove debug symbols from release binaries
# The profile that 'cargo dist' will build with
[profile.dist]
inherits = "release"
lto = "fat" # Full cross-crate LTO (slow build, better codegen)
codegen-units = 1 # Single codegen unit for maximum optimization
lto = "thin"
# Config for 'dist'
[workspace.metadata.dist]
+34 -8
View File
@@ -1,45 +1,71 @@
# Multi-stage Dockerfile for the IronClaw agent (cloud deployment).
#
# Uses cargo-chef for dependency caching — only rebuilds deps when
# Cargo.toml/Cargo.lock change, not on every source edit.
#
# Build:
# docker build --platform linux/amd64 -t ironclaw:latest .
#
# Run:
# docker run --env-file .env -p 3000:3000 ironclaw:latest
# Stage 1: Build
FROM rust:1.92-slim-bookworm AS builder
# Stage 1: Install cargo-chef
FROM rust:1.92-slim-bookworm AS chef
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install wasm-tools
&& cargo install cargo-chef wasm-tools
WORKDIR /app
# Copy manifests first for layer caching
# Stage 2: Generate the dependency recipe (changes only when Cargo.toml/lock change)
FROM chef AS planner
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY benches/ benches/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
# [[bench]] entries in Cargo.toml require bench sources to exist for cargo to parse the manifest
RUN cargo chef prepare --recipe-path recipe.json
# Stage 3: Build dependencies (cached unless Cargo.toml/lock change)
FROM chef AS deps
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
# Stage 4: Build the actual binary (only recompiles ironclaw source)
FROM deps AS builder
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY benches/ benches/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
RUN cargo build --release --bin ironclaw
# Stage 2: Runtime
# Stage 5: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& update-ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
+38 -38
View File
@@ -1,6 +1,6 @@
# IronClaw ↔ OpenClaw Feature Parity Matrix
# OptimClaw ↔ OpenClaw Feature Parity Matrix
This document tracks feature parity between IronClaw (Rust implementation) and OpenClaw (TypeScript reference implementation). Use this to coordinate work across developers.
This document tracks feature parity between OptimClaw (Rust implementation) and OpenClaw (TypeScript reference implementation). Use this to coordinate work across developers.
**Legend:**
@@ -17,7 +17,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 1. Architecture
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Hub-and-spoke architecture | ✅ | ✅ | Web gateway as central hub |
| WebSocket control plane | ✅ | ✅ | Gateway with WebSocket + SSE |
@@ -32,7 +32,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 2. Gateway System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Gateway control plane | ✅ | ✅ | Web gateway with 40+ API endpoints |
| HTTP endpoints for Control UI | ✅ | ✅ | Web dashboard with chat, memory, jobs, logs, extensions |
@@ -62,15 +62,15 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 3. Messaging Channels
| Channel | OpenClaw | IronClaw | Priority | Notes |
| Channel | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| CLI/TUI | ✅ | ✅ | - | Ratatui-based TUI |
| HTTP webhook | ✅ | ✅ | - | axum with secret validation |
| REPL (simple) | ✅ | ✅ | - | For testing |
| WASM channels | ❌ | ✅ | - | IronClaw innovation; host resolves owner scope vs sender identity |
| WASM channels | ❌ | ✅ | - | OptimClaw innovation; host resolves owner scope vs sender identity |
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username, DM topics, setup-time owner auto-verification, owner-scoped persistence |
| Discord | ✅ | | P2 | discord.js, thread parent binding inheritance |
| Discord | ✅ | 🚧 | P2 | Gateway `MESSAGE_CREATE` intake restored via websocket queue + WASM poll; Gateway DMs now respect pairing; thread parent binding inheritance and reply/thread parity still incomplete |
| Signal | ✅ | ✅ | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
@@ -88,7 +88,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### Telegram-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Forum topic creation | ✅ | ❌ | Create topics in forum groups |
| channel_post support | ✅ | ❌ | Bot-to-bot communication |
@@ -100,7 +100,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### Discord-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Forwarded attachment downloads | ✅ | ❌ | Fetch media from forwarded messages |
| Faster reaction state machine | ✅ | ❌ | Watchdog + debounce |
@@ -108,7 +108,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### Slack-Specific Features (since Feb 2025)
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Streaming draft replies | ✅ | ❌ | Partial replies via draft message updates |
| Configurable stream modes | ✅ | ❌ | Per-channel stream behavior |
@@ -117,23 +117,23 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### Mattermost-Specific Features (since Mar 2026)
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Interactive buttons | ✅ | ❌ | Clickable message buttons with signed callback flow |
| Interactive model picker | ✅ | ❌ | In-channel provider/model chooser |
### Feishu/Lark-Specific Features (since Mar 2026)
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Doc/table actions | ✅ | ❌ | `feishu_doc` supports tables, positional insert, color_text, image upload, and file upload |
| Rich-text embedded media extraction | ✅ | ❌ | Pull video/media attachments from post messages |
### Channel Features
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| DM pairing codes | ✅ | ✅ | `ironclaw pairing list/approve`, host APIs |
| DM pairing codes | ✅ | ✅ | `optimclaw pairing list/approve`, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | `allow_from` + pairing store + hardened command/group allowlists |
| Self-message bypass | ✅ | ❌ | Own messages skip pairing |
| Mention-based activation | ✅ | ✅ | bot_username + respond_to_all_group_messages |
@@ -151,7 +151,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 4. CLI Commands
| Command | OpenClaw | IronClaw | Priority | Notes |
| Command | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| `run` (agent) | ✅ | ✅ | - | Default command |
| `tool install/list/remove` | ✅ | ✅ | - | WASM tools |
@@ -189,9 +189,9 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 5. Agent System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Pi agent runtime | ✅ | | IronClaw uses custom runtime |
| Pi agent runtime | ✅ | | OptimClaw uses custom runtime |
| RPC-based execution | ✅ | ✅ | Orchestrator/worker pattern |
| Multi-provider failover | ✅ | ✅ | `FailoverProvider` tries providers sequentially on retryable errors |
| Per-sender sessions | ✅ | ✅ | |
@@ -232,7 +232,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 6. Model & Provider Support
| Provider | OpenClaw | IronClaw | Priority | Notes |
| Provider | OpenClaw | OptimClaw | Priority | Notes |
|----------|----------|----------|----------|-------|
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6, adaptive thinking default |
@@ -245,7 +245,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (OptimClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
| GitHub Copilot | ✅ | ✅ | - | Dedicated provider with OAuth token exchange (`GithubCopilotProvider`) |
| Ollama (local) | ✅ | ✅ | - | via `rig::providers::ollama` (full support) |
@@ -257,7 +257,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
### Model Features
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Auto-discovery | ✅ | ❌ | |
| Failover chains | ✅ | ✅ | `FailoverProvider` with configurable `fallback_model` |
@@ -273,7 +273,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 7. Media Handling
| Feature | OpenClaw | IronClaw | Priority | Notes |
| Feature | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Image processing (Sharp) | ✅ | ❌ | P2 | Resize, format convert |
| Configurable image resize dims | ✅ | ❌ | P2 | Per-agent dimension config |
@@ -296,12 +296,12 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 8. Plugin & Extension System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Dynamic loading | ✅ | ✅ | WASM modules |
| Manifest validation | ✅ | ✅ | WASM metadata |
| HTTP path registration | ✅ | ❌ | Plugin routes |
| Workspace-relative install | ✅ | ✅ | ~/.ironclaw/tools/ |
| Workspace-relative install | ✅ | ✅ | ~/.optimclaw/tools/ |
| Channel plugins | ✅ | ✅ | WASM channels |
| Auth plugins | ✅ | ❌ | |
| Memory plugins | ✅ | ❌ | Custom backends + selectable memory slot |
@@ -321,7 +321,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 9. Configuration System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Primary config file | ✅ `~/.openclaw/openclaw.json` | ✅ `.env` | Different formats |
| JSON5 support | ✅ | ❌ | Comments, trailing commas |
@@ -330,7 +330,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Config validation/schema | ✅ | ✅ | Type-safe Config struct + `openclaw config validate` |
| Hot-reload | ✅ | ❌ | |
| Legacy migration | ✅ | | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.ironclaw/` | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.optimclaw/` | |
| Credentials directory | ✅ | ✅ | Session files |
| Full model compat fields in schema | ✅ | ❌ | pi-ai model compat exposed in config |
@@ -340,7 +340,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 10. Memory & Knowledge System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Vector memory | ✅ | ✅ | pgvector |
| Session-based memory | ✅ | ✅ | |
@@ -351,7 +351,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| OpenAI embeddings | ✅ | ✅ | |
| Gemini embeddings | ✅ | ❌ | |
| Local embeddings | ✅ | ❌ | |
| SQLite-vec backend | ✅ | ❌ | IronClaw uses PostgreSQL |
| SQLite-vec backend | ✅ | ❌ | OptimClaw uses PostgreSQL |
| LanceDB backend | ✅ | ❌ | Configurable auto-capture max length |
| QMD backend | ✅ | ❌ | |
| Atomic reindexing | ✅ | ✅ | |
@@ -369,7 +369,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 11. Mobile Apps
| Feature | OpenClaw | IronClaw | Priority | Notes |
| Feature | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| iOS app (SwiftUI) | ✅ | 🚫 | - | Out of scope initially |
| Android app (Kotlin) | ✅ | 🚫 | - | Out of scope initially |
@@ -390,7 +390,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 12. macOS App
| Feature | OpenClaw | IronClaw | Priority | Notes |
| Feature | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| SwiftUI native app | ✅ | 🚫 | - | Out of scope |
| Menu bar presence | ✅ | 🚫 | - | Animated menubar icon |
@@ -411,7 +411,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 13. Web Interface
| Feature | OpenClaw | IronClaw | Priority | Notes |
| Feature | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Control UI Dashboard | ✅ | ✅ | - | Web gateway with chat, memory, jobs, logs, extensions |
| Channel status view | ✅ | 🚧 | P2 | Gateway status widget, full channel view pending |
@@ -431,7 +431,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 14. Automation
| Feature | OpenClaw | IronClaw | Priority | Notes |
| Feature | OpenClaw | OptimClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Cron jobs | ✅ | ✅ | - | Routines with cron trigger |
| Per-job model fallback override | ✅ | ❌ | P2 | `payload.fallbacks` overrides agent-level fallbacks |
@@ -465,14 +465,14 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 15. Security Features
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Gateway token auth | ✅ | ✅ | Bearer token auth on web gateway |
| Device pairing | ✅ | ❌ | |
| Tailscale identity | ✅ | ❌ | |
| Trusted-proxy auth | ✅ | ❌ | Header-based reverse proxy auth |
| OAuth flows | ✅ | 🚧 | NEAR AI OAuth + Gemini OAuth (PKCE, S256) + hosted extension/MCP OAuth broker; external auth-proxy rollout still pending |
| DM pairing verification | ✅ | ✅ | ironclaw pairing approve, host APIs |
| DM pairing verification | ✅ | ✅ | optimclaw pairing approve, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| Per-group tool policies | ✅ | ❌ | |
| Exec approvals | ✅ | ✅ | TUI overlay |
@@ -483,7 +483,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Loopback-first | ✅ | 🚧 | HTTP binds 0.0.0.0 |
| Docker sandbox | ✅ | ✅ | Orchestrator/worker containers |
| Podman support | ✅ | ❌ | Alternative to Docker |
| WASM sandbox | ❌ | ✅ | IronClaw innovation |
| WASM sandbox | ❌ | ✅ | OptimClaw innovation |
| Sandbox env sanitization | ✅ | 🚧 | Shell tool scrubs env vars (secret detection); docker container env sanitization partial |
| Tool policies | ✅ | ✅ | |
| Elevated mode | ✅ | ❌ | |
@@ -505,7 +505,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## 16. Development & Build System
| Feature | OpenClaw | IronClaw | Notes |
| Feature | OpenClaw | OptimClaw | Notes |
|---------|----------|----------|-------|
| Primary language | TypeScript | Rust | Different ecosystems |
| Build tool | tsdown | cargo | |
@@ -531,7 +531,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ✅ TUI channel with approval overlays
- ✅ HTTP webhook channel
- ✅ DM pairing (ironclaw pairing list/approve, host APIs)
- ✅ DM pairing (optimclaw pairing list/approve, host APIs)
- ✅ WASM tool sandbox
- ✅ Workspace/memory with hybrid search + embeddings batching
- ✅ Prompt injection defense
@@ -605,7 +605,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## Deviations from OpenClaw
IronClaw intentionally differs from OpenClaw in these ways:
OptimClaw intentionally differs from OpenClaw in these ways:
1. **Rust vs TypeScript**: Native performance, memory safety, single binary distribution
2. **WASM sandbox vs Docker**: Lighter weight, faster startup, capability-based security
@@ -613,7 +613,7 @@ IronClaw intentionally differs from OpenClaw in these ways:
4. **NEAR AI focus**: Primary provider with session-based auth
5. **No mobile/desktop apps**: Focus on server-side and CLI initially
6. **WASM channels**: Novel extension mechanism not in OpenClaw
7. **Tinfoil private inference**: IronClaw-only provider for private/encrypted inference
7. **Tinfoil private inference**: OptimClaw-only provider for private/encrypted inference
8. **GitHub WASM tool**: Native GitHub integration as WASM tool
9. **Prompt-based skills**: Different approach than OpenClaw capability bundles (trust gating, attenuation)
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
<h1 align="center">OptimClaw</h1>
<p align="center">
<strong>あなたの味方になる、安全なパーソナルAIアシスタント</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## フィロソフィー
IronClawはシンプルな原則に基づいて構築されています:**あなたのAIアシスタントは、あなたのために働くべきであり、あなたに不利益をもたらすべきではありません。**
OptimClawはシンプルな原則に基づいて構築されています:**あなたのAIアシスタントは、あなたのために働くべきであり、あなたに不利益をもたらすべきではありません。**
AIシステムがデータの取り扱いについて不透明になり、企業の利益に沿って調整されることが増えている世界で、IronClawは異なるアプローチを取ります:
AIシステムがデータの取り扱いについて不透明になり、企業の利益に沿って調整されることが増えている世界で、OptimClawは異なるアプローチを取ります:
- **あなたのデータはあなたのもの** - すべての情報はローカルに保存・暗号化され、あなたの管理下から離れることはありません
- **設計段階からの透明性** - オープンソース、監査可能、隠れたテレメトリやデータ収集なし
- **自己拡張する能力** - ベンダーのアップデートを待たずに、新しいツールをその場で構築
- **多層防御** - 複数のセキュリティレイヤーがプロンプトインジェクションやデータ流出から保護
IronClawは、個人生活にも仕事にも本当に信頼できるAIアシスタントです。
OptimClawは、個人生活にも仕事にも本当に信頼できるAIアシスタントです。
## 機能
@@ -66,7 +66,7 @@ IronClawは、個人生活にも仕事にも本当に信頼できるAIアシス
### 自己拡張
- **動的ツール構築** - 必要なものを説明すると、IronClawがWASMツールとして構築
- **動的ツール構築** - 必要なものを説明すると、OptimClawがWASMツールとして構築
- **MCPプロトコル** - Model Context Protocolサーバーに接続して追加機能を利用
- **プラグインアーキテクチャ** - 再起動なしで新しいWASMツールやチャネルを追加
@@ -86,12 +86,12 @@ IronClawは、個人生活にも仕事にも本当に信頼できるAIアシス
## ダウンロードまたはビルド
最新のアップデートは[リリースページ](https://github.com/nearai/ironclaw/releases/)をご覧ください。
最新のアップデートは[リリースページ](https://github.com/nearai/optimclaw/releases/)をご覧ください。
<details>
<summary>Windowsインストーラーでインストール(Windows</summary>
[Windowsインストーラー](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi)をダウンロードして実行してください。
[Windowsインストーラー](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi)をダウンロードして実行してください。
</details>
@@ -99,7 +99,7 @@ IronClawは、個人生活にも仕事にも本当に信頼できるAIアシス
<summary>PowerShellスクリプトでインストール(Windows</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-install
<summary>シェルスクリプトでインストール(macOS、Linux、Windows/WSL</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
<summary>Homebrewでインストール(macOS/Linux</summary>
```sh
brew install ironclaw
brew install optimclaw
```
</details>
@@ -128,8 +128,8 @@ brew install ironclaw
```bash
# リポジトリをクローン
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
# ビルド
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# データベースを作成
createdb ironclaw
createdb optimclaw
# pgvectorを有効化
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## 設定
セットアップウィザードを実行してIronClawを設定します:
セットアップウィザードを実行してOptimClawを設定します:
```bash
ironclaw onboard
optimclaw onboard
```
ウィザードは、データベース接続、NEAR AI認証(ブラウザOAuth経由)、シークレットの暗号化(システムキーチェーンを使用)を処理します。設定は接続されたデータベースに永続化されます。ブートストラップ変数(例:`DATABASE_URL``LLM_BACKEND`)は、データベース接続前に利用できるよう`~/.ironclaw/.env`に書き込まれます。
ウィザードは、データベース接続、NEAR AI認証(ブラウザOAuth経由)、シークレットの暗号化(システムキーチェーンを使用)を処理します。設定は接続されたデータベースに永続化されます。ブートストラップ変数(例:`DATABASE_URL``LLM_BACKEND`)は、データベース接続前に利用できるよう`~/.optimclaw/.env`に書き込まれます。
### 代替LLMプロバイダー
IronClawはデフォルトでNEAR AIを使用しますが、多くのLLMプロバイダーをすぐに利用できます。組み込みプロバイダーには**Anthropic**、**OpenAI**、**Google Gemini**、**MiniMax**、**Mistral**、**Ollama**(ローカル)が含まれます。**OpenRouter**300以上のモデル)、**Together AI**、**Fireworks AI**、セルフホストサーバー(**vLLM**、**LiteLLM**)などのOpenAI互換サービスもサポートされています。
OptimClawはデフォルトでNEAR AIを使用しますが、多くのLLMプロバイダーをすぐに利用できます。組み込みプロバイダーには**Anthropic**、**OpenAI**、**Google Gemini**、**MiniMax**、**Mistral**、**Ollama**(ローカル)が含まれます。**OpenRouter**300以上のモデル)、**Together AI**、**Fireworks AI**、セルフホストサーバー(**vLLM**、**LiteLLM**)などのOpenAI互換サービスもサポートされています。
ウィザードでプロバイダーを選択するか、環境変数を直接設定してください:
@@ -184,7 +184,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## セキュリティ
IronClawは、データを保護し悪用を防ぐために多層防御を実装しています。
OptimClawは、データを保護し悪用を防ぐために多層防御を実装しています。
### WASMサンドボックス
@@ -280,13 +280,13 @@ WASM ──► 許可リスト ──► リーク ──► 認証情報 ─
```bash
# 初回セットアップ(データベース、認証などを設定)
ironclaw onboard
optimclaw onboard
# インタラクティブREPLを起動
cargo run
# デバッグログ付き
RUST_LOG=ironclaw=debug cargo run
RUST_LOG=optimclaw=debug cargo run
```
## 開発
@@ -299,7 +299,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# テスト実行
createdb ironclaw_test
createdb optimclaw_test
cargo test
# 特定のテストを実行
@@ -311,7 +311,7 @@ cargo test test_name
## OpenClawの系譜
IronClawは[OpenClaw](https://github.com/openclaw/openclaw)にインスパイアされたRust再実装です。完全な対応表は[FEATURE_PARITY.md](FEATURE_PARITY.md)をご覧ください。
OptimClawは[OpenClaw](https://github.com/openclaw/openclaw)にインスパイアされたRust再実装です。完全な対応表は[FEATURE_PARITY.md](FEATURE_PARITY.md)をご覧ください。
主な違い:
+74 -28
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
<h1 align="center">OptimClaw</h1>
<p align="center">
<strong>Your secure personal AI assistant, always on your side</strong>
@@ -10,10 +10,10 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
<a href="https://gitcgr.com/nearai/ironclaw">
<img src="https://gitcgr.com/badge/nearai/ironclaw.svg" alt="gitcgr" />
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
<a href="https://gitcgr.com/nearai/optimclaw">
<img src="https://gitcgr.com/badge/nearai/optimclaw.svg" alt="gitcgr" />
</a>
</p>
@@ -27,6 +27,8 @@
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
<a href="#mesh-cluster">Mesh Cluster</a> •
<a href="#lazy-tools">Lazy Tools</a> •
<a href="#installation">Installation</a> •
<a href="#configuration">Configuration</a> •
<a href="#security">Security</a> •
@@ -37,16 +39,16 @@
## Philosophy
IronClaw is built on a simple principle: **your AI assistant should work for you, not against you**.
OptimClaw is built on a simple principle: **your AI assistant should work for you, not against you**.
In a world where AI systems are increasingly opaque about data handling and aligned with corporate interests, IronClaw takes a different approach:
In a world where AI systems are increasingly opaque about data handling and aligned with corporate interests, OptimClaw takes a different approach:
- **Your data stays yours** - All information is stored locally, encrypted, and never leaves your control
- **Transparency by design** - Open source, auditable, no hidden telemetry or data harvesting
- **Self-expanding capabilities** - Build new tools on the fly without waiting for vendor updates
- **Defense in depth** - Multiple security layers protect against prompt injection and data exfiltration
IronClaw is the AI assistant you can actually trust with your personal and professional life.
OptimClaw is the AI assistant you can actually trust with your personal and professional life.
## Features
@@ -69,7 +71,7 @@ IronClaw is the AI assistant you can actually trust with your personal and profe
### Self-Expanding
- **Dynamic Tool Building** - Describe what you need, and IronClaw builds it as a WASM tool
- **Dynamic Tool Building** - Describe what you need, and OptimClaw builds it as a WASM tool
- **MCP Protocol** - Connect to Model Context Protocol servers for additional capabilities
- **Plugin Architecture** - Drop in new WASM tools and channels without restarting
@@ -79,6 +81,50 @@ IronClaw is the AI assistant you can actually trust with your personal and profe
- **Workspace Filesystem** - Flexible path-based storage for notes, logs, and context
- **Identity Files** - Maintain consistent personality and preferences across sessions
## Mesh Cluster
OptimClaw instances can form an **autonomous AI mesh network** where nodes discover each other automatically, coordinate via a gossip protocol, and route tasks intelligently across the cluster.
Key highlights:
- **Zero-config discovery** -- UDP beacon broadcast finds peers on the local network automatically
- **Post-quantum encryption** -- ML-KEM-768 key exchange with AES-256-GCM authenticated encryption protects all inter-node traffic against both classical and quantum adversaries
- **SWIM gossip membership** -- Reliable failure detection and cluster state convergence in O(log N) rounds
- **Intelligent task routing** -- A scoring algorithm balances load, latency, capability match, session affinity, and region locality to pick the best node for each task
- **Graceful degradation** -- Nodes operate independently if connectivity is lost; no split-brain data corruption
Quick start (two nodes on one machine):
```bash
# Terminal 1
export CLUSTER_ENABLED=true CLUSTER_SECRET="your-32-char-secret-here-change-me" CLUSTER_NODE_ID=node-a
cargo run
# Terminal 2
export CLUSTER_ENABLED=true CLUSTER_SECRET="your-32-char-secret-here-change-me" CLUSTER_NODE_ID=node-b CLUSTER_BIND_PORT=9410
cargo run
```
Monitor via `GET /api/mesh/status` and `GET /api/mesh/nodes`.
See [docs/MESH_CLUSTER.md](docs/MESH_CLUSTER.md) for the full guide covering architecture, configuration reference, security model, and troubleshooting.
## Lazy Tools
Lazy tool loading reduces the system prompt from approximately 13,000 tokens to approximately 4,000 tokens by deferring tool schemas that are not immediately needed.
Enable it with:
```bash
export OPTIMCLAW_LAZY_TOOLS=1
```
When enabled, 12 core tools (echo, time, json, http, web_fetch, file_read, file_write, shell, memory_search, memory_write, message, tool_info) are loaded eagerly. All other tools -- including MCP, WASM, and skill tools -- are listed by name only. The LLM calls `tool_info` to load the full schema for any additional tool on demand.
This is recommended for production deployments and cost-sensitive usage with expensive models.
See [docs/LAZY_TOOLS.md](docs/LAZY_TOOLS.md) for the full guide.
## Installation
### Prerequisites
@@ -89,12 +135,12 @@ IronClaw is the AI assistant you can actually trust with your personal and profe
## Download or Build
Visit [Releases page](https://github.com/nearai/ironclaw/releases/) to see the latest updates.
Visit [Releases page](https://github.com/nearai/optimclaw/releases/) to see the latest updates.
<details>
<summary>Install via Windows Installer (Windows)</summary>
Download the [Windows Installer](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) and run it.
Download the [Windows Installer](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) and run it.
</details>
@@ -102,7 +148,7 @@ Download the [Windows Installer](https://github.com/nearai/ironclaw/releases/lat
<summary>Install via powershell script (Windows)</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
```
</details>
@@ -111,7 +157,7 @@ irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-install
<summary>Install via shell script (macOS, Linux, Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
```
</details>
@@ -119,7 +165,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
brew install optimclaw
```
</details>
@@ -131,8 +177,8 @@ Install it with `cargo`, just make sure you have [Rust](https://rustup.rs) insta
```bash
# Clone the repository
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
# Build
cargo build --release
@@ -149,28 +195,28 @@ For **full release** (after modifying channel sources), run `./scripts/build-all
```bash
# Create database
createdb ironclaw
createdb optimclaw
# Enable pgvector
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## Configuration
Run the setup wizard to configure IronClaw:
Run the setup wizard to configure OptimClaw:
```bash
ironclaw onboard
optimclaw onboard
```
The wizard handles database connection, NEAR AI authentication (via browser OAuth),
and secrets encryption (using your system keychain). Settings are persisted in the
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.ironclaw/.env` so they are available before the database connects.
written to `~/.optimclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
IronClaw defaults to NEAR AI but supports many LLM providers out of the box.
OptimClaw defaults to NEAR AI but supports many LLM providers out of the box.
Built-in providers include **Anthropic**, **OpenAI**, **GitHub Copilot**, **Google Gemini**, **MiniMax**,
**Mistral**, and **Ollama** (local). OpenAI-compatible services like **OpenRouter**
(300+ models), **Together AI**, **Fireworks AI**, and self-hosted servers (**vLLM**,
@@ -194,7 +240,7 @@ See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
## Security
IronClaw implements defense in depth to protect your data and prevent misuse.
OptimClaw implements defense in depth to protect your data and prevent misuse.
### WASM Sandbox
@@ -287,13 +333,13 @@ External content passes through multiple security layers:
```bash
# First-time setup (configures database, auth, etc.)
ironclaw onboard
optimclaw onboard
# Start interactive REPL
cargo run
# With debug logging
RUST_LOG=ironclaw=debug cargo run
RUST_LOG=optimclaw=debug cargo run
```
## Development
@@ -306,7 +352,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# Run tests
createdb ironclaw_test
createdb optimclaw_test
cargo test
# Run specific test
@@ -318,7 +364,7 @@ cargo test test_name
## OpenClaw Heritage
IronClaw is a Rust reimplementation inspired by [OpenClaw](https://github.com/openclaw/openclaw). See [FEATURE_PARITY.md](FEATURE_PARITY.md) for the complete tracking matrix.
OptimClaw is a Rust reimplementation inspired by [OpenClaw](https://github.com/openclaw/openclaw). See [FEATURE_PARITY.md](FEATURE_PARITY.md) for the complete tracking matrix.
Key differences:
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
<h1 align="center">OptimClaw</h1>
<p align="center">
<strong>Ваш защищенный персональный AI-ассистент, всегда на вашей стороне</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="Лицензия: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## Философия
IronClaw построен на простом принципе: **ваш AI-ассистент должен работать на вас, а не против вас**.
OptimClaw построен на простом принципе: **ваш AI-ассистент должен работать на вас, а не против вас**.
В мире, где системы ИИ становятся все более непрозрачными в вопросах обработки данных и ориентируются на корпоративные интересы, IronClaw выбирает другой путь:
В мире, где системы ИИ становятся все более непрозрачными в вопросах обработки данных и ориентируются на корпоративные интересы, OptimClaw выбирает другой путь:
- **Ваши данные остаются вашими** — вся информация хранится локально, зашифрована и никогда не покидает ваш контроль.
- **Прозрачность по умолчанию** — открытый исходный код, возможность аудита, отсутствие скрытой телеметрии или сбора данных.
- **Саморасширяемые возможности** — создавайте новые инструменты «на лету», не дожидаясь обновлений от вендора.
- **Глубокая защита** — несколько уровней безопасности защищают от инъекций промптов и утечки данных.
IronClaw — это AI-ассистент, которому вы действительно можете доверять в личной и профессиональной жизни.
OptimClaw — это AI-ассистент, которому вы действительно можете доверять в личной и профессиональной жизни.
## Возможности
@@ -66,7 +66,7 @@ IronClaw — это AI-ассистент, которому вы действи
### Саморасширяемый
- **Динамическое создание инструментов** — опишите, что вам нужно, и IronClaw создаст это как инструмент WASM.
- **Динамическое создание инструментов** — опишите, что вам нужно, и OptimClaw создаст это как инструмент WASM.
- **Протокол MCP** — подключайтесь к серверам Model Context Protocol для получения дополнительных возможностей.
- **Плагинная архитектура** — добавляйте новые инструменты WASM и каналы без перезагрузки системы.
@@ -86,12 +86,12 @@ IronClaw — это AI-ассистент, которому вы действи
## Загрузка и сборка
Посетите [страницу релизов](https://github.com/nearai/ironclaw/releases/), чтобы увидеть последние обновления.
Посетите [страницу релизов](https://github.com/nearai/optimclaw/releases/), чтобы увидеть последние обновления.
<details>
<summary>Установка через установщик Windows (Windows)</summary>
Загрузите [Windows Installer](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) и запустите его.
Загрузите [Windows Installer](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) и запустите его.
</details>
@@ -99,7 +99,7 @@ IronClaw — это AI-ассистент, которому вы действи
<summary>Установка через powershell-скрипт (Windows)</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-install
<summary>Установка через shell-скрипт (macOS, Linux, Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
<summary>Установка через Homebrew (macOS/Linux)</summary>
```sh
brew install ironclaw
brew install optimclaw
```
</details>
@@ -128,8 +128,8 @@ brew install ironclaw
```bash
# Клонируйте репозиторий
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
# Сборка
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# Создание базы данных
createdb ironclaw
createdb optimclaw
# Включение pgvector
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## Конфигурация
Запустите мастер настройки для конфигурации IronClaw:
Запустите мастер настройки для конфигурации OptimClaw:
```bash
ironclaw onboard
optimclaw onboard
```
Мастер настройки поможет установить соединение с базой данных, пройти аутентификацию NEAR AI (через браузер OAuth) и настроить шифрование секретов (используя системную связку ключей). Настройки сохраняются в базе данных; базовые переменные (например, `DATABASE_URL`, `LLM_BACKEND`) записываются в `~/.ironclaw/.env`, чтобы они были доступны до подключения к БД.
Мастер настройки поможет установить соединение с базой данных, пройти аутентификацию NEAR AI (через браузер OAuth) и настроить шифрование секретов (используя системную связку ключей). Настройки сохраняются в базе данных; базовые переменные (например, `DATABASE_URL`, `LLM_BACKEND`) записываются в `~/.optimclaw/.env`, чтобы они были доступны до подключения к БД.
### Альтернативные LLM-провайдеры
IronClaw по умолчанию использует NEAR AI, но поддерживает множество LLM-провайдеров из коробки.
OptimClaw по умолчанию использует NEAR AI, но поддерживает множество LLM-провайдеров из коробки.
Встроенные провайдеры включают **Anthropic**, **OpenAI**, **Google Gemini**, **MiniMax**,
**Mistral** и **Ollama** (локально). Также поддерживаются OpenAI-совместимые сервисы:
**OpenRouter** (300+ моделей), **Together AI**, **Fireworks AI** и собственные серверы
@@ -188,7 +188,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## Безопасность
IronClaw реализует эшелонированную защиту для обеспечения безопасности ваших данных и предотвращения злоупотреблений.
OptimClaw реализует эшелонированную защиту для обеспечения безопасности ваших данных и предотвращения злоупотреблений.
### Песочница WASM
@@ -282,13 +282,13 @@ WASM ──► Валидатор ──► Сканер ───► Инъек
```bash
# Первоначальная настройка (БД, аутентификация и т.д.)
ironclaw onboard
optimclaw onboard
# Запуск интерактивного REPL
cargo run
# С отладочными логами
RUST_LOG=ironclaw=debug cargo run
RUST_LOG=optimclaw=debug cargo run
```
## Разработка
@@ -301,7 +301,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# Запуск тестов
createdb ironclaw_test
createdb optimclaw_test
cargo test
# Запуск конкретного теста
@@ -313,7 +313,7 @@ cargo test название_теста
## Наследие OpenClaw
IronClaw — это реализация на Rust, вдохновленная проектом [OpenClaw](https://github.com/openclaw/openclaw). Полную матрицу соответствия функций можно найти в [FEATURE_PARITY.md](FEATURE_PARITY.md).
OptimClaw — это реализация на Rust, вдохновленная проектом [OpenClaw](https://github.com/openclaw/openclaw). Полную матрицу соответствия функций можно найти в [FEATURE_PARITY.md](FEATURE_PARITY.md).
Ключевые отличия:
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
</p>
<h1 align="center">IronClaw</h1>
<h1 align="center">OptimClaw</h1>
<p align="center">
<strong>安全可靠的个人 AI 助手,始终站在你这边</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## 设计理念
IronClaw 基于一个简单的原则:**你的 AI 助手应该为你服务,而不是与你为敌。**
OptimClaw 基于一个简单的原则:**你的 AI 助手应该为你服务,而不是与你为敌。**
在 AI 系统对数据处理日益不透明、与企业利益捆绑的今天,IronClaw 选择了一条不同的路:
在 AI 系统对数据处理日益不透明、与企业利益捆绑的今天,OptimClaw 选择了一条不同的路:
- **数据归你所有** — 所有信息存储在本地,加密保护,始终在你掌控之下
- **透明至上** — 完全开源,可审计,没有隐藏的遥测或数据收集
- **自主扩展** — 随时构建新工具,无需等待供应商更新
- **纵深防御** — 多层安全机制抵御提示注入和数据泄露
IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还是工作。
OptimClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还是工作。
## 功能特性
@@ -66,7 +66,7 @@ IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
### 自主扩展
- **动态工具构建** — 描述你的需求,IronClaw 会将其构建为 WASM 工具
- **动态工具构建** — 描述你的需求,OptimClaw 会将其构建为 WASM 工具
- **MCP 协议** — 连接模型上下文协议(Model Context Protocol)服务器以获取额外能力
- **插件架构** — 无需重启即可加载新的 WASM 工具和渠道
@@ -86,12 +86,12 @@ IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
## 下载或编译
访问 [Releases 页面](https://github.com/nearai/ironclaw/releases/) 查看最新版本。
访问 [Releases 页面](https://github.com/nearai/optimclaw/releases/) 查看最新版本。
<details>
<summary>通过 Windows 安装程序安装 (Windows)</summary>
下载 [Windows 安装程序](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) 并运行。
下载 [Windows 安装程序](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) 并运行。
</details>
@@ -99,7 +99,7 @@ IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
<summary>通过 PowerShell 脚本安装 (Windows)</summary>
```sh
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-install
<summary>通过 Shell 脚本安装 (macOS、Linux、Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/release
<summary>通过 Homebrew 安装 (macOS/Linux)</summary>
```sh
brew install ironclaw
brew install optimclaw
```
</details>
@@ -128,8 +128,8 @@ brew install ironclaw
```bash
# 克隆仓库
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
# 编译
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# 创建数据库
createdb ironclaw
createdb optimclaw
# 启用 pgvector 扩展
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## 配置
运行设置向导来配置 IronClaw
运行设置向导来配置 OptimClaw
```bash
ironclaw onboard
optimclaw onboard
```
向导将引导你完成数据库连接、NEAR AI 身份验证(通过浏览器 OAuth)和密钥加密(使用系统钥匙串)。设置会保存在数据库中;引导变量(如 `DATABASE_URL``LLM_BACKEND`)写入 `~/.ironclaw/.env`,以便在数据库连接前可用。
向导将引导你完成数据库连接、NEAR AI 身份验证(通过浏览器 OAuth)和密钥加密(使用系统钥匙串)。设置会保存在数据库中;引导变量(如 `DATABASE_URL``LLM_BACKEND`)写入 `~/.optimclaw/.env`,以便在数据库连接前可用。
### 替代 LLM 提供商
IronClaw 默认使用 NEAR AI,但开箱即用地支持多种 LLM 提供商。
OptimClaw 默认使用 NEAR AI,但开箱即用地支持多种 LLM 提供商。
内置提供商包括 **Anthropic**、**OpenAI**、**GitHub Copilot**、**Google Gemini**、**MiniMax**、**Mistral** 和 **Ollama**(本地部署)。同时也支持 OpenAI 兼容服务,如 **OpenRouter**300+ 模型)、**Together AI**、**Fireworks AI** 以及自托管服务器(**vLLM**、**LiteLLM**)。
在向导中选择你的提供商,或直接设置环境变量:
@@ -185,7 +185,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## 安全机制
IronClaw 实现了纵深防御策略来保护你的数据并防止滥用。
OptimClaw 实现了纵深防御策略来保护你的数据并防止滥用。
### WASM 沙箱
@@ -278,13 +278,13 @@ WASM ──► 白名单 ──► 泄露扫描 ──► 凭据 ──► 执
```bash
# 首次设置(配置数据库、认证等)
ironclaw onboard
optimclaw onboard
# 启动交互式 REPL
cargo run
# 启用调试日志
RUST_LOG=ironclaw=debug cargo run
RUST_LOG=optimclaw=debug cargo run
```
## 开发
@@ -297,7 +297,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# 运行测试
createdb ironclaw_test
createdb optimclaw_test
cargo test
# 运行指定测试
@@ -309,7 +309,7 @@ cargo test test_name
## OpenClaw 传承
IronClaw 是受 [OpenClaw](https://github.com/openclaw/openclaw) 启发的 Rust 重新实现。参见 [FEATURE_PARITY.md](FEATURE_PARITY.md) 了解完整的功能追踪矩阵。
OptimClaw 是受 [OpenClaw](https://github.com/openclaw/openclaw) 启发的 Rust 重新实现。参见 [FEATURE_PARITY.md](FEATURE_PARITY.md) 了解完整的功能追踪矩阵。
主要差异:
+1 -1
View File
@@ -1,5 +1,5 @@
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use ironclaw_safety::{LeakDetector, Sanitizer, Validator};
use optimclaw::safety::{LeakDetector, Sanitizer, Validator};
fn bench_sanitizer(c: &mut Criterion) {
let mut group = c.benchmark_group("sanitizer");
+2 -2
View File
@@ -1,6 +1,6 @@
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use ironclaw::config::SafetyConfig;
use ironclaw_safety::{SafetyLayer, Validator};
use optimclaw::config::SafetyConfig;
use optimclaw::safety::{SafetyLayer, Validator};
fn bench_safety_layer_pipeline(c: &mut Criterion) {
let mut group = c.benchmark_group("safety_pipeline");
+2 -58
View File
@@ -20,9 +20,6 @@ fn main() {
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Embed bundled skills ────────────────────────────────────────────
embed_skills(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
@@ -128,7 +125,7 @@ fn embed_registry_catalog(root: &Path) {
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); // safety: build script
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
@@ -180,60 +177,7 @@ fn embed_registry_catalog(root: &Path) {
bundles_raw,
);
fs::write(&out_path, catalog).unwrap(); // safety: build script
}
/// Collect all `skills/*/SKILL.md` files into an embedded JSON blob.
///
/// Output: `$OUT_DIR/embedded_skills.json` — a JSON array of `{"name": "...", "content": "..."}`.
/// These are loaded at runtime as bundled skills (lowest discovery priority, Trusted trust level).
fn embed_skills(root: &Path) {
use std::fs;
let skills_dir = root.join("skills");
// Rerun when any skill changes
println!("cargo:rerun-if-changed=skills");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); // safety: build script panics on failure
let out_path = out_dir.join("embedded_skills.json");
if !skills_dir.is_dir() {
fs::write(&out_path, "[]").unwrap(); // safety: build script
return;
}
let mut skills: Vec<String> = Vec::new();
let mut entries: Vec<_> = fs::read_dir(&skills_dir)
.unwrap() // safety: build script
.filter_map(|e| e.ok())
.filter(|e| e.path().is_dir())
.collect();
entries.sort_by_key(|e| e.file_name());
for entry in entries {
let skill_md = entry.path().join("SKILL.md");
if !skill_md.is_file() {
continue;
}
// Emit per-file watch
println!("cargo:rerun-if-changed={}", skill_md.display());
let name = entry.file_name().to_string_lossy().to_string();
if let Ok(content) = fs::read_to_string(&skill_md) {
// Escape for JSON embedding
let name_json = serde_json::to_string(&name).unwrap(); // safety: build script
let content_json = serde_json::to_string(&content).unwrap(); // safety: build script
skills.push(format!(
r#"{{"name":{},"content":{}}}"#,
name_json, content_json
));
}
}
let catalog = format!("[{}]", skills.join(","));
fs::write(&out_path, catalog).unwrap(); // safety: build script
fs::write(&out_path, catalog).unwrap();
}
/// Read all .json files from a directory and push their raw contents into `out`.
+11 -216
View File
@@ -20,162 +20,33 @@ version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "const-oid"
version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be"
dependencies = [
"cfg-if",
"cpufeatures",
"curve25519-dalek-derive",
"digest",
"fiat-crypto",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek-derive"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "der"
version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid",
"zeroize",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]]
name = "discord-channel"
version = "0.2.0"
version = "0.2.1"
dependencies = [
"ed25519-dalek",
"hex",
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "fiat-crypto"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
@@ -197,12 +68,6 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hex"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "id-arena"
version = "2.3.0"
@@ -223,9 +88,9 @@ dependencies = [
[[package]]
name = "itoa"
version = "1.0.17"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "leb128"
@@ -233,12 +98,6 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "libc"
version = "0.2.182"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112"
[[package]]
name = "log"
version = "0.4.29"
@@ -253,19 +112,9 @@ checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.3"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "pkcs8"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
]
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "prettyplease"
@@ -288,22 +137,13 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.44"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustc_version"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
dependencies = [
"semver",
]
[[package]]
name = "semver"
version = "1.0.27"
@@ -353,23 +193,6 @@ dependencies = [
"zmij",
]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "signature"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
[[package]]
name = "smallvec"
version = "1.15.1"
@@ -385,22 +208,6 @@ dependencies = [
"smallvec",
]
[[package]]
name = "spki"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
]
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
@@ -412,12 +219,6 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "typenum"
version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb"
[[package]]
name = "unicode-ident"
version = "1.0.24"
@@ -575,30 +376,24 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.39"
version = "0.8.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
checksum = "efbb2a062be311f2ba113ce66f697a4dc589f85e78a4aea276200804cea0ed87"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
version = "0.8.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
checksum = "0e8bc7269b54418e7aeeef514aa68f8690b8c0489a06b0136e5f57c4c5ccab89"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zeroize"
version = "1.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
[[package]]
name = "zmij"
version = "1.0.21"
+2 -4
View File
@@ -1,8 +1,8 @@
[package]
name = "discord-channel"
version = "0.2.0"
version = "0.2.1"
edition = "2021"
description = "Discord channel for IronClaw"
description = "Discord channel for OptimClaw"
license = "MIT OR Apache-2.0"
publish = false
@@ -10,8 +10,6 @@ publish = false
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.36"
ed25519-dalek = { version = "2", default-features = false, features = ["alloc", "fast", "zeroize"] }
hex = "0.4"
[lib]
crate-type = ["cdylib"]
+25 -6
View File
@@ -1,4 +1,4 @@
# Discord Channel for IronClaw
# Discord Channel for OptimClaw
WASM channel for Discord integration - handle slash commands and button interactions via webhooks.
@@ -13,12 +13,12 @@ WASM channel for Discord integration - handle slash commands and button interact
1. Create a Discord Application at <https://discord.com/developers/applications>
2. Create a Bot and get the token
3. Set up Interactions URL to point to your IronClaw instance
3. Set up Interactions URL to point to your OptimClaw instance
4. Copy the Application ID and Public Key
5. Store in IronClaw secrets:
5. Store in OptimClaw secrets:
```bash
ironclaw secret set discord_bot_token YOUR_BOT_TOKEN
optimclaw secret set discord_bot_token YOUR_BOT_TOKEN
```
**Note:** The `discord_bot_token` secret is used for Discord REST API calls.
@@ -51,7 +51,7 @@ curl -X POST \
In your Discord app settings, set:
- Interactions Endpoint URL: `https://your-ironclaw.com/webhook/discord`
- Interactions Endpoint URL: `https://your-optimclaw.com/webhook/discord`
## Usage Examples
@@ -86,6 +86,24 @@ If an internal error occurs (e.g., metadata serialization failure), the tool att
Check the host logs for detailed error information.
## Advanced Usage
### Gateway Mode
The Discord channel now defaults to Discord Gateway transport for inbound message intake.
The bundled identify payload requests intents `4609`, which expands to:
- `GUILDS` (`1`)
- `GUILD_MESSAGES` (`512`)
- `DIRECT_MESSAGES` (`4096`)
Gateway DMs now follow the same pairing policy as webhook DMs. Unpaired users receive a pairing
instruction reply in the DM channel before the message is allowed through to the agent. If you
want stricter access control than pairing, set `owner_id`; that lock still applies to both
webhook and Gateway traffic.
Gateway presence simply reflects a successful authenticated Gateway connection and advertises
`online`. Pairing still controls whether DMs are allowed through to the agent, but it no longer
changes the visible Discord status.
### Mention Polling
The Discord channel can also poll configured channels for `@bot` mentions.
@@ -110,6 +128,7 @@ Example channel config:
- `owner_id`: when set, only that Discord user can interact with the bot.
- `dm_policy`: `open` allows all DMs; `pairing` requires approval.
- `allow_from`: allowlist entries for DM pairing checks (`*`, user id, or username).
- Gateway DMs respect `dm_policy` and pairing just like webhook DMs.
### Embeds
@@ -127,7 +146,7 @@ To send embeds, include an `embeds` array in the `metadata_json` field of the ag
### "401 Unauthorized"
- Check that `discord_bot_token` is set correctly in IronClaw secrets.
- Check that `discord_bot_token` is set correctly in OptimClaw secrets.
- Ensure the bot is added to the server.
### "Interaction Failed"
+17 -2
View File
@@ -1,5 +1,5 @@
{
"version": "0.2.0",
"version": "0.2.1",
"wit_version": "0.3.0",
"type": "channel",
"name": "discord",
@@ -22,7 +22,8 @@
"capabilities": {
"http": {
"allowlist": [
{ "host": "discord.com", "path_prefix": "/api/v10" }
{ "host": "discord.com", "path_prefix": "/api/v10" },
{ "host": "gateway.discord.gg", "path_prefix": "/", "methods": ["GET"] }
],
"credentials": {
"discord_bot_token": {
@@ -36,6 +37,20 @@
"requests_per_hour": 3600
}
},
"websocket": {
"url": "wss://gateway.discord.gg/?v=10&encoding=json",
"connect_on_start": true,
"identify_secret_name": "discord_bot_token",
"identify": {
"_intents_doc": "GUILDS(1) + GUILD_MESSAGES(512) + DIRECT_MESSAGES(4096)",
"intents": 4609,
"properties": {
"os": "linux",
"browser": "ironclaw",
"device": "ironclaw"
}
}
},
"secrets": {
"allowed_names": ["discord_bot_token", "discord_*"]
},
File diff suppressed because it is too large Load Diff
+7
View File
@@ -44,6 +44,7 @@ version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"subtle",
"wit-bindgen",
]
@@ -208,6 +209,12 @@ dependencies = [
"smallvec",
]
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
+2 -1
View File
@@ -2,7 +2,7 @@
name = "feishu-channel"
version = "0.1.0"
edition = "2021"
description = "Feishu/Lark Bot channel for IronClaw"
description = "Feishu/Lark Bot channel for OptimClaw"
license = "MIT OR Apache-2.0"
[lib]
@@ -15,6 +15,7 @@ wit-bindgen = "0.36"
# Serialization
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
subtle = "2.6"
# Exclude from parent workspace (this is a standalone WASM component)
+5 -3
View File
@@ -7,7 +7,7 @@
"auth": {
"secret_name": "feishu_app_id",
"display_name": "Feishu / Lark",
"instructions": "Create a bot at https://open.feishu.cn/app (Feishu) or https://open.larksuite.com/app (Lark). You need the App ID and App Secret. Note: IronClaw supports Event Subscription webhook delivery, but not Feishu's long-connection websocket mode.",
"instructions": "Create a bot at https://open.feishu.cn/app (Feishu) or https://open.larksuite.com/app (Lark). You need the App ID and App Secret. Note: OptimClaw supports Event Subscription webhook delivery, but not Feishu's long-connection websocket mode.",
"setup_url": "https://open.feishu.cn/app",
"token_hint": "App ID looks like cli_XXXX, App Secret is a long alphanumeric string",
"env_var": "FEISHU_APP_ID"
@@ -27,7 +27,7 @@
{
"name": "feishu_verification_token",
"prompt": "Enter your Feishu/Lark Verification Token (from Event Subscription webhook settings)",
"optional": true
"optional": false
}
],
"setup_url": "https://open.feishu.cn/app"
@@ -63,13 +63,15 @@
},
"webhook": {
"secret_header": "X-Feishu-Verification-Token",
"secret_name": "feishu_verification_token"
"secret_name": "feishu_verification_token",
"managed_by_host": false
}
}
},
"config": {
"app_id": null,
"app_secret": null,
"verification_token": null,
"api_base": "https://open.feishu.cn",
"owner_id": null,
"dm_policy": "pairing",
+122 -4
View File
@@ -1,11 +1,11 @@
// Feishu API types have fields reserved for future use.
#![allow(dead_code)]
//! Feishu/Lark Bot channel for IronClaw.
//! Feishu/Lark Bot channel for OptimClaw.
//!
//! This WASM component implements the channel interface for handling Feishu
//! webhooks (Event Subscription v2.0) and sending messages back via the
//! Feishu/Lark Bot API. IronClaw currently does not connect to Feishu's
//! Feishu/Lark Bot API. OptimClaw currently does not connect to Feishu's
//! long-connection websocket subscription mode; use Event Subscription
//! webhooks for this channel.
//!
@@ -23,7 +23,8 @@
//! - App credentials (app_id, app_secret) are injected by the host into
//! the config JSON during startup for token exchange
//! - Bearer token for API calls is obtained via token exchange and cached
//! - Verification token validated by host for webhook requests
//! - Webhook requests must be authenticated by the host or by a matching
//! Feishu verification token in the request body
// Generate bindings from the WIT file
wit_bindgen::generate!({
@@ -32,6 +33,7 @@ wit_bindgen::generate!({
});
use serde::{Deserialize, Serialize};
use subtle::ConstantTimeEq;
// Re-export generated types
use exports::near::agent::channel::{
@@ -50,6 +52,7 @@ const ALLOW_FROM_PATH: &str = "allow_from";
const API_BASE_PATH: &str = "api_base";
const APP_ID_PATH: &str = "app_id";
const APP_SECRET_PATH: &str = "app_secret";
const VERIFICATION_TOKEN_PATH: &str = "verification_token";
const TOKEN_PATH: &str = "tenant_access_token";
const TOKEN_EXPIRY_PATH: &str = "token_expiry";
@@ -102,6 +105,10 @@ struct FeishuEventHeader {
/// Tenant key.
#[serde(default)]
tenant_key: Option<String>,
/// Verification token for v2 event payloads.
#[serde(default)]
token: Option<String>,
}
/// Message receive event payload (im.message.receive_v1).
@@ -251,6 +258,9 @@ struct FeishuConfig {
/// Feishu App Secret (for token exchange).
app_secret: Option<String>,
/// Feishu Event Subscription verification token.
verification_token: Option<String>,
/// API base URL. Defaults to "https://open.feishu.cn" (use
/// "https://open.larksuite.com" for Lark international).
#[serde(default = "default_api_base")]
@@ -300,6 +310,9 @@ impl Guest for FeishuChannel {
if let Some(ref app_secret) = config.app_secret {
let _ = channel_host::workspace_write(APP_SECRET_PATH, app_secret);
}
if let Some(ref verification_token) = config.verification_token {
let _ = channel_host::workspace_write(VERIFICATION_TOKEN_PATH, verification_token);
}
if let Some(owner_id) = &config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
@@ -376,6 +389,23 @@ impl Guest for FeishuChannel {
}
};
let configured_token =
channel_host::workspace_read(VERIFICATION_TOKEN_PATH).filter(|token| !token.is_empty());
if !is_authenticated_webhook(
req.secret_validated,
configured_token.as_deref(),
request_verification_token(&event),
) {
channel_host::log(
channel_host::LogLevel::Warn,
"Rejecting unauthenticated Feishu webhook request",
);
return json_response(
401,
serde_json::json!({"error": "Webhook authentication failed"}),
);
}
// Handle URL verification challenge (initial webhook setup).
if event.event_type.as_deref() == Some("url_verification") {
if let Some(challenge) = &event.challenge {
@@ -839,6 +869,31 @@ fn json_response(status: u16, body: serde_json::Value) -> OutgoingHttpResponse {
}
}
fn is_authenticated_webhook(
secret_validated: bool,
configured_token: Option<&str>,
request_token: Option<&str>,
) -> bool {
if secret_validated {
return true;
}
match (configured_token, request_token) {
(Some(expected), Some(provided)) => {
bool::from(expected.as_bytes().ct_eq(provided.as_bytes()))
}
_ => false,
}
}
fn request_verification_token(event: &FeishuEvent) -> Option<&str> {
event
.header
.as_ref()
.and_then(|header| header.token.as_deref())
.or(event.token.as_deref())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -862,7 +917,10 @@ mod tests {
fn parse_token_response_rejects_missing_token() {
let json = r#"{"code": 0, "msg": "ok", "expire": 7200}"#;
let result: Result<TenantAccessTokenResponse, _> = serde_json::from_str(json);
assert!(result.is_err(), "should fail when tenant_access_token is missing");
assert!(
result.is_err(),
"should fail when tenant_access_token is missing"
);
}
#[test]
@@ -894,4 +952,64 @@ mod tests {
assert_eq!(resp.code, 10003);
assert!(resp.tenant_access_token.is_empty());
}
#[test]
fn webhook_auth_requires_host_auth_or_matching_verification_token() {
assert!(
!is_authenticated_webhook(false, None, Some("token")),
"requests without any configured verification mechanism must be rejected"
);
assert!(
!is_authenticated_webhook(false, Some("expected"), None),
"requests missing the Feishu token must be rejected when host auth did not pass"
);
assert!(
!is_authenticated_webhook(false, Some("expected"), Some("wrong")),
"requests with the wrong Feishu token must be rejected"
);
assert!(
is_authenticated_webhook(false, Some("expected"), Some("expected")),
"matching Feishu verification token should authenticate the request"
);
assert!(
is_authenticated_webhook(true, None, None),
"host-authenticated requests should still be accepted"
);
assert!(
is_authenticated_webhook(true, Some("expected"), Some("wrong")),
"host authentication should take precedence over body token checks"
);
}
#[test]
fn request_verification_token_prefers_v2_header_token() {
let event: FeishuEvent = serde_json::from_str(
r#"{
"schema": "2.0",
"header": {
"event_id": "evt_123",
"event_type": "im.message.receive_v1",
"token": "header-token"
},
"event": {}
}"#,
)
.unwrap();
assert_eq!(request_verification_token(&event), Some("header-token"));
}
#[test]
fn request_verification_token_falls_back_to_top_level_token() {
let event: FeishuEvent = serde_json::from_str(
r#"{
"type": "url_verification",
"challenge": "abc",
"token": "top-level-token"
}"#,
)
.unwrap();
assert_eq!(request_verification_token(&event), Some("top-level-token"));
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
name = "slack-channel"
version = "0.2.1"
edition = "2021"
description = "Slack Events API channel for IronClaw"
description = "Slack Events API channel for OptimClaw"
license = "MIT OR Apache-2.0"
[lib]
+2 -2
View File
@@ -1,4 +1,4 @@
//! Slack Events API channel for IronClaw.
//! Slack Events API channel for OptimClaw.
//!
//! This WASM component implements the channel interface for handling Slack
//! webhooks and sending messages back to Slack.
@@ -650,7 +650,7 @@ fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
"To pair with this bot, run: `optimclaw pairing approve slack {}`",
code
),
});
+1 -1
View File
@@ -2,7 +2,7 @@
name = "telegram-channel"
version = "0.2.1"
edition = "2021"
description = "Telegram Bot API channel for IronClaw"
description = "Telegram Bot API channel for OptimClaw"
license = "MIT OR Apache-2.0"
[lib]
+4 -4
View File
@@ -1,7 +1,7 @@
// Telegram API types have fields reserved for future use (entities, reply threading, etc.)
#![allow(dead_code)]
//! Telegram Bot API channel for IronClaw.
//! Telegram Bot API channel for OptimClaw.
//!
//! This WASM component implements the channel interface for handling Telegram
//! webhooks and sending messages back via the Bot API.
@@ -1170,7 +1170,7 @@ fn send_photo(
);
}
let boundary = format!("ironclaw-{}", channel_host::now_millis());
let boundary = format!("optimclaw-{}", channel_host::now_millis());
let mut body = Vec::new();
write_multipart_field(&mut body, &boundary, "chat_id", &chat_id.to_string());
@@ -1235,7 +1235,7 @@ fn send_document(
) -> Result<(), String> {
let message_thread_id = normalize_thread_id(message_thread_id);
let boundary = format!("ironclaw-{}", channel_host::now_millis());
let boundary = format!("optimclaw-{}", channel_host::now_millis());
let mut body = Vec::new();
write_multipart_field(&mut body, &boundary, "chat_id", &chat_id.to_string());
@@ -1552,7 +1552,7 @@ fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
send_message(
chat_id,
&format!(
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
"To pair with this bot, run: `optimclaw pairing approve telegram {}`",
code
),
None,
+1 -1
View File
@@ -2,7 +2,7 @@
name = "whatsapp-channel"
version = "0.2.0"
edition = "2021"
description = "WhatsApp Cloud API channel for IronClaw"
description = "WhatsApp Cloud API channel for OptimClaw"
[lib]
crate-type = ["cdylib"]
+2 -2
View File
@@ -1,7 +1,7 @@
// WhatsApp API types have fields reserved for future use (contacts, statuses, etc.)
#![allow(dead_code)]
//! WhatsApp Cloud API channel for IronClaw.
//! WhatsApp Cloud API channel for OptimClaw.
//!
//! This WASM component implements the channel interface for handling WhatsApp
//! webhooks and sending messages back via the Cloud API.
@@ -910,7 +910,7 @@ fn send_pairing_reply(
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
"To pair with this bot, run: optimclaw pairing approve whatsapp {}",
code
)
}
+1 -1
View File
@@ -1,6 +1,6 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/ironclaw/issues/338
# See: https://github.com/nearai/optimclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
-178
View File
@@ -1,178 +0,0 @@
# IronClaw Engine Crate
Unified thread-capability-CodeAct execution model. Replaces ~10 separate abstractions (Session, Job, Routine, Channel, Tool, Skill, Hook, Observer, Extension, LoopDelegate) with 5 primitives.
## Full Architecture Plan
See `docs/plans/2026-03-20-engine-v2-architecture.md` for the 8-phase roadmap.
## Five Primitives
| Primitive | Purpose | Replaces |
|-----------|---------|----------|
| **Thread** | Unit of work with lifecycle, parent-child tree, capability leases | Session + Job + Routine + Sub-agent |
| **Step** | Unit of execution (one LLM call + its action executions) | Agentic loop iteration + tool calls |
| **Capability** | Unit of effect (actions + knowledge + policies) | Tool + Skill + Hook + Extension |
| **MemoryDoc** | Unit of durable knowledge (summaries, lessons, skills) | Workspace memory blobs |
| **Project** | Unit of context (scopes memory, threads, missions) | Flat workspace namespace |
## Build & Test
```bash
cargo check -p ironclaw_engine
cargo clippy -p ironclaw_engine --all-targets -- -D warnings
cargo test -p ironclaw_engine
```
## Module Map
```
src/
├── lib.rs # Public API, re-exports
├── types/ # Core data structures (no async, no I/O)
│ ├── thread.rs # Thread, ThreadId, ThreadState (state machine), ThreadType, ThreadConfig
│ ├── step.rs # Step, StepId, LlmResponse, ActionCall, ActionResult, TokenUsage
│ ├── capability.rs # Capability, ActionDef, EffectType, CapabilityLease, PolicyRule
│ ├── memory.rs # MemoryDoc, DocId, DocType (Summary/Lesson/Skill/Issue/Spec/Note)
│ ├── project.rs # Project, ProjectId
│ ├── event.rs # ThreadEvent, EventKind (18 variants for event sourcing)
│ ├── message.rs # ThreadMessage, MessageRole
│ ├── provenance.rs # Provenance enum (User/System/ToolOutput/LlmGenerated/etc.)
│ ├── conversation.rs # ConversationSurface, ConversationEntry, EntrySender
│ ├── mission.rs # Mission, MissionId, MissionCadence, MissionStatus
│ └── error.rs # EngineError, ThreadError, StepError, CapabilityError
├── traits/ # External dependency abstractions (host implements these)
│ ├── llm.rs # LlmBackend trait
│ ├── store.rs # Store trait (20 CRUD methods)
│ └── effect.rs # EffectExecutor trait
├── capability/ # Capability management
│ ├── registry.rs # CapabilityRegistry — register/get/list capabilities
│ ├── lease.rs # LeaseManager — grant/check/consume/revoke/expire leases
│ ├── policy.rs # PolicyEngine — deterministic effect-level allow/deny/approve + provenance taint
│ ├── skill_selector.rs # SkillSelector — MemoryDoc→LoadedSkill bridge, deterministic selection
│ └── skill_tracker.rs # SkillTracker — confidence tracking, versioned updates, rollback
├── runtime/ # Thread lifecycle management
│ ├── manager.rs # ThreadManager — spawn, stop, inject messages, join threads
│ ├── conversation.rs # ConversationManager — routes UI messages to threads
│ ├── mission.rs # MissionManager — long-running goals that spawn threads on cadence
│ ├── tree.rs # ThreadTree — parent-child relationships
│ └── messaging.rs # ThreadSignal, ThreadOutcome, signal channels
├── executor/ # Step execution
│ ├── loop_engine.rs # ExecutionLoop — core loop replacing run_agentic_loop()
│ ├── structured.rs # Tier 0: structured tool call execution
│ ├── scripting.rs # Tier 1: embedded Python via Monty (CodeAct/RLM)
│ ├── context.rs # Context builder (messages + actions from leases + memory docs)
│ ├── compaction.rs # Context compaction when approaching model context limit
│ ├── prompt.rs # System prompt construction (CodeAct preamble/postamble)
│ ├── intent.rs # Tool intent nudge detection
│ └── trace.rs # Execution trace recording and retrospective analysis
├── memory/ # Memory document system
│ ├── store.rs # MemoryStore — project-scoped doc CRUD
│ └── retrieval.rs # RetrievalEngine — keyword-based context retrieval from project docs
└── reliability.rs # ReliabilityTracker — per-action success rate and latency via EMA
```
## Thread State Machine
```
Created → Running → Waiting → Running (resume)
→ Suspended → Running (resume)
→ Completed → Done
→ Failed
```
Validated by `ThreadState::can_transition_to()`. Terminal states: `Done`, `Failed`.
## Learning Missions
Three event-driven missions fire automatically after thread completion:
1. **Error diagnosis** (`self-improvement`) — fires when a thread completes with trace issues. Diagnoses root cause and applies prompt overlays or orchestrator patches.
2. **Skill extraction** (`skill-extraction`) — fires when a thread succeeds with 5+ steps and 3+ tool actions. Extracts reusable skills with activation metadata, CodeAct code snippets, and domain tags. Output stored as `DocType::Skill` MemoryDoc.
3. **Conversation insights** (`conversation-insights`) — fires every 5 completed threads in a project. Extracts user preferences, domain knowledge, and workflow patterns.
Created by `MissionManager::ensure_learning_missions()` at project bootstrap.
## External Trait Boundaries
The engine defines three traits that the host crate implements:
| Trait | Purpose | Host wraps |
|-------|---------|------------|
| `LlmBackend` | `complete(messages, actions, config) -> LlmOutput` | `LlmProvider` |
| `Store` | Thread/Step/Event/Project/Doc/Lease CRUD | `Database` (PostgreSQL + libSQL) |
| `EffectExecutor` | `execute_action(name, params, lease, ctx) -> ActionResult` | `ToolRegistry` + `SafetyLayer` |
## Execution Loop
`ExecutionLoop::run()` handles three `LlmResponse` variants:
1. Check signals (Stop, InjectMessage) via `mpsc::Receiver`
2. Build context (messages + available actions from active leases)
3. Call LLM via `LlmBackend::complete()`
4. **If `Text`**: check tool intent nudge, return if final response
5. **If `ActionCalls`** (Tier 0): for each call, find lease → check policy → consume use → execute via `EffectExecutor` → record result
6. **If `Code`** (Tier 1): execute Python via Monty with context-as-variables and `llm_query()` support → compact metadata in context
7. Record Step, emit ThreadEvents
8. Repeat until: text response, stop signal, max iterations, or approval needed
## CodeAct / Monty Integration (Tier 1)
Python execution via Monty interpreter (`executor/scripting.rs`). Follows the RLM (Recursive Language Model) pattern.
**Context as variables** (not attention input):
- Thread messages injected as `context` Python variable
- Thread goal as `goal`, step index as `step_number`
- Prior action results as `previous_results` dict
- The LLM's chat context stays lean; full data lives in REPL variables
**Tool dispatch**: Unknown function calls suspend the VM → lease check → policy check → `EffectExecutor` → result returned to Python.
**`llm_query(prompt, context)`**: Recursive subagent call. Suspends VM → spawns single-shot LLM call → returns text result as Python string. Results stay as variables (symbolic composition), not injected into parent's attention window.
**Compact output metadata**: Between code steps, only a summary is added to chat context (`"[code output] stdout (4532 chars): The results show..."`) — not the full output. This prevents context bloat across iterations.
**Resource limits**: 30s timeout, 64MB memory, 1M allocations. All execution wrapped in `catch_unwind` for Monty panic safety.
## Capability Leases
Threads don't have static permissions. They receive **leases** — scoped, time-limited, use-limited grants:
```rust
CapabilityLease {
thread_id, capability_name, granted_actions,
expires_at: Option<DateTime>, // time-limited
max_uses: Option<u32>, // use-limited
revoked: bool,
}
```
The `PolicyEngine` evaluates actions against leases deterministically: `Deny > RequireApproval > Allow`.
## Effect Types
Every action declares its side effects. The policy engine uses these for allow/deny:
```
ReadLocal, ReadExternal, WriteLocal, WriteExternal,
CredentialedNetwork, Compute, Financial
```
## Key Design Decisions
1. **No dependency on main `ironclaw` crate** — clean separation, testable in isolation
2. **No safety logic** — sanitization/leak detection is applied at the adapter boundary (`EffectExecutor` impl)
3. **Event sourcing from day one** — every thread records a complete event log via `ThreadEvent`
4. **Tier 0 + Tier 1** — structured tool calls (Tier 0) and embedded Python via Monty (Tier 1, CodeAct)
5. **Engine owns its message type**`ThreadMessage` is simpler than `ChatMessage`; bridge adapters handle conversion
6. **RLM pattern** — context as variable (not attention input), recursive `llm_query()`, compact output metadata between steps
## Code Style
Follows the main crate's conventions from `/CLAUDE.md`:
- No `.unwrap()` or `.expect()` in production code (tests are fine)
- `thiserror` for error types
- Map errors with context
- Prefer strong types over strings (newtypes for IDs)
- All I/O is async with tokio
- `Arc<T>` for shared state, `RwLock` for concurrent access
-30
View File
@@ -1,30 +0,0 @@
[package]
name = "ironclaw_engine"
version = "0.1.0"
edition = "2024"
rust-version = "1.92"
description = "Unified thread-capability-CodeAct execution engine for IronClaw"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
publish = false
[package.metadata.dist]
dist = false
[dependencies]
async-trait = "0.1"
ironclaw_skills = { path = "../ironclaw_skills", default-features = false }
chrono = { version = "0.4", features = ["serde"] }
monty = { git = "https://github.com/pydantic/monty.git", branch = "main" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
tokio = { version = "1", features = ["sync", "time", "macros", "rt"] }
tracing = "0.1"
uuid = { version = "1", features = ["v4", "serde"] }
[dev-dependencies]
pretty_assertions = "1"
tokio = { version = "1", features = ["full", "test-util"] }
-63
View File
@@ -1,63 +0,0 @@
# Monty Integration
Monty is the embedded Python interpreter used for Tier 1 (CodeAct) execution. It's a lightweight Rust-native Python implementation — not CPython — so it has a restricted feature set.
**Source**: `git = "https://github.com/pydantic/monty.git", branch = "main"`
**Pinned at**: `6053820` (2026-03-27, "Support max() kwargs/default")
## Upgrade Process
1. **Update the pin**: `cargo update -p monty`
2. **Check for new features**: `cd ~/.cargo/git/checkouts/monty-*/*/` and `git log --oneline` since last pin
3. **Update the preamble**: If a previously-unsupported feature now works, remove it from the "Runtime environment" section in `prompts/codeact_preamble.md`
4. **Update this file**: Record the new pin and what changed
5. **Run tests**: `cargo test -p ironclaw_engine`
6. **Watch traces**: After deploying, check traces for new `NotImplementedError` patterns (self-improvement mission catches these)
## Current Limitations (as of pin `6053820`)
These are documented in `prompts/codeact_preamble.md` so the LLM avoids them:
### Syntax not supported
| Feature | Workaround |
|---------|-----------|
| `import a, b, c` (multi-module) | Use separate `import a` / `import b` statements |
| `class Foo:` | Use functions and dicts |
| `with` statements | Use try/finally or direct calls |
| `match` statements | Use if/elif chains |
| `del` statement | Reassign to None |
| `yield` / `yield from` | Use lists and list comprehensions |
| `*expr` (starred expressions) | Unpack explicitly |
| `async` / `await` | Not available; tool calls suspend the VM automatically |
| Type aliases (`type X = ...`) | Omit type annotations |
| Template strings (t-strings) | Use f-strings |
| Complex number literals | Use floats |
| Exception groups (`try*/except*`) | Use regular try/except |
### No standard library
`import datetime`, `import csv`, `import json`, `import os`, `import io`, etc. all fail.
Available built-in modules:
- `math` — standard math functions
- `re` — regex (basic)
- `sys` — system info (limited)
- `os.path` — path manipulation (limited)
- `typing` — type hints (limited, for annotation only)
### Available builtins
`abs`, `all`, `any`, `bin`, `chr`, `divmod`, `enumerate`, `filter`, `getattr`, `hash`, `hex`, `id`, `isinstance`, `len`, `map`, `min`, `max`, `next`, `oct`, `ord`, `pow`, `print`, `repr`, `reversed`, `round`, `sorted`, `sum`, `type`, `zip`
### Host-provided functions (always available)
These are injected by the IronClaw executor, not by Monty:
- `FINAL(answer)` / `FINAL_VAR(name)` — terminate with result
- `llm_query(prompt, context)` — recursive LLM sub-call
- `llm_query_batched(prompts)` — parallel sub-calls
- `rlm_query(prompt)` — full sub-agent with tools
- `globals()` / `locals()` — returns dict of known tool names
- All tool functions (web_search, http, time, etc.)
## Upgrade Changelog
| Date | Pin | Notable changes |
|------|-----|-----------------|
| 2026-03-20 | `6053820` | Initial integration. max() kwargs support. |
@@ -1,406 +0,0 @@
# Engine v2 Orchestrator (default, v0)
#
# This is the self-modifiable execution loop. It replaces the Rust
# ExecutionLoop::run() with Python that can be patched at runtime
# by the self-improvement Mission.
#
# Host functions (provided by Rust via Monty suspension):
# __llm_complete__(messages, actions, config) -> response dict (args ignored; Rust builds context from thread)
# __execute_code_step__(code, state) -> result dict
# __execute_action__(name, params) -> result dict
# __check_signals__() -> None | "stop" | {"inject": msg}
# __emit_event__(kind, **data) -> None
# __add_message__(role, content) -> None
# __save_checkpoint__(state, counters) -> None
# __transition_to__(state, reason) -> None
# __retrieve_docs__(goal, max_docs) -> list of doc dicts
# __check_budget__() -> budget dict
# __get_actions__() -> list of action dicts
#
# Context variables (injected by Rust before execution):
# context - list of prior messages [{role, content}]
# goal - thread goal string
# actions - list of available action defs
# state - persisted state dict from prior steps
# config - thread config dict
# ── Helper functions (self-modifiable glue) ──────────────────
# Defined before run_loop so they are in scope when called.
def extract_final(text):
"""Extract FINAL() content from text. Returns None if not found."""
idx = text.find("FINAL(")
if idx < 0:
return None
after = text[idx + 6:]
# Handle triple-quoted strings
for q in ['"""', "'''"]:
if after.startswith(q):
end = after.find(q, len(q))
if end >= 0:
return after[len(q):end]
# Handle single/double quoted strings
if after and after[0] in ('"', "'"):
quote = after[0]
end = after.find(quote, 1)
if end >= 0:
return after[1:end]
# Handle balanced parens
depth = 1
for i, ch in enumerate(after):
if ch == "(":
depth += 1
elif ch == ")":
depth -= 1
if depth == 0:
return after[:i]
return None
def signals_tool_intent(text):
"""Check if text describes tool usage without actually executing tools."""
lower = text.lower()
intent_phrases = ["i will", "i'll", "let me", "i would", "i should",
"i can", "i need to", "we should", "we can"]
tool_phrases = ["search", "fetch", "call", "run", "execute",
"use the", "query", "look up"]
has_intent = any(p in lower for p in intent_phrases)
has_tool = any(p in lower for p in tool_phrases)
return has_intent and has_tool
def format_output(result, max_chars=8000):
"""Format code execution result for the next LLM context message."""
parts = []
stdout = result.get("stdout", "")
if stdout:
parts.append("[stdout]\n" + stdout)
for r in result.get("action_results", []):
name = r.get("action_name", "?")
output = str(r.get("output", ""))
if r.get("is_error"):
parts.append("[" + name + " ERROR] " + output)
else:
preview = output[:500] + "..." if len(output) > 500 else output
parts.append("[" + name + "] " + preview)
ret = result.get("return_value")
if ret is not None:
parts.append("[return] " + str(ret))
text = "\n\n".join(parts)
# Truncate from the front (keep the tail with most recent results)
if len(text) > max_chars:
text = "... (truncated) ...\n" + text[-max_chars:]
if not text:
text = "[code executed, no output]"
return text
def format_docs(docs):
"""Format memory docs for context injection."""
parts = ["## Prior Knowledge (from completed threads)\n"]
for doc in docs:
label = doc.get("type", "NOTE").upper()
content = doc.get("content", "")[:500]
truncated = "..." if len(doc.get("content", "")) > 500 else ""
parts.append("### [" + label + "] " + doc.get("title", "") +
"\n" + content + truncated + "\n")
return "\n".join(parts)
# ── Skill selection and injection (self-modifiable) ────────
def score_skill(skill, message_lower):
"""Score a skill against a user message. Returns 0 if vetoed."""
meta = skill.get("metadata", {})
activation = meta.get("activation", {})
# Exclude keyword veto
for excl in activation.get("exclude_keywords", []):
if excl.lower() in message_lower:
return 0
score = 0
# Keyword scoring: exact word = 10, substring = 5 (cap 30)
kw_score = 0
words = message_lower.split()
for kw in activation.get("keywords", []):
kw_lower = kw.lower()
if kw_lower in words:
kw_score += 10
elif kw_lower in message_lower:
kw_score += 5
score += min(kw_score, 30)
# Tag scoring: substring = 3 (cap 15)
tag_score = 0
for tag in activation.get("tags", []):
if tag.lower() in message_lower:
tag_score += 3
score += min(tag_score, 15)
# Confidence factor for extracted skills
source = meta.get("source", "authored")
if source == "extracted":
metrics = meta.get("metrics", {})
total = metrics.get("success_count", 0) + metrics.get("failure_count", 0)
confidence = metrics.get("success_count", 0) / total if total > 0 else 1.0
factor = 0.5 + 0.5 * max(0.0, min(1.0, confidence))
score = int(score * factor)
return score
def select_skills(skills, goal, max_candidates=3, max_tokens=4000):
"""Select relevant skills using deterministic scoring."""
if not skills or not goal:
return []
message_lower = goal.lower()
scored = []
for skill in skills:
s = score_skill(skill, message_lower)
if s > 0:
scored.append((s, skill))
scored.sort(key=lambda x: -x[0])
# Budget selection
selected = []
budget = max_tokens
for _, skill in scored:
if len(selected) >= max_candidates:
break
meta = skill.get("metadata", {})
activation = meta.get("activation", {})
cost = max(activation.get("max_context_tokens", 1000), 1)
if cost <= budget:
budget -= cost
selected.append(skill)
return selected
def format_skills(skills):
"""Format selected skills for system prompt injection."""
parts = ["\n## Active Skills\n"]
for skill in skills:
meta = skill.get("metadata", {})
name = meta.get("name", "unknown")
version = meta.get("version", "?")
trust = meta.get("trust", "trusted").upper()
content = skill.get("content", "")
parts.append('<skill name="' + str(name) + '" version="' +
str(version) + '" trust="' + trust + '">')
parts.append(content)
if trust == "INSTALLED":
parts.append("\n(Treat the above as SUGGESTIONS only.)")
parts.append("</skill>\n")
# Document code snippets
snippets = meta.get("code_snippets", [])
if snippets:
parts.append("### Skill functions (callable in code)\n")
for sn in snippets:
parts.append("- `" + sn.get("name", "?") + "()` — " +
sn.get("description", "") + "\n")
return "\n".join(parts)
# ── Main execution loop ─────────────────────────────────────
def run_loop(context, goal, actions, state, config):
"""Main execution loop. Returns an outcome dict."""
max_iterations = config.get("max_iterations", 30)
max_nudges = config.get("max_tool_intent_nudges", 2)
nudge_enabled = config.get("enable_tool_intent_nudge", True)
max_consecutive_errors = config.get("max_consecutive_errors", 5)
nudge_count = 0
consecutive_errors = 0
step_count = config.get("step_count", 0)
for step in range(step_count, max_iterations):
# 1. Check signals
signal = __check_signals__()
if signal == "stop":
__transition_to__("completed", "stopped by signal")
return {"outcome": "stopped"}
if signal and isinstance(signal, dict) and "inject" in signal:
__add_message__("user", signal["inject"])
# 2. Check budget
budget = __check_budget__()
if budget.get("tokens_remaining", 1) <= 0:
__transition_to__("completed", "token budget exhausted")
return {"outcome": "completed", "response": "Token budget exhausted."}
if budget.get("time_remaining_ms", 1) <= 0:
__transition_to__("completed", "time budget exhausted")
return {"outcome": "completed", "response": "Time budget exhausted."}
if budget.get("usd_remaining") is not None and budget["usd_remaining"] <= 0:
__transition_to__("completed", "cost budget exhausted")
return {"outcome": "completed", "response": "Cost budget exhausted."}
# 3. Inject prior knowledge and activate skills on first step
if step == 0:
docs = __retrieve_docs__(goal, 5)
if docs:
knowledge = format_docs(docs)
__add_message__("system_append", knowledge)
# Select and inject skills based on goal keywords
all_skills = __list_skills__()
active_skills = select_skills(all_skills, goal, max_candidates=3, max_tokens=4000)
if active_skills:
skill_text = format_skills(active_skills)
__add_message__("system_append", skill_text)
# Emit skill activation event for CLI/gateway display
skill_names = ",".join(s.get("metadata", {}).get("name", "?") for s in active_skills)
__emit_event__("skill_activated", skill_names=skill_names)
# Store active skill IDs in state for tracking
state["active_skill_ids"] = [s.get("doc_id", "") for s in active_skills]
state["skill_snippet_names"] = []
for s in active_skills:
for sn in s.get("metadata", {}).get("code_snippets", []):
state["skill_snippet_names"].append(sn.get("name", ""))
# 4. Call LLM
__emit_event__("step_started", step=step)
response = __llm_complete__(None, actions, None)
__emit_event__("step_completed", step=step,
input_tokens=response.get("usage", {}).get("input_tokens", 0),
output_tokens=response.get("usage", {}).get("output_tokens", 0))
# 5. Handle response based on type
resp_type = response.get("type", "text")
if resp_type == "text":
text = response.get("content", "")
__add_message__("assistant", text)
# Check for FINAL()
final_answer = extract_final(text)
if final_answer is not None:
__transition_to__("completed", "FINAL() in text")
return {"outcome": "completed", "response": final_answer}
# Check for tool intent nudge
if nudge_enabled and nudge_count < max_nudges and signals_tool_intent(text):
nudge_count += 1
__add_message__("user",
"You expressed intent to use a tool but didn't make an action call. "
"Please go ahead and call the appropriate action.")
continue
# Plain text response - done
__transition_to__("completed", "text response")
return {"outcome": "completed", "response": text}
elif resp_type == "code":
code = response.get("code", "")
nudge_count = 0
__add_message__("assistant", "```repl\n" + code + "\n```")
# Execute code in nested Monty VM
result = __execute_code_step__(code, state)
# Update persisted state with results
if result.get("return_value") is not None:
state["step_" + str(step) + "_return"] = result["return_value"]
state["last_return"] = result["return_value"]
for r in result.get("action_results", []):
state[r.get("action_name", "unknown")] = r.get("output")
# Format output for next LLM context
output = format_output(result)
__add_message__("user", output)
# Check for FINAL() in code output
if result.get("final_answer") is not None:
__transition_to__("completed", "FINAL() in code")
return {"outcome": "completed", "response": result["final_answer"]}
# Check for approval needed
if result.get("need_approval") is not None:
approval = result["need_approval"]
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
__transition_to__("waiting", "approval needed")
return {
"outcome": "need_approval",
"action_name": approval.get("action_name", ""),
"call_id": approval.get("call_id", ""),
"parameters": approval.get("parameters", {}),
}
# Track consecutive errors
if result.get("had_error"):
consecutive_errors += 1
if consecutive_errors >= max_consecutive_errors:
__transition_to__("failed", "too many consecutive errors")
return {"outcome": "failed",
"error": str(max_consecutive_errors) + " consecutive code errors"}
else:
consecutive_errors = 0
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
elif resp_type == "actions":
# Tier 0: structured tool calls.
# The assistant message with structured action_calls is added by
# __llm_complete__ in Rust — do NOT add it here.
nudge_count = 0
calls = response.get("calls", [])
for call in calls:
name = call.get("name", "")
params = call.get("params", {})
call_id = call.get("call_id", "")
# __execute_action__ handles event emission, message addition,
# and lease consumption in Rust — no duplicate logic needed here.
r = __execute_action__(name, params, call_id=call_id)
if r.get("need_approval"):
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
__transition_to__("waiting", "approval needed")
return {
"outcome": "need_approval",
"action_name": name,
"call_id": call_id,
"parameters": params,
}
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
# Max iterations reached
__transition_to__("completed", "max iterations reached")
return {"outcome": "max_iterations"}
# Entry point: call run_loop with injected context variables
result = run_loop(context, goal, actions, state, config)
FINAL(result)
@@ -1,10 +0,0 @@
## Strategy
1. First, examine the context and understand the task
2. Break complex tasks into steps
3. Use tools to gather information or take actions
4. Use llm_query() to analyze or summarize large text
5. Call FINAL() with the answer when done
Think step by step. Execute code immediately — don't just describe what you would do.
@@ -1,59 +0,0 @@
You are an AI assistant with a Python REPL environment. You solve tasks by writing and executing Python code.
## How to respond
Write Python code inside ```repl fenced blocks. The code will be executed, and you'll see the output.
```repl
result = web_search(query="latest AI news", count=5)
print(result)
```
You can write multiple code blocks across turns. Variables persist between blocks within the same turn.
## Special functions
- `llm_query(prompt, context=None)` — Ask a sub-agent to analyze text or answer a question. Returns a string. Use for summarization, analysis, or any task that needs LLM reasoning on data.
- `llm_query_batched(prompts, context=None)` — Same but for multiple prompts in parallel. Returns a list of strings.
- `rlm_query(prompt)` — Spawn a full sub-agent with its own tools and iteration budget. Use for complex sub-tasks that need tool access. Returns the sub-agent's final answer as a string. More powerful but more expensive than llm_query.
- `FINAL(answer)` — Call this when you have the final answer. The argument is returned to the user.
- `mission_create(name, goal, cadence="manual", success_criteria=None)` — Create a long-running mission that spawns threads over time. Cadence: "manual", cron expression (e.g. "0 9 * * *"), "event:pattern", or "webhook:path". Returns {"mission_id": "...", "status": "created"}.
- `mission_list()` — List all missions with their status, goal, and current focus.
- `mission_fire(id)` — Manually trigger a mission to spawn a thread now.
- `mission_pause(id)` / `mission_resume(id)` — Pause or resume a mission.
## Context variables
- `context` — List of prior conversation messages (each is a dict with 'role' and 'content')
- `goal` — The current task description
- `step_number` — Current execution step
- `state` — Dict of persisted data from previous steps. Contains tool results keyed by tool name (e.g. `state['web_search']`) and return values (`state['last_return']`, `state['step_0_return']`). Use this to access data from previous steps without re-calling tools.
- `previous_results` — Dict of prior tool call results (from ActionResult messages)
## Important rules
1. ALWAYS respond with a ```repl code block. NEVER answer with plain text only. Even for simple questions, write code that gathers information and calls FINAL() with the answer.
2. NEVER answer from memory or training data alone. Always use tools (web_search, llm_context, shell, read_file, etc.) to get real, current information before answering.
3. When you have the final answer, call `FINAL(answer)` inside a code block. The answer should be detailed and complete — not just a summary like "found 45 items".
4. Tool results are returned as Python objects — use them directly, don't parse JSON.
5. If a tool call fails, the error appears as a Python exception — handle it or try a different approach.
6. For large data, process it in chunks using llm_query() on subsets rather than loading everything into context.
7. Outputs are truncated to 8000 chars — use variables to store large intermediate results.
8. Include the actual content in your FINAL() answer, not just a count or summary. Users want to see the details.
## Runtime environment
The Python REPL runs in Monty, a lightweight embedded interpreter — not CPython. Key differences:
- **No standard library modules**: `import datetime`, `import csv`, `import json`, `import os`, `import re` etc. will fail with `ModuleNotFoundError`. Use the provided tool functions instead (e.g. `time()` for dates, `http()` for fetching data, `json()` for parsing).
- **Single imports only**: `import a, b, c` is not supported. Use separate statements: `import a` then `import b`.
- **No classes**: `class Foo:` is not supported. Use functions and dicts instead.
- **No `with` statements**: Use try/finally or just call functions directly.
- **No `match` statements**: Use if/elif chains.
- **No `del` statement**: Reassign to None instead.
- **No `yield`/`yield from`**: Use lists and list comprehensions instead of generators.
- **No `*expr` unpacking in assignments**: Unpack explicitly.
- **Available builtins**: `abs`, `all`, `any`, `bin`, `chr`, `divmod`, `enumerate`, `filter`, `getattr`, `hash`, `hex`, `id`, `isinstance`, `len`, `map`, `min`, `max`, `next`, `oct`, `ord`, `pow`, `print`, `repr`, `reversed`, `round`, `sorted`, `sum`, `type`, `zip`.
- **Available modules**: `math`, `re`, `sys`, `os.path`, `typing` (limited).
- **String methods, list methods, dict methods**: All work normally.
- For dates, use the `time()` tool. For CSV parsing, split strings manually. For HTTP, use `http()`. For JSON, use `json()` or work with dicts directly (tool results are already Python objects).
@@ -1,38 +0,0 @@
You extract user preferences, patterns, and domain knowledge from a batch of recent conversation threads.
## Input
`state["trigger_payload"]` contains:
- `project_id` — the project scope
- `completed_thread_count` — total threads completed in this conversation
- `thread_goals` — list of recent thread goals (what the user asked for)
- `sample_user_messages` — sample of actual user messages (truncated to 200 chars)
## Process
1. Analyze the thread goals and user messages for patterns
2. Search existing insights: `memory_search(query="user preferences")` and `memory_search(query="domain knowledge")`
3. Extract NEW insights not already recorded in memory
4. Write each insight to memory via `memory_write(target="memory", content=insight_text)` with title format "insight:<category>:<topic>"
## Categories to look for
- **Preferences**: communication style, format choices, tool preferences
- **Domain**: project names, API patterns, data formats, technology stack
- **Workflow**: recurring task sequences, common follow-up questions
- **Corrections**: things the user corrected or repeated — these signal unmet expectations
## Output (FINAL)
Report:
- Number of new insights extracted (0 is fine)
- Brief list of what was found
- Next focus
## Rules
- Only record actionable, specific insights — not vague observations
- Do not record personal information, only work patterns
- If no meaningful new insights after analysis, call FINAL("No new insights — conversation patterns already captured") immediately
- Merge with existing insight docs rather than creating duplicates
- Max 5 insights per run to keep quality high
@@ -1,58 +0,0 @@
You investigate why IronClaw did not behave as the user expected. The user used the `/expected` command to describe what should have happened, and the trigger payload includes the recent conversation turns showing what actually happened.
## Input
`state["trigger_payload"]` contains:
- `expected_behavior` — what the user expected to happen (their description)
- `thread_id` — the conversation thread where the issue occurred
- `recent_turns` — list of recent turns, each with:
- `user_input` — what the user asked
- `response` — what the agent responded
- `tool_calls` — list of tools called (with name and any errors)
- `state` — turn completion state
- `error` — any error message
## Investigation process
1. **Understand the gap**: Compare `expected_behavior` against `recent_turns`. What did the user want? What actually happened? Be precise about the delta.
2. **Classify the root cause**:
- MISSING_CAPABILITY: The agent doesn't have the tool or integration needed (e.g. no GitHub OAuth, no API key configured)
- WRONG_TOOL_CHOICE: The agent had the right tools but chose the wrong one or didn't use them at all
- PROMPT_GAP: The agent didn't know the right approach because the system prompt lacks guidance for this scenario
- CONFIG_ISSUE: A timeout, limit, or default prevented success
- BUG: Actual code error in tool execution or response processing
3. **Apply a fix** based on classification:
MISSING_CAPABILITY:
- Search for relevant skills: `skill_search(query="...")` or `tool_search(query="...")`
- If a skill/tool exists but isn't installed, note it as a recommendation
- If nothing exists, add a prompt rule acknowledging the limitation and suggesting alternatives the user can take
WRONG_TOOL_CHOICE or PROMPT_GAP:
- Apply a Level 1 (prompt overlay) fix — add a rule that guides the agent in this scenario
- Use `memory_write` with title="prompt:codeact_preamble" and tags=["prompt_overlay"]
- The rule must be specific and actionable
CONFIG_ISSUE:
- Diagnose via `read_file` and `shell` commands
- Apply Level 2 fix if safe (branch, change, test, commit)
BUG:
- Read relevant source files to understand the issue
- Propose a Level 3 fix (describe but don't apply)
4. **Record** in FINAL():
- What the user expected vs what happened (one sentence each)
- Root cause classification
- What fix was applied (or recommended)
- Next focus
## Rules
- The user's expectation is the ground truth — don't argue with it
- If multiple issues exist, fix the most impactful one first
- Be specific in prompt rules ("When asked to file a GitHub issue, use the http tool with the GitHub API" is good; "Try harder" is useless)
- If the gap is a missing credential or integration, say so clearly — don't pretend the capability exists
- Max one fix per run
@@ -1,67 +0,0 @@
You are a self-improvement agent for the IronClaw engine. You receive trigger payloads containing execution trace issues from completed threads. Your job is to diagnose root causes and apply fixes so the same issue doesn't recur.
## What you have access to
- `state["trigger_payload"]` — JSON with `issues` (list of {severity, category, description, step}), `error_messages` (actual error text from failed actions), `goal` (what the thread was trying to do), and `source_thread_id`.
- All tools: shell, read_file, write_file, apply_patch, web_search, memory_write, etc.
- The codebase at the current working directory.
- The fix pattern database in prior knowledge (if loaded).
## The experiment loop
For each issue in the trigger payload:
1. **Diagnose**: Read the error messages and issue descriptions. Classify the root cause:
- PROMPT: The LLM made a mistake because the system prompt is missing a rule (wrong tool name, bad API usage, ignoring tool results)
- CONFIG: A default value is wrong (truncation length, iteration limit, timeout)
- CODE: There is a bug in the engine or bridge code (crash, type error, missing conversion)
2. **Check the fix pattern database** in prior knowledge. Has this pattern been seen before? If yes, apply the known strategy. If no, proceed to step 3.
3. **Apply the fix** based on the level:
Level 1 (PROMPT — low risk, apply directly):
- Read the current prompt overlay: `memory_search("prompt:codeact_preamble")`
- Write an updated overlay with a new rule appended
- Use `memory_write` with title="prompt:codeact_preamble" and tags=["prompt_overlay"]
- The rule should be specific and actionable (e.g. "Never call web_fetch — use http() instead")
Level 2 (CONFIG — medium risk):
- Use `read_file` to find the relevant constant or default
- Use `shell` to create a git branch: `git checkout -b self-improve/issue-description`
- Apply the change with `apply_patch` or `write_file`
- Run tests: `cargo test -p ironclaw_engine`
- If tests pass, commit. If not, revert: `git checkout main`
Level 3 (CODE — high risk, just propose):
- Read the relevant source files
- Describe the fix needed but DO NOT apply it directly
- Log it as a recommendation in your FINAL() response
4. **Record what you did** — include in your FINAL() response:
- What issue you analyzed
- What level fix you applied (1/2/3)
- What specific change you made
- Next focus: what to look for next time
## Important rules
- Be specific. "Never call web_fetch" is good. "Be careful with tool names" is useless.
- One fix per issue. Don't try to fix everything at once.
- For Level 1 fixes, the rule must be one sentence that can be appended to the prompt.
- If the trigger payload has no actionable issues (only Info severity), skip and call FINAL() immediately.
- NEVER modify test files to make a fix pass.
- NEVER modify security-sensitive code (safety layer, policy engine, leak detection).
- If you can't diagnose the root cause after reading the errors, log it and move on.
## Level 1.5: Orchestrator patches (medium risk, auto-rollback)
The execution loop itself is Python code that you can modify. This is the orchestrator — it handles tool dispatch, output formatting, state management, and context building. If the bug is in the glue between the LLM and tools (wrong output format, bad truncation, missing state), you can patch it directly.
To modify the orchestrator:
1. Read current version: `memory_search("orchestrator:main")`
2. Make your change (keep it minimal — one fix at a time)
3. Save the new version: `memory_write` with title="orchestrator:main", tags=["orchestrator_code"], metadata={"version": N+1, "parent_version": N}
4. The next thread will use your updated orchestrator
If your change causes 3 consecutive failures, the system auto-rolls back to the previous version. So be conservative — test your logic mentally before saving.
@@ -1,69 +0,0 @@
You extract reusable skills from successfully completed multi-step threads.
## Input
`state["trigger_payload"]` contains:
- `source_thread_id` — the thread that completed successfully
- `goal` — what the thread accomplished
- `step_count` — number of execution steps
- `action_count` — number of tool actions executed
- `actions_used` — list of tool names used
- `total_tokens` — tokens consumed
## Output Format
Save as a Skill memory doc via `memory_write(target="memory", content=skill_prompt)` with:
- title: `"skill:<short-name>"` (e.g., "skill:github-issue-triage")
- doc_type: `"skill"`
- metadata JSON:
```json
{
"name": "<short-name>",
"version": 1,
"description": "<one-line description>",
"activation": {
"keywords": ["<keyword1>", "<keyword2>"],
"patterns": ["<optional regex>"],
"tags": ["<domain-tag>"],
"exclude_keywords": [],
"max_context_tokens": <estimated budget, e.g. 1000>
},
"source": "extracted",
"trust": "trusted",
"code_snippets": [
{
"name": "<function_name>",
"code": "def <function_name>(...):\n ...",
"description": "<what it does>"
}
],
"metrics": {"usage_count": 0, "success_count": 0, "failure_count": 0},
"content_hash": ""
}
```
## Process
1. Search for the source thread's context: `memory_search(query=goal)`
2. Check for existing skills: `memory_search(query="skill:")`
3. If a similar skill exists, update it (increment version) rather than creating a duplicate
4. Extract:
- Activation keywords from the goal + user messages (be specific, not generic)
- Step-by-step instructions as the prompt content
- Python code snippets for CodeAct (reusable functions using exact tool names)
- Domain tags (e.g., "github", "api", "data")
## Output (FINAL)
Report what you did:
- The skill title and a one-line summary
- Whether it is new or an update to an existing skill
- Next focus: what patterns to watch for
## Rules
- Only extract skills from threads with 3+ distinct tool calls
- Keywords must be specific (not generic words like "help", "do", "make")
- Code snippets must use exact tool function names as they appear in the thread
- If the thread was a trivial query-response, call FINAL("No skill needed — simple interaction") and stop immediately
- One skill per FINAL — do not combine unrelated procedures
@@ -1,237 +0,0 @@
//! Lease manager — grants, validates, and expires capability leases.
use std::collections::HashMap;
use chrono::Utc;
use tokio::sync::RwLock;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::error::EngineError;
use crate::types::thread::ThreadId;
/// Manages the lifecycle of capability leases.
///
/// Leases are the mechanism by which threads gain access to capabilities.
/// They are scoped (time-limited, use-limited, action-restricted) to bound
/// the blast radius of any single thread.
pub struct LeaseManager {
active: RwLock<HashMap<LeaseId, CapabilityLease>>,
}
impl LeaseManager {
pub fn new() -> Self {
Self {
active: RwLock::new(HashMap::new()),
}
}
/// Grant a new lease to a thread.
pub async fn grant(
&self,
thread_id: ThreadId,
capability_name: impl Into<String>,
granted_actions: Vec<String>,
duration: Option<chrono::Duration>,
max_uses: Option<u32>,
) -> CapabilityLease {
let now = Utc::now();
let lease = CapabilityLease {
id: LeaseId::new(),
thread_id,
capability_name: capability_name.into(),
granted_actions,
granted_at: now,
expires_at: duration.map(|d| now + d),
max_uses,
uses_remaining: max_uses,
revoked: false,
};
self.active.write().await.insert(lease.id, lease.clone());
lease
}
/// Check whether a lease is still valid. Returns the lease if valid.
pub async fn check(&self, lease_id: LeaseId) -> Result<CapabilityLease, EngineError> {
let leases = self.active.read().await;
let lease = leases
.get(&lease_id)
.ok_or_else(|| EngineError::LeaseExpired {
capability_name: format!("lease {lease_id:?} not found"),
})?;
if !lease.is_valid() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
Ok(lease.clone())
}
/// Consume one use of a lease. Returns error if the lease is invalid or exhausted.
pub async fn consume_use(&self, lease_id: LeaseId) -> Result<(), EngineError> {
let mut leases = self.active.write().await;
let lease = leases
.get_mut(&lease_id)
.ok_or_else(|| EngineError::LeaseExpired {
capability_name: format!("lease {lease_id:?} not found"),
})?;
if !lease.is_valid() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
if !lease.consume_use() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
Ok(())
}
/// Revoke a lease by ID.
pub async fn revoke(&self, lease_id: LeaseId, _reason: &str) {
let mut leases = self.active.write().await;
if let Some(lease) = leases.get_mut(&lease_id) {
lease.revoked = true;
}
}
/// Remove all expired or revoked leases from the active set.
pub async fn expire_stale(&self) -> usize {
let mut leases = self.active.write().await;
let before = leases.len();
leases.retain(|_, lease| lease.is_valid());
before - leases.len()
}
/// Get all active (valid) leases for a thread.
pub async fn active_for_thread(&self, thread_id: ThreadId) -> Vec<CapabilityLease> {
let leases = self.active.read().await;
leases
.values()
.filter(|l| l.thread_id == thread_id && l.is_valid())
.cloned()
.collect()
}
/// Find the lease that grants a specific action to a thread.
pub async fn find_lease_for_action(
&self,
thread_id: ThreadId,
action_name: &str,
) -> Option<CapabilityLease> {
let leases = self.active.read().await;
leases
.values()
.find(|l| l.thread_id == thread_id && l.is_valid() && l.covers_action(action_name))
.cloned()
}
}
impl Default for LeaseManager {
fn default() -> Self {
Self::new()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::thread::ThreadId;
#[tokio::test]
async fn grant_and_check() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
assert!(mgr.check(lease.id).await.is_ok());
}
#[tokio::test]
async fn check_nonexistent_fails() {
let mgr = LeaseManager::new();
assert!(mgr.check(LeaseId::new()).await.is_err());
}
#[tokio::test]
async fn consume_use_works() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, Some(2)).await;
assert!(mgr.consume_use(lease.id).await.is_ok());
assert!(mgr.consume_use(lease.id).await.is_ok());
assert!(mgr.consume_use(lease.id).await.is_err());
}
#[tokio::test]
async fn revoke_invalidates() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
mgr.revoke(lease.id, "test").await;
assert!(mgr.check(lease.id).await.is_err());
}
#[tokio::test]
async fn expire_stale_removes_revoked() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
mgr.revoke(lease.id, "done").await;
let removed = mgr.expire_stale().await;
assert_eq!(removed, 1);
assert!(mgr.active_for_thread(tid).await.is_empty());
}
#[tokio::test]
async fn active_for_thread_filters_correctly() {
let mgr = LeaseManager::new();
let t1 = ThreadId::new();
let t2 = ThreadId::new();
mgr.grant(t1, "github", vec![], None, None).await;
mgr.grant(t1, "memory", vec![], None, None).await;
mgr.grant(t2, "slack", vec![], None, None).await;
assert_eq!(mgr.active_for_thread(t1).await.len(), 2);
assert_eq!(mgr.active_for_thread(t2).await.len(), 1);
}
#[tokio::test]
async fn find_lease_for_action_respects_grants() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
mgr.grant(
tid,
"github",
vec!["create_issue".into(), "list_prs".into()],
None,
None,
)
.await;
assert!(
mgr.find_lease_for_action(tid, "create_issue")
.await
.is_some()
);
assert!(
mgr.find_lease_for_action(tid, "delete_repo")
.await
.is_none()
);
}
#[tokio::test]
async fn expired_lease_not_active() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr
.grant(
tid,
"github",
vec![],
Some(chrono::Duration::seconds(-10)),
None,
)
.await;
assert!(mgr.check(lease.id).await.is_err());
assert!(mgr.active_for_thread(tid).await.is_empty());
}
}
@@ -1,15 +0,0 @@
//! Capability management.
//!
//! - [`CapabilityRegistry`] — stores known capabilities and their actions
//! - [`LeaseManager`] — grants, validates, and expires capability leases
//! - [`PolicyEngine`] — deterministic effect-level allow/deny/approve
pub mod lease;
pub mod planner;
pub mod policy;
pub mod registry;
pub mod skill_tracker;
pub use lease::LeaseManager;
pub use policy::{PolicyDecision, PolicyEngine};
pub use registry::CapabilityRegistry;
@@ -1,84 +0,0 @@
//! Lease planning for new threads.
//!
//! Converts capability registry contents plus thread type into explicit
//! capability grants so new threads do not receive implicit wildcard leases.
use crate::capability::registry::CapabilityRegistry;
use crate::types::thread::ThreadType;
/// Explicit grant plan for a single capability.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CapabilityGrantPlan {
pub capability_name: String,
pub granted_actions: Vec<String>,
}
/// Plans explicit capability leases for new threads.
#[derive(Debug, Default)]
pub struct LeasePlanner;
impl LeasePlanner {
pub fn new() -> Self {
Self
}
/// Build the capability grants for a new thread.
pub fn plan_for_thread(
&self,
_thread_type: ThreadType,
capabilities: &CapabilityRegistry,
) -> Vec<CapabilityGrantPlan> {
capabilities
.list()
.into_iter()
.filter_map(|cap| {
let granted_actions: Vec<String> = cap
.actions
.iter()
.map(|action| action.name.clone())
.collect();
if granted_actions.is_empty() {
None
} else {
Some(CapabilityGrantPlan {
capability_name: cap.name.clone(),
granted_actions,
})
}
})
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{ActionDef, Capability, EffectType};
fn registry() -> CapabilityRegistry {
let mut reg = CapabilityRegistry::new();
reg.register(Capability {
name: "tools".into(),
description: "test".into(),
actions: vec![ActionDef {
name: "read_file".into(),
description: "read".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
reg
}
#[test]
fn foreground_threads_get_explicit_actions() {
let planner = LeasePlanner::new();
let plans = planner.plan_for_thread(ThreadType::Foreground, &registry());
assert_eq!(plans.len(), 1);
assert_eq!(plans[0].capability_name, "tools");
assert_eq!(plans[0].granted_actions, vec!["read_file"]);
}
}
@@ -1,380 +0,0 @@
//! Deterministic policy engine.
//!
//! Evaluates whether an action is allowed, denied, or requires approval
//! based on effect types, capability policies, and thread leases.
//! No LLM calls — purely deterministic.
use crate::types::capability::{
ActionDef, CapabilityLease, EffectType, PolicyCondition, PolicyEffect, PolicyRule,
};
use crate::types::provenance::Provenance;
/// The result of a policy evaluation.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PolicyDecision {
Allow,
Deny { reason: String },
RequireApproval { reason: String },
}
/// Deterministic policy engine.
///
/// Evaluation precedence: Deny > RequireApproval > Allow.
/// Checks are evaluated in order: global policies, then capability policies,
/// then action-level `requires_approval`, then effect-type checks against
/// the lease's allowed effects.
pub struct PolicyEngine {
global_policies: Vec<PolicyRule>,
/// Effect types that are always denied unless explicitly overridden.
pub(crate) denied_effects: Vec<EffectType>,
}
impl PolicyEngine {
pub fn new() -> Self {
Self {
global_policies: Vec::new(),
denied_effects: Vec::new(),
}
}
/// Add a global policy rule.
pub fn add_global_policy(&mut self, rule: PolicyRule) {
self.global_policies.push(rule);
}
/// Add an effect type that is always denied.
pub fn deny_effect(&mut self, effect: EffectType) {
self.denied_effects.push(effect);
}
/// Evaluate whether an action is allowed given a lease and capability policies.
pub fn evaluate(
&self,
action: &ActionDef,
lease: &CapabilityLease,
capability_policies: &[PolicyRule],
) -> PolicyDecision {
// 1. Check lease validity
if !lease.is_valid() {
return PolicyDecision::Deny {
reason: format!("lease for {} is expired/revoked", lease.capability_name),
};
}
// 2. Check lease covers this action
if !lease.covers_action(&action.name) {
return PolicyDecision::Deny {
reason: format!(
"lease for {} does not cover action {}",
lease.capability_name, action.name
),
};
}
// 3. Check denied effect types
for effect in &action.effects {
if self.denied_effects.contains(effect) {
return PolicyDecision::Deny {
reason: format!("effect type {effect:?} is denied by global policy"),
};
}
}
// 4. Evaluate global policies
let mut decision = PolicyDecision::Allow;
for rule in &self.global_policies {
if rule_matches(rule, action) {
decision = merge_decision(decision, rule.effect, &rule.name);
}
}
// 5. Evaluate capability-level policies
for rule in capability_policies {
if rule_matches(rule, action) {
decision = merge_decision(decision, rule.effect, &rule.name);
}
}
// 6. Check action-level requires_approval
if action.requires_approval {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"action requires approval",
);
}
decision
}
/// Evaluate with provenance-aware taint checking.
///
/// Extends the base evaluation with provenance-based rules:
/// - `LlmGenerated` data + `Financial` effect → RequireApproval
/// - `LlmGenerated` data + `WriteExternal` effect → RequireApproval
/// - `ToolOutput` data + `Financial` effect → RequireApproval
pub fn evaluate_with_provenance(
&self,
action: &ActionDef,
lease: &CapabilityLease,
capability_policies: &[PolicyRule],
provenance: &Provenance,
) -> PolicyDecision {
let mut decision = self.evaluate(action, lease, capability_policies);
// Provenance-based taint rules
match provenance {
Provenance::LlmGenerated => {
if action.effects.contains(&EffectType::Financial) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"LLM-generated data cannot trigger financial effects without approval",
);
}
if action.effects.contains(&EffectType::WriteExternal) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"LLM-generated data requires approval for external writes",
);
}
}
Provenance::ToolOutput { .. } => {
if action.effects.contains(&EffectType::Financial) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"tool output data requires approval for financial effects",
);
}
}
// User and System provenance are trusted
Provenance::User | Provenance::System => {}
// MemoryRetrieval is internal, treat as trusted
Provenance::MemoryRetrieval { .. } => {}
}
decision
}
}
impl Default for PolicyEngine {
fn default() -> Self {
Self::new()
}
}
/// Check whether a policy rule's condition matches the given action.
fn rule_matches(rule: &PolicyRule, action: &ActionDef) -> bool {
match &rule.condition {
PolicyCondition::Always => true,
PolicyCondition::ActionMatches { pattern } => action.name.contains(pattern.as_str()),
PolicyCondition::EffectTypeIs(effect) => action.effects.contains(effect),
}
}
/// Merge a new policy effect into the current decision.
/// Deny > RequireApproval > Allow.
fn merge_decision(current: PolicyDecision, effect: PolicyEffect, source: &str) -> PolicyDecision {
match effect {
PolicyEffect::Deny => PolicyDecision::Deny {
reason: source.to_string(),
},
PolicyEffect::RequireApproval => match current {
PolicyDecision::Deny { .. } => current,
_ => PolicyDecision::RequireApproval {
reason: source.to_string(),
},
},
PolicyEffect::Allow => current,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::LeaseId;
use crate::types::thread::ThreadId;
use chrono::Utc;
fn make_action(name: &str, effects: Vec<EffectType>, requires_approval: bool) -> ActionDef {
ActionDef {
name: name.into(),
description: String::new(),
parameters_schema: serde_json::json!({}),
effects,
requires_approval,
}
}
fn make_lease() -> CapabilityLease {
CapabilityLease {
id: LeaseId::new(),
thread_id: ThreadId::new(),
capability_name: "test".into(),
granted_actions: vec![],
granted_at: Utc::now(),
expires_at: None,
max_uses: None,
uses_remaining: None,
revoked: false,
}
}
#[test]
fn allow_by_default() {
let engine = PolicyEngine::new();
let action = make_action("read_file", vec![EffectType::ReadLocal], false);
let lease = make_lease();
assert_eq!(engine.evaluate(&action, &lease, &[]), PolicyDecision::Allow);
}
#[test]
fn denied_effect_type() {
let mut engine = PolicyEngine::new();
engine.deny_effect(EffectType::Financial);
let action = make_action("transfer", vec![EffectType::Financial], false);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn action_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("deploy", vec![EffectType::WriteExternal], true);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::RequireApproval { .. }
));
}
#[test]
fn global_policy_deny_overrides_approval() {
let mut engine = PolicyEngine::new();
engine.add_global_policy(PolicyRule {
name: "no external writes".into(),
condition: PolicyCondition::EffectTypeIs(EffectType::WriteExternal),
effect: PolicyEffect::Deny,
});
let action = make_action("deploy", vec![EffectType::WriteExternal], true);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn capability_policy_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("create_issue", vec![EffectType::WriteExternal], false);
let lease = make_lease();
let cap_policies = vec![PolicyRule {
name: "approve writes".into(),
condition: PolicyCondition::EffectTypeIs(EffectType::WriteExternal),
effect: PolicyEffect::RequireApproval,
}];
assert!(matches!(
engine.evaluate(&action, &lease, &cap_policies),
PolicyDecision::RequireApproval { .. }
));
}
#[test]
fn expired_lease_denied() {
let engine = PolicyEngine::new();
let action = make_action("read", vec![EffectType::ReadLocal], false);
let mut lease = make_lease();
lease.revoked = true;
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn lease_not_covering_action_denied() {
let engine = PolicyEngine::new();
let action = make_action("delete_repo", vec![EffectType::WriteExternal], false);
let mut lease = make_lease();
lease.granted_actions = vec!["create_issue".into()];
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn llm_generated_financial_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("transfer_funds", vec![EffectType::Financial], false);
let lease = make_lease();
let decision =
engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::LlmGenerated);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn llm_generated_write_external_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("post_message", vec![EffectType::WriteExternal], false);
let lease = make_lease();
let decision =
engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::LlmGenerated);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn user_provenance_allows_financial() {
let engine = PolicyEngine::new();
let action = make_action("transfer_funds", vec![EffectType::Financial], false);
let lease = make_lease();
let decision = engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::User);
assert_eq!(decision, PolicyDecision::Allow);
}
#[test]
fn tool_output_financial_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("pay_invoice", vec![EffectType::Financial], false);
let lease = make_lease();
let decision = engine.evaluate_with_provenance(
&action,
&lease,
&[],
&Provenance::ToolOutput {
action_name: "scrape_invoices".into(),
},
);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn action_matches_pattern() {
let mut engine = PolicyEngine::new();
engine.add_global_policy(PolicyRule {
name: "approve deletes".into(),
condition: PolicyCondition::ActionMatches {
pattern: "delete".into(),
},
effect: PolicyEffect::RequireApproval,
});
let action = make_action("delete_repo", vec![EffectType::WriteExternal], false);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::RequireApproval { .. }
));
let action2 = make_action("create_issue", vec![EffectType::WriteExternal], false);
assert_eq!(
engine.evaluate(&action2, &lease, &[]),
PolicyDecision::Allow
);
}
}
@@ -1,170 +0,0 @@
//! Capability registry — stores capability definitions available to the system.
use std::collections::HashMap;
use crate::types::capability::{ActionDef, Capability};
/// Registry of all known capabilities.
///
/// Capabilities are registered at startup (from extensions, built-in tools,
/// etc.) and queried when granting leases or resolving action names.
#[derive(Debug, Default)]
pub struct CapabilityRegistry {
capabilities: HashMap<String, Capability>,
}
impl CapabilityRegistry {
pub fn new() -> Self {
Self::default()
}
/// Register a capability. Overwrites any existing capability with the same name.
pub fn register(&mut self, capability: Capability) {
self.capabilities
.insert(capability.name.clone(), capability);
}
/// Look up a capability by name.
pub fn get(&self, name: &str) -> Option<&Capability> {
self.capabilities.get(name)
}
/// List all registered capabilities.
pub fn list(&self) -> Vec<&Capability> {
self.capabilities.values().collect()
}
/// Look up a specific action across all capabilities.
///
/// Returns `(capability_name, action_def)` if found.
pub fn find_action(&self, action_name: &str) -> Option<(&str, &ActionDef)> {
for cap in self.capabilities.values() {
if let Some(action) = cap.actions.iter().find(|a| a.name == action_name) {
return Some((&cap.name, action));
}
}
None
}
/// Get an action definition from a specific capability.
pub fn get_action(&self, capability_name: &str, action_name: &str) -> Option<&ActionDef> {
self.capabilities
.get(capability_name)?
.actions
.iter()
.find(|a| a.name == action_name)
}
/// Collect all action definitions across all capabilities.
pub fn all_actions(&self) -> Vec<&ActionDef> {
self.capabilities
.values()
.flat_map(|c| c.actions.iter())
.collect()
}
/// Number of registered capabilities.
pub fn len(&self) -> usize {
self.capabilities.len()
}
pub fn is_empty(&self) -> bool {
self.capabilities.is_empty()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::EffectType;
fn test_capability() -> Capability {
Capability {
name: "github".into(),
description: "GitHub integration".into(),
actions: vec![
ActionDef {
name: "create_issue".into(),
description: "Create a GitHub issue".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::WriteExternal, EffectType::CredentialedNetwork],
requires_approval: false,
},
ActionDef {
name: "list_prs".into(),
description: "List pull requests".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadExternal, EffectType::CredentialedNetwork],
requires_approval: false,
},
],
knowledge: vec!["When creating issues, always add labels.".into()],
policies: vec![],
}
}
#[test]
fn register_and_get() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert_eq!(reg.len(), 1);
assert!(reg.get("github").is_some());
assert!(reg.get("slack").is_none());
}
#[test]
fn find_action_across_capabilities() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
let (cap_name, action) = reg.find_action("create_issue").unwrap();
assert_eq!(cap_name, "github");
assert_eq!(action.name, "create_issue");
assert!(reg.find_action("nonexistent").is_none());
}
#[test]
fn get_action_from_capability() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert!(reg.get_action("github", "list_prs").is_some());
assert!(reg.get_action("github", "delete_repo").is_none());
assert!(reg.get_action("slack", "list_prs").is_none());
}
#[test]
fn all_actions_collects_across_capabilities() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
reg.register(Capability {
name: "memory".into(),
description: "Memory tools".into(),
actions: vec![ActionDef {
name: "memory_search".into(),
description: "Search memory".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
assert_eq!(reg.all_actions().len(), 3);
}
#[test]
fn overwrite_on_re_register() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert_eq!(reg.get("github").unwrap().actions.len(), 2);
reg.register(Capability {
name: "github".into(),
description: "Updated".into(),
actions: vec![],
knowledge: vec![],
policies: vec![],
});
assert_eq!(reg.get("github").unwrap().actions.len(), 0);
assert_eq!(reg.len(), 1);
}
}
@@ -1,289 +0,0 @@
//! Skill confidence tracking.
//!
//! Tracks usage and success/failure metrics for auto-extracted skills.
//! After each thread completes, the active skills' metrics are updated
//! based on whether the thread succeeded or failed.
use std::sync::Arc;
use ironclaw_skills::v2::V2SkillMetadata;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocId, DocType, MemoryDoc};
/// Tracks skill usage and updates confidence metrics.
pub struct SkillTracker {
store: Arc<dyn Store>,
}
impl SkillTracker {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Record that a skill was used in a completed thread.
///
/// Loads the skill's MemoryDoc, updates metrics in the metadata JSON,
/// and saves it back. If the doc is not found or has invalid metadata,
/// the error is logged and the operation is skipped.
pub async fn record_usage(&self, doc_id: DocId, success: bool) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
if doc.doc_type != DocType::Skill {
return Err(EngineError::Skill {
reason: format!("doc {} is not a skill (type: {:?})", doc_id.0, doc.doc_type),
});
}
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata for {}: {e}", doc_id.0),
})?;
meta.metrics.usage_count += 1;
if success {
meta.metrics.success_count += 1;
} else {
meta.metrics.failure_count += 1;
}
meta.metrics.last_used = Some(chrono::Utc::now());
let updated_doc = MemoryDoc {
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
/// Update a skill's content and increment its version.
///
/// Sets `parent_version` to the current version before incrementing,
/// enabling rollback if the update causes issues.
pub async fn update_skill(
&self,
doc_id: DocId,
new_content: String,
updater: impl FnOnce(&mut V2SkillMetadata),
) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata: {e}"),
})?;
meta.parent_version = Some(meta.version);
meta.version += 1;
updater(&mut meta);
let updated_doc = MemoryDoc {
content: new_content,
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
/// Rollback a skill to its previous version.
///
/// Decrements the version to `parent_version` if available. This is a
/// simple version decrement — the actual content rollback requires the
/// caller to also restore the content from a backup.
pub async fn rollback_skill(&self, doc_id: DocId) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata: {e}"),
})?;
let parent = meta.parent_version.ok_or_else(|| EngineError::Skill {
reason: format!("skill {} has no parent version to rollback to", doc_id.0),
})?;
meta.version = parent;
meta.parent_version = None;
let updated_doc = MemoryDoc {
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::project::ProjectId;
use ironclaw_skills::SkillTrust;
use ironclaw_skills::v2::{SkillMetrics, V2SkillSource};
fn make_skill_doc(project_id: ProjectId) -> MemoryDoc {
let meta = V2SkillMetadata {
name: "test-skill".to_string(),
version: 1,
description: "test".to_string(),
activation: Default::default(),
source: V2SkillSource::Extracted,
trust: SkillTrust::Trusted,
code_snippets: vec![],
metrics: SkillMetrics {
usage_count: 5,
success_count: 3,
failure_count: 2,
last_used: None,
},
parent_version: None,
content_hash: String::new(),
};
let mut doc = MemoryDoc::new(
project_id,
DocType::Skill,
"skill:test",
"Test skill prompt",
);
doc.metadata = serde_json::to_value(&meta).unwrap();
doc
}
#[tokio::test]
async fn test_record_usage_success() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker.record_usage(doc_id, true).await.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.metrics.usage_count, 6);
assert_eq!(meta.metrics.success_count, 4);
assert_eq!(meta.metrics.failure_count, 2);
assert!(meta.metrics.last_used.is_some());
}
#[tokio::test]
async fn test_record_usage_failure() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker.record_usage(doc_id, false).await.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.metrics.usage_count, 6);
assert_eq!(meta.metrics.success_count, 3);
assert_eq!(meta.metrics.failure_count, 3);
}
#[tokio::test]
async fn test_update_skill_increments_version() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker
.update_skill(doc_id, "Updated content".to_string(), |meta| {
meta.description = "Updated description".to_string();
})
.await
.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
assert_eq!(updated.content, "Updated content");
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.version, 2);
assert_eq!(meta.parent_version, Some(1));
assert_eq!(meta.description, "Updated description");
}
#[tokio::test]
async fn test_rollback_restores_parent_version() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
// First update to version 2
tracker
.update_skill(doc_id, "v2 content".to_string(), |_| {})
.await
.unwrap();
// Now rollback
tracker.rollback_skill(doc_id).await.unwrap();
let rolled = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(rolled.metadata).unwrap();
assert_eq!(meta.version, 1);
assert_eq!(meta.parent_version, None);
}
#[tokio::test]
async fn test_rollback_without_parent_fails() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store);
let result = tracker.rollback_skill(doc_id).await;
assert!(result.is_err());
}
#[tokio::test]
async fn test_record_usage_missing_doc() {
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![]));
let tracker = SkillTracker::new(store);
let result = tracker.record_usage(DocId::new(), true).await;
assert!(result.is_err());
}
}
@@ -1,176 +0,0 @@
//! Context compaction and token counting.
//!
//! When message history approaches the model's context limit, compaction
//! asks the LLM to summarize progress and resets the history. This follows
//! the official RLM pattern (compaction at 85% of context limit).
use std::sync::Arc;
use tracing::debug;
use crate::traits::llm::{LlmBackend, LlmCallConfig};
use crate::types::error::EngineError;
use crate::types::message::{MessageRole, ThreadMessage};
use crate::types::step::{LlmResponse, TokenUsage};
/// Characters per token estimate when no tokenizer is available.
/// Conservative estimate (official RLM uses 4).
const CHARS_PER_TOKEN: usize = 4;
/// Estimate token count for a list of messages.
///
/// Uses character length / `CHARS_PER_TOKEN` as a rough estimate.
/// The official RLM uses tiktoken when available; we use this fallback
/// since we don't depend on a Python tokenizer.
pub fn estimate_tokens(messages: &[ThreadMessage]) -> usize {
let total_chars: usize = messages
.iter()
.map(|m| {
m.content.len() + m.action_name.as_ref().map_or(0, |n| n.len()) + 4 // overhead per message (role token, delimiters)
})
.sum();
total_chars.div_ceil(CHARS_PER_TOKEN)
}
/// Check if compaction should be triggered.
///
/// Returns `true` when estimated token count exceeds `threshold_pct` of
/// the model's context limit.
pub fn should_compact(
messages: &[ThreadMessage],
model_context_limit: usize,
threshold_pct: f64,
) -> bool {
let tokens = estimate_tokens(messages);
let threshold = (model_context_limit as f64 * threshold_pct) as usize;
tokens >= threshold
}
/// The compaction prompt sent to the LLM.
const COMPACTION_PROMPT: &str = "\
Summarize your progress so far in a concise but complete way. Include:
1. What you have accomplished
2. Key intermediate results and variable values
3. What still needs to be done
4. Any errors encountered and how they were handled
Preserve all information needed to continue the task. Be specific about data values.";
/// Compact the message history by asking the LLM to summarize.
///
/// Returns the new (shorter) message list and the token usage from the
/// summarization call. The original messages are replaced with:
/// `[system_prompt, summary, continuation_note]`
///
/// The full original messages are returned separately so the caller can
/// store them (e.g., in a `history` variable or event log).
pub async fn compact_messages(
messages: &[ThreadMessage],
llm: &Arc<dyn LlmBackend>,
compaction_count: u32,
) -> Result<CompactionResult, EngineError> {
// Build a summarization request from existing messages + prompt
let mut summarize_messages = messages.to_vec();
summarize_messages.push(ThreadMessage::user(COMPACTION_PROMPT.to_string()));
let config = LlmCallConfig {
force_text: true,
..LlmCallConfig::default()
};
let output = llm.complete(&summarize_messages, &[], &config).await?;
let summary_text = match output.response {
LlmResponse::Text(t) => t,
LlmResponse::ActionCalls { content, .. } | LlmResponse::Code { content, .. } => {
content.unwrap_or_else(|| "[compaction produced no summary]".into())
}
};
// Preserve the system prompt (first message if it's a system message)
let system_msg = messages
.iter()
.find(|m| m.role == MessageRole::System)
.cloned();
// Build compacted history
let mut compacted = Vec::new();
if let Some(sys) = system_msg {
compacted.push(sys);
}
compacted.push(ThreadMessage::assistant(summary_text.clone()));
compacted.push(ThreadMessage::user(format!(
"Your conversation has been compacted {n} time(s). \
The summary above captures your progress. Continue working on the task.",
n = compaction_count + 1,
)));
let tokens_before = estimate_tokens(messages);
let tokens_after = estimate_tokens(&compacted);
debug!(
tokens_before,
tokens_after,
compaction_count = compaction_count + 1,
"context compacted"
);
Ok(CompactionResult {
compacted_messages: compacted,
summary: summary_text,
tokens_used: output.usage,
tokens_before,
tokens_after,
})
}
/// Result of a compaction operation.
pub struct CompactionResult {
/// The new (shorter) message list.
pub compacted_messages: Vec<ThreadMessage>,
/// The summary text produced by the LLM.
pub summary: String,
/// Tokens used by the summarization LLM call.
pub tokens_used: TokenUsage,
/// Estimated token count before compaction.
pub tokens_before: usize,
/// Estimated token count after compaction.
pub tokens_after: usize,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn estimate_tokens_empty() {
assert_eq!(estimate_tokens(&[]), 0);
}
#[test]
fn estimate_tokens_basic() {
let msgs = vec![
ThreadMessage::system("Hello world"), // 11 chars + 4 overhead = 15 / 4 = 3.75
ThreadMessage::user("Hi"), // 2 chars + 4 = 6 / 4 = 1.5
];
let tokens = estimate_tokens(&msgs);
// (11+4 + 2+4) / 4 = 21/4 = 5.25 → 6 (ceiling)
assert!(tokens > 0);
assert!(tokens < 100);
}
#[test]
fn should_compact_below_threshold() {
let msgs = vec![ThreadMessage::user("short message")];
assert!(!should_compact(&msgs, 128_000, 0.85));
}
#[test]
fn should_compact_above_threshold() {
// Create a message large enough to trigger compaction at low limit
let big = "x".repeat(1000);
let msgs = vec![ThreadMessage::user(big)];
// 1000 chars / 4 = 250 tokens. Context limit 200, threshold 85% = 170
assert!(should_compact(&msgs, 200, 0.85));
}
}
@@ -1,291 +0,0 @@
//! Context building for LLM calls.
//!
//! Assembles the message sequence and action definitions from thread state,
//! active leases, and project memory docs retrieved via the [`RetrievalEngine`].
use std::sync::Arc;
use crate::memory::RetrievalEngine;
use crate::traits::effect::EffectExecutor;
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::error::EngineError;
use crate::types::memory::MemoryDoc;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
/// Maximum number of memory docs to inject into context.
const MAX_CONTEXT_DOCS: usize = 5;
/// Build the context for an LLM call: messages and available actions.
///
/// Retrieves relevant memory docs from the project and injects them as a
/// system message after the main system prompt. This gives the LLM access
/// to lessons learned, skills, and known issues from prior threads.
pub async fn build_step_context(
messages: &[ThreadMessage],
leases: &[CapabilityLease],
effects: &Arc<dyn EffectExecutor>,
retrieval: Option<&RetrievalEngine>,
project_id: ProjectId,
goal: &str,
) -> Result<(Vec<ThreadMessage>, Vec<ActionDef>), EngineError> {
let actions = effects.available_actions(leases).await?;
let mut ctx_messages = messages.to_vec();
// Inject retrieved memory docs into the existing system prompt.
// Many providers require all system messages at the beginning (or a single
// system message), so we append to the first system message rather than
// inserting a separate one.
if let Some(engine) = retrieval {
let docs = engine
.retrieve_context(project_id, goal, MAX_CONTEXT_DOCS)
.await?;
if !docs.is_empty() {
let context_section = format_docs_as_context(&docs);
if !ctx_messages.is_empty()
&& ctx_messages[0].role == crate::types::message::MessageRole::System
{
// Append to existing system prompt
ctx_messages[0].content.push_str("\n\n");
ctx_messages[0].content.push_str(&context_section);
} else {
// No system message — prepend as one
ctx_messages.insert(0, ThreadMessage::system(context_section));
}
}
}
Ok((ctx_messages, actions))
}
/// Format memory docs into a system message for context injection.
fn format_docs_as_context(docs: &[MemoryDoc]) -> String {
let mut parts = vec!["## Prior Knowledge (from completed threads)\n".to_string()];
for doc in docs {
let type_label = match doc.doc_type {
crate::types::memory::DocType::Lesson => "LESSON",
crate::types::memory::DocType::Spec => "MISSING CAPABILITY",
crate::types::memory::DocType::Issue => "KNOWN ISSUE",
crate::types::memory::DocType::Summary => "CONTEXT",
crate::types::memory::DocType::Note => "NOTE",
crate::types::memory::DocType::Skill => "SKILL",
};
// Truncate long docs to avoid context bloat
let content: String = doc.content.chars().take(500).collect();
let truncated = if doc.content.chars().count() > 500 {
"..."
} else {
""
};
parts.push(format!(
"### [{type_label}] {}\n{content}{truncated}\n",
doc.title
));
}
parts.join("\n")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, DocType};
use crate::types::project::{Project, ProjectId};
use crate::types::step::{ActionResult, Step};
use crate::types::thread::{Thread, ThreadId, ThreadState};
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: std::time::Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct DocStore(Vec<MemoryDoc>);
#[async_trait::async_trait]
impl crate::traits::store::Store for DocStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, pid: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(self
.0
.iter()
.filter(|d| d.project_id == pid)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
#[tokio::test]
async fn context_injects_docs_after_system_prompt() {
let project = ProjectId::new();
let store: Arc<dyn crate::traits::store::Store> = Arc::new(DocStore(vec![MemoryDoc::new(
project,
DocType::Lesson,
"web tool alias",
"Use web-search not web_search",
)]));
let retrieval = RetrievalEngine::new(store);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![
ThreadMessage::system("You are an assistant."),
ThreadMessage::user("search the web"),
];
let (ctx_msgs, _) = build_step_context(
&messages,
&[],
&effects,
Some(&retrieval),
project,
"search the web",
)
.await
.unwrap();
// Should have 2 messages: system prompt (with docs appended), user message
assert_eq!(ctx_msgs.len(), 2);
assert_eq!(ctx_msgs[0].role, crate::types::message::MessageRole::System);
assert!(ctx_msgs[0].content.contains("You are an assistant."));
assert!(ctx_msgs[0].content.contains("Prior Knowledge"));
assert!(ctx_msgs[0].content.contains("LESSON"));
assert!(ctx_msgs[0].content.contains("web-search"));
assert_eq!(ctx_msgs[1].role, crate::types::message::MessageRole::User);
}
#[tokio::test]
async fn context_without_retrieval_passes_through() {
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![
ThreadMessage::system("prompt"),
ThreadMessage::user("hello"),
];
let (ctx_msgs, _) =
build_step_context(&messages, &[], &effects, None, ProjectId::new(), "hello")
.await
.unwrap();
// No injection — same number of messages
assert_eq!(ctx_msgs.len(), 2);
}
#[tokio::test]
async fn context_no_docs_means_no_injection() {
let project = ProjectId::new();
let store: Arc<dyn crate::traits::store::Store> = Arc::new(DocStore(vec![]));
let retrieval = RetrievalEngine::new(store);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![ThreadMessage::user("hello")];
let (ctx_msgs, _) =
build_step_context(&messages, &[], &effects, Some(&retrieval), project, "hello")
.await
.unwrap();
assert_eq!(ctx_msgs.len(), 1);
}
}
@@ -1,97 +0,0 @@
//! Tool intent nudge detection.
//!
//! Detects when the LLM expresses intent to use a tool without actually
//! producing action calls (e.g. "Let me search..." or "I'll fetch...").
//! Mirrors the logic in `src/agent/agentic_loop.rs` `llm_signals_tool_intent`.
/// Check if a text response signals tool intent without actual action calls.
///
/// Returns `true` if the text contains phrases like "Let me search...",
/// "I'll fetch...", etc. that indicate the LLM wanted to call a tool.
pub fn signals_tool_intent(response: &str) -> bool {
let lower = response.to_lowercase();
// Skip false positives
let false_positive_phrases = [
"let me explain",
"let me think",
"let me know",
"let me summarize",
"let me clarify",
];
for phrase in &false_positive_phrases {
if lower.contains(phrase) {
return false;
}
}
let intent_prefixes = ["let me ", "i'll ", "i will ", "i'm going to "];
let action_verbs = [
"search", "look up", "check", "fetch", "find", "query", "read", "run", "execute", "call",
"use", "invoke",
];
for prefix in &intent_prefixes {
if let Some(after) = lower.strip_prefix(prefix) {
for verb in &action_verbs {
if after.starts_with(verb) {
return true;
}
}
}
// Also check if the prefix appears mid-sentence (after period or newline)
for sep in [". ", ".\n", "\n"] {
for part in lower.split(sep) {
let trimmed = part.trim();
if let Some(after) = trimmed.strip_prefix(prefix) {
for verb in &action_verbs {
if after.starts_with(verb) {
return true;
}
}
}
}
}
}
false
}
/// The nudge message injected into context when tool intent is detected.
pub const TOOL_INTENT_NUDGE: &str = "You expressed intent to use a tool but didn't make an action call. \
Please go ahead and call the appropriate action.";
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn detects_let_me_search() {
assert!(signals_tool_intent("Let me search for that"));
}
#[test]
fn detects_ill_fetch() {
assert!(signals_tool_intent("I'll fetch the latest data"));
}
#[test]
fn ignores_let_me_explain() {
assert!(!signals_tool_intent("Let me explain how this works"));
}
#[test]
fn ignores_let_me_know() {
assert!(!signals_tool_intent("Let me know if you need more"));
}
#[test]
fn ignores_plain_text() {
assert!(!signals_tool_intent("The answer is 42."));
}
#[test]
fn detects_after_period() {
assert!(signals_tool_intent("Sure. Let me search for that."));
}
}
File diff suppressed because it is too large Load Diff
@@ -1,18 +0,0 @@
//! Step execution.
//!
//! - [`ExecutionLoop`] — core loop replacing `run_agentic_loop()`
//! - [`structured`] — Tier 0 action execution (structured tool calls)
//! - [`context`] — context building for LLM calls
//! - [`intent`] — tool intent nudge detection
pub mod compaction;
pub mod context;
pub mod intent;
pub mod loop_engine;
pub mod orchestrator;
pub mod prompt;
pub mod scripting;
pub mod structured;
pub mod trace;
pub use loop_engine::ExecutionLoop;
File diff suppressed because it is too large Load Diff
@@ -1,282 +0,0 @@
//! System prompt construction for the execution loop.
//!
//! Builds a CodeAct/RLM system prompt that instructs the LLM to write
//! Python code in ```repl blocks with tools available as callable functions.
//!
//! Prompt templates live in `crates/ironclaw_engine/prompts/` as plain
//! markdown files for easy inspection and iteration. They are embedded
//! at compile time via `include_str!` and can be extended at runtime with
//! prompt overlays stored as MemoryDocs.
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::capability::ActionDef;
use crate::types::project::ProjectId;
/// Runtime platform metadata injected into system prompts for self-awareness.
///
/// Provides the agent with knowledge about its own identity and environment
/// so it can answer questions about itself, its capabilities, and its
/// configuration without relying on training data.
#[derive(Debug, Clone, Default)]
pub struct PlatformInfo {
/// Software version (from CARGO_PKG_VERSION).
pub version: Option<String>,
/// LLM backend name (e.g. "nearai", "openai", "anthropic").
pub llm_backend: Option<String>,
/// Active model name.
pub model_name: Option<String>,
/// Database backend (e.g. "libsql", "postgres").
pub database_backend: Option<String>,
/// Active channel names (e.g. ["telegram", "cli"]).
pub active_channels: Vec<String>,
/// Owner identifier.
pub owner_id: Option<String>,
/// Project repository URL.
pub repo_url: Option<String>,
}
impl PlatformInfo {
/// Format as a prompt section. Returns empty string if no info is set.
pub fn to_prompt_section(&self) -> String {
let mut lines = Vec::new();
lines.push("You are **IronClaw**, a secure autonomous AI assistant platform.".into());
if let Some(ref v) = self.version {
lines.push(format!("- Version: {v}"));
}
if let Some(ref repo) = self.repo_url {
lines.push(format!("- Repository: {repo}"));
}
if let Some(ref owner) = self.owner_id {
lines.push(format!("- Owner: {owner}"));
}
if let Some(ref backend) = self.llm_backend {
let model = self.model_name.as_deref().unwrap_or("default");
lines.push(format!("- LLM: {backend} ({model})"));
}
if let Some(ref db) = self.database_backend {
lines.push(format!("- Database: {db}"));
}
if !self.active_channels.is_empty() {
lines.push(format!("- Channels: {}", self.active_channels.join(", ")));
}
if lines.len() <= 1 {
// Only the identity line, no runtime details — still include it
return format!("\n\n## Platform\n\n{}\n", lines[0]);
}
format!("\n\n## Platform\n\n{}\n", lines.join("\n"))
}
}
/// The main instruction block (before tool listing).
const CODEACT_PREAMBLE: &str = include_str!("../../prompts/codeact_preamble.md");
/// The strategy/closing block (after tool listing).
const CODEACT_POSTAMBLE: &str = include_str!("../../prompts/codeact_postamble.md");
/// Well-known title for the CodeAct preamble overlay.
pub const PREAMBLE_OVERLAY_TITLE: &str = "prompt:codeact_preamble";
/// Well-known tag for prompt overlay docs.
pub const PROMPT_OVERLAY_TAG: &str = "prompt_overlay";
/// Maximum size for a prompt overlay document (in chars).
const MAX_PROMPT_OVERLAY_CHARS: usize = 4000;
/// Build the system prompt for CodeAct/RLM execution.
///
/// The prompt instructs the LLM to:
/// - Write Python code in ```repl fenced blocks
/// - Call tools as regular Python functions
/// - Use llm_query(prompt, context) for sub-agent calls
/// - Use FINAL(answer) to return the final answer
/// - Access thread context via the `context` variable
///
/// If a Store is provided, checks for a runtime prompt overlay (a MemoryDoc
/// with tag "prompt_overlay" and title "prompt:codeact_preamble") and appends
/// its content after the compiled preamble. This enables the self-improvement
/// mission to evolve the system prompt at runtime.
pub async fn build_codeact_system_prompt(
actions: &[ActionDef],
store: Option<&Arc<dyn Store>>,
project_id: ProjectId,
platform: Option<&PlatformInfo>,
) -> String {
let mut prompt = String::from(CODEACT_PREAMBLE);
// Inject platform identity and runtime metadata
if let Some(info) = platform {
prompt.push_str(&info.to_prompt_section());
}
// Append runtime prompt overlay if available
if let Some(store) = store
&& let Some(overlay) = load_prompt_overlay(store, project_id).await
{
prompt.push_str("\n\n## Learned Rules (from self-improvement)\n\n");
prompt.push_str(&overlay);
}
// Add tool documentation
if !actions.is_empty() {
prompt.push_str("\n## Available tools (call as Python functions)\n\n");
for action in actions {
prompt.push_str(&format!("- `{}(", action.name));
// Extract parameter names from JSON schema
if let Some(props) = action.parameters_schema.get("properties")
&& let Some(obj) = props.as_object()
{
let params: Vec<&str> = obj.keys().map(String::as_str).collect();
prompt.push_str(&params.join(", "));
}
prompt.push_str(&format!(")` — {}\n", action.description));
}
}
prompt.push_str(CODEACT_POSTAMBLE);
prompt
}
/// Load the prompt overlay from the Store, if one exists for this project.
async fn load_prompt_overlay(store: &Arc<dyn Store>, project_id: ProjectId) -> Option<String> {
let docs = store.list_memory_docs(project_id).await.ok()?;
let overlay = docs.iter().find(|d| {
d.title == PREAMBLE_OVERLAY_TITLE && d.tags.contains(&PROMPT_OVERLAY_TAG.to_string())
})?;
let content: String = overlay
.content
.chars()
.take(MAX_PROMPT_OVERLAY_CHARS)
.collect();
if content.is_empty() {
return None;
}
Some(content)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::memory::{DocId, DocType, MemoryDoc};
#[tokio::test]
async fn prompt_without_store_uses_compiled_preamble() {
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), None).await;
assert!(prompt.contains("Python REPL environment"));
assert!(prompt.contains("Strategy"));
assert!(!prompt.contains("Learned Rules"));
}
#[tokio::test]
async fn prompt_with_overlay_appends_rules() {
let project_id = ProjectId(uuid::Uuid::new_v4());
let overlay = MemoryDoc {
id: DocId::new(),
project_id,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: "9. Never call web_fetch — use http() instead.".into(),
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
assert!(prompt.contains("Learned Rules"));
assert!(prompt.contains("Never call web_fetch"));
}
#[tokio::test]
async fn prompt_overlay_size_is_capped() {
let project_id = ProjectId(uuid::Uuid::new_v4());
// Create an overlay that exceeds MAX_PROMPT_OVERLAY_CHARS using a char
// not found in the compiled preamble/postamble
let huge_content = "\u{2603}".repeat(MAX_PROMPT_OVERLAY_CHARS + 1000); // snowman
let overlay = MemoryDoc {
id: DocId::new(),
project_id,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: huge_content,
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
let snowman_count = prompt.chars().filter(|c| *c == '\u{2603}').count();
assert_eq!(snowman_count, MAX_PROMPT_OVERLAY_CHARS);
}
#[tokio::test]
async fn prompt_ignores_wrong_project_overlay() {
let project_id = ProjectId(uuid::Uuid::new_v4());
let other_project = ProjectId(uuid::Uuid::new_v4());
let overlay = MemoryDoc {
id: DocId::new(),
project_id: other_project,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: "Should not appear".into(),
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
assert!(!prompt.contains("Should not appear"));
assert!(!prompt.contains("Learned Rules"));
}
#[tokio::test]
async fn prompt_with_platform_info_injects_identity() {
let info = PlatformInfo {
version: Some("1.2.3".into()),
llm_backend: Some("nearai".into()),
model_name: Some("qwen3-235b".into()),
database_backend: Some("libsql".into()),
active_channels: vec!["telegram".into(), "cli".into()],
owner_id: Some("alice.near".into()),
repo_url: Some("https://github.com/nearai/ironclaw".into()),
};
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), Some(&info)).await;
assert!(prompt.contains("IronClaw"));
assert!(prompt.contains("1.2.3"));
assert!(prompt.contains("nearai"));
assert!(prompt.contains("qwen3-235b"));
assert!(prompt.contains("libsql"));
assert!(prompt.contains("telegram"));
assert!(prompt.contains("alice.near"));
assert!(prompt.contains("github.com/nearai/ironclaw"));
}
#[tokio::test]
async fn prompt_without_platform_info_has_no_platform_section() {
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), None).await;
assert!(!prompt.contains("## Platform"));
}
}
File diff suppressed because it is too large Load Diff
@@ -1,738 +0,0 @@
//! Tier 0 executor: structured tool calls.
//!
//! Executes action calls by delegating to the `EffectExecutor` trait,
//! checking leases and policies for each call.
use std::sync::Arc;
use crate::capability::lease::LeaseManager;
use crate::capability::policy::{PolicyDecision, PolicyEngine};
use crate::runtime::messaging::ThreadOutcome;
use crate::traits::effect::{EffectExecutor, ThreadExecutionContext};
use crate::types::error::EngineError;
use crate::types::event::EventKind;
use crate::types::step::{ActionCall, ActionResult};
use crate::types::thread::Thread;
/// Result of executing a batch of action calls.
pub struct ActionBatchResult {
/// Results for each action call (in order).
pub results: Vec<ActionResult>,
/// Events generated during execution.
pub events: Vec<EventKind>,
/// If set, execution was interrupted and the thread needs approval.
pub need_approval: Option<ThreadOutcome>,
}
/// Execute a batch of action calls using the Tier 0 (structured) approach.
///
/// For each action call:
/// 1. Find the lease that grants this action
/// 2. Check policy (deny/allow/approve)
/// 3. Consume a lease use
/// 4. Call `EffectExecutor::execute_action()`
/// 5. Record result and emit event
///
/// Stops at the first action that requires approval.
pub async fn execute_action_calls(
calls: &[ActionCall],
thread: &Thread,
effects: &Arc<dyn EffectExecutor>,
leases: &LeaseManager,
policy: &PolicyEngine,
context: &ThreadExecutionContext,
capability_policies: &[crate::types::capability::PolicyRule],
) -> Result<ActionBatchResult, EngineError> {
let mut results = Vec::with_capacity(calls.len());
let mut events = Vec::new();
for call in calls {
// 1. Find the lease for this action
let lease = match leases
.find_lease_for_action(thread.id, &call.action_name)
.await
{
Some(l) => l,
None => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": format!(
"no active lease covers action '{}'", call.action_name
)}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: format!("no lease for action '{}'", call.action_name),
params_summary: None,
});
results.push(error_result);
continue;
}
};
// 2. Find the action definition and check policy
let action_def = effects
.available_actions(std::slice::from_ref(&lease))
.await?
.into_iter()
.find(|a| a.name == call.action_name);
if let Some(ref action_def) = action_def {
let decision = policy.evaluate(action_def, &lease, capability_policies);
match decision {
PolicyDecision::Deny { reason } => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": format!("denied: {reason}")}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: reason,
params_summary: None,
});
results.push(error_result);
continue;
}
PolicyDecision::RequireApproval { .. } => {
events.push(EventKind::ApprovalRequested {
action_name: call.action_name.clone(),
call_id: call.id.clone(),
});
return Ok(ActionBatchResult {
results,
events,
need_approval: Some(ThreadOutcome::NeedApproval {
action_name: call.action_name.clone(),
call_id: call.id.clone(),
parameters: call.parameters.clone(),
}),
});
}
PolicyDecision::Allow => {}
}
}
// 3. Consume a lease use
leases.consume_use(lease.id).await?;
// 4. Execute the action
let result = effects
.execute_action(&call.action_name, call.parameters.clone(), &lease, context)
.await;
match result {
Ok(mut action_result) => {
// EffectExecutor doesn't receive call_id; stamp it from the
// original ActionCall so downstream messages carry the correct ID.
action_result.call_id = call.id.clone();
events.push(EventKind::ActionExecuted {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
duration_ms: action_result.duration.as_millis() as u64,
params_summary: None,
});
results.push(action_result);
}
Err(crate::types::error::EngineError::NeedAuthentication {
credential_name,
action_name,
call_id,
parameters,
}) => {
// Interrupt the batch — thread should pause for authentication.
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: action_name.clone(),
call_id: call_id.clone(),
error: format!("authentication required for credential '{credential_name}'"),
params_summary: None,
});
return Ok(ActionBatchResult {
results,
events,
need_approval: Some(ThreadOutcome::NeedAuthentication {
credential_name,
action_name,
call_id,
parameters,
}),
});
}
Err(e) => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": e.to_string()}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: e.to_string(),
params_summary: None,
});
results.push(error_result);
}
}
}
Ok(ActionBatchResult {
results,
events,
need_approval: None,
})
}
#[cfg(test)]
mod tests {
use super::*;
use crate::traits::effect::ThreadExecutionContext;
use crate::types::capability::{ActionDef, CapabilityLease, EffectType};
use crate::types::project::ProjectId;
use crate::types::step::StepId;
use crate::types::thread::{Thread, ThreadConfig, ThreadType};
use std::sync::Mutex;
use std::time::Duration;
struct MockEffects {
results: Mutex<Vec<Result<ActionResult, EngineError>>>,
actions: Vec<ActionDef>,
}
impl MockEffects {
fn new(actions: Vec<ActionDef>, results: Vec<Result<ActionResult, EngineError>>) -> Self {
Self {
results: Mutex::new(results),
actions,
}
}
}
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_name: &str,
_params: serde_json::Value,
_lease: &CapabilityLease,
_ctx: &ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
let mut results = self.results.lock().unwrap();
if results.is_empty() {
Ok(ActionResult {
call_id: String::new(), // EffectExecutor doesn't set call_id
action_name: String::new(),
output: serde_json::json!({"result": "ok"}),
is_error: false,
duration: Duration::from_millis(1),
})
} else {
results.remove(0)
}
}
async fn available_actions(
&self,
_leases: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(self.actions.clone())
}
}
fn test_action(name: &str) -> ActionDef {
ActionDef {
name: name.into(),
description: "Test tool".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}
}
fn make_exec_context(thread: &Thread) -> ThreadExecutionContext {
ThreadExecutionContext {
thread_id: thread.id,
thread_type: thread.thread_type,
project_id: thread.project_id,
user_id: "test".into(),
step_id: StepId::new(),
}
}
// ── call_id propagation tests ────────────────────────────
#[tokio::test]
async fn call_id_preserved_on_successful_execution() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("web_search")],
vec![Ok(ActionResult {
call_id: String::new(), // EffectExecutor returns empty
action_name: "web_search".into(),
output: serde_json::json!({"results": []}),
is_error: false,
duration: Duration::from_millis(42),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "search", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_r2o5mqBgdNUlH8KzskncUGaX".into(),
action_name: "web_search".into(),
parameters: serde_json::json!({"query": "test"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// call_id must be stamped from ActionCall, not the empty EffectExecutor return
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_r2o5mqBgdNUlH8KzskncUGaX");
assert_eq!(result.results[0].action_name, "web_search");
assert!(!result.results[0].is_error);
// Event should carry the same call_id
let exec_event = result
.events
.iter()
.find(|e| matches!(e, EventKind::ActionExecuted { .. }));
assert!(exec_event.is_some());
if let Some(EventKind::ActionExecuted {
call_id,
action_name,
..
}) = exec_event
{
assert_eq!(call_id, "call_r2o5mqBgdNUlH8KzskncUGaX");
assert_eq!(action_name, "web_search");
}
}
#[tokio::test]
async fn call_id_preserved_on_execution_error() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("shell")],
vec![Err(EngineError::Effect {
reason: "permission denied".into(),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "exec", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_abc123def".into(),
action_name: "shell".into(),
parameters: serde_json::json!({"cmd": "ls"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_abc123def");
assert!(result.results[0].is_error);
let fail_event = result
.events
.iter()
.find(|e| matches!(e, EventKind::ActionFailed { .. }));
assert!(fail_event.is_some());
if let Some(EventKind::ActionFailed { call_id, .. }) = fail_event {
assert_eq!(call_id, "call_abc123def");
}
}
#[tokio::test]
async fn call_id_preserved_when_no_lease() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(vec![], vec![]));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
// No lease granted — action should fail with correct call_id
let calls = vec![ActionCall {
id: "call_no_lease_123".into(),
action_name: "web_search".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_no_lease_123");
assert!(result.results[0].is_error);
if let Some(EventKind::ActionFailed { call_id, error, .. }) = result.events.first() {
assert_eq!(call_id, "call_no_lease_123");
assert!(error.contains("no lease"));
} else {
panic!("expected ActionFailed event");
}
}
#[tokio::test]
async fn multiple_calls_each_get_correct_call_id() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("tool_a"), test_action("tool_b")],
vec![
Ok(ActionResult {
call_id: String::new(),
action_name: "tool_a".into(),
output: serde_json::json!("a_result"),
is_error: false,
duration: Duration::from_millis(1),
}),
Ok(ActionResult {
call_id: String::new(),
action_name: "tool_b".into(),
output: serde_json::json!("b_result"),
is_error: false,
duration: Duration::from_millis(2),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "id_aaaa".into(),
action_name: "tool_a".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "id_bbbb".into(),
action_name: "tool_b".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 2);
assert_eq!(result.results[0].call_id, "id_aaaa");
assert_eq!(result.results[1].call_id, "id_bbbb");
}
// ── NeedAuthentication tests ─────────────────────────────
#[tokio::test]
async fn need_authentication_interrupts_batch() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http")],
vec![Err(EngineError::NeedAuthentication {
credential_name: "github_token".into(),
action_name: "http".into(),
call_id: "call_auth_1".into(),
parameters: serde_json::json!({"url": "https://api.github.com/repos"}),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_auth_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({"url": "https://api.github.com/repos"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Batch should be interrupted with NeedAuthentication outcome
assert!(
result.need_approval.is_some(),
"NeedAuthentication should interrupt the batch"
);
match result.need_approval.unwrap() {
ThreadOutcome::NeedAuthentication {
credential_name,
action_name,
..
} => {
assert_eq!(credential_name, "github_token");
assert_eq!(action_name, "http");
}
other => panic!("expected NeedAuthentication, got {:?}", other),
}
// ActionFailed event should be emitted
assert!(
result
.events
.iter()
.any(|e| matches!(e, EventKind::ActionFailed { .. })),
"should emit ActionFailed event"
);
}
#[tokio::test]
async fn need_authentication_stops_before_subsequent_calls() {
// Two calls: first needs auth, second should never execute
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http"), test_action("echo")],
vec![
Err(EngineError::NeedAuthentication {
credential_name: "api_key".into(),
action_name: "http".into(),
call_id: "call_1".into(),
parameters: serde_json::json!({}),
}),
// This should never be called
Ok(ActionResult {
call_id: String::new(),
action_name: "echo".into(),
output: serde_json::json!("should not appear"),
is_error: false,
duration: Duration::from_millis(1),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "call_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "call_2".into(),
action_name: "echo".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Second call should NOT have executed
assert!(
result.results.is_empty(),
"no results should be returned before the interrupted call"
);
assert!(result.need_approval.is_some());
}
/// Regular EngineError::Effect (not NeedAuthentication) should NOT interrupt —
/// it becomes a normal error result and execution continues.
#[tokio::test]
async fn regular_effect_error_does_not_interrupt() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http"), test_action("echo")],
vec![
Err(EngineError::Effect {
reason: "connection timeout".into(),
}),
Ok(ActionResult {
call_id: String::new(),
action_name: "echo".into(),
output: serde_json::json!("second call ran"),
is_error: false,
duration: Duration::from_millis(1),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "call_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "call_2".into(),
action_name: "echo".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Both calls should have results (error does not interrupt)
assert_eq!(result.results.len(), 2);
assert!(result.results[0].is_error);
assert!(!result.results[1].is_error);
assert!(
result.need_approval.is_none(),
"no interruption for regular errors"
);
}
// ── call_id preservation (OpenAI/Mistral) ─────────────────
/// Provider-specific: OpenAI rejects empty string call_id. Verify no result
/// ever has an empty call_id when the ActionCall provided one.
#[tokio::test]
async fn openai_empty_call_id_never_produced() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("echo")],
vec![Ok(ActionResult {
call_id: String::new(), // EffectExecutor always returns empty
action_name: String::new(),
output: serde_json::json!("hello"),
is_error: false,
duration: Duration::from_millis(1),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
let calls = vec![ActionCall {
id: "aB3xK9mZq".into(), // Mistral-compatible 9-char ID
action_name: "echo".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Must NOT be empty — must be stamped from the ActionCall
assert!(!result.results[0].call_id.is_empty());
assert_eq!(result.results[0].call_id, "aB3xK9mZq");
}
/// Mistral requires call_id matching [a-zA-Z0-9]{9}.
/// Verify the ID passes through unmodified (normalization is LLM-layer concern,
/// but engine must never lose it).
#[tokio::test]
async fn mistral_format_call_id_preserved() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("web_search")],
vec![Ok(ActionResult {
call_id: String::new(),
action_name: "web_search".into(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
// Mistral format: exactly 9 alphanumeric chars
let mistral_id = "xK3mR9bZq";
let calls = vec![ActionCall {
id: mistral_id.into(),
action_name: "web_search".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results[0].call_id, mistral_id);
// Event also preserves the exact format
if let Some(EventKind::ActionExecuted { call_id, .. }) = result.events.first() {
assert_eq!(call_id, mistral_id);
}
}
}
@@ -1,565 +0,0 @@
//! Execution trace recording and analysis.
//!
//! Records full execution traces to JSON files for debugging. Optionally
//! runs a post-execution analysis to detect common issues.
//!
//! Enable with `ENGINE_V2_TRACE=1` env var. Traces are written to
//! `engine_trace_{timestamp}.json` in the current directory.
use std::path::PathBuf;
use chrono::Utc;
use serde::Serialize;
use tracing::{debug, warn};
use crate::types::event::ThreadEvent;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Check if trace recording is enabled.
pub fn is_trace_enabled() -> bool {
std::env::var("ENGINE_V2_TRACE")
.map(|v| v == "1" || v == "true")
.unwrap_or(false)
}
/// A complete execution trace for a single thread.
#[derive(Debug, Serialize)]
pub struct ExecutionTrace {
pub thread_id: ThreadId,
pub goal: String,
pub final_state: ThreadState,
pub step_count: usize,
pub total_tokens: u64,
pub messages: Vec<MessageRecord>,
pub events: Vec<ThreadEvent>,
pub issues: Vec<TraceIssue>,
pub timestamp: chrono::DateTime<Utc>,
}
/// A single doc record, for the trace.
#[derive(Debug, Serialize)]
pub struct DocRecord {
pub doc_type: String,
pub title: String,
pub content: String,
}
/// A message in the trace with role labeling.
#[derive(Debug, Serialize)]
pub struct MessageRecord {
pub role: String,
pub content_length: usize,
pub content_preview: String,
pub full_content: String,
pub action_name: Option<String>,
pub action_call_id: Option<String>,
}
/// An issue detected by the retrospective analyzer.
#[derive(Debug, Serialize)]
pub struct TraceIssue {
pub severity: IssueSeverity,
pub category: String,
pub description: String,
pub step: Option<usize>,
}
#[derive(Debug, PartialEq, Serialize)]
pub enum IssueSeverity {
Error,
Warning,
Info,
}
/// Build a trace from a completed thread.
pub fn build_trace(thread: &Thread) -> ExecutionTrace {
let messages: Vec<MessageRecord> = thread
.messages
.iter()
.map(|m| {
let preview: String = m.content.chars().take(300).collect();
MessageRecord {
role: format!("{:?}", m.role),
content_length: m.content.chars().count(),
content_preview: if m.content.chars().count() > 300 {
format!("{preview}...")
} else {
preview
},
full_content: m.content.clone(),
action_name: m.action_name.clone(),
action_call_id: m.action_call_id.clone(),
}
})
.collect();
let issues = analyze_trace(thread);
ExecutionTrace {
thread_id: thread.id,
goal: thread.goal.clone(),
final_state: thread.state,
step_count: thread.step_count,
total_tokens: thread.total_tokens_used,
messages,
events: thread.events.clone(),
issues,
timestamp: Utc::now(),
}
}
/// Write a trace to a JSON file.
pub fn write_trace(trace: &ExecutionTrace) -> Option<PathBuf> {
let filename = format!("engine_trace_{}.json", Utc::now().format("%Y%m%dT%H%M%S"));
let path = PathBuf::from(&filename);
match serde_json::to_string_pretty(trace) {
Ok(json) => match std::fs::write(&path, json) {
Ok(()) => {
debug!(path = %path.display(), "Execution trace written");
Some(path)
}
Err(e) => {
warn!("Failed to write trace: {e}");
None
}
},
Err(e) => {
warn!("Failed to serialize trace: {e}");
None
}
}
}
/// Print a summary of the trace to the log.
pub fn log_trace_summary(trace: &ExecutionTrace) {
debug!(
thread_id = %trace.thread_id,
goal = %trace.goal,
state = ?trace.final_state,
steps = trace.step_count,
tokens = trace.total_tokens,
messages = trace.messages.len(),
events = trace.events.len(),
issues = trace.issues.len(),
"=== Engine V2 Trace Summary ==="
);
for issue in &trace.issues {
match issue.severity {
IssueSeverity::Error => warn!(
category = %issue.category,
step = ?issue.step,
"ISSUE: {}",
issue.description
),
IssueSeverity::Warning => warn!(
category = %issue.category,
step = ?issue.step,
"WARNING: {}",
issue.description
),
IssueSeverity::Info => debug!(
category = %issue.category,
step = ?issue.step,
"NOTE: {}",
issue.description
),
}
}
}
// ── Retrospective analysis ──────────────────────────────────
/// Analyze a completed thread for common issues.
fn analyze_trace(thread: &Thread) -> Vec<TraceIssue> {
let mut issues = Vec::new();
// 1. Check if the thread failed
if thread.state == ThreadState::Failed {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "thread_failure".into(),
description: "Thread ended in Failed state".into(),
step: None,
});
}
// 2. Check for empty response (no FINAL, no useful output)
let has_assistant_response = thread
.messages
.iter()
.any(|m| m.role == crate::types::message::MessageRole::Assistant && !m.content.is_empty());
if !has_assistant_response {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "no_response".into(),
description: "No assistant message in thread — model may not have generated output"
.into(),
step: None,
});
}
// 3. Check for tool errors
let tool_errors: Vec<&ThreadEvent> = thread
.events
.iter()
.filter(|e| matches!(e.kind, crate::types::event::EventKind::ActionFailed { .. }))
.collect();
if !tool_errors.is_empty() {
for event in &tool_errors {
if let crate::types::event::EventKind::ActionFailed {
action_name, error, ..
} = &event.kind
{
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "tool_error".into(),
description: format!("Tool '{action_name}' failed: {error}"),
step: None,
});
}
}
}
// 4. Check for code execution errors in output messages.
// Code output appears as User-role messages (Monty stdout/stderr) with
// prefixes like "[stdout]" or "[stderr]". Skip the System prompt (index 0)
// and Assistant messages to avoid false positives from example text.
let error_patterns = [
"NameError",
"SyntaxError",
"TypeError",
"NotImplementedError",
];
for (i, msg) in thread.messages.iter().enumerate() {
let is_code_output = msg.role == crate::types::message::MessageRole::User
&& (msg.content.starts_with("[stdout]")
|| msg.content.starts_with("[stderr]")
|| msg.content.starts_with("[code ")
|| msg.content.starts_with("Traceback"));
if is_code_output && error_patterns.iter().any(|p| msg.content.contains(p)) {
let preview: String = msg.content.chars().take(200).collect();
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "code_error".into(),
description: format!("Code execution error in message {i}: {preview}"),
step: None,
});
}
}
// 5. Check for empty call_id on ActionResult messages (causes LLM API rejection).
for (i, msg) in thread.messages.iter().enumerate() {
if msg.role == crate::types::message::MessageRole::ActionResult {
let call_id_empty = msg.action_call_id.as_ref().is_none_or(|id| id.is_empty());
if call_id_empty {
let name = msg.action_name.as_deref().unwrap_or("unknown");
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "empty_call_id".into(),
description: format!(
"ActionResult message {i} (tool '{name}') has empty call_id — will cause LLM API rejection"
),
step: None,
});
}
}
}
// 6. Check for model ignoring tool results (hallucination risk).
// In Tier 0 (structured), results appear as ActionResult messages.
// In Tier 1 (CodeAct), results appear as User messages with "[tool result]" prefixes.
let has_tool_results = thread
.messages
.iter()
.any(|m| m.role == crate::types::message::MessageRole::ActionResult);
let has_tool_output_in_messages = thread.messages.iter().any(|m| {
m.role == crate::types::message::MessageRole::ActionResult
|| m.content.contains(" result]")
|| m.content.contains(" error]")
});
if has_tool_results && !has_tool_output_in_messages {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "missing_tool_output".into(),
description:
"Tool results exist but no tool output in messages — model may not see tool results"
.into(),
step: None,
});
}
// 7. Check for excessive iterations
if thread.step_count > 10 {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "excessive_steps".into(),
description: format!(
"Thread took {} steps — may be stuck in a loop",
thread.step_count
),
step: None,
});
}
// 8. Check for text response without FINAL (model answered from memory)
let text_without_code = thread.events.iter().all(|e| {
!matches!(
e.kind,
crate::types::event::EventKind::ActionExecuted { .. }
)
});
if text_without_code && thread.step_count == 1 && has_assistant_response {
issues.push(TraceIssue {
severity: IssueSeverity::Info,
category: "no_tools_used".into(),
description: "Model answered in one step without using any tools — may be answering from training data".into(),
step: Some(1),
});
}
// 9. Check for LLM not producing code blocks
let code_steps = thread
.events
.iter()
.filter(|e| matches!(e.kind, crate::types::event::EventKind::StepStarted { .. }))
.count();
let text_responses_without_code = thread
.messages
.iter()
.filter(|m| {
m.role == crate::types::message::MessageRole::Assistant
&& !m.content.contains("```")
&& !m.content.contains("FINAL(")
})
.count();
if text_responses_without_code > 0 && code_steps > 0 {
issues.push(TraceIssue {
severity: IssueSeverity::Info,
category: "mixed_mode".into(),
description: format!(
"{text_responses_without_code} text response(s) without code blocks — model may not be following CodeAct prompt"
),
step: None,
});
}
// 10. Extract failure reason from StateChanged → Failed events
for event in &thread.events {
if let crate::types::event::EventKind::StateChanged {
to: ThreadState::Failed,
reason: Some(reason),
..
} = &event.kind
{
if reason.contains("LLM") || reason.contains("Provider") {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "llm_error".into(),
description: format!("LLM provider error: {}", truncate(reason, 300)),
step: None,
});
} else if reason.contains("orchestrator") {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "orchestrator_error".into(),
description: format!("Orchestrator error: {}", truncate(reason, 300)),
step: None,
});
}
}
}
issues
}
fn truncate(s: &str, max_chars: usize) -> String {
let chars: String = s.chars().take(max_chars).collect();
if s.chars().count() > max_chars {
format!("{chars}...")
} else {
chars
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::event::EventKind;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::step::StepId;
use crate::types::thread::{ThreadConfig, ThreadType};
fn make_thread() -> Thread {
Thread::new(
"test goal",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
)
}
// ── empty_call_id detection (OpenAI / Codex rejection) ───
/// OpenAI and Codex reject ActionResult messages with empty call_id.
/// The trace analyzer must flag these as errors.
#[test]
fn detects_empty_call_id_on_action_result() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
// Simulate the bug: empty call_id
thread.add_message(ThreadMessage::action_result("", "web_search", "result"));
let issues = analyze_trace(&thread);
let empty_id_issues: Vec<_> = issues
.iter()
.filter(|i| i.category == "empty_call_id")
.collect();
assert_eq!(empty_id_issues.len(), 1);
assert_eq!(empty_id_issues[0].severity, IssueSeverity::Error);
assert!(empty_id_issues[0].description.contains("web_search"));
}
/// ActionResult with None call_id should also be flagged.
#[test]
fn detects_none_call_id_on_action_result() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
// Manually construct a message with None call_id
thread.add_message(ThreadMessage {
role: crate::types::message::MessageRole::ActionResult,
content: "result".into(),
provenance: crate::types::provenance::Provenance::ToolOutput {
action_name: "shell".into(),
},
action_call_id: None,
action_name: Some("shell".into()),
action_calls: None,
timestamp: chrono::Utc::now(),
});
let issues = analyze_trace(&thread);
assert!(issues.iter().any(|i| i.category == "empty_call_id"));
}
/// No false positive: valid call_id should not be flagged.
#[test]
fn no_false_positive_for_valid_call_id() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
thread.add_message(ThreadMessage::action_result(
"call_abc123",
"web_search",
"result",
));
let issues = analyze_trace(&thread);
assert!(
!issues.iter().any(|i| i.category == "empty_call_id"),
"valid call_id should not be flagged"
);
}
// ── tool_error detection ─────────────────────────────────
/// ActionFailed events should produce tool_error warnings.
#[test]
fn detects_tool_failures_in_events() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("ok"));
thread.events.push(ThreadEvent::new(
thread.id,
EventKind::ActionFailed {
step_id: StepId::new(),
action_name: "web_search".into(),
call_id: "call_123".into(),
error: "No lease for action 'web_search'".into(),
params_summary: None,
},
));
let issues = analyze_trace(&thread);
let tool_errors: Vec<_> = issues
.iter()
.filter(|i| i.category == "tool_error")
.collect();
assert_eq!(tool_errors.len(), 1);
assert!(tool_errors[0].description.contains("web_search"));
}
// ── thread_failure detection ─────────────────────────────
#[test]
fn detects_failed_thread_state() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("trying"));
thread.state = ThreadState::Failed;
let issues = analyze_trace(&thread);
assert!(issues.iter().any(|i| i.category == "thread_failure"));
}
// ── LLM error detection from StateChanged events ─────────
/// Reproduces the exact pattern from the trace: OpenAI rejects empty call_id.
#[test]
fn detects_llm_error_from_state_changed() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("ok"));
thread.state = ThreadState::Failed;
thread.events.push(ThreadEvent::new(
thread.id,
EventKind::StateChanged {
from: ThreadState::Running,
to: ThreadState::Failed,
reason: Some(
"LLM error: Provider openai_codex request failed: HTTP 400 Bad Request: \
Invalid 'input[5].call_id': empty string"
.into(),
),
},
));
let issues = analyze_trace(&thread);
assert!(
issues.iter().any(|i| i.category == "llm_error"),
"should detect LLM provider error in StateChanged reason"
);
}
// ── Multiple empty call_ids ──────────────────────────────
/// Anthropic sends consecutive tool results merged into one User message.
/// If multiple ActionResults have empty call_ids, each must be flagged.
#[test]
fn flags_each_empty_call_id_separately() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("parallel calls"));
thread.add_message(ThreadMessage::action_result("", "tool_a", "result_a"));
thread.add_message(ThreadMessage::action_result("", "tool_b", "result_b"));
thread.add_message(ThreadMessage::action_result(
"call_ok", "tool_c", "result_c",
));
let issues = analyze_trace(&thread);
let empty_issues: Vec<_> = issues
.iter()
.filter(|i| i.category == "empty_call_id")
.collect();
assert_eq!(
empty_issues.len(),
2,
"should flag exactly the 2 empty call_ids"
);
}
}
-239
View File
@@ -1,239 +0,0 @@
//! IronClaw Engine — unified thread-capability-CodeAct execution model.
//!
//! This crate provides the core execution engine for IronClaw, unifying
//! ~10 separate abstractions (Session, Job, Routine, Channel, Tool, Skill,
//! Hook, Observer, Extension, LoopDelegate) around 5 primitives:
//!
//! - **Thread** — unit of work (replaces Session + Job + Routine + Sub-agent)
//! - **Step** — unit of execution (replaces agentic loop iteration + tool calls)
//! - **Capability** — unit of effect (replaces Tool + Skill + Hook + Extension)
//! - **MemoryDoc** — unit of durable knowledge (replaces workspace memory blobs)
//! - **Project** — unit of context (replaces flat workspace namespace)
//!
//! The engine defines traits for external dependencies ([`LlmBackend`],
//! [`Store`], [`EffectExecutor`]) that the host crate implements via bridge
//! adapters over existing infrastructure.
pub mod capability;
pub mod executor;
pub mod memory;
pub mod reliability;
pub mod runtime;
pub mod traits;
pub mod types;
// ── Re-exports: types ───────────────────────────────────────
pub use types::capability::{
ActionDef, Capability, CapabilityLease, EffectType, LeaseId, PolicyCondition, PolicyEffect,
PolicyRule,
};
pub use types::error::{CapabilityError, EngineError, StepError, ThreadError};
pub use types::event::{EventId, EventKind, ThreadEvent};
pub use types::memory::{DocId, DocType, MemoryDoc};
pub use types::message::{MessageRole, ThreadMessage};
pub use types::mission::{Mission, MissionCadence, MissionId, MissionStatus};
pub use types::project::{Project, ProjectId};
pub use types::provenance::Provenance;
pub use types::step::{
ActionCall, ActionResult, ExecutionTier, LlmResponse, Step, StepId, StepStatus, TokenUsage,
};
pub use types::thread::{Thread, ThreadConfig, ThreadId, ThreadState, ThreadType};
// ── Re-exports: traits ──────────────────────────────────────
pub use traits::effect::{EffectExecutor, ThreadExecutionContext};
pub use traits::llm::{LlmBackend, LlmCallConfig, LlmOutput};
pub use traits::store::Store;
// ── Re-exports: capability ────────────────────────────────────
pub use capability::lease::LeaseManager;
pub use capability::planner::{CapabilityGrantPlan, LeasePlanner};
pub use capability::policy::{PolicyDecision, PolicyEngine};
pub use capability::registry::CapabilityRegistry;
// ── Re-exports: runtime ───────────────────────────────────────
pub use executor::prompt::PlatformInfo;
pub use runtime::conversation::ConversationManager;
pub use runtime::manager::ThreadManager;
pub use runtime::messaging::ThreadOutcome;
pub use runtime::mission::MissionManager;
pub use runtime::tree::ThreadTree;
pub use types::conversation::{
ConversationEntry, ConversationId, ConversationSurface, EntrySender,
};
// ── Re-exports: executor ──────────────────────────────────────
pub use executor::ExecutionLoop;
// ── Re-exports: memory ────────────────────────────────────────
pub use memory::MemoryStore;
pub use memory::RetrievalEngine;
// ── Re-exports: reliability ──────────────────────────────────
pub use reliability::ReliabilityTracker;
// ── Test utilities ──────────────────────────────────────────
#[cfg(test)]
pub(crate) mod tests {
use tokio::sync::RwLock;
use crate::traits::store::Store;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::conversation::{ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Shared in-memory Store implementation for tests.
pub struct InMemoryStore {
docs: RwLock<Vec<MemoryDoc>>,
missions: RwLock<Vec<Mission>>,
}
impl InMemoryStore {
pub fn with_docs(docs: Vec<MemoryDoc>) -> Self {
Self {
docs: RwLock::new(docs),
missions: RwLock::new(Vec::new()),
}
}
}
#[async_trait::async_trait]
impl Store for InMemoryStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn list_projects(&self) -> Result<Vec<Project>, EngineError> {
Ok(vec![])
}
async fn save_conversation(&self, _: &ConversationSurface) -> Result<(), EngineError> {
Ok(())
}
async fn load_conversation(
&self,
_: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
Ok(None)
}
async fn list_conversations(
&self,
_: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
Ok(vec![])
}
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError> {
let mut docs = self.docs.write().await;
docs.retain(|d| d.id != doc.id);
docs.push(doc.clone());
Ok(())
}
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(self.docs.read().await.iter().find(|d| d.id == id).cloned())
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(self
.docs
.read()
.await
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
missions.retain(|m| m.id != mission.id);
missions.push(mission.clone());
Ok(())
}
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError> {
Ok(self
.missions
.read()
.await
.iter()
.find(|m| m.id == id)
.cloned())
}
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError> {
Ok(self
.missions
.read()
.await
.iter()
.filter(|m| m.project_id == project_id)
.cloned()
.collect())
}
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
if let Some(m) = missions.iter_mut().find(|m| m.id == id) {
m.status = status;
}
Ok(())
}
}
}
-10
View File
@@ -1,10 +0,0 @@
//! Memory document system.
//!
//! - [`MemoryStore`] — project-scoped document CRUD
//! - [`RetrievalEngine`] — context building from project docs via keyword search
pub mod retrieval;
pub mod store;
pub use retrieval::RetrievalEngine;
pub use store::MemoryStore;
@@ -1,413 +0,0 @@
//! Context retrieval engine.
//!
//! Builds context for thread steps by retrieving relevant memory docs
//! from the project. Uses keyword matching against doc title + content,
//! with priority scoring by doc type (Lessons and Specs rank higher
//! than Summaries for context injection).
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocType, MemoryDoc};
use crate::types::project::ProjectId;
/// Retrieves relevant memory docs for a thread's context.
pub struct RetrievalEngine {
store: Arc<dyn Store>,
}
impl RetrievalEngine {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Retrieve relevant memory docs for the given query within a project.
///
/// Loads all docs for the project, scores them by keyword relevance and
/// doc-type priority, and returns the top `max_docs` results.
pub async fn retrieve_context(
&self,
project_id: ProjectId,
query: &str,
max_docs: usize,
) -> Result<Vec<MemoryDoc>, EngineError> {
if max_docs == 0 {
return Ok(Vec::new());
}
let all_docs = self.store.list_memory_docs(project_id).await?;
if all_docs.is_empty() {
return Ok(Vec::new());
}
let keywords = extract_keywords(query);
if keywords.is_empty() {
// No meaningful keywords — return by doc-type priority alone
let mut scored: Vec<(f64, MemoryDoc)> = all_docs
.into_iter()
.map(|doc| (doc_type_weight(doc.doc_type), doc))
.collect();
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
scored.truncate(max_docs);
return Ok(scored.into_iter().map(|(_, doc)| doc).collect());
}
let mut scored: Vec<(f64, MemoryDoc)> = all_docs
.into_iter()
.map(|doc| {
let keyword_score = keyword_match_score(&doc, &keywords);
let type_weight = doc_type_weight(doc.doc_type);
// Combined score: keyword relevance (0.0-1.0) + type priority bonus
let score = keyword_score + type_weight;
(score, doc)
})
.filter(|(score, _)| *score > 0.0)
.collect();
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
scored.truncate(max_docs);
Ok(scored.into_iter().map(|(_, doc)| doc).collect())
}
}
/// Extract lowercase keywords from a query, filtering out stop words.
fn extract_keywords(query: &str) -> Vec<String> {
const STOP_WORDS: &[&str] = &[
"a", "an", "the", "is", "are", "was", "were", "be", "been", "being", "have", "has", "had",
"do", "does", "did", "will", "would", "could", "should", "may", "might", "shall", "can",
"to", "of", "in", "for", "on", "with", "at", "by", "from", "as", "into", "about", "it",
"its", "this", "that", "these", "those", "i", "you", "he", "she", "we", "they", "what",
"which", "who", "how", "when", "where", "why", "and", "or", "but", "not", "no", "if",
"then", "so", "up", "out", "just",
];
query
.split(|c: char| !c.is_alphanumeric() && c != '_' && c != '-')
.map(|w| w.to_lowercase())
.filter(|w| w.len() >= 2 && !STOP_WORDS.contains(&w.as_str()))
.collect()
}
/// Score how well a doc matches the given keywords (0.0 to 1.0).
fn keyword_match_score(doc: &MemoryDoc, keywords: &[String]) -> f64 {
if keywords.is_empty() {
return 0.0;
}
let title_lower = doc.title.to_lowercase();
let content_lower = doc.content.to_lowercase();
let mut matched = 0usize;
for kw in keywords {
// Title matches are worth more
if title_lower.contains(kw.as_str()) {
matched += 2;
} else if content_lower.contains(kw.as_str()) {
matched += 1;
}
}
// Normalize: max possible score is keywords.len() * 2 (all in title)
let max_score = keywords.len() * 2;
matched as f64 / max_score as f64
}
/// Priority weight by doc type. Higher = more useful for context injection.
fn doc_type_weight(doc_type: DocType) -> f64 {
match doc_type {
DocType::Spec => 0.5, // Missing capability info is highest priority
DocType::Skill => 0.45, // Skills with activation metadata and code snippets
DocType::Lesson => 0.4, // Lessons prevent repeating mistakes
DocType::Issue => 0.2, // Known problems
DocType::Summary => 0.1, // Background context
DocType::Note => 0.05, // Scratch notes, lowest priority
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::event::ThreadEvent;
use crate::types::memory::DocId;
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Mock Store that returns a fixed set of memory docs.
struct DocStore {
docs: tokio::sync::Mutex<Vec<MemoryDoc>>,
}
impl DocStore {
fn new(docs: Vec<MemoryDoc>) -> Arc<Self> {
Arc::new(Self {
docs: tokio::sync::Mutex::new(docs),
})
}
}
#[async_trait::async_trait]
impl crate::traits::store::Store for DocStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
let docs = self.docs.lock().await;
Ok(docs
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
#[test]
fn extract_keywords_filters_stop_words() {
let kws = extract_keywords("what is the latest news about Iran war");
assert!(kws.contains(&"latest".to_string()));
assert!(kws.contains(&"news".to_string()));
assert!(kws.contains(&"iran".to_string()));
assert!(kws.contains(&"war".to_string()));
assert!(!kws.contains(&"the".to_string()));
assert!(!kws.contains(&"is".to_string()));
}
#[test]
fn extract_keywords_handles_special_chars() {
let kws = extract_keywords("web_search web-fetch tool");
assert!(kws.contains(&"web_search".to_string()));
assert!(kws.contains(&"web-fetch".to_string()));
assert!(kws.contains(&"tool".to_string()));
}
#[test]
fn keyword_match_title_beats_content() {
use crate::types::project::ProjectId;
let doc = MemoryDoc::new(
ProjectId::new(),
DocType::Lesson,
"Lesson about web_search errors",
"The tool was not found during execution.",
);
let keywords = vec!["web_search".to_string()];
let score = keyword_match_score(&doc, &keywords);
// Title match = 2/2 = 1.0
assert!((score - 1.0).abs() < f64::EPSILON);
let keywords2 = vec!["execution".to_string()];
let score2 = keyword_match_score(&doc, &keywords2);
// Content-only match = 1/2 = 0.5
assert!((score2 - 0.5).abs() < f64::EPSILON);
}
#[test]
fn doc_type_weight_ordering() {
assert!(doc_type_weight(DocType::Spec) > doc_type_weight(DocType::Lesson));
assert!(doc_type_weight(DocType::Lesson) > doc_type_weight(DocType::Issue));
assert!(doc_type_weight(DocType::Issue) > doc_type_weight(DocType::Summary));
assert!(doc_type_weight(DocType::Summary) > doc_type_weight(DocType::Note));
}
#[tokio::test]
async fn retrieve_returns_relevant_docs_by_keyword() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project,
DocType::Lesson,
"web_search tool alias",
"Use web-search not web_search",
),
MemoryDoc::new(
project,
DocType::Summary,
"weather query",
"Fetched weather data",
),
MemoryDoc::new(
project,
DocType::Issue,
"API timeout",
"External API timed out",
),
]);
let engine = RetrievalEngine::new(store);
let docs = engine
.retrieve_context(project, "web_search error", 5)
.await
.unwrap();
assert!(!docs.is_empty());
// The lesson about web_search should rank first (keyword + type weight)
assert_eq!(docs[0].doc_type, DocType::Lesson);
assert!(docs[0].title.contains("web_search"));
}
#[tokio::test]
async fn retrieve_respects_project_scoping() {
let project_a = ProjectId::new();
let project_b = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project_a,
DocType::Lesson,
"Lesson for project A",
"Some lesson",
),
MemoryDoc::new(
project_b,
DocType::Lesson,
"Lesson for project B",
"Other lesson",
),
]);
let engine = RetrievalEngine::new(store);
let docs_a = engine
.retrieve_context(project_a, "lesson", 5)
.await
.unwrap();
assert_eq!(docs_a.len(), 1);
assert!(docs_a[0].title.contains("project A"));
let docs_b = engine
.retrieve_context(project_b, "lesson", 5)
.await
.unwrap();
assert_eq!(docs_b.len(), 1);
assert!(docs_b[0].title.contains("project B"));
}
#[tokio::test]
async fn retrieve_respects_max_docs_limit() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(project, DocType::Lesson, "Lesson 1", "Content 1"),
MemoryDoc::new(project, DocType::Lesson, "Lesson 2", "Content 2"),
MemoryDoc::new(project, DocType::Lesson, "Lesson 3", "Content 3"),
]);
let engine = RetrievalEngine::new(store);
let docs = engine.retrieve_context(project, "lesson", 2).await.unwrap();
assert_eq!(docs.len(), 2);
}
#[tokio::test]
async fn retrieve_empty_store_returns_empty() {
let project = ProjectId::new();
let store = DocStore::new(vec![]);
let engine = RetrievalEngine::new(store);
let docs = engine
.retrieve_context(project, "anything", 5)
.await
.unwrap();
assert!(docs.is_empty());
}
#[tokio::test]
async fn retrieve_spec_ranks_above_summary() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project,
DocType::Summary,
"Summary of search",
"searched the web",
),
MemoryDoc::new(
project,
DocType::Spec,
"Missing search tool",
"ALIAS: web_search -> web-search",
),
]);
let engine = RetrievalEngine::new(store);
let docs = engine.retrieve_context(project, "search", 5).await.unwrap();
assert_eq!(docs.len(), 2);
// Spec should rank first due to higher type weight
assert_eq!(docs[0].doc_type, DocType::Spec);
}
}
-408
View File
@@ -1,408 +0,0 @@
//! Project-scoped memory document operations.
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocId, DocType, MemoryDoc};
use crate::types::project::ProjectId;
use crate::types::thread::ThreadId;
/// Thin wrapper over the [`Store`] trait for project-scoped doc operations.
pub struct MemoryStore {
store: Arc<dyn Store>,
}
impl MemoryStore {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Create a new memory document.
pub async fn create_doc(
&self,
project_id: ProjectId,
doc_type: DocType,
title: &str,
content: &str,
) -> Result<MemoryDoc, EngineError> {
let doc = MemoryDoc::new(project_id, doc_type, title, content);
self.store.save_memory_doc(&doc).await?;
Ok(doc)
}
/// Create a doc linked to a source thread.
pub async fn create_doc_from_thread(
&self,
project_id: ProjectId,
doc_type: DocType,
title: &str,
content: &str,
source_thread_id: ThreadId,
) -> Result<MemoryDoc, EngineError> {
let doc = MemoryDoc::new(project_id, doc_type, title, content)
.with_source_thread(source_thread_id);
self.store.save_memory_doc(&doc).await?;
Ok(doc)
}
/// Load a single doc by ID.
pub async fn get_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
self.store.load_memory_doc(id).await
}
/// List all docs in a project, optionally filtered by type.
pub async fn list_docs(
&self,
project_id: ProjectId,
doc_type: Option<DocType>,
) -> Result<Vec<MemoryDoc>, EngineError> {
let all = self.store.list_memory_docs(project_id).await?;
match doc_type {
Some(dt) => Ok(all.into_iter().filter(|d| d.doc_type == dt).collect()),
None => Ok(all),
}
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use tokio::sync::RwLock;
use crate::traits::store::Store;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, DocType, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
use super::MemoryStore;
// ── In-memory Store implementation ───────────────────────
struct InMemoryDocStore {
docs: RwLock<Vec<MemoryDoc>>,
threads: RwLock<Vec<Thread>>,
steps: RwLock<Vec<Step>>,
events: RwLock<Vec<ThreadEvent>>,
projects: RwLock<Vec<Project>>,
leases: RwLock<Vec<CapabilityLease>>,
missions: RwLock<Vec<Mission>>,
}
impl InMemoryDocStore {
fn new() -> Self {
Self {
docs: RwLock::new(Vec::new()),
threads: RwLock::new(Vec::new()),
steps: RwLock::new(Vec::new()),
events: RwLock::new(Vec::new()),
projects: RwLock::new(Vec::new()),
leases: RwLock::new(Vec::new()),
missions: RwLock::new(Vec::new()),
}
}
}
#[async_trait::async_trait]
impl Store for InMemoryDocStore {
// ── Thread operations ────────────────────────────────
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError> {
let mut threads = self.threads.write().await;
threads.retain(|t| t.id != thread.id);
threads.push(thread.clone());
Ok(())
}
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError> {
let threads = self.threads.read().await;
Ok(threads.iter().find(|t| t.id == id).cloned())
}
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError> {
let threads = self.threads.read().await;
Ok(threads
.iter()
.filter(|t| t.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
id: ThreadId,
state: ThreadState,
) -> Result<(), EngineError> {
let mut threads = self.threads.write().await;
if let Some(t) = threads.iter_mut().find(|t| t.id == id) {
t.state = state;
}
Ok(())
}
// ── Step operations ──────────────────────────────────
async fn save_step(&self, step: &Step) -> Result<(), EngineError> {
let mut steps = self.steps.write().await;
steps.retain(|s| s.id != step.id);
steps.push(step.clone());
Ok(())
}
async fn load_steps(&self, thread_id: ThreadId) -> Result<Vec<Step>, EngineError> {
let steps = self.steps.read().await;
Ok(steps
.iter()
.filter(|s| s.thread_id == thread_id)
.cloned()
.collect())
}
// ── Event operations ─────────────────────────────────
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError> {
let mut stored = self.events.write().await;
stored.extend(events.iter().cloned());
Ok(())
}
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
let events = self.events.read().await;
Ok(events
.iter()
.filter(|e| e.thread_id == thread_id)
.cloned()
.collect())
}
// ── Project operations ───────────────────────────────
async fn save_project(&self, project: &Project) -> Result<(), EngineError> {
let mut projects = self.projects.write().await;
projects.retain(|p| p.id != project.id);
projects.push(project.clone());
Ok(())
}
async fn load_project(&self, id: ProjectId) -> Result<Option<Project>, EngineError> {
let projects = self.projects.read().await;
Ok(projects.iter().find(|p| p.id == id).cloned())
}
// ── Memory doc operations ────────────────────────────
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError> {
let mut docs = self.docs.write().await;
docs.push(doc.clone());
Ok(())
}
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
let docs = self.docs.read().await;
Ok(docs.iter().find(|d| d.id == id).cloned())
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
let docs = self.docs.read().await;
Ok(docs
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
// ── Capability lease operations ──────────────────────
async fn save_lease(&self, lease: &CapabilityLease) -> Result<(), EngineError> {
let mut leases = self.leases.write().await;
leases.retain(|l| l.id != lease.id);
leases.push(lease.clone());
Ok(())
}
async fn load_active_leases(
&self,
thread_id: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
let leases = self.leases.read().await;
Ok(leases
.iter()
.filter(|l| l.thread_id == thread_id && !l.revoked)
.cloned()
.collect())
}
async fn revoke_lease(&self, lease_id: LeaseId, _reason: &str) -> Result<(), EngineError> {
let mut leases = self.leases.write().await;
if let Some(l) = leases.iter_mut().find(|l| l.id == lease_id) {
l.revoked = true;
}
Ok(())
}
// ── Mission operations ───────────────────────────────
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
missions.retain(|m| m.id != mission.id);
missions.push(mission.clone());
Ok(())
}
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError> {
let missions = self.missions.read().await;
Ok(missions.iter().find(|m| m.id == id).cloned())
}
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError> {
let missions = self.missions.read().await;
Ok(missions
.iter()
.filter(|m| m.project_id == project_id)
.cloned()
.collect())
}
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
if let Some(m) = missions.iter_mut().find(|m| m.id == id) {
m.status = status;
}
Ok(())
}
}
fn make_store() -> MemoryStore {
MemoryStore::new(Arc::new(InMemoryDocStore::new()))
}
// ── Tests ────────────────────────────────────────────────
#[tokio::test]
async fn create_doc_and_get() {
let store = make_store();
let project_id = ProjectId::new();
let doc = store
.create_doc(project_id, DocType::Summary, "Test Doc", "Some content")
.await
.unwrap();
assert_eq!(doc.title, "Test Doc");
assert_eq!(doc.content, "Some content");
assert_eq!(doc.doc_type, DocType::Summary);
assert_eq!(doc.project_id, project_id);
assert!(doc.source_thread_id.is_none());
let loaded = store.get_doc(doc.id).await.unwrap();
let loaded = loaded.unwrap();
assert_eq!(loaded.id, doc.id);
assert_eq!(loaded.title, "Test Doc");
assert_eq!(loaded.content, "Some content");
}
#[tokio::test]
async fn create_doc_from_thread_links_source() {
let store = make_store();
let project_id = ProjectId::new();
let thread_id = ThreadId::new();
let doc = store
.create_doc_from_thread(
project_id,
DocType::Lesson,
"Thread Lesson",
"Learned something",
thread_id,
)
.await
.unwrap();
assert_eq!(doc.source_thread_id, Some(thread_id));
assert_eq!(doc.doc_type, DocType::Lesson);
let loaded = store.get_doc(doc.id).await.unwrap().unwrap();
assert_eq!(loaded.source_thread_id, Some(thread_id));
}
#[tokio::test]
async fn list_docs_by_project() {
let store = make_store();
let project_a = ProjectId::new();
let project_b = ProjectId::new();
store
.create_doc(project_a, DocType::Note, "A1", "content a1")
.await
.unwrap();
store
.create_doc(project_a, DocType::Note, "A2", "content a2")
.await
.unwrap();
store
.create_doc(project_b, DocType::Note, "B1", "content b1")
.await
.unwrap();
let docs_a = store.list_docs(project_a, None).await.unwrap();
assert_eq!(docs_a.len(), 2);
assert!(docs_a.iter().all(|d| d.project_id == project_a));
let docs_b = store.list_docs(project_b, None).await.unwrap();
assert_eq!(docs_b.len(), 1);
assert_eq!(docs_b[0].title, "B1");
}
#[tokio::test]
async fn list_docs_filters_by_type() {
let store = make_store();
let project_id = ProjectId::new();
store
.create_doc(project_id, DocType::Summary, "S1", "summary content")
.await
.unwrap();
store
.create_doc(project_id, DocType::Lesson, "L1", "lesson content")
.await
.unwrap();
store
.create_doc(project_id, DocType::Summary, "S2", "another summary")
.await
.unwrap();
let summaries = store
.list_docs(project_id, Some(DocType::Summary))
.await
.unwrap();
assert_eq!(summaries.len(), 2);
assert!(summaries.iter().all(|d| d.doc_type == DocType::Summary));
let lessons = store
.list_docs(project_id, Some(DocType::Lesson))
.await
.unwrap();
assert_eq!(lessons.len(), 1);
assert_eq!(lessons[0].title, "L1");
}
#[tokio::test]
async fn get_nonexistent_returns_none() {
let store = make_store();
let result = store.get_doc(DocId::new()).await.unwrap();
assert!(result.is_none());
}
}
-194
View File
@@ -1,194 +0,0 @@
//! Tool reliability tracking with exponential moving averages.
//!
//! Tracks per-action success rate and latency using EMA (exponential moving
//! average) to smooth out noise. This data can be injected into the context
//! builder to inform the LLM about unreliable tools.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::RwLock;
/// EMA smoothing factor. Higher = more weight on recent observations.
const EMA_ALPHA: f64 = 0.3;
/// Per-action reliability metrics.
#[derive(Debug, Clone)]
pub struct ActionMetrics {
/// EMA of success rate (0.0 to 1.0).
pub success_rate: f64,
/// EMA of latency in milliseconds.
pub avg_latency_ms: f64,
/// Total number of calls recorded.
pub call_count: u64,
/// Last error message (if any).
pub last_error: Option<String>,
}
impl Default for ActionMetrics {
fn default() -> Self {
Self {
success_rate: 1.0, // assume success until proven otherwise
avg_latency_ms: 0.0,
call_count: 0,
last_error: None,
}
}
}
/// Thread-safe registry of per-action reliability metrics.
#[derive(Clone)]
pub struct ReliabilityTracker {
metrics: Arc<RwLock<HashMap<String, ActionMetrics>>>,
}
impl ReliabilityTracker {
pub fn new() -> Self {
Self {
metrics: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Record a successful action execution.
pub async fn record_success(&self, action_name: &str, latency: Duration) {
let mut metrics = self.metrics.write().await;
let entry = metrics.entry(action_name.to_string()).or_default();
entry.call_count += 1;
let latency_ms = latency.as_millis() as f64;
if entry.call_count == 1 {
// First observation — use raw values
entry.avg_latency_ms = latency_ms;
// success_rate stays at 1.0
} else {
entry.success_rate = ema(entry.success_rate, 1.0);
entry.avg_latency_ms = ema(entry.avg_latency_ms, latency_ms);
}
}
/// Record a failed action execution.
pub async fn record_failure(&self, action_name: &str, error: &str) {
let mut metrics = self.metrics.write().await;
let entry = metrics.entry(action_name.to_string()).or_default();
entry.call_count += 1;
entry.last_error = Some(error.to_string());
if entry.call_count == 1 {
entry.success_rate = 0.0;
} else {
entry.success_rate = ema(entry.success_rate, 0.0);
}
}
/// Get metrics for a specific action.
pub async fn get_metrics(&self, action_name: &str) -> Option<ActionMetrics> {
let metrics = self.metrics.read().await;
metrics.get(action_name).cloned()
}
/// Get all metrics, sorted by success rate (worst first).
pub async fn all_metrics(&self) -> Vec<(String, ActionMetrics)> {
let metrics = self.metrics.read().await;
let mut entries: Vec<(String, ActionMetrics)> = metrics
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
entries.sort_by(|a, b| {
a.1.success_rate
.partial_cmp(&b.1.success_rate)
.unwrap_or(std::cmp::Ordering::Equal)
});
entries
}
/// Get actions with reliability below a threshold.
pub async fn unreliable_actions(&self, threshold: f64) -> Vec<(String, ActionMetrics)> {
let all = self.all_metrics().await;
all.into_iter()
.filter(|(_, m)| m.success_rate < threshold)
.collect()
}
}
impl Default for ReliabilityTracker {
fn default() -> Self {
Self::new()
}
}
/// Compute exponential moving average.
fn ema(prev: f64, new: f64) -> f64 {
EMA_ALPHA * new + (1.0 - EMA_ALPHA) * prev
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ema_moves_toward_new() {
let result = ema(1.0, 0.0);
// 0.3 * 0.0 + 0.7 * 1.0 = 0.7
assert!((result - 0.7).abs() < f64::EPSILON);
}
#[test]
fn ema_converges_on_repeated() {
let mut val = 1.0;
for _ in 0..20 {
val = ema(val, 0.0);
}
// Should converge toward 0.0
assert!(val < 0.01);
}
#[tokio::test]
async fn track_success() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("tool_a", Duration::from_millis(100))
.await;
tracker
.record_success("tool_a", Duration::from_millis(200))
.await;
let m = tracker.get_metrics("tool_a").await.unwrap();
assert_eq!(m.call_count, 2);
assert!((m.success_rate - 1.0).abs() < f64::EPSILON);
assert!(m.avg_latency_ms > 100.0); // EMA of 100 and 200
}
#[tokio::test]
async fn track_failure_lowers_success_rate() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("tool_b", Duration::from_millis(50))
.await;
tracker.record_failure("tool_b", "not found").await;
let m = tracker.get_metrics("tool_b").await.unwrap();
assert_eq!(m.call_count, 2);
assert!(m.success_rate < 1.0);
assert_eq!(m.last_error, Some("not found".into()));
}
#[tokio::test]
async fn unreliable_actions_filters() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("good_tool", Duration::from_millis(10))
.await;
tracker.record_failure("bad_tool", "always fails").await;
let unreliable = tracker.unreliable_actions(0.5).await;
assert_eq!(unreliable.len(), 1);
assert_eq!(unreliable[0].0, "bad_tool");
}
#[tokio::test]
async fn unknown_action_returns_none() {
let tracker = ReliabilityTracker::new();
assert!(tracker.get_metrics("nonexistent").await.is_none());
}
}
@@ -1,816 +0,0 @@
//! Conversation manager — routes UI messages to threads.
//!
//! The ConversationManager is the bridge between channel I/O (user messages,
//! status updates) and the thread execution model. It maintains conversation
//! surfaces and decides whether to spawn new threads or inject messages into
//! existing ones.
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
use tracing::debug;
use crate::runtime::manager::ThreadManager;
use crate::runtime::messaging::ThreadOutcome;
use crate::traits::store::Store;
use crate::types::conversation::{ConversationEntry, ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::thread::{ThreadConfig, ThreadId, ThreadState, ThreadType};
enum ActiveForeground {
Running(ThreadId),
Resumable(ThreadId),
}
/// Manages conversation surfaces and routes messages to threads.
///
/// Each channel message arrives here. The manager decides whether to:
/// 1. Spawn a new foreground thread for the message
/// 2. Inject the message into an existing active thread
/// 3. Create a new conversation if none exists for this channel+user
pub struct ConversationManager {
thread_manager: Arc<ThreadManager>,
store: Arc<dyn Store>,
conversations: RwLock<HashMap<ConversationId, ConversationSurface>>,
/// Maps (channel, user_id) → conversation ID for lookup.
channel_user_index: RwLock<HashMap<(String, String), ConversationId>>,
}
impl ConversationManager {
pub fn new(thread_manager: Arc<ThreadManager>, store: Arc<dyn Store>) -> Self {
Self {
thread_manager,
store,
conversations: RwLock::new(HashMap::new()),
channel_user_index: RwLock::new(HashMap::new()),
}
}
/// Restore persisted conversations for a user into the in-memory index.
pub async fn bootstrap_user(&self, user_id: &str) -> Result<usize, EngineError> {
let conversations = self.store.list_conversations(user_id).await?;
let count = conversations.len();
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
for conversation in conversations {
index.insert(
(conversation.channel.clone(), conversation.user_id.clone()),
conversation.id,
);
convs.insert(conversation.id, conversation);
}
Ok(count)
}
/// Get or create a conversation for a channel+user pair.
pub async fn get_or_create_conversation(
&self,
channel: &str,
user_id: &str,
) -> Result<ConversationId, EngineError> {
// Check index first
let key = (channel.to_string(), user_id.to_string());
{
let index = self.channel_user_index.read().await;
if let Some(conv_id) = index.get(&key) {
return Ok(*conv_id);
}
}
// Check persisted conversations for this user/channel.
if let Some(conv) = self
.store
.list_conversations(user_id)
.await?
.into_iter()
.find(|conv| conv.channel == channel)
{
let conv_id = conv.id;
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
convs.insert(conv_id, conv);
index.insert(key, conv_id);
return Ok(conv_id);
}
// Create new conversation
let conv = ConversationSurface::new(channel, user_id);
let conv_id = conv.id;
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
convs.insert(conv_id, conv.clone());
index.insert(key, conv_id);
self.store.save_conversation(&conv).await?;
debug!(conversation_id = %conv_id, channel, user_id, "created conversation");
Ok(conv_id)
}
/// Handle an incoming user message.
///
/// If the conversation has an active foreground thread, the message is
/// injected into it. Otherwise, a new foreground thread is spawned.
///
/// Returns the thread ID that is handling the message.
pub async fn handle_user_message(
&self,
conversation_id: ConversationId,
content: &str,
project_id: ProjectId,
user_id: &str,
thread_config: ThreadConfig,
) -> Result<ThreadId, EngineError> {
let mut convs = self.conversations.write().await;
let conv = convs.get_mut(&conversation_id).ok_or(EngineError::Store {
reason: format!("conversation {conversation_id} not found"),
})?;
// Record the user entry
conv.add_entry(ConversationEntry::user(content));
// Check for an active foreground thread
let active_foreground = self.find_active_foreground(conv).await;
match active_foreground {
Some(ActiveForeground::Running(thread_id)) => {
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"injecting message into active thread"
);
self.thread_manager
.inject_message(thread_id, ThreadMessage::user(content))
.await?;
self.store.save_conversation(conv).await?;
Ok(thread_id)
}
Some(ActiveForeground::Resumable(thread_id)) => {
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"resuming suspended foreground thread"
);
self.thread_manager
.resume_thread(thread_id, user_id, Some(ThreadMessage::user(content)), None)
.await?;
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread resumed",
));
self.store.save_conversation(conv).await?;
Ok(thread_id)
}
None => {
// Build conversation history from prior entries for context continuity
let history = build_history_from_entries(&conv.entries);
// Spawn new foreground thread with conversation history
let thread_id = self
.thread_manager
.spawn_thread_with_history(
content, // use message as goal
ThreadType::Foreground,
project_id,
thread_config,
None,
user_id,
history,
)
.await?;
conv.track_thread(thread_id);
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread started",
));
self.store.save_conversation(conv).await?;
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"spawned new foreground thread"
);
Ok(thread_id)
}
}
}
/// Record a thread's outcome in its conversation.
pub async fn record_thread_outcome(
&self,
conversation_id: ConversationId,
thread_id: ThreadId,
outcome: &ThreadOutcome,
) -> Result<(), EngineError> {
let mut convs = self.conversations.write().await;
if let Some(conv) = convs.get_mut(&conversation_id) {
match outcome {
ThreadOutcome::Completed { response } => {
if let Some(text) = response {
conv.add_entry(ConversationEntry::agent(thread_id, text));
}
conv.untrack_thread(thread_id);
}
ThreadOutcome::Stopped => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread stopped",
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::MaxIterations => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread reached max iterations",
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::Failed { error } => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Thread failed: {error}"),
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::NeedApproval {
action_name,
call_id: _,
parameters: _,
} => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Approval needed for action: {action_name}"),
));
// Thread stays active — waiting for approval
}
ThreadOutcome::NeedAuthentication {
credential_name,
action_name: _,
call_id: _,
parameters: _,
} => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Authentication required for credential: {credential_name}"),
));
// Thread stays active — waiting for OAuth completion
}
}
self.store.save_conversation(conv).await?;
}
Ok(())
}
/// Clear a conversation's entries and active threads.
///
/// Stops tracking all threads and removes conversation history so the next
/// user message spawns a fresh thread with no prior context.
pub async fn clear_conversation(
&self,
conversation_id: ConversationId,
) -> Result<(), EngineError> {
let mut convs = self.conversations.write().await;
if let Some(conv) = convs.get_mut(&conversation_id) {
conv.active_threads.clear();
conv.entries.clear();
conv.updated_at = chrono::Utc::now();
self.store.save_conversation(conv).await?;
debug!(conversation_id = %conversation_id, "cleared conversation");
}
Ok(())
}
/// Get a snapshot of a conversation.
pub async fn get_conversation(
&self,
conversation_id: ConversationId,
) -> Option<ConversationSurface> {
let convs = self.conversations.read().await;
convs.get(&conversation_id).cloned()
}
/// List all conversations for a user.
pub async fn list_conversations(&self, user_id: &str) -> Vec<ConversationSurface> {
let convs = self.conversations.read().await;
convs
.values()
.filter(|c| c.user_id == user_id)
.cloned()
.collect()
}
/// Find an active foreground thread in a conversation.
async fn find_active_foreground(&self, conv: &ConversationSurface) -> Option<ActiveForeground> {
for &tid in &conv.active_threads {
if self.thread_manager.is_running(tid).await {
return Some(ActiveForeground::Running(tid));
}
if let Ok(Some(thread)) = self.store.load_thread(tid).await
&& thread.thread_type == ThreadType::Foreground
&& thread.state == ThreadState::Suspended
{
return Some(ActiveForeground::Resumable(tid));
}
}
None
}
}
/// Build ThreadMessage history from conversation entries.
///
/// Converts user and agent entries into ThreadMessages so a new thread
/// inherits context from prior turns in the same conversation.
fn build_history_from_entries(
entries: &[ConversationEntry],
) -> Vec<crate::types::message::ThreadMessage> {
use crate::types::conversation::EntrySender;
// Skip the last entry (it's the current user message, added by the caller
// before this function runs). Also skip system entries (thread lifecycle
// notifications aren't useful as LLM context).
let history_entries = if entries.len() > 1 {
&entries[..entries.len() - 1]
} else {
return Vec::new();
};
history_entries
.iter()
.filter_map(|entry| match &entry.sender {
EntrySender::User => Some(crate::types::message::ThreadMessage::user(&entry.content)),
EntrySender::Agent { .. } => Some(crate::types::message::ThreadMessage::assistant(
&entry.content,
)),
EntrySender::System => None, // skip system notifications
})
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::capability::lease::LeaseManager;
use crate::capability::policy::PolicyEngine;
use crate::capability::registry::CapabilityRegistry;
use crate::traits::effect::EffectExecutor;
use crate::traits::llm::{LlmBackend, LlmCallConfig, LlmOutput};
use crate::traits::store::Store;
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::conversation::{ConversationId, ConversationSurface, EntrySender};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::project::Project;
use crate::types::step::{ActionResult, LlmResponse, Step, TokenUsage};
use crate::types::thread::ThreadState;
use std::sync::Mutex;
use std::time::Duration;
// ── Mocks (same as manager tests) ───────────────────────
struct MockLlm(Mutex<Vec<LlmOutput>>);
#[async_trait::async_trait]
impl LlmBackend for MockLlm {
async fn complete(
&self,
_: &[ThreadMessage],
_: &[ActionDef],
_: &LlmCallConfig,
) -> Result<LlmOutput, EngineError> {
let mut r = self.0.lock().unwrap();
if r.is_empty() {
Ok(LlmOutput {
response: LlmResponse::Text("done".into()),
usage: TokenUsage::default(),
})
} else {
Ok(r.remove(0))
}
}
fn model_name(&self) -> &str {
"mock"
}
}
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct MockStore {
conversations: RwLock<HashMap<ConversationId, ConversationSurface>>,
threads: RwLock<HashMap<ThreadId, crate::types::thread::Thread>>,
}
impl MockStore {
fn new() -> Self {
Self {
conversations: RwLock::new(HashMap::new()),
threads: RwLock::new(HashMap::new()),
}
}
}
#[async_trait::async_trait]
impl Store for MockStore {
async fn save_thread(
&self,
thread: &crate::types::thread::Thread,
) -> Result<(), EngineError> {
self.threads.write().await.insert(thread.id, thread.clone());
Ok(())
}
async fn load_thread(
&self,
id: ThreadId,
) -> Result<Option<crate::types::thread::Thread>, EngineError> {
Ok(self.threads.read().await.get(&id).cloned())
}
async fn list_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<crate::types::thread::Thread>, EngineError> {
Ok(self
.threads
.read()
.await
.values()
.filter(|thread| thread.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_conversation(
&self,
conversation: &ConversationSurface,
) -> Result<(), EngineError> {
self.conversations
.write()
.await
.insert(conversation.id, conversation.clone());
Ok(())
}
async fn load_conversation(
&self,
id: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
Ok(self.conversations.read().await.get(&id).cloned())
}
async fn list_conversations(
&self,
user_id: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
Ok(self
.conversations
.read()
.await
.values()
.filter(|conversation| conversation.user_id == user_id)
.cloned()
.collect())
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, _: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(vec![])
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(
&self,
_: crate::types::capability::LeaseId,
_: &str,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
fn make_conv_manager() -> (Arc<ThreadManager>, ConversationManager) {
let store = Arc::new(MockStore::new());
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![LlmOutput {
response: LlmResponse::Text("Hello!".into()),
usage: TokenUsage::default(),
}]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(Arc::clone(&tm), store);
(tm, cm)
}
// ── Tests ───────────────────────────────────────────────
#[tokio::test]
async fn get_or_create_conversation() {
let (_, cm) = make_conv_manager();
let c1 = cm
.get_or_create_conversation("telegram", "user1")
.await
.unwrap();
let c2 = cm
.get_or_create_conversation("telegram", "user1")
.await
.unwrap();
assert_eq!(c1, c2); // same channel+user returns same conversation
let c3 = cm
.get_or_create_conversation("slack", "user1")
.await
.unwrap();
assert_ne!(c1, c3); // different channel → different conversation
}
#[tokio::test]
async fn handle_message_spawns_thread() {
let (tm, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
let tid = cm
.handle_user_message(conv_id, "Hello", project, "user1", ThreadConfig::default())
.await
.unwrap();
// Thread was spawned
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.active_threads.contains(&tid));
assert_eq!(conv.entries.len(), 2); // user message + "Thread started"
// Wait for thread to complete
let outcome = tm.join_thread(tid).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
#[tokio::test]
async fn handle_message_resumes_suspended_thread() {
let store = Arc::new(MockStore::new());
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![LlmOutput {
response: LlmResponse::Text("Recovered".into()),
usage: TokenUsage::default(),
}]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(Arc::clone(&tm), store.clone());
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
let mut thread = crate::types::thread::Thread::new(
"resume",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
thread.transition_to(ThreadState::Running, None).unwrap();
thread.add_message(ThreadMessage::user("earlier"));
thread.step_count = 1;
thread.metadata = serde_json::json!({
"runtime_checkpoint": {
"persisted_state": {"last_return": 7},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
thread
.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)
.unwrap();
store.save_thread(&thread).await.unwrap();
{
let mut convs = cm.conversations.write().await;
let conv = convs.get_mut(&conv_id).unwrap();
conv.track_thread(thread.id);
}
let resumed = cm
.handle_user_message(
conv_id,
"continue from there",
project,
"user1",
ThreadConfig::default(),
)
.await
.unwrap();
assert_eq!(resumed, thread.id);
let outcome = tm.join_thread(thread.id).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
#[tokio::test]
async fn record_outcome_adds_entry() {
let (_, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("cli", "user1").await.unwrap();
let tid = ThreadId::new();
// Manually track a thread
{
let mut convs = cm.conversations.write().await;
let conv = convs.get_mut(&conv_id).unwrap();
conv.track_thread(tid);
}
// Record completion
cm.record_thread_outcome(
conv_id,
tid,
&ThreadOutcome::Completed {
response: Some("Done!".into()),
},
)
.await
.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.active_threads.is_empty());
assert_eq!(conv.entries.len(), 1);
assert_eq!(conv.entries[0].content, "Done!");
// Check sender is agent
assert!(matches!(
conv.entries[0].sender,
EntrySender::Agent { thread_id } if thread_id == tid
));
}
#[tokio::test]
async fn list_conversations_filters_by_user() {
let (_, cm) = make_conv_manager();
cm.get_or_create_conversation("web", "alice").await.unwrap();
cm.get_or_create_conversation("telegram", "alice")
.await
.unwrap();
cm.get_or_create_conversation("web", "bob").await.unwrap();
let alice_convs = cm.list_conversations("alice").await;
assert_eq!(alice_convs.len(), 2);
let bob_convs = cm.list_conversations("bob").await;
assert_eq!(bob_convs.len(), 1);
}
#[tokio::test]
async fn bootstrap_user_loads_persisted_conversations() {
let store = Arc::new(MockStore::new());
let mut conv = ConversationSurface::new("web", "user1");
conv.add_entry(ConversationEntry::user("persisted"));
store.save_conversation(&conv).await.unwrap();
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(tm, store);
let loaded = cm.bootstrap_user("user1").await.unwrap();
assert_eq!(loaded, 1);
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
assert_eq!(conv_id, conv.id);
let saved = cm.get_conversation(conv.id).await.unwrap();
assert_eq!(saved.entries.len(), 1);
assert_eq!(saved.entries[0].content, "persisted");
}
#[tokio::test]
async fn clear_conversation_resets_entries_and_threads() {
let (tm, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
// Spawn a thread so the conversation has entries and active threads
let tid = cm
.handle_user_message(conv_id, "Hello", project, "user1", ThreadConfig::default())
.await
.unwrap();
// Wait for thread to finish
let _ = tm.join_thread(tid).await.unwrap();
// Record outcome so there's an agent entry
cm.record_thread_outcome(
conv_id,
tid,
&ThreadOutcome::Completed {
response: Some("Hi there".into()),
},
)
.await
.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(!conv.entries.is_empty());
// Clear the conversation
cm.clear_conversation(conv_id).await.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.entries.is_empty());
assert!(conv.active_threads.is_empty());
}
}
@@ -1,995 +0,0 @@
//! Thread manager — top-level orchestrator for thread lifecycle.
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
use tracing::{debug, error};
use crate::capability::lease::LeaseManager;
use crate::capability::planner::LeasePlanner;
use crate::capability::policy::PolicyEngine;
use crate::capability::registry::CapabilityRegistry;
use crate::executor::ExecutionLoop;
use crate::runtime::messaging::{self, SignalSender, ThreadOutcome, ThreadSignal};
use crate::runtime::tree::ThreadTree;
use crate::traits::effect::EffectExecutor;
use crate::traits::llm::LlmBackend;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::thread::{Thread, ThreadConfig, ThreadId, ThreadState, ThreadType};
/// Handle to a running thread for checking results.
struct RunningThread {
signal_tx: SignalSender,
handle: tokio::task::JoinHandle<Result<ThreadOutcome, EngineError>>,
}
/// Top-level orchestrator for thread lifecycle.
///
/// Manages thread spawning, supervision, signaling, and tree relationships.
pub struct ThreadManager {
llm: Arc<dyn LlmBackend>,
effects: Arc<dyn EffectExecutor>,
store: Arc<dyn Store>,
pub capabilities: Arc<CapabilityRegistry>,
pub leases: Arc<LeaseManager>,
pub policy: Arc<PolicyEngine>,
lease_planner: LeasePlanner,
tree: RwLock<ThreadTree>,
running: Arc<RwLock<HashMap<ThreadId, RunningThread>>>,
completed: Arc<RwLock<HashMap<ThreadId, ThreadOutcome>>>,
/// Broadcast channel for thread events (for live status updates).
event_tx: tokio::sync::broadcast::Sender<crate::types::event::ThreadEvent>,
}
impl ThreadManager {
pub fn new(
llm: Arc<dyn LlmBackend>,
effects: Arc<dyn EffectExecutor>,
store: Arc<dyn Store>,
capabilities: Arc<CapabilityRegistry>,
leases: Arc<LeaseManager>,
policy: Arc<PolicyEngine>,
) -> Self {
let (event_tx, _) = tokio::sync::broadcast::channel(256);
Self {
llm,
effects,
store,
capabilities,
leases,
policy,
lease_planner: LeasePlanner::new(),
tree: RwLock::new(ThreadTree::new()),
running: Arc::new(RwLock::new(HashMap::new())),
completed: Arc::new(RwLock::new(HashMap::new())),
event_tx,
}
}
/// Subscribe to thread events for live status updates.
pub fn subscribe_events(
&self,
) -> tokio::sync::broadcast::Receiver<crate::types::event::ThreadEvent> {
self.event_tx.subscribe()
}
/// Spawn a new thread and start executing it.
///
/// Grants default capability leases for all registered capabilities.
/// Returns the thread ID immediately; the thread runs in a background task.
///
/// `initial_messages` provides conversation history from prior threads
/// (for context continuity across turns in the same conversation).
pub async fn spawn_thread(
&self,
goal: impl Into<String>,
thread_type: ThreadType,
project_id: ProjectId,
config: ThreadConfig,
parent_id: Option<ThreadId>,
user_id: impl Into<String>,
) -> Result<ThreadId, EngineError> {
self.spawn_thread_with_history(
goal,
thread_type,
project_id,
config,
parent_id,
user_id,
Vec::new(),
)
.await
}
/// Spawn a thread with initial conversation history.
#[allow(clippy::too_many_arguments)]
pub async fn spawn_thread_with_history(
&self,
goal: impl Into<String>,
thread_type: ThreadType,
project_id: ProjectId,
config: ThreadConfig,
parent_id: Option<ThreadId>,
user_id: impl Into<String>,
initial_messages: Vec<crate::types::message::ThreadMessage>,
) -> Result<ThreadId, EngineError> {
let mut thread = Thread::new(goal, thread_type, project_id, config);
if let Some(pid) = parent_id {
thread = thread.with_parent(pid);
}
let thread_id = thread.id;
let user_id = user_id.into();
if let Some(metadata) = thread.metadata.as_object_mut() {
metadata.insert("user_id".into(), serde_json::Value::String(user_id.clone()));
}
// Register in tree
if let Some(pid) = parent_id {
self.tree.write().await.add_child(pid, thread_id);
}
// Grant explicit capability leases based on thread type.
for grant in self
.lease_planner
.plan_for_thread(thread_type, &self.capabilities)
{
let lease = self
.leases
.grant(
thread_id,
grant.capability_name,
grant.granted_actions,
None,
None,
)
.await;
self.store.save_lease(&lease).await?;
thread.capability_leases.push(lease.id);
}
// Add conversation history from prior threads (for context continuity)
for msg in initial_messages {
thread.messages.push(msg);
}
// Add the goal as the current user message so the LLM has context
thread.add_message(crate::types::message::ThreadMessage::user(&thread.goal));
// Persist
self.store.save_thread(&thread).await?;
self.start_thread(thread, user_id, false).await
}
/// Resume a persisted waiting or suspended thread.
pub async fn resume_thread(
&self,
thread_id: ThreadId,
user_id: impl Into<String>,
injected_message: Option<ThreadMessage>,
approval_event: Option<(String, bool)>,
) -> Result<(), EngineError> {
if self.is_running(thread_id).await {
return Err(EngineError::Thread(
crate::types::error::ThreadError::AlreadyRunning(thread_id),
));
}
let mut thread = self
.store
.load_thread(thread_id)
.await?
.ok_or(EngineError::ThreadNotFound(thread_id))?;
if !matches!(
thread.state,
crate::types::thread::ThreadState::Waiting
| crate::types::thread::ThreadState::Suspended
) {
return Err(EngineError::Store {
reason: format!(
"thread {thread_id} is not resumable from {:?}",
thread.state
),
});
}
if let Some((call_id, approved)) = approval_event {
let event = crate::types::event::ThreadEvent::new(
thread_id,
crate::types::event::EventKind::ApprovalReceived { call_id, approved },
);
let _ = self.event_tx.send(event.clone());
thread.events.push(event);
thread.updated_at = chrono::Utc::now();
}
if let Some(message) = injected_message {
thread.add_message(message);
}
self.store.save_thread(&thread).await?;
self.start_thread(thread, user_id.into(), true).await?;
Ok(())
}
async fn start_thread(
&self,
thread: Thread,
user_id: String,
is_resume: bool,
) -> Result<ThreadId, EngineError> {
let thread_id = thread.id;
// Create signal channel
let (tx, rx) = messaging::signal_channel(32);
// Build execution loop
let llm = Arc::clone(&self.llm);
let effects = Arc::clone(&self.effects);
let leases = Arc::clone(&self.leases);
let policy = Arc::clone(&self.policy);
let store_for_retrieval = Arc::clone(&self.store);
let retrieval = crate::memory::RetrievalEngine::new(store_for_retrieval);
let exec_loop = ExecutionLoop::new(thread, llm, effects, leases, policy, rx, user_id)
.with_capabilities(Arc::clone(&self.capabilities))
.with_event_tx(self.event_tx.clone())
.with_retrieval(retrieval)
.with_store(Arc::clone(&self.store));
// Spawn background task
let store_for_task = Arc::clone(&self.store);
let running = Arc::clone(&self.running);
let completed = Arc::clone(&self.completed);
let handle = tokio::spawn(async move {
let mut exec = exec_loop;
let result = exec.run().await;
debug!(thread_id = %thread_id, "thread execution finished");
// Run retrospective trace analysis (non-LLM, always runs).
// Issues are picked up by the self-improvement mission via event listener.
let trace = crate::executor::trace::build_trace(&exec.thread);
if !trace.issues.is_empty() {
crate::executor::trace::log_trace_summary(&trace);
}
// Transition Completed → Done
if exec.thread.state == crate::types::thread::ThreadState::Completed
&& let Err(e) = exec
.thread
.transition_to(crate::types::thread::ThreadState::Done, None)
{
tracing::warn!(thread_id = %thread_id, "failed to transition to Done: {e}");
}
// Write trace file if enabled
if crate::executor::trace::is_trace_enabled() {
crate::executor::trace::log_trace_summary(&trace);
crate::executor::trace::write_trace(&trace);
}
if let Err(e) = store_for_task.append_events(&exec.thread.events).await {
tracing::warn!(
thread_id = %thread_id,
"failed to persist thread events: {e}"
);
}
// Save final thread state to store
if let Err(e) = store_for_task.save_thread(&exec.thread).await {
tracing::warn!(
thread_id = %thread_id,
"failed to save final thread state: {e}"
);
}
let outcome = match result {
Ok(outcome) => outcome,
Err(error) => ThreadOutcome::Failed {
error: error.to_string(),
},
};
completed.write().await.insert(thread_id, outcome.clone());
running.write().await.remove(&thread_id);
Ok(outcome)
});
self.running.write().await.insert(
thread_id,
RunningThread {
signal_tx: tx,
handle,
},
);
if is_resume {
debug!(thread_id = %thread_id, "resumed thread");
}
Ok(thread_id)
}
/// Send a stop signal to a running thread.
pub async fn stop_thread(&self, thread_id: ThreadId) -> Result<(), EngineError> {
let running = self.running.read().await;
if let Some(rt) = running.get(&thread_id) {
let _ = rt.signal_tx.send(ThreadSignal::Stop).await;
Ok(())
} else {
Err(EngineError::ThreadNotFound(thread_id))
}
}
/// Inject a user message into a running thread.
pub async fn inject_message(
&self,
thread_id: ThreadId,
message: ThreadMessage,
) -> Result<(), EngineError> {
let running = self.running.read().await;
if let Some(rt) = running.get(&thread_id) {
let _ = rt
.signal_tx
.send(ThreadSignal::InjectMessage(message))
.await;
Ok(())
} else {
Err(EngineError::ThreadNotFound(thread_id))
}
}
/// Check if a thread is still running.
pub async fn is_running(&self, thread_id: ThreadId) -> bool {
let running = self.running.read().await;
running
.get(&thread_id)
.is_some_and(|rt| !rt.handle.is_finished())
}
/// Wait for a thread to finish and return its outcome.
/// Removes the thread from the running set.
pub async fn join_thread(&self, thread_id: ThreadId) -> Result<ThreadOutcome, EngineError> {
if let Some(outcome) = self.completed.write().await.remove(&thread_id) {
return Ok(outcome);
}
let rt = {
let mut running = self.running.write().await;
running.remove(&thread_id)
};
match rt {
Some(rt) => match rt.handle.await {
Ok(result) => result,
Err(e) => {
error!(thread_id = %thread_id, "thread task panicked: {e}");
Ok(ThreadOutcome::Failed {
error: format!("thread task panicked: {e}"),
})
}
},
None => Err(EngineError::ThreadNotFound(thread_id)),
}
}
/// Get children of a thread.
pub async fn children_of(&self, thread_id: ThreadId) -> Vec<ThreadId> {
let tree = self.tree.read().await;
tree.children_of(thread_id).to_vec()
}
/// Get the parent of a thread.
pub async fn parent_of(&self, thread_id: ThreadId) -> Option<ThreadId> {
let tree = self.tree.read().await;
tree.parent_of(thread_id)
}
/// Clean up finished threads from the running set.
pub async fn cleanup_finished(&self) -> Vec<ThreadId> {
let mut running = self.running.write().await;
let finished: Vec<ThreadId> = running
.iter()
.filter(|(_, rt)| rt.handle.is_finished())
.map(|(id, _)| *id)
.collect();
for id in &finished {
running.remove(id);
}
finished
}
/// Automatically resume checkpointed non-foreground threads.
pub async fn resume_background_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<ThreadId>, EngineError> {
let threads = self.store.list_threads(project_id).await?;
let mut resumed = Vec::new();
for thread in threads {
if thread.state != ThreadState::Suspended {
continue;
}
if thread.thread_type != ThreadType::Research {
continue;
}
if thread.metadata.get("runtime_checkpoint").is_none() {
continue;
}
let Some(user_id) = thread
.metadata
.get("user_id")
.and_then(|value| value.as_str())
.filter(|user_id| !user_id.is_empty())
else {
continue;
};
self.resume_thread(thread.id, user_id.to_string(), None, None)
.await?;
resumed.push(thread.id);
}
Ok(resumed)
}
/// Reconcile persisted non-terminal threads after process startup.
///
/// The current engine does not support mid-thread replay/resume, so any
/// thread left in a non-terminal state is marked failed-safe.
pub async fn recover_project_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<ThreadId>, EngineError> {
const PENDING_APPROVAL_METADATA_KEY: &str = "pending_approval";
const RUNTIME_CHECKPOINT_METADATA_KEY: &str = "runtime_checkpoint";
let threads = self.store.list_threads(project_id).await?;
let mut recovered = Vec::new();
for mut thread in threads {
if thread.state.is_terminal() || thread.state == ThreadState::Completed {
continue;
}
if thread.state == ThreadState::Waiting
&& thread.metadata.get(PENDING_APPROVAL_METADATA_KEY).is_some()
{
continue;
}
if thread
.metadata
.get(RUNTIME_CHECKPOINT_METADATA_KEY)
.is_some()
&& matches!(thread.state, ThreadState::Running | ThreadState::Suspended)
{
if thread.state == ThreadState::Running {
thread.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)?;
}
self.store.append_events(&thread.events).await?;
self.store.save_thread(&thread).await?;
recovered.push(thread.id);
continue;
}
if thread
.transition_to(
ThreadState::Failed,
Some("engine restart before thread completion".into()),
)
.is_ok()
{
self.store.append_events(&thread.events).await?;
self.store.save_thread(&thread).await?;
recovered.push(thread.id);
}
}
Ok(recovered)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::traits::llm::{LlmCallConfig, LlmOutput};
use crate::types::capability::{ActionDef, Capability, CapabilityLease, EffectType};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::project::Project;
use crate::types::step::{ActionResult, LlmResponse, Step, TokenUsage};
use crate::types::thread::ThreadState;
use std::sync::Mutex;
use std::time::Duration;
// ── Mocks ───────────────────────────────────────────────
struct MockLlm {
responses: Mutex<Vec<LlmOutput>>,
}
impl MockLlm {
fn text(msg: &str) -> Arc<Self> {
Arc::new(Self {
responses: Mutex::new(vec![LlmOutput {
response: LlmResponse::Text(msg.into()),
usage: TokenUsage::default(),
}]),
})
}
}
#[async_trait::async_trait]
impl LlmBackend for MockLlm {
async fn complete(
&self,
_: &[crate::types::message::ThreadMessage],
_: &[ActionDef],
_: &LlmCallConfig,
) -> Result<LlmOutput, EngineError> {
let mut r = self.responses.lock().unwrap();
if r.is_empty() {
Ok(LlmOutput {
response: LlmResponse::Text("done".into()),
usage: TokenUsage::default(),
})
} else {
Ok(r.remove(0))
}
}
fn model_name(&self) -> &str {
"mock"
}
}
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct MockStore {
threads: RwLock<HashMap<ThreadId, Thread>>,
events: RwLock<HashMap<ThreadId, Vec<ThreadEvent>>>,
}
impl MockStore {
fn new() -> Self {
Self {
threads: RwLock::new(HashMap::new()),
events: RwLock::new(HashMap::new()),
}
}
}
#[async_trait::async_trait]
impl Store for MockStore {
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError> {
self.threads.write().await.insert(thread.id, thread.clone());
Ok(())
}
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(self.threads.read().await.get(&id).cloned())
}
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(self
.threads
.read()
.await
.values()
.filter(|thread| thread.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError> {
let mut stored = self.events.write().await;
for event in events {
stored
.entry(event.thread_id)
.or_default()
.push(event.clone());
}
Ok(())
}
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(self
.events
.read()
.await
.get(&thread_id)
.cloned()
.unwrap_or_default())
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, _: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(vec![])
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(
&self,
_: crate::types::capability::LeaseId,
_: &str,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
fn make_manager(llm: Arc<dyn LlmBackend>) -> ThreadManager {
let mut caps = CapabilityRegistry::new();
caps.register(Capability {
name: "test".into(),
description: "Test capability".into(),
actions: vec![ActionDef {
name: "test_tool".into(),
description: "Test".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
ThreadManager::new(
llm,
Arc::new(MockEffects),
Arc::new(MockStore::new()),
Arc::new(caps),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
)
}
fn make_manager_with_store(llm: Arc<dyn LlmBackend>, store: Arc<MockStore>) -> ThreadManager {
let mut caps = CapabilityRegistry::new();
caps.register(Capability {
name: "test".into(),
description: "Test capability".into(),
actions: vec![ActionDef {
name: "test_tool".into(),
description: "Test".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
ThreadManager::new(
llm,
Arc::new(MockEffects),
store,
Arc::new(caps),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
)
}
// ── Tests ───────────────────────────────────────────────
#[tokio::test]
async fn spawn_and_join() {
let mgr = make_manager(MockLlm::text("Hello!"));
let project = ProjectId::new();
let tid = mgr
.spawn_thread(
"test",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
let outcome = mgr.join_thread(tid).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { response: Some(r) } if r == "Hello!"));
}
#[tokio::test]
async fn stop_thread_works() {
// LLM that returns many action responses
let responses: Vec<LlmOutput> = (0..100)
.map(|i| LlmOutput {
response: LlmResponse::ActionCalls {
calls: vec![crate::types::step::ActionCall {
id: format!("c{i}"),
action_name: "test_tool".into(),
parameters: serde_json::json!({}),
}],
content: None,
},
usage: TokenUsage::default(),
})
.collect();
let mgr = make_manager(Arc::new(MockLlm {
responses: Mutex::new(responses),
}));
let project = ProjectId::new();
let tid = mgr
.spawn_thread(
"test",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
// Give it a moment to start, then stop
tokio::time::sleep(Duration::from_millis(10)).await;
let _ = mgr.stop_thread(tid).await;
let outcome = mgr.join_thread(tid).await.unwrap();
assert!(matches!(
outcome,
ThreadOutcome::Stopped | ThreadOutcome::Completed { .. } | ThreadOutcome::MaxIterations
));
}
#[tokio::test]
async fn parent_child_tree() {
let mgr = make_manager(MockLlm::text("parent done"));
let project = ProjectId::new();
let parent = mgr
.spawn_thread(
"parent",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
let child = mgr
.spawn_thread(
"child",
ThreadType::Research,
project,
ThreadConfig::default(),
Some(parent),
"user",
)
.await
.unwrap();
assert_eq!(mgr.parent_of(child).await, Some(parent));
assert_eq!(mgr.children_of(parent).await, vec![child]);
}
#[tokio::test]
async fn recover_project_threads_marks_non_terminal_as_failed() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut running = Thread::new(
"running",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
running.transition_to(ThreadState::Running, None).unwrap();
store.save_thread(&running).await.unwrap();
let mut completed = Thread::new(
"done",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
completed
.transition_to(ThreadState::Failed, Some("already terminal".into()))
.unwrap();
store.save_thread(&completed).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert_eq!(recovered, vec![running.id]);
let saved = store.load_thread(running.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Failed);
let events = store.load_events(running.id).await.unwrap();
assert!(!events.is_empty());
}
#[tokio::test]
async fn recover_project_threads_preserves_waiting_approval_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut waiting = Thread::new(
"awaiting approval",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
waiting.transition_to(ThreadState::Running, None).unwrap();
waiting
.transition_to(ThreadState::Waiting, Some("approval".into()))
.unwrap();
waiting.metadata = serde_json::json!({
"pending_approval": {
"request_id": "req-1",
"action_name": "shell",
"call_id": "call-1"
}
});
store.save_thread(&waiting).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert!(recovered.is_empty());
let saved = store.load_thread(waiting.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Waiting);
}
#[tokio::test]
async fn recover_project_threads_suspends_checkpointed_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut running = Thread::new(
"resume me",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
running.transition_to(ThreadState::Running, None).unwrap();
running.metadata = serde_json::json!({
"runtime_checkpoint": {
"persisted_state": {"last_return": 7},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
store.save_thread(&running).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert_eq!(recovered, vec![running.id]);
let saved = store.load_thread(running.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Suspended);
}
#[tokio::test]
async fn resume_background_threads_restarts_suspended_research_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut research = Thread::new(
"background research",
ThreadType::Research,
project,
ThreadConfig::default(),
);
research.transition_to(ThreadState::Running, None).unwrap();
research.metadata = serde_json::json!({
"user_id": "owner",
"runtime_checkpoint": {
"persisted_state": {},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
research
.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)
.unwrap();
store.save_thread(&research).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("done"), Arc::clone(&store));
let resumed = mgr.resume_background_threads(project).await.unwrap();
assert_eq!(resumed, vec![research.id]);
let outcome = mgr.join_thread(research.id).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
// Skill selection and injection tests are in tests/engine_v2_skill_codeact.rs
// (skill selection happens in the Python orchestrator, not in Rust).
}
@@ -1,61 +0,0 @@
//! Thread-to-thread messaging via channels.
use crate::types::message::ThreadMessage;
use crate::types::thread::ThreadId;
/// Signal sent to a running thread via its mailbox.
#[derive(Debug)]
pub enum ThreadSignal {
/// Stop the thread gracefully.
Stop,
/// Pause execution (can be resumed later).
Suspend,
/// Resume a suspended thread.
Resume,
/// Inject a user message into the thread's context.
InjectMessage(ThreadMessage),
/// Notification that a child thread completed.
ChildCompleted {
child_id: ThreadId,
outcome: ThreadOutcome,
},
}
/// Final outcome of a thread's execution.
#[derive(Debug, Clone)]
pub enum ThreadOutcome {
/// Completed with an optional text response.
Completed { response: Option<String> },
/// Thread was stopped by a signal.
Stopped,
/// Max iterations reached without completing.
MaxIterations,
/// Terminal failure.
Failed { error: String },
/// A capability action requires user approval before continuing.
NeedApproval {
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
/// An action needs a credential that requires user authentication (e.g. OAuth).
/// The thread pauses until the credential is available, then resumes.
NeedAuthentication {
credential_name: String,
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
}
/// A mailbox for sending signals to a running thread.
///
/// Each thread gets a `(sender, receiver)` pair. The `ThreadManager` holds
/// the sender; the `ExecutionLoop` holds the receiver.
pub type SignalSender = tokio::sync::mpsc::Sender<ThreadSignal>;
pub type SignalReceiver = tokio::sync::mpsc::Receiver<ThreadSignal>;
/// Create a new signal channel with the given buffer size.
pub fn signal_channel(buffer: usize) -> (SignalSender, SignalReceiver) {
tokio::sync::mpsc::channel(buffer)
}
File diff suppressed because it is too large Load Diff
-17
View File
@@ -1,17 +0,0 @@
//! Thread lifecycle management.
//!
//! - [`ThreadManager`] — top-level orchestrator for spawning and supervising threads
//! - [`ThreadTree`] — parent-child relationship tracking
//! - [`messaging`] — inter-thread signal channel
pub mod conversation;
pub mod manager;
pub mod messaging;
pub mod mission;
pub mod tree;
pub use conversation::ConversationManager;
pub use manager::ThreadManager;
pub use messaging::ThreadOutcome;
pub use mission::MissionManager;
pub use tree::ThreadTree;
-129
View File
@@ -1,129 +0,0 @@
//! Thread tree — parent-child relationship tracking.
use std::collections::HashMap;
use crate::types::thread::ThreadId;
/// Manages parent-child thread relationships.
///
/// Simple in-memory tree. Threads form a forest (multiple roots).
#[derive(Debug, Default)]
pub struct ThreadTree {
/// child → parent
parents: HashMap<ThreadId, ThreadId>,
/// parent → children (ordered by insertion)
children: HashMap<ThreadId, Vec<ThreadId>>,
}
impl ThreadTree {
pub fn new() -> Self {
Self::default()
}
/// Register a parent-child relationship.
pub fn add_child(&mut self, parent_id: ThreadId, child_id: ThreadId) {
self.parents.insert(child_id, parent_id);
self.children.entry(parent_id).or_default().push(child_id);
}
/// Get the parent of a thread, if any.
pub fn parent_of(&self, thread_id: ThreadId) -> Option<ThreadId> {
self.parents.get(&thread_id).copied()
}
/// Get the children of a thread.
pub fn children_of(&self, thread_id: ThreadId) -> &[ThreadId] {
self.children
.get(&thread_id)
.map(Vec::as_slice)
.unwrap_or(&[])
}
/// Walk up the tree to collect all ancestors (parent, grandparent, ...).
pub fn ancestors(&self, thread_id: ThreadId) -> Vec<ThreadId> {
let mut result = Vec::new();
let mut current = thread_id;
while let Some(parent) = self.parents.get(&current) {
result.push(*parent);
current = *parent;
}
result
}
/// Remove a thread from the tree. Does not remove its children.
pub fn remove(&mut self, thread_id: ThreadId) {
if let Some(parent) = self.parents.remove(&thread_id)
&& let Some(siblings) = self.children.get_mut(&parent)
{
siblings.retain(|id| *id != thread_id);
}
// Orphan any children (their parent_id entries become stale)
self.children.remove(&thread_id);
}
/// Check if a thread is a root (no parent).
pub fn is_root(&self, thread_id: ThreadId) -> bool {
!self.parents.contains_key(&thread_id)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn add_and_query() {
let mut tree = ThreadTree::new();
let parent = ThreadId::new();
let child1 = ThreadId::new();
let child2 = ThreadId::new();
tree.add_child(parent, child1);
tree.add_child(parent, child2);
assert_eq!(tree.parent_of(child1), Some(parent));
assert_eq!(tree.parent_of(child2), Some(parent));
assert_eq!(tree.children_of(parent).len(), 2);
assert!(tree.is_root(parent));
assert!(!tree.is_root(child1));
}
#[test]
fn ancestors_walk_up() {
let mut tree = ThreadTree::new();
let root = ThreadId::new();
let mid = ThreadId::new();
let leaf = ThreadId::new();
tree.add_child(root, mid);
tree.add_child(mid, leaf);
let ancestors = tree.ancestors(leaf);
assert_eq!(ancestors, vec![mid, root]);
}
#[test]
fn remove_detaches_from_parent() {
let mut tree = ThreadTree::new();
let parent = ThreadId::new();
let child = ThreadId::new();
tree.add_child(parent, child);
tree.remove(child);
assert_eq!(tree.parent_of(child), None);
assert!(tree.children_of(parent).is_empty());
}
#[test]
fn children_of_unknown_returns_empty() {
let tree = ThreadTree::new();
assert!(tree.children_of(ThreadId::new()).is_empty());
}
#[test]
fn ancestors_of_root_is_empty() {
let tree = ThreadTree::new();
assert!(tree.ancestors(ThreadId::new()).is_empty());
}
}
@@ -1,57 +0,0 @@
//! Effect executor trait.
//!
//! The engine delegates actual action execution to the host through this
//! trait. The main crate implements it by wrapping `ToolRegistry` and
//! `SafetyLayer` — the engine itself has no knowledge of specific tools.
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::error::EngineError;
use crate::types::project::ProjectId;
use crate::types::step::{ActionResult, StepId};
use crate::types::thread::{ThreadId, ThreadType};
/// Contextual information about the thread requesting an effect.
///
/// Passed to the executor so it can make context-dependent decisions
/// (e.g. different tool behavior in background vs foreground threads).
#[derive(Debug, Clone)]
pub struct ThreadExecutionContext {
pub thread_id: ThreadId,
pub thread_type: ThreadType,
pub project_id: ProjectId,
pub user_id: String,
pub step_id: StepId,
}
/// Abstraction over capability action execution.
///
/// The main crate implements this by wrapping its `ToolRegistry`, `SafetyLayer`,
/// and tool execution pipeline. The engine calls `execute_action` and gets back
/// a result — all safety, sanitization, and actual tool invocation happens in
/// the host.
#[async_trait::async_trait]
pub trait EffectExecutor: Send + Sync {
/// Execute a capability action.
///
/// The executor is responsible for:
/// 1. Looking up the actual tool implementation
/// 2. Validating parameters
/// 3. Applying safety checks (sanitization, leak detection)
/// 4. Executing the tool
/// 5. Returning the result
async fn execute_action(
&self,
action_name: &str,
parameters: serde_json::Value,
lease: &CapabilityLease,
context: &ThreadExecutionContext,
) -> Result<ActionResult, EngineError>;
/// List available actions given the current set of active leases.
///
/// Used to build the action definitions sent to the LLM.
async fn available_actions(
&self,
leases: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError>;
}
-56
View File
@@ -1,56 +0,0 @@
//! LLM backend trait.
//!
//! The engine's abstraction over language model providers. Deliberately
//! simpler than the main crate's `LlmProvider` — the engine only needs
//! to make completion calls. Cost tracking, caching, retry, and circuit
//! breaking are host concerns handled by the bridge adapter.
use std::collections::HashMap;
use crate::types::capability::ActionDef;
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::step::{LlmResponse, TokenUsage};
/// Configuration for a single LLM call.
#[derive(Debug, Clone, Default)]
pub struct LlmCallConfig {
/// Maximum tokens to generate.
pub max_tokens: Option<u32>,
/// Sampling temperature.
pub temperature: Option<f32>,
/// When true, the LLM should not return action calls.
pub force_text: bool,
/// Depth in the recursive call tree (0 = root, 1+ = sub-call).
/// Implementations can use this to route to cheaper models for sub-calls.
pub depth: u32,
/// Opaque metadata forwarded to the LLM provider.
pub metadata: HashMap<String, String>,
}
/// Output from a single LLM call.
#[derive(Debug, Clone)]
pub struct LlmOutput {
pub response: LlmResponse,
pub usage: TokenUsage,
}
/// Abstraction over language model providers.
///
/// The main crate implements this by wrapping its `LlmProvider` trait,
/// converting between `ThreadMessage` and `ChatMessage`.
#[async_trait::async_trait]
pub trait LlmBackend: Send + Sync {
/// Call the LLM with conversation messages and available action definitions.
///
/// Returns either a text response or a set of action calls.
async fn complete(
&self,
messages: &[ThreadMessage],
actions: &[ActionDef],
config: &LlmCallConfig,
) -> Result<LlmOutput, EngineError>;
/// The model identifier (e.g. "gpt-4", "claude-opus-4-20250514").
fn model_name(&self) -> &str;
}
-8
View File
@@ -1,8 +0,0 @@
//! External dependency traits.
//!
//! The engine defines these traits; the host (main ironclaw crate)
//! implements them via bridge adapters over existing infrastructure.
pub mod effect;
pub mod llm;
pub mod store;
@@ -1,97 +0,0 @@
//! Storage trait for engine persistence.
//!
//! Defines CRUD operations for all engine types. The main crate implements
//! this by wrapping its dual-backend `Database` trait (PostgreSQL + libSQL).
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::conversation::{ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Persistence abstraction for the engine.
#[async_trait::async_trait]
pub trait Store: Send + Sync {
// ── Thread operations ───────────────────────────────────
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError>;
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError>;
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError>;
async fn update_thread_state(
&self,
id: ThreadId,
state: ThreadState,
) -> Result<(), EngineError>;
// ── Step operations ─────────────────────────────────────
async fn save_step(&self, step: &Step) -> Result<(), EngineError>;
async fn load_steps(&self, thread_id: ThreadId) -> Result<Vec<Step>, EngineError>;
// ── Event operations ────────────────────────────────────
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError>;
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError>;
// ── Project operations ──────────────────────────────────
async fn save_project(&self, project: &Project) -> Result<(), EngineError>;
async fn load_project(&self, id: ProjectId) -> Result<Option<Project>, EngineError>;
async fn list_projects(&self) -> Result<Vec<Project>, EngineError> {
Ok(Vec::new())
}
// ── Conversation operations ─────────────────────────────
async fn save_conversation(
&self,
conversation: &ConversationSurface,
) -> Result<(), EngineError> {
let _ = conversation;
Ok(())
}
async fn load_conversation(
&self,
id: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
let _ = id;
Ok(None)
}
async fn list_conversations(
&self,
user_id: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
let _ = user_id;
Ok(Vec::new())
}
// ── Memory doc operations ───────────────────────────────
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError>;
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError>;
async fn list_memory_docs(&self, project_id: ProjectId) -> Result<Vec<MemoryDoc>, EngineError>;
// ── Capability lease operations ─────────────────────────
async fn save_lease(&self, lease: &CapabilityLease) -> Result<(), EngineError>;
async fn load_active_leases(
&self,
thread_id: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError>;
async fn revoke_lease(&self, lease_id: LeaseId, reason: &str) -> Result<(), EngineError>;
// ── Mission operations ───────────────────────────────────
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError>;
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError>;
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError>;
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError>;
}
@@ -1,254 +0,0 @@
//! Capability — the unit of effect.
//!
//! A capability bundles actions (tools), knowledge (skills), and policies
//! (hooks) into a single installable/activatable unit. Capabilities are
//! granted to threads via leases.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::thread::ThreadId;
/// Strongly-typed lease identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct LeaseId(pub Uuid);
impl LeaseId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for LeaseId {
fn default() -> Self {
Self::new()
}
}
// ── Effect types ────────────────────────────────────────────
/// Classification of side effects that an action may produce.
/// Used by the policy engine for allow/deny decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum EffectType {
/// Read from local filesystem or workspace.
ReadLocal,
/// Read from external APIs (no mutation).
ReadExternal,
/// Write to local filesystem or workspace.
WriteLocal,
/// Write to external services (create PR, send email).
WriteExternal,
/// Authenticated API call requiring credentials.
CredentialedNetwork,
/// Code execution or shell access.
Compute,
/// Financial operations (payments, transfers).
Financial,
}
// ── Action definition ───────────────────────────────────────
/// Definition of a single action within a capability.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ActionDef {
/// Action name (e.g. "create_issue", "web_fetch").
pub name: String,
/// Human-readable description.
pub description: String,
/// JSON Schema for parameters.
pub parameters_schema: serde_json::Value,
/// Effect types this action may produce.
pub effects: Vec<EffectType>,
/// Whether this action requires user approval before execution.
pub requires_approval: bool,
}
// ── Capability ──────────────────────────────────────────────
/// A capability — bundles actions, knowledge, and policies.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Capability {
/// Capability name (e.g. "github", "deployment").
pub name: String,
/// Human-readable description.
pub description: String,
/// Executable actions (replaces tools).
pub actions: Vec<ActionDef>,
/// Domain knowledge blocks (replaces skills).
pub knowledge: Vec<String>,
/// Policy rules (replaces hooks).
pub policies: Vec<PolicyRule>,
}
// ── Policy ──────────────────────────────────────────────────
/// A named policy rule within a capability.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PolicyRule {
pub name: String,
pub condition: PolicyCondition,
pub effect: PolicyEffect,
}
/// When a policy rule applies.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum PolicyCondition {
/// Always applies.
Always,
/// Applies when the action name matches the pattern.
ActionMatches { pattern: String },
/// Applies when the action has a specific effect type.
EffectTypeIs(EffectType),
}
/// What the policy engine decides.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum PolicyEffect {
Allow,
Deny,
RequireApproval,
}
// ── Capability lease ────────────────────────────────────────
/// A time/use-limited grant of capability access to a thread.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CapabilityLease {
pub id: LeaseId,
/// The thread this lease is granted to.
pub thread_id: ThreadId,
/// Which capability this lease covers.
pub capability_name: String,
/// Which actions from the capability are granted (empty = all).
pub granted_actions: Vec<String>,
/// When the lease was granted.
pub granted_at: DateTime<Utc>,
/// When the lease expires (None = no expiry).
pub expires_at: Option<DateTime<Utc>>,
/// Maximum number of action invocations (None = unlimited).
pub max_uses: Option<u32>,
/// Remaining invocations (None = unlimited).
pub uses_remaining: Option<u32>,
/// Whether the lease has been explicitly revoked.
pub revoked: bool,
}
impl CapabilityLease {
/// Check whether this lease is currently valid.
pub fn is_valid(&self) -> bool {
if self.revoked {
return false;
}
if let Some(expires_at) = self.expires_at
&& Utc::now() >= expires_at
{
return false;
}
if let Some(remaining) = self.uses_remaining
&& remaining == 0
{
return false;
}
true
}
/// Check whether a specific action is covered by this lease.
pub fn covers_action(&self, action_name: &str) -> bool {
self.granted_actions.is_empty() || self.granted_actions.iter().any(|a| a == action_name)
}
/// Consume one use of this lease. Returns false if no uses remain.
pub fn consume_use(&mut self) -> bool {
if let Some(ref mut remaining) = self.uses_remaining {
if *remaining == 0 {
return false;
}
*remaining -= 1;
}
true
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_lease() -> CapabilityLease {
CapabilityLease {
id: LeaseId::new(),
thread_id: ThreadId::new(),
capability_name: "test".into(),
granted_actions: vec![],
granted_at: Utc::now(),
expires_at: None,
max_uses: None,
uses_remaining: None,
revoked: false,
}
}
#[test]
fn valid_lease() {
let lease = make_lease();
assert!(lease.is_valid());
}
#[test]
fn revoked_lease_is_invalid() {
let mut lease = make_lease();
lease.revoked = true;
assert!(!lease.is_valid());
}
#[test]
fn expired_lease_is_invalid() {
let mut lease = make_lease();
lease.expires_at = Some(Utc::now() - chrono::Duration::seconds(10));
assert!(!lease.is_valid());
}
#[test]
fn exhausted_lease_is_invalid() {
let mut lease = make_lease();
lease.max_uses = Some(1);
lease.uses_remaining = Some(0);
assert!(!lease.is_valid());
}
#[test]
fn consume_use_decrements() {
let mut lease = make_lease();
lease.max_uses = Some(2);
lease.uses_remaining = Some(2);
assert!(lease.consume_use());
assert_eq!(lease.uses_remaining, Some(1));
assert!(lease.consume_use());
assert_eq!(lease.uses_remaining, Some(0));
assert!(!lease.consume_use());
}
#[test]
fn unlimited_consume_always_succeeds() {
let mut lease = make_lease();
for _ in 0..100 {
assert!(lease.consume_use());
}
}
#[test]
fn covers_action_empty_grants_all() {
let lease = make_lease();
assert!(lease.covers_action("anything"));
}
#[test]
fn covers_action_with_specific_grants() {
let mut lease = make_lease();
lease.granted_actions = vec!["create_issue".into(), "list_prs".into()];
assert!(lease.covers_action("create_issue"));
assert!(lease.covers_action("list_prs"));
assert!(!lease.covers_action("delete_repo"));
}
}
@@ -1,263 +0,0 @@
//! Conversation surface — the UI layer, separate from execution.
//!
//! A conversation is a stream of entries visible to the user. Threads
//! (the execution units) run independently and produce entries that
//! appear in conversations. One conversation can have multiple active
//! threads; one thread can outlive its originating conversation.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::thread::ThreadId;
/// Strongly-typed conversation identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ConversationId(pub Uuid);
impl ConversationId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for ConversationId {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Display for ConversationId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}
/// Strongly-typed entry identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct EntryId(pub Uuid);
impl EntryId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for EntryId {
fn default() -> Self {
Self::new()
}
}
/// Who sent a conversation entry.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum EntrySender {
/// The human user.
User,
/// The agent (from a specific thread).
Agent { thread_id: ThreadId },
/// System notification (thread started, completed, etc.).
System,
}
/// A single entry in a conversation — a message visible to the user.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConversationEntry {
pub id: EntryId,
pub sender: EntrySender,
pub content: String,
/// Which thread produced this entry (if any).
pub origin_thread_id: Option<ThreadId>,
pub timestamp: DateTime<Utc>,
/// Optional metadata (channel-specific formatting, attachments, etc.).
pub metadata: serde_json::Value,
}
impl ConversationEntry {
/// Create a user entry.
pub fn user(content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::User,
content: content.into(),
origin_thread_id: None,
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create an agent entry from a thread.
pub fn agent(thread_id: ThreadId, content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::Agent { thread_id },
content: content.into(),
origin_thread_id: Some(thread_id),
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create a system notification entry.
pub fn system(content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::System,
content: content.into(),
origin_thread_id: None,
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create a system notification linked to a thread.
pub fn system_for_thread(thread_id: ThreadId, content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::System,
content: content.into(),
origin_thread_id: Some(thread_id),
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
}
/// A conversation surface — the UI-facing view of a chat.
///
/// Conversations are NOT execution boundaries. They are streams of entries
/// that may come from multiple concurrent threads. A user can start a new
/// thread while another is still running, and both produce entries in the
/// same conversation.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConversationSurface {
pub id: ConversationId,
/// Which channel this conversation is on (e.g. "telegram", "web", "cli").
pub channel: String,
/// The user who owns this conversation.
pub user_id: String,
/// All entries in chronological order.
pub entries: Vec<ConversationEntry>,
/// Currently active (non-terminal) thread IDs.
pub active_threads: Vec<ThreadId>,
/// Metadata (channel-specific state, external thread IDs, etc.).
pub metadata: serde_json::Value,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
impl ConversationSurface {
pub fn new(channel: impl Into<String>, user_id: impl Into<String>) -> Self {
let now = Utc::now();
Self {
id: ConversationId::new(),
channel: channel.into(),
user_id: user_id.into(),
entries: Vec::new(),
active_threads: Vec::new(),
metadata: serde_json::Value::Null,
created_at: now,
updated_at: now,
}
}
/// Add an entry and update the timestamp.
pub fn add_entry(&mut self, entry: ConversationEntry) {
self.entries.push(entry);
self.updated_at = Utc::now();
}
/// Register a thread as active in this conversation.
pub fn track_thread(&mut self, thread_id: ThreadId) {
if !self.active_threads.contains(&thread_id) {
self.active_threads.push(thread_id);
}
}
/// Remove a thread from the active list (it completed or failed).
pub fn untrack_thread(&mut self, thread_id: ThreadId) {
self.active_threads.retain(|id| *id != thread_id);
}
/// Get the most recent entry, if any.
pub fn last_entry(&self) -> Option<&ConversationEntry> {
self.entries.last()
}
/// Get all entries from a specific thread.
pub fn entries_for_thread(&self, thread_id: ThreadId) -> Vec<&ConversationEntry> {
self.entries
.iter()
.filter(|e| e.origin_thread_id == Some(thread_id))
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn conversation_lifecycle() {
let mut conv = ConversationSurface::new("telegram", "user_123");
assert!(conv.entries.is_empty());
assert!(conv.active_threads.is_empty());
// User sends a message
conv.add_entry(ConversationEntry::user("Hello!"));
assert_eq!(conv.entries.len(), 1);
// Thread starts
let tid = ThreadId::new();
conv.track_thread(tid);
conv.add_entry(ConversationEntry::system_for_thread(tid, "Thread started"));
assert_eq!(conv.active_threads.len(), 1);
// Agent responds
conv.add_entry(ConversationEntry::agent(tid, "Hi there!"));
assert_eq!(conv.entries.len(), 3);
// Thread completes
conv.untrack_thread(tid);
conv.add_entry(ConversationEntry::system_for_thread(
tid,
"Thread completed",
));
assert!(conv.active_threads.is_empty());
assert_eq!(conv.entries.len(), 4);
}
#[test]
fn multiple_concurrent_threads() {
let mut conv = ConversationSurface::new("web", "user_456");
let t1 = ThreadId::new();
let t2 = ThreadId::new();
conv.track_thread(t1);
conv.track_thread(t2);
assert_eq!(conv.active_threads.len(), 2);
conv.add_entry(ConversationEntry::agent(t1, "Research result A"));
conv.add_entry(ConversationEntry::agent(t2, "Research result B"));
conv.add_entry(ConversationEntry::agent(t1, "More from A"));
let t1_entries = conv.entries_for_thread(t1);
assert_eq!(t1_entries.len(), 2);
let t2_entries = conv.entries_for_thread(t2);
assert_eq!(t2_entries.len(), 1);
conv.untrack_thread(t1);
assert_eq!(conv.active_threads.len(), 1);
}
#[test]
fn track_thread_is_idempotent() {
let mut conv = ConversationSurface::new("cli", "user");
let tid = ThreadId::new();
conv.track_thread(tid);
conv.track_thread(tid);
assert_eq!(conv.active_threads.len(), 1);
}
}
-124
View File
@@ -1,124 +0,0 @@
//! Engine error types.
use std::fmt;
use crate::types::capability::EffectType;
use crate::types::thread::{ThreadId, ThreadState};
/// Top-level engine error.
#[derive(Debug, thiserror::Error)]
pub enum EngineError {
#[error("thread error: {0}")]
Thread(#[from] ThreadError),
#[error("step error: {0}")]
Step(#[from] StepError),
#[error("capability error: {0}")]
Capability(#[from] CapabilityError),
#[error("store error: {reason}")]
Store { reason: String },
#[error("LLM error: {reason}")]
Llm { reason: String },
#[error("effect execution error: {reason}")]
Effect { reason: String },
#[error("invalid state transition: {from} -> {to}")]
InvalidTransition { from: ThreadState, to: ThreadState },
#[error("thread not found: {0}")]
ThreadNotFound(ThreadId),
#[error("project not found: {0}")]
ProjectNotFound(ProjectId),
#[error("lease expired for capability: {capability_name}")]
LeaseExpired { capability_name: String },
#[error("lease denied: {reason}")]
LeaseDenied { reason: String },
#[error("max iterations reached: {limit}")]
MaxIterations { limit: usize },
#[error("token limit exceeded: {used} of {limit}")]
TokenLimitExceeded { used: u64, limit: u64 },
#[error("consecutive error threshold exceeded: {count} errors (limit: {threshold})")]
ConsecutiveErrors { count: u32, threshold: u32 },
#[error("thread timeout: {elapsed:?} of {limit:?}")]
Timeout {
elapsed: std::time::Duration,
limit: std::time::Duration,
},
#[error("skill error: {reason}")]
Skill { reason: String },
#[error("authentication required for credential '{credential_name}'")]
NeedAuthentication {
credential_name: String,
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
}
use crate::types::project::ProjectId;
/// Thread-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum ThreadError {
#[error("thread already running: {0}")]
AlreadyRunning(ThreadId),
#[error("thread is in terminal state: {0}")]
Terminal(ThreadState),
#[error("cannot spawn child: parent thread {0} is not running")]
ParentNotRunning(ThreadId),
}
/// Step-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum StepError {
#[error("step timed out after {0:?}")]
Timeout(std::time::Duration),
#[error("action not permitted by capability lease: {action}")]
ActionDenied { action: String },
}
/// Capability-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum CapabilityError {
#[error("capability not found: {0}")]
NotFound(String),
#[error("effect type {effect:?} not permitted by policy")]
EffectDenied { effect: EffectType },
}
// Display impls for types used in error messages that don't already impl Display.
impl fmt::Display for ThreadId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
impl fmt::Display for ThreadState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{self:?}")
}
}
impl fmt::Display for ProjectId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
-217
View File
@@ -1,217 +0,0 @@
//! Event sourcing types.
//!
//! Every significant action within a thread is recorded as an event.
//! This enables replay, debugging, reflection, and trace-based testing.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::capability::LeaseId;
/// Generate a short human-readable summary of tool parameters for display.
///
/// For `http`: shows the URL. For `web_search`: shows the query.
/// For other tools: shows the first string argument, truncated.
/// Returns `None` for empty or unrecognizable params.
pub fn summarize_params(action_name: &str, params: &serde_json::Value) -> Option<String> {
let summary = match action_name {
"http" | "web_fetch" => params.get("url").and_then(|v| v.as_str()).map(|u| {
if u.len() > 80 {
format!("{}...", &u[..77])
} else {
u.to_string()
}
}),
"web_search" | "llm_context" => params
.get("query")
.and_then(|v| v.as_str())
.map(|q| truncate(q, 60)),
"memory_search" => params
.get("query")
.and_then(|v| v.as_str())
.map(|q| truncate(q, 60)),
"memory_write" => params
.get("target")
.and_then(|v| v.as_str())
.map(|t| t.to_string()),
"memory_read" => params
.get("path")
.and_then(|v| v.as_str())
.map(|p| p.to_string()),
"shell" => params
.get("command")
.and_then(|v| v.as_str())
.map(|c| truncate(c, 60)),
"message" => params
.get("content")
.and_then(|v| v.as_str())
.map(|c| truncate(c, 40)),
_ => {
// Generic: show first string value
if let Some(obj) = params.as_object() {
obj.values()
.find_map(|v| v.as_str())
.map(|s| truncate(s, 50))
} else {
None
}
}
};
summary.filter(|s| !s.is_empty())
}
fn truncate(s: &str, max: usize) -> String {
if s.len() <= max {
s.to_string()
} else {
// Find a safe UTF-8 boundary
let mut end = max.min(s.len());
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &s[..end])
}
}
use crate::types::step::{StepId, TokenUsage};
use crate::types::thread::{ThreadId, ThreadState};
/// Strongly-typed event identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct EventId(pub Uuid);
impl EventId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for EventId {
fn default() -> Self {
Self::new()
}
}
/// A recorded event in a thread's execution history.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ThreadEvent {
pub id: EventId,
pub thread_id: ThreadId,
pub timestamp: DateTime<Utc>,
pub kind: EventKind,
}
impl ThreadEvent {
pub fn new(thread_id: ThreadId, kind: EventKind) -> Self {
Self {
id: EventId::new(),
thread_id,
timestamp: Utc::now(),
kind,
}
}
}
/// The specific kind of event that occurred.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum EventKind {
// ── Thread lifecycle ────────────────────────────────────
StateChanged {
from: ThreadState,
to: ThreadState,
reason: Option<String>,
},
// ── Step lifecycle ──────────────────────────────────────
StepStarted {
step_id: StepId,
},
StepCompleted {
step_id: StepId,
tokens: TokenUsage,
},
StepFailed {
step_id: StepId,
error: String,
},
// ── Action execution ────────────────────────────────────
ActionExecuted {
step_id: StepId,
action_name: String,
call_id: String,
duration_ms: u64,
/// Short human-readable summary of parameters (e.g., URL for http tool).
#[serde(default, skip_serializing_if = "Option::is_none")]
params_summary: Option<String>,
},
ActionFailed {
step_id: StepId,
action_name: String,
call_id: String,
error: String,
/// Short human-readable summary of parameters.
#[serde(default, skip_serializing_if = "Option::is_none")]
params_summary: Option<String>,
},
// ── Capability leases ───────────────────────────────────
LeaseGranted {
lease_id: LeaseId,
capability_name: String,
},
LeaseRevoked {
lease_id: LeaseId,
reason: String,
},
LeaseExpired {
lease_id: LeaseId,
},
// ── Messages ────────────────────────────────────────────
MessageAdded {
role: String,
content_preview: String,
},
// ── Thread tree ─────────────────────────────────────────
ChildSpawned {
child_id: ThreadId,
goal: String,
},
ChildCompleted {
child_id: ThreadId,
},
// ── Approval flow ───────────────────────────────────────
ApprovalRequested {
action_name: String,
call_id: String,
},
ApprovalReceived {
call_id: String,
approved: bool,
},
// ── Self-improvement ──────────────────────────────────────
SelfImprovementStarted,
SelfImprovementComplete {
prompt_updated: bool,
patterns_added: usize,
},
SelfImprovementFailed {
error: String,
},
// ── Skill activation ───────────────────────────────────────
SkillActivated {
skill_names: Vec<String>,
},
// ── Orchestrator versioning ───────────────────────────────
OrchestratorRollback {
from_version: u64,
to_version: u64,
reason: String,
},
}
@@ -1,93 +0,0 @@
//! Memory documents — the unit of durable knowledge.
//!
//! Memory docs are structured knowledge produced by reflection on completed
//! threads. They are project-scoped and used for context building (retrieval,
//! not replay of raw history).
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::project::ProjectId;
use crate::types::thread::ThreadId;
/// Strongly-typed document identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct DocId(pub Uuid);
impl DocId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for DocId {
fn default() -> Self {
Self::new()
}
}
/// The kind of knowledge a memory document captures.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum DocType {
/// What a thread accomplished.
Summary,
/// Durable learning from experience.
Lesson,
/// Detected problem for follow-up.
Issue,
/// Missing capability request.
Spec,
/// Working memory / scratch notes.
Note,
/// Reusable skill with activation metadata and optional code snippets.
Skill,
}
/// A memory document — structured durable knowledge.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MemoryDoc {
pub id: DocId,
pub project_id: ProjectId,
pub doc_type: DocType,
pub title: String,
pub content: String,
pub source_thread_id: Option<ThreadId>,
pub tags: Vec<String>,
pub metadata: serde_json::Value,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
impl MemoryDoc {
pub fn new(
project_id: ProjectId,
doc_type: DocType,
title: impl Into<String>,
content: impl Into<String>,
) -> Self {
let now = Utc::now();
Self {
id: DocId::new(),
project_id,
doc_type,
title: title.into(),
content: content.into(),
source_thread_id: None,
tags: Vec::new(),
metadata: serde_json::Value::Object(serde_json::Map::new()),
created_at: now,
updated_at: now,
}
}
pub fn with_source_thread(mut self, thread_id: ThreadId) -> Self {
self.source_thread_id = Some(thread_id);
self
}
pub fn with_tags(mut self, tags: Vec<String>) -> Self {
self.tags = tags;
self
}
}
-109
View File
@@ -1,109 +0,0 @@
//! Thread messages — the engine's own message type.
//!
//! Simpler than the main crate's `ChatMessage`. Bridge adapters handle
//! conversion between `ThreadMessage` and `ChatMessage`.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use crate::types::provenance::Provenance;
use crate::types::step::ActionCall;
/// Role of a message participant.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum MessageRole {
System,
User,
Assistant,
/// Result from a capability action (replaces "Tool" role).
ActionResult,
}
/// A message in a thread's conversation history.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ThreadMessage {
pub role: MessageRole,
pub content: String,
pub provenance: Provenance,
/// For ActionResult messages: the call ID this is responding to.
pub action_call_id: Option<String>,
/// For ActionResult messages: the action name.
pub action_name: Option<String>,
/// For Assistant messages: actions the LLM wants to execute.
pub action_calls: Option<Vec<ActionCall>>,
pub timestamp: DateTime<Utc>,
}
impl ThreadMessage {
/// Create a system message.
pub fn system(content: impl Into<String>) -> Self {
Self {
role: MessageRole::System,
content: content.into(),
provenance: Provenance::System,
action_call_id: None,
action_name: None,
action_calls: None,
timestamp: Utc::now(),
}
}
/// Create a user message.
pub fn user(content: impl Into<String>) -> Self {
Self {
role: MessageRole::User,
content: content.into(),
provenance: Provenance::User,
action_call_id: None,
action_name: None,
action_calls: None,
timestamp: Utc::now(),
}
}
/// Create an assistant text message.
pub fn assistant(content: impl Into<String>) -> Self {
Self {
role: MessageRole::Assistant,
content: content.into(),
provenance: Provenance::LlmGenerated,
action_call_id: None,
action_name: None,
action_calls: None,
timestamp: Utc::now(),
}
}
/// Create an assistant message with action calls.
pub fn assistant_with_actions(content: Option<String>, calls: Vec<ActionCall>) -> Self {
Self {
role: MessageRole::Assistant,
content: content.unwrap_or_default(),
provenance: Provenance::LlmGenerated,
action_call_id: None,
action_name: None,
action_calls: Some(calls),
timestamp: Utc::now(),
}
}
/// Create an action result message.
pub fn action_result(
call_id: impl Into<String>,
action_name: impl Into<String>,
content: impl Into<String>,
) -> Self {
let name: String = action_name.into();
Self {
role: MessageRole::ActionResult,
content: content.into(),
provenance: Provenance::ToolOutput {
action_name: name.clone(),
},
action_call_id: Some(call_id.into()),
action_name: Some(name),
action_calls: None,
timestamp: Utc::now(),
}
}
}
-162
View File
@@ -1,162 +0,0 @@
//! Missions — long-running goals that spawn threads over time.
//!
//! A mission represents an ongoing objective that periodically spawns
//! threads to make progress. Missions can run on a schedule (cron),
//! in response to events, or be triggered manually.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::project::ProjectId;
use crate::types::thread::ThreadId;
/// Strongly-typed mission identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct MissionId(pub Uuid);
impl MissionId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for MissionId {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Display for MissionId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}
/// Lifecycle status of a mission.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum MissionStatus {
/// Mission is actively spawning threads on cadence.
Active,
/// Mission is paused — no new threads will be spawned.
Paused,
/// Mission has achieved its goal.
Completed,
/// Mission has been abandoned or failed irrecoverably.
Failed,
}
/// How a mission triggers new threads.
///
/// The engine defines the trigger *types*. The bridge/host implements the
/// actual trigger infrastructure (cron tickers, webhook endpoints, event
/// matchers). The engine just needs to be told "fire this mission now."
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum MissionCadence {
/// Spawn on a cron schedule (e.g., "0 */6 * * *" for every 6 hours).
Cron {
expression: String,
timezone: Option<String>,
},
/// Spawn in response to a channel message matching a pattern.
OnEvent { event_pattern: String },
/// Spawn in response to a structured system event (from tools or external).
OnSystemEvent { source: String, event_type: String },
/// Spawn when an external webhook is received at a registered path.
/// The bridge registers the webhook endpoint and routes payloads here.
Webhook {
path: String,
secret: Option<String>,
},
/// Only spawn when manually triggered (via mission_fire tool or API).
Manual,
}
/// A mission — a long-running goal that spawns threads over time.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Mission {
pub id: MissionId,
pub project_id: ProjectId,
pub name: String,
pub goal: String,
pub status: MissionStatus,
pub cadence: MissionCadence,
// ── Evolving strategy ──
/// What the next thread should focus on (updated after each thread).
pub current_focus: Option<String>,
/// What approaches have been tried and what happened.
pub approach_history: Vec<String>,
// ── Progress tracking ──
/// History of threads spawned by this mission.
pub thread_history: Vec<ThreadId>,
/// Optional criteria for declaring the mission complete.
pub success_criteria: Option<String>,
// ── Budget ──
/// Maximum threads per day (0 = unlimited).
pub max_threads_per_day: u32,
/// Threads spawned today (reset daily by the cron ticker).
pub threads_today: u32,
// ── Trigger payload ──
/// Payload from the most recent trigger (webhook body, event data, etc.).
/// Injected into the thread's context so the code can access it.
pub last_trigger_payload: Option<serde_json::Value>,
pub metadata: serde_json::Value,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
/// When the next thread should be spawned (for Cron cadence).
pub next_fire_at: Option<DateTime<Utc>>,
}
impl Mission {
pub fn new(
project_id: ProjectId,
name: impl Into<String>,
goal: impl Into<String>,
cadence: MissionCadence,
) -> Self {
let now = Utc::now();
Self {
id: MissionId::new(),
project_id,
name: name.into(),
goal: goal.into(),
status: MissionStatus::Active,
cadence,
current_focus: None,
approach_history: Vec::new(),
thread_history: Vec::new(),
success_criteria: None,
max_threads_per_day: 10,
threads_today: 0,
last_trigger_payload: None,
metadata: serde_json::Value::Object(serde_json::Map::new()),
created_at: now,
updated_at: now,
next_fire_at: None,
}
}
pub fn with_success_criteria(mut self, criteria: impl Into<String>) -> Self {
self.success_criteria = Some(criteria.into());
self
}
/// Record that a thread was spawned for this mission.
pub fn record_thread(&mut self, thread_id: ThreadId) {
self.thread_history.push(thread_id);
self.updated_at = Utc::now();
}
/// Whether the mission is in a terminal state.
pub fn is_terminal(&self) -> bool {
matches!(
self.status,
MissionStatus::Completed | MissionStatus::Failed
)
}
}
-16
View File
@@ -1,16 +0,0 @@
//! Core type definitions for the engine.
//!
//! All data structures live here. No async, no I/O — just types and
//! validation logic.
pub mod capability;
pub mod conversation;
pub mod error;
pub mod event;
pub mod memory;
pub mod message;
pub mod mission;
pub mod project;
pub mod provenance;
pub mod step;
pub mod thread;

Some files were not shown because too many files have changed in this diff Show More