Compare commits

..
101 Commits
Author SHA1 Message Date
[email protected]andClaude Opus 4.6 67d5a47333 fix(engine): transition thread to Waiting on NeedApproval
The orchestrator Python returned {"outcome": "need_approval"} without
calling __transition_to__("waiting"), leaving the thread in Running
state. When the user later approved/denied, resume_thread rejected it
with "thread is not resumable from Running".

- Add __transition_to__("waiting", "approval needed") in both code-step
  and action-call approval paths in default.py
- Add Rust safety net in loop_engine.rs: if orchestrator returns
  NeedApproval but thread isn't Waiting, force the transition

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 14:20:16 -07:00
[email protected]andClaude Opus 4.6 62ea08ac5e fix: document Monty runtime limitations in CodeAct prompt, fix new-thread read-only
- Add "Runtime environment" section to codeact_preamble.md documenting
  Monty's restrictions: no stdlib imports, single imports only, no classes/
  with/match/del/yield, available builtins and modules, workarounds
- Add MONTY.md tracking current pin, all limitations, upgrade process,
  and changelog for future Monty updates
- Fix gateway createNewThread() not resetting read-only state — new
  threads now eagerly enable chat input instead of waiting for async
  loadThreads() callback

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 12:10:01 -07:00
[email protected]andClaude Opus 4.6 5cb073de72 test(e2e): skill-based OAuth flow tests
6 E2E tests covering the full skill credential lifecycle via the
gateway API:

- test_github_skill_loaded: github skill with credential spec loaded
- test_no_github_token_initially: no stored secrets before auth
- test_http_tool_returns_auth_required: http tool signals missing cred
- test_guided_auth_flow: request → auth prompt → paste token → retry
- test_auth_required_sse_event: SSE stream includes auth/skill events
- test_different_users_isolated: per-user credential scoping

Includes mock API server (aiohttp) requiring Bearer auth with token
tracking for assertions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 00:40:57 -07:00
[email protected]andClaude Opus 4.6 5563b53e52 feat(auth): guided credential flow — prompt for token and retry
When a thread completes with authentication_required, the router
enters "auth mode" for that user:

1. Detects credential_name from the error in the thread response
2. Looks up setup_instructions from the skill's credential spec
3. Emits AuthRequired to CLI/gateway with instructions
4. Stores PendingAuth — next user message is treated as a token
5. Stores the token in SecretsStore
6. Retries the original user request automatically

CLI flow:
  › create an issue in github
    ⚿ Authentication required: github_token
      Create a PAT at https://github.com/settings/tokens
    Paste your token below (or type 'cancel'):
  › ghp_abc123...
    ✓ github_token authenticated: Credential stored. Retrying...
    ● http(https://api.github.com/repos/.../issues)
    Issue created: https://github.com/...

Gateway flow: same but AuthRequired SSE event shows the auth modal.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-28 00:32:14 -07:00
[email protected]andClaude Opus 4.6 4d643f47c7 refactor: remove glob re-exports, fix clippy warnings, clean up duplicates
- Remove `pub use ironclaw_safety::*` from src/safety/mod.rs and migrate
  all 20+ call sites to import directly from `ironclaw_safety`
- Remove `pub use ironclaw_skills::*` from src/skills/mod.rs and migrate
  all 15+ call sites to import directly from `ironclaw_skills`
- Fix 4 clippy warnings: 2 shadow imports, 2 collapsible if-let chains
- Add missing SkillActivated arm to WASM channel StatusUpdate match
- Remove duplicate AuthRequired/AuthCompleted arms in repl.rs
- Update CLAUDE.md extracted crates guidance and prompt template rule
- Fix bench imports (safety_check, safety_pipeline)

46 files changed, zero warnings, 3836 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 23:51:24 -07:00
[email protected]andClaude Opus 4.6 a12188e231 fix: handle Python None params in http tool, add params_summary to CodeAct dispatch
Two fixes from live testing:

1. http tool: treat null headers/body as empty (Python's None becomes
   JSON null via Monty). Previously headers=None errored with
   "'headers' must be an object or array of {name, value}".

2. scripting.rs: compute params_summary before dispatching actions in
   the CodeAct path (was always None). Now http calls show their URL
   in the CLI: ● http(https://api.github.com/repos/.../issues)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 23:45:17 -07:00
[email protected]andClaude Opus 4.6 6d6b7fab0e feat(ui): show tool arguments in CLI and gateway
Add params_summary to ActionExecuted/ActionFailed events so the CLI
and gateway can display what tools are doing:

  ● http(https://api.github.com/repos/nearai/ironclaw/issues)
  ● web_search(latest AI news)
  ● memory_read(HEARTBEAT.md)

The summarize_params() helper extracts the most relevant argument
per tool type (URL for http, query for search, path for memory, etc.)
and truncates to 80 chars. Sensitive params are not included.

Router forwards the summary in both StatusUpdate (CLI/REPL) and
AppEvent (web gateway SSE) display names.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 23:28:28 -07:00
[email protected]andClaude Opus 4.6 7018ddafab fix(cli): show auth prompt in REPL when credential is missing
The AuthRequired SSE event was emitted but only reached the web gateway.
The REPL never saw it because it receives events through
forward_event_to_channel which converts ThreadEvents to StatusUpdates.

Fix: when forward_event_to_channel sees an ActionFailed with
"authentication_required" in the error, emit StatusUpdate::AuthRequired
to the channel. Also add AuthRequired/AuthCompleted rendering to the
REPL (was missing — fell through to unmatched arm).

CLI now shows:
  ⚿ Authentication required: github_token
    Store the credential with: ironclaw secret set <name> <value>

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 23:11:46 -07:00
[email protected]andClaude Opus 4.6 84b182b7e2 feat(ui): show activated skills in CLI and gateway
End-to-end skill activation display:

1. Python orchestrator emits __emit_event__("skill_activated", skill_names=...)
   after select_skills() picks skills for the conversation
2. Rust host function parses the comma-separated names into EventKind::SkillActivated
3. Router forwards to channels as StatusUpdate::SkillActivated
4. REPL renders: ◈ skills: github, linear (cyan)
5. Web gateway emits AppEvent::SkillActivated SSE event for frontend display

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 23:09:41 -07:00
[email protected]andClaude Opus 4.6 2ce6e785e7 fix(engine): auto-approve http calls with registered credentials in v2
The v1 approval flow (interactive yes/no prompt) doesn't exist in v2.
When the http tool returned UnlessAutoApproved for credentialed hosts,
the effect adapter blocked with LeaseDenied — making all skill-based
API calls fail.

Fix: credential-backed http calls bypass the v1 approval check. The
user authorized by storing the credential; the v1 interactive prompt
is redundant in v2's lease-based security model.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 22:56:50 -07:00
[email protected]andClaude Opus 4.6 85bcaa64e9 feat(engine): platform self-awareness, event pipeline fix, globals() builtin, prompt templates
Session 9 changes driven by live trace analysis:

- CodeAct event pipeline: handle_execute_code_step now transfers
  CodeExecutionResult events to thread.events and broadcasts via event_tx
  (fixes false-positive no_tools_used trace warnings)
- Monty globals()/locals() builtins: returns dict of available action names
  from capability leases, enabling "tool_name" in globals() probing
- PlatformInfo injection into system prompts (version, LLM backend, model,
  database, channels, owner, repo URL)
- Mission goal prompts moved to prompts/*.md files (include_str! pattern)
- /expected command for triggering self-improvement from user feedback
- Session 9 development history

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 22:39:59 -07:00
[email protected]andClaude Opus 4.6 ae0cae3a22 feat(engine): non-blocking auth signal, NeedAuthentication flow, timeout safety
When the HTTP tool detects a missing credential for a registered host:
1. EffectBridgeAdapter emits SSE AuthRequired event (best-effort, for
   connected frontends — silently dropped for missions/background threads)
2. Error flows back to LLM as normal ActionResult (non-blocking)
3. LLM tells the user to authenticate

This avoids the blocking interruption approach which would hang mission
threads and sub-threads that have no channel context.

Engine additions:
- EngineError::NeedAuthentication variant for structured auth failures
- ThreadOutcome::NeedAuthentication for batch interruption when needed
- structured.rs handles NeedAuthentication by interrupting the batch
  (stops subsequent calls, returns outcome to orchestrator)
- Auth callback on EffectBridgeAdapter (optional, set by router for SSE)
- extract_credential_name parser for HTTP tool error messages
- routine_* tools added to is_v1_only_tool blocklist

Safety: added 5-minute timeout to await_thread_outcome to prevent
infinite hangs (e.g. after denied tool approval where thread fails
to resume).

Tests: 3 structured executor tests (NeedAuthentication interrupts batch,
stops subsequent calls, regular errors don't interrupt) + 7 effect
adapter tests (credential extraction, callback firing, v1-only tools).

Also adds Linear API skill (skills/linear/SKILL.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 22:39:59 -07:00
[email protected]andClaude Opus 4.6 6eed4b722c feat(skills): compile-time skill bundling infrastructure
Add support for embedding skills into the binary at compile time:

- build.rs: embed_skills() collects skills/*/SKILL.md into embedded_skills.json
- src/skills/bundled.rs: loads embedded skills via include_str!
- SkillRegistry: with_bundled_content(), load_from_content(), step 4 in discover_all()
- Bundled skills are Trusted (ship with binary), lowest discovery priority
- 4 new tests for bundled loading, user override, gating, and removal rejection
- Cargo.toml: add serde_json build-dependency

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 20:14:46 -07:00
[email protected]andClaude Opus 4.6 49e83dec5e chore(engine): remove unused skill_selector.rs
Rust-side skill selection was moved to the Python orchestrator in
7f87d179. This module had no production callers — only its own tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 20:04:01 -07:00
[email protected]andClaude Opus 4.6 a7e31a0e60 docs: annotate v1-only code for removal after migration
Mark modules and functions that exist solely for the v1 agent with
"remove after v1 migration" notes:

- src/skills/mod.rs ��� shim, attenuation, credential registration
- src/skills/attenuation.rs — trust-based tool filtering (v1 only)
- ironclaw_skills: selector, gating, registry, catalog modules
- ironclaw_engine: skill_selector.rs (superseded by Python orchestrator)
- src/bridge/skill_migration.rs — one-time v1→v2 conversion

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 20:02:34 -07:00
[email protected]andClaude Opus 4.6 7f87d1799e refactor(engine): move skill selection and injection to Python orchestrator
Skill selection was in Rust (SkillSelector in loop_engine.rs) — now it's
in the Python orchestrator where the self-improvement mission can evolve it.

Rust provides data access via two new host functions:
- __list_skills__() — loads DocType::Skill MemoryDocs from Store
- __record_skill_usage__(doc_id, success) — confidence tracking

Python orchestrator handles everything else:
- score_skill() — keyword/tag/confidence scoring (~40 lines)
- select_skills() — budget-aware top-N selection (~15 lines)
- format_skills() — XML block injection into system prompt (~20 lines)
- Injection at step 0 with active_skill_ids stored in state

Removed from Rust:
- SkillSelector field + builder on ExecutionLoop and ThreadManager
- format_skills_section() from prompt.rs
- Rust-side skill injection block in loop_engine.rs
- SkillSelector wiring in bridge/router.rs

E2E test updated: skills stored in TestStore, Python orchestrator
finds them via __list_skills__() and injects based on goal keywords.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 19:58:27 -07:00
[email protected]andClaude Opus 4.6 ecbe3f5352 chore(engine): remove legacy Playbook doc type, superseded by Skill
Drop DocType::Playbook variant and all references — playbook extraction
mission was already renamed to skill extraction in the previous session.
Updates CLAUDE.md, architecture docs, context builder, retrieval weights,
mission comments, and store adapter path mapping.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 19:46:05 -07:00
[email protected]andClaude Opus 4.6 96266cb46d feat(skills): credential specs in skill frontmatter, HTTP tool hardening, mission leases
Skills can now declare API credentials in YAML frontmatter (SkillCredentialSpec,
SkillCredentialLocation, SkillOAuthConfig, ProviderRefreshStrategy). Valid specs
are registered into SharedCredentialRegistry at startup; the HttpTool auto-injects
credentials for matching hosts — same zero-exposure model as WASM tools.

HTTP tool security hardening:
- Block LLM-provided auth headers for hosts with registered credentials
- Return structured authentication_required error for missing credentials
- Strip sensitive response headers (Set-Cookie, WWW-Authenticate, Authorization)
- Scan response body through LeakDetector before returning to LLM

Mission capability leases: registered mission_create/list/fire/pause/resume/delete
as a "missions" capability so threads receive leases. Removed routine_* aliases
from effect adapter — descriptions mention "routine" for LLM intent mapping.

Includes 10 integration tests (tests/skill_credential_injection.rs) covering
the full pipeline: YAML parsing → validation → registry → HttpTool wiring →
per-user isolation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 19:33:58 -07:00
[email protected]andClaude Opus 4.6 429f6da7e0 chore: clean up legacy playbook references in engine crate
- Rename PLAYBOOK_MIN_STEPS/ACTIONS → SKILL_EXTRACTION_MIN_STEPS/ACTIONS
- Fix pattern DB uses DocType::Note instead of DocType::Playbook
- Update CLAUDE.md: skill-extraction mission, DocType list, module map

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 18:28:19 -07:00
[email protected]andClaude Opus 4.6 baa86a4a3a docs: update engine-v2-architecture for missions and skills
- Replace "Reflection Pipeline" with "Learning Missions" (self-improvement,
  skill-extraction, conversation-insights)
- Add "Skills System" section covering ironclaw_skills crate, deterministic
  selection pipeline, CodeAct integration, confidence tracking, v1 migration
- Update MemoryDoc types table (add Skill, remove Playbook as primary)
- Update Integration Scaling section: Skills replace Capabilities-as-knowledge
  as the concrete implementation
- Update example from Capability YAML to SKILL.md format with credentials
- Fix thread state machine (remove Reflecting state)
- Update key files table and test counts
- Add self-improvement feedback loop diagram

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 18:17:04 -07:00
[email protected] 4b3997cf44 Documenting research around how to extend to more integrations 2026-03-27 16:25:40 -07:00
[email protected]andClaude Opus 4.6 8e2349d12e feat(skills): extract ironclaw_skills crate and integrate with v2 engine
Extract the skills system into a standalone `ironclaw_skills` crate
(following the ironclaw_safety pattern) and wire it into the v2 engine
for deterministic skill selection, CodeAct code injection, and
confidence tracking.

**ironclaw_skills crate** (94 tests):
- Core types: SkillManifest, ActivationCriteria, LoadedSkill, SkillTrust
- V2 types: V2SkillMetadata, CodeSnippet, SkillMetrics, V2SkillSource
- Deterministic 4-phase selector (gating→scoring→budget→attenuation)
- apply_confidence_factor() for extracted skill scoring
- SKILL.md parser, validation/escaping, gating, registry, catalog
- Feature-gated: catalog (reqwest), registry (filesystem)

**Engine integration** (14 new tests):
- DocType::Skill with retrieval weight 0.45
- SkillSelector bridges MemoryDoc→LoadedSkill for shared scoring
- SkillTracker for usage/version/rollback confidence tracking
- System prompt injection via <skill> XML blocks
- CodeAct snippet injection via Monty NameLookup
- Skill extraction mission replaces playbook extraction
- ThreadManager.set_skill_selector() for runtime wiring

**Bridge + migration**:
- skill_migration.rs: v1 SKILL.md → v2 MemoryDoc (idempotent)
- init_engine() migrates v1 skills, builds SkillSelector
- src/skills/mod.rs → re-export shim

**E2E test** (tests/engine_v2_skill_codeact.rs):
- Full CodeAct loop: skill selected → LLM returns Python code →
  Monty executes http() → mock returns canned GitHub JSON →
  FINAL() terminates → thread completes with canned data
- GitHub SKILL.md in skills/github/ as reference implementation

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 16:24:07 -07:00
[email protected]andClaude Opus 4.6 a4f5c56d06 feat(engine): consolidate action execution, remove reflection, add learning missions
Three major changes to the v2 engine:

1. **Consolidated action execution** — `handle_execute_action` in Rust is now
   the single source of truth for lease lookup, policy check, lease consumption,
   action execution, event emission, and ActionResult message recording. The
   Python orchestrator no longer duplicates event/message logic. This fixes the
   empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant
   messages (Codex "No tool call found" error).

2. **Removed reflection system** — Deleted the per-thread reflection pipeline
   (pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection,
   enable_reflection config, and all 3 reflection event kinds. Learning is now
   handled entirely by event-driven missions that fire selectively.

3. **Three learning missions** replace reflection:
   - `self-improvement` — fires on trace issues (error diagnosis, prompt fixes)
   - `playbook-extraction` — fires on successful 5+ step threads (reusable procedures)
   - `conversation-insights` — fires every 5 threads per project (user preferences,
     domain knowledge, workflow patterns)

Additional fixes:
- llm_query()/llm_query_batched() always include system message (Codex compat)
- handle_llm_complete adds assistant message with structured action_calls for
  Tier 0 responses (prevents "No tool call found" errors)
- Gateway broadcasts without thread_id emit as Status events instead of being dropped
- Comprehensive tests for call_id propagation and trace analysis (17 new tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-27 08:57:39 -07:00
[email protected]andClaude Opus 4.6 212fe9817a fix(engine): resolve tool names as callable stubs in CodeAct runtime
When LLM-generated code calls `mission_list()` or any tool function,
Monty's Python execution model first resolves the name (`mission_list`)
as a NameLookup before invoking it as a FunctionCall. The NameLookup
handler always returned Undefined, causing NameError before the function
call could dispatch to the effect executor.

Fix: before starting the Monty VM, collect all known tool names from
the effect executor's available_actions(). In the NameLookup handler,
if the name matches a known tool, return a MontyObject::Function stub
instead of Undefined. Monty then yields FunctionCall for the stub,
which dispatches to the normal tool execution pipeline.

This enables CodeAct code to call any registered tool as a Python
function: mission_list(), mission_create(), routine_list(), web_search(),
memory_search(), etc. — all without explicit imports or __execute_action__
boilerplate.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 23:54:33 -07:00
[email protected]andClaude Opus 4.6 c48605106f feat(web): richer activity indicators for engine v2 execution
The gateway UI showed only generic "Thinking..." during engine v2
execution with no visibility into CodeAct code execution, tool calls,
or reflection. Now the event mapping produces detailed status updates:

Step lifecycle:
- "Calling LLM..." when a step starts (was "Thinking...")
- "Step complete — N in / M out tokens" when done (was "Processing...")

Tool execution:
- Emit ToolStarted + ToolCompleted SSE events so the frontend renders
  proper tool cards with spinner → checkmark/error transitions
- Duration shown in parameters field (e.g., "42ms")

CodeAct visibility:
- "Executing code..." when assistant produces a code block
- "Code executed" / "Code executed (no output)" for successful runs
- "Code error — retrying..." when Monty raises an exception

Reflection:
- "Reflecting on execution..." when post-thread analysis starts
- "Reflection complete — N insight(s) saved" when done

Also refactored thread_event_to_app_event → thread_event_to_app_events
(returns Vec<AppEvent>) to support emitting ToolStarted before
ToolCompleted in a single event handler pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 23:44:55 -07:00
[email protected]andClaude Opus 4.6 fad4272754 fix(engine): route messages to correct conversation by thread scope
Messages sent from a new conversation in the gateway always appeared in
the default assistant conversation because handle_with_engine ignored
the thread_id from the frontend.

Two fixes:

1. Engine conversation scoping — when the message carries a thread_id
   (from the frontend's conversation picker), use it as part of the
   engine conversation key: "gateway:<thread_id>" instead of just
   "gateway". This creates a distinct engine conversation per v1
   thread, so messages don't cross-contaminate.

2. V1 dual-write targeting — write user messages and assistant
   responses to the v1 conversation matching the thread_id (via
   ensure_conversation), not the hardcoded assistant conversation.
   Falls back to the assistant conversation when no thread_id is
   present (e.g., default chat).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 22:46:36 -07:00
[email protected]andClaude Opus 4.6 fa47dc30ab fix(web): support multiple gateway tabs by reducing SSE connections
Each browser tab opened 2 SSE connections (chat events + log events).
With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the
pool and couldn't load any data.

Three changes:

1. Lazy log SSE — only connect when the logs tab is active, disconnect
   when switching away. Most users rarely view logs, so this saves a
   connection slot per tab.

2. Visibility API — close SSE when the browser tab goes to background
   (user switches to another tab), reconnect when it becomes visible.
   Background tabs don't need real-time events.

3. Combined with the existing beforeunload cleanup, this means:
   - Active foreground tab: 1 connection (chat SSE only, +1 if logs tab)
   - Background tabs: 0 connections
   - Closed/refreshed tabs: 0 connections (beforeunload cleanup)

This allows many gateway tabs to coexist within the 6-connection limit.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 19:46:00 -07:00
[email protected]andClaude Opus 4.6 e508d2083b fix(web): close SSE connections on page unload to prevent connection starvation
The browser limits concurrent HTTP/1.1 connections per origin to 6.
Without cleanup, SSE connections from prior page loads linger after
refresh/navigation, eating into the pool. After 2-3 refreshes, all 6
slots are consumed by stale SSE streams and new API fetch calls queue
indefinitely — the UI shows "connected" (SSE works) but data never
loads.

Add a beforeunload handler that closes both eventSource (chat events)
and logEventSource (log stream) so the browser can reuse connections
immediately on page reload.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 19:33:18 -07:00
[email protected]andClaude Opus 4.6 47f83c1bfc feat(web): improve mission detail with markdown goal and thread table
- Goal rendered as full-width markdown block instead of plain-text
  meta item (uses existing renderMarkdown/marked)
- Current focus and success criteria also rendered as markdown
- Spawned threads shown as a clickable table with goal, type, state,
  steps, tokens, and created date instead of a UUID list
- Clicking a thread row opens an inline thread detail view showing
  metadata grid and full message history with markdown rendering
- Back button returns to the mission detail view
- Backend: mission detail now returns full thread summaries (goal,
  state, step_count, tokens) instead of just thread IDs

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 19:15:54 -07:00
[email protected]andClaude Opus 4.6 6d56d9cb11 fix(engine): eagerly initialize engine v2 at startup
The gateway API endpoints (/api/engine/missions, etc.) call bridge
query functions that return empty results when the engine state hasn't
been initialized yet. Previously, initialization only happened lazily
on the first chat message via handle_with_engine().

Now when ENGINE_V2=true, the engine is initialized in Agent::run()
before channels start, so the self-improvement mission and other
engine state is available to gateway API endpoints immediately.

Also rename get_or_init_engine → init_engine and make it public so
it can be called from agent_loop.rs at startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 19:07:35 -07:00
[email protected]andClaude Opus 4.6 7c153d584b feat(web): add Missions tab to gateway UI
Add a full Missions page to the web gateway with list view, detail view,
and action buttons (Fire, Pause, Resume).

Backend: add /api/engine/missions/summary endpoint returning counts by
status (active/paused/completed/failed).

Frontend:
- New "Missions" tab between Jobs and Routines
- Summary cards showing mission counts by status
- Table with name, goal, cadence type, thread count, status, actions
- Detail view with goal, cadence, current focus, success criteria,
  approach history, spawned thread list, and action buttons
- Fire/Pause/Resume actions with toast notifications
- i18n support (English + Chinese)
- CSS following the existing routines/jobs patterns

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 18:53:39 -07:00
[email protected]andClaude Opus 4.6 701974ce58 fix(engine): empty call_id on ActionResult and trace analyzer false positives
Fix structured executor not stamping call_id onto ActionResult — the
EffectExecutor trait doesn't receive call_id, so the structured executor
must copy it from the original ActionCall after execution. Empty call_id
caused OpenAI-compatible providers to reject the next LLM request with
"Invalid 'input[2].call_id': empty string".

Fix trace analyzer false positives:
- code_error check now only scans User-role code output messages
  (prefixed with [stdout]/[stderr]/[code ]/Traceback), not System
  prompt which contains example error text
- missing_tool_output check now recognizes ActionResult messages as
  valid tool output (Tier 0 structured path)
- Add NotImplementedError to detected code error patterns

New trace checks:
- empty_call_id: detect ActionResult messages with missing/empty
  call_id before they reach the LLM API (severity: Error)
- llm_error: extract LLM provider errors from Failed state reason
- orchestrator_error: extract orchestrator errors from Failed state

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 18:38:07 -07:00
[email protected]andClaude Opus 4.6 098addce27 feat(engine): complete v2 side-by-side integration with gateway API
Wire engine v2 into the full submission pipeline and expose threads,
projects, and missions through the web gateway REST API.

Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear
submissions to engine v2 when ENGINE_V2=true. Previously only UserInput
and ApprovalResponse were handled; all other control commands fell
through to disconnected v1 sessions.

Bridge query layer — add 11 read-only query functions and 6 DTO types
so gateway handlers can inspect engine state (threads, steps, events,
projects, missions) without direct access to the EngineState singleton.

Gateway endpoints — new /api/engine/* routes:
  GET  /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events
  GET  /projects, /projects/{id}
  GET  /missions, /missions/{id}
  POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume

SSE events — add ThreadStateChanged, ChildThreadSpawned, and
MissionThreadSpawned AppEvent variants. Expand the bridge event mapper
to forward StateChanged and ChildSpawned engine events to the browser.

Engine crate — add ConversationManager::clear_conversation() for /new
and /clear commands.

Code quality — replace 10 .expect() calls with proper error returns,
remove dead AgentConfig.engine_v2 field, log silent init errors, fix
duplicate doc comment, improve fallthrough documentation.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-26 18:19:10 -07:00
[email protected] b07cc5e144 Merge remote-tracking branch 'origin/staging' into v2-architecture
# Conflicts:
#	Cargo.toml
2026-03-26 14:00:35 -07:00
[email protected] a81c75494d Merge remote-tracking branch 'origin/staging' into v2-architecture
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	crates/ironclaw_common/src/event.rs
#	crates/ironclaw_common/src/lib.rs
#	src/channels/web/server.rs
#	src/channels/web/sse.rs
#	src/channels/web/types.rs
#	src/channels/web/util.rs
#	src/channels/web/ws.rs
#	src/orchestrator/api.rs
#	src/worker/job.rs
#	tests/ws_gateway_integration.rs
2026-03-25 19:17:54 -07:00
[email protected]andClaude Opus 4.6 ef4fb65dbc docs: add engine v2 architecture, self-improvement, and dev history
Three new docs for contributors:

- engine-v2-architecture.md: Two-layer architecture (Rust kernel +
  Python orchestrator), five primitives, execution model with nested
  Monty VMs, bridge layer, memory/reflection, missions, capabilities

- self-improvement.md: Three improvement levels (prompt/orchestrator/
  config/code), autoresearch-inspired Mission loop, versioned
  orchestrator with auto-rollback, fix pattern database, safety model

- development-history.md: Summary of 6 Claude Code sessions that
  built the system, key design decisions and debugging moments,
  architecture evolution from 900-line Rust loop to Python orchestrator

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 16:44:20 -07:00
[email protected]andClaude Opus 4.6 46fd2b5dd9 feat(engine): orchestrator versioning, auto-rollback, and tests
Add version lifecycle for the Python orchestrator:
- Failure tracking via MemoryDoc (orchestrator:failures)
- Auto-rollback: after 3 consecutive failures, skip the latest version
  and fall back to previous (or compiled-in v0)
- Success resets the failure counter
- OrchestratorRollback event for observability

Update self-improvement Mission goal with Level 1.5 instructions for
orchestrator patches — the agent can now modify the execution loop
itself via memory_write with versioned orchestrator docs.

12 new tests: version selection (highest wins), rollback after failures,
rollback to default, failure counting/resetting, outcome parsing for
all 5 ThreadOutcome variants.

189 tests pass, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 16:37:04 -07:00
[email protected]andClaude Opus 4.6 080317aa94 fix(engine): all 177 tests pass with Python orchestrator
- Increment step_count and track tokens in __emit_event__("step_completed")
  so thread bookkeeping matches the old Rust loop behavior
- Remove double-counting of tokens in bootstrap (orchestrator handles it)
- Match nudge text to existing TOOL_INTENT_NUDGE constant
- Fix FINAL result propagation (use stored final_result, not VM return)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 16:31:20 -07:00
[email protected]andClaude Opus 4.6 63756039b4 feat(engine): switch ExecutionLoop::run() to Python orchestrator
Replace the 900-line Rust execution loop with a ~80-line bootstrap
that loads and runs the versioned Python orchestrator via Monty VM.

The orchestrator Python code (orchestrator/default.py) is the v0
compiled-in version. Runtime versions can override it via MemoryDoc
storage (orchestrator:main with tag orchestrator_code).

Key fixes during switchover:
- Use ExtFunctionResult::NotFound for unknown functions so Monty
  falls through to Python-defined functions (extract_final, etc.)
- Move helper function definitions above run_loop for Monty scoping
- Use FINAL result value (not VM return value) in Complete handler
- Rename 'final' variable to 'final_answer' to avoid Python keyword

Status: 171/177 tests pass. 6 remaining failures are step_count and
token tracking bookkeeping — the orchestrator manages these internally
but doesn't yet update the thread's counters via host functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 16:25:24 -07:00
[email protected]andClaude Opus 4.6 cfe856daaf feat(engine): add Python orchestrator module and host functions
Add the orchestrator infrastructure for replacing the Rust execution
loop with versioned Python code. This commit adds the module and host
functions without switching over — the existing Rust loop is unchanged.

New files:
- orchestrator/default.py: v0 Python orchestrator (run_loop + helpers)
- executor/orchestrator.rs: host function dispatch, orchestrator
  loading from Store with version selection, OrchestratorResult parsing

Host functions exposed to orchestrator Python via Monty suspension:
  __llm_complete__, __execute_code_step__ (nested Monty VM),
  __execute_action__, __check_signals__, __emit_event__,
  __add_message__, __save_checkpoint__, __transition_to__,
  __retrieve_docs__, __check_budget__, __get_actions__

Also makes json_to_monty, monty_to_json, monty_to_string pub(crate)
in scripting.rs for cross-module use.

Design doc: docs/plans/2026-03-25-python-orchestrator.md

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 14:51:29 -07:00
[email protected] d7a3e04cec Add checkpoint-based engine thread recovery 2026-03-25 10:02:45 -07:00
[email protected]andClaude Opus 4.6 8180a417ff feat(engine): self-improving engine via Mission system
Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.

Key changes:

Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
  matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
  prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
  async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading

Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-25 00:15:50 -07:00
[email protected]andClaude Opus 4.6 de71cd2ca7 test(engine): add E2E mission flow tests — 7 new tests
Comprehensive mission lifecycle tests:

- fire_mission_builds_meta_prompt_with_goal: verifies thread spawned
  with project context and recorded in history
- outcome_processing_extracts_next_focus: "Next focus: X" in FINAL()
  response → mission.current_focus updated
- outcome_processing_detects_goal_achieved: "Goal achieved: yes" →
  mission status transitions to Completed
- mission_evolves_via_direct_outcome_processing: 3-step evolution:
  step 1 sets focus to "db module", step 2 evolves to "tools module",
  step 3 detects goal achieved → mission completes. Tests the full
  learning loop without background task timing dependencies.
- fire_with_trigger_payload: webhook payload stored on mission and
  threads_today counter incremented
- daily_budget_enforced: max_threads_per_day=1 → first fire succeeds,
  second returns None

157 tests passing (151 prior + 6 new mission E2E).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 11:57:18 -07:00
[email protected]andClaude Opus 4.6 20cb92d4da feat(bridge): map routine_* calls to mission operations in v2
When the model calls routine_create, routine_list, routine_fire,
routine_pause, routine_resume, or routine_delete, the bridge now
routes them to the MissionManager instead of blocking with an error.

Mapping:
  routine_create → mission_create (with cadence parsing)
  routine_list   → mission_list
  routine_fire   → mission_fire
  routine_pause  → mission_pause
  routine_resume → mission_resume
  routine_update → mission_pause/resume (based on params)
  routine_delete → mission_complete (marks as done)

Routine tools removed from v1-only blocklist and restored in
available_actions(). The model can use either "routine" or "mission"
vocabulary — both work.

Still blocked: create_job, cancel_job, build_software (need v1
Scheduler/ContainerJobManager refs).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 11:51:39 -07:00
[email protected]andClaude Opus 4.6 d5b267323b feat(bridge): wire MissionManager into engine v2 for CodeAct access
Missions are now callable from CodeAct Python code:

```python
# Create a daily briefing mission
result = mission_create(
    name="Tech News",
    goal="Daily AI/crypto/software news briefing",
    cadence="0 9 * * *"
)

# List all missions
missions = mission_list()

# Manually fire a mission
mission_fire(id="...")

# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```

Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
  lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
  "webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
  (avoids circular dependency)

System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 09:37:21 -07:00
[email protected]andClaude Opus 4.6 536fafb096 feat(engine): implement MissionManager execution with meta-prompts
The MissionManager now builds evolving meta-prompts and processes
thread outcomes for continuous learning:

fire_mission() upgraded:
- Loads Project MemoryDocs via RetrievalEngine for context
- Builds meta-prompt from: goal, current_focus, approach_history,
  project knowledge docs, trigger payload, thread count
- Spawns thread with meta-prompt as user message
- Background task waits for completion and processes outcome
- Daily thread budget enforcement (max_threads_per_day)

Meta-prompt structure:
  # Mission: {name}
  Goal: {goal}
  ## Current Focus (evolves between threads)
  ## Previous Approaches (what we've tried)
  ## Knowledge from Prior Threads (lessons, playbooks, issues)
  ## Trigger Payload (webhook/event data if applicable)
  ## Instructions (accomplish step, report next focus, check goal)

Outcome processing:
- Extracts "next focus:" from FINAL() response → updates current_focus
- Detects "goal achieved: yes" → completes mission
- Records accomplishment in approach_history
- Failed threads recorded as "FAILED: {error}"

Cron ticker:
- start_cron_ticker() spawns tokio task, ticks every 60s
- Checks active Cron missions, fires those past next_fire_at

151 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 09:24:55 -07:00
[email protected]andClaude Opus 4.6 10cf040b43 feat(engine): extend Mission types with webhook/event triggers + evolving strategy
Mission types updated to support external activation sources:

MissionCadence expanded:
- Cron { expression, timezone } — timezone-aware scheduling
- OnEvent { event_pattern } — channel message pattern matching
- OnSystemEvent { source, event_type } — structured events from tools
- Webhook { path, secret } — external HTTP triggers (GitHub, email, etc.)
- Manual — explicit triggering only

The engine defines trigger TYPES. The bridge implements infrastructure
(cron ticker, webhook endpoints, event matchers). GitHub issues, PRs,
email, Slack events all use the generic Webhook cadence — no
special-casing in the engine. Webhook payload injected as
state["trigger_payload"] in the thread's Python context.

Mission struct extended:
- current_focus: what the next thread should work on (evolving)
- approach_history: what we've tried (for adaptation)
- max_threads_per_day / threads_today: daily budget
- last_trigger_payload: webhook/event data for thread context

Plan updated with trigger type table and webhook integration design.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 09:21:15 -07:00
[email protected]andClaude Opus 4.6 e67f2d686d docs: add Mission system design — goal-oriented autonomous threads
Missions replace routines with evolving, knowledge-accumulating
autonomous agents. Unlike routines (fixed prompt, stateless), Missions:

- Generate prompts from accumulated Project knowledge (lessons,
  playbooks, issues from prior threads)
- Adapt approach when something fails repeatedly
- Track progress toward a goal with success criteria
- Self-manage: pause when stuck, complete when goal achieved

Architecture: MissionManager with cron ticker spawns threads via
ThreadManager. Meta-prompt built from mission goal + Project MemoryDocs
via RetrievalEngine. Reflection feeds back automatically.

6-step implementation plan: cron trigger, meta-prompt builder, bridge
wiring, CodeAct tools, progress tracking, persistence.

Includes two worked examples: daily tech news briefing (ongoing) and
test coverage improvement (goal-driven, self-completing).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 09:14:30 -07:00
[email protected]andClaude Opus 4.6 0b0e770774 feat(bridge): complete Phase 6 — v1-only tool blocking, rate limiting, call limits
Three security/stability improvements in EffectBridgeAdapter:

1. V1-only tool blocking:
   - routine_create, create_job, build_software (and hyphenated variants)
     return helpful error: "use the slash command instead"
   - Filtered out of available_actions() so system prompt doesn't list them
   - Prevents crash from tools needing RoutineEngine/Scheduler refs

2. Per-step tool call limit:
   - Max 50 tool calls per code block (AtomicU32 counter)
   - Prevents amplification: `for i in range(10000): shell(...)`
   - Returns "call limit reached, break into multiple steps"

3. Rate limiting:
   - Per-user per-tool sliding window via RateLimiter
   - Checks tool.rate_limit_config() before every execution
   - Returns "rate limited, try again in Ns"

Architecture plan updated:
- Gateway integration: DONE
- Routines: BLOCKED (gracefully, with slash command fallback)
- Rate limiting: DONE
- Call limit: DONE
- Phase 6 status: DONE (remaining: acceptance tests, two-phase commit)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 08:51:45 -07:00
[email protected]andClaude Opus 4.6 ccec19174d feat(bridge): integrate with web gateway via AppEvent + v1 conversation DB
Three changes to make engine v2 visible in the web gateway:

1. SSE event streaming (AppEvent broadcast):
   - ThreadEvent → AppEvent conversion via thread_event_to_app_event()
   - Events broadcast to SseManager during the poll loop
   - Covers: Thinking, ToolCompleted (success/error), Status, Response
   - Web gateway receives real-time progress without any gateway changes

2. Conversation persistence to v1 database:
   - After thread completes, writes user message + agent response to
     v1 ConversationStore via add_conversation_message()
   - Uses get_or_create_assistant_conversation() for per-user per-channel
   - Web gateway reads from DB as usual — chat history appears

3. Final response broadcast:
   - AppEvent::Response with full text + thread_id sent via SSE
   - Web gateway renders the response in the chat UI

New EngineState fields: sse (Option<Arc<SseManager>>),
db (Option<Arc<dyn Database>>). Both populated from Agent.deps.

Agent.deps visibility widened to pub(crate).

Depends on: ironclaw_common crate with AppEvent type (PR #1615).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 00:54:55 -07:00
[email protected]andClaude Opus 4.6 d059844351 merge: integrate AppEvent refactoring (PR #1615)
Merges refactor/extract-app-event-to-ironclaw-common which extracts
SseEvent into crates/ironclaw_common as AppEvent. This is the
prerequisite for engine v2 gateway integration — the bridge can now
emit AppEvents without depending on web gateway types.

Conflict resolution: workspace members includes all three crates
(ironclaw_common, ironclaw_safety, ironclaw_engine).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 00:43:07 -07:00
[email protected]andClaude Opus 4.6 953b12585d refactor: extract AppEvent to crates/ironclaw_common
SseEvent was defined in src/channels/web/types.rs but imported by 12+
modules across agent, orchestrator, worker, tools, and extensions — it
had become the application-wide event protocol, not a web transport
concern.

Create crates/ironclaw_common as a shared workspace crate and move the
enum there as AppEvent.  Also move the truncate_preview utility which
was similarly leaked from the web gateway into agent modules.

- New crate: crates/ironclaw_common (AppEvent, truncate_preview)
- Rename SseEvent → AppEvent, from_sse_event → from_app_event
- web/types.rs re-exports AppEvent for internal gateway use
- web/util.rs re-exports truncate_preview
- Wire format unchanged (serde renames are on variants, not the enum)

Aligned with the event bus direction on refactor/architectural-hardening
where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 00:28:42 -07:00
[email protected]andClaude Opus 4.6 be2281eb13 docs(engine): document routine/job gap and SIGKILL crash scenario
Routines are entirely v1 — not hooked up to engine v2. When a user
asks "create a routine" as natural language, engine v2 tries to call
routine_create via CodeAct, but the tool needs RoutineEngine + Database
refs that the bridge's minimal JobContext doesn't provide. This caused
a SIGKILL crash during testing.

Options documented: block routine tools in v2 (short term), pass refs
through context (medium), replace with Mission system (long term).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-24 00:08:10 -07:00
[email protected]andClaude Opus 4.6 312f40040b docs(engine): add web gateway integration plan to Phase 6
Documents three gaps between engine v2 and the web gateway:
1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent)
2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB)
3. No cross-channel visibility (REPL ↔ web messages invisible to each other)

Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1
conversation tables after thread completion. Prerequisite: AppEvent
extraction PR (in progress separately).

Also updated DB persistence status: HybridStore with workspace-backed
MemoryDocs is now implemented (partial persistence).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 22:33:42 -07:00
[email protected]andClaude Opus 4.6 af6fff816a fix(bridge): adapt to execute_tool_with_safety params-by-value change
Staging merge changed execute_tool_with_safety to take params by value
instead of by reference (perf optimization from PR #926). Updated
bridge adapter to clone params before passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 22:29:35 -07:00
[email protected] ac7c17ee40 Merge remote-tracking branch 'origin/staging' into v2-architecture 2026-03-23 22:25:43 -07:00
[email protected]andClaude Opus 4.6 3b309ccbc8 feat(bridge): persist reflection docs to workspace for cross-session learning
Replaces InMemoryStore with HybridStore:
- Ephemeral data (threads, steps, events, leases) stays in-memory
- MemoryDocs (lessons, specs, playbooks from reflection) persist to
  the workspace at engine/docs/{type}/{id}.json

On engine init, load_docs_from_workspace() reads existing docs back
into the in-memory cache. This means:
- Lessons learned in session 1 are available in session 2
- The RetrievalEngine injects relevant past lessons into new threads
- The engine genuinely improves over time as reflection accumulates

Workspace paths:
  engine/docs/lessons/{uuid}.json
  engine/docs/specs/{uuid}.json
  engine/docs/playbooks/{uuid}.json
  engine/docs/summaries/{uuid}.json
  engine/docs/issues/{uuid}.json

No new database tables. Uses existing workspace write/read/list.
workspace() accessor widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 22:11:03 -07:00
[email protected]andClaude Opus 4.6 20b258aeea fix(engine): strengthen CodeAct prompt to prevent shallow text answers
The model was answering "Suggested 45 improvements" as a brief text
summary from training data without actually searching or listing them.
The trace showed: no code block, no tool calls, no FINAL().

Prompt changes:
- Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with
  plain text only." (was: "Always write code... plain text for brief
  explanations")
- Rule 2 (NEW): "NEVER answer from memory or training data alone.
  Always use tools to get real, current information before answering."
- Rule 3: FINAL answer "should be detailed and complete — not just a
  summary like 'found 45 items'"
- Rule 8 (NEW): "Include the actual content in your FINAL() answer,
  not just a count or summary. Users want to see the details."

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 21:51:12 -07:00
[email protected]andClaude Opus 4.6 dc367035ce fix(safety): demote leak detector warn-action logs from warn! to debug!
The leak detector's Warn-action matches (high_entropy_hex pattern on
web search results containing commit SHAs, CSS colors, URL hashes)
were logging at warn! level, corrupting the REPL UI with lines like:
  WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5

These are informational false positives — real leaks use LeakAction::Redact
which silently modifies the content. Warn-action matches only log for
debugging purposes and should not appear in production output.

Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 21:44:46 -07:00
[email protected]andClaude Opus 4.6 ad3529e6fb fix(bridge): demote all router info! logging to debug!
"engine v2: initializing" and "engine v2: handling message" were
printing at INFO level, corrupting the REPL UI. All router logging
now uses debug! — only visible with RUST_LOG=ironclaw=debug.

Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 21:40:33 -07:00
[email protected]andClaude Opus 4.6 c1163f41fc fix(engine): demote trace/reflection logging from info to debug
INFO-level log output from background tasks (trace analysis, reflection)
corrupts the REPL terminal UI. The trace summary, issue warnings, and
reflection doc previews were printing mid-approval-card, breaking the
interactive display.

Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!.
Trace analysis and reflection results now only show when
RUST_LOG=ironclaw_engine=debug is set.

Also added logging discipline rule to global CLAUDE.md:
- info! → user-facing status the REPL intentionally renders
- debug! → internal diagnostics (traces, reflection, engine internals)
- Background tasks must NEVER use info! — it breaks the TUI

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 21:14:08 -07:00
[email protected]andClaude Opus 4.6 e82dcbd5e6 feat(bridge): implement tool approval flow for engine v2
Adds a complete approval flow that mirrors v1 behavior, using the
existing v1 security controls (Tool::requires_approval, auto-approve
sets, StatusUpdate::ApprovalNeeded).

## How it works

### Step 1: Tool blocked at execution
When the LLM's code calls a tool (e.g., `shell("ls")`):
1. EffectBridgeAdapter.execute_action() looks up the Tool object
2. Calls tool.requires_approval(&params) — returns ApprovalRequirement
3. If Always → EngineError::LeaseDenied (always blocks)
4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set,
   returns EngineError::LeaseDenied
5. If Never → proceeds to execution

### Step 2: Engine returns NeedApproval
The LeaseDenied error propagates through:
- CodeAct path: becomes Python RuntimeError, code halts, thread returns
  NeedApproval with action_name + parameters
- Structured path: same via ActionResult.is_error

### Step 3: Router stores pending approval
- PendingApproval { action_name, original_content } stored on EngineState
- StatusUpdate::ApprovalNeeded sent to channel (shows approval card in
  CLI/web with tool name, parameters, yes/always/no buttons)
- Returns text: "Tool 'shell' requires approval. Reply yes/always/no."

### Step 4: User responds
handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2:
- 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes
  original message (tool now passes the approval check on second run)
- 'always' → same + logs for session persistence
- 'no' → returns "Denied: tool was not executed."

### Key design choice
Instead of pausing/resuming mid-execution (which needs engine changes
to freeze/restore the Monty VM state), we auto-approve the tool and
re-run the full message. The EffectBridgeAdapter's auto_approved set
persists across runs, so the second execution passes immediately.

This trades one extra LLM call for zero engine modifications.

## Files changed
- src/bridge/router.rs: PendingApproval struct, handle_approval(),
  NeedApproval → StatusUpdate::ApprovalNeeded conversion
- src/bridge/mod.rs: export handle_approval
- src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2
- src/bridge/effect_adapter.rs: fmt fixes

151 tests passing, clippy + fmt clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 21:10:40 -07:00
[email protected]andClaude Opus 4.6 38fc870223 feat(bridge): wire v1 security controls into engine v2 adapter
Zero engine crate changes. All security controls enforced at the bridge
boundary in EffectBridgeAdapter:

1. Tool approval (v1: Tool::requires_approval):
   - Checks each tool's approval requirement with actual params
   - Always → returns EngineError::LeaseDenied (blocks execution)
   - UnlessAutoApproved → checks auto_approved set, blocks if not approved
   - Never → proceeds
   - Per-session auto_approved HashSet (for future "always" handling)

2. Hook interception (v1: BeforeToolCall):
   - Runs HookEvent::ToolCall before every execution
   - HookOutcome::Reject → blocks with reason
   - HookError::Rejected → blocks with reason
   - Hook errors → fail-open (logged, execution continues)

3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm):
   - Leak detection: API keys in tool output are redacted
   - Policy enforcement: content policy rules applied
   - Length truncation: output capped at 100KB
   - XML boundary protection: prevents injection via tool output

4. Sensitive param redaction (v1: redact_params):
   - Tool's sensitive_params() consulted before hooks see parameters
   - Redacted params sent to hooks, original params used for execution

5. available_actions() now sets requires_approval based on each tool's
   default approval requirement, so the engine's PolicyEngine can
   gate tools it hasn't seen before.

6. Actual execution timing measured via Instant::now() (replaces
   placeholder Duration::from_millis(1)).

Accessor visibility: hooks() widened to pub(crate).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 20:34:24 -07:00
[email protected]andClaude Opus 4.6 10098e7958 feat(engine): add missions, reliability tracker, reflection executor, and provenance-aware policy
- Add Mission type and MissionManager for recurring thread scheduling
- Add ReliabilityTracker for per-capability success/failure/latency tracking
- Add reflection executor that spawns CodeAct threads for post-completion reflection
- Extend PolicyEngine with provenance-aware taint checking (LLM-generated data
  requires approval for financial/external-write effects)
- Extend Store trait with mission CRUD methods
- Add conversation surface tracking, compaction token fix, context memory injection
- Wire new modules through lib.rs re-exports and bridge adapters

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 20:21:08 -07:00
[email protected]andClaude Opus 4.6 65a32c81b5 docs(security): cross-reference v1 controls — use, don't reinvent
Updated security plan with detailed audit of ALL existing v1 security
controls and how they map to engine v2 bridge gaps:

Key finding: v1 already has solutions for every security gap identified.
The bridge just needs to wire them in:

- Tool::requires_approval() exists but bridge doesn't call it
- safety.wrap_for_llm() exists but tool results enter context unwrapped
- RateLimiter exists but bridge doesn't check rate limits
- BeforeToolCall hooks exist but bridge doesn't run them
- redact_params() exists but bridge doesn't redact sensitive params
- Shell risk classification (Low/Medium/High) is inherited but ignored

Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter,
not new security infrastructure. The bridge is the security boundary.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 17:30:51 -07:00
[email protected]andClaude Opus 4.6 70c7330bd6 docs: add engine v2 security model and audit
Comprehensive security analysis of engine v2 covering:

Threat model: 4 attacker profiles (malicious input, prompt injection
via tools, poisoned memory, supply chain).

Current state audit: 9 controls working (Monty sandbox, safety layer,
policy engine, leases, provenance, events) and 9 gaps identified.

Critical finding: ALL tools granted by default — CodeAct code can call
shell, write_file, apply_patch without approval. Proposed fix: 3-tier
tool classification (auto/approve-once/always-approve).

CodeAct-specific threats: tool call amplification, prompt injection via
search results, data exfiltration via tool chains, Monty escape.

Self-improvement security: poisoned trace attacks, memory poisoning via
reflection. Mitigations: edit validation, frequency caps, audit trail,
auto-rollback, reflection output scanning.

6-layer security architecture proposed: input validation, capability
gating, output sanitization, execution sandboxing, self-improvement
controls, observability.

Prioritized implementation plan with severity/effort ratings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 17:22:45 -07:00
[email protected]andClaude Opus 4.6 88d1c16d67 docs: add self-improving engine design plan
Designs a system where the engine debugs and improves itself, based on
the pattern observed in the last session: 5 consecutive bug fixes all
followed trace → read → identify → edit → test, using tools the engine
already has access to.

Three levels of self-improvement:
- Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply.
- Level 2 (Config): adjust defaults/mappings. Branch + test + PR.
- Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR.

Architecture: Self-improvement Mission spawns a Reflection thread that
reads traces, reads source, proposes fixes, validates via cargo test,
and either auto-applies (Level 1) or creates a PR (Level 2-3).

Includes: fix pattern database (seeded from our 8 debugging session
fixes), feedback loop diagram, safety model, implementation phases
(A through D), and what exists vs what's new.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 16:02:10 -07:00
[email protected]andClaude Opus 4.6 bc2d5b6b83 docs(engine): update architecture plan with Phase 6 status and approval flow design
Phase 6 updated to reflect what was actually built:
- Bridge adapters (LLM, Effect, InMemoryStore, Router) — all done
- Integration touchpoint (4 lines in handle_message) — done
- Live progress via broadcast events — done
- Conversation persistence across messages — done
- Trace recording + retrospective analysis — done
- 8 bugs found and fixed via trace analysis — documented

Phase 6 remaining work documented:
- Approval flow: detailed 5-step design (send to channel, pause thread,
  route response, resume execution, always handling) with v1 reference
- Database persistence (InMemoryStore → real DB tables)
- Acceptance testing (TestRig + TraceLlm fixtures)
- Two-phase commit for high-stakes effects

Progress table updated: Phase 6 marked as DONE (partial), 134 tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 15:14:27 -07:00
[email protected]andClaude Opus 4.6 7afeaa9cad fix(engine): fix false positive missing_tool_output warning in trace analyzer
The check was looking for "[" + "result]" in System-role messages only,
but tool output metadata is added with patterns like "[shell result]"
and may appear in messages with any role. Changed to scan all messages
for " result]" or " error]" patterns regardless of role.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 14:54:48 -07:00
[email protected]andClaude Opus 4.6 4bfcc9cd61 fix(engine): replace byte-index slicing with char-safe truncation
Panic: 'byte index 80 is not a char boundary; it is inside ''' when
tool output contained multi-byte UTF-8 characters (smart quotes from
web search results).

Fixed 4 unsafe byte-index slices:
- thread.rs:281: message preview &content[..80] → chars().take(80)
- loop_engine.rs:556: tool output &str[..4000] → chars().take(4000)
- loop_engine.rs:579: output tail &str[len-8000..] → chars().skip()
- scripting.rs:82: stdout tail &str[len-N..] → chars().skip()

All now use .chars().take() or .chars().skip() which respect character
boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on
user-supplied or external strings."

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 13:01:03 -07:00
[email protected]andClaude Opus 4.6 c2fe5376df refactor(engine): extract prompt templates to markdown files
Prompt templates moved from inline Rust strings to plain markdown files
at crates/ironclaw_engine/prompts/ for easy inspection and iteration:

- prompts/codeact_preamble.md — main instructions, special functions,
  context variables, rules
- prompts/codeact_postamble.md — strategy section

Loaded at compile time via include_str!(), so no runtime file I/O.
Edit the .md files and rebuild to iterate on prompts.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 12:54:36 -07:00
[email protected]andClaude Opus 4.6 28b05945c1 fix(engine): replace web_fetch example with web_search in CodeAct prompt
The system prompt example used web_fetch(url="...") which doesn't exist
as a tool. The model learned from the example and tried web_fetch,
getting "Tool not found". Changed to web_search(query="...") which is
an actual registered tool.

Found via trace analysis — reflection pipeline correctly identified
this as a "Tool Name Correction" spec doc.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 12:53:02 -07:00
[email protected]andClaude Opus 4.6 3608eb9b25 chore: remove trace files and add to .gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 00:02:17 -07:00
[email protected]andClaude Opus 4.6 45a2f590e5 fix(engine): add state hint on code errors + retrieval engine integration
When code fails with NameError/UnboundLocalError (model trying to
access variables from a previous step), the error output now includes:

  [HINT] Variables don't persist between code blocks. Use the `state`
  dict to access data from previous steps. Available keys: ["web_search",
  "last_return"]

This teaches the model to use `state["web_search"]` instead of `result`
after a NameError, reducing wasted steps from 3-4 to 1.

Also integrates RetrievalEngine into context building and ThreadManager:
- build_step_context() now accepts optional RetrievalEngine to inject
  relevant memory docs (Lessons, Specs, Playbooks) into LLM context
- RetrievalEngine uses keyword matching with doc-type priority scoring
- Memory docs from reflection (Phase 4) now feed back into future threads

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-23 00:02:07 -07:00
[email protected]andClaude Opus 4.6 d2d93f98fe feat(engine): persist variables across code steps via state dict
Monty creates a fresh runtime per code step, so variables are lost
between steps. This caused the model to re-paste tool results from
system messages, wasting tokens.

Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that
accumulates across steps:
- Tool results stored by tool name: state["web_search"] = {results...}
- Return values stored: state["last_return"], state["step_0_return"]
- Injected as a `state` Python variable in each new MontyRun

Now the model can do:
  Step 1: results = web_search(query="...")  # tool result saved in state
  Step 2: data = state["web_search"]         # access previous result
          summary = llm_query("summarize", str(data))
          FINAL(summary)

System prompt updated to document the `state` variable.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 23:16:41 -07:00
[email protected]andClaude Opus 4.6 e8c0d3df52 fix(bridge): parse JSON tool output to prevent double-serialization
From trace analysis: web_search returned a JSON string, which was
wrapped as serde_json::json!(string) creating a Value::String containing
JSON. When Monty got this as MontyObject::String, the Python code
couldn't index it with result['title'] → TypeError.

Fix: try parsing the tool output string as JSON first. If valid, use the
parsed Value (becomes a Python dict/list). If not valid JSON, keep as
string. This means web_search results are directly indexable in Python:
  results = web_search(query="...")
  print(results["results"][0]["title"])  # works now

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 23:08:52 -07:00
[email protected]andClaude Opus 4.6 d8f01693f4 fix(bridge): convert tool name hyphens to underscores for Python compatibility
Root cause from trace analysis: the LLM writes `web_search()` (valid
Python identifier) but the tool registry has `web-search` (with hyphen).
The EffectBridgeAdapter couldn't find the tool → "Tool not found" error
→ model fabricated fake data instead.

Fixes:
- available_actions(): converts tool names from hyphens to underscores
  (web-search → web_search) so the system prompt lists valid Python names
- execute_action(): tries the original name first, then falls back to
  hyphenated form (web_search → web-search) for tool registry lookup
- Same conversion in router's capability registry builder

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 23:03:52 -07:00
[email protected]andClaude Opus 4.6 b1254cf6f9 feat(engine): wire reflection pipeline + trace analysis into thread lifecycle
After every thread completes, ThreadManager now automatically runs:

1. Retrospective trace analysis (non-LLM, always):
   - Detects 8 issue categories (tool errors, code errors, missing
     outputs, excessive steps, hallucination risk, etc.)
   - Logs issues at warn level when found

2. Trace file recording (when ENGINE_V2_TRACE=1):
   - Writes full JSON trace to engine_trace_{timestamp}.json

3. LLM reflection (when enable_reflection=true):
   - Calls reflection pipeline to produce Summary, Lesson, Issue docs
   - Saves docs to store for future context retrieval
   - Enabled by default in the bridge router

All three run inside the spawned tokio task after exec.run() completes,
before saving the final thread state. No external wiring needed.

Removed duplicate trace recording from the router — it's now handled
by ThreadManager automatically.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 22:56:21 -07:00
[email protected]andClaude Opus 4.6 b57178c1d1 feat(engine): execution trace recording + retrospective analysis
Enable with ENGINE_V2_TRACE=1 to get full execution traces and
automatic issue detection after each thread completes.

Trace recording (executor/trace.rs):
- build_trace(): captures full thread state — messages (with full
  content), events, step count, token usage, detected issues
- write_trace(): writes JSON to engine_trace_{timestamp}.json
- log_trace_summary(): logs summary + issues at info/warn level

Retrospective analyzer detects 8 issue categories:
- thread_failure: thread ended in Failed state
- no_response: no assistant message generated
- tool_error: specific tool failures with error details
- code_error: Python errors (NameError, SyntaxError, etc.) in output
- missing_tool_output: tool results exist but not in system messages
- excessive_steps: >10 steps (may be stuck in loop)
- no_tools_used: single-step answer without tools (hallucination risk)
- mixed_mode: text responses without code blocks (prompt not followed)

Thread state now saved to store after execution completes (for trace
access after join_thread).

Usage:
  ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run
  # After each message: trace JSON + issue log in terminal

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 22:53:08 -07:00
[email protected]andClaude Opus 4.6 c62eed9fcd feat(engine): add debug/trace logging for CodeAct execution
Three verbosity levels for debugging the engine:

RUST_LOG=ironclaw_engine=debug:
- LLM call: message count, iteration, force_text
- LLM response: type (text/code/action_calls), token usage
- Code execution: code length, action count, had_error, final_answer
- Text response: length, FINAL() detection

RUST_LOG=ironclaw_engine=trace:
- Full message list sent to LLM (role, length, first 200 chars each)
- Full code block being executed
- stdout preview (first 500 chars)
- Per-tool results (name, success, first 300 chars of output)
- Text response preview (first 500 chars)

Usage:
  ENGINE_V2=true RUST_LOG=ironclaw_engine=debug cargo run
  ENGINE_V2=true RUST_LOG=ironclaw_engine=trace cargo run

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 22:09:23 -07:00
[email protected]andClaude Opus 4.6 51e1bf5533 fix(engine): include tool results in code step output for LLM context
The LLM was ignoring tool results and answering from training data
because the compact output metadata didn't include what tools returned.
Tool results lived only as ActionResult messages (role: Tool) which
some providers flatten or the model ignores.

Now the code step output includes:
- stdout from Python print() statements
- [tool_name result] with the actual output (truncated to 4K per tool)
- [tool_name error] for failed tools
- [return] for the code's return value
- Total output truncated to 8K chars to prevent context bloat

This ensures the model sees web_search results, API responses, etc.
in the next iteration and can reason about them instead of hallucinating.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 21:25:48 -07:00
[email protected]andClaude Opus 4.6 7087964c22 feat(engine): live progress status updates via event broadcast
Engine v2 now shows live progress in the CLI (and any channel):
- "Thinking..." when a step starts
- Tool name + success/error when actions execute
- "Processing results..." when a step completes

Implementation:
- ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256)
- ExecutionLoop.emit_event() writes to thread.events AND broadcasts
- ThreadManager.subscribe_events() returns a receiver
- Router uses tokio::select! to listen for events while waiting for
  thread completion, forwarding them as StatusUpdate to the channel

This replaces the polling approach with zero-latency event streaming.
Agent.channels visibility widened to pub(crate) for bridge access.

102 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 18:18:20 -07:00
[email protected]andClaude Opus 4.6 b621cd503d docs: add crate extraction & cleanup roadmap
Documents architectural recommendations from the engine v2 design
process for future reference:

- Root directory consolidation (channels-src + tools-src → extensions/)
- Crate extraction tiers: zero-coupling (estimation, observability,
  tunnel), trivial-coupling (document_extraction, pairing, hooks),
  medium-coupling (secrets, MCP, db, workspace, llm, skills),
  heavy-coupling (web gateway, agent, extensions)
- src/ module reorganization into logical groups (core, persistence,
  infra, media, support)
- main.rs/app.rs slimming targets (100/500 lines after migration)
- WASM module candidates (document_extraction) and non-candidates
  (REPL, web gateway → separate crates instead)
- Priority ordering for extraction work
- Tracks completed items (ironclaw_safety, ironclaw_engine,
  transcription move)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 14:49:52 -07:00
[email protected] 432ed11e9c Merge remote-tracking branch 'origin/staging' into v2-architecture 2026-03-22 14:33:59 -07:00
[email protected]andClaude Opus 4.6 a325d9fbcd fix(engine): detect FINAL() in text responses + regression tests
Models sometimes write FINAL() outside code blocks — as plain text
after an explanation. The Hyperliquid case: model outputs a long
analysis then FINAL("""...""") at the end, not inside ```repl fences.

Fixes:
- extract_final_from_text(): regex-based FINAL detection in text
  responses, matching the official RLM's find_final_answer() fallback
- Handles: double-quoted, single-quoted, triple-quoted, unquoted,
  nested parens
- Checked in LlmResponse::Text handler BEFORE tool intent nudge
  (FINAL takes priority)

9 new tests:
- codeact_final_in_text_response: FINAL("answer") in plain text
- codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text
- final_double_quoted, final_single_quoted, final_triple_quoted,
  final_unquoted, final_with_nested_parens, final_after_long_text,
  no_final_returns_none

102 tests passing (93 + 9 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 14:31:51 -07:00
[email protected]andClaude Opus 4.6 79120aa9d8 test(bridge): add 11 regression tests for code block extraction
Covers the exact failure modes discovered during live testing:

- extract_repl_block: standard ```repl fenced block
- extract_python_block: ```python marker
- extract_py_block: ```py shorthand
- extract_bare_backtick_block: bare ``` with Python content
- skip_non_python_language: ```json should NOT be extracted
- no_code_blocks_returns_none: plain text, no fences
- multiple_code_blocks_concatenated: two ```repl blocks with
  explanation between them → concatenated with \n\n
- mixed_thinking_and_code: model outputs explanation + two
  ```python blocks (the Hyperliquid case) → both extracted
- repl_preferred_over_bare: ```repl takes priority over bare ```
- empty_code_block_skipped: empty fenced block returns None
- unclosed_block_returns_none: no closing ``` returns None

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 13:17:15 -07:00
[email protected]andClaude Opus 4.6 2e03271cdb fix(bridge): detect code blocks in plain completion path + multi-block support
Two bugs fixed:

1. The no-tools completion path (used by CodeAct since we send empty
   actions) returned LlmResponse::Text without checking for code blocks.
   Code blocks were rendered as markdown text instead of being executed.

2. extract_code_block now:
   - Handles bare ``` fences (skips non-Python languages)
   - Collects ALL code blocks in the response and concatenates them
     (models often split code across multiple blocks with explanation)
   - Tries markers in order: ```repl, ```python, ```py, then bare ```

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 13:08:44 -07:00
[email protected]andClaude Opus 4.6 f3a185cc4a test(engine): add 8 CodeAct/RLM E2E tests with mock LLM
Comprehensive test coverage for the Monty Python execution path:

- codeact_simple_final: Python code calls FINAL('answer') → thread completes
- codeact_tool_call_then_final: code calls test_tool() → FunctionCall
  suspends VM → MockEffects returns result → code resumes → FINAL()
- codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15')
  with no tool calls — pure Python in Monty
- codeact_multi_step: first step prints output (no FINAL), second step
  sees output metadata and calls FINAL — tests iterative REPL flow
- codeact_error_recovery: first step has NameError → error flows to LLM
  as stdout → second step recovers with FINAL — tests error transparency
- codeact_context_variables_available: code accesses `goal` and `context`
  variables injected by the RLM context builder
- codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times
  → 3 FunctionCall suspensions → all results collected → FINAL
- codeact_llm_query_recursive: code calls llm_query('prompt') → VM
  suspends → MockLlm provides sub-agent response → result returned as
  Python string variable

93 tests passing (85 prior + 8 new), zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 13:03:18 -07:00
[email protected]andClaude Opus 4.6 749c208b3c feat(engine): enable CodeAct/RLM mode with code block detection
The engine now operates in CodeAct/RLM mode:

System prompt (executor/prompt.rs):
- Instructs LLM to write Python in ```repl fenced blocks
- Documents available tools as callable Python functions
- Documents llm_query(), llm_query_batched(), FINAL()
- Documents context variables (context, goal, step_number, previous_results)
- Strategy guidance: examine context, break into steps, use tools, call FINAL()

Code block detection (bridge/llm_adapter.rs):
- extract_code_block() scans LLM text responses for ```repl or ```python blocks
- When detected, returns LlmResponse::Code instead of LlmResponse::Text
- The ExecutionLoop routes Code responses through Monty for execution

No structured tool definitions sent to LLM:
- Tools are described in the system prompt as Python functions
- The LLM call sends empty actions array, forcing text-mode responses
- This ensures the LLM writes code blocks (CodeAct) instead of
  structured tool calls (which would bypass the REPL)

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 12:38:07 -07:00
[email protected]andClaude Opus 4.6 4e8b94a555 fix(engine): persist conversation context across messages
The engine was creating a fresh ThreadManager and InMemoryStore per
message, losing all context between turns. A follow-up question like
"what are the latest 10 issues?" had no memory of the prior "how many
issues" response.

Fixes:
- EngineState (ThreadManager, ConversationManager, InMemoryStore) now
  persists across messages via OnceLock, initialized on first use
- ConversationManager builds message history from prior conversation
  entries (user messages + agent responses) and passes it to new threads
- ThreadManager.spawn_thread_with_history() accepts initial_messages
  that are prepended before the current user message
- System notifications (thread started/completed) are filtered out of
  the history (not useful as LLM context)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 01:08:05 -07:00
[email protected]andClaude Opus 4.6 374a21c7fe fix(bridge): match existing LLM request format to prevent 400 errors
The LLM bridge was missing several defaults that the existing
Reasoning.respond_with_tools() sets:

- tool_choice: "auto" when tools are present (required by some providers)
- max_tokens: 4096 (default)
- temperature: 0.7 (default)
- When no tools (force_text): use plain complete() instead of
  complete_with_tools() with empty tools array — matches existing
  no-tools fallback path

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 00:50:57 -07:00
[email protected]andClaude Opus 4.6 9d6d76d9c9 fix(engine): add user message and system prompt to thread before execution
The ExecutionLoop was sending empty messages to the LLM because the
thread was spawned with the user's input as the goal but no messages.

Fixes:
- ThreadManager.spawn_thread() now adds the goal as an initial user
  message before starting the execution loop
- ExecutionLoop.run() injects a default system prompt if none exists

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 00:35:56 -07:00
[email protected]andClaude Opus 4.6 ac4ced02ae feat(engine): Phase 6 — bridge adapters for main crate integration
Strategy C parallel deployment: when ENGINE_V2=true env var is set,
user messages route through the engine instead of the existing agentic
loop. All existing behavior is unchanged when the flag is off.

Bridge module (src/bridge/):
- LlmBridgeAdapter: wraps LlmProvider as engine LlmBackend, converts
  ThreadMessage↔ChatMessage, ActionDef↔ToolDefinition, depth-based
  model routing (primary vs cheap_llm)
- EffectBridgeAdapter: wraps ToolRegistry+SafetyLayer as EffectExecutor,
  routes tool calls through existing execute_tool_with_safety pipeline
- InMemoryStore: HashMap-backed Store impl (no DB tables needed yet)
- EngineRouter: is_engine_v2_enabled() + handle_with_engine() that
  builds engine from Agent deps and processes messages end-to-end

Integration touchpoint (4 lines in agent_loop.rs):
  After hook processing, before session resolution, check ENGINE_V2
  flag and route UserInput through the engine path.

Accessor visibility widened: llm(), cheap_llm(), safety(), tools()
changed from pub(super) to pub(crate) for bridge access.

85 engine tests + main crate clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-22 00:22:13 -07:00
[email protected]andClaude Opus 4.6 f0295f304f docs(engine): simplify execution tiers — Monty-only for CodeAct/RLM
Restructure phases 6-8 to clarify execution model:

- Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker
  Python runtimes for LLM-generated code.
- WASM sandbox is for third-party tool isolation (existing infra, Phase 8)
- Docker containers are for thread-level isolation of high-risk work (Phase 8)
- Two-phase commit moves to Phase 6 (integration) at the adapter boundary

Phase renumbering:
- Old Phase 6 (Tier 2-3) → removed as separate phase
- Old Phase 7 (integration) → Phase 6
- Old Phase 8 (cleanup) → Phase 7
- New Phase 8: WASM tools + Docker thread isolation (infra integration)

Updated progress table: Phases 1-5 marked DONE with test counts and commits.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 23:40:48 -07:00
[email protected]andClaude Opus 4.6 0827235c9c feat(engine): Phase 5 — conversation surface separated from execution
Conversation is now a UI layer, not an execution boundary. Multiple
threads can run concurrently within one conversation; threads can
outlive their originating conversation.

New types (types/conversation.rs):
- ConversationSurface: channel + user + entries + active_threads
- ConversationEntry: sender (User/Agent/System) + content + origin_thread_id
- ConversationId, EntryId (UUID newtypes)
- EntrySender enum (User, Agent{thread_id}, System)

ConversationManager (runtime/conversation.rs):
- get_or_create_conversation(channel, user) — indexed by (channel, user)
- handle_user_message() — injects into active foreground thread or spawns new
- record_thread_outcome() — adds agent/system entries, untracks completed threads
- get_conversation(), list_conversations()

This enables the key architectural insight: a user can ask "what's the
weather?" while a deployment thread is still running. Both produce entries
in the same conversation.

85 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 23:21:54 -07:00
[email protected]andClaude Opus 4.6 4bc7ffdf0c feat(engine): Phase 4 — budget controls, compaction, reflection pipeline
Budget enforcement in ExecutionLoop:
- max_tokens_total: cumulative token limit, checked before each iteration
- max_duration: wall-clock timeout for entire thread
- max_consecutive_errors: consecutive error steps threshold (resets on
  success, matching official RLM behavior)
- All produce ThreadOutcome::Failed with descriptive messages

Context compaction (from RLM paper, 85% threshold):
- estimate_tokens(): char-based estimation (chars/4, matching RLM)
- should_compact(): triggers when tokens >= threshold_pct * context_limit
- compact_messages(): asks LLM to summarize progress, replaces history
  with [system, summary, continuation_note], preserves intermediate results
- Configurable via ThreadConfig: model_context_limit, compaction_threshold

Dual model routing:
- LlmCallConfig gains depth field (0=root, 1+=sub-call)
- Implementations can route to cheaper models for sub-calls
- ExecutionLoop passes thread depth to every LLM call

Reflection pipeline (reflection/pipeline.rs):
- reflect(thread, llm): analyzes completed thread via LLM
- Produces Summary doc (always), Lesson doc (if errors), Issue doc (if failed)
- Builds transcript from thread messages + error events
- Returns ReflectionResult with docs + token usage

ThreadConfig extended with: max_tokens_total, max_consecutive_errors,
model_context_limit, enable_compaction, compaction_threshold, depth, max_depth.

78 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 22:55:34 -07:00
[email protected]andClaude Opus 4.6 ff1107179a docs(engine): update architecture plan with RLM cross-reference learnings
Comprehensive update after cross-referencing against official RLM
(alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), Prime Intellect
(verifiers/RLMEnv), rlm-rs (zircote/rlm-rs), and Google ADK RLM.

Changes:
- Mark Phases 1-3 as DONE with commit refs and test counts
- Add "Key Influences" section documenting all reference implementations
- Phase 3: full table of implemented RLM features with sources
- Phase 3: "Remaining gaps" table with which phase addresses each
- Phase 4: expanded with compaction (85% context), rlm_query() (full
  recursive sub-agent), dual model routing, budget controls (USD,
  timeout, tokens, consecutive errors), lazy loading, pass-by-reference
- Add "RLM Execution Model" cross-cutting section
- Add "Implementation Progress" tracking table
- Remove stale "TO IMPLEMENT" markers (all Phase 3 work is done)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 22:44:47 -07:00
[email protected]andClaude Opus 4.6 953833208e feat(engine): RLM best-practices enhancements from cross-reference analysis
Cross-referenced our implementation against the official RLM (alexzhang13/rlm),
fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation.
Key enhancements:

- FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching
  all three reference implementations. Code can signal completion at any
  point, not just via return value.
- llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn,
  matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch.
- Output truncation increased to 8000 chars (from 120), matching Prime
  Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT].
- Step 0 orientation preamble: auto-injects context metadata (message count,
  total chars, goal, last user message preview) before first code step,
  matching fast-rlm's auto-print pattern.
- Error-to-LLM flow: Python parse errors, runtime errors, NameErrors,
  OS errors, and async errors now flow back as stdout content instead of
  terminating the step, enabling LLM self-correction on next iteration.
  Only VM panics (catch_unwind) terminate as EngineError.

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 22:05:05 -07:00
[email protected]andClaude Opus 4.6 b59a0b9e42 feat(engine): Phase 3 — Monty Python executor with RLM pattern
Add CodeAct execution (Tier 1) using the Monty embedded Python
interpreter, following the Recursive Language Model (RLM) pattern
from arXiv:2512.24601.

Key additions:
- executor/scripting.rs: Monty integration with FunctionCall-based
  tool dispatch, catch_unwind panic safety, resource limits (30s,
  64MB, 1M allocs)
- LlmResponse::Code variant + ExecutionTier::Scripting
- Context-as-variables (RLM 3.4): thread messages, goal, step_number,
  previous_results injected as Python variables — LLM context stays
  lean while code accesses data selectively
- llm_query(prompt, context) (RLM 3.5): recursive subagent calls
  from within Python code — results stored as variables, not injected
  into parent's attention window (symbolic composition)
- Compact output metadata between code steps instead of full stdout
- MontyObject ↔ serde_json::Value bidirectional conversion
- Updated architecture plan with RLM design principles

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 21:32:52 -07:00
[email protected]andClaude Opus 4.6 bf7dfb8c49 feat(engine): Phase 2 — execution loop, capability system, thread runtime
Add the core execution engine to ironclaw_engine crate:

- CapabilityRegistry: register/get/list capabilities and actions
- LeaseManager: async lease lifecycle (grant, check, consume, revoke, expire)
- PolicyEngine: deterministic effect-level allow/deny/approve
- ThreadTree: parent-child relationship tracking
- ThreadSignal/ThreadOutcome: inter-thread messaging via mpsc
- ThreadManager: spawn threads as tokio tasks, stop, inject messages, join
- ExecutionLoop: core loop replacing run_agentic_loop() with signals,
  context building, LLM calls, action execution, and event recording
- Structured executor (Tier 0): lease lookup → policy check → effect execution
- Tool intent nudge detection
- MemoryStore + RetrievalEngine stubs for Phase 4
- Full 8-phase architecture plan in docs/plans/
- CLAUDE.md spec for the engine crate

74 tests passing, zero clippy warnings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 00:16:41 -07:00
[email protected] 8be19a4128 v2 architecture phase 1 2026-03-20 23:32:01 -07:00
577 changed files with 33170 additions and 21204 deletions
+1 -1
View File
@@ -5,7 +5,7 @@ argument-hint: <event_name> [description]
model: opus
---
Add a new SSE event called `$ARGUMENTS` to the OptimClaw web gateway. This involves changes across 5 files in a specific order. Follow each step exactly.
Add a new SSE event called `$ARGUMENTS` to the IronClaw web gateway. This involves changes across 5 files in a specific order. Follow each step exactly.
## Step 1: Add `StatusUpdate` variant
+2 -2
View File
@@ -5,7 +5,7 @@ argument-hint: <tool_name> [description]
model: opus
---
Scaffold a new tool called `$ARGUMENTS` for the OptimClaw agent. First, determine the tool type and then follow the appropriate path.
Scaffold a new tool called `$ARGUMENTS` for the IronClaw agent. First, determine the tool type and then follow the appropriate path.
## Step 0: Determine tool type
@@ -43,7 +43,7 @@ Follow this exact pattern (adjust name and description):
name = "<name>-tool"
version = "0.1.0"
edition = "2021"
description = "<Description> tool for OptimClaw (WASM component)"
description = "<Description> tool for IronClaw (WASM component)"
license = "MIT OR Apache-2.0"
publish = false
+2 -2
View File
@@ -62,7 +62,7 @@ Enter planning mode to design the implementation. The plan MUST cover:
- Happy path (expected input produces expected output)
- Error paths (invalid input, missing data, permission denied)
- Edge cases (empty collections, boundary values, concurrent access)
5. **OptimClaw-specific concerns**:
5. **IronClaw-specific concerns**:
- If the change touches persistence, both database backends must be updated (`postgres.rs` and `libsql_backend.rs`)
- New `Database` trait methods need implementations in both backends
- No `.unwrap()` or `.expect()` in production code
@@ -80,7 +80,7 @@ After the plan is approved:
1. Implement each change from the plan.
2. Write all planned tests.
3. Run OptimClaw's full quality gate:
3. Run IronClaw's full quality gate:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features` (zero warnings)
- `cargo test --lib` (all tests pass)
+3 -3
View File
@@ -96,7 +96,7 @@ Wait for user confirmation (unless `--fix` flag set), then proceed to Phase 3.
Read EVERY changed file in full (not just diff hunks). For PRs touching >20 files, prioritize: service logic > handlers > types > tests > docs. Batch reads in parallel via Agent tool.
### OptimClaw-specific checks (always)
### IronClaw-specific checks (always)
- No `.unwrap()` or `.expect()` in production code
- Prefer `crate::` for cross-module imports (`super::` OK in tests/intra-module)
- Error types use `thiserror`
@@ -167,7 +167,7 @@ gh pr checkout {number}
1. All approved review comment fixes (from Phase 2a)
2. All approved review findings (from Phase 2b)
Follow OptimClaw conventions:
Follow IronClaw conventions:
- `thiserror` for errors
- `crate::` imports
- No `.unwrap()` in production
@@ -180,7 +180,7 @@ After all fixes implemented, proceed to Phase 4.
## Phase 4: Quality Gate
Run the full OptimClaw shipping checklist:
Run the full IronClaw shipping checklist:
```bash
cargo fmt
+2 -2
View File
@@ -60,7 +60,7 @@ Wait for user confirmation before proceeding to implementation.
After user confirms:
1. Implement each fix in the plan.
2. Run OptimClaw's quality gate to verify nothing breaks:
2. Run IronClaw's quality gate to verify nothing breaks:
- `cargo fmt`
- `cargo clippy --all --benches --tests --examples --all-features`
- `cargo test --lib`
@@ -77,5 +77,5 @@ For each comment addressed, reply on the PR with a short message stating what wa
- Group duplicate comments (same issue reported by multiple bots) and reply to all of them.
- Do not make changes beyond what the review comments ask for. Stay focused.
- If a comment suggests a change you disagree with, present your reasoning to the user during the planning phase rather than silently ignoring it.
- Follow OptimClaw conventions: no `.unwrap()` in production code, use `crate::` imports, `thiserror` errors.
- Follow IronClaw conventions: no `.unwrap()` in production code, use `crate::` imports, `thiserror` errors.
- If changes touch persistence, verify both database backends are updated.
+6 -6
View File
@@ -1,5 +1,5 @@
---
description: Deep audit of the OptimClaw crate for vulnerabilities, bugs, unfinished work, inconsistencies, and oversights
description: Deep audit of the IronClaw crate for vulnerabilities, bugs, unfinished work, inconsistencies, and oversights
disable-model-invocation: true
allowed-tools: Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*), Bash(cargo audit:*), Bash(git diff:*), Bash(git log:*), Bash(git show:*), Bash(wc:*), Read, Grep, Glob, Task
argument-hint: "[path/to/crate]"
@@ -93,7 +93,7 @@ Search for `.unwrap()`, `.expect(`, `panic!`, `unreachable!` in non-test code. F
- Is there a code path that reaches this with None/Err?
- Should it be replaced with proper error handling (`?`, `.ok()`, `.unwrap_or_default()`)?
OptimClaw convention: `.unwrap()` and `.expect()` are banned in production code. Any occurrence outside `#[cfg(test)]` blocks is a **High severity** finding.
IronClaw convention: `.unwrap()` and `.expect()` are banned in production code. Any occurrence outside `#[cfg(test)]` blocks is a **High severity** finding.
### 5c. SQL and injection vectors
@@ -102,7 +102,7 @@ Search for string formatting used in SQL queries, shell commands, or HTML:
- String interpolation in query construction vs parameterized queries
- User input flowing into file paths (`Path::new`, `std::fs::`)
OptimClaw has two database backends (PostgreSQL and libSQL). Check both for injection vectors.
IronClaw has two database backends (PostgreSQL and libSQL). Check both for injection vectors.
### 5d. Cryptographic issues
@@ -125,7 +125,7 @@ If the crate uses crypto:
- Are errors swallowed silently? (`let _ = ...`, `.ok()` discarding errors that matter)
- Do error types carry enough context to debug in production?
- Are there error type mismatches? (returning generic `anyhow::Error` where a typed error would prevent confusion)
- Is `thiserror` used consistently for error types (OptimClaw convention)?
- Is `thiserror` used consistently for error types (IronClaw convention)?
## Step 6: Check for inconsistencies
@@ -156,7 +156,7 @@ Look for:
### 6e. Import style
OptimClaw convention: use `crate::` imports, not `super::`. Flag any `super::` imports in non-test code.
IronClaw convention: use `crate::` imports, not `super::`. Flag any `super::` imports in non-test code.
## Step 7: Inspect for change oversights
@@ -172,7 +172,7 @@ OptimClaw convention: use `crate::` imports, not `super::`. Flag any `super::` i
- Are there `impl` blocks that look incomplete?
- Are `Default` implementations sensible?
OptimClaw key traits: `Database` (~60 methods), `Channel`, `Tool`, `LlmProvider`, `SuccessEvaluator`, `EmbeddingProvider`. If any new methods were added to `Database`, verify both `postgres.rs` and `libsql_backend.rs` implement them.
IronClaw key traits: `Database` (~60 methods), `Channel`, `Tool`, `LlmProvider`, `SuccessEvaluator`, `EmbeddingProvider`. If any new methods were added to `Database`, verify both `postgres.rs` and `libsql_backend.rs` implement them.
### 7c. Test coverage gaps
+2 -2
View File
@@ -49,9 +49,9 @@ If the PR touches more than 20 files, still read all of them, but process in thi
Go through the changes with each of these lenses. For every finding, note the file, line range, severity, and a concrete description.
### OptimClaw-specific checks
### IronClaw-specific checks
In addition to the general lenses below, check OptimClaw conventions (see CLAUDE.md):
In addition to the general lenses below, check IronClaw conventions (see CLAUDE.md):
- No `.unwrap()` or `.expect()` in production code (tests are fine)
- Use `crate::` imports, not `super::`
- Error types use `thiserror` in `error.rs`
+1 -1
View File
@@ -3,7 +3,7 @@ description: Run the full Rust quality gate (fmt, clippy, tests) before shipping
allowed-tools: Bash(cargo fmt:*), Bash(cargo clippy:*), Bash(cargo test:*)
---
Run the OptimClaw shipping checklist. This is the mandatory quality gate before any change is considered done.
Run the IronClaw shipping checklist. This is the mandatory quality gate before any change is considered done.
## Steps
+3 -3
View File
@@ -1,15 +1,15 @@
---
description: Trace a data flow or bug through the OptimClaw codebase end-to-end
description: Trace a data flow or bug through the IronClaw codebase end-to-end
allowed-tools: Read, Glob, Grep, Bash(cargo test:*)
argument-hint: <symptom or feature name>
model: sonnet
---
Trace the flow of `$ARGUMENTS` through the OptimClaw codebase. Your job is to map every file and function involved, identify where data transforms or could break, and report the full chain.
Trace the flow of `$ARGUMENTS` through the IronClaw codebase. Your job is to map every file and function involved, identify where data transforms or could break, and report the full chain.
## Architecture Reference
OptimClaw has three main data flow paths. Identify which one(s) are relevant and trace through them:
IronClaw has three main data flow paths. Identify which one(s) are relevant and trace through them:
### Message Flow (user input to LLM response)
```
+2 -2
View File
@@ -11,8 +11,8 @@ SKILL.md files extend the agent's prompt with domain-specific instructions. Each
| Trust Level | Source | Tool Access |
|-------------|--------|-------------|
| **Trusted** | User-placed in `~/.optimclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry (`~/.optimclaw/installed_skills/`) | Read-only tools only (no shell, file write, HTTP) |
| **Trusted** | User-placed in `~/.ironclaw/skills/` or workspace `skills/` | All tools available to the agent |
| **Installed** | Downloaded from ClawHub registry (`~/.ironclaw/installed_skills/`) | Read-only tools only (no shell, file write, HTTP) |
## SKILL.md Format
+1 -1
View File
@@ -7,7 +7,7 @@ paths:
**Keep tool-specific logic out of the main agent codebase.** The main agent provides generic infrastructure; tools are self-contained units that declare requirements through `<name>.capabilities.json` sidecar files (in dev mode: `tools-src/<name>/<name>-tool.capabilities.json`).
Tools can be WASM (sandboxed, credential-injected, single binary) or MCP servers (ecosystem, any language, no sandbox). Both are first-class via `optimclaw tool install`.
Tools can be WASM (sandboxed, credential-injected, single binary) or MCP servers (ecosystem, any language, no sandbox). Both are first-class via `ironclaw tool install`.
See `src/tools/README.md` for full architecture, adding new tools, auth JSON examples, and WASM vs MCP decision guide.
+15 -15
View File
@@ -1,5 +1,5 @@
# Database Configuration
DATABASE_URL=postgres://localhost/optimclaw
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_POOL_SIZE=10
# LLM Provider
@@ -26,19 +26,19 @@ DATABASE_POOL_SIZE=10
# === GitHub Copilot ===
# Uses the OAuth token from your Copilot IDE sign-in (for example
# ~/.config/github-copilot/apps.json on Linux/macOS), or run `optimclaw onboard`
# ~/.config/github-copilot/apps.json on Linux/macOS), or run `ironclaw onboard`
# and choose the GitHub device login flow.
# LLM_BACKEND=github_copilot
# GITHUB_COPILOT_TOKEN=gho_...
# GITHUB_COPILOT_MODEL=gpt-4o
# OptimClaw injects standard VS Code Copilot headers automatically.
# IronClaw injects standard VS Code Copilot headers automatically.
# Optional advanced headers for custom overrides:
# GITHUB_COPILOT_EXTRA_HEADERS=Copilot-Integration-Id:vscode-chat
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
# Session token stored in ~/.optimclaw/session.json automatically.
# Session token stored in ~/.ironclaw/session.json automatically.
# Base URL defaults to https://private.near.ai
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
# Base URL defaults to https://cloud-api.near.ai
@@ -46,7 +46,7 @@ NEARAI_MODEL=Qwen/Qwen3.5-122B-A10B
NEARAI_BASE_URL=https://private.near.ai
NEARAI_AUTH_URL=https://private.near.ai
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
# NEARAI_SESSION_PATH=~/.optimclaw/session.json # optional, default shown
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
# NEARAI_API_KEY=... # API key from cloud.near.ai
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
@@ -63,7 +63,7 @@ NEARAI_AUTH_URL=https://private.near.ai
# LLM_API_KEY=sk-... # optional for local servers
# Custom HTTP headers for OpenAI-compatible providers
# Format: comma-separated key:value pairs
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/optimclaw,X-Title:optimclaw
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
# === OpenRouter (300+ models via OpenAI-compatible) ===
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
@@ -145,7 +145,7 @@ HTTP_HOST=0.0.0.0
HTTP_PORT=8080
HTTP_WEBHOOK_SECRET=your-webhook-secret
# Webhook authentication uses HMAC-SHA256 signature verification.
# Callers must send an X-OptimClaw-Signature header with format: sha256=<hex_digest>
# Callers must send an X-IronClaw-Signature header with format: sha256=<hex_digest>
# where the digest is HMAC-SHA256(HTTP_WEBHOOK_SECRET, raw_request_body) in lowercase hex.
#
# Example (bash):
@@ -153,7 +153,7 @@ HTTP_WEBHOOK_SECRET=your-webhook-secret
# SIG=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "$HTTP_WEBHOOK_SECRET" | cut -d' ' -f2)
# curl -X POST http://localhost:8080/webhook \
# -H "Content-Type: application/json" \
# -H "X-OptimClaw-Signature: sha256=$SIG" \
# -H "X-IronClaw-Signature: sha256=$SIG" \
# -d "$BODY"
#
# DEPRECATED: Passing "secret" in the JSON body still works but will be removed in a future release.
@@ -170,7 +170,7 @@ HTTP_WEBHOOK_SECRET=your-webhook-secret
# SIGNAL_IGNORE_STORIES=true
# Agent Settings
AGENT_NAME=optimclaw
AGENT_NAME=ironclaw
AGENT_MAX_PARALLEL_JOBS=5
AGENT_JOB_TIMEOUT_SECS=3600
AGENT_STUCK_THRESHOLD_SECS=300
@@ -205,7 +205,7 @@ HEARTBEAT_NOTIFY_USER=default
# # commands directly on the host. Without this
# # set to "true", full_access is downgraded to
# # workspace_write.
# SANDBOX_IMAGE=optimclaw-worker:latest
# SANDBOX_IMAGE=ironclaw-worker:latest
# SANDBOX_TIMEOUT_SECS=120
# SANDBOX_MEMORY_LIMIT_MB=2048
@@ -214,11 +214,11 @@ SAFETY_MAX_OUTPUT_LENGTH=100000
SAFETY_INJECTION_CHECK_ENABLED=true
# Restart Feature (Docker containers only)
# Set OPTIMCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
# Without this, the restart tool and /restart command will be disabled.
# OPTIMCLAW_IN_DOCKER=false
# OPTIMCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# OPTIMCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# IRONCLAW_IN_DOCKER=false
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
# Logging
RUST_LOG=optimclaw=debug,tower_http=debug
RUST_LOG=ironclaw=debug,tower_http=debug
+5 -12
View File
@@ -6,7 +6,7 @@
## Change Type
<!-- Check all that apply. Refactor-only PRs are for core team or maintainer-requested work. -->
<!-- Check one -->
- [ ] Bug fix
- [ ] New feature
@@ -18,19 +18,16 @@
## Linked Issue
<!-- Closes #N, Fixes #N, Related #N, or "None". New feature PRs must link an approved issue. -->
<!-- Closes #N, or "None" -->
## Validation
<!-- How did you verify this works? -->
- [ ] `cargo fmt --all -- --check`
- [ ] `cargo clippy --all --benches --tests --examples --all-features -- -D warnings`
- [ ] `cargo build`
- [ ] `cargo fmt`
- [ ] `cargo clippy --all --benches --tests --examples --all-features`
- [ ] Relevant tests pass: <!-- list specific tests -->
- [ ] `cargo test --features integration` if database-backed or integration behavior changed
- [ ] Manual testing: <!-- describe what you tested -->
- [ ] If a coding agent was used and supports it, `review-pr` or `pr-shepherd --fix` was run before requesting review
## Security Impact
@@ -48,10 +45,6 @@
<!-- How to revert if this causes problems? For Track C changes, this is mandatory. -->
## Review Follow-Through
<!-- Review conversations are author-owned. Summarize any known follow-up or areas where reviewer judgment is still needed. -->
---
**Review track**: <!-- A (docs/tests/chore) | B (feature/maintainer-requested refactor) | C (security/runtime/DB/CI) -->
**Review track**: <!-- A (docs/tests/chore) | B (feature/refactor) | C (security/runtime/DB/CI) -->
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
allowed_bots: "optimclaw-ci[bot]"
allowed_bots: "ironclaw-ci[bot]"
claude_args: "--max-turns 50 --model claude-haiku-4-5-20251001 --allowedTools 'Read,Glob,Grep,Agent,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh search:*),Bash(git blame:*),Bash(git log:*),Bash(git diff:*)'"
prompt: |
Code review this pull request. Follow these steps precisely:
+6 -6
View File
@@ -6,11 +6,11 @@
# What it does:
# - Runs unit and integration tests with coverage instrumentation
# - Runs E2E tests with coverage instrumentation
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/optimclaw)
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/ironclaw)
#
# Viewing coverage reports:
# - PRs automatically get coverage comments showing changes in coverage
# - Visit https://codecov.io/gh/nearai/optimclaw for detailed coverage reports
# - Visit https://codecov.io/gh/nearai/ironclaw for detailed coverage reports
# - Coverage reports are generated for three configurations:
# 1. all-features: Full feature set
# 2. default: Default features
@@ -58,7 +58,7 @@ jobs:
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: optimclaw_test
POSTGRES_DB: ironclaw_test
ports:
- 5432:5432
options: >-
@@ -103,11 +103,11 @@ jobs:
PGHOST: localhost
PGUSER: postgres
PGPASSWORD: postgres
PGDATABASE: optimclaw_test
PGDATABASE: ironclaw_test
- name: Set DATABASE_URL for postgres configs
if: matrix.has_postgres
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/optimclaw_test" >> "$GITHUB_ENV"
run: echo "DATABASE_URL=postgres://postgres:postgres@localhost/ironclaw_test" >> "$GITHUB_ENV"
- name: Generate coverage
run: cargo llvm-cov ${{ matrix.flags }} --workspace --lcov --output-path lcov.info
@@ -176,7 +176,7 @@ jobs:
run: |
pytest tests/e2e/ -v --timeout=120
env:
RUST_LOG: optimclaw=info
RUST_LOG: ironclaw=info
RUST_BACKTRACE: "1"
- name: Verify profraw files exist
+5 -5
View File
@@ -14,7 +14,7 @@ on:
jobs:
# ── Step 1: compile once ──────────────────────────────────────────────────
build:
name: Build optimclaw (libsql)
name: Build ironclaw (libsql)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -35,8 +35,8 @@ jobs:
- name: Upload binary
uses: actions/upload-artifact@v4
with:
name: optimclaw-e2e-binary
path: target/debug/optimclaw
name: ironclaw-e2e-binary
path: target/debug/ironclaw
retention-days: 1
# ── Step 2: run test slices in parallel ───────────────────────────────────
@@ -63,11 +63,11 @@ jobs:
- name: Download binary
uses: actions/download-artifact@v4
with:
name: optimclaw-e2e-binary
name: ironclaw-e2e-binary
path: target/debug/
- name: Make binary executable
run: chmod +x target/debug/optimclaw
run: chmod +x target/debug/ironclaw
- uses: actions/setup-python@v5
with:
+1 -1
View File
@@ -56,7 +56,7 @@ jobs:
"src/agent/self_repair.rs"
"src/agent/agentic_loop.rs"
"src/tools/execute.rs"
"crates/optimclaw_safety/src/"
"crates/ironclaw_safety/src/"
)
for pattern in "${HIGH_RISK_PATTERNS[@]}"; do
+2 -2
View File
@@ -166,7 +166,7 @@ jobs:
continue
fi
name=$(echo "$filename" | sed "s/^${kind}-//" | sed 's/-[0-9].*-wasm32-wasip2\.tar\.gz$//')
url="https://github.com/nearai/optimclaw/releases/download/${RELEASE_TAG}/${filename}"
url="https://github.com/nearai/ironclaw/releases/download/${RELEASE_TAG}/${filename}"
manifest="registry/${kind}s/${name}.json"
if [ -f "$manifest" ]; then
@@ -496,7 +496,7 @@ jobs:
continue
fi
name=$(echo "$filename" | sed "s/^${kind}-//" | sed 's/-[0-9].*-wasm32-wasip2\.tar\.gz$//')
url="https://github.com/nearai/optimclaw/releases/download/${RELEASE_TAG}/${filename}"
url="https://github.com/nearai/ironclaw/releases/download/${RELEASE_TAG}/${filename}"
manifest="registry/${kind}s/${name}.json"
if [ -f "$manifest" ]; then
+1 -1
View File
@@ -165,7 +165,7 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v6
- name: Build Docker image
run: docker build -t optimclaw-test:ci .
run: docker build -t ironclaw-test:ci .
version-check:
name: Version Bump Check
+1
View File
@@ -39,3 +39,4 @@ __pycache__/
*.pyc
*.pyo
*.pyd
engine_trace_*.json
+520 -520
View File
File diff suppressed because it is too large Load Diff
+15 -12
View File
@@ -1,6 +1,6 @@
# OptimClaw Development Guide
# IronClaw Development Guide
**OptimClaw** is a secure personal AI assistant — user-first security, self-expanding tools, defense in depth, multi-channel access with proactive background execution.
**IronClaw** is a secure personal AI assistant — user-first security, self-expanding tools, defense in depth, multi-channel access with proactive background execution.
## Build & Test
@@ -9,7 +9,7 @@ cargo fmt # format
cargo clippy --all --benches --tests --examples --all-features # lint (zero warnings)
cargo test # unit tests
cargo test --features integration # + PostgreSQL tests
RUST_LOG=optimclaw=debug cargo run # run with logging
RUST_LOG=ironclaw=debug cargo run # run with logging
```
E2E tests: see `tests/e2e/CLAUDE.md`.
@@ -24,6 +24,8 @@ E2E tests: see `tests/e2e/CLAUDE.md`.
- Prefer strong types over strings (enums, newtypes)
- Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only
- **Prompt templates live in files, not Rust code**: Multi-line prompt strings (mission goals, system prompts, CodeAct preambles) go in `crates/ironclaw_engine/prompts/*.md` and are loaded via `include_str!()`. Never inline large prompt templates as Rust string constants — they're hard to read, review, and iterate on. Single-line format strings are fine inline.
- **Logging levels matter for REPL/TUI**: `info!` and `warn!` output appears in the REPL and corrupts the terminal UI. Use `debug!` for internal diagnostics (trace analysis, reflection results, engine internals). Reserve `info!` for user-facing status that the REPL intentionally renders. Background tasks (reflection, trace analysis) must NEVER use `info!` — it breaks the interactive display.
## Architecture
@@ -35,20 +37,20 @@ All I/O is async with tokio. Use `Arc<T>` for shared state, `RwLock` for concurr
## Extracted Crates
Safety logic lives in `crates/optimclaw_safety/`. The `src/safety/mod.rs` shim re-exports everything for backward compatibility, but **new code should import from `optimclaw_safety` directly** (e.g. `use optimclaw_safety::SafetyLayer`). When touching a file that still uses `crate::safety::*`, migrate its imports to `optimclaw_safety::*`.
Safety logic lives in `crates/ironclaw_safety/`, skills in `crates/ironclaw_skills/`. **Import directly from the extracted crate** (e.g. `use ironclaw_safety::SafetyLayer`, `use ironclaw_skills::SkillRegistry`). Do not use `crate::safety::` or `crate::skills::` for types that originate in extracted crates — `src/safety/mod.rs` and `src/skills/mod.rs` no longer glob-re-export. Local items defined in those modules (e.g. `crate::skills::attenuate_tools`) are fine.
## Project Structure
```
crates/
└── optimclaw_safety/ # Extracted: prompt injection, validation, leak detection, policy
└── ironclaw_safety/ # Extracted: prompt injection, validation, leak detection, policy
src/
├── lib.rs # Library root, module declarations
├── main.rs # Entry point, CLI args, startup
├── app.rs # App startup orchestration (channel wiring, DB init)
├── bootstrap.rs # Base directory resolution (~/.optimclaw), early .env loading
├── settings.rs # User settings persistence (~/.optimclaw/settings.json)
├── bootstrap.rs # Base directory resolution (~/.ironclaw), early .env loading
├── settings.rs # User settings persistence (~/.ironclaw/settings.json)
├── service.rs # OS service management (launchd/systemd daemon install)
├── tracing_fmt.rs # Custom tracing formatter
├── util.rs # Shared utilities
@@ -111,7 +113,7 @@ src/
│ ├── claude_bridge.rs # Claude Code bridge (spawns claude CLI)
│ └── proxy_llm.rs # LlmProvider that proxies through orchestrator
├── safety/ # Re-export shim for crates/optimclaw_safety (see Extracted Crates)
├── safety/ # Re-export shim for crates/ironclaw_safety (see Extracted Crates)
├── llm/ # Multi-provider LLM integration — see src/llm/CLAUDE.md
@@ -191,6 +193,7 @@ When modifying a module with a spec, read the spec first. Code follows spec; spe
| `src/setup/` | `src/setup/README.md` |
| `src/tools/` | `src/tools/README.md` |
| `src/workspace/` | `src/workspace/README.md` |
| `crates/ironclaw_engine/` | `crates/ironclaw_engine/CLAUDE.md` |
| `tests/e2e/` | `tests/e2e/CLAUDE.md` |
## Job State Machine
@@ -206,7 +209,7 @@ Pending -> InProgress -> Completed -> Submitted -> Accepted
SKILL.md files extend the agent's prompt with domain-specific instructions. See `.claude/rules/skills.md` for full details.
- **Trust model**: Trusted (user-placed in `~/.optimclaw/skills/` or workspace `skills/`, full tool access) vs Installed (registry, read-only tools)
- **Trust model**: Trusted (user-placed in `~/.ironclaw/skills/` or workspace `skills/`, full tool access) vs Installed (registry, read-only tools)
- **Selection pipeline**: gating (check bin/env/config requirements) -> scoring (keywords/patterns/tags) -> budget (fit within `SKILLS_MAX_TOKENS`) -> attenuation (trust-based tool ceiling)
- **Skill tools**: `skill_list`, `skill_search`, `skill_install`, `skill_remove`
@@ -228,9 +231,9 @@ Persistent memory with hybrid search (FTS + vector via RRF). Four tools: `memory
## Debugging
```bash
RUST_LOG=optimclaw=trace cargo run # verbose
RUST_LOG=optimclaw::agent=debug cargo run # agent module only
RUST_LOG=optimclaw=debug,tower_http=debug cargo run # + HTTP request logging
RUST_LOG=ironclaw=trace cargo run # verbose
RUST_LOG=ironclaw::agent=debug cargo run # agent module only
RUST_LOG=ironclaw=debug,tower_http=debug cargo run # + HTTP request logging
```
## Current Limitations
+4 -79
View File
@@ -3,49 +3,13 @@
## Getting Started
```bash
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
./scripts/dev-setup.sh
```
This installs the Rust toolchain, WASM targets, git hooks, and runs initial checks.
## How to Contribute
- Bug fixes, docs improvements, and focused cleanup tied to a concrete problem are welcome.
- Search existing issues and PRs before opening a new one to avoid duplicates.
- Keep changes scoped. One bug, one feature, or one documentation improvement per PR.
### Creating Issues
Open an issue when you are reporting a bug, proposing a feature, or documenting a gap in behavior.
For bug reports, include:
- What you expected to happen
- What actually happened
- Clear reproduction steps
- Relevant logs, screenshots, or error output
- Environment details when they matter (OS, database backend, feature flags, commit/branch)
For feature requests:
- Open an issue first before writing code
- Explain the problem being solved, not just the implementation idea
- Wait for maintainer feedback before investing in a large PR
We require an issue for new features so maintainers can prioritize the work and confirm it fits the roadmap before anyone spends time implementing it.
### Fixing Bugs
- Small, targeted bug-fix PRs are welcome
- If there is already an issue, link it in your PR
- If the bug is non-trivial, security-sensitive, or changes behavior across subsystems, open or confirm an issue first so the approach can be aligned before implementation
### Refactor-Only PRs
Refactor-only PRs are not accepted from contributors outside the core team. If a refactor is necessary to land a bug fix or approved feature, keep it minimal and clearly tied to that change.
## Development Workflow
```bash
@@ -55,45 +19,6 @@ cargo test # unit tests
cargo test --features integration # + PostgreSQL tests
```
These commands are for day-to-day iteration while you are developing locally. The pre-submission checks below are intentionally stricter and use CI-style flags so you can catch formatting drift and clippy warnings before requesting review.
## Before You Open a PR
Run the local validation checks required before requesting a review. These are stricter than the commands for iterative development:
```bash
cargo fmt --all -- --check
cargo clippy --all --benches --tests --examples --all-features -- -D warnings
cargo build
cargo test
```
Also run this when your change touches database-backed or integration behavior:
```bash
cargo test --features integration
```
Before asking for review:
- Build and exercise the changed path locally, not just the narrowest unit test
- Keep the PR focused and avoid mixing unrelated concerns
- Fill out the PR template with a clear summary, validation notes, and impact assessment
- If your change affects tracked behavior, update `FEATURE_PARITY.md` in the same branch
- If onboarding or setup behavior changes, update the relevant setup docs in the same branch
- If you are using a coding agent and it supports them, run `review-pr` or `pr-shepherd --fix` before opening or updating the PR
- `codex review --base origin/main` is also encouraged before requesting review
## Review Follow-Through
Review conversations are author-owned.
- Address each review comment with a code change or a clear explanation
- Resolve conversations you have handled; leave them open only when reviewer judgment is still needed
- Do not leave review cleanup for maintainers when the follow-through belongs to the author
If a PR is stale for more than 48 hours after review feedback is posted, maintainers may take over the follow-up work and land the changes needed to accomplish the original PR or issue intent.
## Code Style
- Zero clippy warnings policy
@@ -121,14 +46,14 @@ All PRs follow a risk-based review process:
| Track | Scope | Requirements |
|-------|-------|-------------|
| **A** | Docs, tests, chore, dependency bumps | 1 approval + CI green |
| **B** | Features, maintainer-requested refactors, new tools/channels | 1 approval + CI green + test evidence |
| **B** | Features, refactors, new tools/channels | 1 approval + CI green + test evidence |
| **C** | Security (`src/safety/`, `src/secrets/`), runtime (`src/agent/`, `src/worker/`), database schema, CI workflows | 2 approvals + rollback plan documented |
Select the appropriate track in the PR template based on what your changes touch.
## Database Changes
OptimClaw uses dual-backend persistence (PostgreSQL + libSQL). All new persistence features must support both backends. See `src/db/CLAUDE.md`.
IronClaw uses dual-backend persistence (PostgreSQL + libSQL). All new persistence features must support both backends. See `src/db/CLAUDE.md`.
## Adding Dependencies
+2 -2
View File
@@ -1,6 +1,6 @@
# OptimClaw Coverage Plan: 63.3% to 95%
# IronClaw Coverage Plan: 63.3% to 95%
> Generated 2025-03-06 from [Codecov](https://app.codecov.io/gh/nearai/optimclaw/tree/main/src)
> Generated 2025-03-06 from [Codecov](https://app.codecov.io/gh/nearai/ironclaw/tree/main/src)
## Current State
Generated
+622 -250
View File
File diff suppressed because it is too large Load Diff
+16 -25
View File
@@ -1,5 +1,5 @@
[workspace]
members = [".", "crates/optimclaw_common", "crates/optimclaw_safety"]
members = [".", "crates/ironclaw_common", "crates/ironclaw_safety", "crates/ironclaw_skills", "crates/ironclaw_engine"]
exclude = [
"channels-src/discord",
"channels-src/telegram",
@@ -15,19 +15,19 @@ exclude = [
"tools-src/slack",
"tools-src/telegram",
"fuzz",
"crates/optimclaw_safety/fuzz",
"crates/ironclaw_safety/fuzz",
]
[package]
name = "optimclaw"
name = "ironclaw"
version = "0.22.0"
edition = "2024"
rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/optimclaw"
repository = "https://github.com/nearai/optimclaw"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
[package.metadata.wix]
upgrade-guid = "D0156E61-BA37-451E-8AB9-1A2ECCCFA48F"
@@ -40,7 +40,6 @@ eula = false
tokio = { version = "1", features = ["full"] }
tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3"
tokio-tungstenite = { version = "0.26", features = ["rustls-tls-native-roots"] }
eventsource-stream = "0.2"
# HTTP client
@@ -58,7 +57,6 @@ refinery = { version = "0.8", features = ["tokio-postgres"], optional = true }
tokio-postgres-rustls = { version = "0.13", optional = true }
rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
webpki-roots = { version = "0.26", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
@@ -97,16 +95,18 @@ termimad = "0.34"
# Channel integrations
axum = { version = "0.8", features = ["ws"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["trace", "cors", "set-header", "catch-panic"] }
tower-http = { version = "0.6", features = ["trace", "cors", "set-header"] }
# Cron scheduling for routines
cron = "0.13"
# Shared types
optimclaw_common = { path = "crates/optimclaw_common", version = "0.1.0" }
ironclaw_common = { path = "crates/ironclaw_common", version = "0.1.0" }
# Safety/sanitization
optimclaw_safety = { path = "crates/optimclaw_safety", version = "0.2.0" }
ironclaw_engine = { path = "crates/ironclaw_engine" }
ironclaw_safety = { path = "crates/ironclaw_safety", version = "0.2.0" }
ironclaw_skills = { path = "crates/ironclaw_skills", version = "0.1.0" }
regex = "1"
aho-corasick = "1"
@@ -186,30 +186,22 @@ hex = "0.4.3"
# OpenClaw import (feature gated)
json5 = { version = "0.4", optional = true }
# Mesh cluster (feature gated)
pqcrypto-kyber = { version = "0.8", optional = true }
pqcrypto-traits = { version = "0.3", optional = true }
sys-info = { version = "0.9", optional = true }
hostname = { version = "0.4", optional = true }
quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring"], optional = true }
rcgen = { version = "0.13", optional = true }
# macOS keychain
[target.'cfg(target_os = "macos")'.dependencies]
security-framework = "3"
# PTY allocation for Claude CLI stdout buffering fix (Unix only)
[target.'cfg(unix)'.dependencies]
pty-process = { version = "0.5", features = ["async"] }
# Linux secret-service (GNOME Keyring, KWallet)
[target.'cfg(target_os = "linux")'.dependencies]
secret-service = { version = "4", features = ["rt-tokio-crypto-rust"] }
zbus = "4"
[build-dependencies]
serde_json = "1"
[dev-dependencies]
tokio-test = "0.4"
tracing-test = "0.2"
tokio-tungstenite = "0.26"
testcontainers-modules = { version = "0.11", features = ["postgres"] }
pretty_assertions = "1"
tempfile = "3"
@@ -232,7 +224,6 @@ postgres = [
"dep:tokio-postgres-rustls",
"dep:rustls",
"dep:rustls-native-certs",
"dep:webpki-roots",
"dep:postgres-types",
"dep:refinery",
"dep:pgvector",
@@ -245,7 +236,6 @@ integration = []
html-to-markdown = ["dep:html-to-markdown-rs", "dep:readabilityrs"]
bedrock = ["dep:aws-config", "dep:aws-sdk-bedrockruntime", "dep:aws-smithy-types"]
import = ["dep:json5", "libsql"]
cluster = ["dep:pqcrypto-kyber", "dep:pqcrypto-traits", "dep:sys-info", "dep:hostname", "dep:quinn", "dep:rcgen", "dep:rustls"]
[[test]]
name = "e2e_thread_scheduling"
@@ -261,7 +251,8 @@ strip = true # Remove debug symbols from release binaries
# The profile that 'cargo dist' will build with
[profile.dist]
inherits = "release"
lto = "thin"
lto = "fat" # Full cross-crate LTO (slow build, better codegen)
codegen-units = 1 # Single codegen unit for maximum optimization
# Config for 'dist'
[workspace.metadata.dist]
+8 -34
View File
@@ -1,71 +1,45 @@
# Multi-stage Dockerfile for the IronClaw agent (cloud deployment).
#
# Uses cargo-chef for dependency caching — only rebuilds deps when
# Cargo.toml/Cargo.lock change, not on every source edit.
#
# Build:
# docker build --platform linux/amd64 -t ironclaw:latest .
#
# Run:
# docker run --env-file .env -p 3000:3000 ironclaw:latest
# Stage 1: Install cargo-chef
FROM rust:1.92-slim-bookworm AS chef
# Stage 1: Build
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/* \
&& rustup target add wasm32-wasip2 \
&& cargo install cargo-chef wasm-tools
&& cargo install wasm-tools
WORKDIR /app
# Stage 2: Generate the dependency recipe (changes only when Cargo.toml/lock change)
FROM chef AS planner
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
# Copy source, build script, tests, and supporting directories
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
COPY benches/ benches/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
RUN cargo chef prepare --recipe-path recipe.json
# Stage 3: Build dependencies (cached unless Cargo.toml/lock change)
FROM chef AS deps
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json
# Stage 4: Build the actual binary (only recompiles ironclaw source)
FROM deps AS builder
COPY Cargo.toml Cargo.lock ./
COPY crates/ crates/
COPY build.rs build.rs
COPY src/ src/
COPY tests/ tests/
# [[bench]] entries in Cargo.toml require bench sources to exist for cargo to parse the manifest
COPY benches/ benches/
COPY migrations/ migrations/
COPY registry/ registry/
COPY channels-src/ channels-src/
COPY wit/ wit/
COPY providers.json providers.json
RUN cargo build --release --bin ironclaw
# Stage 5: Runtime
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& update-ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
+38 -38
View File
@@ -1,6 +1,6 @@
# OptimClaw ↔ OpenClaw Feature Parity Matrix
# IronClaw ↔ OpenClaw Feature Parity Matrix
This document tracks feature parity between OptimClaw (Rust implementation) and OpenClaw (TypeScript reference implementation). Use this to coordinate work across developers.
This document tracks feature parity between IronClaw (Rust implementation) and OpenClaw (TypeScript reference implementation). Use this to coordinate work across developers.
**Legend:**
@@ -17,7 +17,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 1. Architecture
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Hub-and-spoke architecture | ✅ | ✅ | Web gateway as central hub |
| WebSocket control plane | ✅ | ✅ | Gateway with WebSocket + SSE |
@@ -32,7 +32,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 2. Gateway System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Gateway control plane | ✅ | ✅ | Web gateway with 40+ API endpoints |
| HTTP endpoints for Control UI | ✅ | ✅ | Web dashboard with chat, memory, jobs, logs, extensions |
@@ -62,15 +62,15 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 3. Messaging Channels
| Channel | OpenClaw | OptimClaw | Priority | Notes |
| Channel | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| CLI/TUI | ✅ | ✅ | - | Ratatui-based TUI |
| HTTP webhook | ✅ | ✅ | - | axum with secret validation |
| REPL (simple) | ✅ | ✅ | - | For testing |
| WASM channels | ❌ | ✅ | - | OptimClaw innovation; host resolves owner scope vs sender identity |
| WASM channels | ❌ | ✅ | - | IronClaw innovation; host resolves owner scope vs sender identity |
| WhatsApp | ✅ | ❌ | P1 | Baileys (Web), same-phone mode with echo detection |
| Telegram | ✅ | ✅ | - | WASM channel(MTProto), DM pairing, caption, /start, bot_username, DM topics, setup-time owner auto-verification, owner-scoped persistence |
| Discord | ✅ | 🚧 | P2 | Gateway `MESSAGE_CREATE` intake restored via websocket queue + WASM poll; Gateway DMs now respect pairing; thread parent binding inheritance and reply/thread parity still incomplete |
| Discord | ✅ | | P2 | discord.js, thread parent binding inheritance |
| Signal | ✅ | ✅ | P2 | signal-cli daemonPC, SSE listener HTTP/JSON-R, user/group allowlists, DM pairing |
| Slack | ✅ | ✅ | - | WASM tool |
| iMessage | ✅ | ❌ | P3 | BlueBubbles or Linq recommended |
@@ -88,7 +88,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
### Telegram-Specific Features (since Feb 2025)
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Forum topic creation | ✅ | ❌ | Create topics in forum groups |
| channel_post support | ✅ | ❌ | Bot-to-bot communication |
@@ -100,7 +100,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
### Discord-Specific Features (since Feb 2025)
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Forwarded attachment downloads | ✅ | ❌ | Fetch media from forwarded messages |
| Faster reaction state machine | ✅ | ❌ | Watchdog + debounce |
@@ -108,7 +108,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
### Slack-Specific Features (since Feb 2025)
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Streaming draft replies | ✅ | ❌ | Partial replies via draft message updates |
| Configurable stream modes | ✅ | ❌ | Per-channel stream behavior |
@@ -117,23 +117,23 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
### Mattermost-Specific Features (since Mar 2026)
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Interactive buttons | ✅ | ❌ | Clickable message buttons with signed callback flow |
| Interactive model picker | ✅ | ❌ | In-channel provider/model chooser |
### Feishu/Lark-Specific Features (since Mar 2026)
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Doc/table actions | ✅ | ❌ | `feishu_doc` supports tables, positional insert, color_text, image upload, and file upload |
| Rich-text embedded media extraction | ✅ | ❌ | Pull video/media attachments from post messages |
### Channel Features
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| DM pairing codes | ✅ | ✅ | `optimclaw pairing list/approve`, host APIs |
| DM pairing codes | ✅ | ✅ | `ironclaw pairing list/approve`, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | `allow_from` + pairing store + hardened command/group allowlists |
| Self-message bypass | ✅ | ❌ | Own messages skip pairing |
| Mention-based activation | ✅ | ✅ | bot_username + respond_to_all_group_messages |
@@ -151,7 +151,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 4. CLI Commands
| Command | OpenClaw | OptimClaw | Priority | Notes |
| Command | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| `run` (agent) | ✅ | ✅ | - | Default command |
| `tool install/list/remove` | ✅ | ✅ | - | WASM tools |
@@ -189,9 +189,9 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 5. Agent System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Pi agent runtime | ✅ | | OptimClaw uses custom runtime |
| Pi agent runtime | ✅ | | IronClaw uses custom runtime |
| RPC-based execution | ✅ | ✅ | Orchestrator/worker pattern |
| Multi-provider failover | ✅ | ✅ | `FailoverProvider` tries providers sequentially on retryable errors |
| Per-sender sessions | ✅ | ✅ | |
@@ -232,7 +232,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 6. Model & Provider Support
| Provider | OpenClaw | OptimClaw | Priority | Notes |
| Provider | OpenClaw | IronClaw | Priority | Notes |
|----------|----------|----------|----------|-------|
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6, adaptive thinking default |
@@ -245,7 +245,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (OptimClaw-only) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
| GitHub Copilot | ✅ | ✅ | - | Dedicated provider with OAuth token exchange (`GithubCopilotProvider`) |
| Ollama (local) | ✅ | ✅ | - | via `rig::providers::ollama` (full support) |
@@ -257,7 +257,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
### Model Features
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Auto-discovery | ✅ | ❌ | |
| Failover chains | ✅ | ✅ | `FailoverProvider` with configurable `fallback_model` |
@@ -273,7 +273,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 7. Media Handling
| Feature | OpenClaw | OptimClaw | Priority | Notes |
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Image processing (Sharp) | ✅ | ❌ | P2 | Resize, format convert |
| Configurable image resize dims | ✅ | ❌ | P2 | Per-agent dimension config |
@@ -296,12 +296,12 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 8. Plugin & Extension System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Dynamic loading | ✅ | ✅ | WASM modules |
| Manifest validation | ✅ | ✅ | WASM metadata |
| HTTP path registration | ✅ | ❌ | Plugin routes |
| Workspace-relative install | ✅ | ✅ | ~/.optimclaw/tools/ |
| Workspace-relative install | ✅ | ✅ | ~/.ironclaw/tools/ |
| Channel plugins | ✅ | ✅ | WASM channels |
| Auth plugins | ✅ | ❌ | |
| Memory plugins | ✅ | ❌ | Custom backends + selectable memory slot |
@@ -321,7 +321,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 9. Configuration System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Primary config file | ✅ `~/.openclaw/openclaw.json` | ✅ `.env` | Different formats |
| JSON5 support | ✅ | ❌ | Comments, trailing commas |
@@ -330,7 +330,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
| Config validation/schema | ✅ | ✅ | Type-safe Config struct + `openclaw config validate` |
| Hot-reload | ✅ | ❌ | |
| Legacy migration | ✅ | | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.optimclaw/` | |
| State directory | ✅ `~/.openclaw-state/` | ✅ `~/.ironclaw/` | |
| Credentials directory | ✅ | ✅ | Session files |
| Full model compat fields in schema | ✅ | ❌ | pi-ai model compat exposed in config |
@@ -340,7 +340,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 10. Memory & Knowledge System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Vector memory | ✅ | ✅ | pgvector |
| Session-based memory | ✅ | ✅ | |
@@ -351,7 +351,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
| OpenAI embeddings | ✅ | ✅ | |
| Gemini embeddings | ✅ | ❌ | |
| Local embeddings | ✅ | ❌ | |
| SQLite-vec backend | ✅ | ❌ | OptimClaw uses PostgreSQL |
| SQLite-vec backend | ✅ | ❌ | IronClaw uses PostgreSQL |
| LanceDB backend | ✅ | ❌ | Configurable auto-capture max length |
| QMD backend | ✅ | ❌ | |
| Atomic reindexing | ✅ | ✅ | |
@@ -369,7 +369,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 11. Mobile Apps
| Feature | OpenClaw | OptimClaw | Priority | Notes |
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| iOS app (SwiftUI) | ✅ | 🚫 | - | Out of scope initially |
| Android app (Kotlin) | ✅ | 🚫 | - | Out of scope initially |
@@ -390,7 +390,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 12. macOS App
| Feature | OpenClaw | OptimClaw | Priority | Notes |
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| SwiftUI native app | ✅ | 🚫 | - | Out of scope |
| Menu bar presence | ✅ | 🚫 | - | Animated menubar icon |
@@ -411,7 +411,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 13. Web Interface
| Feature | OpenClaw | OptimClaw | Priority | Notes |
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Control UI Dashboard | ✅ | ✅ | - | Web gateway with chat, memory, jobs, logs, extensions |
| Channel status view | ✅ | 🚧 | P2 | Gateway status widget, full channel view pending |
@@ -431,7 +431,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 14. Automation
| Feature | OpenClaw | OptimClaw | Priority | Notes |
| Feature | OpenClaw | IronClaw | Priority | Notes |
|---------|----------|----------|----------|-------|
| Cron jobs | ✅ | ✅ | - | Routines with cron trigger |
| Per-job model fallback override | ✅ | ❌ | P2 | `payload.fallbacks` overrides agent-level fallbacks |
@@ -465,14 +465,14 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 15. Security Features
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Gateway token auth | ✅ | ✅ | Bearer token auth on web gateway |
| Device pairing | ✅ | ❌ | |
| Tailscale identity | ✅ | ❌ | |
| Trusted-proxy auth | ✅ | ❌ | Header-based reverse proxy auth |
| OAuth flows | ✅ | 🚧 | NEAR AI OAuth + Gemini OAuth (PKCE, S256) + hosted extension/MCP OAuth broker; external auth-proxy rollout still pending |
| DM pairing verification | ✅ | ✅ | optimclaw pairing approve, host APIs |
| DM pairing verification | ✅ | ✅ | ironclaw pairing approve, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| Per-group tool policies | ✅ | ❌ | |
| Exec approvals | ✅ | ✅ | TUI overlay |
@@ -483,7 +483,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
| Loopback-first | ✅ | 🚧 | HTTP binds 0.0.0.0 |
| Docker sandbox | ✅ | ✅ | Orchestrator/worker containers |
| Podman support | ✅ | ❌ | Alternative to Docker |
| WASM sandbox | ❌ | ✅ | OptimClaw innovation |
| WASM sandbox | ❌ | ✅ | IronClaw innovation |
| Sandbox env sanitization | ✅ | 🚧 | Shell tool scrubs env vars (secret detection); docker container env sanitization partial |
| Tool policies | ✅ | ✅ | |
| Elevated mode | ✅ | ❌ | |
@@ -505,7 +505,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## 16. Development & Build System
| Feature | OpenClaw | OptimClaw | Notes |
| Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------|
| Primary language | TypeScript | Rust | Different ecosystems |
| Build tool | tsdown | cargo | |
@@ -531,7 +531,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
- ✅ TUI channel with approval overlays
- ✅ HTTP webhook channel
- ✅ DM pairing (optimclaw pairing list/approve, host APIs)
- ✅ DM pairing (ironclaw pairing list/approve, host APIs)
- ✅ WASM tool sandbox
- ✅ Workspace/memory with hybrid search + embeddings batching
- ✅ Prompt injection defense
@@ -605,7 +605,7 @@ This document tracks feature parity between OptimClaw (Rust implementation) and
## Deviations from OpenClaw
OptimClaw intentionally differs from OpenClaw in these ways:
IronClaw intentionally differs from OpenClaw in these ways:
1. **Rust vs TypeScript**: Native performance, memory safety, single binary distribution
2. **WASM sandbox vs Docker**: Lighter weight, faster startup, capability-based security
@@ -613,7 +613,7 @@ OptimClaw intentionally differs from OpenClaw in these ways:
4. **NEAR AI focus**: Primary provider with session-based auth
5. **No mobile/desktop apps**: Focus on server-side and CLI initially
6. **WASM channels**: Novel extension mechanism not in OpenClaw
7. **Tinfoil private inference**: OptimClaw-only provider for private/encrypted inference
7. **Tinfoil private inference**: IronClaw-only provider for private/encrypted inference
8. **GitHub WASM tool**: Native GitHub integration as WASM tool
9. **Prompt-based skills**: Different approach than OpenClaw capability bundles (trust gating, attenuation)
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">OptimClaw</h1>
<h1 align="center">IronClaw</h1>
<p align="center">
<strong>あなたの味方になる、安全なパーソナルAIアシスタント</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## フィロソフィー
OptimClawはシンプルな原則に基づいて構築されています:**あなたのAIアシスタントは、あなたのために働くべきであり、あなたに不利益をもたらすべきではありません。**
IronClawはシンプルな原則に基づいて構築されています:**あなたのAIアシスタントは、あなたのために働くべきであり、あなたに不利益をもたらすべきではありません。**
AIシステムがデータの取り扱いについて不透明になり、企業の利益に沿って調整されることが増えている世界で、OptimClawは異なるアプローチを取ります:
AIシステムがデータの取り扱いについて不透明になり、企業の利益に沿って調整されることが増えている世界で、IronClawは異なるアプローチを取ります:
- **あなたのデータはあなたのもの** - すべての情報はローカルに保存・暗号化され、あなたの管理下から離れることはありません
- **設計段階からの透明性** - オープンソース、監査可能、隠れたテレメトリやデータ収集なし
- **自己拡張する能力** - ベンダーのアップデートを待たずに、新しいツールをその場で構築
- **多層防御** - 複数のセキュリティレイヤーがプロンプトインジェクションやデータ流出から保護
OptimClawは、個人生活にも仕事にも本当に信頼できるAIアシスタントです。
IronClawは、個人生活にも仕事にも本当に信頼できるAIアシスタントです。
## 機能
@@ -66,7 +66,7 @@ OptimClawは、個人生活にも仕事にも本当に信頼できるAIアシス
### 自己拡張
- **動的ツール構築** - 必要なものを説明すると、OptimClawがWASMツールとして構築
- **動的ツール構築** - 必要なものを説明すると、IronClawがWASMツールとして構築
- **MCPプロトコル** - Model Context Protocolサーバーに接続して追加機能を利用
- **プラグインアーキテクチャ** - 再起動なしで新しいWASMツールやチャネルを追加
@@ -86,12 +86,12 @@ OptimClawは、個人生活にも仕事にも本当に信頼できるAIアシス
## ダウンロードまたはビルド
最新のアップデートは[リリースページ](https://github.com/nearai/optimclaw/releases/)をご覧ください。
最新のアップデートは[リリースページ](https://github.com/nearai/ironclaw/releases/)をご覧ください。
<details>
<summary>Windowsインストーラーでインストール(Windows</summary>
[Windowsインストーラー](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi)をダウンロードして実行してください。
[Windowsインストーラー](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi)をダウンロードして実行してください。
</details>
@@ -99,7 +99,7 @@ OptimClawは、個人生活にも仕事にも本当に信頼できるAIアシス
<summary>PowerShellスクリプトでインストール(Windows</summary>
```sh
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-insta
<summary>シェルスクリプトでインストール(macOS、Linux、Windows/WSL</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releas
<summary>Homebrewでインストール(macOS/Linux</summary>
```sh
brew install optimclaw
brew install ironclaw
```
</details>
@@ -128,8 +128,8 @@ brew install optimclaw
```bash
# リポジトリをクローン
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
# ビルド
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# データベースを作成
createdb optimclaw
createdb ironclaw
# pgvectorを有効化
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## 設定
セットアップウィザードを実行してOptimClawを設定します:
セットアップウィザードを実行してIronClawを設定します:
```bash
optimclaw onboard
ironclaw onboard
```
ウィザードは、データベース接続、NEAR AI認証(ブラウザOAuth経由)、シークレットの暗号化(システムキーチェーンを使用)を処理します。設定は接続されたデータベースに永続化されます。ブートストラップ変数(例:`DATABASE_URL``LLM_BACKEND`)は、データベース接続前に利用できるよう`~/.optimclaw/.env`に書き込まれます。
ウィザードは、データベース接続、NEAR AI認証(ブラウザOAuth経由)、シークレットの暗号化(システムキーチェーンを使用)を処理します。設定は接続されたデータベースに永続化されます。ブートストラップ変数(例:`DATABASE_URL``LLM_BACKEND`)は、データベース接続前に利用できるよう`~/.ironclaw/.env`に書き込まれます。
### 代替LLMプロバイダー
OptimClawはデフォルトでNEAR AIを使用しますが、多くのLLMプロバイダーをすぐに利用できます。組み込みプロバイダーには**Anthropic**、**OpenAI**、**Google Gemini**、**MiniMax**、**Mistral**、**Ollama**(ローカル)が含まれます。**OpenRouter**300以上のモデル)、**Together AI**、**Fireworks AI**、セルフホストサーバー(**vLLM**、**LiteLLM**)などのOpenAI互換サービスもサポートされています。
IronClawはデフォルトでNEAR AIを使用しますが、多くのLLMプロバイダーをすぐに利用できます。組み込みプロバイダーには**Anthropic**、**OpenAI**、**Google Gemini**、**MiniMax**、**Mistral**、**Ollama**(ローカル)が含まれます。**OpenRouter**300以上のモデル)、**Together AI**、**Fireworks AI**、セルフホストサーバー(**vLLM**、**LiteLLM**)などのOpenAI互換サービスもサポートされています。
ウィザードでプロバイダーを選択するか、環境変数を直接設定してください:
@@ -184,7 +184,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## セキュリティ
OptimClawは、データを保護し悪用を防ぐために多層防御を実装しています。
IronClawは、データを保護し悪用を防ぐために多層防御を実装しています。
### WASMサンドボックス
@@ -280,13 +280,13 @@ WASM ──► 許可リスト ──► リーク ──► 認証情報 ─
```bash
# 初回セットアップ(データベース、認証などを設定)
optimclaw onboard
ironclaw onboard
# インタラクティブREPLを起動
cargo run
# デバッグログ付き
RUST_LOG=optimclaw=debug cargo run
RUST_LOG=ironclaw=debug cargo run
```
## 開発
@@ -299,7 +299,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# テスト実行
createdb optimclaw_test
createdb ironclaw_test
cargo test
# 特定のテストを実行
@@ -311,7 +311,7 @@ cargo test test_name
## OpenClawの系譜
OptimClawは[OpenClaw](https://github.com/openclaw/openclaw)にインスパイアされたRust再実装です。完全な対応表は[FEATURE_PARITY.md](FEATURE_PARITY.md)をご覧ください。
IronClawは[OpenClaw](https://github.com/openclaw/openclaw)にインスパイアされたRust再実装です。完全な対応表は[FEATURE_PARITY.md](FEATURE_PARITY.md)をご覧ください。
主な違い:
+28 -74
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">OptimClaw</h1>
<h1 align="center">IronClaw</h1>
<p align="center">
<strong>Your secure personal AI assistant, always on your side</strong>
@@ -10,10 +10,10 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
<a href="https://gitcgr.com/nearai/optimclaw">
<img src="https://gitcgr.com/badge/nearai/optimclaw.svg" alt="gitcgr" />
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
<a href="https://gitcgr.com/nearai/ironclaw">
<img src="https://gitcgr.com/badge/nearai/ironclaw.svg" alt="gitcgr" />
</a>
</p>
@@ -27,8 +27,6 @@
<p align="center">
<a href="#philosophy">Philosophy</a> •
<a href="#features">Features</a> •
<a href="#mesh-cluster">Mesh Cluster</a> •
<a href="#lazy-tools">Lazy Tools</a> •
<a href="#installation">Installation</a> •
<a href="#configuration">Configuration</a> •
<a href="#security">Security</a> •
@@ -39,16 +37,16 @@
## Philosophy
OptimClaw is built on a simple principle: **your AI assistant should work for you, not against you**.
IronClaw is built on a simple principle: **your AI assistant should work for you, not against you**.
In a world where AI systems are increasingly opaque about data handling and aligned with corporate interests, OptimClaw takes a different approach:
In a world where AI systems are increasingly opaque about data handling and aligned with corporate interests, IronClaw takes a different approach:
- **Your data stays yours** - All information is stored locally, encrypted, and never leaves your control
- **Transparency by design** - Open source, auditable, no hidden telemetry or data harvesting
- **Self-expanding capabilities** - Build new tools on the fly without waiting for vendor updates
- **Defense in depth** - Multiple security layers protect against prompt injection and data exfiltration
OptimClaw is the AI assistant you can actually trust with your personal and professional life.
IronClaw is the AI assistant you can actually trust with your personal and professional life.
## Features
@@ -71,7 +69,7 @@ OptimClaw is the AI assistant you can actually trust with your personal and prof
### Self-Expanding
- **Dynamic Tool Building** - Describe what you need, and OptimClaw builds it as a WASM tool
- **Dynamic Tool Building** - Describe what you need, and IronClaw builds it as a WASM tool
- **MCP Protocol** - Connect to Model Context Protocol servers for additional capabilities
- **Plugin Architecture** - Drop in new WASM tools and channels without restarting
@@ -81,50 +79,6 @@ OptimClaw is the AI assistant you can actually trust with your personal and prof
- **Workspace Filesystem** - Flexible path-based storage for notes, logs, and context
- **Identity Files** - Maintain consistent personality and preferences across sessions
## Mesh Cluster
OptimClaw instances can form an **autonomous AI mesh network** where nodes discover each other automatically, coordinate via a gossip protocol, and route tasks intelligently across the cluster.
Key highlights:
- **Zero-config discovery** -- UDP beacon broadcast finds peers on the local network automatically
- **Post-quantum encryption** -- ML-KEM-768 key exchange with AES-256-GCM authenticated encryption protects all inter-node traffic against both classical and quantum adversaries
- **SWIM gossip membership** -- Reliable failure detection and cluster state convergence in O(log N) rounds
- **Intelligent task routing** -- A scoring algorithm balances load, latency, capability match, session affinity, and region locality to pick the best node for each task
- **Graceful degradation** -- Nodes operate independently if connectivity is lost; no split-brain data corruption
Quick start (two nodes on one machine):
```bash
# Terminal 1
export CLUSTER_ENABLED=true CLUSTER_SECRET="your-32-char-secret-here-change-me" CLUSTER_NODE_ID=node-a
cargo run
# Terminal 2
export CLUSTER_ENABLED=true CLUSTER_SECRET="your-32-char-secret-here-change-me" CLUSTER_NODE_ID=node-b CLUSTER_BIND_PORT=9410
cargo run
```
Monitor via `GET /api/mesh/status` and `GET /api/mesh/nodes`.
See [docs/MESH_CLUSTER.md](docs/MESH_CLUSTER.md) for the full guide covering architecture, configuration reference, security model, and troubleshooting.
## Lazy Tools
Lazy tool loading reduces the system prompt from approximately 13,000 tokens to approximately 4,000 tokens by deferring tool schemas that are not immediately needed.
Enable it with:
```bash
export OPTIMCLAW_LAZY_TOOLS=1
```
When enabled, 12 core tools (echo, time, json, http, web_fetch, file_read, file_write, shell, memory_search, memory_write, message, tool_info) are loaded eagerly. All other tools -- including MCP, WASM, and skill tools -- are listed by name only. The LLM calls `tool_info` to load the full schema for any additional tool on demand.
This is recommended for production deployments and cost-sensitive usage with expensive models.
See [docs/LAZY_TOOLS.md](docs/LAZY_TOOLS.md) for the full guide.
## Installation
### Prerequisites
@@ -135,12 +89,12 @@ See [docs/LAZY_TOOLS.md](docs/LAZY_TOOLS.md) for the full guide.
## Download or Build
Visit [Releases page](https://github.com/nearai/optimclaw/releases/) to see the latest updates.
Visit [Releases page](https://github.com/nearai/ironclaw/releases/) to see the latest updates.
<details>
<summary>Install via Windows Installer (Windows)</summary>
Download the [Windows Installer](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) and run it.
Download the [Windows Installer](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) and run it.
</details>
@@ -148,7 +102,7 @@ Download the [Windows Installer](https://github.com/nearai/optimclaw/releases/la
<summary>Install via powershell script (Windows)</summary>
```sh
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
@@ -157,7 +111,7 @@ irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-insta
<summary>Install via shell script (macOS, Linux, Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
@@ -165,7 +119,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releas
<summary>Install via Homebrew (macOS/Linux)</summary>
```sh
brew install optimclaw
brew install ironclaw
```
</details>
@@ -177,8 +131,8 @@ Install it with `cargo`, just make sure you have [Rust](https://rustup.rs) insta
```bash
# Clone the repository
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
# Build
cargo build --release
@@ -195,28 +149,28 @@ For **full release** (after modifying channel sources), run `./scripts/build-all
```bash
# Create database
createdb optimclaw
createdb ironclaw
# Enable pgvector
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## Configuration
Run the setup wizard to configure OptimClaw:
Run the setup wizard to configure IronClaw:
```bash
optimclaw onboard
ironclaw onboard
```
The wizard handles database connection, NEAR AI authentication (via browser OAuth),
and secrets encryption (using your system keychain). Settings are persisted in the
connected database; bootstrap variables (e.g. `DATABASE_URL`, `LLM_BACKEND`) are
written to `~/.optimclaw/.env` so they are available before the database connects.
written to `~/.ironclaw/.env` so they are available before the database connects.
### Alternative LLM Providers
OptimClaw defaults to NEAR AI but supports many LLM providers out of the box.
IronClaw defaults to NEAR AI but supports many LLM providers out of the box.
Built-in providers include **Anthropic**, **OpenAI**, **GitHub Copilot**, **Google Gemini**, **MiniMax**,
**Mistral**, and **Ollama** (local). OpenAI-compatible services like **OpenRouter**
(300+ models), **Together AI**, **Fireworks AI**, and self-hosted servers (**vLLM**,
@@ -240,7 +194,7 @@ See [docs/LLM_PROVIDERS.md](docs/LLM_PROVIDERS.md) for a full provider guide.
## Security
OptimClaw implements defense in depth to protect your data and prevent misuse.
IronClaw implements defense in depth to protect your data and prevent misuse.
### WASM Sandbox
@@ -333,13 +287,13 @@ External content passes through multiple security layers:
```bash
# First-time setup (configures database, auth, etc.)
optimclaw onboard
ironclaw onboard
# Start interactive REPL
cargo run
# With debug logging
RUST_LOG=optimclaw=debug cargo run
RUST_LOG=ironclaw=debug cargo run
```
## Development
@@ -352,7 +306,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# Run tests
createdb optimclaw_test
createdb ironclaw_test
cargo test
# Run specific test
@@ -364,7 +318,7 @@ cargo test test_name
## OpenClaw Heritage
OptimClaw is a Rust reimplementation inspired by [OpenClaw](https://github.com/openclaw/openclaw). See [FEATURE_PARITY.md](FEATURE_PARITY.md) for the complete tracking matrix.
IronClaw is a Rust reimplementation inspired by [OpenClaw](https://github.com/openclaw/openclaw). See [FEATURE_PARITY.md](FEATURE_PARITY.md) for the complete tracking matrix.
Key differences:
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">OptimClaw</h1>
<h1 align="center">IronClaw</h1>
<p align="center">
<strong>Ваш защищенный персональный AI-ассистент, всегда на вашей стороне</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="Лицензия: MIT OR Apache-2.0" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## Философия
OptimClaw построен на простом принципе: **ваш AI-ассистент должен работать на вас, а не против вас**.
IronClaw построен на простом принципе: **ваш AI-ассистент должен работать на вас, а не против вас**.
В мире, где системы ИИ становятся все более непрозрачными в вопросах обработки данных и ориентируются на корпоративные интересы, OptimClaw выбирает другой путь:
В мире, где системы ИИ становятся все более непрозрачными в вопросах обработки данных и ориентируются на корпоративные интересы, IronClaw выбирает другой путь:
- **Ваши данные остаются вашими** — вся информация хранится локально, зашифрована и никогда не покидает ваш контроль.
- **Прозрачность по умолчанию** — открытый исходный код, возможность аудита, отсутствие скрытой телеметрии или сбора данных.
- **Саморасширяемые возможности** — создавайте новые инструменты «на лету», не дожидаясь обновлений от вендора.
- **Глубокая защита** — несколько уровней безопасности защищают от инъекций промптов и утечки данных.
OptimClaw — это AI-ассистент, которому вы действительно можете доверять в личной и профессиональной жизни.
IronClaw — это AI-ассистент, которому вы действительно можете доверять в личной и профессиональной жизни.
## Возможности
@@ -66,7 +66,7 @@ OptimClaw — это AI-ассистент, которому вы действи
### Саморасширяемый
- **Динамическое создание инструментов** — опишите, что вам нужно, и OptimClaw создаст это как инструмент WASM.
- **Динамическое создание инструментов** — опишите, что вам нужно, и IronClaw создаст это как инструмент WASM.
- **Протокол MCP** — подключайтесь к серверам Model Context Protocol для получения дополнительных возможностей.
- **Плагинная архитектура** — добавляйте новые инструменты WASM и каналы без перезагрузки системы.
@@ -86,12 +86,12 @@ OptimClaw — это AI-ассистент, которому вы действи
## Загрузка и сборка
Посетите [страницу релизов](https://github.com/nearai/optimclaw/releases/), чтобы увидеть последние обновления.
Посетите [страницу релизов](https://github.com/nearai/ironclaw/releases/), чтобы увидеть последние обновления.
<details>
<summary>Установка через установщик Windows (Windows)</summary>
Загрузите [Windows Installer](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) и запустите его.
Загрузите [Windows Installer](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) и запустите его.
</details>
@@ -99,7 +99,7 @@ OptimClaw — это AI-ассистент, которому вы действи
<summary>Установка через powershell-скрипт (Windows)</summary>
```sh
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-insta
<summary>Установка через shell-скрипт (macOS, Linux, Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releas
<summary>Установка через Homebrew (macOS/Linux)</summary>
```sh
brew install optimclaw
brew install ironclaw
```
</details>
@@ -128,8 +128,8 @@ brew install optimclaw
```bash
# Клонируйте репозиторий
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
# Сборка
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# Создание базы данных
createdb optimclaw
createdb ironclaw
# Включение pgvector
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## Конфигурация
Запустите мастер настройки для конфигурации OptimClaw:
Запустите мастер настройки для конфигурации IronClaw:
```bash
optimclaw onboard
ironclaw onboard
```
Мастер настройки поможет установить соединение с базой данных, пройти аутентификацию NEAR AI (через браузер OAuth) и настроить шифрование секретов (используя системную связку ключей). Настройки сохраняются в базе данных; базовые переменные (например, `DATABASE_URL`, `LLM_BACKEND`) записываются в `~/.optimclaw/.env`, чтобы они были доступны до подключения к БД.
Мастер настройки поможет установить соединение с базой данных, пройти аутентификацию NEAR AI (через браузер OAuth) и настроить шифрование секретов (используя системную связку ключей). Настройки сохраняются в базе данных; базовые переменные (например, `DATABASE_URL`, `LLM_BACKEND`) записываются в `~/.ironclaw/.env`, чтобы они были доступны до подключения к БД.
### Альтернативные LLM-провайдеры
OptimClaw по умолчанию использует NEAR AI, но поддерживает множество LLM-провайдеров из коробки.
IronClaw по умолчанию использует NEAR AI, но поддерживает множество LLM-провайдеров из коробки.
Встроенные провайдеры включают **Anthropic**, **OpenAI**, **Google Gemini**, **MiniMax**,
**Mistral** и **Ollama** (локально). Также поддерживаются OpenAI-совместимые сервисы:
**OpenRouter** (300+ моделей), **Together AI**, **Fireworks AI** и собственные серверы
@@ -188,7 +188,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## Безопасность
OptimClaw реализует эшелонированную защиту для обеспечения безопасности ваших данных и предотвращения злоупотреблений.
IronClaw реализует эшелонированную защиту для обеспечения безопасности ваших данных и предотвращения злоупотреблений.
### Песочница WASM
@@ -282,13 +282,13 @@ WASM ──► Валидатор ──► Сканер ───► Инъек
```bash
# Первоначальная настройка (БД, аутентификация и т.д.)
optimclaw onboard
ironclaw onboard
# Запуск интерактивного REPL
cargo run
# С отладочными логами
RUST_LOG=optimclaw=debug cargo run
RUST_LOG=ironclaw=debug cargo run
```
## Разработка
@@ -301,7 +301,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# Запуск тестов
createdb optimclaw_test
createdb ironclaw_test
cargo test
# Запуск конкретного теста
@@ -313,7 +313,7 @@ cargo test название_теста
## Наследие OpenClaw
OptimClaw — это реализация на Rust, вдохновленная проектом [OpenClaw](https://github.com/openclaw/openclaw). Полную матрицу соответствия функций можно найти в [FEATURE_PARITY.md](FEATURE_PARITY.md).
IronClaw — это реализация на Rust, вдохновленная проектом [OpenClaw](https://github.com/openclaw/openclaw). Полную матрицу соответствия функций можно найти в [FEATURE_PARITY.md](FEATURE_PARITY.md).
Ключевые отличия:
+26 -26
View File
@@ -1,8 +1,8 @@
<p align="center">
<img src="optimclaw.png?v=2" alt="OptimClaw" width="200"/>
<img src="ironclaw.png?v=2" alt="IronClaw" width="200"/>
</p>
<h1 align="center">OptimClaw</h1>
<h1 align="center">IronClaw</h1>
<p align="center">
<strong>安全可靠的个人 AI 助手,始终站在你这边</strong>
@@ -10,8 +10,8 @@
<p align="center">
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%20OR%20Apache%202.0-blue.svg" alt="License: MIT OR Apache-2.0" /></a>
<a href="https://t.me/optimclawAI"><img src="https://img.shields.io/badge/Telegram-%40optimclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @optimclawAI" /></a>
<a href="https://www.reddit.com/r/optimclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FoptimclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/optimclawAI" /></a>
<a href="https://t.me/ironclawAI"><img src="https://img.shields.io/badge/Telegram-%40ironclawAI-26A5E4?style=flat&logo=telegram&logoColor=white" alt="Telegram: @ironclawAI" /></a>
<a href="https://www.reddit.com/r/ironclawAI/"><img src="https://img.shields.io/badge/Reddit-r%2FironclawAI-FF4500?style=flat&logo=reddit&logoColor=white" alt="Reddit: r/ironclawAI" /></a>
</p>
<p align="center">
@@ -34,16 +34,16 @@
## 设计理念
OptimClaw 基于一个简单的原则:**你的 AI 助手应该为你服务,而不是与你为敌。**
IronClaw 基于一个简单的原则:**你的 AI 助手应该为你服务,而不是与你为敌。**
在 AI 系统对数据处理日益不透明、与企业利益捆绑的今天,OptimClaw 选择了一条不同的路:
在 AI 系统对数据处理日益不透明、与企业利益捆绑的今天,IronClaw 选择了一条不同的路:
- **数据归你所有** — 所有信息存储在本地,加密保护,始终在你掌控之下
- **透明至上** — 完全开源,可审计,没有隐藏的遥测或数据收集
- **自主扩展** — 随时构建新工具,无需等待供应商更新
- **纵深防御** — 多层安全机制抵御提示注入和数据泄露
OptimClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还是工作。
IronClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还是工作。
## 功能特性
@@ -66,7 +66,7 @@ OptimClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
### 自主扩展
- **动态工具构建** — 描述你的需求,OptimClaw 会将其构建为 WASM 工具
- **动态工具构建** — 描述你的需求,IronClaw 会将其构建为 WASM 工具
- **MCP 协议** — 连接模型上下文协议(Model Context Protocol)服务器以获取额外能力
- **插件架构** — 无需重启即可加载新的 WASM 工具和渠道
@@ -86,12 +86,12 @@ OptimClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
## 下载或编译
访问 [Releases 页面](https://github.com/nearai/optimclaw/releases/) 查看最新版本。
访问 [Releases 页面](https://github.com/nearai/ironclaw/releases/) 查看最新版本。
<details>
<summary>通过 Windows 安装程序安装 (Windows)</summary>
下载 [Windows 安装程序](https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-x86_64-pc-windows-msvc.msi) 并运行。
下载 [Windows 安装程序](https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-x86_64-pc-windows-msvc.msi) 并运行。
</details>
@@ -99,7 +99,7 @@ OptimClaw 是一个你真正可以信赖的 AI 助手,无论是个人生活还
<summary>通过 PowerShell 脚本安装 (Windows)</summary>
```sh
irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.ps1 | iex
irm https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.ps1 | iex
```
</details>
@@ -108,7 +108,7 @@ irm https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-insta
<summary>通过 Shell 脚本安装 (macOS、Linux、Windows/WSL)</summary>
```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releases/latest/download/optimclaw-installer.sh | sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/ironclaw/releases/latest/download/ironclaw-installer.sh | sh
```
</details>
@@ -116,7 +116,7 @@ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/nearai/optimclaw/releas
<summary>通过 Homebrew 安装 (macOS/Linux)</summary>
```sh
brew install optimclaw
brew install ironclaw
```
</details>
@@ -128,8 +128,8 @@ brew install optimclaw
```bash
# 克隆仓库
git clone https://github.com/nearai/optimclaw.git
cd optimclaw
git clone https://github.com/nearai/ironclaw.git
cd ironclaw
# 编译
cargo build --release
@@ -146,25 +146,25 @@ cargo test
```bash
# 创建数据库
createdb optimclaw
createdb ironclaw
# 启用 pgvector 扩展
psql optimclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
psql ironclaw -c "CREATE EXTENSION IF NOT EXISTS vector;"
```
## 配置
运行设置向导来配置 OptimClaw
运行设置向导来配置 IronClaw
```bash
optimclaw onboard
ironclaw onboard
```
向导将引导你完成数据库连接、NEAR AI 身份验证(通过浏览器 OAuth)和密钥加密(使用系统钥匙串)。设置会保存在数据库中;引导变量(如 `DATABASE_URL``LLM_BACKEND`)写入 `~/.optimclaw/.env`,以便在数据库连接前可用。
向导将引导你完成数据库连接、NEAR AI 身份验证(通过浏览器 OAuth)和密钥加密(使用系统钥匙串)。设置会保存在数据库中;引导变量(如 `DATABASE_URL``LLM_BACKEND`)写入 `~/.ironclaw/.env`,以便在数据库连接前可用。
### 替代 LLM 提供商
OptimClaw 默认使用 NEAR AI,但开箱即用地支持多种 LLM 提供商。
IronClaw 默认使用 NEAR AI,但开箱即用地支持多种 LLM 提供商。
内置提供商包括 **Anthropic**、**OpenAI**、**GitHub Copilot**、**Google Gemini**、**MiniMax**、**Mistral** 和 **Ollama**(本地部署)。同时也支持 OpenAI 兼容服务,如 **OpenRouter**300+ 模型)、**Together AI**、**Fireworks AI** 以及自托管服务器(**vLLM**、**LiteLLM**)。
在向导中选择你的提供商,或直接设置环境变量:
@@ -185,7 +185,7 @@ LLM_MODEL=anthropic/claude-sonnet-4
## 安全机制
OptimClaw 实现了纵深防御策略来保护你的数据并防止滥用。
IronClaw 实现了纵深防御策略来保护你的数据并防止滥用。
### WASM 沙箱
@@ -278,13 +278,13 @@ WASM ──► 白名单 ──► 泄露扫描 ──► 凭据 ──► 执
```bash
# 首次设置(配置数据库、认证等)
optimclaw onboard
ironclaw onboard
# 启动交互式 REPL
cargo run
# 启用调试日志
RUST_LOG=optimclaw=debug cargo run
RUST_LOG=ironclaw=debug cargo run
```
## 开发
@@ -297,7 +297,7 @@ cargo fmt
cargo clippy --all --benches --tests --examples --all-features
# 运行测试
createdb optimclaw_test
createdb ironclaw_test
cargo test
# 运行指定测试
@@ -309,7 +309,7 @@ cargo test test_name
## OpenClaw 传承
OptimClaw 是受 [OpenClaw](https://github.com/openclaw/openclaw) 启发的 Rust 重新实现。参见 [FEATURE_PARITY.md](FEATURE_PARITY.md) 了解完整的功能追踪矩阵。
IronClaw 是受 [OpenClaw](https://github.com/openclaw/openclaw) 启发的 Rust 重新实现。参见 [FEATURE_PARITY.md](FEATURE_PARITY.md) 了解完整的功能追踪矩阵。
主要差异:
+1 -1
View File
@@ -1,5 +1,5 @@
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use optimclaw::safety::{LeakDetector, Sanitizer, Validator};
use ironclaw_safety::{LeakDetector, Sanitizer, Validator};
fn bench_sanitizer(c: &mut Criterion) {
let mut group = c.benchmark_group("sanitizer");
+2 -2
View File
@@ -1,6 +1,6 @@
use criterion::{Criterion, black_box, criterion_group, criterion_main};
use optimclaw::config::SafetyConfig;
use optimclaw::safety::{SafetyLayer, Validator};
use ironclaw::config::SafetyConfig;
use ironclaw_safety::{SafetyLayer, Validator};
fn bench_safety_layer_pipeline(c: &mut Criterion) {
let mut group = c.benchmark_group("safety_pipeline");
+58 -2
View File
@@ -20,6 +20,9 @@ fn main() {
// ── Embed registry manifests ────────────────────────────────────────
embed_registry_catalog(&root);
// ── Embed bundled skills ────────────────────────────────────────────
embed_skills(&root);
// ── Build Telegram channel WASM ─────────────────────────────────────
let channel_dir = root.join("channels-src/telegram");
let wasm_out = channel_dir.join("telegram.wasm");
@@ -125,7 +128,7 @@ fn embed_registry_catalog(root: &Path) {
// are emitted inside collect_json_files to track content changes reliably).
println!("cargo:rerun-if-changed=registry/_bundles.json");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap());
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); // safety: build script
let out_path = out_dir.join("embedded_catalog.json");
if !registry_dir.is_dir() {
@@ -177,7 +180,60 @@ fn embed_registry_catalog(root: &Path) {
bundles_raw,
);
fs::write(&out_path, catalog).unwrap();
fs::write(&out_path, catalog).unwrap(); // safety: build script
}
/// Collect all `skills/*/SKILL.md` files into an embedded JSON blob.
///
/// Output: `$OUT_DIR/embedded_skills.json` — a JSON array of `{"name": "...", "content": "..."}`.
/// These are loaded at runtime as bundled skills (lowest discovery priority, Trusted trust level).
fn embed_skills(root: &Path) {
use std::fs;
let skills_dir = root.join("skills");
// Rerun when any skill changes
println!("cargo:rerun-if-changed=skills");
let out_dir = PathBuf::from(env::var("OUT_DIR").unwrap()); // safety: build script panics on failure
let out_path = out_dir.join("embedded_skills.json");
if !skills_dir.is_dir() {
fs::write(&out_path, "[]").unwrap(); // safety: build script
return;
}
let mut skills: Vec<String> = Vec::new();
let mut entries: Vec<_> = fs::read_dir(&skills_dir)
.unwrap() // safety: build script
.filter_map(|e| e.ok())
.filter(|e| e.path().is_dir())
.collect();
entries.sort_by_key(|e| e.file_name());
for entry in entries {
let skill_md = entry.path().join("SKILL.md");
if !skill_md.is_file() {
continue;
}
// Emit per-file watch
println!("cargo:rerun-if-changed={}", skill_md.display());
let name = entry.file_name().to_string_lossy().to_string();
if let Ok(content) = fs::read_to_string(&skill_md) {
// Escape for JSON embedding
let name_json = serde_json::to_string(&name).unwrap(); // safety: build script
let content_json = serde_json::to_string(&content).unwrap(); // safety: build script
skills.push(format!(
r#"{{"name":{},"content":{}}}"#,
name_json, content_json
));
}
}
let catalog = format!("[{}]", skills.join(","));
fs::write(&out_path, catalog).unwrap(); // safety: build script
}
/// Read all .json files from a directory and push their raw contents into `out`.
+217 -12
View File
@@ -20,12 +20,27 @@ version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "base64ct"
version = "1.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2af50177e190e07a26ab74f8b1efbfe2ef87da2116221318cb1c2e82baf7de06"
[[package]]
name = "bitflags"
version = "2.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "843867be96c8daad0d758b57df9392b6d8d271134fce549de6ce169ff98a92af"
[[package]]
name = "block-buffer"
version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3078c7629b62d3f0439517fa394996acacc5cbc91c5a20d8c658e77abd503a71"
dependencies = [
"generic-array",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
@@ -33,20 +48,134 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "discord-channel"
version = "0.2.1"
name = "const-oid"
version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "cpufeatures"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "59ed5838eebb26a2bb2e58f6d5b5316989ae9d08bab10e0e6d103e656d1b0280"
dependencies = [
"libc",
]
[[package]]
name = "crypto-common"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a"
dependencies = [
"generic-array",
"typenum",
]
[[package]]
name = "curve25519-dalek"
version = "4.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97fb8b7c4503de7d6ae7b42ab72a5a59857b4c937ec27a3d4539dba95b5ab2be"
dependencies = [
"cfg-if",
"cpufeatures",
"curve25519-dalek-derive",
"digest",
"fiat-crypto",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek-derive"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "der"
version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid",
"zeroize",
]
[[package]]
name = "digest"
version = "0.10.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292"
dependencies = [
"block-buffer",
"crypto-common",
]
[[package]]
name = "discord-channel"
version = "0.2.0"
dependencies = [
"ed25519-dalek",
"hex",
"serde",
"serde_json",
"wit-bindgen",
]
[[package]]
name = "ed25519"
version = "2.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
dependencies = [
"pkcs8",
"signature",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
dependencies = [
"curve25519-dalek",
"ed25519",
"serde",
"sha2",
"subtle",
"zeroize",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "fiat-crypto"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]]
name = "generic-array"
version = "0.14.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85649ca51fd72272d7821adaf274ad91c288277713d9c18820d8499a7ff69e9a"
dependencies = [
"typenum",
"version_check",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
@@ -68,6 +197,12 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "hex"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "id-arena"
version = "2.3.0"
@@ -88,9 +223,9 @@ dependencies = [
[[package]]
name = "itoa"
version = "1.0.18"
version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "leb128"
@@ -98,6 +233,12 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "884e2677b40cc8c339eaefcb701c32ef1fd2493d71118dc0ca4b6a736c93bd67"
[[package]]
name = "libc"
version = "0.2.182"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112"
[[package]]
name = "log"
version = "0.4.29"
@@ -112,9 +253,19 @@ checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "once_cell"
version = "1.21.4"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "pkcs8"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
]
[[package]]
name = "prettyplease"
@@ -137,13 +288,22 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.45"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustc_version"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92"
dependencies = [
"semver",
]
[[package]]
name = "semver"
version = "1.0.27"
@@ -193,6 +353,23 @@ dependencies = [
"zmij",
]
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "signature"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de"
[[package]]
name = "smallvec"
version = "1.15.1"
@@ -208,6 +385,22 @@ dependencies = [
"smallvec",
]
[[package]]
name = "spki"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
]
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
@@ -219,6 +412,12 @@ dependencies = [
"unicode-ident",
]
[[package]]
name = "typenum"
version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "562d481066bde0658276a35467c4af00bdc6ee726305698a55b86e61d7ad82bb"
[[package]]
name = "unicode-ident"
version = "1.0.24"
@@ -376,24 +575,30 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.47"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "efbb2a062be311f2ba113ce66f697a4dc589f85e78a4aea276200804cea0ed87"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.47"
version = "0.8.39"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e8bc7269b54418e7aeeef514aa68f8690b8c0489a06b0136e5f57c4c5ccab89"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "zeroize"
version = "1.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0"
[[package]]
name = "zmij"
version = "1.0.21"
+4 -2
View File
@@ -1,8 +1,8 @@
[package]
name = "discord-channel"
version = "0.2.1"
version = "0.2.0"
edition = "2021"
description = "Discord channel for OptimClaw"
description = "Discord channel for IronClaw"
license = "MIT OR Apache-2.0"
publish = false
@@ -10,6 +10,8 @@ publish = false
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
wit-bindgen = "0.36"
ed25519-dalek = { version = "2", default-features = false, features = ["alloc", "fast", "zeroize"] }
hex = "0.4"
[lib]
crate-type = ["cdylib"]
+6 -25
View File
@@ -1,4 +1,4 @@
# Discord Channel for OptimClaw
# Discord Channel for IronClaw
WASM channel for Discord integration - handle slash commands and button interactions via webhooks.
@@ -13,12 +13,12 @@ WASM channel for Discord integration - handle slash commands and button interact
1. Create a Discord Application at <https://discord.com/developers/applications>
2. Create a Bot and get the token
3. Set up Interactions URL to point to your OptimClaw instance
3. Set up Interactions URL to point to your IronClaw instance
4. Copy the Application ID and Public Key
5. Store in OptimClaw secrets:
5. Store in IronClaw secrets:
```bash
optimclaw secret set discord_bot_token YOUR_BOT_TOKEN
ironclaw secret set discord_bot_token YOUR_BOT_TOKEN
```
**Note:** The `discord_bot_token` secret is used for Discord REST API calls.
@@ -51,7 +51,7 @@ curl -X POST \
In your Discord app settings, set:
- Interactions Endpoint URL: `https://your-optimclaw.com/webhook/discord`
- Interactions Endpoint URL: `https://your-ironclaw.com/webhook/discord`
## Usage Examples
@@ -86,24 +86,6 @@ If an internal error occurs (e.g., metadata serialization failure), the tool att
Check the host logs for detailed error information.
## Advanced Usage
### Gateway Mode
The Discord channel now defaults to Discord Gateway transport for inbound message intake.
The bundled identify payload requests intents `4609`, which expands to:
- `GUILDS` (`1`)
- `GUILD_MESSAGES` (`512`)
- `DIRECT_MESSAGES` (`4096`)
Gateway DMs now follow the same pairing policy as webhook DMs. Unpaired users receive a pairing
instruction reply in the DM channel before the message is allowed through to the agent. If you
want stricter access control than pairing, set `owner_id`; that lock still applies to both
webhook and Gateway traffic.
Gateway presence simply reflects a successful authenticated Gateway connection and advertises
`online`. Pairing still controls whether DMs are allowed through to the agent, but it no longer
changes the visible Discord status.
### Mention Polling
The Discord channel can also poll configured channels for `@bot` mentions.
@@ -128,7 +110,6 @@ Example channel config:
- `owner_id`: when set, only that Discord user can interact with the bot.
- `dm_policy`: `open` allows all DMs; `pairing` requires approval.
- `allow_from`: allowlist entries for DM pairing checks (`*`, user id, or username).
- Gateway DMs respect `dm_policy` and pairing just like webhook DMs.
### Embeds
@@ -146,7 +127,7 @@ To send embeds, include an `embeds` array in the `metadata_json` field of the ag
### "401 Unauthorized"
- Check that `discord_bot_token` is set correctly in OptimClaw secrets.
- Check that `discord_bot_token` is set correctly in IronClaw secrets.
- Ensure the bot is added to the server.
### "Interaction Failed"
+2 -17
View File
@@ -1,5 +1,5 @@
{
"version": "0.2.1",
"version": "0.2.0",
"wit_version": "0.3.0",
"type": "channel",
"name": "discord",
@@ -22,8 +22,7 @@
"capabilities": {
"http": {
"allowlist": [
{ "host": "discord.com", "path_prefix": "/api/v10" },
{ "host": "gateway.discord.gg", "path_prefix": "/", "methods": ["GET"] }
{ "host": "discord.com", "path_prefix": "/api/v10" }
],
"credentials": {
"discord_bot_token": {
@@ -37,20 +36,6 @@
"requests_per_hour": 3600
}
},
"websocket": {
"url": "wss://gateway.discord.gg/?v=10&encoding=json",
"connect_on_start": true,
"identify_secret_name": "discord_bot_token",
"identify": {
"_intents_doc": "GUILDS(1) + GUILD_MESSAGES(512) + DIRECT_MESSAGES(4096)",
"intents": 4609,
"properties": {
"os": "linux",
"browser": "ironclaw",
"device": "ironclaw"
}
}
},
"secrets": {
"allowed_names": ["discord_bot_token", "discord_*"]
},
File diff suppressed because it is too large Load Diff
-7
View File
@@ -44,7 +44,6 @@ version = "0.1.0"
dependencies = [
"serde",
"serde_json",
"subtle",
"wit-bindgen",
]
@@ -209,12 +208,6 @@ dependencies = [
"smallvec",
]
[[package]]
name = "subtle"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "syn"
version = "2.0.117"
+1 -2
View File
@@ -2,7 +2,7 @@
name = "feishu-channel"
version = "0.1.0"
edition = "2021"
description = "Feishu/Lark Bot channel for OptimClaw"
description = "Feishu/Lark Bot channel for IronClaw"
license = "MIT OR Apache-2.0"
[lib]
@@ -15,7 +15,6 @@ wit-bindgen = "0.36"
# Serialization
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
subtle = "2.6"
# Exclude from parent workspace (this is a standalone WASM component)
+3 -5
View File
@@ -7,7 +7,7 @@
"auth": {
"secret_name": "feishu_app_id",
"display_name": "Feishu / Lark",
"instructions": "Create a bot at https://open.feishu.cn/app (Feishu) or https://open.larksuite.com/app (Lark). You need the App ID and App Secret. Note: OptimClaw supports Event Subscription webhook delivery, but not Feishu's long-connection websocket mode.",
"instructions": "Create a bot at https://open.feishu.cn/app (Feishu) or https://open.larksuite.com/app (Lark). You need the App ID and App Secret. Note: IronClaw supports Event Subscription webhook delivery, but not Feishu's long-connection websocket mode.",
"setup_url": "https://open.feishu.cn/app",
"token_hint": "App ID looks like cli_XXXX, App Secret is a long alphanumeric string",
"env_var": "FEISHU_APP_ID"
@@ -27,7 +27,7 @@
{
"name": "feishu_verification_token",
"prompt": "Enter your Feishu/Lark Verification Token (from Event Subscription webhook settings)",
"optional": false
"optional": true
}
],
"setup_url": "https://open.feishu.cn/app"
@@ -63,15 +63,13 @@
},
"webhook": {
"secret_header": "X-Feishu-Verification-Token",
"secret_name": "feishu_verification_token",
"managed_by_host": false
"secret_name": "feishu_verification_token"
}
}
},
"config": {
"app_id": null,
"app_secret": null,
"verification_token": null,
"api_base": "https://open.feishu.cn",
"owner_id": null,
"dm_policy": "pairing",
+4 -122
View File
@@ -1,11 +1,11 @@
// Feishu API types have fields reserved for future use.
#![allow(dead_code)]
//! Feishu/Lark Bot channel for OptimClaw.
//! Feishu/Lark Bot channel for IronClaw.
//!
//! This WASM component implements the channel interface for handling Feishu
//! webhooks (Event Subscription v2.0) and sending messages back via the
//! Feishu/Lark Bot API. OptimClaw currently does not connect to Feishu's
//! Feishu/Lark Bot API. IronClaw currently does not connect to Feishu's
//! long-connection websocket subscription mode; use Event Subscription
//! webhooks for this channel.
//!
@@ -23,8 +23,7 @@
//! - App credentials (app_id, app_secret) are injected by the host into
//! the config JSON during startup for token exchange
//! - Bearer token for API calls is obtained via token exchange and cached
//! - Webhook requests must be authenticated by the host or by a matching
//! Feishu verification token in the request body
//! - Verification token validated by host for webhook requests
// Generate bindings from the WIT file
wit_bindgen::generate!({
@@ -33,7 +32,6 @@ wit_bindgen::generate!({
});
use serde::{Deserialize, Serialize};
use subtle::ConstantTimeEq;
// Re-export generated types
use exports::near::agent::channel::{
@@ -52,7 +50,6 @@ const ALLOW_FROM_PATH: &str = "allow_from";
const API_BASE_PATH: &str = "api_base";
const APP_ID_PATH: &str = "app_id";
const APP_SECRET_PATH: &str = "app_secret";
const VERIFICATION_TOKEN_PATH: &str = "verification_token";
const TOKEN_PATH: &str = "tenant_access_token";
const TOKEN_EXPIRY_PATH: &str = "token_expiry";
@@ -105,10 +102,6 @@ struct FeishuEventHeader {
/// Tenant key.
#[serde(default)]
tenant_key: Option<String>,
/// Verification token for v2 event payloads.
#[serde(default)]
token: Option<String>,
}
/// Message receive event payload (im.message.receive_v1).
@@ -258,9 +251,6 @@ struct FeishuConfig {
/// Feishu App Secret (for token exchange).
app_secret: Option<String>,
/// Feishu Event Subscription verification token.
verification_token: Option<String>,
/// API base URL. Defaults to "https://open.feishu.cn" (use
/// "https://open.larksuite.com" for Lark international).
#[serde(default = "default_api_base")]
@@ -310,9 +300,6 @@ impl Guest for FeishuChannel {
if let Some(ref app_secret) = config.app_secret {
let _ = channel_host::workspace_write(APP_SECRET_PATH, app_secret);
}
if let Some(ref verification_token) = config.verification_token {
let _ = channel_host::workspace_write(VERIFICATION_TOKEN_PATH, verification_token);
}
if let Some(owner_id) = &config.owner_id {
let _ = channel_host::workspace_write(OWNER_ID_PATH, owner_id);
@@ -389,23 +376,6 @@ impl Guest for FeishuChannel {
}
};
let configured_token =
channel_host::workspace_read(VERIFICATION_TOKEN_PATH).filter(|token| !token.is_empty());
if !is_authenticated_webhook(
req.secret_validated,
configured_token.as_deref(),
request_verification_token(&event),
) {
channel_host::log(
channel_host::LogLevel::Warn,
"Rejecting unauthenticated Feishu webhook request",
);
return json_response(
401,
serde_json::json!({"error": "Webhook authentication failed"}),
);
}
// Handle URL verification challenge (initial webhook setup).
if event.event_type.as_deref() == Some("url_verification") {
if let Some(challenge) = &event.challenge {
@@ -869,31 +839,6 @@ fn json_response(status: u16, body: serde_json::Value) -> OutgoingHttpResponse {
}
}
fn is_authenticated_webhook(
secret_validated: bool,
configured_token: Option<&str>,
request_token: Option<&str>,
) -> bool {
if secret_validated {
return true;
}
match (configured_token, request_token) {
(Some(expected), Some(provided)) => {
bool::from(expected.as_bytes().ct_eq(provided.as_bytes()))
}
_ => false,
}
}
fn request_verification_token(event: &FeishuEvent) -> Option<&str> {
event
.header
.as_ref()
.and_then(|header| header.token.as_deref())
.or(event.token.as_deref())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -917,10 +862,7 @@ mod tests {
fn parse_token_response_rejects_missing_token() {
let json = r#"{"code": 0, "msg": "ok", "expire": 7200}"#;
let result: Result<TenantAccessTokenResponse, _> = serde_json::from_str(json);
assert!(
result.is_err(),
"should fail when tenant_access_token is missing"
);
assert!(result.is_err(), "should fail when tenant_access_token is missing");
}
#[test]
@@ -952,64 +894,4 @@ mod tests {
assert_eq!(resp.code, 10003);
assert!(resp.tenant_access_token.is_empty());
}
#[test]
fn webhook_auth_requires_host_auth_or_matching_verification_token() {
assert!(
!is_authenticated_webhook(false, None, Some("token")),
"requests without any configured verification mechanism must be rejected"
);
assert!(
!is_authenticated_webhook(false, Some("expected"), None),
"requests missing the Feishu token must be rejected when host auth did not pass"
);
assert!(
!is_authenticated_webhook(false, Some("expected"), Some("wrong")),
"requests with the wrong Feishu token must be rejected"
);
assert!(
is_authenticated_webhook(false, Some("expected"), Some("expected")),
"matching Feishu verification token should authenticate the request"
);
assert!(
is_authenticated_webhook(true, None, None),
"host-authenticated requests should still be accepted"
);
assert!(
is_authenticated_webhook(true, Some("expected"), Some("wrong")),
"host authentication should take precedence over body token checks"
);
}
#[test]
fn request_verification_token_prefers_v2_header_token() {
let event: FeishuEvent = serde_json::from_str(
r#"{
"schema": "2.0",
"header": {
"event_id": "evt_123",
"event_type": "im.message.receive_v1",
"token": "header-token"
},
"event": {}
}"#,
)
.unwrap();
assert_eq!(request_verification_token(&event), Some("header-token"));
}
#[test]
fn request_verification_token_falls_back_to_top_level_token() {
let event: FeishuEvent = serde_json::from_str(
r#"{
"type": "url_verification",
"challenge": "abc",
"token": "top-level-token"
}"#,
)
.unwrap();
assert_eq!(request_verification_token(&event), Some("top-level-token"));
}
}
+1 -1
View File
@@ -2,7 +2,7 @@
name = "slack-channel"
version = "0.2.1"
edition = "2021"
description = "Slack Events API channel for OptimClaw"
description = "Slack Events API channel for IronClaw"
license = "MIT OR Apache-2.0"
[lib]
+2 -2
View File
@@ -1,4 +1,4 @@
//! Slack Events API channel for OptimClaw.
//! Slack Events API channel for IronClaw.
//!
//! This WASM component implements the channel interface for handling Slack
//! webhooks and sending messages back to Slack.
@@ -650,7 +650,7 @@ fn send_pairing_reply(channel_id: &str, code: &str) -> Result<(), String> {
let payload = serde_json::json!({
"channel": channel_id,
"text": format!(
"To pair with this bot, run: `optimclaw pairing approve slack {}`",
"To pair with this bot, run: `ironclaw pairing approve slack {}`",
code
),
});
+1 -1
View File
@@ -2,7 +2,7 @@
name = "telegram-channel"
version = "0.2.1"
edition = "2021"
description = "Telegram Bot API channel for OptimClaw"
description = "Telegram Bot API channel for IronClaw"
license = "MIT OR Apache-2.0"
[lib]
+4 -4
View File
@@ -1,7 +1,7 @@
// Telegram API types have fields reserved for future use (entities, reply threading, etc.)
#![allow(dead_code)]
//! Telegram Bot API channel for OptimClaw.
//! Telegram Bot API channel for IronClaw.
//!
//! This WASM component implements the channel interface for handling Telegram
//! webhooks and sending messages back via the Bot API.
@@ -1170,7 +1170,7 @@ fn send_photo(
);
}
let boundary = format!("optimclaw-{}", channel_host::now_millis());
let boundary = format!("ironclaw-{}", channel_host::now_millis());
let mut body = Vec::new();
write_multipart_field(&mut body, &boundary, "chat_id", &chat_id.to_string());
@@ -1235,7 +1235,7 @@ fn send_document(
) -> Result<(), String> {
let message_thread_id = normalize_thread_id(message_thread_id);
let boundary = format!("optimclaw-{}", channel_host::now_millis());
let boundary = format!("ironclaw-{}", channel_host::now_millis());
let mut body = Vec::new();
write_multipart_field(&mut body, &boundary, "chat_id", &chat_id.to_string());
@@ -1552,7 +1552,7 @@ fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
send_message(
chat_id,
&format!(
"To pair with this bot, run: `optimclaw pairing approve telegram {}`",
"To pair with this bot, run: `ironclaw pairing approve telegram {}`",
code
),
None,
+1 -1
View File
@@ -2,7 +2,7 @@
name = "whatsapp-channel"
version = "0.2.0"
edition = "2021"
description = "WhatsApp Cloud API channel for OptimClaw"
description = "WhatsApp Cloud API channel for IronClaw"
[lib]
crate-type = ["cdylib"]
+2 -2
View File
@@ -1,7 +1,7 @@
// WhatsApp API types have fields reserved for future use (contacts, statuses, etc.)
#![allow(dead_code)]
//! WhatsApp Cloud API channel for OptimClaw.
//! WhatsApp Cloud API channel for IronClaw.
//!
//! This WASM component implements the channel interface for handling WhatsApp
//! webhooks and sending messages back via the Cloud API.
@@ -910,7 +910,7 @@ fn send_pairing_reply(
"text": {
"preview_url": false,
"body": format!(
"To pair with this bot, run: optimclaw pairing approve whatsapp {}",
"To pair with this bot, run: ironclaw pairing approve whatsapp {}",
code
)
}
+1 -1
View File
@@ -1,6 +1,6 @@
# Complexity guardrails for AI-assisted development quality.
# These thresholds prevent new violations while preserving existing code.
# See: https://github.com/nearai/optimclaw/issues/338
# See: https://github.com/nearai/ironclaw/issues/338
cognitive-complexity-threshold = 15 # default: 25 (only active when lint is enabled)
too-many-lines-threshold = 100 # default: 100 (only active when lint is enabled)
@@ -1,13 +1,13 @@
[package]
name = "optimclaw_common"
name = "ironclaw_common"
version = "0.1.0"
edition = "2024"
rust-version = "1.92"
description = "Shared types and utilities for the OptimClaw workspace"
description = "Shared types and utilities for the IronClaw workspace"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/optimclaw"
repository = "https://github.com/nearai/optimclaw"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
[package.metadata.dist]
dist = false
@@ -181,6 +181,14 @@ pub enum AppEvent {
thread_id: Option<String>,
},
/// Skills activated for a conversation turn.
#[serde(rename = "skill_activated")]
SkillActivated {
skill_names: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none")]
thread_id: Option<String>,
},
/// Extension activation status change (WASM channels).
#[serde(rename = "extension_status")]
ExtensionStatus {
@@ -206,6 +214,33 @@ pub enum AppEvent {
narrative: String,
decisions: Vec<ToolDecisionDto>,
},
// ── Engine v2 thread lifecycle events ──
/// Engine thread changed state (e.g. Running → Completed).
#[serde(rename = "thread_state_changed")]
ThreadStateChanged {
thread_id: String,
from_state: String,
to_state: String,
#[serde(skip_serializing_if = "Option::is_none")]
reason: Option<String>,
},
/// A child thread was spawned by a parent thread.
#[serde(rename = "child_thread_spawned")]
ChildThreadSpawned {
parent_thread_id: String,
child_thread_id: String,
goal: String,
},
/// A mission spawned a new thread.
#[serde(rename = "mission_thread_spawned")]
MissionThreadSpawned {
mission_id: String,
thread_id: String,
mission_name: String,
},
}
impl AppEvent {
@@ -233,9 +268,13 @@ impl AppEvent {
Self::ImageGenerated { .. } => "image_generated",
Self::Suggestions { .. } => "suggestions",
Self::TurnCost { .. } => "turn_cost",
Self::SkillActivated { .. } => "skill_activated",
Self::ExtensionStatus { .. } => "extension_status",
Self::ReasoningUpdate { .. } => "reasoning_update",
Self::JobReasoning { .. } => "job_reasoning",
Self::ThreadStateChanged { .. } => "thread_state_changed",
Self::ChildThreadSpawned { .. } => "child_thread_spawned",
Self::MissionThreadSpawned { .. } => "mission_thread_spawned",
}
}
}
@@ -351,6 +390,10 @@ mod tests {
cost_usd: String::new(),
thread_id: None,
},
AppEvent::SkillActivated {
skill_names: vec![],
thread_id: None,
},
AppEvent::ExtensionStatus {
extension_name: String::new(),
status: String::new(),
@@ -366,6 +409,22 @@ mod tests {
narrative: String::new(),
decisions: vec![],
},
AppEvent::ThreadStateChanged {
thread_id: String::new(),
from_state: String::new(),
to_state: String::new(),
reason: None,
},
AppEvent::ChildThreadSpawned {
parent_thread_id: String::new(),
child_thread_id: String::new(),
goal: String::new(),
},
AppEvent::MissionThreadSpawned {
mission_id: String::new(),
thread_id: String::new(),
mission_name: String::new(),
},
];
for variant in &variants {
@@ -1,4 +1,4 @@
//! Shared types and utilities for the OptimClaw workspace.
//! Shared types and utilities for the IronClaw workspace.
mod event;
mod util;
+178
View File
@@ -0,0 +1,178 @@
# IronClaw Engine Crate
Unified thread-capability-CodeAct execution model. Replaces ~10 separate abstractions (Session, Job, Routine, Channel, Tool, Skill, Hook, Observer, Extension, LoopDelegate) with 5 primitives.
## Full Architecture Plan
See `docs/plans/2026-03-20-engine-v2-architecture.md` for the 8-phase roadmap.
## Five Primitives
| Primitive | Purpose | Replaces |
|-----------|---------|----------|
| **Thread** | Unit of work with lifecycle, parent-child tree, capability leases | Session + Job + Routine + Sub-agent |
| **Step** | Unit of execution (one LLM call + its action executions) | Agentic loop iteration + tool calls |
| **Capability** | Unit of effect (actions + knowledge + policies) | Tool + Skill + Hook + Extension |
| **MemoryDoc** | Unit of durable knowledge (summaries, lessons, skills) | Workspace memory blobs |
| **Project** | Unit of context (scopes memory, threads, missions) | Flat workspace namespace |
## Build & Test
```bash
cargo check -p ironclaw_engine
cargo clippy -p ironclaw_engine --all-targets -- -D warnings
cargo test -p ironclaw_engine
```
## Module Map
```
src/
├── lib.rs # Public API, re-exports
├── types/ # Core data structures (no async, no I/O)
│ ├── thread.rs # Thread, ThreadId, ThreadState (state machine), ThreadType, ThreadConfig
│ ├── step.rs # Step, StepId, LlmResponse, ActionCall, ActionResult, TokenUsage
│ ├── capability.rs # Capability, ActionDef, EffectType, CapabilityLease, PolicyRule
│ ├── memory.rs # MemoryDoc, DocId, DocType (Summary/Lesson/Skill/Issue/Spec/Note)
│ ├── project.rs # Project, ProjectId
│ ├── event.rs # ThreadEvent, EventKind (18 variants for event sourcing)
│ ├── message.rs # ThreadMessage, MessageRole
│ ├── provenance.rs # Provenance enum (User/System/ToolOutput/LlmGenerated/etc.)
│ ├── conversation.rs # ConversationSurface, ConversationEntry, EntrySender
│ ├── mission.rs # Mission, MissionId, MissionCadence, MissionStatus
│ └── error.rs # EngineError, ThreadError, StepError, CapabilityError
├── traits/ # External dependency abstractions (host implements these)
│ ├── llm.rs # LlmBackend trait
│ ├── store.rs # Store trait (20 CRUD methods)
│ └── effect.rs # EffectExecutor trait
├── capability/ # Capability management
│ ├── registry.rs # CapabilityRegistry — register/get/list capabilities
│ ├── lease.rs # LeaseManager — grant/check/consume/revoke/expire leases
│ ├── policy.rs # PolicyEngine — deterministic effect-level allow/deny/approve + provenance taint
│ ├── skill_selector.rs # SkillSelector — MemoryDoc→LoadedSkill bridge, deterministic selection
│ └── skill_tracker.rs # SkillTracker — confidence tracking, versioned updates, rollback
├── runtime/ # Thread lifecycle management
│ ├── manager.rs # ThreadManager — spawn, stop, inject messages, join threads
│ ├── conversation.rs # ConversationManager — routes UI messages to threads
│ ├── mission.rs # MissionManager — long-running goals that spawn threads on cadence
│ ├── tree.rs # ThreadTree — parent-child relationships
│ └── messaging.rs # ThreadSignal, ThreadOutcome, signal channels
├── executor/ # Step execution
│ ├── loop_engine.rs # ExecutionLoop — core loop replacing run_agentic_loop()
│ ├── structured.rs # Tier 0: structured tool call execution
│ ├── scripting.rs # Tier 1: embedded Python via Monty (CodeAct/RLM)
│ ├── context.rs # Context builder (messages + actions from leases + memory docs)
│ ├── compaction.rs # Context compaction when approaching model context limit
│ ├── prompt.rs # System prompt construction (CodeAct preamble/postamble)
│ ├── intent.rs # Tool intent nudge detection
│ └── trace.rs # Execution trace recording and retrospective analysis
├── memory/ # Memory document system
│ ├── store.rs # MemoryStore — project-scoped doc CRUD
│ └── retrieval.rs # RetrievalEngine — keyword-based context retrieval from project docs
└── reliability.rs # ReliabilityTracker — per-action success rate and latency via EMA
```
## Thread State Machine
```
Created → Running → Waiting → Running (resume)
→ Suspended → Running (resume)
→ Completed → Done
→ Failed
```
Validated by `ThreadState::can_transition_to()`. Terminal states: `Done`, `Failed`.
## Learning Missions
Three event-driven missions fire automatically after thread completion:
1. **Error diagnosis** (`self-improvement`) — fires when a thread completes with trace issues. Diagnoses root cause and applies prompt overlays or orchestrator patches.
2. **Skill extraction** (`skill-extraction`) — fires when a thread succeeds with 5+ steps and 3+ tool actions. Extracts reusable skills with activation metadata, CodeAct code snippets, and domain tags. Output stored as `DocType::Skill` MemoryDoc.
3. **Conversation insights** (`conversation-insights`) — fires every 5 completed threads in a project. Extracts user preferences, domain knowledge, and workflow patterns.
Created by `MissionManager::ensure_learning_missions()` at project bootstrap.
## External Trait Boundaries
The engine defines three traits that the host crate implements:
| Trait | Purpose | Host wraps |
|-------|---------|------------|
| `LlmBackend` | `complete(messages, actions, config) -> LlmOutput` | `LlmProvider` |
| `Store` | Thread/Step/Event/Project/Doc/Lease CRUD | `Database` (PostgreSQL + libSQL) |
| `EffectExecutor` | `execute_action(name, params, lease, ctx) -> ActionResult` | `ToolRegistry` + `SafetyLayer` |
## Execution Loop
`ExecutionLoop::run()` handles three `LlmResponse` variants:
1. Check signals (Stop, InjectMessage) via `mpsc::Receiver`
2. Build context (messages + available actions from active leases)
3. Call LLM via `LlmBackend::complete()`
4. **If `Text`**: check tool intent nudge, return if final response
5. **If `ActionCalls`** (Tier 0): for each call, find lease → check policy → consume use → execute via `EffectExecutor` → record result
6. **If `Code`** (Tier 1): execute Python via Monty with context-as-variables and `llm_query()` support → compact metadata in context
7. Record Step, emit ThreadEvents
8. Repeat until: text response, stop signal, max iterations, or approval needed
## CodeAct / Monty Integration (Tier 1)
Python execution via Monty interpreter (`executor/scripting.rs`). Follows the RLM (Recursive Language Model) pattern.
**Context as variables** (not attention input):
- Thread messages injected as `context` Python variable
- Thread goal as `goal`, step index as `step_number`
- Prior action results as `previous_results` dict
- The LLM's chat context stays lean; full data lives in REPL variables
**Tool dispatch**: Unknown function calls suspend the VM → lease check → policy check → `EffectExecutor` → result returned to Python.
**`llm_query(prompt, context)`**: Recursive subagent call. Suspends VM → spawns single-shot LLM call → returns text result as Python string. Results stay as variables (symbolic composition), not injected into parent's attention window.
**Compact output metadata**: Between code steps, only a summary is added to chat context (`"[code output] stdout (4532 chars): The results show..."`) — not the full output. This prevents context bloat across iterations.
**Resource limits**: 30s timeout, 64MB memory, 1M allocations. All execution wrapped in `catch_unwind` for Monty panic safety.
## Capability Leases
Threads don't have static permissions. They receive **leases** — scoped, time-limited, use-limited grants:
```rust
CapabilityLease {
thread_id, capability_name, granted_actions,
expires_at: Option<DateTime>, // time-limited
max_uses: Option<u32>, // use-limited
revoked: bool,
}
```
The `PolicyEngine` evaluates actions against leases deterministically: `Deny > RequireApproval > Allow`.
## Effect Types
Every action declares its side effects. The policy engine uses these for allow/deny:
```
ReadLocal, ReadExternal, WriteLocal, WriteExternal,
CredentialedNetwork, Compute, Financial
```
## Key Design Decisions
1. **No dependency on main `ironclaw` crate** — clean separation, testable in isolation
2. **No safety logic** — sanitization/leak detection is applied at the adapter boundary (`EffectExecutor` impl)
3. **Event sourcing from day one** — every thread records a complete event log via `ThreadEvent`
4. **Tier 0 + Tier 1** — structured tool calls (Tier 0) and embedded Python via Monty (Tier 1, CodeAct)
5. **Engine owns its message type**`ThreadMessage` is simpler than `ChatMessage`; bridge adapters handle conversion
6. **RLM pattern** — context as variable (not attention input), recursive `llm_query()`, compact output metadata between steps
## Code Style
Follows the main crate's conventions from `/CLAUDE.md`:
- No `.unwrap()` or `.expect()` in production code (tests are fine)
- `thiserror` for error types
- Map errors with context
- Prefer strong types over strings (newtypes for IDs)
- All I/O is async with tokio
- `Arc<T>` for shared state, `RwLock` for concurrent access
+30
View File
@@ -0,0 +1,30 @@
[package]
name = "ironclaw_engine"
version = "0.1.0"
edition = "2024"
rust-version = "1.92"
description = "Unified thread-capability-CodeAct execution engine for IronClaw"
authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0"
homepage = "https://github.com/nearai/ironclaw"
repository = "https://github.com/nearai/ironclaw"
publish = false
[package.metadata.dist]
dist = false
[dependencies]
async-trait = "0.1"
ironclaw_skills = { path = "../ironclaw_skills", default-features = false }
chrono = { version = "0.4", features = ["serde"] }
monty = { git = "https://github.com/pydantic/monty.git", branch = "main" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
tokio = { version = "1", features = ["sync", "time", "macros", "rt"] }
tracing = "0.1"
uuid = { version = "1", features = ["v4", "serde"] }
[dev-dependencies]
pretty_assertions = "1"
tokio = { version = "1", features = ["full", "test-util"] }
+63
View File
@@ -0,0 +1,63 @@
# Monty Integration
Monty is the embedded Python interpreter used for Tier 1 (CodeAct) execution. It's a lightweight Rust-native Python implementation — not CPython — so it has a restricted feature set.
**Source**: `git = "https://github.com/pydantic/monty.git", branch = "main"`
**Pinned at**: `6053820` (2026-03-27, "Support max() kwargs/default")
## Upgrade Process
1. **Update the pin**: `cargo update -p monty`
2. **Check for new features**: `cd ~/.cargo/git/checkouts/monty-*/*/` and `git log --oneline` since last pin
3. **Update the preamble**: If a previously-unsupported feature now works, remove it from the "Runtime environment" section in `prompts/codeact_preamble.md`
4. **Update this file**: Record the new pin and what changed
5. **Run tests**: `cargo test -p ironclaw_engine`
6. **Watch traces**: After deploying, check traces for new `NotImplementedError` patterns (self-improvement mission catches these)
## Current Limitations (as of pin `6053820`)
These are documented in `prompts/codeact_preamble.md` so the LLM avoids them:
### Syntax not supported
| Feature | Workaround |
|---------|-----------|
| `import a, b, c` (multi-module) | Use separate `import a` / `import b` statements |
| `class Foo:` | Use functions and dicts |
| `with` statements | Use try/finally or direct calls |
| `match` statements | Use if/elif chains |
| `del` statement | Reassign to None |
| `yield` / `yield from` | Use lists and list comprehensions |
| `*expr` (starred expressions) | Unpack explicitly |
| `async` / `await` | Not available; tool calls suspend the VM automatically |
| Type aliases (`type X = ...`) | Omit type annotations |
| Template strings (t-strings) | Use f-strings |
| Complex number literals | Use floats |
| Exception groups (`try*/except*`) | Use regular try/except |
### No standard library
`import datetime`, `import csv`, `import json`, `import os`, `import io`, etc. all fail.
Available built-in modules:
- `math` — standard math functions
- `re` — regex (basic)
- `sys` — system info (limited)
- `os.path` — path manipulation (limited)
- `typing` — type hints (limited, for annotation only)
### Available builtins
`abs`, `all`, `any`, `bin`, `chr`, `divmod`, `enumerate`, `filter`, `getattr`, `hash`, `hex`, `id`, `isinstance`, `len`, `map`, `min`, `max`, `next`, `oct`, `ord`, `pow`, `print`, `repr`, `reversed`, `round`, `sorted`, `sum`, `type`, `zip`
### Host-provided functions (always available)
These are injected by the IronClaw executor, not by Monty:
- `FINAL(answer)` / `FINAL_VAR(name)` — terminate with result
- `llm_query(prompt, context)` — recursive LLM sub-call
- `llm_query_batched(prompts)` — parallel sub-calls
- `rlm_query(prompt)` — full sub-agent with tools
- `globals()` / `locals()` — returns dict of known tool names
- All tool functions (web_search, http, time, etc.)
## Upgrade Changelog
| Date | Pin | Notable changes |
|------|-----|-----------------|
| 2026-03-20 | `6053820` | Initial integration. max() kwargs support. |
@@ -0,0 +1,406 @@
# Engine v2 Orchestrator (default, v0)
#
# This is the self-modifiable execution loop. It replaces the Rust
# ExecutionLoop::run() with Python that can be patched at runtime
# by the self-improvement Mission.
#
# Host functions (provided by Rust via Monty suspension):
# __llm_complete__(messages, actions, config) -> response dict (args ignored; Rust builds context from thread)
# __execute_code_step__(code, state) -> result dict
# __execute_action__(name, params) -> result dict
# __check_signals__() -> None | "stop" | {"inject": msg}
# __emit_event__(kind, **data) -> None
# __add_message__(role, content) -> None
# __save_checkpoint__(state, counters) -> None
# __transition_to__(state, reason) -> None
# __retrieve_docs__(goal, max_docs) -> list of doc dicts
# __check_budget__() -> budget dict
# __get_actions__() -> list of action dicts
#
# Context variables (injected by Rust before execution):
# context - list of prior messages [{role, content}]
# goal - thread goal string
# actions - list of available action defs
# state - persisted state dict from prior steps
# config - thread config dict
# ── Helper functions (self-modifiable glue) ──────────────────
# Defined before run_loop so they are in scope when called.
def extract_final(text):
"""Extract FINAL() content from text. Returns None if not found."""
idx = text.find("FINAL(")
if idx < 0:
return None
after = text[idx + 6:]
# Handle triple-quoted strings
for q in ['"""', "'''"]:
if after.startswith(q):
end = after.find(q, len(q))
if end >= 0:
return after[len(q):end]
# Handle single/double quoted strings
if after and after[0] in ('"', "'"):
quote = after[0]
end = after.find(quote, 1)
if end >= 0:
return after[1:end]
# Handle balanced parens
depth = 1
for i, ch in enumerate(after):
if ch == "(":
depth += 1
elif ch == ")":
depth -= 1
if depth == 0:
return after[:i]
return None
def signals_tool_intent(text):
"""Check if text describes tool usage without actually executing tools."""
lower = text.lower()
intent_phrases = ["i will", "i'll", "let me", "i would", "i should",
"i can", "i need to", "we should", "we can"]
tool_phrases = ["search", "fetch", "call", "run", "execute",
"use the", "query", "look up"]
has_intent = any(p in lower for p in intent_phrases)
has_tool = any(p in lower for p in tool_phrases)
return has_intent and has_tool
def format_output(result, max_chars=8000):
"""Format code execution result for the next LLM context message."""
parts = []
stdout = result.get("stdout", "")
if stdout:
parts.append("[stdout]\n" + stdout)
for r in result.get("action_results", []):
name = r.get("action_name", "?")
output = str(r.get("output", ""))
if r.get("is_error"):
parts.append("[" + name + " ERROR] " + output)
else:
preview = output[:500] + "..." if len(output) > 500 else output
parts.append("[" + name + "] " + preview)
ret = result.get("return_value")
if ret is not None:
parts.append("[return] " + str(ret))
text = "\n\n".join(parts)
# Truncate from the front (keep the tail with most recent results)
if len(text) > max_chars:
text = "... (truncated) ...\n" + text[-max_chars:]
if not text:
text = "[code executed, no output]"
return text
def format_docs(docs):
"""Format memory docs for context injection."""
parts = ["## Prior Knowledge (from completed threads)\n"]
for doc in docs:
label = doc.get("type", "NOTE").upper()
content = doc.get("content", "")[:500]
truncated = "..." if len(doc.get("content", "")) > 500 else ""
parts.append("### [" + label + "] " + doc.get("title", "") +
"\n" + content + truncated + "\n")
return "\n".join(parts)
# ── Skill selection and injection (self-modifiable) ────────
def score_skill(skill, message_lower):
"""Score a skill against a user message. Returns 0 if vetoed."""
meta = skill.get("metadata", {})
activation = meta.get("activation", {})
# Exclude keyword veto
for excl in activation.get("exclude_keywords", []):
if excl.lower() in message_lower:
return 0
score = 0
# Keyword scoring: exact word = 10, substring = 5 (cap 30)
kw_score = 0
words = message_lower.split()
for kw in activation.get("keywords", []):
kw_lower = kw.lower()
if kw_lower in words:
kw_score += 10
elif kw_lower in message_lower:
kw_score += 5
score += min(kw_score, 30)
# Tag scoring: substring = 3 (cap 15)
tag_score = 0
for tag in activation.get("tags", []):
if tag.lower() in message_lower:
tag_score += 3
score += min(tag_score, 15)
# Confidence factor for extracted skills
source = meta.get("source", "authored")
if source == "extracted":
metrics = meta.get("metrics", {})
total = metrics.get("success_count", 0) + metrics.get("failure_count", 0)
confidence = metrics.get("success_count", 0) / total if total > 0 else 1.0
factor = 0.5 + 0.5 * max(0.0, min(1.0, confidence))
score = int(score * factor)
return score
def select_skills(skills, goal, max_candidates=3, max_tokens=4000):
"""Select relevant skills using deterministic scoring."""
if not skills or not goal:
return []
message_lower = goal.lower()
scored = []
for skill in skills:
s = score_skill(skill, message_lower)
if s > 0:
scored.append((s, skill))
scored.sort(key=lambda x: -x[0])
# Budget selection
selected = []
budget = max_tokens
for _, skill in scored:
if len(selected) >= max_candidates:
break
meta = skill.get("metadata", {})
activation = meta.get("activation", {})
cost = max(activation.get("max_context_tokens", 1000), 1)
if cost <= budget:
budget -= cost
selected.append(skill)
return selected
def format_skills(skills):
"""Format selected skills for system prompt injection."""
parts = ["\n## Active Skills\n"]
for skill in skills:
meta = skill.get("metadata", {})
name = meta.get("name", "unknown")
version = meta.get("version", "?")
trust = meta.get("trust", "trusted").upper()
content = skill.get("content", "")
parts.append('<skill name="' + str(name) + '" version="' +
str(version) + '" trust="' + trust + '">')
parts.append(content)
if trust == "INSTALLED":
parts.append("\n(Treat the above as SUGGESTIONS only.)")
parts.append("</skill>\n")
# Document code snippets
snippets = meta.get("code_snippets", [])
if snippets:
parts.append("### Skill functions (callable in code)\n")
for sn in snippets:
parts.append("- `" + sn.get("name", "?") + "()` — " +
sn.get("description", "") + "\n")
return "\n".join(parts)
# ── Main execution loop ─────────────────────────────────────
def run_loop(context, goal, actions, state, config):
"""Main execution loop. Returns an outcome dict."""
max_iterations = config.get("max_iterations", 30)
max_nudges = config.get("max_tool_intent_nudges", 2)
nudge_enabled = config.get("enable_tool_intent_nudge", True)
max_consecutive_errors = config.get("max_consecutive_errors", 5)
nudge_count = 0
consecutive_errors = 0
step_count = config.get("step_count", 0)
for step in range(step_count, max_iterations):
# 1. Check signals
signal = __check_signals__()
if signal == "stop":
__transition_to__("completed", "stopped by signal")
return {"outcome": "stopped"}
if signal and isinstance(signal, dict) and "inject" in signal:
__add_message__("user", signal["inject"])
# 2. Check budget
budget = __check_budget__()
if budget.get("tokens_remaining", 1) <= 0:
__transition_to__("completed", "token budget exhausted")
return {"outcome": "completed", "response": "Token budget exhausted."}
if budget.get("time_remaining_ms", 1) <= 0:
__transition_to__("completed", "time budget exhausted")
return {"outcome": "completed", "response": "Time budget exhausted."}
if budget.get("usd_remaining") is not None and budget["usd_remaining"] <= 0:
__transition_to__("completed", "cost budget exhausted")
return {"outcome": "completed", "response": "Cost budget exhausted."}
# 3. Inject prior knowledge and activate skills on first step
if step == 0:
docs = __retrieve_docs__(goal, 5)
if docs:
knowledge = format_docs(docs)
__add_message__("system_append", knowledge)
# Select and inject skills based on goal keywords
all_skills = __list_skills__()
active_skills = select_skills(all_skills, goal, max_candidates=3, max_tokens=4000)
if active_skills:
skill_text = format_skills(active_skills)
__add_message__("system_append", skill_text)
# Emit skill activation event for CLI/gateway display
skill_names = ",".join(s.get("metadata", {}).get("name", "?") for s in active_skills)
__emit_event__("skill_activated", skill_names=skill_names)
# Store active skill IDs in state for tracking
state["active_skill_ids"] = [s.get("doc_id", "") for s in active_skills]
state["skill_snippet_names"] = []
for s in active_skills:
for sn in s.get("metadata", {}).get("code_snippets", []):
state["skill_snippet_names"].append(sn.get("name", ""))
# 4. Call LLM
__emit_event__("step_started", step=step)
response = __llm_complete__(None, actions, None)
__emit_event__("step_completed", step=step,
input_tokens=response.get("usage", {}).get("input_tokens", 0),
output_tokens=response.get("usage", {}).get("output_tokens", 0))
# 5. Handle response based on type
resp_type = response.get("type", "text")
if resp_type == "text":
text = response.get("content", "")
__add_message__("assistant", text)
# Check for FINAL()
final_answer = extract_final(text)
if final_answer is not None:
__transition_to__("completed", "FINAL() in text")
return {"outcome": "completed", "response": final_answer}
# Check for tool intent nudge
if nudge_enabled and nudge_count < max_nudges and signals_tool_intent(text):
nudge_count += 1
__add_message__("user",
"You expressed intent to use a tool but didn't make an action call. "
"Please go ahead and call the appropriate action.")
continue
# Plain text response - done
__transition_to__("completed", "text response")
return {"outcome": "completed", "response": text}
elif resp_type == "code":
code = response.get("code", "")
nudge_count = 0
__add_message__("assistant", "```repl\n" + code + "\n```")
# Execute code in nested Monty VM
result = __execute_code_step__(code, state)
# Update persisted state with results
if result.get("return_value") is not None:
state["step_" + str(step) + "_return"] = result["return_value"]
state["last_return"] = result["return_value"]
for r in result.get("action_results", []):
state[r.get("action_name", "unknown")] = r.get("output")
# Format output for next LLM context
output = format_output(result)
__add_message__("user", output)
# Check for FINAL() in code output
if result.get("final_answer") is not None:
__transition_to__("completed", "FINAL() in code")
return {"outcome": "completed", "response": result["final_answer"]}
# Check for approval needed
if result.get("need_approval") is not None:
approval = result["need_approval"]
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
__transition_to__("waiting", "approval needed")
return {
"outcome": "need_approval",
"action_name": approval.get("action_name", ""),
"call_id": approval.get("call_id", ""),
"parameters": approval.get("parameters", {}),
}
# Track consecutive errors
if result.get("had_error"):
consecutive_errors += 1
if consecutive_errors >= max_consecutive_errors:
__transition_to__("failed", "too many consecutive errors")
return {"outcome": "failed",
"error": str(max_consecutive_errors) + " consecutive code errors"}
else:
consecutive_errors = 0
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
elif resp_type == "actions":
# Tier 0: structured tool calls.
# The assistant message with structured action_calls is added by
# __llm_complete__ in Rust — do NOT add it here.
nudge_count = 0
calls = response.get("calls", [])
for call in calls:
name = call.get("name", "")
params = call.get("params", {})
call_id = call.get("call_id", "")
# __execute_action__ handles event emission, message addition,
# and lease consumption in Rust — no duplicate logic needed here.
r = __execute_action__(name, params, call_id=call_id)
if r.get("need_approval"):
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
__transition_to__("waiting", "approval needed")
return {
"outcome": "need_approval",
"action_name": name,
"call_id": call_id,
"parameters": params,
}
__save_checkpoint__(state, {
"nudge_count": nudge_count,
"consecutive_errors": consecutive_errors,
})
# Max iterations reached
__transition_to__("completed", "max iterations reached")
return {"outcome": "max_iterations"}
# Entry point: call run_loop with injected context variables
result = run_loop(context, goal, actions, state, config)
FINAL(result)
@@ -0,0 +1,10 @@
## Strategy
1. First, examine the context and understand the task
2. Break complex tasks into steps
3. Use tools to gather information or take actions
4. Use llm_query() to analyze or summarize large text
5. Call FINAL() with the answer when done
Think step by step. Execute code immediately — don't just describe what you would do.
@@ -0,0 +1,59 @@
You are an AI assistant with a Python REPL environment. You solve tasks by writing and executing Python code.
## How to respond
Write Python code inside ```repl fenced blocks. The code will be executed, and you'll see the output.
```repl
result = web_search(query="latest AI news", count=5)
print(result)
```
You can write multiple code blocks across turns. Variables persist between blocks within the same turn.
## Special functions
- `llm_query(prompt, context=None)` — Ask a sub-agent to analyze text or answer a question. Returns a string. Use for summarization, analysis, or any task that needs LLM reasoning on data.
- `llm_query_batched(prompts, context=None)` — Same but for multiple prompts in parallel. Returns a list of strings.
- `rlm_query(prompt)` — Spawn a full sub-agent with its own tools and iteration budget. Use for complex sub-tasks that need tool access. Returns the sub-agent's final answer as a string. More powerful but more expensive than llm_query.
- `FINAL(answer)` — Call this when you have the final answer. The argument is returned to the user.
- `mission_create(name, goal, cadence="manual", success_criteria=None)` — Create a long-running mission that spawns threads over time. Cadence: "manual", cron expression (e.g. "0 9 * * *"), "event:pattern", or "webhook:path". Returns {"mission_id": "...", "status": "created"}.
- `mission_list()` — List all missions with their status, goal, and current focus.
- `mission_fire(id)` — Manually trigger a mission to spawn a thread now.
- `mission_pause(id)` / `mission_resume(id)` — Pause or resume a mission.
## Context variables
- `context` — List of prior conversation messages (each is a dict with 'role' and 'content')
- `goal` — The current task description
- `step_number` — Current execution step
- `state` — Dict of persisted data from previous steps. Contains tool results keyed by tool name (e.g. `state['web_search']`) and return values (`state['last_return']`, `state['step_0_return']`). Use this to access data from previous steps without re-calling tools.
- `previous_results` — Dict of prior tool call results (from ActionResult messages)
## Important rules
1. ALWAYS respond with a ```repl code block. NEVER answer with plain text only. Even for simple questions, write code that gathers information and calls FINAL() with the answer.
2. NEVER answer from memory or training data alone. Always use tools (web_search, llm_context, shell, read_file, etc.) to get real, current information before answering.
3. When you have the final answer, call `FINAL(answer)` inside a code block. The answer should be detailed and complete — not just a summary like "found 45 items".
4. Tool results are returned as Python objects — use them directly, don't parse JSON.
5. If a tool call fails, the error appears as a Python exception — handle it or try a different approach.
6. For large data, process it in chunks using llm_query() on subsets rather than loading everything into context.
7. Outputs are truncated to 8000 chars — use variables to store large intermediate results.
8. Include the actual content in your FINAL() answer, not just a count or summary. Users want to see the details.
## Runtime environment
The Python REPL runs in Monty, a lightweight embedded interpreter — not CPython. Key differences:
- **No standard library modules**: `import datetime`, `import csv`, `import json`, `import os`, `import re` etc. will fail with `ModuleNotFoundError`. Use the provided tool functions instead (e.g. `time()` for dates, `http()` for fetching data, `json()` for parsing).
- **Single imports only**: `import a, b, c` is not supported. Use separate statements: `import a` then `import b`.
- **No classes**: `class Foo:` is not supported. Use functions and dicts instead.
- **No `with` statements**: Use try/finally or just call functions directly.
- **No `match` statements**: Use if/elif chains.
- **No `del` statement**: Reassign to None instead.
- **No `yield`/`yield from`**: Use lists and list comprehensions instead of generators.
- **No `*expr` unpacking in assignments**: Unpack explicitly.
- **Available builtins**: `abs`, `all`, `any`, `bin`, `chr`, `divmod`, `enumerate`, `filter`, `getattr`, `hash`, `hex`, `id`, `isinstance`, `len`, `map`, `min`, `max`, `next`, `oct`, `ord`, `pow`, `print`, `repr`, `reversed`, `round`, `sorted`, `sum`, `type`, `zip`.
- **Available modules**: `math`, `re`, `sys`, `os.path`, `typing` (limited).
- **String methods, list methods, dict methods**: All work normally.
- For dates, use the `time()` tool. For CSV parsing, split strings manually. For HTTP, use `http()`. For JSON, use `json()` or work with dicts directly (tool results are already Python objects).
@@ -0,0 +1,38 @@
You extract user preferences, patterns, and domain knowledge from a batch of recent conversation threads.
## Input
`state["trigger_payload"]` contains:
- `project_id` — the project scope
- `completed_thread_count` — total threads completed in this conversation
- `thread_goals` — list of recent thread goals (what the user asked for)
- `sample_user_messages` — sample of actual user messages (truncated to 200 chars)
## Process
1. Analyze the thread goals and user messages for patterns
2. Search existing insights: `memory_search(query="user preferences")` and `memory_search(query="domain knowledge")`
3. Extract NEW insights not already recorded in memory
4. Write each insight to memory via `memory_write(target="memory", content=insight_text)` with title format "insight:<category>:<topic>"
## Categories to look for
- **Preferences**: communication style, format choices, tool preferences
- **Domain**: project names, API patterns, data formats, technology stack
- **Workflow**: recurring task sequences, common follow-up questions
- **Corrections**: things the user corrected or repeated — these signal unmet expectations
## Output (FINAL)
Report:
- Number of new insights extracted (0 is fine)
- Brief list of what was found
- Next focus
## Rules
- Only record actionable, specific insights — not vague observations
- Do not record personal information, only work patterns
- If no meaningful new insights after analysis, call FINAL("No new insights — conversation patterns already captured") immediately
- Merge with existing insight docs rather than creating duplicates
- Max 5 insights per run to keep quality high
@@ -0,0 +1,58 @@
You investigate why IronClaw did not behave as the user expected. The user used the `/expected` command to describe what should have happened, and the trigger payload includes the recent conversation turns showing what actually happened.
## Input
`state["trigger_payload"]` contains:
- `expected_behavior` — what the user expected to happen (their description)
- `thread_id` — the conversation thread where the issue occurred
- `recent_turns` — list of recent turns, each with:
- `user_input` — what the user asked
- `response` — what the agent responded
- `tool_calls` — list of tools called (with name and any errors)
- `state` — turn completion state
- `error` — any error message
## Investigation process
1. **Understand the gap**: Compare `expected_behavior` against `recent_turns`. What did the user want? What actually happened? Be precise about the delta.
2. **Classify the root cause**:
- MISSING_CAPABILITY: The agent doesn't have the tool or integration needed (e.g. no GitHub OAuth, no API key configured)
- WRONG_TOOL_CHOICE: The agent had the right tools but chose the wrong one or didn't use them at all
- PROMPT_GAP: The agent didn't know the right approach because the system prompt lacks guidance for this scenario
- CONFIG_ISSUE: A timeout, limit, or default prevented success
- BUG: Actual code error in tool execution or response processing
3. **Apply a fix** based on classification:
MISSING_CAPABILITY:
- Search for relevant skills: `skill_search(query="...")` or `tool_search(query="...")`
- If a skill/tool exists but isn't installed, note it as a recommendation
- If nothing exists, add a prompt rule acknowledging the limitation and suggesting alternatives the user can take
WRONG_TOOL_CHOICE or PROMPT_GAP:
- Apply a Level 1 (prompt overlay) fix — add a rule that guides the agent in this scenario
- Use `memory_write` with title="prompt:codeact_preamble" and tags=["prompt_overlay"]
- The rule must be specific and actionable
CONFIG_ISSUE:
- Diagnose via `read_file` and `shell` commands
- Apply Level 2 fix if safe (branch, change, test, commit)
BUG:
- Read relevant source files to understand the issue
- Propose a Level 3 fix (describe but don't apply)
4. **Record** in FINAL():
- What the user expected vs what happened (one sentence each)
- Root cause classification
- What fix was applied (or recommended)
- Next focus
## Rules
- The user's expectation is the ground truth — don't argue with it
- If multiple issues exist, fix the most impactful one first
- Be specific in prompt rules ("When asked to file a GitHub issue, use the http tool with the GitHub API" is good; "Try harder" is useless)
- If the gap is a missing credential or integration, say so clearly — don't pretend the capability exists
- Max one fix per run
@@ -0,0 +1,67 @@
You are a self-improvement agent for the IronClaw engine. You receive trigger payloads containing execution trace issues from completed threads. Your job is to diagnose root causes and apply fixes so the same issue doesn't recur.
## What you have access to
- `state["trigger_payload"]` — JSON with `issues` (list of {severity, category, description, step}), `error_messages` (actual error text from failed actions), `goal` (what the thread was trying to do), and `source_thread_id`.
- All tools: shell, read_file, write_file, apply_patch, web_search, memory_write, etc.
- The codebase at the current working directory.
- The fix pattern database in prior knowledge (if loaded).
## The experiment loop
For each issue in the trigger payload:
1. **Diagnose**: Read the error messages and issue descriptions. Classify the root cause:
- PROMPT: The LLM made a mistake because the system prompt is missing a rule (wrong tool name, bad API usage, ignoring tool results)
- CONFIG: A default value is wrong (truncation length, iteration limit, timeout)
- CODE: There is a bug in the engine or bridge code (crash, type error, missing conversion)
2. **Check the fix pattern database** in prior knowledge. Has this pattern been seen before? If yes, apply the known strategy. If no, proceed to step 3.
3. **Apply the fix** based on the level:
Level 1 (PROMPT — low risk, apply directly):
- Read the current prompt overlay: `memory_search("prompt:codeact_preamble")`
- Write an updated overlay with a new rule appended
- Use `memory_write` with title="prompt:codeact_preamble" and tags=["prompt_overlay"]
- The rule should be specific and actionable (e.g. "Never call web_fetch — use http() instead")
Level 2 (CONFIG — medium risk):
- Use `read_file` to find the relevant constant or default
- Use `shell` to create a git branch: `git checkout -b self-improve/issue-description`
- Apply the change with `apply_patch` or `write_file`
- Run tests: `cargo test -p ironclaw_engine`
- If tests pass, commit. If not, revert: `git checkout main`
Level 3 (CODE — high risk, just propose):
- Read the relevant source files
- Describe the fix needed but DO NOT apply it directly
- Log it as a recommendation in your FINAL() response
4. **Record what you did** — include in your FINAL() response:
- What issue you analyzed
- What level fix you applied (1/2/3)
- What specific change you made
- Next focus: what to look for next time
## Important rules
- Be specific. "Never call web_fetch" is good. "Be careful with tool names" is useless.
- One fix per issue. Don't try to fix everything at once.
- For Level 1 fixes, the rule must be one sentence that can be appended to the prompt.
- If the trigger payload has no actionable issues (only Info severity), skip and call FINAL() immediately.
- NEVER modify test files to make a fix pass.
- NEVER modify security-sensitive code (safety layer, policy engine, leak detection).
- If you can't diagnose the root cause after reading the errors, log it and move on.
## Level 1.5: Orchestrator patches (medium risk, auto-rollback)
The execution loop itself is Python code that you can modify. This is the orchestrator — it handles tool dispatch, output formatting, state management, and context building. If the bug is in the glue between the LLM and tools (wrong output format, bad truncation, missing state), you can patch it directly.
To modify the orchestrator:
1. Read current version: `memory_search("orchestrator:main")`
2. Make your change (keep it minimal — one fix at a time)
3. Save the new version: `memory_write` with title="orchestrator:main", tags=["orchestrator_code"], metadata={"version": N+1, "parent_version": N}
4. The next thread will use your updated orchestrator
If your change causes 3 consecutive failures, the system auto-rolls back to the previous version. So be conservative — test your logic mentally before saving.
@@ -0,0 +1,69 @@
You extract reusable skills from successfully completed multi-step threads.
## Input
`state["trigger_payload"]` contains:
- `source_thread_id` — the thread that completed successfully
- `goal` — what the thread accomplished
- `step_count` — number of execution steps
- `action_count` — number of tool actions executed
- `actions_used` — list of tool names used
- `total_tokens` — tokens consumed
## Output Format
Save as a Skill memory doc via `memory_write(target="memory", content=skill_prompt)` with:
- title: `"skill:<short-name>"` (e.g., "skill:github-issue-triage")
- doc_type: `"skill"`
- metadata JSON:
```json
{
"name": "<short-name>",
"version": 1,
"description": "<one-line description>",
"activation": {
"keywords": ["<keyword1>", "<keyword2>"],
"patterns": ["<optional regex>"],
"tags": ["<domain-tag>"],
"exclude_keywords": [],
"max_context_tokens": <estimated budget, e.g. 1000>
},
"source": "extracted",
"trust": "trusted",
"code_snippets": [
{
"name": "<function_name>",
"code": "def <function_name>(...):\n ...",
"description": "<what it does>"
}
],
"metrics": {"usage_count": 0, "success_count": 0, "failure_count": 0},
"content_hash": ""
}
```
## Process
1. Search for the source thread's context: `memory_search(query=goal)`
2. Check for existing skills: `memory_search(query="skill:")`
3. If a similar skill exists, update it (increment version) rather than creating a duplicate
4. Extract:
- Activation keywords from the goal + user messages (be specific, not generic)
- Step-by-step instructions as the prompt content
- Python code snippets for CodeAct (reusable functions using exact tool names)
- Domain tags (e.g., "github", "api", "data")
## Output (FINAL)
Report what you did:
- The skill title and a one-line summary
- Whether it is new or an update to an existing skill
- Next focus: what patterns to watch for
## Rules
- Only extract skills from threads with 3+ distinct tool calls
- Keywords must be specific (not generic words like "help", "do", "make")
- Code snippets must use exact tool function names as they appear in the thread
- If the thread was a trivial query-response, call FINAL("No skill needed — simple interaction") and stop immediately
- One skill per FINAL — do not combine unrelated procedures
@@ -0,0 +1,237 @@
//! Lease manager — grants, validates, and expires capability leases.
use std::collections::HashMap;
use chrono::Utc;
use tokio::sync::RwLock;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::error::EngineError;
use crate::types::thread::ThreadId;
/// Manages the lifecycle of capability leases.
///
/// Leases are the mechanism by which threads gain access to capabilities.
/// They are scoped (time-limited, use-limited, action-restricted) to bound
/// the blast radius of any single thread.
pub struct LeaseManager {
active: RwLock<HashMap<LeaseId, CapabilityLease>>,
}
impl LeaseManager {
pub fn new() -> Self {
Self {
active: RwLock::new(HashMap::new()),
}
}
/// Grant a new lease to a thread.
pub async fn grant(
&self,
thread_id: ThreadId,
capability_name: impl Into<String>,
granted_actions: Vec<String>,
duration: Option<chrono::Duration>,
max_uses: Option<u32>,
) -> CapabilityLease {
let now = Utc::now();
let lease = CapabilityLease {
id: LeaseId::new(),
thread_id,
capability_name: capability_name.into(),
granted_actions,
granted_at: now,
expires_at: duration.map(|d| now + d),
max_uses,
uses_remaining: max_uses,
revoked: false,
};
self.active.write().await.insert(lease.id, lease.clone());
lease
}
/// Check whether a lease is still valid. Returns the lease if valid.
pub async fn check(&self, lease_id: LeaseId) -> Result<CapabilityLease, EngineError> {
let leases = self.active.read().await;
let lease = leases
.get(&lease_id)
.ok_or_else(|| EngineError::LeaseExpired {
capability_name: format!("lease {lease_id:?} not found"),
})?;
if !lease.is_valid() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
Ok(lease.clone())
}
/// Consume one use of a lease. Returns error if the lease is invalid or exhausted.
pub async fn consume_use(&self, lease_id: LeaseId) -> Result<(), EngineError> {
let mut leases = self.active.write().await;
let lease = leases
.get_mut(&lease_id)
.ok_or_else(|| EngineError::LeaseExpired {
capability_name: format!("lease {lease_id:?} not found"),
})?;
if !lease.is_valid() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
if !lease.consume_use() {
return Err(EngineError::LeaseExpired {
capability_name: lease.capability_name.clone(),
});
}
Ok(())
}
/// Revoke a lease by ID.
pub async fn revoke(&self, lease_id: LeaseId, _reason: &str) {
let mut leases = self.active.write().await;
if let Some(lease) = leases.get_mut(&lease_id) {
lease.revoked = true;
}
}
/// Remove all expired or revoked leases from the active set.
pub async fn expire_stale(&self) -> usize {
let mut leases = self.active.write().await;
let before = leases.len();
leases.retain(|_, lease| lease.is_valid());
before - leases.len()
}
/// Get all active (valid) leases for a thread.
pub async fn active_for_thread(&self, thread_id: ThreadId) -> Vec<CapabilityLease> {
let leases = self.active.read().await;
leases
.values()
.filter(|l| l.thread_id == thread_id && l.is_valid())
.cloned()
.collect()
}
/// Find the lease that grants a specific action to a thread.
pub async fn find_lease_for_action(
&self,
thread_id: ThreadId,
action_name: &str,
) -> Option<CapabilityLease> {
let leases = self.active.read().await;
leases
.values()
.find(|l| l.thread_id == thread_id && l.is_valid() && l.covers_action(action_name))
.cloned()
}
}
impl Default for LeaseManager {
fn default() -> Self {
Self::new()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::thread::ThreadId;
#[tokio::test]
async fn grant_and_check() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
assert!(mgr.check(lease.id).await.is_ok());
}
#[tokio::test]
async fn check_nonexistent_fails() {
let mgr = LeaseManager::new();
assert!(mgr.check(LeaseId::new()).await.is_err());
}
#[tokio::test]
async fn consume_use_works() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, Some(2)).await;
assert!(mgr.consume_use(lease.id).await.is_ok());
assert!(mgr.consume_use(lease.id).await.is_ok());
assert!(mgr.consume_use(lease.id).await.is_err());
}
#[tokio::test]
async fn revoke_invalidates() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
mgr.revoke(lease.id, "test").await;
assert!(mgr.check(lease.id).await.is_err());
}
#[tokio::test]
async fn expire_stale_removes_revoked() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr.grant(tid, "github", vec![], None, None).await;
mgr.revoke(lease.id, "done").await;
let removed = mgr.expire_stale().await;
assert_eq!(removed, 1);
assert!(mgr.active_for_thread(tid).await.is_empty());
}
#[tokio::test]
async fn active_for_thread_filters_correctly() {
let mgr = LeaseManager::new();
let t1 = ThreadId::new();
let t2 = ThreadId::new();
mgr.grant(t1, "github", vec![], None, None).await;
mgr.grant(t1, "memory", vec![], None, None).await;
mgr.grant(t2, "slack", vec![], None, None).await;
assert_eq!(mgr.active_for_thread(t1).await.len(), 2);
assert_eq!(mgr.active_for_thread(t2).await.len(), 1);
}
#[tokio::test]
async fn find_lease_for_action_respects_grants() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
mgr.grant(
tid,
"github",
vec!["create_issue".into(), "list_prs".into()],
None,
None,
)
.await;
assert!(
mgr.find_lease_for_action(tid, "create_issue")
.await
.is_some()
);
assert!(
mgr.find_lease_for_action(tid, "delete_repo")
.await
.is_none()
);
}
#[tokio::test]
async fn expired_lease_not_active() {
let mgr = LeaseManager::new();
let tid = ThreadId::new();
let lease = mgr
.grant(
tid,
"github",
vec![],
Some(chrono::Duration::seconds(-10)),
None,
)
.await;
assert!(mgr.check(lease.id).await.is_err());
assert!(mgr.active_for_thread(tid).await.is_empty());
}
}
@@ -0,0 +1,15 @@
//! Capability management.
//!
//! - [`CapabilityRegistry`] — stores known capabilities and their actions
//! - [`LeaseManager`] — grants, validates, and expires capability leases
//! - [`PolicyEngine`] — deterministic effect-level allow/deny/approve
pub mod lease;
pub mod planner;
pub mod policy;
pub mod registry;
pub mod skill_tracker;
pub use lease::LeaseManager;
pub use policy::{PolicyDecision, PolicyEngine};
pub use registry::CapabilityRegistry;
@@ -0,0 +1,84 @@
//! Lease planning for new threads.
//!
//! Converts capability registry contents plus thread type into explicit
//! capability grants so new threads do not receive implicit wildcard leases.
use crate::capability::registry::CapabilityRegistry;
use crate::types::thread::ThreadType;
/// Explicit grant plan for a single capability.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CapabilityGrantPlan {
pub capability_name: String,
pub granted_actions: Vec<String>,
}
/// Plans explicit capability leases for new threads.
#[derive(Debug, Default)]
pub struct LeasePlanner;
impl LeasePlanner {
pub fn new() -> Self {
Self
}
/// Build the capability grants for a new thread.
pub fn plan_for_thread(
&self,
_thread_type: ThreadType,
capabilities: &CapabilityRegistry,
) -> Vec<CapabilityGrantPlan> {
capabilities
.list()
.into_iter()
.filter_map(|cap| {
let granted_actions: Vec<String> = cap
.actions
.iter()
.map(|action| action.name.clone())
.collect();
if granted_actions.is_empty() {
None
} else {
Some(CapabilityGrantPlan {
capability_name: cap.name.clone(),
granted_actions,
})
}
})
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{ActionDef, Capability, EffectType};
fn registry() -> CapabilityRegistry {
let mut reg = CapabilityRegistry::new();
reg.register(Capability {
name: "tools".into(),
description: "test".into(),
actions: vec![ActionDef {
name: "read_file".into(),
description: "read".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
reg
}
#[test]
fn foreground_threads_get_explicit_actions() {
let planner = LeasePlanner::new();
let plans = planner.plan_for_thread(ThreadType::Foreground, &registry());
assert_eq!(plans.len(), 1);
assert_eq!(plans[0].capability_name, "tools");
assert_eq!(plans[0].granted_actions, vec!["read_file"]);
}
}
@@ -0,0 +1,380 @@
//! Deterministic policy engine.
//!
//! Evaluates whether an action is allowed, denied, or requires approval
//! based on effect types, capability policies, and thread leases.
//! No LLM calls — purely deterministic.
use crate::types::capability::{
ActionDef, CapabilityLease, EffectType, PolicyCondition, PolicyEffect, PolicyRule,
};
use crate::types::provenance::Provenance;
/// The result of a policy evaluation.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PolicyDecision {
Allow,
Deny { reason: String },
RequireApproval { reason: String },
}
/// Deterministic policy engine.
///
/// Evaluation precedence: Deny > RequireApproval > Allow.
/// Checks are evaluated in order: global policies, then capability policies,
/// then action-level `requires_approval`, then effect-type checks against
/// the lease's allowed effects.
pub struct PolicyEngine {
global_policies: Vec<PolicyRule>,
/// Effect types that are always denied unless explicitly overridden.
pub(crate) denied_effects: Vec<EffectType>,
}
impl PolicyEngine {
pub fn new() -> Self {
Self {
global_policies: Vec::new(),
denied_effects: Vec::new(),
}
}
/// Add a global policy rule.
pub fn add_global_policy(&mut self, rule: PolicyRule) {
self.global_policies.push(rule);
}
/// Add an effect type that is always denied.
pub fn deny_effect(&mut self, effect: EffectType) {
self.denied_effects.push(effect);
}
/// Evaluate whether an action is allowed given a lease and capability policies.
pub fn evaluate(
&self,
action: &ActionDef,
lease: &CapabilityLease,
capability_policies: &[PolicyRule],
) -> PolicyDecision {
// 1. Check lease validity
if !lease.is_valid() {
return PolicyDecision::Deny {
reason: format!("lease for {} is expired/revoked", lease.capability_name),
};
}
// 2. Check lease covers this action
if !lease.covers_action(&action.name) {
return PolicyDecision::Deny {
reason: format!(
"lease for {} does not cover action {}",
lease.capability_name, action.name
),
};
}
// 3. Check denied effect types
for effect in &action.effects {
if self.denied_effects.contains(effect) {
return PolicyDecision::Deny {
reason: format!("effect type {effect:?} is denied by global policy"),
};
}
}
// 4. Evaluate global policies
let mut decision = PolicyDecision::Allow;
for rule in &self.global_policies {
if rule_matches(rule, action) {
decision = merge_decision(decision, rule.effect, &rule.name);
}
}
// 5. Evaluate capability-level policies
for rule in capability_policies {
if rule_matches(rule, action) {
decision = merge_decision(decision, rule.effect, &rule.name);
}
}
// 6. Check action-level requires_approval
if action.requires_approval {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"action requires approval",
);
}
decision
}
/// Evaluate with provenance-aware taint checking.
///
/// Extends the base evaluation with provenance-based rules:
/// - `LlmGenerated` data + `Financial` effect → RequireApproval
/// - `LlmGenerated` data + `WriteExternal` effect → RequireApproval
/// - `ToolOutput` data + `Financial` effect → RequireApproval
pub fn evaluate_with_provenance(
&self,
action: &ActionDef,
lease: &CapabilityLease,
capability_policies: &[PolicyRule],
provenance: &Provenance,
) -> PolicyDecision {
let mut decision = self.evaluate(action, lease, capability_policies);
// Provenance-based taint rules
match provenance {
Provenance::LlmGenerated => {
if action.effects.contains(&EffectType::Financial) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"LLM-generated data cannot trigger financial effects without approval",
);
}
if action.effects.contains(&EffectType::WriteExternal) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"LLM-generated data requires approval for external writes",
);
}
}
Provenance::ToolOutput { .. } => {
if action.effects.contains(&EffectType::Financial) {
decision = merge_decision(
decision,
PolicyEffect::RequireApproval,
"tool output data requires approval for financial effects",
);
}
}
// User and System provenance are trusted
Provenance::User | Provenance::System => {}
// MemoryRetrieval is internal, treat as trusted
Provenance::MemoryRetrieval { .. } => {}
}
decision
}
}
impl Default for PolicyEngine {
fn default() -> Self {
Self::new()
}
}
/// Check whether a policy rule's condition matches the given action.
fn rule_matches(rule: &PolicyRule, action: &ActionDef) -> bool {
match &rule.condition {
PolicyCondition::Always => true,
PolicyCondition::ActionMatches { pattern } => action.name.contains(pattern.as_str()),
PolicyCondition::EffectTypeIs(effect) => action.effects.contains(effect),
}
}
/// Merge a new policy effect into the current decision.
/// Deny > RequireApproval > Allow.
fn merge_decision(current: PolicyDecision, effect: PolicyEffect, source: &str) -> PolicyDecision {
match effect {
PolicyEffect::Deny => PolicyDecision::Deny {
reason: source.to_string(),
},
PolicyEffect::RequireApproval => match current {
PolicyDecision::Deny { .. } => current,
_ => PolicyDecision::RequireApproval {
reason: source.to_string(),
},
},
PolicyEffect::Allow => current,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::LeaseId;
use crate::types::thread::ThreadId;
use chrono::Utc;
fn make_action(name: &str, effects: Vec<EffectType>, requires_approval: bool) -> ActionDef {
ActionDef {
name: name.into(),
description: String::new(),
parameters_schema: serde_json::json!({}),
effects,
requires_approval,
}
}
fn make_lease() -> CapabilityLease {
CapabilityLease {
id: LeaseId::new(),
thread_id: ThreadId::new(),
capability_name: "test".into(),
granted_actions: vec![],
granted_at: Utc::now(),
expires_at: None,
max_uses: None,
uses_remaining: None,
revoked: false,
}
}
#[test]
fn allow_by_default() {
let engine = PolicyEngine::new();
let action = make_action("read_file", vec![EffectType::ReadLocal], false);
let lease = make_lease();
assert_eq!(engine.evaluate(&action, &lease, &[]), PolicyDecision::Allow);
}
#[test]
fn denied_effect_type() {
let mut engine = PolicyEngine::new();
engine.deny_effect(EffectType::Financial);
let action = make_action("transfer", vec![EffectType::Financial], false);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn action_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("deploy", vec![EffectType::WriteExternal], true);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::RequireApproval { .. }
));
}
#[test]
fn global_policy_deny_overrides_approval() {
let mut engine = PolicyEngine::new();
engine.add_global_policy(PolicyRule {
name: "no external writes".into(),
condition: PolicyCondition::EffectTypeIs(EffectType::WriteExternal),
effect: PolicyEffect::Deny,
});
let action = make_action("deploy", vec![EffectType::WriteExternal], true);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn capability_policy_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("create_issue", vec![EffectType::WriteExternal], false);
let lease = make_lease();
let cap_policies = vec![PolicyRule {
name: "approve writes".into(),
condition: PolicyCondition::EffectTypeIs(EffectType::WriteExternal),
effect: PolicyEffect::RequireApproval,
}];
assert!(matches!(
engine.evaluate(&action, &lease, &cap_policies),
PolicyDecision::RequireApproval { .. }
));
}
#[test]
fn expired_lease_denied() {
let engine = PolicyEngine::new();
let action = make_action("read", vec![EffectType::ReadLocal], false);
let mut lease = make_lease();
lease.revoked = true;
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn lease_not_covering_action_denied() {
let engine = PolicyEngine::new();
let action = make_action("delete_repo", vec![EffectType::WriteExternal], false);
let mut lease = make_lease();
lease.granted_actions = vec!["create_issue".into()];
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::Deny { .. }
));
}
#[test]
fn llm_generated_financial_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("transfer_funds", vec![EffectType::Financial], false);
let lease = make_lease();
let decision =
engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::LlmGenerated);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn llm_generated_write_external_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("post_message", vec![EffectType::WriteExternal], false);
let lease = make_lease();
let decision =
engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::LlmGenerated);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn user_provenance_allows_financial() {
let engine = PolicyEngine::new();
let action = make_action("transfer_funds", vec![EffectType::Financial], false);
let lease = make_lease();
let decision = engine.evaluate_with_provenance(&action, &lease, &[], &Provenance::User);
assert_eq!(decision, PolicyDecision::Allow);
}
#[test]
fn tool_output_financial_requires_approval() {
let engine = PolicyEngine::new();
let action = make_action("pay_invoice", vec![EffectType::Financial], false);
let lease = make_lease();
let decision = engine.evaluate_with_provenance(
&action,
&lease,
&[],
&Provenance::ToolOutput {
action_name: "scrape_invoices".into(),
},
);
assert!(matches!(decision, PolicyDecision::RequireApproval { .. }));
}
#[test]
fn action_matches_pattern() {
let mut engine = PolicyEngine::new();
engine.add_global_policy(PolicyRule {
name: "approve deletes".into(),
condition: PolicyCondition::ActionMatches {
pattern: "delete".into(),
},
effect: PolicyEffect::RequireApproval,
});
let action = make_action("delete_repo", vec![EffectType::WriteExternal], false);
let lease = make_lease();
assert!(matches!(
engine.evaluate(&action, &lease, &[]),
PolicyDecision::RequireApproval { .. }
));
let action2 = make_action("create_issue", vec![EffectType::WriteExternal], false);
assert_eq!(
engine.evaluate(&action2, &lease, &[]),
PolicyDecision::Allow
);
}
}
@@ -0,0 +1,170 @@
//! Capability registry — stores capability definitions available to the system.
use std::collections::HashMap;
use crate::types::capability::{ActionDef, Capability};
/// Registry of all known capabilities.
///
/// Capabilities are registered at startup (from extensions, built-in tools,
/// etc.) and queried when granting leases or resolving action names.
#[derive(Debug, Default)]
pub struct CapabilityRegistry {
capabilities: HashMap<String, Capability>,
}
impl CapabilityRegistry {
pub fn new() -> Self {
Self::default()
}
/// Register a capability. Overwrites any existing capability with the same name.
pub fn register(&mut self, capability: Capability) {
self.capabilities
.insert(capability.name.clone(), capability);
}
/// Look up a capability by name.
pub fn get(&self, name: &str) -> Option<&Capability> {
self.capabilities.get(name)
}
/// List all registered capabilities.
pub fn list(&self) -> Vec<&Capability> {
self.capabilities.values().collect()
}
/// Look up a specific action across all capabilities.
///
/// Returns `(capability_name, action_def)` if found.
pub fn find_action(&self, action_name: &str) -> Option<(&str, &ActionDef)> {
for cap in self.capabilities.values() {
if let Some(action) = cap.actions.iter().find(|a| a.name == action_name) {
return Some((&cap.name, action));
}
}
None
}
/// Get an action definition from a specific capability.
pub fn get_action(&self, capability_name: &str, action_name: &str) -> Option<&ActionDef> {
self.capabilities
.get(capability_name)?
.actions
.iter()
.find(|a| a.name == action_name)
}
/// Collect all action definitions across all capabilities.
pub fn all_actions(&self) -> Vec<&ActionDef> {
self.capabilities
.values()
.flat_map(|c| c.actions.iter())
.collect()
}
/// Number of registered capabilities.
pub fn len(&self) -> usize {
self.capabilities.len()
}
pub fn is_empty(&self) -> bool {
self.capabilities.is_empty()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::EffectType;
fn test_capability() -> Capability {
Capability {
name: "github".into(),
description: "GitHub integration".into(),
actions: vec![
ActionDef {
name: "create_issue".into(),
description: "Create a GitHub issue".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::WriteExternal, EffectType::CredentialedNetwork],
requires_approval: false,
},
ActionDef {
name: "list_prs".into(),
description: "List pull requests".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadExternal, EffectType::CredentialedNetwork],
requires_approval: false,
},
],
knowledge: vec!["When creating issues, always add labels.".into()],
policies: vec![],
}
}
#[test]
fn register_and_get() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert_eq!(reg.len(), 1);
assert!(reg.get("github").is_some());
assert!(reg.get("slack").is_none());
}
#[test]
fn find_action_across_capabilities() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
let (cap_name, action) = reg.find_action("create_issue").unwrap();
assert_eq!(cap_name, "github");
assert_eq!(action.name, "create_issue");
assert!(reg.find_action("nonexistent").is_none());
}
#[test]
fn get_action_from_capability() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert!(reg.get_action("github", "list_prs").is_some());
assert!(reg.get_action("github", "delete_repo").is_none());
assert!(reg.get_action("slack", "list_prs").is_none());
}
#[test]
fn all_actions_collects_across_capabilities() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
reg.register(Capability {
name: "memory".into(),
description: "Memory tools".into(),
actions: vec![ActionDef {
name: "memory_search".into(),
description: "Search memory".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
assert_eq!(reg.all_actions().len(), 3);
}
#[test]
fn overwrite_on_re_register() {
let mut reg = CapabilityRegistry::new();
reg.register(test_capability());
assert_eq!(reg.get("github").unwrap().actions.len(), 2);
reg.register(Capability {
name: "github".into(),
description: "Updated".into(),
actions: vec![],
knowledge: vec![],
policies: vec![],
});
assert_eq!(reg.get("github").unwrap().actions.len(), 0);
assert_eq!(reg.len(), 1);
}
}
@@ -0,0 +1,289 @@
//! Skill confidence tracking.
//!
//! Tracks usage and success/failure metrics for auto-extracted skills.
//! After each thread completes, the active skills' metrics are updated
//! based on whether the thread succeeded or failed.
use std::sync::Arc;
use ironclaw_skills::v2::V2SkillMetadata;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocId, DocType, MemoryDoc};
/// Tracks skill usage and updates confidence metrics.
pub struct SkillTracker {
store: Arc<dyn Store>,
}
impl SkillTracker {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Record that a skill was used in a completed thread.
///
/// Loads the skill's MemoryDoc, updates metrics in the metadata JSON,
/// and saves it back. If the doc is not found or has invalid metadata,
/// the error is logged and the operation is skipped.
pub async fn record_usage(&self, doc_id: DocId, success: bool) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
if doc.doc_type != DocType::Skill {
return Err(EngineError::Skill {
reason: format!("doc {} is not a skill (type: {:?})", doc_id.0, doc.doc_type),
});
}
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata for {}: {e}", doc_id.0),
})?;
meta.metrics.usage_count += 1;
if success {
meta.metrics.success_count += 1;
} else {
meta.metrics.failure_count += 1;
}
meta.metrics.last_used = Some(chrono::Utc::now());
let updated_doc = MemoryDoc {
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
/// Update a skill's content and increment its version.
///
/// Sets `parent_version` to the current version before incrementing,
/// enabling rollback if the update causes issues.
pub async fn update_skill(
&self,
doc_id: DocId,
new_content: String,
updater: impl FnOnce(&mut V2SkillMetadata),
) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata: {e}"),
})?;
meta.parent_version = Some(meta.version);
meta.version += 1;
updater(&mut meta);
let updated_doc = MemoryDoc {
content: new_content,
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
/// Rollback a skill to its previous version.
///
/// Decrements the version to `parent_version` if available. This is a
/// simple version decrement — the actual content rollback requires the
/// caller to also restore the content from a backup.
pub async fn rollback_skill(&self, doc_id: DocId) -> Result<(), EngineError> {
let doc = self
.store
.load_memory_doc(doc_id)
.await?
.ok_or_else(|| EngineError::Skill {
reason: format!("skill doc not found: {}", doc_id.0),
})?;
let mut meta: V2SkillMetadata =
serde_json::from_value(doc.metadata.clone()).map_err(|e| EngineError::Skill {
reason: format!("invalid skill metadata: {e}"),
})?;
let parent = meta.parent_version.ok_or_else(|| EngineError::Skill {
reason: format!("skill {} has no parent version to rollback to", doc_id.0),
})?;
meta.version = parent;
meta.parent_version = None;
let updated_doc = MemoryDoc {
metadata: serde_json::to_value(&meta).map_err(|e| EngineError::Skill {
reason: format!("failed to serialize skill metadata: {e}"),
})?,
updated_at: chrono::Utc::now(),
..doc
};
self.store.save_memory_doc(&updated_doc).await
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::project::ProjectId;
use ironclaw_skills::SkillTrust;
use ironclaw_skills::v2::{SkillMetrics, V2SkillSource};
fn make_skill_doc(project_id: ProjectId) -> MemoryDoc {
let meta = V2SkillMetadata {
name: "test-skill".to_string(),
version: 1,
description: "test".to_string(),
activation: Default::default(),
source: V2SkillSource::Extracted,
trust: SkillTrust::Trusted,
code_snippets: vec![],
metrics: SkillMetrics {
usage_count: 5,
success_count: 3,
failure_count: 2,
last_used: None,
},
parent_version: None,
content_hash: String::new(),
};
let mut doc = MemoryDoc::new(
project_id,
DocType::Skill,
"skill:test",
"Test skill prompt",
);
doc.metadata = serde_json::to_value(&meta).unwrap();
doc
}
#[tokio::test]
async fn test_record_usage_success() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker.record_usage(doc_id, true).await.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.metrics.usage_count, 6);
assert_eq!(meta.metrics.success_count, 4);
assert_eq!(meta.metrics.failure_count, 2);
assert!(meta.metrics.last_used.is_some());
}
#[tokio::test]
async fn test_record_usage_failure() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker.record_usage(doc_id, false).await.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.metrics.usage_count, 6);
assert_eq!(meta.metrics.success_count, 3);
assert_eq!(meta.metrics.failure_count, 3);
}
#[tokio::test]
async fn test_update_skill_increments_version() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
tracker
.update_skill(doc_id, "Updated content".to_string(), |meta| {
meta.description = "Updated description".to_string();
})
.await
.unwrap();
let updated = store.load_memory_doc(doc_id).await.unwrap().unwrap();
assert_eq!(updated.content, "Updated content");
let meta: V2SkillMetadata = serde_json::from_value(updated.metadata).unwrap();
assert_eq!(meta.version, 2);
assert_eq!(meta.parent_version, Some(1));
assert_eq!(meta.description, "Updated description");
}
#[tokio::test]
async fn test_rollback_restores_parent_version() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store.clone());
// First update to version 2
tracker
.update_skill(doc_id, "v2 content".to_string(), |_| {})
.await
.unwrap();
// Now rollback
tracker.rollback_skill(doc_id).await.unwrap();
let rolled = store.load_memory_doc(doc_id).await.unwrap().unwrap();
let meta: V2SkillMetadata = serde_json::from_value(rolled.metadata).unwrap();
assert_eq!(meta.version, 1);
assert_eq!(meta.parent_version, None);
}
#[tokio::test]
async fn test_rollback_without_parent_fails() {
let project_id = ProjectId::new();
let doc = make_skill_doc(project_id);
let doc_id = doc.id;
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![doc]));
let tracker = SkillTracker::new(store);
let result = tracker.rollback_skill(doc_id).await;
assert!(result.is_err());
}
#[tokio::test]
async fn test_record_usage_missing_doc() {
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![]));
let tracker = SkillTracker::new(store);
let result = tracker.record_usage(DocId::new(), true).await;
assert!(result.is_err());
}
}
@@ -0,0 +1,176 @@
//! Context compaction and token counting.
//!
//! When message history approaches the model's context limit, compaction
//! asks the LLM to summarize progress and resets the history. This follows
//! the official RLM pattern (compaction at 85% of context limit).
use std::sync::Arc;
use tracing::debug;
use crate::traits::llm::{LlmBackend, LlmCallConfig};
use crate::types::error::EngineError;
use crate::types::message::{MessageRole, ThreadMessage};
use crate::types::step::{LlmResponse, TokenUsage};
/// Characters per token estimate when no tokenizer is available.
/// Conservative estimate (official RLM uses 4).
const CHARS_PER_TOKEN: usize = 4;
/// Estimate token count for a list of messages.
///
/// Uses character length / `CHARS_PER_TOKEN` as a rough estimate.
/// The official RLM uses tiktoken when available; we use this fallback
/// since we don't depend on a Python tokenizer.
pub fn estimate_tokens(messages: &[ThreadMessage]) -> usize {
let total_chars: usize = messages
.iter()
.map(|m| {
m.content.len() + m.action_name.as_ref().map_or(0, |n| n.len()) + 4 // overhead per message (role token, delimiters)
})
.sum();
total_chars.div_ceil(CHARS_PER_TOKEN)
}
/// Check if compaction should be triggered.
///
/// Returns `true` when estimated token count exceeds `threshold_pct` of
/// the model's context limit.
pub fn should_compact(
messages: &[ThreadMessage],
model_context_limit: usize,
threshold_pct: f64,
) -> bool {
let tokens = estimate_tokens(messages);
let threshold = (model_context_limit as f64 * threshold_pct) as usize;
tokens >= threshold
}
/// The compaction prompt sent to the LLM.
const COMPACTION_PROMPT: &str = "\
Summarize your progress so far in a concise but complete way. Include:
1. What you have accomplished
2. Key intermediate results and variable values
3. What still needs to be done
4. Any errors encountered and how they were handled
Preserve all information needed to continue the task. Be specific about data values.";
/// Compact the message history by asking the LLM to summarize.
///
/// Returns the new (shorter) message list and the token usage from the
/// summarization call. The original messages are replaced with:
/// `[system_prompt, summary, continuation_note]`
///
/// The full original messages are returned separately so the caller can
/// store them (e.g., in a `history` variable or event log).
pub async fn compact_messages(
messages: &[ThreadMessage],
llm: &Arc<dyn LlmBackend>,
compaction_count: u32,
) -> Result<CompactionResult, EngineError> {
// Build a summarization request from existing messages + prompt
let mut summarize_messages = messages.to_vec();
summarize_messages.push(ThreadMessage::user(COMPACTION_PROMPT.to_string()));
let config = LlmCallConfig {
force_text: true,
..LlmCallConfig::default()
};
let output = llm.complete(&summarize_messages, &[], &config).await?;
let summary_text = match output.response {
LlmResponse::Text(t) => t,
LlmResponse::ActionCalls { content, .. } | LlmResponse::Code { content, .. } => {
content.unwrap_or_else(|| "[compaction produced no summary]".into())
}
};
// Preserve the system prompt (first message if it's a system message)
let system_msg = messages
.iter()
.find(|m| m.role == MessageRole::System)
.cloned();
// Build compacted history
let mut compacted = Vec::new();
if let Some(sys) = system_msg {
compacted.push(sys);
}
compacted.push(ThreadMessage::assistant(summary_text.clone()));
compacted.push(ThreadMessage::user(format!(
"Your conversation has been compacted {n} time(s). \
The summary above captures your progress. Continue working on the task.",
n = compaction_count + 1,
)));
let tokens_before = estimate_tokens(messages);
let tokens_after = estimate_tokens(&compacted);
debug!(
tokens_before,
tokens_after,
compaction_count = compaction_count + 1,
"context compacted"
);
Ok(CompactionResult {
compacted_messages: compacted,
summary: summary_text,
tokens_used: output.usage,
tokens_before,
tokens_after,
})
}
/// Result of a compaction operation.
pub struct CompactionResult {
/// The new (shorter) message list.
pub compacted_messages: Vec<ThreadMessage>,
/// The summary text produced by the LLM.
pub summary: String,
/// Tokens used by the summarization LLM call.
pub tokens_used: TokenUsage,
/// Estimated token count before compaction.
pub tokens_before: usize,
/// Estimated token count after compaction.
pub tokens_after: usize,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn estimate_tokens_empty() {
assert_eq!(estimate_tokens(&[]), 0);
}
#[test]
fn estimate_tokens_basic() {
let msgs = vec![
ThreadMessage::system("Hello world"), // 11 chars + 4 overhead = 15 / 4 = 3.75
ThreadMessage::user("Hi"), // 2 chars + 4 = 6 / 4 = 1.5
];
let tokens = estimate_tokens(&msgs);
// (11+4 + 2+4) / 4 = 21/4 = 5.25 → 6 (ceiling)
assert!(tokens > 0);
assert!(tokens < 100);
}
#[test]
fn should_compact_below_threshold() {
let msgs = vec![ThreadMessage::user("short message")];
assert!(!should_compact(&msgs, 128_000, 0.85));
}
#[test]
fn should_compact_above_threshold() {
// Create a message large enough to trigger compaction at low limit
let big = "x".repeat(1000);
let msgs = vec![ThreadMessage::user(big)];
// 1000 chars / 4 = 250 tokens. Context limit 200, threshold 85% = 170
assert!(should_compact(&msgs, 200, 0.85));
}
}
@@ -0,0 +1,291 @@
//! Context building for LLM calls.
//!
//! Assembles the message sequence and action definitions from thread state,
//! active leases, and project memory docs retrieved via the [`RetrievalEngine`].
use std::sync::Arc;
use crate::memory::RetrievalEngine;
use crate::traits::effect::EffectExecutor;
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::error::EngineError;
use crate::types::memory::MemoryDoc;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
/// Maximum number of memory docs to inject into context.
const MAX_CONTEXT_DOCS: usize = 5;
/// Build the context for an LLM call: messages and available actions.
///
/// Retrieves relevant memory docs from the project and injects them as a
/// system message after the main system prompt. This gives the LLM access
/// to lessons learned, skills, and known issues from prior threads.
pub async fn build_step_context(
messages: &[ThreadMessage],
leases: &[CapabilityLease],
effects: &Arc<dyn EffectExecutor>,
retrieval: Option<&RetrievalEngine>,
project_id: ProjectId,
goal: &str,
) -> Result<(Vec<ThreadMessage>, Vec<ActionDef>), EngineError> {
let actions = effects.available_actions(leases).await?;
let mut ctx_messages = messages.to_vec();
// Inject retrieved memory docs into the existing system prompt.
// Many providers require all system messages at the beginning (or a single
// system message), so we append to the first system message rather than
// inserting a separate one.
if let Some(engine) = retrieval {
let docs = engine
.retrieve_context(project_id, goal, MAX_CONTEXT_DOCS)
.await?;
if !docs.is_empty() {
let context_section = format_docs_as_context(&docs);
if !ctx_messages.is_empty()
&& ctx_messages[0].role == crate::types::message::MessageRole::System
{
// Append to existing system prompt
ctx_messages[0].content.push_str("\n\n");
ctx_messages[0].content.push_str(&context_section);
} else {
// No system message — prepend as one
ctx_messages.insert(0, ThreadMessage::system(context_section));
}
}
}
Ok((ctx_messages, actions))
}
/// Format memory docs into a system message for context injection.
fn format_docs_as_context(docs: &[MemoryDoc]) -> String {
let mut parts = vec!["## Prior Knowledge (from completed threads)\n".to_string()];
for doc in docs {
let type_label = match doc.doc_type {
crate::types::memory::DocType::Lesson => "LESSON",
crate::types::memory::DocType::Spec => "MISSING CAPABILITY",
crate::types::memory::DocType::Issue => "KNOWN ISSUE",
crate::types::memory::DocType::Summary => "CONTEXT",
crate::types::memory::DocType::Note => "NOTE",
crate::types::memory::DocType::Skill => "SKILL",
};
// Truncate long docs to avoid context bloat
let content: String = doc.content.chars().take(500).collect();
let truncated = if doc.content.chars().count() > 500 {
"..."
} else {
""
};
parts.push(format!(
"### [{type_label}] {}\n{content}{truncated}\n",
doc.title
));
}
parts.join("\n")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, DocType};
use crate::types::project::{Project, ProjectId};
use crate::types::step::{ActionResult, Step};
use crate::types::thread::{Thread, ThreadId, ThreadState};
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: std::time::Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct DocStore(Vec<MemoryDoc>);
#[async_trait::async_trait]
impl crate::traits::store::Store for DocStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, pid: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(self
.0
.iter()
.filter(|d| d.project_id == pid)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
#[tokio::test]
async fn context_injects_docs_after_system_prompt() {
let project = ProjectId::new();
let store: Arc<dyn crate::traits::store::Store> = Arc::new(DocStore(vec![MemoryDoc::new(
project,
DocType::Lesson,
"web tool alias",
"Use web-search not web_search",
)]));
let retrieval = RetrievalEngine::new(store);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![
ThreadMessage::system("You are an assistant."),
ThreadMessage::user("search the web"),
];
let (ctx_msgs, _) = build_step_context(
&messages,
&[],
&effects,
Some(&retrieval),
project,
"search the web",
)
.await
.unwrap();
// Should have 2 messages: system prompt (with docs appended), user message
assert_eq!(ctx_msgs.len(), 2);
assert_eq!(ctx_msgs[0].role, crate::types::message::MessageRole::System);
assert!(ctx_msgs[0].content.contains("You are an assistant."));
assert!(ctx_msgs[0].content.contains("Prior Knowledge"));
assert!(ctx_msgs[0].content.contains("LESSON"));
assert!(ctx_msgs[0].content.contains("web-search"));
assert_eq!(ctx_msgs[1].role, crate::types::message::MessageRole::User);
}
#[tokio::test]
async fn context_without_retrieval_passes_through() {
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![
ThreadMessage::system("prompt"),
ThreadMessage::user("hello"),
];
let (ctx_msgs, _) =
build_step_context(&messages, &[], &effects, None, ProjectId::new(), "hello")
.await
.unwrap();
// No injection — same number of messages
assert_eq!(ctx_msgs.len(), 2);
}
#[tokio::test]
async fn context_no_docs_means_no_injection() {
let project = ProjectId::new();
let store: Arc<dyn crate::traits::store::Store> = Arc::new(DocStore(vec![]));
let retrieval = RetrievalEngine::new(store);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects);
let messages = vec![ThreadMessage::user("hello")];
let (ctx_msgs, _) =
build_step_context(&messages, &[], &effects, Some(&retrieval), project, "hello")
.await
.unwrap();
assert_eq!(ctx_msgs.len(), 1);
}
}
@@ -0,0 +1,97 @@
//! Tool intent nudge detection.
//!
//! Detects when the LLM expresses intent to use a tool without actually
//! producing action calls (e.g. "Let me search..." or "I'll fetch...").
//! Mirrors the logic in `src/agent/agentic_loop.rs` `llm_signals_tool_intent`.
/// Check if a text response signals tool intent without actual action calls.
///
/// Returns `true` if the text contains phrases like "Let me search...",
/// "I'll fetch...", etc. that indicate the LLM wanted to call a tool.
pub fn signals_tool_intent(response: &str) -> bool {
let lower = response.to_lowercase();
// Skip false positives
let false_positive_phrases = [
"let me explain",
"let me think",
"let me know",
"let me summarize",
"let me clarify",
];
for phrase in &false_positive_phrases {
if lower.contains(phrase) {
return false;
}
}
let intent_prefixes = ["let me ", "i'll ", "i will ", "i'm going to "];
let action_verbs = [
"search", "look up", "check", "fetch", "find", "query", "read", "run", "execute", "call",
"use", "invoke",
];
for prefix in &intent_prefixes {
if let Some(after) = lower.strip_prefix(prefix) {
for verb in &action_verbs {
if after.starts_with(verb) {
return true;
}
}
}
// Also check if the prefix appears mid-sentence (after period or newline)
for sep in [". ", ".\n", "\n"] {
for part in lower.split(sep) {
let trimmed = part.trim();
if let Some(after) = trimmed.strip_prefix(prefix) {
for verb in &action_verbs {
if after.starts_with(verb) {
return true;
}
}
}
}
}
}
false
}
/// The nudge message injected into context when tool intent is detected.
pub const TOOL_INTENT_NUDGE: &str = "You expressed intent to use a tool but didn't make an action call. \
Please go ahead and call the appropriate action.";
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn detects_let_me_search() {
assert!(signals_tool_intent("Let me search for that"));
}
#[test]
fn detects_ill_fetch() {
assert!(signals_tool_intent("I'll fetch the latest data"));
}
#[test]
fn ignores_let_me_explain() {
assert!(!signals_tool_intent("Let me explain how this works"));
}
#[test]
fn ignores_let_me_know() {
assert!(!signals_tool_intent("Let me know if you need more"));
}
#[test]
fn ignores_plain_text() {
assert!(!signals_tool_intent("The answer is 42."));
}
#[test]
fn detects_after_period() {
assert!(signals_tool_intent("Sure. Let me search for that."));
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,18 @@
//! Step execution.
//!
//! - [`ExecutionLoop`] — core loop replacing `run_agentic_loop()`
//! - [`structured`] — Tier 0 action execution (structured tool calls)
//! - [`context`] — context building for LLM calls
//! - [`intent`] — tool intent nudge detection
pub mod compaction;
pub mod context;
pub mod intent;
pub mod loop_engine;
pub mod orchestrator;
pub mod prompt;
pub mod scripting;
pub mod structured;
pub mod trace;
pub use loop_engine::ExecutionLoop;
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,282 @@
//! System prompt construction for the execution loop.
//!
//! Builds a CodeAct/RLM system prompt that instructs the LLM to write
//! Python code in ```repl blocks with tools available as callable functions.
//!
//! Prompt templates live in `crates/ironclaw_engine/prompts/` as plain
//! markdown files for easy inspection and iteration. They are embedded
//! at compile time via `include_str!` and can be extended at runtime with
//! prompt overlays stored as MemoryDocs.
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::capability::ActionDef;
use crate::types::project::ProjectId;
/// Runtime platform metadata injected into system prompts for self-awareness.
///
/// Provides the agent with knowledge about its own identity and environment
/// so it can answer questions about itself, its capabilities, and its
/// configuration without relying on training data.
#[derive(Debug, Clone, Default)]
pub struct PlatformInfo {
/// Software version (from CARGO_PKG_VERSION).
pub version: Option<String>,
/// LLM backend name (e.g. "nearai", "openai", "anthropic").
pub llm_backend: Option<String>,
/// Active model name.
pub model_name: Option<String>,
/// Database backend (e.g. "libsql", "postgres").
pub database_backend: Option<String>,
/// Active channel names (e.g. ["telegram", "cli"]).
pub active_channels: Vec<String>,
/// Owner identifier.
pub owner_id: Option<String>,
/// Project repository URL.
pub repo_url: Option<String>,
}
impl PlatformInfo {
/// Format as a prompt section. Returns empty string if no info is set.
pub fn to_prompt_section(&self) -> String {
let mut lines = Vec::new();
lines.push("You are **IronClaw**, a secure autonomous AI assistant platform.".into());
if let Some(ref v) = self.version {
lines.push(format!("- Version: {v}"));
}
if let Some(ref repo) = self.repo_url {
lines.push(format!("- Repository: {repo}"));
}
if let Some(ref owner) = self.owner_id {
lines.push(format!("- Owner: {owner}"));
}
if let Some(ref backend) = self.llm_backend {
let model = self.model_name.as_deref().unwrap_or("default");
lines.push(format!("- LLM: {backend} ({model})"));
}
if let Some(ref db) = self.database_backend {
lines.push(format!("- Database: {db}"));
}
if !self.active_channels.is_empty() {
lines.push(format!("- Channels: {}", self.active_channels.join(", ")));
}
if lines.len() <= 1 {
// Only the identity line, no runtime details — still include it
return format!("\n\n## Platform\n\n{}\n", lines[0]);
}
format!("\n\n## Platform\n\n{}\n", lines.join("\n"))
}
}
/// The main instruction block (before tool listing).
const CODEACT_PREAMBLE: &str = include_str!("../../prompts/codeact_preamble.md");
/// The strategy/closing block (after tool listing).
const CODEACT_POSTAMBLE: &str = include_str!("../../prompts/codeact_postamble.md");
/// Well-known title for the CodeAct preamble overlay.
pub const PREAMBLE_OVERLAY_TITLE: &str = "prompt:codeact_preamble";
/// Well-known tag for prompt overlay docs.
pub const PROMPT_OVERLAY_TAG: &str = "prompt_overlay";
/// Maximum size for a prompt overlay document (in chars).
const MAX_PROMPT_OVERLAY_CHARS: usize = 4000;
/// Build the system prompt for CodeAct/RLM execution.
///
/// The prompt instructs the LLM to:
/// - Write Python code in ```repl fenced blocks
/// - Call tools as regular Python functions
/// - Use llm_query(prompt, context) for sub-agent calls
/// - Use FINAL(answer) to return the final answer
/// - Access thread context via the `context` variable
///
/// If a Store is provided, checks for a runtime prompt overlay (a MemoryDoc
/// with tag "prompt_overlay" and title "prompt:codeact_preamble") and appends
/// its content after the compiled preamble. This enables the self-improvement
/// mission to evolve the system prompt at runtime.
pub async fn build_codeact_system_prompt(
actions: &[ActionDef],
store: Option<&Arc<dyn Store>>,
project_id: ProjectId,
platform: Option<&PlatformInfo>,
) -> String {
let mut prompt = String::from(CODEACT_PREAMBLE);
// Inject platform identity and runtime metadata
if let Some(info) = platform {
prompt.push_str(&info.to_prompt_section());
}
// Append runtime prompt overlay if available
if let Some(store) = store
&& let Some(overlay) = load_prompt_overlay(store, project_id).await
{
prompt.push_str("\n\n## Learned Rules (from self-improvement)\n\n");
prompt.push_str(&overlay);
}
// Add tool documentation
if !actions.is_empty() {
prompt.push_str("\n## Available tools (call as Python functions)\n\n");
for action in actions {
prompt.push_str(&format!("- `{}(", action.name));
// Extract parameter names from JSON schema
if let Some(props) = action.parameters_schema.get("properties")
&& let Some(obj) = props.as_object()
{
let params: Vec<&str> = obj.keys().map(String::as_str).collect();
prompt.push_str(&params.join(", "));
}
prompt.push_str(&format!(")` — {}\n", action.description));
}
}
prompt.push_str(CODEACT_POSTAMBLE);
prompt
}
/// Load the prompt overlay from the Store, if one exists for this project.
async fn load_prompt_overlay(store: &Arc<dyn Store>, project_id: ProjectId) -> Option<String> {
let docs = store.list_memory_docs(project_id).await.ok()?;
let overlay = docs.iter().find(|d| {
d.title == PREAMBLE_OVERLAY_TITLE && d.tags.contains(&PROMPT_OVERLAY_TAG.to_string())
})?;
let content: String = overlay
.content
.chars()
.take(MAX_PROMPT_OVERLAY_CHARS)
.collect();
if content.is_empty() {
return None;
}
Some(content)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::memory::{DocId, DocType, MemoryDoc};
#[tokio::test]
async fn prompt_without_store_uses_compiled_preamble() {
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), None).await;
assert!(prompt.contains("Python REPL environment"));
assert!(prompt.contains("Strategy"));
assert!(!prompt.contains("Learned Rules"));
}
#[tokio::test]
async fn prompt_with_overlay_appends_rules() {
let project_id = ProjectId(uuid::Uuid::new_v4());
let overlay = MemoryDoc {
id: DocId::new(),
project_id,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: "9. Never call web_fetch — use http() instead.".into(),
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
assert!(prompt.contains("Learned Rules"));
assert!(prompt.contains("Never call web_fetch"));
}
#[tokio::test]
async fn prompt_overlay_size_is_capped() {
let project_id = ProjectId(uuid::Uuid::new_v4());
// Create an overlay that exceeds MAX_PROMPT_OVERLAY_CHARS using a char
// not found in the compiled preamble/postamble
let huge_content = "\u{2603}".repeat(MAX_PROMPT_OVERLAY_CHARS + 1000); // snowman
let overlay = MemoryDoc {
id: DocId::new(),
project_id,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: huge_content,
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
let snowman_count = prompt.chars().filter(|c| *c == '\u{2603}').count();
assert_eq!(snowman_count, MAX_PROMPT_OVERLAY_CHARS);
}
#[tokio::test]
async fn prompt_ignores_wrong_project_overlay() {
let project_id = ProjectId(uuid::Uuid::new_v4());
let other_project = ProjectId(uuid::Uuid::new_v4());
let overlay = MemoryDoc {
id: DocId::new(),
project_id: other_project,
doc_type: DocType::Note,
title: PREAMBLE_OVERLAY_TITLE.into(),
content: "Should not appear".into(),
source_thread_id: None,
tags: vec![PROMPT_OVERLAY_TAG.into()],
metadata: serde_json::json!({}),
created_at: chrono::Utc::now(),
updated_at: chrono::Utc::now(),
};
let store = Arc::new(crate::tests::InMemoryStore::with_docs(vec![overlay]));
let prompt =
build_codeact_system_prompt(&[], Some(&(store as Arc<dyn Store>)), project_id, None)
.await;
assert!(!prompt.contains("Should not appear"));
assert!(!prompt.contains("Learned Rules"));
}
#[tokio::test]
async fn prompt_with_platform_info_injects_identity() {
let info = PlatformInfo {
version: Some("1.2.3".into()),
llm_backend: Some("nearai".into()),
model_name: Some("qwen3-235b".into()),
database_backend: Some("libsql".into()),
active_channels: vec!["telegram".into(), "cli".into()],
owner_id: Some("alice.near".into()),
repo_url: Some("https://github.com/nearai/ironclaw".into()),
};
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), Some(&info)).await;
assert!(prompt.contains("IronClaw"));
assert!(prompt.contains("1.2.3"));
assert!(prompt.contains("nearai"));
assert!(prompt.contains("qwen3-235b"));
assert!(prompt.contains("libsql"));
assert!(prompt.contains("telegram"));
assert!(prompt.contains("alice.near"));
assert!(prompt.contains("github.com/nearai/ironclaw"));
}
#[tokio::test]
async fn prompt_without_platform_info_has_no_platform_section() {
let prompt =
build_codeact_system_prompt(&[], None, ProjectId(uuid::Uuid::nil()), None).await;
assert!(!prompt.contains("## Platform"));
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,738 @@
//! Tier 0 executor: structured tool calls.
//!
//! Executes action calls by delegating to the `EffectExecutor` trait,
//! checking leases and policies for each call.
use std::sync::Arc;
use crate::capability::lease::LeaseManager;
use crate::capability::policy::{PolicyDecision, PolicyEngine};
use crate::runtime::messaging::ThreadOutcome;
use crate::traits::effect::{EffectExecutor, ThreadExecutionContext};
use crate::types::error::EngineError;
use crate::types::event::EventKind;
use crate::types::step::{ActionCall, ActionResult};
use crate::types::thread::Thread;
/// Result of executing a batch of action calls.
pub struct ActionBatchResult {
/// Results for each action call (in order).
pub results: Vec<ActionResult>,
/// Events generated during execution.
pub events: Vec<EventKind>,
/// If set, execution was interrupted and the thread needs approval.
pub need_approval: Option<ThreadOutcome>,
}
/// Execute a batch of action calls using the Tier 0 (structured) approach.
///
/// For each action call:
/// 1. Find the lease that grants this action
/// 2. Check policy (deny/allow/approve)
/// 3. Consume a lease use
/// 4. Call `EffectExecutor::execute_action()`
/// 5. Record result and emit event
///
/// Stops at the first action that requires approval.
pub async fn execute_action_calls(
calls: &[ActionCall],
thread: &Thread,
effects: &Arc<dyn EffectExecutor>,
leases: &LeaseManager,
policy: &PolicyEngine,
context: &ThreadExecutionContext,
capability_policies: &[crate::types::capability::PolicyRule],
) -> Result<ActionBatchResult, EngineError> {
let mut results = Vec::with_capacity(calls.len());
let mut events = Vec::new();
for call in calls {
// 1. Find the lease for this action
let lease = match leases
.find_lease_for_action(thread.id, &call.action_name)
.await
{
Some(l) => l,
None => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": format!(
"no active lease covers action '{}'", call.action_name
)}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: format!("no lease for action '{}'", call.action_name),
params_summary: None,
});
results.push(error_result);
continue;
}
};
// 2. Find the action definition and check policy
let action_def = effects
.available_actions(std::slice::from_ref(&lease))
.await?
.into_iter()
.find(|a| a.name == call.action_name);
if let Some(ref action_def) = action_def {
let decision = policy.evaluate(action_def, &lease, capability_policies);
match decision {
PolicyDecision::Deny { reason } => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": format!("denied: {reason}")}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: reason,
params_summary: None,
});
results.push(error_result);
continue;
}
PolicyDecision::RequireApproval { .. } => {
events.push(EventKind::ApprovalRequested {
action_name: call.action_name.clone(),
call_id: call.id.clone(),
});
return Ok(ActionBatchResult {
results,
events,
need_approval: Some(ThreadOutcome::NeedApproval {
action_name: call.action_name.clone(),
call_id: call.id.clone(),
parameters: call.parameters.clone(),
}),
});
}
PolicyDecision::Allow => {}
}
}
// 3. Consume a lease use
leases.consume_use(lease.id).await?;
// 4. Execute the action
let result = effects
.execute_action(&call.action_name, call.parameters.clone(), &lease, context)
.await;
match result {
Ok(mut action_result) => {
// EffectExecutor doesn't receive call_id; stamp it from the
// original ActionCall so downstream messages carry the correct ID.
action_result.call_id = call.id.clone();
events.push(EventKind::ActionExecuted {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
duration_ms: action_result.duration.as_millis() as u64,
params_summary: None,
});
results.push(action_result);
}
Err(crate::types::error::EngineError::NeedAuthentication {
credential_name,
action_name,
call_id,
parameters,
}) => {
// Interrupt the batch — thread should pause for authentication.
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: action_name.clone(),
call_id: call_id.clone(),
error: format!("authentication required for credential '{credential_name}'"),
params_summary: None,
});
return Ok(ActionBatchResult {
results,
events,
need_approval: Some(ThreadOutcome::NeedAuthentication {
credential_name,
action_name,
call_id,
parameters,
}),
});
}
Err(e) => {
let error_result = ActionResult {
call_id: call.id.clone(),
action_name: call.action_name.clone(),
output: serde_json::json!({"error": e.to_string()}),
is_error: true,
duration: std::time::Duration::ZERO,
};
events.push(EventKind::ActionFailed {
step_id: context.step_id,
action_name: call.action_name.clone(),
call_id: call.id.clone(),
error: e.to_string(),
params_summary: None,
});
results.push(error_result);
}
}
}
Ok(ActionBatchResult {
results,
events,
need_approval: None,
})
}
#[cfg(test)]
mod tests {
use super::*;
use crate::traits::effect::ThreadExecutionContext;
use crate::types::capability::{ActionDef, CapabilityLease, EffectType};
use crate::types::project::ProjectId;
use crate::types::step::StepId;
use crate::types::thread::{Thread, ThreadConfig, ThreadType};
use std::sync::Mutex;
use std::time::Duration;
struct MockEffects {
results: Mutex<Vec<Result<ActionResult, EngineError>>>,
actions: Vec<ActionDef>,
}
impl MockEffects {
fn new(actions: Vec<ActionDef>, results: Vec<Result<ActionResult, EngineError>>) -> Self {
Self {
results: Mutex::new(results),
actions,
}
}
}
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_name: &str,
_params: serde_json::Value,
_lease: &CapabilityLease,
_ctx: &ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
let mut results = self.results.lock().unwrap();
if results.is_empty() {
Ok(ActionResult {
call_id: String::new(), // EffectExecutor doesn't set call_id
action_name: String::new(),
output: serde_json::json!({"result": "ok"}),
is_error: false,
duration: Duration::from_millis(1),
})
} else {
results.remove(0)
}
}
async fn available_actions(
&self,
_leases: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(self.actions.clone())
}
}
fn test_action(name: &str) -> ActionDef {
ActionDef {
name: name.into(),
description: "Test tool".into(),
parameters_schema: serde_json::json!({"type": "object"}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}
}
fn make_exec_context(thread: &Thread) -> ThreadExecutionContext {
ThreadExecutionContext {
thread_id: thread.id,
thread_type: thread.thread_type,
project_id: thread.project_id,
user_id: "test".into(),
step_id: StepId::new(),
}
}
// ── call_id propagation tests ────────────────────────────
#[tokio::test]
async fn call_id_preserved_on_successful_execution() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("web_search")],
vec![Ok(ActionResult {
call_id: String::new(), // EffectExecutor returns empty
action_name: "web_search".into(),
output: serde_json::json!({"results": []}),
is_error: false,
duration: Duration::from_millis(42),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "search", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_r2o5mqBgdNUlH8KzskncUGaX".into(),
action_name: "web_search".into(),
parameters: serde_json::json!({"query": "test"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// call_id must be stamped from ActionCall, not the empty EffectExecutor return
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_r2o5mqBgdNUlH8KzskncUGaX");
assert_eq!(result.results[0].action_name, "web_search");
assert!(!result.results[0].is_error);
// Event should carry the same call_id
let exec_event = result
.events
.iter()
.find(|e| matches!(e, EventKind::ActionExecuted { .. }));
assert!(exec_event.is_some());
if let Some(EventKind::ActionExecuted {
call_id,
action_name,
..
}) = exec_event
{
assert_eq!(call_id, "call_r2o5mqBgdNUlH8KzskncUGaX");
assert_eq!(action_name, "web_search");
}
}
#[tokio::test]
async fn call_id_preserved_on_execution_error() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("shell")],
vec![Err(EngineError::Effect {
reason: "permission denied".into(),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "exec", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_abc123def".into(),
action_name: "shell".into(),
parameters: serde_json::json!({"cmd": "ls"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_abc123def");
assert!(result.results[0].is_error);
let fail_event = result
.events
.iter()
.find(|e| matches!(e, EventKind::ActionFailed { .. }));
assert!(fail_event.is_some());
if let Some(EventKind::ActionFailed { call_id, .. }) = fail_event {
assert_eq!(call_id, "call_abc123def");
}
}
#[tokio::test]
async fn call_id_preserved_when_no_lease() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(vec![], vec![]));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
// No lease granted — action should fail with correct call_id
let calls = vec![ActionCall {
id: "call_no_lease_123".into(),
action_name: "web_search".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 1);
assert_eq!(result.results[0].call_id, "call_no_lease_123");
assert!(result.results[0].is_error);
if let Some(EventKind::ActionFailed { call_id, error, .. }) = result.events.first() {
assert_eq!(call_id, "call_no_lease_123");
assert!(error.contains("no lease"));
} else {
panic!("expected ActionFailed event");
}
}
#[tokio::test]
async fn multiple_calls_each_get_correct_call_id() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("tool_a"), test_action("tool_b")],
vec![
Ok(ActionResult {
call_id: String::new(),
action_name: "tool_a".into(),
output: serde_json::json!("a_result"),
is_error: false,
duration: Duration::from_millis(1),
}),
Ok(ActionResult {
call_id: String::new(),
action_name: "tool_b".into(),
output: serde_json::json!("b_result"),
is_error: false,
duration: Duration::from_millis(2),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "id_aaaa".into(),
action_name: "tool_a".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "id_bbbb".into(),
action_name: "tool_b".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results.len(), 2);
assert_eq!(result.results[0].call_id, "id_aaaa");
assert_eq!(result.results[1].call_id, "id_bbbb");
}
// ── NeedAuthentication tests ─────────────────────────────
#[tokio::test]
async fn need_authentication_interrupts_batch() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http")],
vec![Err(EngineError::NeedAuthentication {
credential_name: "github_token".into(),
action_name: "http".into(),
call_id: "call_auth_1".into(),
parameters: serde_json::json!({"url": "https://api.github.com/repos"}),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![ActionCall {
id: "call_auth_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({"url": "https://api.github.com/repos"}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Batch should be interrupted with NeedAuthentication outcome
assert!(
result.need_approval.is_some(),
"NeedAuthentication should interrupt the batch"
);
match result.need_approval.unwrap() {
ThreadOutcome::NeedAuthentication {
credential_name,
action_name,
..
} => {
assert_eq!(credential_name, "github_token");
assert_eq!(action_name, "http");
}
other => panic!("expected NeedAuthentication, got {:?}", other),
}
// ActionFailed event should be emitted
assert!(
result
.events
.iter()
.any(|e| matches!(e, EventKind::ActionFailed { .. })),
"should emit ActionFailed event"
);
}
#[tokio::test]
async fn need_authentication_stops_before_subsequent_calls() {
// Two calls: first needs auth, second should never execute
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http"), test_action("echo")],
vec![
Err(EngineError::NeedAuthentication {
credential_name: "api_key".into(),
action_name: "http".into(),
call_id: "call_1".into(),
parameters: serde_json::json!({}),
}),
// This should never be called
Ok(ActionResult {
call_id: String::new(),
action_name: "echo".into(),
output: serde_json::json!("should not appear"),
is_error: false,
duration: Duration::from_millis(1),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "call_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "call_2".into(),
action_name: "echo".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Second call should NOT have executed
assert!(
result.results.is_empty(),
"no results should be returned before the interrupted call"
);
assert!(result.need_approval.is_some());
}
/// Regular EngineError::Effect (not NeedAuthentication) should NOT interrupt —
/// it becomes a normal error result and execution continues.
#[tokio::test]
async fn regular_effect_error_does_not_interrupt() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("http"), test_action("echo")],
vec![
Err(EngineError::Effect {
reason: "connection timeout".into(),
}),
Ok(ActionResult {
call_id: String::new(),
action_name: "echo".into(),
output: serde_json::json!("second call ran"),
is_error: false,
duration: Duration::from_millis(1),
}),
],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "tools", vec![], None, None).await;
let calls = vec![
ActionCall {
id: "call_1".into(),
action_name: "http".into(),
parameters: serde_json::json!({}),
},
ActionCall {
id: "call_2".into(),
action_name: "echo".into(),
parameters: serde_json::json!({}),
},
];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Both calls should have results (error does not interrupt)
assert_eq!(result.results.len(), 2);
assert!(result.results[0].is_error);
assert!(!result.results[1].is_error);
assert!(
result.need_approval.is_none(),
"no interruption for regular errors"
);
}
// ── call_id preservation (OpenAI/Mistral) ─────────────────
/// Provider-specific: OpenAI rejects empty string call_id. Verify no result
/// ever has an empty call_id when the ActionCall provided one.
#[tokio::test]
async fn openai_empty_call_id_never_produced() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("echo")],
vec![Ok(ActionResult {
call_id: String::new(), // EffectExecutor always returns empty
action_name: String::new(),
output: serde_json::json!("hello"),
is_error: false,
duration: Duration::from_millis(1),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
let calls = vec![ActionCall {
id: "aB3xK9mZq".into(), // Mistral-compatible 9-char ID
action_name: "echo".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
// Must NOT be empty — must be stamped from the ActionCall
assert!(!result.results[0].call_id.is_empty());
assert_eq!(result.results[0].call_id, "aB3xK9mZq");
}
/// Mistral requires call_id matching [a-zA-Z0-9]{9}.
/// Verify the ID passes through unmodified (normalization is LLM-layer concern,
/// but engine must never lose it).
#[tokio::test]
async fn mistral_format_call_id_preserved() {
let thread = Thread::new(
"test",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
);
let effects: Arc<dyn EffectExecutor> = Arc::new(MockEffects::new(
vec![test_action("web_search")],
vec![Ok(ActionResult {
call_id: String::new(),
action_name: "web_search".into(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})],
));
let leases = Arc::new(LeaseManager::new());
let policy = Arc::new(PolicyEngine::new());
let ctx = make_exec_context(&thread);
leases.grant(thread.id, "cap", vec![], None, None).await;
// Mistral format: exactly 9 alphanumeric chars
let mistral_id = "xK3mR9bZq";
let calls = vec![ActionCall {
id: mistral_id.into(),
action_name: "web_search".into(),
parameters: serde_json::json!({}),
}];
let result = execute_action_calls(&calls, &thread, &effects, &leases, &policy, &ctx, &[])
.await
.unwrap();
assert_eq!(result.results[0].call_id, mistral_id);
// Event also preserves the exact format
if let Some(EventKind::ActionExecuted { call_id, .. }) = result.events.first() {
assert_eq!(call_id, mistral_id);
}
}
}
@@ -0,0 +1,565 @@
//! Execution trace recording and analysis.
//!
//! Records full execution traces to JSON files for debugging. Optionally
//! runs a post-execution analysis to detect common issues.
//!
//! Enable with `ENGINE_V2_TRACE=1` env var. Traces are written to
//! `engine_trace_{timestamp}.json` in the current directory.
use std::path::PathBuf;
use chrono::Utc;
use serde::Serialize;
use tracing::{debug, warn};
use crate::types::event::ThreadEvent;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Check if trace recording is enabled.
pub fn is_trace_enabled() -> bool {
std::env::var("ENGINE_V2_TRACE")
.map(|v| v == "1" || v == "true")
.unwrap_or(false)
}
/// A complete execution trace for a single thread.
#[derive(Debug, Serialize)]
pub struct ExecutionTrace {
pub thread_id: ThreadId,
pub goal: String,
pub final_state: ThreadState,
pub step_count: usize,
pub total_tokens: u64,
pub messages: Vec<MessageRecord>,
pub events: Vec<ThreadEvent>,
pub issues: Vec<TraceIssue>,
pub timestamp: chrono::DateTime<Utc>,
}
/// A single doc record, for the trace.
#[derive(Debug, Serialize)]
pub struct DocRecord {
pub doc_type: String,
pub title: String,
pub content: String,
}
/// A message in the trace with role labeling.
#[derive(Debug, Serialize)]
pub struct MessageRecord {
pub role: String,
pub content_length: usize,
pub content_preview: String,
pub full_content: String,
pub action_name: Option<String>,
pub action_call_id: Option<String>,
}
/// An issue detected by the retrospective analyzer.
#[derive(Debug, Serialize)]
pub struct TraceIssue {
pub severity: IssueSeverity,
pub category: String,
pub description: String,
pub step: Option<usize>,
}
#[derive(Debug, PartialEq, Serialize)]
pub enum IssueSeverity {
Error,
Warning,
Info,
}
/// Build a trace from a completed thread.
pub fn build_trace(thread: &Thread) -> ExecutionTrace {
let messages: Vec<MessageRecord> = thread
.messages
.iter()
.map(|m| {
let preview: String = m.content.chars().take(300).collect();
MessageRecord {
role: format!("{:?}", m.role),
content_length: m.content.chars().count(),
content_preview: if m.content.chars().count() > 300 {
format!("{preview}...")
} else {
preview
},
full_content: m.content.clone(),
action_name: m.action_name.clone(),
action_call_id: m.action_call_id.clone(),
}
})
.collect();
let issues = analyze_trace(thread);
ExecutionTrace {
thread_id: thread.id,
goal: thread.goal.clone(),
final_state: thread.state,
step_count: thread.step_count,
total_tokens: thread.total_tokens_used,
messages,
events: thread.events.clone(),
issues,
timestamp: Utc::now(),
}
}
/// Write a trace to a JSON file.
pub fn write_trace(trace: &ExecutionTrace) -> Option<PathBuf> {
let filename = format!("engine_trace_{}.json", Utc::now().format("%Y%m%dT%H%M%S"));
let path = PathBuf::from(&filename);
match serde_json::to_string_pretty(trace) {
Ok(json) => match std::fs::write(&path, json) {
Ok(()) => {
debug!(path = %path.display(), "Execution trace written");
Some(path)
}
Err(e) => {
warn!("Failed to write trace: {e}");
None
}
},
Err(e) => {
warn!("Failed to serialize trace: {e}");
None
}
}
}
/// Print a summary of the trace to the log.
pub fn log_trace_summary(trace: &ExecutionTrace) {
debug!(
thread_id = %trace.thread_id,
goal = %trace.goal,
state = ?trace.final_state,
steps = trace.step_count,
tokens = trace.total_tokens,
messages = trace.messages.len(),
events = trace.events.len(),
issues = trace.issues.len(),
"=== Engine V2 Trace Summary ==="
);
for issue in &trace.issues {
match issue.severity {
IssueSeverity::Error => warn!(
category = %issue.category,
step = ?issue.step,
"ISSUE: {}",
issue.description
),
IssueSeverity::Warning => warn!(
category = %issue.category,
step = ?issue.step,
"WARNING: {}",
issue.description
),
IssueSeverity::Info => debug!(
category = %issue.category,
step = ?issue.step,
"NOTE: {}",
issue.description
),
}
}
}
// ── Retrospective analysis ──────────────────────────────────
/// Analyze a completed thread for common issues.
fn analyze_trace(thread: &Thread) -> Vec<TraceIssue> {
let mut issues = Vec::new();
// 1. Check if the thread failed
if thread.state == ThreadState::Failed {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "thread_failure".into(),
description: "Thread ended in Failed state".into(),
step: None,
});
}
// 2. Check for empty response (no FINAL, no useful output)
let has_assistant_response = thread
.messages
.iter()
.any(|m| m.role == crate::types::message::MessageRole::Assistant && !m.content.is_empty());
if !has_assistant_response {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "no_response".into(),
description: "No assistant message in thread — model may not have generated output"
.into(),
step: None,
});
}
// 3. Check for tool errors
let tool_errors: Vec<&ThreadEvent> = thread
.events
.iter()
.filter(|e| matches!(e.kind, crate::types::event::EventKind::ActionFailed { .. }))
.collect();
if !tool_errors.is_empty() {
for event in &tool_errors {
if let crate::types::event::EventKind::ActionFailed {
action_name, error, ..
} = &event.kind
{
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "tool_error".into(),
description: format!("Tool '{action_name}' failed: {error}"),
step: None,
});
}
}
}
// 4. Check for code execution errors in output messages.
// Code output appears as User-role messages (Monty stdout/stderr) with
// prefixes like "[stdout]" or "[stderr]". Skip the System prompt (index 0)
// and Assistant messages to avoid false positives from example text.
let error_patterns = [
"NameError",
"SyntaxError",
"TypeError",
"NotImplementedError",
];
for (i, msg) in thread.messages.iter().enumerate() {
let is_code_output = msg.role == crate::types::message::MessageRole::User
&& (msg.content.starts_with("[stdout]")
|| msg.content.starts_with("[stderr]")
|| msg.content.starts_with("[code ")
|| msg.content.starts_with("Traceback"));
if is_code_output && error_patterns.iter().any(|p| msg.content.contains(p)) {
let preview: String = msg.content.chars().take(200).collect();
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "code_error".into(),
description: format!("Code execution error in message {i}: {preview}"),
step: None,
});
}
}
// 5. Check for empty call_id on ActionResult messages (causes LLM API rejection).
for (i, msg) in thread.messages.iter().enumerate() {
if msg.role == crate::types::message::MessageRole::ActionResult {
let call_id_empty = msg.action_call_id.as_ref().is_none_or(|id| id.is_empty());
if call_id_empty {
let name = msg.action_name.as_deref().unwrap_or("unknown");
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "empty_call_id".into(),
description: format!(
"ActionResult message {i} (tool '{name}') has empty call_id — will cause LLM API rejection"
),
step: None,
});
}
}
}
// 6. Check for model ignoring tool results (hallucination risk).
// In Tier 0 (structured), results appear as ActionResult messages.
// In Tier 1 (CodeAct), results appear as User messages with "[tool result]" prefixes.
let has_tool_results = thread
.messages
.iter()
.any(|m| m.role == crate::types::message::MessageRole::ActionResult);
let has_tool_output_in_messages = thread.messages.iter().any(|m| {
m.role == crate::types::message::MessageRole::ActionResult
|| m.content.contains(" result]")
|| m.content.contains(" error]")
});
if has_tool_results && !has_tool_output_in_messages {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "missing_tool_output".into(),
description:
"Tool results exist but no tool output in messages — model may not see tool results"
.into(),
step: None,
});
}
// 7. Check for excessive iterations
if thread.step_count > 10 {
issues.push(TraceIssue {
severity: IssueSeverity::Warning,
category: "excessive_steps".into(),
description: format!(
"Thread took {} steps — may be stuck in a loop",
thread.step_count
),
step: None,
});
}
// 8. Check for text response without FINAL (model answered from memory)
let text_without_code = thread.events.iter().all(|e| {
!matches!(
e.kind,
crate::types::event::EventKind::ActionExecuted { .. }
)
});
if text_without_code && thread.step_count == 1 && has_assistant_response {
issues.push(TraceIssue {
severity: IssueSeverity::Info,
category: "no_tools_used".into(),
description: "Model answered in one step without using any tools — may be answering from training data".into(),
step: Some(1),
});
}
// 9. Check for LLM not producing code blocks
let code_steps = thread
.events
.iter()
.filter(|e| matches!(e.kind, crate::types::event::EventKind::StepStarted { .. }))
.count();
let text_responses_without_code = thread
.messages
.iter()
.filter(|m| {
m.role == crate::types::message::MessageRole::Assistant
&& !m.content.contains("```")
&& !m.content.contains("FINAL(")
})
.count();
if text_responses_without_code > 0 && code_steps > 0 {
issues.push(TraceIssue {
severity: IssueSeverity::Info,
category: "mixed_mode".into(),
description: format!(
"{text_responses_without_code} text response(s) without code blocks — model may not be following CodeAct prompt"
),
step: None,
});
}
// 10. Extract failure reason from StateChanged → Failed events
for event in &thread.events {
if let crate::types::event::EventKind::StateChanged {
to: ThreadState::Failed,
reason: Some(reason),
..
} = &event.kind
{
if reason.contains("LLM") || reason.contains("Provider") {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "llm_error".into(),
description: format!("LLM provider error: {}", truncate(reason, 300)),
step: None,
});
} else if reason.contains("orchestrator") {
issues.push(TraceIssue {
severity: IssueSeverity::Error,
category: "orchestrator_error".into(),
description: format!("Orchestrator error: {}", truncate(reason, 300)),
step: None,
});
}
}
}
issues
}
fn truncate(s: &str, max_chars: usize) -> String {
let chars: String = s.chars().take(max_chars).collect();
if s.chars().count() > max_chars {
format!("{chars}...")
} else {
chars
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::event::EventKind;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::step::StepId;
use crate::types::thread::{ThreadConfig, ThreadType};
fn make_thread() -> Thread {
Thread::new(
"test goal",
ThreadType::Foreground,
ProjectId::new(),
ThreadConfig::default(),
)
}
// ── empty_call_id detection (OpenAI / Codex rejection) ───
/// OpenAI and Codex reject ActionResult messages with empty call_id.
/// The trace analyzer must flag these as errors.
#[test]
fn detects_empty_call_id_on_action_result() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
// Simulate the bug: empty call_id
thread.add_message(ThreadMessage::action_result("", "web_search", "result"));
let issues = analyze_trace(&thread);
let empty_id_issues: Vec<_> = issues
.iter()
.filter(|i| i.category == "empty_call_id")
.collect();
assert_eq!(empty_id_issues.len(), 1);
assert_eq!(empty_id_issues[0].severity, IssueSeverity::Error);
assert!(empty_id_issues[0].description.contains("web_search"));
}
/// ActionResult with None call_id should also be flagged.
#[test]
fn detects_none_call_id_on_action_result() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
// Manually construct a message with None call_id
thread.add_message(ThreadMessage {
role: crate::types::message::MessageRole::ActionResult,
content: "result".into(),
provenance: crate::types::provenance::Provenance::ToolOutput {
action_name: "shell".into(),
},
action_call_id: None,
action_name: Some("shell".into()),
action_calls: None,
timestamp: chrono::Utc::now(),
});
let issues = analyze_trace(&thread);
assert!(issues.iter().any(|i| i.category == "empty_call_id"));
}
/// No false positive: valid call_id should not be flagged.
#[test]
fn no_false_positive_for_valid_call_id() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("calling tool"));
thread.add_message(ThreadMessage::action_result(
"call_abc123",
"web_search",
"result",
));
let issues = analyze_trace(&thread);
assert!(
!issues.iter().any(|i| i.category == "empty_call_id"),
"valid call_id should not be flagged"
);
}
// ── tool_error detection ─────────────────────────────────
/// ActionFailed events should produce tool_error warnings.
#[test]
fn detects_tool_failures_in_events() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("ok"));
thread.events.push(ThreadEvent::new(
thread.id,
EventKind::ActionFailed {
step_id: StepId::new(),
action_name: "web_search".into(),
call_id: "call_123".into(),
error: "No lease for action 'web_search'".into(),
params_summary: None,
},
));
let issues = analyze_trace(&thread);
let tool_errors: Vec<_> = issues
.iter()
.filter(|i| i.category == "tool_error")
.collect();
assert_eq!(tool_errors.len(), 1);
assert!(tool_errors[0].description.contains("web_search"));
}
// ── thread_failure detection ─────────────────────────────
#[test]
fn detects_failed_thread_state() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("trying"));
thread.state = ThreadState::Failed;
let issues = analyze_trace(&thread);
assert!(issues.iter().any(|i| i.category == "thread_failure"));
}
// ── LLM error detection from StateChanged events ─────────
/// Reproduces the exact pattern from the trace: OpenAI rejects empty call_id.
#[test]
fn detects_llm_error_from_state_changed() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("ok"));
thread.state = ThreadState::Failed;
thread.events.push(ThreadEvent::new(
thread.id,
EventKind::StateChanged {
from: ThreadState::Running,
to: ThreadState::Failed,
reason: Some(
"LLM error: Provider openai_codex request failed: HTTP 400 Bad Request: \
Invalid 'input[5].call_id': empty string"
.into(),
),
},
));
let issues = analyze_trace(&thread);
assert!(
issues.iter().any(|i| i.category == "llm_error"),
"should detect LLM provider error in StateChanged reason"
);
}
// ── Multiple empty call_ids ──────────────────────────────
/// Anthropic sends consecutive tool results merged into one User message.
/// If multiple ActionResults have empty call_ids, each must be flagged.
#[test]
fn flags_each_empty_call_id_separately() {
let mut thread = make_thread();
thread.add_message(ThreadMessage::system("sys"));
thread.add_message(ThreadMessage::assistant("parallel calls"));
thread.add_message(ThreadMessage::action_result("", "tool_a", "result_a"));
thread.add_message(ThreadMessage::action_result("", "tool_b", "result_b"));
thread.add_message(ThreadMessage::action_result(
"call_ok", "tool_c", "result_c",
));
let issues = analyze_trace(&thread);
let empty_issues: Vec<_> = issues
.iter()
.filter(|i| i.category == "empty_call_id")
.collect();
assert_eq!(
empty_issues.len(),
2,
"should flag exactly the 2 empty call_ids"
);
}
}
+239
View File
@@ -0,0 +1,239 @@
//! IronClaw Engine — unified thread-capability-CodeAct execution model.
//!
//! This crate provides the core execution engine for IronClaw, unifying
//! ~10 separate abstractions (Session, Job, Routine, Channel, Tool, Skill,
//! Hook, Observer, Extension, LoopDelegate) around 5 primitives:
//!
//! - **Thread** — unit of work (replaces Session + Job + Routine + Sub-agent)
//! - **Step** — unit of execution (replaces agentic loop iteration + tool calls)
//! - **Capability** — unit of effect (replaces Tool + Skill + Hook + Extension)
//! - **MemoryDoc** — unit of durable knowledge (replaces workspace memory blobs)
//! - **Project** — unit of context (replaces flat workspace namespace)
//!
//! The engine defines traits for external dependencies ([`LlmBackend`],
//! [`Store`], [`EffectExecutor`]) that the host crate implements via bridge
//! adapters over existing infrastructure.
pub mod capability;
pub mod executor;
pub mod memory;
pub mod reliability;
pub mod runtime;
pub mod traits;
pub mod types;
// ── Re-exports: types ───────────────────────────────────────
pub use types::capability::{
ActionDef, Capability, CapabilityLease, EffectType, LeaseId, PolicyCondition, PolicyEffect,
PolicyRule,
};
pub use types::error::{CapabilityError, EngineError, StepError, ThreadError};
pub use types::event::{EventId, EventKind, ThreadEvent};
pub use types::memory::{DocId, DocType, MemoryDoc};
pub use types::message::{MessageRole, ThreadMessage};
pub use types::mission::{Mission, MissionCadence, MissionId, MissionStatus};
pub use types::project::{Project, ProjectId};
pub use types::provenance::Provenance;
pub use types::step::{
ActionCall, ActionResult, ExecutionTier, LlmResponse, Step, StepId, StepStatus, TokenUsage,
};
pub use types::thread::{Thread, ThreadConfig, ThreadId, ThreadState, ThreadType};
// ── Re-exports: traits ──────────────────────────────────────
pub use traits::effect::{EffectExecutor, ThreadExecutionContext};
pub use traits::llm::{LlmBackend, LlmCallConfig, LlmOutput};
pub use traits::store::Store;
// ── Re-exports: capability ────────────────────────────────────
pub use capability::lease::LeaseManager;
pub use capability::planner::{CapabilityGrantPlan, LeasePlanner};
pub use capability::policy::{PolicyDecision, PolicyEngine};
pub use capability::registry::CapabilityRegistry;
// ── Re-exports: runtime ───────────────────────────────────────
pub use executor::prompt::PlatformInfo;
pub use runtime::conversation::ConversationManager;
pub use runtime::manager::ThreadManager;
pub use runtime::messaging::ThreadOutcome;
pub use runtime::mission::MissionManager;
pub use runtime::tree::ThreadTree;
pub use types::conversation::{
ConversationEntry, ConversationId, ConversationSurface, EntrySender,
};
// ── Re-exports: executor ──────────────────────────────────────
pub use executor::ExecutionLoop;
// ── Re-exports: memory ────────────────────────────────────────
pub use memory::MemoryStore;
pub use memory::RetrievalEngine;
// ── Re-exports: reliability ──────────────────────────────────
pub use reliability::ReliabilityTracker;
// ── Test utilities ──────────────────────────────────────────
#[cfg(test)]
pub(crate) mod tests {
use tokio::sync::RwLock;
use crate::traits::store::Store;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::conversation::{ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Shared in-memory Store implementation for tests.
pub struct InMemoryStore {
docs: RwLock<Vec<MemoryDoc>>,
missions: RwLock<Vec<Mission>>,
}
impl InMemoryStore {
pub fn with_docs(docs: Vec<MemoryDoc>) -> Self {
Self {
docs: RwLock::new(docs),
missions: RwLock::new(Vec::new()),
}
}
}
#[async_trait::async_trait]
impl Store for InMemoryStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn list_projects(&self) -> Result<Vec<Project>, EngineError> {
Ok(vec![])
}
async fn save_conversation(&self, _: &ConversationSurface) -> Result<(), EngineError> {
Ok(())
}
async fn load_conversation(
&self,
_: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
Ok(None)
}
async fn list_conversations(
&self,
_: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
Ok(vec![])
}
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError> {
let mut docs = self.docs.write().await;
docs.retain(|d| d.id != doc.id);
docs.push(doc.clone());
Ok(())
}
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(self.docs.read().await.iter().find(|d| d.id == id).cloned())
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(self
.docs
.read()
.await
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
missions.retain(|m| m.id != mission.id);
missions.push(mission.clone());
Ok(())
}
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError> {
Ok(self
.missions
.read()
.await
.iter()
.find(|m| m.id == id)
.cloned())
}
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError> {
Ok(self
.missions
.read()
.await
.iter()
.filter(|m| m.project_id == project_id)
.cloned()
.collect())
}
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
if let Some(m) = missions.iter_mut().find(|m| m.id == id) {
m.status = status;
}
Ok(())
}
}
}
+10
View File
@@ -0,0 +1,10 @@
//! Memory document system.
//!
//! - [`MemoryStore`] — project-scoped document CRUD
//! - [`RetrievalEngine`] — context building from project docs via keyword search
pub mod retrieval;
pub mod store;
pub use retrieval::RetrievalEngine;
pub use store::MemoryStore;
@@ -0,0 +1,413 @@
//! Context retrieval engine.
//!
//! Builds context for thread steps by retrieving relevant memory docs
//! from the project. Uses keyword matching against doc title + content,
//! with priority scoring by doc type (Lessons and Specs rank higher
//! than Summaries for context injection).
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocType, MemoryDoc};
use crate::types::project::ProjectId;
/// Retrieves relevant memory docs for a thread's context.
pub struct RetrievalEngine {
store: Arc<dyn Store>,
}
impl RetrievalEngine {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Retrieve relevant memory docs for the given query within a project.
///
/// Loads all docs for the project, scores them by keyword relevance and
/// doc-type priority, and returns the top `max_docs` results.
pub async fn retrieve_context(
&self,
project_id: ProjectId,
query: &str,
max_docs: usize,
) -> Result<Vec<MemoryDoc>, EngineError> {
if max_docs == 0 {
return Ok(Vec::new());
}
let all_docs = self.store.list_memory_docs(project_id).await?;
if all_docs.is_empty() {
return Ok(Vec::new());
}
let keywords = extract_keywords(query);
if keywords.is_empty() {
// No meaningful keywords — return by doc-type priority alone
let mut scored: Vec<(f64, MemoryDoc)> = all_docs
.into_iter()
.map(|doc| (doc_type_weight(doc.doc_type), doc))
.collect();
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
scored.truncate(max_docs);
return Ok(scored.into_iter().map(|(_, doc)| doc).collect());
}
let mut scored: Vec<(f64, MemoryDoc)> = all_docs
.into_iter()
.map(|doc| {
let keyword_score = keyword_match_score(&doc, &keywords);
let type_weight = doc_type_weight(doc.doc_type);
// Combined score: keyword relevance (0.0-1.0) + type priority bonus
let score = keyword_score + type_weight;
(score, doc)
})
.filter(|(score, _)| *score > 0.0)
.collect();
scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
scored.truncate(max_docs);
Ok(scored.into_iter().map(|(_, doc)| doc).collect())
}
}
/// Extract lowercase keywords from a query, filtering out stop words.
fn extract_keywords(query: &str) -> Vec<String> {
const STOP_WORDS: &[&str] = &[
"a", "an", "the", "is", "are", "was", "were", "be", "been", "being", "have", "has", "had",
"do", "does", "did", "will", "would", "could", "should", "may", "might", "shall", "can",
"to", "of", "in", "for", "on", "with", "at", "by", "from", "as", "into", "about", "it",
"its", "this", "that", "these", "those", "i", "you", "he", "she", "we", "they", "what",
"which", "who", "how", "when", "where", "why", "and", "or", "but", "not", "no", "if",
"then", "so", "up", "out", "just",
];
query
.split(|c: char| !c.is_alphanumeric() && c != '_' && c != '-')
.map(|w| w.to_lowercase())
.filter(|w| w.len() >= 2 && !STOP_WORDS.contains(&w.as_str()))
.collect()
}
/// Score how well a doc matches the given keywords (0.0 to 1.0).
fn keyword_match_score(doc: &MemoryDoc, keywords: &[String]) -> f64 {
if keywords.is_empty() {
return 0.0;
}
let title_lower = doc.title.to_lowercase();
let content_lower = doc.content.to_lowercase();
let mut matched = 0usize;
for kw in keywords {
// Title matches are worth more
if title_lower.contains(kw.as_str()) {
matched += 2;
} else if content_lower.contains(kw.as_str()) {
matched += 1;
}
}
// Normalize: max possible score is keywords.len() * 2 (all in title)
let max_score = keywords.len() * 2;
matched as f64 / max_score as f64
}
/// Priority weight by doc type. Higher = more useful for context injection.
fn doc_type_weight(doc_type: DocType) -> f64 {
match doc_type {
DocType::Spec => 0.5, // Missing capability info is highest priority
DocType::Skill => 0.45, // Skills with activation metadata and code snippets
DocType::Lesson => 0.4, // Lessons prevent repeating mistakes
DocType::Issue => 0.2, // Known problems
DocType::Summary => 0.1, // Background context
DocType::Note => 0.05, // Scratch notes, lowest priority
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::event::ThreadEvent;
use crate::types::memory::DocId;
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Mock Store that returns a fixed set of memory docs.
struct DocStore {
docs: tokio::sync::Mutex<Vec<MemoryDoc>>,
}
impl DocStore {
fn new(docs: Vec<MemoryDoc>) -> Arc<Self> {
Arc::new(Self {
docs: tokio::sync::Mutex::new(docs),
})
}
}
#[async_trait::async_trait]
impl crate::traits::store::Store for DocStore {
async fn save_thread(&self, _: &Thread) -> Result<(), EngineError> {
Ok(())
}
async fn load_thread(&self, _: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(None)
}
async fn list_threads(&self, _: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(vec![])
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
let docs = self.docs.lock().await;
Ok(docs
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(&self, _: LeaseId, _: &str) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
#[test]
fn extract_keywords_filters_stop_words() {
let kws = extract_keywords("what is the latest news about Iran war");
assert!(kws.contains(&"latest".to_string()));
assert!(kws.contains(&"news".to_string()));
assert!(kws.contains(&"iran".to_string()));
assert!(kws.contains(&"war".to_string()));
assert!(!kws.contains(&"the".to_string()));
assert!(!kws.contains(&"is".to_string()));
}
#[test]
fn extract_keywords_handles_special_chars() {
let kws = extract_keywords("web_search web-fetch tool");
assert!(kws.contains(&"web_search".to_string()));
assert!(kws.contains(&"web-fetch".to_string()));
assert!(kws.contains(&"tool".to_string()));
}
#[test]
fn keyword_match_title_beats_content() {
use crate::types::project::ProjectId;
let doc = MemoryDoc::new(
ProjectId::new(),
DocType::Lesson,
"Lesson about web_search errors",
"The tool was not found during execution.",
);
let keywords = vec!["web_search".to_string()];
let score = keyword_match_score(&doc, &keywords);
// Title match = 2/2 = 1.0
assert!((score - 1.0).abs() < f64::EPSILON);
let keywords2 = vec!["execution".to_string()];
let score2 = keyword_match_score(&doc, &keywords2);
// Content-only match = 1/2 = 0.5
assert!((score2 - 0.5).abs() < f64::EPSILON);
}
#[test]
fn doc_type_weight_ordering() {
assert!(doc_type_weight(DocType::Spec) > doc_type_weight(DocType::Lesson));
assert!(doc_type_weight(DocType::Lesson) > doc_type_weight(DocType::Issue));
assert!(doc_type_weight(DocType::Issue) > doc_type_weight(DocType::Summary));
assert!(doc_type_weight(DocType::Summary) > doc_type_weight(DocType::Note));
}
#[tokio::test]
async fn retrieve_returns_relevant_docs_by_keyword() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project,
DocType::Lesson,
"web_search tool alias",
"Use web-search not web_search",
),
MemoryDoc::new(
project,
DocType::Summary,
"weather query",
"Fetched weather data",
),
MemoryDoc::new(
project,
DocType::Issue,
"API timeout",
"External API timed out",
),
]);
let engine = RetrievalEngine::new(store);
let docs = engine
.retrieve_context(project, "web_search error", 5)
.await
.unwrap();
assert!(!docs.is_empty());
// The lesson about web_search should rank first (keyword + type weight)
assert_eq!(docs[0].doc_type, DocType::Lesson);
assert!(docs[0].title.contains("web_search"));
}
#[tokio::test]
async fn retrieve_respects_project_scoping() {
let project_a = ProjectId::new();
let project_b = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project_a,
DocType::Lesson,
"Lesson for project A",
"Some lesson",
),
MemoryDoc::new(
project_b,
DocType::Lesson,
"Lesson for project B",
"Other lesson",
),
]);
let engine = RetrievalEngine::new(store);
let docs_a = engine
.retrieve_context(project_a, "lesson", 5)
.await
.unwrap();
assert_eq!(docs_a.len(), 1);
assert!(docs_a[0].title.contains("project A"));
let docs_b = engine
.retrieve_context(project_b, "lesson", 5)
.await
.unwrap();
assert_eq!(docs_b.len(), 1);
assert!(docs_b[0].title.contains("project B"));
}
#[tokio::test]
async fn retrieve_respects_max_docs_limit() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(project, DocType::Lesson, "Lesson 1", "Content 1"),
MemoryDoc::new(project, DocType::Lesson, "Lesson 2", "Content 2"),
MemoryDoc::new(project, DocType::Lesson, "Lesson 3", "Content 3"),
]);
let engine = RetrievalEngine::new(store);
let docs = engine.retrieve_context(project, "lesson", 2).await.unwrap();
assert_eq!(docs.len(), 2);
}
#[tokio::test]
async fn retrieve_empty_store_returns_empty() {
let project = ProjectId::new();
let store = DocStore::new(vec![]);
let engine = RetrievalEngine::new(store);
let docs = engine
.retrieve_context(project, "anything", 5)
.await
.unwrap();
assert!(docs.is_empty());
}
#[tokio::test]
async fn retrieve_spec_ranks_above_summary() {
let project = ProjectId::new();
let store = DocStore::new(vec![
MemoryDoc::new(
project,
DocType::Summary,
"Summary of search",
"searched the web",
),
MemoryDoc::new(
project,
DocType::Spec,
"Missing search tool",
"ALIAS: web_search -> web-search",
),
]);
let engine = RetrievalEngine::new(store);
let docs = engine.retrieve_context(project, "search", 5).await.unwrap();
assert_eq!(docs.len(), 2);
// Spec should rank first due to higher type weight
assert_eq!(docs[0].doc_type, DocType::Spec);
}
}
+408
View File
@@ -0,0 +1,408 @@
//! Project-scoped memory document operations.
use std::sync::Arc;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::memory::{DocId, DocType, MemoryDoc};
use crate::types::project::ProjectId;
use crate::types::thread::ThreadId;
/// Thin wrapper over the [`Store`] trait for project-scoped doc operations.
pub struct MemoryStore {
store: Arc<dyn Store>,
}
impl MemoryStore {
pub fn new(store: Arc<dyn Store>) -> Self {
Self { store }
}
/// Create a new memory document.
pub async fn create_doc(
&self,
project_id: ProjectId,
doc_type: DocType,
title: &str,
content: &str,
) -> Result<MemoryDoc, EngineError> {
let doc = MemoryDoc::new(project_id, doc_type, title, content);
self.store.save_memory_doc(&doc).await?;
Ok(doc)
}
/// Create a doc linked to a source thread.
pub async fn create_doc_from_thread(
&self,
project_id: ProjectId,
doc_type: DocType,
title: &str,
content: &str,
source_thread_id: ThreadId,
) -> Result<MemoryDoc, EngineError> {
let doc = MemoryDoc::new(project_id, doc_type, title, content)
.with_source_thread(source_thread_id);
self.store.save_memory_doc(&doc).await?;
Ok(doc)
}
/// Load a single doc by ID.
pub async fn get_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
self.store.load_memory_doc(id).await
}
/// List all docs in a project, optionally filtered by type.
pub async fn list_docs(
&self,
project_id: ProjectId,
doc_type: Option<DocType>,
) -> Result<Vec<MemoryDoc>, EngineError> {
let all = self.store.list_memory_docs(project_id).await?;
match doc_type {
Some(dt) => Ok(all.into_iter().filter(|d| d.doc_type == dt).collect()),
None => Ok(all),
}
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use tokio::sync::RwLock;
use crate::traits::store::Store;
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, DocType, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
use super::MemoryStore;
// ── In-memory Store implementation ───────────────────────
struct InMemoryDocStore {
docs: RwLock<Vec<MemoryDoc>>,
threads: RwLock<Vec<Thread>>,
steps: RwLock<Vec<Step>>,
events: RwLock<Vec<ThreadEvent>>,
projects: RwLock<Vec<Project>>,
leases: RwLock<Vec<CapabilityLease>>,
missions: RwLock<Vec<Mission>>,
}
impl InMemoryDocStore {
fn new() -> Self {
Self {
docs: RwLock::new(Vec::new()),
threads: RwLock::new(Vec::new()),
steps: RwLock::new(Vec::new()),
events: RwLock::new(Vec::new()),
projects: RwLock::new(Vec::new()),
leases: RwLock::new(Vec::new()),
missions: RwLock::new(Vec::new()),
}
}
}
#[async_trait::async_trait]
impl Store for InMemoryDocStore {
// ── Thread operations ────────────────────────────────
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError> {
let mut threads = self.threads.write().await;
threads.retain(|t| t.id != thread.id);
threads.push(thread.clone());
Ok(())
}
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError> {
let threads = self.threads.read().await;
Ok(threads.iter().find(|t| t.id == id).cloned())
}
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError> {
let threads = self.threads.read().await;
Ok(threads
.iter()
.filter(|t| t.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
id: ThreadId,
state: ThreadState,
) -> Result<(), EngineError> {
let mut threads = self.threads.write().await;
if let Some(t) = threads.iter_mut().find(|t| t.id == id) {
t.state = state;
}
Ok(())
}
// ── Step operations ──────────────────────────────────
async fn save_step(&self, step: &Step) -> Result<(), EngineError> {
let mut steps = self.steps.write().await;
steps.retain(|s| s.id != step.id);
steps.push(step.clone());
Ok(())
}
async fn load_steps(&self, thread_id: ThreadId) -> Result<Vec<Step>, EngineError> {
let steps = self.steps.read().await;
Ok(steps
.iter()
.filter(|s| s.thread_id == thread_id)
.cloned()
.collect())
}
// ── Event operations ─────────────────────────────────
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError> {
let mut stored = self.events.write().await;
stored.extend(events.iter().cloned());
Ok(())
}
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
let events = self.events.read().await;
Ok(events
.iter()
.filter(|e| e.thread_id == thread_id)
.cloned()
.collect())
}
// ── Project operations ───────────────────────────────
async fn save_project(&self, project: &Project) -> Result<(), EngineError> {
let mut projects = self.projects.write().await;
projects.retain(|p| p.id != project.id);
projects.push(project.clone());
Ok(())
}
async fn load_project(&self, id: ProjectId) -> Result<Option<Project>, EngineError> {
let projects = self.projects.read().await;
Ok(projects.iter().find(|p| p.id == id).cloned())
}
// ── Memory doc operations ────────────────────────────
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError> {
let mut docs = self.docs.write().await;
docs.push(doc.clone());
Ok(())
}
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError> {
let docs = self.docs.read().await;
Ok(docs.iter().find(|d| d.id == id).cloned())
}
async fn list_memory_docs(
&self,
project_id: ProjectId,
) -> Result<Vec<MemoryDoc>, EngineError> {
let docs = self.docs.read().await;
Ok(docs
.iter()
.filter(|d| d.project_id == project_id)
.cloned()
.collect())
}
// ── Capability lease operations ──────────────────────
async fn save_lease(&self, lease: &CapabilityLease) -> Result<(), EngineError> {
let mut leases = self.leases.write().await;
leases.retain(|l| l.id != lease.id);
leases.push(lease.clone());
Ok(())
}
async fn load_active_leases(
&self,
thread_id: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
let leases = self.leases.read().await;
Ok(leases
.iter()
.filter(|l| l.thread_id == thread_id && !l.revoked)
.cloned()
.collect())
}
async fn revoke_lease(&self, lease_id: LeaseId, _reason: &str) -> Result<(), EngineError> {
let mut leases = self.leases.write().await;
if let Some(l) = leases.iter_mut().find(|l| l.id == lease_id) {
l.revoked = true;
}
Ok(())
}
// ── Mission operations ───────────────────────────────
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
missions.retain(|m| m.id != mission.id);
missions.push(mission.clone());
Ok(())
}
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError> {
let missions = self.missions.read().await;
Ok(missions.iter().find(|m| m.id == id).cloned())
}
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError> {
let missions = self.missions.read().await;
Ok(missions
.iter()
.filter(|m| m.project_id == project_id)
.cloned()
.collect())
}
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError> {
let mut missions = self.missions.write().await;
if let Some(m) = missions.iter_mut().find(|m| m.id == id) {
m.status = status;
}
Ok(())
}
}
fn make_store() -> MemoryStore {
MemoryStore::new(Arc::new(InMemoryDocStore::new()))
}
// ── Tests ────────────────────────────────────────────────
#[tokio::test]
async fn create_doc_and_get() {
let store = make_store();
let project_id = ProjectId::new();
let doc = store
.create_doc(project_id, DocType::Summary, "Test Doc", "Some content")
.await
.unwrap();
assert_eq!(doc.title, "Test Doc");
assert_eq!(doc.content, "Some content");
assert_eq!(doc.doc_type, DocType::Summary);
assert_eq!(doc.project_id, project_id);
assert!(doc.source_thread_id.is_none());
let loaded = store.get_doc(doc.id).await.unwrap();
let loaded = loaded.unwrap();
assert_eq!(loaded.id, doc.id);
assert_eq!(loaded.title, "Test Doc");
assert_eq!(loaded.content, "Some content");
}
#[tokio::test]
async fn create_doc_from_thread_links_source() {
let store = make_store();
let project_id = ProjectId::new();
let thread_id = ThreadId::new();
let doc = store
.create_doc_from_thread(
project_id,
DocType::Lesson,
"Thread Lesson",
"Learned something",
thread_id,
)
.await
.unwrap();
assert_eq!(doc.source_thread_id, Some(thread_id));
assert_eq!(doc.doc_type, DocType::Lesson);
let loaded = store.get_doc(doc.id).await.unwrap().unwrap();
assert_eq!(loaded.source_thread_id, Some(thread_id));
}
#[tokio::test]
async fn list_docs_by_project() {
let store = make_store();
let project_a = ProjectId::new();
let project_b = ProjectId::new();
store
.create_doc(project_a, DocType::Note, "A1", "content a1")
.await
.unwrap();
store
.create_doc(project_a, DocType::Note, "A2", "content a2")
.await
.unwrap();
store
.create_doc(project_b, DocType::Note, "B1", "content b1")
.await
.unwrap();
let docs_a = store.list_docs(project_a, None).await.unwrap();
assert_eq!(docs_a.len(), 2);
assert!(docs_a.iter().all(|d| d.project_id == project_a));
let docs_b = store.list_docs(project_b, None).await.unwrap();
assert_eq!(docs_b.len(), 1);
assert_eq!(docs_b[0].title, "B1");
}
#[tokio::test]
async fn list_docs_filters_by_type() {
let store = make_store();
let project_id = ProjectId::new();
store
.create_doc(project_id, DocType::Summary, "S1", "summary content")
.await
.unwrap();
store
.create_doc(project_id, DocType::Lesson, "L1", "lesson content")
.await
.unwrap();
store
.create_doc(project_id, DocType::Summary, "S2", "another summary")
.await
.unwrap();
let summaries = store
.list_docs(project_id, Some(DocType::Summary))
.await
.unwrap();
assert_eq!(summaries.len(), 2);
assert!(summaries.iter().all(|d| d.doc_type == DocType::Summary));
let lessons = store
.list_docs(project_id, Some(DocType::Lesson))
.await
.unwrap();
assert_eq!(lessons.len(), 1);
assert_eq!(lessons[0].title, "L1");
}
#[tokio::test]
async fn get_nonexistent_returns_none() {
let store = make_store();
let result = store.get_doc(DocId::new()).await.unwrap();
assert!(result.is_none());
}
}
+194
View File
@@ -0,0 +1,194 @@
//! Tool reliability tracking with exponential moving averages.
//!
//! Tracks per-action success rate and latency using EMA (exponential moving
//! average) to smooth out noise. This data can be injected into the context
//! builder to inform the LLM about unreliable tools.
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::RwLock;
/// EMA smoothing factor. Higher = more weight on recent observations.
const EMA_ALPHA: f64 = 0.3;
/// Per-action reliability metrics.
#[derive(Debug, Clone)]
pub struct ActionMetrics {
/// EMA of success rate (0.0 to 1.0).
pub success_rate: f64,
/// EMA of latency in milliseconds.
pub avg_latency_ms: f64,
/// Total number of calls recorded.
pub call_count: u64,
/// Last error message (if any).
pub last_error: Option<String>,
}
impl Default for ActionMetrics {
fn default() -> Self {
Self {
success_rate: 1.0, // assume success until proven otherwise
avg_latency_ms: 0.0,
call_count: 0,
last_error: None,
}
}
}
/// Thread-safe registry of per-action reliability metrics.
#[derive(Clone)]
pub struct ReliabilityTracker {
metrics: Arc<RwLock<HashMap<String, ActionMetrics>>>,
}
impl ReliabilityTracker {
pub fn new() -> Self {
Self {
metrics: Arc::new(RwLock::new(HashMap::new())),
}
}
/// Record a successful action execution.
pub async fn record_success(&self, action_name: &str, latency: Duration) {
let mut metrics = self.metrics.write().await;
let entry = metrics.entry(action_name.to_string()).or_default();
entry.call_count += 1;
let latency_ms = latency.as_millis() as f64;
if entry.call_count == 1 {
// First observation — use raw values
entry.avg_latency_ms = latency_ms;
// success_rate stays at 1.0
} else {
entry.success_rate = ema(entry.success_rate, 1.0);
entry.avg_latency_ms = ema(entry.avg_latency_ms, latency_ms);
}
}
/// Record a failed action execution.
pub async fn record_failure(&self, action_name: &str, error: &str) {
let mut metrics = self.metrics.write().await;
let entry = metrics.entry(action_name.to_string()).or_default();
entry.call_count += 1;
entry.last_error = Some(error.to_string());
if entry.call_count == 1 {
entry.success_rate = 0.0;
} else {
entry.success_rate = ema(entry.success_rate, 0.0);
}
}
/// Get metrics for a specific action.
pub async fn get_metrics(&self, action_name: &str) -> Option<ActionMetrics> {
let metrics = self.metrics.read().await;
metrics.get(action_name).cloned()
}
/// Get all metrics, sorted by success rate (worst first).
pub async fn all_metrics(&self) -> Vec<(String, ActionMetrics)> {
let metrics = self.metrics.read().await;
let mut entries: Vec<(String, ActionMetrics)> = metrics
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
entries.sort_by(|a, b| {
a.1.success_rate
.partial_cmp(&b.1.success_rate)
.unwrap_or(std::cmp::Ordering::Equal)
});
entries
}
/// Get actions with reliability below a threshold.
pub async fn unreliable_actions(&self, threshold: f64) -> Vec<(String, ActionMetrics)> {
let all = self.all_metrics().await;
all.into_iter()
.filter(|(_, m)| m.success_rate < threshold)
.collect()
}
}
impl Default for ReliabilityTracker {
fn default() -> Self {
Self::new()
}
}
/// Compute exponential moving average.
fn ema(prev: f64, new: f64) -> f64 {
EMA_ALPHA * new + (1.0 - EMA_ALPHA) * prev
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn ema_moves_toward_new() {
let result = ema(1.0, 0.0);
// 0.3 * 0.0 + 0.7 * 1.0 = 0.7
assert!((result - 0.7).abs() < f64::EPSILON);
}
#[test]
fn ema_converges_on_repeated() {
let mut val = 1.0;
for _ in 0..20 {
val = ema(val, 0.0);
}
// Should converge toward 0.0
assert!(val < 0.01);
}
#[tokio::test]
async fn track_success() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("tool_a", Duration::from_millis(100))
.await;
tracker
.record_success("tool_a", Duration::from_millis(200))
.await;
let m = tracker.get_metrics("tool_a").await.unwrap();
assert_eq!(m.call_count, 2);
assert!((m.success_rate - 1.0).abs() < f64::EPSILON);
assert!(m.avg_latency_ms > 100.0); // EMA of 100 and 200
}
#[tokio::test]
async fn track_failure_lowers_success_rate() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("tool_b", Duration::from_millis(50))
.await;
tracker.record_failure("tool_b", "not found").await;
let m = tracker.get_metrics("tool_b").await.unwrap();
assert_eq!(m.call_count, 2);
assert!(m.success_rate < 1.0);
assert_eq!(m.last_error, Some("not found".into()));
}
#[tokio::test]
async fn unreliable_actions_filters() {
let tracker = ReliabilityTracker::new();
tracker
.record_success("good_tool", Duration::from_millis(10))
.await;
tracker.record_failure("bad_tool", "always fails").await;
let unreliable = tracker.unreliable_actions(0.5).await;
assert_eq!(unreliable.len(), 1);
assert_eq!(unreliable[0].0, "bad_tool");
}
#[tokio::test]
async fn unknown_action_returns_none() {
let tracker = ReliabilityTracker::new();
assert!(tracker.get_metrics("nonexistent").await.is_none());
}
}
@@ -0,0 +1,816 @@
//! Conversation manager — routes UI messages to threads.
//!
//! The ConversationManager is the bridge between channel I/O (user messages,
//! status updates) and the thread execution model. It maintains conversation
//! surfaces and decides whether to spawn new threads or inject messages into
//! existing ones.
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
use tracing::debug;
use crate::runtime::manager::ThreadManager;
use crate::runtime::messaging::ThreadOutcome;
use crate::traits::store::Store;
use crate::types::conversation::{ConversationEntry, ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::thread::{ThreadConfig, ThreadId, ThreadState, ThreadType};
enum ActiveForeground {
Running(ThreadId),
Resumable(ThreadId),
}
/// Manages conversation surfaces and routes messages to threads.
///
/// Each channel message arrives here. The manager decides whether to:
/// 1. Spawn a new foreground thread for the message
/// 2. Inject the message into an existing active thread
/// 3. Create a new conversation if none exists for this channel+user
pub struct ConversationManager {
thread_manager: Arc<ThreadManager>,
store: Arc<dyn Store>,
conversations: RwLock<HashMap<ConversationId, ConversationSurface>>,
/// Maps (channel, user_id) → conversation ID for lookup.
channel_user_index: RwLock<HashMap<(String, String), ConversationId>>,
}
impl ConversationManager {
pub fn new(thread_manager: Arc<ThreadManager>, store: Arc<dyn Store>) -> Self {
Self {
thread_manager,
store,
conversations: RwLock::new(HashMap::new()),
channel_user_index: RwLock::new(HashMap::new()),
}
}
/// Restore persisted conversations for a user into the in-memory index.
pub async fn bootstrap_user(&self, user_id: &str) -> Result<usize, EngineError> {
let conversations = self.store.list_conversations(user_id).await?;
let count = conversations.len();
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
for conversation in conversations {
index.insert(
(conversation.channel.clone(), conversation.user_id.clone()),
conversation.id,
);
convs.insert(conversation.id, conversation);
}
Ok(count)
}
/// Get or create a conversation for a channel+user pair.
pub async fn get_or_create_conversation(
&self,
channel: &str,
user_id: &str,
) -> Result<ConversationId, EngineError> {
// Check index first
let key = (channel.to_string(), user_id.to_string());
{
let index = self.channel_user_index.read().await;
if let Some(conv_id) = index.get(&key) {
return Ok(*conv_id);
}
}
// Check persisted conversations for this user/channel.
if let Some(conv) = self
.store
.list_conversations(user_id)
.await?
.into_iter()
.find(|conv| conv.channel == channel)
{
let conv_id = conv.id;
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
convs.insert(conv_id, conv);
index.insert(key, conv_id);
return Ok(conv_id);
}
// Create new conversation
let conv = ConversationSurface::new(channel, user_id);
let conv_id = conv.id;
let mut convs = self.conversations.write().await;
let mut index = self.channel_user_index.write().await;
convs.insert(conv_id, conv.clone());
index.insert(key, conv_id);
self.store.save_conversation(&conv).await?;
debug!(conversation_id = %conv_id, channel, user_id, "created conversation");
Ok(conv_id)
}
/// Handle an incoming user message.
///
/// If the conversation has an active foreground thread, the message is
/// injected into it. Otherwise, a new foreground thread is spawned.
///
/// Returns the thread ID that is handling the message.
pub async fn handle_user_message(
&self,
conversation_id: ConversationId,
content: &str,
project_id: ProjectId,
user_id: &str,
thread_config: ThreadConfig,
) -> Result<ThreadId, EngineError> {
let mut convs = self.conversations.write().await;
let conv = convs.get_mut(&conversation_id).ok_or(EngineError::Store {
reason: format!("conversation {conversation_id} not found"),
})?;
// Record the user entry
conv.add_entry(ConversationEntry::user(content));
// Check for an active foreground thread
let active_foreground = self.find_active_foreground(conv).await;
match active_foreground {
Some(ActiveForeground::Running(thread_id)) => {
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"injecting message into active thread"
);
self.thread_manager
.inject_message(thread_id, ThreadMessage::user(content))
.await?;
self.store.save_conversation(conv).await?;
Ok(thread_id)
}
Some(ActiveForeground::Resumable(thread_id)) => {
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"resuming suspended foreground thread"
);
self.thread_manager
.resume_thread(thread_id, user_id, Some(ThreadMessage::user(content)), None)
.await?;
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread resumed",
));
self.store.save_conversation(conv).await?;
Ok(thread_id)
}
None => {
// Build conversation history from prior entries for context continuity
let history = build_history_from_entries(&conv.entries);
// Spawn new foreground thread with conversation history
let thread_id = self
.thread_manager
.spawn_thread_with_history(
content, // use message as goal
ThreadType::Foreground,
project_id,
thread_config,
None,
user_id,
history,
)
.await?;
conv.track_thread(thread_id);
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread started",
));
self.store.save_conversation(conv).await?;
debug!(
conversation_id = %conversation_id,
thread_id = %thread_id,
"spawned new foreground thread"
);
Ok(thread_id)
}
}
}
/// Record a thread's outcome in its conversation.
pub async fn record_thread_outcome(
&self,
conversation_id: ConversationId,
thread_id: ThreadId,
outcome: &ThreadOutcome,
) -> Result<(), EngineError> {
let mut convs = self.conversations.write().await;
if let Some(conv) = convs.get_mut(&conversation_id) {
match outcome {
ThreadOutcome::Completed { response } => {
if let Some(text) = response {
conv.add_entry(ConversationEntry::agent(thread_id, text));
}
conv.untrack_thread(thread_id);
}
ThreadOutcome::Stopped => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread stopped",
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::MaxIterations => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
"Thread reached max iterations",
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::Failed { error } => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Thread failed: {error}"),
));
conv.untrack_thread(thread_id);
}
ThreadOutcome::NeedApproval {
action_name,
call_id: _,
parameters: _,
} => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Approval needed for action: {action_name}"),
));
// Thread stays active — waiting for approval
}
ThreadOutcome::NeedAuthentication {
credential_name,
action_name: _,
call_id: _,
parameters: _,
} => {
conv.add_entry(ConversationEntry::system_for_thread(
thread_id,
format!("Authentication required for credential: {credential_name}"),
));
// Thread stays active — waiting for OAuth completion
}
}
self.store.save_conversation(conv).await?;
}
Ok(())
}
/// Clear a conversation's entries and active threads.
///
/// Stops tracking all threads and removes conversation history so the next
/// user message spawns a fresh thread with no prior context.
pub async fn clear_conversation(
&self,
conversation_id: ConversationId,
) -> Result<(), EngineError> {
let mut convs = self.conversations.write().await;
if let Some(conv) = convs.get_mut(&conversation_id) {
conv.active_threads.clear();
conv.entries.clear();
conv.updated_at = chrono::Utc::now();
self.store.save_conversation(conv).await?;
debug!(conversation_id = %conversation_id, "cleared conversation");
}
Ok(())
}
/// Get a snapshot of a conversation.
pub async fn get_conversation(
&self,
conversation_id: ConversationId,
) -> Option<ConversationSurface> {
let convs = self.conversations.read().await;
convs.get(&conversation_id).cloned()
}
/// List all conversations for a user.
pub async fn list_conversations(&self, user_id: &str) -> Vec<ConversationSurface> {
let convs = self.conversations.read().await;
convs
.values()
.filter(|c| c.user_id == user_id)
.cloned()
.collect()
}
/// Find an active foreground thread in a conversation.
async fn find_active_foreground(&self, conv: &ConversationSurface) -> Option<ActiveForeground> {
for &tid in &conv.active_threads {
if self.thread_manager.is_running(tid).await {
return Some(ActiveForeground::Running(tid));
}
if let Ok(Some(thread)) = self.store.load_thread(tid).await
&& thread.thread_type == ThreadType::Foreground
&& thread.state == ThreadState::Suspended
{
return Some(ActiveForeground::Resumable(tid));
}
}
None
}
}
/// Build ThreadMessage history from conversation entries.
///
/// Converts user and agent entries into ThreadMessages so a new thread
/// inherits context from prior turns in the same conversation.
fn build_history_from_entries(
entries: &[ConversationEntry],
) -> Vec<crate::types::message::ThreadMessage> {
use crate::types::conversation::EntrySender;
// Skip the last entry (it's the current user message, added by the caller
// before this function runs). Also skip system entries (thread lifecycle
// notifications aren't useful as LLM context).
let history_entries = if entries.len() > 1 {
&entries[..entries.len() - 1]
} else {
return Vec::new();
};
history_entries
.iter()
.filter_map(|entry| match &entry.sender {
EntrySender::User => Some(crate::types::message::ThreadMessage::user(&entry.content)),
EntrySender::Agent { .. } => Some(crate::types::message::ThreadMessage::assistant(
&entry.content,
)),
EntrySender::System => None, // skip system notifications
})
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::capability::lease::LeaseManager;
use crate::capability::policy::PolicyEngine;
use crate::capability::registry::CapabilityRegistry;
use crate::traits::effect::EffectExecutor;
use crate::traits::llm::{LlmBackend, LlmCallConfig, LlmOutput};
use crate::traits::store::Store;
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::conversation::{ConversationId, ConversationSurface, EntrySender};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::project::Project;
use crate::types::step::{ActionResult, LlmResponse, Step, TokenUsage};
use crate::types::thread::ThreadState;
use std::sync::Mutex;
use std::time::Duration;
// ── Mocks (same as manager tests) ───────────────────────
struct MockLlm(Mutex<Vec<LlmOutput>>);
#[async_trait::async_trait]
impl LlmBackend for MockLlm {
async fn complete(
&self,
_: &[ThreadMessage],
_: &[ActionDef],
_: &LlmCallConfig,
) -> Result<LlmOutput, EngineError> {
let mut r = self.0.lock().unwrap();
if r.is_empty() {
Ok(LlmOutput {
response: LlmResponse::Text("done".into()),
usage: TokenUsage::default(),
})
} else {
Ok(r.remove(0))
}
}
fn model_name(&self) -> &str {
"mock"
}
}
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct MockStore {
conversations: RwLock<HashMap<ConversationId, ConversationSurface>>,
threads: RwLock<HashMap<ThreadId, crate::types::thread::Thread>>,
}
impl MockStore {
fn new() -> Self {
Self {
conversations: RwLock::new(HashMap::new()),
threads: RwLock::new(HashMap::new()),
}
}
}
#[async_trait::async_trait]
impl Store for MockStore {
async fn save_thread(
&self,
thread: &crate::types::thread::Thread,
) -> Result<(), EngineError> {
self.threads.write().await.insert(thread.id, thread.clone());
Ok(())
}
async fn load_thread(
&self,
id: ThreadId,
) -> Result<Option<crate::types::thread::Thread>, EngineError> {
Ok(self.threads.read().await.get(&id).cloned())
}
async fn list_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<crate::types::thread::Thread>, EngineError> {
Ok(self
.threads
.read()
.await
.values()
.filter(|thread| thread.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, _: &[ThreadEvent]) -> Result<(), EngineError> {
Ok(())
}
async fn load_events(&self, _: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(vec![])
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_conversation(
&self,
conversation: &ConversationSurface,
) -> Result<(), EngineError> {
self.conversations
.write()
.await
.insert(conversation.id, conversation.clone());
Ok(())
}
async fn load_conversation(
&self,
id: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
Ok(self.conversations.read().await.get(&id).cloned())
}
async fn list_conversations(
&self,
user_id: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
Ok(self
.conversations
.read()
.await
.values()
.filter(|conversation| conversation.user_id == user_id)
.cloned()
.collect())
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, _: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(vec![])
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(
&self,
_: crate::types::capability::LeaseId,
_: &str,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
fn make_conv_manager() -> (Arc<ThreadManager>, ConversationManager) {
let store = Arc::new(MockStore::new());
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![LlmOutput {
response: LlmResponse::Text("Hello!".into()),
usage: TokenUsage::default(),
}]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(Arc::clone(&tm), store);
(tm, cm)
}
// ── Tests ───────────────────────────────────────────────
#[tokio::test]
async fn get_or_create_conversation() {
let (_, cm) = make_conv_manager();
let c1 = cm
.get_or_create_conversation("telegram", "user1")
.await
.unwrap();
let c2 = cm
.get_or_create_conversation("telegram", "user1")
.await
.unwrap();
assert_eq!(c1, c2); // same channel+user returns same conversation
let c3 = cm
.get_or_create_conversation("slack", "user1")
.await
.unwrap();
assert_ne!(c1, c3); // different channel → different conversation
}
#[tokio::test]
async fn handle_message_spawns_thread() {
let (tm, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
let tid = cm
.handle_user_message(conv_id, "Hello", project, "user1", ThreadConfig::default())
.await
.unwrap();
// Thread was spawned
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.active_threads.contains(&tid));
assert_eq!(conv.entries.len(), 2); // user message + "Thread started"
// Wait for thread to complete
let outcome = tm.join_thread(tid).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
#[tokio::test]
async fn handle_message_resumes_suspended_thread() {
let store = Arc::new(MockStore::new());
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![LlmOutput {
response: LlmResponse::Text("Recovered".into()),
usage: TokenUsage::default(),
}]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(Arc::clone(&tm), store.clone());
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
let mut thread = crate::types::thread::Thread::new(
"resume",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
thread.transition_to(ThreadState::Running, None).unwrap();
thread.add_message(ThreadMessage::user("earlier"));
thread.step_count = 1;
thread.metadata = serde_json::json!({
"runtime_checkpoint": {
"persisted_state": {"last_return": 7},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
thread
.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)
.unwrap();
store.save_thread(&thread).await.unwrap();
{
let mut convs = cm.conversations.write().await;
let conv = convs.get_mut(&conv_id).unwrap();
conv.track_thread(thread.id);
}
let resumed = cm
.handle_user_message(
conv_id,
"continue from there",
project,
"user1",
ThreadConfig::default(),
)
.await
.unwrap();
assert_eq!(resumed, thread.id);
let outcome = tm.join_thread(thread.id).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
#[tokio::test]
async fn record_outcome_adds_entry() {
let (_, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("cli", "user1").await.unwrap();
let tid = ThreadId::new();
// Manually track a thread
{
let mut convs = cm.conversations.write().await;
let conv = convs.get_mut(&conv_id).unwrap();
conv.track_thread(tid);
}
// Record completion
cm.record_thread_outcome(
conv_id,
tid,
&ThreadOutcome::Completed {
response: Some("Done!".into()),
},
)
.await
.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.active_threads.is_empty());
assert_eq!(conv.entries.len(), 1);
assert_eq!(conv.entries[0].content, "Done!");
// Check sender is agent
assert!(matches!(
conv.entries[0].sender,
EntrySender::Agent { thread_id } if thread_id == tid
));
}
#[tokio::test]
async fn list_conversations_filters_by_user() {
let (_, cm) = make_conv_manager();
cm.get_or_create_conversation("web", "alice").await.unwrap();
cm.get_or_create_conversation("telegram", "alice")
.await
.unwrap();
cm.get_or_create_conversation("web", "bob").await.unwrap();
let alice_convs = cm.list_conversations("alice").await;
assert_eq!(alice_convs.len(), 2);
let bob_convs = cm.list_conversations("bob").await;
assert_eq!(bob_convs.len(), 1);
}
#[tokio::test]
async fn bootstrap_user_loads_persisted_conversations() {
let store = Arc::new(MockStore::new());
let mut conv = ConversationSurface::new("web", "user1");
conv.add_entry(ConversationEntry::user("persisted"));
store.save_conversation(&conv).await.unwrap();
let tm = Arc::new(ThreadManager::new(
Arc::new(MockLlm(Mutex::new(vec![]))),
Arc::new(MockEffects),
store.clone(),
Arc::new(CapabilityRegistry::new()),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
));
let cm = ConversationManager::new(tm, store);
let loaded = cm.bootstrap_user("user1").await.unwrap();
assert_eq!(loaded, 1);
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
assert_eq!(conv_id, conv.id);
let saved = cm.get_conversation(conv.id).await.unwrap();
assert_eq!(saved.entries.len(), 1);
assert_eq!(saved.entries[0].content, "persisted");
}
#[tokio::test]
async fn clear_conversation_resets_entries_and_threads() {
let (tm, cm) = make_conv_manager();
let conv_id = cm.get_or_create_conversation("web", "user1").await.unwrap();
let project = ProjectId::new();
// Spawn a thread so the conversation has entries and active threads
let tid = cm
.handle_user_message(conv_id, "Hello", project, "user1", ThreadConfig::default())
.await
.unwrap();
// Wait for thread to finish
let _ = tm.join_thread(tid).await.unwrap();
// Record outcome so there's an agent entry
cm.record_thread_outcome(
conv_id,
tid,
&ThreadOutcome::Completed {
response: Some("Hi there".into()),
},
)
.await
.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(!conv.entries.is_empty());
// Clear the conversation
cm.clear_conversation(conv_id).await.unwrap();
let conv = cm.get_conversation(conv_id).await.unwrap();
assert!(conv.entries.is_empty());
assert!(conv.active_threads.is_empty());
}
}
@@ -0,0 +1,995 @@
//! Thread manager — top-level orchestrator for thread lifecycle.
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
use tracing::{debug, error};
use crate::capability::lease::LeaseManager;
use crate::capability::planner::LeasePlanner;
use crate::capability::policy::PolicyEngine;
use crate::capability::registry::CapabilityRegistry;
use crate::executor::ExecutionLoop;
use crate::runtime::messaging::{self, SignalSender, ThreadOutcome, ThreadSignal};
use crate::runtime::tree::ThreadTree;
use crate::traits::effect::EffectExecutor;
use crate::traits::llm::LlmBackend;
use crate::traits::store::Store;
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::project::ProjectId;
use crate::types::thread::{Thread, ThreadConfig, ThreadId, ThreadState, ThreadType};
/// Handle to a running thread for checking results.
struct RunningThread {
signal_tx: SignalSender,
handle: tokio::task::JoinHandle<Result<ThreadOutcome, EngineError>>,
}
/// Top-level orchestrator for thread lifecycle.
///
/// Manages thread spawning, supervision, signaling, and tree relationships.
pub struct ThreadManager {
llm: Arc<dyn LlmBackend>,
effects: Arc<dyn EffectExecutor>,
store: Arc<dyn Store>,
pub capabilities: Arc<CapabilityRegistry>,
pub leases: Arc<LeaseManager>,
pub policy: Arc<PolicyEngine>,
lease_planner: LeasePlanner,
tree: RwLock<ThreadTree>,
running: Arc<RwLock<HashMap<ThreadId, RunningThread>>>,
completed: Arc<RwLock<HashMap<ThreadId, ThreadOutcome>>>,
/// Broadcast channel for thread events (for live status updates).
event_tx: tokio::sync::broadcast::Sender<crate::types::event::ThreadEvent>,
}
impl ThreadManager {
pub fn new(
llm: Arc<dyn LlmBackend>,
effects: Arc<dyn EffectExecutor>,
store: Arc<dyn Store>,
capabilities: Arc<CapabilityRegistry>,
leases: Arc<LeaseManager>,
policy: Arc<PolicyEngine>,
) -> Self {
let (event_tx, _) = tokio::sync::broadcast::channel(256);
Self {
llm,
effects,
store,
capabilities,
leases,
policy,
lease_planner: LeasePlanner::new(),
tree: RwLock::new(ThreadTree::new()),
running: Arc::new(RwLock::new(HashMap::new())),
completed: Arc::new(RwLock::new(HashMap::new())),
event_tx,
}
}
/// Subscribe to thread events for live status updates.
pub fn subscribe_events(
&self,
) -> tokio::sync::broadcast::Receiver<crate::types::event::ThreadEvent> {
self.event_tx.subscribe()
}
/// Spawn a new thread and start executing it.
///
/// Grants default capability leases for all registered capabilities.
/// Returns the thread ID immediately; the thread runs in a background task.
///
/// `initial_messages` provides conversation history from prior threads
/// (for context continuity across turns in the same conversation).
pub async fn spawn_thread(
&self,
goal: impl Into<String>,
thread_type: ThreadType,
project_id: ProjectId,
config: ThreadConfig,
parent_id: Option<ThreadId>,
user_id: impl Into<String>,
) -> Result<ThreadId, EngineError> {
self.spawn_thread_with_history(
goal,
thread_type,
project_id,
config,
parent_id,
user_id,
Vec::new(),
)
.await
}
/// Spawn a thread with initial conversation history.
#[allow(clippy::too_many_arguments)]
pub async fn spawn_thread_with_history(
&self,
goal: impl Into<String>,
thread_type: ThreadType,
project_id: ProjectId,
config: ThreadConfig,
parent_id: Option<ThreadId>,
user_id: impl Into<String>,
initial_messages: Vec<crate::types::message::ThreadMessage>,
) -> Result<ThreadId, EngineError> {
let mut thread = Thread::new(goal, thread_type, project_id, config);
if let Some(pid) = parent_id {
thread = thread.with_parent(pid);
}
let thread_id = thread.id;
let user_id = user_id.into();
if let Some(metadata) = thread.metadata.as_object_mut() {
metadata.insert("user_id".into(), serde_json::Value::String(user_id.clone()));
}
// Register in tree
if let Some(pid) = parent_id {
self.tree.write().await.add_child(pid, thread_id);
}
// Grant explicit capability leases based on thread type.
for grant in self
.lease_planner
.plan_for_thread(thread_type, &self.capabilities)
{
let lease = self
.leases
.grant(
thread_id,
grant.capability_name,
grant.granted_actions,
None,
None,
)
.await;
self.store.save_lease(&lease).await?;
thread.capability_leases.push(lease.id);
}
// Add conversation history from prior threads (for context continuity)
for msg in initial_messages {
thread.messages.push(msg);
}
// Add the goal as the current user message so the LLM has context
thread.add_message(crate::types::message::ThreadMessage::user(&thread.goal));
// Persist
self.store.save_thread(&thread).await?;
self.start_thread(thread, user_id, false).await
}
/// Resume a persisted waiting or suspended thread.
pub async fn resume_thread(
&self,
thread_id: ThreadId,
user_id: impl Into<String>,
injected_message: Option<ThreadMessage>,
approval_event: Option<(String, bool)>,
) -> Result<(), EngineError> {
if self.is_running(thread_id).await {
return Err(EngineError::Thread(
crate::types::error::ThreadError::AlreadyRunning(thread_id),
));
}
let mut thread = self
.store
.load_thread(thread_id)
.await?
.ok_or(EngineError::ThreadNotFound(thread_id))?;
if !matches!(
thread.state,
crate::types::thread::ThreadState::Waiting
| crate::types::thread::ThreadState::Suspended
) {
return Err(EngineError::Store {
reason: format!(
"thread {thread_id} is not resumable from {:?}",
thread.state
),
});
}
if let Some((call_id, approved)) = approval_event {
let event = crate::types::event::ThreadEvent::new(
thread_id,
crate::types::event::EventKind::ApprovalReceived { call_id, approved },
);
let _ = self.event_tx.send(event.clone());
thread.events.push(event);
thread.updated_at = chrono::Utc::now();
}
if let Some(message) = injected_message {
thread.add_message(message);
}
self.store.save_thread(&thread).await?;
self.start_thread(thread, user_id.into(), true).await?;
Ok(())
}
async fn start_thread(
&self,
thread: Thread,
user_id: String,
is_resume: bool,
) -> Result<ThreadId, EngineError> {
let thread_id = thread.id;
// Create signal channel
let (tx, rx) = messaging::signal_channel(32);
// Build execution loop
let llm = Arc::clone(&self.llm);
let effects = Arc::clone(&self.effects);
let leases = Arc::clone(&self.leases);
let policy = Arc::clone(&self.policy);
let store_for_retrieval = Arc::clone(&self.store);
let retrieval = crate::memory::RetrievalEngine::new(store_for_retrieval);
let exec_loop = ExecutionLoop::new(thread, llm, effects, leases, policy, rx, user_id)
.with_capabilities(Arc::clone(&self.capabilities))
.with_event_tx(self.event_tx.clone())
.with_retrieval(retrieval)
.with_store(Arc::clone(&self.store));
// Spawn background task
let store_for_task = Arc::clone(&self.store);
let running = Arc::clone(&self.running);
let completed = Arc::clone(&self.completed);
let handle = tokio::spawn(async move {
let mut exec = exec_loop;
let result = exec.run().await;
debug!(thread_id = %thread_id, "thread execution finished");
// Run retrospective trace analysis (non-LLM, always runs).
// Issues are picked up by the self-improvement mission via event listener.
let trace = crate::executor::trace::build_trace(&exec.thread);
if !trace.issues.is_empty() {
crate::executor::trace::log_trace_summary(&trace);
}
// Transition Completed → Done
if exec.thread.state == crate::types::thread::ThreadState::Completed
&& let Err(e) = exec
.thread
.transition_to(crate::types::thread::ThreadState::Done, None)
{
tracing::warn!(thread_id = %thread_id, "failed to transition to Done: {e}");
}
// Write trace file if enabled
if crate::executor::trace::is_trace_enabled() {
crate::executor::trace::log_trace_summary(&trace);
crate::executor::trace::write_trace(&trace);
}
if let Err(e) = store_for_task.append_events(&exec.thread.events).await {
tracing::warn!(
thread_id = %thread_id,
"failed to persist thread events: {e}"
);
}
// Save final thread state to store
if let Err(e) = store_for_task.save_thread(&exec.thread).await {
tracing::warn!(
thread_id = %thread_id,
"failed to save final thread state: {e}"
);
}
let outcome = match result {
Ok(outcome) => outcome,
Err(error) => ThreadOutcome::Failed {
error: error.to_string(),
},
};
completed.write().await.insert(thread_id, outcome.clone());
running.write().await.remove(&thread_id);
Ok(outcome)
});
self.running.write().await.insert(
thread_id,
RunningThread {
signal_tx: tx,
handle,
},
);
if is_resume {
debug!(thread_id = %thread_id, "resumed thread");
}
Ok(thread_id)
}
/// Send a stop signal to a running thread.
pub async fn stop_thread(&self, thread_id: ThreadId) -> Result<(), EngineError> {
let running = self.running.read().await;
if let Some(rt) = running.get(&thread_id) {
let _ = rt.signal_tx.send(ThreadSignal::Stop).await;
Ok(())
} else {
Err(EngineError::ThreadNotFound(thread_id))
}
}
/// Inject a user message into a running thread.
pub async fn inject_message(
&self,
thread_id: ThreadId,
message: ThreadMessage,
) -> Result<(), EngineError> {
let running = self.running.read().await;
if let Some(rt) = running.get(&thread_id) {
let _ = rt
.signal_tx
.send(ThreadSignal::InjectMessage(message))
.await;
Ok(())
} else {
Err(EngineError::ThreadNotFound(thread_id))
}
}
/// Check if a thread is still running.
pub async fn is_running(&self, thread_id: ThreadId) -> bool {
let running = self.running.read().await;
running
.get(&thread_id)
.is_some_and(|rt| !rt.handle.is_finished())
}
/// Wait for a thread to finish and return its outcome.
/// Removes the thread from the running set.
pub async fn join_thread(&self, thread_id: ThreadId) -> Result<ThreadOutcome, EngineError> {
if let Some(outcome) = self.completed.write().await.remove(&thread_id) {
return Ok(outcome);
}
let rt = {
let mut running = self.running.write().await;
running.remove(&thread_id)
};
match rt {
Some(rt) => match rt.handle.await {
Ok(result) => result,
Err(e) => {
error!(thread_id = %thread_id, "thread task panicked: {e}");
Ok(ThreadOutcome::Failed {
error: format!("thread task panicked: {e}"),
})
}
},
None => Err(EngineError::ThreadNotFound(thread_id)),
}
}
/// Get children of a thread.
pub async fn children_of(&self, thread_id: ThreadId) -> Vec<ThreadId> {
let tree = self.tree.read().await;
tree.children_of(thread_id).to_vec()
}
/// Get the parent of a thread.
pub async fn parent_of(&self, thread_id: ThreadId) -> Option<ThreadId> {
let tree = self.tree.read().await;
tree.parent_of(thread_id)
}
/// Clean up finished threads from the running set.
pub async fn cleanup_finished(&self) -> Vec<ThreadId> {
let mut running = self.running.write().await;
let finished: Vec<ThreadId> = running
.iter()
.filter(|(_, rt)| rt.handle.is_finished())
.map(|(id, _)| *id)
.collect();
for id in &finished {
running.remove(id);
}
finished
}
/// Automatically resume checkpointed non-foreground threads.
pub async fn resume_background_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<ThreadId>, EngineError> {
let threads = self.store.list_threads(project_id).await?;
let mut resumed = Vec::new();
for thread in threads {
if thread.state != ThreadState::Suspended {
continue;
}
if thread.thread_type != ThreadType::Research {
continue;
}
if thread.metadata.get("runtime_checkpoint").is_none() {
continue;
}
let Some(user_id) = thread
.metadata
.get("user_id")
.and_then(|value| value.as_str())
.filter(|user_id| !user_id.is_empty())
else {
continue;
};
self.resume_thread(thread.id, user_id.to_string(), None, None)
.await?;
resumed.push(thread.id);
}
Ok(resumed)
}
/// Reconcile persisted non-terminal threads after process startup.
///
/// The current engine does not support mid-thread replay/resume, so any
/// thread left in a non-terminal state is marked failed-safe.
pub async fn recover_project_threads(
&self,
project_id: ProjectId,
) -> Result<Vec<ThreadId>, EngineError> {
const PENDING_APPROVAL_METADATA_KEY: &str = "pending_approval";
const RUNTIME_CHECKPOINT_METADATA_KEY: &str = "runtime_checkpoint";
let threads = self.store.list_threads(project_id).await?;
let mut recovered = Vec::new();
for mut thread in threads {
if thread.state.is_terminal() || thread.state == ThreadState::Completed {
continue;
}
if thread.state == ThreadState::Waiting
&& thread.metadata.get(PENDING_APPROVAL_METADATA_KEY).is_some()
{
continue;
}
if thread
.metadata
.get(RUNTIME_CHECKPOINT_METADATA_KEY)
.is_some()
&& matches!(thread.state, ThreadState::Running | ThreadState::Suspended)
{
if thread.state == ThreadState::Running {
thread.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)?;
}
self.store.append_events(&thread.events).await?;
self.store.save_thread(&thread).await?;
recovered.push(thread.id);
continue;
}
if thread
.transition_to(
ThreadState::Failed,
Some("engine restart before thread completion".into()),
)
.is_ok()
{
self.store.append_events(&thread.events).await?;
self.store.save_thread(&thread).await?;
recovered.push(thread.id);
}
}
Ok(recovered)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::traits::llm::{LlmCallConfig, LlmOutput};
use crate::types::capability::{ActionDef, Capability, CapabilityLease, EffectType};
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::project::Project;
use crate::types::step::{ActionResult, LlmResponse, Step, TokenUsage};
use crate::types::thread::ThreadState;
use std::sync::Mutex;
use std::time::Duration;
// ── Mocks ───────────────────────────────────────────────
struct MockLlm {
responses: Mutex<Vec<LlmOutput>>,
}
impl MockLlm {
fn text(msg: &str) -> Arc<Self> {
Arc::new(Self {
responses: Mutex::new(vec![LlmOutput {
response: LlmResponse::Text(msg.into()),
usage: TokenUsage::default(),
}]),
})
}
}
#[async_trait::async_trait]
impl LlmBackend for MockLlm {
async fn complete(
&self,
_: &[crate::types::message::ThreadMessage],
_: &[ActionDef],
_: &LlmCallConfig,
) -> Result<LlmOutput, EngineError> {
let mut r = self.responses.lock().unwrap();
if r.is_empty() {
Ok(LlmOutput {
response: LlmResponse::Text("done".into()),
usage: TokenUsage::default(),
})
} else {
Ok(r.remove(0))
}
}
fn model_name(&self) -> &str {
"mock"
}
}
struct MockEffects;
#[async_trait::async_trait]
impl EffectExecutor for MockEffects {
async fn execute_action(
&self,
_: &str,
_: serde_json::Value,
_: &CapabilityLease,
_: &crate::traits::effect::ThreadExecutionContext,
) -> Result<ActionResult, EngineError> {
Ok(ActionResult {
call_id: String::new(),
action_name: String::new(),
output: serde_json::json!({}),
is_error: false,
duration: Duration::from_millis(1),
})
}
async fn available_actions(
&self,
_: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError> {
Ok(vec![])
}
}
struct MockStore {
threads: RwLock<HashMap<ThreadId, Thread>>,
events: RwLock<HashMap<ThreadId, Vec<ThreadEvent>>>,
}
impl MockStore {
fn new() -> Self {
Self {
threads: RwLock::new(HashMap::new()),
events: RwLock::new(HashMap::new()),
}
}
}
#[async_trait::async_trait]
impl Store for MockStore {
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError> {
self.threads.write().await.insert(thread.id, thread.clone());
Ok(())
}
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError> {
Ok(self.threads.read().await.get(&id).cloned())
}
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError> {
Ok(self
.threads
.read()
.await
.values()
.filter(|thread| thread.project_id == project_id)
.cloned()
.collect())
}
async fn update_thread_state(
&self,
_: ThreadId,
_: ThreadState,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_step(&self, _: &Step) -> Result<(), EngineError> {
Ok(())
}
async fn load_steps(&self, _: ThreadId) -> Result<Vec<Step>, EngineError> {
Ok(vec![])
}
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError> {
let mut stored = self.events.write().await;
for event in events {
stored
.entry(event.thread_id)
.or_default()
.push(event.clone());
}
Ok(())
}
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError> {
Ok(self
.events
.read()
.await
.get(&thread_id)
.cloned()
.unwrap_or_default())
}
async fn save_project(&self, _: &Project) -> Result<(), EngineError> {
Ok(())
}
async fn load_project(&self, _: ProjectId) -> Result<Option<Project>, EngineError> {
Ok(None)
}
async fn save_memory_doc(&self, _: &MemoryDoc) -> Result<(), EngineError> {
Ok(())
}
async fn load_memory_doc(&self, _: DocId) -> Result<Option<MemoryDoc>, EngineError> {
Ok(None)
}
async fn list_memory_docs(&self, _: ProjectId) -> Result<Vec<MemoryDoc>, EngineError> {
Ok(vec![])
}
async fn save_lease(&self, _: &CapabilityLease) -> Result<(), EngineError> {
Ok(())
}
async fn load_active_leases(
&self,
_: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError> {
Ok(vec![])
}
async fn revoke_lease(
&self,
_: crate::types::capability::LeaseId,
_: &str,
) -> Result<(), EngineError> {
Ok(())
}
async fn save_mission(
&self,
_: &crate::types::mission::Mission,
) -> Result<(), EngineError> {
Ok(())
}
async fn load_mission(
&self,
_: crate::types::mission::MissionId,
) -> Result<Option<crate::types::mission::Mission>, EngineError> {
Ok(None)
}
async fn list_missions(
&self,
_: ProjectId,
) -> Result<Vec<crate::types::mission::Mission>, EngineError> {
Ok(vec![])
}
async fn update_mission_status(
&self,
_: crate::types::mission::MissionId,
_: crate::types::mission::MissionStatus,
) -> Result<(), EngineError> {
Ok(())
}
}
fn make_manager(llm: Arc<dyn LlmBackend>) -> ThreadManager {
let mut caps = CapabilityRegistry::new();
caps.register(Capability {
name: "test".into(),
description: "Test capability".into(),
actions: vec![ActionDef {
name: "test_tool".into(),
description: "Test".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
ThreadManager::new(
llm,
Arc::new(MockEffects),
Arc::new(MockStore::new()),
Arc::new(caps),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
)
}
fn make_manager_with_store(llm: Arc<dyn LlmBackend>, store: Arc<MockStore>) -> ThreadManager {
let mut caps = CapabilityRegistry::new();
caps.register(Capability {
name: "test".into(),
description: "Test capability".into(),
actions: vec![ActionDef {
name: "test_tool".into(),
description: "Test".into(),
parameters_schema: serde_json::json!({}),
effects: vec![EffectType::ReadLocal],
requires_approval: false,
}],
knowledge: vec![],
policies: vec![],
});
ThreadManager::new(
llm,
Arc::new(MockEffects),
store,
Arc::new(caps),
Arc::new(LeaseManager::new()),
Arc::new(PolicyEngine::new()),
)
}
// ── Tests ───────────────────────────────────────────────
#[tokio::test]
async fn spawn_and_join() {
let mgr = make_manager(MockLlm::text("Hello!"));
let project = ProjectId::new();
let tid = mgr
.spawn_thread(
"test",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
let outcome = mgr.join_thread(tid).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { response: Some(r) } if r == "Hello!"));
}
#[tokio::test]
async fn stop_thread_works() {
// LLM that returns many action responses
let responses: Vec<LlmOutput> = (0..100)
.map(|i| LlmOutput {
response: LlmResponse::ActionCalls {
calls: vec![crate::types::step::ActionCall {
id: format!("c{i}"),
action_name: "test_tool".into(),
parameters: serde_json::json!({}),
}],
content: None,
},
usage: TokenUsage::default(),
})
.collect();
let mgr = make_manager(Arc::new(MockLlm {
responses: Mutex::new(responses),
}));
let project = ProjectId::new();
let tid = mgr
.spawn_thread(
"test",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
// Give it a moment to start, then stop
tokio::time::sleep(Duration::from_millis(10)).await;
let _ = mgr.stop_thread(tid).await;
let outcome = mgr.join_thread(tid).await.unwrap();
assert!(matches!(
outcome,
ThreadOutcome::Stopped | ThreadOutcome::Completed { .. } | ThreadOutcome::MaxIterations
));
}
#[tokio::test]
async fn parent_child_tree() {
let mgr = make_manager(MockLlm::text("parent done"));
let project = ProjectId::new();
let parent = mgr
.spawn_thread(
"parent",
ThreadType::Foreground,
project,
ThreadConfig::default(),
None,
"user",
)
.await
.unwrap();
let child = mgr
.spawn_thread(
"child",
ThreadType::Research,
project,
ThreadConfig::default(),
Some(parent),
"user",
)
.await
.unwrap();
assert_eq!(mgr.parent_of(child).await, Some(parent));
assert_eq!(mgr.children_of(parent).await, vec![child]);
}
#[tokio::test]
async fn recover_project_threads_marks_non_terminal_as_failed() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut running = Thread::new(
"running",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
running.transition_to(ThreadState::Running, None).unwrap();
store.save_thread(&running).await.unwrap();
let mut completed = Thread::new(
"done",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
completed
.transition_to(ThreadState::Failed, Some("already terminal".into()))
.unwrap();
store.save_thread(&completed).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert_eq!(recovered, vec![running.id]);
let saved = store.load_thread(running.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Failed);
let events = store.load_events(running.id).await.unwrap();
assert!(!events.is_empty());
}
#[tokio::test]
async fn recover_project_threads_preserves_waiting_approval_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut waiting = Thread::new(
"awaiting approval",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
waiting.transition_to(ThreadState::Running, None).unwrap();
waiting
.transition_to(ThreadState::Waiting, Some("approval".into()))
.unwrap();
waiting.metadata = serde_json::json!({
"pending_approval": {
"request_id": "req-1",
"action_name": "shell",
"call_id": "call-1"
}
});
store.save_thread(&waiting).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert!(recovered.is_empty());
let saved = store.load_thread(waiting.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Waiting);
}
#[tokio::test]
async fn recover_project_threads_suspends_checkpointed_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut running = Thread::new(
"resume me",
ThreadType::Foreground,
project,
ThreadConfig::default(),
);
running.transition_to(ThreadState::Running, None).unwrap();
running.metadata = serde_json::json!({
"runtime_checkpoint": {
"persisted_state": {"last_return": 7},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
store.save_thread(&running).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("ignored"), Arc::clone(&store));
let recovered = mgr.recover_project_threads(project).await.unwrap();
assert_eq!(recovered, vec![running.id]);
let saved = store.load_thread(running.id).await.unwrap().unwrap();
assert_eq!(saved.state, ThreadState::Suspended);
}
#[tokio::test]
async fn resume_background_threads_restarts_suspended_research_threads() {
let store = Arc::new(MockStore::new());
let project = ProjectId::new();
let mut research = Thread::new(
"background research",
ThreadType::Research,
project,
ThreadConfig::default(),
);
research.transition_to(ThreadState::Running, None).unwrap();
research.metadata = serde_json::json!({
"user_id": "owner",
"runtime_checkpoint": {
"persisted_state": {},
"nudge_count": 0,
"consecutive_errors": 0,
"compaction_count": 0
}
});
research
.transition_to(
ThreadState::Suspended,
Some("engine restart; resumable from checkpoint".into()),
)
.unwrap();
store.save_thread(&research).await.unwrap();
let mgr = make_manager_with_store(MockLlm::text("done"), Arc::clone(&store));
let resumed = mgr.resume_background_threads(project).await.unwrap();
assert_eq!(resumed, vec![research.id]);
let outcome = mgr.join_thread(research.id).await.unwrap();
assert!(matches!(outcome, ThreadOutcome::Completed { .. }));
}
// Skill selection and injection tests are in tests/engine_v2_skill_codeact.rs
// (skill selection happens in the Python orchestrator, not in Rust).
}
@@ -0,0 +1,61 @@
//! Thread-to-thread messaging via channels.
use crate::types::message::ThreadMessage;
use crate::types::thread::ThreadId;
/// Signal sent to a running thread via its mailbox.
#[derive(Debug)]
pub enum ThreadSignal {
/// Stop the thread gracefully.
Stop,
/// Pause execution (can be resumed later).
Suspend,
/// Resume a suspended thread.
Resume,
/// Inject a user message into the thread's context.
InjectMessage(ThreadMessage),
/// Notification that a child thread completed.
ChildCompleted {
child_id: ThreadId,
outcome: ThreadOutcome,
},
}
/// Final outcome of a thread's execution.
#[derive(Debug, Clone)]
pub enum ThreadOutcome {
/// Completed with an optional text response.
Completed { response: Option<String> },
/// Thread was stopped by a signal.
Stopped,
/// Max iterations reached without completing.
MaxIterations,
/// Terminal failure.
Failed { error: String },
/// A capability action requires user approval before continuing.
NeedApproval {
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
/// An action needs a credential that requires user authentication (e.g. OAuth).
/// The thread pauses until the credential is available, then resumes.
NeedAuthentication {
credential_name: String,
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
}
/// A mailbox for sending signals to a running thread.
///
/// Each thread gets a `(sender, receiver)` pair. The `ThreadManager` holds
/// the sender; the `ExecutionLoop` holds the receiver.
pub type SignalSender = tokio::sync::mpsc::Sender<ThreadSignal>;
pub type SignalReceiver = tokio::sync::mpsc::Receiver<ThreadSignal>;
/// Create a new signal channel with the given buffer size.
pub fn signal_channel(buffer: usize) -> (SignalSender, SignalReceiver) {
tokio::sync::mpsc::channel(buffer)
}
File diff suppressed because it is too large Load Diff
+17
View File
@@ -0,0 +1,17 @@
//! Thread lifecycle management.
//!
//! - [`ThreadManager`] — top-level orchestrator for spawning and supervising threads
//! - [`ThreadTree`] — parent-child relationship tracking
//! - [`messaging`] — inter-thread signal channel
pub mod conversation;
pub mod manager;
pub mod messaging;
pub mod mission;
pub mod tree;
pub use conversation::ConversationManager;
pub use manager::ThreadManager;
pub use messaging::ThreadOutcome;
pub use mission::MissionManager;
pub use tree::ThreadTree;
+129
View File
@@ -0,0 +1,129 @@
//! Thread tree — parent-child relationship tracking.
use std::collections::HashMap;
use crate::types::thread::ThreadId;
/// Manages parent-child thread relationships.
///
/// Simple in-memory tree. Threads form a forest (multiple roots).
#[derive(Debug, Default)]
pub struct ThreadTree {
/// child → parent
parents: HashMap<ThreadId, ThreadId>,
/// parent → children (ordered by insertion)
children: HashMap<ThreadId, Vec<ThreadId>>,
}
impl ThreadTree {
pub fn new() -> Self {
Self::default()
}
/// Register a parent-child relationship.
pub fn add_child(&mut self, parent_id: ThreadId, child_id: ThreadId) {
self.parents.insert(child_id, parent_id);
self.children.entry(parent_id).or_default().push(child_id);
}
/// Get the parent of a thread, if any.
pub fn parent_of(&self, thread_id: ThreadId) -> Option<ThreadId> {
self.parents.get(&thread_id).copied()
}
/// Get the children of a thread.
pub fn children_of(&self, thread_id: ThreadId) -> &[ThreadId] {
self.children
.get(&thread_id)
.map(Vec::as_slice)
.unwrap_or(&[])
}
/// Walk up the tree to collect all ancestors (parent, grandparent, ...).
pub fn ancestors(&self, thread_id: ThreadId) -> Vec<ThreadId> {
let mut result = Vec::new();
let mut current = thread_id;
while let Some(parent) = self.parents.get(&current) {
result.push(*parent);
current = *parent;
}
result
}
/// Remove a thread from the tree. Does not remove its children.
pub fn remove(&mut self, thread_id: ThreadId) {
if let Some(parent) = self.parents.remove(&thread_id)
&& let Some(siblings) = self.children.get_mut(&parent)
{
siblings.retain(|id| *id != thread_id);
}
// Orphan any children (their parent_id entries become stale)
self.children.remove(&thread_id);
}
/// Check if a thread is a root (no parent).
pub fn is_root(&self, thread_id: ThreadId) -> bool {
!self.parents.contains_key(&thread_id)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn add_and_query() {
let mut tree = ThreadTree::new();
let parent = ThreadId::new();
let child1 = ThreadId::new();
let child2 = ThreadId::new();
tree.add_child(parent, child1);
tree.add_child(parent, child2);
assert_eq!(tree.parent_of(child1), Some(parent));
assert_eq!(tree.parent_of(child2), Some(parent));
assert_eq!(tree.children_of(parent).len(), 2);
assert!(tree.is_root(parent));
assert!(!tree.is_root(child1));
}
#[test]
fn ancestors_walk_up() {
let mut tree = ThreadTree::new();
let root = ThreadId::new();
let mid = ThreadId::new();
let leaf = ThreadId::new();
tree.add_child(root, mid);
tree.add_child(mid, leaf);
let ancestors = tree.ancestors(leaf);
assert_eq!(ancestors, vec![mid, root]);
}
#[test]
fn remove_detaches_from_parent() {
let mut tree = ThreadTree::new();
let parent = ThreadId::new();
let child = ThreadId::new();
tree.add_child(parent, child);
tree.remove(child);
assert_eq!(tree.parent_of(child), None);
assert!(tree.children_of(parent).is_empty());
}
#[test]
fn children_of_unknown_returns_empty() {
let tree = ThreadTree::new();
assert!(tree.children_of(ThreadId::new()).is_empty());
}
#[test]
fn ancestors_of_root_is_empty() {
let tree = ThreadTree::new();
assert!(tree.ancestors(ThreadId::new()).is_empty());
}
}
@@ -0,0 +1,57 @@
//! Effect executor trait.
//!
//! The engine delegates actual action execution to the host through this
//! trait. The main crate implements it by wrapping `ToolRegistry` and
//! `SafetyLayer` — the engine itself has no knowledge of specific tools.
use crate::types::capability::{ActionDef, CapabilityLease};
use crate::types::error::EngineError;
use crate::types::project::ProjectId;
use crate::types::step::{ActionResult, StepId};
use crate::types::thread::{ThreadId, ThreadType};
/// Contextual information about the thread requesting an effect.
///
/// Passed to the executor so it can make context-dependent decisions
/// (e.g. different tool behavior in background vs foreground threads).
#[derive(Debug, Clone)]
pub struct ThreadExecutionContext {
pub thread_id: ThreadId,
pub thread_type: ThreadType,
pub project_id: ProjectId,
pub user_id: String,
pub step_id: StepId,
}
/// Abstraction over capability action execution.
///
/// The main crate implements this by wrapping its `ToolRegistry`, `SafetyLayer`,
/// and tool execution pipeline. The engine calls `execute_action` and gets back
/// a result — all safety, sanitization, and actual tool invocation happens in
/// the host.
#[async_trait::async_trait]
pub trait EffectExecutor: Send + Sync {
/// Execute a capability action.
///
/// The executor is responsible for:
/// 1. Looking up the actual tool implementation
/// 2. Validating parameters
/// 3. Applying safety checks (sanitization, leak detection)
/// 4. Executing the tool
/// 5. Returning the result
async fn execute_action(
&self,
action_name: &str,
parameters: serde_json::Value,
lease: &CapabilityLease,
context: &ThreadExecutionContext,
) -> Result<ActionResult, EngineError>;
/// List available actions given the current set of active leases.
///
/// Used to build the action definitions sent to the LLM.
async fn available_actions(
&self,
leases: &[CapabilityLease],
) -> Result<Vec<ActionDef>, EngineError>;
}
+56
View File
@@ -0,0 +1,56 @@
//! LLM backend trait.
//!
//! The engine's abstraction over language model providers. Deliberately
//! simpler than the main crate's `LlmProvider` — the engine only needs
//! to make completion calls. Cost tracking, caching, retry, and circuit
//! breaking are host concerns handled by the bridge adapter.
use std::collections::HashMap;
use crate::types::capability::ActionDef;
use crate::types::error::EngineError;
use crate::types::message::ThreadMessage;
use crate::types::step::{LlmResponse, TokenUsage};
/// Configuration for a single LLM call.
#[derive(Debug, Clone, Default)]
pub struct LlmCallConfig {
/// Maximum tokens to generate.
pub max_tokens: Option<u32>,
/// Sampling temperature.
pub temperature: Option<f32>,
/// When true, the LLM should not return action calls.
pub force_text: bool,
/// Depth in the recursive call tree (0 = root, 1+ = sub-call).
/// Implementations can use this to route to cheaper models for sub-calls.
pub depth: u32,
/// Opaque metadata forwarded to the LLM provider.
pub metadata: HashMap<String, String>,
}
/// Output from a single LLM call.
#[derive(Debug, Clone)]
pub struct LlmOutput {
pub response: LlmResponse,
pub usage: TokenUsage,
}
/// Abstraction over language model providers.
///
/// The main crate implements this by wrapping its `LlmProvider` trait,
/// converting between `ThreadMessage` and `ChatMessage`.
#[async_trait::async_trait]
pub trait LlmBackend: Send + Sync {
/// Call the LLM with conversation messages and available action definitions.
///
/// Returns either a text response or a set of action calls.
async fn complete(
&self,
messages: &[ThreadMessage],
actions: &[ActionDef],
config: &LlmCallConfig,
) -> Result<LlmOutput, EngineError>;
/// The model identifier (e.g. "gpt-4", "claude-opus-4-20250514").
fn model_name(&self) -> &str;
}
+8
View File
@@ -0,0 +1,8 @@
//! External dependency traits.
//!
//! The engine defines these traits; the host (main ironclaw crate)
//! implements them via bridge adapters over existing infrastructure.
pub mod effect;
pub mod llm;
pub mod store;
@@ -0,0 +1,97 @@
//! Storage trait for engine persistence.
//!
//! Defines CRUD operations for all engine types. The main crate implements
//! this by wrapping its dual-backend `Database` trait (PostgreSQL + libSQL).
use crate::types::capability::{CapabilityLease, LeaseId};
use crate::types::conversation::{ConversationId, ConversationSurface};
use crate::types::error::EngineError;
use crate::types::event::ThreadEvent;
use crate::types::memory::{DocId, MemoryDoc};
use crate::types::mission::{Mission, MissionId, MissionStatus};
use crate::types::project::{Project, ProjectId};
use crate::types::step::Step;
use crate::types::thread::{Thread, ThreadId, ThreadState};
/// Persistence abstraction for the engine.
#[async_trait::async_trait]
pub trait Store: Send + Sync {
// ── Thread operations ───────────────────────────────────
async fn save_thread(&self, thread: &Thread) -> Result<(), EngineError>;
async fn load_thread(&self, id: ThreadId) -> Result<Option<Thread>, EngineError>;
async fn list_threads(&self, project_id: ProjectId) -> Result<Vec<Thread>, EngineError>;
async fn update_thread_state(
&self,
id: ThreadId,
state: ThreadState,
) -> Result<(), EngineError>;
// ── Step operations ─────────────────────────────────────
async fn save_step(&self, step: &Step) -> Result<(), EngineError>;
async fn load_steps(&self, thread_id: ThreadId) -> Result<Vec<Step>, EngineError>;
// ── Event operations ────────────────────────────────────
async fn append_events(&self, events: &[ThreadEvent]) -> Result<(), EngineError>;
async fn load_events(&self, thread_id: ThreadId) -> Result<Vec<ThreadEvent>, EngineError>;
// ── Project operations ──────────────────────────────────
async fn save_project(&self, project: &Project) -> Result<(), EngineError>;
async fn load_project(&self, id: ProjectId) -> Result<Option<Project>, EngineError>;
async fn list_projects(&self) -> Result<Vec<Project>, EngineError> {
Ok(Vec::new())
}
// ── Conversation operations ─────────────────────────────
async fn save_conversation(
&self,
conversation: &ConversationSurface,
) -> Result<(), EngineError> {
let _ = conversation;
Ok(())
}
async fn load_conversation(
&self,
id: ConversationId,
) -> Result<Option<ConversationSurface>, EngineError> {
let _ = id;
Ok(None)
}
async fn list_conversations(
&self,
user_id: &str,
) -> Result<Vec<ConversationSurface>, EngineError> {
let _ = user_id;
Ok(Vec::new())
}
// ── Memory doc operations ───────────────────────────────
async fn save_memory_doc(&self, doc: &MemoryDoc) -> Result<(), EngineError>;
async fn load_memory_doc(&self, id: DocId) -> Result<Option<MemoryDoc>, EngineError>;
async fn list_memory_docs(&self, project_id: ProjectId) -> Result<Vec<MemoryDoc>, EngineError>;
// ── Capability lease operations ─────────────────────────
async fn save_lease(&self, lease: &CapabilityLease) -> Result<(), EngineError>;
async fn load_active_leases(
&self,
thread_id: ThreadId,
) -> Result<Vec<CapabilityLease>, EngineError>;
async fn revoke_lease(&self, lease_id: LeaseId, reason: &str) -> Result<(), EngineError>;
// ── Mission operations ───────────────────────────────────
async fn save_mission(&self, mission: &Mission) -> Result<(), EngineError>;
async fn load_mission(&self, id: MissionId) -> Result<Option<Mission>, EngineError>;
async fn list_missions(&self, project_id: ProjectId) -> Result<Vec<Mission>, EngineError>;
async fn update_mission_status(
&self,
id: MissionId,
status: MissionStatus,
) -> Result<(), EngineError>;
}
@@ -0,0 +1,254 @@
//! Capability — the unit of effect.
//!
//! A capability bundles actions (tools), knowledge (skills), and policies
//! (hooks) into a single installable/activatable unit. Capabilities are
//! granted to threads via leases.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::thread::ThreadId;
/// Strongly-typed lease identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct LeaseId(pub Uuid);
impl LeaseId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for LeaseId {
fn default() -> Self {
Self::new()
}
}
// ── Effect types ────────────────────────────────────────────
/// Classification of side effects that an action may produce.
/// Used by the policy engine for allow/deny decisions.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum EffectType {
/// Read from local filesystem or workspace.
ReadLocal,
/// Read from external APIs (no mutation).
ReadExternal,
/// Write to local filesystem or workspace.
WriteLocal,
/// Write to external services (create PR, send email).
WriteExternal,
/// Authenticated API call requiring credentials.
CredentialedNetwork,
/// Code execution or shell access.
Compute,
/// Financial operations (payments, transfers).
Financial,
}
// ── Action definition ───────────────────────────────────────
/// Definition of a single action within a capability.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ActionDef {
/// Action name (e.g. "create_issue", "web_fetch").
pub name: String,
/// Human-readable description.
pub description: String,
/// JSON Schema for parameters.
pub parameters_schema: serde_json::Value,
/// Effect types this action may produce.
pub effects: Vec<EffectType>,
/// Whether this action requires user approval before execution.
pub requires_approval: bool,
}
// ── Capability ──────────────────────────────────────────────
/// A capability — bundles actions, knowledge, and policies.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Capability {
/// Capability name (e.g. "github", "deployment").
pub name: String,
/// Human-readable description.
pub description: String,
/// Executable actions (replaces tools).
pub actions: Vec<ActionDef>,
/// Domain knowledge blocks (replaces skills).
pub knowledge: Vec<String>,
/// Policy rules (replaces hooks).
pub policies: Vec<PolicyRule>,
}
// ── Policy ──────────────────────────────────────────────────
/// A named policy rule within a capability.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PolicyRule {
pub name: String,
pub condition: PolicyCondition,
pub effect: PolicyEffect,
}
/// When a policy rule applies.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum PolicyCondition {
/// Always applies.
Always,
/// Applies when the action name matches the pattern.
ActionMatches { pattern: String },
/// Applies when the action has a specific effect type.
EffectTypeIs(EffectType),
}
/// What the policy engine decides.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum PolicyEffect {
Allow,
Deny,
RequireApproval,
}
// ── Capability lease ────────────────────────────────────────
/// A time/use-limited grant of capability access to a thread.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CapabilityLease {
pub id: LeaseId,
/// The thread this lease is granted to.
pub thread_id: ThreadId,
/// Which capability this lease covers.
pub capability_name: String,
/// Which actions from the capability are granted (empty = all).
pub granted_actions: Vec<String>,
/// When the lease was granted.
pub granted_at: DateTime<Utc>,
/// When the lease expires (None = no expiry).
pub expires_at: Option<DateTime<Utc>>,
/// Maximum number of action invocations (None = unlimited).
pub max_uses: Option<u32>,
/// Remaining invocations (None = unlimited).
pub uses_remaining: Option<u32>,
/// Whether the lease has been explicitly revoked.
pub revoked: bool,
}
impl CapabilityLease {
/// Check whether this lease is currently valid.
pub fn is_valid(&self) -> bool {
if self.revoked {
return false;
}
if let Some(expires_at) = self.expires_at
&& Utc::now() >= expires_at
{
return false;
}
if let Some(remaining) = self.uses_remaining
&& remaining == 0
{
return false;
}
true
}
/// Check whether a specific action is covered by this lease.
pub fn covers_action(&self, action_name: &str) -> bool {
self.granted_actions.is_empty() || self.granted_actions.iter().any(|a| a == action_name)
}
/// Consume one use of this lease. Returns false if no uses remain.
pub fn consume_use(&mut self) -> bool {
if let Some(ref mut remaining) = self.uses_remaining {
if *remaining == 0 {
return false;
}
*remaining -= 1;
}
true
}
}
#[cfg(test)]
mod tests {
use super::*;
fn make_lease() -> CapabilityLease {
CapabilityLease {
id: LeaseId::new(),
thread_id: ThreadId::new(),
capability_name: "test".into(),
granted_actions: vec![],
granted_at: Utc::now(),
expires_at: None,
max_uses: None,
uses_remaining: None,
revoked: false,
}
}
#[test]
fn valid_lease() {
let lease = make_lease();
assert!(lease.is_valid());
}
#[test]
fn revoked_lease_is_invalid() {
let mut lease = make_lease();
lease.revoked = true;
assert!(!lease.is_valid());
}
#[test]
fn expired_lease_is_invalid() {
let mut lease = make_lease();
lease.expires_at = Some(Utc::now() - chrono::Duration::seconds(10));
assert!(!lease.is_valid());
}
#[test]
fn exhausted_lease_is_invalid() {
let mut lease = make_lease();
lease.max_uses = Some(1);
lease.uses_remaining = Some(0);
assert!(!lease.is_valid());
}
#[test]
fn consume_use_decrements() {
let mut lease = make_lease();
lease.max_uses = Some(2);
lease.uses_remaining = Some(2);
assert!(lease.consume_use());
assert_eq!(lease.uses_remaining, Some(1));
assert!(lease.consume_use());
assert_eq!(lease.uses_remaining, Some(0));
assert!(!lease.consume_use());
}
#[test]
fn unlimited_consume_always_succeeds() {
let mut lease = make_lease();
for _ in 0..100 {
assert!(lease.consume_use());
}
}
#[test]
fn covers_action_empty_grants_all() {
let lease = make_lease();
assert!(lease.covers_action("anything"));
}
#[test]
fn covers_action_with_specific_grants() {
let mut lease = make_lease();
lease.granted_actions = vec!["create_issue".into(), "list_prs".into()];
assert!(lease.covers_action("create_issue"));
assert!(lease.covers_action("list_prs"));
assert!(!lease.covers_action("delete_repo"));
}
}
@@ -0,0 +1,263 @@
//! Conversation surface — the UI layer, separate from execution.
//!
//! A conversation is a stream of entries visible to the user. Threads
//! (the execution units) run independently and produce entries that
//! appear in conversations. One conversation can have multiple active
//! threads; one thread can outlive its originating conversation.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::thread::ThreadId;
/// Strongly-typed conversation identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct ConversationId(pub Uuid);
impl ConversationId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for ConversationId {
fn default() -> Self {
Self::new()
}
}
impl std::fmt::Display for ConversationId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}
/// Strongly-typed entry identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct EntryId(pub Uuid);
impl EntryId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for EntryId {
fn default() -> Self {
Self::new()
}
}
/// Who sent a conversation entry.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum EntrySender {
/// The human user.
User,
/// The agent (from a specific thread).
Agent { thread_id: ThreadId },
/// System notification (thread started, completed, etc.).
System,
}
/// A single entry in a conversation — a message visible to the user.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConversationEntry {
pub id: EntryId,
pub sender: EntrySender,
pub content: String,
/// Which thread produced this entry (if any).
pub origin_thread_id: Option<ThreadId>,
pub timestamp: DateTime<Utc>,
/// Optional metadata (channel-specific formatting, attachments, etc.).
pub metadata: serde_json::Value,
}
impl ConversationEntry {
/// Create a user entry.
pub fn user(content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::User,
content: content.into(),
origin_thread_id: None,
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create an agent entry from a thread.
pub fn agent(thread_id: ThreadId, content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::Agent { thread_id },
content: content.into(),
origin_thread_id: Some(thread_id),
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create a system notification entry.
pub fn system(content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::System,
content: content.into(),
origin_thread_id: None,
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
/// Create a system notification linked to a thread.
pub fn system_for_thread(thread_id: ThreadId, content: impl Into<String>) -> Self {
Self {
id: EntryId::new(),
sender: EntrySender::System,
content: content.into(),
origin_thread_id: Some(thread_id),
timestamp: Utc::now(),
metadata: serde_json::Value::Null,
}
}
}
/// A conversation surface — the UI-facing view of a chat.
///
/// Conversations are NOT execution boundaries. They are streams of entries
/// that may come from multiple concurrent threads. A user can start a new
/// thread while another is still running, and both produce entries in the
/// same conversation.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ConversationSurface {
pub id: ConversationId,
/// Which channel this conversation is on (e.g. "telegram", "web", "cli").
pub channel: String,
/// The user who owns this conversation.
pub user_id: String,
/// All entries in chronological order.
pub entries: Vec<ConversationEntry>,
/// Currently active (non-terminal) thread IDs.
pub active_threads: Vec<ThreadId>,
/// Metadata (channel-specific state, external thread IDs, etc.).
pub metadata: serde_json::Value,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
impl ConversationSurface {
pub fn new(channel: impl Into<String>, user_id: impl Into<String>) -> Self {
let now = Utc::now();
Self {
id: ConversationId::new(),
channel: channel.into(),
user_id: user_id.into(),
entries: Vec::new(),
active_threads: Vec::new(),
metadata: serde_json::Value::Null,
created_at: now,
updated_at: now,
}
}
/// Add an entry and update the timestamp.
pub fn add_entry(&mut self, entry: ConversationEntry) {
self.entries.push(entry);
self.updated_at = Utc::now();
}
/// Register a thread as active in this conversation.
pub fn track_thread(&mut self, thread_id: ThreadId) {
if !self.active_threads.contains(&thread_id) {
self.active_threads.push(thread_id);
}
}
/// Remove a thread from the active list (it completed or failed).
pub fn untrack_thread(&mut self, thread_id: ThreadId) {
self.active_threads.retain(|id| *id != thread_id);
}
/// Get the most recent entry, if any.
pub fn last_entry(&self) -> Option<&ConversationEntry> {
self.entries.last()
}
/// Get all entries from a specific thread.
pub fn entries_for_thread(&self, thread_id: ThreadId) -> Vec<&ConversationEntry> {
self.entries
.iter()
.filter(|e| e.origin_thread_id == Some(thread_id))
.collect()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn conversation_lifecycle() {
let mut conv = ConversationSurface::new("telegram", "user_123");
assert!(conv.entries.is_empty());
assert!(conv.active_threads.is_empty());
// User sends a message
conv.add_entry(ConversationEntry::user("Hello!"));
assert_eq!(conv.entries.len(), 1);
// Thread starts
let tid = ThreadId::new();
conv.track_thread(tid);
conv.add_entry(ConversationEntry::system_for_thread(tid, "Thread started"));
assert_eq!(conv.active_threads.len(), 1);
// Agent responds
conv.add_entry(ConversationEntry::agent(tid, "Hi there!"));
assert_eq!(conv.entries.len(), 3);
// Thread completes
conv.untrack_thread(tid);
conv.add_entry(ConversationEntry::system_for_thread(
tid,
"Thread completed",
));
assert!(conv.active_threads.is_empty());
assert_eq!(conv.entries.len(), 4);
}
#[test]
fn multiple_concurrent_threads() {
let mut conv = ConversationSurface::new("web", "user_456");
let t1 = ThreadId::new();
let t2 = ThreadId::new();
conv.track_thread(t1);
conv.track_thread(t2);
assert_eq!(conv.active_threads.len(), 2);
conv.add_entry(ConversationEntry::agent(t1, "Research result A"));
conv.add_entry(ConversationEntry::agent(t2, "Research result B"));
conv.add_entry(ConversationEntry::agent(t1, "More from A"));
let t1_entries = conv.entries_for_thread(t1);
assert_eq!(t1_entries.len(), 2);
let t2_entries = conv.entries_for_thread(t2);
assert_eq!(t2_entries.len(), 1);
conv.untrack_thread(t1);
assert_eq!(conv.active_threads.len(), 1);
}
#[test]
fn track_thread_is_idempotent() {
let mut conv = ConversationSurface::new("cli", "user");
let tid = ThreadId::new();
conv.track_thread(tid);
conv.track_thread(tid);
assert_eq!(conv.active_threads.len(), 1);
}
}
+124
View File
@@ -0,0 +1,124 @@
//! Engine error types.
use std::fmt;
use crate::types::capability::EffectType;
use crate::types::thread::{ThreadId, ThreadState};
/// Top-level engine error.
#[derive(Debug, thiserror::Error)]
pub enum EngineError {
#[error("thread error: {0}")]
Thread(#[from] ThreadError),
#[error("step error: {0}")]
Step(#[from] StepError),
#[error("capability error: {0}")]
Capability(#[from] CapabilityError),
#[error("store error: {reason}")]
Store { reason: String },
#[error("LLM error: {reason}")]
Llm { reason: String },
#[error("effect execution error: {reason}")]
Effect { reason: String },
#[error("invalid state transition: {from} -> {to}")]
InvalidTransition { from: ThreadState, to: ThreadState },
#[error("thread not found: {0}")]
ThreadNotFound(ThreadId),
#[error("project not found: {0}")]
ProjectNotFound(ProjectId),
#[error("lease expired for capability: {capability_name}")]
LeaseExpired { capability_name: String },
#[error("lease denied: {reason}")]
LeaseDenied { reason: String },
#[error("max iterations reached: {limit}")]
MaxIterations { limit: usize },
#[error("token limit exceeded: {used} of {limit}")]
TokenLimitExceeded { used: u64, limit: u64 },
#[error("consecutive error threshold exceeded: {count} errors (limit: {threshold})")]
ConsecutiveErrors { count: u32, threshold: u32 },
#[error("thread timeout: {elapsed:?} of {limit:?}")]
Timeout {
elapsed: std::time::Duration,
limit: std::time::Duration,
},
#[error("skill error: {reason}")]
Skill { reason: String },
#[error("authentication required for credential '{credential_name}'")]
NeedAuthentication {
credential_name: String,
action_name: String,
call_id: String,
parameters: serde_json::Value,
},
}
use crate::types::project::ProjectId;
/// Thread-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum ThreadError {
#[error("thread already running: {0}")]
AlreadyRunning(ThreadId),
#[error("thread is in terminal state: {0}")]
Terminal(ThreadState),
#[error("cannot spawn child: parent thread {0} is not running")]
ParentNotRunning(ThreadId),
}
/// Step-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum StepError {
#[error("step timed out after {0:?}")]
Timeout(std::time::Duration),
#[error("action not permitted by capability lease: {action}")]
ActionDenied { action: String },
}
/// Capability-specific errors.
#[derive(Debug, thiserror::Error)]
pub enum CapabilityError {
#[error("capability not found: {0}")]
NotFound(String),
#[error("effect type {effect:?} not permitted by policy")]
EffectDenied { effect: EffectType },
}
// Display impls for types used in error messages that don't already impl Display.
impl fmt::Display for ThreadId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
impl fmt::Display for ThreadState {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{self:?}")
}
}
impl fmt::Display for ProjectId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
}
}
+217
View File
@@ -0,0 +1,217 @@
//! Event sourcing types.
//!
//! Every significant action within a thread is recorded as an event.
//! This enables replay, debugging, reflection, and trace-based testing.
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use uuid::Uuid;
use crate::types::capability::LeaseId;
/// Generate a short human-readable summary of tool parameters for display.
///
/// For `http`: shows the URL. For `web_search`: shows the query.
/// For other tools: shows the first string argument, truncated.
/// Returns `None` for empty or unrecognizable params.
pub fn summarize_params(action_name: &str, params: &serde_json::Value) -> Option<String> {
let summary = match action_name {
"http" | "web_fetch" => params.get("url").and_then(|v| v.as_str()).map(|u| {
if u.len() > 80 {
format!("{}...", &u[..77])
} else {
u.to_string()
}
}),
"web_search" | "llm_context" => params
.get("query")
.and_then(|v| v.as_str())
.map(|q| truncate(q, 60)),
"memory_search" => params
.get("query")
.and_then(|v| v.as_str())
.map(|q| truncate(q, 60)),
"memory_write" => params
.get("target")
.and_then(|v| v.as_str())
.map(|t| t.to_string()),
"memory_read" => params
.get("path")
.and_then(|v| v.as_str())
.map(|p| p.to_string()),
"shell" => params
.get("command")
.and_then(|v| v.as_str())
.map(|c| truncate(c, 60)),
"message" => params
.get("content")
.and_then(|v| v.as_str())
.map(|c| truncate(c, 40)),
_ => {
// Generic: show first string value
if let Some(obj) = params.as_object() {
obj.values()
.find_map(|v| v.as_str())
.map(|s| truncate(s, 50))
} else {
None
}
}
};
summary.filter(|s| !s.is_empty())
}
fn truncate(s: &str, max: usize) -> String {
if s.len() <= max {
s.to_string()
} else {
// Find a safe UTF-8 boundary
let mut end = max.min(s.len());
while end > 0 && !s.is_char_boundary(end) {
end -= 1;
}
format!("{}...", &s[..end])
}
}
use crate::types::step::{StepId, TokenUsage};
use crate::types::thread::{ThreadId, ThreadState};
/// Strongly-typed event identifier.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct EventId(pub Uuid);
impl EventId {
pub fn new() -> Self {
Self(Uuid::new_v4())
}
}
impl Default for EventId {
fn default() -> Self {
Self::new()
}
}
/// A recorded event in a thread's execution history.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ThreadEvent {
pub id: EventId,
pub thread_id: ThreadId,
pub timestamp: DateTime<Utc>,
pub kind: EventKind,
}
impl ThreadEvent {
pub fn new(thread_id: ThreadId, kind: EventKind) -> Self {
Self {
id: EventId::new(),
thread_id,
timestamp: Utc::now(),
kind,
}
}
}
/// The specific kind of event that occurred.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum EventKind {
// ── Thread lifecycle ────────────────────────────────────
StateChanged {
from: ThreadState,
to: ThreadState,
reason: Option<String>,
},
// ── Step lifecycle ──────────────────────────────────────
StepStarted {
step_id: StepId,
},
StepCompleted {
step_id: StepId,
tokens: TokenUsage,
},
StepFailed {
step_id: StepId,
error: String,
},
// ── Action execution ────────────────────────────────────
ActionExecuted {
step_id: StepId,
action_name: String,
call_id: String,
duration_ms: u64,
/// Short human-readable summary of parameters (e.g., URL for http tool).
#[serde(default, skip_serializing_if = "Option::is_none")]
params_summary: Option<String>,
},
ActionFailed {
step_id: StepId,
action_name: String,
call_id: String,
error: String,
/// Short human-readable summary of parameters.
#[serde(default, skip_serializing_if = "Option::is_none")]
params_summary: Option<String>,
},
// ── Capability leases ───────────────────────────────────
LeaseGranted {
lease_id: LeaseId,
capability_name: String,
},
LeaseRevoked {
lease_id: LeaseId,
reason: String,
},
LeaseExpired {
lease_id: LeaseId,
},
// ── Messages ────────────────────────────────────────────
MessageAdded {
role: String,
content_preview: String,
},
// ── Thread tree ─────────────────────────────────────────
ChildSpawned {
child_id: ThreadId,
goal: String,
},
ChildCompleted {
child_id: ThreadId,
},
// ── Approval flow ───────────────────────────────────────
ApprovalRequested {
action_name: String,
call_id: String,
},
ApprovalReceived {
call_id: String,
approved: bool,
},
// ── Self-improvement ──────────────────────────────────────
SelfImprovementStarted,
SelfImprovementComplete {
prompt_updated: bool,
patterns_added: usize,
},
SelfImprovementFailed {
error: String,
},
// ── Skill activation ───────────────────────────────────────
SkillActivated {
skill_names: Vec<String>,
},
// ── Orchestrator versioning ───────────────────────────────
OrchestratorRollback {
from_version: u64,
to_version: u64,
reason: String,
},
}

Some files were not shown because too many files have changed in this diff Show More