diff --git a/docs/plans/2026-03-20-engine-v2-architecture.md b/docs/plans/2026-03-20-engine-v2-architecture.md index 9948ebe6..3de81684 100644 --- a/docs/plans/2026-03-20-engine-v2-architecture.md +++ b/docs/plans/2026-03-20-engine-v2-architecture.md @@ -1,7 +1,8 @@ # IronClaw Engine v2: Unified Thread-Capability-CodeAct Architecture **Date:** 2026-03-20 -**Status:** Draft +**Updated:** 2026-03-22 +**Status:** In Progress (Phases 1-3 complete) **Goal:** Replace IronClaw's ~10 fragmented abstractions with a unified execution model built on 5 primitives: Thread, Step, Capability, MemoryDoc, Project. Developed as a standalone crate (`ironclaw_engine`) that can be swapped in when it passes all acceptance tests. --- @@ -29,6 +30,15 @@ IronClaw currently has Session, Job, Routine, Channel, Tool, Skill, Hook, Observ 8. **Recursive subagent spawning** (RLM pattern) — code can call `llm_query()` to spawn child threads inline. Results are stored as variables, not injected into the parent's context window 9. **Event sourcing from day one** — every thread records a complete execution trace for replay/debugging/reflection +## Key Influences + +- **RLM paper** (arXiv:2512.24601, Zhang/Kraska/Khattab, MIT) — context as variable, FINAL() termination, recursive sub-calls, output truncation, compaction +- **Official RLM impl** (alexzhang13/rlm) — 30 max iterations, 20K char truncation, compaction at 85% context, scaffold restoration, FINAL_VAR regex fallback, consecutive error counting, budget/timeout/token limits with inheritance to child RLMs +- **fast-rlm** (avbiswas/fast-rlm) — Step 0 orientation preamble, parallel `asyncio.gather` sub-calls, dual model routing (stronger root, cheaper sub), dual system prompts (leaf vs non-leaf), 2K char truncation (aggressive but fast), fresh runtime per sub-agent +- **Prime Intellect** (verifiers/RLMEnv) — answer dictionary pattern (`{"content": "", "ready": True}`), tools restricted to sub-LLMs only, `llm_batch()` for parallel dispatch, 8K char truncation, FIFO-based sandbox communication, per-REPL-call 120s timeout +- **rlm-rs** (zircote/rlm-rs) — Rust CLI using pass-by-reference chunk IDs, tree-sitter code-aware chunking, hybrid BGE-M3+BM25 search with RRF, SQLite persistence +- **Google ADK RLM** — lazy Path objects (data stays on disk/GCS until code accesses it), massive parallelism with global concurrency limits + ## The Five Primitives | Primitive | Purpose | Replaces | @@ -46,6 +56,7 @@ Single crate: `crates/ironclaw_engine/` ``` crates/ironclaw_engine/ Cargo.toml + CLAUDE.md src/ lib.rs # Public API, re-exports @@ -53,352 +64,198 @@ crates/ironclaw_engine/ mod.rs error.rs # EngineError, ThreadError, StepError, CapabilityError thread.rs # Thread, ThreadId, ThreadState, ThreadType, ThreadConfig - step.rs # Step, StepId, StepStatus, ExecutionTier, ActionCall, ActionResult + step.rs # Step, StepId, StepStatus, ExecutionTier, ActionCall, ActionResult, LlmResponse capability.rs # Capability, ActionDef, EffectType, CapabilityLease, PolicyRule memory.rs # MemoryDoc, DocId, DocType project.rs # Project, ProjectId - event.rs # ThreadEvent, EventKind (event sourcing) + event.rs # ThreadEvent, EventKind (16 variants for event sourcing) provenance.rs # Provenance enum (User, System, ToolOutput, LlmGenerated, etc.) message.rs # ThreadMessage, MessageRole conversation.rs # ConversationSurface, ConversationEntry (Phase 5) mission.rs # Mission, MissionId (Phase 4) - traits/ # External dependency abstractions + traits/ # External dependency abstractions (host implements these) mod.rs llm.rs # LlmBackend trait - store.rs # Store trait (thread/step/event/project/doc/lease CRUD) + store.rs # Store trait (18 CRUD methods) effect.rs # EffectExecutor trait - code_runner.rs # CodeRunner trait (Phase 3) capability/ # Capability management mod.rs registry.rs # CapabilityRegistry lease.rs # LeaseManager (grant, check, consume, revoke, expire) - policy.rs # PolicyEngine (deterministic effect-level allow/deny) - provenance.rs # ProvenanceTracker (taint analysis at effect boundaries, Phase 4) + policy.rs # PolicyEngine (deterministic effect-level allow/deny/approve) + provenance.rs # ProvenanceTracker (taint analysis, Phase 4) runtime/ # Thread lifecycle management mod.rs - manager.rs # ThreadManager (spawn, supervise, stop, inject messages) + manager.rs # ThreadManager (spawn, supervise, stop, inject, join) tree.rs # ThreadTree (parent-child relationships) - messaging.rs # ThreadMailbox, ThreadSignal (inter-thread communication) - conversation.rs # ConversationManager (UI surface → thread routing, Phase 5) + messaging.rs # ThreadSignal, ThreadOutcome, signal channels + conversation.rs # ConversationManager (Phase 5) executor/ # Step execution mod.rs - loop_engine.rs # ExecutionLoop (core loop replacing run_agentic_loop) + loop_engine.rs # ExecutionLoop (core loop, handles Text/ActionCalls/Code) structured.rs # Tier 0: structured tool calls - scripting.rs # Tier 1: embedded Python via Monty (Phase 3) - context.rs # Context builder (thread state + project docs + capabilities) + scripting.rs # Tier 1: embedded Python via Monty (RLM pattern) + context.rs # Context builder (messages + actions from leases) intent.rs # Tool intent nudge detection memory/ # Memory document system mod.rs - store.rs # MemoryStore (project-scoped doc operations) - retrieval.rs # RetrievalEngine (context building from project docs) + store.rs # MemoryStore (project-scoped doc CRUD) + retrieval.rs # RetrievalEngine (stub, Phase 4) - reflection/ # Post-thread reflection pipeline + reflection/ # Post-thread reflection (stub, Phase 4) mod.rs - pipeline.rs # ReflectionPipeline (summarize, extract lessons, detect issues) - learning.rs # Tool reliability learning, playbook promotion - - testing/ # Test utilities (cfg(test)) - mod.rs - mock_llm.rs # MockLlmBackend (queued responses) - mock_store.rs # MockStore (in-memory HashMap storage) - mock_effect.rs # MockEffectExecutor (configurable results) ``` -Dependencies (minimal — no main crate dependency): +Dependencies: - `tokio` (sync, time, macros, rt), `serde` + `serde_json`, `thiserror`, `tracing`, `uuid`, `chrono`, `async-trait` -- `monty` (git dep) — embedded Python interpreter for CodeAct (Tier 1) +- `monty` (git dep from pydantic/monty) — embedded Python interpreter for CodeAct --- -## Phase 1: Foundation (Types + Traits + Thread Lifecycle) +## Phase 1: Foundation — DONE -**Goal:** Get the crate compiling with all core types, trait definitions, and thread state machine. No execution yet. +**Commit:** `8be19a4` -### 1.1 Crate scaffolding -- Create `crates/ironclaw_engine/Cargo.toml` (follow `ironclaw_safety` pattern) -- Add to workspace `members` in root `Cargo.toml` +All core types, trait definitions, and thread state machine. 32 tests. -### 1.2 Core types -All files in `src/types/`. Pure data structures with `Serialize`/`Deserialize`. - -**`error.rs`** — Error hierarchy: -- `EngineError` (top-level: Thread, Step, Capability, Store, Llm, Effect, InvalidTransition, NotFound, LeaseExpired, LeaseDenied, MaxIterations) -- `ThreadError` (AlreadyRunning, Terminal, ParentNotRunning) -- `StepError` (Timeout, ActionDenied) -- `CapabilityError` (NotFound, EffectDenied) - -**`thread.rs`** — Thread state machine: -``` -Created → Running → Waiting → Running (resume) - → Suspended → Running (resume) - → Completed → Reflecting → Done - → Failed -``` -- `ThreadState::can_transition_to(target) -> bool` -- `ThreadState::is_terminal() -> bool` (Done, Failed) -- `ThreadState::is_active() -> bool` (Running, Waiting) -- `Thread::transition_to(state) -> Result<(), EngineError>` (validates + records event) - -**`step.rs`** — Step + LLM response types: -- `LlmResponse::Text(String)` or `LlmResponse::ActionCalls { calls, content }` -- `ActionCall { id, action_name, parameters }` -- `ActionResult { call_id, action_name, output, is_error, duration }` -- `TokenUsage { input_tokens, output_tokens, cache_read_tokens, cache_write_tokens }` -- `ExecutionTier` enum — Phase 1: only `Structured` - -**`capability.rs`** — Effect typing + leases: -- `EffectType` enum: ReadLocal, ReadExternal, WriteLocal, WriteExternal, CredentialedNetwork, Compute, Financial -- `ActionDef { name, description, parameters_schema, effects: Vec, requires_approval }` -- `Capability { name, description, actions, knowledge: Vec, policies: Vec }` -- `CapabilityLease { id, thread_id, capability_name, granted_actions, granted_at, expires_at, max_uses, uses_remaining, revoked }` -- `PolicyRule { name, condition: PolicyCondition, effect: PolicyEffect }` -- `PolicyCondition` enum: Always, ActionMatches { pattern }, EffectTypeIs(EffectType) -- `PolicyEffect` enum: Allow, Deny, RequireApproval - -**`message.rs`** — Engine's own message type (simpler than `ChatMessage`): -- `MessageRole` enum: System, User, Assistant, ActionResult -- `ThreadMessage { role, content, provenance, action_call_id, action_name, action_calls, timestamp }` -- Constructors: `system()`, `user()`, `assistant()`, `assistant_with_actions()`, `action_result()` - -**`event.rs`** — Event sourcing: -- `EventKind` enum: StateChanged, StepStarted, StepCompleted, StepFailed, ActionExecuted, ActionFailed, LeaseGranted, LeaseRevoked, LeaseExpired, MessageAdded, ChildSpawned, ChildCompleted, ApprovalRequested, ApprovalReceived - -**`project.rs`**, **`memory.rs`**, **`provenance.rs`** — Straightforward structs. - -### 1.3 Trait definitions -- `LlmBackend` — `complete(messages, actions, config) -> LlmOutput`, `model_name() -> &str` -- `Store` — Thread/Step/Event/Project/MemoryDoc/Lease CRUD (~20 methods) -- `EffectExecutor` — `execute_action(name, params, lease, ctx) -> ActionResult`, `available_actions(leases) -> Vec` - -### 1.4 Tests -- Thread state machine: all valid/invalid transitions -- ThreadMessage constructors -- CapabilityLease expiry checks (time-based, use-based) - -### Verification -```bash -cargo check -p ironclaw_engine -cargo clippy -p ironclaw_engine --all-targets -- -D warnings -cargo test -p ironclaw_engine -``` +- Types: Thread (state machine), Step (LlmResponse, ActionCall, ActionResult, TokenUsage), Capability (ActionDef, EffectType, CapabilityLease, PolicyRule), MemoryDoc (DocType), Project, ThreadEvent (EventKind), ThreadMessage, Provenance, EngineError +- Traits: LlmBackend, Store (18 methods), EffectExecutor +- Tests: state machine transitions (valid/invalid), lease expiry (time/use), message constructors --- -## Phase 2: Execution Engine (Tier 0 — Structured Tool Calls) +## Phase 2: Execution Engine (Tier 0) — DONE -**Goal:** A working execution loop that is functionally equivalent to the current `run_agentic_loop()`. Thread spawning, capability leasing, policy enforcement, event logging. +**Commit:** `bf7dfb8` -### 2.1 Capability management -- `CapabilityRegistry` — register/get/list capabilities and their actions -- `LeaseManager` — grant leases (scoped, time-limited, use-limited), check validity, consume uses, revoke, expire stale. State: `RwLock>` -- `PolicyEngine` — deterministic evaluation: `evaluate(action_def, lease, thread_context) -> PolicyDecision`. Check order: global policies → capability policies → action-level `requires_approval` → effect type against lease. Deny > RequireApproval > Allow +Working execution loop equivalent to `run_agentic_loop()`. 74 tests. -### 2.2 Thread runtime -- `ThreadTree` — in-memory parent-child tracking. `add_child()`, `parent_of()`, `children_of()`, `remove()`, `ancestors()` -- `ThreadMailbox` + `ThreadSignal` — `mpsc`-based inter-thread messaging. Signals: Stop, Suspend, Resume, InjectMessage, ChildCompleted -- `ThreadManager` — orchestrator. `spawn_thread()` creates thread + leases + `ExecutionLoop`, wraps in tokio task. `stop_thread()`, `inject_message()`, `get_thread_state()`. Holds `Arc` + capability/lease/policy - -### 2.3 Execution loop -- `build_step_context()` — assemble messages + action definitions from thread state + active leases -- `execute_action_calls()` — Tier 0: for each ActionCall, find lease → check policy → consume use → call `EffectExecutor` → record result + event. Returns NeedApproval if policy requires it -- `ExecutionLoop::run()` — core loop mirroring `run_agentic_loop()`: - 1. `signal_rx.try_recv()` → handle Stop, Suspend, InjectMessage - 2. `build_step_context()` → messages + actions - 3. `llm.complete()` → LlmOutput - 4. If `LlmResponse::Text` → check tool intent nudge, return if final - 5. If `LlmResponse::ActionCalls` → `execute_action_calls()`, add results to messages - 6. Record Step, emit events - 7. Check max_iterations, force_text on final iterations - 8. Repeat - -### 2.4 Memory stubs -- `MemoryStore` — thin wrapper: `create_doc()`, `get_doc()`, `update_doc()`, `list_by_type()` -- `RetrievalEngine` — stub returning empty vec - -### 2.5 Tests (comprehensive) -- Simple text response: MockLlm returns text → thread Created→Running→Completed→Done -- Tool call then text: MockLlm returns ActionCalls then text → effect executor called, result in messages -- Multi-tool parallel: Multiple ActionCalls in one response → all executed, all results recorded -- Max iterations: MockLlm always returns actions → loop stops at limit -- Stop signal: Send Stop → clean termination -- Inject message: Send InjectMessage during loop → appears in context -- Lease expiry (uses): max_uses=1 → first OK, second fails -- Lease expiry (time): expires_at in past → immediate failure -- Policy deny: Financial effect blocked → ActionDenied -- Policy require approval: → returns NeedApproval outcome -- Event sourcing: run loop → verify all events recorded in order -- Tool intent nudge: "Let me search..." text → nudge injected, capped at max -- Child thread spawning: spawn from parent → tree relationships correct, child completion event on parent - -### Verification -```bash -cargo test -p ironclaw_engine -# All existing tests still pass: -cargo test -``` +- **CapabilityRegistry** — register/get/list capabilities and actions (5 tests) +- **LeaseManager** — grant, check, consume, revoke, expire. `RwLock` (7 tests) +- **PolicyEngine** — deterministic: global policies → capability policies → action requires_approval → effect type. Deny > RequireApproval > Allow (8 tests) +- **ThreadTree** — parent-child relationships (5 tests) +- **ThreadSignal/ThreadOutcome** — mpsc-based inter-thread messaging +- **ThreadManager** — spawn as tokio tasks, stop, inject messages, join (3 tests) +- **ExecutionLoop** — signals → context → LLM call → handle Text/ActionCalls → record step + events → repeat (6 tests) +- **execute_action_calls()** — lease lookup → policy → consume → EffectExecutor +- **signals_tool_intent()** — nudge detection (6 tests) +- **MemoryStore** + **RetrievalEngine** — stubs for Phase 4 --- -## Phase 3: CodeAct Executor (Tier 1 — Monty Python + RLM Pattern) +## Phase 3: CodeAct Executor (Tier 1 — Monty + RLM) — DONE -**Goal:** LLMs write Python code that composes tools, uses control flow, queries thread context as data, and recursively spawns sub-agents. Uses the Monty interpreter (Pydantic) for sandboxed in-process execution. Follows the Recursive Language Model (RLM) pattern: context as a variable, not attention input. +**Commits:** `b59a0b9`, `9538332` -**Status:** Implemented (Phases 3.1–3.3). Phases 3.4–3.5 are the RLM enhancements. +LLMs write Python code that composes tools, queries thread context as data, and recursively spawns sub-agents. Uses Monty interpreter with the RLM (Recursive Language Model) pattern. -### 3.1 Monty integration (DONE) +### What was built -`executor/scripting.rs` — Embeds the Monty Python interpreter (git dep, v0.0.8). +**Monty integration** (`executor/scripting.rs`): +- Embeds Pydantic's Monty Python interpreter (git dep, v0.0.8) +- `MontyRun::new(code, "step.py", input_names)` → `runner.start(inputs, tracker, print)` → loop over `RunProgress` suspension points +- Resource limits: 30s timeout, 64MB memory, 1M allocations, recursion depth 1000 +- All execution wrapped in `catch_unwind` (Monty can panic) +- `monty_to_json()` / `json_to_monty()` bidirectional conversion -**Execution model:** -1. `MontyRun::new(code, "step.py", input_names)` — parse Python code -2. `runner.start(inputs, tracker, print_writer)` — begin execution with resource limits -3. Loop over `RunProgress` suspension points: - - `FunctionCall` → find lease → check policy → call `EffectExecutor` → resume with result - - `NameLookup` → resolve or raise `NameError` - - `OsCall` → deny with `OSError` - - `ResolveFutures` → error (async not supported) - - `Complete` → return value + captured stdout -4. All execution wrapped in `catch_unwind` (Monty 0.0.x can panic) +**RLM features** (cross-referenced against official RLM, fast-rlm, Prime Intellect): -**Resource limits:** 30s timeout, 64MB memory, 1M allocations, recursion depth 1000. +| Feature | Implementation | Reference | +|---|---|---| +| Context as variables | `context`, `goal`, `step_number`, `previous_results` injected as Monty inputs | RLM paper §3 | +| `FINAL(answer)` | FunctionCall handler sets `final_answer`, loop exits | Official RLM, fast-rlm | +| `FINAL_VAR(name)` | FunctionCall handler stores var name reference | Official RLM | +| `llm_query(prompt, context)` | FunctionCall → single-shot `LlmBackend::complete()` with force_text | All three impls | +| `llm_query_batched(prompts)` | FunctionCall → parallel `tokio::spawn` for each prompt, collect results | fast-rlm asyncio.gather, Prime Intellect llm_batch | +| Output truncation (8K chars) | `compact_output_metadata()` with `[TRUNCATED: last N chars]` or `[FULL OUTPUT]` prefix | Prime Intellect 8192, Official 20K, fast-rlm 2K | +| Step 0 orientation | Auto-inject context metadata (msg count, total chars, goal, preview) before first code step | fast-rlm Step 0 auto-print | +| Error-to-LLM flow | Parse/runtime/name/OS errors return as stdout content, not EngineError. LLM can self-correct. | Official RLM (errors in stderr shown to LLM) | +| Tool dispatch | Unknown functions suspend VM → lease → policy → EffectExecutor → resume | Original design | +| OS call denial | `RunProgress::OsCall` → `OSError` exception | Original design | +| Async denial | `RunProgress::ResolveFutures` → error in stdout | Original design | -**Tool dispatch:** Unknown function calls in Python suspend the VM via `RunProgress::FunctionCall`. The engine routes through the same lease → policy → `EffectExecutor` pipeline as structured tool calls: -```python -result = web_fetch(url="https://example.com") # suspends → EffectExecutor -data = memory_search(query="deployment") # suspends → EffectExecutor -for item in result["items"]: # control flow in Python - memory_write(key=item["id"], value=item["summary"]) -``` +**LlmResponse::Code** variant + **ExecutionTier::Scripting** — the `ExecutionLoop` routes `Code` to `scripting::execute_code()`. -**Type conversion:** `monty_to_json()` / `json_to_monty()` bidirectional conversion between `MontyObject` and `serde_json::Value`. +### Remaining gaps (future phases) -### 3.2 LlmResponse::Code variant (DONE) - -New `LlmResponse::Code { code, content }` variant alongside `Text` and `ActionCalls`. The `ExecutionLoop` routes `Code` responses to `scripting::execute_code()` instead of `structured::execute_action_calls()`. - -### 3.3 ExecutionLoop integration (DONE) - -The loop handles `LlmResponse::Code`: -- Records assistant message with code -- Sets `step.tier = ExecutionTier::Scripting` -- Executes via `scripting::execute_code()` -- Records events and action results -- Captures stdout + return value as context for next iteration -- Handles `NeedApproval` outcome (pauses thread) - -### 3.4 RLM: Context as variables (TO IMPLEMENT) - -Inspired by Recursive Language Models (arXiv:2512.24601). The key insight: **the prompt is an environment variable, not attention input.** The LLM never sees the full thread context in its window — it writes code to access it selectively. - -**Implementation:** -- Pass thread state as Monty input variables via `MontyRun::new(code, "step.py", input_names)`: - - `context` — full thread message history as a Python list of dicts - - `goal` — the thread's goal string - - `step_number` — current step index - - `previous_results` — dict of `{call_id: result}` from prior steps -- Use compact output metadata between code steps: `"[code output: 4,532 chars]"` instead of full stdout in chat history -- The LLM's chat context stays lean; the full data lives in REPL variables - -**Before (current):** Full context in LLM attention window -``` -System: You are an agent... -User: Analyze these 1000 items... -[1000 items in context] -Assistant: ```python result = web_fetch(...)``` -``` - -**After (RLM pattern):** Context as a variable -``` -System: You have access to `context` (1000 items) and `previous_results`. -Write Python to accomplish the goal. -Assistant: ```python -items = context # never loaded into LLM window -for batch in [items[i:i+100] for i in range(0, len(items), 100)]: - result = llm_query("summarize these items", batch) - # result is a variable, not injected into parent context -``` - -### 3.5 RLM: Recursive `llm_query()` within code (TO IMPLEMENT) - -Expose `llm_query(prompt, context)` as a callable inside the Monty environment. When code calls it, Monty suspends via `FunctionCall`. The engine: -1. Spawns a child thread with the given prompt and context -2. Runs the child to completion (inline, blocking the parent's code) -3. Returns the child's result as a `MontyObject` - -This enables the core RLM patterns: -```python -# Partition + Map + Reduce -chunks = [context[i:i+1000] for i in range(0, len(context), 1000)] -summaries = [] -for chunk in chunks: - summary = llm_query("Summarize this section", chunk) - summaries.append(summary) # variable, not in parent's LLM context -final = llm_query("Combine these summaries", summaries) - -# Verification -answer = llm_query("What is X?", context) -verified = llm_query(f"Is this answer correct: {answer}", context) -``` - -**Key RLM properties preserved:** -- **Symbolic handle to context** — the parent LLM never sees child outputs in its attention window -- **Unbounded output** — variables in the REPL can exceed the context window -- **Recursive decomposition** — the model decides how to partition work, not the architect - -### 3.6 Tests -- **Simple code execution:** `x = 1 + 2` → returns 3, no tool calls -- **Tool call from code:** `result = web_fetch(url="...")` → `FunctionCall` suspension → effect executor called → result returned to Python -- **Multiple tool calls in loop:** `for i in range(3): fetch(url=urls[i])` → 3 effect executor calls -- **Context as variable:** Code accesses `context[0]` → correct value from thread messages -- **Compact metadata:** After code step, context has metadata summary not full stdout -- **`llm_query()` recursive call:** Code calls `llm_query("summarize", data)` → child thread spawned → result returned as variable -- **Resource limits:** Infinite loop → Monty `TimeoutError` -- **OS call denied:** `import os; os.listdir(".")` → `OSError` -- **VM panic recovery:** Monty panics → `catch_unwind` returns `EngineError`, thread doesn't crash -- **Policy deny in code:** Code calls denied action → Python `RuntimeError` raised -- **Approval needed in code:** Code calls approval-required action → `NeedApproval` returned, code halted +| Gap | Where it fits | Source | +|---|---|---| +| `rlm_query()` (child gets own REPL + full RLM loop) | Phase 4 — needs ThreadManager in CodeAct runtime | Official RLM | +| Dual model routing (cheaper model for sub-calls) | Phase 4 — LlmBackend needs `complete_with_model(model, ...)` | fast-rlm, Official RLM | +| Compaction at 85% context limit | Phase 4 — summarize history, reset messages | Official RLM | +| Persistent REPL state across code steps | Monty limitation (fresh MontyRun per step) — monitor Monty roadmap | Official RLM LocalREPL | +| Scaffold restoration (prevent code overwriting context/llm_query) | Not needed — Monty creates fresh execution per step | Official RLM | +| `SHOW_VARS()` listing | Monty limitation — no namespace access from host | Official RLM | +| Consecutive error counting + threshold | Phase 4 — add `max_consecutive_errors` to ThreadConfig | Official RLM | +| USD budget tracking | Phase 4 — needs cost data from LlmBackend | Official RLM, fast-rlm | +| answer dictionary pattern (`{"content":"","ready":True}`) | Alternative to FINAL() — lower priority, FINAL() works | Prime Intellect | +| Tools restricted to sub-LLMs only | Design decision for Phase 4 — evaluate tradeoffs | Prime Intellect | +| Lazy Path objects (data on disk until accessed) | Phase 4 retrieval — avoid loading full context upfront | Google ADK | +| Pass-by-reference chunk IDs for sub-agents | Phase 4 retrieval — sub-agents get IDs not content | rlm-rs | +| Code-aware chunking (tree-sitter) | Phase 4 retrieval — for code repositories | rlm-rs | --- ## Phase 4: Memory, Reflection, and Learning -**Goal:** The agent learns from its work. Completed threads produce structured knowledge (summaries, lessons, playbooks) that improve future threads. +**Goal:** The agent learns from its work. Completed threads produce structured knowledge. Context building uses project-scoped retrieval, not raw history replay. ### 4.1 Project-scoped retrieval - `RetrievalEngine::retrieve_context(project_id, query, max_docs)` — keyword + semantic search over project's memory docs -- Context builder uses retrieval: thread state + project docs (summaries, lessons, playbooks) + capability descriptions -- The LLM gets relevant project knowledge, not raw history +- Context builder: thread state + project docs (summaries, lessons, playbooks) + capability descriptions +- **Lazy loading** (Google ADK pattern): data stays in storage until code explicitly accesses it via variables +- **Pass-by-reference** (rlm-rs pattern): sub-agents receive chunk IDs, fetch content on demand ### 4.2 Reflection pipeline After thread completes (state → Completed), optionally spawns a Reflection-type thread: -1. **Summarize** → produce `DocType::Summary` doc -2. **Extract lessons** → scan for failures, workarounds, discoveries → produce `DocType::Lesson` docs -3. **Detect issues** → find problems that weren't resolved → produce `DocType::Issue` docs -4. **Detect missing capabilities** → "no tool available" patterns → produce `DocType::Spec` docs -5. **Promote playbooks** → successful multi-step procedures → produce `DocType::Playbook` docs +1. **Summarize** → `DocType::Summary` +2. **Extract lessons** → `DocType::Lesson` (from failures, workarounds, discoveries) +3. **Detect issues** → `DocType::Issue` (unresolved problems) +4. **Detect missing capabilities** → `DocType::Spec` ("no tool available" patterns) +5. **Promote playbooks** → `DocType::Playbook` (successful multi-step procedures) Reflection is itself a thread running CodeAct — it's recursive. -### 4.3 Provenance tracking -Every data value tagged with origin: -- `Provenance::User` — direct user input -- `Provenance::System` — system prompt, config -- `Provenance::ToolOutput { action_name }` — result from a capability action -- `Provenance::LlmGenerated` — LLM output -- `Provenance::Reflection { source_thread_id }` — from reflection pipeline -- `Provenance::MemoryRetrieval { doc_id }` — from project memory +### 4.3 Compaction (from RLM) +When message history tokens reach **85% of model context limit** (per official RLM): +1. Ask LLM to "summarize progress so far" with instructions to preserve intermediate results +2. Replace message history with `[system, summary, "continue..."]` +3. Append full trajectory to a `history` variable accessible from code +- Requires token counting — add `count_tokens(messages, model)` utility (tiktoken or char-estimate fallback, per official RLM `token_utils.py`) -The policy engine uses provenance at effect boundaries: -- LlmGenerated data cannot flow into Financial effects without approval -- ToolOutput from untrusted sources triggers extra validation -- User-provenance data is trusted (no taint) +### 4.4 `rlm_query()` — full recursive sub-agent +Unlike `llm_query()` (single-shot text completion), `rlm_query(prompt)` spawns a **child thread with its own CodeAct executor**: +- Child gets own REPL, own context variable, own iteration budget +- Child can call `llm_query()` and tools but NOT `rlm_query()` (depth limit) +- Budget/timeout inheritance: child gets `remaining_budget - spent`, `remaining_timeout - elapsed` +- Returns child's `FINAL()` answer as a string variable -### 4.4 Missions (long-running goals) +### 4.5 Dual model routing +`LlmBackend` gains optional depth-based model selection: +- depth=0 (root): use primary model (e.g., GPT-5, Claude Opus) +- depth=1+ (sub-calls): use cheaper model (e.g., GPT-5-mini, Claude Haiku) +- Configurable via `ThreadConfig` or `LlmCallConfig` + +### 4.6 Budget controls (from RLM cross-reference) +Add to `ThreadConfig`: +- `max_budget_usd: Option` — cumulative USD cost limit (needs cost data from LlmBackend) +- `max_timeout: Option` — wall-clock timeout for entire thread +- `max_tokens_total: Option` — cumulative input+output token limit +- `max_consecutive_errors: Option` — consecutive steps with errors before termination +- All limits inherited by child threads with remaining budget + +### 4.7 Provenance tracking +Every data value tagged with origin. Policy engine uses provenance at effect boundaries: +- LlmGenerated → Financial effects: require approval +- ToolOutput from untrusted sources: extra validation +- User-provenance: trusted + +### 4.8 Missions (long-running goals) ```rust pub struct Mission { pub id: MissionId, @@ -406,27 +263,24 @@ pub struct Mission { pub goal: String, pub status: MissionStatus, // Active, Paused, Completed, Failed pub cadence: MissionCadence, // Cron, OnEvent, OnPush, Manual - pub thread_history: Vec, // past threads spawned by this mission + pub thread_history: Vec, pub success_criteria: Option, } ``` -Missions spawn threads on cadence, track progress across runs, and adapt based on reflection docs. -### 4.5 Tool reliability learning -Track per-action metrics: -- Success rate (EMA) -- Avg latency -- Common failure patterns -- Last N results +### 4.9 Tool reliability learning +Track per-action EMA metrics (success rate, latency, failure patterns). Feed into context builder. -Feed into context builder so the LLM knows "this tool has been flaky recently." - -### 4.6 Tests -- Reflection produces correct doc types for a completed thread with failures +### 4.10 Tests +- Reflection produces correct doc types from a completed thread with failures - Retrieval returns project-scoped docs, not cross-project +- Compaction triggers at 85% context, preserves intermediate results +- `rlm_query()` spawns child thread, returns answer, respects budget inheritance +- Dual model routing: root uses primary, sub-calls use cheaper +- Budget exceeded → `BudgetExceededError` with partial answer +- Consecutive errors threshold → termination - Provenance taint blocks financial effects from LLM-generated data - Mission spawns thread on cadence, tracks history -- Tool reliability metrics update correctly after successes/failures --- @@ -477,33 +331,32 @@ The existing `Channel` trait stays. A bridge adapter translates: **Goal:** Full CodeAct with WASM sandbox (Tier 2) and Docker container (Tier 3). Two-phase commit for high-stakes effects. ### 6.1 Tier 2: WASM sandbox -- Embed Python interpreter (RustPython) or use Starlark compiled to WASM - Leverage existing `wasmtime` infrastructure from `src/tools/wasm/` - Fuel metering, memory limits, network allowlisting (all existing) - Runtime API exposed via WIT interface (extend existing `wit/tool.wit`) +- Candidate runtimes: RustPython compiled to WASM, or Monty if it gains WASM support ### 6.2 Tier 3: Docker container - Leverage existing `src/sandbox/` + `src/orchestrator/` infrastructure -- Full Python runtime with `thread.*` and `tools.*` available via HTTP proxy +- Full CPython with pip packages, shell access, filesystem +- `llm_query()` / `llm_query_batched()` / tool calls available via HTTP proxy to orchestrator - Network access through existing sandbox proxy (domain allowlist, credential injection) +- This is the production path for complex CodeAct (data science, system admin, etc.) ### 6.3 Automatic tier selection -Analyze LLM-generated code: -- Pure `tools.*` calls, no I/O → Tier 1 (embedded) -- Uses `tools.web_fetch` or HTTP → Tier 2 (WASM, allowlisted network) -- Uses `tools.shell`, `import os`, filesystem → Tier 3 (Docker) -- Falls back gracefully: if Tier 1 fails with capability error, promote to Tier 2/3 +Analyze LLM-generated code statically or try-and-promote: +- Pure function calls, no I/O → Tier 1 (Monty, in-process) +- Uses HTTP/network tools → Tier 2 (WASM, allowlisted network) +- Uses `import os`, shell, filesystem, pip packages → Tier 3 (Docker) +- Falls back gracefully: if Tier 1 fails with capability error, promote to Tier 3 ### 6.4 Two-phase commit For `WriteExternal` + `Financial` effects: -1. **Simulate** — dry-run the effect, return preview +1. **Simulate** — dry-run, return preview 2. **Approve** — user or policy approves 3. **Execute** — actual effect -Replaces current binary approve/deny with richer commit policies: -- `CommitPolicy::Direct` — execute immediately (ReadLocal, ReadExternal) -- `CommitPolicy::Approved` — needs approval before execution (WriteExternal) -- `CommitPolicy::TwoPhase` — simulate → approve → execute (Financial, production deploys) +Commit policies: `Direct` (ReadLocal, ReadExternal), `Approved` (WriteExternal), `TwoPhase` (Financial, production deploys). ### 6.5 Tests - Tier selection routes correctly based on code analysis @@ -516,55 +369,34 @@ Replaces current binary approve/deny with richer commit policies: ## Phase 7: Main Crate Integration -**Goal:** Bridge adapters connect the engine to existing IronClaw infrastructure. Feature-flagged swap. +**Goal:** Bridge adapters connect the engine to existing IronClaw infrastructure. ### 7.1 Bridge adapters (`src/bridge/`) -- `LlmBridgeAdapter` — wraps `Arc`, converts `ThreadMessage` ↔ `ChatMessage`, `ActionDef` ↔ `ToolDefinition` +- `LlmBridgeAdapter` — wraps `Arc`, converts `ThreadMessage` ↔ `ChatMessage`, `ActionDef` ↔ `ToolDefinition`. Implements depth-based model routing via existing `cheap_llm` in `AgentDeps` - `StoreBridgeAdapter` — wraps `Arc`, maps engine CRUD to existing sub-traits. New tables for threads/projects/docs/leases/events (migration V14+) -- `EffectBridgeAdapter` — wraps `ToolRegistry` + `SafetyLayer`. On `execute_action()`: lookup tool → validate params via safety → execute → sanitize output → return. This is where safety logic lives (not in the engine) +- `EffectBridgeAdapter` — wraps `ToolRegistry` + `SafetyLayer`. On `execute_action()`: lookup tool → validate params → execute → sanitize output → return. Safety logic lives here, not in the engine ### 7.2 Database migrations -New tables: -- `engine_threads` (id, goal, type, state, project_id, parent_id, config_json, metadata, timestamps) -- `engine_steps` (id, thread_id, sequence, status, tier, request_json, response_json, results_json, tokens_json, timestamps) -- `engine_events` (id, thread_id, timestamp, kind_json) -- `engine_projects` (id, name, description, metadata, timestamps) -- `engine_memory_docs` (id, project_id, doc_type, title, content, source_thread_id, tags_json, metadata, timestamps) -- `engine_capability_leases` (id, thread_id, capability_name, granted_actions_json, granted_at, expires_at, max_uses, uses_remaining, revoked) +New tables (both PostgreSQL and libSQL): +- `engine_threads`, `engine_steps`, `engine_events`, `engine_projects`, `engine_memory_docs`, `engine_capability_leases` -Both PostgreSQL and libSQL backends (per existing dual-backend requirement). +### 7.3 Integration strategy +Implement `EngineV2Delegate` that wraps `ExecutionLoop` but presents the `LoopDelegate` interface. The existing dispatcher calls `run_agentic_loop()` with either ChatDelegate or EngineV2Delegate. This enables gradual migration without a flag day. -### 7.3 Feature-flagged swap -```rust -// In app.rs or agent_loop.rs: -#[cfg(feature = "engine_v2")] -{ - let engine = ironclaw_engine::ThreadManager::new( - Arc::new(LlmBridgeAdapter::new(llm_provider)), - Arc::new(StoreBridgeAdapter::new(database)), - Arc::new(EffectBridgeAdapter::new(tool_registry, safety)), - ); - // Use engine for thread management -} -``` - -### 7.4 Alternative LoopDelegate -Implement a `EngineV2Delegate` that wraps the engine's `ExecutionLoop` but presents the `LoopDelegate` interface. This enables gradual migration — the existing dispatcher calls `run_agentic_loop()` with either the old ChatDelegate or the new EngineV2Delegate. - -### 7.5 Acceptance testing -Use existing `TestRig` + `TraceLlm` infrastructure: +### 7.4 Acceptance testing +Use existing `TestRig` + `TraceLlm`: - Load pre-recorded LLM trace fixtures -- Drive the engine via bridge adapters +- Drive engine via bridge adapters - Compare output with `verify_trace_expects()` -- All existing fixture tests must pass with identical results +- All existing fixture tests must pass -When all tests pass: remove feature flag, make engine the default, deprecate old path. +When all tests pass: make engine the default, deprecate old path. -### 7.6 Tests -- Bridge adapter conversion: ThreadMessage ↔ ChatMessage round-trips correctly +### 7.5 Tests +- Bridge adapter conversion: ThreadMessage ↔ ChatMessage round-trips - End-to-end: TestRig drives engine, same output as old loop -- Migration: new tables created for both PostgreSQL and libSQL -- Feature flag: both paths compile and pass tests +- Migration: new tables for both backends +- Both paths compile and pass --- @@ -582,55 +414,65 @@ When all tests pass: remove feature flag, make engine the default, deprecate old ### 8.2 Slim down main crate - Agent module becomes thin adapter over engine -- `app.rs` orchestrates engine startup instead of manually wiring channels/tools/sessions +- `app.rs` orchestrates engine startup - Remove `LoopDelegate` and its three implementations - Remove `SessionManager`, `Scheduler` (replaced by `ThreadManager`) ### 8.3 Sub-crate extraction -Once engine boundaries are stable, split internal modules into sub-crates if beneficial: -- `ironclaw_types` — shared types usable by WASM extensions +Once boundaries stabilize, split if beneficial: +- `ironclaw_types` — shared types for WASM extensions - `ironclaw_capability` — if used by tooling/CLI independently -- `ironclaw_codeact` — if the code runner grows complex +- `ironclaw_codeact` — if code runner grows complex --- ## Cross-Cutting Concerns ### Security Model -- **Capability leases** replace static permissions. Scoped per-thread, time-limited, use-limited. Blast radius bounded by lease -- **Effect typing** on every action. Policy engine uses effect types (not tool names) for allow/deny -- **Provenance tracking** (Phase 4). Data tagged with origin; taint analysis at effect boundaries +- **Capability leases** replace static permissions. Scoped, time-limited, use-limited. Blast radius bounded +- **Effect typing** on every action. Policy engine uses effect types for allow/deny +- **Provenance tracking** (Phase 4). Taint analysis at effect boundaries - **Two-phase commit** (Phase 6) for WriteExternal + Financial effects -- **Safety at adapter boundary**. The engine is pure orchestration; `SafetyLayer` (sanitization, leak detection, injection checking) is applied in `EffectBridgeAdapter` +- **Safety at adapter boundary**. Engine is pure orchestration; `SafetyLayer` applied in `EffectBridgeAdapter` +- **Monty sandboxing**: no filesystem (OsCall denied), no network (no imports), resource-limited, catch_unwind for panics ### Observability -- **Event sourcing** replaces ad-hoc `ObserverEvent`. Every thread has a complete event log -- **Trace-based testing** (Phase 4+). Use event logs as golden traces for regression testing -- **Thread-structural events** (thread.started, step.completed, action.executed) vs current per-subsystem events +- **Event sourcing** replaces ad-hoc `ObserverEvent`. Every thread has complete event log (16 event kinds) +- **Trace-based testing** (Phase 4+). Event logs as golden traces +- **Thread-structural events** (thread.started, step.completed, action.executed) vs per-subsystem + +### RLM Execution Model +- **Context as variable**: thread messages/goal/results injected as Python variables, not LLM attention input +- **Output truncation**: 8K chars between steps (configurable), with `[TRUNCATED]`/`[FULL OUTPUT]` prefixes +- **Step 0 orientation**: auto-inject context metadata before first code step +- **FINAL()/FINAL_VAR()**: explicit termination from within code +- **llm_query()/llm_query_batched()**: recursive/parallel sub-agent calls +- **Error transparency**: Python errors flow to LLM for self-correction, not step termination +- **Symbolic composition**: sub-agent results stored as variables, not injected into parent context ### Backward Compatibility -- Engine runs alongside existing code (feature flag) +- Engine runs alongside existing code via `EngineV2Delegate` adapter - Bridge adapters translate between engine and existing types -- WASM tools/channels unchanged — they implement `Tool`/`Channel` traits, which the bridge wraps -- MCP tools unchanged — same adapter principle -- Existing tests unmodified — they test the old path; new tests validate the engine +- WASM tools/channels unchanged (bridge wraps `Tool`/`Channel` traits) +- MCP tools unchanged (same adapter principle) +- Existing tests unmodified — they test the old path --- -## Implementation Order Summary +## Implementation Progress -| Phase | Scope | Depends on | Key deliverable | -|-------|-------|------------|-----------------| -| **1** | Types + traits + state machine | Nothing | Compiling crate with all type definitions | -| **2** | Tier 0 executor + capability + runtime | Phase 1 | Working execution loop equivalent to `run_agentic_loop()` | -| **3** | CodeAct (Tier 1 embedded scripting) | Phase 2 | LLMs write code that composes tools | -| **4** | Reflection + retrieval + provenance + missions | Phase 2 | Agent learns from work, project-scoped memory | -| **5** | Conversation surface + channel integration | Phase 2 | UI separated from execution | -| **6** | Tier 2-3 + two-phase commit | Phase 3 | Full sandboxed code execution | -| **7** | Main crate bridge + acceptance tests | Phase 2+ | Engine passes all existing tests via adapters | -| **8** | Cleanup + migration | Phase 7 | Old abstractions removed | +| Phase | Scope | Status | Tests | Commits | +|-------|-------|--------|-------|---------| +| **1** | Types + traits + state machine | **DONE** | 32 | `8be19a4` | +| **2** | Tier 0 executor + capability + runtime | **DONE** | 74 | `bf7dfb8` | +| **3** | CodeAct (Monty + RLM pattern) | **DONE** | 74 | `b59a0b9`, `9538332` | +| **4** | Reflection + retrieval + compaction + rlm_query + budget | Planned | — | — | +| **5** | Conversation surface + channel integration | Planned | — | — | +| **6** | Tier 2-3 + two-phase commit | Planned | — | — | +| **7** | Main crate bridge + acceptance tests | Planned | — | — | +| **8** | Cleanup + migration | Planned | — | — | -Phases 3, 4, 5 can proceed in parallel after Phase 2 is complete. +Phases 4, 5, 6 can proceed in parallel. Phase 7 depends on Phase 2+ being stable. Phase 8 depends on Phase 7 passing acceptance tests. --- @@ -648,5 +490,5 @@ cargo clippy --all --benches --tests --examples --all-features cargo test # Phase 7+ acceptance: -cargo test --features engine_v2 # engine-driven tests match existing fixtures +cargo test # engine-driven tests match existing fixtures via EngineV2Delegate ```