mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 15:40:18 +00:00
docs: add engine v2 architecture, self-improvement, and dev history
Three new docs for contributors: - engine-v2-architecture.md: Two-layer architecture (Rust kernel + Python orchestrator), five primitives, execution model with nested Monty VMs, bridge layer, memory/reflection, missions, capabilities - self-improvement.md: Three improvement levels (prompt/orchestrator/ config/code), autoresearch-inspired Mission loop, versioned orchestrator with auto-rollback, fix pattern database, safety model - development-history.md: Summary of 6 Claude Code sessions that built the system, key design decisions and debugging moments, architecture evolution from 900-line Rust loop to Python orchestrator Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
This commit is contained in:
@@ -0,0 +1,124 @@
|
||||
# Development History
|
||||
|
||||
Summary of the Claude Code sessions that built the engine v2, self-improvement system, and Python orchestrator. This helps new contributors understand *why* things were designed the way they are.
|
||||
|
||||
## Session 1: Engine v2 Foundation (2026-03-20 to 2026-03-22)
|
||||
|
||||
Built the core engine crate (`crates/ironclaw_engine/`) from scratch in 6 phases:
|
||||
|
||||
- **Phase 1**: Core types (Thread, Step, Capability, MemoryDoc, Project), trait definitions (LlmBackend, Store, EffectExecutor), thread state machine. 32 tests.
|
||||
- **Phase 2**: Execution engine (Tier 0) — CapabilityRegistry, LeaseManager, PolicyEngine, ThreadManager, ExecutionLoop with structured tool calls. 74 tests.
|
||||
- **Phase 3**: CodeAct executor (Tier 1) — Monty Python interpreter integration, RLM pattern (context-as-variables, FINAL(), llm_query(), output truncation, Step 0 orientation). 74 tests.
|
||||
- **Phase 4**: Memory and reflection — RetrievalEngine, reflection pipeline (Summary/Lesson/Issue/Spec/Playbook docs), context compaction, rlm_query() recursive sub-agents, budget controls. 78 tests.
|
||||
- **Phase 5**: Conversation surface — ConversationManager routing UI messages to threads. 85 tests.
|
||||
- **Phase 6**: Bridge adapters — LlmBridgeAdapter, EffectBridgeAdapter, HybridStore, EngineRouter. Parallel deployment via `ENGINE_V2=true`. 151 tests.
|
||||
|
||||
**Key design decision**: The engine has zero dependency on the main ironclaw crate. All interaction goes through three traits (LlmBackend, Store, EffectExecutor) implemented by bridge adapters.
|
||||
|
||||
## Session 2: Debugging via Traces (2026-03-22 to 2026-03-23)
|
||||
|
||||
Ran the engine end-to-end with real LLMs and discovered 8 bugs through trace analysis:
|
||||
|
||||
1. Tool name hyphens vs underscores (`web-search` vs `web_search`)
|
||||
2. Double-serialization of JSON tool output
|
||||
3. UTF-8 byte-index slicing panics on multi-byte characters
|
||||
4. Code block detection missing in plain completion path
|
||||
5. Missing system prompt on thread spawn
|
||||
6. Empty messages sent to LLM
|
||||
7. `web_fetch` example in prompt (nonexistent tool)
|
||||
8. False positive `missing_tool_output` trace warning
|
||||
|
||||
**Key insight**: Every fix followed the same loop (trace → human reads → human edits Rust → rebuild). This became the motivation for the self-improving engine design.
|
||||
|
||||
## Session 3: Mission System (2026-03-24)
|
||||
|
||||
Built the Mission system for long-running goals that spawn threads over time:
|
||||
|
||||
- `MissionManager` with create/pause/resume/complete lifecycle
|
||||
- `MissionCadence`: Cron, OnEvent, OnSystemEvent, Webhook, Manual
|
||||
- `build_meta_prompt()` — assembles mission goal + current focus + approach history + project docs + trigger payload
|
||||
- `process_mission_outcome()` — extracts next_focus and goal-achieved status from thread responses
|
||||
- Cron ticker (60s interval)
|
||||
- 7 E2E mission flow tests
|
||||
|
||||
**Key design decision**: Missions evolve their strategy via `current_focus` and `approach_history`. Each thread gets a meta-prompt that includes what was tried before.
|
||||
|
||||
## Session 4: Review Fixes + Self-Improvement Foundation (2026-03-25, morning)
|
||||
|
||||
Fixed 4 review comments (P1/P2 severity) in the engine v2 bridge:
|
||||
|
||||
1. **SSE events scoped to user** — `broadcast_for_user()` instead of `broadcast()`
|
||||
2. **Per-user pending approvals** — HashMap keyed by user_id instead of global Option
|
||||
3. **Reset tool-call limit counter** — reset before each thread, not monotonic
|
||||
4. **Only auto-approve on "always"** — one-off "yes" no longer persists
|
||||
|
||||
Then built the self-improvement foundation:
|
||||
|
||||
- Runtime prompt overlay via MemoryDoc (prompt builder becomes async + Store-aware)
|
||||
- `fire_on_system_event()` — wires the previously-unimplemented OnSystemEvent cadence
|
||||
- `start_event_listener()` — subscribes to thread events, fires matching missions
|
||||
- `ensure_self_improvement_mission()` — creates the built-in self-improvement Mission
|
||||
- `process_self_improvement_output()` — saves prompt overlays and fix patterns
|
||||
- Seed fix pattern database with 8 known patterns
|
||||
|
||||
## Session 5: Autoresearch-Inspired Redesign (2026-03-25, afternoon)
|
||||
|
||||
Studied [karpathy/autoresearch](https://github.com/karpathy/autoresearch) and redesigned the self-improvement approach:
|
||||
|
||||
**Before**: Vague goal prompt, structured JSON output, reactive only.
|
||||
**After**: Concrete `program.md`-style prompt with exact loop steps, plain text + tool-use (agent uses tools directly like autoresearch), enriched trigger payload with actual error messages.
|
||||
|
||||
Key takeaways applied from autoresearch:
|
||||
- The entire "research org" is a markdown prompt with an explicit loop
|
||||
- The agent uses tools directly (shell, grep, git) rather than emitting structured output
|
||||
- Results tracked in a simple append-only log
|
||||
- "NEVER STOP" — the agent is autonomous within constraints
|
||||
|
||||
## Session 6: Python Orchestrator (2026-03-25, evening)
|
||||
|
||||
The pivotal architectural change. Motivated by the question: *"What if we move some part of the engine inside CodeAct itself?"*
|
||||
|
||||
**The realization**: All the bugs from Session 2 were in the "glue" between the LLM and tools — output formatting, tool dispatch, state management, truncation. These functions are Python-natural. If they were Python, the self-improvement Mission could fix them without a Rust rebuild.
|
||||
|
||||
**Research**: Verified that Monty supports nested VM execution (`rlm_query()` already does exactly this — suspends parent VM, runs child ExecutionLoop, resumes parent). No shared state, ~50KB per suspended VM.
|
||||
|
||||
**Implementation** (4 commits):
|
||||
|
||||
1. **Host function module** (`executor/orchestrator.rs`) — 11 host functions exposed to Python via Monty suspension: `__llm_complete__`, `__execute_code_step__`, `__execute_action__`, `__check_signals__`, `__emit_event__`, `__add_message__`, `__save_checkpoint__`, `__transition_to__`, `__retrieve_docs__`, `__check_budget__`, `__get_actions__`.
|
||||
|
||||
2. **Default orchestrator** (`orchestrator/default.py`) — The v0 Python orchestrator that replicates the Rust loop logic. Helper functions (extract_final, format_output, signals_tool_intent) defined before run_loop for Monty scoping.
|
||||
|
||||
3. **Switchover** — Replaced the 900-line `ExecutionLoop::run()` with an 80-line bootstrap. Key debugging: Monty's `ExtFunctionResult::NotFound` (not `Error`) for user-defined functions, FINAL result propagation, step_count tracking via `__emit_event__("step_completed")`.
|
||||
|
||||
4. **Versioning + rollback** — Failure tracking via MemoryDoc, auto-rollback after 3 consecutive failures, `OrchestratorRollback` event. Self-improvement Mission goal updated with Level 1.5 orchestrator patch instructions.
|
||||
|
||||
**Key debugging moment**: The orchestrator's helper functions (`extract_final`, `format_output`) were defined after `run_loop` in the Python file. Monty couldn't find them because the default `FunctionCall` handler returned `ExtFunctionResult::Error` instead of `ExtFunctionResult::NotFound`. The fix: return `NotFound` for unknown functions so Monty falls through to its own namespace resolution. Then move helpers above `run_loop` to avoid any ordering issues.
|
||||
|
||||
**Final state**: 189 tests pass, zero clippy warnings. The Python orchestrator is the execution engine. The Rust layer is the kernel.
|
||||
|
||||
## Architecture Evolution
|
||||
|
||||
```
|
||||
Session 1-2: Rust loop (900 lines) → works but bugs in glue layer
|
||||
Session 3: + Missions (long-running goals, evolving strategy)
|
||||
Session 4: + Self-improvement Mission (fires on issues, fixes prompts)
|
||||
Session 5: + Autoresearch-style goal prompt (concrete, not vague)
|
||||
Session 6: Rust loop → Python orchestrator (self-modifiable)
|
||||
900 lines Rust → 80 lines Rust bootstrap + 230 lines Python
|
||||
```
|
||||
|
||||
## Key Commits
|
||||
|
||||
| Commit | Description |
|
||||
|--------|-------------|
|
||||
| `8be19a4` | Phase 1: Foundation types + traits |
|
||||
| `bf7dfb8` | Phase 2: Tier 0 execution engine |
|
||||
| `b59a0b9` | Phase 3: CodeAct (Monty + RLM) |
|
||||
| `4bc7ffd` | Phase 4: Memory + reflection + budgets |
|
||||
| `0827235` | Phase 5: Conversation surface |
|
||||
| `ac4ced0` | Phase 6: Bridge adapters (parallel deploy) |
|
||||
| `8180a417` | Self-improving engine via Mission system |
|
||||
| `cfe856da` | Python orchestrator module + host functions |
|
||||
| `63756039` | Switch ExecutionLoop to Python orchestrator |
|
||||
| `080317aa` | All 177 tests pass with orchestrator |
|
||||
| `46fd2b5d` | Versioning, auto-rollback, 189 tests |
|
||||
@@ -0,0 +1,252 @@
|
||||
# Engine v2 Architecture
|
||||
|
||||
This document describes the IronClaw Engine v2 architecture for new contributors. It covers the execution model, the Python orchestrator, the bridge layer, and how everything fits together.
|
||||
|
||||
## Overview
|
||||
|
||||
IronClaw Engine v2 replaces ~10 fragmented abstractions (Session, Job, Routine, Channel, Tool, Skill, Hook, Observer, Extension, LoopDelegate) with a unified model built on 5 primitives. The engine lives in `crates/ironclaw_engine/` as a standalone crate with no dependency on the main `ironclaw` crate.
|
||||
|
||||
The key architectural innovation: **the execution loop is Python code running inside the Monty interpreter, not Rust**. Rust provides the infrastructure (LLM calls, tool execution, safety, persistence). Python provides the orchestration (tool dispatch, output formatting, state management). This makes the glue layer self-modifiable at runtime by the self-improvement Mission.
|
||||
|
||||
## Five Primitives
|
||||
|
||||
| Primitive | Purpose | Replaces |
|
||||
|-----------|---------|----------|
|
||||
| **Thread** | Unit of work with lifecycle, parent-child tree, capability leases | Session + Job + Routine + Sub-agent |
|
||||
| **Step** | Unit of execution (one LLM call + its action executions) | Agentic loop iteration + tool calls |
|
||||
| **Capability** | Unit of effect (actions + knowledge + policies) | Tool + Skill + Hook + Extension |
|
||||
| **MemoryDoc** | Unit of durable knowledge (summaries, lessons, playbooks) | Workspace memory blobs |
|
||||
| **Project** | Unit of context (scopes memory, threads, missions) | Flat workspace namespace |
|
||||
|
||||
## Execution Model
|
||||
|
||||
### The Two-Layer Architecture
|
||||
|
||||
```
|
||||
Rust Layer (stable kernel — rarely changes)
|
||||
├── LlmBackend trait → make LLM API calls
|
||||
├── EffectExecutor trait → run tools with safety/policy/hooks
|
||||
├── Store trait → persist threads, steps, events, docs
|
||||
├── LeaseManager → grant/check/consume/revoke capability leases
|
||||
├── PolicyEngine → deterministic allow/deny/require-approval
|
||||
├── ThreadManager → spawn, stop, inject messages, join threads
|
||||
├── Monty VM → embedded Python interpreter
|
||||
└── Safety layer → sanitization, leak detection, policy enforcement
|
||||
|
||||
Python Layer (self-modifiable orchestrator — where bugs get fixed)
|
||||
├── The step loop → call LLM → handle response → repeat
|
||||
├── Tool dispatch → name resolution, alias mapping
|
||||
├── Output formatting → truncation, context assembly
|
||||
├── State management → persisted_state dict across code steps
|
||||
├── FINAL() extraction → parse termination signals from text
|
||||
├── Tool intent nudging → detect when LLM describes instead of acts
|
||||
└── Doc injection → format memory docs for context
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Bootstrap** (`ExecutionLoop::run()` in `loop_engine.rs`, ~80 lines):
|
||||
- Transition thread to Running state
|
||||
- Inject CodeAct system prompt (with runtime prompt overlay if available)
|
||||
- Load versioned Python orchestrator from Store (or compiled-in default)
|
||||
- Execute orchestrator via Monty VM
|
||||
- Map return value to `ThreadOutcome`
|
||||
- Persist final state
|
||||
|
||||
2. **Orchestrator** (`orchestrator/default.py`, ~230 lines):
|
||||
- Calls host functions to interact with Rust infrastructure
|
||||
- Runs the step loop: check signals → check budget → call LLM → handle response
|
||||
- For text responses: extract FINAL(), check nudge, or complete
|
||||
- For code responses: run user code in nested Monty VM, format output
|
||||
- For action calls: execute each action, handle approval flow
|
||||
- Returns outcome dict: `{outcome, response, error, ...}`
|
||||
|
||||
3. **Host functions** (Rust, called via Monty's suspension mechanism):
|
||||
- `__llm_complete__` → call `LlmBackend::complete()`
|
||||
- `__execute_code_step__` → run user CodeAct code in a nested Monty VM
|
||||
- `__execute_action__` → execute a tool with lease + policy + safety
|
||||
- `__check_signals__` → poll for stop/inject signals
|
||||
- `__emit_event__` → broadcast ThreadEvent + record in thread
|
||||
- `__add_message__` → append message to thread history
|
||||
- `__save_checkpoint__` → persist state to thread metadata
|
||||
- `__transition_to__` → validated thread state transition
|
||||
- `__retrieve_docs__` → query memory docs from Store
|
||||
- `__check_budget__` → remaining tokens/time/USD
|
||||
- `__get_actions__` → available tool definitions from leases
|
||||
|
||||
### Nested Execution (CodeAct)
|
||||
|
||||
When the LLM responds with Python code, the orchestrator calls `__execute_code_step__(code, state)`. This suspends the orchestrator VM and creates a **second Monty VM** for the user's code:
|
||||
|
||||
```
|
||||
Orchestrator VM (Monty #1)
|
||||
→ calls __execute_code_step__(code, state)
|
||||
→ suspends
|
||||
→ Rust creates Monty #2 (user code VM)
|
||||
→ User code calls web_search() → suspends → Rust executes tool → resumes
|
||||
→ User code calls FINAL("answer") → terminates
|
||||
→ Rust collects results
|
||||
→ Orchestrator VM resumes with results dict
|
||||
→ Orchestrator formats output, decides next step
|
||||
```
|
||||
|
||||
This is the same mechanism as `rlm_query()` (recursive sub-agent). Each VM owns its own heap — no shared state, no locks.
|
||||
|
||||
### Thread State Machine
|
||||
|
||||
```
|
||||
Created → Running → Waiting → Running (resume)
|
||||
→ Suspended → Running (resume)
|
||||
→ Completed → Reflecting → Done
|
||||
→ Failed
|
||||
```
|
||||
|
||||
Terminal states: `Done`, `Failed`. Validated by `ThreadState::can_transition_to()`.
|
||||
|
||||
## Bridge Layer (`src/bridge/`)
|
||||
|
||||
The bridge connects the engine to existing IronClaw infrastructure:
|
||||
|
||||
| Adapter | Wraps | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `LlmBridgeAdapter` | `LlmProvider` | Converts `ThreadMessage` ↔ `ChatMessage`, depth-based model routing, code block detection |
|
||||
| `EffectBridgeAdapter` | `ToolRegistry` + `SafetyLayer` | Tool execution with all v1 security controls, name normalization (underscore ↔ hyphen), rate limiting |
|
||||
| `HybridStore` | `Workspace` | In-memory for ephemeral data, workspace files for MemoryDocs |
|
||||
| `EngineRouter` | `Agent` | Routes messages through engine when `ENGINE_V2=true`, manages SSE events |
|
||||
|
||||
### Enabling Engine v2
|
||||
|
||||
Set `ENGINE_V2=true` environment variable. The router in `src/bridge/router.rs` intercepts messages and routes them through the engine instead of the v1 agent loop.
|
||||
|
||||
For trace debugging: `ENGINE_V2_TRACE=1` writes full JSON traces to `engine_trace_*.json`.
|
||||
|
||||
## Memory and Reflection
|
||||
|
||||
### MemoryDoc Types
|
||||
|
||||
| Type | Purpose | Produced By |
|
||||
|------|---------|-------------|
|
||||
| `Summary` | What a thread accomplished | Reflection (always) |
|
||||
| `Lesson` | Durable learning from experience | Reflection (on errors) |
|
||||
| `Playbook` | Reusable multi-step procedure | Reflection (on success with 2+ tools) |
|
||||
| `Issue` | Detected problem for follow-up | Reflection (on failure) |
|
||||
| `Spec` | Missing capability request | Reflection (on "not found" errors) |
|
||||
| `Note` | Working memory / scratch | Self-improvement, orchestrator code |
|
||||
|
||||
### Reflection Pipeline
|
||||
|
||||
After a thread completes with `enable_reflection: true`:
|
||||
|
||||
1. **Trace analysis** (non-LLM, always runs) — detects 8 issue categories
|
||||
2. **LLM reflection** — spawns a Reflection-type CodeAct thread with read-only tools
|
||||
3. **Doc production** — creates Summary, Lesson, Issue, Spec, Playbook docs
|
||||
4. **Persistence** — saves docs to Store (HybridStore → workspace files)
|
||||
5. **Event firing** — if issues detected, fires OnSystemEvent missions (self-improvement)
|
||||
|
||||
### Context Injection
|
||||
|
||||
On each LLM call, `build_step_context()` retrieves up to 5 relevant MemoryDocs from the project and appends them to the system prompt as "## Prior Knowledge". This gives the LLM access to lessons, playbooks, and known issues from prior threads.
|
||||
|
||||
## Missions
|
||||
|
||||
Missions are long-running goals that spawn threads over time. They replace v1 Routines.
|
||||
|
||||
```
|
||||
Mission
|
||||
├── goal: "Increase test coverage to 80%"
|
||||
├── cadence: Cron("0 9 * * *") | OnSystemEvent | Manual | Webhook
|
||||
├── current_focus: "Write tests for auth module" (evolves)
|
||||
├── approach_history: ["Analyzed codebase", "Added 15 tests for db"]
|
||||
├── thread_history: [thread_1, thread_2, ...]
|
||||
└── max_threads_per_day: 10
|
||||
```
|
||||
|
||||
### How Missions Fire
|
||||
|
||||
- **Cron**: Background ticker checks every 60s, fires missions with past `next_fire_at`
|
||||
- **OnSystemEvent**: Event listener subscribes to ThreadManager events, fires matching missions when threads complete with issues
|
||||
- **Manual**: `mission_fire(id)` from CodeAct or API
|
||||
- **Webhook**: Bridge routes incoming webhooks to matching missions
|
||||
|
||||
### Meta-Prompt Generation
|
||||
|
||||
When a mission fires, `build_meta_prompt()` assembles:
|
||||
- Mission goal + success criteria
|
||||
- Current focus (what to work on next)
|
||||
- Approach history (what was tried and what happened)
|
||||
- Project knowledge (relevant MemoryDocs)
|
||||
- Trigger payload (event data, trace issues)
|
||||
|
||||
The thread runs with this context and returns: what it accomplished, what to focus on next, whether the goal is achieved. `process_mission_outcome()` extracts these and updates the mission.
|
||||
|
||||
## Capability System
|
||||
|
||||
### Leases
|
||||
|
||||
Threads don't have static permissions. They receive **leases** — scoped, time-limited, use-limited grants:
|
||||
|
||||
```rust
|
||||
CapabilityLease {
|
||||
thread_id,
|
||||
capability_name,
|
||||
granted_actions: ["web_search", "read_file", ...],
|
||||
expires_at: Option<DateTime>,
|
||||
max_uses: Option<u32>,
|
||||
revoked: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Policy Engine
|
||||
|
||||
The PolicyEngine evaluates actions against leases deterministically:
|
||||
|
||||
1. Check global denied effects (e.g., deny all Financial)
|
||||
2. Check capability-level policies (per-action rules)
|
||||
3. Check action's `requires_approval` flag
|
||||
4. Check effect types against lease grant
|
||||
|
||||
Decision priority: **Deny > RequireApproval > Allow**
|
||||
|
||||
### Effect Types
|
||||
|
||||
Every action declares its side effects:
|
||||
```
|
||||
ReadLocal, ReadExternal, WriteLocal, WriteExternal,
|
||||
CredentialedNetwork, Compute, Financial
|
||||
```
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `crates/ironclaw_engine/orchestrator/default.py` | The Python execution loop (v0) |
|
||||
| `crates/ironclaw_engine/src/executor/orchestrator.rs` | Host functions + versioning + loading |
|
||||
| `crates/ironclaw_engine/src/executor/loop_engine.rs` | Bootstrap (loads + runs orchestrator) |
|
||||
| `crates/ironclaw_engine/src/executor/scripting.rs` | Monty VM integration, user code execution |
|
||||
| `crates/ironclaw_engine/src/runtime/manager.rs` | ThreadManager (spawn, stop, join, reflection) |
|
||||
| `crates/ironclaw_engine/src/runtime/mission.rs` | MissionManager (lifecycle, firing, self-improvement) |
|
||||
| `crates/ironclaw_engine/src/types/` | All core data structures |
|
||||
| `crates/ironclaw_engine/src/traits/` | LlmBackend, Store, EffectExecutor |
|
||||
| `src/bridge/router.rs` | Engine v2 entry point from main crate |
|
||||
| `src/bridge/effect_adapter.rs` | Tool execution bridge with safety |
|
||||
| `src/bridge/llm_adapter.rs` | LLM provider bridge |
|
||||
| `src/bridge/store_adapter.rs` | HybridStore (in-memory + workspace) |
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
cargo check -p ironclaw_engine # compiles
|
||||
cargo clippy -p ironclaw_engine --all-targets -- -D warnings # zero warnings
|
||||
cargo test -p ironclaw_engine # 189 tests
|
||||
cargo clippy --all --all-features # full crate
|
||||
cargo test # full suite
|
||||
```
|
||||
|
||||
## Design Influences
|
||||
|
||||
- **RLM paper** (arXiv:2512.24601) — context as variable, FINAL() termination, recursive sub-calls
|
||||
- **karpathy/autoresearch** — the self-improvement loop as a program.md, fixed-budget evaluation, git as state machine
|
||||
- **Official RLM impl** (alexzhang13/rlm) — 30 max iterations, compaction at 85%, budget inheritance
|
||||
- **fast-rlm** (avbiswas/fast-rlm) — Step 0 orientation, parallel sub-calls, dual model routing
|
||||
|
||||
See also: `docs/plans/2026-03-20-engine-v2-architecture.md` for the full 8-phase roadmap.
|
||||
@@ -0,0 +1,218 @@
|
||||
# Self-Improving Engine
|
||||
|
||||
This document describes how IronClaw improves itself at runtime — fixing bugs, evolving prompts, and patching its own execution loop without a Rust rebuild.
|
||||
|
||||
## The Problem
|
||||
|
||||
During development, 5 consecutive debugging sessions revealed the same pattern:
|
||||
|
||||
1. A thread runs and hits a bug (wrong tool name, bad output format, UTF-8 crash)
|
||||
2. The LLM tries to work around it but can't fix the Rust code
|
||||
3. A human reads the trace, identifies the root cause, edits Rust, rebuilds
|
||||
4. The fix takes effect on the next run
|
||||
|
||||
Every step of this loop is something the engine can do. The key insight: **if the orchestration layer were Python (not Rust), the engine could fix its own bugs at runtime**.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Three Self-Improvement Levels
|
||||
|
||||
| Level | What Changes | Risk | Who Approves | Mechanism |
|
||||
|-------|-------------|------|-------------|-----------|
|
||||
| **1: Prompt** | System prompt rules | Low | Auto | MemoryDoc overlay appended to compiled preamble |
|
||||
| **1.5: Orchestrator** | Python execution loop | Medium | Auto (3-failure rollback) | Versioned MemoryDoc, loaded at thread start |
|
||||
| **2: Config** | Engine defaults, constants | Medium | Auto if tests pass | Git branch + cargo test |
|
||||
| **3: Code** | Rust source in engine/bridge | High | Human via PR | Proposed, not applied |
|
||||
|
||||
### The Self-Improvement Mission
|
||||
|
||||
A built-in Mission with `OnSystemEvent` cadence fires when threads complete with issues:
|
||||
|
||||
```
|
||||
Thread completes → Trace analysis (8 issue categories)
|
||||
→ Reflection (produces Lesson/Spec/Issue docs)
|
||||
→ Emit "thread_completed_with_issues" event
|
||||
↓
|
||||
MissionManager event listener
|
||||
↓
|
||||
Self-improvement Mission fires
|
||||
↓
|
||||
Mission thread (CodeAct, all tools)
|
||||
├── Reads trigger payload (trace issues + error messages)
|
||||
├── Checks fix pattern database for known solutions
|
||||
├── Diagnoses root cause (PROMPT / ORCHESTRATOR / CONFIG / CODE)
|
||||
└── Applies fix at appropriate level
|
||||
```
|
||||
|
||||
### Trigger Payload
|
||||
|
||||
The event listener builds a JSON payload containing:
|
||||
|
||||
```json
|
||||
{
|
||||
"source_thread_id": "uuid",
|
||||
"goal": "what the thread was trying to do",
|
||||
"issues": [
|
||||
{"severity": "Error", "category": "tool_error", "description": "Tool web_search not found", "step": 1}
|
||||
],
|
||||
"error_messages": ["web_search: no lease for action 'web_search'"],
|
||||
"reflection": {"docs_produced": 3, "doc_types": ["Summary", "Lesson", "Spec"]}
|
||||
}
|
||||
```
|
||||
|
||||
The self-improvement thread receives this as `state["trigger_payload"]` and uses it to diagnose and fix the issue.
|
||||
|
||||
## Level 1: Prompt Evolution
|
||||
|
||||
The system prompt is built from:
|
||||
1. **Compiled preamble** (`include_str!` from `prompts/codeact_preamble.md`) — never modified at runtime
|
||||
2. **Runtime overlay** (MemoryDoc with tag `prompt_overlay`) — appended after the preamble
|
||||
3. **Tool documentation** — dynamically generated from available actions
|
||||
4. **Compiled postamble** — strategy section
|
||||
|
||||
The self-improvement thread can append rules to the overlay:
|
||||
|
||||
```python
|
||||
# In the self-improvement thread:
|
||||
memory_write(
|
||||
title="prompt:codeact_preamble",
|
||||
content="9. Never call web_fetch — use http() instead.\n10. Always access state dict for prior results.",
|
||||
tags=["prompt_overlay"]
|
||||
)
|
||||
```
|
||||
|
||||
The overlay is capped at 4000 characters. Next thread picks up the updated prompt.
|
||||
|
||||
## Level 1.5: Orchestrator Patches
|
||||
|
||||
The execution loop itself is Python code stored as a versioned MemoryDoc:
|
||||
|
||||
```
|
||||
v0 (compiled-in default.py)
|
||||
→ v1 (self-improvement fix: better output formatting)
|
||||
→ v2 (self-improvement fix: tool name alias)
|
||||
→ v3 (bad fix, causes crashes)
|
||||
↑ auto-rollback after 3 failures → back to v2
|
||||
```
|
||||
|
||||
### Versioning
|
||||
|
||||
Each orchestrator version is a MemoryDoc:
|
||||
- Title: `orchestrator:main`
|
||||
- Tag: `orchestrator_code`
|
||||
- Metadata: `{"version": N, "parent_version": N-1}`
|
||||
|
||||
Loading priority: highest version number wins. If the latest version has 3+ consecutive failures (tracked via `orchestrator:failures` doc), it's skipped and the previous version is loaded.
|
||||
|
||||
### Auto-Rollback
|
||||
|
||||
```
|
||||
Thread starts → load_orchestrator() checks failure tracker
|
||||
├── Latest version has < 3 failures → use it
|
||||
├── Latest version has >= 3 failures → skip, try previous
|
||||
└── All versions failed → use compiled-in v0
|
||||
|
||||
Thread succeeds → reset failure counter
|
||||
Thread fails → increment failure counter for current version
|
||||
```
|
||||
|
||||
### What the Orchestrator Controls
|
||||
|
||||
The Python orchestrator handles all the "glue" between the LLM and tools:
|
||||
|
||||
- **Tool dispatch**: How function calls are resolved and executed
|
||||
- **Output formatting**: How tool results are presented to the LLM
|
||||
- **State management**: How variables persist across code steps
|
||||
- **Truncation**: How large outputs are compacted
|
||||
- **FINAL() extraction**: How termination signals are parsed
|
||||
- **Nudge detection**: When to prompt the LLM to write code instead of describing
|
||||
|
||||
These are exactly the functions that had bugs during development (wrong tool names, JSON double-serialization, UTF-8 panics, missing state). Now they can be fixed at runtime.
|
||||
|
||||
## Level 2: Configuration Tuning
|
||||
|
||||
The self-improvement thread can create git branches and modify engine defaults:
|
||||
|
||||
```python
|
||||
# In the self-improvement thread:
|
||||
shell("git checkout -b self-improve/increase-truncation")
|
||||
read_file("crates/ironclaw_engine/src/executor/scripting.rs")
|
||||
apply_patch(...)
|
||||
result = shell("cargo test -p ironclaw_engine")
|
||||
if "test result: ok" in result:
|
||||
shell("git commit -am 'Increase output truncation to 12000 chars'")
|
||||
else:
|
||||
shell("git checkout main")
|
||||
```
|
||||
|
||||
## Level 3: Code Patches
|
||||
|
||||
For Rust bugs in the engine or bridge, the self-improvement thread describes the fix but does not apply it directly. The recommendation appears in the thread's FINAL() response and in the mission's approach_history.
|
||||
|
||||
## Fix Pattern Database
|
||||
|
||||
A Playbook MemoryDoc maps known trace symptoms to fix strategies:
|
||||
|
||||
| Trace Pattern | Fix Strategy | Location |
|
||||
|---|---|---|
|
||||
| Tool X not found | Add name alias or prompt hint | prompt overlay or effect_adapter |
|
||||
| TypeError: str indices must be integers | Parse JSON before wrapping | output conversion |
|
||||
| NameError: name 'X' not defined | Add prompt hint about state dict | prompt overlay |
|
||||
| byte index N is not a char boundary | Replace byte slicing with chars() | string truncation |
|
||||
| Model calls nonexistent tool | Add prompt rule with correct name | prompt overlay |
|
||||
| Model ignores tool results | Improve output metadata format | orchestrator |
|
||||
| Excessive steps (>5) for simple task | Add prompt rule or fix tool schema | prompt overlay |
|
||||
| Code error in REPL output | Add prompt hint about correct API | prompt overlay |
|
||||
|
||||
The database grows over time — after successfully fixing an issue, the self-improvement thread adds a new pattern entry.
|
||||
|
||||
## Safety Boundaries
|
||||
|
||||
**Hard boundaries (never auto-modify):**
|
||||
- Security-sensitive code (safety layer, policy engine, leak detection)
|
||||
- Database schemas / migrations
|
||||
- Test files (never weaken tests to make a fix pass)
|
||||
- Files outside `crates/ironclaw_engine/` and `src/bridge/` without human approval
|
||||
|
||||
**Orchestrator safety:**
|
||||
- Auto-rollback after 3 consecutive failures
|
||||
- Compiled-in v0 always available as last resort
|
||||
- Each version tracked with parent_version for audit trail
|
||||
- Resource limits (5min timeout, 128MB memory) on orchestrator VM
|
||||
|
||||
## Creating the Self-Improvement Mission
|
||||
|
||||
On engine init (`src/bridge/router.rs`), `ensure_self_improvement_mission()` is called. It:
|
||||
|
||||
1. Checks if a self-improvement mission already exists for the project
|
||||
2. If not, creates one with `OnSystemEvent { source: "engine", event_type: "thread_completed_with_issues" }`
|
||||
3. Seeds the fix pattern database with known patterns
|
||||
4. Starts the event listener (`start_event_listener()`)
|
||||
|
||||
The mission is capped at 5 threads per day (`max_threads_per_day: 5`).
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `crates/ironclaw_engine/orchestrator/default.py` | The v0 orchestrator (self-modifiable) |
|
||||
| `crates/ironclaw_engine/src/executor/orchestrator.rs` | Loading, versioning, rollback, host functions |
|
||||
| `crates/ironclaw_engine/src/executor/prompt.rs` | Prompt overlay loading |
|
||||
| `crates/ironclaw_engine/src/runtime/mission.rs` | Self-improvement mission, OnSystemEvent wiring, fix patterns |
|
||||
| `docs/plans/2026-03-23-self-improving-engine.md` | Original design doc |
|
||||
| `docs/plans/2026-03-25-python-orchestrator.md` | Python orchestrator design doc |
|
||||
|
||||
## Debugging Self-Improvement
|
||||
|
||||
Enable trace logging to see the self-improvement loop in action:
|
||||
|
||||
```bash
|
||||
ENGINE_V2=true ENGINE_V2_TRACE=1 RUST_LOG=ironclaw_engine=debug cargo run
|
||||
```
|
||||
|
||||
Look for:
|
||||
- `"loaded runtime orchestrator"` — which version was loaded
|
||||
- `"orchestrator version has too many failures, skipping"` — rollback in action
|
||||
- `"self-improvement: updated prompt overlay"` — Level 1 fix applied
|
||||
- `"event listener: failed to fire self-improvement"` — event listener errors
|
||||
- `SelfImprovementStarted` / `SelfImprovementComplete` events in traces
|
||||
Reference in New Issue
Block a user