optimclaw/docs/self-improvement.md

Self-Improving Engine

This document describes how IronClaw improves itself at runtime — fixing bugs, evolving prompts, and patching its own execution loop without a Rust rebuild.

The Problem

During development, 5 consecutive debugging sessions revealed the same pattern:

1. A thread runs and hits a bug (wrong tool name, bad output format, UTF-8 crash)
2. The LLM tries to work around it but can't fix the Rust code
3. A human reads the trace, identifies the root cause, edits Rust, rebuilds
4. The fix takes effect on the next run

Every step of this loop is something the engine can do. The key insight: if the orchestration layer were Python (not Rust), the engine could fix its own bugs at runtime.

Architecture

Three Self-Improvement Levels

Level	What Changes	Risk	Who Approves	Mechanism
1: Prompt	System prompt rules	Low	Auto	MemoryDoc overlay appended to compiled preamble
1.5: Orchestrator	Python execution loop	Medium	Auto (3-failure rollback)	Versioned MemoryDoc, loaded at thread start
2: Config	Engine defaults, constants	Medium	Auto if tests pass	Git branch + cargo test
3: Code	Rust source in engine/bridge	High	Human via PR	Proposed, not applied

The Self-Improvement Mission

A built-in Mission with OnSystemEvent cadence fires when threads complete with issues:

Thread completes → Trace analysis (8 issue categories)
                → Reflection (produces Lesson/Spec/Issue docs)
                → Emit "thread_completed_with_issues" event
                        ↓
              MissionManager event listener
                        ↓
              Self-improvement Mission fires
                        ↓
              Mission thread (CodeAct, all tools)
                  ├── Reads trigger payload (trace issues + error messages)
                  ├── Checks fix pattern database for known solutions
                  ├── Diagnoses root cause (PROMPT / ORCHESTRATOR / CONFIG / CODE)
                  └── Applies fix at appropriate level

Trigger Payload

The event listener builds a JSON payload containing:

{
  "source_thread_id": "uuid",
  "goal": "what the thread was trying to do",
  "issues": [
    {"severity": "Error", "category": "tool_error", "description": "Tool web_search not found", "step": 1}
  ],
  "error_messages": ["web_search: no lease for action 'web_search'"],
  "reflection": {"docs_produced": 3, "doc_types": ["Summary", "Lesson", "Spec"]}
}

The self-improvement thread receives this as state["trigger_payload"] and uses it to diagnose and fix the issue.

Level 1: Prompt Evolution

The system prompt is built from:

1. Compiled preamble (include_str! from prompts/codeact_preamble.md) — never modified at runtime
2. Runtime overlay (MemoryDoc with tag prompt_overlay) — appended after the preamble
3. Tool documentation — dynamically generated from available actions
4. Compiled postamble — strategy section

The self-improvement thread can append rules to the overlay:

# In the self-improvement thread:
memory_write(
    title="prompt:codeact_preamble",
    content="9. Never call web_fetch — use http() instead.\n10. Always access state dict for prior results.",
    tags=["prompt_overlay"]
)

The overlay is capped at 4000 characters. Next thread picks up the updated prompt.

Level 1.5: Orchestrator Patches

The execution loop itself is Python code stored as a versioned MemoryDoc:

v0 (compiled-in default.py)
  → v1 (self-improvement fix: better output formatting)
    → v2 (self-improvement fix: tool name alias)
      → v3 (bad fix, causes crashes)
          ↑ auto-rollback after 3 failures → back to v2

Versioning

Each orchestrator version is a MemoryDoc:

• Title: orchestrator:main
• Tag: orchestrator_code
• Metadata: {"version": N, "parent_version": N-1}

Loading priority: highest version number wins. If the latest version has 3+ consecutive failures (tracked via orchestrator:failures doc), it's skipped and the previous version is loaded.

Auto-Rollback

Thread starts → load_orchestrator() checks failure tracker
  ├── Latest version has < 3 failures → use it
  ├── Latest version has >= 3 failures → skip, try previous
  └── All versions failed → use compiled-in v0

Thread succeeds → reset failure counter
Thread fails → increment failure counter for current version

What the Orchestrator Controls

The Python orchestrator handles all the "glue" between the LLM and tools:

• Tool dispatch: How function calls are resolved and executed
• Output formatting: How tool results are presented to the LLM
• State management: How variables persist across code steps
• Truncation: How large outputs are compacted
• FINAL() extraction: How termination signals are parsed
• Nudge detection: When to prompt the LLM to write code instead of describing

These are exactly the functions that had bugs during development (wrong tool names, JSON double-serialization, UTF-8 panics, missing state). Now they can be fixed at runtime.

Level 2: Configuration Tuning

The self-improvement thread can create git branches and modify engine defaults:

# In the self-improvement thread:
shell("git checkout -b self-improve/increase-truncation")
read_file("crates/ironclaw_engine/src/executor/scripting.rs")
apply_patch(...)
result = shell("cargo test -p ironclaw_engine")
if "test result: ok" in result:
    shell("git commit -am 'Increase output truncation to 12000 chars'")
else:
    shell("git checkout main")

Level 3: Code Patches

For Rust bugs in the engine or bridge, the self-improvement thread describes the fix but does not apply it directly. The recommendation appears in the thread's FINAL() response and in the mission's approach_history.

Fix Pattern Database

A Note MemoryDoc maps known trace symptoms to fix strategies:

Trace Pattern	Fix Strategy	Location
Tool X not found	Add name alias or prompt hint	prompt overlay or effect_adapter
TypeError: str indices must be integers	Parse JSON before wrapping	output conversion
NameError: name 'X' not defined	Add prompt hint about state dict	prompt overlay
byte index N is not a char boundary	Replace byte slicing with chars()	string truncation
Model calls nonexistent tool	Add prompt rule with correct name	prompt overlay
Model ignores tool results	Improve output metadata format	orchestrator
Excessive steps (>5) for simple task	Add prompt rule or fix tool schema	prompt overlay
Code error in REPL output	Add prompt hint about correct API	prompt overlay

The database grows over time — after successfully fixing an issue, the self-improvement thread adds a new pattern entry.

Safety Boundaries

Hard boundaries (never auto-modify):

• Security-sensitive code (safety layer, policy engine, leak detection)
• Database schemas / migrations
• Test files (never weaken tests to make a fix pass)
• Files outside crates/ironclaw_engine/ and src/bridge/ without human approval

Orchestrator safety:

• Auto-rollback after 3 consecutive failures
• Compiled-in v0 always available as last resort
• Each version tracked with parent_version for audit trail
• Resource limits (5min timeout, 128MB memory) on orchestrator VM

Creating the Self-Improvement Mission

On engine init (src/bridge/router.rs), ensure_self_improvement_mission() is called. It:

1. Checks if a self-improvement mission already exists for the project
2. If not, creates one with OnSystemEvent { source: "engine", event_type: "thread_completed_with_issues" }
3. Seeds the fix pattern database with known patterns
4. Starts the event listener (start_event_listener())

The mission is capped at 5 threads per day (max_threads_per_day: 5).

Key Files

File	Purpose
crates/ironclaw_engine/orchestrator/default.py	The v0 orchestrator (self-modifiable)
crates/ironclaw_engine/src/executor/orchestrator.rs	Loading, versioning, rollback, host functions
crates/ironclaw_engine/src/executor/prompt.rs	Prompt overlay loading
crates/ironclaw_engine/src/runtime/mission.rs	Self-improvement mission, OnSystemEvent wiring, fix patterns
docs/plans/2026-03-23-self-improving-engine.md	Original design doc
docs/plans/2026-03-25-python-orchestrator.md	Python orchestrator design doc

Debugging Self-Improvement

Enable trace logging to see the self-improvement loop in action:

ENGINE_V2=true ENGINE_V2_TRACE=1 RUST_LOG=ironclaw_engine=debug cargo run

Look for:

• "loaded runtime orchestrator" — which version was loaded
• "orchestrator version has too many failures, skipping" — rollback in action
• "self-improvement: updated prompt overlay" — Level 1 fix applied
• "event listener: failed to fire self-improvement" — event listener errors
• SelfImprovementStarted / SelfImprovementComplete events in traces