The orchestrator Python returned {"outcome": "need_approval"} without
calling __transition_to__("waiting"), leaving the thread in Running
state. When the user later approved/denied, resume_thread rejected it
with "thread is not resumable from Running".
- Add __transition_to__("waiting", "approval needed") in both code-step
and action-call approval paths in default.py
- Add Rust safety net in loop_engine.rs: if orchestrator returns
NeedApproval but thread isn't Waiting, force the transition
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Add "Runtime environment" section to codeact_preamble.md documenting
Monty's restrictions: no stdlib imports, single imports only, no classes/
with/match/del/yield, available builtins and modules, workarounds
- Add MONTY.md tracking current pin, all limitations, upgrade process,
and changelog for future Monty updates
- Fix gateway createNewThread() not resetting read-only state — new
threads now eagerly enable chat input instead of waiting for async
loadThreads() callback
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Remove `pub use ironclaw_safety::*` from src/safety/mod.rs and migrate
all 20+ call sites to import directly from `ironclaw_safety`
- Remove `pub use ironclaw_skills::*` from src/skills/mod.rs and migrate
all 15+ call sites to import directly from `ironclaw_skills`
- Fix 4 clippy warnings: 2 shadow imports, 2 collapsible if-let chains
- Add missing SkillActivated arm to WASM channel StatusUpdate match
- Remove duplicate AuthRequired/AuthCompleted arms in repl.rs
- Update CLAUDE.md extracted crates guidance and prompt template rule
- Fix bench imports (safety_check, safety_pipeline)
46 files changed, zero warnings, 3836 tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Two fixes from live testing:
1. http tool: treat null headers/body as empty (Python's None becomes
JSON null via Monty). Previously headers=None errored with
"'headers' must be an object or array of {name, value}".
2. scripting.rs: compute params_summary before dispatching actions in
the CodeAct path (was always None). Now http calls show their URL
in the CLI: ● http(https://api.github.com/repos/.../issues)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add params_summary to ActionExecuted/ActionFailed events so the CLI
and gateway can display what tools are doing:
● http(https://api.github.com/repos/nearai/ironclaw/issues)
● web_search(latest AI news)
● memory_read(HEARTBEAT.md)
The summarize_params() helper extracts the most relevant argument
per tool type (URL for http, query for search, path for memory, etc.)
and truncates to 80 chars. Sensitive params are not included.
Router forwards the summary in both StatusUpdate (CLI/REPL) and
AppEvent (web gateway SSE) display names.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
End-to-end skill activation display:
1. Python orchestrator emits __emit_event__("skill_activated", skill_names=...)
after select_skills() picks skills for the conversation
2. Rust host function parses the comma-separated names into EventKind::SkillActivated
3. Router forwards to channels as StatusUpdate::SkillActivated
4. REPL renders: ◈ skills: github, linear (cyan)
5. Web gateway emits AppEvent::SkillActivated SSE event for frontend display
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Session 9 changes driven by live trace analysis:
- CodeAct event pipeline: handle_execute_code_step now transfers
CodeExecutionResult events to thread.events and broadcasts via event_tx
(fixes false-positive no_tools_used trace warnings)
- Monty globals()/locals() builtins: returns dict of available action names
from capability leases, enabling "tool_name" in globals() probing
- PlatformInfo injection into system prompts (version, LLM backend, model,
database, channels, owner, repo URL)
- Mission goal prompts moved to prompts/*.md files (include_str! pattern)
- /expected command for triggering self-improvement from user feedback
- Session 9 development history
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When the HTTP tool detects a missing credential for a registered host:
1. EffectBridgeAdapter emits SSE AuthRequired event (best-effort, for
connected frontends — silently dropped for missions/background threads)
2. Error flows back to LLM as normal ActionResult (non-blocking)
3. LLM tells the user to authenticate
This avoids the blocking interruption approach which would hang mission
threads and sub-threads that have no channel context.
Engine additions:
- EngineError::NeedAuthentication variant for structured auth failures
- ThreadOutcome::NeedAuthentication for batch interruption when needed
- structured.rs handles NeedAuthentication by interrupting the batch
(stops subsequent calls, returns outcome to orchestrator)
- Auth callback on EffectBridgeAdapter (optional, set by router for SSE)
- extract_credential_name parser for HTTP tool error messages
- routine_* tools added to is_v1_only_tool blocklist
Safety: added 5-minute timeout to await_thread_outcome to prevent
infinite hangs (e.g. after denied tool approval where thread fails
to resume).
Tests: 3 structured executor tests (NeedAuthentication interrupts batch,
stops subsequent calls, regular errors don't interrupt) + 7 effect
adapter tests (credential extraction, callback firing, v1-only tools).
Also adds Linear API skill (skills/linear/SKILL.md).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Rust-side skill selection was moved to the Python orchestrator in
7f87d179. This module had no production callers — only its own tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Skill selection was in Rust (SkillSelector in loop_engine.rs) — now it's
in the Python orchestrator where the self-improvement mission can evolve it.
Rust provides data access via two new host functions:
- __list_skills__() — loads DocType::Skill MemoryDocs from Store
- __record_skill_usage__(doc_id, success) — confidence tracking
Python orchestrator handles everything else:
- score_skill() — keyword/tag/confidence scoring (~40 lines)
- select_skills() — budget-aware top-N selection (~15 lines)
- format_skills() — XML block injection into system prompt (~20 lines)
- Injection at step 0 with active_skill_ids stored in state
Removed from Rust:
- SkillSelector field + builder on ExecutionLoop and ThreadManager
- format_skills_section() from prompt.rs
- Rust-side skill injection block in loop_engine.rs
- SkillSelector wiring in bridge/router.rs
E2E test updated: skills stored in TestStore, Python orchestrator
finds them via __list_skills__() and injects based on goal keywords.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Drop DocType::Playbook variant and all references — playbook extraction
mission was already renamed to skill extraction in the previous session.
Updates CLAUDE.md, architecture docs, context builder, retrieval weights,
mission comments, and store adapter path mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Skills can now declare API credentials in YAML frontmatter (SkillCredentialSpec,
SkillCredentialLocation, SkillOAuthConfig, ProviderRefreshStrategy). Valid specs
are registered into SharedCredentialRegistry at startup; the HttpTool auto-injects
credentials for matching hosts — same zero-exposure model as WASM tools.
HTTP tool security hardening:
- Block LLM-provided auth headers for hosts with registered credentials
- Return structured authentication_required error for missing credentials
- Strip sensitive response headers (Set-Cookie, WWW-Authenticate, Authorization)
- Scan response body through LeakDetector before returning to LLM
Mission capability leases: registered mission_create/list/fire/pause/resume/delete
as a "missions" capability so threads receive leases. Removed routine_* aliases
from effect adapter — descriptions mention "routine" for LLM intent mapping.
Includes 10 integration tests (tests/skill_credential_injection.rs) covering
the full pipeline: YAML parsing → validation → registry → HttpTool wiring →
per-user isolation.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Three major changes to the v2 engine:
1. **Consolidated action execution** — `handle_execute_action` in Rust is now
the single source of truth for lease lookup, policy check, lease consumption,
action execution, event emission, and ActionResult message recording. The
Python orchestrator no longer duplicates event/message logic. This fixes the
empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant
messages (Codex "No tool call found" error).
2. **Removed reflection system** — Deleted the per-thread reflection pipeline
(pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection,
enable_reflection config, and all 3 reflection event kinds. Learning is now
handled entirely by event-driven missions that fire selectively.
3. **Three learning missions** replace reflection:
- `self-improvement` — fires on trace issues (error diagnosis, prompt fixes)
- `playbook-extraction` — fires on successful 5+ step threads (reusable procedures)
- `conversation-insights` — fires every 5 threads per project (user preferences,
domain knowledge, workflow patterns)
Additional fixes:
- llm_query()/llm_query_batched() always include system message (Codex compat)
- handle_llm_complete adds assistant message with structured action_calls for
Tier 0 responses (prevents "No tool call found" errors)
- Gateway broadcasts without thread_id emit as Status events instead of being dropped
- Comprehensive tests for call_id propagation and trace analysis (17 new tests)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When LLM-generated code calls `mission_list()` or any tool function,
Monty's Python execution model first resolves the name (`mission_list`)
as a NameLookup before invoking it as a FunctionCall. The NameLookup
handler always returned Undefined, causing NameError before the function
call could dispatch to the effect executor.
Fix: before starting the Monty VM, collect all known tool names from
the effect executor's available_actions(). In the NameLookup handler,
if the name matches a known tool, return a MontyObject::Function stub
instead of Undefined. Monty then yields FunctionCall for the stub,
which dispatches to the normal tool execution pipeline.
This enables CodeAct code to call any registered tool as a Python
function: mission_list(), mission_create(), routine_list(), web_search(),
memory_search(), etc. — all without explicit imports or __execute_action__
boilerplate.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Fix structured executor not stamping call_id onto ActionResult — the
EffectExecutor trait doesn't receive call_id, so the structured executor
must copy it from the original ActionCall after execution. Empty call_id
caused OpenAI-compatible providers to reject the next LLM request with
"Invalid 'input[2].call_id': empty string".
Fix trace analyzer false positives:
- code_error check now only scans User-role code output messages
(prefixed with [stdout]/[stderr]/[code ]/Traceback), not System
prompt which contains example error text
- missing_tool_output check now recognizes ActionResult messages as
valid tool output (Tier 0 structured path)
- Add NotImplementedError to detected code error patterns
New trace checks:
- empty_call_id: detect ActionResult messages with missing/empty
call_id before they reach the LLM API (severity: Error)
- llm_error: extract LLM provider errors from Failed state reason
- orchestrator_error: extract orchestrator errors from Failed state
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Wire engine v2 into the full submission pipeline and expose threads,
projects, and missions through the web gateway REST API.
Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear
submissions to engine v2 when ENGINE_V2=true. Previously only UserInput
and ApprovalResponse were handled; all other control commands fell
through to disconnected v1 sessions.
Bridge query layer — add 11 read-only query functions and 6 DTO types
so gateway handlers can inspect engine state (threads, steps, events,
projects, missions) without direct access to the EngineState singleton.
Gateway endpoints — new /api/engine/* routes:
GET /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events
GET /projects, /projects/{id}
GET /missions, /missions/{id}
POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume
SSE events — add ThreadStateChanged, ChildThreadSpawned, and
MissionThreadSpawned AppEvent variants. Expand the bridge event mapper
to forward StateChanged and ChildSpawned engine events to the browser.
Engine crate — add ConversationManager::clear_conversation() for /new
and /clear commands.
Code quality — replace 10 .expect() calls with proper error returns,
remove dead AgentConfig.engine_v2 field, log silent init errors, fix
duplicate doc comment, improve fallthrough documentation.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add version lifecycle for the Python orchestrator:
- Failure tracking via MemoryDoc (orchestrator:failures)
- Auto-rollback: after 3 consecutive failures, skip the latest version
and fall back to previous (or compiled-in v0)
- Success resets the failure counter
- OrchestratorRollback event for observability
Update self-improvement Mission goal with Level 1.5 instructions for
orchestrator patches — the agent can now modify the execution loop
itself via memory_write with versioned orchestrator docs.
12 new tests: version selection (highest wins), rollback after failures,
rollback to default, failure counting/resetting, outcome parsing for
all 5 ThreadOutcome variants.
189 tests pass, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Increment step_count and track tokens in __emit_event__("step_completed")
so thread bookkeeping matches the old Rust loop behavior
- Remove double-counting of tokens in bootstrap (orchestrator handles it)
- Match nudge text to existing TOOL_INTENT_NUDGE constant
- Fix FINAL result propagation (use stored final_result, not VM return)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Replace the 900-line Rust execution loop with a ~80-line bootstrap
that loads and runs the versioned Python orchestrator via Monty VM.
The orchestrator Python code (orchestrator/default.py) is the v0
compiled-in version. Runtime versions can override it via MemoryDoc
storage (orchestrator:main with tag orchestrator_code).
Key fixes during switchover:
- Use ExtFunctionResult::NotFound for unknown functions so Monty
falls through to Python-defined functions (extract_final, etc.)
- Move helper function definitions above run_loop for Monty scoping
- Use FINAL result value (not VM return value) in Complete handler
- Rename 'final' variable to 'final_answer' to avoid Python keyword
Status: 171/177 tests pass. 6 remaining failures are step_count and
token tracking bookkeeping — the orchestrator manages these internally
but doesn't yet update the thread's counters via host functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add the orchestrator infrastructure for replacing the Rust execution
loop with versioned Python code. This commit adds the module and host
functions without switching over — the existing Rust loop is unchanged.
New files:
- orchestrator/default.py: v0 Python orchestrator (run_loop + helpers)
- executor/orchestrator.rs: host function dispatch, orchestrator
loading from Store with version selection, OrchestratorResult parsing
Host functions exposed to orchestrator Python via Monty suspension:
__llm_complete__, __execute_code_step__ (nested Monty VM),
__execute_action__, __check_signals__, __emit_event__,
__add_message__, __save_checkpoint__, __transition_to__,
__retrieve_docs__, __check_budget__, __get_actions__
Also makes json_to_monty, monty_to_json, monty_to_string pub(crate)
in scripting.rs for cross-module use.
Design doc: docs/plans/2026-03-25-python-orchestrator.md
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.
Key changes:
Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading
Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Missions are now callable from CodeAct Python code:
```python
# Create a daily briefing mission
result = mission_create(
name="Tech News",
goal="Daily AI/crypto/software news briefing",
cadence="0 9 * * *"
)
# List all missions
missions = mission_list()
# Manually fire a mission
mission_fire(id="...")
# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```
Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
"webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
(avoids circular dependency)
System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.
151 tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The model was answering "Suggested 45 improvements" as a brief text
summary from training data without actually searching or listing them.
The trace showed: no code block, no tool calls, no FINAL().
Prompt changes:
- Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with
plain text only." (was: "Always write code... plain text for brief
explanations")
- Rule 2 (NEW): "NEVER answer from memory or training data alone.
Always use tools to get real, current information before answering."
- Rule 3: FINAL answer "should be detailed and complete — not just a
summary like 'found 45 items'"
- Rule 8 (NEW): "Include the actual content in your FINAL() answer,
not just a count or summary. Users want to see the details."
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
INFO-level log output from background tasks (trace analysis, reflection)
corrupts the REPL terminal UI. The trace summary, issue warnings, and
reflection doc previews were printing mid-approval-card, breaking the
interactive display.
Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!.
Trace analysis and reflection results now only show when
RUST_LOG=ironclaw_engine=debug is set.
Also added logging discipline rule to global CLAUDE.md:
- info! → user-facing status the REPL intentionally renders
- debug! → internal diagnostics (traces, reflection, engine internals)
- Background tasks must NEVER use info! — it breaks the TUI
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Adds a complete approval flow that mirrors v1 behavior, using the
existing v1 security controls (Tool::requires_approval, auto-approve
sets, StatusUpdate::ApprovalNeeded).
## How it works
### Step 1: Tool blocked at execution
When the LLM's code calls a tool (e.g., `shell("ls")`):
1. EffectBridgeAdapter.execute_action() looks up the Tool object
2. Calls tool.requires_approval(¶ms) — returns ApprovalRequirement
3. If Always → EngineError::LeaseDenied (always blocks)
4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set,
returns EngineError::LeaseDenied
5. If Never → proceeds to execution
### Step 2: Engine returns NeedApproval
The LeaseDenied error propagates through:
- CodeAct path: becomes Python RuntimeError, code halts, thread returns
NeedApproval with action_name + parameters
- Structured path: same via ActionResult.is_error
### Step 3: Router stores pending approval
- PendingApproval { action_name, original_content } stored on EngineState
- StatusUpdate::ApprovalNeeded sent to channel (shows approval card in
CLI/web with tool name, parameters, yes/always/no buttons)
- Returns text: "Tool 'shell' requires approval. Reply yes/always/no."
### Step 4: User responds
handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2:
- 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes
original message (tool now passes the approval check on second run)
- 'always' → same + logs for session persistence
- 'no' → returns "Denied: tool was not executed."
### Key design choice
Instead of pausing/resuming mid-execution (which needs engine changes
to freeze/restore the Monty VM state), we auto-approve the tool and
re-run the full message. The EffectBridgeAdapter's auto_approved set
persists across runs, so the second execution passes immediately.
This trades one extra LLM call for zero engine modifications.
## Files changed
- src/bridge/router.rs: PendingApproval struct, handle_approval(),
NeedApproval → StatusUpdate::ApprovalNeeded conversion
- src/bridge/mod.rs: export handle_approval
- src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2
- src/bridge/effect_adapter.rs: fmt fixes
151 tests passing, clippy + fmt clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Add Mission type and MissionManager for recurring thread scheduling
- Add ReliabilityTracker for per-capability success/failure/latency tracking
- Add reflection executor that spawns CodeAct threads for post-completion reflection
- Extend PolicyEngine with provenance-aware taint checking (LLM-generated data
requires approval for financial/external-write effects)
- Extend Store trait with mission CRUD methods
- Add conversation surface tracking, compaction token fix, context memory injection
- Wire new modules through lib.rs re-exports and bridge adapters
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The check was looking for "[" + "result]" in System-role messages only,
but tool output metadata is added with patterns like "[shell result]"
and may appear in messages with any role. Changed to scan all messages
for " result]" or " error]" patterns regardless of role.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Panic: 'byte index 80 is not a char boundary; it is inside ''' when
tool output contained multi-byte UTF-8 characters (smart quotes from
web search results).
Fixed 4 unsafe byte-index slices:
- thread.rs:281: message preview &content[..80] → chars().take(80)
- loop_engine.rs:556: tool output &str[..4000] → chars().take(4000)
- loop_engine.rs:579: output tail &str[len-8000..] → chars().skip()
- scripting.rs:82: stdout tail &str[len-N..] → chars().skip()
All now use .chars().take() or .chars().skip() which respect character
boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on
user-supplied or external strings."
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Prompt templates moved from inline Rust strings to plain markdown files
at crates/ironclaw_engine/prompts/ for easy inspection and iteration:
- prompts/codeact_preamble.md — main instructions, special functions,
context variables, rules
- prompts/codeact_postamble.md — strategy section
Loaded at compile time via include_str!(), so no runtime file I/O.
Edit the .md files and rebuild to iterate on prompts.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The system prompt example used web_fetch(url="...") which doesn't exist
as a tool. The model learned from the example and tried web_fetch,
getting "Tool not found". Changed to web_search(query="...") which is
an actual registered tool.
Found via trace analysis — reflection pipeline correctly identified
this as a "Tool Name Correction" spec doc.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When code fails with NameError/UnboundLocalError (model trying to
access variables from a previous step), the error output now includes:
[HINT] Variables don't persist between code blocks. Use the `state`
dict to access data from previous steps. Available keys: ["web_search",
"last_return"]
This teaches the model to use `state["web_search"]` instead of `result`
after a NameError, reducing wasted steps from 3-4 to 1.
Also integrates RetrievalEngine into context building and ThreadManager:
- build_step_context() now accepts optional RetrievalEngine to inject
relevant memory docs (Lessons, Specs, Playbooks) into LLM context
- RetrievalEngine uses keyword matching with doc-type priority scoring
- Memory docs from reflection (Phase 4) now feed back into future threads
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Monty creates a fresh runtime per code step, so variables are lost
between steps. This caused the model to re-paste tool results from
system messages, wasting tokens.
Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that
accumulates across steps:
- Tool results stored by tool name: state["web_search"] = {results...}
- Return values stored: state["last_return"], state["step_0_return"]
- Injected as a `state` Python variable in each new MontyRun
Now the model can do:
Step 1: results = web_search(query="...") # tool result saved in state
Step 2: data = state["web_search"] # access previous result
summary = llm_query("summarize", str(data))
FINAL(summary)
System prompt updated to document the `state` variable.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
After every thread completes, ThreadManager now automatically runs:
1. Retrospective trace analysis (non-LLM, always):
- Detects 8 issue categories (tool errors, code errors, missing
outputs, excessive steps, hallucination risk, etc.)
- Logs issues at warn level when found
2. Trace file recording (when ENGINE_V2_TRACE=1):
- Writes full JSON trace to engine_trace_{timestamp}.json
3. LLM reflection (when enable_reflection=true):
- Calls reflection pipeline to produce Summary, Lesson, Issue docs
- Saves docs to store for future context retrieval
- Enabled by default in the bridge router
All three run inside the spawned tokio task after exec.run() completes,
before saving the final thread state. No external wiring needed.
Removed duplicate trace recording from the router — it's now handled
by ThreadManager automatically.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Enable with ENGINE_V2_TRACE=1 to get full execution traces and
automatic issue detection after each thread completes.
Trace recording (executor/trace.rs):
- build_trace(): captures full thread state — messages (with full
content), events, step count, token usage, detected issues
- write_trace(): writes JSON to engine_trace_{timestamp}.json
- log_trace_summary(): logs summary + issues at info/warn level
Retrospective analyzer detects 8 issue categories:
- thread_failure: thread ended in Failed state
- no_response: no assistant message generated
- tool_error: specific tool failures with error details
- code_error: Python errors (NameError, SyntaxError, etc.) in output
- missing_tool_output: tool results exist but not in system messages
- excessive_steps: >10 steps (may be stuck in loop)
- no_tools_used: single-step answer without tools (hallucination risk)
- mixed_mode: text responses without code blocks (prompt not followed)
Thread state now saved to store after execution completes (for trace
access after join_thread).
Usage:
ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run
# After each message: trace JSON + issue log in terminal
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The LLM was ignoring tool results and answering from training data
because the compact output metadata didn't include what tools returned.
Tool results lived only as ActionResult messages (role: Tool) which
some providers flatten or the model ignores.
Now the code step output includes:
- stdout from Python print() statements
- [tool_name result] with the actual output (truncated to 4K per tool)
- [tool_name error] for failed tools
- [return] for the code's return value
- Total output truncated to 8K chars to prevent context bloat
This ensures the model sees web_search results, API responses, etc.
in the next iteration and can reason about them instead of hallucinating.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Engine v2 now shows live progress in the CLI (and any channel):
- "Thinking..." when a step starts
- Tool name + success/error when actions execute
- "Processing results..." when a step completes
Implementation:
- ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256)
- ExecutionLoop.emit_event() writes to thread.events AND broadcasts
- ThreadManager.subscribe_events() returns a receiver
- Router uses tokio::select! to listen for events while waiting for
thread completion, forwarding them as StatusUpdate to the channel
This replaces the polling approach with zero-latency event streaming.
Agent.channels visibility widened to pub(crate) for bridge access.
102 tests passing, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Models sometimes write FINAL() outside code blocks — as plain text
after an explanation. The Hyperliquid case: model outputs a long
analysis then FINAL("""...""") at the end, not inside ```repl fences.
Fixes:
- extract_final_from_text(): regex-based FINAL detection in text
responses, matching the official RLM's find_final_answer() fallback
- Handles: double-quoted, single-quoted, triple-quoted, unquoted,
nested parens
- Checked in LlmResponse::Text handler BEFORE tool intent nudge
(FINAL takes priority)
9 new tests:
- codeact_final_in_text_response: FINAL("answer") in plain text
- codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text
- final_double_quoted, final_single_quoted, final_triple_quoted,
final_unquoted, final_with_nested_parens, final_after_long_text,
no_final_returns_none
102 tests passing (93 + 9 new), zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Comprehensive test coverage for the Monty Python execution path:
- codeact_simple_final: Python code calls FINAL('answer') → thread completes
- codeact_tool_call_then_final: code calls test_tool() → FunctionCall
suspends VM → MockEffects returns result → code resumes → FINAL()
- codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15')
with no tool calls — pure Python in Monty
- codeact_multi_step: first step prints output (no FINAL), second step
sees output metadata and calls FINAL — tests iterative REPL flow
- codeact_error_recovery: first step has NameError → error flows to LLM
as stdout → second step recovers with FINAL — tests error transparency
- codeact_context_variables_available: code accesses `goal` and `context`
variables injected by the RLM context builder
- codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times
→ 3 FunctionCall suspensions → all results collected → FINAL
- codeact_llm_query_recursive: code calls llm_query('prompt') → VM
suspends → MockLlm provides sub-agent response → result returned as
Python string variable
93 tests passing (85 prior + 8 new), zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The engine now operates in CodeAct/RLM mode:
System prompt (executor/prompt.rs):
- Instructs LLM to write Python in ```repl fenced blocks
- Documents available tools as callable Python functions
- Documents llm_query(), llm_query_batched(), FINAL()
- Documents context variables (context, goal, step_number, previous_results)
- Strategy guidance: examine context, break into steps, use tools, call FINAL()
Code block detection (bridge/llm_adapter.rs):
- extract_code_block() scans LLM text responses for ```repl or ```python blocks
- When detected, returns LlmResponse::Code instead of LlmResponse::Text
- The ExecutionLoop routes Code responses through Monty for execution
No structured tool definitions sent to LLM:
- Tools are described in the system prompt as Python functions
- The LLM call sends empty actions array, forcing text-mode responses
- This ensures the LLM writes code blocks (CodeAct) instead of
structured tool calls (which would bypass the REPL)
85 tests passing, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The engine was creating a fresh ThreadManager and InMemoryStore per
message, losing all context between turns. A follow-up question like
"what are the latest 10 issues?" had no memory of the prior "how many
issues" response.
Fixes:
- EngineState (ThreadManager, ConversationManager, InMemoryStore) now
persists across messages via OnceLock, initialized on first use
- ConversationManager builds message history from prior conversation
entries (user messages + agent responses) and passes it to new threads
- ThreadManager.spawn_thread_with_history() accepts initial_messages
that are prepended before the current user message
- System notifications (thread started/completed) are filtered out of
the history (not useful as LLM context)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The ExecutionLoop was sending empty messages to the LLM because the
thread was spawned with the user's input as the goal but no messages.
Fixes:
- ThreadManager.spawn_thread() now adds the goal as an initial user
message before starting the execution loop
- ExecutionLoop.run() injects a default system prompt if none exists
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Restructure phases 6-8 to clarify execution model:
- Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker
Python runtimes for LLM-generated code.
- WASM sandbox is for third-party tool isolation (existing infra, Phase 8)
- Docker containers are for thread-level isolation of high-risk work (Phase 8)
- Two-phase commit moves to Phase 6 (integration) at the adapter boundary
Phase renumbering:
- Old Phase 6 (Tier 2-3) → removed as separate phase
- Old Phase 7 (integration) → Phase 6
- Old Phase 8 (cleanup) → Phase 7
- New Phase 8: WASM tools + Docker thread isolation (infra integration)
Updated progress table: Phases 1-5 marked DONE with test counts and commits.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
