Comprehensive test coverage for the Monty Python execution path:
- codeact_simple_final: Python code calls FINAL('answer') → thread completes
- codeact_tool_call_then_final: code calls test_tool() → FunctionCall
suspends VM → MockEffects returns result → code resumes → FINAL()
- codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15')
with no tool calls — pure Python in Monty
- codeact_multi_step: first step prints output (no FINAL), second step
sees output metadata and calls FINAL — tests iterative REPL flow
- codeact_error_recovery: first step has NameError → error flows to LLM
as stdout → second step recovers with FINAL — tests error transparency
- codeact_context_variables_available: code accesses `goal` and `context`
variables injected by the RLM context builder
- codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times
→ 3 FunctionCall suspensions → all results collected → FINAL
- codeact_llm_query_recursive: code calls llm_query('prompt') → VM
suspends → MockLlm provides sub-agent response → result returned as
Python string variable
93 tests passing (85 prior + 8 new), zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The engine now operates in CodeAct/RLM mode:
System prompt (executor/prompt.rs):
- Instructs LLM to write Python in ```repl fenced blocks
- Documents available tools as callable Python functions
- Documents llm_query(), llm_query_batched(), FINAL()
- Documents context variables (context, goal, step_number, previous_results)
- Strategy guidance: examine context, break into steps, use tools, call FINAL()
Code block detection (bridge/llm_adapter.rs):
- extract_code_block() scans LLM text responses for ```repl or ```python blocks
- When detected, returns LlmResponse::Code instead of LlmResponse::Text
- The ExecutionLoop routes Code responses through Monty for execution
No structured tool definitions sent to LLM:
- Tools are described in the system prompt as Python functions
- The LLM call sends empty actions array, forcing text-mode responses
- This ensures the LLM writes code blocks (CodeAct) instead of
structured tool calls (which would bypass the REPL)
85 tests passing, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The engine was creating a fresh ThreadManager and InMemoryStore per
message, losing all context between turns. A follow-up question like
"what are the latest 10 issues?" had no memory of the prior "how many
issues" response.
Fixes:
- EngineState (ThreadManager, ConversationManager, InMemoryStore) now
persists across messages via OnceLock, initialized on first use
- ConversationManager builds message history from prior conversation
entries (user messages + agent responses) and passes it to new threads
- ThreadManager.spawn_thread_with_history() accepts initial_messages
that are prepended before the current user message
- System notifications (thread started/completed) are filtered out of
the history (not useful as LLM context)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The ExecutionLoop was sending empty messages to the LLM because the
thread was spawned with the user's input as the goal but no messages.
Fixes:
- ThreadManager.spawn_thread() now adds the goal as an initial user
message before starting the execution loop
- ExecutionLoop.run() injects a default system prompt if none exists
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Restructure phases 6-8 to clarify execution model:
- Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker
Python runtimes for LLM-generated code.
- WASM sandbox is for third-party tool isolation (existing infra, Phase 8)
- Docker containers are for thread-level isolation of high-risk work (Phase 8)
- Two-phase commit moves to Phase 6 (integration) at the adapter boundary
Phase renumbering:
- Old Phase 6 (Tier 2-3) → removed as separate phase
- Old Phase 7 (integration) → Phase 6
- Old Phase 8 (cleanup) → Phase 7
- New Phase 8: WASM tools + Docker thread isolation (infra integration)
Updated progress table: Phases 1-5 marked DONE with test counts and commits.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Conversation is now a UI layer, not an execution boundary. Multiple
threads can run concurrently within one conversation; threads can
outlive their originating conversation.
New types (types/conversation.rs):
- ConversationSurface: channel + user + entries + active_threads
- ConversationEntry: sender (User/Agent/System) + content + origin_thread_id
- ConversationId, EntryId (UUID newtypes)
- EntrySender enum (User, Agent{thread_id}, System)
ConversationManager (runtime/conversation.rs):
- get_or_create_conversation(channel, user) — indexed by (channel, user)
- handle_user_message() — injects into active foreground thread or spawns new
- record_thread_outcome() — adds agent/system entries, untracks completed threads
- get_conversation(), list_conversations()
This enables the key architectural insight: a user can ask "what's the
weather?" while a deployment thread is still running. Both produce entries
in the same conversation.
85 tests passing, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Cross-referenced our implementation against the official RLM (alexzhang13/rlm),
fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation.
Key enhancements:
- FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching
all three reference implementations. Code can signal completion at any
point, not just via return value.
- llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn,
matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch.
- Output truncation increased to 8000 chars (from 120), matching Prime
Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT].
- Step 0 orientation preamble: auto-injects context metadata (message count,
total chars, goal, last user message preview) before first code step,
matching fast-rlm's auto-print pattern.
- Error-to-LLM flow: Python parse errors, runtime errors, NameErrors,
OS errors, and async errors now flow back as stdout content instead of
terminating the step, enabling LLM self-correction on next iteration.
Only VM panics (catch_unwind) terminate as EngineError.
74 tests passing, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* perf(safety): make XML attribute escaping single-pass
* test(safety): annotate assertion for no-panics CI
* test(safety): inline no-panics suppression comment
These tests guard against catastrophic regex backtracking (seconds/minutes),
not 12ms differences. CI runners with coverage instrumentation (cargo-llvm-cov)
consistently exceed the 100ms threshold due to overhead, causing flaky failures.
500ms still catches real regressions while tolerating CI variability.
[skip-regression-check]
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: eliminate panic paths in production code and document infallible operations
PolicyRule::new() now returns Result instead of panicking on invalid
caller-supplied regex. CreateJobTool returns ToolError when job_manager
is unconfigured instead of panicking. Remaining infallible unwrap/expect
calls (hardcoded regexes, compile-time constants, guarded accesses)
are annotated with SAFETY comments. Where possible, unwraps are replaced
with safer patterns: split_last(), if-let, match-destructure, and
reusing peek() values.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: use inline lowercase safety comments to match CI pattern
The no-panics CI check greps for '// safety:' (lowercase, inline)
to suppress false positives. Switch from block SAFETY comments to
inline safety comments on the .unwrap() lines.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* test: add regression tests for panic-path fixes
- PolicyRule::new returns Err on invalid regex (not panic)
- CreateJobTool::execute_sandbox returns ToolError when job_manager is None
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: add inline // safety: comments on all infallible unwrap/expect lines
The CI no-panics check requires '// safety:' on the same line as
unwrap()/expect() to suppress false positives. Move safety annotations
from block comments to inline comments on every infallible production
unwrap/expect across all touched files.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* chore: trigger CI with skip-regression-check label
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: remove redundant block-level SAFETY comments
Each unwrap/expect now carries its own inline // safety: annotation,
making the standalone block comments above them redundant.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: extract safety module into ironclaw_safety crate
Move prompt injection defense, input validation, secret leak detection,
and safety policy enforcement into a standalone crate under crates/.
The safety module was a leaf dependency with no async, no database, and
no other ironclaw traits — only pure computation with pattern matching.
SafetyConfig (2 fields) moves into the crate; env-var resolution stays
in ironclaw's config module as a free function. src/safety/mod.rs becomes
a thin re-export so all existing `crate::safety::*` imports keep working.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* docs: update CLAUDE.md for ironclaw_safety crate extraction
Add guidance to migrate imports from crate::safety to ironclaw_safety
when touching files. Update project structure to reflect crates/ dir.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* refactor: move safety fuzz targets into ironclaw_safety crate
Split fuzz infrastructure:
- crates/ironclaw_safety/fuzz/ — 5 safety-only targets (sanitizer,
validator, leak_detector, credential_detect, config_env) depending
only on ironclaw_safety for faster builds
- fuzz/ — keeps fuzz_tool_params which needs ironclaw::tools
Add seed corpus files (51 total) covering each pattern family:
sanitizer injection patterns, validator edge cases, leak detector
secret formats, credential detect HTTP param shapes.
Add new fuzz_credential_detect target exercising
params_contain_manual_credentials with arbitrary JSON.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address PR review — single-pass XML escaping and versioned path dep
Rewrite escape_xml_attr from chained .replace() to single-pass char
iteration (O(n) instead of O(4n) with intermediate allocations). Add
version = "0.1.0" to ironclaw_safety path dep to satisfy cargo-deny
wildcards = "deny".
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
