Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.
Key changes:
Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading
Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When the model calls routine_create, routine_list, routine_fire,
routine_pause, routine_resume, or routine_delete, the bridge now
routes them to the MissionManager instead of blocking with an error.
Mapping:
routine_create → mission_create (with cadence parsing)
routine_list → mission_list
routine_fire → mission_fire
routine_pause → mission_pause
routine_resume → mission_resume
routine_update → mission_pause/resume (based on params)
routine_delete → mission_complete (marks as done)
Routine tools removed from v1-only blocklist and restored in
available_actions(). The model can use either "routine" or "mission"
vocabulary — both work.
Still blocked: create_job, cancel_job, build_software (need v1
Scheduler/ContainerJobManager refs).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Missions are now callable from CodeAct Python code:
```python
# Create a daily briefing mission
result = mission_create(
name="Tech News",
goal="Daily AI/crypto/software news briefing",
cadence="0 9 * * *"
)
# List all missions
missions = mission_list()
# Manually fire a mission
mission_fire(id="...")
# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```
Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
"webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
(avoids circular dependency)
System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.
151 tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Three changes to make engine v2 visible in the web gateway:
1. SSE event streaming (AppEvent broadcast):
- ThreadEvent → AppEvent conversion via thread_event_to_app_event()
- Events broadcast to SseManager during the poll loop
- Covers: Thinking, ToolCompleted (success/error), Status, Response
- Web gateway receives real-time progress without any gateway changes
2. Conversation persistence to v1 database:
- After thread completes, writes user message + agent response to
v1 ConversationStore via add_conversation_message()
- Uses get_or_create_assistant_conversation() for per-user per-channel
- Web gateway reads from DB as usual — chat history appears
3. Final response broadcast:
- AppEvent::Response with full text + thread_id sent via SSE
- Web gateway renders the response in the chat UI
New EngineState fields: sse (Option<Arc<SseManager>>),
db (Option<Arc<dyn Database>>). Both populated from Agent.deps.
Agent.deps visibility widened to pub(crate).
Depends on: ironclaw_common crate with AppEvent type (PR #1615).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Merges refactor/extract-app-event-to-ironclaw-common which extracts
SseEvent into crates/ironclaw_common as AppEvent. This is the
prerequisite for engine v2 gateway integration — the bridge can now
emit AppEvents without depending on web gateway types.
Conflict resolution: workspace members includes all three crates
(ironclaw_common, ironclaw_safety, ironclaw_engine).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
SseEvent was defined in src/channels/web/types.rs but imported by 12+
modules across agent, orchestrator, worker, tools, and extensions — it
had become the application-wide event protocol, not a web transport
concern.
Create crates/ironclaw_common as a shared workspace crate and move the
enum there as AppEvent. Also move the truncate_preview utility which
was similarly leaked from the web gateway into agent modules.
- New crate: crates/ironclaw_common (AppEvent, truncate_preview)
- Rename SseEvent → AppEvent, from_sse_event → from_app_event
- web/types.rs re-exports AppEvent for internal gateway use
- web/util.rs re-exports truncate_preview
- Wire format unchanged (serde renames are on variants, not the enum)
Aligned with the event bus direction on refactor/architectural-hardening
where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Routines are entirely v1 — not hooked up to engine v2. When a user
asks "create a routine" as natural language, engine v2 tries to call
routine_create via CodeAct, but the tool needs RoutineEngine + Database
refs that the bridge's minimal JobContext doesn't provide. This caused
a SIGKILL crash during testing.
Options documented: block routine tools in v2 (short term), pass refs
through context (medium), replace with Mission system (long term).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Documents three gaps between engine v2 and the web gateway:
1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent)
2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB)
3. No cross-channel visibility (REPL ↔ web messages invisible to each other)
Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1
conversation tables after thread completion. Prerequisite: AppEvent
extraction PR (in progress separately).
Also updated DB persistence status: HybridStore with workspace-backed
MemoryDocs is now implemented (partial persistence).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Staging merge changed execute_tool_with_safety to take params by value
instead of by reference (perf optimization from PR #926). Updated
bridge adapter to clone params before passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(agent): persist /model selection to .env, TOML, and DB
The /model command only wrote selected_model to the DB and config.toml,
but env vars from ~/.ironclaw/.env (e.g. NEARAI_MODEL) have the highest
priority in LlmConfig::resolve_model(). The .env value was never
updated, so it always shadowed the new model on restart.
Now persist_selected_model updates all three persistence layers:
1. The backend-specific model env var in ~/.ironclaw/.env (only if the
var already exists, to avoid injecting new vars)
2. The config.toml file (created if absent, since TOML > DB priority)
3. The DB settings table (for completeness)
Also adds diagnostic logging when the DB store is unavailable.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(agent): address PR review — backend from deps, exact .env match
Review feedback:
- Use resolved llm_backend from AgentDeps instead of re-reading from
disk/env (fixes DB-only backend detection, eliminates redundant I/O)
- Match .env var with exact "KEY=" prefix and skip commented lines
(prevents false matches on NEARAI_MODEL_VERSION etc.)
- TOML is now loaded once (no double-read for backend + model update)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Replaces InMemoryStore with HybridStore:
- Ephemeral data (threads, steps, events, leases) stays in-memory
- MemoryDocs (lessons, specs, playbooks from reflection) persist to
the workspace at engine/docs/{type}/{id}.json
On engine init, load_docs_from_workspace() reads existing docs back
into the in-memory cache. This means:
- Lessons learned in session 1 are available in session 2
- The RetrievalEngine injects relevant past lessons into new threads
- The engine genuinely improves over time as reflection accumulates
Workspace paths:
engine/docs/lessons/{uuid}.json
engine/docs/specs/{uuid}.json
engine/docs/playbooks/{uuid}.json
engine/docs/summaries/{uuid}.json
engine/docs/issues/{uuid}.json
No new database tables. Uses existing workspace write/read/list.
workspace() accessor widened to pub(crate).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(tools): add missing description, parameters, and improve credential prompts
Silence three categories of startup warnings emitted by
CapabilitiesFile::validate() and WasmToolLoader:
1. "description" field missing → add tool descriptions to all manifests
2. "parameters" field missing → add action-enum parameter schemas
3. Short credential prompts (<30 chars) → append source URLs
Affects: github, gmail, google-calendar, google-docs, google-drive,
google-sheets, google-slides, slack, telegram, llm-context, feishu.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor(tools): auto-compact WASM tool schemas from module exports
Replace the manual `parameters` field in capabilities JSON with automatic
schema compaction. WasmToolSchemas::compact_schema() derives a compact
advertised schema from the WASM module's schema() export by keeping only
required and enum-constrained properties. The full schema remains
available via tool_info(detail: "schema").
This eliminates:
- The `parameters` field from CapabilitiesFile and all 11 sidecar JSONs
- The "missing parameters" startup warning from the loader
- Manual maintenance of duplicate schema data
The `description` field in capabilities JSON is retained.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(tests): remove cap_file.parameters reference in test_rig
The parameters field was removed from CapabilitiesFile in the previous
commit. Update test_rig.rs to match — schema is now auto-compacted from
the WASM module export, no sidecar override needed.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(tools): handle oneOf schemas in compact_schema, add tool name to warning
Address PR review feedback:
- compact_schema now collects properties from oneOf/anyOf/allOf variants,
fixing GitHub-style schemas that have no top-level properties
- Use HashSet for required lookup instead of Vec::contains
- Add tool name to "Capabilities file not found" warning for consistency
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(tools): merge oneOf const values into enum, cap property collection
Address review feedback from @serrrfirat:
1. Merge const values across oneOf variants into a single enum array,
so the LLM sees all valid actions (not just the first variant's const).
2. Cap property collection at 100 to bound allocations.
3. Also keep properties with const constraint (single-variant case).
4. Update doc comment to describe variant collection and design choices
around variant-level required fields.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
The model was answering "Suggested 45 improvements" as a brief text
summary from training data without actually searching or listing them.
The trace showed: no code block, no tool calls, no FINAL().
Prompt changes:
- Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with
plain text only." (was: "Always write code... plain text for brief
explanations")
- Rule 2 (NEW): "NEVER answer from memory or training data alone.
Always use tools to get real, current information before answering."
- Rule 3: FINAL answer "should be detailed and complete — not just a
summary like 'found 45 items'"
- Rule 8 (NEW): "Include the actual content in your FINAL() answer,
not just a count or summary. Users want to see the details."
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The leak detector's Warn-action matches (high_entropy_hex pattern on
web search results containing commit SHAs, CSS colors, URL hashes)
were logging at warn! level, corrupting the REPL UI with lines like:
WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5
These are informational false positives — real leaks use LeakAction::Redact
which silently modifies the content. Warn-action matches only log for
debugging purposes and should not appear in production output.
Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
"engine v2: initializing" and "engine v2: handling message" were
printing at INFO level, corrupting the REPL UI. All router logging
now uses debug! — only visible with RUST_LOG=ironclaw=debug.
Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
INFO-level log output from background tasks (trace analysis, reflection)
corrupts the REPL terminal UI. The trace summary, issue warnings, and
reflection doc previews were printing mid-approval-card, breaking the
interactive display.
Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!.
Trace analysis and reflection results now only show when
RUST_LOG=ironclaw_engine=debug is set.
Also added logging discipline rule to global CLAUDE.md:
- info! → user-facing status the REPL intentionally renders
- debug! → internal diagnostics (traces, reflection, engine internals)
- Background tasks must NEVER use info! — it breaks the TUI
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Adds a complete approval flow that mirrors v1 behavior, using the
existing v1 security controls (Tool::requires_approval, auto-approve
sets, StatusUpdate::ApprovalNeeded).
## How it works
### Step 1: Tool blocked at execution
When the LLM's code calls a tool (e.g., `shell("ls")`):
1. EffectBridgeAdapter.execute_action() looks up the Tool object
2. Calls tool.requires_approval(¶ms) — returns ApprovalRequirement
3. If Always → EngineError::LeaseDenied (always blocks)
4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set,
returns EngineError::LeaseDenied
5. If Never → proceeds to execution
### Step 2: Engine returns NeedApproval
The LeaseDenied error propagates through:
- CodeAct path: becomes Python RuntimeError, code halts, thread returns
NeedApproval with action_name + parameters
- Structured path: same via ActionResult.is_error
### Step 3: Router stores pending approval
- PendingApproval { action_name, original_content } stored on EngineState
- StatusUpdate::ApprovalNeeded sent to channel (shows approval card in
CLI/web with tool name, parameters, yes/always/no buttons)
- Returns text: "Tool 'shell' requires approval. Reply yes/always/no."
### Step 4: User responds
handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2:
- 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes
original message (tool now passes the approval check on second run)
- 'always' → same + logs for session persistence
- 'no' → returns "Denied: tool was not executed."
### Key design choice
Instead of pausing/resuming mid-execution (which needs engine changes
to freeze/restore the Monty VM state), we auto-approve the tool and
re-run the full message. The EffectBridgeAdapter's auto_approved set
persists across runs, so the second execution passes immediately.
This trades one extra LLM call for zero engine modifications.
## Files changed
- src/bridge/router.rs: PendingApproval struct, handle_approval(),
NeedApproval → StatusUpdate::ApprovalNeeded conversion
- src/bridge/mod.rs: export handle_approval
- src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2
- src/bridge/effect_adapter.rs: fmt fixes
151 tests passing, clippy + fmt clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat: multi-tenant auth with per-user scoping
Multi-user authentication and authorization for IronClaw gateway:
- Token-based auth mapping tokens to user IDs via GATEWAY_USER_TOKENS
- Per-user SSE broadcast scoping
- Per-user rate limiting with poisoned lock recovery
- Handler auth and ownership checks for jobs, settings, routines
- Extension secrets scoped per-user
- Chat handlers use authenticated identity
- Reverse proxy deployment documentation
- Comprehensive integration tests for auth, SSE, rate limiting, and job isolation
* fix: scope memory tools per-user in multi-tenant mode
Memory tools (search, write, read, tree) held a single workspace
created at startup with GATEWAY_USER_ID. In multi-tenant mode, all
users' tool calls searched the default user's scope.
Add WorkspaceResolver trait that resolves workspaces per-request using
JobContext.user_id. In single-user mode, returns the startup workspace.
In multi-tenant mode (GATEWAY_USER_TOKENS configured), creates and
caches per-user workspaces on demand.
Includes regression tests for workspace resolution and user isolation.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: comprehensive multi-tenant isolation audit
Address all review findings from @serrrfirat plus 7 additional gaps
found via full security audit:
Reviewer findings (5):
- WorkspacePool now applies search config, memory layers, embedding
cache, identity read scopes, and global config scopes (was bare)
- jobs_summary_handler uses per-user queries instead of global counters
- jobs_prompt_handler restructured to not 404 agent jobs + ownership check
- jobs_restart_handler agent branch now verifies user ownership
- agent_job_summary_for_user added to Database trait + both backends
Audit findings (7):
- Delete dead handlers/memory.rs (stale copies with no auth)
- Add AuthenticatedUser to logs_events, logs_level_get, logs_level_set
- Add AuthenticatedUser to extensions_tools_handler, gateway_status_handler
- Add auth + ownership checks to all 6 routines handlers
- Add auth to all 4 skills handlers with audit logging on mutations
- Scope extension setup SSE broadcast to user (broadcast_for_user)
- Fix pre-existing test compilation errors in extensions/manager.rs
17 new multi-tenant isolation tests covering:
- WorkspacePool config propagation and scope merging
- Jobs handler per-user isolation (summary, restart, prompt, cancel)
- Routines handler auth enforcement and cross-user rejection
- Auth middleware enforcement on logs, skills, status endpoints
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: second-pass multi-tenant audit — scope SSE broadcasts, DB queries, dead handlers
Second audit pass applying learned patterns across the codebase:
- OAuth callback SSE broadcasts now use broadcast_for_user (lines 773, 912)
- jobs_list_handler uses list_agent_jobs_for_user instead of fetching
all users' jobs and filtering in Rust
- list_agent_jobs_for_user added to Database trait + postgres + libsql
- Dead handler files (extensions.rs, static_files.rs) hardened with
AuthenticatedUser to prevent auth regression if migrated
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address review findings — token hashing, broadcast scoping, error handling
Security fixes:
- Hash tokens with SHA-256 at construction time so authentication
compares fixed-size 32-byte digests, eliminating length-oracle
timing leaks
- Scope auth SSE broadcasts per-user in chat_auth_token_handler —
AuthRequired/AuthCompleted events were leaking across tenants
- Propagate DB errors in restart handlers instead of silently
swallowing via `if let Ok(Some(...))` pattern
Code quality:
- Log SSE serialization failures instead of silently producing empty
strings via unwrap_or_default()
- Remove dead `pub type AuthState = MultiAuthState` alias
- Replace `.unwrap()` with `Arc::clone(db)` in app.rs multi-tenant
workspace setup (db is guaranteed Some in context, but unwrap
violates project convention)
- Fix telegram setup test to inject UserIdentity into request
extensions (handler now requires AuthenticatedUser)
- Add safety comments on test-only expect/unwrap calls for CI
- Apply cargo fmt to fix pre-existing formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address review findings — unify workspace pool, fix SSE regression, cache job owners
- Unify WorkspacePool and PerUserWorkspaceResolver: WorkspacePool now
implements WorkspaceResolver, eliminating duplicate per-user workspace
construction logic. app.rs uses WorkspacePool directly.
- Fix sse_tx: None scheduler regression: change scheduler/worker SSE
broadcasting from broadcast::Sender<SseEvent> to Arc<SseManager>,
restoring SSE event delivery for scheduled agent jobs.
- Cache job owner in orchestrator: add job_owner_cache to
OrchestratorState so job_event_handler avoids a DB round-trip on
every event after the first per job.
- Deduplicate ext_user_id computation in main.rs.
- Remove unused _gateway_state variable.
- Fix pre-existing test: first_token() returns None in multi-user mode
by design; align test assertion.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fix formatting in app.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: extract memory handlers back into handlers/memory.rs
Move memory API handlers out of server.rs into their own module,
consistent with how jobs, routines, and skills handlers are organized.
The resolve_workspace() helper moves with them since it is only used
by memory handlers.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Zero engine crate changes. All security controls enforced at the bridge
boundary in EffectBridgeAdapter:
1. Tool approval (v1: Tool::requires_approval):
- Checks each tool's approval requirement with actual params
- Always → returns EngineError::LeaseDenied (blocks execution)
- UnlessAutoApproved → checks auto_approved set, blocks if not approved
- Never → proceeds
- Per-session auto_approved HashSet (for future "always" handling)
2. Hook interception (v1: BeforeToolCall):
- Runs HookEvent::ToolCall before every execution
- HookOutcome::Reject → blocks with reason
- HookError::Rejected → blocks with reason
- Hook errors → fail-open (logged, execution continues)
3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm):
- Leak detection: API keys in tool output are redacted
- Policy enforcement: content policy rules applied
- Length truncation: output capped at 100KB
- XML boundary protection: prevents injection via tool output
4. Sensitive param redaction (v1: redact_params):
- Tool's sensitive_params() consulted before hooks see parameters
- Redacted params sent to hooks, original params used for execution
5. available_actions() now sets requires_approval based on each tool's
default approval requirement, so the engine's PolicyEngine can
gate tools it hasn't seen before.
6. Actual execution timing measured via Instant::now() (replaces
placeholder Duration::from_millis(1)).
Accessor visibility: hooks() widened to pub(crate).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Add Mission type and MissionManager for recurring thread scheduling
- Add ReliabilityTracker for per-capability success/failure/latency tracking
- Add reflection executor that spawns CodeAct threads for post-completion reflection
- Extend PolicyEngine with provenance-aware taint checking (LLM-generated data
requires approval for financial/external-write effects)
- Extend Store trait with mission CRUD methods
- Add conversation surface tracking, compaction token fix, context memory injection
- Wire new modules through lib.rs re-exports and bridge adapters
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Updated security plan with detailed audit of ALL existing v1 security
controls and how they map to engine v2 bridge gaps:
Key finding: v1 already has solutions for every security gap identified.
The bridge just needs to wire them in:
- Tool::requires_approval() exists but bridge doesn't call it
- safety.wrap_for_llm() exists but tool results enter context unwrapped
- RateLimiter exists but bridge doesn't check rate limits
- BeforeToolCall hooks exist but bridge doesn't run them
- redact_params() exists but bridge doesn't redact sensitive params
- Shell risk classification (Low/Medium/High) is inherited but ignored
Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter,
not new security infrastructure. The bridge is the security boundary.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Designs a system where the engine debugs and improves itself, based on
the pattern observed in the last session: 5 consecutive bug fixes all
followed trace → read → identify → edit → test, using tools the engine
already has access to.
Three levels of self-improvement:
- Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply.
- Level 2 (Config): adjust defaults/mappings. Branch + test + PR.
- Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR.
Architecture: Self-improvement Mission spawns a Reflection thread that
reads traces, reads source, proposes fixes, validates via cargo test,
and either auto-applies (Level 1) or creates a PR (Level 2-3).
Includes: fix pattern database (seeded from our 8 debugging session
fixes), feedback loop diagram, safety model, implementation phases
(A through D), and what exists vs what's new.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The check was looking for "[" + "result]" in System-role messages only,
but tool output metadata is added with patterns like "[shell result]"
and may appear in messages with any role. Changed to scan all messages
for " result]" or " error]" patterns regardless of role.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Panic: 'byte index 80 is not a char boundary; it is inside ''' when
tool output contained multi-byte UTF-8 characters (smart quotes from
web search results).
Fixed 4 unsafe byte-index slices:
- thread.rs:281: message preview &content[..80] → chars().take(80)
- loop_engine.rs:556: tool output &str[..4000] → chars().take(4000)
- loop_engine.rs:579: output tail &str[len-8000..] → chars().skip()
- scripting.rs:82: stdout tail &str[len-N..] → chars().skip()
All now use .chars().take() or .chars().skip() which respect character
boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on
user-supplied or external strings."
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Prompt templates moved from inline Rust strings to plain markdown files
at crates/ironclaw_engine/prompts/ for easy inspection and iteration:
- prompts/codeact_preamble.md — main instructions, special functions,
context variables, rules
- prompts/codeact_postamble.md — strategy section
Loaded at compile time via include_str!(), so no runtime file I/O.
Edit the .md files and rebuild to iterate on prompts.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The system prompt example used web_fetch(url="...") which doesn't exist
as a tool. The model learned from the example and tried web_fetch,
getting "Tool not found". Changed to web_search(query="...") which is
an actual registered tool.
Found via trace analysis — reflection pipeline correctly identified
this as a "Tool Name Correction" spec doc.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* Default new lightweight routines to tools-enabled
* Fix fmt and clippy on lightweight routine PR
* Use grouped execution field in routine no-tools fixture
* Align CLI routine defaults with tools-enabled lightweight mode
* feat(cli): add ironclaw models subcommands (list/status/set/set-provider)
Implements model management CLI (part of #83):
- `models list [provider] [--verbose] [--json]` — list providers; fetches
live model list from the provider API when a specific provider is given
- `models status [--json]` — show active provider/model
- `models set <model>` — set default model with validation
- `models set-provider <id> [--model <name>]` — set provider with alias
normalization
- fix conflicts
* fix(deps): update tar to 0.4.45 (RUSTSEC-2026-0067, RUSTSEC-2026-0068)
---------
Co-authored-by: firat.sertgoz <[email protected]>
When code fails with NameError/UnboundLocalError (model trying to
access variables from a previous step), the error output now includes:
[HINT] Variables don't persist between code blocks. Use the `state`
dict to access data from previous steps. Available keys: ["web_search",
"last_return"]
This teaches the model to use `state["web_search"]` instead of `result`
after a NameError, reducing wasted steps from 3-4 to 1.
Also integrates RetrievalEngine into context building and ThreadManager:
- build_step_context() now accepts optional RetrievalEngine to inject
relevant memory docs (Lessons, Specs, Playbooks) into LLM context
- RetrievalEngine uses keyword matching with doc-type priority scoring
- Memory docs from reflection (Phase 4) now feed back into future threads
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Monty creates a fresh runtime per code step, so variables are lost
between steps. This caused the model to re-paste tool results from
system messages, wasting tokens.
Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that
accumulates across steps:
- Tool results stored by tool name: state["web_search"] = {results...}
- Return values stored: state["last_return"], state["step_0_return"]
- Injected as a `state` Python variable in each new MontyRun
Now the model can do:
Step 1: results = web_search(query="...") # tool result saved in state
Step 2: data = state["web_search"] # access previous result
summary = llm_query("summarize", str(data))
FINAL(summary)
System prompt updated to document the `state` variable.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
From trace analysis: web_search returned a JSON string, which was
wrapped as serde_json::json!(string) creating a Value::String containing
JSON. When Monty got this as MontyObject::String, the Python code
couldn't index it with result['title'] → TypeError.
Fix: try parsing the tool output string as JSON first. If valid, use the
parsed Value (becomes a Python dict/list). If not valid JSON, keep as
string. This means web_search results are directly indexable in Python:
results = web_search(query="...")
print(results["results"][0]["title"]) # works now
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Root cause from trace analysis: the LLM writes `web_search()` (valid
Python identifier) but the tool registry has `web-search` (with hyphen).
The EffectBridgeAdapter couldn't find the tool → "Tool not found" error
→ model fabricated fake data instead.
Fixes:
- available_actions(): converts tool names from hyphens to underscores
(web-search → web_search) so the system prompt lists valid Python names
- execute_action(): tries the original name first, then falls back to
hyphenated form (web_search → web-search) for tool registry lookup
- Same conversion in router's capability registry builder
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat(workspace): multi-scope workspace reads
Adds the ability for a workspace to read from multiple user scopes
while keeping writes isolated to the primary scope. Configuration
via WORKSPACE_READ_SCOPES env var (comma-separated user IDs).
Includes identity file isolation (read_primary), multi-scope search,
list, and read operations, WorkspaceConfig refactor, and comprehensive
integration tests.
* fix: address review feedback for multi-scope workspace reads
- fix(memory): deduplicate timezone parsing for daily_log target
parse_timezone was called twice when target was "daily_log" without a
layer — once in path resolution, again in the fallback. Now computed
once and reused.
- fix(config): add character validation for WORKSPACE_READ_SCOPES and
layer scopes — both enforce [a-zA-Z0-9_-] to prevent path traversal
or injection via scope strings used as user_id in SQL queries.
- fix(config): use chars().take(32) instead of byte-index slicing for
scope length error messages (UTF-8 safety).
- fix(error): remove unused WorkspaceError::NotFound variant
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: downgrade search log to debug, add comments on list iteration
- Downgrade hybrid_search_multi tracing::info! to debug! — fires on
every multi-scope search with the default backend, too noisy for info
- Add comments explaining why list/list_all iterate per-scope instead
of using _multi trait methods (identity path filtering needs scope
attribution that merged results lose)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
After every thread completes, ThreadManager now automatically runs:
1. Retrospective trace analysis (non-LLM, always):
- Detects 8 issue categories (tool errors, code errors, missing
outputs, excessive steps, hallucination risk, etc.)
- Logs issues at warn level when found
2. Trace file recording (when ENGINE_V2_TRACE=1):
- Writes full JSON trace to engine_trace_{timestamp}.json
3. LLM reflection (when enable_reflection=true):
- Calls reflection pipeline to produce Summary, Lesson, Issue docs
- Saves docs to store for future context retrieval
- Enabled by default in the bridge router
All three run inside the spawned tokio task after exec.run() completes,
before saving the final thread state. No external wiring needed.
Removed duplicate trace recording from the router — it's now handled
by ThreadManager automatically.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Enable with ENGINE_V2_TRACE=1 to get full execution traces and
automatic issue detection after each thread completes.
Trace recording (executor/trace.rs):
- build_trace(): captures full thread state — messages (with full
content), events, step count, token usage, detected issues
- write_trace(): writes JSON to engine_trace_{timestamp}.json
- log_trace_summary(): logs summary + issues at info/warn level
Retrospective analyzer detects 8 issue categories:
- thread_failure: thread ended in Failed state
- no_response: no assistant message generated
- tool_error: specific tool failures with error details
- code_error: Python errors (NameError, SyntaxError, etc.) in output
- missing_tool_output: tool results exist but not in system messages
- excessive_steps: >10 steps (may be stuck in loop)
- no_tools_used: single-step answer without tools (hallucination risk)
- mixed_mode: text responses without code blocks (prompt not followed)
Thread state now saved to store after execution completes (for trace
access after join_thread).
Usage:
ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run
# After each message: trace JSON + issue log in terminal
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The LLM was ignoring tool results and answering from training data
because the compact output metadata didn't include what tools returned.
Tool results lived only as ActionResult messages (role: Tool) which
some providers flatten or the model ignores.
Now the code step output includes:
- stdout from Python print() statements
- [tool_name result] with the actual output (truncated to 4K per tool)
- [tool_name error] for failed tools
- [return] for the code's return value
- Total output truncated to 8K chars to prevent context bloat
This ensures the model sees web_search results, API responses, etc.
in the next iteration and can reason about them instead of hallucinating.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
