test: add 29 E2E trace tests for issues #571-575 (#593)

* test: add 29 E2E trace tests for worker, threading, tools, workspace, and routines (#571-575)

Add comprehensive E2E test coverage across five test files:
- e2e_worker_coverage (7 tests): parallel tool calls, error feedback, unknown tools,
  invalid params, rate limiting, iteration limits, planning mode
- e2e_thread_scheduling (3 tests + 2 deferred): multi-turn state, undo/redo, concurrent dispatch
- e2e_builtin_tool_coverage (8 tests): time parse/diff/invalid, routine CRUD/history,
  job create/status/list/cancel, HTTP replay
- e2e_workspace_coverage (6 tests): chunked search, multi-doc search, hybrid search,
  directory tree, document lifecycle, identity in system prompt
- e2e_routine_heartbeat (5 tests): cron triggers, event matching, cooldown enforcement,
  heartbeat findings, empty checklist skip

Infrastructure: extend TestRig with database/workspace/trace_llm accessors, register
job and routine tools by default, add with_extra_tools() for custom stub tools.

Includes 24 JSON trace fixtures across worker/, threading/, tools/, and workspace/.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use 6-field cron format in routine_create_list fixture

The cron 0.13 crate accepts both 6 and 7 fields, but the routine_create
tool documents 6-field format. Align the fixture to match.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: eliminate vacuous passes and silently-skipped assertions in E2E tests

- job_create_status: replace job_status (needs dynamic UUID) with list_jobs,
  assert both succeed via completed() not just started()
- job_list_cancel: keep cancel_job but explicitly assert it fails with
  invalid canned job_id "latest", verify create_job + list_jobs succeed
- unknown_tool_name: add !is_empty() guard before .all() to prevent
  vacuous pass on empty iterator
- workspace tests: change `if let Some(ws)` to `.expect()` so assertions
  are never silently skipped when workspace/trace_llm is available

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add template substitution to TraceLlm for dynamic tool result forwarding

Add {{call_id.json_path}} template syntax to trace fixtures, enabling
tool results from one step to flow into subsequent steps' arguments.
TraceLlm extracts variables from Role::Tool messages (stripping the
safety layer's <tool_output> XML wrapper and unescaping entities) and
substitutes them in canned tool_call arguments before returning.

This fixes job_create_status and job_list_cancel tests to properly test
job_status and cancel_job with real dynamic UUIDs from create_job,
instead of using invalid canned IDs that silently failed.

Also adds tool result content assertions to job_create_status to verify
the actual tool output contains expected data (job_id, title).

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on E2E tests

- undo_redo_cycle: assert exactly 3 turns instead of >= 2
- tool_error_feedback: use tempfile::tempdir() instead of hardcoded /tmp path,
  patch fixture path at runtime for CI portability
- worker_timeout → iteration_limit: rename to accurately describe what's tested
- post_plan_work_remaining → simple_echo_flow: rename, test doesn't exercise planning
- identity_in_system_prompt: seed IDENTITY.md before test, assert system prompt
  contains the seeded content instead of just checking Role::System exists

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: strengthen workspace E2E test assertions per PR review

- write_chunk_search: assert memory_search was called and returned
  payment/architecture-related results
- multi_document_search: assert memory_search was called for
  cross-document search
- hybrid_search_with_embeddings: assert both memory_write and
  memory_search were called to confirm write-then-search pipeline
- directory_tree: assert tree output contains expected alpha/beta
  project paths

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

tests/e2e_builtin_tool_coverage.rs

@@ -0,0 +1,332 @@
+//! E2E trace tests: builtin tool coverage (#573).
+//!
+//! Covers time (parse, diff, invalid), routine (create, list, update, delete,
+//! history), job (create, status, list, cancel), and HTTP replay.
+
+#[cfg(feature = "libsql")]
+mod support;
+
+#[cfg(feature = "libsql")]
+mod tests {
+    use std::time::Duration;
+
+    use crate::support::test_rig::TestRigBuilder;
+    use crate::support::trace_llm::LlmTrace;
+
+    // -----------------------------------------------------------------------
+    // Test 1: time_parse_and_diff
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn time_parse_and_diff() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/time_parse_diff.json"
+        ))
+        .expect("failed to load time_parse_diff.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Parse a time and compute a diff").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Time tool should have been called twice (parse + diff).
+        let started = rig.tool_calls_started();
+        let time_count = started.iter().filter(|n| n.as_str() == "time").count();
+        assert!(
+            time_count >= 2,
+            "Expected >= 2 time tool calls, got {time_count}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 2: time_parse_invalid
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn time_parse_invalid() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/time_parse_invalid.json"
+        ))
+        .expect("failed to load time_parse_invalid.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Parse an invalid timestamp").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // The time tool call should have failed (invalid timestamp).
+        let completed = rig.tool_calls_completed();
+        let time_results: Vec<_> = completed
+            .iter()
+            .filter(|(name, _)| name == "time")
+            .collect();
+        assert!(!time_results.is_empty(), "Expected time tool to be called");
+        assert!(
+            time_results.iter().any(|(_, ok)| !ok),
+            "Expected at least one failed time call: {time_results:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 3: routine_create_list
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn routine_create_list() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/routine_create_list.json"
+        ))
+        .expect("failed to load routine_create_list.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Create a daily routine and list all routines")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Both routine_create and routine_list should have succeeded.
+        let completed = rig.tool_calls_completed();
+        assert!(
+            completed.iter().any(|(n, ok)| n == "routine_create" && *ok),
+            "routine_create should succeed: {completed:?}"
+        );
+        assert!(
+            completed.iter().any(|(n, ok)| n == "routine_list" && *ok),
+            "routine_list should succeed: {completed:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 4: routine_update_delete
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn routine_update_delete() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/routine_update_delete.json"
+        ))
+        .expect("failed to load routine_update_delete.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Create, update, and delete a routine")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"routine_create".to_string()),
+            "routine_create not started"
+        );
+        assert!(
+            started.contains(&"routine_update".to_string()),
+            "routine_update not started"
+        );
+        assert!(
+            started.contains(&"routine_delete".to_string()),
+            "routine_delete not started"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 5: routine_history
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn routine_history() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/routine_history.json"
+        ))
+        .expect("failed to load routine_history.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Create a routine and check its history")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"routine_create".to_string()),
+            "routine_create missing"
+        );
+        assert!(
+            started.contains(&"routine_history".to_string()),
+            "routine_history missing"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 6: job_create_status
+    // -----------------------------------------------------------------------
+    // Uses {{call_cj_1.job_id}} template to forward the dynamic UUID from
+    // create_job's result into job_status's arguments.
+
+    #[tokio::test]
+    async fn job_create_status() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/job_create_status.json"
+        ))
+        .expect("failed to load job_create_status.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Create a job and check its status").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Both tools should have succeeded.
+        let completed = rig.tool_calls_completed();
+        assert!(
+            completed.iter().any(|(n, ok)| n == "create_job" && *ok),
+            "create_job should succeed: {completed:?}"
+        );
+        assert!(
+            completed.iter().any(|(n, ok)| n == "job_status" && *ok),
+            "job_status should succeed: {completed:?}"
+        );
+
+        // Verify tool results contain expected content.
+        let results = rig.tool_results();
+        let create_result = results
+            .iter()
+            .find(|(n, _)| n == "create_job")
+            .expect("create_job result missing");
+        assert!(
+            create_result.1.contains("job_id"),
+            "create_job should return a job_id: {:?}",
+            create_result.1
+        );
+        let status_result = results
+            .iter()
+            .find(|(n, _)| n == "job_status")
+            .expect("job_status result missing");
+        assert!(
+            status_result.1.contains("Test analysis job"),
+            "job_status should return the job title: {:?}",
+            status_result.1
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 7: job_list_cancel
+    // -----------------------------------------------------------------------
+    // Uses {{call_cj_lc.job_id}} template to forward the dynamic UUID from
+    // create_job into cancel_job.
+
+    #[tokio::test]
+    async fn job_list_cancel() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/job_list_cancel.json"
+        ))
+        .expect("failed to load job_list_cancel.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Create a job, list jobs, then cancel it")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // All three tools should have succeeded.
+        let completed = rig.tool_calls_completed();
+        assert!(
+            completed.iter().any(|(n, ok)| n == "create_job" && *ok),
+            "create_job should succeed: {completed:?}"
+        );
+        assert!(
+            completed.iter().any(|(n, ok)| n == "list_jobs" && *ok),
+            "list_jobs should succeed: {completed:?}"
+        );
+        assert!(
+            completed.iter().any(|(n, ok)| n == "cancel_job" && *ok),
+            "cancel_job should succeed: {completed:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 8: http_get_with_replay
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn http_get_with_replay() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/tools/http_get_replay.json"
+        ))
+        .expect("failed to load http_get_replay.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Make an http GET request").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // HTTP tool should have succeeded with the replayed exchange.
+        let completed = rig.tool_calls_completed();
+        assert!(
+            completed.iter().any(|(n, ok)| n == "http" && *ok),
+            "http tool should succeed: {completed:?}"
+        );
+
+        rig.shutdown();
+    }
+}

tests/e2e_routine_heartbeat.rs

@@ -0,0 +1,416 @@
+//! E2E tests: routine engine and heartbeat (#575).
+//!
+//! These tests construct RoutineEngine and HeartbeatRunner directly
+//! with a TraceLlm and libSQL database, bypassing the full TestRig.
+
+#[cfg(feature = "libsql")]
+mod support;
+
+#[cfg(feature = "libsql")]
+mod tests {
+    use std::sync::Arc;
+    use std::time::Duration;
+
+    use chrono::Utc;
+    use uuid::Uuid;
+
+    use ironclaw::agent::routine::{
+        NotifyConfig, Routine, RoutineAction, RoutineGuardrails, Trigger,
+    };
+    use ironclaw::agent::routine_engine::RoutineEngine;
+    use ironclaw::agent::{HeartbeatConfig, HeartbeatRunner};
+    use ironclaw::channels::IncomingMessage;
+    use ironclaw::config::{RoutineConfig, SafetyConfig};
+    use ironclaw::db::Database;
+    use ironclaw::safety::SafetyLayer;
+    use ironclaw::workspace::Workspace;
+    use ironclaw::workspace::hygiene::HygieneConfig;
+
+    use crate::support::trace_llm::{LlmTrace, TraceLlm, TraceResponse, TraceStep};
+
+    /// Create a temp libSQL database with migrations applied.
+    async fn create_test_db() -> (Arc<dyn Database>, tempfile::TempDir) {
+        use ironclaw::db::libsql::LibSqlBackend;
+
+        let temp_dir = tempfile::tempdir().expect("tempdir");
+        let db_path = temp_dir.path().join("test.db");
+        let backend = LibSqlBackend::new_local(&db_path)
+            .await
+            .expect("LibSqlBackend");
+        backend.run_migrations().await.expect("migrations");
+        let db: Arc<dyn Database> = Arc::new(backend);
+        (db, temp_dir)
+    }
+
+    /// Create a workspace backed by the test database.
+    fn create_workspace(db: &Arc<dyn Database>) -> Arc<Workspace> {
+        Arc::new(Workspace::new_with_db("default", db.clone()))
+    }
+
+    /// Helper to insert a routine directly into the database.
+    fn make_routine(name: &str, trigger: Trigger, prompt: &str) -> Routine {
+        Routine {
+            id: Uuid::new_v4(),
+            name: name.to_string(),
+            description: format!("Test routine: {name}"),
+            user_id: "default".to_string(),
+            enabled: true,
+            trigger,
+            action: RoutineAction::Lightweight {
+                prompt: prompt.to_string(),
+                context_paths: vec![],
+                max_tokens: 1000,
+            },
+            guardrails: RoutineGuardrails {
+                cooldown: Duration::from_secs(0),
+                max_concurrent: 5,
+                dedup_window: None,
+            },
+            notify: NotifyConfig::default(),
+            last_run_at: None,
+            next_fire_at: None,
+            run_count: 0,
+            consecutive_failures: 0,
+            state: serde_json::json!({}),
+            created_at: Utc::now(),
+            updated_at: Utc::now(),
+        }
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 1: cron_routine_fires
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn cron_routine_fires() {
+        let (db, _tmp) = create_test_db().await;
+        let ws = create_workspace(&db);
+
+        // Create a TraceLlm that responds with ROUTINE_OK.
+        let trace = LlmTrace::single_turn(
+            "test-cron-fire",
+            "check",
+            vec![TraceStep {
+                request_hint: None,
+                response: TraceResponse::Text {
+                    content: "ROUTINE_OK".to_string(),
+                    input_tokens: 50,
+                    output_tokens: 5,
+                },
+                expected_tool_results: vec![],
+            }],
+        );
+        let llm = Arc::new(TraceLlm::from_trace(trace));
+
+        let (notify_tx, mut notify_rx) = tokio::sync::mpsc::channel(16);
+
+        let engine = Arc::new(RoutineEngine::new(
+            RoutineConfig::default(),
+            db.clone(),
+            llm,
+            ws,
+            notify_tx,
+            None,
+        ));
+
+        // Insert a cron routine with next_fire_at in the past.
+        let mut routine = make_routine(
+            "cron-test",
+            Trigger::Cron {
+                schedule: "* * * * *".to_string(),
+            },
+            "Check system status.",
+        );
+        routine.next_fire_at = Some(Utc::now() - chrono::Duration::minutes(5));
+        db.create_routine(&routine).await.expect("create_routine");
+
+        // Fire cron triggers.
+        engine.check_cron_triggers().await;
+
+        // Give the spawned task time to execute.
+        tokio::time::sleep(Duration::from_millis(500)).await;
+
+        // Verify a run was recorded.
+        let runs = db
+            .list_routine_runs(routine.id, 10)
+            .await
+            .expect("list_routine_runs");
+        assert!(
+            !runs.is_empty(),
+            "Expected at least one routine run after cron trigger"
+        );
+
+        // Notification may or may not be sent depending on config;
+        // just verify no panic occurred. Drain the channel.
+        let _ = notify_rx.try_recv();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 2: event_trigger_matches
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn event_trigger_matches() {
+        let (db, _tmp) = create_test_db().await;
+        let ws = create_workspace(&db);
+
+        let trace = LlmTrace::single_turn(
+            "test-event-match",
+            "deploy",
+            vec![TraceStep {
+                request_hint: None,
+                response: TraceResponse::Text {
+                    content: "Deployment detected".to_string(),
+                    input_tokens: 50,
+                    output_tokens: 10,
+                },
+                expected_tool_results: vec![],
+            }],
+        );
+        let llm = Arc::new(TraceLlm::from_trace(trace));
+        let (notify_tx, _notify_rx) = tokio::sync::mpsc::channel(16);
+
+        let engine = Arc::new(RoutineEngine::new(
+            RoutineConfig::default(),
+            db.clone(),
+            llm,
+            ws,
+            notify_tx,
+            None,
+        ));
+
+        // Insert an event routine matching "deploy.*production".
+        let routine = make_routine(
+            "deploy-watcher",
+            Trigger::Event {
+                channel: None,
+                pattern: "deploy.*production".to_string(),
+            },
+            "Report on deployment.",
+        );
+        db.create_routine(&routine).await.expect("create_routine");
+
+        // Refresh the event cache so the engine knows about the routine.
+        engine.refresh_event_cache().await;
+
+        // Positive match: message containing "deploy to production".
+        let matching_msg = IncomingMessage {
+            id: Uuid::new_v4(),
+            channel: "test".to_string(),
+            user_id: "default".to_string(),
+            user_name: None,
+            content: "deploy to production now".to_string(),
+            thread_id: None,
+            received_at: Utc::now(),
+            metadata: serde_json::json!({}),
+        };
+        let fired = engine.check_event_triggers(&matching_msg).await;
+        assert!(
+            fired >= 1,
+            "Expected >= 1 routine fired on match, got {fired}"
+        );
+
+        // Give spawn time.
+        tokio::time::sleep(Duration::from_millis(500)).await;
+
+        // Negative match: message that doesn't match.
+        let non_matching_msg = IncomingMessage {
+            id: Uuid::new_v4(),
+            channel: "test".to_string(),
+            user_id: "default".to_string(),
+            user_name: None,
+            content: "check the staging environment".to_string(),
+            thread_id: None,
+            received_at: Utc::now(),
+            metadata: serde_json::json!({}),
+        };
+        let fired_neg = engine.check_event_triggers(&non_matching_msg).await;
+        assert_eq!(fired_neg, 0, "Expected 0 routines fired on non-match");
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 3: routine_cooldown
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn routine_cooldown() {
+        let (db, _tmp) = create_test_db().await;
+        let ws = create_workspace(&db);
+
+        // Need two LLM responses (one for the first fire).
+        let trace = LlmTrace::single_turn(
+            "test-cooldown",
+            "check",
+            vec![TraceStep {
+                request_hint: None,
+                response: TraceResponse::Text {
+                    content: "ROUTINE_OK".to_string(),
+                    input_tokens: 50,
+                    output_tokens: 5,
+                },
+                expected_tool_results: vec![],
+            }],
+        );
+        let llm = Arc::new(TraceLlm::from_trace(trace));
+        let (notify_tx, _notify_rx) = tokio::sync::mpsc::channel(16);
+
+        let engine = Arc::new(RoutineEngine::new(
+            RoutineConfig::default(),
+            db.clone(),
+            llm,
+            ws,
+            notify_tx,
+            None,
+        ));
+
+        // Insert an event routine with 1-hour cooldown.
+        let mut routine = make_routine(
+            "cooldown-test",
+            Trigger::Event {
+                channel: None,
+                pattern: "test-cooldown".to_string(),
+            },
+            "Check status.",
+        );
+        routine.guardrails.cooldown = Duration::from_secs(3600);
+        db.create_routine(&routine).await.expect("create_routine");
+        engine.refresh_event_cache().await;
+
+        // First fire should work.
+        let msg = IncomingMessage {
+            id: Uuid::new_v4(),
+            channel: "test".to_string(),
+            user_id: "default".to_string(),
+            user_name: None,
+            content: "test-cooldown trigger".to_string(),
+            thread_id: None,
+            received_at: Utc::now(),
+            metadata: serde_json::json!({}),
+        };
+        let fired1 = engine.check_event_triggers(&msg).await;
+        assert!(fired1 >= 1, "First fire should work");
+
+        // Give spawn time, then update last_run_at to simulate recent execution.
+        tokio::time::sleep(Duration::from_millis(300)).await;
+
+        // Update the routine's last_run_at to now (simulating it just ran).
+        db.update_routine_runtime(routine.id, Utc::now(), None, 1, 0, &serde_json::json!({}))
+            .await
+            .expect("update_routine_runtime");
+
+        // Refresh cache to pick up updated last_run_at.
+        engine.refresh_event_cache().await;
+
+        // Second fire should be blocked by cooldown.
+        let fired2 = engine.check_event_triggers(&msg).await;
+        assert_eq!(fired2, 0, "Second fire should be blocked by cooldown");
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 4: heartbeat_findings
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn heartbeat_findings() {
+        let (db, _tmp) = create_test_db().await;
+        let ws = create_workspace(&db);
+
+        // Write a real heartbeat checklist.
+        ws.write(
+            "HEARTBEAT.md",
+            "# Heartbeat Checklist\n\n- [ ] Check if the server is running\n- [ ] Review error logs",
+        )
+        .await
+        .expect("write heartbeat");
+
+        // LLM responds with findings (not HEARTBEAT_OK).
+        let trace = LlmTrace::single_turn(
+            "test-heartbeat-findings",
+            "heartbeat",
+            vec![TraceStep {
+                request_hint: None,
+                response: TraceResponse::Text {
+                    content: "The server has elevated error rates. Review the logs immediately."
+                        .to_string(),
+                    input_tokens: 100,
+                    output_tokens: 20,
+                },
+                expected_tool_results: vec![],
+            }],
+        );
+        let llm = Arc::new(TraceLlm::from_trace(trace));
+        let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
+            max_output_length: 100_000,
+            injection_check_enabled: false,
+        }));
+
+        let (tx, mut rx) = tokio::sync::mpsc::channel(16);
+
+        let hygiene_config = HygieneConfig {
+            enabled: false,
+            retention_days: 30,
+            cadence_hours: 24,
+            state_dir: _tmp.path().to_path_buf(),
+        };
+
+        let runner =
+            HeartbeatRunner::new(HeartbeatConfig::default(), hygiene_config, ws, llm, safety)
+                .with_response_channel(tx);
+
+        let result = runner.check_heartbeat().await;
+        match result {
+            ironclaw::agent::HeartbeatResult::NeedsAttention(msg) => {
+                assert!(
+                    msg.contains("error"),
+                    "Expected 'error' in attention message: {msg}"
+                );
+            }
+            other => panic!("Expected NeedsAttention, got: {other:?}"),
+        }
+
+        // No notification since we called check_heartbeat directly (not run).
+        let _ = rx.try_recv();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 5: heartbeat_empty_skip
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn heartbeat_empty_skip() {
+        let (db, _tmp) = create_test_db().await;
+        let ws = create_workspace(&db);
+
+        // Write an effectively empty heartbeat (just headers and comments).
+        ws.write(
+            "HEARTBEAT.md",
+            "# Heartbeat Checklist\n\n<!-- No tasks yet -->\n",
+        )
+        .await
+        .expect("write heartbeat");
+
+        // LLM should NOT be called, so provide a trace that would panic if called.
+        let trace = LlmTrace::single_turn("test-heartbeat-skip", "skip", vec![]);
+        let llm = Arc::new(TraceLlm::from_trace(trace));
+        let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
+            max_output_length: 100_000,
+            injection_check_enabled: false,
+        }));
+
+        let hygiene_config = HygieneConfig {
+            enabled: false,
+            retention_days: 30,
+            cadence_hours: 24,
+            state_dir: _tmp.path().to_path_buf(),
+        };
+
+        let runner =
+            HeartbeatRunner::new(HeartbeatConfig::default(), hygiene_config, ws, llm, safety);
+
+        let result = runner.check_heartbeat().await;
+        assert!(
+            matches!(result, ironclaw::agent::HeartbeatResult::Skipped),
+            "Expected Skipped for empty checklist, got: {result:?}"
+        );
+    }
+}

tests/e2e_thread_scheduling.rs

@@ -0,0 +1,155 @@
+//! E2E trace tests: thread/scheduler operations (#572).
+//!
+//! Covers multi-turn state persistence, undo/redo, and concurrent dispatch.
+//! Tests for thread_interruption and max_parallel_exceeded are deferred.
+
+#[cfg(feature = "libsql")]
+mod support;
+
+#[cfg(feature = "libsql")]
+mod tests {
+    use std::time::Duration;
+
+    use crate::support::test_rig::TestRigBuilder;
+    use crate::support::trace_llm::LlmTrace;
+
+    // -----------------------------------------------------------------------
+    // Test 1: multi_turn_state
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn multi_turn_state() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/threading/multi_turn_state.json"
+        ))
+        .expect("failed to load multi_turn_state.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        let all_responses = rig
+            .run_and_verify_trace(&trace, Duration::from_secs(30))
+            .await;
+
+        // Should have 3 turns of responses.
+        assert_eq!(
+            all_responses.len(),
+            3,
+            "Expected 3 turns, got {}",
+            all_responses.len()
+        );
+
+        // Verify memory tools were used across turns.
+        let started = rig.tool_calls_started();
+        let mw_count = started
+            .iter()
+            .filter(|n| n.as_str() == "memory_write")
+            .count();
+        let ms_count = started
+            .iter()
+            .filter(|n| n.as_str() == "memory_search")
+            .count();
+        assert!(
+            mw_count >= 2,
+            "Expected >= 2 memory_write calls: {started:?}"
+        );
+        assert!(
+            ms_count >= 1,
+            "Expected >= 1 memory_search calls: {started:?}"
+        );
+
+        // Verify DB is accessible (conversation persistence is tested by
+        // the agent's internal session management).
+        let _db = rig.database();
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 2: thread_interruption -- DEFERRED
+    // -----------------------------------------------------------------------
+    // Needs interrupt signaling infrastructure in TestChannel.
+
+    // -----------------------------------------------------------------------
+    // Test 3: undo_redo_cycle
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn undo_redo_cycle() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/threading/undo_redo.json"
+        ))
+        .expect("failed to load undo_redo.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        let all_responses = rig
+            .run_and_verify_trace(&trace, Duration::from_secs(30))
+            .await;
+
+        // Should get responses for all 3 turns (echo, /undo, /redo).
+        assert_eq!(
+            all_responses.len(),
+            3,
+            "Expected 3 turn responses, got {}",
+            all_responses.len()
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 4: concurrent_dispatch
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn concurrent_dispatch() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/threading/concurrent_dispatch.json"
+        ))
+        .expect("failed to load concurrent_dispatch.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        let all_responses = rig
+            .run_and_verify_trace(&trace, Duration::from_secs(30))
+            .await;
+
+        // Should have 2 turns.
+        assert_eq!(
+            all_responses.len(),
+            2,
+            "Expected 2 turns, got {}",
+            all_responses.len()
+        );
+
+        // Both echo calls should have succeeded.
+        let completed = rig.tool_calls_completed();
+        let echo_successes = completed
+            .iter()
+            .filter(|(name, ok)| name == "echo" && *ok)
+            .count();
+        assert!(
+            echo_successes >= 2,
+            "Expected >= 2 successful echo calls: {completed:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 5: max_parallel_exceeded -- DEFERRED
+    // -----------------------------------------------------------------------
+    // Needs max_parallel config exposed through TestRigBuilder.
+}

tests/e2e_worker_coverage.rs

@@ -0,0 +1,325 @@
+//! E2E trace tests: worker execution paths (#571).
+//!
+//! Covers parallel tool calls, error feedback loops, unknown tools,
+//! invalid parameters, rate limiting, iteration limits, and planning mode.
+
+#[cfg(feature = "libsql")]
+mod support;
+
+#[cfg(feature = "libsql")]
+mod tests {
+    use std::sync::Arc;
+    use std::time::Duration;
+
+    use async_trait::async_trait;
+    use serde_json::json;
+
+    use ironclaw::context::JobContext;
+    use ironclaw::tools::{Tool, ToolError, ToolOutput};
+
+    use crate::support::test_rig::TestRigBuilder;
+    use crate::support::trace_llm::LlmTrace;
+
+    // -- Stub tools for rate-limit and timeout tests --------------------------
+
+    /// A tool that always returns RateLimited.
+    struct StubRateLimitTool;
+
+    #[async_trait]
+    impl Tool for StubRateLimitTool {
+        fn name(&self) -> &str {
+            "stub_rate_limit"
+        }
+        fn description(&self) -> &str {
+            "Always returns rate limited error"
+        }
+        fn parameters_schema(&self) -> serde_json::Value {
+            json!({ "type": "object", "properties": {} })
+        }
+        async fn execute(
+            &self,
+            _params: serde_json::Value,
+            _ctx: &JobContext,
+        ) -> Result<ToolOutput, ToolError> {
+            Err(ToolError::RateLimited(Some(Duration::from_secs(60))))
+        }
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 1: parallel_three_tools
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn parallel_three_tools() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/parallel_three_tools.json"
+        ))
+        .expect("failed to load parallel_three_tools.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Run three tools in parallel").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify all three tools were started.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"echo".to_string()),
+            "echo not started: {started:?}"
+        );
+        assert!(
+            started.contains(&"time".to_string()),
+            "time not started: {started:?}"
+        );
+        assert!(
+            started.contains(&"json".to_string()),
+            "json not started: {started:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 2: tool_error_feedback
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn tool_error_feedback() {
+        // Use a tempdir for the recovery file. The fixture's recovery path
+        // is updated to write here via the test_dir variable.
+        let tmp = tempfile::tempdir().expect("create temp dir");
+        let test_dir = tmp.path().to_str().expect("tempdir path");
+
+        // Patch the fixture's recovery path to use our tempdir.
+        let fixture_str = std::fs::read_to_string(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/tool_error_feedback.json"
+        ))
+        .expect("read fixture");
+        let fixture_str = fixture_str.replace(
+            "/tmp/ironclaw_error_feedback_test/recovered.txt",
+            &format!("{test_dir}/recovered.txt"),
+        );
+        let trace: LlmTrace = serde_json::from_str(&fixture_str).expect("parse patched fixture");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write a file to a bad path then recover")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify the recovery file exists in the tempdir.
+        let content = std::fs::read_to_string(format!("{test_dir}/recovered.txt"))
+            .expect("recovered.txt should exist");
+        assert!(
+            content.contains("recovered"),
+            "Expected 'recovered' in file, got: {content:?}"
+        );
+
+        // At least one tool call should have failed (the bad path).
+        let completed = rig.tool_calls_completed();
+        let failures: Vec<_> = completed.iter().filter(|(_, ok)| !ok).collect();
+        assert!(
+            !failures.is_empty(),
+            "Expected at least one failed tool call, got: {completed:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 3: unknown_tool_name
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn unknown_tool_name() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/unknown_tool.json"
+        ))
+        .expect("failed to load unknown_tool.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Deploy to production").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // The deploy_to_production tool should have been attempted but failed.
+        let completed = rig.tool_calls_completed();
+        let deploy_results: Vec<_> = completed
+            .iter()
+            .filter(|(name, _)| name == "deploy_to_production")
+            .collect();
+        assert!(
+            !deploy_results.is_empty(),
+            "deploy_to_production should have been attempted: {completed:?}"
+        );
+        assert!(
+            deploy_results.iter().all(|(_, ok)| !ok),
+            "deploy_to_production should fail: {deploy_results:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 4: invalid_tool_params
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn invalid_tool_params() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/invalid_params.json"
+        ))
+        .expect("failed to load invalid_params.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Echo something with wrong params first")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Echo should have been called at least twice (bad then good).
+        let started = rig.tool_calls_started();
+        let echo_count = started.iter().filter(|n| n.as_str() == "echo").count();
+        assert!(
+            echo_count >= 2,
+            "Expected >= 2 echo calls, got {echo_count}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 5: rate_limit_cascade
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn rate_limit_cascade() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/rate_limit_cascade.json"
+        ))
+        .expect("failed to load rate_limit_cascade.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .with_extra_tools(vec![Arc::new(StubRateLimitTool) as Arc<dyn Tool>])
+            .build()
+            .await;
+
+        rig.send_message("Call the rate limited tool").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Both calls should have failed due to rate limiting.
+        let completed = rig.tool_calls_completed();
+        let rl_calls: Vec<_> = completed
+            .iter()
+            .filter(|(name, _)| name == "stub_rate_limit")
+            .collect();
+        assert!(
+            !rl_calls.is_empty(),
+            "Expected stub_rate_limit calls: {completed:?}"
+        );
+        assert!(
+            rl_calls.iter().all(|(_, ok)| !ok),
+            "All stub_rate_limit calls should fail: {rl_calls:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 6: iteration_limit
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn iteration_limit() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/worker_timeout.json"
+        ))
+        .expect("failed to load worker_timeout.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .with_max_tool_iterations(2)
+            .build()
+            .await;
+
+        rig.send_message("Keep calling tools until the limit").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        // We should still get a response even with iteration limit.
+        assert!(
+            !responses.is_empty(),
+            "Expected at least one response with iteration limit"
+        );
+
+        // Metrics should show we hit the iteration limit.
+        let metrics = rig.collect_metrics().await;
+        assert!(
+            metrics.tool_calls.len() <= 2,
+            "Expected at most 2 tool calls with limit=2, got {}",
+            metrics.tool_calls.len()
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 7: simple_echo_flow
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn simple_echo_flow() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/worker/plan_remaining_work.json"
+        ))
+        .expect("failed to load plan_remaining_work.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Plan and execute a task").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify echo was called during execution.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"echo".to_string()),
+            "echo should be called: {started:?}"
+        );
+
+        rig.shutdown();
+    }
+}

tests/e2e_workspace_coverage.rs

@@ -0,0 +1,320 @@
+//! E2E trace tests: workspace persistence (#574).
+//!
+//! Covers chunking, multi-document search, hybrid search, directory tree,
+//! document lifecycle (write/read/overwrite), and identity in system prompt.
+
+#[cfg(feature = "libsql")]
+mod support;
+
+#[cfg(feature = "libsql")]
+mod tests {
+    use std::time::Duration;
+
+    use crate::support::test_rig::TestRigBuilder;
+    use crate::support::trace_llm::LlmTrace;
+
+    // -----------------------------------------------------------------------
+    // Test 1: write_chunk_search
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn write_chunk_search() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/write_chunk_search.json"
+        ))
+        .expect("failed to load write_chunk_search.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write a long architecture document and search it")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify the document was persisted via workspace.
+        let ws = rig.workspace().expect("workspace must be available");
+        let doc = ws
+            .read("context/architecture.md")
+            .await
+            .expect("architecture.md should exist");
+        assert!(
+            doc.content.contains("Payment Service"),
+            "Document should contain 'Payment Service'"
+        );
+        assert!(
+            doc.content.len() > 1000,
+            "Document should be long (>1000 chars), got {}",
+            doc.content.len()
+        );
+
+        // Verify memory_search was called and returned relevant results.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"memory_search".to_string()),
+            "memory_search should be called: {started:?}"
+        );
+        let results = rig.tool_results();
+        let search_results: Vec<_> = results
+            .iter()
+            .filter(|(name, _)| name == "memory_search")
+            .collect();
+        assert!(!search_results.is_empty(), "Expected memory_search results");
+        assert!(
+            search_results
+                .iter()
+                .any(|(_, preview)| preview.contains("Payment Service")
+                    || preview.contains("payment")
+                    || preview.contains("architecture")),
+            "memory_search should return results related to payment/architecture: {search_results:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 2: multi_document_search
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn multi_document_search() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/multi_doc_search.json"
+        ))
+        .expect("failed to load multi_doc_search.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write three docs and search across them")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify all three documents were written.
+        let ws = rig.workspace().expect("workspace must be available");
+        let frontend = ws.read("context/frontend.md").await;
+        let backend = ws.read("context/backend.md").await;
+        let devops = ws.read("context/devops.md").await;
+        assert!(frontend.is_ok(), "frontend.md should exist");
+        assert!(backend.is_ok(), "backend.md should exist");
+        assert!(devops.is_ok(), "devops.md should exist");
+
+        // Verify cross-document memory_search was called.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"memory_search".to_string()),
+            "memory_search should be called in multi_document_search: {started:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 3: hybrid_search_with_embeddings
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn hybrid_search_with_embeddings() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/hybrid_search.json"
+        ))
+        .expect("failed to load hybrid_search.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write and semantically search for ML content")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify both memory_write and memory_search were used.
+        // Without a real embedding provider the FTS path handles keyword matches;
+        // we assert both tools ran to confirm the write-then-search pipeline.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"memory_write".to_string()),
+            "memory_write should be called: {started:?}"
+        );
+        assert!(
+            started.contains(&"memory_search".to_string()),
+            "memory_search should be called: {started:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 4: directory_tree
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn directory_tree() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/directory_tree.json"
+        ))
+        .expect("failed to load directory_tree.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write files in a hierarchy and show the tree")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify tree tool was called.
+        let started = rig.tool_calls_started();
+        assert!(
+            started.contains(&"memory_tree".to_string()),
+            "memory_tree should be called: {started:?}"
+        );
+
+        // Verify the tree result contains the expected directory hierarchy.
+        let results = rig.tool_results();
+        let tree_results: Vec<_> = results
+            .iter()
+            .filter(|(name, _)| name == "memory_tree")
+            .collect();
+        assert!(!tree_results.is_empty(), "Expected memory_tree results");
+
+        let tree_output: String = tree_results
+            .iter()
+            .map(|(_, preview)| preview.as_str())
+            .collect();
+        assert!(
+            tree_output.contains("alpha") || tree_output.contains("Alpha"),
+            "memory_tree output should contain 'alpha' project, got: {tree_output:?}"
+        );
+        assert!(
+            tree_output.contains("beta") || tree_output.contains("Beta"),
+            "memory_tree output should contain 'beta' project, got: {tree_output:?}"
+        );
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 5: document_lifecycle
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn document_lifecycle() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/doc_lifecycle.json"
+        ))
+        .expect("failed to load doc_lifecycle.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        rig.send_message("Write, read, overwrite, and read a document")
+            .await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify the document has the updated content.
+        let ws = rig.workspace().expect("workspace must be available");
+        let doc = ws
+            .read("context/lifecycle.md")
+            .await
+            .expect("lifecycle.md should exist");
+        assert!(
+            doc.content.contains("Version 2"),
+            "Document should contain 'Version 2', got: {:?}",
+            doc.content
+        );
+
+        // memory_write and memory_read should each be called twice.
+        let started = rig.tool_calls_started();
+        let write_count = started
+            .iter()
+            .filter(|n| n.as_str() == "memory_write")
+            .count();
+        let read_count = started
+            .iter()
+            .filter(|n| n.as_str() == "memory_read")
+            .count();
+        assert_eq!(write_count, 2, "Expected 2 memory_write calls");
+        assert_eq!(read_count, 2, "Expected 2 memory_read calls");
+
+        rig.shutdown();
+    }
+
+    // -----------------------------------------------------------------------
+    // Test 6: identity_in_system_prompt
+    // -----------------------------------------------------------------------
+
+    #[tokio::test]
+    async fn identity_in_system_prompt() {
+        let trace = LlmTrace::from_file(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/tests/fixtures/llm_traces/workspace/identity_prompt.json"
+        ))
+        .expect("failed to load identity_prompt.json");
+
+        let rig = TestRigBuilder::new()
+            .with_trace(trace.clone())
+            .build()
+            .await;
+
+        // Seed an IDENTITY.md so the system prompt has real content to inject.
+        let ws = rig.workspace().expect("workspace must be available");
+        ws.write(
+            "IDENTITY.md",
+            "I am TestBot, a helpful testing assistant created for E2E verification.",
+        )
+        .await
+        .expect("write IDENTITY.md");
+
+        rig.send_message("Who are you?").await;
+        let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
+
+        rig.verify_trace_expects(&trace, &responses);
+
+        // Verify the TraceLlm captured requests include a system message
+        // with the seeded identity content.
+        let trace_llm = rig.trace_llm().expect("trace_llm must be available");
+        let captured = trace_llm.captured_requests();
+        assert!(
+            !captured.is_empty(),
+            "Expected at least one captured request"
+        );
+        let first_request = &captured[0];
+        let system_msg = first_request
+            .iter()
+            .find(|msg| matches!(msg.role, ironclaw::llm::Role::System));
+        assert!(
+            system_msg.is_some(),
+            "Expected a system message in the first request"
+        );
+        assert!(
+            system_msg.unwrap().content.contains("TestBot"),
+            "System prompt should contain seeded identity 'TestBot', got: {:?}",
+            &system_msg.unwrap().content[..200.min(system_msg.unwrap().content.len())]
+        );
+
+        rig.shutdown();
+    }
+}

tests/fixtures/llm_traces/threading/concurrent_dispatch.json

@@ -0,0 +1,70 @@
+{
+  "model_name": "test-concurrent-dispatch",
+  "expects": {
+    "tools_used": [
+      "echo"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 2
+  },
+  "turns": [
+    {
+      "user_input": "Echo 'first message'",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_echo_first",
+                "name": "echo",
+                "arguments": {
+                  "message": "first message"
+                }
+              }
+            ],
+            "input_tokens": 100,
+            "output_tokens": 25
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Echoed: first message",
+            "input_tokens": 200,
+            "output_tokens": 15
+          }
+        }
+      ]
+    },
+    {
+      "user_input": "Echo 'second message'",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_echo_second",
+                "name": "echo",
+                "arguments": {
+                  "message": "second message"
+                }
+              }
+            ],
+            "input_tokens": 300,
+            "output_tokens": 25
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Echoed: second message",
+            "input_tokens": 400,
+            "output_tokens": 15
+          }
+        }
+      ]
+    }
+  ]
+}

tests/fixtures/llm_traces/threading/multi_turn_state.json

@@ -0,0 +1,102 @@
+{
+  "model_name": "test-multi-turn-state",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_search"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 3
+  },
+  "turns": [
+    {
+      "user_input": "Remember that project Alpha uses PostgreSQL.",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_mw_1",
+                "name": "memory_write",
+                "arguments": {
+                  "content": "# Project Alpha\n\nDatabase: PostgreSQL",
+                  "target": "context/project_alpha.md"
+                }
+              }
+            ],
+            "input_tokens": 100,
+            "output_tokens": 30
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "I've saved the note that Project Alpha uses PostgreSQL.",
+            "input_tokens": 200,
+            "output_tokens": 20
+          }
+        }
+      ]
+    },
+    {
+      "user_input": "Also note that it uses Redis for caching.",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_mw_2",
+                "name": "memory_write",
+                "arguments": {
+                  "content": "# Project Alpha\n\nDatabase: PostgreSQL\nCache: Redis",
+                  "target": "context/project_alpha.md"
+                }
+              }
+            ],
+            "input_tokens": 300,
+            "output_tokens": 30
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Updated the Project Alpha notes to include Redis caching.",
+            "input_tokens": 400,
+            "output_tokens": 20
+          }
+        }
+      ]
+    },
+    {
+      "user_input": "What database does Project Alpha use?",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_ms_1",
+                "name": "memory_search",
+                "arguments": {
+                  "query": "Project Alpha database"
+                }
+              }
+            ],
+            "input_tokens": 500,
+            "output_tokens": 25
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Project Alpha uses PostgreSQL as its database and Redis for caching.",
+            "input_tokens": 600,
+            "output_tokens": 20
+          }
+        }
+      ]
+    }
+  ]
+}

tests/fixtures/llm_traces/threading/undo_redo.json

@@ -0,0 +1,66 @@
+{
+  "model_name": "test-undo-redo",
+  "expects": {
+    "tools_used": [
+      "echo"
+    ],
+    "min_responses": 1
+  },
+  "turns": [
+    {
+      "user_input": "Echo the word 'original'",
+      "steps": [
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_echo_orig",
+                "name": "echo",
+                "arguments": {
+                  "message": "original"
+                }
+              }
+            ],
+            "input_tokens": 100,
+            "output_tokens": 25
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Echoed: original",
+            "input_tokens": 200,
+            "output_tokens": 15
+          }
+        }
+      ]
+    },
+    {
+      "user_input": "/undo",
+      "steps": [
+        {
+          "response": {
+            "type": "text",
+            "content": "Undone.",
+            "input_tokens": 50,
+            "output_tokens": 5
+          }
+        }
+      ]
+    },
+    {
+      "user_input": "/redo",
+      "steps": [
+        {
+          "response": {
+            "type": "text",
+            "content": "Redone.",
+            "input_tokens": 50,
+            "output_tokens": 5
+          }
+        }
+      ]
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/http_get_replay.json

@@ -0,0 +1,51 @@
+{
+  "model_name": "test-http-get-replay",
+  "expects": {
+    "tools_used": ["http"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "http_exchanges": [
+    {
+      "request": {
+        "method": "GET",
+        "url": "https://httpbin.org/get?test=1",
+        "headers": [],
+        "body": null
+      },
+      "response": {
+        "status": 200,
+        "headers": [["content-type", "application/json"]],
+        "body": "{\"args\": {\"test\": \"1\"}, \"url\": \"https://httpbin.org/get?test=1\"}"
+      }
+    }
+  ],
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "http" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_http_1",
+            "name": "http",
+            "arguments": {
+              "method": "GET",
+              "url": "https://httpbin.org/get?test=1"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The HTTP GET request to httpbin returned a 200 OK with the args confirming test=1.",
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/job_create_status.json

@@ -0,0 +1,50 @@
+{
+  "model_name": "test-job-create-status",
+  "expects": {
+    "tools_used": ["create_job", "job_status"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "job" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_cj_1",
+            "name": "create_job",
+            "arguments": {
+              "title": "Test analysis job",
+              "description": "Analyze the test data and summarize findings."
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 30
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_js_1",
+            "name": "job_status",
+            "arguments": { "job_id": "{{call_cj_1.job_id}}" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "Created a new job titled 'Test analysis job'. Its current status shows it's been registered in the system.",
+        "input_tokens": 300,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/job_list_cancel.json

@@ -0,0 +1,63 @@
+{
+  "model_name": "test-job-list-cancel",
+  "expects": {
+    "tools_used": ["create_job", "list_jobs", "cancel_job"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_cj_lc",
+            "name": "create_job",
+            "arguments": {
+              "title": "Cancellable job",
+              "description": "A job that will be cancelled."
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_lj_1",
+            "name": "list_jobs",
+            "arguments": {}
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_cancel_1",
+            "name": "cancel_job",
+            "arguments": { "job_id": "{{call_cj_lc.job_id}}" }
+          }
+        ],
+        "input_tokens": 300,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "Created a job, verified it appeared in the list, then cancelled it successfully.",
+        "input_tokens": 400,
+        "output_tokens": 20
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/routine_create_list.json

@@ -0,0 +1,53 @@
+{
+  "model_name": "test-routine-create-list",
+  "expects": {
+    "tools_used": ["routine_create", "routine_list"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "routine" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rc_1",
+            "name": "routine_create",
+            "arguments": {
+              "name": "daily-check",
+              "trigger_type": "cron",
+              "schedule": "0 0 9 * * *",
+              "prompt": "Check system status and report any issues.",
+              "description": "Daily system health check"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 35
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rl_1",
+            "name": "routine_list",
+            "arguments": {}
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "I created a daily-check routine that runs at 9 AM every day. The routine list shows it as active.",
+        "input_tokens": 300,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/routine_history.json

@@ -0,0 +1,50 @@
+{
+  "model_name": "test-routine-history",
+  "expects": {
+    "tools_used": ["routine_create", "routine_history"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rc_h",
+            "name": "routine_create",
+            "arguments": {
+              "name": "history-test",
+              "trigger_type": "manual",
+              "prompt": "Test routine for history."
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rh_1",
+            "name": "routine_history",
+            "arguments": { "name": "history-test" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The history-test routine was created. Its run history is empty since it hasn't been triggered yet.",
+        "input_tokens": 300,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/routine_update_delete.json

@@ -0,0 +1,68 @@
+{
+  "model_name": "test-routine-update-delete",
+  "expects": {
+    "tools_used": ["routine_create", "routine_update", "routine_delete"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rc_ud",
+            "name": "routine_create",
+            "arguments": {
+              "name": "temp-routine",
+              "trigger_type": "manual",
+              "prompt": "Temporary routine for testing."
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 30
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_ru_1",
+            "name": "routine_update",
+            "arguments": {
+              "name": "temp-routine",
+              "prompt": "Updated prompt for the temporary routine.",
+              "description": "Updated description"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 30
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rd_1",
+            "name": "routine_delete",
+            "arguments": { "name": "temp-routine" }
+          }
+        ],
+        "input_tokens": 300,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "Created, updated, and then deleted the temp-routine successfully.",
+        "input_tokens": 400,
+        "output_tokens": 20
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/time_parse_diff.json

@@ -0,0 +1,47 @@
+{
+  "model_name": "test-time-parse-diff",
+  "expects": {
+    "tools_used": ["time"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "time" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_time_parse",
+            "name": "time",
+            "arguments": { "operation": "parse", "timestamp": "2024-01-15T10:30:00Z" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_time_diff",
+            "name": "time",
+            "arguments": { "operation": "diff", "timestamp": "2024-01-15T10:30:00Z", "timestamp2": "2024-01-16T14:45:00Z" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The timestamp 2024-01-15T10:30:00Z was parsed successfully. The difference between the two timestamps is 1 day, 4 hours, and 15 minutes (101700 seconds).",
+        "input_tokens": 300,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/tools/time_parse_invalid.json

@@ -0,0 +1,32 @@
+{
+  "model_name": "test-time-parse-invalid",
+  "expects": {
+    "tools_used": ["time"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "parse" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_time_bad",
+            "name": "time",
+            "arguments": { "operation": "parse", "timestamp": "not-a-valid-timestamp" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The timestamp 'not-a-valid-timestamp' could not be parsed. Please provide a valid ISO 8601 timestamp like '2024-01-15T10:30:00Z'.",
+        "input_tokens": 200,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/invalid_params.json

@@ -0,0 +1,46 @@
+{
+  "model_name": "test-invalid-params",
+  "expects": {
+    "tools_used": ["echo"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "echo" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_bad_echo",
+            "name": "echo",
+            "arguments": { "message": 12345 }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_good_echo",
+            "name": "echo",
+            "arguments": { "message": "corrected message" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The echo tool initially received a number instead of a string. After correcting the parameter type, the echo returned: corrected message.",
+        "input_tokens": 300,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/parallel_three_tools.json

@@ -0,0 +1,43 @@
+{
+  "model_name": "test-parallel-three-tools",
+  "expects": {
+    "tools_used": ["echo", "time", "json"],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "parallel" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_echo_1",
+            "name": "echo",
+            "arguments": { "message": "hello from parallel" }
+          },
+          {
+            "id": "call_time_1",
+            "name": "time",
+            "arguments": { "operation": "now" }
+          },
+          {
+            "id": "call_json_1",
+            "name": "json",
+            "arguments": { "operation": "parse", "data": "{\"key\": \"value\"}" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 40
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "All three tools executed in parallel: echo returned the greeting, time gave the current timestamp, and json parsed the object successfully.",
+        "input_tokens": 200,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/plan_remaining_work.json

@@ -0,0 +1,31 @@
+{
+  "model_name": "test-plan-remaining-work",
+  "expects": {
+    "tools_used": ["echo"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_echo_plan",
+            "name": "echo",
+            "arguments": { "message": "planning step executed" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "I have completed the planning phase. The echo tool confirmed the step was executed successfully.",
+        "input_tokens": 200,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/rate_limit_cascade.json

@@ -0,0 +1,46 @@
+{
+  "model_name": "test-rate-limit-cascade",
+  "expects": {
+    "tools_used": ["stub_rate_limit"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "rate" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rl_1",
+            "name": "stub_rate_limit",
+            "arguments": {}
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_rl_2",
+            "name": "stub_rate_limit",
+            "arguments": {}
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The tool is rate limited. I was unable to complete the request due to repeated rate limiting.",
+        "input_tokens": 300,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/tool_error_feedback.json

@@ -0,0 +1,46 @@
+{
+  "model_name": "test-tool-error-feedback",
+  "expects": {
+    "tools_used": ["write_file"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "write" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_bad_write",
+            "name": "write_file",
+            "arguments": { "path": "/nonexistent_root_dir_xyz/impossible/file.txt", "content": "test" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_good_write",
+            "name": "write_file",
+            "arguments": { "path": "/tmp/ironclaw_error_feedback_test/recovered.txt", "content": "recovered content" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The first write failed because the directory didn't exist. I retried with a valid path and the file was written successfully.",
+        "input_tokens": 300,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/unknown_tool.json

@@ -0,0 +1,31 @@
+{
+  "model_name": "test-unknown-tool",
+  "expects": {
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": { "last_user_message_contains": "deploy" },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_unknown",
+            "name": "deploy_to_production",
+            "arguments": { "target": "us-east-1" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "I don't have a deploy_to_production tool available. I can only use the tools that are registered in my tool registry.",
+        "input_tokens": 200,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/worker/worker_timeout.json

@@ -0,0 +1,45 @@
+{
+  "model_name": "test-worker-timeout",
+  "expects": {
+    "tools_used": ["echo"],
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_echo_1",
+            "name": "echo",
+            "arguments": { "message": "iteration 1" }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_echo_2",
+            "name": "echo",
+            "arguments": { "message": "iteration 2" }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 25
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "Completed 2 iterations of tool calls.",
+        "input_tokens": 300,
+        "output_tokens": 20
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/directory_tree.json

@@ -0,0 +1,70 @@
+{
+  "model_name": "test-directory-tree",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_tree"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_t1",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Alpha Project\n\nMain readme for the Alpha project.",
+              "target": "projects/alpha/readme.md"
+            }
+          },
+          {
+            "id": "call_mw_t2",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Alpha Config\n\nConfiguration details for Alpha.",
+              "target": "projects/alpha/config.md"
+            }
+          },
+          {
+            "id": "call_mw_t3",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Beta Project\n\nMain readme for the Beta project.",
+              "target": "projects/beta/readme.md"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 50
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mt_1",
+            "name": "memory_tree",
+            "arguments": {
+              "path": "projects"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The workspace tree under 'projects/' shows two subdirectories: alpha (with readme.md and config.md) and beta (with readme.md).",
+        "input_tokens": 300,
+        "output_tokens": 25
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/doc_lifecycle.json

@@ -0,0 +1,87 @@
+{
+  "model_name": "test-doc-lifecycle",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_read"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_lc1",
+            "name": "memory_write",
+            "arguments": {
+              "content": "Version 1: Initial content",
+              "target": "context/lifecycle.md"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mr_lc1",
+            "name": "memory_read",
+            "arguments": {
+              "path": "context/lifecycle.md"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_lc2",
+            "name": "memory_write",
+            "arguments": {
+              "content": "Version 2: Updated content with changes",
+              "target": "context/lifecycle.md"
+            }
+          }
+        ],
+        "input_tokens": 300,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mr_lc2",
+            "name": "memory_read",
+            "arguments": {
+              "path": "context/lifecycle.md"
+            }
+          }
+        ],
+        "input_tokens": 400,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "Document lifecycle complete: wrote Version 1, read it back, overwrote with Version 2, and confirmed the update. The document now contains 'Version 2: Updated content with changes'.",
+        "input_tokens": 500,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/hybrid_search.json

@@ -0,0 +1,54 @@
+{
+  "model_name": "test-hybrid-search",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_search"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_hybrid",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Machine Learning Pipeline\n\nOur ML pipeline uses PyTorch for model training and ONNX for inference. Feature engineering is done with Pandas and the feature store uses Feast. Model versioning is handled by MLflow with experiment tracking. The training infrastructure runs on GPU-enabled Kubernetes pods.",
+              "target": "context/ml-pipeline.md"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 35
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_ms_hybrid",
+            "name": "memory_search",
+            "arguments": {
+              "query": "deep learning model training infrastructure"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "The hybrid search found the ML pipeline document. Even though the exact phrase 'deep learning' isn't in the document, the semantic similarity between 'deep learning model training' and 'PyTorch model training' helped surface the relevant content.",
+        "input_tokens": 300,
+        "output_tokens": 35
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/identity_prompt.json

@@ -0,0 +1,16 @@
+{
+  "model_name": "test-identity-prompt",
+  "expects": {
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "text",
+        "content": "I am IronClaw, your personal AI assistant. I can help you with various tasks.",
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/multi_doc_search.json

@@ -0,0 +1,70 @@
+{
+  "model_name": "test-multi-doc-search",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_search"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_d1",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Frontend Stack\n\nWe use React with TypeScript for the web application. State management is handled by Zustand. The build system is Vite.",
+              "target": "context/frontend.md"
+            }
+          },
+          {
+            "id": "call_mw_d2",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# Backend Stack\n\nThe backend uses Rust with Actix-web framework. Database is PostgreSQL with SQLx for queries.",
+              "target": "context/backend.md"
+            }
+          },
+          {
+            "id": "call_mw_d3",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# DevOps\n\nCI/CD via GitHub Actions. Deployment to AWS using Terraform. Monitoring with Datadog.",
+              "target": "context/devops.md"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 60
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_ms_multi",
+            "name": "memory_search",
+            "arguments": {
+              "query": "TypeScript React Rust"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "I wrote three documents covering the frontend (React/TypeScript), backend (Rust/Actix), and devops stacks. The search for 'TypeScript React Rust' matched the frontend and backend documents.",
+        "input_tokens": 300,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/fixtures/llm_traces/workspace/write_chunk_search.json

@@ -0,0 +1,57 @@
+{
+  "model_name": "test-write-chunk-search",
+  "expects": {
+    "tools_used": [
+      "memory_write",
+      "memory_search"
+    ],
+    "all_tools_succeeded": true,
+    "min_responses": 1
+  },
+  "steps": [
+    {
+      "request_hint": {
+        "last_user_message_contains": "document"
+      },
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_mw_long",
+            "name": "memory_write",
+            "arguments": {
+              "content": "# System Architecture\n\nThis document describes the complete architecture of our distributed system.\n\n## Overview\n\nThe system consists of multiple microservices communicating via message queues. Each service is independently deployable and follows the single responsibility principle. The main services include: User Service, Order Service, Payment Service, Notification Service, and Analytics Service.\n\n## User Service\n\nThe User Service manages user authentication, authorization, and profile management. It uses JWT tokens for session management and bcrypt for password hashing. The service exposes a REST API on port 8001 and maintains its own PostgreSQL database for user data. Rate limiting is applied at 100 requests per minute per user.\n\n## Order Service\n\nThe Order Service handles the complete order lifecycle from creation to fulfillment. Orders go through states: Created, Confirmed, Processing, Shipped, Delivered, or Cancelled. Each state transition is recorded as an event in the event store. The service uses an event-sourced architecture with CQRS for read optimization.\n\n## Payment Service\n\nThe Payment Service integrates with multiple payment providers including Stripe, PayPal, and cryptocurrency gateways. It implements the saga pattern for distributed transactions, ensuring consistency across the Order and Inventory services. Failed payments trigger automatic retry with exponential backoff.\n\n## Notification Service\n\nThe Notification Service sends alerts via email, SMS, push notifications, and webhooks. It uses a template engine for message formatting and supports multiple languages. Notifications are queued in RabbitMQ with priority levels and delivery guarantees.\n\n## Analytics Service\n\nThe Analytics Service collects metrics from all other services via event streams. It processes data in real-time using Apache Kafka and stores aggregated results in ClickHouse for fast querying. Dashboards are served via a Grafana integration.\n\n## Infrastructure\n\nAll services are containerized with Docker and orchestrated via Kubernetes. The cluster runs on AWS EKS with auto-scaling policies based on CPU and memory utilization. Service mesh is provided by Istio for traffic management, security, and observability.\n\n## Database Strategy\n\nEach microservice owns its database schema. Cross-service data access happens only through published APIs or event streams. Database migrations are managed via Flyway and executed during deployment. Read replicas are used for analytics queries to avoid impacting production workloads.\n\n## Security\n\nAll inter-service communication uses mTLS certificates managed by cert-manager. External traffic passes through AWS ALB with WAF rules. Secrets are stored in HashiCorp Vault and injected at runtime. Regular penetration testing is conducted quarterly.\n\n## Monitoring\n\nThe observability stack includes Prometheus for metrics, Jaeger for distributed tracing, and ELK for log aggregation. SLOs are defined per service with error budgets tracked via SLI dashboards. PagerDuty handles on-call alerting with escalation policies.\n\n## Deployment\n\nCI/CD pipelines run on GitHub Actions with stages for lint, test, build, and deploy. Feature flags are managed via LaunchDarkly for gradual rollouts. Blue-green deployments minimize downtime, with automatic rollback on health check failures.\n\n## Performance Requirements\n\nThe system targets 99.9% uptime with P99 latency under 200ms for API calls. Load testing with k6 validates performance before each release. CDN caching reduces origin load for static assets and frequently accessed API responses.",
+              "target": "context/architecture.md"
+            }
+          }
+        ],
+        "input_tokens": 100,
+        "output_tokens": 50
+      }
+    },
+    {
+      "response": {
+        "type": "tool_calls",
+        "tool_calls": [
+          {
+            "id": "call_ms_chunk",
+            "name": "memory_search",
+            "arguments": {
+              "query": "payment service saga pattern"
+            }
+          }
+        ],
+        "input_tokens": 200,
+        "output_tokens": 20
+      }
+    },
+    {
+      "response": {
+        "type": "text",
+        "content": "I wrote the architecture document and searched for payment-related content. The search found the Payment Service section describing the saga pattern for distributed transactions.",
+        "input_tokens": 300,
+        "output_tokens": 30
+      }
+    }
+  ]
+}

tests/support/test_rig.rs

@@ -20,6 +20,7 @@ use ironclaw::config::Config;
 use ironclaw::db::Database;
 use ironclaw::error::ChannelError;
 use ironclaw::llm::{LlmProvider, SessionConfig, SessionManager};
+use ironclaw::tools::Tool;
 
 use crate::support::instrumented_llm::InstrumentedLlm;
 use crate::support::metrics::{ToolInvocation, TraceMetrics};
@@ -108,6 +109,15 @@ pub struct TestRig {
     max_tool_iterations: usize,
     /// Handle to the background agent task (wrapped in Option so Drop can take it).
     agent_handle: Option<tokio::task::JoinHandle<()>>,
+    /// Database handle for direct queries in tests.
+    #[cfg(feature = "libsql")]
+    db: Arc<dyn Database>,
+    /// Workspace handle for direct memory operations in tests.
+    #[cfg(feature = "libsql")]
+    workspace: Option<Arc<ironclaw::workspace::Workspace>>,
+    /// The underlying TraceLlm for inspecting captured requests.
+    #[cfg(feature = "libsql")]
+    trace_llm: Option<Arc<TraceLlm>>,
     /// Temp directory guard -- keeps the libSQL database file alive.
     #[cfg(feature = "libsql")]
     _temp_dir: tempfile::TempDir,
@@ -352,6 +362,7 @@ pub struct TestRigBuilder {
     llm: Option<Arc<dyn LlmProvider>>,
     max_tool_iterations: usize,
     injection_check: bool,
+    extra_tools: Vec<Arc<dyn Tool>>,
 }
 
 impl TestRigBuilder {
@@ -362,6 +373,7 @@ impl TestRigBuilder {
             llm: None,
             max_tool_iterations: 10,
             injection_check: false,
+            extra_tools: Vec::new(),
         }
     }
 
@@ -383,6 +395,12 @@ impl TestRigBuilder {
         self
     }
 
+    /// Register additional custom tools (e.g. stub tools for testing).
+    pub fn with_extra_tools(mut self, tools: Vec<Arc<dyn Tool>>) -> Self {
+        self.extra_tools = tools;
+        self
+    }
+
     /// Enable prompt injection detection in the safety layer.
     ///
     /// When enabled, tool outputs are scanned for injection patterns
@@ -436,10 +454,13 @@ impl TestRigBuilder {
             .map(|t| t.http_exchanges.clone())
             .unwrap_or_default();
 
+        let mut trace_llm_ref: Option<Arc<TraceLlm>> = None;
         let base_llm: Arc<dyn LlmProvider> = if let Some(llm) = self.llm {
             llm
         } else if let Some(trace) = self.trace {
-            Arc::new(TraceLlm::from_trace(trace))
+            let tlm = Arc::new(TraceLlm::from_trace(trace));
+            trace_llm_ref = Some(Arc::clone(&tlm));
+            tlm
         } else {
             let trace = LlmTrace::single_turn(
                 "test-rig-default",
@@ -454,7 +475,9 @@ impl TestRigBuilder {
                     expected_tool_results: Vec::new(),
                 }],
             );
-            Arc::new(TraceLlm::from_trace(trace))
+            let tlm = Arc::new(TraceLlm::from_trace(trace));
+            trace_llm_ref = Some(Arc::clone(&tlm));
+            tlm
         };
         let instrumented = Arc::new(InstrumentedLlm::new(base_llm));
         let llm: Arc<dyn LlmProvider> = Arc::clone(&instrumented) as Arc<dyn LlmProvider>;
@@ -474,7 +497,55 @@ impl TestRigBuilder {
             .await
             .expect("AppBuilder::build_all() failed in test rig");
 
-        // 6. Construct AgentDeps from AppComponents (mirrors main.rs).
+        // 6. Register job tools, routine tools, and extra tools.
+        {
+            use ironclaw::context::ContextManager;
+
+            let ctx_mgr = Arc::new(ContextManager::new(
+                components.config.agent.max_parallel_jobs,
+            ));
+            components.tools.register_job_tools(
+                ctx_mgr,
+                None,
+                None,
+                components.db.clone(),
+                None,
+                None,
+                None,
+                None,
+            );
+
+            // Routine tools: create a RoutineEngine with the LLM and workspace.
+            if let (Some(db_arc), Some(ws)) = (&components.db, &components.workspace) {
+                use ironclaw::agent::routine_engine::RoutineEngine;
+                use ironclaw::config::RoutineConfig;
+
+                let routine_config = RoutineConfig::default();
+                let (notify_tx, _notify_rx) = tokio::sync::mpsc::channel(16);
+                let engine = Arc::new(RoutineEngine::new(
+                    routine_config,
+                    Arc::clone(db_arc),
+                    components.llm.clone(),
+                    Arc::clone(ws),
+                    notify_tx,
+                    None,
+                ));
+                components
+                    .tools
+                    .register_routine_tools(Arc::clone(db_arc), engine);
+            }
+
+            // Register any extra test-specific tools.
+            for tool in self.extra_tools {
+                components.tools.register(tool).await;
+            }
+        }
+
+        // Save references for test accessors.
+        let db_ref = components.db.clone().expect("test rig requires a database");
+        let workspace_ref = components.workspace.clone();
+
+        // 7. Construct AgentDeps from AppComponents (mirrors main.rs).
         let deps = AgentDeps {
             store: components.db,
             llm: components.llm,
@@ -535,6 +606,9 @@ impl TestRigBuilder {
             start_time: Instant::now(),
             max_tool_iterations: self.max_tool_iterations,
             agent_handle: Some(agent_handle),
+            db: db_ref,
+            workspace: workspace_ref,
+            trace_llm: trace_llm_ref,
             _temp_dir: temp_dir,
         }
     }
@@ -547,6 +621,24 @@ impl Default for TestRigBuilder {
 }
 
 impl TestRig {
+    /// Get the database handle for direct queries.
+    #[cfg(feature = "libsql")]
+    pub fn database(&self) -> &Arc<dyn Database> {
+        &self.db
+    }
+
+    /// Get the workspace handle for direct memory operations.
+    #[cfg(feature = "libsql")]
+    pub fn workspace(&self) -> Option<&Arc<ironclaw::workspace::Workspace>> {
+        self.workspace.as_ref()
+    }
+
+    /// Get the underlying TraceLlm for inspecting captured requests.
+    #[cfg(feature = "libsql")]
+    pub fn trace_llm(&self) -> Option<&Arc<TraceLlm>> {
+        self.trace_llm.as_ref()
+    }
+
     /// Check if any captured status events contain safety/injection warnings.
     pub fn has_safety_warnings(&self) -> bool {
         self.captured_status_events().iter().any(|s| {

tests/support/trace_llm.rs

@@ -308,6 +308,13 @@ impl TraceLlm {
     // -- internal helpers ---------------------------------------------------
 
     /// Advance the step index and return the current step, or an error if exhausted.
+    ///
+    /// Before returning, applies template substitution on tool_call arguments:
+    /// `{{call_id.json_path}}` is replaced with the value extracted from the
+    /// tool result message whose `tool_call_id` matches `call_id`. The
+    /// `json_path` is a dot-separated path into the JSON content of that tool
+    /// result (e.g., `{{call_cj_1.job_id}}` extracts `.job_id` from the result
+    /// of tool call `call_cj_1`).
     fn next_step(&self, messages: &[ChatMessage]) -> Result<TraceStep, LlmError> {
         // Capture the request messages.
         self.captured_requests
@@ -316,7 +323,7 @@ impl TraceLlm {
             .push(messages.to_vec());
 
         let idx = self.index.fetch_add(1, Ordering::Relaxed);
-        let step = self
+        let mut step = self
             .steps
             .get(idx)
             .ok_or_else(|| LlmError::RequestFailed {
@@ -334,6 +341,19 @@ impl TraceLlm {
             self.validate_hint(hint, messages);
         }
 
+        // Apply template substitution on tool_call arguments.
+        if let TraceResponse::ToolCalls {
+            ref mut tool_calls, ..
+        } = step.response
+        {
+            let vars = Self::extract_tool_result_vars(messages);
+            if !vars.is_empty() {
+                for tc in tool_calls.iter_mut() {
+                    Self::substitute_templates(&mut tc.arguments, &vars);
+                }
+            }
+        }
+
         Ok(step)
     }
 
@@ -365,6 +385,121 @@ impl TraceLlm {
             );
         }
     }
+
+    /// Build a map of `"call_id.json_path" -> resolved_value` from tool result
+    /// messages in the conversation. Each `Role::Tool` message with a
+    /// `tool_call_id` has its content parsed as JSON; all top-level
+    /// string/number/bool values are indexed so that `{{call_id.key}}` can be
+    /// resolved.
+    ///
+    /// Tool results may be wrapped in `<tool_output>` XML tags by the safety
+    /// layer, so we strip those before parsing.
+    fn extract_tool_result_vars(
+        messages: &[ChatMessage],
+    ) -> std::collections::HashMap<String, String> {
+        let mut vars = std::collections::HashMap::new();
+        for msg in messages {
+            if msg.role != Role::Tool {
+                continue;
+            }
+            let call_id = match &msg.tool_call_id {
+                Some(id) => id,
+                None => continue,
+            };
+            // Strip <tool_output ...>...</tool_output> wrapper if present.
+            let content = Self::unwrap_tool_output(&msg.content);
+            // Try parsing the content as JSON.
+            let json: serde_json::Value = match serde_json::from_str(&content) {
+                Ok(v) => v,
+                Err(_) => continue,
+            };
+            if let Some(obj) = json.as_object() {
+                for (key, val) in obj {
+                    let str_val = match val {
+                        serde_json::Value::String(s) => s.clone(),
+                        serde_json::Value::Number(n) => n.to_string(),
+                        serde_json::Value::Bool(b) => b.to_string(),
+                        _ => continue,
+                    };
+                    vars.insert(format!("{call_id}.{key}"), str_val);
+                }
+            }
+        }
+        vars
+    }
+
+    /// Strip `<tool_output name="..." sanitized="...">...\n</tool_output>`
+    /// wrapper and unescape XML entities from safety-layer output.
+    fn unwrap_tool_output(content: &str) -> std::borrow::Cow<'_, str> {
+        let trimmed = content.trim();
+        if let Some(rest) = trimmed.strip_prefix("<tool_output")
+            && let Some(tag_end) = rest.find('>')
+        {
+            let inner = &rest[tag_end + 1..];
+            if let Some(close) = inner.rfind("</tool_output>") {
+                let body = inner[..close].trim();
+                // Reverse XML escaping applied by safety layer.
+                if body.contains("&amp;") || body.contains("&lt;") || body.contains("&gt;") {
+                    return std::borrow::Cow::Owned(
+                        body.replace("&amp;", "&")
+                            .replace("&lt;", "<")
+                            .replace("&gt;", ">"),
+                    );
+                }
+                return std::borrow::Cow::Borrowed(body);
+            }
+        }
+        std::borrow::Cow::Borrowed(content)
+    }
+
+    /// Walk a JSON value and replace any string matching `{{call_id.path}}`
+    /// with the resolved value from the vars map. Operates in-place.
+    fn substitute_templates(
+        value: &mut serde_json::Value,
+        vars: &std::collections::HashMap<String, String>,
+    ) {
+        match value {
+            serde_json::Value::String(s) => {
+                // Full-value replacement: if the entire string is `{{...}}`,
+                // replace the whole value (preserving type if possible).
+                if s.starts_with("{{") && s.ends_with("}}") && s.matches("{{").count() == 1 {
+                    let key = s[2..s.len() - 2].trim();
+                    if let Some(resolved) = vars.get(key) {
+                        *s = resolved.clone();
+                        return;
+                    }
+                }
+                // Inline replacement: replace all `{{...}}` occurrences within the string.
+                let mut result = s.clone();
+                while let Some(start) = result.find("{{") {
+                    if let Some(end) = result[start..].find("}}") {
+                        let end = start + end + 2;
+                        let key = result[start + 2..end - 2].trim();
+                        if let Some(resolved) = vars.get(key) {
+                            result = format!("{}{}{}", &result[..start], resolved, &result[end..]);
+                        } else {
+                            // Unresolved template — leave as-is and stop to avoid infinite loop.
+                            break;
+                        }
+                    } else {
+                        break;
+                    }
+                }
+                *s = result;
+            }
+            serde_json::Value::Object(map) => {
+                for val in map.values_mut() {
+                    Self::substitute_templates(val, vars);
+                }
+            }
+            serde_json::Value::Array(arr) => {
+                for val in arr.iter_mut() {
+                    Self::substitute_templates(val, vars);
+                }
+            }
+            _ => {}
+        }
+    }
 }
 
 #[async_trait]