feat(agent): thread per-tool reasoning through provider, session, and all surfaces (#1513)

* feat(agent): thread per-tool reasoning from LLM through to REPL, HTTP, SSE, and DB

Add end-to-end agent reasoning summaries so users can see *why* the
agent chose specific tools, not just what it did.

- Add `reasoning: Option<String>` to `ToolCall` (all providers)
- Populate from LLM response content in `Reasoning::respond_with_tools`
  and `select_tools`, with per-tool override when providers supply it
- Extend `Turn` with `narrative` and `TurnToolCall` with `rationale` +
  `tool_call_id` for identity-based result matching
- Persist reasoning in DB via existing tool_calls JSON (no migration)
- Add `StatusUpdate::ReasoningUpdate` and `SseEvent::ReasoningUpdate` +
  `SseEvent::JobReasoning` for real-time streaming
- Emit reasoning events in both chat dispatcher and worker job path
- Add `/reasoning [N|all]` command for inspecting turn reasoning
- Surface `narrative` and `rationale` in HTTP `/api/chat/history`

Based on the design from #361 and #456, reconstructed cleanly with
Option<String> to minimize blast radius (vs mandatory String that broke
compilation in #456).

Closes #456

Co-Authored-By: panosAthDBX <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review feedback from Gemini and Copilot

- Fix `_ => Ok(None)` in agent_loop.rs to avoid accidental shutdown
- Fix fallback in record_tool_result_for/record_tool_error_for to use
  first pending call instead of last_mut (parallel execution safety)
- Include per-tool decisions in WASM channel reasoning messages
- Apply truncate_at_tool_tags + clean_response to shared_reasoning in
  select_tools (parity with respond_with_tools)
- Persist turn-level narrative to DB in tool_calls JSON wrapper
- Parse both old (array) and new (object) tool_calls formats in
  build_turns_from_db_messages for backward compatibility
- Populate reasoning from action.reasoning in execute_plan ToolCalls

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address second round of review comments + merge fixes

- Add reasoning: None to new github_copilot.rs ToolCall sites (from staging merge)
- Run cargo fmt on 4 files with formatting diffs
- Truncate narrative to 1000 chars before DB persistence
- Clone turn data and drop session lock in /reasoning command
- Extract ToolDecisionDto::from_json_array shared helper (deduplicate
  worker/job.rs and orchestrator/api.rs)
- Add unit tests for wrapped tool_calls JSON format with narrative

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address third round of review comments (Copilot + serrrfirat)

- Reword ToolCall.reasoning docstring to reflect provider-supplied or
  fallback contract
- Sanitize narrative through SafetyLayer before storage/emission
- Clean per-tool reasoning via truncate_at_tool_tags + clean_response
  in select_tools (parity with shared reasoning)
- Convert 4 approval-path recording sites in thread_ops.rs to
  identity-based record_tool_result_for/record_tool_error_for
- Preserve tool_call_id and reasoning through restore_from_messages
- Fix has_result/has_error to reject JSON null values
- Truncate tool_call_id to 128 chars before DB persistence
- Add 4 unit tests for record_tool_result_for/error_for edge cases

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address zmanian review — sanitize JobDelegate reasoning + warn on dropped results

- Sanitize narrative and per-tool rationale through SafetyLayer in
  JobDelegate reasoning events (parity with ChatDelegate)
- Add tracing::warn when record_tool_result_for/error_for drops a
  result because no matching or pending tool call exists
- Add 3 unit tests for reasoning normalization (thinking tags,
  tool tags, empty-after-cleaning)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address 4 remaining unreplied review comments

- Clean per-tool reasoning in respond_with_tools via truncate_at_tool_tags
  + clean_response (parity with select_tools)
- Handle wrapped JSON format in rebuild_chat_messages_from_db so cold
  hydration works after persist_tool_calls format change
- Update persist_tool_calls doc comment to describe new JSON shape
- Sanitize per-tool rationale through SafetyLayer in ChatDelegate before
  emission and storage (parity with JobDelegate)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address zmanian review round 2

- Add tracing::debug on fallback-to-pending path in record_tool_result_for
  and record_tool_error_for (item 1)
- Add comment explaining why /reasoning is special-cased in agent_loop.rs
  (item 4)
- Items 2 (narrative persistence), 3 (rationale sanitization), and 5
  (catch-all fix) were already addressed in prior commits

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: panosAthDBX <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

src/worker/job.rs

@@ -18,6 +18,7 @@ use crate::agent::agentic_loop::{
 };
 use crate::agent::scheduler::WorkerMessage;
 use crate::agent::task::TaskOutput;
+use crate::channels::web::types::ToolDecisionDto;
 use crate::context::{ContextManager, JobState};
 use crate::db::Database;
 use crate::error::Error;
@@ -200,6 +201,19 @@ impl Worker {
                         .map(|s| s.to_string()),
                     fallback_deliverable: data.get("fallback_deliverable").cloned(),
                 }),
+                "reasoning" => {
+                    let narrative = data
+                        .get("narrative")
+                        .and_then(|v| v.as_str())
+                        .unwrap_or("")
+                        .to_string();
+                    let decisions = ToolDecisionDto::from_json_array(&data["decisions"]);
+                    Some(AppEvent::JobReasoning {
+                        job_id: job_id_str,
+                        narrative,
+                        decisions,
+                    })
+                }
                 _ => None,
             };
             if let Some(event) = event {
@@ -897,6 +911,11 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
                         id: selection.tool_call_id.clone(),
                         name: selection.tool_name.clone(),
                         arguments: selection.parameters.clone(),
+                        reasoning: if action.reasoning.is_empty() {
+                            None
+                        } else {
+                            Some(action.reasoning.clone())
+                        },
                     }],
                 ));
 
@@ -1357,6 +1376,48 @@ impl<'a> LoopDelegate for JobDelegate<'a> {
             );
         }
 
+        // Emit reasoning event if any tool calls carry reasoning.
+        // Sanitize narrative and per-tool rationale through SafetyLayer
+        // (parity with ChatDelegate in dispatcher.rs).
+        let sanitized_narrative = content
+            .as_deref()
+            .filter(|c| !c.trim().is_empty())
+            .map(|c| {
+                self.worker
+                    .deps
+                    .safety
+                    .sanitize_tool_output("job_narrative", c)
+                    .content
+            })
+            .filter(|c| !c.trim().is_empty())
+            .unwrap_or_default();
+        let decisions: Vec<serde_json::Value> = tool_calls
+            .iter()
+            .filter_map(|tc| {
+                tc.reasoning.as_ref().map(|r| {
+                    let sanitized = self
+                        .worker
+                        .deps
+                        .safety
+                        .sanitize_tool_output("tool_rationale", r)
+                        .content;
+                    serde_json::json!({
+                        "tool_name": tc.name,
+                        "rationale": sanitized,
+                    })
+                })
+            })
+            .collect();
+        if !decisions.is_empty() {
+            self.worker.log_event(
+                "reasoning",
+                serde_json::json!({
+                    "narrative": sanitized_narrative,
+                    "decisions": decisions,
+                }),
+            );
+        }
+
         // Add assistant message with tool_calls (OpenAI protocol)
         reason_ctx
             .messages
@@ -1371,7 +1432,7 @@ impl<'a> LoopDelegate for JobDelegate<'a> {
             .map(|tc| ToolSelection {
                 tool_name: tc.name.clone(),
                 parameters: tc.arguments.clone(),
-                reasoning: String::new(),
+                reasoning: tc.reasoning.clone().unwrap_or_default(),
                 alternatives: vec![],
                 tool_call_id: tc.id.clone(),
             })
@@ -1424,6 +1485,11 @@ fn selections_to_tool_calls(selections: &[ToolSelection]) -> Vec<ToolCall> {
             id: s.tool_call_id.clone(),
             name: s.tool_name.clone(),
             arguments: s.parameters.clone(),
+            reasoning: if s.reasoning.is_empty() {
+                None
+            } else {
+                Some(s.reasoning.clone())
+            },
         })
         .collect()
 }