feat: merge http/web_fetch tools, add tool output stash for large responses (#578)

* feat: merge http/web_fetch tools, add tool output stash for large responses

Merge `web_fetch` into `http` tool with smart approval: plain GETs (no
headers, no body) run without approval and follow redirects with SSRF
re-validation per hop; all other requests require approval as before.

Add `tool_output_stash` on JobContext so full tool outputs are preserved
before safety-layer truncation. The `json` tool gains a
`source_tool_call_id` parameter to reference stashed outputs, enabling
reliable parsing of large API responses that exceed the 100KB context
limit.

Other improvements:
- Descriptive User-Agent header using CARGO_PKG_VERSION
- Truncation now keeps partial data + hint about source_tool_call_id
- System prompt reinforces tool_calls over narration
- json tool query/stringify handle pre-parsed (non-string) data

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: delete dead web_fetch.rs (merged into http tool)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: rename shadowed data binding for clarity in json tool

Address PR review: rename owned `data` to `data_value` before
re-binding as `let data = &data_value` to make ownership explicit.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(ci): mark network-dependent trace tests as #[ignore]

The weather_sf and baseball_stats tests hit live external APIs (wttr.in,
ESPN) which are unreliable in CI. Mark them #[ignore] so they don't
block the pipeline. Run locally with `--ignored` to include them.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replay recorded HTTP exchanges in trace tests instead of hitting live APIs

Wire ReplayingHttpInterceptor into TestRig when the trace fixture
contains http_exchanges. This replays recorded responses instead of
making live network calls, making tests deterministic and CI-stable.

Add captured HTTP responses to weather_sf.json (wttr.in) and
baseball_stats.json (ESPN API) fixtures.

Revert #[ignore] on both tests — they now run offline.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: recover inline bracket-format tool calls from LLM text responses

When flatten_tool_messages converts tool calls to text like
`[Called tool `http` with arguments: {...}]` for NEAR AI compatibility,
the LLM sometimes echoes this format back in its text responses instead
of using proper tool_calls. Add recovery for this bracket format in
recover_tool_calls_from_content and strip it in clean_response so
users don't see raw tool call syntax.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>

tests/e2e_recorded_trace.rs

@@ -15,4 +15,17 @@ mod recorded_trace_tests {
     async fn recorded_telegram_check() {
         run_recorded_trace("telegram_check.json").await;
     }
+
+    /// Recorded trace: weather query for San Francisco.
+    #[tokio::test]
+    async fn recorded_weather_sf() {
+        run_recorded_trace("weather_sf.json").await;
+    }
+
+    /// Recorded trace: baseball stats with large HTTP response exercising
+    /// tool_output_stash + source_tool_call_id for untruncated data access.
+    #[tokio::test]
+    async fn recorded_baseball_stats() {
+        run_recorded_trace("baseball_stats.json").await;
+    }
 }

tests/support/test_rig.rs

@@ -429,7 +429,13 @@ impl TestRigBuilder {
         let session = Arc::new(SessionManager::new(SessionConfig::default()));
         let log_broadcaster = Arc::new(LogBroadcaster::new());
 
-        // 4. Create TraceLlm + InstrumentedLlm.
+        // 4. Create TraceLlm + InstrumentedLlm, extract HTTP exchanges for replay.
+        let http_exchanges = self
+            .trace
+            .as_ref()
+            .map(|t| t.http_exchanges.clone())
+            .unwrap_or_default();
+
         let base_llm: Arc<dyn LlmProvider> = if let Some(llm) = self.llm {
             llm
         } else if let Some(trace) = self.trace {
@@ -483,7 +489,13 @@ impl TestRigBuilder {
             hooks: components.hooks,
             cost_guard: components.cost_guard,
             sse_tx: None,
-            http_interceptor: None,
+            http_interceptor: if http_exchanges.is_empty() {
+                None
+            } else {
+                Some(Arc::new(
+                    ironclaw::llm::recording::ReplayingHttpInterceptor::new(http_exchanges),
+                ))
+            },
         };
 
         // 7. Create TestChannel and ChannelManager.

tests/tool_schema_validation.rs

@@ -68,7 +68,6 @@ async fn core_registration_covers_expected_tools() {
         "read_file",
         "shell",
         "time",
-        "web_fetch",
         "write_file",
     ];