feat(agent): queue and merge messages during active turns (#1412)

* feat(agent): queue and merge messages during active turns Replace the hard rejection ("Turn in progress") when messages arrive during an active turn with a bounded queue (max 10) that auto-drains after the turn completes. Queued messages are merged with newlines into a single turn so the LLM receives full context from rapid consecutive inputs instead of producing fragmented responses from partial context. Key changes: - Thread.pending_messages (VecDeque) with queue_message/drain_pending_messages - Drain loop in agent_loop.rs merges all queued messages per iteration - interrupt() and /clear both clear the pending queue - MAX_PENDING_MESSAGES constant with cap enforced inside queue_message() - Drain loop continues on soft errors, stops on NeedApproval/Interrupted - Drain loop logs respond() failures instead of silently swallowing them Fixes #259 — debounces rapid inbound messages during processing Fixes #826 — drain loop is bounded by MAX_PENDING_MESSAGES cap Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review — drain loop busy-loop guard and stale state re-check - Add Ok(SubmissionResult::Ok) to drain loop break conditions to prevent a tight busy-loop if process_user_input returns a queued-ack (e.g. from a corrupted/hydrated session stuck in Processing state) - Re-check thread.state under the mutable lock in the Processing arm to guard against the turn completing between the snapshot read and the queue operation Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: clear attachments on drain-loop queued message processing Queued messages are text-only (queued as strings during Processing state). The drain loop was reusing the original IncomingMessage reference which carried the first message's attachments, causing augment_with_attachments to incorrectly re-apply them to unrelated queued text. Clone the message with cleared attachments for drain-loop turns. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review round 2 — stale state fallthrough and thread-not-found guard - Processing arm: when re-checked state is no longer Processing, fall through to normal processing instead of dropping user input - Processing arm: return error when thread not found instead of false "queued" ack - Document intermediate drain-loop responses as best-effort for one-shot channels (HttpChannel) - Add regression tests for both edge cases Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review feedback for message queue drain loop [skip-regression-check] — test modifications present but hook has SIGPIPE/pipefail false negative when awk exits early on match - Replace wildcard match in drain loop with explicit `while let Ok(Response)` guard — stops on Error variant too, preventing confusing interleaved output after soft errors (review issue #1) - Reject queueing messages with attachments during Processing state instead of silently dropping them (review issue #2) - Document response routing limitation: all drain-loop responses route via original message identity (review issue #3) - Document why SubmissionResult::Ok is correct for queued ack and how it interacts with drain loop break condition (review issue #4) - Rewrite two dead regression tests to assert actual behavior: thread-gone returns error, state-changed does not queue (review #5) - Document MAX_PENDING_MESSAGES=10 as acceptable for personal assistant use case (review issue #6) - Fix misleading one-shot channel comment — HttpChannel consumes sender on first call, subsequent calls are dropped (review issue #8) - Simplify drain loop intermediate response since while-let guard guarantees Response variant Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: add missing extension_manager field in webhook EngineContext The fire_webhook method's EngineContext initializer was missing the extension_manager field added in staging, causing CI compilation failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: gate TestRig::session_manager() behind libsql feature flag The field is #[cfg(feature = "libsql")] so the accessor must match. All callers are already inside #[cfg(feature = "libsql")] blocks. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: re-queue drained messages on drain loop failure If process_user_input fails after drain_pending_messages() removed all queued content, that user input was permanently lost. Now the merged content is re-queued at the front of pending_messages on any non-Response result so it will be processed on the next successful turn. Adds Thread::requeue_drained() helper and unit test. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: remove unreachable!() from drain loop, add lock-drop comments - Extract content binding in `while let` pattern instead of using a separate match with unreachable!() — satisfies the no-panic-in- production convention (zmanian review item #1) - Add comment clarifying session lock is dropped at Processing arm boundary before fall-through (zmanian review item #5) - Document bounded cap overshoot on requeue_drained (review item #2) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(security): validate queued messages and touch updated_at on queue ops - Run safety validation, policy checks, and secret scanning on messages before queueing during Processing state. Previously, content with leaked secrets could be stored in pending_messages and serialized without hitting the inbound scanner. - Touch updated_at in queue_message(), drain_pending_messages(), and requeue_drained() so thread timestamps reflect queue activity. [skip-regression-check] — safety validation requires full Agent; updated_at is a data-level fix on existing tested methods Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-08-25 14:53:34 +00:00 · 2026-03-21 21:53:14 -07:00
parent 89394ebd29
commit ccdea40e9d
6 changed files with 702 additions and 16 deletions
@@ -0,0 +1,104 @@
+{
+  "model_name": "advanced-message-queue-during-tools",
+  "turns": [
+    {
+      "user_input": "Echo hello for me",
+      "steps": [
+        {
+          "request_hint": { "last_user_message_contains": "Echo hello" },
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_echo_setup",
+                "name": "echo",
+                "arguments": { "message": "hello" }
+              }
+            ],
+            "input_tokens": 80,
+            "output_tokens": 20
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "I echoed hello for you. The tool returned: hello",
+            "input_tokens": 120,
+            "output_tokens": 25
+          }
+        }
+      ],
+      "expects": {
+        "tools_used": ["echo"],
+        "all_tools_succeeded": true,
+        "response_contains": ["hello"]
+      }
+    },
+    {
+      "user_input": "Now echo world and check the time",
+      "steps": [
+        {
+          "request_hint": { "last_user_message_contains": "echo world" },
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_echo_main",
+                "name": "echo",
+                "arguments": { "message": "world" }
+              }
+            ],
+            "input_tokens": 160,
+            "output_tokens": 20
+          }
+        },
+        {
+          "response": {
+            "type": "tool_calls",
+            "tool_calls": [
+              {
+                "id": "call_time_main",
+                "name": "time",
+                "arguments": {}
+              }
+            ],
+            "input_tokens": 200,
+            "output_tokens": 15
+          }
+        },
+        {
+          "response": {
+            "type": "text",
+            "content": "Done! I echoed world and checked the time for you.",
+            "input_tokens": 250,
+            "output_tokens": 20
+          }
+        }
+      ],
+      "expects": {
+        "tools_used": ["echo", "time"],
+        "all_tools_succeeded": true
+      }
+    },
+    {
+      "user_input": "What is 2+2?",
+      "steps": [
+        {
+          "response": {
+            "type": "text",
+            "content": "2+2 equals 4.",
+            "input_tokens": 80,
+            "output_tokens": 10
+          }
+        }
+      ],
+      "expects": {
+        "response_contains": ["4"]
+      }
+    }
+  ],
+  "expects": {
+    "tools_used": ["echo", "time"],
+    "min_responses": 3
+  }
+}