feat(agent): queue and merge messages during active turns (#1412)

* feat(agent): queue and merge messages during active turns Replace the hard rejection ("Turn in progress") when messages arrive during an active turn with a bounded queue (max 10) that auto-drains after the turn completes. Queued messages are merged with newlines into a single turn so the LLM receives full context from rapid consecutive inputs instead of producing fragmented responses from partial context. Key changes: - Thread.pending_messages (VecDeque) with queue_message/drain_pending_messages - Drain loop in agent_loop.rs merges all queued messages per iteration - interrupt() and /clear both clear the pending queue - MAX_PENDING_MESSAGES constant with cap enforced inside queue_message() - Drain loop continues on soft errors, stops on NeedApproval/Interrupted - Drain loop logs respond() failures instead of silently swallowing them Fixes #259 — debounces rapid inbound messages during processing Fixes #826 — drain loop is bounded by MAX_PENDING_MESSAGES cap Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review — drain loop busy-loop guard and stale state re-check - Add Ok(SubmissionResult::Ok) to drain loop break conditions to prevent a tight busy-loop if process_user_input returns a queued-ack (e.g. from a corrupted/hydrated session stuck in Processing state) - Re-check thread.state under the mutable lock in the Processing arm to guard against the turn completing between the snapshot read and the queue operation Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: clear attachments on drain-loop queued message processing Queued messages are text-only (queued as strings during Processing state). The drain loop was reusing the original IncomingMessage reference which carried the first message's attachments, causing augment_with_attachments to incorrectly re-apply them to unrelated queued text. Clone the message with cleared attachments for drain-loop turns. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review round 2 — stale state fallthrough and thread-not-found guard - Processing arm: when re-checked state is no longer Processing, fall through to normal processing instead of dropping user input - Processing arm: return error when thread not found instead of false "queued" ack - Document intermediate drain-loop responses as best-effort for one-shot channels (HttpChannel) - Add regression tests for both edge cases Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review feedback for message queue drain loop [skip-regression-check] — test modifications present but hook has SIGPIPE/pipefail false negative when awk exits early on match - Replace wildcard match in drain loop with explicit `while let Ok(Response)` guard — stops on Error variant too, preventing confusing interleaved output after soft errors (review issue #1) - Reject queueing messages with attachments during Processing state instead of silently dropping them (review issue #2) - Document response routing limitation: all drain-loop responses route via original message identity (review issue #3) - Document why SubmissionResult::Ok is correct for queued ack and how it interacts with drain loop break condition (review issue #4) - Rewrite two dead regression tests to assert actual behavior: thread-gone returns error, state-changed does not queue (review #5) - Document MAX_PENDING_MESSAGES=10 as acceptable for personal assistant use case (review issue #6) - Fix misleading one-shot channel comment — HttpChannel consumes sender on first call, subsequent calls are dropped (review issue #8) - Simplify drain loop intermediate response since while-let guard guarantees Response variant Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: add missing extension_manager field in webhook EngineContext The fire_webhook method's EngineContext initializer was missing the extension_manager field added in staging, causing CI compilation failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: gate TestRig::session_manager() behind libsql feature flag The field is #[cfg(feature = "libsql")] so the accessor must match. All callers are already inside #[cfg(feature = "libsql")] blocks. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: re-queue drained messages on drain loop failure If process_user_input fails after drain_pending_messages() removed all queued content, that user input was permanently lost. Now the merged content is re-queued at the front of pending_messages on any non-Response result so it will be processed on the next successful turn. Adds Thread::requeue_drained() helper and unit test. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: remove unreachable!() from drain loop, add lock-drop comments - Extract content binding in `while let` pattern instead of using a separate match with unreachable!() — satisfies the no-panic-in- production convention (zmanian review item #1) - Add comment clarifying session lock is dropped at Processing arm boundary before fall-through (zmanian review item #5) - Document bounded cap overshoot on requeue_drained (review item #2) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(security): validate queued messages and touch updated_at on queue ops - Run safety validation, policy checks, and secret scanning on messages before queueing during Processing state. Previously, content with leaked secrets could be stored in pending_messages and serialized without hitting the inbound scanner. - Touch updated_at in queue_message(), drain_pending_messages(), and requeue_drained() so thread timestamps reflect queue activity. [skip-regression-check] — safety validation requires full Agent; updated_at is a data-level fix on existing tested methods Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-08-31 16:49:34 +00:00 · 2026-03-21 21:53:14 -07:00
parent 89394ebd29
commit ccdea40e9d
6 changed files with 702 additions and 16 deletions
@@ -10,7 +10,7 @@
 //! - Compaction: Summarize old turns to save context
 //! - Resume: Continue from a saved checkpoint

-use std::collections::{HashMap, HashSet};
+use std::collections::{HashMap, HashSet, VecDeque};

 use chrono::{DateTime, TimeDelta, Utc};
 use serde::{Deserialize, Serialize};
@@ -222,8 +222,17 @@ pub struct Thread {
    /// Pending auth token request (thread is in auth mode).
    #[serde(default)]
    pub pending_auth: Option<PendingAuth>,
+    /// Messages queued while the thread was processing a turn.
+    #[serde(default, skip_serializing_if = "VecDeque::is_empty")]
+    pub pending_messages: VecDeque<String>,
 }

+/// Maximum number of messages that can be queued while a thread is processing.
+/// 10 merged messages can produce a large combined input for the LLM, but this
+/// is acceptable for the personal assistant use case where a single user sends
+/// rapid follow-ups. The drain loop processes them as one newline-delimited turn.
+pub const MAX_PENDING_MESSAGES: usize = 10;
+
 impl Thread {
    /// Create a new thread.
    pub fn new(session_id: Uuid) -> Self {
@@ -238,6 +247,7 @@ impl Thread {
            metadata: serde_json::Value::Null,
            pending_approval: None,
            pending_auth: None,
+            pending_messages: VecDeque::new(),
        }
    }

@@ -254,6 +264,7 @@ impl Thread {
            metadata: serde_json::Value::Null,
            pending_approval: None,
            pending_auth: None,
+            pending_messages: VecDeque::new(),
        }
    }

@@ -272,6 +283,47 @@ impl Thread {
        self.turns.last_mut()
    }

+    /// Queue a message for processing after the current turn completes.
+    /// Returns `false` if the queue is at capacity ([`MAX_PENDING_MESSAGES`]).
+    pub fn queue_message(&mut self, content: String) -> bool {
+        if self.pending_messages.len() >= MAX_PENDING_MESSAGES {
+            return false;
+        }
+        self.pending_messages.push_back(content);
+        self.updated_at = Utc::now();
+        true
+    }
+
+    /// Take the next pending message from the queue.
+    pub fn take_pending_message(&mut self) -> Option<String> {
+        self.pending_messages.pop_front()
+    }
+
+    /// Drain all pending messages from the queue.
+    /// Multiple messages are joined with newlines so the LLM receives
+    /// full context from rapid consecutive inputs (#259).
+    pub fn drain_pending_messages(&mut self) -> Option<String> {
+        if self.pending_messages.is_empty() {
+            return None;
+        }
+        let parts: Vec<String> = self.pending_messages.drain(..).collect();
+        self.updated_at = Utc::now();
+        Some(parts.join("\n"))
+    }
+
+    /// Re-queue previously drained content at the front of the queue.
+    /// Used to preserve user input when the drain loop fails to process
+    /// merged messages (soft error, hard error, interrupt).
+    ///
+    /// This intentionally bypasses [`MAX_PENDING_MESSAGES`] — the content
+    /// was already counted against the cap before draining. The overshoot
+    /// is bounded to 1 entry (the re-queued merged string) plus any new
+    /// messages that arrived during the failed attempt.
+    pub fn requeue_drained(&mut self, content: String) {
+        self.pending_messages.push_front(content);
+        self.updated_at = Utc::now();
+    }
+
    /// Start a new turn with user input.
    pub fn start_turn(&mut self, user_input: impl Into<String>) -> &mut Turn {
        let turn_number = self.turns.len();
@@ -335,11 +387,12 @@ impl Thread {
        self.pending_auth.take()
    }

-    /// Interrupt the current turn.
+    /// Interrupt the current turn and discard any queued messages.
    pub fn interrupt(&mut self) {
        if let Some(turn) = self.turns.last_mut() {
            turn.interrupt();
        }
+        self.pending_messages.clear();
        self.state = ThreadState::Interrupted;
        self.updated_at = Utc::now();
    }
@@ -1392,4 +1445,165 @@ mod tests {
        );
        assert!(tool_result_content.ends_with("..."));
    }
+
+    #[test]
+    fn test_thread_message_queue() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Queue is initially empty
+        assert!(thread.pending_messages.is_empty());
+        assert!(thread.take_pending_message().is_none());
+
+        // Queue messages and verify FIFO ordering
+        assert!(thread.queue_message("first".to_string()));
+        assert!(thread.queue_message("second".to_string()));
+        assert!(thread.queue_message("third".to_string()));
+        assert_eq!(thread.pending_messages.len(), 3);
+
+        assert_eq!(thread.take_pending_message(), Some("first".to_string()));
+        assert_eq!(thread.take_pending_message(), Some("second".to_string()));
+        assert_eq!(thread.take_pending_message(), Some("third".to_string()));
+        assert!(thread.take_pending_message().is_none());
+
+        // Fill to capacity — all 10 should succeed
+        for i in 0..MAX_PENDING_MESSAGES {
+            assert!(thread.queue_message(format!("msg-{}", i)));
+        }
+        assert_eq!(thread.pending_messages.len(), MAX_PENDING_MESSAGES);
+
+        // 11th message rejected by queue_message itself
+        assert!(!thread.queue_message("overflow".to_string()));
+        assert_eq!(thread.pending_messages.len(), MAX_PENDING_MESSAGES);
+
+        // Drain and verify order
+        for i in 0..MAX_PENDING_MESSAGES {
+            assert_eq!(thread.take_pending_message(), Some(format!("msg-{}", i)));
+        }
+        assert!(thread.take_pending_message().is_none());
+    }
+
+    #[test]
+    fn test_thread_message_queue_serialization() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Empty queue should not appear in serialization (skip_serializing_if)
+        let json = serde_json::to_string(&thread).unwrap();
+        assert!(!json.contains("pending_messages"));
+
+        // Non-empty queue should serialize and deserialize
+        thread.queue_message("queued msg".to_string());
+        let json = serde_json::to_string(&thread).unwrap();
+        assert!(json.contains("pending_messages"));
+        assert!(json.contains("queued msg"));
+
+        let restored: Thread = serde_json::from_str(&json).unwrap();
+        assert_eq!(restored.pending_messages.len(), 1);
+        assert_eq!(restored.pending_messages[0], "queued msg");
+    }
+
+    #[test]
+    fn test_thread_message_queue_default_on_old_data() {
+        // Deserialization of old data without pending_messages should default to empty
+        let thread = Thread::new(Uuid::new_v4());
+        let json = serde_json::to_string(&thread).unwrap();
+
+        // The field is absent (skip_serializing_if), simulating old data
+        assert!(!json.contains("pending_messages"));
+        let restored: Thread = serde_json::from_str(&json).unwrap();
+        assert!(restored.pending_messages.is_empty());
+    }
+
+    #[test]
+    fn test_interrupt_clears_pending_messages() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Start a turn so there's something to interrupt
+        thread.start_turn("initial input");
+
+        // Queue several messages while "processing"
+        thread.queue_message("queued-1".to_string());
+        thread.queue_message("queued-2".to_string());
+        thread.queue_message("queued-3".to_string());
+        assert_eq!(thread.pending_messages.len(), 3);
+
+        // Interrupt should clear the queue
+        thread.interrupt();
+        assert!(thread.pending_messages.is_empty());
+        assert_eq!(thread.state, ThreadState::Interrupted);
+    }
+
+    #[test]
+    fn test_thread_state_idle_after_full_drain() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Simulate a full drain cycle: start turn, queue messages, complete turn,
+        // then drain all queued messages as a single merged turn (#259).
+        thread.start_turn("turn 1");
+        assert_eq!(thread.state, ThreadState::Processing);
+
+        thread.queue_message("queued-a".to_string());
+        thread.queue_message("queued-b".to_string());
+
+        // Complete the turn (simulates process_user_input finishing)
+        thread.complete_turn("response 1");
+        assert_eq!(thread.state, ThreadState::Idle);
+
+        // Drain: merge all queued messages and process as a single turn
+        let merged = thread.drain_pending_messages().unwrap();
+        assert_eq!(merged, "queued-a\nqueued-b");
+        thread.start_turn(&merged);
+        thread.complete_turn("response for merged");
+
+        // Queue is fully drained, thread is idle
+        assert!(thread.drain_pending_messages().is_none());
+        assert!(thread.pending_messages.is_empty());
+        assert_eq!(thread.state, ThreadState::Idle);
+    }
+
+    #[test]
+    fn test_drain_pending_messages_merges_with_newlines() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Empty queue returns None
+        assert!(thread.drain_pending_messages().is_none());
+
+        // Single message returned as-is (no trailing newline)
+        thread.queue_message("only one".to_string());
+        assert_eq!(
+            thread.drain_pending_messages(),
+            Some("only one".to_string()),
+        );
+        assert!(thread.pending_messages.is_empty());
+
+        // Multiple messages joined with newlines
+        thread.queue_message("hey".to_string());
+        thread.queue_message("can you check the server".to_string());
+        thread.queue_message("it started 10 min ago".to_string());
+        assert_eq!(
+            thread.drain_pending_messages(),
+            Some("hey\ncan you check the server\nit started 10 min ago".to_string()),
+        );
+        assert!(thread.pending_messages.is_empty());
+
+        // Queue is empty after drain
+        assert!(thread.drain_pending_messages().is_none());
+    }
+
+    #[test]
+    fn test_requeue_drained_preserves_content_at_front() {
+        let mut thread = Thread::new(Uuid::new_v4());
+
+        // Re-queue into empty queue
+        thread.requeue_drained("failed batch".to_string());
+        assert_eq!(thread.pending_messages.len(), 1);
+        assert_eq!(thread.pending_messages[0], "failed batch");
+
+        // New messages go behind the re-queued content
+        thread.queue_message("new msg".to_string());
+        assert_eq!(thread.pending_messages.len(), 2);
+
+        // Drain should return re-queued content first (front of queue)
+        let merged = thread.drain_pending_messages().unwrap();
+        assert_eq!(merged, "failed batch\nnew msg");
+    }
 }