Three major changes to the v2 engine:
1. **Consolidated action execution** — `handle_execute_action` in Rust is now
the single source of truth for lease lookup, policy check, lease consumption,
action execution, event emission, and ActionResult message recording. The
Python orchestrator no longer duplicates event/message logic. This fixes the
empty call_id bug (OpenAI HTTP 400) and the missing tool_calls on assistant
messages (Codex "No tool call found" error).
2. **Removed reflection system** — Deleted the per-thread reflection pipeline
(pipeline.rs, executor.rs), ThreadState::Reflecting, ThreadType::Reflection,
enable_reflection config, and all 3 reflection event kinds. Learning is now
handled entirely by event-driven missions that fire selectively.
3. **Three learning missions** replace reflection:
- `self-improvement` — fires on trace issues (error diagnosis, prompt fixes)
- `playbook-extraction` — fires on successful 5+ step threads (reusable procedures)
- `conversation-insights` — fires every 5 threads per project (user preferences,
domain knowledge, workflow patterns)
Additional fixes:
- llm_query()/llm_query_batched() always include system message (Codex compat)
- handle_llm_complete adds assistant message with structured action_calls for
Tier 0 responses (prevents "No tool call found" errors)
- Gateway broadcasts without thread_id emit as Status events instead of being dropped
- Comprehensive tests for call_id propagation and trace analysis (17 new tests)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
When LLM-generated code calls `mission_list()` or any tool function,
Monty's Python execution model first resolves the name (`mission_list`)
as a NameLookup before invoking it as a FunctionCall. The NameLookup
handler always returned Undefined, causing NameError before the function
call could dispatch to the effect executor.
Fix: before starting the Monty VM, collect all known tool names from
the effect executor's available_actions(). In the NameLookup handler,
if the name matches a known tool, return a MontyObject::Function stub
instead of Undefined. Monty then yields FunctionCall for the stub,
which dispatches to the normal tool execution pipeline.
This enables CodeAct code to call any registered tool as a Python
function: mission_list(), mission_create(), routine_list(), web_search(),
memory_search(), etc. — all without explicit imports or __execute_action__
boilerplate.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The gateway UI showed only generic "Thinking..." during engine v2
execution with no visibility into CodeAct code execution, tool calls,
or reflection. Now the event mapping produces detailed status updates:
Step lifecycle:
- "Calling LLM..." when a step starts (was "Thinking...")
- "Step complete — N in / M out tokens" when done (was "Processing...")
Tool execution:
- Emit ToolStarted + ToolCompleted SSE events so the frontend renders
proper tool cards with spinner → checkmark/error transitions
- Duration shown in parameters field (e.g., "42ms")
CodeAct visibility:
- "Executing code..." when assistant produces a code block
- "Code executed" / "Code executed (no output)" for successful runs
- "Code error — retrying..." when Monty raises an exception
Reflection:
- "Reflecting on execution..." when post-thread analysis starts
- "Reflection complete — N insight(s) saved" when done
Also refactored thread_event_to_app_event → thread_event_to_app_events
(returns Vec<AppEvent>) to support emitting ToolStarted before
ToolCompleted in a single event handler pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Messages sent from a new conversation in the gateway always appeared in
the default assistant conversation because handle_with_engine ignored
the thread_id from the frontend.
Two fixes:
1. Engine conversation scoping — when the message carries a thread_id
(from the frontend's conversation picker), use it as part of the
engine conversation key: "gateway:<thread_id>" instead of just
"gateway". This creates a distinct engine conversation per v1
thread, so messages don't cross-contaminate.
2. V1 dual-write targeting — write user messages and assistant
responses to the v1 conversation matching the thread_id (via
ensure_conversation), not the hardcoded assistant conversation.
Falls back to the assistant conversation when no thread_id is
present (e.g., default chat).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Each browser tab opened 2 SSE connections (chat events + log events).
With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the
pool and couldn't load any data.
Three changes:
1. Lazy log SSE — only connect when the logs tab is active, disconnect
when switching away. Most users rarely view logs, so this saves a
connection slot per tab.
2. Visibility API — close SSE when the browser tab goes to background
(user switches to another tab), reconnect when it becomes visible.
Background tabs don't need real-time events.
3. Combined with the existing beforeunload cleanup, this means:
- Active foreground tab: 1 connection (chat SSE only, +1 if logs tab)
- Background tabs: 0 connections
- Closed/refreshed tabs: 0 connections (beforeunload cleanup)
This allows many gateway tabs to coexist within the 6-connection limit.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The browser limits concurrent HTTP/1.1 connections per origin to 6.
Without cleanup, SSE connections from prior page loads linger after
refresh/navigation, eating into the pool. After 2-3 refreshes, all 6
slots are consumed by stale SSE streams and new API fetch calls queue
indefinitely — the UI shows "connected" (SSE works) but data never
loads.
Add a beforeunload handler that closes both eventSource (chat events)
and logEventSource (log stream) so the browser can reuse connections
immediately on page reload.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Goal rendered as full-width markdown block instead of plain-text
meta item (uses existing renderMarkdown/marked)
- Current focus and success criteria also rendered as markdown
- Spawned threads shown as a clickable table with goal, type, state,
steps, tokens, and created date instead of a UUID list
- Clicking a thread row opens an inline thread detail view showing
metadata grid and full message history with markdown rendering
- Back button returns to the mission detail view
- Backend: mission detail now returns full thread summaries (goal,
state, step_count, tokens) instead of just thread IDs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The gateway API endpoints (/api/engine/missions, etc.) call bridge
query functions that return empty results when the engine state hasn't
been initialized yet. Previously, initialization only happened lazily
on the first chat message via handle_with_engine().
Now when ENGINE_V2=true, the engine is initialized in Agent::run()
before channels start, so the self-improvement mission and other
engine state is available to gateway API endpoints immediately.
Also rename get_or_init_engine → init_engine and make it public so
it can be called from agent_loop.rs at startup.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add a full Missions page to the web gateway with list view, detail view,
and action buttons (Fire, Pause, Resume).
Backend: add /api/engine/missions/summary endpoint returning counts by
status (active/paused/completed/failed).
Frontend:
- New "Missions" tab between Jobs and Routines
- Summary cards showing mission counts by status
- Table with name, goal, cadence type, thread count, status, actions
- Detail view with goal, cadence, current focus, success criteria,
approach history, spawned thread list, and action buttons
- Fire/Pause/Resume actions with toast notifications
- i18n support (English + Chinese)
- CSS following the existing routines/jobs patterns
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Fix structured executor not stamping call_id onto ActionResult — the
EffectExecutor trait doesn't receive call_id, so the structured executor
must copy it from the original ActionCall after execution. Empty call_id
caused OpenAI-compatible providers to reject the next LLM request with
"Invalid 'input[2].call_id': empty string".
Fix trace analyzer false positives:
- code_error check now only scans User-role code output messages
(prefixed with [stdout]/[stderr]/[code ]/Traceback), not System
prompt which contains example error text
- missing_tool_output check now recognizes ActionResult messages as
valid tool output (Tier 0 structured path)
- Add NotImplementedError to detected code error patterns
New trace checks:
- empty_call_id: detect ActionResult messages with missing/empty
call_id before they reach the LLM API (severity: Error)
- llm_error: extract LLM provider errors from Failed state reason
- orchestrator_error: extract orchestrator errors from Failed state
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Wire engine v2 into the full submission pipeline and expose threads,
projects, and missions through the web gateway REST API.
Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear
submissions to engine v2 when ENGINE_V2=true. Previously only UserInput
and ApprovalResponse were handled; all other control commands fell
through to disconnected v1 sessions.
Bridge query layer — add 11 read-only query functions and 6 DTO types
so gateway handlers can inspect engine state (threads, steps, events,
projects, missions) without direct access to the EngineState singleton.
Gateway endpoints — new /api/engine/* routes:
GET /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events
GET /projects, /projects/{id}
GET /missions, /missions/{id}
POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume
SSE events — add ThreadStateChanged, ChildThreadSpawned, and
MissionThreadSpawned AppEvent variants. Expand the bridge event mapper
to forward StateChanged and ChildSpawned engine events to the browser.
Engine crate — add ConversationManager::clear_conversation() for /new
and /clear commands.
Code quality — replace 10 .expect() calls with proper error returns,
remove dead AgentConfig.engine_v2 field, log silent init errors, fix
duplicate doc comment, improve fallthrough documentation.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(extensions): channel-relay auth dead-end, add observability and relay URL override
Fix a bug where clicking Activate on the Slack relay extension produces
a dead-end "Authentication required" error with no OAuth URL. The root
cause: `auth_channel_relay()` used `is_relay_channel()` to check auth
status, but that function returns true as soon as the extension is
*installed* (in-memory set), before OAuth completes. This short-circuits
the OAuth flow so the authorization URL is never offered.
Changes:
1. **Bug fix** — `auth_channel_relay()` now uses `has_stored_team_id()`
which only checks the persistent settings store for an actual team_id.
The extension list `authenticated` field uses the same check so the UI
accurately reflects OAuth completion status.
2. **Observability** — Added debug/warn/info tracing to all channel-relay
code paths that were previously silent on failure:
- `activate_channel_relay`: team_id retrieval, relay config, signing
secret fetch, hot_add, cache operations
- `auth_channel_relay`: auth check, OAuth initiation, nonce storage
- `extensions_activate_handler`: request entry, auth fallback flow
- `slack_relay_oauth_callback_handler`: team_id persistence (was
silently ignored with `let _`)
- `RelayClient`: initiate_oauth, get_signing_secret, proxy_provider
all log URL, status, and errors
- `has_stored_team_id`: store read success/failure
3. **Per-extension relay URL override** — Users can now override the
CHANNEL_RELAY_URL via Settings > Extensions > Reconfigure. Stored
under `extensions.{name}.relay_url` in settings. Both auth and
activate read this override before falling back to the env default.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address review feedback — clear relay_url override and improve log message
1. Allow clearing the relay_url override: when an optional setup field
with a setting_path is submitted empty, delete the stored setting so
the system reverts to the env/default value. Previously empty values
were silently skipped, making it impossible to undo an override from
the UI.
2. Improve the OAuth callback team_id persistence error log to be
self-contained without referencing implementation details.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: collapse nested if per clippy::collapsible_if
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address review feedback — security, scope consistency, and error handling
1. OAuth callback team_id persistence is now fatal: if set_setting fails,
the callback returns an error instead of proceeding to activate (which
would re-read from the store and fail anyway).
2. effective_relay_url uses owner scope (self.user_id) for reads, matching
configure() which writes under the same scope. Prevents multi-user
mismatch where an override saved via Reconfigure was invisible during
auth/activation.
3. has_stored_team_id uses owner scope for the same reason — the OAuth
callback stores team_id under state.owner_id (= self.user_id).
4. Security: effective_relay_url validates the override URL — only
http/https without embedded credentials (userinfo) is accepted. This
prevents API-key exfiltration if a user points relay_url at an
attacker-controlled host. Logs only host portion, not full URL.
5. Fixed effective_relay_url docstring to match behavior (returns Option,
callers handle the fallback).
6. get_setup_schema for ChannelRelay now logs a warning on settings store
errors instead of silently returning None.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling
Finishes the remaining isolation work from phases 2–4 of #59:
Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.
Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.
Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: use selected_model setting key to match /model command persistence
The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override
Three follow-up fixes for multi-tenant isolation:
1. Multi-user heartbeat now runs memory hygiene per user before each
heartbeat check, matching single-user heartbeat behavior.
2. /model command in multi-tenant mode only persists to per-user
settings (selected_model) without calling set_model() on the shared
LlmProvider. The per-request model_override in the dispatcher reads
from the same setting. Added multi_tenant flag to AgentConfig
(auto-detected from GATEWAY_USER_TOKENS).
3. RigAdapter now supports per-request model overrides by injecting the
model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
API servers use last-key-wins for duplicate JSON keys, so the override
takes effect via serde's flatten serialization order.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review — cost model attribution, heartbeat concurrency, pruning
Fixes from review comments on #1614:
- Cost tracking now uses the override model name (not active_model_name)
when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: /status ownership, model persistence scoping, heartbeat robustness
Addresses second round of PR review on #1614:
- /status <job_id> DB path now validates job.user_id == requesting user
before returning data (was missing ownership check, security fix).
- persist_selected_model takes user_id param instead of owner_id, and
skips .env/TOML writes in multi-tenant mode (these are shared global
files). handle_system_command now receives user_id from caller.
- JoinSet collection handles Err(JoinError) explicitly instead of
silently dropping panicked tasks.
- Notification forwarder extracts owner_id from response metadata in
multi-tenant mode for per-user routing instead of broadcasting to
the agent owner.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap
Round 3 review fixes:
- Cost tracking passes None for cost_per_token when model override is
active, letting CostGuard look up pricing by model name instead of
using the default provider's rates (serrrfirat).
- fire_manual() now uses per-user workspace, matching spawn_fire()
pattern (serrrfirat).
- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).
- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
the LLM provider (serrrfirat + Copilot).
- Fixed inject_model_override doc comment accuracy (Copilot).
- Added comment explaining multi-tenant notification routing priority
(Copilot).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat: user-scoped webhook endpoint for multi-tenant isolation
Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.
The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.
Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
services that can't send bearer tokens)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat: add TenantCtx for compile-time tenant isolation
Implements zmanian's architectural proposal from #1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.
TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.
AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).
TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.
Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Three new docs for contributors:
- engine-v2-architecture.md: Two-layer architecture (Rust kernel +
Python orchestrator), five primitives, execution model with nested
Monty VMs, bridge layer, memory/reflection, missions, capabilities
- self-improvement.md: Three improvement levels (prompt/orchestrator/
config/code), autoresearch-inspired Mission loop, versioned
orchestrator with auto-rollback, fix pattern database, safety model
- development-history.md: Summary of 6 Claude Code sessions that
built the system, key design decisions and debugging moments,
architecture evolution from 900-line Rust loop to Python orchestrator
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Add version lifecycle for the Python orchestrator:
- Failure tracking via MemoryDoc (orchestrator:failures)
- Auto-rollback: after 3 consecutive failures, skip the latest version
and fall back to previous (or compiled-in v0)
- Success resets the failure counter
- OrchestratorRollback event for observability
Update self-improvement Mission goal with Level 1.5 instructions for
orchestrator patches — the agent can now modify the execution loop
itself via memory_write with versioned orchestrator docs.
12 new tests: version selection (highest wins), rollback after failures,
rollback to default, failure counting/resetting, outcome parsing for
all 5 ThreadOutcome variants.
189 tests pass, zero clippy warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Increment step_count and track tokens in __emit_event__("step_completed")
so thread bookkeeping matches the old Rust loop behavior
- Remove double-counting of tokens in bootstrap (orchestrator handles it)
- Match nudge text to existing TOOL_INTENT_NUDGE constant
- Fix FINAL result propagation (use stored final_result, not VM return)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Replace the 900-line Rust execution loop with a ~80-line bootstrap
that loads and runs the versioned Python orchestrator via Monty VM.
The orchestrator Python code (orchestrator/default.py) is the v0
compiled-in version. Runtime versions can override it via MemoryDoc
storage (orchestrator:main with tag orchestrator_code).
Key fixes during switchover:
- Use ExtFunctionResult::NotFound for unknown functions so Monty
falls through to Python-defined functions (extract_final, etc.)
- Move helper function definitions above run_loop for Monty scoping
- Use FINAL result value (not VM return value) in Complete handler
- Rename 'final' variable to 'final_answer' to avoid Python keyword
Status: 171/177 tests pass. 6 remaining failures are step_count and
token tracking bookkeeping — the orchestrator manages these internally
but doesn't yet update the thread's counters via host functions.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
