Add the orchestrator infrastructure for replacing the Rust execution
loop with versioned Python code. This commit adds the module and host
functions without switching over — the existing Rust loop is unchanged.
New files:
- orchestrator/default.py: v0 Python orchestrator (run_loop + helpers)
- executor/orchestrator.rs: host function dispatch, orchestrator
loading from Store with version selection, OrchestratorResult parsing
Host functions exposed to orchestrator Python via Monty suspension:
__llm_complete__, __execute_code_step__ (nested Monty VM),
__execute_action__, __check_signals__, __emit_event__,
__add_message__, __save_checkpoint__, __transition_to__,
__retrieve_docs__, __check_budget__, __get_actions__
Also makes json_to_monty, monty_to_json, monty_to_string pub(crate)
in scripting.rs for cross-module use.
Design doc: docs/plans/2026-03-25-python-orchestrator.md
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* Fix REPL single-message hang and cap CI test duration
* Fix Clippy nested-if lint in REPL startup
* Fix single-message approval flow
* Handle empty single-message REPL exits
* Wait for one-shot event routines before exit
* Fix MCP lifecycle trace user scope
* Fix REPL single-message hang and cap CI test duration
* Fix Clippy nested-if lint in REPL startup
* Fix single-message approval flow
* Handle empty single-message REPL exits
* Wait for one-shot event routines before exit
* feat(agent): thread per-tool reasoning from LLM through to REPL, HTTP, SSE, and DB
Add end-to-end agent reasoning summaries so users can see *why* the
agent chose specific tools, not just what it did.
- Add `reasoning: Option<String>` to `ToolCall` (all providers)
- Populate from LLM response content in `Reasoning::respond_with_tools`
and `select_tools`, with per-tool override when providers supply it
- Extend `Turn` with `narrative` and `TurnToolCall` with `rationale` +
`tool_call_id` for identity-based result matching
- Persist reasoning in DB via existing tool_calls JSON (no migration)
- Add `StatusUpdate::ReasoningUpdate` and `SseEvent::ReasoningUpdate` +
`SseEvent::JobReasoning` for real-time streaming
- Emit reasoning events in both chat dispatcher and worker job path
- Add `/reasoning [N|all]` command for inspecting turn reasoning
- Surface `narrative` and `rationale` in HTTP `/api/chat/history`
Based on the design from #361 and #456, reconstructed cleanly with
Option<String> to minimize blast radius (vs mandatory String that broke
compilation in #456).
Closes#456
Co-Authored-By: panosAthDBX <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review feedback from Gemini and Copilot
- Fix `_ => Ok(None)` in agent_loop.rs to avoid accidental shutdown
- Fix fallback in record_tool_result_for/record_tool_error_for to use
first pending call instead of last_mut (parallel execution safety)
- Include per-tool decisions in WASM channel reasoning messages
- Apply truncate_at_tool_tags + clean_response to shared_reasoning in
select_tools (parity with respond_with_tools)
- Persist turn-level narrative to DB in tool_calls JSON wrapper
- Parse both old (array) and new (object) tool_calls formats in
build_turns_from_db_messages for backward compatibility
- Populate reasoning from action.reasoning in execute_plan ToolCalls
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address second round of review comments + merge fixes
- Add reasoning: None to new github_copilot.rs ToolCall sites (from staging merge)
- Run cargo fmt on 4 files with formatting diffs
- Truncate narrative to 1000 chars before DB persistence
- Clone turn data and drop session lock in /reasoning command
- Extract ToolDecisionDto::from_json_array shared helper (deduplicate
worker/job.rs and orchestrator/api.rs)
- Add unit tests for wrapped tool_calls JSON format with narrative
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address third round of review comments (Copilot + serrrfirat)
- Reword ToolCall.reasoning docstring to reflect provider-supplied or
fallback contract
- Sanitize narrative through SafetyLayer before storage/emission
- Clean per-tool reasoning via truncate_at_tool_tags + clean_response
in select_tools (parity with shared reasoning)
- Convert 4 approval-path recording sites in thread_ops.rs to
identity-based record_tool_result_for/record_tool_error_for
- Preserve tool_call_id and reasoning through restore_from_messages
- Fix has_result/has_error to reject JSON null values
- Truncate tool_call_id to 128 chars before DB persistence
- Add 4 unit tests for record_tool_result_for/error_for edge cases
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address zmanian review — sanitize JobDelegate reasoning + warn on dropped results
- Sanitize narrative and per-tool rationale through SafetyLayer in
JobDelegate reasoning events (parity with ChatDelegate)
- Add tracing::warn when record_tool_result_for/error_for drops a
result because no matching or pending tool call exists
- Add 3 unit tests for reasoning normalization (thinking tags,
tool tags, empty-after-cleaning)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address 4 remaining unreplied review comments
- Clean per-tool reasoning in respond_with_tools via truncate_at_tool_tags
+ clean_response (parity with select_tools)
- Handle wrapped JSON format in rebuild_chat_messages_from_db so cold
hydration works after persist_tool_calls format change
- Update persist_tool_calls doc comment to describe new JSON shape
- Sanitize per-tool rationale through SafetyLayer in ChatDelegate before
emission and storage (parity with JobDelegate)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address zmanian review round 2
- Add tracing::debug on fallback-to-pending path in record_tool_result_for
and record_tool_error_for (item 1)
- Add comment explaining why /reasoning is special-cased in agent_loop.rs
(item 4)
- Items 2 (narrative persistence), 3 (rationale sanitization), and 5
(catch-all fix) were already addressed in prior commits
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: panosAthDBX <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: ensure LLM calls always end with user message (closes#763)
Claude 4.6 models (claude-sonnet-4-6, claude-opus-4-6) no longer support
assistant message prefill — any LLM call where the conversation ends on an
assistant message is rejected with HTTP 400 "This model does not support
assistant message prefill".
The same root cause also triggers NEAR AI's "No user query found in messages"
400 error for the routine engine path.
Two fixes:
1. src/worker/container.rs — before_llm_call()
After poll_and_inject_prompt(), if no user follow-up arrived and
handle_text_response() left an assistant message at the end of the
conversation, inject a sentinel "Continue." user message before
the next LLM call.
2. src/agent/routine_engine.rs — execute_lightweight_with_tools()
Before the force_text final completion call, ensure messages end
with a user-role message. Tool result messages (Role::Tool) satisfy
Anthropic but not NEAR AI; assistant messages satisfy neither.
Also updates the worker system prompt to instruct the agent to include
the phrase "The job is complete" in its final message, so the agentic
loop can detect termination reliably.
Tested with claude-sonnet-4-6 and claude-opus-4-6.
Workaround: ANTHROPIC_MODEL=claude-sonnet-4-20250514 (still supports prefill).
* fix: broaden sentinel guard to any non-user message (per review)
Gemini suggested the Role::Assistant check in before_llm_call() is too
specific. Changed to !Role::User to match the routine_engine.rs fix and
cover tool results too.
* fix: address zmanian review — JobDelegate sentinel, shared helper, NearAI complete() flattening
- Extract ensure_ends_with_user_message() to src/util.rs with 4 unit tests
(empty list, after assistant, after tool result, no-op when already user)
- Add sentinel guard to JobDelegate::before_llm_call() in src/worker/job.rs
so scheduler jobs (CreateJob / /job path) no longer hit Claude 4.6 / NEAR AI 400s
- Replace inline guards in ContainerDelegate and routine_engine.rs with the
shared helper — all 3 call sites now use one implementation
- Fix complete() in nearai_chat.rs to apply flatten_tool_messages when
flatten_tool_messages=true — previously only complete_with_tools() flattened,
so force_text paths could still send role:"tool" messages to NEAR AI
- Update stale comment in container.rs: "assistant message" → "non-user message"
- Add flatten tests in nearai_chat.rs covering the complete() path
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* ci: fix fmt and tar advisory
---------
Co-authored-by: Jacob Lasky <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
Wire the self-improvement loop as a Mission with OnSystemEvent cadence,
inspired by karpathy/autoresearch's program.md approach. The mission
fires when threads complete with issues, receives trace data as trigger
payload, and uses tools directly to diagnose and fix problems.
Key changes:
Engine self-improvement (Phase A+B from design doc):
- Add fire_on_system_event() to MissionManager for OnSystemEvent cadence
- Add start_event_listener() that subscribes to thread events and fires
matching missions when non-Mission threads complete with trace issues
- Add ensure_self_improvement_mission() with autoresearch-style goal
prompt (concrete loop steps, not vague instructions)
- Add process_self_improvement_output() for structured JSON fallback
- Seed fix pattern database with 8 known patterns from debugging
- Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now
async + Store-aware, appends learned rules from prompt_overlay docs)
- Pass Store to ExecutionLoop for overlay loading
Bridge review fixes (P1/P2):
- Scope engine v2 SSE events to requesting user (broadcast_for_user)
- Per-user pending approvals via HashMap instead of global Option
- Reset tool-call limit counter before each thread execution
- Only persist auto-approval when user chose "always", not one-off "yes"
- Remove dead store/mission_manager fields from EngineState
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: extract AppEvent to crates/ironclaw_common
SseEvent was defined in src/channels/web/types.rs but imported by 12+
modules across agent, orchestrator, worker, tools, and extensions — it
had become the application-wide event protocol, not a web transport
concern.
Create crates/ironclaw_common as a shared workspace crate and move the
enum there as AppEvent. Also move the truncate_preview utility which
was similarly leaked from the web gateway into agent modules.
- New crate: crates/ironclaw_common (AppEvent, truncate_preview)
- Rename SseEvent → AppEvent, from_sse_event → from_app_event
- web/types.rs re-exports AppEvent for internal gateway use
- web/util.rs re-exports truncate_preview
- Wire format unchanged (serde renames are on variants, not the enum)
Aligned with the event bus direction on refactor/architectural-hardening
where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: add AppEvent::event_type() helper, deduplicate match blocks
Address Gemini review: extract the variant→string match into a single
method on AppEvent, replacing the duplicated 22-arm matches in sse.rs
and types.rs.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: rename leftover sse vars/tests to match AppEvent rename
Address Copilot review: rename sse_event vars to app_event in
orchestrator/api.rs and ws.rs, rename test functions from
test_ws_server_from_sse_* to test_ws_server_from_app_event_*, and
update stale SSE comments.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: add Deserialize to AppEvent, round-trip test, fix stale comments
Address zmanian review:
- Add Deserialize derive to AppEvent so downstream consumers can
deserialize incoming events
- Add event_type_matches_serde_type_field test that round-trips every
variant through serde and asserts event_type() matches the serialized
"type" field — catches drift between serde renames and the manual match
- Add round_trip_deserialize test for basic Serialize/Deserialize parity
- Update remaining "SSE" references in comments across server.rs,
manager.rs, ws_gateway_integration.rs, and worker/job.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat(cli): show credential auth status in `tool info`
`ironclaw tool info` now checks the secrets store and shows whether
each required credential is configured or missing, consolidated into
a single Auth section that deduplicates across http.credentials,
auth, and setup.required_secrets. Secrets already shown in Auth are
filtered from the Secrets section to avoid redundancy.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(cli): address review feedback on tool info auth status
- Fix clippy collapsible-if by using `if let` + `&&`
- Use HashMap<String, usize> for O(1) dedup instead of HashSet + linear scan
- Add --user flag to `tool info` for checking non-default user credentials
- Show "? unknown" on secrets store errors instead of silently reporting missing
- Surface secrets store init failure via eprintln instead of silent .ok()
- Sort auth entries by secret name for deterministic output
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(cli): only filter secrets when auth section renders, add regression test
When the secrets store fails to initialize, the Auth section is not
rendered. Previously, secret names were still filtered from the Secrets
section, causing credential names to disappear entirely. Now secrets
are only filtered when the Auth section will actually be displayed.
Adds test verifying auth secret deduplication across auth, setup, and
http.credentials sections, plus secrets store existence checks.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor(cli): extract collect_auth_secrets helper, always render Auth section
Address review feedback:
- Extract dedup logic into `collect_auth_secrets()` so the test exercises
the same code path as production (not a re-implementation)
- Always render the Auth section when auth secrets exist, showing
"? unknown" status when the secrets store is unavailable instead of
hiding credential names entirely
- Lazily init secrets store only when capabilities contain auth secrets,
avoiding spurious warnings for tools with no auth
- Add test for empty capabilities edge case
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style(cli): move HashMap/HashSet imports to top of file
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(cli): use correct tagged JSON format for credential location in test
The CredentialLocationSchema uses serde tagged enum format
({"type": "bearer"}), not a bare string ("AuthorizationBearer").
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
When the model calls routine_create, routine_list, routine_fire,
routine_pause, routine_resume, or routine_delete, the bridge now
routes them to the MissionManager instead of blocking with an error.
Mapping:
routine_create → mission_create (with cadence parsing)
routine_list → mission_list
routine_fire → mission_fire
routine_pause → mission_pause
routine_resume → mission_resume
routine_update → mission_pause/resume (based on params)
routine_delete → mission_complete (marks as done)
Routine tools removed from v1-only blocklist and restored in
available_actions(). The model can use either "routine" or "mission"
vocabulary — both work.
Still blocked: create_job, cancel_job, build_software (need v1
Scheduler/ContainerJobManager refs).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: remove stale stream_token gate from channel-relay activation
The relay architecture now uses instance-scoped bearer auth + webhook
callbacks, not streaming. The `relay:<name>:stream_token` secret was
never written by the current OAuth flow, so activation always failed
with AuthRequired.
Replace stream_token with the team_id setting (already stored by the
OAuth callback) as the persistent "auth completed" marker:
- is_relay_channel(): check team_id setting instead of stream_token secret
- activate_channel_relay(): gate on team_id emptiness, not stream_token
- removal flow: delete team_id setting + oauth_state secret
- configure(): return empty allowed-secrets set (relay is OAuth-only)
- configure_token(): return AuthRequired (no manual token entry)
- list(): surface activation_error for relay channels (was hardcoded None)
- Clean up stale comments referencing stream_token / "stored token"
- Update test to match OAuth-only model (no secrets to pass)
Made-with: Cursor
* fix: address CI and review feedback
- Fix pre-existing tunnel/mod.rs test compilation (missing GatewayConfig
fields: memory_layers, user_tokens, workspace_read_scopes)
- Log warnings on failed team_id/oauth_state cleanup during removal
instead of silently ignoring errors (gemini review)
- Also delete legacy stream_token secret during removal for backward
compatibility with pre-webhook installs (codex review)
Made-with: Cursor
Missions are now callable from CodeAct Python code:
```python
# Create a daily briefing mission
result = mission_create(
name="Tech News",
goal="Daily AI/crypto/software news briefing",
cadence="0 9 * * *"
)
# List all missions
missions = mission_list()
# Manually fire a mission
mission_fire(id="...")
# Pause/resume
mission_pause(id="...")
mission_resume(id="...")
```
Implementation:
- MissionManager created on engine init, cron ticker started
- EffectBridgeAdapter intercepts mission_* function calls before tool
lookup and routes to MissionManager
- parse_cadence() handles: "manual", cron expressions, "event:pattern",
"webhook:path"
- Mission functions documented in CodeAct system prompt
- MissionManager set on adapter via set_mission_manager() after init
(avoids circular dependency)
System prompt updated with mission_create, mission_list, mission_fire,
mission_pause, mission_resume documentation.
151 tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
