* feat: receive relay events via webhook callbacks instead of SSE
Replace the SSE pull model with push-based webhook callbacks from
channel-relay. Eliminates the reconnect loop, stream token auth,
and SSE parser — events arrive via HTTP POST to /relay/events.
- Add webhook handler with HMAC signature verification
- Simplify RelayChannel to use mpsc from webhook handler
- Remove SSE connect/reconnect/parse logic from RelayClient
- Add register_callback() to RelayClient for callback URL registration
- Update activation flow to create event channel and register callback
- Wire relay webhook endpoint into web gateway
* fix: address review feedback on webhook callback PR
- Return 503 when relay event channel is full/closed (enables retry)
- Reject malformed timestamps with 400 instead of proceeding
- Allow relay activation without settings store (no-store/ephemeral mode)
- Check installed_relay_extensions set in is_relay_channel for no-db mode
- Fix staging test constructors for new RelayChannel signature
* security: adapt relay client to new channel-relay auth model
Adapts the relay integration to the hardened channel-relay security model:
- Switch from X-API-Key header to Authorization: Bearer sk-agent-*
for all relay API calls (chat-api token verification)
- Remove register_callback() — PUT /callbacks endpoint removed
- Remove event_callback_url from initiate_oauth() — parameter removed
- Make signing_secret a required field in RelayConfig (new env var:
CHANNEL_RELAY_SIGNING_SECRET)
- Update integration tests for Bearer auth and removed endpoints
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: use server-side approval tokens, remove caller-supplied routing
- Approval flow now calls POST /approvals to register server-side
record, then embeds only the opaque approval_token in button value
- Remove instance_id parameter from proxy_provider() — channel-relay
no longer accepts it (uses verified identity)
- Remove instance_id and user_id from initiate_oauth() — channel-relay
derives them from the Bearer token
- Add create_approval() to RelayClient
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: pass webhook_url during OAuth so callback_url is set on connection
The channel-relay OAuth flow now accepts webhook_url to set the
callback_url during connection creation. IronClaw computes its webhook
URL from callback_base + webhook_path and passes it during initiate_oauth.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove webhook_url from OAuth initiation
Channel-relay now derives the callback URL from chat-api's instance_url.
IronClaw no longer supplies webhook_url during OAuth — the relay is the
authority on where events get delivered.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* chore: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove all URL params from OAuth initiation
IronClaw no longer supplies any URLs to channel-relay. The relay
derives all URLs from the trusted instance_url in chat-api.
initiate_oauth() takes no parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: restore CSRF nonce for OAuth callback validation
Re-add nonce generation and secret storage in auth_channel_relay.
The nonce is passed to channel-relay as state_nonce param (not a URL).
Channel-relay embeds it in the signed state and appends it to the
redirect URL so IronClaw's callback handler can validate and activate.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: per-instance callback signing secrets
relay_signing_secret() now prefers OPENCLAW_GATEWAY_TOKEN (per-instance)
over the shared CHANNEL_RELAY_SIGNING_SECRET. A compromised instance
can no longer forge callbacks to other instances on the same relay.
CHANNEL_RELAY_SIGNING_SECRET is now optional in RelayConfig.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: clean per-instance callback secrets, no shared secrets, no fallbacks
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: pass team_id to get_signing_secret for workspace-scoped lookup
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove sender_id from create_approval — relay derives it
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: remove stale relay sender_id validation
* fix: harden relay webhook activation lifecycle
---------
Co-authored-by: Pierre <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
The HTTP tool returned `ApprovalRequirement::Always` for requests with
credentials, but `Always` is hardcoded to ignore the session auto-approve
set. This meant users who clicked "always" were re-prompted on every
subsequent HTTP call — the UI offered "always" but the backend ignored it.
Two fixes:
1. HTTP credentialed requests now return `UnlessAutoApproved` instead of
`Always`, so the session auto-approve set is respected.
2. `StatusUpdate::ApprovalNeeded` now carries `allow_always: bool`. All
channel UIs (Telegram, Slack, Signal, REPL, Web) conditionally hide
the "always" option when a tool truly requires per-invocation approval
(`ApprovalRequirement::Always`, e.g. destructive shell commands).
Also boxes `PendingApproval` in `AgenticLoopResult::NeedApproval` to fix
a pre-existing clippy `large_enum_variant` warning.
Regression tests included (test_credentialed_requests_respect_auto_approve,
test_allow_always_matches_approval_requirement) but CI heuristic cannot
detect them in cross-fork PR diffs.
[skip-regression-check]
Co-authored-by: Tyler <[email protected]>
* fix: skip NEAR AI session check when backend is not nearai
When a user configures a non-NEAR AI backend (e.g. Anthropic), the
doctor command was incorrectly failing with "session file not found"
even though no NEAR AI session is needed. The check now skips with a
descriptive message when LLM_BACKEND is not nearai/near_ai/near.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* fix(ci): avoid holding sync MutexGuard across await in doctor test
Convert check_nearai_session_skips_for_non_nearai_backend from
#[tokio::test] to #[test] with block_on, matching the pattern used by
all other ENV_MUTEX tests. Fixes clippy::await_holding_lock error.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Kristian Glass <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
- Fix empty text part for assistant messages with tool calls
(curate_contents could drop entire model turn)
- Propagate cache_read/creation_input_tokens in complete_with_tools
- Log warning on save_credential failure instead of silently ignoring
- Fix doc comment to mention underscore in header name pattern
- Handle gemini-oauth (hyphen variant) in setup wizard display
- Fix docs: thinkingConfig uses thinkingBudget/thinkingLevel, not
includeThoughts
* feat(self-repair): wire stuck_threshold, store, and builder (#647)
Wire the previously dead-code fields in DefaultSelfRepair:
- stuck_threshold: detect_stuck_jobs() now filters by duration, only
reporting jobs stuck longer than the configured threshold
- with_store(): wired in agent_loop.rs from AgentDeps.store for
tool failure tracking via Database trait
- with_builder(): wired from register_builder_tool() return value
through AppComponents and AgentDeps for automatic tool rebuilding
- tools: passed alongside builder for hot-reload logging
Remove all #[allow(dead_code)] annotations. Add regression tests for
threshold-based filtering (both above and below threshold).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: add missing `builder` field to AgentDeps in gateway workflow harness
After rebase onto staging, AgentDeps gained a `builder` field for
self-repair tool rebuilding. The gateway workflow test harness was
missing this field, causing CI compilation failure.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: retrigger CI
* fix: force CI refresh after path_routing_tests dedup
* test: add E2E test for stuck job repair and tool rebuild cycle
Tests the full self-repair flow requested in review:
1. Job transitions Pending -> InProgress -> Stuck
2. detect_stuck_jobs() finds it (zero threshold)
3. repair_stuck_job() recovers it back to InProgress
4. A broken tool is repaired via MockBuilder
5. Verify builder was invoked and repair succeeded
Uses a MockBuilder (impl SoftwareBuilder) that returns successful
BuildResult without requiring an LLM or filesystem. Uses libsql
test database for the store (increment_repair_attempts, mark_tool_repaired).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(self-repair): measure stuck_duration from Stuck transition, not started_at
- Use ctx.transitions to find the most recent Stuck transition timestamp
instead of ctx.started_at (which reflects job start, not stuck time)
- Fix StuckJob.last_activity to use stuck transition timestamp
- Remove misleading "hot-reloaded into registry" log
- Remove stray "// ci fix" comment in memory.rs
- Add regression test: backdated started_at must not inflate stuck_duration
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: re-trigger CI with latest changes
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: add type annotation to Ok(()) in test to resolve E0282
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* feat(testing): add FaultInjector framework for StubLlm (#1220)
Adds a configurable fault injection framework for testing retry, failover,
and circuit breaker behavior. The FaultInjector attaches to StubLlm and
provides per-call control over failure type, timing, and sequencing.
Components:
- FaultType: maps to LlmError variants (RequestFailed, RateLimited,
AuthFailed, InvalidResponse, IoError, ContextLengthExceeded, SessionExpired)
- FaultAction: Succeed, Fail(FaultType), Delay(Duration)
- FaultMode: SequenceOnce (play then succeed), SequenceLoop (repeat forever),
Random (seeded xorshift64 PRNG for reproducibility)
- FaultInjector: thread-safe (AtomicU32 counter + Mutex RNG)
Integration:
- StubLlm gains optional fault_injector field via with_fault_injector()
- When set, takes precedence over should_fail/error_kind
- Backward compatible: existing StubLlm usage unchanged
Closes#1220
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor(testing): address review feedback on FaultInjector
- Remove redundant .abs() in random fault comparison
- Extract check_faults() helper to DRY up StubLlm methods
- Guard xorshift seed=0 (fixed point) by mapping to 1
- Add StubLlm integration test (stub_llm_fault_injector_sequence)
- Remove dead seed field from FaultMode::Random
- Move pub mod fault_injection to top of mod.rs
- Add Debug impl for FaultInjector
- Add empty_sequence_always_succeeds test
- Add random_seed_zero_does_not_always_fail test
* fix(testing): address #1233 review -- seed-0 bug, reset(), Debug derive
- Store seed in FaultMode::Random so reset() can re-init the RNG
- Add reset() method for test reproducibility (re-seeds RNG, zeros counter)
- Strengthen seed=0 regression test to 100 iterations with stricter assertion
- Add reset_restores_random_rng_from_stored_seed test
- Debug impl and empty_sequence test were already present from prior commit
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* ci: re-trigger CI with latest changes
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: trigger new run with skip-regression-check label
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(testing): address PR #1233 review -- error_rate validation and edge cases
- Validate error_rate is in 0.0..=1.0 and not NaN (panics on invalid input)
- Fix error_rate==1.0 edge case: use <= instead of < so 1.0 always fails
- Add regression tests for error_rate validation (NaN, negative, >1.0)
- Add tests for error_rate boundary values (0.0 never fails, 1.0 always fails)
- Add delay action test using tokio::time::pause() for deterministic timing
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: remove debug_assert guards that panic on valid error paths (#1312)
Two debug_assert! calls added in #1312 fire on expected runtime error
paths (not programmer bugs), turning graceful error returns into panics
in debug/test builds:
- state.rs: Completed→Cancelled is a user-facing error handled by
transition_to() returning Err — not a bug
- execute.rs: empty tool_name from malformed LLM output is handled by
ToolError::NotFound — not a bug
Removes both asserts; keeps the circuit-breaker assert (genuinely guards
a caller invariant).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: tighten empty tool name test to assert ToolError::NotFound variant
Address review feedback: assert the specific error variant instead of
just is_err() so the regression test actually enforces the expected
error path.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat(gateway): full settings page polish with all tiers
- Backend: add ActiveConfigSnapshot to expose resolved LLM backend,
model, and enabled channels via /api/gateway/status
- Add missing Agent settings (daily cost cap, actions/hour, local tools)
- Add Sandbox, Routines, Safety, Skills, and Search setting groups
- Settings import/export (JSON download + file upload)
- Active env defaults shown as placeholders in Inference settings
- Styled confirmation modals replace window.confirm() for remove actions
- Global restart banner persists across settings subtab switches
- Client-side validation with min/max constraints on number inputs
- Accessibility: aria-label on inputs, role=status on save indicators
- Settings search filters rows across current subtab
- Smooth CSS transitions for conditional field visibility (showWhen)
- Tunnel settings in Channels subtab
- Mobile responsive settings layout at 768px breakpoint
- i18n keys for toolbar, search, and import/export in en + zh-CN
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat(gateway): polish settings page and remove registered tools debug section
Remove the "Registered Tools" table from the extensions tab (debug info
not useful to end users), clean up associated CSS/i18n/JS. Additional
settings page UI polish: extension card state styling, layout refinements.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address PR review feedback [skip-regression-check]
- Use refreshCurrentSettingsTab() in SSE event handlers to reduce duplication
- Remove unused formatGroupName/formatSettingLabel helpers
- Use i18n keys for MCP Configure/Reconfigure buttons
- Add data-i18n-placeholder to settings search input
- Remove data-i18n from confirm modal button (set dynamically by showConfirmModal)
- Fix cargo fmt in main.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): update tests for unified settings tab layout [skip-regression-check]
- Update TABS list: replace extensions/skills with settings
- Add settings_subtab/settings_subpanel selectors to helpers
- Update test_connection, test_skills, test_extensions, test_wasm_lifecycle
to navigate via Settings > subtab instead of top-level tabs
- Move MCP card tests to use go_to_mcp() helper (MCP is now a separate subtab)
- Remove tools table tests and mock_ext_apis tools= parameter
- Fix CSP violation: replace inline onclick on confirm modal cancel button
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address second round of PR review feedback [skip-regression-check]
- Use I18n.t() for MCP empty state, export/import toasts, confirm modal
- Fix CLI channel card using wrong channel key ('repl' -> 'cli')
- Fix settings search counting hidden rows as visible
- Add aria-label i18n for settings search input
- Add common.loadFailed i18n key (en + zh-CN)
- Update E2E tests: WASM channel tests use Channels subtab,
remove tests use custom confirm modal instead of window.confirm
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): fix WASM channel card selector and skills remove confirm [skip-regression-check]
- WASM channel tests: filter by display name to avoid matching built-in
channel cards in the Channels subtab
- Skills remove test: click confirm modal button instead of using
window.confirm (skill removal now uses custom confirm modal)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address third round of PR review feedback [skip-regression-check]
- approval_needed SSE: refresh any active settings subtab, not just
Extensions — approvals can surface from Channels/MCP setup flows too
- renderCardsSkeleton: remove nested .extensions-list wrapper that
caused skeleton cards to render constrained inside grid cells
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): fix auth_completed reload test race condition [skip-regression-check]
Use expect_response to deterministically wait for the /api/extensions
reload triggered by handleAuthCompleted → refreshCurrentSettingsTab,
instead of a fixed 600ms sleep that was too short under CI load.
Also remove stale /api/extensions/tools route handler.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): debug auth_completed reload test with function counter [skip-regression-check]
Inject a counter wrapper around refreshCurrentSettingsTab to verify it's
actually called, and wait for the async fetch to complete before
asserting the reload count.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat(gateway): localize all settings labels, descriptions, and channel cards [skip-regression-check]
Move 120+ hardcoded strings in settings definitions (INFERENCE_SETTINGS,
AGENT_SETTINGS, NETWORKING_SETTINGS) and channel card labels to i18n
keys. Render functions now resolve labels via I18n.t() so the settings
page translates when switching locales.
Covers: group titles, setting labels/descriptions, built-in channel
names/descriptions, and the "No settings found" empty state.
Both en.js and zh-CN.js updated with all new cfg.* and channels.* keys.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): localize remaining hardcoded UI strings [skip-regression-check]
- Fix export error toast using wrong i18n key (importFailed → exportFailed)
- Replace "Failed to load settings:" with I18n.t('common.loadFailed')
- Localize renderBuiltinChannelCard: "Built-in", "Active", "Inactive"
- Localize settings placeholders: "env: ", "env default", "use env default"
- Localize "✓ Saved" indicator
- Add new i18n keys to both en.js and zh-CN.js
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): confirm modal a11y, Esc/click-outside, search guard [skip-regression-check]
- Add role="dialog", aria-modal="true", aria-labelledby to confirm modal
- Focus confirm button when modal opens
- Close modal on Escape key or overlay click
- Skip settings search on non-settings panels (Extensions/MCP/Skills/Channels)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): boolean tri-state, search reset on subtab switch, stale model suggestions [skip-regression-check]
Address PR review feedback:
- Boolean settings now use a tri-state select (env default / On / Off)
instead of a checkbox, matching the pattern used by other select settings
and allowing users to revert to the env default
- Clear search input when switching settings subtabs so stale filters
don't carry over to the new panel
- Always assign model suggestions (even empty array) so stale IDs from a
previous successful /v1/models fetch don't persist when the endpoint
later returns empty
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): auth_completed handler, bedrock_cross_region select, integer-only number inputs [skip-regression-check]
Address PR review feedback:
- auth_completed SSE listener now delegates to handleAuthCompleted(data)
instead of inlining logic with a bare closeConfigureModal() call, so
only the matching extension's modal is dismissed
- bedrock_cross_region changed from free text to select with the four
valid values (us/eu/apac/global), matching backend validation
- Number settings now use step=1 and parseInt() instead of parseFloat(),
preventing fractional values that the backend (u32/u64) would reject
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* fix: full_job routine runs stay running until linked job completion (#1317)
Previously, execute_full_job() returned RunStatus::Ok immediately after
dispatching the job, causing routine runs to be marked as completed before
the linked worker job had actually finished. This meant failure notifications
were never sent and max_concurrent guardrails stopped applying once the run
was prematurely finalized.
Changes:
- execute_full_job() now returns RunStatus::Running instead of Ok
- execute_routine() skips finalization for Running status (leaves run open)
- New sync_dispatched_runs() polls on each cron tick, checks linked job
state, and finalizes runs when jobs reach terminal states
- New list_dispatched_routine_runs() DB method on both backends
- Deferred notifications are sent when the run is actually finalized
- consecutive_failures is preserved (not reset) while outcome is unknown
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review feedback (watcher predicate, running_count safety)
- FullJobWatcher: use is_parallel_blocking() instead of is_active() so
the watcher exits when a job reaches Completed (not terminal but
finished executing). Fixes infinite-poll for routine jobs.
- Remove running_count decrement from sync_dispatched_runs() — in normal
flow execute_routine() handles it; sync only runs for crash recovery
where the counter is already 0.
- Update PR description to match actual FullJobWatcher behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: sync only at startup to prevent double-completion race
- Move sync_dispatched_runs() out of cron loop into startup-only path.
During normal operation FullJobWatcher handles finalization inline;
running sync on every tick would race with the watcher.
- Update complete_dispatched_run() to properly advance runtime fields
(last_run_at, next_fire_at, run_count) for crash recovery — in that
scenario execute_routine() never reached its runtime update.
- Fix stale doc comment on complete_dispatched_run().
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: use boot_time filter for safe periodic sync of orphaned runs
- Add boot_time field to RoutineEngine, set to Utc::now() at creation.
- sync_dispatched_runs() now filters runs by started_at < boot_time,
so it only processes orphans from a previous process — never races
with FullJobWatcher instances from the current process.
- Move sync back into the cron loop (safe with boot_time filter) and
run it BEFORE check_cron_triggers to avoid picking up freshly
dispatched runs.
- Fix doc comments to match actual behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: add FullJobWatcher to track full_job lifecycle for concurrency (#1318)
full_job routines previously bypassed max_concurrent and global concurrency
limits because execute_full_job() returned RunStatus::Ok immediately after
dispatch. This meant running_count was decremented and the routine_run row
was finalized before the actual job completed.
Introduce FullJobWatcher struct that polls store.get_job() every 5s until
the linked job reaches a non-active state, then maps the final JobState to
RunStatus. execute_full_job now creates and awaits the watcher, keeping both
the DB-level running row and the in-memory running_count elevated for the
full job duration.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* test: full_job concurrency regression tests (issue #1318)
Add two integration tests verifying full_job routine concurrency:
1. full_job_max_concurrent_blocks_second_fire_while_first_active:
Inserts a Running routine_run (simulating an in-flight full_job) and
verifies fire_manual returns MaxConcurrent error for max_concurrent=1.
2. global_concurrency_counts_live_full_job_runs:
Elevates running_count to simulate a live full_job holding the global
slot, verifies check_cron_triggers skips due routines, then releases
the slot and verifies the routine fires.
Also makes running_count_for_test() unconditionally public so integration
tests (separate crate) can access it.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fmt and clippy fixes for full_job concurrency tests
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review feedback on FullJobWatcher
- Add #[doc(hidden)] to running_count_for_test() to hide from public API
- Derive MAX_POLLS from POLL_INTERVAL to keep constants coupled
- Check job state before first sleep to finalize promptly for fast jobs
- Update execute_full_job doc comment to reflect blocking behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: consume matched event routine messages
* style: run rustfmt for event routine fix
* fix: preserve preprocessing for routine-triggered messages
* fix: match routines against rewritten input
* refactor: narrow check_event_triggers API and simplify routine_engine_slot
Address Copilot review feedback:
- Change check_event_triggers to accept (user_id, channel, content) instead
of &IncomingMessage, eliminating the need to clone the full message
(including attachments) when hooks rewrite content.
- Remove routine_trigger_message and the Cow<IncomingMessage> indirection;
the event-trigger check now inlines the is_internal + UserInput guard and
passes the post-hook content string directly.
- Make routine_engine_slot non-optional since Agent::new() always
initializes it. Removes the redundant Option wrapper and simplifies
accessor/setter methods.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: add debug_assert invariant guards to critical code paths (closes#1215)
Add three debug_assert! calls to catch impossible-in-correct-code states
early in debug builds without affecting release performance:
- execute_tool_with_safety: assert tool_name is non-empty at entry
- JobContext::transition_to: assert state machine transition is valid
- CircuitBreakerProvider::record_success: assert circuit is not Open
(check_allowed() must gate all calls before record_success())
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* test: add regression test for empty tool name invariant guard
Covers the debug_assert!(!tool_name.is_empty()) added in execute_tool_with_safety.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
---------
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
* feat: upgrade MiniMax default model to M2.7
- Add MiniMax-M2.7 and MiniMax-M2.7-highspeed to model list
- Set MiniMax-M2.7 as default model
- Keep all previous models as alternatives
- Update related tests
* fix: use canonical model name in test per review
Use MiniMax-M2.7-highspeed (canonical casing) in the reasoning
models test for consistency with the documentation and provider
configuration.
[skip-regression-check]
- Fix parse_custom_headers to preserve commas in values by splitting
only on commas followed by a header-name:colon pattern (manual scan
instead of simple split(','))
- Use matches! macro for backend exclusion check in app.rs
- Merge SSE metadata extraction into single pass (was iterating twice)
- Replace fragile substring-based context_length with explicit match
on known Gemini model IDs via gemini_context_length()
- Add missing models to regression test (8 models, not 5)
- Add '// safety: hardcoded literal' to regex .expect() to satisfy
the no-panic-in-prod CI check
- Fix cargo fmt whitespace in collapsible if-let chain
- Register gemini_oauth as a dedicated backend in config/llm.rs (skip
registry fallback, preserve backend name, suppress unknown-backend warning)
- Fix app.rs credential guard to exclude backends with dedicated configs
(gemini_oauth, bedrock) from the provider.is_none() check
- Auto-discover Cloud Code project_id via loadCodeAssist when credentials
lack it (e.g. created by the original Gemini CLI)
- Persist discovered project_id to credentials file for subsequent runs
- Add safety settings (BLOCK_NONE), gated behind GEMINI_SAFETY_BLOCK_NONE env
- Add thinkingConfig: budget-based for Gemini 2.5, level-based for Gemini 3.x
(without includeThoughts to avoid empty responses from reasoning.rs stripping)
- Add thought signature injection for Gemini 3.x preview APIs
- Add history curation to filter invalid model outputs before re-sending
- Add extended generationConfig env vars (topP, topK, seed, penalties,
responseMimeType, responseJsonSchema, cachedContent)
- Add custom headers support via GEMINI_CLI_CUSTOM_HEADERS
- Add API key auth mode (GEMINI_API_KEY + GEMINI_API_KEY_AUTH_MECHANISM)
- Add SSE metadata extraction (modelVersion, credits, promptFeedback,
groundingMetadata, citationMetadata, cachedContentTokenCount)
- Add countTokens API support
- Add new models to wizard (gemini-3.1-pro-preview-customtools,
gemini-3-pro-preview, gemini-3.1-flash-lite-preview)
- Update docs/LLM_PROVIDERS.md with new models and routing rules
- Rewrite regression tests with comprehensive coverage (23 unit tests pass)
* fix: Rate limiter returns retry after None instead of a duration
linter fix
* review fixes
* fix: rate limiter returns None for retry_after duration
Add regression test to src/llm/retry.rs that verifies RateLimited errors
always have a fallback duration (never None) due to the 60-second fallback
applied in all rate limit error creation sites (nearai_chat.rs,
anthropic_oauth.rs, embeddings.rs).
The production code fix adds `.or(Some(Duration::from_secs(60)))` to ensure
the error message never displays "retry after None" to the user.
[skip-regression-check]
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
---------
Co-authored-by: Claude Haiku 4.5 <[email protected]>
- Remove duplicate build_nearai_model_fetch_config() definition from setup/wizard.rs
(function already exists in llm/models.rs and is imported)
- Add missing cheap_model and smart_routing_cascade fields to LlmConfig
initializer in build_nearai_model_fetch_config() (llm/models.rs)
- Pass request_timeout_secs to create_registry_provider() call
(llm/mod.rs:432)
All clippy checks pass with zero warnings (--no-default-features --features libsql).
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
Resolved merge conflicts in 5 files:
1. src/agent/job_monitor.rs - Used is_internal flag approach (HEAD) for safe internal message marking. Removed metadata-based approach which could be spoofed by external channels.
2. src/agent/agent_loop.rs - Used is_internal check (HEAD) for routing internal messages, consistent with security model where is_internal field cannot be spoofed.
3. src/agent/dispatcher.rs - Included notify_metadata in job context (main), needed for job routing through JobMonitorRoute.
4. src/setup/wizard.rs - Added build_nearai_model_fetch_config() function (main) for model selection during setup.
5. src/tools/builtin/job.rs - Used both comments from HEAD (clarifying notify_channel and notify_user logic) while removing metadata field from JobMonitorRoute (consistent with job_monitor.rs).
All conflicts resolved with security-first approach: use is_internal boolean field for internal message marking (cannot be spoofed), while passing routing metadata through context.
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
* feat: add LLM_CHEAP_MODEL for generic smart routing across all backends
Add generic cheap model support that works with any LLM backend, not just
NearAI. New env vars: LLM_CHEAP_MODEL (cheap model for any backend) and
SMART_ROUTING_CASCADE (top-level cascade flag).
Resolution order: LLM_CHEAP_MODEL > NEARAI_CHEAP_MODEL (backward compat).
Registry-based providers (OpenAI, Anthropic, Groq, etc.) clone their
RegistryProviderConfig with the cheap model swapped in. Bedrock returns
an explicit error (not yet supported). All error paths use ok_or_else
with proper LlmError variants -- no unwrap/expect in production code.
* refactor: address Gemini review — remove unnecessary async, extract cheap_model_name()
- Remove async from create_cheap_provider_for_backend() and
create_cheap_llm_provider() — neither contains .await calls
- Extract duplicated cheap model resolution logic into
LlmConfig::cheap_model_name() helper method (DRY)
- Revert tests from tokio::test async back to sync #[test]
- Add test_cheap_model_name_resolution() unit test for the helper
---------
Co-authored-by: SMKRV <[email protected]>
* feat(orchestrator): read ORCHESTRATOR_PORT env var for configurable API port
The orchestrator internal API port was hardcoded to 50051 in two places
(ContainerJobConfig and OrchestratorApi::start call), making it impossible
to run multiple IronClaw instances on the same host — the second instance
fails with "Address already in use".
NETWORK_SECURITY.md already documents ORCHESTRATOR_PORT as configurable,
and ContainerJobConfig.orchestrator_port is propagated to worker containers
via IRONCLAW_ORCHESTRATOR_URL, but the env var was never actually read.
Extract resolve_orchestrator_port() that reads ORCHESTRATOR_PORT and falls
back to 50051. Includes tests for valid, invalid, and out-of-range values.
* test: add ENV_LOCK mutex for env-var test serialization
Address Gemini review: add std::sync::Mutex to serialize env var access
across test threads. Keep unsafe blocks — required in Rust edition 2024
where std::env::set_var/remove_var are unsafe functions.
---------
Co-authored-by: SMKRV <[email protected]>
* feat(transcription): add Chat Completions API provider for audio transcription
The existing transcription pipeline only supports the OpenAI Whisper API
(/v1/audio/transcriptions with multipart upload). Providers like OpenRouter
expose audio transcription through the Chat Completions API instead, using
base64-encoded audio in the `input_audio` content type.
Add `ChatCompletionsTranscriptionProvider` that sends audio as base64 in
a chat completion request and extracts the transcript from the response.
Compatible with OpenRouter, OpenAI GPT-4o-audio, and any provider that
supports audio input via Chat Completions.
Config changes:
- TRANSCRIPTION_PROVIDER=chat_completions selects the new provider
- TRANSCRIPTION_API_KEY overrides provider-specific keys
- LLM_API_KEY used as fallback for chat_completions provider
- Default model per provider (whisper-1 for openai, gemini-2.0-flash for
chat_completions)
* style: address review feedback — formatting, idiomatic patterns
- Fix rustfmt formatting for provider constructor chain
- Use or_else for resolve_api_key priority chain (Gemini review)
- Use trim_end_matches('/') instead of while loop (Gemini review)
---------
Co-authored-by: SMKRV <[email protected]>
* fix(jobs): make completed->completed transition idempotent to prevent race errors
Both execution_loop and the worker wrapper in execute() can race to call
mark_completed(). Previously the second call hit "Cannot transition from
completed to completed" and errored the job despite successful completion.
This narrowly allows only the Completed->Completed self-transition as
idempotent (early return with debug log, no duplicate history entry).
All other self-transitions remain rejected to preserve state machine
strictness.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: fix assert! formatting in idempotent completion test
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* fix(llm): persist refreshed Anthropic OAuth token after Keychain re-read (#1136)
The Anthropic OAuth provider stored its token as an immutable SecretString.
When a 401 triggered a Keychain re-read, the fresh token was used for a
single retry but never persisted — every subsequent request reused the
expired original token, causing repeated auth failures.
Changes:
- Wrap token in RwLock<SecretString> so it can be updated after refresh
- Persist refreshed token via update_token() on successful retry
- Add 500ms delay before Keychain re-read to give Claude Code time to
complete its async token refresh write (reduces race window)
- Add regression test verifying token updates persist across reads
Closes#1136
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fix formatting
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix(worker): prevent orphaned tool_results and fix parallel merging
Two fixes for tool result handling in the Worker:
1. Preserve reasoning text from select_tools() in the RespondResult
content field so it appears in the assistant_with_tool_calls message
pushed by execute_tool_calls. Without this, the LLM's reasoning
context was lost when using the select_tools path.
2. Merge consecutive tool_result messages into a single User message
in rig_adapter's convert_messages(). When parallel tools execute,
each produces a separate ChatMessage with role: Tool. Without
merging, these become consecutive User messages which Anthropic
rejects. Now consecutive tool results are merged into one User
message with multiple ToolResult content items.
Includes regression tests for both fixes.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(worker): use find_map for first non-empty reasoning extraction
The previous code only checked the first ToolSelection's reasoning,
missing cases where the first selection has empty reasoning but
subsequent ones do not. Switch to find_map to get the first non-empty
reasoning across all selections.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* feat(heartbeat): fire_at time-of-day scheduling with IANA timezone support
- HEARTBEAT_FIRE_AT=HH:MM — fire heartbeat at a specific time of day instead
of on a rolling interval; format is 24h HH:MM (e.g. "14:00")
- HEARTBEAT_TIMEZONE=Region/City — IANA timezone name for fire_at (e.g.
"Pacific/Auckland", "America/New_York"). Defaults to UTC.
- When fire_at is set, interval_secs is ignored
- Config also readable from settings.toml [heartbeat] section
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* feat(heartbeat): wire fire_at + timezone into HeartbeatConfig runner
Missed file from heartbeat scheduling commit. HeartbeatConfig struct in
agent/heartbeat.rs now carries fire_at: Option<NaiveTime> and timezone: Tz
so the runner can schedule against a fixed time of day.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* fix: add chrono-tz dependency for heartbeat fire_at timezone support
The chrono-tz crate was used in the heartbeat fire_at commits but
its Cargo.toml entry was lost during rebase conflict resolution.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: rustfmt fix for chained method call
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* test(heartbeat): add fire_at scheduling and DST safety tests
- test_default_config_has_no_fire_at: interval-based default unchanged
- test_with_fire_at_builder: builder sets time and timezone
- test_duration_until_next_fire_is_bounded: result always 1s–24h
- test_duration_until_next_fire_dst_timezone_no_panic: US Eastern DST
- test_resolved_tz_defaults_to_utc: missing timezone falls back to UTC
- test_resolved_tz_parses_iana: IANA string resolves correctly
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(heartbeat): restore drift-free interval, add settings.json fallback for fire_at
- Interval path: restore tokio::time::interval (drift-free) instead of
tokio::time::sleep which drifts by loop body execution time
- fire_at config: fall back to settings.heartbeat.fire_at when
HEARTBEAT_FIRE_AT env var is not set, consistent with other settings
Addresses Gemini Code Assist review feedback.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: IronClaw <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
* feat: add Codex auth.json token reuse for LLM authentication
When LLM_USE_CODEX_AUTH=true, IronClaw reads the Codex CLI's auth.json
(default ~/.codex/auth.json) and extracts the API key or OAuth access
token. This lets IronClaw piggyback on a Codex login without
implementing its own OAuth flow.
New env vars:
- LLM_USE_CODEX_AUTH: enable Codex auth fallback (default: false)
- CODEX_AUTH_PATH: override path to auth.json
* fix: handle ChatGPT auth mode correctly
Switch base_url to chatgpt.com/backend-api/codex when auth.json
contains ChatGPT OAuth tokens. The access_token is a JWT that only
works against the private ChatGPT backend, not the public OpenAI API.
Refactored codex_auth.rs to return CodexCredentials (token +
is_chatgpt_mode) instead of just a string key.
* fix: Codex auth takes highest priority over secrets store
When LLM_USE_CODEX_AUTH=true, Codex credentials are now loaded before
checking env vars or the secrets store overlay. Previously the secrets
store key (injected during onboarding) would shadow the Codex token.
* feat: Responses API provider for ChatGPT backend
- New CodexChatGptProvider speaks the Responses API protocol
- Auto-detects model from /models endpoint (gpt-4o -> gpt-5.2-codex)
- Adds store=false (required by ChatGPT backend)
- Error handling with timeout for HTTP 400 responses
- Message format translation: Chat Completions -> Responses API
- SSE response parsing for text, tool calls, and usage stats
- 7 unit tests for message conversion and SSE parsing
* fix: SSE parser uses item_id instead of call_id for tool call deltas
The Responses API sends function_call_arguments.delta events with
item_id (e.g. fc_...) not call_id (e.g. call_...). The parser now
keys pending tool calls by item_id from output_item.added and
tracks call_id separately for result matching.
* fix: strip empty string values from tool call arguments
gpt-5.2-codex fills optional tool parameters with empty strings
(e.g. timestamp: ""), which IronClaw's tool validation rejects.
Strip them before passing to tool execution.
* fix: prevent apiKey mode fallback to ChatGPT token
When auth_mode is explicitly 'apiKey' but the key is missing/empty,
do not fall through to check for a ChatGPT access_token. This prevents
returning credentials with is_chatgpt_mode: true and routing to the
wrong LLM provider.
* refactor: reuse single reqwest::Client across model discovery and LLM calls
Create Client once in with_auto_model, pass &Client to
fetch_default_model, and move it into the provider struct.
Eliminates the redundant Client::new() that wasted a connection pool.
* fix: bump client_version to 1.0.0 to unlock gpt-5.3-codex and gpt-5.4
The /models endpoint gates newer models behind client_version.
Version 0.1.0 only returns up to gpt-5.2-codex, while 1.0.0+
also returns gpt-5.3-codex and gpt-5.4.
* feat: user-configured LLM_MODEL takes priority over auto-detection
Fetch the full model list from /models endpoint. If LLM_MODEL is set,
validate it against the supported list and warn with available models
if not found. If LLM_MODEL is not set, auto-detect the highest-priority
model. Also bumps client_version to 1.0.0 to unlock gpt-5.3/5.4.
* fix: add 10s timeout to model discovery HTTP request
Prevents startup from blocking indefinitely if chatgpt.com
is slow or unreachable. Uses reqwest per-request timeout.
* docs: add private API warning for ChatGPT backend endpoint
The chatgpt.com/backend-api/codex endpoint is private and
undocumented. Add warning in module docs and a runtime log
on first use to inform users of potential ToS implications.
* feat: implement OAuth 401 token refresh for Codex ChatGPT provider
On HTTP 401, if a refresh_token is available, the provider now
automatically refreshes the access token via auth.openai.com/oauth/token
(same protocol as Codex CLI) and retries the request once. Refreshed
tokens are persisted back to auth.json.
Changes:
- codex_auth: read refresh_token, add refresh_access_token() and
persist_refreshed_tokens()
- codex_chatgpt: RwLock for api_key, 401 detection + retry in
send_request, send_http_request helper
- config/llm: thread refresh_token/auth_path through RegistryProviderConfig
- llm/mod: pass refresh params to with_auto_model
* refactor: lazy model detection via OnceCell, remove block_in_place
Model is no longer resolved during provider construction. Instead,
resolve_model() uses tokio::sync::OnceCell to lazily fetch from
/models on the first LLM call. This eliminates the block_in_place
+ block_on workaround in create_codex_chatgpt_from_registry.
- with_auto_model (async) -> with_lazy_model (sync constructor)
- resolve_model() added with OnceCell-based lazy init
- build_request_body takes model as parameter
- model_name() returns resolved or configured_model as fallback
* feat: support multimodal content (images) in Codex ChatGPT provider
message_to_input_items now checks content_parts for user messages.
ContentPart::Text maps to input_text and ContentPart::ImageUrl maps
to input_image, matching the Responses API format used by Codex CLI.
Falls back to plain text when content_parts is empty.
Also updates client_version to 0.111.0 for /models endpoint.
Adds test: test_message_conversion_user_with_image
* refactor: move codex_auth module from src/ to src/llm/
codex_auth is only used by the LLM layer (codex_chatgpt provider
and config/llm). Moving it under src/llm/ reflects its actual scope.
- Remove pub mod codex_auth from lib.rs
- Add pub mod codex_auth to llm/mod.rs
- Update imports: super::codex_auth, crate::llm::codex_auth
* Fix codex provider style issues
* Use SecretString throughout codex auth refresh flow
* Use SecretString for codex access tokens
* Reuse provider client for codex token refresh
* Stream Codex SSE responses incrementally
* Fix Windows clippy and SQLite test linkage
* Trigger checks after regression skip label
* Tighten codex auth module handling
Artem