* docs: add comments explaining CLI_ENABLED=false in service templates (#990)
Clarify that CLI_ENABLED=false is needed in daemon mode (launchd/systemd)
to prevent blocking on stdin when running as a background service.
Closes#990
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* perf: use Arc<Vec<f32>> in embedding cache to avoid clones on miss path (#1429)
Store embeddings as Arc<Vec<f32>> internally so that cache insertions
share the allocation with the return value via Arc::clone instead of
cloning the entire float vector (6-12 KB per embedding).
- embed() miss path: Arc::try_unwrap avoids a clone when returning
(the cache holds one Arc ref, the return path holds the other;
try_unwrap succeeds when the thundering-herd path doesn't fire)
- embed_batch() miss path: cache first via Arc::clone, then
try_unwrap for results — embeddings skipped due to capacity
limits are returned without any clone
- Hit path still clones (trait returns Vec<f32>); a future trait
change to Arc<Vec<f32>> could eliminate this too
Closes#1429
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fix formatting in embedding_cache.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review — correct doc comment and remove dead try_unwrap
- Reword CacheEntry doc comment to accurately reflect that hit/miss paths
still clone into a fresh Vec<f32> for callers; Arc sharing only helps
in embed_batch when embeddings are skipped from caching
- Remove Arc::try_unwrap in embed() which could never succeed (cache
always holds an Arc ref, so refcount >= 2)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: revert embed() to plain Vec, keep Arc only in embed_batch()
In embed(), Arc adds overhead (allocation + refcount) without saving
any clones — the original pattern (clone for cache, return by move)
was already optimal. Arc only helps in embed_batch() where
capacity-skipped embeddings can be returned via try_unwrap.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: move clone+Arc::new outside mutex in embed()
Clone the embedding and wrap in Arc before acquiring the lock so the
mutex is held only for the HashMap insert, not during the O(n) copy.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: drop Arc, use cache-then-move pattern instead
Arc was the wrong abstraction — the trait returns Vec<f32>, so Arc
can't avoid clones on return paths. Instead:
- embed(): skip clone in thundering-herd case (just touch timestamp)
- embed_batch(): cache first (clone only cacheable subset), then move
originals into results (zero-copy). For N misses with K cacheable:
old = 2N clones, new = K clones.
- CacheEntry reverted to plain Vec<f32>, no Arc overhead
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* docs: add comments explaining CLI_ENABLED=false in service templates (#990)
Clarify that CLI_ENABLED=false is needed in daemon mode (launchd/systemd)
to prevent blocking on stdin when running as a background service.
Closes#990
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* Add owner-scoped full-job routine permissions
* Address PR review feedback
* Fix owner gate test timing
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat: structured fallback deliverables for failed/stuck jobs (#221)
When a job fails or gets stuck, build a FallbackDeliverable that captures
partial results, action statistics, cost, timing, and repair attempts.
This replaces opaque error strings with structured data users can act on.
- Add FallbackDeliverable, LastAction, ActionStats types in context/fallback.rs
- Store fallback in JobContext.metadata["fallback_deliverable"] on failure
- Surface fallback in job_status tool output and SSE job_result events
- Update mark_failed() and mark_stuck() in worker to build fallback
- 8 unit tests covering zero/mixed actions, truncation, timing, serialization
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address review comments on fallback deliverables
- Fix doc comment: "200 chars" -> "200 bytes (UTF-8 safe)" since
truncate_str operates on byte length, not character count.
- Add code comment documenting that SSE fallback_deliverable is
currently always None (forward-compatible infrastructure).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* refactor: take Option<&FallbackDeliverable> instead of &Option<…>
Addresses Gemini review feedback: idiomatic Rust prefers
Option<&T> over &Option<T> for borrowed optional values.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: guard against non-object metadata and add fallback test
- store_fallback_in_metadata now resets metadata to {} when it's any
non-object type (string, array, number), not just null. Prevents
panic on index assignment.
- Add test_job_status_includes_fallback_deliverable to verify the
fallback field is surfaced in job_status tool output.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: use sanitized output in fallback preview + add integration tests
Security fix: FallbackDeliverable::build() now uses output_sanitized
instead of output_raw, preventing secrets/PII from leaking through
the job_status tool and SSE job_result events.
Also adds:
- test_fallback_uses_sanitized_output: proves raw secrets don't leak
- test_store_fallback_in_metadata_roundtrip: full serialize/deserialize
- test_store_fallback_handles_non_object_metadata: edge case coverage
- test_store_fallback_none_is_noop: None input is safe
Addresses serrrfirat review feedback on PR #236.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: harden fallback deliverables against review findings
- Truncate failure_reason to 1000 bytes to prevent metadata bloat
- Add tracing::warn on fallback serialization failure (was silently discarded)
- Fix module/struct docs to cover stuck jobs, remove stale SSE claim
- Fix job.rs test to use real FallbackDeliverable field names
- Add tests for failure_reason truncation and completed_at=None elapsed time
- Fix pre-existing clippy warning in settings.rs (field_reassign_with_default)
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address Copilot review findings on fallback deliverables
- Fix output_raw/output_sanitized field swap in ActionRecord::succeed()
so sanitized data actually goes into the sanitized field (security)
- Return None instead of empty Memory when get_memory fails in
build_fallback, with tracing::warn for observability
- Replace manual elapsed calculation with ctx.elapsed() which already
clamps negative durations
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: resolve rebase conflicts and update tests for parameter swap
- Add fallback field to SseEvent::JobResult in job_monitor
- Fix type annotation in fallback deliverable test
- Update test_action_record_succeed_sets_fields for new parameter order
- Use create_job_for_user in test (API changed on main)
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* chore: trigger CI re-check after rebase
* fix: fall back to error message for failed action output_preview
When the last action is a failed tool call, output_sanitized is None,
leaving output_preview empty. Now falls back to the action's error
message so users see what went wrong.
[skip-regression-check]
* ci: add safety comments to test code for no-panics check
The CI no-panics grep check cannot distinguish test code inside
src/ files from production code. Add // safety: test annotations
to .unwrap(), .expect(), and assert!() calls in #[cfg(test)] modules.
* fix: clarify succeed() doc and avoid clone in output_preview
- Fix doc comment: output_raw is stored as pretty-printed JSON string,
not a raw JSON value
- Borrow string slice directly in fallback preview to avoid cloning
potentially large sanitized outputs before truncation
* refactor: reuse floor_char_boundary in truncate_str
Replace hand-rolled UTF-8 boundary logic with existing
crate::util::floor_char_boundary to reduce duplication.
* fix: rename SSE fallback field to fallback_deliverable for consistency
The SSE JobResult field was named `fallback` while everywhere else
(metadata key, job_status tool) uses `fallback_deliverable`. Align
the SSE wire format to avoid forcing clients to handle two names.
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* feat: LRU embedding cache for workspace search (#165)
Add CachedEmbeddingProvider that wraps any EmbeddingProvider with an
in-memory LRU cache keyed by SHA-256(model_name + text). This avoids
redundant HTTP calls when the same text is embedded multiple times
(common during reindexing and repeated searches).
- Cache uses HashMap + last_accessed tracking with manual LRU eviction
(same pattern as llm::response_cache::CachedProvider)
- Lock is never held during HTTP calls to prevent blocking
- embed_batch() partitions into hits/misses and only fetches misses
- Default 10,000 entries (~58 MB for 1536-dim vectors)
- Configurable via EMBEDDING_CACHE_SIZE env var
- Workspace.with_embeddings() auto-wraps; with_embeddings_uncached()
available for tests
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address review comments on embedding cache
- Validate embed_batch return count matches expected miss count
- Replace unwrap_or_default() with proper error propagation
- Fix batch eviction: run final eviction pass after insert to enforce cap
- Fix test: use different-length inputs to verify ordering correctness
- Reject EMBEDDING_CACHE_SIZE=0 in config validation (minimum is 1)
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: replace .expect() with proper error handling in embed_batch
The all-cache-hits early-return path used .expect("all cache hits") which
violates the project convention of no .unwrap()/.expect() in production
code. Replaced with the same ok_or_else pattern used in the normal path.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: clarify memory sizing docs and use saturating_add for eviction
- Update memory comments in embedding_cache.rs, config/embeddings.rs,
and workspace/mod.rs to note the ~58 MB figure is payload-only
(actual memory is higher due to HashMap/key/allocation overhead)
- Use saturating_add(1) instead of + 1 for eviction threshold to
prevent overflow if max_entries is usize::MAX
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address Copilot review on embedding cache
- Avoid double-clone per miss in embed_batch: move embedding into
results, clone only for the cache entry
- Evict per-insert instead of after all inserts to keep peak memory
bounded during large batches
- Clamp max_entries to at least 1 in constructor to prevent unexpected
eviction behavior when set to 0 via the public API
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: reduce embedding_cache module visibility to private
Types are already re-exported via `pub use`, so the module itself
doesn't need to be public. Reduces unnecessary API surface.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address serrrfirat review feedback on embedding cache
- Add TODO comment for O(n) LRU eviction scalability
- Add thundering herd note at lock release in embed()
- Warn when cache max_entries exceeds 100k
- Use with_embeddings_uncached() in integration test
- Add tests: error_does_not_pollute_cache, embed_batch_empty_input
- Update README with cache-aware with_embeddings() docs
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: prevent u32 wrapping in FailThenSucceedMock failure counter
fetch_sub(1) wraps to u32::MAX when called past zero, silently
breaking the mock for 3+ calls. Switch to load-then-store to avoid
the wrapping bug in both embed() and embed_batch().
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address Copilot and serrrfirat review findings on embedding cache
- Switch tokio::sync::Mutex to std::sync::Mutex (lock never held across
.await — cheaper synchronous lock)
- Extract DEFAULT_EMBEDDING_CACHE_SIZE constant to avoid 10_000 duplication
between EmbeddingCacheConfig and EmbeddingsConfig
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* test: add all-misses batch test for embedding cache
Adds embed_batch_all_misses test covering the case where every text in a
batch is a cache miss — fulfilling the commitment from serrrfirat's review.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* chore: trigger CI re-check after rebase
* fix: use raw [u8;32] cache keys and pre-allocate HashMap capacity
Address Copilot review findings:
- cache_key() now returns [u8; 32] instead of hex String, avoiding a
64-byte allocation per lookup
- HashMap::with_capacity(max_entries) avoids incremental reallocation
- Fix pre-existing staging compilation error in cli/routines.rs
(missing max_tool_rounds/use_tools fields)
[skip-regression-check]
* fix: make cache accessors sync and update doc for [u8;32] keys
Address Copilot review:
- len(), is_empty(), clear() are now sync since they only take a
std::sync::Mutex lock with no .await points
- Update cache_size doc comment to reflect [u8;32] keys instead of
String keys
[skip-regression-check]
* fix: remove clone_on_copy for [u8; 32] cache keys
[skip-regression-check]
* ci: add safety comments to test code for no-panics check
The CI no-panics grep check cannot distinguish test code inside
src/ files from production code. Add // safety: test annotations
to .unwrap(), .expect(), and assert!() calls in #[cfg(test)] modules.
* fix: correct cache doc and demote hit/miss logs to trace
- Fix misleading "String keys" in memory comment (cache uses [u8; 32])
- Demote per-request hit/miss logs from debug to trace to reduce noise
on hot paths (batch summary stays at trace too)
* docs: add missing Arc import in workspace README example
* perf: batch eviction in embed_batch to avoid O(n×m) cost
Replace per-insert evict_lru call with a single evict_k_oldest pass
that computes eviction count upfront and removes the k oldest entries
in one O(n) scan. Avoids O(n×m) HashMap iterations while holding the
mutex during batch inserts.
* fix: cap batch cache inserts at max_entries and use O(n) selection
- evict_k_oldest now uses select_nth_unstable_by_key for O(n) average
partial selection instead of O(n log n) full sort
- embed_batch caps cached entries at max_entries when misses exceed
capacity, preventing the cache from growing unbounded
- Added test: batch_exceeding_capacity_respects_max_entries
* fix: flatten test assert for fmt compatibility
Shorten assert message to fit single line so cargo fmt doesn't
split the safety annotation onto a separate line.
* fix: address review feedback and improve embedding cache (takeover #235)
- Fix merge conflict: add missing allow_always field in PendingApproval
- Thread EmbeddingCacheConfig through CLI memory commands so they respect
EMBEDDING_CACHE_SIZE instead of silently using default (fixes#235 review)
- Cap HashMap pre-allocation at min(max_entries, 1024) to avoid upfront
memory waste at large cache sizes
- Fix FailThenSucceedMock race: replace load+store with atomic fetch_update
- Remove noisy '// safety: test' comments (40+ lines of diff noise)
- Fix collapsed lines from comment removal
- Simplify redundant Ok(...collect()?) to just collect()
Co-Authored-By: ztsalexey <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(embedding-cache): skip eviction on concurrent duplicate insert
When the lock is released for the HTTP call, another caller may insert
the same key. Re-check under lock and just update the existing entry
without evicting, avoiding unnecessary cache churn under concurrency.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: ztsalexey <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: ztsalexey <[email protected]>
* feat: receive relay events via webhook callbacks instead of SSE
Replace the SSE pull model with push-based webhook callbacks from
channel-relay. Eliminates the reconnect loop, stream token auth,
and SSE parser — events arrive via HTTP POST to /relay/events.
- Add webhook handler with HMAC signature verification
- Simplify RelayChannel to use mpsc from webhook handler
- Remove SSE connect/reconnect/parse logic from RelayClient
- Add register_callback() to RelayClient for callback URL registration
- Update activation flow to create event channel and register callback
- Wire relay webhook endpoint into web gateway
* fix: address review feedback on webhook callback PR
- Return 503 when relay event channel is full/closed (enables retry)
- Reject malformed timestamps with 400 instead of proceeding
- Allow relay activation without settings store (no-store/ephemeral mode)
- Check installed_relay_extensions set in is_relay_channel for no-db mode
- Fix staging test constructors for new RelayChannel signature
* security: adapt relay client to new channel-relay auth model
Adapts the relay integration to the hardened channel-relay security model:
- Switch from X-API-Key header to Authorization: Bearer sk-agent-*
for all relay API calls (chat-api token verification)
- Remove register_callback() — PUT /callbacks endpoint removed
- Remove event_callback_url from initiate_oauth() — parameter removed
- Make signing_secret a required field in RelayConfig (new env var:
CHANNEL_RELAY_SIGNING_SECRET)
- Update integration tests for Bearer auth and removed endpoints
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: use server-side approval tokens, remove caller-supplied routing
- Approval flow now calls POST /approvals to register server-side
record, then embeds only the opaque approval_token in button value
- Remove instance_id parameter from proxy_provider() — channel-relay
no longer accepts it (uses verified identity)
- Remove instance_id and user_id from initiate_oauth() — channel-relay
derives them from the Bearer token
- Add create_approval() to RelayClient
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: pass webhook_url during OAuth so callback_url is set on connection
The channel-relay OAuth flow now accepts webhook_url to set the
callback_url during connection creation. IronClaw computes its webhook
URL from callback_base + webhook_path and passes it during initiate_oauth.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove webhook_url from OAuth initiation
Channel-relay now derives the callback URL from chat-api's instance_url.
IronClaw no longer supplies webhook_url during OAuth — the relay is the
authority on where events get delivered.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* chore: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove all URL params from OAuth initiation
IronClaw no longer supplies any URLs to channel-relay. The relay
derives all URLs from the trusted instance_url in chat-api.
initiate_oauth() takes no parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: restore CSRF nonce for OAuth callback validation
Re-add nonce generation and secret storage in auth_channel_relay.
The nonce is passed to channel-relay as state_nonce param (not a URL).
Channel-relay embeds it in the signed state and appends it to the
redirect URL so IronClaw's callback handler can validate and activate.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: per-instance callback signing secrets
relay_signing_secret() now prefers OPENCLAW_GATEWAY_TOKEN (per-instance)
over the shared CHANNEL_RELAY_SIGNING_SECRET. A compromised instance
can no longer forge callbacks to other instances on the same relay.
CHANNEL_RELAY_SIGNING_SECRET is now optional in RelayConfig.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: clean per-instance callback secrets, no shared secrets, no fallbacks
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: pass team_id to get_signing_secret for workspace-scoped lookup
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* security: remove sender_id from create_approval — relay derives it
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: remove stale relay sender_id validation
* fix: harden relay webhook activation lifecycle
---------
Co-authored-by: Pierre <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
The HTTP tool returned `ApprovalRequirement::Always` for requests with
credentials, but `Always` is hardcoded to ignore the session auto-approve
set. This meant users who clicked "always" were re-prompted on every
subsequent HTTP call — the UI offered "always" but the backend ignored it.
Two fixes:
1. HTTP credentialed requests now return `UnlessAutoApproved` instead of
`Always`, so the session auto-approve set is respected.
2. `StatusUpdate::ApprovalNeeded` now carries `allow_always: bool`. All
channel UIs (Telegram, Slack, Signal, REPL, Web) conditionally hide
the "always" option when a tool truly requires per-invocation approval
(`ApprovalRequirement::Always`, e.g. destructive shell commands).
Also boxes `PendingApproval` in `AgenticLoopResult::NeedApproval` to fix
a pre-existing clippy `large_enum_variant` warning.
Regression tests included (test_credentialed_requests_respect_auto_approve,
test_allow_always_matches_approval_requirement) but CI heuristic cannot
detect them in cross-fork PR diffs.
[skip-regression-check]
Co-authored-by: Tyler <[email protected]>
* fix(feishu): parse flat token response from tenant_access_token API
The Feishu /auth/v3/tenant_access_token/internal endpoint returns a flat
JSON response with tenant_access_token and expire at the top level, not
nested under a "data" field. The previous code used FeishuApiResponse<T>
which expects a "data" wrapper, causing all token exchanges to fail with
"Token response missing data" despite receiving a valid HTTP 200 response.
- Replace TenantAccessTokenData with TenantAccessTokenResponse that includes
code/msg/tenant_access_token/expire at the top level
- Deserialize token response directly instead of via FeishuApiResponse<T> wrapper
- Add empty-token guard to catch malformed responses
- No changes to FeishuApiResponse<T> or other API call paths
Fixes#1391
* fix(feishu): address review feedback on token response parsing
- Remove #[serde(default)] from tenant_access_token and expire fields
so deserialization fails explicitly when critical fields are missing
- Add expire > 0 validation guard to prevent refresh loops or overflow
- Use saturating_add/saturating_mul for expiry calculation
- Add 5 regression tests for TenantAccessTokenResponse deserialization
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: reidliu <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: skip NEAR AI session check when backend is not nearai
When a user configures a non-NEAR AI backend (e.g. Anthropic), the
doctor command was incorrectly failing with "session file not found"
even though no NEAR AI session is needed. The check now skips with a
descriptive message when LLM_BACKEND is not nearai/near_ai/near.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* fix(ci): avoid holding sync MutexGuard across await in doctor test
Convert check_nearai_session_skips_for_non_nearai_backend from
#[tokio::test] to #[test] with block_on, matching the pattern used by
all other ENV_MUTEX tests. Fixes clippy::await_holding_lock error.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Kristian Glass <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Bump registry version to pass check-version-bumps.sh after
channels-src/telegram/ changes.
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: navigate telegram E2E tests to channels subtab
wasm_channel extensions (like telegram) are now rendered in the
Settings → Channels subtab, not the Extensions subtab. Update
test_telegram_hot_activation to navigate there and use the correct
card selector. Also mock /api/gateway/status which loadChannelsStatus
fetches.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: select telegram card by name, not first card in channels subtab
Built-in channel cards (Web Gateway, HTTP, etc.) render first in the
channels subtab content, so .first matches them instead of the
telegram extension card. Select by has_text="Telegram" to target
the correct card.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor: make gateway_status_handler parameterizable in mock helper
Address review feedback: extract default gateway status handler and
accept an optional gateway_status_handler kwarg in mock_extension_lists
for test flexibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
- Add `builder: None` to AgentDeps initializer in e2e_telegram_message_routing
test (field added in #712 but test not updated)
- Update go_to_extensions() in test_telegram_hot_activation to navigate via
settings tab -> extensions subtab (extensions tab was moved to settings)
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat(self-repair): wire stuck_threshold, store, and builder (#647)
Wire the previously dead-code fields in DefaultSelfRepair:
- stuck_threshold: detect_stuck_jobs() now filters by duration, only
reporting jobs stuck longer than the configured threshold
- with_store(): wired in agent_loop.rs from AgentDeps.store for
tool failure tracking via Database trait
- with_builder(): wired from register_builder_tool() return value
through AppComponents and AgentDeps for automatic tool rebuilding
- tools: passed alongside builder for hot-reload logging
Remove all #[allow(dead_code)] annotations. Add regression tests for
threshold-based filtering (both above and below threshold).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: add missing `builder` field to AgentDeps in gateway workflow harness
After rebase onto staging, AgentDeps gained a `builder` field for
self-repair tool rebuilding. The gateway workflow test harness was
missing this field, causing CI compilation failure.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: retrigger CI
* fix: force CI refresh after path_routing_tests dedup
* test: add E2E test for stuck job repair and tool rebuild cycle
Tests the full self-repair flow requested in review:
1. Job transitions Pending -> InProgress -> Stuck
2. detect_stuck_jobs() finds it (zero threshold)
3. repair_stuck_job() recovers it back to InProgress
4. A broken tool is repaired via MockBuilder
5. Verify builder was invoked and repair succeeded
Uses a MockBuilder (impl SoftwareBuilder) that returns successful
BuildResult without requiring an LLM or filesystem. Uses libsql
test database for the store (increment_repair_attempts, mark_tool_repaired).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(self-repair): measure stuck_duration from Stuck transition, not started_at
- Use ctx.transitions to find the most recent Stuck transition timestamp
instead of ctx.started_at (which reflects job start, not stuck time)
- Fix StuckJob.last_activity to use stuck transition timestamp
- Remove misleading "hot-reloaded into registry" log
- Remove stray "// ci fix" comment in memory.rs
- Add regression test: backdated started_at must not inflate stuck_duration
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: re-trigger CI with latest changes
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: add type annotation to Ok(()) in test to resolve E0282
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* feat(testing): add FaultInjector framework for StubLlm (#1220)
Adds a configurable fault injection framework for testing retry, failover,
and circuit breaker behavior. The FaultInjector attaches to StubLlm and
provides per-call control over failure type, timing, and sequencing.
Components:
- FaultType: maps to LlmError variants (RequestFailed, RateLimited,
AuthFailed, InvalidResponse, IoError, ContextLengthExceeded, SessionExpired)
- FaultAction: Succeed, Fail(FaultType), Delay(Duration)
- FaultMode: SequenceOnce (play then succeed), SequenceLoop (repeat forever),
Random (seeded xorshift64 PRNG for reproducibility)
- FaultInjector: thread-safe (AtomicU32 counter + Mutex RNG)
Integration:
- StubLlm gains optional fault_injector field via with_fault_injector()
- When set, takes precedence over should_fail/error_kind
- Backward compatible: existing StubLlm usage unchanged
Closes#1220
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* refactor(testing): address review feedback on FaultInjector
- Remove redundant .abs() in random fault comparison
- Extract check_faults() helper to DRY up StubLlm methods
- Guard xorshift seed=0 (fixed point) by mapping to 1
- Add StubLlm integration test (stub_llm_fault_injector_sequence)
- Remove dead seed field from FaultMode::Random
- Move pub mod fault_injection to top of mod.rs
- Add Debug impl for FaultInjector
- Add empty_sequence_always_succeeds test
- Add random_seed_zero_does_not_always_fail test
* fix(testing): address #1233 review -- seed-0 bug, reset(), Debug derive
- Store seed in FaultMode::Random so reset() can re-init the RNG
- Add reset() method for test reproducibility (re-seeds RNG, zeros counter)
- Strengthen seed=0 regression test to 100 iterations with stricter assertion
- Add reset_restores_random_rng_from_stored_seed test
- Debug impl and empty_sequence test were already present from prior commit
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* ci: re-trigger CI with latest changes
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: trigger new run with skip-regression-check label
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(testing): address PR #1233 review -- error_rate validation and edge cases
- Validate error_rate is in 0.0..=1.0 and not NaN (panics on invalid input)
- Fix error_rate==1.0 edge case: use <= instead of < so 1.0 always fails
- Add regression tests for error_rate validation (NaN, negative, >1.0)
- Add tests for error_rate boundary values (0.0 never fails, 1.0 always fails)
- Add delay action test using tokio::time::pause() for deterministic timing
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat(telegram): support auto split large message
* fix(telegram): strengthen split_message test assertion
Replace word-by-word contains check with assert_eq! on rejoined chunks,
ensuring split_message preserves content exactly.
send_response is still used (lines 745, 753) so it is intentionally kept.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(telegram): add missing split_message tests and document limitations
- Add test for sentence-boundary splitting
- Add test for hard-cut on pathological input (no spaces)
- Add test for multi-byte character safety (emoji)
- Document CJK sentence punctuation limitation
- Document trim behavior at chunk boundaries
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* ci: re-trigger CI with latest changes
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Hans <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: remove debug_assert guards that panic on valid error paths (#1312)
Two debug_assert! calls added in #1312 fire on expected runtime error
paths (not programmer bugs), turning graceful error returns into panics
in debug/test builds:
- state.rs: Completed→Cancelled is a user-facing error handled by
transition_to() returning Err — not a bug
- execute.rs: empty tool_name from malformed LLM output is handled by
ToolError::NotFound — not a bug
Removes both asserts; keeps the circuit-breaker assert (genuinely guards
a caller invariant).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: tighten empty tool name test to assert ToolError::NotFound variant
Address review feedback: assert the specific error variant instead of
just is_err() so the regression test actually enforces the expected
error path.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* feat(gateway): full settings page polish with all tiers
- Backend: add ActiveConfigSnapshot to expose resolved LLM backend,
model, and enabled channels via /api/gateway/status
- Add missing Agent settings (daily cost cap, actions/hour, local tools)
- Add Sandbox, Routines, Safety, Skills, and Search setting groups
- Settings import/export (JSON download + file upload)
- Active env defaults shown as placeholders in Inference settings
- Styled confirmation modals replace window.confirm() for remove actions
- Global restart banner persists across settings subtab switches
- Client-side validation with min/max constraints on number inputs
- Accessibility: aria-label on inputs, role=status on save indicators
- Settings search filters rows across current subtab
- Smooth CSS transitions for conditional field visibility (showWhen)
- Tunnel settings in Channels subtab
- Mobile responsive settings layout at 768px breakpoint
- i18n keys for toolbar, search, and import/export in en + zh-CN
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat(gateway): polish settings page and remove registered tools debug section
Remove the "Registered Tools" table from the extensions tab (debug info
not useful to end users), clean up associated CSS/i18n/JS. Additional
settings page UI polish: extension card state styling, layout refinements.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address PR review feedback [skip-regression-check]
- Use refreshCurrentSettingsTab() in SSE event handlers to reduce duplication
- Remove unused formatGroupName/formatSettingLabel helpers
- Use i18n keys for MCP Configure/Reconfigure buttons
- Add data-i18n-placeholder to settings search input
- Remove data-i18n from confirm modal button (set dynamically by showConfirmModal)
- Fix cargo fmt in main.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): update tests for unified settings tab layout [skip-regression-check]
- Update TABS list: replace extensions/skills with settings
- Add settings_subtab/settings_subpanel selectors to helpers
- Update test_connection, test_skills, test_extensions, test_wasm_lifecycle
to navigate via Settings > subtab instead of top-level tabs
- Move MCP card tests to use go_to_mcp() helper (MCP is now a separate subtab)
- Remove tools table tests and mock_ext_apis tools= parameter
- Fix CSP violation: replace inline onclick on confirm modal cancel button
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address second round of PR review feedback [skip-regression-check]
- Use I18n.t() for MCP empty state, export/import toasts, confirm modal
- Fix CLI channel card using wrong channel key ('repl' -> 'cli')
- Fix settings search counting hidden rows as visible
- Add aria-label i18n for settings search input
- Add common.loadFailed i18n key (en + zh-CN)
- Update E2E tests: WASM channel tests use Channels subtab,
remove tests use custom confirm modal instead of window.confirm
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): fix WASM channel card selector and skills remove confirm [skip-regression-check]
- WASM channel tests: filter by display name to avoid matching built-in
channel cards in the Channels subtab
- Skills remove test: click confirm modal button instead of using
window.confirm (skill removal now uses custom confirm modal)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): address third round of PR review feedback [skip-regression-check]
- approval_needed SSE: refresh any active settings subtab, not just
Extensions — approvals can surface from Channels/MCP setup flows too
- renderCardsSkeleton: remove nested .extensions-list wrapper that
caused skeleton cards to render constrained inside grid cells
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): fix auth_completed reload test race condition [skip-regression-check]
Use expect_response to deterministically wait for the /api/extensions
reload triggered by handleAuthCompleted → refreshCurrentSettingsTab,
instead of a fixed 600ms sleep that was too short under CI load.
Also remove stale /api/extensions/tools route handler.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(e2e): debug auth_completed reload test with function counter [skip-regression-check]
Inject a counter wrapper around refreshCurrentSettingsTab to verify it's
actually called, and wait for the async fetch to complete before
asserting the reload count.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat(gateway): localize all settings labels, descriptions, and channel cards [skip-regression-check]
Move 120+ hardcoded strings in settings definitions (INFERENCE_SETTINGS,
AGENT_SETTINGS, NETWORKING_SETTINGS) and channel card labels to i18n
keys. Render functions now resolve labels via I18n.t() so the settings
page translates when switching locales.
Covers: group titles, setting labels/descriptions, built-in channel
names/descriptions, and the "No settings found" empty state.
Both en.js and zh-CN.js updated with all new cfg.* and channels.* keys.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): localize remaining hardcoded UI strings [skip-regression-check]
- Fix export error toast using wrong i18n key (importFailed → exportFailed)
- Replace "Failed to load settings:" with I18n.t('common.loadFailed')
- Localize renderBuiltinChannelCard: "Built-in", "Active", "Inactive"
- Localize settings placeholders: "env: ", "env default", "use env default"
- Localize "✓ Saved" indicator
- Add new i18n keys to both en.js and zh-CN.js
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): confirm modal a11y, Esc/click-outside, search guard [skip-regression-check]
- Add role="dialog", aria-modal="true", aria-labelledby to confirm modal
- Focus confirm button when modal opens
- Close modal on Escape key or overlay click
- Skip settings search on non-settings panels (Extensions/MCP/Skills/Channels)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): boolean tri-state, search reset on subtab switch, stale model suggestions [skip-regression-check]
Address PR review feedback:
- Boolean settings now use a tri-state select (env default / On / Off)
instead of a checkbox, matching the pattern used by other select settings
and allowing users to revert to the env default
- Clear search input when switching settings subtabs so stale filters
don't carry over to the new panel
- Always assign model suggestions (even empty array) so stale IDs from a
previous successful /v1/models fetch don't persist when the endpoint
later returns empty
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(gateway): auth_completed handler, bedrock_cross_region select, integer-only number inputs [skip-regression-check]
Address PR review feedback:
- auth_completed SSE listener now delegates to handleAuthCompleted(data)
instead of inlining logic with a bare closeConfigureModal() call, so
only the matching extension's modal is dismissed
- bedrock_cross_region changed from free text to select with the four
valid values (us/eu/apac/global), matching backend validation
- Number settings now use step=1 and parseInt() instead of parseFloat(),
preventing fractional values that the backend (u32/u64) would reject
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
* fix: full_job routine runs stay running until linked job completion (#1317)
Previously, execute_full_job() returned RunStatus::Ok immediately after
dispatching the job, causing routine runs to be marked as completed before
the linked worker job had actually finished. This meant failure notifications
were never sent and max_concurrent guardrails stopped applying once the run
was prematurely finalized.
Changes:
- execute_full_job() now returns RunStatus::Running instead of Ok
- execute_routine() skips finalization for Running status (leaves run open)
- New sync_dispatched_runs() polls on each cron tick, checks linked job
state, and finalizes runs when jobs reach terminal states
- New list_dispatched_routine_runs() DB method on both backends
- Deferred notifications are sent when the run is actually finalized
- consecutive_failures is preserved (not reset) while outcome is unknown
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review feedback (watcher predicate, running_count safety)
- FullJobWatcher: use is_parallel_blocking() instead of is_active() so
the watcher exits when a job reaches Completed (not terminal but
finished executing). Fixes infinite-poll for routine jobs.
- Remove running_count decrement from sync_dispatched_runs() — in normal
flow execute_routine() handles it; sync only runs for crash recovery
where the counter is already 0.
- Update PR description to match actual FullJobWatcher behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: sync only at startup to prevent double-completion race
- Move sync_dispatched_runs() out of cron loop into startup-only path.
During normal operation FullJobWatcher handles finalization inline;
running sync on every tick would race with the watcher.
- Update complete_dispatched_run() to properly advance runtime fields
(last_run_at, next_fire_at, run_count) for crash recovery — in that
scenario execute_routine() never reached its runtime update.
- Fix stale doc comment on complete_dispatched_run().
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: use boot_time filter for safe periodic sync of orphaned runs
- Add boot_time field to RoutineEngine, set to Utc::now() at creation.
- sync_dispatched_runs() now filters runs by started_at < boot_time,
so it only processes orphans from a previous process — never races
with FullJobWatcher instances from the current process.
- Move sync back into the cron loop (safe with boot_time filter) and
run it BEFORE check_cron_triggers to avoid picking up freshly
dispatched runs.
- Fix doc comments to match actual behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: add FullJobWatcher to track full_job lifecycle for concurrency (#1318)
full_job routines previously bypassed max_concurrent and global concurrency
limits because execute_full_job() returned RunStatus::Ok immediately after
dispatch. This meant running_count was decremented and the routine_run row
was finalized before the actual job completed.
Introduce FullJobWatcher struct that polls store.get_job() every 5s until
the linked job reaches a non-active state, then maps the final JobState to
RunStatus. execute_full_job now creates and awaits the watcher, keeping both
the DB-level running row and the in-memory running_count elevated for the
full job duration.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* test: full_job concurrency regression tests (issue #1318)
Add two integration tests verifying full_job routine concurrency:
1. full_job_max_concurrent_blocks_second_fire_while_first_active:
Inserts a Running routine_run (simulating an in-flight full_job) and
verifies fire_manual returns MaxConcurrent error for max_concurrent=1.
2. global_concurrency_counts_live_full_job_runs:
Elevates running_count to simulate a live full_job holding the global
slot, verifies check_cron_triggers skips due routines, then releases
the slot and verifies the routine fires.
Also makes running_count_for_test() unconditionally public so integration
tests (separate crate) can access it.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fmt and clippy fixes for full_job concurrency tests
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address PR review feedback on FullJobWatcher
- Add #[doc(hidden)] to running_count_for_test() to hide from public API
- Derive MAX_POLLS from POLL_INTERVAL to keep constants coupled
- Check job state before first sleep to finalize promptly for fast jobs
- Update execute_full_job doc comment to reflect blocking behavior
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
One flaky test (test_builtin_echo_tool timeout) was stopping the entire
e2e coverage suite via -x, preventing 118+ remaining tests from running
and generating coverage data.
Tests are independent (each gets a fresh browser context via the
function-scoped page fixture), so removing -x is safe.
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: consume matched event routine messages
* style: run rustfmt for event routine fix
* fix: preserve preprocessing for routine-triggered messages
* fix: match routines against rewritten input
* refactor: narrow check_event_triggers API and simplify routine_engine_slot
Address Copilot review feedback:
- Change check_event_triggers to accept (user_id, channel, content) instead
of &IncomingMessage, eliminating the need to clone the full message
(including attachments) when hooks rewrite content.
- Remove routine_trigger_message and the Cow<IncomingMessage> indirection;
the event-trigger check now inlines the is_internal + UserInput guard and
passes the post-hook content string directly.
- Make routine_engine_slot non-optional since Agent::new() always
initializes it. Removes the redundant Option wrapper and simplifies
accessor/setter methods.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: add debug_assert invariant guards to critical code paths (closes#1215)
Add three debug_assert! calls to catch impossible-in-correct-code states
early in debug builds without affecting release performance:
- execute_tool_with_safety: assert tool_name is non-empty at entry
- JobContext::transition_to: assert state machine transition is valid
- CircuitBreakerProvider::record_success: assert circuit is not Open
(check_allowed() must gate all calls before record_success())
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* test: add regression test for empty tool name invariant guard
Covers the debug_assert!(!tool_name.is_empty()) added in execute_tool_with_safety.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
---------
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
* feat: upgrade MiniMax default model to M2.7
- Add MiniMax-M2.7 and MiniMax-M2.7-highspeed to model list
- Set MiniMax-M2.7 as default model
- Keep all previous models as alternatives
- Update related tests
* fix: use canonical model name in test per review
Use MiniMax-M2.7-highspeed (canonical casing) in the reasoning
models test for consistency with the documentation and provider
configuration.
[skip-regression-check]
These tests guard against catastrophic regex backtracking (seconds/minutes),
not 12ms differences. CI runners with coverage instrumentation (cargo-llvm-cov)
consistently exceed the 100ms threshold due to overhead, causing flaky failures.
500ms still catches real regressions while tolerating CI variability.
[skip-regression-check]
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
* fix: Rate limiter returns retry after None instead of a duration
linter fix
* review fixes
* fix: rate limiter returns None for retry_after duration
Add regression test to src/llm/retry.rs that verifies RateLimited errors
always have a fallback duration (never None) due to the 60-second fallback
applied in all rate limit error creation sites (nearai_chat.rs,
anthropic_oauth.rs, embeddings.rs).
The production code fix adds `.or(Some(Duration::from_secs(60)))` to ensure
the error message never displays "retry after None" to the user.
[skip-regression-check]
Co-Authored-By: Claude Haiku 4.5 <[email protected]>
---------
Co-authored-by: Claude Haiku 4.5 <[email protected]>
