22 Commits

Include renames

Author	SHA1	Message	Date
3 people	706c3a1b47	refactor: extract AppEvent to crates/ironclaw_common (#1615) ... * refactor: extract AppEvent to crates/ironclaw_common SseEvent was defined in src/channels/web/types.rs but imported by 12+ modules across agent, orchestrator, worker, tools, and extensions — it had become the application-wide event protocol, not a web transport concern. Create crates/ironclaw_common as a shared workspace crate and move the enum there as AppEvent. Also move the truncate_preview utility which was similarly leaked from the web gateway into agent modules. - New crate: crates/ironclaw_common (AppEvent, truncate_preview) - Rename SseEvent → AppEvent, from_sse_event → from_app_event - web/types.rs re-exports AppEvent for internal gateway use - web/util.rs re-exports truncate_preview - Wire format unchanged (serde renames are on variants, not the enum) Aligned with the event bus direction on refactor/architectural-hardening where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: add AppEvent::event_type() helper, deduplicate match blocks Address Gemini review: extract the variant→string match into a single method on AppEvent, replacing the duplicated 22-arm matches in sse.rs and types.rs. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: rename leftover sse vars/tests to match AppEvent rename Address Copilot review: rename sse_event vars to app_event in orchestrator/api.rs and ws.rs, rename test functions from test_ws_server_from_sse_* to test_ws_server_from_app_event_*, and update stale SSE comments. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: add Deserialize to AppEvent, round-trip test, fix stale comments Address zmanian review: - Add Deserialize derive to AppEvent so downstream consumers can deserialize incoming events - Add event_type_matches_serde_type_field test that round-trips every variant through serde and asserts event_type() matches the serialized "type" field — catches drift between serde renames and the manual match - Add round_trip_deserialize test for basic Serialize/Deserialize parity - Update remaining "SSE" references in comments across server.rs, manager.rs, ws_gateway_integration.rs, and worker/job.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>	2026-03-24 23:02:46 -07:00
Henry ParkandGitHub	82822d7b25	fix: restore owner-scoped gateway startup (#1625) ... * fix: restore owner-scoped gateway startup * fix: split gateway owner and sender scope * fix: keep multi-user gateway sender identity * test: cover gateway sender scope regression * test: harden e2e startup teardown race * fix: align gateway owner scope across auth modes	2026-03-24 16:11:53 -07:00
4 people	b441ebec02	feat: multi-tenant auth with per-user workspace isolation (#1118) ... * feat: multi-tenant auth with per-user scoping Multi-user authentication and authorization for IronClaw gateway: - Token-based auth mapping tokens to user IDs via GATEWAY_USER_TOKENS - Per-user SSE broadcast scoping - Per-user rate limiting with poisoned lock recovery - Handler auth and ownership checks for jobs, settings, routines - Extension secrets scoped per-user - Chat handlers use authenticated identity - Reverse proxy deployment documentation - Comprehensive integration tests for auth, SSE, rate limiting, and job isolation * fix: scope memory tools per-user in multi-tenant mode Memory tools (search, write, read, tree) held a single workspace created at startup with GATEWAY_USER_ID. In multi-tenant mode, all users' tool calls searched the default user's scope. Add WorkspaceResolver trait that resolves workspaces per-request using JobContext.user_id. In single-user mode, returns the startup workspace. In multi-tenant mode (GATEWAY_USER_TOKENS configured), creates and caches per-user workspaces on demand. Includes regression tests for workspace resolution and user isolation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: comprehensive multi-tenant isolation audit Address all review findings from @serrrfirat plus 7 additional gaps found via full security audit: Reviewer findings (5): - WorkspacePool now applies search config, memory layers, embedding cache, identity read scopes, and global config scopes (was bare) - jobs_summary_handler uses per-user queries instead of global counters - jobs_prompt_handler restructured to not 404 agent jobs + ownership check - jobs_restart_handler agent branch now verifies user ownership - agent_job_summary_for_user added to Database trait + both backends Audit findings (7): - Delete dead handlers/memory.rs (stale copies with no auth) - Add AuthenticatedUser to logs_events, logs_level_get, logs_level_set - Add AuthenticatedUser to extensions_tools_handler, gateway_status_handler - Add auth + ownership checks to all 6 routines handlers - Add auth to all 4 skills handlers with audit logging on mutations - Scope extension setup SSE broadcast to user (broadcast_for_user) - Fix pre-existing test compilation errors in extensions/manager.rs 17 new multi-tenant isolation tests covering: - WorkspacePool config propagation and scope merging - Jobs handler per-user isolation (summary, restart, prompt, cancel) - Routines handler auth enforcement and cross-user rejection - Auth middleware enforcement on logs, skills, status endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: second-pass multi-tenant audit — scope SSE broadcasts, DB queries, dead handlers Second audit pass applying learned patterns across the codebase: - OAuth callback SSE broadcasts now use broadcast_for_user (lines 773, 912) - jobs_list_handler uses list_agent_jobs_for_user instead of fetching all users' jobs and filtering in Rust - list_agent_jobs_for_user added to Database trait + postgres + libsql - Dead handler files (extensions.rs, static_files.rs) hardened with AuthenticatedUser to prevent auth regression if migrated Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — token hashing, broadcast scoping, error handling Security fixes: - Hash tokens with SHA-256 at construction time so authentication compares fixed-size 32-byte digests, eliminating length-oracle timing leaks - Scope auth SSE broadcasts per-user in chat_auth_token_handler — AuthRequired/AuthCompleted events were leaking across tenants - Propagate DB errors in restart handlers instead of silently swallowing via `if let Ok(Some(...))` pattern Code quality: - Log SSE serialization failures instead of silently producing empty strings via unwrap_or_default() - Remove dead `pub type AuthState = MultiAuthState` alias - Replace `.unwrap()` with `Arc::clone(db)` in app.rs multi-tenant workspace setup (db is guaranteed Some in context, but unwrap violates project convention) - Fix telegram setup test to inject UserIdentity into request extensions (handler now requires AuthenticatedUser) - Add safety comments on test-only expect/unwrap calls for CI - Apply cargo fmt to fix pre-existing formatting Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — unify workspace pool, fix SSE regression, cache job owners - Unify WorkspacePool and PerUserWorkspaceResolver: WorkspacePool now implements WorkspaceResolver, eliminating duplicate per-user workspace construction logic. app.rs uses WorkspacePool directly. - Fix sse_tx: None scheduler regression: change scheduler/worker SSE broadcasting from broadcast::Sender<SseEvent> to Arc<SseManager>, restoring SSE event delivery for scheduled agent jobs. - Cache job owner in orchestrator: add job_owner_cache to OrchestratorState so job_event_handler avoids a DB round-trip on every event after the first per job. - Deduplicate ext_user_id computation in main.rs. - Remove unused _gateway_state variable. - Fix pre-existing test: first_token() returns None in multi-user mode by design; align test assertion. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in app.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: extract memory handlers back into handlers/memory.rs Move memory API handlers out of server.rs into their own module, consistent with how jobs, routines, and skills handlers are organized. The resolve_workspace() helper moves with them since it is only used by memory handlers. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: [email protected] <[email protected]>	2026-03-23 20:50:05 -07:00
4 people	6d847c6009	feat(webhooks): add public webhook trigger endpoint for routines (#736) ... * feat(webhooks): add public webhook trigger endpoint for routines Add POST /api/webhooks/{path} endpoint that matches incoming webhooks against routines with Trigger::Webhook, validates secrets using constant-time comparison (subtle crate), and fires the matched routine through the message pipeline. Closes #651 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(webhooks): address PR review feedback - access control, targeted query, rate limiting [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(ci): add missing webhook_rate_limiter field and fix formatting Add the webhook_rate_limiter field to the GatewayState initializer in gateway_workflow_harness.rs and fix rustfmt formatting for the webhook tuple in types.rs. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): require webhook secret, add rate limiting, improve tests Extract validate_webhook_secret() from the handler so the security-critical secret validation logic (mandatory secret, constant-time comparison) is directly testable without mocking the database layer. Improves the error message for misconfigured routines to guide users toward the fix. Replaces the previous unit tests (which only tested Rust pattern matching and status code constants) with tests that exercise the actual validation function against all rejection paths: missing secret (403), non-webhook trigger (403), wrong secret (401), empty secret (401), and different-length secret (401). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Route webhook triggers through RoutineEngine instead of chat pipeline Adds fire_webhook() to RoutineEngine and updates the webhook handler to use it. This ensures webhook-triggered routines get proper run tracking, guardrail enforcement (cooldown + max_concurrent), notifications, and FullJob dispatch support. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in webhook handler Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: [email protected] <[email protected]>	2026-03-20 15:50:31 -07:00
3 people	4566181f40	feat(gateway): unified settings page with subtabs (#1191) ... * feat(gateway): full settings page polish with all tiers - Backend: add ActiveConfigSnapshot to expose resolved LLM backend, model, and enabled channels via /api/gateway/status - Add missing Agent settings (daily cost cap, actions/hour, local tools) - Add Sandbox, Routines, Safety, Skills, and Search setting groups - Settings import/export (JSON download + file upload) - Active env defaults shown as placeholders in Inference settings - Styled confirmation modals replace window.confirm() for remove actions - Global restart banner persists across settings subtab switches - Client-side validation with min/max constraints on number inputs - Accessibility: aria-label on inputs, role=status on save indicators - Settings search filters rows across current subtab - Smooth CSS transitions for conditional field visibility (showWhen) - Tunnel settings in Channels subtab - Mobile responsive settings layout at 768px breakpoint - i18n keys for toolbar, search, and import/export in en + zh-CN Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(gateway): polish settings page and remove registered tools debug section Remove the "Registered Tools" table from the extensions tab (debug info not useful to end users), clean up associated CSS/i18n/JS. Additional settings page UI polish: extension card state styling, layout refinements. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): address PR review feedback [skip-regression-check] - Use refreshCurrentSettingsTab() in SSE event handlers to reduce duplication - Remove unused formatGroupName/formatSettingLabel helpers - Use i18n keys for MCP Configure/Reconfigure buttons - Add data-i18n-placeholder to settings search input - Remove data-i18n from confirm modal button (set dynamically by showConfirmModal) - Fix cargo fmt in main.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(e2e): update tests for unified settings tab layout [skip-regression-check] - Update TABS list: replace extensions/skills with settings - Add settings_subtab/settings_subpanel selectors to helpers - Update test_connection, test_skills, test_extensions, test_wasm_lifecycle to navigate via Settings > subtab instead of top-level tabs - Move MCP card tests to use go_to_mcp() helper (MCP is now a separate subtab) - Remove tools table tests and mock_ext_apis tools= parameter - Fix CSP violation: replace inline onclick on confirm modal cancel button Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): address second round of PR review feedback [skip-regression-check] - Use I18n.t() for MCP empty state, export/import toasts, confirm modal - Fix CLI channel card using wrong channel key ('repl' -> 'cli') - Fix settings search counting hidden rows as visible - Add aria-label i18n for settings search input - Add common.loadFailed i18n key (en + zh-CN) - Update E2E tests: WASM channel tests use Channels subtab, remove tests use custom confirm modal instead of window.confirm Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(e2e): fix WASM channel card selector and skills remove confirm [skip-regression-check] - WASM channel tests: filter by display name to avoid matching built-in channel cards in the Channels subtab - Skills remove test: click confirm modal button instead of using window.confirm (skill removal now uses custom confirm modal) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): address third round of PR review feedback [skip-regression-check] - approval_needed SSE: refresh any active settings subtab, not just Extensions — approvals can surface from Channels/MCP setup flows too - renderCardsSkeleton: remove nested .extensions-list wrapper that caused skeleton cards to render constrained inside grid cells Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(e2e): fix auth_completed reload test race condition [skip-regression-check] Use expect_response to deterministically wait for the /api/extensions reload triggered by handleAuthCompleted → refreshCurrentSettingsTab, instead of a fixed 600ms sleep that was too short under CI load. Also remove stale /api/extensions/tools route handler. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(e2e): debug auth_completed reload test with function counter [skip-regression-check] Inject a counter wrapper around refreshCurrentSettingsTab to verify it's actually called, and wait for the async fetch to complete before asserting the reload count. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(gateway): localize all settings labels, descriptions, and channel cards [skip-regression-check] Move 120+ hardcoded strings in settings definitions (INFERENCE_SETTINGS, AGENT_SETTINGS, NETWORKING_SETTINGS) and channel card labels to i18n keys. Render functions now resolve labels via I18n.t() so the settings page translates when switching locales. Covers: group titles, setting labels/descriptions, built-in channel names/descriptions, and the "No settings found" empty state. Both en.js and zh-CN.js updated with all new cfg.* and channels.* keys. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): localize remaining hardcoded UI strings [skip-regression-check] - Fix export error toast using wrong i18n key (importFailed → exportFailed) - Replace "Failed to load settings:" with I18n.t('common.loadFailed') - Localize renderBuiltinChannelCard: "Built-in", "Active", "Inactive" - Localize settings placeholders: "env: ", "env default", "use env default" - Localize "✓ Saved" indicator - Add new i18n keys to both en.js and zh-CN.js Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): confirm modal a11y, Esc/click-outside, search guard [skip-regression-check] - Add role="dialog", aria-modal="true", aria-labelledby to confirm modal - Focus confirm button when modal opens - Close modal on Escape key or overlay click - Skip settings search on non-settings panels (Extensions/MCP/Skills/Channels) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): boolean tri-state, search reset on subtab switch, stale model suggestions [skip-regression-check] Address PR review feedback: - Boolean settings now use a tri-state select (env default / On / Off) instead of a checkbox, matching the pattern used by other select settings and allowing users to revert to the env default - Clear search input when switching settings subtabs so stale filters don't carry over to the new panel - Always assign model suggestions (even empty array) so stale IDs from a previous successful /v1/models fetch don't persist when the endpoint later returns empty Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): auth_completed handler, bedrock_cross_region select, integer-only number inputs [skip-regression-check] Address PR review feedback: - auth_completed SSE listener now delegates to handleAuthCompleted(data) instead of inlining logic with a bare closeConfigureModal() call, so only the matching extension's modal is dismissed - bedrock_cross_region changed from free text to select with the four valid values (us/eu/apac/global), matching backend validation - Number settings now use step=1 and parseInt() instead of parseFloat(), preventing fractional values that the backend (u32/u64) would reject Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-03-18 16:18:29 -07:00
4 people	b0214fef41	feat: add channel-relay integration for Slack (#790) ... * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 <[email protected]> * feat: add channel-relay integration for Slack via external relay service - Add RelayChannel and RelayClient for connecting to channel-relay SSE streams - Add RelayConfig with env-based configuration (CHANNEL_RELAY_URL, CHANNEL_RELAY_API_KEY) - Add channel-relay extension lifecycle: install, OAuth auth, activate with hot-add - Add proxy message sending through channel-relay for Slack chat.postMessage - Add extension registry entry for Slack relay with OAuth auth hint - Add relay integration test with mock SSE server - Wire relay channel into app startup with reconnect on stored credentials - Add AuthRequired extension error variant for cleaner auth flow detection [skip-regression-check] * chore: apply cargo fmt * fix: remove remaining Telegram test references in relay channel * fix: address PR #790 review feedback — parser handle leak, CSRF, circuit breaker - Fix parser handle leak on reconnect by sharing Arc<RwLock> instead of creating a local copy in start() (shutdown now aborts the correct task) - Add CSRF state nonce to OAuth flow: generate in auth_channel_relay, validate in slack_relay_oauth_callback_handler, one-time use - Remove dead proxy_slack method, update integration test to use proxy_provider - Add reconnect circuit breaker (max_consecutive_failures, default 50) - Fix stale docs (Telegram refs), extract event_types constants --------- Co-authored-by: Henry Park <[email protected]> Co-authored-by: Claude Sonnet 4.6 <[email protected]>	2026-03-10 16:34:54 -07:00
3 people	34f69b31dc	fix: prevent session lock contention blocking message processing (#783) ... * fix: prevent session lock contention blocking message processing ## Problem After container restart, POST /api/chat/send returns 202 ACCEPTED but messages don't appear in conversation_messages and agent never responds. Messages get stuck in "stale state" after restart. Root cause: Session lock was held for entire duration of chat_threads_handler and chat_history_handler, including during slow database queries. This blocked the agent loop from acquiring the session lock to process incoming messages, causing them to hang indefinitely. ## Solution 1. **Release session lock early in chat_threads_handler**: Only acquire lock when reading active_thread at response time, not during DB queries for thread list. DB operations no longer block message processing. 2. **Release session lock early in chat_history_handler**: Only acquire lock when accessing in-memory thread state, not during paginated DB queries or thread ownership checks. DB operations no longer block message processing. 3. **Add comprehensive logging**: Track message flow from receipt through session resolution, thread hydration, and state transitions. Helps diagnose future issues: - Message queued to agent loop (chat_send_handler) - Processing message from channel (handle_message) - Hydrating thread from DB (maybe_hydrate_thread) - Resolving session and thread (resolve_thread) - Checking thread state (process_user_input) - Persisting user message (persist_user_message) ## Impact - Message processing no longer blocks on session lock contention - API response times for thread list/history queries unaffected (DB queries still happen, but lock is not held) - Better diagnostics for future debugging ## Testing - All 2756 tests pass - Code compiles with zero clippy warnings - No changes to user-facing API or behavior, only lock timing Co-Authored-By: Claude Haiku 4.5 <[email protected]> * security: redact PII from info-level logs Downgrade user_id and channel logging to debug level to prevent exposing Personally Identifiable Information (PII) in production logs. The user_id field can contain sensitive information such as phone numbers (e.g., for Signal messages). Logging PII in cleartext at the info level creates a security and privacy risk, as these logs may be stored in persistent storage, indexed by log management systems, or accessible to unauthorized personnel. Changes: - Info level: logs only message_id (UUID) for tracking - Debug level: logs user_id, channel, thread_id for troubleshooting This maintains debugging capability for developers while protecting user privacy in production logs. Co-Authored-By: Claude Haiku 4.5 <[email protected]> --------- Co-authored-by: Claude Haiku 4.5 <[email protected]>	2026-03-10 08:11:30 -07:00
3 people	9f71bd0d44	feat: unified thread model for web gateway (#607) ... * feat: unified thread model for web gateway Every piece of activity (user chat, routine run, heartbeat alert, external channel message) now lives in its own thread, properly isolated, with meaningful titles and visual distinction. Key changes: - Add `channel` field to ConversationSummary and ThreadInfo so the gateway can distinguish thread origins (gateway, telegram, routine, heartbeat). - Add `list_conversations_all_channels` to Database trait (both postgres and libsql) so chat_threads_handler shows cross-channel threads. - Routine runs get a persistent conversation per routine via `get_or_create_routine_conversation`; notifications carry thread_id. - Heartbeat gets a persistent conversation via `get_or_create_heartbeat_conversation`; HeartbeatRunner accepts an optional Database store and binds notifications to the thread. - Fix broadcast() in web gateway to propagate response.thread_id instead of hardcoding empty string. - Fix isCurrentThread(null) returning true (the core notification leak bug) — now returns false so events without a thread_id don't leak into the active thread. - Rewrite frontend thread sidebar: meaningful titles with channel-specific fallbacks, relative timestamps instead of turn counts, channel badges for non-gateway threads, unread notification dots, read-only indicator for external channel threads. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — TOCTOU races, stale comment, debounce, broadcast warning - Fix TOCTOU race in get_or_create_routine_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_routine unique index + SELECT-back. - Fix TOCTOU race in get_or_create_heartbeat_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_heartbeat unique index + SELECT-back. - Fix TOCTOU race in get_or_create_routine_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Fix TOCTOU race in get_or_create_heartbeat_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Add V11 migration with partial unique indexes for postgres. - Add matching unique indexes to libsql schema. - Update stale comment on isCurrentThread (said "always shown" but logic now returns false for missing thread_id). - Debounce loadThreads() on off-thread SSE events to prevent request storms. - Log warning in broadcast() when thread_id is None (clients will drop it). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: sort in-memory thread fallback by updated_at descending The in-memory thread list fallback (when no DB is available) used HashMap::values() which has no guaranteed ordering. Sort by updated_at descending to match the SQL query ordering. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: retry libsql connect() on transient "unable to open database file" The cron ticker's background task occasionally fails with "unable to open database file" when creating a new SQLite connection concurrently with the main thread. Add retry with exponential backoff (50ms, 100ms, 200ms) to handle transient VFS/locking issues in libsql's local mode. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use ON CONFLICT with index expressions instead of named constraints PostgreSQL ON CONFLICT ON CONSTRAINT requires a named table constraint, but V11 migration creates unique indexes. Switch to the expression form (ON CONFLICT (columns) WHERE condition) which works with unique indexes. Also fix dead code in threadTitle() where thread.title was already checked on the previous line. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt chain collapse in heartbeat.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: skip broadcast when thread_id is None instead of sending empty Clients drop SSE events with empty thread_id anyway, so avoid the unnecessary network traffic by returning early. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add libsql routine/heartbeat conversation idempotency tests Add tests proving get_or_create_routine_conversation returns the same conversation ID across multiple invocations with the same routine_id. Add debug logging to routine engine to track conversation resolution. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: show "New chat" title for empty threads - threadTitle() returns "New chat" when turn_count is 0 - Assistant thread label updates dynamically from API data - Default HTML label changed from "Assistant" to "New chat" - New threads naturally sort to top via last_activity DESC [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: thread sorting, routine isolation, and UI polish - Fix libsql timestamp format mismatch causing broken thread sort order. SQLite defaults used `datetime('now')` (space-separated) while Rust code used RFC3339 (T-separated), breaking string-based ORDER BY. All INSERTs now use RFC3339, and queries use `datetime()` to normalize comparison. - Route manual routine triggers through RoutineEngine.fire_manual() instead of injecting as regular chat messages, so routines always run in their dedicated conversation thread. - Add RoutineEngineSlot to GatewayState for gateway<->engine communication. - Derive routine thread titles from conversation metadata (routine_name) instead of showing truncated UUID hashes. - Make chat_new_thread_handler persist to DB synchronously so loadThreads() sees newly created threads immediately. - Fix enableChatInput() no-op and wrong element ID in disableChatInputReadOnly(). - Fix handlers/chat.rs stale gateway-only query (use list_conversations_all_channels). - Sort in-memory threads by DateTime before converting to RFC3339 strings. - Trigger debouncedLoadThreads() on thinking/status SSE events for non-current threads so routine/heartbeat threads appear in sidebar promptly. - Remove "Threads" text from sidebar header. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: routine history display, orphaned tool_results, duplicate system messages Three independent fixes with regression tests: 1. Routine conversations now display in the web UI. build_turns_from_db_messages() handles standalone assistant messages (no preceding user message) by creating turns with empty user_input. Frontend skips empty user bubbles. 2. Worker select_tools and execute_plan paths now push an assistant_with_tool_calls message before tool execution, preventing sanitize_tool_messages from rewriting tool_results as orphaned user messages. 3. Reasoning::plan() and respond_with_tools() merge system messages from context into a single system prompt instead of creating [system, system, ...] sequences that strict LLM providers (Qwen) reject. Also: sidebar padding/spacing improvements, wider thread panel (240px). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #607 review — RwLock held across await, missing ownership check, heartbeat config - Clone Arc<RoutineEngine> out of RwLock before .await in trigger handler - Add user_id ownership check to fire_manual() with NotAuthorized error - Wire heartbeat notify_user/notify_channel from config to AgentHeartbeatConfig Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: gitignore trace_*.json files and remove stale traces Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: remove trace JSON files from repo Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: proper HTTP status codes for routine errors, read-only input guard, respond thread_id - Map RoutineError::NotFound → 404, NotAuthorized → 403, Disabled → 409 - Guard enableChatInput() against re-enabling on read-only threads - Skip respond() when thread_id is None (matches broadcast() behavior) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-03-07 19:53:43 +00:00
4 people	30790439ee	perf: build system prompt once per turn, skip tools on force-text (#583) ... * perf: build system prompt once per turn, skip tools on force-text, fix nudge role (#565) Three fixes to agentic loop prompt handling: 1. Build system prompt once per turn instead of every tool iteration. `build_system_prompt_with_tools` is now pub; callers pass the result via `ReasoningContext::system_prompt` to avoid rebuilding ~1,500 tokens per iteration. 2. Skip `## Available Tools` section when `force_text = true`. The dispatcher passes a no-tools prompt variant on the final iteration, saving ~460 tokens and removing misleading instructions. 3. Change nudge message from `Role::System` to `Role::User`. A second system message mid-conversation is unsupported by most providers. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: revert nudge role change to keep ChatMessage::system Copilot review correctly identified that using Role::User for the nudge breaks compact_messages_for_retry, which uses rposition for Role::User to find the last real user message. Role::Assistant would cause back-to-back assistant messages. Since no production issues were reported with the original system role, revert to ChatMessage::system. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review — omit tool guidance when tools empty, rename shadowed var - Conditionalize "Call tools…" guidelines and "## Tool Call Style" section in the system prompt so they are only included when tools are non-empty. Previously the force-text (no-tools) prompt still contained misleading tool-calling instructions. (Copilot review comment) - Rename `system_prompt` → `cached_prompt` in dispatcher to avoid shadowing the earlier workspace identity `system_prompt` variable. (Copilot review) - Add regression tests: `test_system_prompt_with_tools_contains_tool_guidance` and extended assertions in `test_system_prompt_without_tools_omits_tools_section`. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: [email protected] <[email protected]>	2026-03-07 09:15:00 +00:00
3 people	45ec691f4c	Improve test infrastructure: StubChannel, gateway helpers, security tests, search edge cases (#623) ... * feat(testing): add StubChannel test double for Channel trait Adds StubChannel to src/testing.rs alongside StubLlm. Supports message injection via mpsc sender, response/status capture, and configurable health check toggling. Includes handle methods for use after ownership transfer to ChannelManager. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(testing): wire StubChannel into TestHarnessBuilder Add with_stub_channel() builder method that creates a StubChannel pre-registered in a ChannelManager. Tests can inject messages via the sender and verify routing through the manager. The channel field on TestHarness is Optional, defaulting to None for backward compat. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: gate external-service tests behind integration feature flag Replace silent try_connect() skip pattern with explicit feature gating. cargo test now runs only self-contained tests. cargo test --features integration runs tests requiring PostgreSQL. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(channels): add ChannelManager unit tests using StubChannel Cover add/start_all stream merging, respond routing, unknown channel errors, health_check_all with mixed health, empty-channels error path, and injection channel merging -- all via StubChannel test double. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: document test tier separation (unit/integration/live) Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add architecture boundary check script Grep-based checks for three architecture boundaries: - Direct database driver usage (tokio_postgres/libsql) outside src/db/ - .unwrap()/.expect() in production code (warning only) - Direct std::env::var reads outside config layer (warning only) The DB driver check is a hard violation; the other two are warnings for gradual cleanup. Run with: bash scripts/check-boundaries.sh Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(search): add RRF edge case tests for empty inputs, limits, and config modes Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(security): add regression tests for skill installer ZIP and SSRF protections Add 11 regression tests covering the security controls in skill_tools: ZIP extraction safety: - Valid SKILL.md extraction works correctly - Non-SKILL.md entries are ignored (returns error) - Path traversal entries (../../SKILL.md) do not match - Nested path entries (subdir/SKILL.md) do not match - Oversized entries (>1MB uncompressed) are rejected SSRF prevention: - Loopback addresses (127.0.0.1) are blocked - Private ranges (10.x, 172.16.x, 192.168.x) are blocked - Link-local addresses (169.254.x) are blocked - Public IPs (8.8.8.8, 1.1.1.1) are allowed - IPv4-mapped IPv6 unwrapping logic works correctly - Metadata endpoints and .internal/.local hostnames are blocked - Normal hostnames (github.com, clawhub.dev) are allowed Also documents a known gap: url::Url::host_str() returns bracketed IPv6 addresses that std::net::IpAddr cannot parse, so IPv4-mapped IPv6 URLs currently bypass IP-based checks in validate_fetch_url. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(testing): extract TestGatewayBuilder to eliminate gateway test duplication Both ws_gateway_integration.rs and openai_compat_integration.rs manually constructed GatewayState with 19+ fields. Extracted to a shared builder in src/channels/web/test_helpers.rs that provides sensible defaults and lets tests override only what they need. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: add implementation plans for testing batches 1 and 2 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): close IPv6 SSRF bypass in validate_fetch_url validate_fetch_url used host_str() which returns bracketed IPv6 (e.g. "[::ffff:7f00:1]") that IpAddr::parse() cannot handle, silently skipping IP-based SSRF checks for all IPv6 URLs. Switch to url::Host enum matching to extract proper IpAddr values without string parsing. IPv4-mapped IPv6 addresses like ::ffff:127.0.0.1 are now correctly unwrapped and blocked. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(skills): add activation criteria limits enforcement tests Adds test_activation_criteria_enforce_limits to verify that enforce_limits() correctly trims excess patterns (>5), keywords (>20), and tags (>10), and filters out short keywords/tags (<3 chars). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(wasm): add security regression tests for WASM tool loader Add 6 tests covering: tool name path separator rejection, empty name rejection, nonexistent file handling, invalid WASM bytes rejection, dotfile discovery behavior, and subdirectory non-recursion. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: address PR review feedback - Remove plan files from repo (ilblackdragon review) - Replace CLAUDE.md test tier rules with pointer to check-boundaries.sh - Add Check 4 to check-boundaries.sh: enforces integration tests are gated behind the 'integration' feature flag Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add try_connect silent-skip pattern check to check-boundaries.sh Check 5 catches try_connect() and similar silent-skip patterns in integration tests. Tests should use feature gates to fail loudly when prerequisites are missing, not silently return. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): harden skill fetch SSRF checks * fix(scripts): use bash arrays in check-boundaries.sh tier violation check Refactor Check 4 in check-boundaries.sh to use bash arrays and printf instead of string concatenation with echo -e. This is more robust with special characters in filenames and avoids portability concerns with echo -e. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-03-07 08:30:47 +00:00
3 people	902492bcdb	feat(web): show error details for failed tool calls (#490) ... * feat(web): show error details and input params for failed tool calls Failed tool calls in the gateway UI previously showed only a red X icon with an empty expandable body. This change: - Adds optional `error` and `parameters` fields to `ToolCompleted` SSE events so the browser receives failure details in real-time - Auto-expands failed tool cards to make errors immediately visible - Adds `StatusUpdate::tool_completed()` constructor that centralizes the 5 duplicated construction sites and applies `redact_params()` to prevent sensitive values (e.g. secret_save's "value" param) from leaking through SSE broadcasts - Adds `sensitive_params()` trait method to `Tool` for declaring which parameters must be redacted before logging, hooks, and UI display - Adds `redact_params()` utility and wires it through hooks, approvals, ActionRecord storage, and debug logs in dispatcher/worker - Adds `SecretListTool` and `SecretDeleteTool` for LLM-driven secret management (values never returned, only names/metadata) - Fixes auth flow: setup-only extensions show configure modal instead of OAuth card; auth_completed SSE dismisses both UI paths - CI: release workflow creates PR instead of pushing directly to main - Registry: MissingChecksum error enables source fallback for bootstrapping when checksums haven't been populated yet Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: apply cargo fmt Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: keep original params in PendingApproval for execution, redact only for display Address two PR review comments: 1. execute_chat_tool_standalone now redacts sensitive params before logging, matching the pattern already used in worker.rs. 2. PendingApproval previously stored redacted parameters, which meant approved tool calls received "[REDACTED]" instead of the actual values. Add a display_parameters field for UI/logs and keep parameters as the original values used for execution. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments - worker.rs: redact sensitive params before BeforeToolCall hook, matching dispatcher.rs — hooks in the autonomous job path now receive redacted params instead of raw values - registry.rs: fix docstring for register_secrets_tools (list, delete, not save/list/delete — no SecretSaveTool is registered) - app.js: fix double toast/loadExtensions in submitConfigureModal — for non-OAuth success the auth_completed SSE already handles both, so skip them in the HTTP response handler to avoid duplicates [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>	2026-03-04 15:38:26 -08:00
3 people	f18fb5173b	feat(web): fix jobs UI parity for non-sandbox mode (#491) ... * feat(web): fix jobs UI parity for non-sandbox mode The web gateway Jobs UI was built primarily for sandbox (Docker) jobs. When running without sandbox (common for NEAR AI hosted envs), multiple features were broken. This change fixes all of them: - Agent jobs now broadcast live SSE events to the web UI (Activity tab) - Agent job restart via scheduler.dispatch_job (not chat message) - Follow-up prompts for agent jobs via WorkerMessage injection - Capability flags (can_restart, can_prompt, job_kind) in job detail API - Rate-limit retry with cap (10 consecutive) and Retry-After header parsing - Plan interruption on user message (breaks out of plan, re-evaluates) - Correct SSE status field in mark_completed/mark_failed/mark_stuck - SseManager preserved across rebuild_state calls Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in db/mod.rs and nearai_chat.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>	2026-03-03 22:23:30 +08:00
3 people	78878ad7ef	Remove restart infrastructure, generalize WASM channel setup (#493) ... * refactor: remove restart infrastructure and generalize Telegram-specific code Remove the gateway restart mechanism (hot-activation works, restart won't fix activation failures) and generalize Telegram-specific hardcoded checks so all WASM channels get equal treatment. Part 1 - Remove restart infrastructure: - Remove needs_restart from ActionResponse, restart_requested from GatewayState - Remove gateway_restart_handler, /api/gateway/restart route, exit code 75 - Remove restart overlay JS/CSS (dead code - restartGateway() never called) - Surface actual activation errors instead of suggesting restart Part 2 - Generalize Telegram-specific code: - Replace telegram_owner_id: Option<i64> with generic wasm_channel_owner_ids: HashMap<String, i64> (backwards-compatible via TELEGRAM_OWNER_ID env var) - Pairing status check now applies to all active WASM channels - All channels get 3-step stepper in web UI, remove "coming soon" note - Remove dead setup_telegram() code (~700 lines) - Telegram's capabilities.json declares required_secrets, so the generic setup_wasm_channel() path handles it Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test: add Settings::set() test for wasm_channel_owner_ids Addresses review feedback: verify that setting per-channel owner IDs via the dotted-path Settings::set() API works correctly with the new HashMap<String, i64> type. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): refresh extension stepper after pairing approval loadPairingRequests only refreshed the pairing section, not the stepper status. Call loadExtensions() instead so the stepper updates from "Awaiting Pairing" to "Active" immediately after approval. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>	2026-03-03 22:10:32 +08:00
3 people	996c6a8cc9	feat(web): improve WASM channel setup flow (#380) ... * feat(web): improve WASM channel setup flow with stepper UI and auto-configure Streamline the WASM channel setup experience in the web gateway: - Auto-open configure modal after installing a WASM channel - Add progress stepper (Installed → Configured → Active) on channel cards - Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart) - Show "Awaiting Pairing" status for Telegram until first user is paired - Add SSE extension_status events for real-time status updates - Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard - Always mount webhook routes at startup so hot-added channels work without restart - Add pairing request polling (10s interval) on extensions tab - Track activation errors per channel with inline error display Includes review fixes: activation_error priority over active status, stepper failed state rendering, restart poll timeout, configure modal double-submit guard, and SSE sender ordering constraint documentation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: address PR review comments - Move PairingStore construction outside .map() loop - Extract createReconfigureButton() helper to reduce duplication Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>	2026-02-25 22:27:13 -08:00
3 people	436066415b	feat: embedded registry catalog and WASM bundle install pipeline (#283) ... * feat: embedded registry catalog and WASM bundle install pipeline Embed registry manifests at compile time so the extension catalog is available without network access. Add tar.gz bundle support for WASM extension downloads (tools and channels), a /api/extensions/registry endpoint, CI job to build and publish WASM bundles on release, and ephemeral in-memory secrets fallback so the extension manager works even without a persistent secrets store. Key changes: - build.rs: collect registry/*.json into embedded_catalog.json at compile time - src/registry/embedded.rs + catalog.rs: load embedded or on-disk catalog - src/extensions/manager.rs: download_and_install_wasm handles tar.gz bundles, bare .wasm files, and separate capabilities downloads; wasm channel install - src/channels/web/server.rs: /api/extensions/registry endpoint + no-cache headers - src/app.rs: ephemeral InMemorySecretsStore fallback for extension manager - registry/*.json: populate artifact download URLs for release bundles - .github/workflows/release.yml: build-wasm-extensions CI job - Simplified setup wizard and CLI registry commands Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — archive hardening, decompression bomb guard, test fix - Add 100 MB decompressed entry size cap to tar.gz extraction in both manager.rs and installer.rs to prevent decompression bombs - Add archive.set_preserve_permissions(false) and set_unpack_xattrs(false) for defense-in-depth against malicious archives - Fix test assertion logic in catalog.rs (|| → || with correct negation) - Replace silent tar fallback in CI with explicit if/else for capabilities - Add warning when installing without SHA256 verification Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: resolve clippy warning in settings.rs and enforce zero-warnings policy Use struct initializer with ..Default::default() instead of field reassignment. Update CLAUDE.md to codify zero clippy warnings policy — all warnings must be fixed before committing, including pre-existing ones. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review round 2 — build reliability, caps validation, naming - build.rs: emit per-file rerun-if-changed for reliable content tracking; fix bundles fallback to match BundlesFile shape ({"bundles":{}}) - embedded.rs: parse catalog once via OnceLock instead of double-parsing - manager.rs + installer.rs: add 1 MB size cap on capabilities_url downloads with proper error surfacing - secrets/store.rs: rename misleading `pub mod testing` to `pub mod in_memory` - server.rs: track installed extensions by (name, kind) tuple to avoid false positives across different extension kinds Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-21 05:43:28 +00:00
3 people	b68d67bd35	feat: show token usage and cost tracker in gateway status popover (#284) ... * feat: show token usage, cost tracker, and uptime in gateway status popover The "Connected" hover popover in the web gateway now displays three sections: connection info (SSE/WS counts, uptime), daily cost tracker (spend + actions/hr), and per-model token usage (input/output counts with cost per model). Also fixes the field name mismatch between the backend response and JS rendering that prevented the popover from showing correct data. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — escape HTML in popover, add model_usage test - Escape model name and cost strings with escapeHtml() before inserting into innerHTML to prevent XSS via crafted model names - Add test_model_usage_per_model_tracking test covering multi-model token/cost accumulation in CostGuard Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-21 03:45:04 +00:00
3 people	448383cfb0	refactor: remove Responses API, consolidate to Chat Completions (#272) ... * fix: strip reasoning from LLM responses and persist assistant messages reliably - Filter out `type: "reasoning"` output items from NEAR AI Responses API parsing so chain-of-thought never reaches the UI (nearai.rs) - Rewrite clean_response with regex-based tag stripping that is code-aware (preserves tags inside fenced blocks and inline backticks), supports 9+ tag names (think, thought, reasoning, reflection, etc.), handles <final> extraction, pipe-delimited tags, and case/whitespace tolerance (reasoning.rs) - Add Reasoning::complete() helper so all non-agentic LLM call sites (summarize, suggest, heartbeat, compaction) get automatic response cleaning; thread SafetyLayer through to those callers - Change persist_turn from fire-and-forget tokio::spawn to awaited async so both user and assistant messages are written before returning, preventing data loss on shutdown/restart - Pass input_count through seed_response_chain so response chaining delta calculation is accurate after thread hydration on restart - Make NearAiResponse.usage optional and preserve response_id in alt response path for chaining continuity - Persist session token to DB during onboarding wizard so runtime loads it without legacy-key fallback; suppress spurious warning on fresh installs - Fix dev tool double-registration when builder already registers them - Load dotenv/ironclaw env for doctor and status subcommands - Reduce startup log noise (demote info→debug for skills, remove redundant info lines) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Nudge to not loop over tools continuesly * refactor: remove Responses API, consolidate NEAR AI to Chat Completions only The Responses API provider (nearai.rs, 1278 lines) added significant complexity (response chaining state machine, delta message calculation, previous_response_id persistence) for marginal benefit. This consolidates to the Chat Completions API only, upgrading NearAiChatProvider with dual auth (session token + API key) and 401 retry for session token renewal. - Delete src/llm/nearai.rs (Responses API provider) - Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models - Remove response_id from CompletionResponse and ToolCompletionResponse - Remove seed_response_chain/get_response_chain_id from LlmProvider trait - Remove response chain persistence from agent (thread_ops, session) - Remove NearAiApiMode enum and NEARAI_API_MODE config - Clean up all wrapper providers (retry, circuit_breaker, failover, cache) - Update documentation (CLAUDE.md, .env.example) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: runtime log level control via gateway UI and URL parameter Add server-side log level switching using tracing_subscriber::reload::Layer so the EnvFilter can be swapped at runtime without restarting. Expose via GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs toolbar, and a ?log_level=debug URL parameter for one-click activation. Also applies cargo fmt to pre-existing files (llm/, tests/). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-20 20:43:32 +00:00
3 people	bac2d75713	feat: Secure prompt-based skills system (Phases 1-4) (#51) ... * feat: Add secure prompt-based skills system (Phase 1 MVP) Implement a skills system that extends the agent with prompt-level instructions from local directories. Skills declare activation criteria, tool permissions, and trust tiers that determine authority attenuation. Core security model: the minimum trust level of any active skill determines a tool ceiling -- tools above the ceiling are removed from the LLM's tool list entirely at the API level, preventing prompt-based manipulation. New modules: - skills/mod.rs: Core types (SkillTrust, SkillManifest, LoadedSkill) - skills/scanner.rs: Content scanner for manipulation detection - skills/registry.rs: Filesystem discovery and manifest parsing - skills/selector.rs: Deterministic two-phase prefilter (no LLM) - skills/attenuation.rs: Trust-based tool filtering Integration: - Agent loop selects skills per-turn and applies tool attenuation - Reasoning engine injects skill context with structural isolation - Config supports SKILLS_ENABLED, SKILLS_DIR, SKILLS_MAX_ACTIVE, SKILLS_MAX_CONTEXT_TOKENS environment variables - Disabled by default (SKILLS_ENABLED=false) 41 new tests covering all modules. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address all adversarial review findings for skills system Security fixes: - Escape skill name/version in XML attributes to prevent trust spoofing - Escape prompt content to prevent </skill> tag breakout - Require integrity hash for Verified/Community tier skills - Validate skill names against [a-zA-Z0-9][a-zA-Z0-9._-]{0,63} - Add 64 KiB file size limit on prompt.md Bug fixes: - Use actual SkillsConfig from AgentDeps instead of SkillsConfig::default() - Add skills_config field to AgentDeps, wired through from main.rs Performance: - Pre-compile regex patterns at load time (cached on LoadedSkill) - Selector uses pre-compiled patterns instead of recompiling per message - Switch all std::fs to tokio::fs for non-blocking async I/O Hardening: - Cap keyword score at 30 points to prevent keyword stuffing attacks - Enforce max 20 keywords and 5 patterns per skill - Normalize line endings (CRLF/CR to LF) before hashing - Also includes cargo fmt formatting fixes for adjacent code Tests: 54 skills tests pass (up from 41), zero new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address medium/low severity findings from adversarial review Fixes all 18 medium/low severity findings identified by the security review: - mod.rs: Add MAX_TAGS_PER_SKILL cap (10) in enforce_limits(); use RegexBuilder with 64 KiB size_limit to prevent ReDoS; replace case-enumerated escape_skill_content with regex matching all case variants plus whitespace/null byte injection between </ and skill; document allowed_patterns as unenforced until Phase 2; document Marketplace URL validation as Phase 3 concern - registry.rs: Add MAX_MANIFEST_FILE_SIZE (16 KiB) check before reading; add symlink detection via symlink_metadata to reject symlinks in discover_local; add MAX_DISCOVERED_SKILLS (100) cap; validate prompt_hash format (sha256: + 64 hex chars); warn on name collision before overwriting; accept SkillSource parameter in load_skill instead of always using Local; add InvalidHashFormat, ManifestTooLarge, SymlinkDetected error variants - selector.rs: Add MAX_TAG_SCORE (15) cap parallel to keyword cap; warn when declared max_context_tokens diverges >2x from actual prompt size - scanner.rs: Add mixed-script homoglyph detection (Cyrillic, Greek, Armenian unicode ranges); document token-boundary bypass and semantic paraphrasing as known limitations - attenuation.rs: Document READ_ONLY_TOOLS maintenance requirements - agent_loop.rs: Surface scan warnings via structured tracing; add structured audit events for skill activation and tool attenuation 61 tests pass, 0 new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address 12 findings from second adversarial security review HIGH: - Escape opening <skill tags in prompt content (prevents fake skill block injection) - Scan manifest metadata fields (description, author, tags, reasons) not just prompt - Block trust downgrade on name collision (existing Local can't be replaced by Community) MEDIUM: - Eliminate TOCTOU gap: read files then check size instead of metadata-then-read - Reject file-level symlinks in load_skill (prompt.md, skill.toml) - Truncate and filter manifest.skill.tags (prevent unlimited tag scoring) - Cap regex pattern score at 40 (prevent 5x20=100 dominating keyword+tag) - Add doc comment about skill_list tool exposing metadata (sanitization required) - Move Community disclaimer inside <skill> tags (not outside structural boundary) - Filter keywords/tags shorter than 3 chars (prevent broad matching) LOW: - Enforce minimum token_cost of 1 (max_context_tokens=0 can't bypass budget) - Remove redundant try_exists checks in discover_local (let load_skill handle errors) 70 skills tests passing. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add HTTP endpoint scoping for skills (Phase 1) Skills that declare an [http] section in skill.toml now have their HTTP requests constrained to declared endpoints at runtime. This addresses the gap where allowed_patterns was parsed but never enforced -- once the http tool was visible via attenuation, the LLM could reach any URL. Enforcement reuses EndpointPattern/AllowlistValidator from the WASM capability system. Semantics: if no active skill declares [http], all requests pass through (backward compat). If any skill declares [http], URLs must match at least one skill's allowlist (union). Community skills' [http] declarations are silently ignored (defense in depth). Shell commands using curl/wget are also validated against scopes. Scanner gains detection for known exfiltration domains (webhook.site, ngrok.io, etc.), overly broad wildcards, and credential/host mismatches. Closes #38 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt to http_scoping.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt across codebase Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add parameter-level permission enforcement for skills (Phase 2) Activates enforcement of `allowed_patterns` in skill.toml permissions. Previously these patterns were parsed but not enforced -- a Verified skill declaring `permissions.shell` with `allowed_patterns = [{command = "cargo *"}]` could still run any shell command. Now the enforcer validates tool parameters against declared glob patterns before execution. Key changes: - New `enforcer.rs` module with `SkillPermissionEnforcer`, `glob_to_regex()`, and `validate_tool_call()` with union semantics across active skills - Typed pattern enums (`ShellPattern`, `FilePathPattern`, `MemoryTargetPattern`) replace the previous `Vec<serde_json::Value>` in `ToolPermissionDeclaration` - Scanner gains `scan_permission_patterns()` detecting dangerous patterns (rm, sudo, curl, bare wildcards, command chaining, sensitive paths, identity files) - Registry blocks non-Local skills with critical permission pattern warnings - Agent loop threads enforcer into `execute_chat_tool` alongside HTTP scoping Trust interaction: Community patterns ignored, Verified enforced, Local without patterns unrestricted, Local with patterns enforced as guidance. Union semantics across skills -- tool call allowed if ANY skill's patterns permit it. 34 new tests. All 818 library tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add worker permission enforcement and LLM behavioral analysis (Phase 3+4) Phase 3 - Worker-side permission enforcement: - Add SerializedToolPermission/SerializedPattern DTOs for HTTP boundary crossing - Extend JobDescription, ContainerHandle, and orchestrator API to carry permissions - CreateJobTool snapshots and forwards skill permissions to spawned workers - Worker runtime builds SkillPermissionEnforcer and checks before tool execution - Load-time token budget enforcement rejects prompts exceeding 2x declared budget - Deduplicate enforcer construction: from_active_skills() delegates to from_serialized() Phase 4 - LLM behavioral analysis: - BehavioralAnalyzer with cached, LLM-based semantic content analysis - Structured output parsing (FINDING|CATEGORY|SEVERITY|DESCRIPTION or CLEAN) - Content-hash caching with bounded size (MAX_CACHE_ENTRIES=256) - Graceful degradation when LLM unavailable - Integrated into load_skill() for non-Local skills; critical findings block loading Review fixes: - Real cache tests with CountingLlm mock (test_cache_hit, test_cache_miss, test_cache_bounded) - UTF-8-safe truncate() in worker runtime - Few-shot examples in behavioral analysis prompt - Documented max_context_tokens=0 opt-out and create_job() permission gap 848 tests passing, no new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address review feedback from serrrfirat on skills-phase2 - Fix truncate_cmd UTF-8 panic: use char-boundary-aware slicing - Remove redundant effective_tools branching in reasoning.rs - Document cache eviction as known limitation (arbitrary, not LRU) - Add safety comment on SkillTrust enum ordering (security-critical) - Simplify active_skills selection (prefilter_skills handles empty input) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining skills review feedback * refactor: replace skills system with OpenClaw SKILL.md format + 2-state trust Replace the 5-gate, 3-tier trust hierarchy (scanner, behavioral analyzer, parameter-level enforcer, HTTP endpoint scoping) with a simplified 3-layer security model: gating -> attenuation -> Docker confinement. Key changes: - SKILL.md format (YAML frontmatter + markdown prompt) replaces skill.toml + prompt.md - 2-state trust (Installed/Trusted) replaces 3-tier (Community/Verified/Local) - New parser.rs for SKILL.md parsing with serde_yaml - New gating.rs for requirements checking (bins/env/config) - Simplified registry with 2-location discovery (workspace + user dirs) - Removed scanner, behavioral_analyzer, enforcer, http_scoping (~4,100 lines) - Removed skill_permissions propagation through job/orchestrator/worker pipeline - Added serde_yaml dependency for YAML frontmatter parsing Net: -5,298 lines, 59 skills tests pass, 907 total tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add in-app skill management tools and ClawHub catalog integration Add 4 chat-callable tools (skill_list, skill_search, skill_install, skill_remove) plus matching web gateway endpoints for managing skills at runtime. The catalog fetches from ClawHub's public registry API at runtime rather than bundling entries at compile time. Key changes: - SkillRegistry gains mutation methods (install_skill, remove_skill, reload, find_by_name) with Arc<RwLock> for concurrent access - New catalog module queries ClawHub /api/v1/search with in-memory caching (5-min TTL, configurable via CLAWHUB_REGISTRY env var) - skill_list and skill_search added to READ_ONLY_TOOLS for safe use under Installed trust ceiling - Web gateway gets /api/skills, /api/skills/search, /api/skills/install, and /api/skills/{name} DELETE endpoints Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #51 review feedback from ilblackdragon Security: - Add SSRF protection to fetch_skill_content: require HTTPS, reject private/loopback/link-local IPs and internal hostnames, disable redirects. Gateway install handler now reuses the same validation. - URL-encode slug in skill_download_url to prevent query injection. - Require X-Confirm-Action header on gateway skill install/remove endpoints (equivalent to chat tool requires_approval gate). Correctness: - Eliminate all block_in_place/block_on usage in skill tools and gateway handlers. Split install into prepare_install_to_disk (static async, no lock) + commit_install (sync, brief write lock). Same pattern for remove: validate_remove + delete_skill_files + commit_remove. - Write normalized content to disk in install_skill (was writing original un-normalized content, causing hash mismatch on re-read). - Fix token estimation from 0.75 to 0.25 tokens/byte (~4 chars per token) in registry.rs, selector.rs, and standalone loader. Dependencies: - Replace deprecated serde_yaml 0.9 with serde_yml 0.0.12. - Remove unused toml dependency. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-18 00:28:38 +00:00
4 people	bbb68f7490	Add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models) (#31) ... * feat: add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models) * - Reject model mismatches: validate req.model against the active model and return 404 model_not_found instead of silently ignoring it - Add x-ironclaw-streaming: simulated response header so clients know streaming is not true token-by-token delivery - Use SSE event type "error" for mid-stream LLM failures so clients can distinguish errors from content chunks - Mark docker-compose credentials as dev-only - Add integration tests for model mismatch, streaming header, and body size limit (axum's default 2MB) * fix: address Copilot review feedback on OpenAI-compat API - Wire chat_rate_limiter into /v1/chat/completions handler - Execute LLM before starting SSE stream so failures return proper HTTP errors instead of SSE error events - Validate tool-role messages require tool_call_id and name fields - Surface list_models() errors in models_handler via map_llm_error - Reject unknown roles with 400 instead of defaulting to User Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: firat.sertgoz <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-13 18:26:49 +04:00
3 people	33ef0a6ea5	fix: security hardening across all layers (#35) ... * fix: comprehensive security hardening across all layers Critical: - Replace --dangerously-skip-permissions with explicit tool allowlist via settings.json (Claude Code bridge) - Constant-time token comparison (subtle crate) in web auth and orchestrator auth to prevent timing attacks High: - Revoke tokens and clean up handles on container creation failure - Drop SETUID/SETGID capabilities from containers (keep only CHOWN) - Disable redirect following in HTTP tool and WASM wrapper (SSRF) - Reject URL userinfo (@) in WASM allowlist parser (host confusion) - Fix binary body bypassing leak detection (from_utf8 -> from_utf8_lossy) - Protect identity files from LLM overwrites (prompt injection defense) - Prevent tool shadowing: built-in tools cannot be replaced dynamically - User-scoped job APIs: list/detail/cancel/restart/prompt/events/files - CORS restricted to localhost origins, WebSocket origin validation - Sandbox shell fail-closed: no silent fallback to unsandboxed execution - Scrub secrets from log broadcaster before SSE broadcast - XSS sanitization on rendered markdown in web UI - WASM epoch ticker thread so timeout deadlines actually fire Medium: - Cap state transition history at 200 entries - SSE/WebSocket connection limit (100 max) - Request body size limit (1MB) - Response body size limit enforcement in WASM HTTP - UTF-8 safe string truncation (routine engine, shell tool) - Fix PolicyAction::Sanitize to actually run the sanitizer - TOCTOU fix in scheduler and context manager (hold write lock) - Project file serving moved behind auth - Path traversal guard on project_id - Session file permissions set to 0600 on unix - AtomicUsize for routine running_count (panic-safe) - Completion detection hardened against false positives and tool injection - Tool output no longer drives job completion (only LLM response) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address security review findings across all layers - Fix path traversal sandbox bypass via lexical normalization (file.rs) - Fix SSRF via DNS rebinding with pre-request hostname resolution (http.rs) - Add token budget enforcement on LLM calls (reasoning.rs, state.rs) - Fix cross-user chat history leak with ownership verification (store.rs, server.rs) - Add sliding-window rate limiter on gateway chat endpoint (server.rs) - Harden extension install: HTTPS-only, 50MB cap, WASM magic validation (manager.rs) - Add destructive command blocklist that overrides shell auto-approval (shell.rs) - Add 5MB response body size cap to HTTP tool (http.rs) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: deduplicate shared helpers and remove dead code Extract floor_char_boundary and llm_signals_completion into src/util.rs, unifying diverging phrase lists from agent/worker.rs and worker/runtime.rs. Remove dead RespondResult::usage(), duplicate PROTECTED_IDENTITY_FILES constant, double LeakDetector scanning in WebLogLayer, and invalid 0.0.0.0 origin from WebSocket allow list. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review findings and CI test failures - Fix record_failed_approve: .truncate(true) wiped the attempts file before reading, so failed pairing attempts never accumulated and rate limiting never triggered. - Guard wizard WASM test: skip gracefully when channel build artifacts are absent (CI doesn't compile wasm32-wasip2 targets). - Fix DNS rebinding check: use port 0 instead of hardcoded 443, since the port is irrelevant for hostname resolution. - Remove hardcoded CORS port 3001: the dynamic addr.port() entries already cover the actual server port. - Require WebSocket Origin header: reject connections that omit it entirely, since browsers always send Origin for WS upgrades and a missing header indicates a non-browser client bypassing the check. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR review findings - store.rs: reintroduce file locking around read-modify-write in record_failed_approve (concurrent callers could clobber each other). - sse.rs: replace load+check+fetch_add with atomic fetch_update in both subscribe_raw() and subscribe() to prevent overshooting max_connections. - ws.rs: decrement WS tracker before early return when subscribe_raw() returns None (connection limit reached), fixing a counter leak. - server.rs: parse WS Origin host exactly instead of prefix matching, preventing bypass via crafted origins like http://localhost.evil.com. - workspace_integration.rs: skip tests gracefully when Postgres is unreachable instead of panicking (fixes 10 CI failures). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add Origin header to WS integration tests The Origin header requirement added in a3b0190 broke the WS gateway integration tests. Test clients now send Origin: http://127.0.0.1:{port} to match the server's localhost validation. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-13 05:25:20 +00:00
3 people	ced83d5b4d	feat: Sandbox jobs (#4) ... * Orchestrating jobs and running them in sandboxes * Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback - Query /v1/models API for context_length and set max_tokens to half (floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7 need much larger budgets - Guard against empty LLM content (reasoning models can burn all tokens on chain-of-thought and return content: null) - Simplify notification routing: try configured channel first, fall back to broadcast_all so heartbeat alerts always reach someone - Add ModelMetadata struct and model_metadata() to LlmProvider trait - Refactor NearAiChatProvider::list_models into shared fetch_models() - Add standalone test_heartbeat example for isolated debugging Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add job detail view with drill-down from jobs list Click a job row to see full details across four sub-tabs: Overview (metadata grid, description, state transitions timeline), Actions (expandable tool call cards with input/output JSON), Thinking (conversation messages styled by role), and Files (embedded workspace tree browser). Co-Authored-By: Claude Opus 4.6 <[email protected]> * Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400 Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the content field instead of using the OpenAI tool_calls array. This XML leaks through to channels as text, and Telegram's Markdown parser chokes on the underscores, returning 400 "can't parse entities". Two fixes: - Generalize clean_response() to strip <tool_call>, <function_call>, <tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside the existing <thinking> tag stripping - Add Telegram send_message helper with parse_mode fallback: try Markdown first, retry as plain text on "can't parse entities" 400 errors Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add SystemCommand submission type for thread-state-independent commands System commands (/help, /model, /version, /tools, /ping, /debug) now bypass thread-state checks and safety validation via a dedicated Submission::SystemCommand variant. Previously these flowed through process_user_input() which blocked them during Processing/AwaitingApproval /Completed states. - Add /model [name] for runtime model switching with provider validation - Add active_model_name()/set_model() to LlmProvider trait with RwLock hot-swap in both NEAR AI providers - Rewrite /help with aligned columns grouped by category - Expand REPL tab-completion from 10 to 23 slash commands - Remove REPL-local /help interception (now handled by agent) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files The sandbox e2e pipeline (agent -> container -> built website -> browsable URL) was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need minutes, no auto-created project directory meant container output vanished, and no HTTP route to browse the built files. - Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler, worker/runtime) with the per-tool value - Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer) - Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified, so every sandbox job gets a persistent bind mount - Include `project_dir` and `browse_url` in sandbox tool output JSON - Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes to the web gateway with path traversal protection and MIME type detection - Add `mime_guess` dependency for content-type detection Co-Authored-By: Claude Opus 4.6 <[email protected]> * Apply cargo fmt to wizard.rs after merge Co-Authored-By: Claude Opus 4.6 <[email protected]> * Persist sandbox jobs in DB, fix web UI, unify job model Sandbox container jobs were invisible to the web UI because they lived only in ContainerJobManager's in-memory HashMap while the API queried ContextManager. This persists them to the agent_jobs table and fixes all six front-end bugs (empty job list, broken back button, empty actions/thinking tabs, wrong files tab, stuck status, no persistence). Key changes: - V4 migration adds project_dir and user_id columns to agent_jobs - Embedded migrations via refinery (no external CLI needed) - SandboxJobRecord CRUD in Store with fire-and-forget DB writes - Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager - Web API queries DB for sandbox jobs, merges with ContextManager direct jobs - New endpoints: restart, project file list/read with path traversal protection - Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in chat stream, source badges, restart button for failed/interrupted jobs - Gateway defaults to enabled, prints Web UI URL on startup - Stale jobs marked "interrupted" on restart for visibility and restartability Co-Authored-By: Claude Opus 4.6 <[email protected]> * Secure in-chat auth: tokens never touch the LLM or chat history Remove the token parameter from tool_auth so the LLM cannot pass raw API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket (auth_token) endpoints that route tokens directly to ext_mgr.auth(), completely bypassing the message pipeline, turns, history, and compaction. Web UI shows an auth card (password input + OAuth button) when the agent enters auth mode, submitted via the dedicated endpoint. CLI auth mode interception is unchanged (already secure). New StatusUpdate::AuthRequired/AuthCompleted variants propagate through all channels (SSE, WebSocket, REPL, WASM). Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add Claude Code mode for sandbox jobs Run Claude Code CLI inside Docker containers as an alternative to the standard worker mode. The bridge spawns `claude -p` with stream-json output, posts events to the orchestrator, and supports follow-up prompts via `--resume`. Key additions: - `claude-bridge` CLI subcommand and ClaudeBridgeRuntime - JobMode enum (Worker vs ClaudeCode) with per-mode container config - Orchestrator endpoints for Claude events and prompt polling - SSE event variants for real-time Claude Code streaming to frontend - Claude Code sub-tab in web UI with terminal-style output and input bar - Database migration for job_mode column and claude_code_events table - ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.) - Mode parameter on run_in_sandbox tool schema Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs When sandbox mode is on, the LLM would call create_job (creating a pending "direct" entry) then run_in_sandbox (creating a second "sandbox" entry), producing two jobs in the list for a single user request. Now register_job_tools() skips create_job when sandbox is enabled since run_in_sandbox already creates tracked jobs. Also improved the run_in_sandbox description to guide the LLM to use it directly and to mention wait=false for async execution. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Web gateway UI quality-of-life improvements Phase 1: Send button disabled state to prevent double-sends, copy button on code blocks, confirm() guards on destructive actions, SSE-driven job list auto-refresh, log filters re-applied on tab switch, jobEvents memory leak fix (cap at 500, cleanup after 60s). Phase 2: Toast notification system replacing chat-based system messages, memory search highlighting with centered snippets, keyboard shortcuts (Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur), activity tab toolbar with event type filter and auto-scroll toggle. Phase 3: Thread sidebar with load/switch/create, thread_id passed with messages, collapsible to hamburger. Memory inline editing with textarea, Save/Cancel, POST to /api/memory/write. Phase 4: Gateway status popover on hover (polls every 30s), extension install form (name/URL/kind), markdown rendering in memory viewer for .md files, mobile responsive layout at 768px breakpoint. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add routines system, remove non-sandbox job mode from web UI Routines: scheduled & reactive job system with cron and event triggers, lightweight (single LLM call) and full-job execution modes, guardrails (cooldown, max concurrent, dedup), and LLM-facing tools for CRUD. Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs are now exclusively sandbox-backed (DB + container). Simplify job detail response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo), fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab event rendering. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML Three fixes: 1. Chat input stays disabled after agent finishes: the "Done" status SSE event now calls enableChatInput() as a safety net when the response event is empty or lost. Same for auth_completed and cancelAuth(). 2. tool_activate never triggers auth: when activation fails due to missing authentication, it now auto-initiates the auth flow (same pattern as the web API handler). detect_auth_awaiting() also matches tool_activate results now. 3. Models like GLM-4.7 emit tool calls as XML tags in content (<tool_call>tool_list</tool_call>) instead of using the structured tool_calls array. recover_tool_calls_from_content() extracts and validates these before falling back to plain text. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add routines web UI tab, update docs for sandbox-jobs branch Add full routines management to the web gateway (list, detail, trigger, toggle, delete) with 7 new API endpoints, response types, and frontend (HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new subsystems, config, TODOs), and README.md (architecture diagram, features, components, fix onboard command). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Bind Telegram bot to owner account during setup Without owner binding, anyone who discovers the bot can send it messages. The setup wizard now prompts the user to message their bot, captures their Telegram user ID via getUpdates, and persists it as telegram_owner_id in settings. On startup, the owner_id is injected into the WASM channel config so the existing owner restriction logic drops messages from non-owners. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Move settings from disk to PostgreSQL database Settings previously lived in three JSON files on disk (settings.json, mcp-servers.json, session.json). This made them inaccessible from the web UI and caused redundant disk reads (Settings::load() called 8+ times during startup). Now all settings live in a `settings` table (user_id + key -> JSONB) with only 4 bootstrap fields remaining on disk (database_url, pool size, secrets key source, onboard_completed) since they're needed before the DB connection exists. - Add V8 migration for settings table - Add BootstrapConfig (thin disk file) and Settings DB round-trip - Add Store CRUD methods for settings (get/set/delete/list/bulk) - Refactor Config to load from DB (env > DB > default cascade) - Add SessionManager DB persistence for session tokens - Add DB-backed MCP server config load/save functions - Add 6 settings web API endpoints (list/get/set/delete/export/import) - Add one-time disk-to-DB migration on first boot - Make CLI config commands async with DB access (disk fallback) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth - Add Workspace::seed_if_empty() to create core identity files (README, MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called on every boot without overwriting existing user edits - Remove duplicate gateway log lines from web/mod.rs (main.rs has the useful clickable ?token= URL) - Auto-authenticate from ?token= URL parameter in the web UI and strip the token from the address bar after successful auth Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Harden sandbox security (path traversal + orchestrator auth) Two vulnerabilities fixed: 1. project_dir path traversal: The create_job tool let the LLM specify arbitrary host paths for Docker bind mounts. Removed project_dir from the tool schema entirely, and added canonicalization + prefix validation at both resolve_project_dir() and the job_manager bind mount point. 2. Orchestrator API auth bypass: worker_auth_middleware was defined but never applied. Each handler manually called validate_token(), so any new endpoint that forgot would be publicly accessible. Applied the middleware as route_layer on all /worker/ routes, removed manual auth from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps 0.0.0.0 since containers reach host via docker bridge, not loopback). Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining Implements the 4-phase plan for overhauling the web gateway chat: - Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below - Phase 2: Cursor-based history pagination with infinite scroll - Phase 3: NEAR AI previous_response_id chaining (delta-only messages), with fallback to full history on chain errors, and DB persistence of chain state across restarts - Phase 4: SSE thread isolation (events filtered by thread_id) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Add per-request HTTP timeout to WASM host, redact credentials in errors Three fixes for WASM channel reliability: 1. Per-request timeout: Add optional timeout-ms parameter to http-request in both channel and tool WIT interfaces. Telegram long-poll now specifies 35s (outliving the 30s server-side hold), while regular API calls use the 30s default. Fixes the triple-30s timeout race that caused polling failures. 2. Credential redaction: reqwest::Error includes the full URL (with injected bot tokens) in its Display output. Scrub credential values from error messages before logging or returning to WASM. 3. Webhook route registration: Remove tunnel URL gate so webhook routes are always available when webhook channels exist, not only when TUNNEL_URL is configured. Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: Fix clippy warnings in WASM tools and channels - slack channel: allow dead_code on signing_secret_name (forward compat field) - gmail tool: use div_ceil() instead of manual (n+2)/3 - google-calendar tool: extract CreateEventParams/UpdateEventParams structs to fix too-many-arguments warnings Co-Authored-By: Claude Opus 4.6 <[email protected]> * Fix approval flow * fix: Rebuild bundled telegram.wasm with updated WIT interface The bundled WASM binary must match the host's WIT definition. Previous binary was compiled against the old 4-arg http-request; this rebuild includes the new timeout-ms parameter. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Load WASM channels from disk instead of bundling in binary Remove include_bytes! embedding of telegram.wasm. Channels are now loaded from their build output directories (channels-src/<name>/target/) during onboarding, then from ~/.ironclaw/channels/ at runtime. - bundled.rs: locate_channel_artifacts() finds WASM + capabilities from build output; IRONCLAW_CHANNELS_SRC env var overrides the default path - available_channel_names(): only lists channels with build artifacts - bundled_channel_names(): lists all known channels (manifest) - Setup wizard uses available_channel_names() to offer installable channels - Add *.wasm to .gitignore, remove tracked telegram.wasm Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Persist gateway auth token, fix thread hydration race, polish auth screen Three web gateway UX fixes: 1. Token persistence: Store auth token in sessionStorage so refreshing the page doesn't force re-authentication. Hide the auth screen immediately when a saved token exists to prevent flash. 2. Thread hydration: Remove the !msgs.is_empty() bail-out in maybe_hydrate_thread so that even brand-new (empty) assistant threads get hydrated with their correct DB UUID. Previously resolve_thread would mint a fresh UUID, causing messages to land in the wrong conversation and duplicate threads to appear. 3. Auth screen: Redesign as a centered card with brand, tagline, labeled input, and hint text. Also adds 34 new tests covering session/thread lifecycle, thread resolution isolation (user, channel, external ID), hydration edge cases, serialization round-trips, approval flows, and stale mapping recovery. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Use bindgen! for WASM tool wrapper, add dev tool loading Three changes: 1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen! instead of manual linker.root().func_wrap(). This fixes the "component imports instance 'near:agent/host', but a matching implementation was not found in the linker" error. All 6 host functions (log, now-millis, workspace-read, http-request, secret-exists, tool-invoke) are now properly registered under the near:agent/host namespace. Also adds WASI support, credential injection, and leak detection for HTTP requests made by WASM tools. 2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the loader now also scans tools-src/*/target/wasm32-wasip2/release/ for build artifacts that are newer than installed copies. This means during development you just rebuild the WASM and restart the host; no manual copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir. 3. Wire up load_dev_tools() in main.rs alongside the existing load_from_dir() call. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Wire main startup and CLI to use DB-backed settings main.rs now reloads Config from the database after connecting, attaches the store to the session manager for dual-write tokens, and loads MCP servers from DB instead of disk. ExtensionManager and MCP CLI commands use DB when available with disk fallback. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-11 08:31:25 +00:00
3 people	642c320b13	Add WebSocket gateway and control plane (#8) ... * Add WebSocket gateway and control plane endpoint Adds bidirectional WebSocket transport to the web gateway alongside the existing SSE stream. Clients can send messages, approvals, and pings over a single persistent connection at /api/chat/ws. - Enable axum `ws` feature for built-in WebSocket support - Add WsClientMessage/WsServerMessage types with tagged JSON protocol - Add subscribe_raw() to SseManager for non-SSE consumers - Create ws.rs with connection handler (split sender/receiver tasks) - Add WsConnectionTracker for active connection counting - Add /api/gateway/status control plane endpoint (SSE + WS counts) - 35 new tests covering message types, broadcast, and handler logic https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b * Add e2e WebSocket gateway integration tests - Add tokio-tungstenite dev-dependency for WebSocket client in tests - Update start_server to return actual bound SocketAddr (enables port 0) - Add 10 e2e tests covering full HTTP upgrade → WebSocket → message flow: ping/pong, message routing to agent, broadcast event delivery, connection tracking, invalid message handling, auth rejection, gateway status endpoint, and multi-event sequencing https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b --------- Co-authored-by: Claude <[email protected]>	2026-02-09 01:00:35 +00:00