mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 23:50:17 +00:00
633b234e44055fb2d46476752dff4121880c57b2
61
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
633b234e44 |
docs: add comprehensive subdirectory CLAUDE.md files and update root (#589)
* docs: add comprehensive subdirectory CLAUDE.md files and update root The repo has grown significantly. This adds module-level CLAUDE.md files for the five most complex subsystems, and updates the root CLAUDE.md to reflect the actual current state of the codebase. New files: - src/agent/CLAUDE.md — full module map (19 files), session/thread/turn model, agentic loop flow, compaction strategies with correct thresholds, scheduler invariants, self-repair details, complete submission command reference table - src/channels/web/CLAUDE.md — complete API route table (50+ endpoints), SSE event type reference, auth/rate limiting gotchas, connection limits, CORS headers, step-by-step endpoint addition guide - src/db/CLAUDE.md — dual-backend build commands, sub-trait structure (7 sub-traits, ~67 methods), SQL dialect differences, boolean/timestamp gotchas, complete schema table, in-memory test helper, shared handle pattern - src/llm/CLAUDE.md — corrected LlmProvider trait signatures, provider chain decorator order, NEAR AI dual-auth and session renewal details, circuit breaker thresholds, previously undocumented smart_routing.rs and recording.rs - tests/e2e/CLAUDE.md — conftest fixtures and async scoping, environment injected into the binary, mock_llm canned responses, writing guide with correct asyncio usage, gotchas section Root CLAUDE.md updates: - Added E2E test setup and integration test commands - Documented ~15 undocumented modules: cli/, registry/, hooks/, tunnel/, observability/, webhook_server.rs, cost_guard.rs, job_monitor.rs, etc. - Corrected libSQL backend path (libsql/ directory, 8 sub-modules) - Updated Database trait method count (~67, split across 7 sub-traits) - Fixed stale references: config.rs → config/channels.rs, main.rs → app.rs - Added Hook, Observer, Tunnel traits to extensibility section - Added tunnel and observability env vars to Configuration section - Removed resolved TODO (webhook trigger is now shipped) - Added Module Specifications entries for all 5 new CLAUDE.md files Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * docs: address PR review comments and reduce CLAUDE.md size - Fix 7-sub-trait count (was 6) and ~78 async methods (was ~60/~67) in both CLAUDE.md and src/db/CLAUDE.md - Add missing types.rs to secrets/ file tree (CLAUDE.md) - Add missing tls.rs to src/db/CLAUDE.md Files table - Fix method counts: ConversationStore 12, JobStore 13, RoutineStore 15 - Add Windows venv activation note to E2E setup commands - Collapse agent/, web/, llm/, db/ file trees to one-liners (detail lives in their respective CLAUDE.md files) - Replace verbose Database and LLM Providers sections with summaries linking to src/db/CLAUDE.md and src/llm/CLAUDE.md - Root CLAUDE.md: 43,868 → 35,270 chars (fixes >40k perf warning) [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
|
45ec691f4c |
Improve test infrastructure: StubChannel, gateway helpers, security tests, search edge cases (#623)
* feat(testing): add StubChannel test double for Channel trait Adds StubChannel to src/testing.rs alongside StubLlm. Supports message injection via mpsc sender, response/status capture, and configurable health check toggling. Includes handle methods for use after ownership transfer to ChannelManager. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(testing): wire StubChannel into TestHarnessBuilder Add with_stub_channel() builder method that creates a StubChannel pre-registered in a ChannelManager. Tests can inject messages via the sender and verify routing through the manager. The channel field on TestHarness is Optional, defaulting to None for backward compat. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: gate external-service tests behind integration feature flag Replace silent try_connect() skip pattern with explicit feature gating. cargo test now runs only self-contained tests. cargo test --features integration runs tests requiring PostgreSQL. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(channels): add ChannelManager unit tests using StubChannel Cover add/start_all stream merging, respond routing, unknown channel errors, health_check_all with mixed health, empty-channels error path, and injection channel merging -- all via StubChannel test double. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: document test tier separation (unit/integration/live) Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add architecture boundary check script Grep-based checks for three architecture boundaries: - Direct database driver usage (tokio_postgres/libsql) outside src/db/ - .unwrap()/.expect() in production code (warning only) - Direct std::env::var reads outside config layer (warning only) The DB driver check is a hard violation; the other two are warnings for gradual cleanup. Run with: bash scripts/check-boundaries.sh Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(search): add RRF edge case tests for empty inputs, limits, and config modes Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(security): add regression tests for skill installer ZIP and SSRF protections Add 11 regression tests covering the security controls in skill_tools: ZIP extraction safety: - Valid SKILL.md extraction works correctly - Non-SKILL.md entries are ignored (returns error) - Path traversal entries (../../SKILL.md) do not match - Nested path entries (subdir/SKILL.md) do not match - Oversized entries (>1MB uncompressed) are rejected SSRF prevention: - Loopback addresses (127.0.0.1) are blocked - Private ranges (10.x, 172.16.x, 192.168.x) are blocked - Link-local addresses (169.254.x) are blocked - Public IPs (8.8.8.8, 1.1.1.1) are allowed - IPv4-mapped IPv6 unwrapping logic works correctly - Metadata endpoints and .internal/.local hostnames are blocked - Normal hostnames (github.com, clawhub.dev) are allowed Also documents a known gap: url::Url::host_str() returns bracketed IPv6 addresses that std::net::IpAddr cannot parse, so IPv4-mapped IPv6 URLs currently bypass IP-based checks in validate_fetch_url. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(testing): extract TestGatewayBuilder to eliminate gateway test duplication Both ws_gateway_integration.rs and openai_compat_integration.rs manually constructed GatewayState with 19+ fields. Extracted to a shared builder in src/channels/web/test_helpers.rs that provides sensible defaults and lets tests override only what they need. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: add implementation plans for testing batches 1 and 2 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): close IPv6 SSRF bypass in validate_fetch_url validate_fetch_url used host_str() which returns bracketed IPv6 (e.g. "[::ffff:7f00:1]") that IpAddr::parse() cannot handle, silently skipping IP-based SSRF checks for all IPv6 URLs. Switch to url::Host enum matching to extract proper IpAddr values without string parsing. IPv4-mapped IPv6 addresses like ::ffff:127.0.0.1 are now correctly unwrapped and blocked. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(skills): add activation criteria limits enforcement tests Adds test_activation_criteria_enforce_limits to verify that enforce_limits() correctly trims excess patterns (>5), keywords (>20), and tags (>10), and filters out short keywords/tags (<3 chars). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(wasm): add security regression tests for WASM tool loader Add 6 tests covering: tool name path separator rejection, empty name rejection, nonexistent file handling, invalid WASM bytes rejection, dotfile discovery behavior, and subdirectory non-recursion. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: address PR review feedback - Remove plan files from repo (ilblackdragon review) - Replace CLAUDE.md test tier rules with pointer to check-boundaries.sh - Add Check 4 to check-boundaries.sh: enforces integration tests are gated behind the 'integration' feature flag Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add try_connect silent-skip pattern check to check-boundaries.sh Check 5 catches try_connect() and similar silent-skip patterns in integration tests. Tests should use feature gates to fail loudly when prerequisites are missing, not silently return. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): harden skill fetch SSRF checks * fix(scripts): use bash arrays in check-boundaries.sh tier violation check Refactor Check 4 in check-boundaries.sh to use bash arrays and printf instead of string concatenation with echo -e. This is more robust with special characters in filenames and avoids portability concerns with echo -e. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
469a252051 |
feat(gateway): show IronClaw version in status popover [skip-regression-check] (#636)
Add version field to gateway status API response (from Cargo.toml via
env!("CARGO_PKG_VERSION")) and display it at the top of the hover
popover on the "Connected" indicator.
Co-authored-by: Claude Opus 4.6 <[email protected]>
|
||
|
|
b425213c53 |
feat(e2e): extensions tab tests, CI parallelization, and 3 production bug fixes (#584)
* feat(e2e): extensions tab tests, CI parallelization, and 3 bug fixes ## E2E test coverage - Add tests/e2e/scenarios/test_extensions.py with 57 tests covering all extensions tab flows: installed WASM tool/MCP/channel cards, configure modal (open, fields, cancel, save, OAuth, error), auth card (token, OAuth, submit, cancel, error, multi-extension coexistence), activate flow, install/remove flows, WASM channel stepper states, and tab reload behaviour. All network calls intercepted via page.route() — no real binaries or external registries needed. - Expand tests/e2e/helpers.py with 50+ new CSS selectors for the extensions tab UI. - Add tests/e2e/README.md documentation on the page.route() mocking pattern, LIFO handler ordering, and page.evaluate() injection. ## CI parallelization - Split .github/workflows/e2e.yml into a build job (compile once, upload artifact) and a 3-way parallel test matrix (core / features / extensions), matching the pattern in test.yml. Reduces wall-clock time from ~15–20 min serial to ~10–12 min. Adds an e2e roll-up job for branch protection. ## Bug fixes in app.js (found via test-driven code review) - Fix null crash: renderExtensionCard() called ext.tools.length without a null guard; add ext.tools && check (regression: test_ext_tools_null). - Fix modal UX: submitConfigureModal() closed the overlay before checking success, making failures unrecoverable without reopening; close only on success, re-enable buttons and keep modal open on failure (regression: test_configure_modal_stays_open_on_save_failure). - Fix URL injection: all window.open() calls for server-supplied auth_url now go through openOAuthUrl() which rejects non-HTTPS schemes (regression: test_oauth_url_injection_blocked). Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * refactor(e2e): prune extensions tests 57→46 by merging redundant setups Merge 11 tests that shared identical fixture+navigation overhead: - Group A: 3 empty-state tests → test_extensions_empty_tab_layout - Group B: card_renders absorbs ext_tools_list_shown (same _WASM_TOOL fixture) - Group B: auth_dot_unauthed + unauthed_shows_configure_btn → test_installed_wasm_tool_unauthed_state - Group D: installed + configured states → test_wasm_channel_setup_states (identical UI) - Group D: failed_state + stepper_failed_circle → test_wasm_channel_failed_renders - Group G: 5 field badge tests → test_configure_modal_field_variants (4 fields, one pass) - Group H: submit_success + enter_key_submits → test_auth_card_submit_success Coverage preserved: all assertions kept, no unique behaviors removed. Extensions CI job estimated to drop from ~7 min to ~5 min. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(e2e): fix configure_input selector scoping in merged field variants test modal.locator(".configure-modal input[type='password']") scoped the absolute selector inside .configure-modal, effectively searching for a nested .configure-modal which never exists → count() == 0. Use page.locator() instead, consistent with all other tests in the file. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(e2e): address PR review comments — replace fixed sleeps with deterministic waits - Remove unnecessary wait_for_timeout(1000) from test_remove_cancelled_keeps_card (window.confirm = () => false is synchronous; DOM is unchanged when click() returns) - Replace wait_for_timeout(800) with wait_for_function() for window._lastOpenedUrl checks in configure_modal_save_oauth and activate_with_auth_url_opens_popup - Replace wait_for_timeout(300) with nth(1).wait_for(visible) in test_auth_card_multiple_extensions_coexist - Remove wait_for_timeout(800/300) in test_auth_card_submit_empty_noop and test_auth_completed_sse_dismisses_card (both check synchronous JS side-effects) - Add comment in test_oauth_url_injection_blocked explaining why timeout is kept (negative assertion — cannot use wait_for_function for absence of event) Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(e2e): address remaining PR review comments - Remove unused `import pytest` from test_extensions.py - Fix unawaited coroutine bug: convert lambda route handlers to async def in test_extensions_tab_reloads_on_revisit and test_auth_completed_sse_triggers_extensions_reload (lambda r: r.fulfill(...) returns an unawaited coroutine; requests silently fell through to real server) - Fix README.md example to use async def handler (same bug in docs) - Harden openOAuthUrl() in app.js: use URL constructor instead of .startsWith() so non-string server-supplied values (objects, null, etc.) are safely rejected rather than throwing TypeError Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(e2e): address second round of PR review comments - Add timeout-minutes to CI build job to prevent hung workflows - Use parsed.href instead of raw url in openOAuthUrl for safety - Remove unused MessageEvent variable in auth_completed test - Replace wait_for_timeout(800) with expect_response in activate test - Replace wait_for_timeout(300) with tab panel wait_for in reload test [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
|
6a2a6cd050 |
fix(security): use OsRng for all security-critical key and token generation (#519)
* fix(security): use OsRng for all security-critical key and token generation Replace rand::thread_rng() with rand::rngs::OsRng in all security-critical code paths that generate cryptographic key material, bearer tokens, PKCE verifiers, CSRF state parameters, and webhook secrets. thread_rng() uses a userspace CSPRNG (ChaCha) seeded from OS entropy, which is fine for non-security contexts but adds an unnecessary intermediate layer for key material where direct OS entropy (OsRng) is the correct choice. Files changed: - src/secrets/keychain.rs: master encryption key generation - src/secrets/crypto.rs: per-secret HKDF salt generation - src/orchestrator/auth.rs: per-job bearer token generation - src/channels/web/mod.rs: gateway auth token fallback - src/cli/oauth_defaults.rs: OAuth PKCE verifier and CSRF state - src/tools/mcp/auth.rs: MCP OAuth PKCE verifier - src/extensions/manager.rs: auto-generated extension secrets - src/setup/channels.rs: webhook secret generation Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(security): address PR review feedback for OsRng migration - Remove shadowing inner `use rand::rngs::OsRng` in `generate_salt()`; use module-level `aes_gcm::aead::OsRng` import instead (same type, avoids divergence risk if rand_core versions drift) - Fix missed callsites in `pairing/store.rs`: `random_code()` and `generate_unique_code()` now use `OsRng` for pairing auth codes - Add regression tests for `generate_salt()`: correct length, non-zero output, uniqueness across calls Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
Nick PismenkovandGitHub
|
9ae04f14e3 |
feat: restart (#531)
* feat: restart * review fixes * add IRONCLAW_IN_DOCKER env variable * review fixes * fix tests * set default value as false |
||
|
|
704d63f16a |
feat(oauth): route callbacks through web gateway for hosted instances (#555)
* feat: route OAuth callbacks through web gateway for hosted instances On hosted instances (e.g., NEAR AI), OAuth callbacks can't reach the local TCP listener on port 9876. This adds a gateway-routed OAuth flow that works behind reverse proxies and load balancers. Backend changes: - Add /oauth/callback as a public route on the web gateway - PendingOAuthFlow registry shared between ExtensionManager and handler - Gateway mode auto-detected via IRONCLAW_OAUTH_CALLBACK_URL env var - Platform state format (instance:nonce) for nginx routing - Token exchange proxy support via IRONCLAW_OAUTH_EXCHANGE_URL - Local TCP listener mode preserved as backward-compatible fallback UX improvements: - Hide Configure button for tools with auto-resolved OAuth credentials (builtin defaults or platform-injected env vars) - Skip client_id/client_secret fields in setup schema when auto-resolved - Show Reconfigure only after successful authentication Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(oauth): harden gateway callback and refactor AuthResult - Add 60s timeout to exchange_via_proxy HTTP client (matching exchange_oauth_code) - Read GATEWAY_AUTH_TOKEN once at ExtensionManager construction instead of per-flow from env (prevents coupling and clarifies token provenance) - Extract oauth_error_page() helper to deduplicate error landing pages - Remove IRONCLAW_FORCE_GATEWAY_CALLBACK env var (auto-detection suffices) - Refactor AuthResult into typed AuthStatus enum with constructors, eliminating stringly-typed status and Option fields that were always None - Adapt all handlers (chat, extensions, ws) to new AuthResult/AuthStatus API - Use setup_url (not validation_endpoint) for awaiting_token responses [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(oauth): address review feedback — empty token guard, test flakiness, doc typos - Fail early in exchange_via_proxy() when gateway_token is empty instead of sending an unauthenticated request to the exchange proxy - Fix test_oauth_callback_strips_instance_prefix to use an expired flow so it never attempts a real HTTP token exchange (prevents CI flakiness) - Fix doc comments: /auth/callback → /oauth/callback in PendingOAuthFlow and ExtensionManager pending_oauth_flows docs [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: clarify strip_instance_prefix safety, wrapper credential fix, test assertion - Add comment to strip_instance_prefix noting nonces are base64url (no colons) - Expand wrapper.rs comment explaining the credential_user_id bug fix - Fix test_oauth_callback_strips_instance_prefix assertion: landing_html does not include provider_name on error pages [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
902492bcdb |
feat(web): show error details for failed tool calls (#490)
* feat(web): show error details and input params for failed tool calls Failed tool calls in the gateway UI previously showed only a red X icon with an empty expandable body. This change: - Adds optional `error` and `parameters` fields to `ToolCompleted` SSE events so the browser receives failure details in real-time - Auto-expands failed tool cards to make errors immediately visible - Adds `StatusUpdate::tool_completed()` constructor that centralizes the 5 duplicated construction sites and applies `redact_params()` to prevent sensitive values (e.g. secret_save's "value" param) from leaking through SSE broadcasts - Adds `sensitive_params()` trait method to `Tool` for declaring which parameters must be redacted before logging, hooks, and UI display - Adds `redact_params()` utility and wires it through hooks, approvals, ActionRecord storage, and debug logs in dispatcher/worker - Adds `SecretListTool` and `SecretDeleteTool` for LLM-driven secret management (values never returned, only names/metadata) - Fixes auth flow: setup-only extensions show configure modal instead of OAuth card; auth_completed SSE dismisses both UI paths - CI: release workflow creates PR instead of pushing directly to main - Registry: MissingChecksum error enables source fallback for bootstrapping when checksums haven't been populated yet Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: apply cargo fmt Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: keep original params in PendingApproval for execution, redact only for display Address two PR review comments: 1. execute_chat_tool_standalone now redacts sensitive params before logging, matching the pattern already used in worker.rs. 2. PendingApproval previously stored redacted parameters, which meant approved tool calls received "[REDACTED]" instead of the actual values. Add a display_parameters field for UI/logs and keep parameters as the original values used for execution. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments - worker.rs: redact sensitive params before BeforeToolCall hook, matching dispatcher.rs — hooks in the autonomous job path now receive redacted params instead of raw values - registry.rs: fix docstring for register_secrets_tools (list, delete, not save/list/delete — no SecretSaveTool is registered) - app.js: fix double toast/loadExtensions in submitConfigureModal — for non-OAuth success the auth_completed SSE already handles both, so skip them in the HTTP response handler to avoid duplicates [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
13697976db |
feat(extensions): improve auth UX and add load-time validation (#536)
* feat(extensions): add load-time validation for auth capabilities Catch common misconfigurations (missing auth section, missing setup_url, short prompts) at startup via tracing::warn instead of silently failing at auth time. * feat(extensions): improve auth prompts, setup_url, and showAuthCard Add setup_url and descriptive prompts to channel and tool capabilities files. Fix showAuthCard in web gateway and improve extension manager auth flow messaging. * refactor(extensions): extract MIN_PROMPT_LENGTH constant in validate() Address review feedback: replace magic number 30 with a named constant for readability and maintainability. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
cbcd5adcc0 |
fix(security): restrict query-token auth to SSE endpoints only (#528)
* fix(security): restrict query-token auth to SSE endpoints only Query-string `?token=xxx` auth was accepted on all endpoints, exposing the main auth token in server logs, Referer headers, and browser history for state-changing routes. Now only GET /api/chat/events and GET /api/logs/events accept query tokens; all other endpoints require the Authorization header. Supersedes #364. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add WebSocket endpoint to query-token allowlist, add URL-encoding tests The WS upgrade at /api/chat/ws also can't set custom headers, so it needs query-token auth like the SSE endpoints. Also adds tests for URL-encoded token values to cover the form_urlencoded parser. Addresses review feedback from Gemini (partially, /api/jobs/{id}/events is a JSON endpoint not SSE, so it correctly stays excluded) and Copilot (URL-encoded token test). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
e4e78d8a87 |
fix(web): reset job list UI on restart failure (#499)
* fix(web): reset job list UI on restart failure The restartJob() catch handler was missing a loadJobs() call, so the job row stayed in a stale highlighted state after a failed restart attempt. Add loadJobs() to match the success path behavior. Closes #485 Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: use .finally() for loadJobs() instead of duplicating Move loadJobs() to a .finally() block so it runs on both success and failure without duplication. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
a181c8b384 |
fix(web): mobile browser bar obscures chat input (#508)
* fix(web): use dvh units to prevent mobile browser bar from obscuring chat input On mobile browsers (Brave/Android, Safari/iOS), the bottom navigation bar covers the chat input because 100vh includes space behind browser chrome. Switch to 100dvh (dynamic viewport height) with vh fallback for older browsers, and add safe-area-inset padding for notched devices. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Fix padding declaration in chat input style --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
35a79caf87 |
fix(web): assign unique thread_id to manual routine triggers (#500)
* fix(web): assign unique thread_id to manual routine triggers
Manual routine triggers via the web API created an IncomingMessage
without a thread_id, causing session_manager.resolve_thread() to
route the output to whatever thread was last associated with the
(user, "gateway", None) key. This sets a unique thread_id of the
form "routine-{id}-{timestamp}" so each manual trigger gets its own
dedicated thread.
Closes #484
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: add ownership check to routine trigger handler (IDOR)
Address review feedback: verify routine.user_id matches the
authenticated user before allowing the trigger, preventing
unauthorized cross-user routine execution.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
|
||
|
|
85999b25a8 |
fix(web): refresh routine UI after Run Now trigger (#501)
* fix(web): refresh routine UI after "Run Now" trigger triggerRoutine() only showed a toast but did not refresh the routine data after triggering. This adds openRoutineDetail() / loadRoutines() calls after the toast, matching the pattern used by toggleRoutine(). Closes #483 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: only refresh detail view if triggered routine matches current view Check currentRoutineId === id before refreshing the detail panel to avoid refreshing the wrong routine's view. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
b60e5e907a |
fix(skills): use slug for skill download URL from ClawHub (#502)
* fix(web): use slug for skill download URL from ClawHub The skill install handler was using req.name (display name like "Markdown Converter") instead of the slug (like "owner/markdown-converter") when constructing the download URL. The registry endpoint expects a slug, so display names caused 502 errors. - Add optional `slug` field to SkillInstallRequest - Prefer slug over name when building the download URL - JS installSkill() now sends slug from search results Closes #482 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: guard against empty slug string in skill download URL Filter out empty slug strings so we fall back to name instead of constructing an invalid download URL. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
d562dc8d90 |
fix(workspace): thread document path through search results (#503)
* fix(workspace): thread document path through search results Memory search results were showing chunk UUIDs instead of source file paths. Thread document_path through RankedResult, SearchResult, and the RRF fusion pipeline so handlers can display the actual file path. Fixes #481 Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: use into_iter to move values instead of cloning Address review feedback: consume results with into_iter() to move String fields directly instead of cloning them. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
18b59ae9a7 |
feat: add OAuth support for WASM tools in web gateway (#489)
* feat: add OAuth support for WASM tools in web gateway Extract reusable OAuth functions (build_oauth_url, exchange_oauth_code, store_oauth_tokens, validate_oauth_token) from CLI into shared oauth_defaults module, then wire them into the web gateway's ExtensionManager. Key changes: - Install auto-activates WASM tools (no separate Activate button) - Configure button triggers OAuth flow via save_setup_secrets - Scope merging: installing a second Google tool triggers re-auth with merged scopes from all tools sharing the same secret_name - Cancel-and-retry: aborting stale OAuth listeners prevents port conflicts - Post-auth validation: wrong account detected via validation_endpoint - Reconfigure always re-auths (deletes old token before starting fresh) - UI shows error toast on OAuth failure, refreshes extension list Flow: Install → Active → Configure (enter client_id/secret) → Save → OAuth popup → authorize → done. Second Google tool install auto-triggers scope expansion OAuth. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments - Add custom headers support to ValidationEndpointSchema (fixes missing Notion-Version header regression) - Guard activate handler auth check with status == "awaiting_authorization" to prevent unexpected OAuth popups - Add window dimensions to OAuth popup in activateExtension() - Simplify UTF-8 truncation boundary check Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address Copilot PR review comments (security, UX, bugs) - Add CSRF state parameter to OAuth flow (random state in auth URL, validated in callback) - Restore MCP server Activate button in web UI (was hidden for all non-channel extensions) - Abort JoinHandle in cleanup_expired_auths to prevent port 9876 conflicts - Fix Google-specific error message for non-Google OAuth providers - Add has_auth field to ExtensionInfo API response (fixes Configure button visibility) - Use oauth_defaults::callback_url() instead of hardcoded redirect_uri (both CLI and manager) - Update auth check comment to match actual behavior (scope expansion + first-time auth) - Add unit tests for build_oauth_url (basic, PKCE, extra params, state uniqueness) - Check all required setup secrets (client_id + client_secret) before starting OAuth Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
f18fb5173b |
feat(web): fix jobs UI parity for non-sandbox mode (#491)
* feat(web): fix jobs UI parity for non-sandbox mode The web gateway Jobs UI was built primarily for sandbox (Docker) jobs. When running without sandbox (common for NEAR AI hosted envs), multiple features were broken. This change fixes all of them: - Agent jobs now broadcast live SSE events to the web UI (Activity tab) - Agent job restart via scheduler.dispatch_job (not chat message) - Follow-up prompts for agent jobs via WorkerMessage injection - Capability flags (can_restart, can_prompt, job_kind) in job detail API - Rate-limit retry with cap (10 consecutive) and Retry-After header parsing - Plan interruption on user message (breaks out of plan, re-evaluates) - Correct SSE status field in mark_completed/mark_failed/mark_stuck - SseManager preserved across rebuild_state calls Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in db/mod.rs and nearai_chat.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
78878ad7ef |
Remove restart infrastructure, generalize WASM channel setup (#493)
* refactor: remove restart infrastructure and generalize Telegram-specific code Remove the gateway restart mechanism (hot-activation works, restart won't fix activation failures) and generalize Telegram-specific hardcoded checks so all WASM channels get equal treatment. Part 1 - Remove restart infrastructure: - Remove needs_restart from ActionResponse, restart_requested from GatewayState - Remove gateway_restart_handler, /api/gateway/restart route, exit code 75 - Remove restart overlay JS/CSS (dead code - restartGateway() never called) - Surface actual activation errors instead of suggesting restart Part 2 - Generalize Telegram-specific code: - Replace telegram_owner_id: Option<i64> with generic wasm_channel_owner_ids: HashMap<String, i64> (backwards-compatible via TELEGRAM_OWNER_ID env var) - Pairing status check now applies to all active WASM channels - All channels get 3-step stepper in web UI, remove "coming soon" note - Remove dead setup_telegram() code (~700 lines) - Telegram's capabilities.json declares required_secrets, so the generic setup_wasm_channel() path handles it Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test: add Settings::set() test for wasm_channel_owner_ids Addresses review feedback: verify that setting per-channel owner IDs via the dotted-path Settings::set() API works correctly with the new HashMap<String, i64> type. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): refresh extension stepper after pairing approval loadPairingRequests only refreshed the pairing section, not the stepper status. Call loadExtensions() instead so the stepper updates from "Awaiting Pairing" to "Active" immediately after approval. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
20073ccf57 |
fix(web): auto-scroll and Enter key completion for slash command autocomplete (#475)
- Add scrollIntoView to keep arrow-key-selected item visible in dropdown - Make Enter complete the first matching command when autocomplete is visible, instead of requiring explicit arrow-key navigation first Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
98467a553e |
fix(telegram): remove restart button, validate token on setup (#434)
* fix(web): remove gateway restart button from channel activation failure cards When a WASM channel (e.g. Telegram) fails to hot-activate after setup, the extension card showed a "Restart" button that calls POST /api/gateway/restart. This triggers a process exit and relies on an external supervisor to relaunch, which doesn't work reliably when running inside Docker. Remove the Restart button entirely from the failed-activation card for all channels — Reconfigure is the correct recovery action (re-enter credentials). Also fix two bugs found during review: - setServerLogLevel/loadServerLogLevel called .json() on the already-parsed object returned by apiFetch, causing a silent TypeError that prevented the log level selector from updating - buildBreadcrumb embedded paths in inline onclick JS strings using escapeHtml, which doesn't escape single quotes; switched to data-path attribute pattern to avoid JS string injection from paths containing quotes And simplify: collapse the dead Telegram-specific branch in submitConfigureModal toast messaging — all channels now show "Configured and activated X" on success. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(telegram): propagate token validation errors from on_start Both webhook and polling mode in on_start() swallowed activation errors from register_webhook/delete_webhook — using `if let Err(e)` to log but then returning Ok regardless. This caused a bad bot token to show as "configured and active" instead of failing activation. Telegram returns {"ok": true} when deleteWebhook is called with no existing webhook (idempotent), so any error (e.g. 401 Unauthorized) genuinely means an invalid token. The WASM is rebuilt automatically via build.rs on cargo build. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(telegram): validate bot token before storing, fix misleading toast Add upfront GET /getMe validation in save_setup_secrets() before writing the bot token to the secrets store. This catches bad tokens immediately for both fresh installs and reconfigures — the reconfigure path (refresh_active_channel) skips on_start entirely and would never catch an invalid token without this check. URL-encode the token before interpolating into the getMe URL path. Also update the activation-failure toast from "Restart required to activate" (misleading now that the Restart button is gone) to "Use Reconfigure to re-enter credentials and activate". Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(telegram): collapse nested if, fix formatting (clippy + fmt) Collapse `if name == "telegram" { if let Some(...) }` into a single let-chain condition as suggested by clippy's collapsible_if lint. Also apply rustfmt line-length fixes in the same block. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
|
6481448d50 |
feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar - Remove active-jobs-bar UI element (HTML, CSS, JS polling) - Move job handlers from server.rs to handlers/jobs.rs - Remove user_id scoping (single-user gateway) - Add list_agent_jobs() and agent_job_summary() to Database trait (both postgres and libsql backends) for non-sandbox job visibility - Wire SchedulerSlot into CreateJobTool so execute_local dispatches via scheduler (persists to DB + spawns worker) instead of creating phantom ContextManager-only jobs - Update /status and /list slash commands to read from DB for consistency with Jobs tab - Fix worker mark_completed: skip if already terminal or stuck - Add agent job cancel via DB update in both web handler and slash cmd - Add Stuck → Completed guard with tracing in worker completion path Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: address PR review comments - Log warning when get_context fails in worker completion path - Extract duplicated status-counting logic into AgentJobSummary::add_count() helper, used by both postgres and libsql backends Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Nick Pismenkov <[email protected]> |
||
|
|
afb49597ac |
feat(extensions): add OAuth setup UI for WASM tools + display name labels (#437)
Add setup.required_secrets to tool capabilities.json files so users can configure OAuth client credentials (Google, Slack, Okta, Telegram) through the Extensions UI Setup modal instead of environment variables. - Add ToolSetupSchema/ToolSecretSetupSchema types to capabilities_schema.rs - Extend get_setup_schema(), save_setup_secrets(), list() to handle WasmTool - Extract load_tool_capabilities() helper to reduce duplication - Auto-activate tools after saving setup secrets - Show display_name labels (Channel/Tool/MCP) in extension cards - Update button labels: "Setup" when unconfigured, "Reconfigure" when set - Replace "Set" badge with checkmark in configure modal - Fix innerHTML XSS pattern in slash autocomplete (use textContent) - Add tests for ToolSetupSchema parsing and resolve_nested promotion - Update registry display names (e.g. "Telegram Channel" vs "Telegram Tool") Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
|
9ce09f71b0 |
feat(web): slash command autocomplete + /status /list + fix chat input locking (#404)
* feat(web): slash command autocomplete, /status /list /cancel, fix input locking Backend: - Add JobStatus, JobList, JobCancel Submission variants to submission.rs - Parse /status [id], /progress [id], /list, /cancel <id> as control commands - Dispatch to existing handle_check_status/handle_list_jobs/handle_cancel_job handlers via new process_job_status/process_job_list/process_job_cancel methods - Add 4 parser tests (34 total, all passing) Web UI: - Add slash command autocomplete: type / in chat input to see all 18 commands with descriptions; arrow-key navigation, Tab/Enter to select, Escape to close - Remove chat input locking: drop textarea.disabled + sendBtn.disabled so users can always type and send (including /interrupt while agent is processing) - Remove quick-action toolbar buttons (↩↪⏸⊖🗑📋) added in previous session - Remove dead #chat-status bar (min-height 28px black bar always visible when empty) Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * refactor: address PR review comments - Remove Submission::JobList variant; parse /list directly as JobStatus { job_id: None } (simpler, eliminates redundant enum variant, match arm, is_control branch, and wrapper function) - Cache autocomplete matches in _slashMatches to avoid re-filtering SLASH_COMMANDS on every keydown while autocomplete is open Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Pierre LE GUEN <[email protected]> |
||
|
|
a65b282066 |
fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the web UI because the routines list endpoint filtered by GATEWAY_USER_ID, which didn't match the Telegram user's ID stored on the routine. Add list_all_routines() to the RoutineStore trait (both libSQL and PostgreSQL backends) and use it in the web dashboard handlers so all routines are visible regardless of which channel created them. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
Henry ParkandGitHub
|
ddd01a628c | feat(web): persist tool calls, restore approvals on thread switch, and UI fixes (#382) | ||
|
ibhagwanandGitHub
|
c592a8f2de | feat: add IRONCLAW_BASE_DIR env var with LazyLock caching (#397) | ||
|
|
a24fd3e8a3 |
Add automated QA: schema validator, CI matrix, Docker build, and P1 test coverage (#353)
* Add automated QA: tool schema validator, feature-flag CI matrix, Docker build P0 items from the automated QA plan (#352): - Add validate_tool_schema() that checks OpenAI strict-mode rules (type: object, required keys in properties, nested object/array recursion) with 10 unit tests and 6 integration tests covering all core built-in tools - CI test matrix now runs with --all-features, default features, and --no-default-features --features libsql to catch dead code behind wrong cfg gates - CI clippy now runs the same 3-feature matrix with --all flags - Docker build job added to catch missing files in Dockerfile Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P1 automated QA tests and fix LeakDetector prefix shadowing bug P1 test coverage: config round-trip (settings + bootstrap), shell tool arg handling, safety adversarial tests (sanitizer, leak detector, allowlist), turn persistence (conversations, metadata, pagination, jobs), and a clippy fix for libsql-only builds. Fixed a real bug where AhoCorasick non-overlapping prefix iteration caused shorter prefixes (e.g. "sk-") to shadow longer ones (e.g. "sk-ant-api"), preventing Anthropic API key and SSH private key detection. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P2 automated QA tests: chaos, lifecycle, collision, and recovery Cover all P2 items from the automated QA plan: - Circuit breaker chaos tests (hanging provider, rapid cycles, mixed errors) - Failover chaos tests (hanging failover, all-fail, tools path, single provider) - Value estimator boundary tests (negative cost, zero price, zero earnings) - Context length recovery test (ContextLengthExceeded -> compact -> retry) - WASM channel lifecycle tests (write/commit/read round-trip, namespace isolation) - Extension registry collision tests (same-name different-kind coexistence) - Extension filesystem collision tests (separate dirs, detect_kind priority) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add P3 concurrent stress tests for ContextManager and SessionManager Tests verify thread safety of double-checked locking, TOCTOU prevention, and RwLock-based concurrent access patterns under load. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add dispatcher loop guard and self-repair stuck job tests Dispatcher: test force_text mechanism prevents infinite tool call loops, verify iteration bound arithmetic guarantees termination for all configs. Self-repair: test stuck job detection, recovery within attempt limits, manual escalation when limit exceeded, graceful degradation without store/builder dependencies. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure design doc Python + Playwright framework with mock LLM server for deterministic browser-level testing of the web gateway. Covers connection/auth, chat round-trip with SSE streaming, and skills lifecycle scenarios. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add E2E testing infrastructure implementation plan 10-task plan covering: scaffolding, mock LLM server, helpers, conftest fixtures, connection/chat/skills test scenarios, CI workflow, README, and integration run. Co-Authored-By: Claude Opus 4.6 <[email protected]> * scaffold: E2E test project with pyproject.toml Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E helpers with DOM selectors and port discovery Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: mock OpenAI-compat LLM server for E2E tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E conftest with session fixtures for mock LLM and ironclaw Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 1 -- connection and tab navigation tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 2 -- chat message round-trip tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: E2E scenario 3 -- skills search, install, remove tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add weekly E2E test workflow with Playwright Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: E2E test README with setup and usage instructions Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test integration fixes from first run - Use temp file DB instead of :memory: (libSQL :memory: doesn't persist tables across execute_batch) - Fix installed skills selector: #skills-list not #installed-skills - Add pytest-timeout to dependencies - Improve skills install/remove test with wait_for instead of fixed sleeps 8 passed, 1 skipped (skills install depends on ClawHub availability) Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add OpenAI strict-mode schema validator for all built-in tools (QA 1.1) Add src/tools/schema_validator.rs with validate_strict_schema() that checks tool parameter schemas against OpenAI function calling strict-mode rules: type object at top level, required keys in properties, enum type consistency, array items definitions, nested object recursion, and additionalProperties. 17 tests validate all 34+ built-in tool schemas across 5 test groups: - 9 simple tools (echo, time, json, http, shell, file read/write/list/patch) - 4 job tools (create, list, status, cancel) - 4 skill tools (list, search, install, remove) - 13 inline schemas for extension, routine, and complex job tools - 4 memory tool schemas Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add E2E scenarios for SSE reconnect, HTML injection, and tool approval (QA 3.3/5/6) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: E2E test reliability for HTML injection and SSE reconnect - HTML injection: test sanitization directly via JS injection instead of depending on full LLM round-trip (avoids intermittent 404 from mock) - SSE reconnect: increase wait times for DB persistence and relax assertion to check total message count after history reload Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt formatting Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add WASM and MCP tool schema validation tests (QA 1.1) Extends the schema validator with representative WASM tool schemas (weather, HTTP client, batch processor, status), MCP tool schemas (default, file read, SQL query, strict mode), and defect detection tests for common external schema issues (missing type, typo in required, array without items, enum type mismatch). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add auth middleware and compaction module tests Auth middleware (8 new tests): valid/invalid bearer tokens, query param fallback, case sensitivity, empty tokens, whitespace handling. Compaction module (16 new tests): truncation strategy, summarize strategy with mock LLM, workspace fallback, format_turns helper, sequential compactions, coherence after compaction, token decrease verification. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add config round-trip integration tests (QA 1.2) Test the full bootstrap .env lifecycle: write via the same format as save_bootstrap_env/upsert_bootstrap_var, read back via dotenvy, and assert values match. Covers LLM backend selection, embedding disable flag, onboard completion flag, session token keys, multi-key preservation across upsert, and special characters (spaces, equals, quotes, backslashes, hashes). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add value estimator boundary tests and dispatcher loop guard (QA 4.3/4.4) Value estimator (14 new tests): zero/negative prices, large values, negative cost, exact margin boundaries, custom margin configuration. Dispatcher loop guard (2 new tests): verifies the dispatch loop terminates when all tool calls fail (regression guard for PR #252 infinite loop) and when max iterations are reached. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add failover edge cases and provider chaos tests (QA 2.6/4.1) Failover edge cases (4 new tests): cooldown at zero nanos, half-open failure reopens circuit, all providers fail gracefully (no panic), single failing provider with cooldown. Provider chaos tests (15 new tests): flakey provider with retries, hanging provider with timeout, garbage provider, circuit breaker trip/recover, failover chain cascading, non-transient error stops chain, full stack integration (retry + failover + circuit breaker). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback on QA tests - Fix Bearer auth case-sensitivity per RFC 6750 (auth.rs) - Refactor bootstrap.rs to expose path-parameterized variants so config_round_trip tests call real code instead of reimplementations - Remove deprecated event_loop fixture, use dynamic ports, minimal env, session-scoped browser, and wire HEADED=1 in E2E conftest - Add cross-referencing doc comments between schema validators - Simplify array validation logic in tool.rs - Bump e2e.yml checkout@v4 to @v6 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt and fix clippy warning in signal.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: improve E2E fixture error reporting and prevent stdin blocking - Add --no-onboard flag to prevent wizard from blocking in CI - Pipe /dev/null to stdin to prevent any stdin reads from hanging - Add RUST_BACKTRACE=1 for crash diagnostics - On server startup timeout, dump stderr to pytest output so CI logs show why the server failed to start Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set session-scoped event loop for E2E async fixtures pytest-asyncio 1.3.0 defaults asyncio_default_fixture_loop_scope to None (function scope), causing session-scoped async fixtures to be re-evaluated per test function with independent event loops. Each test then independently attempts to start the ironclaw server, times out at 120s, and wastes ~24 minutes of CI before the job is cancelled. Setting asyncio_default_fixture_loop_scope = "session" ensures all session-scoped async fixtures share a single event loop, so the server starts once and is reused across all tests. Also adds -x flag to pytest in CI to stop on first failure instead of running all 19 tests when the fixture is broken. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: set test loop scope to session to match fixture loop scope With asyncio_default_fixture_loop_scope=session but asyncio_default_test_loop_scope=function (the default), tests run on a per-function event loop while fixtures produce objects (Playwright pages, browser contexts) on the session event loop. This event loop mismatch causes the test to hang indefinitely awaiting Playwright operations that are bound to the wrong loop. Setting both scopes to "session" ensures a single event loop is shared across all fixtures and tests, eliminating the deadlock. Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add roll-up jobs to match branch protection required checks Branch protection expects "Code Style (fmt + clippy)" and "Run Tests" status checks, but only individual job names were reported. Add roll-up jobs that aggregate results and report the expected names. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
996c6a8cc9 |
feat(web): improve WASM channel setup flow (#380)
* feat(web): improve WASM channel setup flow with stepper UI and auto-configure Streamline the WASM channel setup experience in the web gateway: - Auto-open configure modal after installing a WASM channel - Add progress stepper (Installed → Configured → Active) on channel cards - Replace generic Activate button with state-specific actions (Setup, Reconfigure, Restart) - Show "Awaiting Pairing" status for Telegram until first user is paired - Add SSE extension_status events for real-time status updates - Add gateway restart endpoint (POST /api/gateway/restart) with idempotency guard - Always mount webhook routes at startup so hot-added channels work without restart - Add pairing request polling (10s interval) on extensions tab - Track activation errors per channel with inline error display Includes review fixes: activation_error priority over active status, stepper failed state rendering, restart poll timeout, configure modal double-submit guard, and SSE sender ordering constraint documentation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: address PR review comments - Move PairingStore construction outside .map() loop - Extract createReconfigureButton() helper to reduce duplication Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
443b120272 |
feat(web): inline tool activity cards with auto-collapsing (#376)
* feat(web): inline tool activity cards with auto-collapsing Add Claude/Codex-style inline tool activity cards to the web UI that show tool execution progress directly in the chat conversation. While processing: - Animated thinking dots with message text (e.g. "Calling LLM...") - Individual tool cards with live spinner and elapsed timer - Cards show tool name, duration, and expandable output preview After response arrives: - Activity group auto-collapses to "Used N tools (Xs)" - Click summary to expand and see individual tool cards - Click card header to see tool output in monospace Also includes: - "Calling LLM..." thinking status from dispatcher (all channels) - 5-minute max timer guard to prevent leaks on dropped SSE - Handles parallel tools, same tool twice, failures, thread switching Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): use frozen duration for completed tools in activity summary The collapsed activity summary was showing inflated total duration because finalizeActivityGroup() recalculated elapsed time from Date.now() for already-completed tools. Now each tool card stores its final duration at completion time and the summary uses that frozen value instead. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
0c5f082d16 |
feat(web): display logs newest-first in web gateway UI (#369)
Reverse log display order so the most recent entries appear at the top, removing the need to scroll to see latest activity. Frontend: rename appendLogEntry to prependLogEntry, use prepend() for DOM insertion, cap oldest entries from the bottom, and auto-scroll to top. Backend: update recent_entries() doc comment to clarify the oldest-first return order works correctly with the frontend's prepend. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
4e2dd76ae5 |
Fix skills system: enable by default, fix registry and install (#300)
* feat: add Docker detection module with platform guidance Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add Docker sandbox step to setup wizard Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: show Docker status in boot screen Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: check Docker availability at startup When SANDBOX_ENABLED=true, proactively detect whether Docker is installed and running before creating the ContainerJobManager. If Docker is unavailable, log a warning with platform-specific guidance and disable the sandbox for the session. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: enable sandbox by default, improve wizard explanation, document detection limits - SandboxConfig defaults to enabled=true (startup check disables gracefully if Docker is unavailable) - Wizard step explains why Docker matters: isolation for LLM-generated code vs running directly on the host - Document detection confidence per platform in detect.rs module docs: high on macOS/Linux, medium on Windows (named pipe edge cases) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: cargo fmt + update test_builder_defaults for enabled-by-default Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: deduplicate wizard Docker status handling per review Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: fix skills system - enable by default, fix registry connectivity and install - Enable skills system by default (SKILLS_ENABLED no longer required) - Bypass Vercel TLS fingerprint blocking by pointing DEFAULT_REGISTRY_URL directly at the Convex backend (wry-manatee-359.convex.site) - Handle ZIP archives from ClawHub download API - the registry returns ZIP files containing SKILL.md, not raw text. Uses flate2 (existing dep) to extract SKILL.md from the archive. - Surface catalog search errors in the UI with a yellow warning banner instead of silently returning empty results - Handle both {"results":[...]} envelope and bare [...] array JSON formats from the search API - Add ClawHub links and metadata to search result cards (clickable skill names linking to clawhub.ai, relevance score, "updated X ago" recency) - Fix 3 pre-existing clippy warnings in tests/html_to_markdown.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address security review feedback on ZIP extraction and SSRF - Cap download size to 10 MB before reading response body - Guard against ZIP bombs: cap uncompressed_size at 1 MB, wrap DeflateDecoder with .take() read limit - Use checked_add for ZIP header offset arithmetic to prevent overflow - Remove .unwrap() on try_into() -- use direct array construction - Handle IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) in SSRF checks - Don't leak internal registry URLs in user-facing catalog_error messages - Fix non-ASCII panic in catalog response debug logging (use .get() instead of byte slicing) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add /skills command and enrich search results with ClawHub metadata - Parse /skills and /skills search <query> as SystemCommands in submission.rs - Add skill_catalog to AgentDeps and wire it through main.rs - Handle "skills" command in commands.rs: list installed skills and search ClawHub - Add /skills and /skills search <q> entries to /help output - Add SkillDetail, SkillStats, SkillOwner structs to catalog.rs - Add fetch_skill_detail() calling GET /api/v1/skills/{slug} on Convex backend - Add enrich_search_results() to fetch stars/downloads/owner for top 5 results in parallel - Fix SkillDetailResponse wrapper struct to match actual API shape: {"skill":{...},"owner":{...}} - Surface stars, downloads, owner in web UI skill search cards (app.js) - Surface enriched data in skills web handler and skill_search tool output Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: cargo fmt after merge conflict resolution Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: separate installed_skills dir for correct trust on restart, remove duplicate handlers Trust level bug: skills installed from ClawHub were written to user_dir (~/.ironclaw/skills/) which is discovered as Trusted on restart. Now installs go to ~/.ironclaw/installed_skills/ which is discovered as Installed, matching the documented skill directory layout. Changes: - SkillsConfig: add installed_dir field (SKILLS_INSTALLED_DIR env var, default ~/.ironclaw/installed_skills/) - SkillRegistry: add with_installed_dir() builder, installed_dir()/ install_target_dir() accessors, and discover installed_dir with SkillTrust::Installed in discover_all() - All install paths (web handler, skill tool) use install_target_dir() instead of user_dir() so new installs land in the correct directory - 3 new registry tests: test_installed_dir_uses_installed_trust, test_install_target_dir_prefers_installed_dir, test_user_dir_stays_trusted_with_installed_dir Duplicate handler cleanup: handlers/skills.rs was the canonical implementation but the handlers module was never compiled (not declared in web/mod.rs), so server.rs had its own duplicate inline definitions that the router used. Wire up the handlers module, delete the 260-line duplicate in server.rs, and have server.rs import skills handlers from handlers::skills. Fix pre-existing compile error in handlers/extensions.rs (missing needs_setup field). Add #[allow(dead_code)] on not-yet-migrated handler modules to suppress warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: probe more Docker socket paths on macOS Docker Desktop 4.13+ (stabilised in 4.18) no longer creates the /var/run/docker.sock symlink by default. The API socket lives at ~/.docker/run/docker.sock, which bollard's connect_with_local_defaults() does not try. Add a fallback probe list covering the common macOS container runtimes: - ~/.docker/run/docker.sock — Docker Desktop 4.13+ - ~/.colima/default/docker.sock — Colima - ~/.rd/docker.sock — Rancher Desktop Remove the bogus ~/.docker/desktop/docker.sock path that was added previously; it is not an API socket on any known Docker installation. Fixes the false-negative "Docker is installed but not running" warning reported by Illia on macOS with Docker Desktop 4.18+. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * Harden Docker detection for rootless Linux and Windows fallback --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
Robert YanandGitHub
|
c68dc2ff2a | feat: update dashboard favicon (#309) | ||
|
|
82f24bf08f |
fix: block send until thread is selected (#306)
* fix: block send until thread is selected Prevents messages from ending up in orphan threads when user sends while currentThreadId is null during page load. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: guard enableChatInput against null thread + add user feedback Prevents SSE events from re-enabling input before a thread is selected. Adds status message when user tries to send without a thread. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
510fba4c92 |
fix: reload chat history on SSE reconnect (#307)
When SSE auto-reconnects after a server restart, the chat now re-syncs from the database so no messages are lost. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
04d3b005b1 |
feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch FullJob routines previously fell back to lightweight mode (single LLM call, no tools) with a warning. This wires them to the existing Scheduler/Worker infrastructure so they dispatch real jobs with full tool access. Fire-and-forget model: the routine creates a job via ContextManager, schedules it, links the routine_run to the job_id, and completes immediately. The job runs independently with full tool access. - Add RoutineError::JobDispatchFailed variant - Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL) - Add execute_full_job() in routine_engine with context_manager/scheduler - Wire context_manager + scheduler into RoutineEngine from agent_loop - Fix pre-existing clippy warnings in tests/html_to_markdown.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: persist job to DB before scheduling in execute_full_job The worker emits job_actions and llm_calls rows that reference agent_jobs via foreign key. Without persisting the job first, those inserts can fail. Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations Move the create + persist + schedule sequence into a single Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs) don't duplicate the logic. FullJob routines now pass max_iterations via job metadata, and the worker reads it (defaulting to 50 if unset). Also removes the context_manager field from RoutineEngine since dispatch_job handles everything internally. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: clamp max_iterations to 500 and log category update failures Address PR review feedback: - worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500) to prevent unbounded LLM token usage from malicious/buggy configs - commands.rs: log warning on category update failure instead of silently discarding the error Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
ea57447649 |
feat: hot-activate WASM channels, channel-first prompts, unified artifact resolution (#297)
* refactor: unify WASM artifact resolution into registry/artifacts.rs Consolidate duplicated WASM find/build/install logic from 5+ files into a single src/registry/artifacts.rs module. This fixes two bugs: - registry/installer.rs now respects CARGO_TARGET_DIR (was hardcoded) - channels/wasm/bundled.rs now searches all WASM triples (was wasip2 only) Also includes: extension manager hot-activation for WASM channels, extension guidance in LLM prompts, channel manager hot-add support, webhook router channel lookup, and minor cleanups. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: send approval prompts as messages on WASM channels (Telegram, Slack) WASM channels mapped ApprovalNeeded status to a typing indicator, so users on Telegram never saw tool approval prompts — the agent got stuck in AwaitingApproval and all subsequent messages failed with "Waiting for approval". - Intercept ApprovalNeeded in WasmChannel::handle_status_update and send the prompt as an actual message via call_on_respond, showing tool name, description, parameters, and yes/no/always instructions - Guard against empty LLM responses after clean_response() strips reasoning_content think-tags (defense-in-depth for reasoning models) - Add reasoning_content fallback to NearAiChatProvider::complete() for consistency with complete_with_tools() - Add debug logging when empty responses are suppressed - Improve error logging for channel respond() failures - Register WASM channel webhook routes before credential checks so platforms don't deactivate webhook URLs with 404s Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #297 review comments - ChannelManager::add: use async write().await instead of try_write() - resolve_target_dir: resolve relative CARGO_TARGET_DIR against crate_dir - install_wasm_files: log warning on capabilities copy failure - refresh_active_channel: load capabilities file for webhook secret name - activate_wasm_channel: validate name against path traversal - Fix cargo fmt formatting in nearai_chat.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: wire up channel runtime for hot-activation and address PR review round 2 - Wire up set_channel_runtime() in main.rs so hot-activation actually works (with_channel_runtime was never called — hot-activation was dead code) - Change ExtensionManager channel runtime fields to RwLock<Option<...>> interior mutability so set_channel_runtime(&self) works after Arc wrapping - Fix artifact tests to use resolve_target_dir() instead of hardcoding "target/" (breaks when CARGO_TARGET_DIR is set) - Fix bundled.rs build hint: cargo component build (not cargo build --target) - Fix wasm_artifact_path doc: binary_name should not include .wasm extension Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use char-aware truncation to prevent UTF-8 panic in approval prompt &s[..77] panics on multi-byte UTF-8 (CJK, emoji). Use s.chars().take(77) for safe truncation at character boundaries. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
b3bf50f10e |
feat: add pairing/permission system to all WASM channels and fix extension registry (#286)
Port Telegram's permission model (owner_id, dm_policy, allow_from, pairing codes)
to Discord, Slack, and WhatsApp WASM channels. Add web UI for configuration and
pairing approval. Fix extension registry issues preventing Discord install and
causing Slack activation to hit the wrong endpoint.
WASM channels:
- Discord: add DiscordConfig, permission checks, ephemeral pairing replies,
fix capabilities.json (header_name→name), downgrade wit-bindgen to 0.36
- Slack: expand SlackConfig with permission fields, add check_sender_permission
and send_pairing_reply via chat.postMessage
- WhatsApp: expand WhatsAppConfig with permission fields, add permission checks
and pairing reply via Cloud API
- Telegram: reformat capabilities.json, add setup.required_secrets
Extension system:
- Add Discord to KNOWN_CHANNELS in bundled.rs and to extension registry
- Rename "slack" MCP→"slack-mcp", "slack-channel"→"slack" to fix name collision
- Add ExtensionSource::Bundled variant handling in discovery.rs
- Add get_setup_schema/save_setup_secrets to ExtensionManager
- Add needs_setup field to InstalledExtension
Web gateway:
- Add GET/POST /api/extensions/{name}/setup for configuration modal
- Add GET /api/pairing/{channel} and POST /api/pairing/{channel}/approve
- Add configure modal UI (password fields, provided badges, auto-generate hints)
- Add pairing request UI on active WASM channel cards
- Show "Restart to activate" label instead of Activate button for WASM channels
Co-authored-by: Claude Opus 4.6 <[email protected]>
|
||
|
|
436066415b |
feat: embedded registry catalog and WASM bundle install pipeline (#283)
* feat: embedded registry catalog and WASM bundle install pipeline Embed registry manifests at compile time so the extension catalog is available without network access. Add tar.gz bundle support for WASM extension downloads (tools and channels), a /api/extensions/registry endpoint, CI job to build and publish WASM bundles on release, and ephemeral in-memory secrets fallback so the extension manager works even without a persistent secrets store. Key changes: - build.rs: collect registry/*.json into embedded_catalog.json at compile time - src/registry/embedded.rs + catalog.rs: load embedded or on-disk catalog - src/extensions/manager.rs: download_and_install_wasm handles tar.gz bundles, bare .wasm files, and separate capabilities downloads; wasm channel install - src/channels/web/server.rs: /api/extensions/registry endpoint + no-cache headers - src/app.rs: ephemeral InMemorySecretsStore fallback for extension manager - registry/*.json: populate artifact download URLs for release bundles - .github/workflows/release.yml: build-wasm-extensions CI job - Simplified setup wizard and CLI registry commands Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — archive hardening, decompression bomb guard, test fix - Add 100 MB decompressed entry size cap to tar.gz extraction in both manager.rs and installer.rs to prevent decompression bombs - Add archive.set_preserve_permissions(false) and set_unpack_xattrs(false) for defense-in-depth against malicious archives - Fix test assertion logic in catalog.rs (|| → || with correct negation) - Replace silent tar fallback in CI with explicit if/else for capabilities - Add warning when installing without SHA256 verification Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: resolve clippy warning in settings.rs and enforce zero-warnings policy Use struct initializer with ..Default::default() instead of field reassignment. Update CLAUDE.md to codify zero clippy warnings policy — all warnings must be fixed before committing, including pre-existing ones. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review round 2 — build reliability, caps validation, naming - build.rs: emit per-file rerun-if-changed for reliable content tracking; fix bundles fallback to match BundlesFile shape ({"bundles":{}}) - embedded.rs: parse catalog once via OnceLock instead of double-parsing - manager.rs + installer.rs: add 1 MB size cap on capabilities_url downloads with proper error surfacing - secrets/store.rs: rename misleading `pub mod testing` to `pub mod in_memory` - server.rs: track installed extensions by (name, kind) tuple to avoid false positives across different extension kinds Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
b68d67bd35 |
feat: show token usage and cost tracker in gateway status popover (#284)
* feat: show token usage, cost tracker, and uptime in gateway status popover The "Connected" hover popover in the web gateway now displays three sections: connection info (SSE/WS counts, uptime), daily cost tracker (spend + actions/hr), and per-model token usage (input/output counts with cost per model). Also fixes the field name mismatch between the backend response and JS rendering that prevented the popover from showing correct data. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — escape HTML in popover, add model_usage test - Escape model name and cost strings with escapeHtml() before inserting into innerHTML to prevent XSS via crafted model names - Add test_model_usage_per_model_tracking test covering multi-model token/cost accumulation in CostGuard Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
250551799b |
style: adopt agent-market design language for web UI (#282)
* fix: move Logs to status bar and fix chat history ordering after restart
Move the Logs tab out of the main tab bar and into the right-side status
area as a compact pill button next to "Connected". Remove it from the
Ctrl+1-N shortcut order (now Ctrl+1-5).
Fix chat message ordering in libSQL backend: datetime('now') has only
second precision, so back-to-back user+assistant inserts got identical
timestamps causing non-deterministic ORDER BY. Now passes explicit
millisecond-precision timestamps and uses rowid as tiebreaker for
existing data.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* refactor: separate WASM extensions from MCP servers on Extensions page
Reorganize the Extensions tab into 5 distinct sections: Installed
Extensions, Available WASM Extensions, Install WASM Extension (by
tar.gz URL), MCP Servers (with Add Custom form), and Registered Tools.
Registry entries are now filtered client-side by kind so WASM tools/
channels and MCP servers each have dedicated UI sections.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: adopt agent-market design language for web UI
Refresh the web gateway visual identity with a cleaner, modern aesthetic:
deeper blacks, green accent palette, DM Sans + IBM Plex Mono typography,
larger border-radii, glassmorphic navigation, refined hover effects,
pill badges, and green focus rings. CSS-only change plus Google Fonts.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* Update src/channels/web/static/style.css
Co-authored-by: Copilot <[email protected]>
* Update src/channels/web/static/style.css
Co-authored-by: Copilot <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
|
||
|
|
3b6105d5ea |
feat: add TEE attestation shield to web gateway UI (#275)
Show a shield indicator in the tab bar when the instance is running inside a TEE deployment. On hover, fetches and displays the TDX attestation report (image digest, TLS cert fingerprint, report data, VM config) from the management API. Co-authored-by: Cursor <[email protected]> |
||
|
Pierre LE GUENandGitHub
|
df8616b604 |
fix: add X-Accel-Buffering header to SSE endpoints (#277)
Nginx buffers responses by default, breaking SSE connections that go through a reverse proxy. Add X-Accel-Buffering: no header to chat and log SSE handlers to match what compose-api and chat-api already do. |
||
|
|
448383cfb0 |
refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably - Filter out `type: "reasoning"` output items from NEAR AI Responses API parsing so chain-of-thought never reaches the UI (nearai.rs) - Rewrite clean_response with regex-based tag stripping that is code-aware (preserves tags inside fenced blocks and inline backticks), supports 9+ tag names (think, thought, reasoning, reflection, etc.), handles <final> extraction, pipe-delimited tags, and case/whitespace tolerance (reasoning.rs) - Add Reasoning::complete() helper so all non-agentic LLM call sites (summarize, suggest, heartbeat, compaction) get automatic response cleaning; thread SafetyLayer through to those callers - Change persist_turn from fire-and-forget tokio::spawn to awaited async so both user and assistant messages are written before returning, preventing data loss on shutdown/restart - Pass input_count through seed_response_chain so response chaining delta calculation is accurate after thread hydration on restart - Make NearAiResponse.usage optional and preserve response_id in alt response path for chaining continuity - Persist session token to DB during onboarding wizard so runtime loads it without legacy-key fallback; suppress spurious warning on fresh installs - Fix dev tool double-registration when builder already registers them - Load dotenv/ironclaw env for doctor and status subcommands - Reduce startup log noise (demote info→debug for skills, remove redundant info lines) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Nudge to not loop over tools continuesly * refactor: remove Responses API, consolidate NEAR AI to Chat Completions only The Responses API provider (nearai.rs, 1278 lines) added significant complexity (response chaining state machine, delta message calculation, previous_response_id persistence) for marginal benefit. This consolidates to the Chat Completions API only, upgrading NearAiChatProvider with dual auth (session token + API key) and 401 retry for session token renewal. - Delete src/llm/nearai.rs (Responses API provider) - Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models - Remove response_id from CompletionResponse and ToolCompletionResponse - Remove seed_response_chain/get_response_chain_id from LlmProvider trait - Remove response chain persistence from agent (thread_ops, session) - Remove NearAiApiMode enum and NEARAI_API_MODE config - Clean up all wrapper providers (retry, circuit_breaker, failover, cache) - Update documentation (CLAUDE.md, .env.example) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: runtime log level control via gateway UI and URL parameter Add server-side log level switching using tracing_subscriber::reload::Layer so the EnvFilter can be swapped at runtime without restarting. Expose via GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs toolbar, and a ?log_level=debug URL parameter for one-click activation. Also applies cargo fmt to pre-existing files (llm/, tests/). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
097a26ace6 |
fix: harden openai-compatible provider, approval replay, and embeddings defaults (#237)
* fix: harden openai-compatible tool flow and local defaults * fix: close approval replay gaps and harden openai-compatible flow * fix: address review feedback and code improvements (takeover #112) - Make ChatCompletionResponse.id Optional<String> to handle providers that omit or null the field - Propagate HTTP client builder errors instead of silently dropping timeout configuration (openai_compatible_chat, nearai_chat) - Add EMBEDDING_DIMENSION env var with smart per-model defaults instead of hardcoding 768/1536 everywhere - Remove duplicated dimension inference logic from main.rs Co-Authored-By: panosAthDBX <[email protected]> Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden src/llm/ module from crate audit findings - Replace 9x .expect() on RwLock with graceful poison recovery (nearai.rs: 7, nearai_chat.rs: 2) — eliminates production panics - Propagate HTTP client builder errors in nearai.rs instead of silently dropping timeout config (NearAiProvider::new now returns Result) - Make nearai_chat ChatCompletionResponse.id Optional<String> (mirrors openai_compatible_chat.rs fix for providers that omit id) - Make nearai_chat usage fields optional with defensive parse_usage() helper (was required u32 fields that crash on null/missing) - Truncate error responses to 512 chars in nearai_chat.rs error messages to prevent log bloat and potential data leakage - Delegate 4 missing LlmProvider methods in FailoverProvider (model_metadata, seed_response_chain, get_response_chain_id, calculate_cost) to last-used provider instead of trait defaults Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(llm): add RetryProvider, remove openai_compatible_chat, harden decorators - Add composable RetryProvider decorator wrapping any LlmProvider with exponential backoff + jitter, respecting RateLimited retry_after hints - Remove openai_compatible_chat.rs — replaced by rig adapter + RetryProvider - Remove internal retry loop from nearai.rs (was causing double-retry with external RetryProvider, up to 16 attempts instead of 4) - Remove internal retry loop from nearai_chat.rs (same issue) - Wire RetryProvider into main.rs composition chain: each provider gets its own retry wrapper before failover - Move normalize_tool_name to rig_adapter.rs for all rig-based providers - Reconcile is_retryable() vs is_transient() error classification: ModelNotAvailable no longer retryable, Json no longer transient - Fix unchecked Duration subtraction panic in circuit_breaker.rs - Make failover.rs use shared is_retryable() from retry.rs - Remove stale #[allow(dead_code)] on NearAiResponse::id (field is used) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback — error handling, dimension validation, libSQL warning - Replace response.text().await.unwrap_or_default() with proper error propagation in nearai.rs and nearai_chat.rs (4 call sites). Failures now return LlmError::RequestFailed with context instead of silently proceeding with an empty string. - Add embedding dimension validation in OllamaEmbeddings::embed_batch(): returns EmbeddingError if Ollama returns embeddings with a dimension that doesn't match the configured value. - Add runtime warning when libSQL backend is used with non-1536 embedding dimension, since the libSQL schema uses F32_BLOB(1536) and cannot store different-dimension vectors. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Apply suggestions from code review Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: panosAthDbx <[email protected]> Co-authored-by: panosAthDBX <[email protected]> Co-authored-by: panosAthDBX <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Copilot <[email protected]> |
||
|
|
e42b1e5ec1 |
fix: Network Security Findings (#201)
* docs(security): add network security reference for all listeners Catalogs every network-facing surface (web gateway, webhook server, orchestrator API, OAuth callback, sandbox proxy) with auth mechanisms, bind addresses, egress controls, known findings, and a review checklist for PRs that touch network-facing code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): address three network security findings - Use constant-time comparison (ct_eq) for webhook secret validation, matching the pattern in web gateway and orchestrator auth - Add X-Content-Type-Options and X-Frame-Options security headers to the web gateway via SetResponseHeaderLayer - Warn at startup when HTTP webhook server binds to 0.0.0.0 - Update NETWORK_SECURITY.md to mark findings 1, 4, 5 as resolved Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): address PR #201 review findings - Reorder web gateway layers so security headers (X-Content-Type-Options, X-Frame-Options) are outermost and apply to all responses including DefaultBodyLimit 413 rejections - Move 0.0.0.0 warning to final bind address resolution so it fires for WASM-only webhook servers that fall back to the default address - Add webhook handler auth tests: correct secret -> 200, wrong secret -> 401, missing secret -> 401 - Rewrite NETWORK_SECURITY.md: replace brittle line-number references with function/struct name anchors, add threat model section, document graceful shutdown per listener, fill content gaps (health endpoint responses, content-type validation, CSRF analysis, WS auth flow, MCP trust boundary, orchestrator rate limiting), change findings F-4/F-5 from "Resolved" to "Mitigated" with caveats Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt and clippy warnings from main merge Fix formatting in llm/mod.rs and llm/rig_adapter.rs introduced by PR #132, and collapse nested if in rig_adapter.rs per clippy. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
ccf60055f4 |
feat: support per-request model override in /v1/chat/completions (#103)
* feat: support per-request model override for /v1/chat/completions - add optional model override to completion request types\n- forward request model through gateway, worker, and orchestrator proxy paths\n- use request model in NEAR AI providers with fallback to active model\n- replace model-mismatch integration test with override propagation checks\n- update FEATURE_PARITY.md note for OpenAI-compatible API behavior\n\nRefs #49 * Wire gateway OpenAI-compatible routes to active LLM provider * Validate OpenAI model name length before streaming * Address PR103 review feedback on model override and validation * Report effective model in OpenAI-compatible responses * Use async mutexes in OpenAI compatibility integration tests * fix tests for per-request model field in response cache * fix formatting and clippy lint after main merge * Fix model override reporting and cache correctness --------- Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
ffb1cc9be8 |
refactor: architecture improvements for contributor velocity (#198)
* refactor: split large files and consolidate test stubs for contributor velocity - Extract 7 Database sub-traits (ConversationStore, JobStore, SandboxStore, RoutineStore, ToolFailureStore, SettingsStore, WorkspaceStore) with Database as a supertrait combining them all - Split libsql_backend.rs (2769 lines) into src/db/libsql/ directory with one file per sub-trait implementation - Split config.rs (1753 lines) into src/config/ directory with 16 domain files - Consolidate 3 duplicate test LLM stubs into shared StubLlm in src/testing.rs - Split server.rs handlers into src/channels/web/handlers/ directory - Extract main.rs init phases into AppBuilder (src/app.rs) - Add developer setup script (scripts/dev-setup.sh) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: move heartbeat test from examples/ to tests/ Convert standalone example binary into a proper #[ignore] integration test, matching the convention of the other integration tests. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt formatting for CI Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review comments from Copilot - tunnel.rs: replace .ok().flatten() with ? to propagate env var errors - secrets.rs: remove misleading "process-wide cache" comment - database.rs: use uppercase "DATABASE_URL" in error key - testing.rs: gate harness tests with #[cfg(feature = "libsql")] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Illia Polosukhin <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
cfb579a4bb |
feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows (#57)
* feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows Adds JobEventsTool and JobPromptTool so the main agent can read container event logs and send follow-up prompts to running Claude Code sessions. A background JobMonitor forwards container assistant messages into the agent loop via a new inject channel on ChannelManager. CreateJobTool now accepts a project_dir parameter for mounting existing cloned repos into containers, and spawns the monitor automatically for async jobs. Also: Dockerfile bumped to Rust 1.88 (rig-core needs let chains), GITHUB_TOKEN forwarded into containers for gh CLI auth, and truncate() fixed for multi-byte char boundary panics. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Add ownership checks to JobEventsTool and JobPromptTool via ContextManager to prevent users from accessing other users' jobs (IDOR) - Combine Dockerfile gh CLI install into single apt-get layer - Handle truncate() edge case when max falls inside first multi-byte char - Log actual count of registered job management tools - Document fire-and-forget job monitor lifecycle - Add tests for ownership rejection and schema validation Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Replace hardcoded GITHUB_TOKEN with on-demand credential delivery Containers now fetch credentials via authenticated GET /worker/{id}/credentials endpoint instead of receiving them baked into env vars at creation time. Secrets are decrypted from SecretsStore on demand, scoped per-job via CredentialGrant, and revoked automatically when the job completes. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address sandbox audit findings (CONNECT tunnel, readonly_rootfs, type consolidation) - Implement real CONNECT tunnel with bidirectional TCP piping via hyper upgrade - Fix readonly_rootfs to apply for both ReadOnly and WorkspaceWrite policies - Consolidate duplicate CredentialMapping/CredentialLocation into secrets::types - Share reqwest::Client across proxy requests instead of per-request allocation - Store Docker connection and reuse across executions - Remove .unwrap() from proxy response builders with safe fallbacks - Add output truncation to direct (non-container) execution (64KB limit) - Delete dead src/tools/sandbox.rs (ToolSandbox never used) - Fix connect_docker error message to list all attempted socket paths - Update proxy credential injection to handle all CredentialLocation variants - Use glob-based host_patterns matching for credential lookup in proxy policy Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Dockerfile: install curl+ca-certificates before fetching GitHub CLI GPG key - JobEventsTool/JobPromptTool: reject missing context (prevents IDOR bypass) - parse_credentials: validate env var names against denylist and pattern - resolve_project_dir: require explicit paths to exist before validation - Credential serving: lower log level from info to debug Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address orchestrator audit findings (constant-time auth, error handling, tests) - auth: constant-time token comparison via subtle::ConstantTimeEq - auth: replace hand-rolled hex_encode with std::fmt::Write fold - api: report_status now updates ContainerHandle (was a no-op) - api: log complete_job errors instead of silently discarding - job_manager: log Docker cleanup errors in stop_job/complete_job - job_manager: extract validate_bind_mount_path with proper error on missing home_dir and mandatory base dir creation before canonicalize - job_manager: cache Docker connection across operations - error: remove dead OrchestratorError::AuthFailed and ContainerTimeout - Add 13 new tests (prompt queue, credentials, events, status, paths) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use floor_char_boundary in sandbox manager truncate to prevent multi-byte panics String::truncate() panics when the index falls mid-way through a multi-byte UTF-8 character. Use the same floor_char_boundary utility already used in worker/runtime.rs and tools/builtin/shell.rs. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: default base_url to private.near.ai for Responses API mode Session tokens only authenticate against private.near.ai, not cloud-api.near.ai. The default base_url now matches the api_mode: - Responses (session token): https://private.near.ai - ChatCompletions (API key): https://cloud-api.near.ai This broke when the multi-provider merge introduced cloud-api.near.ai as the unconditional default. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use private.near.ai as default base URL for all API modes private.near.ai now supports both Responses and ChatCompletions endpoints, so there is no reason to route through cloud-api.near.ai. This also fixes session token auth which only works against private.near.ai. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden libSQL concurrency, fix Claude Code Docker auth and permissions Three fixes for the sandbox/Claude Code pipeline: 1. SQLite "database is locked": set WAL journal mode in migrations and PRAGMA busy_timeout=5000 on every connection across LibSqlBackend, LibSqlSecretsStore, and LibSqlWasmToolStore (~83 async call sites). 2. Claude Code container auth: extract OAuth token from macOS Keychain (or Linux ~/.claude/.credentials.json) at startup and inject via CLAUDE_CODE_OAUTH_TOKEN env var. Removes the broken bind-mount approach that failed on uid mismatch. 3. Claude Code tool permissions: wire CLAUDE_CODE_ALLOWED_TOOLS env var through to the worker binary (was hardcoded to empty vec), and expand defaults to include all standard tools (Read, Write, Edit, Glob, Grep, NotebookEdit, Bash, Task, WebFetch, WebSearch). Also adds --verbose flag to claude CLI (required with stream-json + -p), failover provider model switching, and nearai models endpoint fix. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: stream event parsing, job ID prefix resolution, session renewal in list_models Three fixes for the Docker/gateway pipeline: 1. Claude Code stream event parsing (claude_bridge.rs): Rewrite ClaudeStreamEvent to match actual NDJSON format where content blocks are nested under message.content[], not at the top level. Add handler for "user" events (tool_result blocks) and emit result text as a "message" event so reviews appear in gateway activity view. 2. Job ID prefix resolution (job.rs): Add resolve_job_id() that accepts short hex prefixes (like git short SHAs) in addition to full UUIDs. The LLM sees truncated IDs in job monitor messages like "[Job f2854dd8]" and can now use them directly with job_status/cancel/events/prompt tools. 3. Session renewal in list_models (nearai.rs): list_models() now retries with OAuth renewal on 401, matching send_request()'s existing behavior. Previously it returned SessionExpired immediately, causing the setup wizard to fall back to defaults instead of prompting re-authentication. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: /model command now lists available models Previously /model with no args only showed the current model name. Now it fetches and displays all available models from the provider, marking the active one, so users can see what's available before switching with /model <name>. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #57 review findings (set_var UB, tunnel timeout, restart creds) - Replace unsafe `std::env::set_var` in worker runtime and Claude bridge with `Command::envs()` injection via a new `extra_env` field on `JobContext`, avoiding undefined behavior in the multi-threaded tokio runtime. - Add 30-minute timeout to CONNECT tunnel `copy_bidirectional` in the sandbox proxy to prevent stuck connections from leaking spawned tasks. - Persist credential grants (as JSON in the description column) on `SandboxJobRecord` so `jobs_restart_handler` can restore them instead of passing `vec![]`, which caused restarted containers to lose access to their original secrets. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR #57 review comments - Normalize host_patterns to lowercase in proxy policy matching - Push LIMIT into SQL for list_job_events (Database trait + both backends) - Remove unused was_explicit binding in job tool - Return 500 instead of 200 in make_response fallback path - Update copy_auth_from_mount docstring for env-var default - Use entry.file_type() instead of is_dir() to avoid following symlinks Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address third round of PR #57 review comments - Restore glob patterns in default_claude_code_allowed_tools (Bash -> Bash(*)) - Add tracing::warn for credential grant serialize/deserialize failures - Wrap extra_env in Arc<HashMap> to avoid deep cloning per tool call - Document unsupported credential locations (AuthorizationBasic, UrlPath) - Document TOCTOU window in validate_bind_mount_path - Expand doc comments on JobEventsTool and JobPromptTool Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fourth round of PR #57 review comments - Document CONNECT tunnel task lifecycle (timeout is the cleanup mechanism) - Remove secret names from error-level credential logs to prevent leaking - Expand DANGEROUS_ENV_VARS denylist with language runtime hijack vectors Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fifth round of PR #57 review comments - Promote job monitor startup log to info level for observability - Require minimum 4-char prefix in resolve_job_id to limit enumeration - Cap credential grants at 20 per job to bound column storage - Clamp job events limit to 1..1000 to prevent memory abuse Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing closing brace for SkillsConfig impl block The merge resolution dropped the closing `}` for `impl SkillsConfig`, causing a compilation error in CI. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
bac2d75713 |
feat: Secure prompt-based skills system (Phases 1-4) (#51)
* feat: Add secure prompt-based skills system (Phase 1 MVP) Implement a skills system that extends the agent with prompt-level instructions from local directories. Skills declare activation criteria, tool permissions, and trust tiers that determine authority attenuation. Core security model: the minimum trust level of any active skill determines a tool ceiling -- tools above the ceiling are removed from the LLM's tool list entirely at the API level, preventing prompt-based manipulation. New modules: - skills/mod.rs: Core types (SkillTrust, SkillManifest, LoadedSkill) - skills/scanner.rs: Content scanner for manipulation detection - skills/registry.rs: Filesystem discovery and manifest parsing - skills/selector.rs: Deterministic two-phase prefilter (no LLM) - skills/attenuation.rs: Trust-based tool filtering Integration: - Agent loop selects skills per-turn and applies tool attenuation - Reasoning engine injects skill context with structural isolation - Config supports SKILLS_ENABLED, SKILLS_DIR, SKILLS_MAX_ACTIVE, SKILLS_MAX_CONTEXT_TOKENS environment variables - Disabled by default (SKILLS_ENABLED=false) 41 new tests covering all modules. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address all adversarial review findings for skills system Security fixes: - Escape skill name/version in XML attributes to prevent trust spoofing - Escape prompt content to prevent </skill> tag breakout - Require integrity hash for Verified/Community tier skills - Validate skill names against [a-zA-Z0-9][a-zA-Z0-9._-]{0,63} - Add 64 KiB file size limit on prompt.md Bug fixes: - Use actual SkillsConfig from AgentDeps instead of SkillsConfig::default() - Add skills_config field to AgentDeps, wired through from main.rs Performance: - Pre-compile regex patterns at load time (cached on LoadedSkill) - Selector uses pre-compiled patterns instead of recompiling per message - Switch all std::fs to tokio::fs for non-blocking async I/O Hardening: - Cap keyword score at 30 points to prevent keyword stuffing attacks - Enforce max 20 keywords and 5 patterns per skill - Normalize line endings (CRLF/CR to LF) before hashing - Also includes cargo fmt formatting fixes for adjacent code Tests: 54 skills tests pass (up from 41), zero new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address medium/low severity findings from adversarial review Fixes all 18 medium/low severity findings identified by the security review: - mod.rs: Add MAX_TAGS_PER_SKILL cap (10) in enforce_limits(); use RegexBuilder with 64 KiB size_limit to prevent ReDoS; replace case-enumerated escape_skill_content with regex matching all case variants plus whitespace/null byte injection between </ and skill; document allowed_patterns as unenforced until Phase 2; document Marketplace URL validation as Phase 3 concern - registry.rs: Add MAX_MANIFEST_FILE_SIZE (16 KiB) check before reading; add symlink detection via symlink_metadata to reject symlinks in discover_local; add MAX_DISCOVERED_SKILLS (100) cap; validate prompt_hash format (sha256: + 64 hex chars); warn on name collision before overwriting; accept SkillSource parameter in load_skill instead of always using Local; add InvalidHashFormat, ManifestTooLarge, SymlinkDetected error variants - selector.rs: Add MAX_TAG_SCORE (15) cap parallel to keyword cap; warn when declared max_context_tokens diverges >2x from actual prompt size - scanner.rs: Add mixed-script homoglyph detection (Cyrillic, Greek, Armenian unicode ranges); document token-boundary bypass and semantic paraphrasing as known limitations - attenuation.rs: Document READ_ONLY_TOOLS maintenance requirements - agent_loop.rs: Surface scan warnings via structured tracing; add structured audit events for skill activation and tool attenuation 61 tests pass, 0 new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address 12 findings from second adversarial security review HIGH: - Escape opening <skill tags in prompt content (prevents fake skill block injection) - Scan manifest metadata fields (description, author, tags, reasons) not just prompt - Block trust downgrade on name collision (existing Local can't be replaced by Community) MEDIUM: - Eliminate TOCTOU gap: read files then check size instead of metadata-then-read - Reject file-level symlinks in load_skill (prompt.md, skill.toml) - Truncate and filter manifest.skill.tags (prevent unlimited tag scoring) - Cap regex pattern score at 40 (prevent 5x20=100 dominating keyword+tag) - Add doc comment about skill_list tool exposing metadata (sanitization required) - Move Community disclaimer inside <skill> tags (not outside structural boundary) - Filter keywords/tags shorter than 3 chars (prevent broad matching) LOW: - Enforce minimum token_cost of 1 (max_context_tokens=0 can't bypass budget) - Remove redundant try_exists checks in discover_local (let load_skill handle errors) 70 skills tests passing. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add HTTP endpoint scoping for skills (Phase 1) Skills that declare an [http] section in skill.toml now have their HTTP requests constrained to declared endpoints at runtime. This addresses the gap where allowed_patterns was parsed but never enforced -- once the http tool was visible via attenuation, the LLM could reach any URL. Enforcement reuses EndpointPattern/AllowlistValidator from the WASM capability system. Semantics: if no active skill declares [http], all requests pass through (backward compat). If any skill declares [http], URLs must match at least one skill's allowlist (union). Community skills' [http] declarations are silently ignored (defense in depth). Shell commands using curl/wget are also validated against scopes. Scanner gains detection for known exfiltration domains (webhook.site, ngrok.io, etc.), overly broad wildcards, and credential/host mismatches. Closes #38 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt to http_scoping.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt across codebase Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add parameter-level permission enforcement for skills (Phase 2) Activates enforcement of `allowed_patterns` in skill.toml permissions. Previously these patterns were parsed but not enforced -- a Verified skill declaring `permissions.shell` with `allowed_patterns = [{command = "cargo *"}]` could still run any shell command. Now the enforcer validates tool parameters against declared glob patterns before execution. Key changes: - New `enforcer.rs` module with `SkillPermissionEnforcer`, `glob_to_regex()`, and `validate_tool_call()` with union semantics across active skills - Typed pattern enums (`ShellPattern`, `FilePathPattern`, `MemoryTargetPattern`) replace the previous `Vec<serde_json::Value>` in `ToolPermissionDeclaration` - Scanner gains `scan_permission_patterns()` detecting dangerous patterns (rm, sudo, curl, bare wildcards, command chaining, sensitive paths, identity files) - Registry blocks non-Local skills with critical permission pattern warnings - Agent loop threads enforcer into `execute_chat_tool` alongside HTTP scoping Trust interaction: Community patterns ignored, Verified enforced, Local without patterns unrestricted, Local with patterns enforced as guidance. Union semantics across skills -- tool call allowed if ANY skill's patterns permit it. 34 new tests. All 818 library tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add worker permission enforcement and LLM behavioral analysis (Phase 3+4) Phase 3 - Worker-side permission enforcement: - Add SerializedToolPermission/SerializedPattern DTOs for HTTP boundary crossing - Extend JobDescription, ContainerHandle, and orchestrator API to carry permissions - CreateJobTool snapshots and forwards skill permissions to spawned workers - Worker runtime builds SkillPermissionEnforcer and checks before tool execution - Load-time token budget enforcement rejects prompts exceeding 2x declared budget - Deduplicate enforcer construction: from_active_skills() delegates to from_serialized() Phase 4 - LLM behavioral analysis: - BehavioralAnalyzer with cached, LLM-based semantic content analysis - Structured output parsing (FINDING|CATEGORY|SEVERITY|DESCRIPTION or CLEAN) - Content-hash caching with bounded size (MAX_CACHE_ENTRIES=256) - Graceful degradation when LLM unavailable - Integrated into load_skill() for non-Local skills; critical findings block loading Review fixes: - Real cache tests with CountingLlm mock (test_cache_hit, test_cache_miss, test_cache_bounded) - UTF-8-safe truncate() in worker runtime - Few-shot examples in behavioral analysis prompt - Documented max_context_tokens=0 opt-out and create_job() permission gap 848 tests passing, no new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address review feedback from serrrfirat on skills-phase2 - Fix truncate_cmd UTF-8 panic: use char-boundary-aware slicing - Remove redundant effective_tools branching in reasoning.rs - Document cache eviction as known limitation (arbitrary, not LRU) - Add safety comment on SkillTrust enum ordering (security-critical) - Simplify active_skills selection (prefilter_skills handles empty input) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining skills review feedback * refactor: replace skills system with OpenClaw SKILL.md format + 2-state trust Replace the 5-gate, 3-tier trust hierarchy (scanner, behavioral analyzer, parameter-level enforcer, HTTP endpoint scoping) with a simplified 3-layer security model: gating -> attenuation -> Docker confinement. Key changes: - SKILL.md format (YAML frontmatter + markdown prompt) replaces skill.toml + prompt.md - 2-state trust (Installed/Trusted) replaces 3-tier (Community/Verified/Local) - New parser.rs for SKILL.md parsing with serde_yaml - New gating.rs for requirements checking (bins/env/config) - Simplified registry with 2-location discovery (workspace + user dirs) - Removed scanner, behavioral_analyzer, enforcer, http_scoping (~4,100 lines) - Removed skill_permissions propagation through job/orchestrator/worker pipeline - Added serde_yaml dependency for YAML frontmatter parsing Net: -5,298 lines, 59 skills tests pass, 907 total tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add in-app skill management tools and ClawHub catalog integration Add 4 chat-callable tools (skill_list, skill_search, skill_install, skill_remove) plus matching web gateway endpoints for managing skills at runtime. The catalog fetches from ClawHub's public registry API at runtime rather than bundling entries at compile time. Key changes: - SkillRegistry gains mutation methods (install_skill, remove_skill, reload, find_by_name) with Arc<RwLock> for concurrent access - New catalog module queries ClawHub /api/v1/search with in-memory caching (5-min TTL, configurable via CLAWHUB_REGISTRY env var) - skill_list and skill_search added to READ_ONLY_TOOLS for safe use under Installed trust ceiling - Web gateway gets /api/skills, /api/skills/search, /api/skills/install, and /api/skills/{name} DELETE endpoints Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #51 review feedback from ilblackdragon Security: - Add SSRF protection to fetch_skill_content: require HTTPS, reject private/loopback/link-local IPs and internal hostnames, disable redirects. Gateway install handler now reuses the same validation. - URL-encode slug in skill_download_url to prevent query injection. - Require X-Confirm-Action header on gateway skill install/remove endpoints (equivalent to chat tool requires_approval gate). Correctness: - Eliminate all block_in_place/block_on usage in skill tools and gateway handlers. Split install into prepare_install_to_disk (static async, no lock) + commit_install (sync, brief write lock). Same pattern for remove: validate_remove + delete_skill_files + commit_remove. - Write normalized content to disk in install_skill (was writing original un-normalized content, causing hash mismatch on re-read). - Fix token estimation from 0.75 to 0.25 tokens/byte (~4 chars per token) in registry.rs, selector.rs, and standalone loader. Dependencies: - Replace deprecated serde_yaml 0.9 with serde_yml 0.0.12. - Remove unused toml dependency. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |