mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 07:30:11 +00:00
8a320ae9db4f7fdada609a30528bee6116cbe71c
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
8a320ae9db |
fix(routines): complete full_job execution reliability overhaul (#1650)
* fix(routines): persist full LLM transcript and remove sandbox gate for full_job Routine execution output was invisible — routine_fire returned a one-liner, routine_history had no actual output, and the conversation thread contained only a summary. Full-job routines also hard-failed without Docker. Three fixes: 1. **Full transcript persistence**: execute_lightweight now persists every message (prompt, LLM responses, tool calls with params, tool results) to the routine's conversation thread as it executes, not just a summary after the fact. 2. **Routine output visibility**: routine_history includes conversation_id and recent_output messages. routine_fire tells the user to check routine_history. Web detail page has a "View Execution Thread" button that navigates to the chat tab. ROUTINE_OK stores "No issues found" instead of None. Full-job summary pulls actual job output instead of generic "Job X finished". 3. **Remove SandboxReadiness gate**: full_job routines dispatch through the scheduler like regular /job commands — no Docker required. The SandboxReadiness enum is removed entirely. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: apply cargo fmt Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(worker): treat AutonomousUnavailable tool errors as recoverable The job worker crashed the entire job when a tool was denied for autonomous execution (e.g. secret_list). The error was already recorded in reason_ctx for the LLM to see, but process_tool_result_job returned Err which propagated through the agentic loop and terminated the job. Now all tool errors (including AutonomousUnavailable) return Ok, letting the LLM see the denial and try a different approach. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(llm): sanitize tool names for OpenAI Codex Responses API The Codex API requires tool names to match `^[a-zA-Z0-9_-]+$` but MCP/extension tools can have dots in their names (e.g. `mcp.server.tool`). This caused HTTP 400 errors when the job worker sent tool calls back to the LLM. Sanitize tool names in both `convert_tool_definition` and `convert_message` (function_call items) by replacing invalid characters with underscores. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(routines): inject execution context into full_job description [skip-regression-check] When a full_job routine dispatches a job, the LLM had no context that it was already executing inside a routine. It wasted iterations on infrastructure (discovering tools, creating routines, setting up auth) instead of doing the actual work. Prepend a clear directive to the job description telling the LLM that tools and the routine are already configured, and to execute the task directly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(mcp): auto-refresh expired OAuth tokens on access [skip-regression-check] When IronClaw restarts, MCP servers fail with "Secret has expired" because get_access_token() checks token expiry locally and returns an error before any HTTP request is made — so the existing 401-retry refresh logic never triggers. Now get_access_token() catches SecretError::Expired and automatically calls refresh_access_token() using the stored refresh token. If the refresh succeeds, the new token is returned transparently. If it fails, the error message includes both the expiry and the refresh failure. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(mcp): align refresh token naming and set expiry on stored tokens Two bugs prevented MCP OAuth token auto-refresh on restart: 1. Naming mismatch: the hosted OAuth flow stored the refresh token as `{token_secret_name}_refresh_token` (e.g. `mcp_notion_access_token_refresh_token`) but `McpServerConfig::refresh_token_secret_name()` returned `mcp_notion_refresh_token`. The refresh token was there but unfindable. 2. Missing expiry: `store_tokens` in auth.rs never called `with_expiry()` even though `AccessToken::expires_in` was available. Combined with the fix from the previous commit (auto-refresh on Expired), tokens stored via the MCP auth flow will now also trigger refresh correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): show activity and transitions for agent jobs in job detail [skip-regression-check] The job events endpoint only checked sandbox jobs for ownership, returning 404 for agent jobs dispatched from routines. The detail handler also returned empty transitions for agent jobs. - events handler: fall back to agent job ownership check - detail handler: populate transitions from job's state history Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(routines): expose max_iterations for full_job routines (default 25) The max_iterations parameter was hardcoded to 10 and not configurable via routine_create or routine_update, causing complex tasks to hit the iteration cap. - Add max_iterations to full_job execution schema (1-200, default 25) - Thread it through parse → build → RoutineAction - Support updating via routine_update - Raise default from 10 to 25 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(routines): break self-dialogue loop after full_job plan execution After plan execution, the completion-check Q&A ("Is the job complete?" / "No, not complete...") was left in the message context, causing the agentic loop to repeat the same analysis instead of calling tools. Replace the stale dialogue with an action-oriented continuation prompt that instructs the LLM to use tools for remaining work. Also strip <suggestions> tags from all job output since they're only meaningful for interactive chat sessions. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(repl): prevent test hang in single-message mode In single-message mode, start() stored a clone of the mpsc sender in self.msg_tx for approval injection. After the thread sent /quit and exited, the stored clone kept the stream alive, so stream.next() blocked forever in the test assertion that the stream ends. Skip storing the sender in single-message mode since interactive approval is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(jobs): treat text responses as final answer in agentic loop When the LLM produces a non-empty text response with no tool intent (already filtered by the nudge mechanism), it is the job's final answer. Previously, handle_text_response only exited the loop if the text matched rigid completion phrases like "job is complete". Natural summaries like "Weekly review completed and saved to Notion" were added to context and the loop continued, causing the LLM to restate the same summary until max_iterations was hit. Now any non-empty text response marks the job complete and stops the loop, matching the chat dispatcher behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * perf(tests): reduce skills catalog network failure test from 10s to 1s The test_search_returns_error_on_network_failure test connects to an unreachable RFC 5737 TEST-NET IP and waited for the full 10s production REQUEST_TIMEOUT. Add with_url_and_timeout test helper and use a 1s timeout instead. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(tools): accept 'message' as alias for 'content' in message tool LLMs frequently call the message tool with {"message": "..."} instead of {"content": "..."}. Fall back to the 'message' key when 'content' is missing to avoid InvalidParameters errors during autonomous job execution. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(tools): attach thread_id for gateway broadcast in message tool When the message tool broadcasts to all channels (channel=null), it sent an OutgoingResponse without a thread_id. The gateway silently dropped these messages (returned Ok but never sent the SSE event), so they appeared in repl but not in the web UI. The thread_id was only populated when channel was explicitly "gateway". Now it is always populated from notify_thread_id metadata, so broadcast_all delivers to the gateway correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(gateway): return error instead of silently dropping messages Gateway broadcast() and respond() previously returned Ok(()) when thread_id was missing, silently swallowing the message. Callers (message tool, agent loop) believed delivery succeeded when it didn't. Now returns ChannelError::MissingRoutingTarget so callers can detect and report the failure. Four regression tests verify the contract: respond/broadcast with and without thread_id. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: resolve rebase conflicts with staging Restore sandbox_readiness field removed by pre-rebase commits (staging still uses it). Update repl test to match staging's single-message behavior (no longer sends /quit). Add missing reasoning field to ToolCall in codex test. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(tools): log error when routine conversation lookup fails The routine_history tool silently swallowed errors from get_or_create_routine_conversation, returning empty output without any diagnostic logging. Add tracing::warn so failures are visible in logs. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR #1650 review comments - E2E test: accept submitted/accepted as success states in job assertion - TimeTool: remove operation from required schema (defaults to "now") - jobs handler: log DB errors server-side, return generic message to client - routines handler: use read-only find_routine_conversation on GET - codex provider: reverse-map sanitized tool names so MCP tools resolve Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address zmanian review feedback on PR #1650 - MCP refresh token: fall back to legacy secret name (mcp_{name}_refresh_token) so existing users don't need to re-authenticate after the naming fix - Job worker: replace fragile messages.pop() with truncate-to-saved-count to avoid maintenance hazard if message flow changes - Document cost implications of max_iterations 10->25 default bump - Revert Cargo.toml dist profile change (thin LTO comment, codegen-units=16) as it's unrelated to this PR Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: resolve rebase conflicts and address new Copilot comments - Fix no_silent_drop tests for updated GatewayConfig (user_id moved to GatewayChannel::new second arg, user_tokens removed) - Fix handle_text_response param name (_reason_ctx -> reason_ctx) - Fix missing has_text_response field in test JobDelegate - Propagate row.get errors in find_routine_conversation instead of unwrap_or_default - Only fall back to legacy refresh token name on NotFound/Expired, propagate real errors (DB, decryption) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
f059d50331 |
fix: preserve AuthError type in oauth_http_client cache (#1152)
* fix(mcp): cache oauth client init error as AuthError * Update src/tools/mcp/auth.rs Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix(mcp): use AuthError::Http in oauth client cache and add regression test * test(mcp): annotate test assert for no-panics CI matcher --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> |
||
|
|
62d16e69ac |
fix(mcp): handle 400 auth errors, clear auth mode after OAuth, trim tokens (#1158)
* fix(mcp): handle 400 auth errors, clear auth mode after OAuth, trim tokens Three bugs prevented MCP server authentication (e.g. GitHub MCP) from working correctly: 1. **400 treated as auth-required**: GitHub's MCP endpoint returns 400 "Authorization header is badly formatted" instead of 401 when auth is missing. Broadened auth detection in activate_mcp, send_request, and discover_via_401 to also match 400+authorization errors. 2. **Auth mode not cleared after OAuth callback**: The OAuth callback handler and setup submit handler did not call clear_auth_mode(), leaving pending_auth on the thread. The next user message was intercepted as a token instead of triggering an LLM turn. 3. **Token trimming**: Tokens with leading/trailing whitespace or newlines produced malformed Authorization headers. Now trimmed before storage (configure) and before use (build_request_headers). Adds E2E tests with a mock MCP server (JSON-RPC + OAuth discovery + DCR + token exchange) covering install -> activate -> OAuth callback -> LLM turn lifecycle, plus a GitHub-style 400 error variant. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(mcp): add TTL to PendingAuth and clear auth mode on all failure paths Auth mode (pending_auth on a Thread) had no timeout and several code paths that failed to clear it, causing user messages to be swallowed indefinitely. This adds defense-in-depth: - Add created_at + 5-minute TTL to PendingAuth; auto-clear on next message if expired (safety net for edge cases like user closing browser mid-OAuth) - Clear auth mode on OAuth callback failure paths (unknown/consumed state, expired flow) - Move clear_auth_mode before configure() match in setup_submit so it runs on failure too (addresses Copilot review feedback) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(ci): exclude test hunks from unwrap/assert pre-commit check The pre-commit safety script only excluded files in tests/ but not #[cfg(test)] mod tests blocks inside src/ files. Use the git diff @@ hunk header context (which includes the enclosing function name) to detect and skip test hunks. Also removes unnecessary // safety: comments from test assertions. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: restore formatting in test assertions The replace_all edit that removed // safety: comments collapsed newlines. Restore proper line breaks. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address Copilot review - tighten pre-commit filter, document TTL sync - pre-commit-safety.sh: only exclude `mod tests` hunks (not `fn test_*`) to avoid hiding unwrap/assert in production functions like test_server() - session.rs: extract AUTH_MODE_TTL_SECS constant and add doc comment linking to OAUTH_FLOW_EXPIRY to prevent silent drift [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(mcp): return error on expired auth input, clear auth on all OAuth paths - When auth mode TTL expires and the user sends a message (possibly a pasted token), return an explicit "expired, please retry" response instead of forwarding the content to the LLM/history - Add clear_auth_mode() to all early-return paths in oauth_callback_handler (provider error, missing state/code, no extension manager) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
f53c1bb10b |
fix(mcp): address 14 audit findings across MCP module (#1094)
* fix(mcp): address 14 audit findings across MCP module - Replace panicking assert! in new_with_config with Result return (Critical) - Fix initialize() race condition using tokio::sync::OnceCell (High) - Fix localhost check bypass via proper URL parsing (High) - Extract shared stream_transport_send() to deduplicate stdio/unix send logic - Use atomic write (tmp+rename) for config file persistence - Filter SSE responses by request_id to prevent wrong-response dispatch - Share a single reqwest::Client for OAuth via fallible OnceLock - Log notification send errors instead of silently discarding - Fix unwrap_or(0) that could steal id=0 responses - Store InitializeResult in OnceCell so callers can access server capabilities - Add redirect logging in OAuth discovery - Reuse is_localhost_url() in auth.rs - Add McpToolWrapper unit tests and regression tests - URL-encode PKCE challenge for consistency Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: retrigger CI with skip-regression-check label Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
8ac24e775b |
style: fix formatting in cli/mod.rs and mcp/auth.rs (#1071)
* style: fix formatting in cli/mod.rs and mcp/auth.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(cli): add missing use_tools and max_tool_rounds fields to routines create The routines CLI create command was missing the new Lightweight fields added after the cron->routines rename merged. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(clippy): move path_routing_tests after production code in memory.rs Fixes items_after_test_module lint by moving the test module to the end of the file, after all production structs and impls. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
nearfamiliarcowandGitHub
|
4faf81ab61 |
fix(mcp): include OAuth state parameter in authorization URLs (#1049)
Some MCP servers (e.g. Attio) require the `state` parameter in OAuth
authorization requests and reject requests without it:
{"error":"invalid_request","error_description":"Invalid value provided for: state"}
While OAuth 2.1 makes `state` optional when PKCE is used, the MCP
specification does not forbid servers from requiring it. This caused a
hard failure when authenticating with any MCP server that enforces the
state parameter.
Generate a 128-bit cryptographically random state (via OsRng, base64url
encoded without padding) and inject it into extra_params before building
the authorization URL. This covers both pre-configured OAuth and Dynamic
Client Registration (DCR) code paths.
The callback listener intentionally does not validate the echoed state
because: (1) PKCE already binds the authorization code to the token
exchange, preventing code injection attacks, and (2) not all MCP servers
echo state back — strict validation would break those servers. Other
OAuth flows in the codebase (tool.rs, extensions/manager.rs) that
generate and validate state are unaffected.
|
||
|
|
02f85a8ad5 |
feat(mcp): transport abstraction, stdio/UDS transports, and OAuth fixes (#721)
* feat(mcp): transport abstraction, stdio/UDS transports, and OAuth fixes Extract McpTransport trait from HTTP-coupled McpClient, enabling pluggable transport backends. Implements stdio and Unix domain socket transports for local MCP server integration, fixes OAuth discovery per RFC 9728, and adds SSRF protection. Transport abstraction (Step 2): - McpTransport trait with send(), shutdown(), supports_http_features() - HttpMcpTransport extracted from McpClient with SSE parsing, session tracking - Shared JSON-RPC framing helpers (write_jsonrpc_line, spawn_jsonrpc_reader) - McpClient refactored to hold Arc<dyn McpTransport> Stdio transport (#652, Step 4): - StdioMcpTransport spawns child process, communicates via stdin/stdout - McpProcessManager for lifecycle management with exponential backoff restart - Background stderr drain task for debug logging Unix domain socket transport (#134, Step 5): - UnixMcpTransport connects to existing Unix sockets - Reuses shared JSON-RPC framing from transport.rs HTML error body sanitization (#263, Step 1): - sanitize_error_body() detects HTML, strips control chars, truncates to 500 Custom headers (#639, Step 3): - headers field on McpServerConfig, merged into every HTTP request - --header CLI arg for `mcp add` Config and CLI updates (Step 6): - McpTransportConfig tagged enum (Http/Stdio/Unix) with serde support - EffectiveTransport for zero-copy config dispatch - CLI: --transport, --command, --arg, --env, --socket flags for `mcp add` - `mcp list` shows transport type OAuth fixes (#299, Step 8): - Multi-strategy discovery (401-based, RFC 9728, direct) - RFC 8707 resource parameter in auth and refresh flows - SSRF protection with IPv4-mapped IPv6 bypass detection - Well-known URI construction per RFC 8414 Closes #652, #134, #639, #263, #299 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(mcp): address audit findings from crate review - Fix SSRF bypass: make validate_url_safe async with DNS resolution to block hostnames that resolve to private/link-local IPs - Fix UTF-8 truncation: use char-based truncation in sanitize_error_body to avoid panicking on multi-byte characters - Fix SSE parser: process only complete lines to handle chunks split across boundaries, add 10MB buffer size limit - Add debug_assert for transport type mismatch in new_with_config - Propagate custom headers in new_with_transport constructor - Deduplicate effective_transport() calls in CLI list command - Gate test-only accessors with #[cfg(test)] to eliminate dead_code warnings - Document JSON-RPC notification id:0 limitation in protocol.rs - Document total backoff wait time (31s) in process.rs - Add regression test for multi-byte UTF-8 truncation Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(mcp): address PR review findings from Copilot, Gemini, and zmanian Moderate/High fixes: - Plumb custom headers through new_authenticated constructor - Restrict HTTP to localhost only in validate_url_safe (prevent plaintext credential leaks over non-localhost HTTP) - Add mcp_process_manager.shutdown_all() to app shutdown path to prevent orphaning stdio child processes - Validate discovered authorization_url before opening browser (prevent malicious MCP server redirecting to phishing page) Medium fixes: - Upgrade debug_assert to assert in new_with_config (fires in release) - Remove pending map entry on Ok(Err(_)) in stdio/unix send() to avoid stale entries and unnecessary 30s waits - Shut down old transport in try_restart() before spawning replacement - Redact env var values in mcp list --verbose (may contain secrets) - Drain pending requests on shutdown to wake waiters immediately - Add IPv6 link-local, site-local, unique-local, and documentation ranges to is_dangerous_ip SSRF protection Low fixes: - Truncate logged JSON parse error lines to 200 chars (prevent sensitive data in logs) - Remove misleading shutdown comment in unix_transport - Use tempfile::tempdir() instead of hardcoded /tmp/ path in test - Adopt main's improved sanitize_error_body (HTML tag stripping, 200-char truncation with char_indices) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(mcp): gate unix_transport with #[cfg(unix)] for Windows compat - Add #[cfg(unix)] to unix_transport module declaration - Add #[cfg(unix)]/#[cfg(not(unix))] branches in app.rs for Unix socket MCP server setup - Remove unused sanitize_error_body import in client.rs tests [skip-regression-check] --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
cf96a3253c |
fix(tests): replace hardcoded /tmp paths with tempdir + add 300 unit tests (#659)
* test: add unit tests across 20 modules for coverage push Add 300+ unit tests covering config, context, evaluation, extensions, LLM, secrets, tools/builder, and tools/mcp modules. All tests are pure unit tests (no mocks) exercising serde roundtrips, edge cases, error paths, and business logic. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(tests): replace hardcoded /tmp paths with tempfile::tempdir The e2e_metrics_test::test_metrics_collected_from_tool_trace test was failing because setup_test_dir() created /tmp/ironclaw_metrics_test but the fixture referenced /tmp/ironclaw_e2e_test/hello.txt (path mismatch). Added LlmTrace::replace_paths() to substitute fixture paths at runtime, then converted all 12 test files from hardcoded /tmp/ironclaw_* paths to tempfile::tempdir(). Tests are now isolated, parallel-safe, and leave no debris on disk. Regression test: test_metrics_collected_from_tool_trace now passes consistently regardless of prior /tmp state. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
6a2a6cd050 |
fix(security): use OsRng for all security-critical key and token generation (#519)
* fix(security): use OsRng for all security-critical key and token generation Replace rand::thread_rng() with rand::rngs::OsRng in all security-critical code paths that generate cryptographic key material, bearer tokens, PKCE verifiers, CSRF state parameters, and webhook secrets. thread_rng() uses a userspace CSPRNG (ChaCha) seeded from OS entropy, which is fine for non-security contexts but adds an unnecessary intermediate layer for key material where direct OS entropy (OsRng) is the correct choice. Files changed: - src/secrets/keychain.rs: master encryption key generation - src/secrets/crypto.rs: per-secret HKDF salt generation - src/orchestrator/auth.rs: per-job bearer token generation - src/channels/web/mod.rs: gateway auth token fallback - src/cli/oauth_defaults.rs: OAuth PKCE verifier and CSRF state - src/tools/mcp/auth.rs: MCP OAuth PKCE verifier - src/extensions/manager.rs: auto-generated extension secrets - src/setup/channels.rs: webhook secret generation Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix(security): address PR review feedback for OsRng migration - Remove shadowing inner `use rand::rngs::OsRng` in `generate_salt()`; use module-level `aes_gcm::aead::OsRng` import instead (same type, avoids divergence risk if rand_core versions drift) - Fix missed callsites in `pairing/store.rs`: `random_code()` and `generate_unique_code()` now use `OsRng` for pairing auth codes - Add regression tests for `generate_salt()`: correct length, non-zero output, uniqueness across calls Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> |
||
|
|
18b59ae9a7 |
feat: add OAuth support for WASM tools in web gateway (#489)
* feat: add OAuth support for WASM tools in web gateway Extract reusable OAuth functions (build_oauth_url, exchange_oauth_code, store_oauth_tokens, validate_oauth_token) from CLI into shared oauth_defaults module, then wire them into the web gateway's ExtensionManager. Key changes: - Install auto-activates WASM tools (no separate Activate button) - Configure button triggers OAuth flow via save_setup_secrets - Scope merging: installing a second Google tool triggers re-auth with merged scopes from all tools sharing the same secret_name - Cancel-and-retry: aborting stale OAuth listeners prevents port conflicts - Post-auth validation: wrong account detected via validation_endpoint - Reconfigure always re-auths (deletes old token before starting fresh) - UI shows error toast on OAuth failure, refreshes extension list Flow: Install → Active → Configure (enter client_id/secret) → Save → OAuth popup → authorize → done. Second Google tool install auto-triggers scope expansion OAuth. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments - Add custom headers support to ValidationEndpointSchema (fixes missing Notion-Version header regression) - Guard activate handler auth check with status == "awaiting_authorization" to prevent unexpected OAuth popups - Add window dimensions to OAuth popup in activateExtension() - Simplify UTF-8 truncation boundary check Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address Copilot PR review comments (security, UX, bugs) - Add CSRF state parameter to OAuth flow (random state in auth URL, validated in callback) - Restore MCP server Activate button in web UI (was hidden for all non-channel extensions) - Abort JoinHandle in cleanup_expired_auths to prevent port 9876 conflicts - Fix Google-specific error message for non-Google OAuth providers - Add has_auth field to ExtensionInfo API response (fixes Configure button visibility) - Use oauth_defaults::callback_url() instead of hardcoded redirect_uri (both CLI and manager) - Update auth check comment to match actual behavior (scope expansion + first-time auth) - Add unit tests for build_oauth_url (basic, PKCE, extra params, state uniqueness) - Check all required setup secrets (client_id + client_secret) before starting OAuth Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
37c0158765 |
Fix: allow OAuth callback to work on remote servers (fixes #186) (#212)
* fix: allow OAuth callback to work on remote servers via OAUTH_CALLBACK_HOST Fixes #186. The OAuth callback URL was hardcoded to `http://127.0.0.1:9876` in two places (NEAR AI login and MCP server auth). On a remote server this URL is unreachable from the user's browser, making authentication impossible. Changes: - Add `callback_host()` to `oauth_defaults` that reads `OAUTH_CALLBACK_HOST` (default: `127.0.0.1`) - Update `bind_callback_listener()` to bind to `0.0.0.0` when a non-loopback host is configured, so the port is reachable from outside the machine - Update `session.rs` and `mcp/auth.rs` to use `callback_host()` instead of hardcoded `127.0.0.1` / `localhost` Usage on a remote server: export OAUTH_CALLBACK_HOST=<your-server-ip> ironclaw login * fix: address PR review comments for OAuth callback security * fix: address serrrfirat review comments on PR #212 --------- Co-authored-by: firat.sertgoz <[email protected]> |
||
|
|
a53b2c10b5 |
fix: Fix wasm tool schemas and runtime (#42)
* feat: Move debug log truncation from agent loop to REPL channel Full tool output now flows through StatusUpdate so the web gateway gets untruncated content. The REPL channel truncates at display time (200 chars for tool results, thinking, and status messages). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Flatten WASM tool schemas and fix host HTTP runtime contention LLMs can't reliably follow oneOf + const discriminator patterns in JSON Schema, causing tools like Google Calendar to receive malformed params (e.g., {"operation":"list_events","data":{"calendarId":"primary"}} instead of {"action":"list_events","calendar_id":"primary"}). Replace all 9 WASM tool schemas with flat action enum + top-level properties. The serde #[serde(tag = "action")] deserialization works identically. Also fixes WASM host HTTP requests (channels and tools) stalling during startup by replacing Handle::current().block_on() with a dedicated single-threaded runtime per request, avoiding I/O driver contention. Reduces verbose LLM debug logging (full request/response payloads) and changes tower_http default from debug to warn. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Built-in OAuth credentials and combined Google scopes Add infrastructure for shipping default OAuth credentials with the binary, similar to how gcloud/rclone bake in their client_id. Credentials are set at compile time via IRONCLAW_GOOGLE_CLIENT_ID / IRONCLAW_GOOGLE_CLIENT_SECRET env vars, or can be hardcoded in src/cli/oauth_defaults.rs. The fallback chain is: capabilities file > runtime env var > built-in defaults. Also, when authing any Google tool, scopes from ALL installed Google tools are now combined into a single OAuth request (they all share the same google_oauth_token secret). One login covers Gmail, Calendar, Drive, etc. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Ship default Google OAuth credentials for zero-config auth Google Desktop App credentials are not secret (per Google's own docs). Hardcode them so `ironclaw tool auth <google-tool>` works out of the box without requiring users to register their own OAuth app. Credentials can still be overridden at compile time (IRONCLAW_GOOGLE_CLIENT_ID) or runtime (GOOGLE_OAUTH_CLIENT_ID). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Consistent OAuth callback port and polished landing page - Use fixed port 9876 instead of scanning 9876-9886 (one redirect URI to register in provider OAuth apps, deterministic behavior) - Replace broken unicode checkmark with SVG icons (charset was missing, rendered as mojibake) - Dark themed landing page with proper card layout for both success and error states - Add charset=utf-8 to Content-Type headers Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Unify OAuth callback server across all auth flows All three OAuth flows (WASM tool auth, MCP server auth, NEAR AI login) now share the same code from cli::oauth_defaults: - Fixed port 9876 (one redirect URI to register per provider) - Shared landing page HTML (dark card with SVG icons, proper charset) - Parameterized wait_for_callback(listener, path, param, display_name) Removes ~120 lines of duplicated callback/HTML code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Support for oauth token refresh * refactor: Replace bootstrap.json with ~/.ironclaw/.env for DATABASE_URL Kill the 4-field BootstrapConfig JSON file. Only DATABASE_URL actually needs disk persistence (chicken-and-egg before DB connect). The other three fields are now derived: pool_size defaults to 10 via env var, secrets master key is auto-detected (env then keychain probe), and onboard_completed is inferred from DATABASE_URL presence. The new format is a standard .env file loaded via dotenvy early in main, so DATABASE_URL is available as a regular env var everywhere. Handles three upgrade paths: - Clean start: wizard writes .env, reload after wizard completes - Returning user: .env loaded at startup, business as usual - Legacy upgrade: bootstrap.json auto-migrated to .env on first run Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review findings - Fix UTF-8 panic in truncate_for_preview (byte-slice on char boundary) - Cap WASM guest timeout_ms at 5 minutes to prevent resource exhaustion - Fix localhost detection in requires_auth() to avoid substring matches (e.g. "notlocalhost.com" no longer matches) - Fix query param injection to insert before URL fragment - Fix extract_host_from_url for IPv6 bracket notation - Remove misleading schema defaults: Slack limit, Slides insertion_index, Docs index (per-action defaults documented in descriptions instead) Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Fix cargo fmt formatting Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: IPv6 loopback support for OAuth listener and localhost detection - bind_callback_listener: try [::1] first, fall back to 127.0.0.1, so OAuth redirects work on systems where localhost resolves to ::1 - is_localhost_url: replace manual string parsing with url::Url for correct handling of IPv6 brackets, ports, userinfo, etc. - Add url crate as direct dependency (already a transitive dep) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review feedback on runtime reuse, onboard check, and OAuth binding - Remove session file check from check_onboard_needed(); DATABASE_URL is sufficient - Detect AddrInUse on IPv6 bind and fail immediately instead of falling through to IPv4 - Reuse dedicated tokio runtime across HTTP calls in both tool and channel WASM wrappers Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: HTML-escape provider name in OAuth landing page, simplify Slack limit description - Add html_escape() to prevent XSS in landing_html() where provider_name was interpolated directly into HTML (defense-in-depth, source is trusted but escaping costs nothing) - Remove per-action default numbers from Slack limit field description to avoid confusing LLMs with conflicting defaults Addresses review feedback from zmanian on PR #42. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Save all bootstrap fields from wizard, fix config module comment - Wizard now saves secrets_master_key_source and database_pool_size to bootstrap.json (was only saving database_url and onboard_completed, which broke secrets after fresh onboard since SecretsConfig::resolve reads key source from bootstrap) - Update config.rs module doc to reflect bootstrap.json priority chain instead of the removed ~/.ironclaw/.env approach Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Replace BootstrapConfig with .env-based bootstrap DATABASE_URL is the only setting that needs disk persistence before the database is available. Instead of a custom bootstrap.json with 4 fields, use a standard ~/.ironclaw/.env file loaded via dotenvy. - Remove BootstrapConfig struct entirely - Restore ironclaw_env_path(), load_ironclaw_env(), save_database_url() - SecretsConfig::resolve() now auto-detects (env var then keychain probe) instead of reading a saved source from bootstrap.json - DatabaseConfig::resolve() reads DATABASE_URL from env only (dotenvy loads ~/.ironclaw/.env into the environment early in startup) - check_onboard_needed() is now sync (just checks env vars) - Wizard save_and_summarize() works for both postgres and libsql backends - One-time migration from bootstrap.json to .env preserved Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Ensure load_ironclaw_env() runs in all Config paths, fix .env priority - Config::from_env() and Config::from_db() now call load_ironclaw_env() internally (after dotenvy::dotenv()), so CLI commands like `memory` and `config` correctly load DATABASE_URL from ~/.ironclaw/.env - Fix load order: standard ./.env first (higher priority), then ~/.ironclaw/.env, matching the documented priority chain - Collapse nested if/if-let into let-chains (clippy::collapsible_if) in oauth_defaults.rs, tool.rs, and secrets/store.rs - Fix rename_to_migrated to take &Path instead of &PathBuf Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review comments (quoting, SSRF, error mapping) - Quote DATABASE_URL in .env writes so `#` in passwords isn't treated as a dotenv comment (e.g., `DATABASE_URL="postgres://..."`) - Add SSRF defenses to refresh_oauth_token(): require HTTPS, reject private/loopback IPs (with DNS resolution), disable redirects. token_url comes from tool capabilities JSON, so a malicious tool could otherwise exfiltrate refresh tokens. - Fix IPv4 bind error mapping: only map AddrInUse to PortInUse, use generic Io variant for other bind failures Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
5df0d13b59 |
Bump MSRV to 1.92, add GCP deployment files (#40)
* Bump MSRV to 1.92 and add GCP deployment files rig-core 0.30 uses let_chains (stabilized post-1.87), which breaks builds on Rust 1.85. Bump rust-version in Cargo.toml and both Dockerfiles to 1.92 (verified working). Add cloud deployment scaffolding: - Dockerfile: multi-stage build for the main agent container - deploy/cloud-sql-proxy.service: systemd unit for Cloud SQL Auth Proxy - deploy/ironclaw.service: systemd unit for the IronClaw container - deploy/setup.sh: VM bootstrap script (Docker, proxy, services) - deploy/env.example: reference environment configuration Co-Authored-By: Claude Opus 4.6 <[email protected]> * Address review feedback: harden deploy scaffolding - Add comment explaining GATEWAY_HOST=0.0.0.0 and when to use 127.0.0.1 - Document /opt/ironclaw ownership model (root-owned, Docker reads as root) - Switch cloud-sql-proxy service from User=root to DynamicUser=yes Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (Rust 1.93) and fix CI test workflow - Fix 97 collapsible_if warnings using let-chains syntax (auto-fixed) - Fix ptr_arg: change &PathBuf to &Path in pairing store functions - Fix suspicious_open_options: add .truncate(false) to OpenOptions - Fix too_many_arguments: add clippy allow on execute_status - Fix unnecessary_unwrap: use if-let in repository.rs hybrid_search - Gate unused EchoTool with #[cfg(test)] - Add PairingStore argument to ChannelStoreData::new() test call sites - Add skip guard for bundled channel test when WASM artifacts unavailable - Split CI test workflow to exclude PostgreSQL-dependent integration tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address review feedback from ilblackdragon - Add root check to setup.sh (exits with error if not root) - Add warning comment to env.example about placeholder passwords - Dockerfile.worker already uses rust:1.92 (no change needed) - PR #41 overlap noted; will rebase after #41 merges Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: resolve 47 collapsible_if clippy warnings Collapse nested if statements across the codebase to satisfy clippy::collapsible_if on Rust 1.93. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.6
|
ae3c86a7ea |
Add in-chat extension discovery, auth, and activation system
Introduces a unified extension abstraction over MCP servers and WASM tools with six agent-callable tools (tool_search, tool_install, tool_auth, tool_activate, tool_list, tool_remove) so users can add capabilities conversationally without CLI commands. Includes built-in registry of 11 MCP servers, online discovery via URL probing and GitHub search, OAuth 2.1 flows for MCP servers, and manual token auth for WASM tools. Co-Authored-By: Claude Opus 4.6 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.5
|
974bc8d407 |
Add hosted MCP server support with OAuth 2.1 and token refresh
Enables connecting to official MCP servers (like Notion) instead of building custom WASM tools. Uses OAuth 2.1 with PKCE and supports Dynamic Client Registration for zero-config authentication. Key features: - OAuth 2.1 flow with PKCE for secure browser-based auth - Dynamic Client Registration (DCR) for servers without pre-configured clients - Automatic token refresh on 401 responses - Session management with Mcp-Session-Id headers - SSE streaming response handling New CLI commands: - `mcp add <name> <url>` - Add an MCP server - `mcp remove <name>` - Remove an MCP server - `mcp list` - List configured servers - `mcp auth <name>` - Authenticate with a server - `mcp test <name>` - Test connection Also removes the Notion WASM tool example since it's superseded by the Notion MCP server which provides 13 official tools. Co-Authored-By: Claude Opus 4.5 <[email protected]> |