6 Commits

Include renames

Author	SHA1	Message	Date
3 people	f0a0642e7d	feat: multi-provider inference + libSQL onboarding selection (#92) ... * feat: add interactive database backend selection during onboarding Previously the onboarding wizard silently defaulted to PostgreSQL because libsql wasn't in the default feature set. Now both backends ship by default and the wizard presents a selection prompt when both are available. DATABASE_BACKEND env var still bypasses the prompt for headless/CI use. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: resolve libSQL onboarding crash, keychain double-prompt, and setup audit findings Three bugs fixed: 1. libSQL onboarding crash ("Missing required setting 'database_url'"): DatabaseConfig::resolve() only checked DATABASE_BACKEND env var, falling back to Postgres default. Now reads settings.database_backend, plus settings.libsql_path and settings.libsql_url as fallbacks. 2. OS keychain prompts twice during startup: Config::from_env() and Config::from_db() both called get_master_key(). Now caches the key in SECRETS_MASTER_KEY env var after first read so from_db() skips keychain. 3. "Path not found: nearai.session" warning: from_db_map() tried to apply app-specific DB keys (nearai.session_token) to the Settings struct. Now skips keys that don't map to known Settings fields. Also fixed bootstrap migration key mismatch (nearai.session -> nearai.session_token). Setup module audit fixes (14 findings): - Replace unreachable!() with proper error in provider match - Extract setup_api_key_provider() to deduplicate setup_anthropic/setup_openai - Add SAFETY comments to all unsafe std::env::set_var blocks - Fix .unwrap() calls with proper error handling - Remove incorrect #[allow(dead_code)] on used TelegramUpdate::update_id - Log warnings instead of silently discarding HTTP errors in Telegram binding - Guard select_many against empty options, fix mask_api_key for non-ASCII - Update stale doc comment in mod.rs, rename misleading variable - Add 7 new tests (model fetcher fallbacks, channel discovery, secret gen) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback (set_var safety, parse warnings, db_map efficiency) 1. Replace unsafe set_var keychain caching with OnceLock<String> in SecretsConfig::resolve(). Eliminates the env var write from main.rs entirely, using a process-wide OnceLock cache instead. 2. Log tracing::warn when database_backend or llm_backend settings fail to parse, instead of silently falling back to defaults. 3. Remove O(K*S) get() pre-check in from_db_map(). Instead, let set() run and match on "Path not found" errors to skip unknown keys, avoiding full Settings serialization per key. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address critical/high audit findings across WASM sub-crates - Telegram: remove .unwrap() panic on workspace_read (owner_id check) - WhatsApp: use configured api_version instead of hardcoded v18.0 - WhatsApp: log config parse errors before falling back to defaults - Slack: log serialization errors in emit_message and json_response - Google Docs: safe array access for batch update replies - Google Sheets: safe array access for add_sheet replies - Google Calendar: fix doc comment secret name mismatch - Gmail: avoid unnecessary String allocation in UNREAD check Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second-round PR review feedback - Validate custom model ID is non-empty (loop until valid input) - Warn on unknown DATABASE_BACKEND env var before defaulting to Postgres - Force re-selection when llm_backend contains unknown provider value - Use ok_or_else for proper String error type in google-sheets Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden setup module error handling and secret safety - Introduce ChannelSetupError typed enum replacing raw String errors across all channel setup functions (setup_telegram, setup_http, setup_tunnel, setup_wasm_channel, validate_telegram_token) - Add From<ChannelSetupError> for SetupError to simplify call sites - Convert setup_telegram retry from recursion to loop (unbounded stack) - Stop printing HTTP webhook secret plaintext to terminal - Use secret_input() for Turso auth token (was visible input()) - Replace dirs::home_dir().unwrap_or_default() with proper error - Fix UTF-8 panic in model name truncation (byte-index to chars-based) - Log warning in secret_exists() instead of silently swallowing errors - Deduplicate generate_webhook_secret() to delegate to shared helper Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: replace unreachable!() with error return in setup wizard The provider match in step_inference_provider was guarded by is_known but used unreachable!() as the catch-all. If a new provider is added to the is_known check without a corresponding match arm, this would panic at runtime. Return a typed error instead. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: remove unsafe set_var, use thread-safe overlay for injected secrets Address PR #92 review comments: - Replace all 5 unsafe `std::env::set_var()` calls with safe alternatives - Add INJECTED_VARS OnceLock<HashMap> overlay in config.rs, checked by optional_env() before falling back to std::env::var() - Cache wizard API key in SetupWizard.llm_api_key field instead of env - Pass explicit key param to fetch_anthropic_models/fetch_openai_models - Persist env-provided API keys to secrets store during onboarding Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining PR review comments (clippy, TODO, secrets backend ordering) - Fix empty line after doc comment (clippy: empty_line_after_doc_comments) - Collapse nested if in optional_env overlay check (clippy: collapsible_if) - Remove dangling TODO(#XX) placeholder issue ref in channels.rs - Fix init_secrets_context to respect selected database_backend when both postgres and libsql features are compiled, preventing wrong-backend secrets storage when DATABASE_URL is set but libsql was chosen Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address latest PR review comments (SecretString, empty env, docs, embeddings) - Change wizard llm_api_key from String to SecretString to prevent accidental logging of API keys - Fix inject_llm_keys_from_secrets skipping when env var is set but empty, matching optional_env's treatment of empty as unset - Fix inverted doc comment on INJECTED_VARS (env checked first, overlay is the fallback, not the other way around) - Update stale "env vars" comments in main.rs to reflect overlay pattern - Fix step_embeddings not seeing cached OpenAI key from wizard session Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: OAuth callback listener binds IPv4 first to match redirect URLs The listener was binding to [::1] (IPv6) first, but NEAR AI and other OAuth flows redirect to http://127.0.0.1:9876/... (IPv4 explicit). On macOS and most systems, [::1] and 127.0.0.1 are separate addresses, so the browser's connection to 127.0.0.1 was refused when the listener was on [::1]. Reversed the bind order: try 127.0.0.1 first, fall back to [::1] if IPv4 is unavailable. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: cache keychain key eagerly to avoid redundant macOS password dialogs Replace has_master_key() with get_master_key() in step_security() and immediately build SecretsCrypto from the result. This eliminates redundant keychain accesses later in init_secrets_context(), each of which triggers macOS system dialogs (keychain unlock + app authorization). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: persist DATABASE_BACKEND to ~/.ironclaw/.env for libSQL startup The wizard saved database_backend only to the database, but Config::from_env() needs it BEFORE connecting to any database (to decide which backend to use). Without it, the backend defaults to Postgres and then fails with "Missing required setting database_url". Now save all database bootstrap vars (DATABASE_BACKEND, DATABASE_URL, LIBSQL_PATH, LIBSQL_URL) to ~/.ironclaw/.env via save_bootstrap_env(). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: status command shows libSQL backend and skips keychain probe The status command only checked DATABASE_URL (postgres), showing "not configured" for libSQL users. Now detects the DATABASE_BACKEND env var and reports libSQL path and Turso sync status. Also remove the keychain probe from status. get_generic_password() triggers macOS unlock+authorization dialogs which is terrible UX for a read-only diagnostic command. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt formatting in bootstrap test Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-15 08:24:51 +00:00
3 people	a53b2c10b5	fix: Fix wasm tool schemas and runtime (#42) ... * feat: Move debug log truncation from agent loop to REPL channel Full tool output now flows through StatusUpdate so the web gateway gets untruncated content. The REPL channel truncates at display time (200 chars for tool results, thinking, and status messages). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Flatten WASM tool schemas and fix host HTTP runtime contention LLMs can't reliably follow oneOf + const discriminator patterns in JSON Schema, causing tools like Google Calendar to receive malformed params (e.g., {"operation":"list_events","data":{"calendarId":"primary"}} instead of {"action":"list_events","calendar_id":"primary"}). Replace all 9 WASM tool schemas with flat action enum + top-level properties. The serde #[serde(tag = "action")] deserialization works identically. Also fixes WASM host HTTP requests (channels and tools) stalling during startup by replacing Handle::current().block_on() with a dedicated single-threaded runtime per request, avoiding I/O driver contention. Reduces verbose LLM debug logging (full request/response payloads) and changes tower_http default from debug to warn. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Built-in OAuth credentials and combined Google scopes Add infrastructure for shipping default OAuth credentials with the binary, similar to how gcloud/rclone bake in their client_id. Credentials are set at compile time via IRONCLAW_GOOGLE_CLIENT_ID / IRONCLAW_GOOGLE_CLIENT_SECRET env vars, or can be hardcoded in src/cli/oauth_defaults.rs. The fallback chain is: capabilities file > runtime env var > built-in defaults. Also, when authing any Google tool, scopes from ALL installed Google tools are now combined into a single OAuth request (they all share the same google_oauth_token secret). One login covers Gmail, Calendar, Drive, etc. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Ship default Google OAuth credentials for zero-config auth Google Desktop App credentials are not secret (per Google's own docs). Hardcode them so `ironclaw tool auth <google-tool>` works out of the box without requiring users to register their own OAuth app. Credentials can still be overridden at compile time (IRONCLAW_GOOGLE_CLIENT_ID) or runtime (GOOGLE_OAUTH_CLIENT_ID). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Consistent OAuth callback port and polished landing page - Use fixed port 9876 instead of scanning 9876-9886 (one redirect URI to register in provider OAuth apps, deterministic behavior) - Replace broken unicode checkmark with SVG icons (charset was missing, rendered as mojibake) - Dark themed landing page with proper card layout for both success and error states - Add charset=utf-8 to Content-Type headers Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Unify OAuth callback server across all auth flows All three OAuth flows (WASM tool auth, MCP server auth, NEAR AI login) now share the same code from cli::oauth_defaults: - Fixed port 9876 (one redirect URI to register per provider) - Shared landing page HTML (dark card with SVG icons, proper charset) - Parameterized wait_for_callback(listener, path, param, display_name) Removes ~120 lines of duplicated callback/HTML code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Support for oauth token refresh * refactor: Replace bootstrap.json with ~/.ironclaw/.env for DATABASE_URL Kill the 4-field BootstrapConfig JSON file. Only DATABASE_URL actually needs disk persistence (chicken-and-egg before DB connect). The other three fields are now derived: pool_size defaults to 10 via env var, secrets master key is auto-detected (env then keychain probe), and onboard_completed is inferred from DATABASE_URL presence. The new format is a standard .env file loaded via dotenvy early in main, so DATABASE_URL is available as a regular env var everywhere. Handles three upgrade paths: - Clean start: wizard writes .env, reload after wizard completes - Returning user: .env loaded at startup, business as usual - Legacy upgrade: bootstrap.json auto-migrated to .env on first run Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review findings - Fix UTF-8 panic in truncate_for_preview (byte-slice on char boundary) - Cap WASM guest timeout_ms at 5 minutes to prevent resource exhaustion - Fix localhost detection in requires_auth() to avoid substring matches (e.g. "notlocalhost.com" no longer matches) - Fix query param injection to insert before URL fragment - Fix extract_host_from_url for IPv6 bracket notation - Remove misleading schema defaults: Slack limit, Slides insertion_index, Docs index (per-action defaults documented in descriptions instead) Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Fix cargo fmt formatting Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: IPv6 loopback support for OAuth listener and localhost detection - bind_callback_listener: try [::1] first, fall back to 127.0.0.1, so OAuth redirects work on systems where localhost resolves to ::1 - is_localhost_url: replace manual string parsing with url::Url for correct handling of IPv6 brackets, ports, userinfo, etc. - Add url crate as direct dependency (already a transitive dep) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review feedback on runtime reuse, onboard check, and OAuth binding - Remove session file check from check_onboard_needed(); DATABASE_URL is sufficient - Detect AddrInUse on IPv6 bind and fail immediately instead of falling through to IPv4 - Reuse dedicated tokio runtime across HTTP calls in both tool and channel WASM wrappers Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: HTML-escape provider name in OAuth landing page, simplify Slack limit description - Add html_escape() to prevent XSS in landing_html() where provider_name was interpolated directly into HTML (defense-in-depth, source is trusted but escaping costs nothing) - Remove per-action default numbers from Slack limit field description to avoid confusing LLMs with conflicting defaults Addresses review feedback from zmanian on PR #42. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Save all bootstrap fields from wizard, fix config module comment - Wizard now saves secrets_master_key_source and database_pool_size to bootstrap.json (was only saving database_url and onboard_completed, which broke secrets after fresh onboard since SecretsConfig::resolve reads key source from bootstrap) - Update config.rs module doc to reflect bootstrap.json priority chain instead of the removed ~/.ironclaw/.env approach Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Replace BootstrapConfig with .env-based bootstrap DATABASE_URL is the only setting that needs disk persistence before the database is available. Instead of a custom bootstrap.json with 4 fields, use a standard ~/.ironclaw/.env file loaded via dotenvy. - Remove BootstrapConfig struct entirely - Restore ironclaw_env_path(), load_ironclaw_env(), save_database_url() - SecretsConfig::resolve() now auto-detects (env var then keychain probe) instead of reading a saved source from bootstrap.json - DatabaseConfig::resolve() reads DATABASE_URL from env only (dotenvy loads ~/.ironclaw/.env into the environment early in startup) - check_onboard_needed() is now sync (just checks env vars) - Wizard save_and_summarize() works for both postgres and libsql backends - One-time migration from bootstrap.json to .env preserved Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Ensure load_ironclaw_env() runs in all Config paths, fix .env priority - Config::from_env() and Config::from_db() now call load_ironclaw_env() internally (after dotenvy::dotenv()), so CLI commands like `memory` and `config` correctly load DATABASE_URL from ~/.ironclaw/.env - Fix load order: standard ./.env first (higher priority), then ~/.ironclaw/.env, matching the documented priority chain - Collapse nested if/if-let into let-chains (clippy::collapsible_if) in oauth_defaults.rs, tool.rs, and secrets/store.rs - Fix rename_to_migrated to take &Path instead of &PathBuf Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR review comments (quoting, SSRF, error mapping) - Quote DATABASE_URL in .env writes so `#` in passwords isn't treated as a dotenv comment (e.g., `DATABASE_URL="postgres://..."`) - Add SSRF defenses to refresh_oauth_token(): require HTTPS, reject private/loopback IPs (with DNS resolution), disable redirects. token_url comes from tool capabilities JSON, so a malicious tool could otherwise exfiltrate refresh tokens. - Fix IPv4 bind error mapping: only map AddrInUse to PortInUse, use generic Io variant for other bind failures Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>	2026-02-14 21:21:22 +00:00
4 people	e843c18141	feat: add libSQL/Turso embedded database backend (#47) ... * feat: add libSQL/Turso database backend with full feature parity Introduce a Database trait abstraction (~60 async methods) enabling compile-time backend selection between PostgreSQL and libSQL/Turso. Convert all modules from concrete Store to Arc<dyn Database>, add LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire libsql stores throughout CLI and main entry points, and make the setup wizard backend-agnostic. Key changes: - src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend with native SQLite-dialect SQL, and idempotent migration system - src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods) - src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods) - src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring - src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore> - Feature-gate postgres-only tests and examples Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: enable onboarding wizard for libSQL builds Refactor the setup wizard to work with both postgres and libsql feature flags. Previously the wizard was gated behind #[cfg(feature = "postgres")] only, so libsql-only builds would print an error on `ironclaw onboard`. - Add libsql fields to Settings (database_backend, libsql_path, libsql_url) - Split wizard database/migration/secrets methods into feature-gated variants - Add step_database_libsql() with local path and Turso remote replica prompts - Update setup/mod.rs and main.rs feature gates to any(postgres, libsql) - Extend check_onboard_needed() to detect libsql database presence Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback for libSQL backend - P0: Switch libsql_backend to connection-per-operation pattern to fix shared Connection concurrency issue across tokio tasks - P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race - P0: Document encryption-at-rest limitations and json_patch divergence - P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated empty strings with NULL - P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent RFC 3339 timestamps across all queries - P2: Use explicit _rowid column in FTS5 triggers and joins for stability across VACUUM operations - P2: Add tracing::warn when embedding provided but vector search disabled in hybrid_search - Extract shared connect_from_config() helper to deduplicate DB connection logic across main.rs, cli/config.rs, and cli/mcp.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing JobContext fields and resolve fmt/clippy warnings Add total_tokens_used and max_tokens fields to JobContext in libsql_backend.rs, apply cargo fmt, and fix clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: review fixes for libSQL backend (shared connections, panics, indexes) - Replace .expect() with proper error propagation in 3 call sites - Share Arc<Database> between backend and stores instead of single Connection - Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore - Wrap store() INSERT + SELECT-back in a transaction - Add ~22 missing indexes for parity with PostgreSQL schema - Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration - Fix super:: import to use crate:: style - Gate mask_password_in_url behind #[cfg(feature = "postgres")] - Rewrite secrets store init with or_else chain for runtime backend selection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (collapsible_if, too_many_arguments) Collapse nested if blocks into let_chains to satisfy clippy's collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments on libsql_row_to_tool_at since refactoring the positional index pattern would be a larger change. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]>	2026-02-14 02:05:05 +00:00
bkutasiandGitHub	45f547c711	fix: resolve runtime panic in Linux keychain integration (#32) ... * fix: resolve runtime panic in Linux keychain integration - Convert Linux keychain functions from sync (rt.block_on) to async - Remove nested runtime panic when called from async context - Make keychain API consistent across platforms (macOS, Linux, fallback) - Propagate async through config loading and CLI commands Fixes panic on Linux during 'ironclaw onboard' at Step 2 (Security). * fix: await async Config::from_env in test_heartbeat example	2026-02-12 00:15:41 +00:00
Illia PolosukhinandClaude Opus 4.6	8be390afab	Rename setup CLI command to onboard for compatibility ... Serde alias on `onboard_completed` preserves existing settings.json files that still have the old `setup_completed` key. Co-Authored-By: Claude Opus 4.6 <[email protected]>	2026-02-06 09:56:32 -08:00
Illia PolosukhinandClaude Opus 4.6	f3c85f57fc	Add proactivity features: memory CLI, session pruning, self-repair notifications, slash commands, status diagnostics, context warnings ... Closes the proactivity gap with six features: - Memory CLI (`ironclaw memory search/read/write/tree/status`) for direct workspace access without starting the full agent - Session pruning background task that evicts idle sessions (configurable TTL, default 7 days) - Self-repair notifications broadcast recovery results through channel manager instead of silent logging - `/heartbeat`, `/summarize`, `/suggest` slash commands for manual heartbeat trigger, thread summarization, and next-step suggestions - `ironclaw status` diagnostics command checking DB, session, secrets, embeddings, WASM tools, channels, heartbeat, and MCP servers - Context pressure warning that notifies users before auto-compaction fires Co-Authored-By: Claude Opus 4.6 <[email protected]>	2026-02-05 19:28:13 -08:00