Compare commits

..
Author SHA1 Message Date
Illia PolosukhinandClaude Opus 4.6 1aca76b1a7 style: fix rustfmt formatting in bootstrap test
Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-15 00:22:18 -08:00
Illia PolosukhinandClaude Opus 4.6 414c1b28b9 fix: status command shows libSQL backend and skips keychain probe
The status command only checked DATABASE_URL (postgres), showing
"not configured" for libSQL users. Now detects the DATABASE_BACKEND
env var and reports libSQL path and Turso sync status.

Also remove the keychain probe from status. get_generic_password()
triggers macOS unlock+authorization dialogs which is terrible UX
for a read-only diagnostic command.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-15 00:13:33 -08:00
Illia PolosukhinandClaude Opus 4.6 d771f99f9e fix: persist DATABASE_BACKEND to ~/.ironclaw/.env for libSQL startup
The wizard saved database_backend only to the database, but
Config::from_env() needs it BEFORE connecting to any database (to
decide which backend to use). Without it, the backend defaults to
Postgres and then fails with "Missing required setting database_url".

Now save all database bootstrap vars (DATABASE_BACKEND, DATABASE_URL,
LIBSQL_PATH, LIBSQL_URL) to ~/.ironclaw/.env via save_bootstrap_env().

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-15 00:12:01 -08:00
Illia PolosukhinandClaude Opus 4.6 1facda4a75 fix: cache keychain key eagerly to avoid redundant macOS password dialogs
Replace has_master_key() with get_master_key() in step_security() and
immediately build SecretsCrypto from the result. This eliminates redundant
keychain accesses later in init_secrets_context(), each of which triggers
macOS system dialogs (keychain unlock + app authorization).

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-15 00:07:02 -08:00
Illia PolosukhinandClaude Opus 4.6 e8caab1a12 fix: OAuth callback listener binds IPv4 first to match redirect URLs
The listener was binding to [::1] (IPv6) first, but NEAR AI and other
OAuth flows redirect to http://127.0.0.1:9876/... (IPv4 explicit).
On macOS and most systems, [::1] and 127.0.0.1 are separate addresses,
so the browser's connection to 127.0.0.1 was refused when the listener
was on [::1]. Reversed the bind order: try 127.0.0.1 first, fall back
to [::1] if IPv4 is unavailable.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 22:16:40 -08:00
Illia PolosukhinandClaude Opus 4.6 fac91aec3a fix: address latest PR review comments (SecretString, empty env, docs, embeddings)
- Change wizard llm_api_key from String to SecretString to prevent
  accidental logging of API keys
- Fix inject_llm_keys_from_secrets skipping when env var is set but
  empty, matching optional_env's treatment of empty as unset
- Fix inverted doc comment on INJECTED_VARS (env checked first, overlay
  is the fallback, not the other way around)
- Update stale "env vars" comments in main.rs to reflect overlay pattern
- Fix step_embeddings not seeing cached OpenAI key from wizard session

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 17:58:12 -08:00
Illia PolosukhinandClaude Opus 4.6 0aae66c9dc fix: address remaining PR review comments (clippy, TODO, secrets backend ordering)
- Fix empty line after doc comment (clippy: empty_line_after_doc_comments)
- Collapse nested if in optional_env overlay check (clippy: collapsible_if)
- Remove dangling TODO(#XX) placeholder issue ref in channels.rs
- Fix init_secrets_context to respect selected database_backend when both
  postgres and libsql features are compiled, preventing wrong-backend
  secrets storage when DATABASE_URL is set but libsql was chosen

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 17:19:30 -08:00
Illia PolosukhinandClaude Opus 4.6 aa808ca94e fix: remove unsafe set_var, use thread-safe overlay for injected secrets
Address PR #92 review comments:
- Replace all 5 unsafe `std::env::set_var()` calls with safe alternatives
- Add INJECTED_VARS OnceLock<HashMap> overlay in config.rs, checked by
  optional_env() before falling back to std::env::var()
- Cache wizard API key in SetupWizard.llm_api_key field instead of env
- Pass explicit key param to fetch_anthropic_models/fetch_openai_models
- Persist env-provided API keys to secrets store during onboarding

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 14:14:22 -08:00
Illia PolosukhinandClaude Opus 4.6 c7e6833d14 Merge remote-tracking branch 'origin/main' into fix/setup-audit-fixes
Resolve conflicts between main's simplified config (no bootstrap param,
env-only DatabaseConfig) and our branch's typed ChannelSetupError.

- config.rs: take main's simpler resolve() signatures (no bootstrap)
- main.rs: remove dead check_onboard_needed block and CACHED_KEYCHAIN_KEY ref
- channels.rs: keep ChannelSetupError types, restore settings params from main
- wizard.rs: pass &self.settings to setup_telegram, use ? with From impl
- settings.rs: fix test_llm_backend_round_trip (use std::fs::write, tempfile::tempdir)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 14:05:54 -08:00
9fed8453c7 fix: shell destructive-command check bypassed by Value::Object arguments (#72)
Co-authored-by: Yi LIU <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-14 21:54:13 +00:00
Illia PolosukhinandClaude Opus 4.6 1885d61d46 fix: replace unreachable!() with error return in setup wizard
The provider match in step_inference_provider was guarded by
is_known but used unreachable!() as the catch-all. If a new
provider is added to the is_known check without a corresponding
match arm, this would panic at runtime. Return a typed error
instead.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 13:51:37 -08:00
Illia PolosukhinandClaude Opus 4.6 da47903108 fix: harden setup module error handling and secret safety
- Introduce ChannelSetupError typed enum replacing raw String errors
  across all channel setup functions (setup_telegram, setup_http,
  setup_tunnel, setup_wasm_channel, validate_telegram_token)
- Add From<ChannelSetupError> for SetupError to simplify call sites
- Convert setup_telegram retry from recursion to loop (unbounded stack)
- Stop printing HTTP webhook secret plaintext to terminal
- Use secret_input() for Turso auth token (was visible input())
- Replace dirs::home_dir().unwrap_or_default() with proper error
- Fix UTF-8 panic in model name truncation (byte-index to chars-based)
- Log warning in secret_exists() instead of silently swallowing errors
- Deduplicate generate_webhook_secret() to delegate to shared helper

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 13:43:25 -08:00
eaef335db6 fix: propagate real tool_call_id instead of hardcoded placeholder (#73)
The worker (both agent/worker.rs and worker/runtime.rs) was passing the
literal string "tool_call_id" to ChatMessage::tool_result instead of
the actual tool call ID from the LLM response. This breaks
OpenAI-compatible providers that match tool results to their
corresponding calls by ID.

- Add tool_call_id field to ToolSelection struct
- Propagate ToolCall.id through select_tools() into ToolSelection
- Replace all hardcoded "tool_call_id" usages with selection.tool_call_id
- Generate unique IDs for plan-based synthetic selections
- Add test verifying tool_call_id is preserved

Co-authored-by: Yi LIU <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-14 21:39:25 +00:00
Eric WinerandGitHub 225af29db2 Reformat architecture diagram in README (#64) 2026-02-14 21:22:58 +00:00
a53b2c10b5 fix: Fix wasm tool schemas and runtime (#42)
* feat: Move debug log truncation from agent loop to REPL channel

Full tool output now flows through StatusUpdate so the web gateway
gets untruncated content. The REPL channel truncates at display time
(200 chars for tool results, thinking, and status messages).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Flatten WASM tool schemas and fix host HTTP runtime contention

LLMs can't reliably follow oneOf + const discriminator patterns in JSON
Schema, causing tools like Google Calendar to receive malformed params
(e.g., {"operation":"list_events","data":{"calendarId":"primary"}} instead
of {"action":"list_events","calendar_id":"primary"}). Replace all 9 WASM
tool schemas with flat action enum + top-level properties. The serde
#[serde(tag = "action")] deserialization works identically.

Also fixes WASM host HTTP requests (channels and tools) stalling during
startup by replacing Handle::current().block_on() with a dedicated
single-threaded runtime per request, avoiding I/O driver contention.

Reduces verbose LLM debug logging (full request/response payloads) and
changes tower_http default from debug to warn.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Built-in OAuth credentials and combined Google scopes

Add infrastructure for shipping default OAuth credentials with the binary,
similar to how gcloud/rclone bake in their client_id. Credentials are set
at compile time via IRONCLAW_GOOGLE_CLIENT_ID / IRONCLAW_GOOGLE_CLIENT_SECRET
env vars, or can be hardcoded in src/cli/oauth_defaults.rs.

The fallback chain is: capabilities file > runtime env var > built-in defaults.

Also, when authing any Google tool, scopes from ALL installed Google tools
are now combined into a single OAuth request (they all share the same
google_oauth_token secret). One login covers Gmail, Calendar, Drive, etc.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Ship default Google OAuth credentials for zero-config auth

Google Desktop App credentials are not secret (per Google's own docs).
Hardcode them so `ironclaw tool auth <google-tool>` works out of the box
without requiring users to register their own OAuth app.

Credentials can still be overridden at compile time
(IRONCLAW_GOOGLE_CLIENT_ID) or runtime (GOOGLE_OAUTH_CLIENT_ID).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Consistent OAuth callback port and polished landing page

- Use fixed port 9876 instead of scanning 9876-9886 (one redirect URI
  to register in provider OAuth apps, deterministic behavior)
- Replace broken unicode checkmark with SVG icons (charset was missing,
  rendered as mojibake)
- Dark themed landing page with proper card layout for both success
  and error states
- Add charset=utf-8 to Content-Type headers

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: Unify OAuth callback server across all auth flows

All three OAuth flows (WASM tool auth, MCP server auth, NEAR AI login)
now share the same code from cli::oauth_defaults:

- Fixed port 9876 (one redirect URI to register per provider)
- Shared landing page HTML (dark card with SVG icons, proper charset)
- Parameterized wait_for_callback(listener, path, param, display_name)

Removes ~120 lines of duplicated callback/HTML code.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Support for oauth token refresh

* refactor: Replace bootstrap.json with ~/.ironclaw/.env for DATABASE_URL

Kill the 4-field BootstrapConfig JSON file. Only DATABASE_URL actually
needs disk persistence (chicken-and-egg before DB connect). The other
three fields are now derived: pool_size defaults to 10 via env var,
secrets master key is auto-detected (env then keychain probe), and
onboard_completed is inferred from DATABASE_URL presence.

The new format is a standard .env file loaded via dotenvy early in
main, so DATABASE_URL is available as a regular env var everywhere.

Handles three upgrade paths:
- Clean start: wizard writes .env, reload after wizard completes
- Returning user: .env loaded at startup, business as usual
- Legacy upgrade: bootstrap.json auto-migrated to .env on first run

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Address PR review findings

- Fix UTF-8 panic in truncate_for_preview (byte-slice on char boundary)
- Cap WASM guest timeout_ms at 5 minutes to prevent resource exhaustion
- Fix localhost detection in requires_auth() to avoid substring matches
  (e.g. "notlocalhost.com" no longer matches)
- Fix query param injection to insert before URL fragment
- Fix extract_host_from_url for IPv6 bracket notation
- Remove misleading schema defaults: Slack limit, Slides insertion_index,
  Docs index (per-action defaults documented in descriptions instead)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: Fix cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: IPv6 loopback support for OAuth listener and localhost detection

- bind_callback_listener: try [::1] first, fall back to 127.0.0.1,
  so OAuth redirects work on systems where localhost resolves to ::1
- is_localhost_url: replace manual string parsing with url::Url for
  correct handling of IPv6 brackets, ports, userinfo, etc.
- Add url crate as direct dependency (already a transitive dep)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Address PR review feedback on runtime reuse, onboard check, and OAuth binding

- Remove session file check from check_onboard_needed(); DATABASE_URL is sufficient
- Detect AddrInUse on IPv6 bind and fail immediately instead of falling through to IPv4
- Reuse dedicated tokio runtime across HTTP calls in both tool and channel WASM wrappers

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: HTML-escape provider name in OAuth landing page, simplify Slack limit description

- Add html_escape() to prevent XSS in landing_html() where provider_name
  was interpolated directly into HTML (defense-in-depth, source is trusted
  but escaping costs nothing)
- Remove per-action default numbers from Slack limit field description to
  avoid confusing LLMs with conflicting defaults

Addresses review feedback from zmanian on PR #42.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Save all bootstrap fields from wizard, fix config module comment

- Wizard now saves secrets_master_key_source and database_pool_size to
  bootstrap.json (was only saving database_url and onboard_completed,
  which broke secrets after fresh onboard since SecretsConfig::resolve
  reads key source from bootstrap)
- Update config.rs module doc to reflect bootstrap.json priority chain
  instead of the removed ~/.ironclaw/.env approach

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: Replace BootstrapConfig with .env-based bootstrap

DATABASE_URL is the only setting that needs disk persistence before
the database is available. Instead of a custom bootstrap.json with 4
fields, use a standard ~/.ironclaw/.env file loaded via dotenvy.

- Remove BootstrapConfig struct entirely
- Restore ironclaw_env_path(), load_ironclaw_env(), save_database_url()
- SecretsConfig::resolve() now auto-detects (env var then keychain probe)
  instead of reading a saved source from bootstrap.json
- DatabaseConfig::resolve() reads DATABASE_URL from env only (dotenvy
  loads ~/.ironclaw/.env into the environment early in startup)
- check_onboard_needed() is now sync (just checks env vars)
- Wizard save_and_summarize() works for both postgres and libsql backends
- One-time migration from bootstrap.json to .env preserved

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Ensure load_ironclaw_env() runs in all Config paths, fix .env priority

- Config::from_env() and Config::from_db() now call load_ironclaw_env()
  internally (after dotenvy::dotenv()), so CLI commands like `memory`
  and `config` correctly load DATABASE_URL from ~/.ironclaw/.env
- Fix load order: standard ./.env first (higher priority), then
  ~/.ironclaw/.env, matching the documented priority chain
- Collapse nested if/if-let into let-chains (clippy::collapsible_if)
  in oauth_defaults.rs, tool.rs, and secrets/store.rs
- Fix rename_to_migrated to take &Path instead of &PathBuf

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Address PR review comments (quoting, SSRF, error mapping)

- Quote DATABASE_URL in .env writes so `#` in passwords isn't treated
  as a dotenv comment (e.g., `DATABASE_URL="postgres://..."`)
- Add SSRF defenses to refresh_oauth_token(): require HTTPS, reject
  private/loopback IPs (with DNS resolution), disable redirects.
  token_url comes from tool capabilities JSON, so a malicious tool
  could otherwise exfiltrate refresh tokens.
- Fix IPv4 bind error mapping: only map AddrInUse to PortInUse,
  use generic Io variant for other bind failures

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-14 21:21:22 +00:00
Illia PolosukhinandClaude Opus 4.6 85196cd527 fix: address second-round PR review feedback
- Validate custom model ID is non-empty (loop until valid input)
- Warn on unknown DATABASE_BACKEND env var before defaulting to Postgres
- Force re-selection when llm_backend contains unknown provider value
- Use ok_or_else for proper String error type in google-sheets

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 13:20:06 -08:00
Illia PolosukhinandClaude Opus 4.6 a0e01f04d3 fix: address critical/high audit findings across WASM sub-crates
- Telegram: remove .unwrap() panic on workspace_read (owner_id check)
- WhatsApp: use configured api_version instead of hardcoded v18.0
- WhatsApp: log config parse errors before falling back to defaults
- Slack: log serialization errors in emit_message and json_response
- Google Docs: safe array access for batch update replies
- Google Sheets: safe array access for add_sheet replies
- Google Calendar: fix doc comment secret name mismatch
- Gmail: avoid unnecessary String allocation in UNREAD check

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 12:54:20 -08:00
408ae8a29a feat: add multi-provider LLM failover with retry backoff (#28)
* feat: add multi-provider LLM failover

Add FailoverProvider that wraps multiple LlmProvider instances and
tries each in sequence on transient failures. Non-retryable errors
(auth, context length, model not available) propagate immediately.

- New `FailoverProvider` with generic `try_providers` helper
- `is_retryable()` classifies transient errors (request failed,
  rate limited, invalid response, session renewal, HTTP, IO)
- Configurable via `NEARAI_FALLBACK_MODEL` env var
- Returns `Result` from constructor (no panics in production)
- Updates FEATURE_PARITY.md: failover chains , cooldown 

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: track last-used provider for accurate cost/model reporting

After failover, model_name() and cost_per_token() now reflect the
provider that actually handled the request, not always the primary.
Also corrects is_retryable() docs to list ModelNotAvailable as retryable.

Addresses PR #28 review comments.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: add retry with exponential backoff for LLM providers

Add retry logic with exponential backoff and jitter to both NearAiProvider
and NearAiChatProvider for transient errors (HTTP 429, 500, 502, 503, 504).

Extract shared retry helpers (is_retryable_status, retry_backoff_delay)
into src/llm/retry.rs so both providers reuse the same logic.

Configurable via NEARAI_MAX_RETRIES env var (default: 3).

* docs: clarify max_retries means N retries, not N total attempts

* warn when fallback model equals primary model

* fix: saturating_mul in backoff delay, dedupe to_lowercase allocation

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-14 15:43:38 +04:00
Illia PolosukhinandClaude Opus 4.6 e982699e09 fix: address PR review feedback (set_var safety, parse warnings, db_map efficiency)
1. Replace unsafe set_var keychain caching with OnceLock<String> in
   SecretsConfig::resolve(). Eliminates the env var write from main.rs
   entirely, using a process-wide OnceLock cache instead.

2. Log tracing::warn when database_backend or llm_backend settings
   fail to parse, instead of silently falling back to defaults.

3. Remove O(K*S) get() pre-check in from_db_map(). Instead, let set()
   run and match on "Path not found" errors to skip unknown keys,
   avoiding full Settings serialization per key.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-14 01:09:20 -08:00
Illia Polosukhin 5e73dbbdc8 Merge remote-tracking branch 'origin/main' into feat/onboarding-libsql-selection 2026-02-14 00:55:48 -08:00
Illia PolosukhinandClaude Opus 4.6 92863bf860 fix: resolve libSQL onboarding crash, keychain double-prompt, and setup audit findings
Three bugs fixed:

1. libSQL onboarding crash ("Missing required setting 'database_url'"):
   DatabaseConfig::resolve() only checked DATABASE_BACKEND env var, falling
   back to Postgres default. Now reads settings.database_backend, plus
   settings.libsql_path and settings.libsql_url as fallbacks.

2. OS keychain prompts twice during startup: Config::from_env() and
   Config::from_db() both called get_master_key(). Now caches the key in
   SECRETS_MASTER_KEY env var after first read so from_db() skips keychain.

3. "Path not found: nearai.session" warning: from_db_map() tried to apply
   app-specific DB keys (nearai.session_token) to the Settings struct.
   Now skips keys that don't map to known Settings fields. Also fixed
   bootstrap migration key mismatch (nearai.session -> nearai.session_token).

Setup module audit fixes (14 findings):
- Replace unreachable!() with proper error in provider match
- Extract setup_api_key_provider() to deduplicate setup_anthropic/setup_openai
- Add SAFETY comments to all unsafe std::env::set_var blocks
- Fix .unwrap() calls with proper error handling
- Remove incorrect #[allow(dead_code)] on used TelegramUpdate::update_id
- Log warnings instead of silently discarding HTTP errors in Telegram binding
- Guard select_many against empty options, fix mask_api_key for non-ASCII
- Update stale doc comment in mod.rs, rename misleading variable
- Add 7 new tests (model fetcher fallbacks, channel discovery, secret gen)

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-13 21:46:19 -08:00
Zaki ManianGitHubClaude Opus 4.6Illia Polosukhingemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
d9ff86d7e0 docs: Add review discipline guidelines to CLAUDE.md (#68)
* docs: Add review discipline guidelines to CLAUDE.md

Codifies lessons learned from Illia's review fixes on the libSQL
backend PR -- patterns we missed that should be caught systematically
going forward.

- Ban .expect() alongside .unwrap() in production code
- Add mechanical grep checks before committing
- New "Review & Fix Discipline" section covering:
  - Fix all instances of a pattern, not just the one flagged
  - Propagate architectural changes to satellite types
  - Schema translation must include indexes and seed data
  - Feature flag testing with each feature in isolation

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-14 04:25:53 +00:00
Illia PolosukhinandClaude Opus 4.6 46c1daca5e feat: add interactive database backend selection during onboarding
Previously the onboarding wizard silently defaulted to PostgreSQL because
libsql wasn't in the default feature set. Now both backends ship by default
and the wizard presents a selection prompt when both are available.

DATABASE_BACKEND env var still bypasses the prompt for headless/CI use.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-13 18:27:53 -08:00
e843c18141 feat: add libSQL/Turso embedded database backend (#47)
* feat: add libSQL/Turso database backend with full feature parity

Introduce a Database trait abstraction (~60 async methods) enabling
compile-time backend selection between PostgreSQL and libSQL/Turso.
Convert all modules from concrete Store to Arc<dyn Database>, add
LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire
libsql stores throughout CLI and main entry points, and make the
setup wizard backend-agnostic.

Key changes:
- src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend
  with native SQLite-dialect SQL, and idempotent migration system
- src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods)
- src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods)
- src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring
- src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore>
- Feature-gate postgres-only tests and examples

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable onboarding wizard for libSQL builds

Refactor the setup wizard to work with both postgres and libsql feature
flags. Previously the wizard was gated behind #[cfg(feature = "postgres")]
only, so libsql-only builds would print an error on `ironclaw onboard`.

- Add libsql fields to Settings (database_backend, libsql_path, libsql_url)
- Split wizard database/migration/secrets methods into feature-gated variants
- Add step_database_libsql() with local path and Turso remote replica prompts
- Update setup/mod.rs and main.rs feature gates to any(postgres, libsql)
- Extend check_onboard_needed() to detect libsql database presence

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback for libSQL backend

- P0: Switch libsql_backend to connection-per-operation pattern to fix
  shared Connection concurrency issue across tokio tasks
- P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race
- P0: Document encryption-at-rest limitations and json_patch divergence
- P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated
  empty strings with NULL
- P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent
  RFC 3339 timestamps across all queries
- P2: Use explicit _rowid column in FTS5 triggers and joins for stability
  across VACUUM operations
- P2: Add tracing::warn when embedding provided but vector search disabled
  in hybrid_search
- Extract shared connect_from_config() helper to deduplicate DB connection
  logic across main.rs, cli/config.rs, and cli/mcp.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add missing JobContext fields and resolve fmt/clippy warnings

Add total_tokens_used and max_tokens fields to JobContext in
libsql_backend.rs, apply cargo fmt, and fix clippy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: review fixes for libSQL backend (shared connections, panics, indexes)

- Replace .expect() with proper error propagation in 3 call sites
- Share Arc<Database> between backend and stores instead of single Connection
- Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore
- Wrap store() INSERT + SELECT-back in a transaction
- Add ~22 missing indexes for parity with PostgreSQL schema
- Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration
- Fix super:: import to use crate:: style
- Gate mask_password_in_url behind #[cfg(feature = "postgres")]
- Rewrite secrets store init with or_else chain for runtime backend selection

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Resolve clippy lints (collapsible_if, too_many_arguments)

Collapse nested if blocks into let_chains to satisfy clippy's
collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments
on libsql_row_to_tool_at since refactoring the positional index
pattern would be a larger change.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-14 02:05:05 +00:00
54e9206f0b feat: Move debug log truncation from agent loop to REPL channel (#65)
* feat: Move debug log truncation from agent loop to REPL channel

Full tool output now flows through StatusUpdate so the web gateway
gets untruncated content. The REPL channel truncates at display time
(200 chars for tool results, thinking, and status messages).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: truncating fmt layer for terminal, full logs for web gateway

Instead of truncating debug output at each LLM call site (fragile),
use a custom MakeWriter on the fmt layer that caps each tracing event
at 500 bytes before flushing to stderr. The web gateway WebLogLayer
still receives full untruncated content for /api/logs/events SSE.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: UTF-8 safe truncation in truncate_for_preview, remove double truncation

- Use char_indices() instead of byte-based slicing to find the cut
  point, preventing panics on multi-byte characters (emoji, CJK, etc.)
- Remove redundant truncation in REPL channel (agent loop already
  truncates ToolResult previews to 200 chars)
- Add 9 unit tests covering edge cases: empty, exact length, multi-byte
  UTF-8 (emoji, CJK), mixed scripts, newline collapsing, whitespace

Addresses PR #65 review comments.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-13 23:24:07 +00:00
5df0d13b59 Bump MSRV to 1.92, add GCP deployment files (#40)
* Bump MSRV to 1.92 and add GCP deployment files

rig-core 0.30 uses let_chains (stabilized post-1.87), which breaks
builds on Rust 1.85. Bump rust-version in Cargo.toml and both
Dockerfiles to 1.92 (verified working).

Add cloud deployment scaffolding:
- Dockerfile: multi-stage build for the main agent container
- deploy/cloud-sql-proxy.service: systemd unit for Cloud SQL Auth Proxy
- deploy/ironclaw.service: systemd unit for the IronClaw container
- deploy/setup.sh: VM bootstrap script (Docker, proxy, services)
- deploy/env.example: reference environment configuration

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Address review feedback: harden deploy scaffolding

- Add comment explaining GATEWAY_HOST=0.0.0.0 and when to use 127.0.0.1
- Document /opt/ironclaw ownership model (root-owned, Docker reads as root)
- Switch cloud-sql-proxy service from User=root to DynamicUser=yes

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Resolve clippy lints (Rust 1.93) and fix CI test workflow

- Fix 97 collapsible_if warnings using let-chains syntax (auto-fixed)
- Fix ptr_arg: change &PathBuf to &Path in pairing store functions
- Fix suspicious_open_options: add .truncate(false) to OpenOptions
- Fix too_many_arguments: add clippy allow on execute_status
- Fix unnecessary_unwrap: use if-let in repository.rs hybrid_search
- Gate unused EchoTool with #[cfg(test)]
- Add PairingStore argument to ChannelStoreData::new() test call sites
- Add skip guard for bundled channel test when WASM artifacts unavailable
- Split CI test workflow to exclude PostgreSQL-dependent integration tests

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Address review feedback from ilblackdragon

- Add root check to setup.sh (exits with error if not root)
- Add warning comment to env.example about placeholder passwords
- Dockerfile.worker already uses rust:1.92 (no change needed)
- PR #41 overlap noted; will rebase after #41 merges

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: resolve 47 collapsible_if clippy warnings

Collapse nested if statements across the codebase to satisfy
clippy::collapsible_if on Rust 1.93.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-13 22:21:50 +04:00
94 changed files with 6632 additions and 3674 deletions
+1
View File
@@ -1,6 +1,7 @@
.env .env
.env.local .env.local
.env.*
target/ target/
+33 -1
View File
@@ -198,8 +198,9 @@ When designing new features or systems, always prefer generic/extensible archite
### Error Handling ### Error Handling
- Use `thiserror` for error types in `error.rs` - Use `thiserror` for error types in `error.rs`
- Never use `.unwrap()` in production code (tests are fine) - Never use `.unwrap()` or `.expect()` in production code (tests are fine)
- Map errors with context: `.map_err(|e| SomeError::Variant { reason: e.to_string() })?` - Map errors with context: `.map_err(|e| SomeError::Variant { reason: e.to_string() })?`
- Before committing, grep for `.unwrap()` and `.expect(` in changed files to catch violations mechanically
### Async ### Async
- All I/O is async with tokio - All I/O is async with tokio
@@ -637,6 +638,37 @@ RUST_LOG=ironclaw=debug,tower_http=debug cargo run
- Keep functions focused, extract helpers when logic is reused - Keep functions focused, extract helpers when logic is reused
- Comments for non-obvious logic only - Comments for non-obvious logic only
## Review & Fix Discipline
Hard-won lessons from code review -- follow these when fixing bugs or addressing review feedback.
### Fix the pattern, not just the instance
When a reviewer flags a bug (e.g., TOCTOU race in INSERT + SELECT-back), search the entire codebase for all instances of that same pattern. A fix in `SecretsStore::create()` that doesn't also fix `WasmToolStore::store()` is half a fix.
### Propagate architectural fixes to satellite types
If a core type changes its concurrency model (e.g., `LibSqlBackend` switches to connection-per-operation), every type that was handed a resource from the old model (e.g., `LibSqlSecretsStore`, `LibSqlWasmToolStore` holding a single `Connection`) must also be updated. Grep for the old type across the codebase.
### Schema translation is more than DDL
When translating a database schema between backends (PostgreSQL to libSQL, etc.), check for:
- **Indexes** -- diff `CREATE INDEX` statements between the two schemas
- **Seed data** -- check for `INSERT INTO` in migrations (e.g., `leak_detection_patterns`)
- **Semantic differences** -- document where SQL functions behave differently (e.g., `json_patch` vs `jsonb_set`)
### Feature flag testing
When adding feature-gated code, test compilation with each feature in isolation:
```bash
cargo check # default features
cargo check --no-default-features --features libsql # libsql only
cargo check --all-features # all features
```
Dead code behind the wrong `#[cfg]` gate will only show up when building with a single feature.
### Mechanical verification before committing
Run these checks on changed files before committing:
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
## Workspace & Memory System ## Workspace & Memory System
Inspired by [OpenClaw](https://github.com/openclaw/openclaw), the workspace provides persistent memory for agents with a flexible filesystem-like structure. Inspired by [OpenClaw](https://github.com/openclaw/openclaw), the workspace provides persistent memory for agents with a flexible filesystem-like structure.
Generated
+3 -11
View File
@@ -2210,11 +2210,11 @@ dependencies = [
"hyper 1.8.1", "hyper 1.8.1",
"hyper-util", "hyper-util",
"rustls", "rustls",
"rustls-native-certs",
"rustls-pki-types", "rustls-pki-types",
"tokio", "tokio",
"tokio-rustls", "tokio-rustls",
"tower-service", "tower-service",
"webpki-roots",
] ]
[[package]] [[package]]
@@ -2548,6 +2548,7 @@ dependencies = [
"tower-http 0.6.8", "tower-http 0.6.8",
"tracing", "tracing",
"tracing-subscriber", "tracing-subscriber",
"url",
"urlencoding", "urlencoding",
"uuid", "uuid",
"wasmparser 0.220.1", "wasmparser 0.220.1",
@@ -4033,6 +4034,7 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
"quinn", "quinn",
"rustls", "rustls",
"rustls-native-certs",
"rustls-pki-types", "rustls-pki-types",
"serde", "serde",
"serde_json", "serde_json",
@@ -4050,7 +4052,6 @@ dependencies = [
"wasm-bindgen-futures", "wasm-bindgen-futures",
"wasm-streams", "wasm-streams",
"web-sys", "web-sys",
"webpki-roots",
] ]
[[package]] [[package]]
@@ -6237,15 +6238,6 @@ dependencies = [
"wasm-bindgen", "wasm-bindgen",
] ]
[[package]]
name = "webpki-roots"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12bed680863276c63889429bfd6cab3b99943659923822de1c8a39c49e4d722c"
dependencies = [
"rustls-pki-types",
]
[[package]] [[package]]
name = "which" name = "which"
version = "4.4.2" version = "4.4.2"
+5 -4
View File
@@ -2,7 +2,7 @@
name = "ironclaw" name = "ironclaw"
version = "0.1.3" version = "0.1.3"
edition = "2024" edition = "2024"
rust-version = "1.85" rust-version = "1.92"
description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly" description = "Secure personal AI assistant that protects your data and expands its capabilities on the fly"
authors = ["NEAR AI <[email protected]>"] authors = ["NEAR AI <[email protected]>"]
license = "MIT OR Apache-2.0" license = "MIT OR Apache-2.0"
@@ -22,7 +22,7 @@ tokio-stream = { version = "0.1", features = ["sync"] }
futures = "0.3" futures = "0.3"
# HTTP client # HTTP client
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls", "stream"] } reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-native-roots", "stream"] }
# Serialization # Serialization
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
@@ -84,7 +84,8 @@ fs4 = "0.6"
# Secrecy for sensitive values # Secrecy for sensitive values
secrecy = { version = "0.10", features = ["serde"] } secrecy = { version = "0.10", features = ["serde"] }
# URL encoding for OAuth flow # URL parsing and encoding
url = "2"
urlencoding = "2" urlencoding = "2"
# Open URLs in browser # Open URLs in browser
@@ -138,7 +139,7 @@ pretty_assertions = "1"
tempfile = "3" tempfile = "3"
[features] [features]
default = ["postgres"] default = ["postgres", "libsql"]
postgres = [ postgres = [
"dep:deadpool-postgres", "dep:deadpool-postgres",
"dep:tokio-postgres", "dep:tokio-postgres",
+46
View File
@@ -0,0 +1,46 @@
# Multi-stage Dockerfile for the IronClaw agent (cloud deployment).
#
# Build:
# docker build --platform linux/amd64 -t ironclaw:latest .
#
# Run:
# docker run --env-file .env -p 3000:3000 ironclaw:latest
# Stage 1: Build
FROM rust:1.92-slim-bookworm AS builder
RUN apt-get update && apt-get install -y --no-install-recommends \
pkg-config libssl-dev cmake gcc g++ \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
# Copy manifests first for layer caching
COPY Cargo.toml Cargo.lock ./
# Copy source and build artifacts
COPY src/ src/
COPY migrations/ migrations/
COPY wit/ wit/
RUN cargo build --release --bin ironclaw
# Stage 2: Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates libssl3 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/target/release/ironclaw /usr/local/bin/ironclaw
COPY --from=builder /app/migrations /app/migrations
# Non-root user
RUN useradd -m -u 1000 -s /bin/bash ironclaw
USER ironclaw
EXPOSE 3000
ENV RUST_LOG=ironclaw=info
ENTRYPOINT ["ironclaw"]
+2 -2
View File
@@ -9,7 +9,7 @@
# The image includes common development tools so workers can build software, # The image includes common development tools so workers can build software,
# run tests, and execute shell commands. # run tests, and execute shell commands.
FROM rust:1.85-bookworm AS builder FROM rust:1.92-bookworm AS builder
WORKDIR /build WORKDIR /build
COPY . . COPY . .
@@ -40,7 +40,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
ENV RUSTUP_HOME=/usr/local/rustup \ ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \ CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH PATH=/usr/local/cargo/bin:$PATH
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.85.0 \ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.92.0 \
&& chmod -R a+r /usr/local/rustup /usr/local/cargo && chmod -R a+r /usr/local/rustup /usr/local/cargo
# Install Claude Code CLI (for claude-bridge mode) # Install Claude Code CLI (for claude-bridge mode)
+3 -3
View File
@@ -133,7 +133,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
|---------|----------|----------|-------| |---------|----------|----------|-------|
| Pi agent runtime | ✅ | | IronClaw uses custom runtime | | Pi agent runtime | ✅ | | IronClaw uses custom runtime |
| RPC-based execution | ✅ | ✅ | Orchestrator/worker pattern | | RPC-based execution | ✅ | ✅ | Orchestrator/worker pattern |
| Multi-provider failover | ✅ | | Provider fallback chains | | Multi-provider failover | ✅ | | `FailoverProvider` tries providers sequentially on retryable errors |
| Per-sender sessions | ✅ | ✅ | | | Per-sender sessions | ✅ | ✅ | |
| Global sessions | ✅ | ❌ | Optional shared context | | Global sessions | ✅ | ❌ | Optional shared context |
| Session pruning | ✅ | ❌ | Auto cleanup old sessions | | Session pruning | ✅ | ❌ | Auto cleanup old sessions |
@@ -173,7 +173,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Feature | OpenClaw | IronClaw | Notes | | Feature | OpenClaw | IronClaw | Notes |
|---------|----------|----------|-------| |---------|----------|----------|-------|
| Auto-discovery | ✅ | ❌ | | | Auto-discovery | ✅ | ❌ | |
| Failover chains | ✅ | | Provider fallback | | Failover chains | ✅ | | `FailoverProvider` with configurable `fallback_model` |
| Cooldown management | ✅ | ❌ | Skip failed providers | | Cooldown management | ✅ | ❌ | Skip failed providers |
| Per-session model override | ✅ | ✅ | Model selector in TUI | | Per-session model override | ✅ | ✅ | Model selector in TUI |
| Model selection UI | ✅ | ✅ | TUI keyboard shortcut | | Model selection UI | ✅ | ✅ | TUI keyboard shortcut |
@@ -419,7 +419,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ❌ Slack channel (real implementation) - ❌ Slack channel (real implementation)
- ✅ Telegram channel (WASM, DM pairing, caption, /start) - ✅ Telegram channel (WASM, DM pairing, caption, /start)
- ❌ WhatsApp channel - ❌ WhatsApp channel
- Multi-provider failover - Multi-provider failover (`FailoverProvider` with retryable error classification)
- ❌ Hooks system (beforeInbound, beforeToolCall, etc.) - ❌ Hooks system (beforeInbound, beforeToolCall, etc.)
### P2 - Medium Priority ### P2 - Medium Priority
+8 -8
View File
@@ -181,7 +181,7 @@ External content passes through multiple security layers:
## Architecture ## Architecture
``` ```
┌──────────────────────────────────────────────────────────────────── ┌────────────────────────────────────────────────────────────────┐
│ Channels │ │ Channels │
│ ┌──────┐ ┌──────┐ ┌─────────────┐ ┌─────────────┐ │ │ ┌──────┐ ┌──────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ REPL │ │ HTTP │ │WASM Channels│ │ Web Gateway │ │ │ │ REPL │ │ HTTP │ │WASM Channels│ │ Web Gateway │ │
@@ -191,20 +191,20 @@ External content passes through multiple security layers:
│ │ │ │ │ │
│ ┌─────────▼─────────┐ │ │ ┌─────────▼─────────┐ │
│ │ Agent Loop │ Intent routing │ │ │ Agent Loop │ Intent routing │
│ └────┬─────────────┘ │ └────┬─────────────┘ │
│ │ │ │ │ │ │ │
│ ┌──────────▼───┐ ┌──▼──────────────┐ │ ┌──────────▼───┐ ┌──▼──────────────┐ │
│ │ Scheduler │ │ Routines Engine │ │ │ │ Scheduler │ │ Routines Engine │ │
│ │(parallel jobs)│ │(cron, event, wh) │ │ │ │(parallel jobs)│ │(cron, event, wh) │ │
│ └──────┬───────┘ └────────┬─────────┘ │ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │ │ │ │ │
│ ┌─────────────┼───────────────────┘ │ ┌─────────────┼───────────────────┘ │
│ │ │ │ │ │ │ │
│ ┌───▼────┐ ┌────▼────────────────┐ │ ┌───▼────┐ ┌────▼────────────────┐ │
│ │ Local │ │ Orchestrator │ │ │ │ Local │ │ Orchestrator │ │
│ │Workers │ │ ┌───────────────┐ │ │ │ │Workers │ │ ┌───────────────┐ │ │
│ │(in-proc)│ │ │ Docker Sandbox│ │ │ │ │(in-proc)│ │ │ Docker Sandbox│ │ │
│ └───┬────┘ │ │ Containers │ │ │ └───┬────┘ │ │ Containers │ │ │
│ │ │ │ ┌───────────┐ │ │ │ │ │ │ │ ┌───────────┐ │ │ │
│ │ │ │ │Worker / CC│ │ │ │ │ │ │ │ │Worker / CC│ │ │ │
│ │ │ │ └───────────┘ │ │ │ │ │ │ │ └───────────┘ │ │ │
@@ -216,7 +216,7 @@ External content passes through multiple security layers:
│ │ Tool Registry │ │ │ │ Tool Registry │ │
│ │ Built-in, MCP, WASM │ │ │ │ Built-in, MCP, WASM │ │
│ └──────────────────────┘ │ │ └──────────────────────┘ │
└──────────────────────────────────────────────────────────────────── └────────────────────────────────────────────────────────────────┘
``` ```
### Core Components ### Core Components
+14 -2
View File
@@ -338,7 +338,13 @@ fn emit_message(
team_id, team_id,
}; };
let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|_| "{}".to_string()); let metadata_json = serde_json::to_string(&metadata).unwrap_or_else(|e| {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize Slack metadata: {}", e),
);
"{}".to_string()
});
// Strip @ mentions of the bot from the text for cleaner messages // Strip @ mentions of the bot from the text for cleaner messages
let cleaned_text = strip_bot_mention(&text); let cleaned_text = strip_bot_mention(&text);
@@ -366,7 +372,13 @@ fn strip_bot_mention(text: &str) -> String {
/// Create a JSON HTTP response. /// Create a JSON HTTP response.
fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse { fn json_response(status: u16, value: serde_json::Value) -> OutgoingHttpResponse {
let body = serde_json::to_vec(&value).unwrap_or_default(); let body = serde_json::to_vec(&value).unwrap_or_else(|e| {
channel_host::log(
channel_host::LogLevel::Error,
&format!("Failed to serialize JSON response: {}", e),
);
Vec::new()
});
let headers = serde_json::json!({"Content-Type": "application/json"}); let headers = serde_json::json!({"Content-Type": "application/json"});
OutgoingHttpResponse { OutgoingHttpResponse {
+9 -19
View File
@@ -285,11 +285,7 @@ impl Guest for TelegramChannel {
} }
// Persist dm_policy and allow_from for DM pairing in handle_message // Persist dm_policy and allow_from for DM pairing in handle_message
let dm_policy = config let dm_policy = config.dm_policy.as_deref().unwrap_or("pairing").to_string();
.dm_policy
.as_deref()
.unwrap_or("pairing")
.to_string();
let _ = channel_host::workspace_write(DM_POLICY_PATH, &dm_policy); let _ = channel_host::workspace_write(DM_POLICY_PATH, &dm_policy);
let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default()) let allow_from_json = serde_json::to_string(&config.allow_from.unwrap_or_default())
@@ -844,8 +840,8 @@ fn send_pairing_reply(chat_id: i64, code: &str) -> Result<(), String> {
"parse_mode": "Markdown", "parse_mode": "Markdown",
}); });
let payload_bytes = serde_json::to_vec(&payload) let payload_bytes =
.map_err(|e| format!("Failed to serialize payload: {}", e))?; serde_json::to_vec(&payload).map_err(|e| format!("Failed to serialize payload: {}", e))?;
let headers = serde_json::json!({ let headers = serde_json::json!({
"Content-Type": "application/json" "Content-Type": "application/json"
@@ -915,15 +911,10 @@ fn handle_message(message: TelegramMessage) {
let is_private = message.chat.chat_type == "private"; let is_private = message.chat.chat_type == "private";
// Owner validation: when owner_id is set, only that user can message // Owner validation: when owner_id is set, only that user can message
let owner_configured = channel_host::workspace_read(OWNER_ID_PATH) let owner_id_str = channel_host::workspace_read(OWNER_ID_PATH).filter(|s| !s.is_empty());
.map(|s| !s.is_empty())
.unwrap_or(false);
if owner_configured { if let Some(ref id_str) = owner_id_str {
if let Ok(owner_id) = channel_host::workspace_read(OWNER_ID_PATH) if let Ok(owner_id) = id_str.parse::<i64>() {
.unwrap()
.parse::<i64>()
{
if from.id != owner_id { if from.id != owner_id {
channel_host::log( channel_host::log(
channel_host::LogLevel::Debug, channel_host::LogLevel::Debug,
@@ -937,8 +928,8 @@ fn handle_message(message: TelegramMessage) {
} }
} else if is_private { } else if is_private {
// No owner_id: apply dm_policy for private chats // No owner_id: apply dm_policy for private chats
let dm_policy = channel_host::workspace_read(DM_POLICY_PATH) let dm_policy =
.unwrap_or_else(|| "pairing".to_string()); channel_host::workspace_read(DM_POLICY_PATH).unwrap_or_else(|| "pairing".to_string());
if dm_policy != "open" { if dm_policy != "open" {
// Build effective allow list: config allow_from + pairing store // Build effective allow list: config allow_from + pairing store
@@ -1001,8 +992,7 @@ fn handle_message(message: TelegramMessage) {
if !respond_to_all { if !respond_to_all {
let has_command = content.starts_with('/'); let has_command = content.starts_with('/');
let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH) let bot_username = channel_host::workspace_read(BOT_USERNAME_PATH).unwrap_or_default();
.unwrap_or_default();
let has_bot_mention = if bot_username.is_empty() { let has_bot_mention = if bot_username.is_empty() {
content.contains('@') content.contains('@')
} else { } else {
+21 -4
View File
@@ -254,10 +254,19 @@ struct WhatsAppChannel;
impl Guest for WhatsAppChannel { impl Guest for WhatsAppChannel {
fn on_start(config_json: String) -> Result<ChannelConfig, String> { fn on_start(config_json: String) -> Result<ChannelConfig, String> {
let config: WhatsAppConfig = serde_json::from_str(&config_json).unwrap_or(WhatsAppConfig { let config: WhatsAppConfig = match serde_json::from_str(&config_json) {
Ok(c) => c,
Err(e) => {
channel_host::log(
channel_host::LogLevel::Warn,
&format!("Failed to parse WhatsApp config, using defaults: {}", e),
);
WhatsAppConfig {
api_version: default_api_version(), api_version: default_api_version(),
reply_to_message: default_reply_to_message(), reply_to_message: default_reply_to_message(),
}); }
}
};
channel_host::log( channel_host::log(
channel_host::LogLevel::Info, channel_host::LogLevel::Info,
@@ -267,6 +276,9 @@ impl Guest for WhatsAppChannel {
), ),
); );
// Persist api_version in workspace so on_respond() can read it
let _ = channel_host::workspace_write("channels/whatsapp/api_version", &config.api_version);
// WhatsApp Cloud API is webhook-only, no polling available // WhatsApp Cloud API is webhook-only, no polling available
Ok(ChannelConfig { Ok(ChannelConfig {
display_name: "WhatsApp".to_string(), display_name: "WhatsApp".to_string(),
@@ -327,11 +339,16 @@ impl Guest for WhatsAppChannel {
let metadata: WhatsAppMessageMetadata = serde_json::from_str(&response.metadata_json) let metadata: WhatsAppMessageMetadata = serde_json::from_str(&response.metadata_json)
.map_err(|e| format!("Failed to parse metadata: {}", e))?; .map_err(|e| format!("Failed to parse metadata: {}", e))?;
// Read api_version from workspace (set during on_start), fallback to default
let api_version = channel_host::workspace_read("channels/whatsapp/api_version")
.filter(|s| !s.is_empty())
.unwrap_or_else(|| "v18.0".to_string());
// Build WhatsApp API URL with token placeholder // Build WhatsApp API URL with token placeholder
// Host will replace {WHATSAPP_ACCESS_TOKEN} with actual token in Authorization header // Host will replace {WHATSAPP_ACCESS_TOKEN} with actual token in Authorization header
let api_url = format!( let api_url = format!(
"https://graph.facebook.com/v18.0/{}/messages", "https://graph.facebook.com/{}/{}/messages",
metadata.phone_number_id api_version, metadata.phone_number_id
); );
// Build sendMessage payload // Build sendMessage payload
+13
View File
@@ -0,0 +1,13 @@
[Unit]
Description=Cloud SQL Auth Proxy
After=network.target
[Service]
Type=simple
DynamicUser=yes
ExecStart=/usr/local/bin/cloud-sql-proxy ironclaw-prod:us-central1:ironclaw-db --port=5432
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
+27
View File
@@ -0,0 +1,27 @@
# WARNING: Replace all CHANGE_ME values before deploying.
# Do not use placeholder passwords in production.
DATABASE_URL=postgres://ironclaw:CHANGE_ME@localhost:5432/ironclaw
# NEAR AI
NEARAI_SESSION_TOKEN=CHANGE_ME
NEARAI_MODEL=claude-3-5-sonnet-20241022
NEARAI_BASE_URL=https://cloud-api.near.ai
NEARAI_AUTH_URL=https://private.near.ai
NEARAI_API_MODE=chat_completions
# Agent
AGENT_NAME=ironclaw
CLI_ENABLED=false
# Web Gateway
GATEWAY_ENABLED=true
# 0.0.0.0 binds to all interfaces (required for Docker --network=host).
# Use 127.0.0.1 if running outside Docker or for local-only access.
GATEWAY_HOST=0.0.0.0
GATEWAY_PORT=3000
GATEWAY_AUTH_TOKEN=CHANGE_ME
# Disabled for initial deploy
SANDBOX_ENABLED=false
HEARTBEAT_ENABLED=false
EMBEDDING_ENABLED=false
+20
View File
@@ -0,0 +1,20 @@
[Unit]
Description=IronClaw AI Assistant
After=cloud-sql-proxy.service docker.service
Requires=cloud-sql-proxy.service
[Service]
Type=simple
ExecStartPre=/usr/bin/docker pull us-central1-docker.pkg.dev/ironclaw-prod/ironclaw/agent:latest
ExecStart=/usr/bin/docker run --rm \
--name ironclaw \
--env-file /opt/ironclaw/.env \
--network=host \
us-central1-docker.pkg.dev/ironclaw-prod/ironclaw/agent:latest \
--no-onboard
ExecStop=/usr/bin/docker stop ironclaw
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# VM bootstrap script for IronClaw on GCP Compute Engine.
#
# Run on a fresh Debian 12 VM after SSH:
# sudo bash setup.sh
#
# Prerequisites:
# - VM has the ironclaw-vm service account attached
# - Cloud SQL Auth Proxy accessible via IAM
# - Artifact Registry image pushed
set -euo pipefail
# Must run as root
if [ "$(id -u)" -ne 0 ]; then
echo "ERROR: This script must be run as root (sudo bash setup.sh)"
exit 1
fi
echo "==> Installing Docker"
apt-get update
apt-get install -y docker.io
systemctl enable docker
systemctl start docker
echo "==> Installing Cloud SQL Auth Proxy"
curl -fsSL -o /usr/local/bin/cloud-sql-proxy \
https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.14.3/cloud-sql-proxy.linux.amd64
chmod +x /usr/local/bin/cloud-sql-proxy
echo "==> Installing systemd services"
cp /tmp/deploy/cloud-sql-proxy.service /etc/systemd/system/
cp /tmp/deploy/ironclaw.service /etc/systemd/system/
systemctl daemon-reload
echo "==> Starting Cloud SQL Auth Proxy"
systemctl enable cloud-sql-proxy
systemctl start cloud-sql-proxy
echo "==> Configuring Docker registry auth"
# The VM service account provides Artifact Registry access
gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
echo "==> Creating config directory"
# Owned by root, readable only by root. Docker reads --env-file as root
# before dropping to uid 1000 (ironclaw) inside the container.
mkdir -p /opt/ironclaw
chmod 700 /opt/ironclaw
if [ ! -f /opt/ironclaw/.env ]; then
echo "WARNING: /opt/ironclaw/.env does not exist."
echo "Create it with your configuration before starting IronClaw."
echo "See deploy/env.example for the required variables."
echo ""
echo "Then run: systemctl enable ironclaw && systemctl start ironclaw"
else
chmod 600 /opt/ironclaw/.env
echo "==> Starting IronClaw"
systemctl enable ironclaw
systemctl start ironclaw
fi
echo "==> Setup complete"
echo ""
echo "Verify with:"
echo " systemctl status cloud-sql-proxy"
echo " systemctl status ironclaw"
echo " docker logs ironclaw"
-1
View File
@@ -81,7 +81,6 @@ async fn main() -> anyhow::Result<()> {
let session = create_session_manager(SessionConfig { let session = create_session_manager(SessionConfig {
auth_base_url: config.llm.nearai.auth_base_url.clone(), auth_base_url: config.llm.nearai.auth_base_url.clone(),
session_path: config.llm.nearai.session_path.clone(), session_path: config.llm.nearai.session_path.clone(),
..Default::default()
}) })
.await; .await;
let llm = create_llm_provider(&config.llm, session)?; let llm = create_llm_provider(&config.llm, session)?;
+121 -50
View File
@@ -28,7 +28,7 @@ use crate::tools::ToolRegistry;
use crate::workspace::Workspace; use crate::workspace::Workspace;
/// Collapse a tool output string into a single-line preview for display. /// Collapse a tool output string into a single-line preview for display.
fn truncate_for_preview(output: &str, max_chars: usize) -> String { pub(crate) fn truncate_for_preview(output: &str, max_chars: usize) -> String {
let collapsed: String = output let collapsed: String = output
.chars() .chars()
.take(max_chars + 50) .take(max_chars + 50)
@@ -37,8 +37,14 @@ fn truncate_for_preview(output: &str, max_chars: usize) -> String {
.split_whitespace() .split_whitespace()
.collect::<Vec<_>>() .collect::<Vec<_>>()
.join(" "); .join(" ");
if collapsed.len() > max_chars { // char_indices gives us byte offsets at char boundaries, so the slice is always valid UTF-8.
format!("{}...", &collapsed[..max_chars]) if collapsed.chars().count() > max_chars {
let byte_offset = collapsed
.char_indices()
.nth(max_chars)
.map(|(i, _)| i)
.unwrap_or(collapsed.len());
format!("{}...", &collapsed[..byte_offset])
} else { } else {
collapsed collapsed
} }
@@ -654,9 +660,9 @@ impl Agent {
} }
// Restore response chain from conversation metadata // Restore response chain from conversation metadata
if let Some(store) = self.store() { if let Some(store) = self.store()
if let Ok(Some(metadata)) = store.get_conversation_metadata(thread_uuid).await { && let Ok(Some(metadata)) = store.get_conversation_metadata(thread_uuid).await
if let Some(rid) = metadata && let Some(rid) = metadata
.get("last_response_id") .get("last_response_id")
.and_then(|v| v.as_str()) .and_then(|v| v.as_str())
.map(String::from) .map(String::from)
@@ -666,8 +672,6 @@ impl Agent {
.seed_response_chain(&thread_uuid.to_string(), rid); .seed_response_chain(&thread_uuid.to_string(), rid);
tracing::debug!("Restored response chain for thread {}", thread_uuid); tracing::debug!("Restored response chain for thread {}", thread_uuid);
} }
}
}
// Insert into session and register with session manager // Insert into session and register with session manager
{ {
@@ -954,14 +958,13 @@ impl Agent {
return; return;
} }
if let Some(ref resp) = response { if let Some(ref resp) = response
if let Err(e) = store && let Err(e) = store
.add_conversation_message(thread_id, "assistant", resp) .add_conversation_message(thread_id, "assistant", resp)
.await .await
{ {
tracing::warn!("Failed to persist assistant message: {}", e); tracing::warn!("Failed to persist assistant message: {}", e);
} }
}
}); });
} }
@@ -1058,8 +1061,9 @@ impl Agent {
// Check if interrupted // Check if interrupted
{ {
let sess = session.lock().await; let sess = session.lock().await;
if let Some(thread) = sess.threads.get(&thread_id) { if let Some(thread) = sess.threads.get(&thread_id)
if thread.state == ThreadState::Interrupted { && thread.state == ThreadState::Interrupted
{
return Err(crate::error::JobError::ContextError { return Err(crate::error::JobError::ContextError {
id: thread_id, id: thread_id,
reason: "Interrupted".to_string(), reason: "Interrupted".to_string(),
@@ -1067,7 +1071,6 @@ impl Agent {
.into()); .into());
} }
} }
}
// Refresh tool definitions each iteration so newly built tools become visible // Refresh tool definitions each iteration so newly built tools become visible
let tool_defs = self.tools().tool_definitions().await; let tool_defs = self.tools().tool_definitions().await;
@@ -1140,20 +1143,21 @@ impl Agent {
// Record tool calls in the thread // Record tool calls in the thread
{ {
let mut sess = session.lock().await; let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) { if let Some(thread) = sess.threads.get_mut(&thread_id)
if let Some(turn) = thread.last_turn_mut() { && let Some(turn) = thread.last_turn_mut()
{
for tc in &tool_calls { for tc in &tool_calls {
turn.record_tool_call(&tc.name, tc.arguments.clone()); turn.record_tool_call(&tc.name, tc.arguments.clone());
} }
} }
} }
}
// Execute each tool (with approval checking) // Execute each tool (with approval checking)
for tc in tool_calls { for tc in tool_calls {
// Check if tool requires approval // Check if tool requires approval
if let Some(tool) = self.tools().get(&tc.name).await { if let Some(tool) = self.tools().get(&tc.name).await
if tool.requires_approval() { && tool.requires_approval()
{
// Check if auto-approved for this session // Check if auto-approved for this session
let mut is_auto_approved = { let mut is_auto_approved = {
let sess = session.lock().await; let sess = session.lock().await;
@@ -1163,9 +1167,14 @@ impl Agent {
// For shell commands, override auto-approval for // For shell commands, override auto-approval for
// destructive patterns that should always require // destructive patterns that should always require
// explicit per-invocation approval. // explicit per-invocation approval.
if is_auto_approved && tc.name == "shell" { if is_auto_approved
if let Some(cmd) = tc && tc.name == "shell"
&& let Some(cmd) = tc
.arguments .arguments
.get("command")
.and_then(|c| c.as_str().map(String::from))
.or_else(|| {
tc.arguments
.as_str() .as_str()
.and_then(|s| { .and_then(|s| {
serde_json::from_str::<serde_json::Value>(s).ok() serde_json::from_str::<serde_json::Value>(s).ok()
@@ -1174,18 +1183,15 @@ impl Agent {
v.get("command") v.get("command")
.and_then(|c| c.as_str().map(String::from)) .and_then(|c| c.as_str().map(String::from))
}) })
})
&& crate::tools::builtin::shell::requires_explicit_approval(&cmd)
{ {
if crate::tools::builtin::shell::requires_explicit_approval(
&cmd,
) {
tracing::info!( tracing::info!(
"Shell command '{}' requires explicit approval despite auto-approve", "Shell command '{}' requires explicit approval despite auto-approve",
cmd.chars().take(80).collect::<String>() cmd.chars().take(80).collect::<String>()
); );
is_auto_approved = false; is_auto_approved = false;
} }
}
}
if !is_auto_approved { if !is_auto_approved {
// Need approval - store pending request and return // Need approval - store pending request and return
@@ -1201,7 +1207,6 @@ impl Agent {
return Ok(AgenticLoopResult::NeedApproval { pending }); return Ok(AgenticLoopResult::NeedApproval { pending });
} }
} }
}
let _ = self let _ = self
.channels .channels
@@ -1230,27 +1235,28 @@ impl Agent {
) )
.await; .await;
if let Ok(ref output) = tool_result { if let Ok(ref output) = tool_result
if !output.is_empty() { && !output.is_empty()
{
let _ = self let _ = self
.channels .channels
.send_status( .send_status(
&message.channel, &message.channel,
StatusUpdate::ToolResult { StatusUpdate::ToolResult {
name: tc.name.clone(), name: tc.name.clone(),
preview: truncate_for_preview(output, 200), preview: output.clone(),
}, },
&message.metadata, &message.metadata,
) )
.await; .await;
} }
}
// Record result in thread // Record result in thread
{ {
let mut sess = session.lock().await; let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) { if let Some(thread) = sess.threads.get_mut(&thread_id)
if let Some(turn) = thread.last_turn_mut() { && let Some(turn) = thread.last_turn_mut()
{
match &tool_result { match &tool_result {
Ok(output) => { Ok(output) => {
turn.record_tool_result(serde_json::json!(output)); turn.record_tool_result(serde_json::json!(output));
@@ -1261,7 +1267,6 @@ impl Agent {
} }
} }
} }
}
// If tool_auth returned awaiting_token, enter auth mode // If tool_auth returned awaiting_token, enter auth mode
// and short-circuit: return the instructions directly so // and short-circuit: return the instructions directly so
@@ -1640,8 +1645,9 @@ impl Agent {
}; };
// Verify request ID if provided // Verify request ID if provided
if let Some(req_id) = request_id { if let Some(req_id) = request_id
if req_id != pending.request_id { && req_id != pending.request_id
{
// Put it back and return error // Put it back and return error
let mut sess = session.lock().await; let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) { if let Some(thread) = sess.threads.get_mut(&thread_id) {
@@ -1651,7 +1657,6 @@ impl Agent {
"Request ID mismatch. Use the correct request ID.", "Request ID mismatch. Use the correct request ID.",
)); ));
} }
}
if approved { if approved {
// If always, add to auto-approved set // If always, add to auto-approved set
@@ -1704,21 +1709,21 @@ impl Agent {
) )
.await; .await;
if let Ok(ref output) = tool_result { if let Ok(ref output) = tool_result
if !output.is_empty() { && !output.is_empty()
{
let _ = self let _ = self
.channels .channels
.send_status( .send_status(
&message.channel, &message.channel,
StatusUpdate::ToolResult { StatusUpdate::ToolResult {
name: pending.tool_name.clone(), name: pending.tool_name.clone(),
preview: truncate_for_preview(output, 200), preview: output.clone(),
}, },
&message.metadata, &message.metadata,
) )
.await; .await;
} }
}
// Build context including the tool result // Build context including the tool result
let mut context_messages = pending.context_messages; let mut context_messages = pending.context_messages;
@@ -1726,8 +1731,9 @@ impl Agent {
// Record result in thread // Record result in thread
{ {
let mut sess = session.lock().await; let mut sess = session.lock().await;
if let Some(thread) = sess.threads.get_mut(&thread_id) { if let Some(thread) = sess.threads.get_mut(&thread_id)
if let Some(turn) = thread.last_turn_mut() { && let Some(turn) = thread.last_turn_mut()
{
match &tool_result { match &tool_result {
Ok(output) => { Ok(output) => {
turn.record_tool_result(serde_json::json!(output)); turn.record_tool_result(serde_json::json!(output));
@@ -1738,7 +1744,6 @@ impl Agent {
} }
} }
} }
}
// If tool_auth returned awaiting_token, enter auth mode and // If tool_auth returned awaiting_token, enter auth mode and
// return instructions directly (skip agentic loop continuation). // return instructions directly (skip agentic loop continuation).
@@ -2094,8 +2099,9 @@ impl Agent {
} }
// Persist new job to database (fire-and-forget) // Persist new job to database (fire-and-forget)
if let Some(store) = self.store() { if let Some(store) = self.store()
if let Ok(ctx) = self.context_manager.get_context(job_id).await { && let Ok(ctx) = self.context_manager.get_context(job_id).await
{
let store = store.clone(); let store = store.clone();
tokio::spawn(async move { tokio::spawn(async move {
if let Err(e) = store.save_job(&ctx).await { if let Err(e) = store.save_job(&ctx).await {
@@ -2103,7 +2109,6 @@ impl Agent {
} }
}); });
} }
}
// Schedule for execution // Schedule for execution
self.scheduler.schedule(job_id).await?; self.scheduler.schedule(job_id).await?;
@@ -2182,12 +2187,12 @@ impl Agent {
let mut output = String::from("Jobs:\n"); let mut output = String::from("Jobs:\n");
for job_id in jobs { for job_id in jobs {
if let Ok(ctx) = self.context_manager.get_context(job_id).await { if let Ok(ctx) = self.context_manager.get_context(job_id).await
if ctx.user_id == user_id { && ctx.user_id == user_id
{
output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state)); output.push_str(&format!(" {} - {} ({:?})\n", job_id, ctx.title, ctx.state));
} }
} }
}
Ok(output) Ok(output)
} }
@@ -2636,4 +2641,70 @@ mod tests {
assert!(detect_auth_awaiting("tool_activate", &result).is_none()); assert!(detect_auth_awaiting("tool_activate", &result).is_none());
} }
// --- truncate_for_preview tests ---
use super::truncate_for_preview;
#[test]
fn test_truncate_short_input() {
assert_eq!(truncate_for_preview("hello", 10), "hello");
}
#[test]
fn test_truncate_empty_input() {
assert_eq!(truncate_for_preview("", 10), "");
}
#[test]
fn test_truncate_exact_length() {
assert_eq!(truncate_for_preview("hello", 5), "hello");
}
#[test]
fn test_truncate_over_limit() {
let result = truncate_for_preview("hello world, this is long", 10);
assert!(result.ends_with("..."));
// "hello worl" = 10 chars + "..."
assert_eq!(result, "hello worl...");
}
#[test]
fn test_truncate_collapses_newlines() {
let result = truncate_for_preview("line1\nline2\nline3", 100);
assert!(!result.contains('\n'));
assert_eq!(result, "line1 line2 line3");
}
#[test]
fn test_truncate_collapses_whitespace() {
let result = truncate_for_preview("hello world", 100);
assert_eq!(result, "hello world");
}
#[test]
fn test_truncate_multibyte_utf8() {
// Each emoji is 4 bytes. Truncating at char boundary must not panic.
let input = "😀😁😂🤣😃😄😅😆😉😊";
let result = truncate_for_preview(input, 5);
assert!(result.ends_with("..."));
// First 5 chars = 5 emoji
assert_eq!(result, "😀😁😂🤣😃...");
}
#[test]
fn test_truncate_cjk_characters() {
// CJK chars are 3 bytes each in UTF-8.
let input = "你好世界测试数据很长的字符串";
let result = truncate_for_preview(input, 4);
assert_eq!(result, "你好世界...");
}
#[test]
fn test_truncate_mixed_multibyte_and_ascii() {
let input = "hello 世界 foo";
let result = truncate_for_preview(input, 8);
// 'h','e','l','l','o',' ','世','界' = 8 chars
assert_eq!(result, "hello 世界...");
}
} }
+1
View File
@@ -26,6 +26,7 @@ pub mod task;
pub mod undo; pub mod undo;
pub mod worker; pub mod worker;
pub(crate) use agent_loop::truncate_for_preview;
pub use agent_loop::{Agent, AgentDeps}; pub use agent_loop::{Agent, AgentDeps};
pub use compaction::{CompactionResult, ContextCompactor}; pub use compaction::{CompactionResult, ContextCompactor};
pub use context_monitor::{CompactionStrategy, ContextBreakdown, ContextMonitor}; pub use context_monitor::{CompactionStrategy, ContextBreakdown, ContextMonitor};
+1 -2
View File
@@ -103,11 +103,10 @@ impl RoutineEngine {
if let Trigger::Event { if let Trigger::Event {
channel: Some(ch), .. channel: Some(ch), ..
} = &routine.trigger } = &routine.trigger
&& ch != &message.channel
{ {
if ch != &message.channel {
continue; continue;
} }
}
// Regex match // Regex match
if !re.is_match(&message.content) { if !re.is_match(&message.content) {
+3 -3
View File
@@ -119,8 +119,9 @@ impl SelfRepair for DefaultSelfRepair {
let mut stuck_jobs = Vec::new(); let mut stuck_jobs = Vec::new();
for job_id in stuck_ids { for job_id in stuck_ids {
if let Ok(ctx) = self.context_manager.get_context(job_id).await { if let Ok(ctx) = self.context_manager.get_context(job_id).await
if ctx.state == JobState::Stuck { && ctx.state == JobState::Stuck
{
let stuck_duration = ctx let stuck_duration = ctx
.started_at .started_at
.map(|start| { .map(|start| {
@@ -139,7 +140,6 @@ impl SelfRepair for DefaultSelfRepair {
}); });
} }
} }
}
stuck_jobs stuck_jobs
} }
+3 -3
View File
@@ -346,12 +346,12 @@ impl Thread {
let mut turn = Turn::new(turn_number, &msg.content); let mut turn = Turn::new(turn_number, &msg.content);
// Check if next is assistant response // Check if next is assistant response
if let Some(next) = iter.peek() { if let Some(next) = iter.peek()
if next.role == crate::llm::Role::Assistant { && next.role == crate::llm::Role::Assistant
{
let response = iter.next().expect("peeked"); let response = iter.next().expect("peeked");
turn.complete(&response.content); turn.complete(&response.content);
} }
}
self.turns.push(turn); self.turns.push(turn);
turn_number += 1; turn_number += 1;
+3 -3
View File
@@ -199,13 +199,13 @@ impl SessionManager {
{ {
let sessions = self.sessions.read().await; let sessions = self.sessions.read().await;
for user_id in &stale_users { for user_id in &stale_users {
if let Some(session) = sessions.get(user_id) { if let Some(session) = sessions.get(user_id)
if let Ok(sess) = session.try_lock() { && let Ok(sess) = session.try_lock()
{
stale_thread_ids.extend(sess.threads.keys()); stale_thread_ids.extend(sess.threads.keys());
} }
} }
} }
}
// Remove sessions // Remove sessions
let count = { let count = {
+10 -11
View File
@@ -93,28 +93,27 @@ impl SubmissionParser {
// /thread <uuid> - switch thread // /thread <uuid> - switch thread
if let Some(rest) = lower.strip_prefix("/thread ") { if let Some(rest) = lower.strip_prefix("/thread ") {
let rest = rest.trim(); let rest = rest.trim();
if rest != "new" { if rest != "new"
if let Ok(id) = Uuid::parse_str(rest) { && let Ok(id) = Uuid::parse_str(rest)
{
return Submission::SwitchThread { thread_id: id }; return Submission::SwitchThread { thread_id: id };
} }
} }
}
// /resume <uuid> - resume from checkpoint // /resume <uuid> - resume from checkpoint
if let Some(rest) = lower.strip_prefix("/resume ") { if let Some(rest) = lower.strip_prefix("/resume ")
if let Ok(id) = Uuid::parse_str(rest.trim()) { && let Ok(id) = Uuid::parse_str(rest.trim())
{
return Submission::Resume { checkpoint_id: id }; return Submission::Resume { checkpoint_id: id };
} }
}
// Try structured JSON approval (from web gateway's /api/chat/approval endpoint) // Try structured JSON approval (from web gateway's /api/chat/approval endpoint)
if trimmed.starts_with('{') { if trimmed.starts_with('{')
if let Ok(submission) = serde_json::from_str::<Submission>(trimmed) { && let Ok(submission) = serde_json::from_str::<Submission>(trimmed)
if matches!(submission, Submission::ExecApproval { .. }) { && matches!(submission, Submission::ExecApproval { .. })
{
return submission; return submission;
} }
}
}
// Approval responses (simple yes/no/always for pending approvals) // Approval responses (simple yes/no/always for pending approvals)
// These are short enough to check explicitly // These are short enough to check explicitly
+28 -6
View File
@@ -227,12 +227,12 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
} }
// Check for cancellation // Check for cancellation
if let Ok(ctx) = self.context_manager().get_context(self.job_id).await { if let Ok(ctx) = self.context_manager().get_context(self.job_id).await
if ctx.state == JobState::Cancelled { && ctx.state == JobState::Cancelled
{
tracing::info!("Worker for job {} detected cancellation", self.job_id); tracing::info!("Worker for job {} detected cancellation", self.job_id);
return Ok(()); return Ok(());
} }
}
iteration += 1; iteration += 1;
if iteration > max_iterations { if iteration > max_iterations {
@@ -299,6 +299,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
parameters: tc.arguments.clone(), parameters: tc.arguments.clone(),
reasoning: String::new(), reasoning: String::new(),
alternatives: vec![], alternatives: vec![],
tool_call_id: tc.id.clone(),
}; };
self.process_tool_result(reason_ctx, &selection, result) self.process_tool_result(reason_ctx, &selection, result)
@@ -565,7 +566,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
); );
reason_ctx.messages.push(ChatMessage::tool_result( reason_ctx.messages.push(ChatMessage::tool_result(
"tool_call_id", &selection.tool_call_id,
&selection.tool_name, &selection.tool_name,
wrapped, wrapped,
)); ));
@@ -597,7 +598,7 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
} }
reason_ctx.messages.push(ChatMessage::tool_result( reason_ctx.messages.push(ChatMessage::tool_result(
"tool_call_id", &selection.tool_call_id,
&selection.tool_name, &selection.tool_name,
format!("Error: {}", e), format!("Error: {}", e),
)); ));
@@ -647,12 +648,15 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
.execute_tool(&action.tool_name, &action.parameters) .execute_tool(&action.tool_name, &action.parameters)
.await; .await;
// Create a synthetic ToolSelection for process_tool_result // Create a synthetic ToolSelection for process_tool_result.
// Plan actions don't originate from an LLM tool_call response so
// there is no real tool_call_id; generate a unique one.
let selection = ToolSelection { let selection = ToolSelection {
tool_name: action.tool_name.clone(), tool_name: action.tool_name.clone(),
parameters: action.parameters.clone(), parameters: action.parameters.clone(),
reasoning: action.reasoning.clone(), reasoning: action.reasoning.clone(),
alternatives: vec![], alternatives: vec![],
tool_call_id: format!("plan_{}_{}", self.job_id, i),
}; };
// Process the result // Process the result
@@ -774,8 +778,26 @@ impl From<TaskOutput> for Result<String, Error> {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::llm::ToolSelection;
use crate::util::llm_signals_completion; use crate::util::llm_signals_completion;
#[test]
fn test_tool_selection_preserves_call_id() {
let selection = ToolSelection {
tool_name: "memory_search".to_string(),
parameters: serde_json::json!({"query": "test"}),
reasoning: "Need to search memory".to_string(),
alternatives: vec![],
tool_call_id: "call_abc123".to_string(),
};
assert_eq!(selection.tool_call_id, "call_abc123");
assert_ne!(
selection.tool_call_id, "tool_call_id",
"tool_call_id must not be the hardcoded placeholder string"
);
}
#[test] #[test]
fn test_completion_positive_signals() { fn test_completion_positive_signals() {
assert!(llm_signals_completion("The job is complete.")); assert!(llm_signals_completion("The job is complete."));
+295 -156
View File
@@ -1,147 +1,145 @@
//! Bootstrap configuration for IronClaw. //! Bootstrap helpers for IronClaw.
//! //!
//! These are the only settings that MUST live on disk because they're needed //! The only setting that truly needs disk persistence before the database is
//! before the database connection is established. Everything else lives in the //! available is `DATABASE_URL` (chicken-and-egg: can't connect to DB without
//! `settings` table in PostgreSQL. //! it). Everything else is auto-detected or read from env vars.
//! //!
//! File: `~/.ironclaw/bootstrap.json` //! File: `~/.ironclaw/.env` (standard dotenvy format)
use std::path::PathBuf; use std::path::PathBuf;
use serde::{Deserialize, Serialize}; /// Path to the IronClaw-specific `.env` file: `~/.ironclaw/.env`.
pub fn ironclaw_env_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join(".env")
}
use crate::settings::KeySource; /// Load env vars from `~/.ironclaw/.env` (in addition to the standard `.env`).
/// Minimal config needed to connect to the database and decrypt secrets.
/// ///
/// This is the only JSON file IronClaw reads from disk at startup. /// Call this **after** `dotenvy::dotenv()` so that the standard `./.env`
/// All other configuration lives in the `settings` table in PostgreSQL. /// takes priority over `~/.ironclaw/.env`. dotenvy never overwrites
#[derive(Debug, Clone, Serialize, Deserialize)] /// existing env vars, so the effective priority is:
pub struct BootstrapConfig { ///
/// Database connection URL (postgres://...). /// explicit env vars > `./.env` > `~/.ironclaw/.env`
#[serde(default)] ///
pub database_url: Option<String>, /// If `~/.ironclaw/.env` doesn't exist but the legacy `bootstrap.json` does,
/// extracts `DATABASE_URL` from it and writes the `.env` file (one-time
/// upgrade from the old config format).
pub fn load_ironclaw_env() {
let path = ironclaw_env_path();
/// Database connection pool size. if !path.exists() {
#[serde(default)] // One-time upgrade: extract DATABASE_URL from legacy bootstrap.json
pub database_pool_size: Option<usize>, migrate_bootstrap_json_to_env(&path);
/// Source for the secrets master key.
#[serde(default)]
pub secrets_master_key_source: KeySource,
/// Whether onboarding wizard has been completed.
#[serde(default)]
pub onboard_completed: bool,
} }
impl Default for BootstrapConfig { if path.exists() {
fn default() -> Self { let _ = dotenvy::from_path(&path);
Self {
database_url: None,
database_pool_size: None,
secrets_master_key_source: KeySource::None,
onboard_completed: false,
}
} }
} }
impl BootstrapConfig { /// If `bootstrap.json` exists, pull `database_url` out of it and write `.env`.
/// Default bootstrap file path: `~/.ironclaw/bootstrap.json`. fn migrate_bootstrap_json_to_env(env_path: &std::path::Path) {
pub fn default_path() -> PathBuf { let ironclaw_dir = env_path
dirs::home_dir() .parent()
.unwrap_or_else(|| PathBuf::from(".")) .unwrap_or_else(|| std::path::Path::new("."));
.join(".ironclaw") let bootstrap_path = ironclaw_dir.join("bootstrap.json");
.join("bootstrap.json")
if !bootstrap_path.exists() {
return;
} }
/// Legacy settings.json path (for migration detection). let content = match std::fs::read_to_string(&bootstrap_path) {
pub fn legacy_settings_path() -> PathBuf { Ok(c) => c,
dirs::home_dir() Err(_) => return,
.unwrap_or_else(|| PathBuf::from(".")) };
.join(".ironclaw")
.join("settings.json")
}
/// Load from the default path, falling back to legacy settings.json, // Minimal parse: just grab database_url from the JSON
/// then to defaults if neither exists. let parsed: serde_json::Value = match serde_json::from_str(&content) {
pub fn load() -> Self { Ok(v) => v,
let bootstrap_path = Self::default_path(); Err(_) => return,
if bootstrap_path.exists() { };
return Self::load_from(&bootstrap_path);
}
// Fall back to legacy settings.json (extract just the 4 bootstrap fields) if let Some(url) = parsed.get("database_url").and_then(|v| v.as_str()) {
let legacy_path = Self::legacy_settings_path(); if let Some(parent) = env_path.parent()
if legacy_path.exists() { && let Err(e) = std::fs::create_dir_all(parent)
return Self::load_from_legacy(&legacy_path); {
eprintln!("Warning: failed to create {}: {}", parent.display(), e);
return;
} }
if let Err(e) = std::fs::write(env_path, format!("DATABASE_URL=\"{}\"\n", url)) {
Self::default() eprintln!("Warning: failed to migrate bootstrap.json to .env: {}", e);
return;
} }
rename_to_migrated(&bootstrap_path);
/// Load from a specific path. eprintln!(
pub fn load_from(path: &PathBuf) -> Self { "Migrated DATABASE_URL from bootstrap.json to {}",
match std::fs::read_to_string(path) { env_path.display()
Ok(data) => serde_json::from_str(&data).unwrap_or_default(), );
Err(_) => Self::default(),
} }
} }
/// Extract bootstrap fields from a legacy settings.json. /// Write database bootstrap vars to `~/.ironclaw/.env`.
fn load_from_legacy(path: &PathBuf) -> Self { ///
match std::fs::read_to_string(path) { /// These settings form the chicken-and-egg layer: they must be available
Ok(data) => { /// from the filesystem (env vars) BEFORE any database connection, because
// The legacy Settings struct is a superset; serde will ignore extra fields. /// they determine which database to connect to. Everything else is stored
serde_json::from_str(&data).unwrap_or_default() /// in the database itself.
} ///
Err(_) => Self::default(), /// Creates the parent directory if it doesn't exist.
} /// Values are double-quoted so that `#` (common in URL-encoded passwords)
} /// and other shell-special characters are preserved by dotenvy.
pub fn save_bootstrap_env(vars: &[(&str, &str)]) -> std::io::Result<()> {
/// Save to the default path. let path = ironclaw_env_path();
pub fn save(&self) -> std::io::Result<()> {
self.save_to(&Self::default_path())
}
/// Save to a specific path.
pub fn save_to(&self, path: &PathBuf) -> std::io::Result<()> {
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?; std::fs::create_dir_all(parent)?;
} }
let json = serde_json::to_string_pretty(self) let mut content = String::new();
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string()))?; for (key, value) in vars {
std::fs::write(path, json) content.push_str(&format!("{}=\"{}\"\n", key, value));
} }
std::fs::write(&path, content)
} }
/// One-time migration from disk config files to the database settings table. /// Write `DATABASE_URL` to `~/.ironclaw/.env`.
/// ///
/// On first boot after upgrade, checks if: /// Convenience wrapper around `save_bootstrap_env` for single-value migration
/// 1. `~/.ironclaw/settings.json` exists /// paths. Prefer `save_bootstrap_env` for new code.
/// 2. The DB settings table is empty for this user pub fn save_database_url(url: &str) -> std::io::Result<()> {
save_bootstrap_env(&[("DATABASE_URL", url)])
}
/// One-time migration of legacy `~/.ironclaw/settings.json` into the database.
/// ///
/// If both conditions hold, migrates settings, MCP servers, and session data /// Only runs when a `settings.json` exists on disk AND the DB has no settings
/// to the database, writes `bootstrap.json`, and renames old files to `.migrated`. /// yet. After the wizard writes directly to the DB, this path is only hit by
/// users upgrading from the old disk-only configuration.
///
/// After syncing, renames `settings.json` to `.migrated` so it won't trigger again.
pub async fn migrate_disk_to_db( pub async fn migrate_disk_to_db(
store: &dyn crate::db::Database, store: &dyn crate::db::Database,
user_id: &str, user_id: &str,
) -> Result<(), MigrationError> { ) -> Result<(), MigrationError> {
let legacy_settings_path = BootstrapConfig::legacy_settings_path(); let ironclaw_dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let legacy_settings_path = ironclaw_dir.join("settings.json");
if !legacy_settings_path.exists() { if !legacy_settings_path.exists() {
tracing::debug!("No legacy settings.json found, skipping disk-to-DB migration"); tracing::debug!("No legacy settings.json found, skipping disk-to-DB migration");
return Ok(()); return Ok(());
} }
// Only migrate if DB is empty for this user // If DB already has settings, this is not a first boot, the wizard already
// wrote directly to the DB. Just clean up the stale file.
let has_settings = store.has_settings(user_id).await.map_err(|e| { let has_settings = store.has_settings(user_id).await.map_err(|e| {
MigrationError::Database(format!("Failed to check existing settings: {}", e)) MigrationError::Database(format!("Failed to check existing settings: {}", e))
})?; })?;
if has_settings { if has_settings {
tracing::debug!( tracing::info!("DB already has settings, renaming stale settings.json");
"DB already has settings for user '{}', skipping migration", rename_to_migrated(&legacy_settings_path);
user_id
);
return Ok(()); return Ok(());
} }
@@ -160,22 +158,14 @@ pub async fn migrate_disk_to_db(
tracing::info!("Migrated {} settings to database", db_map.len()); tracing::info!("Migrated {} settings to database", db_map.len());
} }
// 2. Write bootstrap.json with the 4 essential fields // 2. Write DATABASE_URL to ~/.ironclaw/.env
let bootstrap = BootstrapConfig { if let Some(ref url) = settings.database_url {
database_url: settings.database_url.clone(), save_database_url(url)
database_pool_size: settings.database_pool_size, .map_err(|e| MigrationError::Io(format!("Failed to write .env: {}", e)))?;
secrets_master_key_source: settings.secrets_master_key_source, tracing::info!("Wrote DATABASE_URL to {}", ironclaw_env_path().display());
onboard_completed: settings.onboard_completed, }
};
bootstrap
.save()
.map_err(|e| MigrationError::Io(format!("Failed to write bootstrap.json: {}", e)))?;
tracing::info!("Wrote bootstrap.json");
// 3. Migrate mcp-servers.json if it exists // 3. Migrate mcp-servers.json if it exists
let ironclaw_dir = dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw");
let mcp_path = ironclaw_dir.join("mcp-servers.json"); let mcp_path = ironclaw_dir.join("mcp-servers.json");
if mcp_path.exists() { if mcp_path.exists() {
match std::fs::read_to_string(&mcp_path) { match std::fs::read_to_string(&mcp_path) {
@@ -211,7 +201,7 @@ pub async fn migrate_disk_to_db(
Ok(content) => match serde_json::from_str::<serde_json::Value>(&content) { Ok(content) => match serde_json::from_str::<serde_json::Value>(&content) {
Ok(value) => { Ok(value) => {
store store
.set_setting(user_id, "nearai.session", &value) .set_setting(user_id, "nearai.session_token", &value)
.await .await
.map_err(|e| { .map_err(|e| {
MigrationError::Database(format!( MigrationError::Database(format!(
@@ -236,12 +226,19 @@ pub async fn migrate_disk_to_db(
// 5. Rename settings.json to .migrated (don't delete, safety net) // 5. Rename settings.json to .migrated (don't delete, safety net)
rename_to_migrated(&legacy_settings_path); rename_to_migrated(&legacy_settings_path);
// 6. Clean up old bootstrap.json if it exists (superseded by .env)
let old_bootstrap = ironclaw_dir.join("bootstrap.json");
if old_bootstrap.exists() {
rename_to_migrated(&old_bootstrap);
tracing::info!("Renamed old bootstrap.json to .migrated");
}
tracing::info!("Disk-to-DB migration complete"); tracing::info!("Disk-to-DB migration complete");
Ok(()) Ok(())
} }
/// Rename a file to `<name>.migrated` as a safety net. /// Rename a file to `<name>.migrated` as a safety net.
fn rename_to_migrated(path: &PathBuf) { fn rename_to_migrated(path: &std::path::Path) {
let mut migrated = path.as_os_str().to_owned(); let mut migrated = path.as_os_str().to_owned();
migrated.push(".migrated"); migrated.push(".migrated");
if let Err(e) = std::fs::rename(path, &migrated) { if let Err(e) = std::fs::rename(path, &migrated) {
@@ -264,62 +261,204 @@ mod tests {
use tempfile::tempdir; use tempfile::tempdir;
#[test] #[test]
fn test_bootstrap_save_load() { fn test_save_and_load_database_url() {
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
let path = dir.path().join("bootstrap.json"); let env_path = dir.path().join(".env");
let config = BootstrapConfig { // Write in the quoted format that save_database_url uses
database_url: Some("postgres://localhost/test".to_string()), let url = "postgres://localhost:5432/ironclaw_test";
database_pool_size: Some(5), std::fs::write(&env_path, format!("DATABASE_URL=\"{}\"\n", url)).unwrap();
secrets_master_key_source: KeySource::Keychain,
onboard_completed: true,
};
config.save_to(&path).unwrap(); // Verify the content is a valid dotenv line (quoted)
let content = std::fs::read_to_string(&env_path).unwrap();
let loaded = BootstrapConfig::load_from(&path);
assert_eq!( assert_eq!(
loaded.database_url, content,
Some("postgres://localhost/test".to_string()) "DATABASE_URL=\"postgres://localhost:5432/ironclaw_test\"\n"
); );
assert_eq!(loaded.database_pool_size, Some(5));
assert_eq!(loaded.secrets_master_key_source, KeySource::Keychain); // Verify dotenvy can parse it (strips quotes automatically)
assert!(loaded.onboard_completed); let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].0, "DATABASE_URL");
assert_eq!(parsed[0].1, url);
} }
#[test] #[test]
fn test_bootstrap_from_legacy_settings() { fn test_save_database_url_with_hash_in_password() {
let dir = tempdir().unwrap(); let dir = tempdir().unwrap();
let path = dir.path().join("settings.json"); let env_path = dir.path().join(".env");
// Write a legacy settings.json with many extra fields // URLs with # in the password are common (URL-encoded special chars).
let legacy = serde_json::json!({ // Without quoting, dotenvy treats # as a comment delimiter.
"database_url": "postgres://localhost/ironclaw", let url = "postgres://user:p%23ss@localhost:5432/ironclaw";
"database_pool_size": 10, std::fs::write(&env_path, format!("DATABASE_URL=\"{}\"\n", url)).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 1);
assert_eq!(parsed[0].0, "DATABASE_URL");
assert_eq!(parsed[0].1, url);
}
#[test]
fn test_save_database_url_creates_parent_dirs() {
let dir = tempdir().unwrap();
let nested = dir.path().join("deep").join("nested");
let env_path = nested.join(".env");
// Parent doesn't exist yet
assert!(!nested.exists());
// The global function uses a fixed path, so we test the logic directly
std::fs::create_dir_all(&nested).unwrap();
std::fs::write(&env_path, "DATABASE_URL=postgres://test\n").unwrap();
assert!(env_path.exists());
let content = std::fs::read_to_string(&env_path).unwrap();
assert!(content.contains("DATABASE_URL=postgres://test"));
}
#[test]
fn test_ironclaw_env_path() {
let path = ironclaw_env_path();
assert!(path.ends_with(".ironclaw/.env"));
}
#[test]
fn test_migrate_bootstrap_json_to_env() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
let bootstrap_path = dir.path().join("bootstrap.json");
// Write a legacy bootstrap.json
let bootstrap_json = serde_json::json!({
"database_url": "postgres://localhost/ironclaw_upgrade",
"database_pool_size": 5,
"secrets_master_key_source": "keychain", "secrets_master_key_source": "keychain",
"onboard_completed": true, "onboard_completed": true
"selected_model": "claude-3-5-sonnet",
"agent": { "name": "testbot", "max_parallel_jobs": 3 },
"heartbeat": { "enabled": true }
}); });
std::fs::write(&path, serde_json::to_string_pretty(&legacy).unwrap()).unwrap(); std::fs::write(
&bootstrap_path,
serde_json::to_string_pretty(&bootstrap_json).unwrap(),
)
.unwrap();
let config = BootstrapConfig::load_from_legacy(&path); assert!(!env_path.exists());
assert!(bootstrap_path.exists());
// Run the migration
migrate_bootstrap_json_to_env(&env_path);
// .env should now exist with DATABASE_URL
assert!(env_path.exists());
let content = std::fs::read_to_string(&env_path).unwrap();
assert_eq!( assert_eq!(
config.database_url, content,
Some("postgres://localhost/ironclaw".to_string()) "DATABASE_URL=\"postgres://localhost/ironclaw_upgrade\"\n"
); );
assert_eq!(config.database_pool_size, Some(10));
assert_eq!(config.secrets_master_key_source, KeySource::Keychain); // bootstrap.json should be renamed to .migrated
assert!(config.onboard_completed); assert!(!bootstrap_path.exists());
assert!(dir.path().join("bootstrap.json.migrated").exists());
} }
#[test] #[test]
fn test_bootstrap_defaults() { fn test_migrate_bootstrap_json_no_database_url() {
let config = BootstrapConfig::default(); let dir = tempdir().unwrap();
assert!(config.database_url.is_none()); let env_path = dir.path().join(".env");
assert!(config.database_pool_size.is_none()); let bootstrap_path = dir.path().join("bootstrap.json");
assert_eq!(config.secrets_master_key_source, KeySource::None);
assert!(!config.onboard_completed); // bootstrap.json with no database_url
let bootstrap_json = serde_json::json!({
"onboard_completed": false
});
std::fs::write(
&bootstrap_path,
serde_json::to_string_pretty(&bootstrap_json).unwrap(),
)
.unwrap();
migrate_bootstrap_json_to_env(&env_path);
// .env should NOT be created
assert!(!env_path.exists());
// bootstrap.json should remain (no migration happened)
assert!(bootstrap_path.exists());
}
#[test]
fn test_migrate_bootstrap_json_missing() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// No bootstrap.json at all
migrate_bootstrap_json_to_env(&env_path);
// Nothing should happen
assert!(!env_path.exists());
}
#[test]
fn test_save_bootstrap_env_multiple_vars() {
let dir = tempdir().unwrap();
let env_path = dir.path().join("nested").join(".env");
std::fs::create_dir_all(env_path.parent().unwrap()).unwrap();
let vars = [
("DATABASE_BACKEND", "libsql"),
("LIBSQL_PATH", "/home/user/.ironclaw/ironclaw.db"),
];
// Write manually to the temp path (save_bootstrap_env uses the global path)
let mut content = String::new();
for (key, value) in &vars {
content.push_str(&format!("{}=\"{}\"\n", key, value));
}
std::fs::write(&env_path, &content).unwrap();
// Verify dotenvy can parse all entries
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
assert_eq!(parsed.len(), 2);
assert_eq!(
parsed[0],
("DATABASE_BACKEND".to_string(), "libsql".to_string())
);
assert_eq!(
parsed[1],
(
"LIBSQL_PATH".to_string(),
"/home/user/.ironclaw/ironclaw.db".to_string()
)
);
}
#[test]
fn test_save_bootstrap_env_overwrites_previous() {
let dir = tempdir().unwrap();
let env_path = dir.path().join(".env");
// Write initial content
std::fs::write(&env_path, "DATABASE_URL=\"postgres://old\"\n").unwrap();
// Overwrite with new vars (simulating save_bootstrap_env behavior)
let content = "DATABASE_BACKEND=\"libsql\"\nLIBSQL_PATH=\"/new/path.db\"\n";
std::fs::write(&env_path, content).unwrap();
let parsed: Vec<(String, String)> = dotenvy::from_path_iter(&env_path)
.unwrap()
.filter_map(|r| r.ok())
.collect();
// Old DATABASE_URL should be gone
assert_eq!(parsed.len(), 2);
assert!(parsed.iter().all(|(k, _)| k != "DATABASE_URL"));
} }
} }
+17 -7
View File
@@ -33,9 +33,16 @@ use termimad::MadSkin;
use tokio::sync::mpsc; use tokio::sync::mpsc;
use tokio_stream::wrappers::ReceiverStream; use tokio_stream::wrappers::ReceiverStream;
use crate::agent::truncate_for_preview;
use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate}; use crate::channels::{Channel, IncomingMessage, MessageStream, OutgoingResponse, StatusUpdate};
use crate::error::ChannelError; use crate::error::ChannelError;
/// Max characters for tool result previews in the terminal.
const CLI_TOOL_RESULT_MAX: usize = 200;
/// Max characters for thinking/status messages in the terminal.
const CLI_STATUS_MAX: usize = 200;
/// Slash commands available in the REPL. /// Slash commands available in the REPL.
const SLASH_COMMANDS: &[&str] = &[ const SLASH_COMMANDS: &[&str] = &[
"/help", "/help",
@@ -261,7 +268,7 @@ impl Channel for ReplChannel {
std::thread::spawn(move || { std::thread::spawn(move || {
// Single message mode: send it and return // Single message mode: send it and return
if let Some(msg) = single_message { if let Some(msg) = single_message {
let incoming = IncomingMessage::new("repl", "user", &msg); let incoming = IncomingMessage::new("repl", "default", &msg);
let _ = tx.blocking_send(incoming); let _ = tx.blocking_send(incoming);
return; return;
} }
@@ -329,21 +336,21 @@ impl Channel for ReplChannel {
_ => {} _ => {}
} }
let msg = IncomingMessage::new("repl", "user", line); let msg = IncomingMessage::new("repl", "default", line);
if tx.blocking_send(msg).is_err() { if tx.blocking_send(msg).is_err() {
break; break;
} }
} }
Err(ReadlineError::Interrupted) => { Err(ReadlineError::Interrupted) => {
// Ctrl+C: send /interrupt // Ctrl+C: send /interrupt
let msg = IncomingMessage::new("repl", "user", "/interrupt"); let msg = IncomingMessage::new("repl", "default", "/interrupt");
if tx.blocking_send(msg).is_err() { if tx.blocking_send(msg).is_err() {
break; break;
} }
} }
Err(ReadlineError::Eof) => { Err(ReadlineError::Eof) => {
// Ctrl+D: send /quit so the agent loop runs graceful shutdown // Ctrl+D: send /quit so the agent loop runs graceful shutdown
let msg = IncomingMessage::new("repl", "user", "/quit"); let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg); let _ = tx.blocking_send(msg);
break; break;
} }
@@ -400,7 +407,8 @@ impl Channel for ReplChannel {
match status { match status {
StatusUpdate::Thinking(msg) => { StatusUpdate::Thinking(msg) => {
eprintln!(" \x1b[90m\u{25CB} {msg}\x1b[0m"); let display = truncate_for_preview(&msg, CLI_STATUS_MAX);
eprintln!(" \x1b[90m\u{25CB} {display}\x1b[0m");
} }
StatusUpdate::ToolStarted { name } => { StatusUpdate::ToolStarted { name } => {
eprintln!(" \x1b[33m\u{25CB} {name}\x1b[0m"); eprintln!(" \x1b[33m\u{25CB} {name}\x1b[0m");
@@ -413,7 +421,8 @@ impl Channel for ReplChannel {
} }
} }
StatusUpdate::ToolResult { name: _, preview } => { StatusUpdate::ToolResult { name: _, preview } => {
eprintln!(" \x1b[90m{preview}\x1b[0m"); let display = truncate_for_preview(&preview, CLI_TOOL_RESULT_MAX);
eprintln!(" \x1b[90m{display}\x1b[0m");
} }
StatusUpdate::StreamChunk(chunk) => { StatusUpdate::StreamChunk(chunk) => {
// Print separator on the false-to-true transition // Print separator on the false-to-true transition
@@ -438,7 +447,8 @@ impl Channel for ReplChannel {
} }
StatusUpdate::Status(msg) => { StatusUpdate::Status(msg) => {
if debug || msg.contains("approval") || msg.contains("Approval") { if debug || msg.contains("approval") || msg.contains("Approval") {
eprintln!(" \x1b[90m{msg}\x1b[0m"); let display = truncate_for_preview(&msg, CLI_STATUS_MAX);
eprintln!(" \x1b[90m{display}\x1b[0m");
} }
} }
StatusUpdate::ApprovalNeeded { StatusUpdate::ApprovalNeeded {
+93 -24
View File
@@ -76,6 +76,9 @@ struct ChannelStoreData {
credentials: HashMap<String, String>, credentials: HashMap<String, String>,
/// Pairing store for DM pairing (guest access control). /// Pairing store for DM pairing (guest access control).
pairing_store: Arc<PairingStore>, pairing_store: Arc<PairingStore>,
/// Dedicated tokio runtime for HTTP requests, lazily initialized.
/// Reused across multiple `http_request` calls within one execution.
http_runtime: Option<tokio::runtime::Runtime>,
} }
impl ChannelStoreData { impl ChannelStoreData {
@@ -96,6 +99,7 @@ impl ChannelStoreData {
table: ResourceTable::new(), table: ResourceTable::new(),
credentials, credentials,
pairing_store, pairing_store,
http_runtime: None,
} }
} }
@@ -134,15 +138,15 @@ impl ChannelStoreData {
if result.contains('{') && result.contains('}') { if result.contains('{') && result.contains('}') {
// Only warn if it looks like an unresolved placeholder (not JSON braces) // Only warn if it looks like an unresolved placeholder (not JSON braces)
let brace_pattern = regex::Regex::new(r"\{[A-Z_]+\}").ok(); let brace_pattern = regex::Regex::new(r"\{[A-Z_]+\}").ok();
if let Some(re) = brace_pattern { if let Some(re) = brace_pattern
if re.is_match(&result) { && re.is_match(&result)
{
tracing::warn!( tracing::warn!(
context = %context, context = %context,
"String may contain unresolved credential placeholders" "String may contain unresolved credential placeholders"
); );
} }
} }
}
result result
} }
@@ -283,10 +287,25 @@ impl near::agent::channel_host::Host for ChannelStoreData {
.map(|h| h.max_response_bytes) .map(|h| h.max_response_bytes)
.unwrap_or(10 * 1024 * 1024); .unwrap_or(10 * 1024 * 1024);
// Make the HTTP request using blocking I/O // Make the HTTP request using a dedicated single-threaded runtime.
// We're already in a spawn_blocking context, so we can use block_on // We're inside spawn_blocking, so we can't rely on the main runtime's
let result = tokio::runtime::Handle::current().block_on(async { // I/O driver (it may be busy with WASM compilation or other startup work).
let client = reqwest::Client::new(); // A dedicated runtime gives us our own I/O driver and avoids contention.
// The runtime is lazily created and reused across calls within one execution.
if self.http_runtime.is_none() {
self.http_runtime = Some(
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.map_err(|e| format!("Failed to create HTTP runtime: {e}"))?,
);
}
let rt = self.http_runtime.as_ref().expect("just initialized");
let result = rt.block_on(async {
let client = reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.map_err(|e| format!("Failed to build HTTP client: {e}"))?;
let mut request = match method.to_uppercase().as_str() { let mut request = match method.to_uppercase().as_str() {
"GET" => client.get(&url), "GET" => client.get(&url),
@@ -308,9 +327,9 @@ impl near::agent::channel_host::Host for ChannelStoreData {
request = request.body(body_bytes); request = request.body(body_bytes);
} }
// Send request with caller-specified timeout (default 30s). // Send request with caller-specified timeout (default 30s, max 5min).
// Cap at callback_timeout to prevent outliving the host wrapper. let timeout_ms = timeout_ms.unwrap_or(30_000).min(300_000) as u64;
let timeout = std::time::Duration::from_millis(timeout_ms.unwrap_or(30_000) as u64); let timeout = std::time::Duration::from_millis(timeout_ms);
let response = request.timeout(timeout).send().await.map_err(|e| { let response = request.timeout(timeout).send().await.map_err(|e| {
// Walk the full error chain so we get the actual root cause // Walk the full error chain so we get the actual root cause
// (DNS, TLS, connection refused, etc.) instead of just // (DNS, TLS, connection refused, etc.) instead of just
@@ -338,14 +357,14 @@ impl near::agent::channel_host::Host for ChannelStoreData {
// Enforce max response body size to prevent memory exhaustion. // Enforce max response body size to prevent memory exhaustion.
let max_response = max_response_bytes; let max_response = max_response_bytes;
if let Some(cl) = response.content_length() { if let Some(cl) = response.content_length()
if cl as usize > max_response { && cl as usize > max_response
{
return Err(format!( return Err(format!(
"Response body too large: {} bytes exceeds limit of {} bytes", "Response body too large: {} bytes exceeds limit of {} bytes",
cl, max_response cl, max_response
)); ));
} }
}
let body = response let body = response
.bytes() .bytes()
.await .await
@@ -795,7 +814,21 @@ impl WasmChannel {
.await; .await;
match result { match result {
Ok(Ok((config, _host_state))) => { Ok(Ok((config, mut host_state))) => {
// Surface WASM guest logs (errors/warnings from webhook setup, etc.)
for entry in host_state.take_logs() {
match entry.level {
crate::tools::wasm::LogLevel::Error => {
tracing::error!(channel = %self.name, "{}", entry.message);
}
crate::tools::wasm::LogLevel::Warn => {
tracing::warn!(channel = %self.name, "{}", entry.message);
}
_ => {
tracing::debug!(channel = %self.name, "{}", entry.message);
}
}
}
tracing::info!( tracing::info!(
channel = %self.name, channel = %self.name,
display_name = %config.display_name, display_name = %config.display_name,
@@ -1495,8 +1528,8 @@ impl WasmChannel {
match result { match result {
Ok(emitted_messages) => { Ok(emitted_messages) => {
// Process any emitted messages // Process any emitted messages
if !emitted_messages.is_empty() { if !emitted_messages.is_empty()
if let Err(e) = Self::dispatch_emitted_messages( && let Err(e) = Self::dispatch_emitted_messages(
&channel_name, &channel_name,
emitted_messages, emitted_messages,
&message_tx, &message_tx,
@@ -1509,7 +1542,6 @@ impl WasmChannel {
); );
} }
} }
}
Err(e) => { Err(e) => {
tracing::warn!( tracing::warn!(
channel = %channel_name, channel = %channel_name,
@@ -1738,8 +1770,9 @@ impl Channel for WasmChannel {
*self.endpoints.write().await = endpoints; *self.endpoints.write().await = endpoints;
// Start polling if configured // Start polling if configured
if let Some(poll_config) = &config.poll { if let Some(poll_config) = &config.poll
if poll_config.enabled { && poll_config.enabled
{
let interval = self let interval = self
.capabilities .capabilities
.validate_poll_interval(poll_config.interval_ms) .validate_poll_interval(poll_config.interval_ms)
@@ -1754,7 +1787,6 @@ impl Channel for WasmChannel {
self.start_polling(Duration::from_millis(interval as u64), poll_shutdown_rx); self.start_polling(Duration::from_millis(interval as u64), poll_shutdown_rx);
} }
}
tracing::info!( tracing::info!(
channel = %self.name, channel = %self.name,
@@ -2616,15 +2648,52 @@ mod tests {
assert_eq!(store.redact_credentials(input), input); assert_eq!(store.redact_credentials(input), input);
} }
/// Verify that the block_on-inside-spawn_blocking pattern used by the WASM /// Verify that WASM HTTP host functions work using a dedicated
/// channel HTTP host function doesn't deadlock or panic. /// current-thread runtime inside spawn_blocking.
#[tokio::test] #[tokio::test]
async fn test_block_on_inside_spawn_blocking_does_not_deadlock() { async fn test_dedicated_runtime_inside_spawn_blocking() {
let result = tokio::task::spawn_blocking(|| { let result = tokio::task::spawn_blocking(|| {
tokio::runtime::Handle::current().block_on(async { 42 }) let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("failed to build runtime");
rt.block_on(async { 42 })
}) })
.await .await
.expect("spawn_blocking panicked"); .expect("spawn_blocking panicked");
assert_eq!(result, 42); assert_eq!(result, 42);
} }
/// Verify a real HTTP request works using the dedicated-runtime pattern.
/// This catches DNS, TLS, and I/O driver issues that trivial tests miss.
#[tokio::test]
#[ignore] // requires network
async fn test_dedicated_runtime_real_http() {
let result = tokio::task::spawn_blocking(|| {
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("failed to build runtime");
rt.block_on(async {
let client = reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(10))
.build()
.expect("failed to build client");
let resp = client
.get("https://api.telegram.org/bot000/getMe")
.timeout(std::time::Duration::from_secs(10))
.send()
.await;
match resp {
Ok(r) => r.status().as_u16(),
Err(e) if e.is_timeout() => panic!("request timed out: {e}"),
Err(e) => panic!("unexpected error: {e}"),
}
})
})
.await
.expect("spawn_blocking panicked");
// 404 because "000" is not a valid bot token
assert_eq!(result, 404);
}
} }
+8 -10
View File
@@ -25,26 +25,24 @@ pub async fn auth_middleware(
next: Next, next: Next,
) -> Response { ) -> Response {
// Try Authorization header first (constant-time comparison) // Try Authorization header first (constant-time comparison)
if let Some(auth_header) = headers.get("authorization") { if let Some(auth_header) = headers.get("authorization")
if let Ok(value) = auth_header.to_str() { && let Ok(value) = auth_header.to_str()
if let Some(token) = value.strip_prefix("Bearer ") { && let Some(token) = value.strip_prefix("Bearer ")
if bool::from(token.as_bytes().ct_eq(auth.token.as_bytes())) { && bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await; return next.run(request).await;
} }
}
}
}
// Fall back to query parameter for SSE EventSource (constant-time comparison) // Fall back to query parameter for SSE EventSource (constant-time comparison)
if let Some(query) = request.uri().query() { if let Some(query) = request.uri().query() {
for pair in query.split('&') { for pair in query.split('&') {
if let Some(token) = pair.strip_prefix("token=") { if let Some(token) = pair.strip_prefix("token=")
if bool::from(token.as_bytes().ct_eq(auth.token.as_bytes())) { && bool::from(token.as_bytes().ct_eq(auth.token.as_bytes()))
{
return next.run(request).await; return next.run(request).await;
} }
} }
} }
}
(StatusCode::UNAUTHORIZED, "Invalid or missing auth token").into_response() (StatusCode::UNAUTHORIZED, "Invalid or missing auth token").into_response()
} }
+6 -6
View File
@@ -473,11 +473,11 @@ pub async fn chat_completions_handler(
if let Some(mt) = req.max_tokens { if let Some(mt) = req.max_tokens {
tool_req = tool_req.with_max_tokens(mt); tool_req = tool_req.with_max_tokens(mt);
} }
if let Some(ref tc) = req.tool_choice { if let Some(ref tc) = req.tool_choice
if let Some(choice) = normalize_tool_choice(tc) { && let Some(choice) = normalize_tool_choice(tc)
{
tool_req = tool_req.with_tool_choice(choice); tool_req = tool_req.with_tool_choice(choice);
} }
}
let resp = llm let resp = llm
.complete_with_tools(tool_req) .complete_with_tools(tool_req)
@@ -591,11 +591,11 @@ async fn handle_streaming(
if let Some(mt) = req.max_tokens { if let Some(mt) = req.max_tokens {
tool_req = tool_req.with_max_tokens(mt); tool_req = tool_req.with_max_tokens(mt);
} }
if let Some(ref tc) = req.tool_choice { if let Some(ref tc) = req.tool_choice
if let Some(choice) = normalize_tool_choice(tc) { && let Some(choice) = normalize_tool_choice(tc)
{
tool_req = tool_req.with_tool_choice(choice); tool_req = tool_req.with_tool_choice(choice);
} }
}
LlmResult::WithTools( LlmResult::WithTools(
llm.complete_with_tools(tool_req) llm.complete_with_tools(tool_req)
.await .await
+26 -27
View File
@@ -525,13 +525,13 @@ pub async fn clear_auth_mode(state: &GatewayState) {
if let Some(ref sm) = state.session_manager { if let Some(ref sm) = state.session_manager {
let session = sm.get_or_create_session(&state.user_id).await; let session = sm.get_or_create_session(&state.user_id).await;
let mut sess = session.lock().await; let mut sess = session.lock().await;
if let Some(thread_id) = sess.active_thread { if let Some(thread_id) = sess.active_thread
if let Some(thread) = sess.threads.get_mut(&thread_id) { && let Some(thread) = sess.threads.get_mut(&thread_id)
{
thread.pending_auth = None; thread.pending_auth = None;
} }
} }
} }
}
async fn chat_events_handler( async fn chat_events_handler(
State(state): State<Arc<GatewayState>>, State(state): State<Arc<GatewayState>>,
@@ -626,8 +626,9 @@ async fn chat_history_handler(
// Verify the thread belongs to the authenticated user before returning any data. // Verify the thread belongs to the authenticated user before returning any data.
// In-memory threads are already scoped by user via session_manager, but DB // In-memory threads are already scoped by user via session_manager, but DB
// lookups could expose another user's conversation if the UUID is guessed. // lookups could expose another user's conversation if the UUID is guessed.
if query.thread_id.is_some() { if query.thread_id.is_some()
if let Some(ref store) = state.store { && let Some(ref store) = state.store
{
let owned = store let owned = store
.conversation_belongs_to_user(thread_id, &state.user_id) .conversation_belongs_to_user(thread_id, &state.user_id)
.await .await
@@ -636,11 +637,11 @@ async fn chat_history_handler(
return Err((StatusCode::NOT_FOUND, "Thread not found".to_string())); return Err((StatusCode::NOT_FOUND, "Thread not found".to_string()));
} }
} }
}
// For paginated requests (before cursor set), always go to DB // For paginated requests (before cursor set), always go to DB
if before_cursor.is_some() { if before_cursor.is_some()
if let Some(ref store) = state.store { && let Some(ref store) = state.store
{
let (messages, has_more) = store let (messages, has_more) = store
.list_conversation_messages_paginated(thread_id, before_cursor, limit as i64) .list_conversation_messages_paginated(thread_id, before_cursor, limit as i64)
.await .await
@@ -655,11 +656,11 @@ async fn chat_history_handler(
oldest_timestamp, oldest_timestamp,
})); }));
} }
}
// Try in-memory first (freshest data for active threads) // Try in-memory first (freshest data for active threads)
if let Some(thread) = sess.threads.get(&thread_id) { if let Some(thread) = sess.threads.get(&thread_id)
if !thread.turns.is_empty() { && !thread.turns.is_empty()
{
let turns: Vec<TurnInfo> = thread let turns: Vec<TurnInfo> = thread
.turns .turns
.iter() .iter()
@@ -689,7 +690,6 @@ async fn chat_history_handler(
oldest_timestamp: None, oldest_timestamp: None,
})); }));
} }
}
// Fall back to DB for historical threads not in memory (paginated) // Fall back to DB for historical threads not in memory (paginated)
if let Some(ref store) = state.store { if let Some(ref store) = state.store {
@@ -738,13 +738,13 @@ fn build_turns_from_db_messages(messages: &[crate::history::ConversationMessage]
}; };
// Check if next message is an assistant response // Check if next message is an assistant response
if let Some(next) = iter.peek() { if let Some(next) = iter.peek()
if next.role == "assistant" { && next.role == "assistant"
{
let assistant_msg = iter.next().expect("peeked"); let assistant_msg = iter.next().expect("peeked");
turn.response = Some(assistant_msg.content.clone()); turn.response = Some(assistant_msg.content.clone());
turn.completed_at = Some(assistant_msg.created_at.to_rfc3339()); turn.completed_at = Some(assistant_msg.created_at.to_rfc3339());
} }
}
// Incomplete turn (user message without response) // Incomplete turn (user message without response)
if turn.response.is_none() { if turn.response.is_none() {
@@ -1126,8 +1126,9 @@ async fn jobs_detail_handler(
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job from DB first, scoped to the authenticated user. // Try sandbox job from DB first, scoped to the authenticated user.
if let Some(ref store) = state.store { if let Some(ref store) = state.store
if let Ok(Some(job)) = store.get_sandbox_job(job_id).await { && let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id { if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string())); return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
} }
@@ -1185,7 +1186,6 @@ async fn jobs_detail_handler(
transitions, transitions,
})); }));
} }
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string())) Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
} }
@@ -1198,18 +1198,19 @@ async fn jobs_cancel_handler(
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Try sandbox job cancellation, scoped to the authenticated user. // Try sandbox job cancellation, scoped to the authenticated user.
if let Some(ref store) = state.store { if let Some(ref store) = state.store
if let Ok(Some(job)) = store.get_sandbox_job(job_id).await { && let Ok(Some(job)) = store.get_sandbox_job(job_id).await
{
if job.user_id != state.user_id { if job.user_id != state.user_id {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string())); return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
} }
if job.status == "running" || job.status == "creating" { if job.status == "running" || job.status == "creating" {
// Stop the container if we have a job manager. // Stop the container if we have a job manager.
if let Some(ref jm) = state.job_manager { if let Some(ref jm) = state.job_manager
if let Err(e) = jm.stop_job(job_id).await { && let Err(e) = jm.stop_job(job_id).await
{
tracing::warn!(job_id = %job_id, error = %e, "Failed to stop container during cancellation"); tracing::warn!(job_id = %job_id, error = %e, "Failed to stop container during cancellation");
} }
}
store store
.update_sandbox_job_status( .update_sandbox_job_status(
job_id, job_id,
@@ -1227,7 +1228,6 @@ async fn jobs_cancel_handler(
"job_id": job_id, "job_id": job_id,
}))); })));
} }
}
Err((StatusCode::NOT_FOUND, "Job not found".to_string())) Err((StatusCode::NOT_FOUND, "Job not found".to_string()))
} }
@@ -1334,15 +1334,14 @@ async fn jobs_prompt_handler(
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?; .map_err(|_| (StatusCode::BAD_REQUEST, "Invalid job ID".to_string()))?;
// Verify user owns this job. // Verify user owns this job.
if let Some(ref store) = state.store { if let Some(ref store) = state.store
if !store && !store
.sandbox_job_belongs_to_user(job_id, &state.user_id) .sandbox_job_belongs_to_user(job_id, &state.user_id)
.await .await
.unwrap_or(false) .unwrap_or(false)
{ {
return Err((StatusCode::NOT_FOUND, "Job not found".to_string())); return Err((StatusCode::NOT_FOUND, "Job not found".to_string()));
} }
}
let content = body let content = body
.get("content") .get("content")
+15 -47
View File
@@ -48,8 +48,6 @@ pub enum ConfigCommand {
/// Connects to the database to read/write settings. Falls back to disk /// Connects to the database to read/write settings. Falls back to disk
/// if the database is not available. /// if the database is not available.
pub async fn run_config_command(cmd: ConfigCommand) -> anyhow::Result<()> { pub async fn run_config_command(cmd: ConfigCommand) -> anyhow::Result<()> {
let _ = dotenvy::dotenv();
// Try to connect to the DB for settings access // Try to connect to the DB for settings access
let db: Option<Arc<dyn crate::db::Database>> = match connect_db().await { let db: Option<Arc<dyn crate::db::Database>> = match connect_db().await {
Ok(d) => Some(d), Ok(d) => Some(d),
@@ -92,7 +90,7 @@ async fn load_settings(store: Option<&dyn crate::db::Database>) -> Settings {
_ => {} _ => {}
} }
} }
Settings::load() Settings::default()
} }
/// List all settings. /// List all settings.
@@ -110,11 +108,11 @@ async fn list_settings(
println!(); println!();
for (key, value) in all { for (key, value) in all {
if let Some(ref f) = filter { if let Some(ref f) = filter
if !key.starts_with(f) { && !key.starts_with(f)
{
continue; continue;
} }
}
let display_value = if value.len() > 60 { let display_value = if value.len() > 60 {
format!("{}...", &value[..57]) format!("{}...", &value[..57])
@@ -155,8 +153,9 @@ async fn set_setting(
.set(path, value) .set(path, value)
.map_err(|e| anyhow::anyhow!("{}", e))?; .map_err(|e| anyhow::anyhow!("{}", e))?;
// Save to DB if available, otherwise disk let store = store.ok_or_else(|| {
if let Some(store) = store { anyhow::anyhow!("Database connection required to save settings. Check DATABASE_URL.")
})?;
let json_value = match serde_json::from_str::<serde_json::Value>(value) { let json_value = match serde_json::from_str::<serde_json::Value>(value) {
Ok(v) => v, Ok(v) => v,
Err(_) => serde_json::Value::String(value.to_string()), Err(_) => serde_json::Value::String(value.to_string()),
@@ -165,9 +164,6 @@ async fn set_setting(
.set_setting(DEFAULT_USER_ID, path, &json_value) .set_setting(DEFAULT_USER_ID, path, &json_value)
.await .await
.map_err(|e| anyhow::anyhow!("Failed to save to database: {}", e))?; .map_err(|e| anyhow::anyhow!("Failed to save to database: {}", e))?;
} else {
settings.save()?;
}
println!("Set {} = {}", path, value); println!("Set {} = {}", path, value);
Ok(()) Ok(())
@@ -180,17 +176,13 @@ async fn reset_setting(store: Option<&dyn crate::db::Database>, path: &str) -> a
.get(path) .get(path)
.ok_or_else(|| anyhow::anyhow!("Unknown setting: {}", path))?; .ok_or_else(|| anyhow::anyhow!("Unknown setting: {}", path))?;
// Delete from DB (falling back to default) or reset on disk let store = store.ok_or_else(|| {
if let Some(store) = store { anyhow::anyhow!("Database connection required to reset settings. Check DATABASE_URL.")
})?;
store store
.delete_setting(DEFAULT_USER_ID, path) .delete_setting(DEFAULT_USER_ID, path)
.await .await
.map_err(|e| anyhow::anyhow!("Failed to delete setting from database: {}", e))?; .map_err(|e| anyhow::anyhow!("Failed to delete setting from database: {}", e))?;
} else {
let mut settings = Settings::load();
settings.reset(path).map_err(|e| anyhow::anyhow!("{}", e))?;
settings.save()?;
}
println!("Reset {} to default: {}", path, default_value); println!("Reset {} to default: {}", path, default_value);
Ok(()) Ok(())
@@ -200,37 +192,13 @@ async fn reset_setting(store: Option<&dyn crate::db::Database>, path: &str) -> a
fn show_path(has_db: bool) -> anyhow::Result<()> { fn show_path(has_db: bool) -> anyhow::Result<()> {
if has_db { if has_db {
println!("Settings stored in: database (settings table)"); println!("Settings stored in: database (settings table)");
} else {
println!("Settings stored in: PostgreSQL (not connected, using defaults)");
}
println!( println!(
"Bootstrap config: {}", "Env config: {}",
crate::bootstrap::BootstrapConfig::default_path().display() crate::bootstrap::ironclaw_env_path().display()
); );
} else {
let path = Settings::default_path();
println!("Settings stored in: {} (disk fallback)", path.display());
if path.exists() {
let metadata = std::fs::metadata(&path)?;
println!(" Size: {} bytes", metadata.len());
if let Ok(modified) = metadata.modified() {
use std::time::SystemTime;
let duration = SystemTime::now()
.duration_since(modified)
.unwrap_or_default();
let secs = duration.as_secs();
if secs < 60 {
println!(" Modified: {} seconds ago", secs);
} else if secs < 3600 {
println!(" Modified: {} minutes ago", secs / 60);
} else if secs < 86400 {
println!(" Modified: {} hours ago", secs / 3600);
} else {
println!(" Modified: {} days ago", secs / 86400);
}
}
} else {
println!(" (does not exist, using defaults)");
}
}
Ok(()) Ok(())
} }
+1
View File
@@ -12,6 +12,7 @@
mod config; mod config;
mod mcp; mod mcp;
pub mod memory; pub mod memory;
pub mod oauth_defaults;
mod pairing; mod pairing;
pub mod status; pub mod status;
mod tool; mod tool;
+342
View File
@@ -0,0 +1,342 @@
//! Shared OAuth infrastructure: built-in credentials, callback server, landing pages.
//!
//! Every OAuth flow in the codebase (WASM tool auth, MCP server auth, NEAR AI login)
//! uses the same callback port, landing page, and listener logic from this module.
//!
//! # Built-in Credentials
//!
//! Many CLI tools (gcloud, rclone, gdrive) ship with default OAuth credentials
//! so users don't need to register their own OAuth app. Google explicitly
//! documents that client_secret for "Desktop App" / "Installed App" types
//! is NOT actually secret.
//!
//! Default credentials are hardcoded below. They can be overridden at:
//!
//! - **Compile time**: Set IRONCLAW_GOOGLE_CLIENT_ID / IRONCLAW_GOOGLE_CLIENT_SECRET
//! env vars before building to replace the hardcoded defaults.
//! - **Runtime**: Users can set GOOGLE_OAUTH_CLIENT_ID / GOOGLE_OAUTH_CLIENT_SECRET
//! env vars, which take priority over built-in defaults.
use std::time::Duration;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener;
// ── Built-in credentials ────────────────────────────────────────────────
pub struct OAuthCredentials {
pub client_id: &'static str,
pub client_secret: &'static str,
}
/// Google OAuth "Desktop App" credentials, shared across all Google tools.
/// Compile-time env vars override the hardcoded defaults below.
const GOOGLE_CLIENT_ID: &str = match option_env!("IRONCLAW_GOOGLE_CLIENT_ID") {
Some(v) => v,
None => "564604149681-efo25d43rs85v0tibdepsmdv5dsrhhr0.apps.googleusercontent.com",
};
const GOOGLE_CLIENT_SECRET: &str = match option_env!("IRONCLAW_GOOGLE_CLIENT_SECRET") {
Some(v) => v,
None => "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",
};
/// Returns built-in OAuth credentials for a provider, keyed by secret_name.
///
/// The secret_name comes from the tool's capabilities.json `auth.secret_name` field.
/// Returns `None` if no built-in credentials are configured for that provider.
pub fn builtin_credentials(secret_name: &str) -> Option<OAuthCredentials> {
match secret_name {
"google_oauth_token" => Some(OAuthCredentials {
client_id: GOOGLE_CLIENT_ID,
client_secret: GOOGLE_CLIENT_SECRET,
}),
_ => None,
}
}
// ── Shared callback server ──────────────────────────────────────────────
/// Fixed port for all OAuth callbacks.
///
/// Every redirect URI registered with providers must use this port:
/// `http://localhost:9876/callback` (or `/auth/callback` for NEAR AI).
pub const OAUTH_CALLBACK_PORT: u16 = 9876;
/// Error from the OAuth callback listener.
#[derive(Debug, thiserror::Error)]
pub enum OAuthCallbackError {
#[error("Port {0} is in use (another auth flow running?): {1}")]
PortInUse(u16, String),
#[error("Authorization denied by user")]
Denied,
#[error("Timed out waiting for authorization")]
Timeout,
#[error("IO error: {0}")]
Io(String),
}
/// Bind the OAuth callback listener on the fixed port.
///
/// Binds to IPv4 `127.0.0.1` first because callback URLs use `127.0.0.1`
/// explicitly (e.g., NEAR AI redirects to `http://127.0.0.1:9876/auth/callback`).
/// Falls back to IPv6 `[::1]` only if IPv4 binding fails for a reason other
/// than `AddrInUse`. If the port is already occupied, fails immediately.
pub async fn bind_callback_listener() -> Result<TcpListener, OAuthCallbackError> {
let ipv4_addr = format!("127.0.0.1:{}", OAUTH_CALLBACK_PORT);
match TcpListener::bind(&ipv4_addr).await {
Ok(listener) => return Ok(listener),
Err(e) if e.kind() == std::io::ErrorKind::AddrInUse => {
return Err(OAuthCallbackError::PortInUse(
OAUTH_CALLBACK_PORT,
e.to_string(),
));
}
Err(_) => {
// IPv4 not available, fall back to IPv6
}
}
TcpListener::bind(format!("[::1]:{}", OAUTH_CALLBACK_PORT))
.await
.map_err(|e| {
if e.kind() == std::io::ErrorKind::AddrInUse {
OAuthCallbackError::PortInUse(OAUTH_CALLBACK_PORT, e.to_string())
} else {
OAuthCallbackError::Io(e.to_string())
}
})
}
/// Wait for an OAuth callback and extract a query parameter value.
///
/// Listens for a GET request matching `path_prefix` (e.g., "/callback" or "/auth/callback"),
/// extracts the value of `param_name` (e.g., "code" or "token"), and shows a branded
/// landing page using `display_name` (e.g., "Google", "Notion", "NEAR AI").
///
/// Times out after 5 minutes.
pub async fn wait_for_callback(
listener: TcpListener,
path_prefix: &str,
param_name: &str,
display_name: &str,
) -> Result<String, OAuthCallbackError> {
let path_prefix = path_prefix.to_string();
let param_name = param_name.to_string();
let display_name = display_name.to_string();
tokio::time::timeout(Duration::from_secs(300), async move {
loop {
let (mut socket, _) = listener
.accept()
.await
.map_err(|e| OAuthCallbackError::Io(e.to_string()))?;
let mut reader = BufReader::new(&mut socket);
let mut request_line = String::new();
reader
.read_line(&mut request_line)
.await
.map_err(|e| OAuthCallbackError::Io(e.to_string()))?;
if let Some(path) = request_line.split_whitespace().nth(1)
&& path.starts_with(&path_prefix)
&& let Some(query) = path.split('?').nth(1)
{
// Check for error first
if query.contains("error=") {
let html = landing_html(&display_name, false);
let response = format!(
"HTTP/1.1 400 Bad Request\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
{}",
html
);
let _ = socket.write_all(response.as_bytes()).await;
return Err(OAuthCallbackError::Denied);
}
// Look for the target parameter
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == param_name {
let value = urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned();
let html = landing_html(&display_name, true);
let response = format!(
"HTTP/1.1 200 OK\r\n\
Content-Type: text/html; charset=utf-8\r\n\
Connection: close\r\n\
\r\n\
{}",
html
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(value);
}
}
}
// Not the callback we're looking for
let response = "HTTP/1.1 404 Not Found\r\nConnection: close\r\n\r\n";
let _ = socket.write_all(response.as_bytes()).await;
}
})
.await
.map_err(|_| OAuthCallbackError::Timeout)?
}
/// Escape a string for safe interpolation into HTML content.
fn html_escape(s: &str) -> String {
let mut out = String::with_capacity(s.len());
for c in s.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
'\'' => out.push_str("&#x27;"),
_ => out.push(c),
}
}
out
}
/// HTML landing page shown in the browser after an OAuth redirect.
pub fn landing_html(provider_name: &str, success: bool) -> String {
let safe_name = html_escape(provider_name);
let (icon, heading, subtitle, accent) = if success {
(
r##"<div style="width:64px;height:64px;border-radius:50%;background:#22c55e;display:flex;align-items:center;justify-content:center;margin:0 auto 24px">
<svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="#fff" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><polyline points="20 6 9 17 4 12"/></svg>
</div>"##,
format!("{} Connected", safe_name),
"You can close this window and return to your terminal.",
"#22c55e",
)
} else {
(
r##"<div style="width:64px;height:64px;border-radius:50%;background:#ef4444;display:flex;align-items:center;justify-content:center;margin:0 auto 24px">
<svg width="32" height="32" viewBox="0 0 24 24" fill="none" stroke="#fff" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"><line x1="18" y1="6" x2="6" y2="18"/><line x1="6" y1="6" x2="18" y2="18"/></svg>
</div>"##,
"Authorization Failed".to_string(),
"The request was denied. You can close this window and try again.",
"#ef4444",
)
};
format!(
r#"<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>IronClaw - {heading}</title>
<style>
* {{ margin:0; padding:0; box-sizing:border-box }}
body {{
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
background: #0a0a0a;
color: #e5e5e5;
display: flex;
justify-content: center;
align-items: center;
min-height: 100vh;
}}
.card {{
text-align: center;
padding: 48px 40px;
max-width: 420px;
border: 1px solid #262626;
border-radius: 16px;
background: #141414;
}}
h1 {{
font-size: 22px;
font-weight: 600;
margin-bottom: 8px;
color: #fafafa;
}}
p {{
font-size: 14px;
color: #a3a3a3;
line-height: 1.5;
}}
.accent {{ color: {accent}; }}
.brand {{
margin-top: 32px;
font-size: 12px;
color: #525252;
letter-spacing: 0.5px;
text-transform: uppercase;
}}
</style>
</head>
<body>
<div class="card">
{icon}
<h1>{heading}</h1>
<p>{subtitle}</p>
<div class="brand">IronClaw</div>
</div>
</body>
</html>"#,
heading = heading,
icon = icon,
subtitle = subtitle,
accent = accent,
)
}
#[cfg(test)]
mod tests {
use crate::cli::oauth_defaults::{builtin_credentials, landing_html};
#[test]
fn test_unknown_provider_returns_none() {
assert!(builtin_credentials("unknown_token").is_none());
}
#[test]
fn test_google_returns_based_on_compile_env() {
let creds = builtin_credentials("google_oauth_token");
assert!(creds.is_some());
let creds = creds.unwrap();
assert!(!creds.client_id.is_empty());
assert!(!creds.client_secret.is_empty());
}
#[test]
fn test_landing_html_success_contains_key_elements() {
let html = landing_html("Google", true);
assert!(html.contains("Google Connected"));
assert!(html.contains("charset"));
assert!(html.contains("IronClaw"));
assert!(html.contains("#22c55e")); // green accent
assert!(!html.contains("Failed"));
}
#[test]
fn test_landing_html_escapes_provider_name() {
let html = landing_html("<script>alert(1)</script>", true);
assert!(!html.contains("<script>"));
assert!(html.contains("&lt;script&gt;"));
}
#[test]
fn test_landing_html_error_contains_key_elements() {
let html = landing_html("Notion", false);
assert!(html.contains("Authorization Failed"));
assert!(html.contains("charset"));
assert!(html.contains("IronClaw"));
assert!(html.contains("#ef4444")); // red accent
assert!(!html.contains("Connected"));
}
}
+40 -20
View File
@@ -9,7 +9,7 @@ use crate::settings::Settings;
/// Run the status command, printing system health info. /// Run the status command, printing system health info.
pub async fn run_status_command() -> anyhow::Result<()> { pub async fn run_status_command() -> anyhow::Result<()> {
let settings = Settings::load(); let settings = Settings::default();
println!("IronClaw Status"); println!("IronClaw Status");
println!("===============\n"); println!("===============\n");
@@ -22,17 +22,37 @@ pub async fn run_status_command() -> anyhow::Result<()> {
); );
// Database // Database
let db_url_set = settings.database_url.is_some() || std::env::var("DATABASE_URL").is_ok();
print!(" Database: "); print!(" Database: ");
if db_url_set { let db_backend = std::env::var("DATABASE_BACKEND")
// Try to connect .ok()
.unwrap_or_else(|| "postgres".to_string());
match db_backend.as_str() {
"libsql" | "turso" | "sqlite" => {
let path = std::env::var("LIBSQL_PATH")
.map(std::path::PathBuf::from)
.unwrap_or_else(|_| crate::config::default_libsql_path());
if path.exists() {
let turso = if std::env::var("LIBSQL_URL").is_ok() {
" + Turso sync"
} else {
""
};
println!("libSQL ({}{})", path.display(), turso);
} else {
println!("libSQL (file missing: {})", path.display());
}
}
_ => {
if std::env::var("DATABASE_URL").is_ok() {
match check_database().await { match check_database().await {
Ok(()) => println!("connected"), Ok(()) => println!("connected (PostgreSQL)"),
Err(e) => println!("error ({})", e), Err(e) => println!("error ({})", e),
} }
} else { } else {
println!("not configured"); println!("not configured");
} }
}
}
// Session / Auth // Session / Auth
print!(" Session: "); print!(" Session: ");
@@ -43,15 +63,17 @@ pub async fn run_status_command() -> anyhow::Result<()> {
println!("not found (run `ironclaw onboard`)"); println!("not found (run `ironclaw onboard`)");
} }
// Secrets // Secrets (auto-detect from env only; skip keychain probe to avoid
// triggering macOS system password dialogs on a simple status check)
print!(" Secrets: "); print!(" Secrets: ");
let secrets_configured = settings.secrets_master_key_source != crate::settings::KeySource::None if std::env::var("SECRETS_MASTER_KEY").is_ok() {
|| std::env::var("SECRETS_MASTER_KEY").is_ok() println!("configured (env)");
|| crate::secrets::keychain::has_master_key().await;
if secrets_configured {
println!("configured ({:?})", settings.secrets_master_key_source);
} else { } else {
println!("not configured"); // We don't probe the keychain here because get_generic_password()
// triggers macOS unlock+authorization dialogs, which is bad UX for
// a read-only status command. If onboarding completed with keychain
// storage, the key is there; we just can't cheaply verify it.
println!("env not set (keychain may be configured)");
} }
// Embeddings // Embeddings
@@ -129,20 +151,18 @@ pub async fn run_status_command() -> anyhow::Result<()> {
Err(_) => println!("none configured"), Err(_) => println!("none configured"),
} }
// Settings path // Config path
println!("\n Settings: {}", Settings::default_path().display()); println!(
"\n Config: {}",
crate::bootstrap::ironclaw_env_path().display()
);
Ok(()) Ok(())
} }
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
async fn check_database() -> anyhow::Result<()> { async fn check_database() -> anyhow::Result<()> {
let _ = dotenvy::dotenv(); let url = std::env::var("DATABASE_URL").map_err(|_| anyhow::anyhow!("DATABASE_URL not set"))?;
let settings = Settings::load();
let url = std::env::var("DATABASE_URL")
.ok()
.or(settings.database_url)
.ok_or_else(|| anyhow::anyhow!("no URL"))?;
let config: deadpool_postgres::Config = deadpool_postgres::Config { let config: deadpool_postgres::Config = deadpool_postgres::Config {
url: Some(url), url: Some(url),
+137 -115
View File
@@ -423,13 +423,13 @@ async fn extract_crate_name(cargo_toml: &Path) -> anyhow::Result<String> {
// Simple TOML parsing for [package] name // Simple TOML parsing for [package] name
for line in content.lines() { for line in content.lines() {
let line = line.trim(); let line = line.trim();
if line.starts_with("name") { if line.starts_with("name")
if let Some((_, value)) = line.split_once('=') { && let Some((_, value)) = line.split_once('=')
{
let name = value.trim().trim_matches('"').trim_matches('\''); let name = value.trim().trim_matches('"').trim_matches('\'');
return Ok(name.to_string()); return Ok(name.to_string());
} }
} }
}
anyhow::bail!( anyhow::bail!(
"Could not extract package name from {}", "Could not extract package name from {}",
@@ -491,12 +491,12 @@ async fn list_tools(dir: Option<PathBuf>, verbose: bool) -> anyhow::Result<()> {
if has_caps { if has_caps {
let caps_path = path.with_extension("capabilities.json"); let caps_path = path.with_extension("capabilities.json");
if let Ok(content) = fs::read_to_string(&caps_path).await { if let Ok(content) = fs::read_to_string(&caps_path).await
if let Ok(caps) = CapabilitiesFile::from_json(&content) { && let Ok(caps) = CapabilitiesFile::from_json(&content)
{
print_capabilities_summary(&caps); print_capabilities_summary(&caps);
} }
} }
}
println!(); println!();
} else { } else {
let caps_indicator = if has_caps { "" } else { "" }; let caps_indicator = if has_caps { "" } else { "" };
@@ -607,17 +607,17 @@ fn print_capabilities_summary(caps: &CapabilitiesFile) {
} }
} }
if let Some(ref secrets) = caps.secrets { if let Some(ref secrets) = caps.secrets
if !secrets.allowed_names.is_empty() { && !secrets.allowed_names.is_empty()
{
parts.push(format!("secrets: {}", secrets.allowed_names.len())); parts.push(format!("secrets: {}", secrets.allowed_names.len()));
} }
}
if let Some(ref ws) = caps.workspace { if let Some(ref ws) = caps.workspace
if !ws.allowed_prefixes.is_empty() { && !ws.allowed_prefixes.is_empty()
{
parts.push("workspace: read".to_string()); parts.push("workspace: read".to_string());
} }
}
if !parts.is_empty() { if !parts.is_empty() {
println!(" Perms: {}", parts.join(", ")); println!(" Perms: {}", parts.join(", "));
@@ -653,33 +653,33 @@ fn print_capabilities_detail(caps: &CapabilitiesFile) {
} }
} }
if let Some(ref secrets) = caps.secrets { if let Some(ref secrets) = caps.secrets
if !secrets.allowed_names.is_empty() { && !secrets.allowed_names.is_empty()
{
println!(" Secrets (existence check only):"); println!(" Secrets (existence check only):");
for name in &secrets.allowed_names { for name in &secrets.allowed_names {
println!(" {}", name); println!(" {}", name);
} }
} }
}
if let Some(ref tool_invoke) = caps.tool_invoke { if let Some(ref tool_invoke) = caps.tool_invoke
if !tool_invoke.aliases.is_empty() { && !tool_invoke.aliases.is_empty()
{
println!(" Tool aliases:"); println!(" Tool aliases:");
for (alias, real_name) in &tool_invoke.aliases { for (alias, real_name) in &tool_invoke.aliases {
println!(" {} -> {}", alias, real_name); println!(" {} -> {}", alias, real_name);
} }
} }
}
if let Some(ref ws) = caps.workspace { if let Some(ref ws) = caps.workspace
if !ws.allowed_prefixes.is_empty() { && !ws.allowed_prefixes.is_empty()
{
println!(" Workspace read prefixes:"); println!(" Workspace read prefixes:");
for prefix in &ws.allowed_prefixes { for prefix in &ws.allowed_prefixes {
println!(" {}", prefix); println!(" {}", prefix);
} }
} }
} }
}
/// Configure authentication for a tool. /// Configure authentication for a tool.
async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> { async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyhow::Result<()> {
@@ -802,9 +802,10 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
} }
// Check for environment variable // Check for environment variable
if let Some(ref env_var) = auth.env_var { if let Some(ref env_var) = auth.env_var
if let Ok(token) = std::env::var(env_var) { && let Ok(token) = std::env::var(env_var)
if !token.is_empty() { && !token.is_empty()
{
println!(" Found {} in environment.", env_var); println!(" Found {} in environment.", env_var);
println!(); println!();
@@ -828,22 +829,73 @@ async fn auth_tool(name: String, dir: Option<PathBuf>, user_id: String) -> anyho
} }
// Save the token // Save the token
save_token(secrets_store.as_ref(), &user_id, &auth, &token).await?; save_token(secrets_store.as_ref(), &user_id, &auth, &token, None, None).await?;
print_success(display_name); print_success(display_name);
return Ok(()); return Ok(());
} }
}
}
// Check for OAuth configuration // Check for OAuth configuration
if let Some(ref oauth) = auth.oauth { if let Some(ref oauth) = auth.oauth {
return auth_tool_oauth(secrets_store.as_ref(), &user_id, &auth, oauth).await; // For providers with shared tokens (e.g., all Google tools share google_oauth_token),
// combine scopes from all installed tools so one auth covers everything.
let combined = combine_provider_scopes(&tools_dir, &auth.secret_name, oauth).await;
if combined.scopes.len() > oauth.scopes.len() {
let extra = combined.scopes.len() - oauth.scopes.len();
println!(
" Including scopes from {} other installed tool(s) sharing this credential.",
extra
);
println!();
}
return auth_tool_oauth(secrets_store.as_ref(), &user_id, &auth, &combined).await;
} }
// Fall back to manual entry // Fall back to manual entry
auth_tool_manual(secrets_store.as_ref(), &user_id, &auth).await auth_tool_manual(secrets_store.as_ref(), &user_id, &auth).await
} }
/// Scan the tools directory for all capabilities files sharing the same secret_name
/// and combine their OAuth scopes. This way, authing any Google tool requests scopes
/// for ALL installed Google tools, so one login covers everything.
async fn combine_provider_scopes(
tools_dir: &Path,
secret_name: &str,
base_oauth: &crate::tools::wasm::OAuthConfigSchema,
) -> crate::tools::wasm::OAuthConfigSchema {
let mut all_scopes: std::collections::HashSet<String> =
base_oauth.scopes.iter().cloned().collect();
if let Ok(mut entries) = tokio::fs::read_dir(tools_dir).await {
while let Ok(Some(entry)) = entries.next_entry().await {
let path = entry.path();
if path.extension().and_then(|e| e.to_str()) != Some("json") {
continue;
}
let name = path
.file_name()
.and_then(|n| n.to_str())
.unwrap_or_default();
if !name.ends_with(".capabilities.json") {
continue;
}
if let Ok(content) = tokio::fs::read_to_string(&path).await
&& let Ok(caps) = CapabilitiesFile::from_json(&content)
&& let Some(auth) = &caps.auth
&& auth.secret_name == secret_name
&& let Some(oauth) = &auth.oauth
{
all_scopes.extend(oauth.scopes.iter().cloned());
}
}
}
let mut combined = base_oauth.clone();
combined.scopes = all_scopes.into_iter().collect();
combined.scopes.sort(); // deterministic ordering
combined
}
/// OAuth browser-based login flow. /// OAuth browser-based login flow.
async fn auth_tool_oauth( async fn auth_tool_oauth(
store: &(dyn SecretsStore + Send + Sync), store: &(dyn SecretsStore + Send + Sync),
@@ -854,12 +906,14 @@ async fn auth_tool_oauth(
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD}; use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore; use rand::RngCore;
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener; use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
let display_name = auth.display_name.as_deref().unwrap_or(&auth.secret_name); let display_name = auth.display_name.as_deref().unwrap_or(&auth.secret_name);
// Get client_id from config or env // Get client_id: capabilities file > runtime env var > built-in defaults
let builtin = oauth_defaults::builtin_credentials(&auth.secret_name);
let client_id = oauth let client_id = oauth
.client_id .client_id
.clone() .clone()
@@ -869,41 +923,32 @@ async fn auth_tool_oauth(
.as_ref() .as_ref()
.and_then(|env| std::env::var(env).ok()) .and_then(|env| std::env::var(env).ok())
}) })
.or_else(|| builtin.as_ref().map(|c| c.client_id.to_string()))
.ok_or_else(|| { .ok_or_else(|| {
anyhow::anyhow!( anyhow::anyhow!(
"OAuth client_id not configured.\n\ "OAuth client_id not configured.\n\
Set it in the capabilities file or via environment variable." Set {} env var, or build with IRONCLAW_GOOGLE_CLIENT_ID.",
oauth.client_id_env.as_deref().unwrap_or("the client_id")
) )
})?; })?;
// Get client_secret if provided // Get client_secret: capabilities file > runtime env var > built-in defaults
let client_secret = oauth.client_secret.clone().or_else(|| { let client_secret = oauth
.client_secret
.clone()
.or_else(|| {
oauth oauth
.client_secret_env .client_secret_env
.as_ref() .as_ref()
.and_then(|env| std::env::var(env).ok()) .and_then(|env| std::env::var(env).ok())
}); })
.or_else(|| builtin.as_ref().map(|c| c.client_secret.to_string()));
println!(" Starting OAuth authentication..."); println!(" Starting OAuth authentication...");
println!(); println!();
// Find an available port for the callback let listener = oauth_defaults::bind_callback_listener().await?;
let mut listener = None; let redirect_uri = format!("http://localhost:{}/callback", OAUTH_CALLBACK_PORT);
let mut port = 0;
for p in 9876..=9886 {
match TcpListener::bind(format!("127.0.0.1:{}", p)).await {
Ok(l) => {
listener = Some(l);
port = p;
break;
}
Err(_) => continue,
}
}
let listener = listener.ok_or_else(|| anyhow::anyhow!("Could not find available port"))?;
let redirect_uri = format!("http://localhost:{}/callback", port);
// Generate PKCE verifier and challenge // Generate PKCE verifier and challenge
let (code_verifier, code_challenge) = if oauth.use_pkce { let (code_verifier, code_challenge) = if oauth.use_pkce {
@@ -962,65 +1007,8 @@ async fn auth_tool_oauth(
println!(" Waiting for authorization..."); println!(" Waiting for authorization...");
// Wait for callback with timeout let code =
let timeout = std::time::Duration::from_secs(300); oauth_defaults::wait_for_callback(listener, "/callback", "code", display_name).await?;
let code = tokio::time::timeout(timeout, async {
loop {
let (mut socket, _) = listener.accept().await?;
let mut reader = BufReader::new(&mut socket);
let mut request_line = String::new();
reader.read_line(&mut request_line).await?;
// Parse GET /callback?code=xxx HTTP/1.1
if let Some(path) = request_line.split_whitespace().nth(1) {
if path.starts_with("/callback") {
if let Some(query) = path.split('?').nth(1) {
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == "code" {
let code = urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned();
// Send success response
let response = format!(
"HTTP/1.1 200 OK\r\n\
Content-Type: text/html\r\n\
\r\n\
<!DOCTYPE html><html><body style=\"font-family: sans-serif; \
display: flex; justify-content: center; align-items: center; \
height: 100vh; margin: 0; background: #191919; color: white;\">\
<div style=\"text-align: center;\">\
<h1>✓ {} Connected!</h1>\
<p>You can close this window.</p>\
</div></body></html>",
display_name
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok::<_, anyhow::Error>(code);
}
}
// Check for error
if query.contains("error=") {
let response =
"HTTP/1.1 400 Bad Request\r\n\r\nAuthorization denied";
let _ = socket.write_all(response.as_bytes()).await;
return Err(anyhow::anyhow!("Authorization denied by user"));
}
}
}
}
let response = "HTTP/1.1 404 Not Found\r\n\r\n";
let _ = socket.write_all(response.as_bytes()).await;
}
})
.await
.map_err(|_| anyhow::anyhow!("Timed out waiting for authorization"))??;
println!(); println!();
println!(" Exchanging code for token..."); println!(" Exchanging code for token...");
@@ -1071,8 +1059,19 @@ async fn auth_tool_oauth(
) )
})?; })?;
// Save the token let refresh_token = token_data.get("refresh_token").and_then(|v| v.as_str());
save_token(store, user_id, auth, access_token).await?; let expires_in = token_data.get("expires_in").and_then(|v| v.as_u64());
// Save the token (with refresh token and expiry if provided)
save_token(
store,
user_id,
auth,
access_token,
refresh_token,
expires_in,
)
.await?;
// Extract any additional info for display // Extract any additional info for display
let workspace_name = token_data let workspace_name = token_data
@@ -1174,8 +1173,8 @@ async fn auth_tool_manual(
} }
} }
// Save the token // Save the token (manual path: no refresh token or expiry)
save_token(store, user_id, auth, &token).await?; save_token(store, user_id, auth, &token, None, None).await?;
print_success(display_name); print_success(display_name);
Ok(()) Ok(())
} }
@@ -1266,11 +1265,16 @@ async fn validate_token(
} }
/// Save token to secrets store. /// Save token to secrets store.
///
/// Optionally stores a refresh token (as `{secret_name}_refresh_token`) and
/// sets `expires_at` on the access token so the runtime can auto-refresh.
async fn save_token( async fn save_token(
store: &(dyn SecretsStore + Send + Sync), store: &(dyn SecretsStore + Send + Sync),
user_id: &str, user_id: &str,
auth: &crate::tools::wasm::AuthCapabilitySchema, auth: &crate::tools::wasm::AuthCapabilitySchema,
token: &str, token: &str,
refresh_token: Option<&str>,
expires_in: Option<u64>,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
let mut params = CreateSecretParams::new(&auth.secret_name, token); let mut params = CreateSecretParams::new(&auth.secret_name, token);
@@ -1278,11 +1282,29 @@ async fn save_token(
params = params.with_provider(provider); params = params.with_provider(provider);
} }
if let Some(secs) = expires_in {
let expires_at = chrono::Utc::now() + chrono::Duration::seconds(secs as i64);
params = params.with_expiry(expires_at);
}
store store
.create(user_id, params) .create(user_id, params)
.await .await
.map_err(|e| anyhow::anyhow!("Failed to save token: {}", e))?; .map_err(|e| anyhow::anyhow!("Failed to save token: {}", e))?;
// Store refresh token separately (no expiry, it's long-lived)
if let Some(rt) = refresh_token {
let refresh_name = format!("{}_refresh_token", auth.secret_name);
let mut refresh_params = CreateSecretParams::new(&refresh_name, rt);
if let Some(ref provider) = auth.provider {
refresh_params = refresh_params.with_provider(provider);
}
store
.create(user_id, refresh_params)
.await
.map_err(|e| anyhow::anyhow!("Failed to save refresh token: {}", e))?;
}
Ok(()) Ok(())
} }
+125 -63
View File
@@ -1,11 +1,13 @@
//! Configuration for IronClaw. //! Configuration for IronClaw.
//! //!
//! Settings are loaded with priority: env var > database > default. //! Settings are loaded with priority: env var > database > default.
//! The database replaces the old `settings.json` file for all settings //! `DATABASE_URL` lives in `~/.ironclaw/.env` (loaded via dotenvy early
//! except the 4 bootstrap fields (database_url, pool_size, secrets key //! in startup). Everything else comes from env vars, the DB settings
//! source, onboard_completed) which live in `~/.ironclaw/bootstrap.json`. //! table, or auto-detection.
use std::collections::HashMap;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::OnceLock;
use std::time::Duration; use std::time::Duration;
use secrecy::{ExposeSecret, SecretString}; use secrecy::{ExposeSecret, SecretString};
@@ -13,6 +15,13 @@ use secrecy::{ExposeSecret, SecretString};
use crate::error::ConfigError; use crate::error::ConfigError;
use crate::settings::Settings; use crate::settings::Settings;
/// Thread-safe overlay for injected env vars (secrets loaded from DB).
///
/// Used by `inject_llm_keys_from_secrets()` to make API keys available to
/// `optional_env()` without unsafe `set_var` calls. `optional_env()` checks
/// real env vars first, then falls back to this overlay.
static INJECTED_VARS: OnceLock<HashMap<String, String>> = OnceLock::new();
/// Main configuration for the agent. /// Main configuration for the agent.
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct Config { pub struct Config {
@@ -40,9 +49,9 @@ impl Config {
pub async fn from_db( pub async fn from_db(
store: &dyn crate::db::Database, store: &dyn crate::db::Database,
user_id: &str, user_id: &str,
bootstrap: &crate::bootstrap::BootstrapConfig,
) -> Result<Self, ConfigError> { ) -> Result<Self, ConfigError> {
let _ = dotenvy::dotenv(); let _ = dotenvy::dotenv();
crate::bootstrap::load_ironclaw_env();
// Load all settings from DB into a Settings struct // Load all settings from DB into a Settings struct
let db_settings = match store.get_all_settings(user_id).await { let db_settings = match store.get_all_settings(user_id).await {
@@ -53,7 +62,7 @@ impl Config {
} }
}; };
Self::build(bootstrap, &db_settings).await Self::build(&db_settings).await
} }
/// Load configuration from environment variables only (no database). /// Load configuration from environment variables only (no database).
@@ -61,20 +70,20 @@ impl Config {
/// Used during early startup before the database is connected, /// Used during early startup before the database is connected,
/// and by CLI commands that don't have DB access. /// and by CLI commands that don't have DB access.
/// Falls back to legacy `settings.json` on disk if present. /// Falls back to legacy `settings.json` on disk if present.
///
/// Loads both `./.env` (standard, higher priority) and `~/.ironclaw/.env`
/// (lower priority) via dotenvy, which never overwrites existing vars.
pub async fn from_env() -> Result<Self, ConfigError> { pub async fn from_env() -> Result<Self, ConfigError> {
let _ = dotenvy::dotenv(); let _ = dotenvy::dotenv();
let bootstrap = crate::bootstrap::BootstrapConfig::load(); crate::bootstrap::load_ironclaw_env();
let settings = Settings::load(); let settings = Settings::load();
Self::build(&bootstrap, &settings).await Self::build(&settings).await
} }
/// Build config from bootstrap + settings (shared by from_env and from_db). /// Build config from settings (shared by from_env and from_db).
async fn build( async fn build(settings: &Settings) -> Result<Self, ConfigError> {
bootstrap: &crate::bootstrap::BootstrapConfig,
settings: &Settings,
) -> Result<Self, ConfigError> {
Ok(Self { Ok(Self {
database: DatabaseConfig::resolve(bootstrap)?, database: DatabaseConfig::resolve()?,
llm: LlmConfig::resolve(settings)?, llm: LlmConfig::resolve(settings)?,
embeddings: EmbeddingsConfig::resolve(settings)?, embeddings: EmbeddingsConfig::resolve(settings)?,
tunnel: TunnelConfig::resolve(settings)?, tunnel: TunnelConfig::resolve(settings)?,
@@ -82,7 +91,7 @@ impl Config {
agent: AgentConfig::resolve(settings)?, agent: AgentConfig::resolve(settings)?,
safety: SafetyConfig::resolve()?, safety: SafetyConfig::resolve()?,
wasm: WasmConfig::resolve()?, wasm: WasmConfig::resolve()?,
secrets: SecretsConfig::resolve(bootstrap).await?, secrets: SecretsConfig::resolve().await?,
builder: BuilderModeConfig::resolve()?, builder: BuilderModeConfig::resolve()?,
heartbeat: HeartbeatConfig::resolve(settings)?, heartbeat: HeartbeatConfig::resolve(settings)?,
routines: RoutineConfig::resolve()?, routines: RoutineConfig::resolve()?,
@@ -107,14 +116,14 @@ impl TunnelConfig {
let public_url = optional_env("TUNNEL_URL")? let public_url = optional_env("TUNNEL_URL")?
.or_else(|| settings.tunnel.public_url.clone().filter(|s| !s.is_empty())); .or_else(|| settings.tunnel.public_url.clone().filter(|s| !s.is_empty()));
if let Some(ref url) = public_url { if let Some(ref url) = public_url
if !url.starts_with("https://") { && !url.starts_with("https://")
{
return Err(ConfigError::InvalidValue { return Err(ConfigError::InvalidValue {
key: "TUNNEL_URL".to_string(), key: "TUNNEL_URL".to_string(),
message: "must start with https:// (webhooks require HTTPS)".to_string(), message: "must start with https:// (webhooks require HTTPS)".to_string(),
}); });
} }
}
Ok(Self { public_url }) Ok(Self { public_url })
} }
@@ -179,7 +188,7 @@ pub struct DatabaseConfig {
} }
impl DatabaseConfig { impl DatabaseConfig {
fn resolve(bootstrap: &crate::bootstrap::BootstrapConfig) -> Result<Self, ConfigError> { fn resolve() -> Result<Self, ConfigError> {
let backend: DatabaseBackend = if let Some(b) = optional_env("DATABASE_BACKEND")? { let backend: DatabaseBackend = if let Some(b) = optional_env("DATABASE_BACKEND")? {
b.parse().map_err(|e| ConfigError::InvalidValue { b.parse().map_err(|e| ConfigError::InvalidValue {
key: "DATABASE_BACKEND".to_string(), key: "DATABASE_BACKEND".to_string(),
@@ -191,8 +200,8 @@ impl DatabaseConfig {
// PostgreSQL URL is required only when using the postgres backend. // PostgreSQL URL is required only when using the postgres backend.
// For libsql backend, default to an empty placeholder. // For libsql backend, default to an empty placeholder.
// DATABASE_URL is loaded from ~/.ironclaw/.env via dotenvy early in startup.
let url = optional_env("DATABASE_URL")? let url = optional_env("DATABASE_URL")?
.or_else(|| bootstrap.database_url.clone())
.or_else(|| { .or_else(|| {
if backend == DatabaseBackend::LibSql { if backend == DatabaseBackend::LibSql {
Some("unused://libsql".to_string()) Some("unused://libsql".to_string())
@@ -205,15 +214,7 @@ impl DatabaseConfig {
hint: "Run 'ironclaw onboard' or set DATABASE_URL environment variable".to_string(), hint: "Run 'ironclaw onboard' or set DATABASE_URL environment variable".to_string(),
})?; })?;
let pool_size = optional_env("DATABASE_POOL_SIZE")? let pool_size = parse_optional_env("DATABASE_POOL_SIZE", 10)?;
.map(|s| s.parse())
.transpose()
.map_err(|e| ConfigError::InvalidValue {
key: "DATABASE_POOL_SIZE".to_string(),
message: format!("must be a positive integer: {e}"),
})?
.or(bootstrap.database_pool_size)
.unwrap_or(10);
let libsql_path = optional_env("LIBSQL_PATH")?.map(PathBuf::from).or_else(|| { let libsql_path = optional_env("LIBSQL_PATH")?.map(PathBuf::from).or_else(|| {
if backend == DatabaseBackend::LibSql { if backend == DatabaseBackend::LibSql {
@@ -397,16 +398,37 @@ pub struct NearAiConfig {
pub api_mode: NearAiApiMode, pub api_mode: NearAiApiMode,
/// API key for cloud-api (required for chat_completions mode) /// API key for cloud-api (required for chat_completions mode)
pub api_key: Option<SecretString>, pub api_key: Option<SecretString>,
/// Optional fallback model for failover (default: None).
/// When set, a secondary provider is created with this model and wrapped
/// in a `FailoverProvider` so transient errors on the primary model
/// automatically fall through to the fallback.
pub fallback_model: Option<String>,
/// Maximum number of retries for transient errors (default: 3).
/// With the default of 3, the provider makes up to 4 total attempts
/// (1 initial + 3 retries) before giving up.
pub max_retries: u32,
} }
impl LlmConfig { impl LlmConfig {
fn resolve(settings: &Settings) -> Result<Self, ConfigError> { fn resolve(settings: &Settings) -> Result<Self, ConfigError> {
// Determine backend (default: NearAi) // Determine backend: env var > settings > default (NearAi)
let backend: LlmBackend = if let Some(b) = optional_env("LLM_BACKEND")? { let backend: LlmBackend = if let Some(b) = optional_env("LLM_BACKEND")? {
b.parse().map_err(|e| ConfigError::InvalidValue { b.parse().map_err(|e| ConfigError::InvalidValue {
key: "LLM_BACKEND".to_string(), key: "LLM_BACKEND".to_string(),
message: e, message: e,
})? })?
} else if let Some(ref b) = settings.llm_backend {
match b.parse() {
Ok(backend) => backend,
Err(e) => {
tracing::warn!(
"Invalid llm_backend '{}' in settings: {}. Using default NearAi.",
b,
e
);
LlmBackend::NearAi
}
}
} else { } else {
LlmBackend::NearAi LlmBackend::NearAi
}; };
@@ -441,6 +463,8 @@ impl LlmConfig {
.unwrap_or_else(default_session_path), .unwrap_or_else(default_session_path),
api_mode, api_mode,
api_key: nearai_api_key, api_key: nearai_api_key,
fallback_model: optional_env("NEARAI_FALLBACK_MODEL")?,
max_retries: parse_optional_env("NEARAI_MAX_RETRIES", 3)?,
}; };
// Resolve provider-specific configs based on backend // Resolve provider-specific configs based on backend
@@ -473,6 +497,7 @@ impl LlmConfig {
let ollama = if backend == LlmBackend::Ollama { let ollama = if backend == LlmBackend::Ollama {
let base_url = optional_env("OLLAMA_BASE_URL")? let base_url = optional_env("OLLAMA_BASE_URL")?
.or_else(|| settings.ollama_base_url.clone())
.unwrap_or_else(|| "http://localhost:11434".to_string()); .unwrap_or_else(|| "http://localhost:11434".to_string());
let model = optional_env("OLLAMA_MODEL")?.unwrap_or_else(|| "llama3".to_string()); let model = optional_env("OLLAMA_MODEL")?.unwrap_or_else(|| "llama3".to_string());
Some(OllamaConfig { base_url, model }) Some(OllamaConfig { base_url, model })
@@ -481,8 +506,9 @@ impl LlmConfig {
}; };
let openai_compatible = if backend == LlmBackend::OpenAiCompatible { let openai_compatible = if backend == LlmBackend::OpenAiCompatible {
let base_url = let base_url = optional_env("LLM_BASE_URL")?
optional_env("LLM_BASE_URL")?.ok_or_else(|| ConfigError::MissingRequired { .or_else(|| settings.openai_compatible_base_url.clone())
.ok_or_else(|| ConfigError::MissingRequired {
key: "LLM_BASE_URL".to_string(), key: "LLM_BASE_URL".to_string(),
hint: "Set LLM_BASE_URL when LLM_BACKEND=openai_compatible".to_string(), hint: "Set LLM_BASE_URL when LLM_BACKEND=openai_compatible".to_string(),
})?; })?;
@@ -852,53 +878,42 @@ impl std::fmt::Debug for SecretsConfig {
} }
} }
/// Process-wide cache for the keychain master key.
///
/// Avoids re-prompting the OS keychain on every `SecretsConfig::resolve()` call
/// (e.g. `Config::from_env()` then `Config::from_db()`). Thread-safe alternative
/// to caching in a process env var.
impl SecretsConfig { impl SecretsConfig {
async fn resolve(bootstrap: &crate::bootstrap::BootstrapConfig) -> Result<Self, ConfigError> { /// Auto-detect secrets master key from env var, then OS keychain.
///
/// Sequential probe: SECRETS_MASTER_KEY env var first, then OS keychain.
/// No saved "source" needed; just try each source in order.
async fn resolve() -> Result<Self, ConfigError> {
use crate::settings::KeySource; use crate::settings::KeySource;
let (master_key, source) = if let Some(env_key) = optional_env("SECRETS_MASTER_KEY")? { let (master_key, source) = if let Some(env_key) = optional_env("SECRETS_MASTER_KEY")? {
(Some(SecretString::from(env_key)), KeySource::Env) (Some(SecretString::from(env_key)), KeySource::Env)
} else { } else {
match bootstrap.secrets_master_key_source { // Probe the OS keychain; if a key is stored, use it
KeySource::Keychain => {
// Try to load from OS keychain (async on Linux)
match crate::secrets::keychain::get_master_key().await { match crate::secrets::keychain::get_master_key().await {
Ok(key_bytes) => { Ok(key_bytes) => {
let key_hex: String = let key_hex: String = key_bytes.iter().map(|b| format!("{:02x}", b)).collect();
key_bytes.iter().map(|b| format!("{:02x}", b)).collect();
(Some(SecretString::from(key_hex)), KeySource::Keychain) (Some(SecretString::from(key_hex)), KeySource::Keychain)
} }
Err(_) => { Err(_) => (None, KeySource::None),
// Keychain configured but key not found
// This might happen if keychain was cleared
tracing::warn!(
"Secrets configured for keychain but key not found. \
Run 'ironclaw onboard' to reconfigure."
);
(None, KeySource::None)
}
}
}
KeySource::Env => {
tracing::warn!(
"Secrets configured for env var but SECRETS_MASTER_KEY not set."
);
(None, KeySource::None)
}
KeySource::None => (None, KeySource::None),
} }
}; };
let enabled = master_key.is_some(); let enabled = master_key.is_some();
if let Some(ref key) = master_key { if let Some(ref key) = master_key
if key.expose_secret().len() < 32 { && key.expose_secret().len() < 32
{
return Err(ConfigError::InvalidValue { return Err(ConfigError::InvalidValue {
key: "SECRETS_MASTER_KEY".to_string(), key: "SECRETS_MASTER_KEY".to_string(),
message: "must be at least 32 bytes for AES-256-GCM".to_string(), message: "must be at least 32 bytes for AES-256-GCM".to_string(),
}); });
} }
}
Ok(Self { Ok(Self {
master_key, master_key,
@@ -1351,19 +1366,66 @@ impl ClaudeCodeConfig {
} }
} }
/// Load API keys from the encrypted secrets store into a thread-safe overlay.
///
/// This bridges the gap between secrets stored during onboarding and the
/// env-var-first resolution in `LlmConfig::resolve()`. Keys in the overlay
/// are read by `optional_env()` before falling back to `std::env::var()`,
/// so explicit env vars always win.
pub async fn inject_llm_keys_from_secrets(
secrets: &dyn crate::secrets::SecretsStore,
user_id: &str,
) {
let mappings = [
("llm_openai_api_key", "OPENAI_API_KEY"),
("llm_anthropic_api_key", "ANTHROPIC_API_KEY"),
("llm_compatible_api_key", "LLM_API_KEY"),
];
let mut injected = HashMap::new();
for (secret_name, env_var) in mappings {
match std::env::var(env_var) {
Ok(val) if !val.is_empty() => continue,
_ => {}
}
match secrets.get_decrypted(user_id, secret_name).await {
Ok(decrypted) => {
injected.insert(env_var.to_string(), decrypted.expose().to_string());
tracing::debug!("Loaded secret '{}' for env var '{}'", secret_name, env_var);
}
Err(_) => {
// Secret doesn't exist, that's fine
}
}
}
let _ = INJECTED_VARS.set(injected);
}
// Helper functions // Helper functions
fn optional_env(key: &str) -> Result<Option<String>, ConfigError> { fn optional_env(key: &str) -> Result<Option<String>, ConfigError> {
// Check real env vars first (always win over injected secrets)
match std::env::var(key) { match std::env::var(key) {
Ok(val) if val.is_empty() => Ok(None), Ok(val) if val.is_empty() => {}
Ok(val) => Ok(Some(val)), Ok(val) => return Ok(Some(val)),
Err(std::env::VarError::NotPresent) => Ok(None), Err(std::env::VarError::NotPresent) => {}
Err(e) => Err(ConfigError::ParseError(format!( Err(e) => {
return Err(ConfigError::ParseError(format!(
"failed to read {key}: {e}" "failed to read {key}: {e}"
))), )));
} }
} }
// Fall back to thread-safe overlay (secrets injected from DB)
if let Some(val) = INJECTED_VARS.get().and_then(|map| map.get(key)) {
return Ok(Some(val.clone()));
}
Ok(None)
}
fn parse_optional_env<T>(key: &str, default: T) -> Result<T, ConfigError> fn parse_optional_env<T>(key: &str, default: T) -> Result<T, ConfigError>
where where
T: std::str::FromStr, T: std::str::FromStr,
+6 -6
View File
@@ -772,12 +772,12 @@ impl Database for LibSqlBackend {
.await .await
.map_err(|e| DatabaseError::Query(e.to_string()))? .map_err(|e| DatabaseError::Query(e.to_string()))?
{ {
if let Ok(id_str) = row.get::<String>(0) { if let Ok(id_str) = row.get::<String>(0)
if let Ok(id) = id_str.parse() { && let Ok(id) = id_str.parse()
{
ids.push(id); ids.push(id);
} }
} }
}
Ok(ids) Ok(ids)
} }
@@ -2199,11 +2199,11 @@ impl Database for LibSqlBackend {
e.content_preview = None; e.content_preview = None;
} }
// Update to latest timestamp // Update to latest timestamp
if let (Some(existing), Some(new)) = (&e.updated_at, &updated_at) { if let (Some(existing), Some(new)) = (&e.updated_at, &updated_at)
if new > existing { && new > existing
{
e.updated_at = Some(*new); e.updated_at = Some(*new);
} }
}
}) })
.or_insert(WorkspaceEntry { .or_insert(WorkspaceEntry {
path: entry_path, path: entry_path,
+3 -4
View File
@@ -144,14 +144,13 @@ impl SuccessEvaluator for RuleBasedEvaluator {
// Check for critical errors // Check for critical errors
for action in actions.iter().filter(|a| !a.success) { for action in actions.iter().filter(|a| !a.success) {
if let Some(ref error) = action.error { if let Some(ref error) = action.error
if error.to_lowercase().contains("critical") && (error.to_lowercase().contains("critical")
|| error.to_lowercase().contains("fatal") || error.to_lowercase().contains("fatal"))
{ {
issues.push(format!("Critical error in {}: {}", action.tool_name, error)); issues.push(format!("Critical error in {}: {}", action.tool_name, error));
} }
} }
}
// Check job state // Check job state
if job.state != crate::context::JobState::Completed if job.state != crate::context::JobState::Completed
+8 -8
View File
@@ -492,14 +492,14 @@ impl ExtensionManager {
} }
// Check Content-Length header before downloading the full body // Check Content-Length header before downloading the full body
if let Some(len) = response.content_length() { if let Some(len) = response.content_length()
if len as usize > MAX_WASM_SIZE { && len as usize > MAX_WASM_SIZE
{
return Err(ExtensionError::InstallFailed(format!( return Err(ExtensionError::InstallFailed(format!(
"WASM binary too large ({} bytes, max {} bytes)", "WASM binary too large ({} bytes, max {} bytes)",
len, MAX_WASM_SIZE len, MAX_WASM_SIZE
))); )));
} }
}
let bytes = response let bytes = response
.bytes() .bytes()
@@ -768,11 +768,12 @@ impl ExtensionManager {
}; };
// Check env var first // Check env var first
if let Some(ref env_var) = auth.env_var { if let Some(ref env_var) = auth.env_var
if let Ok(value) = std::env::var(env_var) { && let Ok(value) = std::env::var(env_var)
{
// Store the env var value as a secret // Store the env var value as a secret
let params = CreateSecretParams::new(&auth.secret_name, &value) let params =
.with_provider(name.to_string()); CreateSecretParams::new(&auth.secret_name, &value).with_provider(name.to_string());
self.secrets self.secrets
.create(&self.user_id, params) .create(&self.user_id, params)
.await .await
@@ -789,7 +790,6 @@ impl ExtensionManager {
status: "authenticated".to_string(), status: "authenticated".to_string(),
}); });
} }
}
// Check if already authenticated // Check if already authenticated
if self if self
+5
View File
@@ -36,6 +36,11 @@ pub struct Store {
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
impl Store { impl Store {
/// Wrap an existing pool (useful when the caller already has a connection).
pub fn from_pool(pool: Pool) -> Self {
Self { pool }
}
/// Create a new store and connect to the database. /// Create a new store and connect to the database.
pub async fn new(config: &DatabaseConfig) -> Result<Self, DatabaseError> { pub async fn new(config: &DatabaseConfig) -> Result<Self, DatabaseError> {
let mut cfg = Config::new(); let mut cfg = Config::new();
+1
View File
@@ -59,6 +59,7 @@ pub mod secrets;
pub mod settings; pub mod settings;
pub mod setup; pub mod setup;
pub mod tools; pub mod tools;
pub mod tracing_fmt;
pub mod util; pub mod util;
pub mod worker; pub mod worker;
pub mod workspace; pub mod workspace;
+483
View File
@@ -0,0 +1,483 @@
//! Multi-provider LLM failover.
//!
//! Wraps multiple LlmProvider instances and tries each in sequence
//! until one succeeds. Transparent to callers --- same LlmProvider trait.
use std::future::Future;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use async_trait::async_trait;
use rust_decimal::Decimal;
use crate::error::LlmError;
use crate::llm::provider::{
CompletionRequest, CompletionResponse, LlmProvider, ToolCompletionRequest,
ToolCompletionResponse,
};
/// Returns `true` if the error is transient and the request should be retried
/// on the next provider in the failover chain.
///
/// Retryable: `RequestFailed`, `RateLimited`, `InvalidResponse`,
/// `SessionRenewalFailed`, `ModelNotAvailable`, `Http`, `Io`.
///
/// `ModelNotAvailable` is retryable because the next provider in the chain may
/// offer a different model, so it's worth trying.
///
/// Non-retryable errors (`AuthFailed`, `SessionExpired`, `ContextLengthExceeded`)
/// propagate immediately because a different provider won't fix them.
fn is_retryable(err: &LlmError) -> bool {
matches!(
err,
LlmError::RequestFailed { .. }
| LlmError::RateLimited { .. }
| LlmError::InvalidResponse { .. }
| LlmError::SessionRenewalFailed { .. }
// ModelNotAvailable is retryable: the next provider may offer a different model.
| LlmError::ModelNotAvailable { .. }
| LlmError::Http(_)
| LlmError::Io(_)
)
}
/// An LLM provider that wraps multiple providers and tries each in sequence
/// on transient failures.
///
/// The first provider in the list is the primary. If it fails with a retryable
/// error, the next provider is tried, and so on. Non-retryable errors
/// (e.g. `AuthFailed`, `ContextLengthExceeded`) propagate immediately.
pub struct FailoverProvider {
providers: Vec<Arc<dyn LlmProvider>>,
/// Index of the provider that last handled a request successfully.
/// Used by `model_name()` and `cost_per_token()` so downstream cost
/// tracking reflects the provider that actually served the request.
last_used: AtomicUsize,
}
impl FailoverProvider {
/// Create a new failover provider.
///
/// Returns an error if `providers` is empty.
pub fn new(providers: Vec<Arc<dyn LlmProvider>>) -> Result<Self, LlmError> {
if providers.is_empty() {
return Err(LlmError::RequestFailed {
provider: "failover".to_string(),
reason: "FailoverProvider requires at least one provider".to_string(),
});
}
Ok(Self {
providers,
last_used: AtomicUsize::new(0),
})
}
/// Try each provider in sequence until one succeeds or all fail.
async fn try_providers<T, F, Fut>(&self, mut call: F) -> Result<T, LlmError>
where
F: FnMut(Arc<dyn LlmProvider>) -> Fut,
Fut: Future<Output = Result<T, LlmError>>,
{
let mut last_error: Option<LlmError> = None;
for (i, provider) in self.providers.iter().enumerate() {
let result = call(Arc::clone(provider)).await;
match result {
Ok(response) => {
self.last_used.store(i, Ordering::Relaxed);
return Ok(response);
}
Err(err) => {
if !is_retryable(&err) {
return Err(err);
}
if i + 1 < self.providers.len() {
tracing::warn!(
provider = %provider.model_name(),
error = %err,
next_provider = %self.providers[i + 1].model_name(),
"Provider failed with retryable error, trying next provider"
);
}
last_error = Some(err);
}
}
}
// SAFETY: providers is non-empty (checked in `new`), so at least one
// iteration ran and `last_error` is `Some`.
Err(last_error.expect("providers list is non-empty"))
}
}
#[async_trait]
impl LlmProvider for FailoverProvider {
fn model_name(&self) -> &str {
self.providers[self.last_used.load(Ordering::Relaxed)].model_name()
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
self.providers[self.last_used.load(Ordering::Relaxed)].cost_per_token()
}
async fn complete(&self, request: CompletionRequest) -> Result<CompletionResponse, LlmError> {
self.try_providers(|provider| {
let req = request.clone();
async move { provider.complete(req).await }
})
.await
}
async fn complete_with_tools(
&self,
request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
self.try_providers(|provider| {
let req = request.clone();
async move { provider.complete_with_tools(req).await }
})
.await
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
let mut all_models = Vec::new();
for provider in &self.providers {
match provider.list_models().await {
Ok(models) => all_models.extend(models),
Err(err) => {
tracing::warn!(
provider = %provider.model_name(),
error = %err,
"Failed to list models from provider, skipping"
);
}
}
}
all_models.sort();
all_models.dedup();
Ok(all_models)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
use std::time::Duration;
use crate::llm::provider::{CompletionResponse, FinishReason, ToolCompletionResponse};
/// A mock LLM provider that returns a predetermined result.
struct MockProvider {
name: String,
input_cost: Decimal,
output_cost: Decimal,
complete_result: Mutex<Option<Result<CompletionResponse, LlmError>>>,
tool_complete_result: Mutex<Option<Result<ToolCompletionResponse, LlmError>>>,
}
impl MockProvider {
fn succeeding(name: &str, content: &str) -> Self {
Self {
name: name.to_string(),
input_cost: Decimal::ZERO,
output_cost: Decimal::ZERO,
complete_result: Mutex::new(Some(Ok(CompletionResponse {
content: content.to_string(),
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
}))),
tool_complete_result: Mutex::new(Some(Ok(ToolCompletionResponse {
content: Some(content.to_string()),
tool_calls: vec![],
input_tokens: 10,
output_tokens: 5,
finish_reason: FinishReason::Stop,
response_id: None,
}))),
}
}
fn succeeding_with_cost(
name: &str,
content: &str,
input_cost: Decimal,
output_cost: Decimal,
) -> Self {
Self {
input_cost,
output_cost,
..Self::succeeding(name, content)
}
}
fn failing_retryable(name: &str) -> Self {
Self {
name: name.to_string(),
input_cost: Decimal::ZERO,
output_cost: Decimal::ZERO,
complete_result: Mutex::new(Some(Err(LlmError::RequestFailed {
provider: name.to_string(),
reason: "server error".to_string(),
}))),
tool_complete_result: Mutex::new(Some(Err(LlmError::RequestFailed {
provider: name.to_string(),
reason: "server error".to_string(),
}))),
}
}
fn failing_non_retryable(name: &str) -> Self {
Self {
name: name.to_string(),
input_cost: Decimal::ZERO,
output_cost: Decimal::ZERO,
complete_result: Mutex::new(Some(Err(LlmError::AuthFailed {
provider: name.to_string(),
}))),
tool_complete_result: Mutex::new(Some(Err(LlmError::AuthFailed {
provider: name.to_string(),
}))),
}
}
fn failing_rate_limited(name: &str) -> Self {
Self {
name: name.to_string(),
input_cost: Decimal::ZERO,
output_cost: Decimal::ZERO,
complete_result: Mutex::new(Some(Err(LlmError::RateLimited {
provider: name.to_string(),
retry_after: Some(Duration::from_secs(30)),
}))),
tool_complete_result: Mutex::new(Some(Err(LlmError::RateLimited {
provider: name.to_string(),
retry_after: Some(Duration::from_secs(30)),
}))),
}
}
}
#[async_trait]
impl LlmProvider for MockProvider {
fn model_name(&self) -> &str {
&self.name
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
(self.input_cost, self.output_cost)
}
async fn complete(
&self,
_request: CompletionRequest,
) -> Result<CompletionResponse, LlmError> {
self.complete_result
.lock()
.unwrap()
.take()
.expect("MockProvider::complete called more than once")
}
async fn complete_with_tools(
&self,
_request: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
self.tool_complete_result
.lock()
.unwrap()
.take()
.expect("MockProvider::complete_with_tools called more than once")
}
async fn list_models(&self) -> Result<Vec<String>, LlmError> {
Ok(vec![self.name.clone()])
}
}
fn make_request() -> CompletionRequest {
CompletionRequest::new(vec![crate::llm::ChatMessage::user("hello")])
}
fn make_tool_request() -> ToolCompletionRequest {
ToolCompletionRequest::new(vec![crate::llm::ChatMessage::user("hello")], vec![])
}
// Test 1: Primary succeeds, no failover occurs.
#[tokio::test]
async fn primary_succeeds_no_failover() {
let primary = Arc::new(MockProvider::succeeding("primary", "primary response"));
let fallback = Arc::new(MockProvider::succeeding("fallback", "fallback response"));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
let response = failover.complete(make_request()).await.unwrap();
assert_eq!(response.content, "primary response");
}
// Test 2: Primary fails with retryable error, fallback succeeds.
#[tokio::test]
async fn primary_fails_retryable_fallback_succeeds() {
let primary = Arc::new(MockProvider::failing_retryable("primary"));
let fallback = Arc::new(MockProvider::succeeding("fallback", "fallback response"));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
let response = failover.complete(make_request()).await.unwrap();
assert_eq!(response.content, "fallback response");
}
// Test 3: All providers fail, returns last error.
#[tokio::test]
async fn all_providers_fail_returns_last_error() {
let primary = Arc::new(MockProvider::failing_retryable("primary"));
let fallback = Arc::new(MockProvider::failing_retryable("fallback"));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
let err = failover.complete(make_request()).await.unwrap_err();
match err {
LlmError::RequestFailed { provider, .. } => {
assert_eq!(provider, "fallback");
}
other => panic!("expected RequestFailed, got: {other:?}"),
}
}
// Test 4: Non-retryable error fails immediately, no failover.
#[tokio::test]
async fn non_retryable_error_fails_immediately() {
let primary = Arc::new(MockProvider::failing_non_retryable("primary"));
let fallback = Arc::new(MockProvider::succeeding("fallback", "fallback response"));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
let err = failover.complete(make_request()).await.unwrap_err();
match err {
LlmError::AuthFailed { provider } => {
assert_eq!(provider, "primary");
}
other => panic!("expected AuthFailed, got: {other:?}"),
}
}
// Test 5: Three providers, first two fail (retryable), third succeeds.
#[tokio::test]
async fn three_providers_first_two_fail_third_succeeds() {
let p1 = Arc::new(MockProvider::failing_retryable("provider-1"));
let p2 = Arc::new(MockProvider::failing_rate_limited("provider-2"));
let p3 = Arc::new(MockProvider::succeeding("provider-3", "third time lucky"));
let failover = FailoverProvider::new(vec![p1, p2, p3]).unwrap();
let response = failover.complete(make_request()).await.unwrap();
assert_eq!(response.content, "third time lucky");
}
// Test: complete_with_tools follows same failover logic.
#[tokio::test]
async fn complete_with_tools_failover() {
let primary = Arc::new(MockProvider::failing_retryable("primary"));
let fallback = Arc::new(MockProvider::succeeding("fallback", "tools fallback"));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
let response = failover
.complete_with_tools(make_tool_request())
.await
.unwrap();
assert_eq!(response.content.as_deref(), Some("tools fallback"));
}
// Test: model_name and cost_per_token reflect the last-used provider.
#[tokio::test]
async fn model_name_and_cost_track_last_used_provider() {
let fallback_cost = Decimal::new(15, 6); // 0.000015
let primary = Arc::new(MockProvider::failing_retryable("primary-model"));
let fallback = Arc::new(MockProvider::succeeding_with_cost(
"fallback-model",
"ok",
fallback_cost,
fallback_cost,
));
let failover = FailoverProvider::new(vec![primary, fallback]).unwrap();
// Before any call, defaults to primary (index 0).
assert_eq!(failover.model_name(), "primary-model");
assert_eq!(failover.cost_per_token(), (Decimal::ZERO, Decimal::ZERO));
// After failover, should reflect the fallback provider.
let _ = failover.complete(make_request()).await.unwrap();
assert_eq!(failover.model_name(), "fallback-model");
assert_eq!(failover.cost_per_token(), (fallback_cost, fallback_cost));
}
// Test: list_models aggregates from all providers.
#[tokio::test]
async fn list_models_aggregates_all() {
let p1 = Arc::new(MockProvider::succeeding("model-a", "ok"));
let p2 = Arc::new(MockProvider::succeeding("model-b", "ok"));
let failover = FailoverProvider::new(vec![p1, p2]).unwrap();
let models = failover.list_models().await.unwrap();
assert!(models.contains(&"model-a".to_string()));
assert!(models.contains(&"model-b".to_string()));
}
// Test: is_retryable correctly classifies errors.
#[test]
fn retryable_classification() {
// Retryable
assert!(is_retryable(&LlmError::RequestFailed {
provider: "p".into(),
reason: "err".into(),
}));
assert!(is_retryable(&LlmError::RateLimited {
provider: "p".into(),
retry_after: None,
}));
assert!(is_retryable(&LlmError::InvalidResponse {
provider: "p".into(),
reason: "bad json".into(),
}));
assert!(is_retryable(&LlmError::SessionRenewalFailed {
provider: "p".into(),
reason: "timeout".into(),
}));
assert!(is_retryable(&LlmError::Io(std::io::Error::new(
std::io::ErrorKind::ConnectionReset,
"reset"
))));
assert!(is_retryable(&LlmError::ModelNotAvailable {
provider: "p".into(),
model: "m".into(),
}));
// Non-retryable
assert!(!is_retryable(&LlmError::AuthFailed {
provider: "p".into(),
}));
assert!(!is_retryable(&LlmError::SessionExpired {
provider: "p".into(),
}));
assert!(!is_retryable(&LlmError::ContextLengthExceeded {
used: 100_000,
limit: 50_000,
}));
}
// Test: empty providers list returns error (not panic).
#[test]
fn empty_providers_returns_error() {
let result = FailoverProvider::new(vec![]);
assert!(result.is_err());
}
}
+22 -12
View File
@@ -8,13 +8,16 @@
//! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API //! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API
mod costs; mod costs;
pub mod failover;
mod nearai; mod nearai;
mod nearai_chat; mod nearai_chat;
mod provider; mod provider;
mod reasoning; mod reasoning;
mod retry;
mod rig_adapter; mod rig_adapter;
pub mod session; pub mod session;
pub use failover::FailoverProvider;
pub use nearai::{ModelInfo, NearAiProvider}; pub use nearai::{ModelInfo, NearAiProvider};
pub use nearai_chat::NearAiChatProvider; pub use nearai_chat::NearAiChatProvider;
pub use provider::{ pub use provider::{
@@ -33,7 +36,7 @@ use std::sync::Arc;
use rig::client::CompletionClient; use rig::client::CompletionClient;
use secrecy::ExposeSecret; use secrecy::ExposeSecret;
use crate::config::{LlmBackend, LlmConfig, NearAiApiMode}; use crate::config::{LlmBackend, LlmConfig, NearAiApiMode, NearAiConfig};
use crate::error::LlmError; use crate::error::LlmError;
/// Create an LLM provider based on configuration. /// Create an LLM provider based on configuration.
@@ -46,7 +49,7 @@ pub fn create_llm_provider(
session: Arc<SessionManager>, session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> { ) -> Result<Arc<dyn LlmProvider>, LlmError> {
match config.backend { match config.backend {
LlmBackend::NearAi => create_nearai_provider(config, session), LlmBackend::NearAi => create_llm_provider_with_config(&config.nearai, session),
LlmBackend::OpenAi => create_openai_provider(config), LlmBackend::OpenAi => create_openai_provider(config),
LlmBackend::Anthropic => create_anthropic_provider(config), LlmBackend::Anthropic => create_anthropic_provider(config),
LlmBackend::Ollama => create_ollama_provider(config), LlmBackend::Ollama => create_ollama_provider(config),
@@ -54,21 +57,28 @@ pub fn create_llm_provider(
} }
} }
fn create_nearai_provider( /// Create an LLM provider from a `NearAiConfig` directly.
config: &LlmConfig, ///
/// This is useful when constructing additional providers for failover,
/// where only the model name differs from the primary config.
pub fn create_llm_provider_with_config(
config: &NearAiConfig,
session: Arc<SessionManager>, session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> { ) -> Result<Arc<dyn LlmProvider>, LlmError> {
match config.nearai.api_mode { match config.api_mode {
NearAiApiMode::Responses => { NearAiApiMode::Responses => {
tracing::info!("Using NEAR AI Responses API (chat-api) with session auth"); tracing::info!(
Ok(Arc::new(NearAiProvider::new( model = %config.model,
config.nearai.clone(), "Using Responses API (chat-api) with session auth"
session, );
))) Ok(Arc::new(NearAiProvider::new(config.clone(), session)))
} }
NearAiApiMode::ChatCompletions => { NearAiApiMode::ChatCompletions => {
tracing::info!("Using NEAR AI Chat Completions API (cloud-api) with API key auth"); tracing::info!(
Ok(Arc::new(NearAiChatProvider::new(config.nearai.clone())?)) model = %config.model,
"Using Chat Completions API (cloud-api) with API key auth"
);
Ok(Arc::new(NearAiChatProvider::new(config.clone())?))
} }
} }
} }
+74 -32
View File
@@ -19,6 +19,7 @@ use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall, ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse, ToolCompletionRequest, ToolCompletionResponse,
}; };
use crate::llm::retry::{is_retryable_status, retry_backoff_delay};
use crate::llm::session::SessionManager; use crate::llm::session::SessionManager;
/// Information about an available model from NEAR AI API. /// Information about an available model from NEAR AI API.
@@ -209,8 +210,9 @@ impl NearAiProvider {
data: Option<Vec<ModelEntry>>, data: Option<Vec<ModelEntry>>,
} }
if let Ok(resp) = serde_json::from_str::<ModelsResponse>(&response_text) { if let Ok(resp) = serde_json::from_str::<ModelsResponse>(&response_text)
if let Some(entries) = resp.models.or(resp.data) { && let Some(entries) = resp.models.or(resp.data)
{
let models: Vec<ModelInfo> = entries let models: Vec<ModelInfo> = entries
.into_iter() .into_iter()
.filter_map(|e| { .filter_map(|e| {
@@ -224,7 +226,6 @@ impl NearAiProvider {
return Ok(models); return Ok(models);
} }
} }
}
// Try direct array format // Try direct array format
if let Ok(entries) = serde_json::from_str::<Vec<ModelEntry>>(&response_text) { if let Ok(entries) = serde_json::from_str::<Vec<ModelEntry>>(&response_text) {
@@ -270,16 +271,26 @@ impl NearAiProvider {
} }
} }
/// Inner request implementation without retry logic. /// Inner request implementation with retry logic for transient errors.
///
/// Retries on HTTP 429, 500, 502, 503, 504 with exponential backoff.
/// Does not retry on client errors (400, 401, 403, 404) or parse errors.
async fn send_request_inner<T: Serialize + std::fmt::Debug, R: for<'de> Deserialize<'de>>( async fn send_request_inner<T: Serialize + std::fmt::Debug, R: for<'de> Deserialize<'de>>(
&self, &self,
path: &str, path: &str,
body: &T, body: &T,
) -> Result<R, LlmError> { ) -> Result<R, LlmError> {
let url = self.api_url(path); let url = self.api_url(path);
let max_retries = self.config.max_retries;
for attempt in 0..=max_retries {
let token = self.session.get_token().await?; let token = self.session.get_token().await?;
tracing::debug!("Sending request to NEAR AI: {}", url); tracing::debug!(
"Sending request to NEAR AI: {} (attempt {})",
url,
attempt + 1
);
tracing::debug!("Request body: {:?}", body); tracing::debug!("Request body: {:?}", body);
let response = self let response = self
@@ -289,11 +300,28 @@ impl NearAiProvider {
.header("Content-Type", "application/json") .header("Content-Type", "application/json")
.json(body) .json(body)
.send() .send()
.await .await;
.map_err(|e| {
let response = match response {
Ok(r) => r,
Err(e) => {
tracing::error!("NEAR AI request failed: {}", e); tracing::error!("NEAR AI request failed: {}", e);
e // Network errors (timeout, connection refused) are transient
})?; if attempt < max_retries {
let delay = retry_backoff_delay(attempt);
tracing::warn!(
"NEAR AI request error (attempt {}/{}), retrying in {:?}: {}",
attempt + 1,
max_retries + 1,
delay,
e,
);
tokio::time::sleep(delay).await;
continue;
}
return Err(e.into());
}
};
let status = response.status(); let status = response.status();
let response_text = response.text().await.unwrap_or_default(); let response_text = response.text().await.unwrap_or_default();
@@ -302,11 +330,13 @@ impl NearAiProvider {
tracing::debug!("NEAR AI response body: {}", response_text); tracing::debug!("NEAR AI response body: {}", response_text);
if !status.is_success() { if !status.is_success() {
let status_code = status.as_u16();
// Check for session expiration (401 with specific message patterns) // Check for session expiration (401 with specific message patterns)
if status.as_u16() == 401 { if status_code == 401 {
let is_session_expired = response_text.to_lowercase().contains("session") let lower = response_text.to_lowercase();
&& (response_text.to_lowercase().contains("expired") let is_session_expired = lower.contains("session")
|| response_text.to_lowercase().contains("invalid")); && (lower.contains("expired") || lower.contains("invalid"));
if is_session_expired { if is_session_expired {
return Err(LlmError::SessionExpired { return Err(LlmError::SessionExpired {
@@ -314,15 +344,29 @@ impl NearAiProvider {
}); });
} }
// Generic 401 without session expiration indication // Generic 401 -- not retryable
return Err(LlmError::AuthFailed { return Err(LlmError::AuthFailed {
provider: "nearai".to_string(), provider: "nearai".to_string(),
}); });
} }
// Try to parse as JSON error // Check if this is a transient error worth retrying
if is_retryable_status(status_code) && attempt < max_retries {
let delay = retry_backoff_delay(attempt);
tracing::warn!(
"NEAR AI returned HTTP {} (attempt {}/{}), retrying in {:?}",
status_code,
attempt + 1,
max_retries + 1,
delay,
);
tokio::time::sleep(delay).await;
continue;
}
// Non-retryable error or exhausted retries
if let Ok(error) = serde_json::from_str::<NearAiErrorResponse>(&response_text) { if let Ok(error) = serde_json::from_str::<NearAiErrorResponse>(&response_text) {
if status.as_u16() == 429 { if status_code == 429 {
return Err(LlmError::RateLimited { return Err(LlmError::RateLimited {
provider: "nearai".to_string(), provider: "nearai".to_string(),
retry_after: None, retry_after: None,
@@ -340,8 +384,8 @@ impl NearAiProvider {
}); });
} }
// Try to parse as our expected type // Success -- parse the response
match serde_json::from_str::<R>(&response_text) { return match serde_json::from_str::<R>(&response_text) {
Ok(parsed) => Ok(parsed), Ok(parsed) => Ok(parsed),
Err(e) => { Err(e) => {
tracing::debug!("Response is not expected JSON format: {}", e); tracing::debug!("Response is not expected JSON format: {}", e);
@@ -351,7 +395,15 @@ impl NearAiProvider {
reason: format!("Parse error: {}. Raw: {}", e, response_text), reason: format!("Parse error: {}. Raw: {}", e, response_text),
}) })
} }
};
} }
// This is unreachable because the loop always returns, but the compiler
// cannot prove that. Return a generic error as a safety net.
Err(LlmError::RequestFailed {
provider: "nearai".to_string(),
reason: "retry loop exited unexpectedly".to_string(),
})
} }
} }
@@ -456,7 +508,7 @@ impl LlmProvider for NearAiProvider {
Err(e) => return Err(e), Err(e) => return Err(e),
}; };
tracing::debug!("NEAR AI response: {:?}", response); tracing::debug!("NEAR AI response: output_items={}", response.output.len());
// Extract text from response output // Extract text from response output
// Try multiple formats since API response shape may vary // Try multiple formats since API response shape may vary
@@ -464,11 +516,6 @@ impl LlmProvider for NearAiProvider {
.output .output
.iter() .iter()
.filter_map(|item| { .filter_map(|item| {
tracing::debug!(
"Processing output item: type={}, text={:?}",
item.item_type,
item.text
);
if item.item_type == "message" { if item.item_type == "message" {
// First check for direct text field on item // First check for direct text field on item
if let Some(ref text) = item.text { if let Some(ref text) = item.text {
@@ -479,11 +526,6 @@ impl LlmProvider for NearAiProvider {
contents contents
.iter() .iter()
.filter_map(|c| { .filter_map(|c| {
tracing::debug!(
"Content item: type={}, text={:?}",
c.content_type,
c.text
);
// Accept various content types that might contain text // Accept various content types that might contain text
match c.content_type.as_str() { match c.content_type.as_str() {
"output_text" | "text" => c.text.clone(), "output_text" | "text" => c.text.clone(),
@@ -694,8 +736,9 @@ impl LlmProvider for NearAiProvider {
} }
} }
} }
} else if item.item_type == "function_call" { } else if item.item_type == "function_call"
if let (Some(name), Some(call_id)) = (&item.name, &item.call_id) { && let (Some(name), Some(call_id)) = (&item.name, &item.call_id)
{
// Parse arguments JSON string into Value // Parse arguments JSON string into Value
let arguments = item let arguments = item
.arguments .arguments
@@ -710,7 +753,6 @@ impl LlmProvider for NearAiProvider {
}); });
} }
} }
}
let finish_reason = if tool_calls.is_empty() { let finish_reason = if tool_calls.is_empty() {
FinishReason::Stop FinishReason::Stop
+68 -14
View File
@@ -16,6 +16,7 @@ use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, ModelMetadata, ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, ModelMetadata,
Role, ToolCall, ToolCompletionRequest, ToolCompletionResponse, Role, ToolCall, ToolCompletionRequest, ToolCompletionResponse,
}; };
use crate::llm::retry::{is_retryable_status, retry_backoff_delay};
/// NEAR AI Chat Completions API provider. /// NEAR AI Chat Completions API provider.
pub struct NearAiChatProvider { pub struct NearAiChatProvider {
@@ -62,17 +63,27 @@ impl NearAiChatProvider {
.unwrap_or_default() .unwrap_or_default()
} }
/// Send a request to the chat completions API. /// Send a request to the chat completions API with retry on transient errors.
///
/// Retries on HTTP 429, 500, 502, 503, 504 with exponential backoff.
/// Does not retry on client errors (400, 401, 403, 404) or parse errors.
async fn send_request<T: Serialize, R: for<'de> Deserialize<'de>>( async fn send_request<T: Serialize, R: for<'de> Deserialize<'de>>(
&self, &self,
body: &T, body: &T,
) -> Result<R, LlmError> { ) -> Result<R, LlmError> {
let url = self.api_url("chat/completions"); let url = self.api_url("chat/completions");
let max_retries = self.config.max_retries;
tracing::debug!("Sending request to NEAR AI Chat: {}", url); for attempt in 0..=max_retries {
tracing::debug!(
"Sending request to NEAR AI Chat: {} (attempt {})",
url,
attempt + 1,
);
// Log the request body for debugging tool call issues if tracing::enabled!(tracing::Level::DEBUG)
if let Ok(json) = serde_json::to_string(body) { && let Ok(json) = serde_json::to_string(body)
{
tracing::debug!("NEAR AI Chat request body: {}", json); tracing::debug!("NEAR AI Chat request body: {}", json);
} }
@@ -83,14 +94,30 @@ impl NearAiChatProvider {
.header("Content-Type", "application/json") .header("Content-Type", "application/json")
.json(body) .json(body)
.send() .send()
.await .await;
.map_err(|e| {
let response = match response {
Ok(r) => r,
Err(e) => {
tracing::error!("NEAR AI Chat request failed: {}", e); tracing::error!("NEAR AI Chat request failed: {}", e);
LlmError::RequestFailed { if attempt < max_retries {
let delay = retry_backoff_delay(attempt);
tracing::warn!(
"NEAR AI Chat request error (attempt {}/{}), retrying in {:?}: {}",
attempt + 1,
max_retries + 1,
delay,
e,
);
tokio::time::sleep(delay).await;
continue;
}
return Err(LlmError::RequestFailed {
provider: "nearai_chat".to_string(), provider: "nearai_chat".to_string(),
reason: e.to_string(), reason: e.to_string(),
});
} }
})?; };
let status = response.status(); let status = response.status();
let response_text = response.text().await.unwrap_or_default(); let response_text = response.text().await.unwrap_or_default();
@@ -99,12 +126,31 @@ impl NearAiChatProvider {
tracing::debug!("NEAR AI Chat response body: {}", response_text); tracing::debug!("NEAR AI Chat response body: {}", response_text);
if !status.is_success() { if !status.is_success() {
if status.as_u16() == 401 { let status_code = status.as_u16();
// Auth errors are not retryable
if status_code == 401 {
return Err(LlmError::AuthFailed { return Err(LlmError::AuthFailed {
provider: "nearai_chat".to_string(), provider: "nearai_chat".to_string(),
}); });
} }
if status.as_u16() == 429 {
// Transient errors: retry with backoff
if is_retryable_status(status_code) && attempt < max_retries {
let delay = retry_backoff_delay(attempt);
tracing::warn!(
"NEAR AI Chat returned HTTP {} (attempt {}/{}), retrying in {:?}",
status_code,
attempt + 1,
max_retries + 1,
delay,
);
tokio::time::sleep(delay).await;
continue;
}
// Non-retryable or exhausted retries
if status_code == 429 {
return Err(LlmError::RateLimited { return Err(LlmError::RateLimited {
provider: "nearai_chat".to_string(), provider: "nearai_chat".to_string(),
retry_after: None, retry_after: None,
@@ -116,9 +162,17 @@ impl NearAiChatProvider {
}); });
} }
serde_json::from_str(&response_text).map_err(|e| LlmError::InvalidResponse { // Success — parse the response
return serde_json::from_str(&response_text).map_err(|e| LlmError::InvalidResponse {
provider: "nearai_chat".to_string(), provider: "nearai_chat".to_string(),
reason: format!("JSON parse error: {}. Raw: {}", e, response_text), reason: format!("JSON parse error: {}. Raw: {}", e, response_text),
});
}
// Safety net: unreachable because the loop always returns
Err(LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
reason: "retry loop exited unexpectedly".to_string(),
}) })
} }
@@ -395,11 +449,11 @@ fn flatten_tool_messages(messages: Vec<ChatCompletionMessage>) -> Vec<ChatComple
if let (true, Some(calls)) = (msg.role == "assistant", &msg.tool_calls) { if let (true, Some(calls)) = (msg.role == "assistant", &msg.tool_calls) {
// Convert assistant tool_calls into descriptive text // Convert assistant tool_calls into descriptive text
let mut parts: Vec<String> = Vec::new(); let mut parts: Vec<String> = Vec::new();
if let Some(ref text) = msg.content { if let Some(ref text) = msg.content
if !text.is_empty() { && !text.is_empty()
{
parts.push(text.clone()); parts.push(text.clone());
} }
}
for tc in calls { for tc in calls {
parts.push(format!( parts.push(format!(
"[Called tool `{}` with arguments: {}]", "[Called tool `{}` with arguments: {}]",
+11 -5
View File
@@ -113,6 +113,12 @@ pub struct ToolSelection {
pub reasoning: String, pub reasoning: String,
/// Alternative tools considered. /// Alternative tools considered.
pub alternatives: Vec<String>, pub alternatives: Vec<String>,
/// The tool call ID from the LLM response.
///
/// OpenAI-compatible providers assign each tool call a unique ID that must
/// be echoed back in the corresponding tool result message. Without this,
/// the provider cannot match results to their originating calls.
pub tool_call_id: String,
} }
/// Token usage from a single LLM call. /// Token usage from a single LLM call.
@@ -244,6 +250,7 @@ impl Reasoning {
parameters: tool_call.arguments, parameters: tool_call.arguments,
reasoning: reasoning.clone(), reasoning: reasoning.clone(),
alternatives: vec![], alternatives: vec![],
tool_call_id: tool_call.id,
}) })
.collect(); .collect();
@@ -581,9 +588,10 @@ fn recover_tool_calls_from_content(
} }
// Try JSON first: {"name":"x","arguments":{}} // Try JSON first: {"name":"x","arguments":{}}
if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(inner) { if let Ok(parsed) = serde_json::from_str::<serde_json::Value>(inner)
if let Some(name) = parsed.get("name").and_then(|v| v.as_str()) { && let Some(name) = parsed.get("name").and_then(|v| v.as_str())
if tool_names.contains(name) { && tool_names.contains(name)
{
let arguments = parsed let arguments = parsed
.get("arguments") .get("arguments")
.cloned() .cloned()
@@ -595,8 +603,6 @@ fn recover_tool_calls_from_content(
}); });
continue; continue;
} }
}
}
// Bare tool name (e.g. "<tool_call>tool_list</tool_call>") // Bare tool name (e.g. "<tool_call>tool_list</tool_call>")
let name = inner.trim(); let name = inner.trim();
+96
View File
@@ -0,0 +1,96 @@
//! Shared retry helpers for LLM providers.
//!
//! Provides exponential backoff with jitter and retryable status classification
//! used by both `NearAiProvider` and `NearAiChatProvider`.
use std::time::Duration;
use rand::Rng;
/// Returns `true` if the HTTP status code is transient and worth retrying.
pub(crate) fn is_retryable_status(status: u16) -> bool {
matches!(status, 429 | 500 | 502 | 503 | 504)
}
/// Calculate exponential backoff delay with random jitter.
///
/// Base delay is 1 second, doubled each attempt, with +/-25% jitter.
/// - attempt 0: ~1s (0.75s - 1.25s)
/// - attempt 1: ~2s (1.5s - 2.5s)
/// - attempt 2: ~4s (3.0s - 5.0s)
pub(crate) fn retry_backoff_delay(attempt: u32) -> Duration {
let base_ms: u64 = 1000u64.saturating_mul(2u64.saturating_pow(attempt));
let jitter_range = base_ms / 4; // 25%
let jitter = if jitter_range > 0 {
let offset = rand::thread_rng().gen_range(0..=jitter_range * 2);
offset as i64 - jitter_range as i64
} else {
0
};
let delay_ms = (base_ms as i64 + jitter).max(100) as u64;
Duration::from_millis(delay_ms)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_is_retryable_status() {
// Transient errors should be retryable
assert!(is_retryable_status(429));
assert!(is_retryable_status(500));
assert!(is_retryable_status(502));
assert!(is_retryable_status(503));
assert!(is_retryable_status(504));
// Client errors should not be retryable
assert!(!is_retryable_status(400));
assert!(!is_retryable_status(401));
assert!(!is_retryable_status(403));
assert!(!is_retryable_status(404));
assert!(!is_retryable_status(422));
// Success codes should not be retryable
assert!(!is_retryable_status(200));
assert!(!is_retryable_status(201));
}
#[test]
fn test_retry_backoff_delay_exponential_growth() {
// Run multiple samples to verify the range, accounting for jitter
for _ in 0..20 {
let d0 = retry_backoff_delay(0);
let d1 = retry_backoff_delay(1);
let d2 = retry_backoff_delay(2);
// Attempt 0: base 1000ms, jitter +/-250ms -> [750, 1250]
assert!(d0.as_millis() >= 750, "attempt 0 too low: {:?}", d0);
assert!(d0.as_millis() <= 1250, "attempt 0 too high: {:?}", d0);
// Attempt 1: base 2000ms, jitter +/-500ms -> [1500, 2500]
assert!(d1.as_millis() >= 1500, "attempt 1 too low: {:?}", d1);
assert!(d1.as_millis() <= 2500, "attempt 1 too high: {:?}", d1);
// Attempt 2: base 4000ms, jitter +/-1000ms -> [3000, 5000]
assert!(d2.as_millis() >= 3000, "attempt 2 too low: {:?}", d2);
assert!(d2.as_millis() <= 5000, "attempt 2 too high: {:?}", d2);
}
}
#[test]
fn test_retry_backoff_delay_minimum() {
// Even at attempt 0, delay should be at least 100ms (the minimum floor)
for _ in 0..20 {
let delay = retry_backoff_delay(0);
assert!(delay.as_millis() >= 100);
}
}
#[test]
fn test_retry_backoff_delay_no_overflow() {
// Very high attempt numbers should not panic from overflow
let delay = retry_backoff_delay(30);
assert!(delay.as_millis() >= 100);
}
}
+22 -167
View File
@@ -31,8 +31,6 @@ pub struct SessionConfig {
pub auth_base_url: String, pub auth_base_url: String,
/// Path to session file (e.g., ~/.ironclaw/session.json). /// Path to session file (e.g., ~/.ironclaw/session.json).
pub session_path: PathBuf, pub session_path: PathBuf,
/// Port range for OAuth callback server.
pub callback_port_range: (u16, u16),
} }
impl Default for SessionConfig { impl Default for SessionConfig {
@@ -40,7 +38,6 @@ impl Default for SessionConfig {
Self { Self {
auth_base_url: "https://private.near.ai".to_string(), auth_base_url: "https://private.near.ai".to_string(),
session_path: default_session_path(), session_path: default_session_path(),
callback_port_range: (9876, 9886),
} }
} }
} }
@@ -83,8 +80,9 @@ impl SessionManager {
}; };
// Try to load existing session synchronously during construction // Try to load existing session synchronously during construction
if let Ok(data) = std::fs::read_to_string(&manager.config.session_path) { if let Ok(data) = std::fs::read_to_string(&manager.config.session_path)
if let Ok(session) = serde_json::from_str::<SessionData>(&data) { && let Ok(session) = serde_json::from_str::<SessionData>(&data)
{
// We can't await here, so we use try_write // We can't await here, so we use try_write
if let Ok(mut guard) = manager.token.try_write() { if let Ok(mut guard) = manager.token.try_write() {
*guard = Some(SecretString::from(session.session_token)); *guard = Some(SecretString::from(session.session_token));
@@ -94,7 +92,6 @@ impl SessionManager {
); );
} }
} }
}
manager manager
} }
@@ -222,38 +219,21 @@ impl SessionManager {
/// Start the OAuth login flow. /// Start the OAuth login flow.
/// ///
/// 1. Find an available port for the callback server /// 1. Bind the fixed callback port
/// 2. Print the auth URL and attempt to open browser /// 2. Print the auth URL and attempt to open browser
/// 3. Wait for OAuth callback with session token /// 3. Wait for OAuth callback with session token
/// 4. Save and return the token /// 4. Save and return the token
async fn initiate_login(&self) -> Result<(), LlmError> { async fn initiate_login(&self) -> Result<(), LlmError> {
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
use tokio::net::TcpListener;
// Find an available port let listener = oauth_defaults::bind_callback_listener()
let mut listener = None; .await
let mut port = 0; .map_err(|e| LlmError::SessionRenewalFailed {
for p in self.config.callback_port_range.0..=self.config.callback_port_range.1 {
match TcpListener::bind(format!("127.0.0.1:{}", p)).await {
Ok(l) => {
listener = Some(l);
port = p;
break;
}
Err(_) => continue,
}
}
let listener = listener.ok_or_else(|| LlmError::SessionRenewalFailed {
provider: "nearai".to_string(), provider: "nearai".to_string(),
reason: format!( reason: e.to_string(),
"Could not find available port in range {}-{}",
self.config.callback_port_range.0, self.config.callback_port_range.1
),
})?; })?;
let callback_url = format!("http://127.0.0.1:{}", port); let callback_url = format!("http://127.0.0.1:{}", OAUTH_CALLBACK_PORT);
// Show auth provider menu // Show auth provider menu
println!(); println!();
@@ -333,138 +313,16 @@ impl SessionManager {
println!(); println!();
println!("Waiting for authentication..."); println!("Waiting for authentication...");
// Wait for callback with timeout // The NEAR AI API redirects to: {frontend_callback}/auth/callback?token=X&...
// The API redirects to: {frontend_callback}/auth/callback?token=X&session_id=X&expires_at=X&is_new_user=X let session_token =
let timeout = std::time::Duration::from_secs(300); // 5 minutes oauth_defaults::wait_for_callback(listener, "/auth/callback", "token", "NEAR AI")
let selected_provider = auth_provider.to_string();
let (session_token, auth_provider) = tokio::time::timeout(timeout, async move {
loop {
let (mut socket, _) = listener.accept().await.map_err(|e| {
LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: format!("Failed to accept connection: {}", e),
}
})?;
let mut reader = BufReader::new(&mut socket);
let mut request_line = String::new();
reader.read_line(&mut request_line).await.map_err(|e| {
LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: format!("Failed to read request: {}", e),
}
})?;
// Parse GET /auth/callback?token=xxx&session_id=xxx&expires_at=xxx&is_new_user=xxx HTTP/1.1
if let Some(path) = request_line.split_whitespace().nth(1) {
if path.starts_with("/auth/callback") {
// Parse query parameters
if let Some(query) = path.split('?').nth(1) {
let mut token = None;
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == "token" {
token = Some(
urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned(),
);
}
}
if let Some(token) = token {
// Send success response with nice styling
let response = concat!(
"HTTP/1.1 200 OK\r\n",
"Content-Type: text/html; charset=utf-8\r\n",
"Connection: close\r\n",
"\r\n",
"<!DOCTYPE html>\n",
"<html>\n",
"<head>\n",
" <meta charset=\"utf-8\">\n",
" <title>NEAR AI - Authentication Successful</title>\n",
" <style>\n",
" * { margin: 0; padding: 0; box-sizing: border-box; }\n",
" body {\n",
" font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;\n",
" background: linear-gradient(135deg, #1a1a2e 0%, #16213e 100%);\n",
" min-height: 100vh;\n",
" display: flex;\n",
" align-items: center;\n",
" justify-content: center;\n",
" color: #fff;\n",
" }\n",
" .container {\n",
" text-align: center;\n",
" padding: 3rem;\n",
" background: rgba(255,255,255,0.05);\n",
" border-radius: 16px;\n",
" backdrop-filter: blur(10px);\n",
" border: 1px solid rgba(255,255,255,0.1);\n",
" max-width: 400px;\n",
" }\n",
" .checkmark {\n",
" width: 80px;\n",
" height: 80px;\n",
" background: linear-gradient(135deg, #00d9a5 0%, #00b386 100%);\n",
" border-radius: 50%;\n",
" display: flex;\n",
" align-items: center;\n",
" justify-content: center;\n",
" margin: 0 auto 1.5rem;\n",
" font-size: 40px;\n",
" }\n",
" h1 {\n",
" font-size: 1.5rem;\n",
" font-weight: 600;\n",
" margin-bottom: 0.75rem;\n",
" }\n",
" p {\n",
" color: rgba(255,255,255,0.7);\n",
" font-size: 0.95rem;\n",
" line-height: 1.5;\n",
" }\n",
" .brand {\n",
" margin-top: 2rem;\n",
" padding-top: 1.5rem;\n",
" border-top: 1px solid rgba(255,255,255,0.1);\n",
" font-size: 0.8rem;\n",
" color: rgba(255,255,255,0.4);\n",
" }\n",
" </style>\n",
"</head>\n",
"<body>\n",
" <div class=\"container\">\n",
" <div class=\"checkmark\">&#10003;</div>\n",
" <h1>Authentication Successful</h1>\n",
" <p>You can close this window and return to the terminal.</p>\n",
" <div class=\"brand\">NEAR AI Agent</div>\n",
" </div>\n",
"</body>\n",
"</html>"
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok::<_, LlmError>((token, Some(selected_provider.clone())));
}
}
}
}
// Not the callback we're looking for, send 404
let response = "HTTP/1.1 404 Not Found\r\nConnection: close\r\n\r\n";
let _ = socket.write_all(response.as_bytes()).await;
}
})
.await .await
.map_err(|_| LlmError::SessionRenewalFailed { .map_err(|e| LlmError::SessionRenewalFailed {
provider: "nearai".to_string(), provider: "nearai".to_string(),
reason: "Authentication timed out after 5 minutes".to_string(), reason: e.to_string(),
})??; })?;
let auth_provider = Some(auth_provider.to_string());
// Save the token // Save the token
self.save_session(&session_token, auth_provider.as_deref()) self.save_session(&session_token, auth_provider.as_deref())
@@ -642,17 +500,16 @@ pub async fn create_session_manager(config: SessionConfig) -> Arc<SessionManager
let manager = SessionManager::new_async(config).await; let manager = SessionManager::new_async(config).await;
// Check for legacy env var and migrate if present and no file token // Check for legacy env var and migrate if present and no file token
if !manager.has_token().await { if !manager.has_token().await
if let Ok(token) = std::env::var("NEARAI_SESSION_TOKEN") { && let Ok(token) = std::env::var("NEARAI_SESSION_TOKEN")
if !token.is_empty() { && !token.is_empty()
{
tracing::info!("Migrating session token from NEARAI_SESSION_TOKEN env var to file"); tracing::info!("Migrating session token from NEARAI_SESSION_TOKEN env var to file");
manager.set_token(SecretString::from(token.clone())).await; manager.set_token(SecretString::from(token.clone())).await;
if let Err(e) = manager.save_session(&token, None).await { if let Err(e) = manager.save_session(&token, None).await {
tracing::warn!("Failed to save migrated session: {}", e); tracing::warn!("Failed to save migrated session: {}", e);
} }
} }
}
}
Arc::new(manager) Arc::new(manager)
} }
@@ -671,7 +528,6 @@ mod tests {
let config = SessionConfig { let config = SessionConfig {
auth_base_url: "https://example.com".to_string(), auth_base_url: "https://example.com".to_string(),
session_path: session_path.clone(), session_path: session_path.clone(),
callback_port_range: (9900, 9910),
}; };
let manager = SessionManager::new_async(config.clone()).await; let manager = SessionManager::new_async(config.clone()).await;
@@ -712,7 +568,6 @@ mod tests {
let config = SessionConfig { let config = SessionConfig {
auth_base_url: "https://example.com".to_string(), auth_base_url: "https://example.com".to_string(),
session_path: dir.path().join("nonexistent.json"), session_path: dir.path().join("nonexistent.json"),
callback_port_range: (9900, 9910),
}; };
let manager = SessionManager::new_async(config).await; let manager = SessionManager::new_async(config).await;
+122 -88
View File
@@ -22,7 +22,10 @@ use ironclaw::{
config::Config, config::Config,
context::ContextManager, context::ContextManager,
extensions::ExtensionManager, extensions::ExtensionManager,
llm::{SessionConfig, create_llm_provider, create_session_manager}, llm::{
FailoverProvider, LlmProvider, SessionConfig, create_llm_provider,
create_llm_provider_with_config, create_session_manager,
},
orchestrator::{ orchestrator::{
ContainerJobConfig, ContainerJobManager, OrchestratorApi, TokenStore, ContainerJobConfig, ContainerJobManager, OrchestratorApi, TokenStore,
api::OrchestratorState, api::OrchestratorState,
@@ -45,7 +48,6 @@ use ironclaw::secrets::PostgresSecretsStore;
use ironclaw::secrets::SecretsCrypto; use ironclaw::secrets::SecretsCrypto;
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
use ironclaw::setup::{SetupConfig, SetupWizard}; use ironclaw::setup::{SetupConfig, SetupWizard};
#[tokio::main] #[tokio::main]
async fn main() -> anyhow::Result<()> { async fn main() -> anyhow::Result<()> {
let cli = Cli::parse(); let cli = Cli::parse();
@@ -90,7 +92,6 @@ async fn main() -> anyhow::Result<()> {
.init(); .init();
// Memory commands need database (and optionally embeddings) // Memory commands need database (and optionally embeddings)
let _ = dotenvy::dotenv();
let config = Config::from_env() let config = Config::from_env()
.await .await
.map_err(|e| anyhow::anyhow!("{}", e))?; .map_err(|e| anyhow::anyhow!("{}", e))?;
@@ -99,7 +100,6 @@ async fn main() -> anyhow::Result<()> {
let session = ironclaw::llm::create_session_manager(ironclaw::llm::SessionConfig { let session = ironclaw::llm::create_session_manager(ironclaw::llm::SessionConfig {
auth_base_url: config.llm.nearai.auth_base_url.clone(), auth_base_url: config.llm.nearai.auth_base_url.clone(),
session_path: config.llm.nearai.session_path.clone(), session_path: config.llm.nearai.session_path.clone(),
..Default::default()
}) })
.await; .await;
@@ -152,7 +152,6 @@ async fn main() -> anyhow::Result<()> {
return run_pairing_command(pairing_cmd.clone()).map_err(|e| anyhow::anyhow!("{}", e)); return run_pairing_command(pairing_cmd.clone()).map_err(|e| anyhow::anyhow!("{}", e));
} }
Some(Command::Status) => { Some(Command::Status) => {
let _ = dotenvy::dotenv();
tracing_subscriber::fmt() tracing_subscriber::fmt()
.with_env_filter( .with_env_filter(
EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("warn")), EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("warn")),
@@ -243,8 +242,10 @@ async fn main() -> anyhow::Result<()> {
skip_auth, skip_auth,
channels_only, channels_only,
}) => { }) => {
// Load .env before running onboarding wizard // Load .env files before running onboarding wizard.
// Standard ./.env first (higher priority), then ~/.ironclaw/.env.
let _ = dotenvy::dotenv(); let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
{ {
@@ -267,22 +268,22 @@ async fn main() -> anyhow::Result<()> {
} }
} }
// Load .env if present // Load .env files early so DATABASE_URL (and any other vars) are
// available to all subsequent env-based config resolution.
// Standard ./.env first (higher priority), then ~/.ironclaw/.env.
let _ = dotenvy::dotenv(); let _ = dotenvy::dotenv();
ironclaw::bootstrap::load_ironclaw_env();
// Enhanced first-run detection // Enhanced first-run detection
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
if !cli.no_onboard { if !cli.no_onboard
if let Some(reason) = check_onboard_needed().await { && let Some(reason) = check_onboard_needed()
{
println!("Onboarding needed: {}", reason); println!("Onboarding needed: {}", reason);
println!(); println!();
let mut wizard = SetupWizard::new(); let mut wizard = SetupWizard::new();
wizard.run().await?; wizard.run().await?;
} }
}
// Load bootstrap config (4 fields that must live on disk)
let bootstrap = ironclaw::bootstrap::BootstrapConfig::load();
// Load initial config from env + disk (before DB is available) // Load initial config from env + disk (before DB is available)
let mut config = match Config::from_env().await { let mut config = match Config::from_env().await {
@@ -303,7 +304,6 @@ async fn main() -> anyhow::Result<()> {
let session_config = SessionConfig { let session_config = SessionConfig {
auth_base_url: config.llm.nearai.auth_base_url.clone(), auth_base_url: config.llm.nearai.auth_base_url.clone(),
session_path: config.llm.nearai.session_path.clone(), session_path: config.llm.nearai.session_path.clone(),
..Default::default()
}; };
let session = create_session_manager(session_config).await; let session = create_session_manager(session_config).await;
@@ -314,7 +314,7 @@ async fn main() -> anyhow::Result<()> {
// Initialize tracing // Initialize tracing
let env_filter = EnvFilter::try_from_default_env() let env_filter = EnvFilter::try_from_default_env()
.unwrap_or_else(|_| EnvFilter::new("ironclaw=info,tower_http=debug")); .unwrap_or_else(|_| EnvFilter::new("ironclaw=info,tower_http=warn"));
// Create log broadcaster before tracing init so the WebLogLayer can capture all events. // Create log broadcaster before tracing init so the WebLogLayer can capture all events.
// This gets wired to the gateway's /api/logs/events SSE endpoint later. // This gets wired to the gateway's /api/logs/events SSE endpoint later.
@@ -322,7 +322,11 @@ async fn main() -> anyhow::Result<()> {
tracing_subscriber::registry() tracing_subscriber::registry()
.with(env_filter) .with(env_filter)
.with(tracing_subscriber::fmt::layer().with_target(false)) .with(
tracing_subscriber::fmt::layer()
.with_target(false)
.with_writer(ironclaw::tracing_fmt::TruncatingStderr::default()),
)
.with(WebLogLayer::new(Arc::clone(&log_broadcaster))) .with(WebLogLayer::new(Arc::clone(&log_broadcaster)))
.init(); .init();
@@ -418,7 +422,7 @@ async fn main() -> anyhow::Result<()> {
} }
// Reload config from DB now that we have a connection. // Reload config from DB now that we have a connection.
match Config::from_db(db.as_ref(), "default", &bootstrap).await { match Config::from_db(db.as_ref(), "default").await {
Ok(db_config) => { Ok(db_config) => {
config = db_config; config = db_config;
tracing::info!("Configuration reloaded from database"); tracing::info!("Configuration reloaded from database");
@@ -439,10 +443,97 @@ async fn main() -> anyhow::Result<()> {
tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e); tracing::warn!("Failed to cleanup stale sandbox jobs: {}", e);
} }
} }
// Create secrets store early: needed for injecting LLM API keys from encrypted
// storage before creating the LLM provider, and later for MCP auth + WASM channels.
//
// When both `postgres` and `libsql` features are compiled, the runtime-selected
// backend determines which store is created: whichever DB init branch ran will
// have set its handle (pg_pool or libsql_db), and the or_else chain picks it up.
let secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>> =
if let Some(master_key) = config.secrets.master_key() {
match SecretsCrypto::new(master_key.clone()) {
Ok(crypto) => {
let crypto = Arc::new(crypto);
let store: Option<Arc<dyn SecretsStore + Send + Sync>> = None;
#[cfg(feature = "libsql")]
let store = store.or_else(|| {
libsql_db.take().map(|db| {
Arc::new(LibSqlSecretsStore::new(db, Arc::clone(&crypto)))
as Arc<dyn SecretsStore + Send + Sync>
})
});
#[cfg(feature = "postgres")]
let store = store.or_else(|| {
pg_pool.as_ref().map(|pool| {
Arc::new(PostgresSecretsStore::new(pool.clone(), Arc::clone(&crypto)))
as Arc<dyn SecretsStore + Send + Sync>
})
});
store
}
Err(e) => {
tracing::warn!("Failed to initialize secrets crypto: {}", e);
#[cfg(feature = "libsql")]
let _ = libsql_db.take();
None
}
}
} else {
#[cfg(feature = "libsql")]
let _ = libsql_db.take();
None
};
// Inject LLM API keys from the encrypted secrets store into a thread-safe
// overlay so that optional_env() (used by LlmConfig::resolve()) picks them
// up. Then re-resolve LlmConfig with the newly available keys (backend may
// have been set during onboarding but the API key is in the secrets store).
if let Some(ref secrets) = secrets_store {
ironclaw::config::inject_llm_keys_from_secrets(secrets.as_ref(), "default").await;
// Re-resolve LlmConfig now that secrets overlay has been populated
if let Some(ref db_ref) = db {
match Config::from_db(db_ref.as_ref(), "default").await {
Ok(refreshed) => {
config = refreshed;
tracing::debug!("LlmConfig re-resolved after secret injection");
}
Err(e) => {
tracing::warn!("Failed to re-resolve config after secret injection: {}", e);
}
}
}
}
// Initialize LLM provider (clone session so we can reuse it for embeddings) // Initialize LLM provider (clone session so we can reuse it for embeddings)
let llm = create_llm_provider(&config.llm, session.clone())?; let llm = create_llm_provider(&config.llm, session.clone())?;
tracing::info!("LLM provider initialized: {}", llm.model_name()); tracing::info!("LLM provider initialized: {}", llm.model_name());
// Wrap in failover if a fallback model is configured
let llm: Arc<dyn LlmProvider> =
if let Some(fallback_model) = config.llm.nearai.fallback_model.as_ref() {
if fallback_model == &config.llm.nearai.model {
tracing::warn!(
"fallback_model is the same as primary model, failover may not be effective"
);
}
let mut fallback_config = config.llm.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(&fallback_config, session.clone())?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
"LLM failover enabled"
);
Arc::new(FailoverProvider::new(vec![llm, fallback])?)
} else {
llm
};
// Initialize safety layer // Initialize safety layer
let safety = Arc::new(SafetyLayer::new(&config.safety)); let safety = Arc::new(SafetyLayer::new(&config.safety));
tracing::info!("Safety layer initialized"); tracing::info!("Safety layer initialized");
@@ -516,49 +607,6 @@ async fn main() -> anyhow::Result<()> {
tracing::info!("Builder mode enabled"); tracing::info!("Builder mode enabled");
} }
// Create secrets store if master key is configured (needed for MCP auth and WASM channels).
//
// When both `postgres` and `libsql` features are compiled, the runtime-selected
// backend determines which store is created: whichever DB init branch ran will
// have set its handle (pg_pool or libsql_db), and the or_else chain picks it up.
let secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>> =
if let Some(master_key) = config.secrets.master_key() {
match SecretsCrypto::new(master_key.clone()) {
Ok(crypto) => {
let crypto = Arc::new(crypto);
let store: Option<Arc<dyn SecretsStore + Send + Sync>> = None;
#[cfg(feature = "libsql")]
let store = store.or_else(|| {
libsql_db.take().map(|db| {
Arc::new(LibSqlSecretsStore::new(db, Arc::clone(&crypto)))
as Arc<dyn SecretsStore + Send + Sync>
})
});
#[cfg(feature = "postgres")]
let store = store.or_else(|| {
pg_pool.as_ref().map(|pool| {
Arc::new(PostgresSecretsStore::new(pool.clone(), Arc::clone(&crypto)))
as Arc<dyn SecretsStore + Send + Sync>
})
});
store
}
Err(e) => {
tracing::warn!("Failed to initialize secrets crypto: {}", e);
#[cfg(feature = "libsql")]
let _ = libsql_db.take();
None
}
}
} else {
#[cfg(feature = "libsql")]
let _ = libsql_db.take();
None
};
let mcp_session_manager = Arc::new(McpSessionManager::new()); let mcp_session_manager = Arc::new(McpSessionManager::new());
// Create WASM tool runtime (sync, just builds the wasmtime engine) // Create WASM tool runtime (sync, just builds the wasmtime engine)
@@ -579,7 +627,10 @@ async fn main() -> anyhow::Result<()> {
// Both register into the shared ToolRegistry (RwLock-based) so concurrent writes are safe. // Both register into the shared ToolRegistry (RwLock-based) so concurrent writes are safe.
let wasm_tools_future = async { let wasm_tools_future = async {
if let Some(ref runtime) = wasm_tool_runtime { if let Some(ref runtime) = wasm_tool_runtime {
let loader = WasmToolLoader::new(Arc::clone(runtime), Arc::clone(&tools)); let mut loader = WasmToolLoader::new(Arc::clone(runtime), Arc::clone(&tools));
if let Some(ref secrets) = secrets_store {
loader = loader.with_secrets_store(Arc::clone(secrets));
}
// Load installed tools from ~/.ironclaw/tools/ // Load installed tools from ~/.ironclaw/tools/
match loader.load_from_dir(&config.wasm.tools_dir).await { match loader.load_from_dir(&config.wasm.tools_dir).await {
@@ -902,14 +953,14 @@ async fn main() -> anyhow::Result<()> {
// Inject owner_id for Telegram so the bot only responds // Inject owner_id for Telegram so the bot only responds
// to the bound user account. // to the bound user account.
if channel_name == "telegram" { if channel_name == "telegram"
if let Some(owner_id) = config.channels.telegram_owner_id { && let Some(owner_id) = config.channels.telegram_owner_id
{
config_updates.insert( config_updates.insert(
"owner_id".to_string(), "owner_id".to_string(),
serde_json::json!(owner_id), serde_json::json!(owner_id),
); );
} }
}
if !config_updates.is_empty() { if !config_updates.is_empty() {
channel_arc.update_config(config_updates).await; channel_arc.update_config(config_updates).await;
@@ -999,8 +1050,9 @@ async fn main() -> anyhow::Result<()> {
// Extract its routes for the unified server; the channel itself just // Extract its routes for the unified server; the channel itself just
// provides the mpsc stream. // provides the mpsc stream.
let mut webhook_server_addr: Option<std::net::SocketAddr> = None; let mut webhook_server_addr: Option<std::net::SocketAddr> = None;
if !cli.cli_only { if !cli.cli_only
if let Some(ref http_config) = config.channels.http { && let Some(ref http_config) = config.channels.http
{
let http_channel = HttpChannel::new(http_config.clone()); let http_channel = HttpChannel::new(http_config.clone());
webhook_routes.push(http_channel.routes()); webhook_routes.push(http_channel.routes());
let (host, port) = http_channel.addr(); let (host, port) = http_channel.addr();
@@ -1016,7 +1068,6 @@ async fn main() -> anyhow::Result<()> {
http_config.port http_config.port
); );
} }
}
// Start the unified webhook server if any routes were registered. // Start the unified webhook server if any routes were registered.
let mut webhook_server = if !webhook_routes.is_empty() { let mut webhook_server = if !webhook_routes.is_empty() {
@@ -1166,13 +1217,11 @@ async fn main() -> anyhow::Result<()> {
/// Check if onboarding is needed and return the reason. /// Check if onboarding is needed and return the reason.
/// ///
/// Returns `Some(reason)` if onboarding should be triggered, `None` otherwise. /// Returns `Some(reason)` if onboarding should be triggered, `None` otherwise.
/// Called after `load_ironclaw_env()`, so DATABASE_URL from `~/.ironclaw/.env`
/// is already in the environment.
#[cfg(any(feature = "postgres", feature = "libsql"))] #[cfg(any(feature = "postgres", feature = "libsql"))]
async fn check_onboard_needed() -> Option<&'static str> { fn check_onboard_needed() -> Option<&'static str> {
let bootstrap = ironclaw::bootstrap::BootstrapConfig::load(); let has_db = std::env::var("DATABASE_URL").is_ok()
// Database not configured (and not in env)
let has_db = bootstrap.database_url.is_some()
|| std::env::var("DATABASE_URL").is_ok()
|| std::env::var("LIBSQL_PATH").is_ok() || std::env::var("LIBSQL_PATH").is_ok()
|| ironclaw::config::default_libsql_path().exists(); || ironclaw::config::default_libsql_path().exists();
@@ -1180,21 +1229,6 @@ async fn check_onboard_needed() -> Option<&'static str> {
return Some("Database not configured"); return Some("Database not configured");
} }
// Secrets not configured (and not in env)
if bootstrap.secrets_master_key_source == ironclaw::settings::KeySource::None
&& std::env::var("SECRETS_MASTER_KEY").is_err()
&& !ironclaw::secrets::keychain::has_master_key().await
{
// Only require secrets setup if user hasn't explicitly disabled it
// For now, we don't require it for first run
}
// First run (onboarding never completed and no session)
let session_path = ironclaw::llm::session::default_session_path();
if !bootstrap.onboard_completed && !session_path.exists() {
return Some("First run");
}
None None
} }
+3 -3
View File
@@ -339,8 +339,9 @@ async fn get_prompt_handler(
Path(job_id): Path<Uuid>, Path(job_id): Path<Uuid>,
) -> Result<(StatusCode, Json<serde_json::Value>), StatusCode> { ) -> Result<(StatusCode, Json<serde_json::Value>), StatusCode> {
let mut queue = state.prompt_queue.lock().await; let mut queue = state.prompt_queue.lock().await;
if let Some(prompts) = queue.get_mut(&job_id) { if let Some(prompts) = queue.get_mut(&job_id)
if let Some(prompt) = prompts.pop_front() { && let Some(prompt) = prompts.pop_front()
{
return Ok(( return Ok((
StatusCode::OK, StatusCode::OK,
Json(serde_json::json!({ Json(serde_json::json!({
@@ -349,7 +350,6 @@ async fn get_prompt_handler(
})), })),
)); ));
} }
}
// Return 204 with an empty body. The Json wrapper requires some value // Return 204 with an empty body. The Json wrapper requires some value
// but the status code signals "nothing here". // but the status code signals "nothing here".
+6 -6
View File
@@ -229,8 +229,9 @@ impl ContainerJobManager {
.unwrap_or_else(|| PathBuf::from(".")) .unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw") .join(".ironclaw")
.join("projects"); .join("projects");
if let Ok(canonical_base) = projects_base.canonicalize() { if let Ok(canonical_base) = projects_base.canonicalize()
if !canonical.starts_with(&canonical_base) { && !canonical.starts_with(&canonical_base)
{
return Err(OrchestratorError::ContainerCreationFailed { return Err(OrchestratorError::ContainerCreationFailed {
job_id, job_id,
reason: format!( reason: format!(
@@ -240,7 +241,6 @@ impl ContainerJobManager {
), ),
}); });
} }
}
binds.push(format!("{}:/workspace:rw", canonical.display())); binds.push(format!("{}:/workspace:rw", canonical.display()));
env_vec.push("IRONCLAW_WORKSPACE=/workspace".to_string()); env_vec.push("IRONCLAW_WORKSPACE=/workspace".to_string());
} }
@@ -442,8 +442,9 @@ impl ContainerJobManager {
let containers = self.containers.read().await; let containers = self.containers.read().await;
containers.get(&job_id).map(|h| h.container_id.clone()) containers.get(&job_id).map(|h| h.container_id.clone())
}; };
if let Some(cid) = container_id { if let Some(cid) = container_id
if !cid.is_empty() { && !cid.is_empty()
{
match connect_docker().await { match connect_docker().await {
Ok(docker) => { Ok(docker) => {
if let Err(e) = docker if let Err(e) = docker
@@ -473,7 +474,6 @@ impl ContainerJobManager {
} }
} }
} }
}
self.token_store.revoke(job_id).await; self.token_store.revoke(job_id).await;
tracing::info!(job_id = %job_id, "Completed worker container"); tracing::info!(job_id = %job_id, "Completed worker container");
+3 -3
View File
@@ -147,12 +147,12 @@ impl LeakDetector {
// Build prefix matcher for patterns that start with a known prefix // Build prefix matcher for patterns that start with a known prefix
let mut prefixes = Vec::new(); let mut prefixes = Vec::new();
for (idx, pattern) in patterns.iter().enumerate() { for (idx, pattern) in patterns.iter().enumerate() {
if let Some(prefix) = extract_literal_prefix(pattern.regex.as_str()) { if let Some(prefix) = extract_literal_prefix(pattern.regex.as_str())
if prefix.len() >= 3 { && prefix.len() >= 3
{
prefixes.push((prefix, idx)); prefixes.push((prefix, idx));
} }
} }
}
let prefix_matcher = if !prefixes.is_empty() { let prefix_matcher = if !prefixes.is_empty() {
let prefix_strings: Vec<&str> = prefixes.iter().map(|(s, _)| s.as_str()).collect(); let prefix_strings: Vec<&str> = prefixes.iter().map(|(s, _)| s.as_str()).collect();
+4 -5
View File
@@ -494,11 +494,11 @@ impl ContainerRunner {
/// 3. `~/.docker/run/docker.sock` (Docker Desktop on macOS) /// 3. `~/.docker/run/docker.sock` (Docker Desktop on macOS)
pub async fn connect_docker() -> Result<Docker> { pub async fn connect_docker() -> Result<Docker> {
// First try bollard defaults (checks DOCKER_HOST, then /var/run/docker.sock) // First try bollard defaults (checks DOCKER_HOST, then /var/run/docker.sock)
if let Ok(docker) = Docker::connect_with_local_defaults() { if let Ok(docker) = Docker::connect_with_local_defaults()
if docker.ping().await.is_ok() { && docker.ping().await.is_ok()
{
return Ok(docker); return Ok(docker);
} }
}
// Try Docker Desktop socket (macOS) // Try Docker Desktop socket (macOS)
if let Some(home) = std::env::var_os("HOME") { if let Some(home) = std::env::var_os("HOME") {
@@ -507,13 +507,12 @@ pub async fn connect_docker() -> Result<Docker> {
let sock_str = desktop_sock.to_string_lossy(); let sock_str = desktop_sock.to_string_lossy();
if let Ok(docker) = if let Ok(docker) =
Docker::connect_with_socket(&sock_str, 120, bollard::API_DEFAULT_VERSION) Docker::connect_with_socket(&sock_str, 120, bollard::API_DEFAULT_VERSION)
&& docker.ping().await.is_ok()
{ {
if docker.ping().await.is_ok() {
return Ok(docker); return Ok(docker);
} }
} }
} }
}
Err(SandboxError::DockerNotAvailable { Err(SandboxError::DockerNotAvailable {
reason: "Socket not found: /var/run/docker.sock".to_string(), reason: "Socket not found: /var/run/docker.sock".to_string(),
+6 -6
View File
@@ -259,12 +259,12 @@ async fn handle_connect(
let decision = state.decider.decide(&network_req).await; let decision = state.decider.decide(&network_req).await;
if !decision.is_allowed() { if !decision.is_allowed()
if let NetworkDecision::Deny { reason } = decision { && let NetworkDecision::Deny { reason } = decision
{
tracing::info!("Proxy: blocked CONNECT {} - {}", host, reason); tracing::info!("Proxy: blocked CONNECT {} - {}", host, reason);
return error_response(StatusCode::FORBIDDEN, reason); return error_response(StatusCode::FORBIDDEN, reason);
} }
}
tracing::debug!("Proxy: allowing CONNECT to {}", host); tracing::debug!("Proxy: allowing CONNECT to {}", host);
@@ -294,12 +294,12 @@ async fn forward_request(
// Copy headers (except hop-by-hop headers) // Copy headers (except hop-by-hop headers)
for (name, value) in req.headers() { for (name, value) in req.headers() {
if !is_hop_by_hop_header(name.as_str()) { if !is_hop_by_hop_header(name.as_str())
if let Ok(v) = value.to_str() { && let Ok(v) = value.to_str()
{
builder = builder.header(name.as_str(), v); builder = builder.header(name.as_str(), v);
} }
} }
}
// Inject credentials if needed // Inject credentials if needed
if let NetworkDecision::AllowWithCredentials { if let NetworkDecision::AllowWithCredentials {
+2 -3
View File
@@ -109,13 +109,12 @@ impl NetworkPolicyDecider for DefaultPolicyDecider {
async fn decide(&self, request: &NetworkRequest) -> NetworkDecision { async fn decide(&self, request: &NetworkRequest) -> NetworkDecision {
// First check if the domain is allowed // First check if the domain is allowed
let validation = self.allowlist.is_allowed(&request.host); let validation = self.allowlist.is_allowed(&request.host);
if !validation.is_allowed() { if !validation.is_allowed()
if let crate::sandbox::proxy::allowlist::DomainValidationResult::Denied(reason) = && let crate::sandbox::proxy::allowlist::DomainValidationResult::Denied(reason) =
validation validation
{ {
return NetworkDecision::Deny { reason }; return NetworkDecision::Deny { reason };
} }
}
// Check if we need to inject credentials // Check if we need to inject credentials
if let Some(mapping) = self.find_credential(&request.host) { if let Some(mapping) = self.find_credential(&request.host) {
+1 -1
View File
@@ -261,7 +261,7 @@ pub use platform::{delete_master_key, get_master_key, has_master_key, store_mast
/// Parse a hex string to bytes. /// Parse a hex string to bytes.
fn hex_to_bytes(hex: &str) -> Result<Vec<u8>, SecretError> { fn hex_to_bytes(hex: &str) -> Result<Vec<u8>, SecretError> {
if hex.len() % 2 != 0 { if !hex.len().is_multiple_of(2) {
return Err(SecretError::KeychainError( return Err(SecretError::KeychainError(
"Invalid hex string length".to_string(), "Invalid hex string length".to_string(),
)); ));
+54 -17
View File
@@ -153,11 +153,11 @@ impl SecretsStore for PostgresSecretsStore {
let secret = row_to_secret(&r); let secret = row_to_secret(&r);
// Check expiration // Check expiration
if let Some(expires_at) = secret.expires_at { if let Some(expires_at) = secret.expires_at
if expires_at < Utc::now() { && expires_at < Utc::now()
{
return Err(SecretError::Expired); return Err(SecretError::Expired);
} }
}
Ok(secret) Ok(secret)
} }
@@ -276,12 +276,12 @@ impl SecretsStore for PostgresSecretsStore {
} }
// Simple glob: * matches any suffix // Simple glob: * matches any suffix
if let Some(prefix) = pattern.strip_suffix('*') { if let Some(prefix) = pattern.strip_suffix('*')
if secret_name.starts_with(prefix) { && secret_name.starts_with(prefix)
{
return Ok(true); return Ok(true);
} }
} }
}
Ok(false) Ok(false)
} }
@@ -432,11 +432,11 @@ impl SecretsStore for LibSqlSecretsStore {
Some(row) => { Some(row) => {
let secret = libsql_row_to_secret(&row)?; let secret = libsql_row_to_secret(&row)?;
if let Some(expires_at) = secret.expires_at { if let Some(expires_at) = secret.expires_at
if expires_at < Utc::now() { && expires_at < Utc::now()
{
return Err(SecretError::Expired); return Err(SecretError::Expired);
} }
}
Ok(secret) Ok(secret)
} }
@@ -541,12 +541,12 @@ impl SecretsStore for LibSqlSecretsStore {
return Ok(true); return Ok(true);
} }
if let Some(prefix) = pattern.strip_suffix('*') { if let Some(prefix) = pattern.strip_suffix('*')
if secret_name.starts_with(prefix) { && secret_name.starts_with(prefix)
{
return Ok(true); return Ok(true);
} }
} }
}
Ok(false) Ok(false)
} }
@@ -695,12 +695,21 @@ pub mod testing {
} }
async fn get(&self, user_id: &str, name: &str) -> Result<Secret, SecretError> { async fn get(&self, user_id: &str, name: &str) -> Result<Secret, SecretError> {
self.secrets let secret = self
.secrets
.read() .read()
.await .await
.get(&(user_id.to_string(), name.to_string())) .get(&(user_id.to_string(), name.to_string()))
.cloned() .cloned()
.ok_or_else(|| SecretError::NotFound(name.to_string())) .ok_or_else(|| SecretError::NotFound(name.to_string()))?;
if let Some(expires_at) = secret.expires_at
&& expires_at < Utc::now()
{
return Err(SecretError::Expired);
}
Ok(secret)
} }
async fn get_decrypted( async fn get_decrypted(
@@ -761,12 +770,12 @@ pub mod testing {
if pattern == secret_name { if pattern == secret_name {
return Ok(true); return Ok(true);
} }
if let Some(prefix) = pattern.strip_suffix('*') { if let Some(prefix) = pattern.strip_suffix('*')
if secret_name.starts_with(prefix) { && secret_name.starts_with(prefix)
{
return Ok(true); return Ok(true);
} }
} }
}
Ok(false) Ok(false)
} }
} }
@@ -889,6 +898,34 @@ mod tests {
); );
} }
#[tokio::test]
async fn test_expired_secret_returns_error() {
let store = test_store();
let expires_at = chrono::Utc::now() - chrono::Duration::hours(1);
let params = CreateSecretParams::new("expired_key", "value").with_expiry(expires_at);
store.create("user1", params).await.unwrap();
let result = store.get("user1", "expired_key").await;
assert!(result.is_err());
assert!(matches!(
result.unwrap_err(),
crate::secrets::SecretError::Expired
));
}
#[tokio::test]
async fn test_non_expired_secret_succeeds() {
let store = test_store();
let expires_at = chrono::Utc::now() + chrono::Duration::hours(1);
let params = CreateSecretParams::new("fresh_key", "value").with_expiry(expires_at);
store.create("user1", params).await.unwrap();
let result = store.get("user1", "fresh_key").await;
assert!(result.is_ok());
}
#[tokio::test] #[tokio::test]
async fn test_user_isolation() { async fn test_user_isolation() {
let store = test_store(); let store = test_store();
+69 -77
View File
@@ -40,8 +40,18 @@ pub struct Settings {
#[serde(default)] #[serde(default)]
pub secrets_master_key_source: KeySource, pub secrets_master_key_source: KeySource,
// === Step 3: NEAR AI Auth === // === Step 3: Inference Provider ===
// Session stored separately in session.json /// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible".
#[serde(default)]
pub llm_backend: Option<String>,
/// Ollama base URL (when llm_backend = "ollama").
#[serde(default)]
pub ollama_base_url: Option<String>,
/// OpenAI-compatible endpoint base URL (when llm_backend = "openai_compatible").
#[serde(default)]
pub openai_compatible_base_url: Option<String>,
// === Step 4: Model Selection === // === Step 4: Model Selection ===
/// Currently selected model. /// Currently selected model.
@@ -499,20 +509,16 @@ impl Default for BuilderSettings {
} }
impl Settings { impl Settings {
/// Get the default settings file path (~/.ironclaw/settings.json).
pub fn default_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".ironclaw")
.join("settings.json")
}
/// Reconstruct Settings from a flat key-value map (as stored in the DB). /// Reconstruct Settings from a flat key-value map (as stored in the DB).
/// ///
/// Each key is a dotted path (e.g., "agent.name"), value is a JSONB value. /// Each key is a dotted path (e.g., "agent.name"), value is a JSONB value.
/// Missing keys get their default value. /// Missing keys get their default value.
pub fn from_db_map(map: &std::collections::HashMap<String, serde_json::Value>) -> Self { pub fn from_db_map(map: &std::collections::HashMap<String, serde_json::Value>) -> Self {
// Start with defaults, then overlay each DB setting // Start with defaults, then overlay each DB setting.
//
// The settings table stores both Settings struct fields and app-specific
// data (e.g. nearai.session_token). Skip keys that don't correspond to
// a known Settings path.
let mut settings = Self::default(); let mut settings = Self::default();
for (key, value) in map { for (key, value) in map {
@@ -521,11 +527,16 @@ impl Settings {
serde_json::Value::String(s) => s.clone(), serde_json::Value::String(s) => s.clone(),
serde_json::Value::Bool(b) => b.to_string(), serde_json::Value::Bool(b) => b.to_string(),
serde_json::Value::Number(n) => n.to_string(), serde_json::Value::Number(n) => n.to_string(),
serde_json::Value::Null => "null".to_string(), serde_json::Value::Null => continue, // null means default, skip
other => other.to_string(), other => other.to_string(),
}; };
if let Err(e) = settings.set(key, &value_str) { match settings.set(key, &value_str) {
Ok(()) => {}
// The settings table stores both Settings fields and app-specific
// data (e.g. nearai.session_token). Silently skip unknown paths.
Err(e) if e.starts_with("Path not found") => {}
Err(e) => {
tracing::warn!( tracing::warn!(
"Failed to apply DB setting '{}' = '{}': {}", "Failed to apply DB setting '{}' = '{}': {}",
key, key,
@@ -534,6 +545,7 @@ impl Settings {
); );
} }
} }
}
settings settings
} }
@@ -552,50 +564,27 @@ impl Settings {
map map
} }
/// Get the default settings file path (~/.ironclaw/settings.json).
pub fn default_path() -> std::path::PathBuf {
dirs::home_dir()
.unwrap_or_else(|| std::path::PathBuf::from("."))
.join(".ironclaw")
.join("settings.json")
}
/// Load settings from disk, returning default if not found. /// Load settings from disk, returning default if not found.
pub fn load() -> Self { pub fn load() -> Self {
Self::load_from(&Self::default_path()) Self::load_from(&Self::default_path())
} }
/// Load settings from a specific path. /// Load settings from a specific path (used by bootstrap legacy migration).
pub fn load_from(path: &PathBuf) -> Self { pub fn load_from(path: &std::path::Path) -> Self {
match std::fs::read_to_string(path) { match std::fs::read_to_string(path) {
Ok(data) => serde_json::from_str(&data).unwrap_or_default(), Ok(data) => serde_json::from_str(&data).unwrap_or_default(),
Err(_) => Self::default(), Err(_) => Self::default(),
} }
} }
/// Save settings to disk.
pub fn save(&self) -> std::io::Result<()> {
self.save_to(&Self::default_path())
}
/// Save settings to a specific path.
pub fn save_to(&self, path: &PathBuf) -> std::io::Result<()> {
// Ensure parent directory exists
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let json = serde_json::to_string_pretty(self)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e.to_string()))?;
std::fs::write(path, json)
}
/// Get the selected model, falling back to the provided default.
pub fn model_or(&self, default: &str) -> String {
self.selected_model
.clone()
.unwrap_or_else(|| default.to_string())
}
/// Set the selected model and save.
pub fn set_model(&mut self, model: &str) -> std::io::Result<()> {
self.selected_model = Some(model.to_string());
self.save()
}
/// Get a setting value by dotted path (e.g., "agent.max_parallel_jobs"). /// Get a setting value by dotted path (e.g., "agent.max_parallel_jobs").
pub fn get(&self, path: &str) -> Option<String> { pub fn get(&self, path: &str) -> Option<String> {
let json = serde_json::to_value(self).ok()?; let json = serde_json::to_value(self).ok()?;
@@ -780,42 +769,22 @@ fn collect_settings(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::tempdir;
#[test] #[test]
fn test_settings_save_load() { fn test_db_map_round_trip() {
let dir = tempdir().unwrap();
let path = dir.path().join("settings.json");
let settings = Settings { let settings = Settings {
selected_model: Some("claude-3-5-sonnet-20241022".to_string()), selected_model: Some("claude-3-5-sonnet-20241022".to_string()),
..Default::default() ..Default::default()
}; };
settings.save_to(&path).unwrap(); let map = settings.to_db_map();
let restored = Settings::from_db_map(&map);
let loaded = Settings::load_from(&path);
assert_eq!( assert_eq!(
loaded.selected_model, restored.selected_model,
Some("claude-3-5-sonnet-20241022".to_string()) Some("claude-3-5-sonnet-20241022".to_string())
); );
} }
#[test]
fn test_model_or_default() {
let settings = Settings::default();
assert_eq!(
settings.model_or("default-model"),
"default-model".to_string()
);
let settings = Settings {
selected_model: Some("my-model".to_string()),
..Default::default()
};
assert_eq!(settings.model_or("default-model"), "my-model".to_string());
}
#[test] #[test]
fn test_get_setting() { fn test_get_setting() {
let settings = Settings::default(); let settings = Settings::default();
@@ -886,16 +855,13 @@ mod tests {
} }
#[test] #[test]
fn test_telegram_owner_id_round_trip() { fn test_telegram_owner_id_db_round_trip() {
let dir = tempdir().unwrap();
let path = dir.path().join("settings.json");
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.channels.telegram_owner_id = Some(123456789); settings.channels.telegram_owner_id = Some(123456789);
settings.save_to(&path).unwrap();
let loaded = Settings::load_from(&path); let map = settings.to_db_map();
assert_eq!(loaded.channels.telegram_owner_id, Some(123456789)); let restored = Settings::from_db_map(&map);
assert_eq!(restored.channels.telegram_owner_id, Some(123456789));
} }
#[test] #[test]
@@ -912,4 +878,30 @@ mod tests {
.unwrap(); .unwrap();
assert_eq!(settings.channels.telegram_owner_id, Some(987654321)); assert_eq!(settings.channels.telegram_owner_id, Some(987654321));
} }
#[test]
fn test_llm_backend_round_trip() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("settings.json");
let settings = Settings {
llm_backend: Some("anthropic".to_string()),
ollama_base_url: Some("http://localhost:11434".to_string()),
openai_compatible_base_url: Some("http://my-vllm:8000/v1".to_string()),
..Default::default()
};
let json = serde_json::to_string_pretty(&settings).unwrap();
std::fs::write(&path, json).unwrap();
let loaded = Settings::load_from(&path);
assert_eq!(loaded.llm_backend, Some("anthropic".to_string()));
assert_eq!(
loaded.ollama_base_url,
Some("http://localhost:11434".to_string())
);
assert_eq!(
loaded.openai_compatible_base_url,
Some("http://my-vllm:8000/v1".to_string())
);
}
} }
+141 -102
View File
@@ -15,11 +15,27 @@ use serde::Deserialize;
#[cfg(feature = "postgres")] #[cfg(feature = "postgres")]
use crate::secrets::SecretsCrypto; use crate::secrets::SecretsCrypto;
use crate::secrets::{CreateSecretParams, SecretsStore}; use crate::secrets::{CreateSecretParams, SecretsStore};
use crate::settings::Settings; use crate::settings::{Settings, TunnelSettings};
use crate::setup::prompts::{ use crate::setup::prompts::{
confirm, input, optional_input, print_error, print_info, print_success, secret_input, confirm, input, optional_input, print_error, print_info, print_success, secret_input,
}; };
/// Typed errors for channel setup flows.
#[derive(Debug, thiserror::Error)]
pub enum ChannelSetupError {
#[error("I/O error: {0}")]
Io(#[from] std::io::Error),
#[error("{0}")]
Network(String),
#[error("{0}")]
Secrets(String),
#[error("{0}")]
Validation(String),
}
/// Context for saving secrets during setup. /// Context for saving secrets during setup.
pub struct SecretsContext { pub struct SecretsContext {
store: Arc<dyn SecretsStore>, store: Arc<dyn SecretsStore>,
@@ -45,32 +61,39 @@ impl SecretsContext {
} }
/// Save a secret to the database. /// Save a secret to the database.
pub async fn save_secret(&self, name: &str, value: &SecretString) -> Result<(), String> { pub async fn save_secret(
&self,
name: &str,
value: &SecretString,
) -> Result<(), ChannelSetupError> {
let params = CreateSecretParams::new(name, value.expose_secret()); let params = CreateSecretParams::new(name, value.expose_secret());
self.store self.store
.create(&self.user_id, params) .create(&self.user_id, params)
.await .await
.map_err(|e| format!("Failed to save secret: {}", e))?; .map_err(|e| ChannelSetupError::Secrets(format!("Failed to save secret: {}", e)))?;
Ok(()) Ok(())
} }
/// Check if a secret exists. /// Check if a secret exists.
pub async fn secret_exists(&self, name: &str) -> bool { pub async fn secret_exists(&self, name: &str) -> bool {
self.store match self.store.exists(&self.user_id, name).await {
.exists(&self.user_id, name) Ok(exists) => exists,
.await Err(e) => {
.unwrap_or(false) tracing::warn!(secret = name, error = %e, "Failed to check if secret exists, assuming absent");
false
}
}
} }
/// Read a secret from the database (decrypted). /// Read a secret from the database (decrypted).
pub async fn get_secret(&self, name: &str) -> Result<SecretString, String> { pub async fn get_secret(&self, name: &str) -> Result<SecretString, ChannelSetupError> {
let decrypted = self let decrypted = self
.store .store
.get_decrypted(&self.user_id, name) .get_decrypted(&self.user_id, name)
.await .await
.map_err(|e| format!("Failed to read secret: {}", e))?; .map_err(|e| ChannelSetupError::Secrets(format!("Failed to read secret: {}", e)))?;
Ok(SecretString::from(decrypted.expose().to_string())) Ok(SecretString::from(decrypted.expose().to_string()))
} }
} }
@@ -107,7 +130,6 @@ struct TelegramGetUpdatesResponse {
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
struct TelegramUpdate { struct TelegramUpdate {
#[allow(dead_code)]
update_id: i64, update_id: i64,
message: Option<TelegramUpdateMessage>, message: Option<TelegramUpdateMessage>,
} }
@@ -131,7 +153,10 @@ struct TelegramUpdateUser {
/// 2. Entering the bot token /// 2. Entering the bot token
/// 3. Validating the token /// 3. Validating the token
/// 4. Saving the token to the database /// 4. Saving the token to the database
pub async fn setup_telegram(secrets: &SecretsContext) -> Result<TelegramSetupResult, String> { pub async fn setup_telegram(
secrets: &SecretsContext,
settings: &Settings,
) -> Result<TelegramSetupResult, ChannelSetupError> {
println!("Telegram Setup:"); println!("Telegram Setup:");
println!(); println!();
print_info("To create a Telegram bot:"); print_info("To create a Telegram bot:");
@@ -143,10 +168,10 @@ pub async fn setup_telegram(secrets: &SecretsContext) -> Result<TelegramSetupRes
// Check if token already exists // Check if token already exists
if secrets.secret_exists("telegram_bot_token").await { if secrets.secret_exists("telegram_bot_token").await {
print_info("Existing Telegram token found in database."); print_info("Existing Telegram token found in database.");
if !confirm("Replace existing token?", false).map_err(|e| e.to_string())? { if !confirm("Replace existing token?", false)? {
// Still offer to configure webhook secret and owner binding // Still offer to configure webhook secret and owner binding
let webhook_secret = setup_telegram_webhook_secret(secrets).await?; let webhook_secret = setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
let owner_id = bind_telegram_owner_flow(secrets).await?; let owner_id = bind_telegram_owner_flow(secrets, settings).await?;
return Ok(TelegramSetupResult { return Ok(TelegramSetupResult {
enabled: true, enabled: true,
bot_username: None, bot_username: None,
@@ -156,7 +181,8 @@ pub async fn setup_telegram(secrets: &SecretsContext) -> Result<TelegramSetupRes
} }
} }
let token = secret_input("Bot token (from @BotFather)").map_err(|e| e.to_string())?; loop {
let token = secret_input("Bot token (from @BotFather)")?;
// Validate the token // Validate the token
print_info("Validating bot token..."); print_info("Validating bot token...");
@@ -176,27 +202,27 @@ pub async fn setup_telegram(secrets: &SecretsContext) -> Result<TelegramSetupRes
let owner_id = bind_telegram_owner(&token).await?; let owner_id = bind_telegram_owner(&token).await?;
// Offer webhook secret configuration // Offer webhook secret configuration
let webhook_secret = setup_telegram_webhook_secret(secrets).await?; let webhook_secret =
setup_telegram_webhook_secret(secrets, &settings.tunnel).await?;
Ok(TelegramSetupResult { return Ok(TelegramSetupResult {
enabled: true, enabled: true,
bot_username: username, bot_username: username,
webhook_secret, webhook_secret,
owner_id, owner_id,
}) });
} }
Err(e) => { Err(e) => {
print_error(&format!("Token validation failed: {}", e)); print_error(&format!("Token validation failed: {}", e));
if confirm("Try again?", true).map_err(|e| e.to_string())? { if !confirm("Try again?", true)? {
Box::pin(setup_telegram(secrets)).await return Ok(TelegramSetupResult {
} else {
Ok(TelegramSetupResult {
enabled: false, enabled: false,
bot_username: None, bot_username: None,
webhook_secret: None, webhook_secret: None,
owner_id: None, owner_id: None,
}) });
}
} }
} }
} }
@@ -206,14 +232,14 @@ pub async fn setup_telegram(secrets: &SecretsContext) -> Result<TelegramSetupRes
/// ///
/// Polls `getUpdates` until a message arrives, then captures the sender's user ID. /// Polls `getUpdates` until a message arrives, then captures the sender's user ID.
/// Returns `None` if the user declines or the flow times out. /// Returns `None` if the user declines or the flow times out.
async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, String> { async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, ChannelSetupError> {
println!(); println!();
print_info("Account Binding (recommended):"); print_info("Account Binding (recommended):");
print_info("Binding restricts the bot so only YOU can use it."); print_info("Binding restricts the bot so only YOU can use it.");
print_info("Without this, anyone who finds your bot can send it messages."); print_info("Without this, anyone who finds your bot can send it messages.");
println!(); println!();
if !confirm("Bind bot to your Telegram account?", true).map_err(|e| e.to_string())? { if !confirm("Bind bot to your Telegram account?", true)? {
print_info("Skipping account binding. Bot will accept messages from all users."); print_info("Skipping account binding. Bot will accept messages from all users.");
return Ok(None); return Ok(None);
} }
@@ -224,14 +250,16 @@ async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, String
let client = Client::builder() let client = Client::builder()
.timeout(std::time::Duration::from_secs(35)) .timeout(std::time::Duration::from_secs(35))
.build() .build()
.map_err(|e| format!("Failed to create HTTP client: {}", e))?; .map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
// Clear any existing webhook so getUpdates works // Clear any existing webhook so getUpdates works
let delete_url = format!( let delete_url = format!(
"https://api.telegram.org/bot{}/deleteWebhook", "https://api.telegram.org/bot{}/deleteWebhook",
token.expose_secret() token.expose_secret()
); );
let _ = client.post(&delete_url).send().await; if let Err(e) = client.post(&delete_url).send().await {
tracing::warn!("Failed to delete webhook (getUpdates may not work): {e}");
}
let updates_url = format!( let updates_url = format!(
"https://api.telegram.org/bot{}/getUpdates", "https://api.telegram.org/bot{}/getUpdates",
@@ -246,25 +274,30 @@ async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, String
.query(&[("timeout", "30"), ("allowed_updates", "[\"message\"]")]) .query(&[("timeout", "30"), ("allowed_updates", "[\"message\"]")])
.send() .send()
.await .await
.map_err(|e| format!("getUpdates request failed: {}", e))?; .map_err(|e| ChannelSetupError::Network(format!("getUpdates request failed: {}", e)))?;
if !response.status().is_success() { if !response.status().is_success() {
return Err(format!("getUpdates returned status {}", response.status())); return Err(ChannelSetupError::Network(format!(
"getUpdates returned status {}",
response.status()
)));
} }
let body: TelegramGetUpdatesResponse = response let body: TelegramGetUpdatesResponse = response.json().await.map_err(|e| {
.json() ChannelSetupError::Network(format!("Failed to parse getUpdates response: {}", e))
.await })?;
.map_err(|e| format!("Failed to parse getUpdates response: {}", e))?;
if !body.ok { if !body.ok {
return Err("Telegram API returned error for getUpdates".to_string()); return Err(ChannelSetupError::Network(
"Telegram API returned error for getUpdates".to_string(),
));
} }
// Find the first message with a sender // Find the first message with a sender
for update in &body.result { for update in &body.result {
if let Some(ref msg) = update.message { if let Some(ref msg) = update.message
if let Some(ref from) = msg.from { && let Some(ref from) = msg.from
{
let display_name = from let display_name = from
.username .username
.as_ref() .as_ref()
@@ -281,17 +314,19 @@ async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, String
"https://api.telegram.org/bot{}/getUpdates", "https://api.telegram.org/bot{}/getUpdates",
token.expose_secret() token.expose_secret()
); );
let _ = client if let Err(e) = client
.get(&ack_url) .get(&ack_url)
.query(&[("offset", &(update.update_id + 1).to_string())]) .query(&[("offset", &(update.update_id + 1).to_string())])
.send() .send()
.await; .await
{
tracing::warn!("Failed to acknowledge Telegram update: {e}");
}
return Ok(Some(from.id)); return Ok(Some(from.id));
} }
} }
} }
}
print_error("Timed out waiting for a message. You can re-run setup to try again."); print_error("Timed out waiting for a message. You can re-run setup to try again.");
print_info("Bot will accept messages from all users until owner is bound."); print_info("Bot will accept messages from all users until owner is bound.");
@@ -301,12 +336,13 @@ async fn bind_telegram_owner(token: &SecretString) -> Result<Option<i64>, String
/// Bind flow when the token already exists (reads from secrets store). /// Bind flow when the token already exists (reads from secrets store).
/// ///
/// Retrieves the saved bot token and delegates to `bind_telegram_owner`. /// Retrieves the saved bot token and delegates to `bind_telegram_owner`.
async fn bind_telegram_owner_flow(secrets: &SecretsContext) -> Result<Option<i64>, String> { async fn bind_telegram_owner_flow(
// Check current settings first secrets: &SecretsContext,
let settings = Settings::load(); settings: &Settings,
) -> Result<Option<i64>, ChannelSetupError> {
if settings.channels.telegram_owner_id.is_some() { if settings.channels.telegram_owner_id.is_some() {
print_info("Bot is already bound to a Telegram account."); print_info("Bot is already bound to a Telegram account.");
if !confirm("Re-bind to a different account?", false).map_err(|e| e.to_string())? { if !confirm("Re-bind to a different account?", false)? {
return Ok(settings.channels.telegram_owner_id); return Ok(settings.channels.telegram_owner_id);
} }
} }
@@ -321,12 +357,10 @@ async fn bind_telegram_owner_flow(secrets: &SecretsContext) -> Result<Option<i64
/// ///
/// This is shared across all channels that need webhook endpoints. /// This is shared across all channels that need webhook endpoints.
/// Returns the tunnel URL if configured. /// Returns the tunnel URL if configured.
pub fn setup_tunnel() -> Result<Option<String>, String> { pub fn setup_tunnel(settings: &Settings) -> Result<Option<String>, ChannelSetupError> {
// Check if already configured
let settings = Settings::load();
if let Some(ref url) = settings.tunnel.public_url { if let Some(ref url) = settings.tunnel.public_url {
print_info(&format!("Existing tunnel configured: {}", url)); print_info(&format!("Existing tunnel configured: {}", url));
if !confirm("Change tunnel configuration?", false).map_err(|e| e.to_string())? { if !confirm("Change tunnel configuration?", false)? {
return Ok(Some(url.clone())); return Ok(Some(url.clone()));
} }
} }
@@ -346,30 +380,24 @@ pub fn setup_tunnel() -> Result<Option<String>, String> {
print_info("Security comes from provider-specific secrets (e.g., Telegram webhook secret)."); print_info("Security comes from provider-specific secrets (e.g., Telegram webhook secret).");
println!(); println!();
if !confirm("Configure a tunnel?", false).map_err(|e| e.to_string())? { if !confirm("Configure a tunnel?", false)? {
return Ok(None); return Ok(None);
} }
let tunnel_url = let tunnel_url = input("Tunnel URL (e.g., https://abc123.ngrok.io)")?;
input("Tunnel URL (e.g., https://abc123.ngrok.io)").map_err(|e| e.to_string())?;
// Validate URL format // Validate URL format
if !tunnel_url.starts_with("https://") { if !tunnel_url.starts_with("https://") {
print_error("URL must start with https:// (webhooks require HTTPS)"); print_error("URL must start with https:// (webhooks require HTTPS)");
return Err("Invalid tunnel URL: must use HTTPS".to_string()); return Err(ChannelSetupError::Validation(
"Invalid tunnel URL: must use HTTPS".to_string(),
));
} }
// Remove trailing slash if present // Remove trailing slash if present
let tunnel_url = tunnel_url.trim_end_matches('/').to_string(); let tunnel_url = tunnel_url.trim_end_matches('/').to_string();
// Save to settings print_success(&format!("Tunnel URL configured: {}", tunnel_url));
let mut settings = Settings::load();
settings.tunnel.public_url = Some(tunnel_url.clone());
settings
.save()
.map_err(|e| format!("Failed to save settings: {}", e))?;
print_success(&format!("Tunnel URL saved: {}", tunnel_url));
print_info(""); print_info("");
print_info("Make sure your tunnel is running before starting the agent."); print_info("Make sure your tunnel is running before starting the agent.");
print_info("You can also set TUNNEL_URL environment variable to override."); print_info("You can also set TUNNEL_URL environment variable to override.");
@@ -380,10 +408,11 @@ pub fn setup_tunnel() -> Result<Option<String>, String> {
/// Set up Telegram webhook secret for signature validation. /// Set up Telegram webhook secret for signature validation.
/// ///
/// Returns the webhook secret if configured. /// Returns the webhook secret if configured.
async fn setup_telegram_webhook_secret(secrets: &SecretsContext) -> Result<Option<String>, String> { async fn setup_telegram_webhook_secret(
// Check if tunnel is configured secrets: &SecretsContext,
let settings = Settings::load(); tunnel: &TunnelSettings,
if settings.tunnel.public_url.is_none() { ) -> Result<Option<String>, ChannelSetupError> {
if tunnel.public_url.is_none() {
print_info(""); print_info("");
print_info("No tunnel configured. Telegram will use polling mode (30s+ delay)."); print_info("No tunnel configured. Telegram will use polling mode (30s+ delay).");
print_info("Run setup again to configure a tunnel for instant delivery."); print_info("Run setup again to configure a tunnel for instant delivery.");
@@ -395,7 +424,7 @@ async fn setup_telegram_webhook_secret(secrets: &SecretsContext) -> Result<Optio
print_info("A webhook secret adds an extra layer of security by validating"); print_info("A webhook secret adds an extra layer of security by validating");
print_info("that requests actually come from Telegram's servers."); print_info("that requests actually come from Telegram's servers.");
if !confirm("Generate a webhook secret?", true).map_err(|e| e.to_string())? { if !confirm("Generate a webhook secret?", true)? {
return Ok(None); return Ok(None);
} }
@@ -414,11 +443,13 @@ async fn setup_telegram_webhook_secret(secrets: &SecretsContext) -> Result<Optio
/// Validate a Telegram bot token by calling the getMe API. /// Validate a Telegram bot token by calling the getMe API.
/// ///
/// Returns the bot's username if valid. /// Returns the bot's username if valid.
pub async fn validate_telegram_token(token: &SecretString) -> Result<Option<String>, String> { pub async fn validate_telegram_token(
token: &SecretString,
) -> Result<Option<String>, ChannelSetupError> {
let client = Client::builder() let client = Client::builder()
.timeout(std::time::Duration::from_secs(10)) .timeout(std::time::Duration::from_secs(10))
.build() .build()
.map_err(|e| format!("Failed to create HTTP client: {}", e))?; .map_err(|e| ChannelSetupError::Network(format!("Failed to create HTTP client: {}", e)))?;
let url = format!( let url = format!(
"https://api.telegram.org/bot{}/getMe", "https://api.telegram.org/bot{}/getMe",
@@ -429,21 +460,26 @@ pub async fn validate_telegram_token(token: &SecretString) -> Result<Option<Stri
.get(&url) .get(&url)
.send() .send()
.await .await
.map_err(|e| format!("Request failed: {}", e))?; .map_err(|e| ChannelSetupError::Network(format!("Request failed: {}", e)))?;
if !response.status().is_success() { if !response.status().is_success() {
return Err(format!("API returned status {}", response.status())); return Err(ChannelSetupError::Network(format!(
"API returned status {}",
response.status()
)));
} }
let body: TelegramGetMeResponse = response let body: TelegramGetMeResponse = response
.json() .json()
.await .await
.map_err(|e| format!("Failed to parse response: {}", e))?; .map_err(|e| ChannelSetupError::Network(format!("Failed to parse response: {}", e)))?;
if body.ok { if body.ok {
Ok(body.result.and_then(|u| u.username)) Ok(body.result.and_then(|u| u.username))
} else { } else {
Err("Telegram API returned error".to_string()) Err(ChannelSetupError::Network(
"Telegram API returned error".to_string(),
))
} }
} }
@@ -456,38 +492,34 @@ pub struct HttpSetupResult {
} }
/// Set up HTTP webhook channel. /// Set up HTTP webhook channel.
pub async fn setup_http(secrets: &SecretsContext) -> Result<HttpSetupResult, String> { pub async fn setup_http(secrets: &SecretsContext) -> Result<HttpSetupResult, ChannelSetupError> {
println!("HTTP Webhook Setup:"); println!("HTTP Webhook Setup:");
println!(); println!();
print_info("The HTTP webhook allows external services to send messages to the agent."); print_info("The HTTP webhook allows external services to send messages to the agent.");
println!(); println!();
let port_str = optional_input("Port", Some("default: 8080")).map_err(|e| e.to_string())?; let port_str = optional_input("Port", Some("default: 8080"))?;
let port: u16 = port_str let port: u16 = port_str
.as_deref() .as_deref()
.unwrap_or("8080") .unwrap_or("8080")
.parse() .parse()
.map_err(|e| format!("Invalid port: {}", e))?; .map_err(|e| ChannelSetupError::Validation(format!("Invalid port: {}", e)))?;
if port < 1024 { if port < 1024 {
print_info("Note: Ports below 1024 may require root privileges"); print_info("Note: Ports below 1024 may require root privileges");
} }
let host = optional_input("Host", Some("default: 0.0.0.0")) let host =
.map_err(|e| e.to_string())? optional_input("Host", Some("default: 0.0.0.0"))?.unwrap_or_else(|| "0.0.0.0".to_string());
.unwrap_or_else(|| "0.0.0.0".to_string());
// Generate a webhook secret // Generate a webhook secret
if confirm("Generate a webhook secret for authentication?", true).map_err(|e| e.to_string())? { if confirm("Generate a webhook secret for authentication?", true)? {
let secret = generate_webhook_secret(); let secret = generate_webhook_secret();
secrets secrets
.save_secret("http_webhook_secret", &SecretString::from(secret.clone())) .save_secret("http_webhook_secret", &SecretString::from(secret))
.await?; .await?;
print_success("Webhook secret generated and saved to database"); print_success("Webhook secret generated and saved to database");
print_info(&format!( print_info("Retrieve it later with: ironclaw secret get http_webhook_secret");
"Secret: {} (store this for your webhook clients)",
secret
));
} }
print_success(&format!("HTTP webhook will listen on {}:{}", host, port)); print_success(&format!("HTTP webhook will listen on {}:{}", host, port));
@@ -501,11 +533,7 @@ pub async fn setup_http(secrets: &SecretsContext) -> Result<HttpSetupResult, Str
/// Generate a random webhook secret. /// Generate a random webhook secret.
pub fn generate_webhook_secret() -> String { pub fn generate_webhook_secret() -> String {
use rand::RngCore; generate_secret_with_length(32)
let mut rng = rand::thread_rng();
let mut bytes = [0u8; 32];
rng.fill_bytes(&mut bytes);
bytes.iter().map(|b| format!("{:02x}", b)).collect()
} }
/// Result of WASM channel setup. /// Result of WASM channel setup.
@@ -523,7 +551,7 @@ pub async fn setup_wasm_channel(
secrets: &SecretsContext, secrets: &SecretsContext,
channel_name: &str, channel_name: &str,
setup: &crate::channels::wasm::SetupSchema, setup: &crate::channels::wasm::SetupSchema,
) -> Result<WasmChannelSetupResult, String> { ) -> Result<WasmChannelSetupResult, ChannelSetupError> {
println!("{} Setup:", channel_name); println!("{} Setup:", channel_name);
println!(); println!();
@@ -534,7 +562,7 @@ pub async fn setup_wasm_channel(
"Existing {} found in database.", "Existing {} found in database.",
secret_config.name secret_config.name
)); ));
if !confirm("Replace existing value?", false).map_err(|e| e.to_string())? { if !confirm("Replace existing value?", false)? {
continue; continue;
} }
} }
@@ -542,8 +570,7 @@ pub async fn setup_wasm_channel(
// Get the value from user or auto-generate // Get the value from user or auto-generate
let value = if secret_config.optional { let value = if secret_config.optional {
let input_value = let input_value =
optional_input(&secret_config.prompt, Some("leave empty to auto-generate")) optional_input(&secret_config.prompt, Some("leave empty to auto-generate"))?;
.map_err(|e| e.to_string())?;
if let Some(v) = input_value { if let Some(v) = input_value {
if !v.is_empty() { if !v.is_empty() {
@@ -570,18 +597,21 @@ pub async fn setup_wasm_channel(
} }
} else { } else {
// Required secret // Required secret
let input_value = secret_input(&secret_config.prompt).map_err(|e| e.to_string())?; let input_value = secret_input(&secret_config.prompt)?;
// Validate if pattern is provided // Validate if pattern is provided
if let Some(ref pattern) = secret_config.validation { if let Some(ref pattern) = secret_config.validation {
let re = regex::Regex::new(pattern) let re = regex::Regex::new(pattern).map_err(|e| {
.map_err(|e| format!("Invalid validation pattern: {}", e))?; ChannelSetupError::Validation(format!("Invalid validation pattern: {}", e))
})?;
if !re.is_match(input_value.expose_secret()) { if !re.is_match(input_value.expose_secret()) {
print_error(&format!( print_error(&format!(
"Value does not match expected format: {}", "Value does not match expected format: {}",
pattern pattern
)); ));
return Err("Validation failed".to_string()); return Err(ChannelSetupError::Validation(
"Validation failed".to_string(),
));
} }
} }
@@ -593,14 +623,11 @@ pub async fn setup_wasm_channel(
print_success(&format!("{} saved to database", secret_config.name)); print_success(&format!("{} saved to database", secret_config.name));
} }
// Optionally validate the configuration // TODO: Substitute secrets into the validation URL and make a
// GET request to verify the configured credentials actually work.
if let Some(ref validation_endpoint) = setup.validation_endpoint { if let Some(ref validation_endpoint) = setup.validation_endpoint {
print_info("Validating configuration...");
// The validation endpoint may contain placeholders like {telegram_bot_token}
// For now, we skip validation since we'd need to substitute secrets
// A full implementation would fetch secrets and substitute them
print_info(&format!( print_info(&format!(
"Validation endpoint configured: {} (validation skipped)", "Validation endpoint configured: {} (validation not yet implemented)",
validation_endpoint validation_endpoint
)); ));
} }
@@ -624,11 +651,23 @@ fn generate_secret_with_length(length: usize) -> String {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use crate::setup::channels::generate_webhook_secret;
#[test] #[test]
fn test_generate_webhook_secret() { fn test_generate_webhook_secret() {
let secret = generate_webhook_secret(); let secret = generate_webhook_secret();
assert_eq!(secret.len(), 64); // 32 bytes = 64 hex chars assert_eq!(secret.len(), 64); // 32 bytes = 64 hex chars
} }
#[test]
fn test_generate_secret_with_length() {
use super::generate_secret_with_length;
let s = generate_secret_with_length(16);
assert_eq!(s.len(), 32); // 16 bytes = 32 hex chars
assert!(s.chars().all(|c| c.is_ascii_hexdigit()));
let s2 = generate_secret_with_length(1);
assert_eq!(s2.len(), 2);
}
} }
+3 -2
View File
@@ -3,7 +3,7 @@
//! Provides a guided setup experience for: //! Provides a guided setup experience for:
//! 1. Database connection //! 1. Database connection
//! 2. Security (secrets master key) //! 2. Security (secrets master key)
//! 3. NEAR AI authentication //! 3. Inference provider selection
//! 4. Model selection //! 4. Model selection
//! 5. Embeddings //! 5. Embeddings
//! 6. Channel configuration (HTTP, Telegram, etc.) //! 6. Channel configuration (HTTP, Telegram, etc.)
@@ -24,7 +24,8 @@ mod prompts;
mod wizard; mod wizard;
pub use channels::{ pub use channels::{
SecretsContext, setup_http, setup_telegram, setup_tunnel, validate_telegram_token, ChannelSetupError, SecretsContext, setup_http, setup_telegram, setup_tunnel,
validate_telegram_token,
}; };
pub use prompts::{ pub use prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step, confirm, input, optional_input, print_error, print_header, print_info, print_step,
+9 -3
View File
@@ -21,6 +21,7 @@ use secrecy::SecretString;
/// Display a numbered menu and get user selection. /// Display a numbered menu and get user selection.
/// ///
/// Returns the index (0-based) of the selected option. /// Returns the index (0-based) of the selected option.
/// Pressing Enter without input selects the first option (index 0).
/// ///
/// # Example /// # Example
/// ///
@@ -54,11 +55,12 @@ pub fn select_one(prompt: &str, options: &[&str]) -> io::Result<usize> {
} }
// Parse number // Parse number
if let Ok(num) = input.parse::<usize>() { if let Ok(num) = input.parse::<usize>()
if num >= 1 && num <= options.len() { && num >= 1
&& num <= options.len()
{
return Ok(num - 1); return Ok(num - 1);
} }
}
writeln!( writeln!(
stdout, stdout,
@@ -83,6 +85,10 @@ pub fn select_one(prompt: &str, options: &[&str]) -> io::Result<usize> {
/// ])?; /// ])?;
/// ``` /// ```
pub fn select_many(prompt: &str, options: &[(&str, bool)]) -> io::Result<Vec<usize>> { pub fn select_many(prompt: &str, options: &[(&str, bool)]) -> io::Result<Vec<usize>> {
if options.is_empty() {
return Ok(vec![]);
}
let mut stdout = io::stdout(); let mut stdout = io::stdout();
let mut selected: Vec<bool> = options.iter().map(|(_, s)| *s).collect(); let mut selected: Vec<bool> = options.iter().map(|(_, s)| *s).collect();
let mut cursor_pos = 0; let mut cursor_pos = 0;
+921 -107
View File
File diff suppressed because it is too large Load Diff
+6 -6
View File
@@ -326,8 +326,9 @@ impl TestHarness {
} }
// Verify expected output // Verify expected output
if let Some(ref expected) = test.expected_output { if let Some(ref expected) = test.expected_output
if &actual != expected { && &actual != expected
{
return TestResult { return TestResult {
name: test.name.clone(), name: test.name.clone(),
passed: false, passed: false,
@@ -340,7 +341,6 @@ impl TestHarness {
actual_output: Some(actual), actual_output: Some(actual),
}; };
} }
}
// Verify expected fields // Verify expected fields
if let Some(ref fields) = test.expected_fields { if let Some(ref fields) = test.expected_fields {
@@ -357,8 +357,9 @@ impl TestHarness {
}; };
} }
if let Some(ref expected_value) = field.value { if let Some(ref expected_value) = field.value
if field_value != Some(expected_value) { && field_value != Some(expected_value)
{
return TestResult { return TestResult {
name: test.name.clone(), name: test.name.clone(),
passed: false, passed: false,
@@ -372,7 +373,6 @@ impl TestHarness {
} }
} }
} }
}
TestResult { TestResult {
name: test.name.clone(), name: test.name.clone(),
+3 -3
View File
@@ -54,13 +54,13 @@ fn validate_url(url: &str) -> Result<reqwest::Url, ToolError> {
} }
// Check literal IP addresses // Check literal IP addresses
if let Ok(ip) = host.parse::<IpAddr>() { if let Ok(ip) = host.parse::<IpAddr>()
if is_disallowed_ip(&ip) { && is_disallowed_ip(&ip)
{
return Err(ToolError::NotAuthorized( return Err(ToolError::NotAuthorized(
"private or local IPs are not allowed".to_string(), "private or local IPs are not allowed".to_string(),
)); ));
} }
}
// Resolve hostname and check all resolved IPs against the blocklist. // Resolve hostname and check all resolved IPs against the blocklist.
// This prevents DNS rebinding where a hostname resolves to a private IP. // This prevents DNS rebinding where a hostname resolves to a private IP.
+3 -3
View File
@@ -158,8 +158,9 @@ impl CreateJobTool {
}); });
// Persist the job mode to DB // Persist the job mode to DB
if mode == JobMode::ClaudeCode { if mode == JobMode::ClaudeCode
if let Some(store) = self.store.clone() { && let Some(store) = self.store.clone()
{
let job_id_copy = job_id; let job_id_copy = job_id;
tokio::spawn(async move { tokio::spawn(async move {
if let Err(e) = store if let Err(e) = store
@@ -170,7 +171,6 @@ impl CreateJobTool {
} }
}); });
} }
}
// Create the container job with the pre-determined job_id. // Create the container job with the pre-determined job_id.
let _token = jm let _token = jm
+45 -3
View File
@@ -343,13 +343,13 @@ impl ShellTool {
// Use sandbox if configured; fail-closed (never silently fall through // Use sandbox if configured; fail-closed (never silently fall through
// to unsandboxed execution when sandbox was intended). // to unsandboxed execution when sandbox was intended).
if let Some(ref sandbox) = self.sandbox { if let Some(ref sandbox) = self.sandbox
if sandbox.is_initialized() || sandbox.config().enabled { && (sandbox.is_initialized() || sandbox.config().enabled)
{
return self return self
.execute_sandboxed(sandbox, cmd, &cwd, timeout_duration) .execute_sandboxed(sandbox, cmd, &cwd, timeout_duration)
.await; .await;
} }
}
// Only execute directly when no sandbox was configured at all. // Only execute directly when no sandbox was configured at all.
let (output, code) = self.execute_direct(cmd, &cwd, timeout_duration).await?; let (output, code) = self.execute_direct(cmd, &cwd, timeout_duration).await?;
@@ -527,6 +527,48 @@ mod tests {
)); ));
} }
/// Replicate the extraction logic from agent_loop.rs to prove it works
/// when `arguments` is a `serde_json::Value::Object` (the common case
/// that was previously broken because `Value::Object.as_str()` returns None).
#[test]
fn test_destructive_command_extraction_from_object_args() {
let arguments = serde_json::json!({"command": "rm -rf /tmp/stuff"});
let cmd = arguments
.get("command")
.and_then(|c| c.as_str().map(String::from))
.or_else(|| {
arguments
.as_str()
.and_then(|s| serde_json::from_str::<serde_json::Value>(s).ok())
.and_then(|v| v.get("command").and_then(|c| c.as_str().map(String::from)))
});
assert_eq!(cmd.as_deref(), Some("rm -rf /tmp/stuff"));
assert!(requires_explicit_approval(cmd.as_deref().unwrap()));
}
/// Verify extraction still works when `arguments` is a JSON string
/// (rare, but possible if the LLM provider returns string-encoded JSON).
#[test]
fn test_destructive_command_extraction_from_string_args() {
let arguments =
serde_json::Value::String(r#"{"command": "git push --force origin main"}"#.to_string());
let cmd = arguments
.get("command")
.and_then(|c| c.as_str().map(String::from))
.or_else(|| {
arguments
.as_str()
.and_then(|s| serde_json::from_str::<serde_json::Value>(s).ok())
.and_then(|v| v.get("command").and_then(|c| c.as_str().map(String::from)))
});
assert_eq!(cmd.as_deref(), Some("git push --force origin main"));
assert!(requires_explicit_approval(cmd.as_deref().unwrap()));
}
#[test] #[test]
fn test_sandbox_policy_builder() { fn test_sandbox_policy_builder() {
let tool = ShellTool::new() let tool = ShellTool::new()
+13 -70
View File
@@ -11,9 +11,9 @@ use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use rand::RngCore; use rand::RngCore;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::TcpListener; use tokio::net::TcpListener;
use crate::cli::oauth_defaults::{self, OAUTH_CALLBACK_PORT};
use crate::secrets::{CreateSecretParams, SecretsStore}; use crate::secrets::{CreateSecretParams, SecretsStore};
use crate::tools::mcp::config::McpServerConfig; use crate::tools::mcp::config::McpServerConfig;
@@ -466,14 +466,12 @@ pub async fn authorize_mcp_server(
Ok(token) Ok(token)
} }
/// Find an available port for the OAuth callback. /// Bind the OAuth callback listener on the shared fixed port.
pub async fn find_available_port() -> Result<(TcpListener, u16), AuthError> { pub async fn find_available_port() -> Result<(TcpListener, u16), AuthError> {
for port in 9876..=9886 { let listener = oauth_defaults::bind_callback_listener()
if let Ok(listener) = TcpListener::bind(format!("127.0.0.1:{}", port)).await { .await
return Ok((listener, port)); .map_err(|_| AuthError::PortUnavailable)?;
} Ok((listener, OAUTH_CALLBACK_PORT))
}
Err(AuthError::PortUnavailable)
} }
/// Build the authorization URL with all required parameters. /// Build the authorization URL with all required parameters.
@@ -522,71 +520,16 @@ pub async fn wait_for_authorization_callback(
listener: TcpListener, listener: TcpListener,
server_name: &str, server_name: &str,
) -> Result<String, AuthError> { ) -> Result<String, AuthError> {
let timeout = Duration::from_secs(300); oauth_defaults::wait_for_callback(listener, "/callback", "code", server_name)
tokio::time::timeout(timeout, async {
loop {
let (mut socket, _) = listener
.accept()
.await .await
.map_err(|e| AuthError::Http(e.to_string()))?; .map_err(|e| match e {
oauth_defaults::OAuthCallbackError::Denied => AuthError::AuthorizationDenied,
let mut reader = BufReader::new(&mut socket); oauth_defaults::OAuthCallbackError::Timeout => AuthError::Timeout,
let mut request_line = String::new(); oauth_defaults::OAuthCallbackError::PortInUse(_, msg) => {
reader AuthError::Http(format!("Port error: {}", msg))
.read_line(&mut request_line)
.await
.map_err(|e| AuthError::Http(e.to_string()))?;
// Parse GET /callback?code=xxx HTTP/1.1
if let Some(path) = request_line.split_whitespace().nth(1) {
if path.starts_with("/callback") {
if let Some(query) = path.split('?').nth(1) {
// Check for error first
if query.contains("error=") {
let response = "HTTP/1.1 400 Bad Request\r\n\r\nAuthorization denied";
let _ = socket.write_all(response.as_bytes()).await;
return Err(AuthError::AuthorizationDenied);
}
// Look for code
for param in query.split('&') {
let parts: Vec<&str> = param.splitn(2, '=').collect();
if parts.len() == 2 && parts[0] == "code" {
let code = urlencoding::decode(parts[1])
.unwrap_or_else(|_| parts[1].into())
.into_owned();
// Send success response
let response = format!(
"HTTP/1.1 200 OK\r\n\
Content-Type: text/html\r\n\
\r\n\
<!DOCTYPE html><html><body style=\"font-family: sans-serif; \
display: flex; justify-content: center; align-items: center; \
height: 100vh; margin: 0; background: #191919; color: white;\">\
<div style=\"text-align: center;\">\
<h1> {} Connected!</h1>\
<p>You can close this window.</p>\
</div></body></html>",
server_name
);
let _ = socket.write_all(response.as_bytes()).await;
let _ = socket.shutdown().await;
return Ok(code);
}
}
}
}
}
let response = "HTTP/1.1 404 Not Found\r\n\r\n";
let _ = socket.write_all(response.as_bytes()).await;
} }
oauth_defaults::OAuthCallbackError::Io(msg) => AuthError::Http(msg),
}) })
.await
.map_err(|_| AuthError::Timeout)?
} }
/// Exchange the authorization code for an access token. /// Exchange the authorization code for an access token.
+21 -20
View File
@@ -184,33 +184,36 @@ impl McpClient {
} }
// Add Mcp-Session-Id header if we have a session // Add Mcp-Session-Id header if we have a session
if let Some(ref session_manager) = self.session_manager { if let Some(ref session_manager) = self.session_manager
if let Some(session_id) = session_manager.get_session_id(&self.server_name).await { && let Some(session_id) = session_manager.get_session_id(&self.server_name).await
{
req_builder = req_builder.header("Mcp-Session-Id", session_id); req_builder = req_builder.header("Mcp-Session-Id", session_id);
} }
}
let response = req_builder let response = req_builder.send().await.map_err(|e| {
.send() let mut chain = format!("MCP request failed: {}", e);
.await let mut source = std::error::Error::source(&e);
.map_err(|e| ToolError::ExternalService(format!("MCP request failed: {}", e)))?; while let Some(cause) = source {
chain.push_str(&format!(" -> {}", cause));
source = cause.source();
}
ToolError::ExternalService(chain)
})?;
// Check for 401 Unauthorized - try to refresh token on first attempt // Check for 401 Unauthorized - try to refresh token on first attempt
if response.status() == reqwest::StatusCode::UNAUTHORIZED { if response.status() == reqwest::StatusCode::UNAUTHORIZED {
if attempt == 0 { if attempt == 0 {
// Try to refresh the token // Try to refresh the token
if let Some(ref secrets) = self.secrets { if let Some(ref secrets) = self.secrets
if let Some(ref config) = self.server_config { && let Some(ref config) = self.server_config
{
tracing::debug!( tracing::debug!(
"MCP token expired, attempting refresh for '{}'", "MCP token expired, attempting refresh for '{}'",
self.server_name self.server_name
); );
match refresh_access_token(config, secrets, &self.user_id).await { match refresh_access_token(config, secrets, &self.user_id).await {
Ok(_) => { Ok(_) => {
tracing::info!( tracing::info!("MCP token refreshed for '{}'", self.server_name);
"MCP token refreshed for '{}'",
self.server_name
);
// Continue to next iteration to retry with new token // Continue to next iteration to retry with new token
continue; continue;
} }
@@ -225,7 +228,6 @@ impl McpClient {
} }
} }
} }
}
return Err(ToolError::ExternalService(format!( return Err(ToolError::ExternalService(format!(
"MCP server '{}' requires authentication. Run: ironclaw mcp auth {}", "MCP server '{}' requires authentication. Run: ironclaw mcp auth {}",
self.server_name, self.server_name self.server_name, self.server_name
@@ -245,8 +247,8 @@ impl McpClient {
/// Parse the HTTP response into an MCP response. /// Parse the HTTP response into an MCP response.
async fn parse_response(&self, response: reqwest::Response) -> Result<McpResponse, ToolError> { async fn parse_response(&self, response: reqwest::Response) -> Result<McpResponse, ToolError> {
// Extract session ID from response header // Extract session ID from response header
if let Some(ref session_manager) = self.session_manager { if let Some(ref session_manager) = self.session_manager
if let Some(session_id) = response && let Some(session_id) = response
.headers() .headers()
.get("Mcp-Session-Id") .get("Mcp-Session-Id")
.and_then(|v| v.to_str().ok()) .and_then(|v| v.to_str().ok())
@@ -255,7 +257,6 @@ impl McpClient {
.update_session_id(&self.server_name, Some(session_id.to_string())) .update_session_id(&self.server_name, Some(session_id.to_string()))
.await; .await;
} }
}
if !response.status().is_success() { if !response.status().is_success() {
let status = response.status(); let status = response.status();
@@ -316,12 +317,12 @@ impl McpClient {
/// This should be called once per session to establish capabilities. /// This should be called once per session to establish capabilities.
pub async fn initialize(&self) -> Result<InitializeResult, ToolError> { pub async fn initialize(&self) -> Result<InitializeResult, ToolError> {
// Check if already initialized // Check if already initialized
if let Some(ref session_manager) = self.session_manager { if let Some(ref session_manager) = self.session_manager
if session_manager.is_initialized(&self.server_name).await { && session_manager.is_initialized(&self.server_name).await
{
// Return cached/default capabilities // Return cached/default capabilities
return Ok(InitializeResult::default()); return Ok(InitializeResult::default());
} }
}
// Ensure we have a session // Ensure we have a session
if let Some(ref session_manager) = self.session_manager { if let Some(ref session_manager) = self.session_manager {
+82 -1
View File
@@ -88,8 +88,18 @@ impl McpServerConfig {
} }
/// Check if this server requires authentication. /// Check if this server requires authentication.
///
/// Returns true if OAuth is pre-configured OR if this is a remote HTTPS server
/// (which likely supports Dynamic Client Registration even without pre-configured OAuth).
pub fn requires_auth(&self) -> bool { pub fn requires_auth(&self) -> bool {
self.oauth.is_some() if self.oauth.is_some() {
return true;
}
// Remote HTTPS servers need auth handling (DCR, token refresh, 401 detection).
// Localhost/127.0.0.1 servers are assumed to be dev servers without auth.
let url_lower = self.url.to_lowercase();
let is_localhost = is_localhost_url(&url_lower);
url_lower.starts_with("https://") && !is_localhost
} }
/// Get the secret name used to store the access token. /// Get the secret name used to store the access token.
@@ -402,11 +412,43 @@ pub async fn remove_mcp_server_db(
Ok(()) Ok(())
} }
/// Check if a URL points to a loopback address (localhost, 127.0.0.1, [::1]).
///
/// Uses `url::Url` for proper parsing so edge cases (IPv6, userinfo, ports)
/// are handled correctly without manual string splitting.
fn is_localhost_url(url: &str) -> bool {
let Ok(parsed) = url::Url::parse(url) else {
return false;
};
match parsed.host() {
Some(url::Host::Domain(d)) => d.eq_ignore_ascii_case("localhost"),
Some(url::Host::Ipv4(ip)) => ip.is_loopback(),
Some(url::Host::Ipv6(ip)) => ip.is_loopback(),
None => false,
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::tempdir; use tempfile::tempdir;
#[test]
fn test_is_localhost_url() {
assert!(is_localhost_url("http://localhost:3000/path"));
assert!(is_localhost_url("https://localhost/path"));
assert!(is_localhost_url("http://127.0.0.1:8080"));
assert!(is_localhost_url("http://127.0.0.1"));
assert!(!is_localhost_url("https://notlocalhost.com/path"));
assert!(!is_localhost_url("https://example-localhost.io"));
assert!(!is_localhost_url("https://mcp.notion.com"));
assert!(is_localhost_url("http://user:pass@localhost:3000/path"));
// IPv6 loopback
assert!(is_localhost_url("http://[::1]:8080/path"));
assert!(is_localhost_url("http://[::1]/path"));
assert!(!is_localhost_url("http://[::2]:8080/path"));
}
#[test] #[test]
fn test_server_config_validation() { fn test_server_config_validation() {
// Valid HTTPS server // Valid HTTPS server
@@ -514,4 +556,43 @@ mod tests {
"mcp_notion_refresh_token" "mcp_notion_refresh_token"
); );
} }
#[test]
fn test_requires_auth_with_oauth() {
let config = McpServerConfig::new("notion", "https://mcp.notion.com")
.with_oauth(OAuthConfig::new("client-123"));
assert!(config.requires_auth());
}
#[test]
fn test_requires_auth_remote_https_without_oauth() {
// Remote HTTPS servers need auth even without pre-configured OAuth (DCR)
let config = McpServerConfig::new("github-copilot", "https://api.githubcopilot.com/mcp/");
assert!(config.requires_auth());
let config = McpServerConfig::new("notion", "https://mcp.notion.com");
assert!(config.requires_auth());
}
#[test]
fn test_requires_auth_localhost_no_auth() {
// Localhost servers are dev servers, no auth needed
let config = McpServerConfig::new("local", "http://localhost:8080");
assert!(!config.requires_auth());
let config = McpServerConfig::new("local", "http://127.0.0.1:3000/mcp");
assert!(!config.requires_auth());
// Even HTTPS localhost doesn't require auth
let config = McpServerConfig::new("local", "https://localhost:8443");
assert!(!config.requires_auth());
}
#[test]
fn test_requires_auth_http_remote_no_auth() {
// HTTP remote servers won't pass validation, but if they existed
// they wouldn't trigger HTTPS auth detection
let config = McpServerConfig::new("bad", "http://mcp.example.com");
assert!(!config.requires_auth());
}
} }
+18 -5
View File
@@ -11,6 +11,7 @@ use crate::extensions::ExtensionManager;
use crate::llm::{LlmProvider, ToolDefinition}; use crate::llm::{LlmProvider, ToolDefinition};
use crate::orchestrator::job_manager::ContainerJobManager; use crate::orchestrator::job_manager::ContainerJobManager;
use crate::safety::SafetyLayer; use crate::safety::SafetyLayer;
use crate::secrets::SecretsStore;
use crate::tools::builder::{BuildSoftwareTool, BuilderConfig, LlmSoftwareBuilder}; use crate::tools::builder::{BuildSoftwareTool, BuilderConfig, LlmSoftwareBuilder};
use crate::tools::builtin::{ use crate::tools::builtin::{
ApplyPatchTool, CancelJobTool, CreateJobTool, EchoTool, HttpTool, JobStatusTool, JsonTool, ApplyPatchTool, CancelJobTool, CreateJobTool, EchoTool, HttpTool, JobStatusTool, JsonTool,
@@ -20,8 +21,8 @@ use crate::tools::builtin::{
}; };
use crate::tools::tool::{Tool, ToolDomain}; use crate::tools::tool::{Tool, ToolDomain};
use crate::tools::wasm::{ use crate::tools::wasm::{
Capabilities, ResourceLimits, WasmError, WasmStorageError, WasmToolRuntime, WasmToolStore, Capabilities, OAuthRefreshConfig, ResourceLimits, WasmError, WasmStorageError, WasmToolRuntime,
WasmToolWrapper, WasmToolStore, WasmToolWrapper,
}; };
use crate::workspace::Workspace; use crate::workspace::Workspace;
@@ -96,11 +97,11 @@ impl ToolRegistry {
if let Ok(mut tools) = self.tools.try_write() { if let Ok(mut tools) = self.tools.try_write() {
tools.insert(name.clone(), tool); tools.insert(name.clone(), tool);
// Mark as built-in so it can't be shadowed later // Mark as built-in so it can't be shadowed later
if PROTECTED_TOOL_NAMES.contains(&name.as_str()) { if PROTECTED_TOOL_NAMES.contains(&name.as_str())
if let Ok(mut builtins) = self.builtin_names.try_write() { && let Ok(mut builtins) = self.builtin_names.try_write()
{
builtins.insert(name.clone()); builtins.insert(name.clone());
} }
}
tracing::debug!("Registered tool: {}", name); tracing::debug!("Registered tool: {}", name);
} }
} }
@@ -366,6 +367,12 @@ impl ToolRegistry {
if let Some(s) = reg.schema { if let Some(s) = reg.schema {
wrapper = wrapper.with_schema(s); wrapper = wrapper.with_schema(s);
} }
if let Some(store) = reg.secrets_store {
wrapper = wrapper.with_secrets_store(store);
}
if let Some(oauth) = reg.oauth_refresh {
wrapper = wrapper.with_oauth_refresh(oauth);
}
// Register the tool // Register the tool
self.register(Arc::new(wrapper)).await; self.register(Arc::new(wrapper)).await;
@@ -421,6 +428,8 @@ impl ToolRegistry {
limits: None, limits: None,
description: Some(&tool_with_binary.tool.description), description: Some(&tool_with_binary.tool.description),
schema: Some(tool_with_binary.tool.parameters_schema.clone()), schema: Some(tool_with_binary.tool.parameters_schema.clone()),
secrets_store: None,
oauth_refresh: None,
}) })
.await .await
.map_err(WasmRegistrationError::Wasm)?; .map_err(WasmRegistrationError::Wasm)?;
@@ -462,6 +471,10 @@ pub struct WasmToolRegistration<'a> {
pub description: Option<&'a str>, pub description: Option<&'a str>,
/// Optional parameter schema override. /// Optional parameter schema override.
pub schema: Option<serde_json::Value>, pub schema: Option<serde_json::Value>,
/// Secrets store for credential injection at request time.
pub secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
/// OAuth refresh configuration for auto-refreshing expired tokens.
pub oauth_refresh: Option<OAuthRefreshConfig>,
} }
impl Default for ToolRegistry { impl Default for ToolRegistry {
+10 -9
View File
@@ -209,11 +209,11 @@ impl EndpointPattern {
} }
// Check path prefix // Check path prefix
if let Some(ref prefix) = self.path_prefix { if let Some(ref prefix) = self.path_prefix
if !url_path.starts_with(prefix) { && !url_path.starts_with(prefix)
{
return false; return false;
} }
}
// Check method // Check method
if !self.methods.is_empty() { if !self.methods.is_empty() {
@@ -237,15 +237,16 @@ impl EndpointPattern {
} }
// Support wildcard: *.example.com matches sub.example.com // Support wildcard: *.example.com matches sub.example.com
if let Some(suffix) = self.host.strip_prefix("*.") { if let Some(suffix) = self.host.strip_prefix("*.")
if url_host.ends_with(suffix) && url_host.len() > suffix.len() { && url_host.ends_with(suffix)
&& url_host.len() > suffix.len()
{
// Ensure there's a dot before the suffix (or it's the whole thing) // Ensure there's a dot before the suffix (or it's the whole thing)
let prefix = &url_host[..url_host.len() - suffix.len()]; let prefix = &url_host[..url_host.len() - suffix.len()];
if prefix.ends_with('.') || prefix.is_empty() { if prefix.ends_with('.') || prefix.is_empty() {
return true; return true;
} }
} }
}
false false
} }
@@ -291,12 +292,12 @@ impl SecretsCapability {
if pattern == name { if pattern == name {
return true; return true;
} }
if let Some(prefix) = pattern.strip_suffix('*') { if let Some(prefix) = pattern.strip_suffix('*')
if name.starts_with(prefix) { && name.starts_with(prefix)
{
return true; return true;
} }
} }
}
false false
} }
} }
+9 -8
View File
@@ -158,18 +158,18 @@ impl CredentialInjector {
if pattern == name { if pattern == name {
return true; return true;
} }
if let Some(prefix) = pattern.strip_suffix('*') { if let Some(prefix) = pattern.strip_suffix('*')
if name.starts_with(prefix) { && name.starts_with(prefix)
{
return true; return true;
} }
} }
}
false false
} }
} }
/// Inject a single credential into the result. /// Inject a single credential into the result.
fn inject_credential( pub(crate) fn inject_credential(
result: &mut InjectedCredentials, result: &mut InjectedCredentials,
location: &CredentialLocation, location: &CredentialLocation,
secret: &DecryptedSecret, secret: &DecryptedSecret,
@@ -208,20 +208,21 @@ fn inject_credential(
} }
/// Check if a host matches a pattern (supports wildcards). /// Check if a host matches a pattern (supports wildcards).
fn host_matches_pattern(host: &str, pattern: &str) -> bool { pub(crate) fn host_matches_pattern(host: &str, pattern: &str) -> bool {
if pattern == host { if pattern == host {
return true; return true;
} }
// Support wildcard: *.example.com matches sub.example.com // Support wildcard: *.example.com matches sub.example.com
if let Some(suffix) = pattern.strip_prefix("*.") { if let Some(suffix) = pattern.strip_prefix("*.")
if host.ends_with(suffix) && host.len() > suffix.len() { && host.ends_with(suffix)
&& host.len() > suffix.len()
{
let prefix = &host[..host.len() - suffix.len()]; let prefix = &host[..host.len() - suffix.len()];
if prefix.ends_with('.') || prefix.is_empty() { if prefix.ends_with('.') || prefix.is_empty() {
return true; return true;
} }
} }
}
false false
} }
+179 -7
View File
@@ -39,10 +39,11 @@ use std::sync::Arc;
use tokio::fs; use tokio::fs;
use crate::secrets::SecretsStore;
use crate::tools::registry::{ToolRegistry, WasmRegistrationError, WasmToolRegistration}; use crate::tools::registry::{ToolRegistry, WasmRegistrationError, WasmToolRegistration};
use crate::tools::wasm::capabilities_schema::CapabilitiesFile; use crate::tools::wasm::capabilities_schema::CapabilitiesFile;
use crate::tools::wasm::{ use crate::tools::wasm::{
Capabilities, WasmError, WasmStorageError, WasmToolRuntime, WasmToolStore, Capabilities, OAuthRefreshConfig, WasmError, WasmStorageError, WasmToolRuntime, WasmToolStore,
}; };
/// Error during WASM tool loading. /// Error during WASM tool loading.
@@ -77,12 +78,23 @@ pub enum WasmLoadError {
pub struct WasmToolLoader { pub struct WasmToolLoader {
runtime: Arc<WasmToolRuntime>, runtime: Arc<WasmToolRuntime>,
registry: Arc<ToolRegistry>, registry: Arc<ToolRegistry>,
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
} }
impl WasmToolLoader { impl WasmToolLoader {
/// Create a new loader with the given runtime and registry. /// Create a new loader with the given runtime and registry.
pub fn new(runtime: Arc<WasmToolRuntime>, registry: Arc<ToolRegistry>) -> Self { pub fn new(runtime: Arc<WasmToolRuntime>, registry: Arc<ToolRegistry>) -> Self {
Self { runtime, registry } Self {
runtime,
registry,
secrets_store: None,
}
}
/// Set the secrets store for credential injection in WASM tools.
pub fn with_secrets_store(mut self, store: Arc<dyn SecretsStore + Send + Sync>) -> Self {
self.secrets_store = Some(store);
self
} }
/// Load a single WASM tool from a file pair. /// Load a single WASM tool from a file pair.
@@ -108,22 +120,24 @@ impl WasmToolLoader {
} }
let wasm_bytes = fs::read(wasm_path).await?; let wasm_bytes = fs::read(wasm_path).await?;
// Read capabilities (optional) // Read capabilities (optional) and extract OAuth refresh config
let capabilities = if let Some(cap_path) = capabilities_path { let (capabilities, oauth_refresh) = if let Some(cap_path) = capabilities_path {
if cap_path.exists() { if cap_path.exists() {
let cap_bytes = fs::read(cap_path).await?; let cap_bytes = fs::read(cap_path).await?;
let cap_file = CapabilitiesFile::from_bytes(&cap_bytes) let cap_file = CapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| WasmLoadError::InvalidCapabilities(e.to_string()))?; .map_err(|e| WasmLoadError::InvalidCapabilities(e.to_string()))?;
cap_file.to_capabilities() let caps = cap_file.to_capabilities();
let oauth = resolve_oauth_refresh_config(&cap_file);
(caps, oauth)
} else { } else {
tracing::warn!( tracing::warn!(
path = %cap_path.display(), path = %cap_path.display(),
"Capabilities file not found, using default (no permissions)" "Capabilities file not found, using default (no permissions)"
); );
Capabilities::default() (Capabilities::default(), None)
} }
} else { } else {
Capabilities::default() (Capabilities::default(), None)
}; };
// Register the tool // Register the tool
@@ -136,6 +150,8 @@ impl WasmToolLoader {
limits: None, limits: None,
description: None, description: None,
schema: None, schema: None,
secrets_store: self.secrets_store.clone(),
oauth_refresh,
}) })
.await?; .await?;
@@ -293,6 +309,50 @@ impl WasmToolLoader {
} }
} }
/// Extract OAuth refresh configuration from a parsed capabilities file.
///
/// Returns `None` if there's no `auth.oauth` section or if the client_id
/// can't be resolved from any source (inline, env var, or built-in defaults).
///
/// Fallback chain for client_id:
/// `oauth.client_id` > env var (`oauth.client_id_env`) > `builtin_credentials()`
fn resolve_oauth_refresh_config(cap_file: &CapabilitiesFile) -> Option<OAuthRefreshConfig> {
let auth = cap_file.auth.as_ref()?;
let oauth = auth.oauth.as_ref()?;
let builtin = crate::cli::oauth_defaults::builtin_credentials(&auth.secret_name);
let client_id = oauth
.client_id
.clone()
.or_else(|| {
oauth
.client_id_env
.as_ref()
.and_then(|env| std::env::var(env).ok())
})
.or_else(|| builtin.as_ref().map(|c| c.client_id.to_string()))?;
let client_secret = oauth
.client_secret
.clone()
.or_else(|| {
oauth
.client_secret_env
.as_ref()
.and_then(|env| std::env::var(env).ok())
})
.or_else(|| builtin.as_ref().map(|c| c.client_secret.to_string()));
Some(OAuthRefreshConfig {
token_url: oauth.token_url.clone(),
client_id,
client_secret,
secret_name: auth.secret_name.clone(),
provider: auth.provider.clone(),
})
}
/// Results from loading multiple tools. /// Results from loading multiple tools.
#[derive(Debug, Default)] #[derive(Debug, Default)]
pub struct LoadResults { pub struct LoadResults {
@@ -618,4 +678,116 @@ mod tests {
); );
} }
} }
#[test]
fn test_resolve_oauth_refresh_config_with_oauth() {
use crate::tools::wasm::capabilities_schema::{
AuthCapabilitySchema, CapabilitiesFile, OAuthConfigSchema,
};
let caps = CapabilitiesFile {
auth: Some(AuthCapabilitySchema {
secret_name: "google_oauth_token".to_string(),
provider: Some("google".to_string()),
oauth: Some(OAuthConfigSchema {
authorization_url: "https://accounts.google.com/o/oauth2/v2/auth".to_string(),
token_url: "https://oauth2.googleapis.com/token".to_string(),
client_id: Some("test-client-id".to_string()),
client_secret: Some("test-client-secret".to_string()),
..Default::default()
}),
..Default::default()
}),
..Default::default()
};
let config = super::resolve_oauth_refresh_config(&caps);
assert!(config.is_some());
let config = config.unwrap();
assert_eq!(config.token_url, "https://oauth2.googleapis.com/token");
assert_eq!(config.client_id, "test-client-id");
assert_eq!(config.client_secret, Some("test-client-secret".to_string()));
assert_eq!(config.secret_name, "google_oauth_token");
assert_eq!(config.provider, Some("google".to_string()));
}
#[test]
fn test_resolve_oauth_refresh_config_no_auth() {
use crate::tools::wasm::capabilities_schema::CapabilitiesFile;
let caps = CapabilitiesFile::default();
let config = super::resolve_oauth_refresh_config(&caps);
assert!(config.is_none());
}
#[test]
fn test_resolve_oauth_refresh_config_no_oauth() {
use crate::tools::wasm::capabilities_schema::{AuthCapabilitySchema, CapabilitiesFile};
let caps = CapabilitiesFile {
auth: Some(AuthCapabilitySchema {
secret_name: "manual_token".to_string(),
..Default::default()
}),
..Default::default()
};
let config = super::resolve_oauth_refresh_config(&caps);
assert!(config.is_none());
}
#[test]
fn test_resolve_oauth_refresh_config_no_client_id() {
use crate::tools::wasm::capabilities_schema::{
AuthCapabilitySchema, CapabilitiesFile, OAuthConfigSchema,
};
// A non-Google provider with no client_id anywhere should return None
let caps = CapabilitiesFile {
auth: Some(AuthCapabilitySchema {
secret_name: "unknown_provider_token".to_string(),
oauth: Some(OAuthConfigSchema {
authorization_url: "https://example.com/auth".to_string(),
token_url: "https://example.com/token".to_string(),
// No client_id, no client_id_env, no builtin
..Default::default()
}),
..Default::default()
}),
..Default::default()
};
let config = super::resolve_oauth_refresh_config(&caps);
assert!(config.is_none());
}
#[test]
fn test_resolve_oauth_refresh_config_builtin_google() {
use crate::tools::wasm::capabilities_schema::{
AuthCapabilitySchema, CapabilitiesFile, OAuthConfigSchema,
};
// google_oauth_token should fall back to built-in credentials
let caps = CapabilitiesFile {
auth: Some(AuthCapabilitySchema {
secret_name: "google_oauth_token".to_string(),
provider: Some("google".to_string()),
oauth: Some(OAuthConfigSchema {
authorization_url: "https://accounts.google.com/o/oauth2/v2/auth".to_string(),
token_url: "https://oauth2.googleapis.com/token".to_string(),
// No inline client_id, should fall back to builtin
..Default::default()
}),
..Default::default()
}),
..Default::default()
};
let config = super::resolve_oauth_refresh_config(&caps);
assert!(config.is_some());
let config = config.unwrap();
assert!(!config.client_id.is_empty());
assert!(config.client_secret.is_some());
}
} }
+1 -1
View File
@@ -94,7 +94,7 @@ pub use limits::{
WasmResourceLimiter, WasmResourceLimiter,
}; };
pub use runtime::{PreparedModule, WasmRuntimeConfig, WasmToolRuntime}; pub use runtime::{PreparedModule, WasmRuntimeConfig, WasmToolRuntime};
pub use wrapper::WasmToolWrapper; pub use wrapper::{OAuthRefreshConfig, WasmToolWrapper};
// Capabilities (V2) // Capabilities (V2)
pub use capabilities::{ pub use capabilities::{
+1
View File
@@ -953,6 +953,7 @@ fn libsql_row_to_tool_with_offset(row: &libsql::Row) -> Result<StoredWasmTool, W
} }
#[cfg(feature = "libsql")] #[cfg(feature = "libsql")]
#[allow(clippy::too_many_arguments)]
fn libsql_row_to_tool_at( fn libsql_row_to_tool_at(
row: &libsql::Row, row: &libsql::Row,
id_idx: i32, id_idx: i32,
+878 -15
View File
File diff suppressed because it is too large Load Diff
+319
View File
@@ -0,0 +1,319 @@
//! Truncating terminal writer for tracing.
//!
//! Tracing events from LLM providers can dump 10KB+ JSON bodies to stderr.
//! Rather than truncating at every call site (fragile, easy to miss), we
//! handle it at the writer level: the fmt layer gets a `TruncatingStderr`
//! that caps each event before flushing, while the web gateway `WebLogLayer`
//! still sees the full, untruncated content.
//!
//! ```text
//! tracing::debug!("body: {huge_json}")
//! |
//! v
//! tracing_subscriber::registry()
//! |
//! +-- fmt::layer().with_writer(TruncatingStderr) <-- caps at 500B
//! | \-- stderr (truncated)
//! |
//! \-- WebLogLayer (unchanged)
//! \-- SSE broadcast (full)
//! ```
use std::io::{self, Write};
use tracing_subscriber::fmt::MakeWriter;
/// Maximum bytes per tracing event written to the terminal.
const TERMINAL_MAX_EVENT_BYTES: usize = 500;
/// A `MakeWriter` that creates per-event buffers which truncate on flush.
///
/// Each call to `make_writer()` returns an `EventBuffer`. All `write()`
/// calls accumulate into the buffer. When the buffer drops (after the fmt
/// layer finishes writing one event), it flushes to stderr, truncating if
/// the total exceeds `TERMINAL_MAX_EVENT_BYTES`.
#[derive(Clone)]
pub struct TruncatingStderr {
max_bytes: usize,
}
impl Default for TruncatingStderr {
fn default() -> Self {
Self {
max_bytes: TERMINAL_MAX_EVENT_BYTES,
}
}
}
impl TruncatingStderr {
#[cfg(test)]
fn with_max_bytes(max_bytes: usize) -> Self {
Self { max_bytes }
}
}
impl<'a> MakeWriter<'a> for TruncatingStderr {
type Writer = EventBuffer;
fn make_writer(&'a self) -> Self::Writer {
EventBuffer {
buf: Vec::with_capacity(256),
max_bytes: self.max_bytes,
#[cfg(test)]
sink: None,
}
}
}
/// Per-event buffer that truncates on drop.
pub struct EventBuffer {
buf: Vec<u8>,
max_bytes: usize,
/// Test-only: capture output instead of writing to stderr.
#[cfg(test)]
sink: Option<std::sync::Arc<std::sync::Mutex<Vec<u8>>>>,
}
impl Write for EventBuffer {
fn write(&mut self, data: &[u8]) -> io::Result<usize> {
self.buf.extend_from_slice(data);
Ok(data.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
/// Find the last valid UTF-8 char boundary at or before `pos` in `bytes`.
///
/// Walks backwards from `pos` until we find a byte that isn't a UTF-8
/// continuation byte (0x80..0xBF). Returns 0 if the entire prefix is
/// somehow invalid (shouldn't happen with valid UTF-8 input from tracing).
fn utf8_floor(bytes: &[u8], pos: usize) -> usize {
let mut i = pos;
// UTF-8 continuation bytes have the form 10xxxxxx (0x80..0xBF).
// Walk backwards past them to find the start of the last character.
while i > 0 && bytes[i] & 0xC0 == 0x80 {
i -= 1;
}
i
}
impl Drop for EventBuffer {
fn drop(&mut self) {
if self.buf.is_empty() {
return;
}
let output = if self.buf.len() <= self.max_bytes {
&self.buf[..]
} else {
// Truncate at a UTF-8 safe boundary
let cut = utf8_floor(&self.buf, self.max_bytes);
let suffix = format!("...[{}B total]\n", self.buf.len());
let mut truncated = Vec::with_capacity(cut + suffix.len());
// Strip trailing newline from the cut portion (we add our own via suffix)
let cut_slice = &self.buf[..cut];
let trimmed = if cut_slice.last() == Some(&b'\n') {
&cut_slice[..cut_slice.len() - 1]
} else {
cut_slice
};
truncated.extend_from_slice(trimmed);
truncated.extend_from_slice(suffix.as_bytes());
#[cfg(test)]
if let Some(ref sink) = self.sink {
let mut s = sink.lock().expect("test sink lock poisoned");
s.extend_from_slice(&truncated);
return;
}
let _ = io::stderr().write_all(&truncated);
return;
};
#[cfg(test)]
if let Some(ref sink) = self.sink {
let mut s = sink.lock().expect("test sink lock poisoned");
s.extend_from_slice(output);
return;
}
let _ = io::stderr().write_all(output);
}
}
#[cfg(test)]
mod tests {
use std::sync::{Arc, Mutex};
use crate::tracing_fmt::{EventBuffer, TruncatingStderr, utf8_floor};
use std::io::Write;
/// Helper: create an EventBuffer that captures output to a shared Vec
/// instead of writing to stderr.
fn test_buffer(max_bytes: usize) -> (EventBuffer, Arc<Mutex<Vec<u8>>>) {
let sink = Arc::new(Mutex::new(Vec::new()));
let buf = EventBuffer {
buf: Vec::new(),
max_bytes,
sink: Some(Arc::clone(&sink)),
};
(buf, sink)
}
#[test]
fn test_short_event_not_truncated() {
let (mut buf, sink) = test_buffer(500);
buf.write_all(b"hello world\n").unwrap();
drop(buf);
let output = sink.lock().unwrap();
assert_eq!(&*output, b"hello world\n");
}
#[test]
fn test_long_event_truncated() {
let (mut buf, sink) = test_buffer(20);
let data = "abcdefghijklmnopqrstuvwxyz0123456789\n";
buf.write_all(data.as_bytes()).unwrap();
let total = data.len();
drop(buf);
let output = sink.lock().unwrap();
let output_str = String::from_utf8_lossy(&output);
// Should contain the suffix with total byte count
assert!(
output_str.contains(&format!("...[{}B total]", total)),
"expected truncation suffix, got: {}",
output_str
);
// Should be shorter than the original
assert!(output.len() < total);
}
#[test]
fn test_utf8_boundary_safe() {
// "Helloé" = [72, 101, 108, 108, 111, 195, 169]
// ^-- 2-byte UTF-8 char
// If we truncate at 6 bytes, we'd land in the middle of 'é'.
// utf8_floor should back up to byte 5 (start of 'é' = 195).
let (mut buf, sink) = test_buffer(6);
let data = "Helloé world";
buf.write_all(data.as_bytes()).unwrap();
drop(buf);
let output = sink.lock().unwrap();
let output_str = String::from_utf8(output.clone());
assert!(
output_str.is_ok(),
"output should be valid UTF-8, got bytes: {:?}",
&*output
);
let s = output_str.unwrap();
assert!(
s.contains("...["),
"should be truncated with suffix, got: {}",
s
);
// The truncated prefix must be valid UTF-8 up to the cut point.
// "Hello" (5 bytes) is the last valid cut before the 2-byte é.
assert!(
s.starts_with("Hello"),
"should start with 'Hello', got: {}",
s
);
}
#[test]
fn test_utf8_floor_basic() {
// ASCII: every byte is a valid boundary
assert_eq!(utf8_floor(b"hello", 3), 3);
// 2-byte UTF-8 char é = [0xC3, 0xA9]
// Landing on the continuation byte (0xA9) should back up to 0xC3
let bytes = "".as_bytes(); // [72, 0xC3, 0xA9]
assert_eq!(utf8_floor(bytes, 2), 1); // backs up to start of é
// 3-byte UTF-8 char (e.g. あ = [0xE3, 0x81, 0x82])
let bytes = "aあ".as_bytes(); // [97, 0xE3, 0x81, 0x82]
assert_eq!(utf8_floor(bytes, 2), 1); // backs up past continuation to 0xE3
assert_eq!(utf8_floor(bytes, 3), 1); // same: 0x82 is continuation, 0x81 is too
}
#[test]
fn test_multiple_writes_accumulated() {
let (mut buf, sink) = test_buffer(500);
buf.write_all(b"hello ").unwrap();
buf.write_all(b"world\n").unwrap();
drop(buf);
let output = sink.lock().unwrap();
assert_eq!(&*output, b"hello world\n");
}
#[test]
fn test_empty_buffer_no_output() {
let (_buf, sink) = test_buffer(500);
// drop without writing
drop(_buf);
let output = sink.lock().unwrap();
assert!(output.is_empty());
}
#[test]
fn test_default_max_bytes() {
let writer = TruncatingStderr::default();
assert_eq!(writer.max_bytes, 500);
}
#[test]
fn test_custom_max_bytes() {
let writer = TruncatingStderr::with_max_bytes(100);
assert_eq!(writer.max_bytes, 100);
}
#[test]
fn test_exactly_at_limit_not_truncated() {
let (mut buf, sink) = test_buffer(5);
buf.write_all(b"hello").unwrap();
drop(buf);
let output = sink.lock().unwrap();
assert_eq!(&*output, b"hello");
}
#[test]
fn test_one_over_limit_truncated() {
let (mut buf, sink) = test_buffer(5);
buf.write_all(b"hello!").unwrap();
drop(buf);
let output = sink.lock().unwrap();
let s = String::from_utf8_lossy(&output);
assert!(s.contains("...[6B total]"), "got: {}", s);
}
#[test]
fn test_4byte_utf8_boundary() {
// 4-byte UTF-8 char: 𝄞 (musical symbol) = [0xF0, 0x9D, 0x84, 0x9E]
let data = "AB𝄞CD";
// bytes: [65, 66, 0xF0, 0x9D, 0x84, 0x9E, 67, 68]
// Truncating at byte 4 lands in the middle of the 4-byte char
let (mut buf, sink) = test_buffer(4);
buf.write_all(data.as_bytes()).unwrap();
drop(buf);
let output = sink.lock().unwrap();
let s = String::from_utf8(output.clone());
assert!(s.is_ok(), "output must be valid UTF-8, got: {:?}", &*output);
let s = s.unwrap();
// Should back up to byte 2 (just "AB"), since bytes 2..5 are all part of 𝄞
assert!(s.starts_with("AB"), "expected 'AB', got: {}", s);
assert!(s.contains("...["), "should be truncated, got: {}", s);
}
}
+3 -3
View File
@@ -326,8 +326,9 @@ impl ClaudeBridgeRuntime {
match serde_json::from_str::<ClaudeStreamEvent>(&line) { match serde_json::from_str::<ClaudeStreamEvent>(&line) {
Ok(event) => { Ok(event) => {
// Capture session_id from system init // Capture session_id from system init
if event.event_type == "system" { if event.event_type == "system"
if let Some(ref sid) = event.session_id { && let Some(ref sid) = event.session_id
{
session_id = Some(sid.clone()); session_id = Some(sid.clone());
tracing::info!( tracing::info!(
job_id = %self.config.job_id, job_id = %self.config.job_id,
@@ -335,7 +336,6 @@ impl ClaudeBridgeRuntime {
"Captured Claude session ID" "Captured Claude session ID"
); );
} }
}
// Convert to our event payload and forward // Convert to our event payload and forward
let payloads = stream_event_to_payloads(&event); let payloads = stream_event_to_payloads(&event);
+3 -2
View File
@@ -313,6 +313,7 @@ Work independently to complete this job. Report when done."#,
parameters: tc.arguments.clone(), parameters: tc.arguments.clone(),
reasoning: String::new(), reasoning: String::new(),
alternatives: vec![], alternatives: vec![],
tool_call_id: tc.id.clone(),
}; };
self.process_result(reason_ctx, &selection, result); self.process_result(reason_ctx, &selection, result);
} }
@@ -422,7 +423,7 @@ Work independently to complete this job. Report when done."#,
); );
reason_ctx.messages.push(ChatMessage::tool_result( reason_ctx.messages.push(ChatMessage::tool_result(
"tool_call_id", &selection.tool_call_id,
&selection.tool_name, &selection.tool_name,
wrapped, wrapped,
)); ));
@@ -436,7 +437,7 @@ Work independently to complete this job. Report when done."#,
Err(e) => { Err(e) => {
tracing::warn!("Tool {} failed: {}", selection.tool_name, e); tracing::warn!("Tool {} failed: {}", selection.tool_name, e);
reason_ctx.messages.push(ChatMessage::tool_result( reason_ctx.messages.push(ChatMessage::tool_result(
"tool_call_id", &selection.tool_call_id,
&selection.tool_name, &selection.tool_name,
format!("Error: {}", e), format!("Error: {}", e),
)); ));
+6 -6
View File
@@ -532,20 +532,21 @@ impl Workspace {
]; ];
for (path, header) in identity_files { for (path, header) in identity_files {
if let Ok(doc) = self.read(path).await { if let Ok(doc) = self.read(path).await
if !doc.content.is_empty() { && !doc.content.is_empty()
{
parts.push(format!("{}\n\n{}", header, doc.content)); parts.push(format!("{}\n\n{}", header, doc.content));
} }
} }
}
// Add today's memory context (last 2 days of daily logs) // Add today's memory context (last 2 days of daily logs)
let today = Utc::now().date_naive(); let today = Utc::now().date_naive();
let yesterday = today.pred_opt().unwrap_or(today); let yesterday = today.pred_opt().unwrap_or(today);
for date in [today, yesterday] { for date in [today, yesterday] {
if let Ok(doc) = self.daily_log(date).await { if let Ok(doc) = self.daily_log(date).await
if !doc.content.is_empty() { && !doc.content.is_empty()
{
let header = if date == today { let header = if date == today {
"## Today's Notes" "## Today's Notes"
} else { } else {
@@ -554,7 +555,6 @@ impl Workspace {
parts.push(format!("{}\n\n{}", header, doc.content)); parts.push(format!("{}\n\n{}", header, doc.content));
} }
} }
}
Ok(parts.join("\n\n---\n\n")) Ok(parts.join("\n\n---\n\n"))
} }
+3 -3
View File
@@ -201,13 +201,13 @@ pub fn reciprocal_rank_fusion(
.collect(); .collect();
// Normalize scores to 0-1 range // Normalize scores to 0-1 range
if let Some(max_score) = results.iter().map(|r| r.score).reduce(f32::max) { if let Some(max_score) = results.iter().map(|r| r.score).reduce(f32::max)
if max_score > 0.0 { && max_score > 0.0
{
for result in &mut results { for result in &mut results {
result.score /= max_score; result.score /= max_score;
} }
} }
}
// Filter by minimum score // Filter by minimum score
if config.min_score > 0.0 { if config.min_score > 0.0 {
+3 -3
View File
@@ -302,13 +302,13 @@ async fn test_chat_completions_streaming() {
if data == "[DONE]" { if data == "[DONE]" {
continue; continue;
} }
if let Ok(chunk) = serde_json::from_str::<serde_json::Value>(data) { if let Ok(chunk) = serde_json::from_str::<serde_json::Value>(data)
if let Some(content) = chunk["choices"][0]["delta"]["content"].as_str() { && let Some(content) = chunk["choices"][0]["delta"]["content"].as_str()
{
full_content.push_str(content); full_content.push_str(content);
} }
} }
} }
}
assert!( assert!(
full_content.contains("Stream test"), full_content.contains("Stream test"),
"Expected reassembled content to contain 'Stream test', got: '{}'", "Expected reassembled content to contain 'Stream test', got: '{}'",
+2 -2
View File
@@ -136,7 +136,7 @@ fn parse_message(v: &serde_json::Value) -> Message {
date: get_header(payload, "Date"), date: get_header(payload, "Date"),
body: extract_body(payload), body: extract_body(payload),
snippet: v["snippet"].as_str().unwrap_or("").to_string(), snippet: v["snippet"].as_str().unwrap_or("").to_string(),
is_unread: label_ids.contains(&"UNREAD".to_string()), is_unread: label_ids.iter().any(|l| l == "UNREAD"),
label_ids, label_ids,
} }
} }
@@ -198,7 +198,7 @@ pub fn list_messages(
to: get_header(payload, "To"), to: get_header(payload, "To"),
date: get_header(payload, "Date"), date: get_header(payload, "Date"),
snippet: msg["snippet"].as_str().unwrap_or("").to_string(), snippet: msg["snippet"].as_str().unwrap_or("").to_string(),
is_unread: label_ids.contains(&"UNREAD".to_string()), is_unread: label_ids.iter().any(|l| l == "UNREAD"),
label_ids, label_ids,
}); });
} }
+15 -78
View File
@@ -53,119 +53,56 @@ impl exports::near::agent::tool::Guest for GmailTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "list_messages" }, "action": {
"type": "string",
"enum": ["list_messages", "get_message", "send_message", "create_draft", "reply_to_message", "trash_message"],
"description": "The Gmail operation to perform"
},
"query": { "query": {
"type": "string", "type": "string",
"description": "Gmail search query (same syntax as Gmail search box). Examples: 'is:unread', 'from:[email protected]', 'subject:meeting after:2025/01/01'" "description": "Gmail search query (same syntax as Gmail search box, e.g., 'is:unread', 'from:[email protected]'). Used by: list_messages"
}, },
"max_results": { "max_results": {
"type": "integer", "type": "integer",
"description": "Maximum number of messages to return (default: 20)", "description": "Maximum number of messages to return (default: 20). Used by: list_messages",
"default": 20 "default": 20
}, },
"label_ids": { "label_ids": {
"type": "array", "type": "array",
"items": { "type": "string" }, "items": { "type": "string" },
"description": "Label IDs to filter by (e.g., 'INBOX', 'SENT', 'DRAFT')" "description": "Label IDs to filter by (e.g., 'INBOX', 'SENT', 'DRAFT'). Used by: list_messages"
}
}, },
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_message" },
"message_id": { "message_id": {
"type": "string", "type": "string",
"description": "The message ID to retrieve" "description": "Message ID. Required for: get_message, reply_to_message, trash_message"
}
}, },
"required": ["action", "message_id"]
},
{
"properties": {
"action": { "const": "send_message" },
"to": { "to": {
"type": "string", "type": "string",
"description": "Recipient email address(es), comma-separated" "description": "Recipient email address(es), comma-separated. Required for: send_message, create_draft"
}, },
"subject": { "subject": {
"type": "string", "type": "string",
"description": "Email subject" "description": "Email subject. Required for: send_message, create_draft"
}, },
"body": { "body": {
"type": "string", "type": "string",
"description": "Email body (plain text)" "description": "Email body (plain text). Required for: send_message, create_draft, reply_to_message"
}, },
"cc": { "cc": {
"type": "string", "type": "string",
"description": "CC recipients, comma-separated" "description": "CC recipients, comma-separated. Used by: send_message, create_draft"
}, },
"bcc": { "bcc": {
"type": "string", "type": "string",
"description": "BCC recipients, comma-separated" "description": "BCC recipients, comma-separated. Used by: send_message, create_draft"
}
},
"required": ["action", "to", "subject", "body"]
},
{
"properties": {
"action": { "const": "create_draft" },
"to": {
"type": "string",
"description": "Recipient email address(es), comma-separated"
},
"subject": {
"type": "string",
"description": "Email subject"
},
"body": {
"type": "string",
"description": "Email body (plain text)"
},
"cc": {
"type": "string",
"description": "CC recipients, comma-separated"
},
"bcc": {
"type": "string",
"description": "BCC recipients, comma-separated"
}
},
"required": ["action", "to", "subject", "body"]
},
{
"properties": {
"action": { "const": "reply_to_message" },
"message_id": {
"type": "string",
"description": "The message ID to reply to"
},
"body": {
"type": "string",
"description": "Reply body (plain text)"
}, },
"reply_all": { "reply_all": {
"type": "boolean", "type": "boolean",
"description": "If true, reply to all recipients (default: false)", "description": "If true, reply to all recipients (default: false). Used by: reply_to_message",
"default": false "default": false
} }
},
"required": ["action", "message_id", "body"]
},
{
"properties": {
"action": { "const": "trash_message" },
"message_id": {
"type": "string",
"description": "The message ID to move to trash"
} }
},
"required": ["action", "message_id"]
}
]
}"# }"#
.to_string() .to_string()
} }
+23 -113
View File
@@ -6,7 +6,7 @@
//! # Capabilities Required //! # Capabilities Required
//! //!
//! - HTTP: `www.googleapis.com/calendar/v3/*` (GET, POST, PUT, PATCH, DELETE) //! - HTTP: `www.googleapis.com/calendar/v3/*` (GET, POST, PUT, PATCH, DELETE)
//! - Secrets: `google_calendar_token` (OAuth 2.0 token, injected automatically) //! - Secrets: `google_oauth_token` (OAuth 2.0 token, injected automatically)
//! //!
//! # Supported Actions //! # Supported Actions
//! //!
@@ -52,166 +52,76 @@ impl exports::near::agent::tool::Guest for GoogleCalendarTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "list_events" }, "action": {
"type": "string",
"enum": ["list_events", "get_event", "create_event", "update_event", "delete_event"],
"description": "The calendar operation to perform"
},
"calendar_id": { "calendar_id": {
"type": "string", "type": "string",
"description": "Calendar ID (default: 'primary')", "description": "Calendar ID (default: 'primary')",
"default": "primary" "default": "primary"
}, },
"event_id": {
"type": "string",
"description": "Event ID. Required for: get_event, update_event, delete_event"
},
"time_min": { "time_min": {
"type": "string", "type": "string",
"description": "Lower bound for event start time (RFC3339, e.g., '2025-01-15T00:00:00Z')" "description": "Lower bound for event start time (RFC3339, e.g., '2025-01-15T00:00:00Z'). Used by: list_events"
}, },
"time_max": { "time_max": {
"type": "string", "type": "string",
"description": "Upper bound for event end time (RFC3339)" "description": "Upper bound for event end time (RFC3339). Used by: list_events"
}, },
"max_results": { "max_results": {
"type": "integer", "type": "integer",
"description": "Maximum number of events to return (default: 25)", "description": "Maximum number of events to return (default: 25). Used by: list_events",
"default": 25 "default": 25
}, },
"query": { "query": {
"type": "string", "type": "string",
"description": "Free text search terms to filter events" "description": "Free text search terms to filter events. Used by: list_events"
}
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_event" },
"calendar_id": {
"type": "string",
"description": "Calendar ID (default: 'primary')",
"default": "primary"
},
"event_id": {
"type": "string",
"description": "The event ID to retrieve"
}
},
"required": ["action", "event_id"]
},
{
"properties": {
"action": { "const": "create_event" },
"calendar_id": {
"type": "string",
"description": "Calendar ID (default: 'primary')",
"default": "primary"
}, },
"summary": { "summary": {
"type": "string", "type": "string",
"description": "Event title" "description": "Event title. Required for: create_event. Optional for: update_event"
}, },
"description": { "description": {
"type": "string", "type": "string",
"description": "Event description" "description": "Event description. Used by: create_event, update_event"
}, },
"location": { "location": {
"type": "string", "type": "string",
"description": "Event location" "description": "Event location. Used by: create_event, update_event"
}, },
"start_datetime": { "start_datetime": {
"type": "string", "type": "string",
"description": "Start time as RFC3339 (e.g., '2025-01-15T09:00:00-05:00'). Use start_date for all-day events." "description": "Start time (RFC3339, e.g., '2025-01-15T09:00:00-05:00'). For all-day events use start_date. Used by: create_event, update_event"
}, },
"end_datetime": { "end_datetime": {
"type": "string", "type": "string",
"description": "End time as RFC3339. Use end_date for all-day events." "description": "End time (RFC3339). For all-day events use end_date. Used by: create_event, update_event"
}, },
"start_date": { "start_date": {
"type": "string", "type": "string",
"description": "Start date for all-day events (e.g., '2025-01-15')" "description": "Start date for all-day events (e.g., '2025-01-15'). Used by: create_event, update_event"
}, },
"end_date": { "end_date": {
"type": "string", "type": "string",
"description": "End date for all-day events (exclusive, e.g., '2025-01-16' for a single day)" "description": "End date for all-day events (exclusive, e.g., '2025-01-16'). Used by: create_event, update_event"
}, },
"timezone": { "timezone": {
"type": "string", "type": "string",
"description": "Timezone (e.g., 'America/New_York')" "description": "Timezone (e.g., 'America/New_York'). Used by: create_event, update_event"
}, },
"attendees": { "attendees": {
"type": "array", "type": "array",
"items": { "type": "string" }, "items": { "type": "string" },
"description": "Attendee email addresses" "description": "Attendee email addresses. Used by: create_event, update_event"
} }
},
"required": ["action", "summary"]
},
{
"properties": {
"action": { "const": "update_event" },
"calendar_id": {
"type": "string",
"description": "Calendar ID (default: 'primary')",
"default": "primary"
},
"event_id": {
"type": "string",
"description": "The event ID to update"
},
"summary": {
"type": "string",
"description": "New event title"
},
"description": {
"type": "string",
"description": "New event description"
},
"location": {
"type": "string",
"description": "New event location"
},
"start_datetime": {
"type": "string",
"description": "New start time (RFC3339)"
},
"end_datetime": {
"type": "string",
"description": "New end time (RFC3339)"
},
"start_date": {
"type": "string",
"description": "New start date for all-day events"
},
"end_date": {
"type": "string",
"description": "New end date for all-day events"
},
"timezone": {
"type": "string",
"description": "Timezone for datetime fields"
},
"attendees": {
"type": "array",
"items": { "type": "string" },
"description": "Replace attendees with these email addresses"
} }
},
"required": ["action", "event_id"]
},
{
"properties": {
"action": { "const": "delete_event" },
"calendar_id": {
"type": "string",
"description": "Calendar ID (default: 'primary')",
"default": "primary"
},
"event_id": {
"type": "string",
"description": "The event ID to delete"
}
},
"required": ["action", "event_id"]
}
]
}"# }"#
.to_string() .to_string()
} }
+6 -1
View File
@@ -269,8 +269,13 @@ pub fn replace_text(
let parsed = batch_update_raw(document_id, vec![request])?; let parsed = batch_update_raw(document_id, vec![request])?;
let occurrences = parsed["replies"][0]["replaceAllText"]["occurrencesChanged"] let first_reply = parsed["replies"].as_array().and_then(|arr| arr.first());
let occurrences = first_reply
.map(|r| {
r["replaceAllText"]["occurrencesChanged"]
.as_i64() .as_i64()
.unwrap_or(0)
})
.unwrap_or(0); .unwrap_or(0);
Ok(ReplaceResult { Ok(ReplaceResult {
+30 -161
View File
@@ -74,251 +74,120 @@ impl exports::near::agent::tool::Guest for GoogleDocsTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "create_document" }, "action": {
"type": "string",
"enum": ["create_document", "get_document", "read_content", "insert_text", "delete_content", "replace_text", "format_text", "format_paragraph", "insert_table", "create_list", "batch_update"],
"description": "The Google Docs operation to perform"
},
"title": { "title": {
"type": "string", "type": "string",
"description": "Document title" "description": "Document title. Required for: create_document"
}
}, },
"required": ["action", "title"]
},
{
"properties": {
"action": { "const": "get_document" },
"document_id": { "document_id": {
"type": "string", "type": "string",
"description": "The document ID (same as Google Drive file ID)" "description": "The document ID (same as Google Drive file ID). Required for all actions except create_document"
}
},
"required": ["action", "document_id"]
},
{
"properties": {
"action": { "const": "read_content" },
"document_id": {
"type": "string",
"description": "The document ID"
}
},
"required": ["action", "document_id"]
},
{
"properties": {
"action": { "const": "insert_text" },
"document_id": {
"type": "string",
"description": "The document ID"
}, },
"text": { "text": {
"type": "string", "type": "string",
"description": "Text to insert" "description": "Text to insert. Required for: insert_text"
}, },
"index": { "index": {
"type": "integer", "type": "integer",
"description": "Character index to insert at (1 for start of body). Use -1 to append at end.", "description": "Character index (1 for start of body, -1 to append at end). Required for: insert_table. Used by: insert_text (default: -1)"
"default": -1
}, },
"segment_id": { "segment_id": {
"type": "string", "type": "string",
"description": "Segment ID (empty string for body, or a header/footer ID)", "description": "Segment ID (empty for body, or a header/footer ID). Used by: insert_text, delete_content",
"default": "" "default": ""
}
},
"required": ["action", "document_id", "text"]
},
{
"properties": {
"action": { "const": "delete_content" },
"document_id": {
"type": "string",
"description": "The document ID"
}, },
"start_index": { "start_index": {
"type": "integer", "type": "integer",
"description": "Start index (inclusive)" "description": "Start index (inclusive). Required for: delete_content, format_text, format_paragraph, create_list"
}, },
"end_index": { "end_index": {
"type": "integer", "type": "integer",
"description": "End index (exclusive)" "description": "End index (exclusive). Required for: delete_content, format_text, format_paragraph, create_list"
},
"segment_id": {
"type": "string",
"description": "Segment ID (empty for body)",
"default": ""
}
},
"required": ["action", "document_id", "start_index", "end_index"]
},
{
"properties": {
"action": { "const": "replace_text" },
"document_id": {
"type": "string",
"description": "The document ID"
}, },
"find": { "find": {
"type": "string", "type": "string",
"description": "Text to search for" "description": "Text to search for. Required for: replace_text"
}, },
"replace": { "replace": {
"type": "string", "type": "string",
"description": "Replacement text" "description": "Replacement text. Required for: replace_text"
}, },
"match_case": { "match_case": {
"type": "boolean", "type": "boolean",
"description": "Case-sensitive match (default: true)", "description": "Case-sensitive match (default: true). Used by: replace_text",
"default": true "default": true
}
},
"required": ["action", "document_id", "find", "replace"]
},
{
"properties": {
"action": { "const": "format_text" },
"document_id": {
"type": "string",
"description": "The document ID"
},
"start_index": {
"type": "integer",
"description": "Start index (inclusive)"
},
"end_index": {
"type": "integer",
"description": "End index (exclusive)"
}, },
"bold": { "bold": {
"type": "boolean", "type": "boolean",
"description": "Make text bold" "description": "Make text bold. Used by: format_text"
}, },
"italic": { "italic": {
"type": "boolean", "type": "boolean",
"description": "Make text italic" "description": "Make text italic. Used by: format_text"
}, },
"underline": { "underline": {
"type": "boolean", "type": "boolean",
"description": "Underline text" "description": "Underline text. Used by: format_text"
}, },
"strikethrough": { "strikethrough": {
"type": "boolean", "type": "boolean",
"description": "Strikethrough text" "description": "Strikethrough text. Used by: format_text"
}, },
"font_size": { "font_size": {
"type": "number", "type": "number",
"description": "Font size in points (e.g., 12, 14, 18)" "description": "Font size in points (e.g., 12, 14, 18). Used by: format_text"
}, },
"font_family": { "font_family": {
"type": "string", "type": "string",
"description": "Font family (e.g., 'Arial', 'Times New Roman', 'Courier New')" "description": "Font family (e.g., 'Arial', 'Times New Roman'). Used by: format_text"
}, },
"foreground_color": { "foreground_color": {
"type": "string", "type": "string",
"description": "Text color as hex (e.g., '#FF0000' for red)" "description": "Text color as hex (e.g., '#FF0000'). Used by: format_text"
}, },
"background_color": { "background_color": {
"type": "string", "type": "string",
"description": "Text background/highlight color as hex" "description": "Text background/highlight color as hex. Used by: format_text"
}
},
"required": ["action", "document_id", "start_index", "end_index"]
},
{
"properties": {
"action": { "const": "format_paragraph" },
"document_id": {
"type": "string",
"description": "The document ID"
},
"start_index": {
"type": "integer",
"description": "Start index (inclusive)"
},
"end_index": {
"type": "integer",
"description": "End index (exclusive)"
}, },
"named_style": { "named_style": {
"type": "string", "type": "string",
"enum": ["NORMAL_TEXT", "TITLE", "SUBTITLE", "HEADING_1", "HEADING_2", "HEADING_3", "HEADING_4", "HEADING_5", "HEADING_6"], "enum": ["NORMAL_TEXT", "TITLE", "SUBTITLE", "HEADING_1", "HEADING_2", "HEADING_3", "HEADING_4", "HEADING_5", "HEADING_6"],
"description": "Paragraph style (heading level)" "description": "Paragraph style (heading level). Used by: format_paragraph"
}, },
"alignment": { "alignment": {
"type": "string", "type": "string",
"enum": ["START", "CENTER", "END", "JUSTIFIED"], "enum": ["START", "CENTER", "END", "JUSTIFIED"],
"description": "Text alignment" "description": "Text alignment. Used by: format_paragraph"
}, },
"line_spacing": { "line_spacing": {
"type": "number", "type": "number",
"description": "Line spacing as percentage (e.g., 100 for single, 150 for 1.5x, 200 for double)" "description": "Line spacing as percentage (100=single, 150=1.5x, 200=double). Used by: format_paragraph"
}
},
"required": ["action", "document_id", "start_index", "end_index"]
},
{
"properties": {
"action": { "const": "insert_table" },
"document_id": {
"type": "string",
"description": "The document ID"
}, },
"rows": { "rows": {
"type": "integer", "type": "integer",
"description": "Number of rows" "description": "Number of rows. Required for: insert_table"
}, },
"columns": { "columns": {
"type": "integer", "type": "integer",
"description": "Number of columns" "description": "Number of columns. Required for: insert_table"
},
"index": {
"type": "integer",
"description": "Character index to insert the table at"
}
},
"required": ["action", "document_id", "rows", "columns", "index"]
},
{
"properties": {
"action": { "const": "create_list" },
"document_id": {
"type": "string",
"description": "The document ID"
},
"start_index": {
"type": "integer",
"description": "Start index (inclusive)"
},
"end_index": {
"type": "integer",
"description": "End index (exclusive)"
}, },
"bullet_preset": { "bullet_preset": {
"type": "string", "type": "string",
"enum": ["BULLET_DISC_CIRCLE_SQUARE", "BULLET_CHECKBOX", "BULLET_ARROW_DIAMOND_DISC", "NUMBERED_DECIMAL_ALPHA_ROMAN", "NUMBERED_DECIMAL_NESTED", "NUMBERED_UPPERALPHA_ALPHA_ROMAN"], "enum": ["BULLET_DISC_CIRCLE_SQUARE", "BULLET_CHECKBOX", "BULLET_ARROW_DIAMOND_DISC", "NUMBERED_DECIMAL_ALPHA_ROMAN", "NUMBERED_DECIMAL_NESTED", "NUMBERED_UPPERALPHA_ALPHA_ROMAN"],
"description": "Bullet style preset (default: BULLET_DISC_CIRCLE_SQUARE)", "description": "Bullet style preset (default: BULLET_DISC_CIRCLE_SQUARE). Used by: create_list",
"default": "BULLET_DISC_CIRCLE_SQUARE" "default": "BULLET_DISC_CIRCLE_SQUARE"
}
},
"required": ["action", "document_id", "start_index", "end_index"]
},
{
"properties": {
"action": { "const": "batch_update" },
"document_id": {
"type": "string",
"description": "The document ID"
}, },
"requests": { "requests": {
"type": "array", "type": "array",
"items": { "type": "object" }, "items": { "type": "object" },
"description": "Array of raw Docs API batchUpdate request objects" "description": "Array of raw Docs API batchUpdate request objects. Required for: batch_update"
} }
},
"required": ["action", "document_id", "requests"]
} }
]
}"# }"#
.to_string() .to_string()
} }
+27 -147
View File
@@ -62,215 +62,95 @@ impl exports::near::agent::tool::Guest for GoogleDriveTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "list_files" }, "action": {
"type": "string",
"enum": ["list_files", "get_file", "download_file", "upload_file", "update_file", "create_folder", "delete_file", "trash_file", "share_file", "list_permissions", "remove_permission", "list_shared_drives"],
"description": "The Google Drive operation to perform"
},
"file_id": {
"type": "string",
"description": "File ID. Required for: get_file, download_file, update_file, delete_file, trash_file, share_file, list_permissions, remove_permission"
},
"query": { "query": {
"type": "string", "type": "string",
"description": "Drive search query. Examples: \"name contains 'report'\", \"mimeType = 'application/pdf'\", \"'folderId' in parents\", \"sharedWithMe = true\"" "description": "Drive search query (e.g., \"name contains 'report'\", \"mimeType = 'application/pdf'\"). Used by: list_files"
}, },
"page_size": { "page_size": {
"type": "integer", "type": "integer",
"description": "Max results (default: 25, max: 1000)", "description": "Max results (default: 25, max: 1000). Used by: list_files, list_shared_drives",
"default": 25 "default": 25
}, },
"order_by": { "order_by": {
"type": "string", "type": "string",
"description": "Sort order (e.g., 'modifiedTime desc', 'name')" "description": "Sort order (e.g., 'modifiedTime desc', 'name'). Used by: list_files"
}, },
"corpora": { "corpora": {
"type": "string", "type": "string",
"enum": ["user", "drive", "domain", "allDrives"], "enum": ["user", "drive", "domain", "allDrives"],
"description": "Search scope: 'user' (personal, default), 'drive' (specific shared drive), 'domain' (org-wide), 'allDrives' (everything)", "description": "Search scope: 'user' (default), 'drive' (shared drive), 'domain', 'allDrives'. Used by: list_files",
"default": "user" "default": "user"
}, },
"drive_id": { "drive_id": {
"type": "string", "type": "string",
"description": "Shared drive ID (required when corpora is 'drive')" "description": "Shared drive ID (required when corpora is 'drive'). Used by: list_files"
}, },
"page_token": { "page_token": {
"type": "string", "type": "string",
"description": "Token for next page of results" "description": "Token for next page of results. Used by: list_files"
}
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_file" },
"file_id": {
"type": "string",
"description": "The file ID"
}
},
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "download_file" },
"file_id": {
"type": "string",
"description": "The file ID to download"
}, },
"export_mime_type": { "export_mime_type": {
"type": "string", "type": "string",
"description": "Export format for Google Workspace files (e.g., 'text/plain', 'text/csv', 'application/pdf')" "description": "Export format for Google Workspace files (e.g., 'text/plain', 'text/csv'). Used by: download_file"
}
}, },
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "upload_file" },
"name": { "name": {
"type": "string", "type": "string",
"description": "File name" "description": "File/folder name. Required for: upload_file, create_folder. Optional for: update_file"
}, },
"content": { "content": {
"type": "string", "type": "string",
"description": "File content (text)" "description": "File content (text). Required for: upload_file"
}, },
"mime_type": { "mime_type": {
"type": "string", "type": "string",
"description": "MIME type (default: 'text/plain')", "description": "MIME type (default: 'text/plain'). Used by: upload_file",
"default": "text/plain" "default": "text/plain"
}, },
"parent_id": { "parent_id": {
"type": "string", "type": "string",
"description": "Parent folder ID (omit for root)" "description": "Parent folder ID (omit for root). Used by: upload_file, create_folder"
}, },
"description": { "description": {
"type": "string", "type": "string",
"description": "File description" "description": "File/folder description. Used by: upload_file, update_file, create_folder"
}
},
"required": ["action", "name", "content"]
},
{
"properties": {
"action": { "const": "update_file" },
"file_id": {
"type": "string",
"description": "The file ID to update"
},
"name": {
"type": "string",
"description": "New file name"
},
"description": {
"type": "string",
"description": "New description"
}, },
"move_to_parent": { "move_to_parent": {
"type": "string", "type": "string",
"description": "Move file to this folder ID" "description": "Move file to this folder ID. Used by: update_file"
}, },
"starred": { "starred": {
"type": "boolean", "type": "boolean",
"description": "Star or unstar the file" "description": "Star or unstar the file. Used by: update_file"
}
},
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "create_folder" },
"name": {
"type": "string",
"description": "Folder name"
},
"parent_id": {
"type": "string",
"description": "Parent folder ID (omit for root)"
},
"description": {
"type": "string",
"description": "Folder description"
}
},
"required": ["action", "name"]
},
{
"properties": {
"action": { "const": "delete_file" },
"file_id": {
"type": "string",
"description": "The file ID to permanently delete"
}
},
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "trash_file" },
"file_id": {
"type": "string",
"description": "The file ID to move to trash"
}
},
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "share_file" },
"file_id": {
"type": "string",
"description": "The file ID to share"
}, },
"email": { "email": {
"type": "string", "type": "string",
"description": "Recipient email address" "description": "Recipient email address. Required for: share_file"
}, },
"role": { "role": {
"type": "string", "type": "string",
"enum": ["reader", "commenter", "writer", "organizer"], "enum": ["reader", "commenter", "writer", "organizer"],
"description": "Permission level (default: 'reader')", "description": "Permission level (default: 'reader'). Used by: share_file",
"default": "reader" "default": "reader"
}, },
"message": { "message": {
"type": "string", "type": "string",
"description": "Optional message in sharing notification" "description": "Optional message in sharing notification. Used by: share_file"
}
},
"required": ["action", "file_id", "email"]
},
{
"properties": {
"action": { "const": "list_permissions" },
"file_id": {
"type": "string",
"description": "The file ID to check permissions for"
}
},
"required": ["action", "file_id"]
},
{
"properties": {
"action": { "const": "remove_permission" },
"file_id": {
"type": "string",
"description": "The file ID"
}, },
"permission_id": { "permission_id": {
"type": "string", "type": "string",
"description": "The permission ID to remove (get from list_permissions)" "description": "Permission ID to remove (from list_permissions). Required for: remove_permission"
} }
},
"required": ["action", "file_id", "permission_id"]
},
{
"properties": {
"action": { "const": "list_shared_drives" },
"page_size": {
"type": "integer",
"description": "Max results (default: 25)",
"default": 25
} }
},
"required": ["action"]
}
]
}"# }"#
.to_string() .to_string()
} }
+7 -1
View File
@@ -330,7 +330,13 @@ pub fn add_sheet(spreadsheet_id: &str, title: &str) -> Result<AddSheetResult, St
let parsed = batch_update(spreadsheet_id, requests)?; let parsed = batch_update(spreadsheet_id, requests)?;
let reply = &parsed["replies"][0]["addSheet"]["properties"]; let reply = parsed["replies"]
.as_array()
.and_then(|arr| arr.first())
.map(|r| &r["addSheet"]["properties"]);
let reply = reply.ok_or_else(|| "No reply from batch update".to_string())?;
Ok(AddSheetResult { Ok(AddSheetResult {
sheet: SheetInfo { sheet: SheetInfo {
sheet_id: reply["sheetId"].as_i64().unwrap_or(0), sheet_id: reply["sheetId"].as_i64().unwrap_or(0),
+28 -164
View File
@@ -70,236 +70,100 @@ impl exports::near::agent::tool::Guest for GoogleSheetsTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "create_spreadsheet" }, "action": {
"type": "string",
"enum": ["create_spreadsheet", "get_spreadsheet", "read_values", "batch_read_values", "write_values", "append_values", "clear_values", "add_sheet", "delete_sheet", "rename_sheet", "format_cells"],
"description": "The Google Sheets operation to perform"
},
"spreadsheet_id": {
"type": "string",
"description": "Spreadsheet ID (same as Google Drive file ID). Required for all actions except create_spreadsheet"
},
"title": { "title": {
"type": "string", "type": "string",
"description": "Spreadsheet title" "description": "Title/name. Required for: create_spreadsheet, add_sheet, rename_sheet"
}, },
"sheet_names": { "sheet_names": {
"type": "array", "type": "array",
"items": { "type": "string" }, "items": { "type": "string" },
"description": "Names for sheets (tabs). Defaults to ['Sheet1'] if omitted." "description": "Names for sheets (tabs, defaults to ['Sheet1']). Used by: create_spreadsheet"
}
},
"required": ["action", "title"]
},
{
"properties": {
"action": { "const": "get_spreadsheet" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID (same as Google Drive file ID)"
}
},
"required": ["action", "spreadsheet_id"]
},
{
"properties": {
"action": { "const": "read_values" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
}, },
"range": { "range": {
"type": "string", "type": "string",
"description": "A1 notation range (e.g., 'Sheet1!A1:D10', 'A1:B5')" "description": "A1 notation range (e.g., 'Sheet1!A1:D10'). Required for: read_values, write_values, append_values, clear_values"
}
},
"required": ["action", "spreadsheet_id", "range"]
},
{
"properties": {
"action": { "const": "batch_read_values" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
}, },
"ranges": { "ranges": {
"type": "array", "type": "array",
"items": { "type": "string" }, "items": { "type": "string" },
"description": "List of A1 notation ranges to read" "description": "List of A1 notation ranges. Required for: batch_read_values"
}
},
"required": ["action", "spreadsheet_id", "ranges"]
},
{
"properties": {
"action": { "const": "write_values" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"range": {
"type": "string",
"description": "A1 notation range (e.g., 'Sheet1!A1')"
}, },
"values": { "values": {
"type": "array", "type": "array",
"items": { "type": "array" }, "items": { "type": "array" },
"description": "2D array of values (rows of columns)" "description": "2D array of values (rows of columns). Required for: write_values, append_values"
}, },
"value_input_option": { "value_input_option": {
"type": "string", "type": "string",
"enum": ["RAW", "USER_ENTERED"], "enum": ["RAW", "USER_ENTERED"],
"description": "How to interpret input. USER_ENTERED (default) parses like typing in the UI. RAW stores as-is.", "description": "How to interpret input (USER_ENTERED parses like the UI, RAW stores as-is, default: USER_ENTERED). Used by: write_values, append_values",
"default": "USER_ENTERED" "default": "USER_ENTERED"
}
},
"required": ["action", "spreadsheet_id", "range", "values"]
},
{
"properties": {
"action": { "const": "append_values" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"range": {
"type": "string",
"description": "A1 notation range to find the table (e.g., 'Sheet1!A:E')"
},
"values": {
"type": "array",
"items": { "type": "array" },
"description": "Rows to append (2D array)"
},
"value_input_option": {
"type": "string",
"enum": ["RAW", "USER_ENTERED"],
"description": "How to interpret input (default: USER_ENTERED)",
"default": "USER_ENTERED"
}
},
"required": ["action", "spreadsheet_id", "range", "values"]
},
{
"properties": {
"action": { "const": "clear_values" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"range": {
"type": "string",
"description": "A1 notation range to clear"
}
},
"required": ["action", "spreadsheet_id", "range"]
},
{
"properties": {
"action": { "const": "add_sheet" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"title": {
"type": "string",
"description": "Name for the new sheet (tab)"
}
},
"required": ["action", "spreadsheet_id", "title"]
},
{
"properties": {
"action": { "const": "delete_sheet" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
}, },
"sheet_id": { "sheet_id": {
"type": "integer", "type": "integer",
"description": "Numeric sheet ID (get from get_spreadsheet, NOT the sheet name)" "description": "Numeric sheet ID (from get_spreadsheet, NOT the sheet name). Required for: delete_sheet, rename_sheet, format_cells"
}
},
"required": ["action", "spreadsheet_id", "sheet_id"]
},
{
"properties": {
"action": { "const": "rename_sheet" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"sheet_id": {
"type": "integer",
"description": "Numeric sheet ID"
},
"title": {
"type": "string",
"description": "New name for the sheet"
}
},
"required": ["action", "spreadsheet_id", "sheet_id", "title"]
},
{
"properties": {
"action": { "const": "format_cells" },
"spreadsheet_id": {
"type": "string",
"description": "The spreadsheet ID"
},
"sheet_id": {
"type": "integer",
"description": "Numeric sheet ID"
}, },
"start_row": { "start_row": {
"type": "integer", "type": "integer",
"description": "Start row (0-indexed, inclusive)" "description": "Start row (0-indexed, inclusive). Required for: format_cells"
}, },
"end_row": { "end_row": {
"type": "integer", "type": "integer",
"description": "End row (0-indexed, exclusive)" "description": "End row (0-indexed, exclusive). Required for: format_cells"
}, },
"start_column": { "start_column": {
"type": "integer", "type": "integer",
"description": "Start column (0-indexed, inclusive)" "description": "Start column (0-indexed, inclusive). Required for: format_cells"
}, },
"end_column": { "end_column": {
"type": "integer", "type": "integer",
"description": "End column (0-indexed, exclusive)" "description": "End column (0-indexed, exclusive). Required for: format_cells"
}, },
"bold": { "bold": {
"type": "boolean", "type": "boolean",
"description": "Make text bold" "description": "Make text bold. Used by: format_cells"
}, },
"italic": { "italic": {
"type": "boolean", "type": "boolean",
"description": "Make text italic" "description": "Make text italic. Used by: format_cells"
}, },
"font_size": { "font_size": {
"type": "integer", "type": "integer",
"description": "Font size in points" "description": "Font size in points. Used by: format_cells"
}, },
"text_color": { "text_color": {
"type": "string", "type": "string",
"description": "Text color as hex (e.g., '#FF0000' for red)" "description": "Text color as hex (e.g., '#FF0000'). Used by: format_cells"
}, },
"background_color": { "background_color": {
"type": "string", "type": "string",
"description": "Cell background color as hex (e.g., '#FFFF00' for yellow)" "description": "Cell background color as hex (e.g., '#FFFF00'). Used by: format_cells"
}, },
"horizontal_alignment": { "horizontal_alignment": {
"type": "string", "type": "string",
"enum": ["LEFT", "CENTER", "RIGHT"], "enum": ["LEFT", "CENTER", "RIGHT"],
"description": "Horizontal text alignment" "description": "Horizontal text alignment. Used by: format_cells"
}, },
"number_format": { "number_format": {
"type": "string", "type": "string",
"description": "Number format pattern (e.g., '#,##0.00', 'yyyy-mm-dd', '$#,##0')" "description": "Number format pattern (e.g., '#,##0.00', 'yyyy-mm-dd'). Used by: format_cells"
}, },
"number_format_type": { "number_format_type": {
"type": "string", "type": "string",
"enum": ["NUMBER", "CURRENCY", "PERCENT", "DATE", "TIME", "TEXT"], "enum": ["NUMBER", "CURRENCY", "PERCENT", "DATE", "TIME", "TEXT"],
"description": "Type of number format (default: NUMBER)" "description": "Type of number format (default: NUMBER). Used by: format_cells"
} }
},
"required": ["action", "spreadsheet_id", "sheet_id", "start_row", "end_row", "start_column", "end_column"]
} }
]
}"# }"#
.to_string() .to_string()
} }
+35 -237
View File
@@ -79,326 +79,124 @@ impl exports::near::agent::tool::Guest for GoogleSlidesTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "create_presentation" }, "action": {
"type": "string",
"enum": ["create_presentation", "get_presentation", "get_thumbnail", "create_slide", "delete_object", "insert_text", "delete_text", "replace_all_text", "create_shape", "insert_image", "format_text", "format_paragraph", "replace_shapes_with_image", "batch_update"],
"description": "The Google Slides operation to perform"
},
"title": { "title": {
"type": "string", "type": "string",
"description": "Presentation title" "description": "Presentation title. Required for: create_presentation"
}
}, },
"required": ["action", "title"]
},
{
"properties": {
"action": { "const": "get_presentation" },
"presentation_id": { "presentation_id": {
"type": "string", "type": "string",
"description": "The presentation ID (same as Google Drive file ID)" "description": "Presentation ID (same as Google Drive file ID). Required for all actions except create_presentation"
}
},
"required": ["action", "presentation_id"]
},
{
"properties": {
"action": { "const": "get_thumbnail" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
}, },
"slide_object_id": { "slide_object_id": {
"type": "string", "type": "string",
"description": "The slide's object ID" "description": "Slide object ID. Required for: get_thumbnail, create_shape, insert_image"
}
}, },
"required": ["action", "presentation_id", "slide_object_id"] "object_id": {
},
{
"properties": {
"action": { "const": "create_slide" },
"presentation_id": {
"type": "string", "type": "string",
"description": "The presentation ID" "description": "Object ID of a slide element. Required for: delete_object, insert_text, delete_text, format_text, format_paragraph"
},
"text": {
"type": "string",
"description": "Text to insert. Required for: insert_text"
}, },
"insertion_index": { "insertion_index": {
"type": "integer", "type": "integer",
"description": "Position to insert (0-based). Omit to append at end." "description": "Position to insert at (0-based). Used by: create_slide (omit to append at end), insert_text (default: 0)"
}, },
"layout": { "layout": {
"type": "string", "type": "string",
"enum": ["BLANK", "TITLE", "TITLE_AND_BODY", "TITLE_AND_TWO_COLUMNS", "TITLE_ONLY", "SECTION_HEADER", "CAPTION_ONLY", "BIG_NUMBER", "ONE_COLUMN_TEXT", "MAIN_POINT"], "enum": ["BLANK", "TITLE", "TITLE_AND_BODY", "TITLE_AND_TWO_COLUMNS", "TITLE_ONLY", "SECTION_HEADER", "CAPTION_ONLY", "BIG_NUMBER", "ONE_COLUMN_TEXT", "MAIN_POINT"],
"description": "Predefined layout (default: BLANK)", "description": "Predefined slide layout (default: BLANK). Used by: create_slide",
"default": "BLANK" "default": "BLANK"
}
},
"required": ["action", "presentation_id"]
},
{
"properties": {
"action": { "const": "delete_object" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"object_id": {
"type": "string",
"description": "Object ID of the slide or element to delete"
}
},
"required": ["action", "presentation_id", "object_id"]
},
{
"properties": {
"action": { "const": "insert_text" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"object_id": {
"type": "string",
"description": "Object ID of the shape or text box"
},
"text": {
"type": "string",
"description": "Text to insert"
},
"insertion_index": {
"type": "integer",
"description": "Character index to insert at (0-based). Default: 0.",
"default": 0
}
},
"required": ["action", "presentation_id", "object_id", "text"]
},
{
"properties": {
"action": { "const": "delete_text" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"object_id": {
"type": "string",
"description": "Object ID of the shape"
}, },
"start_index": { "start_index": {
"type": "integer", "type": "integer",
"description": "Start index (inclusive, 0-based)", "description": "Start index (inclusive, 0-based). Used by: delete_text, format_text, format_paragraph"
"default": 0
}, },
"end_index": { "end_index": {
"type": "integer", "type": "integer",
"description": "End index (exclusive). Omit to delete from start_index to end." "description": "End index (exclusive). Used by: delete_text, format_text, format_paragraph"
}
},
"required": ["action", "presentation_id", "object_id"]
},
{
"properties": {
"action": { "const": "replace_all_text" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
}, },
"find": { "find": {
"type": "string", "type": "string",
"description": "Text to search for" "description": "Text to search for. Required for: replace_all_text, replace_shapes_with_image"
}, },
"replace": { "replace": {
"type": "string", "type": "string",
"description": "Replacement text" "description": "Replacement text. Required for: replace_all_text"
}, },
"match_case": { "match_case": {
"type": "boolean", "type": "boolean",
"description": "Case-sensitive match (default: true)", "description": "Case-sensitive match (default: true). Used by: replace_all_text, replace_shapes_with_image",
"default": true "default": true
}
},
"required": ["action", "presentation_id", "find", "replace"]
},
{
"properties": {
"action": { "const": "create_shape" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"slide_object_id": {
"type": "string",
"description": "Slide object ID to place the shape on"
}, },
"shape_type": { "shape_type": {
"type": "string", "type": "string",
"enum": ["TEXT_BOX", "RECTANGLE", "ROUND_RECTANGLE", "ELLIPSE"], "enum": ["TEXT_BOX", "RECTANGLE", "ROUND_RECTANGLE", "ELLIPSE"],
"description": "Shape type (default: TEXT_BOX)", "description": "Shape type (default: TEXT_BOX). Used by: create_shape",
"default": "TEXT_BOX" "default": "TEXT_BOX"
}, },
"x": { "x": {
"type": "number", "type": "number",
"description": "X position in points from left edge" "description": "X position in points from left edge. Required for: create_shape, insert_image"
}, },
"y": { "y": {
"type": "number", "type": "number",
"description": "Y position in points from top edge" "description": "Y position in points from top edge. Required for: create_shape, insert_image"
}, },
"width": { "width": {
"type": "number", "type": "number",
"description": "Width in points" "description": "Width in points. Required for: create_shape, insert_image"
}, },
"height": { "height": {
"type": "number", "type": "number",
"description": "Height in points" "description": "Height in points. Required for: create_shape, insert_image"
}
},
"required": ["action", "presentation_id", "slide_object_id", "x", "y", "width", "height"]
},
{
"properties": {
"action": { "const": "insert_image" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"slide_object_id": {
"type": "string",
"description": "Slide object ID to place the image on"
}, },
"image_url": { "image_url": {
"type": "string", "type": "string",
"description": "Publicly accessible image URL" "description": "Publicly accessible image URL. Required for: insert_image, replace_shapes_with_image"
},
"x": {
"type": "number",
"description": "X position in points"
},
"y": {
"type": "number",
"description": "Y position in points"
},
"width": {
"type": "number",
"description": "Width in points"
},
"height": {
"type": "number",
"description": "Height in points"
}
},
"required": ["action", "presentation_id", "slide_object_id", "image_url", "x", "y", "width", "height"]
},
{
"properties": {
"action": { "const": "format_text" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"object_id": {
"type": "string",
"description": "Object ID of the shape"
},
"start_index": {
"type": "integer",
"description": "Start index (inclusive). Omit to format all text."
},
"end_index": {
"type": "integer",
"description": "End index (exclusive). Omit to format to end."
}, },
"bold": { "bold": {
"type": "boolean", "type": "boolean",
"description": "Make text bold" "description": "Make text bold. Used by: format_text"
}, },
"italic": { "italic": {
"type": "boolean", "type": "boolean",
"description": "Make text italic" "description": "Make text italic. Used by: format_text"
}, },
"underline": { "underline": {
"type": "boolean", "type": "boolean",
"description": "Underline text" "description": "Underline text. Used by: format_text"
}, },
"font_size": { "font_size": {
"type": "number", "type": "number",
"description": "Font size in points (e.g., 12, 18, 24)" "description": "Font size in points (e.g., 12, 18, 24). Used by: format_text"
}, },
"font_family": { "font_family": {
"type": "string", "type": "string",
"description": "Font family (e.g., 'Arial', 'Roboto', 'Times New Roman')" "description": "Font family (e.g., 'Arial', 'Roboto'). Used by: format_text"
}, },
"foreground_color": { "foreground_color": {
"type": "string", "type": "string",
"description": "Text color as hex (e.g., '#FF0000' for red)" "description": "Text color as hex (e.g., '#FF0000'). Used by: format_text"
}
},
"required": ["action", "presentation_id", "object_id"]
},
{
"properties": {
"action": { "const": "format_paragraph" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"object_id": {
"type": "string",
"description": "Object ID of the shape"
}, },
"alignment": { "alignment": {
"type": "string", "type": "string",
"enum": ["START", "CENTER", "END", "JUSTIFIED"], "enum": ["START", "CENTER", "END", "JUSTIFIED"],
"description": "Paragraph alignment" "description": "Paragraph alignment. Required for: format_paragraph"
},
"start_index": {
"type": "integer",
"description": "Start index (inclusive). Omit to format all."
},
"end_index": {
"type": "integer",
"description": "End index (exclusive). Omit to format to end."
}
},
"required": ["action", "presentation_id", "object_id", "alignment"]
},
{
"properties": {
"action": { "const": "replace_shapes_with_image" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
},
"find": {
"type": "string",
"description": "Text to match in shapes"
},
"image_url": {
"type": "string",
"description": "Image URL to replace matched shapes with"
},
"match_case": {
"type": "boolean",
"description": "Case-sensitive match (default: true)",
"default": true
}
},
"required": ["action", "presentation_id", "find", "image_url"]
},
{
"properties": {
"action": { "const": "batch_update" },
"presentation_id": {
"type": "string",
"description": "The presentation ID"
}, },
"requests": { "requests": {
"type": "array", "type": "array",
"items": { "type": "object" }, "items": { "type": "object" },
"description": "Array of raw Slides API batchUpdate request objects" "description": "Array of raw Slides API batchUpdate request objects. Required for: batch_update"
} }
},
"required": ["action", "presentation_id", "requests"]
} }
]
}"# }"#
.to_string() .to_string()
} }
+7 -38
View File
@@ -54,56 +54,25 @@ impl exports::near::agent::tool::Guest for OktaTool {
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "get_profile" } "action": {
"type": "string",
"enum": ["get_profile", "update_profile", "list_apps", "search_apps", "get_app_sso_link", "get_org_info"],
"description": "The Okta operation to perform"
}, },
"required": ["action"]
},
{
"properties": {
"action": { "const": "update_profile" },
"fields": { "fields": {
"type": "object", "type": "object",
"description": "Profile fields to update. Common: firstName, lastName, email, mobilePhone, displayName, nickName, title, department, organization" "description": "Profile fields to update (e.g., firstName, lastName, email, mobilePhone, displayName, title, department). Required for: update_profile"
}
}, },
"required": ["action", "fields"]
},
{
"properties": {
"action": { "const": "list_apps" }
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "search_apps" },
"query": { "query": {
"type": "string", "type": "string",
"description": "Case-insensitive search query to match against app labels and names" "description": "Case-insensitive search query to match against app labels and names. Required for: search_apps"
}
}, },
"required": ["action", "query"]
},
{
"properties": {
"action": { "const": "get_app_sso_link" },
"app": { "app": {
"type": "string", "type": "string",
"description": "App instance ID (e.g., '0oa1xxx') or app label (e.g., 'Google Workspace')" "description": "App instance ID (e.g., '0oa1xxx') or app label (e.g., 'Google Workspace'). Required for: get_app_sso_link"
} }
},
"required": ["action", "app"]
},
{
"properties": {
"action": { "const": "get_org_info" }
},
"required": ["action"]
} }
]
}"# }"#
.to_string() .to_string()
} }
+12 -52
View File
@@ -53,84 +53,44 @@ impl exports::near::agent::tool::Guest for SlackTool {
} }
fn schema() -> String { fn schema() -> String {
// JSON Schema for the tool's parameters
r#"{ r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "send_message" }, "action": {
"type": "string",
"enum": ["send_message", "list_channels", "get_channel_history", "post_reaction", "get_user_info"],
"description": "The Slack operation to perform"
},
"channel": { "channel": {
"type": "string", "type": "string",
"description": "Channel ID or name (e.g., '#general' or 'C1234567890')" "description": "Channel ID or name (e.g., '#general' or 'C1234567890'). Required for: send_message, get_channel_history, post_reaction"
}, },
"text": { "text": {
"type": "string", "type": "string",
"description": "Message text (supports Slack mrkdwn formatting)" "description": "Message text (supports Slack mrkdwn formatting). Required for: send_message"
}, },
"thread_ts": { "thread_ts": {
"type": "string", "type": "string",
"description": "Optional thread timestamp to reply in a thread" "description": "Thread timestamp to reply in a thread. Used by: send_message"
}
},
"required": ["action", "channel", "text"]
},
{
"properties": {
"action": { "const": "list_channels" },
"limit": {
"type": "integer",
"description": "Maximum number of channels to return (default: 100)",
"default": 100
}
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_channel_history" },
"channel": {
"type": "string",
"description": "Channel ID (e.g., 'C1234567890')"
}, },
"limit": { "limit": {
"type": "integer", "type": "integer",
"description": "Maximum number of messages to return (default: 20)", "description": "Maximum number of results to return. Used by: list_channels, get_channel_history"
"default": 20
}
},
"required": ["action", "channel"]
},
{
"properties": {
"action": { "const": "post_reaction" },
"channel": {
"type": "string",
"description": "Channel ID containing the message"
}, },
"timestamp": { "timestamp": {
"type": "string", "type": "string",
"description": "Timestamp of the message to react to" "description": "Timestamp of the message to react to. Required for: post_reaction"
}, },
"emoji": { "emoji": {
"type": "string", "type": "string",
"description": "Emoji name without colons (e.g., 'thumbsup')" "description": "Emoji name without colons (e.g., 'thumbsup'). Required for: post_reaction"
}
}, },
"required": ["action", "channel", "timestamp", "emoji"]
},
{
"properties": {
"action": { "const": "get_user_info" },
"user_id": { "user_id": {
"type": "string", "type": "string",
"description": "User ID (e.g., 'U1234567890')" "description": "User ID (e.g., 'U1234567890'). Required for: get_user_info"
} }
},
"required": ["action", "user_id"]
} }
]
}"# }"#
.to_string() .to_string()
} }
+17 -107
View File
@@ -248,154 +248,64 @@ fn get_api_hash() -> Result<String, String> {
const SCHEMA: &str = r#"{ const SCHEMA: &str = r#"{
"type": "object", "type": "object",
"required": ["action"], "required": ["action"],
"oneOf": [
{
"properties": { "properties": {
"action": { "const": "login" }, "action": {
"type": "string",
"enum": ["login", "submit_auth_code", "submit_2fa_password", "get_me", "get_contacts", "get_chats", "get_messages", "send_message", "forward_message", "delete_message", "search_messages", "get_updates"],
"description": "The Telegram operation to perform"
},
"phone_number": { "phone_number": {
"type": "string", "type": "string",
"description": "Phone number in international format (e.g., '+1234567890')" "description": "Phone number in international format (e.g., '+1234567890'). Required for: login"
}
}, },
"required": ["action", "phone_number"]
},
{
"properties": {
"action": { "const": "submit_auth_code" },
"code": { "code": {
"type": "string", "type": "string",
"description": "Verification code received via SMS or Telegram" "description": "Verification code received via SMS or Telegram. Required for: submit_auth_code"
}
}, },
"required": ["action", "code"]
},
{
"properties": {
"action": { "const": "submit_2fa_password" },
"password": { "password": {
"type": "string", "type": "string",
"description": "Two-factor authentication password" "description": "Two-factor authentication password. Required for: submit_2fa_password"
}
}, },
"required": ["action", "password"]
},
{
"properties": {
"action": { "const": "get_me" }
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_contacts" }
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_chats" },
"limit": {
"type": "integer",
"description": "Maximum number of chats to return (default: 20)",
"default": 20
}
},
"required": ["action"]
},
{
"properties": {
"action": { "const": "get_messages" },
"chat_id": { "chat_id": {
"type": "integer", "type": "integer",
"description": "Chat ID (negative for groups/channels)" "description": "Chat ID (negative for groups/channels). Required for: get_messages, send_message. Optional for: search_messages"
}, },
"limit": { "limit": {
"type": "integer", "type": "integer",
"description": "Maximum number of messages (default: 20)", "description": "Maximum number of results (default: 20). Used by: get_chats, get_messages, search_messages",
"default": 20 "default": 20
}, },
"from_message_id": { "from_message_id": {
"type": "integer", "type": "integer",
"description": "Start from this message ID for pagination" "description": "Start from this message ID for pagination. Used by: get_messages"
}
},
"required": ["action", "chat_id"]
},
{
"properties": {
"action": { "const": "send_message" },
"chat_id": {
"type": "integer",
"description": "Chat ID to send the message to"
}, },
"text": { "text": {
"type": "string", "type": "string",
"description": "Message text" "description": "Message text. Required for: send_message"
}
}, },
"required": ["action", "chat_id", "text"]
},
{
"properties": {
"action": { "const": "forward_message" },
"from_chat_id": { "from_chat_id": {
"type": "integer", "type": "integer",
"description": "Source chat ID" "description": "Source chat ID. Required for: forward_message"
}, },
"to_chat_id": { "to_chat_id": {
"type": "integer", "type": "integer",
"description": "Destination chat ID" "description": "Destination chat ID. Required for: forward_message"
}, },
"message_ids": { "message_ids": {
"type": "array", "type": "array",
"items": { "type": "integer" }, "items": { "type": "integer" },
"description": "Message IDs to forward" "description": "Message IDs. Required for: forward_message, delete_message"
}
},
"required": ["action", "from_chat_id", "to_chat_id", "message_ids"]
},
{
"properties": {
"action": { "const": "delete_message" },
"message_ids": {
"type": "array",
"items": { "type": "integer" },
"description": "Message IDs to delete"
}, },
"revoke": { "revoke": {
"type": "boolean", "type": "boolean",
"description": "Also delete for other participants (default: false)", "description": "Also delete for other participants (default: false). Used by: delete_message",
"default": false "default": false
}
}, },
"required": ["action", "message_ids"]
},
{
"properties": {
"action": { "const": "search_messages" },
"query": { "query": {
"type": "string", "type": "string",
"description": "Search query" "description": "Search query. Required for: search_messages"
},
"chat_id": {
"type": "integer",
"description": "Chat ID to search within (omit for global search)"
},
"limit": {
"type": "integer",
"description": "Maximum number of results (default: 20)",
"default": 20
} }
},
"required": ["action", "query"]
},
{
"properties": {
"action": { "const": "get_updates" }
},
"required": ["action"]
} }
]
}"#; }"#;
export!(TelegramTool); export!(TelegramTool);