Compare commits

...
Author SHA1 Message Date
[email protected]andClaude Opus 4.6 86853244ca docs: add Gemini OAuth env vars to .env.example [skip-regression-check]
Document GEMINI_MODEL, GEMINI_CREDENTIALS_PATH, GEMINI_API_KEY, and
all extended generation config env vars in the example config file.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 21:59:21 -07:00
[email protected]andClaude Opus 4.6 dd9057d069 fix(llm): support smart routing cheap model for gemini_oauth backend
Add explicit gemini_oauth handling in create_cheap_provider_for_backend()
to create a GeminiOauthProvider with the cheap model swapped in. Without
this, setting LLM_CHEAP_MODEL with gemini_oauth backend would fail with
a confusing "no registry provider config available" error.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 20:58:57 -07:00
[email protected]andClaude Opus 4.6 3a72b71d97 style(gemini_oauth): rustfmt formatting [skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 15:37:42 -07:00
[email protected]andClaude Opus 4.6 94f88231a1 fix(gemini_oauth): curate_contents per-part filtering and dead code removal
Fix curate_contents to filter invalid parts individually instead of
dropping entire model turn sequences. Previously a single empty text
part would discard all consecutive model turns including valid
functionCall parts, breaking the tool-call flow.

Also remove unused MID_STREAM_* constants.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 15:37:10 -07:00
[email protected]andClaude Opus 4.6 99566bb997 fix(gemini_oauth): align header parser doc with implementation [skip-regression-check]
Update parse_custom_headers doc comments to include underscore in the
header-name character class, matching the actual implementation.
Also fix formatting from merge.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 15:26:50 -07:00
[email protected]andClaude Opus 4.6 cbf96f25c9 merge: resolve origin/staging into feat/gemini-cli-oauth
Merge staging to pick up GitHub Copilot provider, OpenAI Codex provider,
and other recent changes. Both gemini_oauth and openai_codex backends are
now registered as dedicated configs with proper credential guards.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 15:18:13 -07:00
b97d82dbe6 feat(extensions): support text setup fields in web configure modal (#496)
* feat(extensions): support text setup fields in web configure modal

* fix(extensions): use exported wasm setup schema types

* fix(extensions): validate extension name in setup APIs

* fix(extensions): restrict setup setting_path writes

* refactor(web): use enum for setup field input type

* fix: restore registry versions reverted during merge [skip-regression-check]

The merge auto-resolved registry JSON conflicts in favor of the PR's
older 0.2.0 versions. Restore discord, github, and web-search to
0.2.1 from staging.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: 您的GitHub用户名 <[email protected]>
Co-authored-by: [email protected] <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 15:10:09 -07:00
9d538136b5 fix(oauth): reject malformed ic2.* states in decode_hosted_oauth_state (#1441) (#1454)
* fix(oauth): reject malformed ic2.* states instead of falling through to legacy handler (#1441)

When decode_hosted_oauth_state() encountered a versioned state (ic2.*)
that failed to fully parse (bad base64, invalid JSON, missing separator),
it silently fell through to legacy handling which used the full malformed
envelope as the flow_id. This never matched the raw nonce stored in
pending_oauth_flows, breaking the OAuth callback.

Restructure the versioned decode path so any ic2.* state must parse as a
valid envelope or return Err — never fall through to legacy handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(oauth): address PR review — avoid alloc in strip_prefix, strengthen JSON parse test

- Replace `strip_prefix(&format!(...))` with a `HOSTED_STATE_PREFIX_DOT`
  constant to avoid per-call allocation.
- Fix "valid base64 but not JSON" test to compute the correct checksum so
  it actually exercises the JSON parse error path instead of stopping at
  the checksum check.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: add missing fallback_deliverable field in job_monitor tests

The SseEvent::JobResult struct gained a fallback_deliverable field in
the structured fallback deliverables feature, but the job_monitor test
constructors were not updated.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix(oauth): remove HOSTED_STATE_PREFIX_DOT to avoid drift with HOSTED_STATE_PREFIX

concat! requires literals and cannot reference const items, so a
separate _DOT constant would duplicate the prefix string. Revert to
deriving the dotted prefix via format!() — both encode and decode now
use the same single HOSTED_STATE_PREFIX constant, keeping them
mechanically consistent.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
2026-03-21 14:39:52 -07:00
8ad7d78a70 fix: parameter coercion and validation for oneOf/anyOf/allOf schemas (#1397)
* fix: parameter coercion and validation for oneOf/anyOf/allOf schemas

WASM extension tools with multi-action schemas (e.g. github extension)
fail when the LLM passes numeric parameters as strings because the
coercion layer skips JSON Schema combinators. This causes serde
deserialization errors like `invalid type: string "100", expected u32`.

Add discriminated-union resolution to the coercion layer: for oneOf/anyOf,
match the active variant by const or single-element enum discriminators;
for allOf, merge all variants' properties. Also propagate combinator
awareness to schema validators, WASM wrapper helpers, and tool discovery
so they no longer reject or ignore valid combinator-based schemas.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add e2e tests for oneOf discriminated union parameter coercion

Add three end-to-end tests using a fixture tool that mirrors the github
WASM tool's oneOf schema with #[serde(tag = "action")] deserialization.
Each test sends string-typed numeric/boolean params through the full
agent loop, verifying that coercion resolves them before serde runs:

- list_issues: limit "100" → 100 (integer in oneOf variant)
- get_issue: issue_number "42" → 42 (integer in different variant)
- create_pull_request: draft "true" → true (boolean in variant)

Without the coercion fix these fail with:
  invalid type: string "100", expected u32

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test: add real WASM github tool e2e tests with HTTP interception

Load the actual compiled github WASM binary, send params with string-typed
numbers through the coercion layer, and verify the WASM tool constructs
correct HTTP API calls via a new HTTP interceptor in the WASM wrapper.

Changes:
- Add `http_interceptor` field to `StoreData` and `WasmToolWrapper` so
  WASM tool HTTP requests can be captured/mocked in tests
- Make `prepare_tool_params` and `coercion` module public for integration tests
- Add 3 e2e tests loading the real github WASM binary:
  - list_issues: `limit: "50"` → URL contains `per_page=50`
  - get_issue: `issue_number: "42"` → URL contains `/issues/42`
  - list_pull_requests: `limit: "25"` → URL contains `per_page=25`

Tests gracefully skip if the WASM binary isn't compiled.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor: simplify WASM e2e tests to use TestRig with with_wasm_tool()

Replace the manual WasmToolWrapper construction with TestRig integration:

- Add `with_wasm_tool(name, wasm_path, capabilities_path)` to TestRigBuilder
  that loads real WASM binaries and wires the shared HTTP interceptor
- Build the HTTP interceptor before tool registration so it can be shared
  between AgentDeps and WASM tool wrappers
- Rewrite github WASM e2e tests to use the standard trace pattern:
  TraceLlm sends tool calls with string params, http_exchanges specify
  expected outgoing requests and canned responses

The test code is now identical to other trace-based e2e tests — no custom
interceptors or manual WASM construction needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address review comments on combinator schema support

- Validate `has_combinators` checks array type (`.as_array().is_some()`)
  instead of bare `.is_some()` to reject malformed `{ "oneOf": {} }`
- Validate top-level `required` keys against merged combinator variant
  properties when no top-level `properties` exists (both validators)
- Deduplicate oneOf/anyOf handling into single loop in coercion.rs
- Revert `pub mod coercion` to private; only re-export `prepare_tool_params`
- Call `after_response` on interceptor after real HTTP when `before_request`
  returns None (recording mode correctness)
- Fix formatting (CI failure)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address second round of review comments

- Fix headers deserialization bug: deserialize resp.headers_json as
  HashMap<String, String> then convert to Vec, not directly as Vec
- Sort interceptor headers for deterministic trace fixtures
- Update after_response comment: RecordingHttpInterceptor does exercise
  this path (returns None from before_request)
- Mark WASM tests #[ignore] instead of silent skip — avoids false-green
  CI while keeping them runnable with --ignored
- Fix with_wasm_tool signature: Option<PathBuf> instead of
  Option<impl Into<PathBuf>> which doesn't compile in nested position
- Fix with_wasm_tool doc comment to match actual behavior
- Revert prepare_tool_params to pub(crate) — no longer needed publicly

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: coerce empty strings to null for optional tool parameters

LLMs often send "" instead of null/omitting optional parameters, causing
parse errors in tools that expect typed values (e.g., timezone, schedule).

PR #1127 fixed this per-field in the time tool. This commit adds
dispatcher-level coercion so all tools benefit:

- Non-required properties with value "" are coerced to null at the
  object level (based on the schema's `required` array)
- Explicitly nullable schemas (`type: ["string", "null"]`) coerce ""
  to null in the per-value coercion path
- Required string-only fields keep "" unchanged

Closes #755

Co-Authored-By: spiritj <[email protected]>
Co-Authored-By: Xing Ji <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* feat: complete coercion coverage for $ref, nested combinators, and additionalProperties

Close remaining coercion gaps so 3rd-party tools (MCP servers, complex
WASM tools) work correctly:

- $ref resolution: inline all #/definitions/<name> and #/$defs/<name>
  references in a pre-pass before coercion, with depth limit (16) for
  circular ref safety
- Nested combinators: resolve_effective_properties now recurses into
  variants that themselves contain allOf/oneOf/anyOf (depth limit 4)
- additionalProperties inheritance: check allOf variants and matched
  oneOf/anyOf variant for additionalProperties schemas

New tests:
- resolves_ref_and_coerces_referenced_properties
- resolves_nested_refs_in_oneof_variants
- coerces_nested_combinators_allof_containing_oneof
- coerces_array_items_with_oneof_discriminator
- circular_ref_does_not_infinite_loop

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address third round of review comments

- Validators: tighten has_combinators to require at least one object-typed
  variant (has type:"object" or properties), rejecting non-object combinator
  schemas like { "oneOf": [{"type":"integer"}] }
- Empty-string coercion: only coerce "" → null when schema allows null or
  doesn't allow string; pure type:"string" fields keep "" as meaningful
- Fix comment: "coerce to null" → "return unchanged" for empty strings
  with no type match (code returns None, not null)
- Redact credentials before passing to after_response interceptor to
  prevent secret leakage into recorded trace files
- Switch to tokio::fs::read for async WASM binary loading in test rig
- Add doc comment explaining soft URL check in WASM e2e tests

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* ci: retrigger after staging merge [skip-regression-check]

* fix: merge staging, report non-array combinator values as errors

Merge latest staging to fix CI (missing fallback_deliverable field).
Add explicit error reporting when oneOf/anyOf/allOf values are not
arrays in both strict and lenient validators.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: recurse into combinator variants that have properties but no explicit type

Both validators only recursed into variants with `type: "object"`,
missing variants that define `properties` without an explicit type
(common in allOf patterns). Now recurse when variant has either.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: spiritj <[email protected]>
Co-authored-by: Xing Ji <[email protected]>
2026-03-21 12:41:46 -07:00
Artem a070c069cb fix: add missing allow_always field after staging merge 2026-03-19 22:05:56 +03:00
Artem abf13a3ee2 Merge remote-tracking branch 'origin/staging' into feat/gemini-cli-oauth 2026-03-19 21:57:29 +03:00
Artem 21abbe5691 fix: address Copilot PR review feedback
- Fix empty text part for assistant messages with tool calls
  (curate_contents could drop entire model turn)
- Propagate cache_read/creation_input_tokens in complete_with_tools
- Log warning on save_credential failure instead of silently ignoring
- Fix doc comment to mention underscore in header name pattern
- Handle gemini-oauth (hyphen variant) in setup wizard display
- Fix docs: thinkingConfig uses thinkingBudget/thinkingLevel, not
  includeThoughts
2026-03-19 07:26:55 +03:00
Artem 9be29b2c22 fix: address PR review feedback from gemini-code-assist
- Fix parse_custom_headers to preserve commas in values by splitting
  only on commas followed by a header-name:colon pattern (manual scan
  instead of simple split(','))
- Use matches! macro for backend exclusion check in app.rs
- Merge SSE metadata extraction into single pass (was iterating twice)
- Replace fragile substring-based context_length with explicit match
  on known Gemini model IDs via gemini_context_length()
- Add missing models to regression test (8 models, not 5)
2026-03-18 14:27:12 +03:00
Artem 09a2320e64 fix: CI violations — add safety comment on expect, fix fmt
- Add '// safety: hardcoded literal' to regex .expect() to satisfy
  the no-panic-in-prod CI check
- Fix cargo fmt whitespace in collapsible if-let chain
2026-03-18 13:57:19 +03:00
Artem d4bfc2db58 feat(gemini_oauth): full Cloud Code API integration with project discovery
- Register gemini_oauth as a dedicated backend in config/llm.rs (skip
  registry fallback, preserve backend name, suppress unknown-backend warning)
- Fix app.rs credential guard to exclude backends with dedicated configs
  (gemini_oauth, bedrock) from the provider.is_none() check
- Auto-discover Cloud Code project_id via loadCodeAssist when credentials
  lack it (e.g. created by the original Gemini CLI)
- Persist discovered project_id to credentials file for subsequent runs
- Add safety settings (BLOCK_NONE), gated behind GEMINI_SAFETY_BLOCK_NONE env
- Add thinkingConfig: budget-based for Gemini 2.5, level-based for Gemini 3.x
  (without includeThoughts to avoid empty responses from reasoning.rs stripping)
- Add thought signature injection for Gemini 3.x preview APIs
- Add history curation to filter invalid model outputs before re-sending
- Add extended generationConfig env vars (topP, topK, seed, penalties,
  responseMimeType, responseJsonSchema, cachedContent)
- Add custom headers support via GEMINI_CLI_CUSTOM_HEADERS
- Add API key auth mode (GEMINI_API_KEY + GEMINI_API_KEY_AUTH_MECHANISM)
- Add SSE metadata extraction (modelVersion, credits, promptFeedback,
  groundingMetadata, citationMetadata, cachedContentTokenCount)
- Add countTokens API support
- Add new models to wizard (gemini-3.1-pro-preview-customtools,
  gemini-3-pro-preview, gemini-3.1-flash-lite-preview)
- Update docs/LLM_PROVIDERS.md with new models and routing rules
- Rewrite regression tests with comprehensive coverage (23 unit tests pass)
2026-03-18 13:53:58 +03:00
Artem 6e86f50bc5 Merge remote-tracking branch 'origin/main' into feat/gemini-cli-oauth 2026-03-17 18:21:02 +03:00
Artem 12b0e90a7a feat(gemini-oauth): implement code review v3 refinements
- Add force_refresh() for 401 retry (bypass timestamp check)
- Standardize Gemini model list across docs, wizard, and provider
- Restore gemini-3 check for thinkingConfig
- Redact sensitive tokens in GoogleTokenRefreshResponse Debug output
- Use dynamic version for GOOG_API_CLIENT
- Improve model_metadata() context length heuristics
- Use strip_prefix("data:") for safer SSE parsing
- Skip re-auth in wizard if keeping existing provider
2026-03-10 13:15:05 +03:00
Artem 4a2950e777 style: fix formatting in Gemini OAuth regression tests 2026-03-09 16:47:09 +03:00
Artem b35771d505 Add dedicated regression tests for Gemini OAuth fixes 2026-03-09 16:44:30 +03:00
Artem e4e747ba54 fix: address code review issues in gemini-cli OAuth integration
- Add cache_read_input_tokens/cache_creation_input_tokens fields (value 0)
- Implement manual Debug for OAuthCredential to redact tokens
- Fix hardcoded /tmp: use GeminiOauthConfig::default_credentials_path()
- Replace emoji output with plain text markers
- Propagate Client::builder() errors instead of silent fallback
- Use tokio::fs for all file I/O in CredentialManager (was std::fs)
- Use if let Some(ref pid) to avoid consuming credential.project_id
- Extract uses_cloud_code_api() helper; route by major version (gemini-2+)
- Concatenate multiple system messages into systemInstruction
- Include functionCall parts in assistant message conversion
- Add 401 retry loop with allow_retry flag for auth failures
- Remove biased from tokio::select! in OAuth callback handler
- Remove hardcoded context_length 1M; vary by model family
- Change GOOG_API_CLIENT from Node.js spoof to gl-rust/1.0.0
- Implement list_models() with static model list
- Move create_gemini_oauth_provider() before test module (clippy)
- Fix 9 additional clippy warnings (collapsible_if, map_or, needless_borrow)
- Run cargo fmt
2026-03-09 16:31:15 +03:00
Artem fdb0077736 Merge origin/main into feat/gemini-cli-oauth and resolve conflicts 2026-03-09 15:20:57 +03:00
Artem 8452102454 feat(gemini): implement function calling, generationConfig, and update models
- Implement function calling support (functionDeclarations, functionResponse)
- Add functionCall SSE parsing and empty stream retry support
- Add generationConfig (temperature, maxOutputTokens)
- Add thinkingConfig for Gemini 3 and thinking models
- Add toolConfig (functionCallingConfig.mode)
- Fix .expect() panics with .ok_or_else()
- Restrict oauth credentials file permissions to 0600
- Update docs and FEATURE_PARITY.md
- Update wizard to current Gemini 3.1 and 2.5 models
2026-03-04 23:05:16 +03:00
Artem 727283afe3 feat: integrate Gemini CLI OAuth with Cloud Code API
- Add gemini_oauth.rs: full OAuth flow with PKCE, token refresh,
  and Cloud Code project discovery (loadCodeAssist + onboardUser)
- Route preview/gemini-3 models through cloudcode-pa.googleapis.com
  with proper project ID injection in request payload
- Trigger OAuth login during onboarding wizard (not first chat message)
- Support manual redirect URL paste as fallback (tokio::select race)
- Parse 429 rate-limit errors with retry_after from Google response
- Add static model list: gemini-1.5/2.0/2.5/3.0/3.1 variants
- Add GeminiOauthConfig with default credentials path (~/.gemini/)
2026-03-02 23:50:45 +03:00
28 changed files with 5751 additions and 271 deletions
+18 -1
View File
@@ -4,7 +4,7 @@ DATABASE_POOL_SIZE=10
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, github_copilot, tinfoil, openai_codex
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, github_copilot, tinfoil, openai_codex, gemini_oauth
# LLM_REQUEST_TIMEOUT_SECS=120 # Increase for local LLMs (Ollama, vLLM, LM Studio)
# === Anthropic Direct ===
@@ -110,6 +110,23 @@ NEARAI_AUTH_URL=https://private.near.ai
# OPENAI_CODEX_AUTH_URL=https://auth.openai.com # override (rare)
# OPENAI_CODEX_API_URL=https://chatgpt.com/backend-api/codex # override (rare)
# === Google Gemini (OAuth, Gemini CLI compatible) ===
# LLM_BACKEND=gemini_oauth
# GEMINI_MODEL=gemini-2.5-flash # default
# GEMINI_CREDENTIALS_PATH=~/.gemini/oauth_creds.json # default
# GEMINI_API_KEY=... # optional: use API key instead of OAuth
# GEMINI_API_KEY_AUTH_MECHANISM=query # "query" (default) or "header"
# GEMINI_SAFETY_BLOCK_NONE=true # disable safety filters (default: false)
# GEMINI_CLI_CUSTOM_HEADERS=Key:Value,Key2:Value2
# GEMINI_TOP_P=0.95
# GEMINI_TOP_K=40
# GEMINI_SEED=42
# GEMINI_PRESENCE_PENALTY=0.0
# GEMINI_FREQUENCY_PENALTY=0.0
# GEMINI_RESPONSE_MIME_TYPE=application/json
# GEMINI_RESPONSE_JSON_SCHEMA={"type":"object"}
# GEMINI_CACHED_CONTENT=cachedContents/abc123
# For full provider setup guide see docs/LLM_PROVIDERS.md
# Channel Configuration
+14 -5
View File
@@ -3,6 +3,7 @@
This document tracks feature parity between IronClaw (Rust implementation) and OpenClaw (TypeScript reference implementation). Use this to coordinate work across developers.
**Legend:**
- ✅ Implemented
- 🚧 Partial (in progress or incomplete)
- ❌ Not implemented
@@ -204,7 +205,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Skills (modular capabilities) | ✅ | ✅ | Prompt-based skills with trust gating, attenuation, activation criteria, catalog, selector |
| Skill routing blocks | ✅ | 🚧 | ActivationCriteria (keywords, patterns, tags) but no "Use when / Don't use when" blocks |
| Skill path compaction | ✅ | ❌ | ~ prefix to reduce prompt tokens |
| Thinking modes (off/minimal/low/medium/high/xhigh/adaptive) | ✅ | | Configurable reasoning depth |
| Thinking modes (off/minimal/low/medium/high/xhigh/adaptive) | ✅ | 🚧 | thinkingConfig for Gemini models (thinkingBudget/thinkingLevel); no per-level control yet |
| Per-model thinkingDefault override | ✅ | ❌ | Override thinking level per model; Anthropic Claude 4.6 defaults to adaptive |
| Block-level streaming | ✅ | ❌ | |
| Tool-level streaming | ✅ | ❌ | |
@@ -236,9 +237,13 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6, adaptive thinking default |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy; GPT-5.4 + Codex OAuth |
| AWS Bedrock | ✅ | ❌ | P3 | |
| Google Gemini | ✅ | | P3 | |
| NVIDIA API | ✅ | | P3 | New provider |
| AWS Bedrock | ✅ | ✅ | - | Native Converse API via aws-sdk-bedrockruntime (requires `--features bedrock`) |
| Google Gemini | ✅ | | - | OAuth (PKCE + S256), function calling, thinkingConfig, generationConfig |
| io.net | ✅ | | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
@@ -466,7 +471,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Device pairing | ✅ | ❌ | |
| Tailscale identity | ✅ | ❌ | |
| Trusted-proxy auth | ✅ | ❌ | Header-based reverse proxy auth |
| OAuth flows | ✅ | 🚧 | NEAR AI OAuth plus hosted extension/MCP OAuth broker; external auth-proxy rollout still pending |
| OAuth flows | ✅ | 🚧 | NEAR AI OAuth + Gemini OAuth (PKCE, S256) + hosted extension/MCP OAuth broker; external auth-proxy rollout still pending |
| DM pairing verification | ✅ | ✅ | ironclaw pairing approve, host APIs |
| Allowlist/blocklist | ✅ | 🚧 | allow_from + pairing store |
| Per-group tool policies | ✅ | ❌ | |
@@ -523,6 +528,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
## Implementation Priorities
### P0 - Core (Already Done)
- ✅ TUI channel with approval overlays
- ✅ HTTP webhook channel
- ✅ DM pairing (ironclaw pairing list/approve, host APIs)
@@ -550,6 +556,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ✅ OpenAI-compatible / OpenRouter provider support
### P1 - High Priority
- ❌ Slack channel (real implementation)
- ✅ Telegram channel (WASM, DM pairing, caption, /start)
- ❌ WhatsApp channel
@@ -557,6 +564,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ✅ Hooks system (core lifecycle hooks + bundled/plugin/workspace hooks + outbound webhooks)
### P2 - Medium Priority
- ❌ Media handling (images, PDFs)
- ✅ Ollama/local model support (via rig::providers::ollama)
- ❌ Configuration hot-reload
@@ -565,6 +573,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
- ❌ Partial output preservation on abort
### P3 - Lower Priority
- ❌ Discord channel
- ❌ Matrix channel
- ❌ Other messaging platforms
+48 -3
View File
@@ -1,8 +1,8 @@
# LLM Provider Configuration
IronClaw defaults to NEAR AI for model access, but supports any OpenAI-compatible
endpoint as well as Anthropic and Ollama directly. This guide covers the most common
configurations.
endpoint as well as Anthropic, Ollama, and Google Gemini directly. This guide covers
the most common configurations.
## Provider Overview
@@ -11,7 +11,7 @@ configurations.
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| Google Gemini | `gemini_oauth` | OAuth (browser) | Gemini models; function calling |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
@@ -62,6 +62,51 @@ Popular models: `gpt-4o`, `gpt-4o-mini`, `o3-mini`
---
## Google Gemini (OAuth)
Uses Google OAuth with PKCE (S256) for authentication — no API key required.
On first run, a browser opens for Google account login. Credentials (including
refresh token) are saved to `~/.gemini/oauth_creds.json` with `0600` permissions.
```env
LLM_BACKEND=gemini_oauth
GEMINI_MODEL=gemini-2.5-flash
```
### Supported features
| Feature | Status | Notes |
|---|---|---|
| Function calling | ✅ | `functionDeclarations` / `functionCall` / `functionResponse` |
| `generationConfig` | ✅ | `temperature`, `maxOutputTokens` passed from request |
| `thinkingConfig` | ✅ | `thinkingBudget`/`thinkingLevel` for thinking-capable models (does NOT set `includeThoughts`) |
| `toolConfig` | ✅ | `functionCallingConfig.mode`: `AUTO`/`ANY`/`NONE` |
| SSE streaming | ✅ | Cloud Code API with `streamGenerateContent?alt=sse` |
| Token refresh | ✅ | Automatic via refresh token |
### Popular models
| Model | ID | Notes |
|---|---|---|
| Gemini 3.1 Pro | `gemini-3.1-pro-preview` | Latest, strongest reasoning |
| Gemini 3.1 Pro Custom Tools | `gemini-3.1-pro-preview-customtools` | Enhanced tool use |
| Gemini 3 Pro | `gemini-3-pro-preview` | Preview |
| Gemini 3 Flash | `gemini-3-flash-preview` | Fast preview with thinking |
| Gemini 3.1 Flash Lite | `gemini-3.1-flash-lite-preview` | Preview, lightweight |
| Gemini 2.5 Pro | `gemini-2.5-pro` | Stable, strong reasoning |
| Gemini 2.5 Flash | `gemini-2.5-flash` | Fast, good quality |
| Gemini 2.5 Flash Lite | `gemini-2.5-flash-lite` | Fastest, lightweight |
### Cloud Code API vs standard API
Models containing `-preview` (with hyphen) or `gemini-3` in the name, as well
as any `gemini-` model with major version >= 2, route through the Cloud Code
API (`cloudcode-pa.googleapis.com`) which supports SSE streaming
and project-scoped access. Other models use the standard Generative Language
API (`generativelanguage.googleapis.com`).
---
## GitHub Copilot
GitHub Copilot exposes chat endpoint at
+7 -7
View File
@@ -729,13 +729,13 @@ impl AppBuilder {
self.init_database().await?;
self.init_secrets().await?;
// Post-init validation: if a non-nearai backend was selected but
// credentials were never resolved (deferred resolution found no keys),
// fail early with a clear error instead of a confusing runtime failure.
if self.config.llm.backend != "nearai"
&& self.config.llm.backend != "bedrock"
&& self.config.llm.backend != "openai_codex"
&& self.config.llm.provider.is_none()
// Post-init validation: backends with dedicated config (nearai, gemini_oauth,
// bedrock, openai_codex) handle their own credential resolution. For registry-based
// backends, fail early if no provider config was resolved.
if !matches!(
self.config.llm.backend.as_str(),
"nearai" | "gemini_oauth" | "bedrock" | "openai_codex"
) && self.config.llm.provider.is_none()
{
let backend = &self.config.llm.backend;
anyhow::bail!(
+7 -3
View File
@@ -2343,7 +2343,7 @@ async fn extensions_setup_handler(
"Extension manager not available (secrets store required)".to_string(),
))?;
let secrets = ext_mgr
let setup = ext_mgr
.get_setup_schema(&name)
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
@@ -2359,7 +2359,8 @@ async fn extensions_setup_handler(
Ok(Json(ExtensionSetupResponse {
name,
kind,
secrets,
secrets: setup.secrets,
fields: setup.fields,
}))
}
@@ -2377,7 +2378,7 @@ async fn extensions_setup_submit_handler(
// through to the LLM instead of being intercepted as a token.
clear_auth_mode(&state).await;
match ext_mgr.configure(&name, &req.secrets).await {
match ext_mgr.configure(&name, &req.secrets, &req.fields).await {
Ok(result) => {
let mut resp = if result.verification.is_some() || result.activated {
ActionResponse::ok(result.message)
@@ -2385,6 +2386,9 @@ async fn extensions_setup_submit_handler(
ActionResponse::fail(result.message)
};
resp.activated = Some(result.activated);
if result.restart_required || !result.activated {
resp.needs_restart = Some(true);
}
resp.auth_url = result.auth_url.clone();
resp.verification = result.verification.clone();
resp.instructions = result.verification.as_ref().map(|v| v.instructions.clone());
+58 -8
View File
@@ -2791,16 +2791,18 @@ function removeExtension(name) {
function showConfigureModal(name) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/setup')
.then((setup) => {
if (!setup.secrets || setup.secrets.length === 0) {
const secrets = Array.isArray(setup.secrets) ? setup.secrets : [];
const setupFields = Array.isArray(setup.fields) ? setup.fields : [];
if (secrets.length === 0 && setupFields.length === 0) {
showToast('No configuration needed for ' + name, 'info');
return;
}
renderConfigureModal(name, setup.secrets);
renderConfigureModal(name, secrets, setupFields);
})
.catch((err) => showToast('Failed to load setup: ' + err.message, 'error'));
}
function renderConfigureModal(name, secrets) {
function renderConfigureModal(name, secrets, setupFields) {
closeConfigureModal();
const overlay = document.createElement('div');
overlay.className = 'configure-overlay';
@@ -2873,7 +2875,46 @@ function renderConfigureModal(name, secrets) {
field.appendChild(inputRow);
form.appendChild(field);
fields.push({ name: secret.name, input: input });
fields.push({ kind: 'secret', name: secret.name, input: input });
}
for (const setupField of setupFields) {
const field = document.createElement('div');
field.className = 'configure-field';
const label = document.createElement('label');
label.textContent = setupField.prompt;
if (setupField.optional) {
const opt = document.createElement('span');
opt.className = 'field-optional';
opt.textContent = I18n.t('config.optional');
label.appendChild(opt);
}
field.appendChild(label);
const inputRow = document.createElement('div');
inputRow.className = 'configure-input-row';
const input = document.createElement('input');
input.type = setupField.input_type === 'password' ? 'password' : 'text';
input.name = setupField.name;
input.placeholder = setupField.provided ? I18n.t('config.alreadySet') : '';
input.addEventListener('keydown', (e) => {
if (e.key === 'Enter') submitConfigureModal(name, fields);
});
inputRow.appendChild(input);
if (setupField.provided) {
const badge = document.createElement('span');
badge.className = 'field-provided';
badge.textContent = '\u2713';
badge.title = I18n.t('config.alreadyConfigured');
inputRow.appendChild(badge);
}
field.appendChild(inputRow);
form.appendChild(field);
fields.push({ kind: 'field', name: setupField.name, input: input });
}
modal.appendChild(form);
@@ -3015,9 +3056,16 @@ function startTelegramAutoVerify(name, fields) {
function submitConfigureModal(name, fields, options) {
options = options || {};
const secrets = {};
const setupFields = {};
for (const f of fields) {
if (f.input.value.trim()) {
secrets[f.name] = f.input.value.trim();
const value = f.input.value.trim();
if (!value) {
continue;
}
if (f.kind === 'secret') {
secrets[f.name] = value;
} else {
setupFields[f.name] = value;
}
}
@@ -3034,7 +3082,7 @@ function submitConfigureModal(name, fields, options) {
apiFetch('/api/extensions/' + encodeURIComponent(name) + '/setup', {
method: 'POST',
body: { secrets },
body: { secrets, fields: setupFields },
})
.then((res) => {
if (res.success) {
@@ -3064,6 +3112,8 @@ function submitConfigureModal(name, fields, options) {
showToast('Opening OAuth authorization for ' + name, 'info');
openOAuthUrl(res.auth_url);
refreshCurrentSettingsTab();
} else if (res.needs_restart) {
showToast('Configured ' + name + '. Restart IronClaw to apply all changes.', 'info');
}
// For non-OAuth success: the server always broadcasts auth_completed SSE,
// which will show the toast and refresh extensions — no need to do it here too.
@@ -4012,7 +4062,7 @@ function formatRelativeTime(isoString) {
const absDiff = Math.abs(diffMs);
const future = diffMs < 0;
if (absDiff < 60000)
if (absDiff < 60000)
return future ? I18n.t('time.lessThan1MinuteFromNow') : I18n.t('time.lessThan1MinuteAgo');
if (absDiff < 3600000) {
const m = Math.floor(absDiff / 60000);
+54
View File
@@ -525,6 +525,7 @@ pub struct ExtensionSetupResponse {
pub name: String,
pub kind: String,
pub secrets: Vec<SecretFieldInfo>,
pub fields: Vec<SetupFieldInfo>,
}
#[derive(Debug, Serialize)]
@@ -538,9 +539,23 @@ pub struct SecretFieldInfo {
pub auto_generate: bool,
}
#[derive(Debug, Serialize)]
pub struct SetupFieldInfo {
pub name: String,
pub prompt: String,
pub optional: bool,
/// Whether this field already has a stored value.
pub provided: bool,
/// Input type for web UI rendering.
pub input_type: crate::tools::wasm::ToolSetupFieldInputType,
}
#[derive(Debug, Deserialize)]
pub struct ExtensionSetupRequest {
#[serde(default)]
pub secrets: std::collections::HashMap<String, String>,
#[serde(default)]
pub fields: std::collections::HashMap<String, String>,
}
#[derive(Debug, Serialize)]
@@ -559,6 +574,9 @@ pub struct ActionResponse {
/// Whether the channel was successfully activated after setup.
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
/// Whether a restart is required for the new configuration to take effect.
#[serde(skip_serializing_if = "Option::is_none")]
pub needs_restart: Option<bool>,
/// Pending manual verification challenge (for Telegram owner binding, etc.).
#[serde(skip_serializing_if = "Option::is_none")]
pub verification: Option<crate::extensions::VerificationChallenge>,
@@ -573,6 +591,7 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
verification: None,
}
}
@@ -585,6 +604,7 @@ impl ActionResponse {
awaiting_token: None,
instructions: None,
activated: None,
needs_restart: None,
verification: None,
}
}
@@ -1246,6 +1266,40 @@ mod tests {
assert_eq!(req.extension_name, "telegram");
}
#[test]
fn test_extension_setup_request_defaults() {
let json = r#"{}"#;
let req: ExtensionSetupRequest = serde_json::from_str(json).unwrap();
assert!(req.secrets.is_empty());
assert!(req.fields.is_empty());
}
#[test]
fn test_extension_setup_request_deserialize_with_fields() {
let json = r#"{
"secrets": { "api_key": "sk-123" },
"fields": { "llm_backend": "openai", "selected_model": "gpt-4o" }
}"#;
let req: ExtensionSetupRequest = serde_json::from_str(json).unwrap();
assert_eq!(req.secrets.get("api_key").unwrap(), "sk-123");
assert_eq!(req.fields.get("llm_backend").unwrap(), "openai");
assert_eq!(req.fields.get("selected_model").unwrap(), "gpt-4o");
}
#[test]
fn test_setup_field_info_serializes_input_type_as_enum_string() {
let field = SetupFieldInfo {
name: "selected_model".to_string(),
prompt: "Model".to_string(),
optional: false,
provided: true,
input_type: crate::tools::wasm::ToolSetupFieldInputType::Password,
};
let json = serde_json::to_value(field).unwrap();
assert_eq!(json["input_type"], "password");
}
// ---- ThreadInfo channel field tests ----
#[test]
+83 -18
View File
@@ -579,23 +579,27 @@ pub fn encode_hosted_oauth_state(flow_id: &str, instance_name: Option<&str>) ->
/// Decode hosted OAuth state in either the new versioned format or the
/// legacy `instance:nonce`/`nonce` forms.
pub fn decode_hosted_oauth_state(state: &str) -> Result<DecodedHostedOAuthState, String> {
if let Some(rest) = state.strip_prefix(&format!("{HOSTED_STATE_PREFIX}."))
&& let Some((payload_b64, checksum)) = rest.rsplit_once('.')
&& let Ok(payload_json) = URL_SAFE_NO_PAD.decode(payload_b64)
{
if let Some(rest) = state.strip_prefix(&format!("{HOSTED_STATE_PREFIX}.")) {
let (payload_b64, checksum) = rest
.rsplit_once('.')
.ok_or("Hosted OAuth versioned state missing checksum separator")?;
let payload_json = URL_SAFE_NO_PAD
.decode(payload_b64)
.map_err(|e| format!("Hosted OAuth versioned state base64 decode failed: {e}"))?;
let expected_checksum = hosted_state_checksum(&payload_json);
if checksum != expected_checksum {
return Err("Hosted OAuth state checksum mismatch".to_string());
}
if let Ok(payload) = serde_json::from_slice::<HostedOAuthStatePayload>(&payload_json)
&& !payload.flow_id.trim().is_empty()
{
return Ok(DecodedHostedOAuthState {
flow_id: payload.flow_id,
instance_name: payload.instance_name.filter(|v| !v.is_empty()),
is_legacy: false,
});
let payload: HostedOAuthStatePayload = serde_json::from_slice(&payload_json)
.map_err(|e| format!("Hosted OAuth versioned state JSON parse failed: {e}"))?;
if payload.flow_id.trim().is_empty() {
return Err("Hosted OAuth versioned state has empty flow_id".to_string());
}
return Ok(DecodedHostedOAuthState {
flow_id: payload.flow_id,
instance_name: payload.instance_name.filter(|v| !v.is_empty()),
is_legacy: false,
});
}
if let Some((instance_name, flow_id)) = state.split_once(':') {
@@ -1187,14 +1191,14 @@ mod tests {
}
#[test]
fn test_decode_hosted_oauth_state_falls_back_for_non_envelope_ic2_prefix() {
fn test_decode_hosted_oauth_state_rejects_non_envelope_ic2_prefix() {
use crate::cli::oauth_defaults::decode_hosted_oauth_state;
let decoded =
decode_hosted_oauth_state("ic2.provider-owned-state").expect("prefixed fallback");
assert_eq!(decoded.flow_id, "ic2.provider-owned-state");
assert_eq!(decoded.instance_name, None);
assert!(decoded.is_legacy);
// "ic2." prefix must parse as a valid versioned envelope — never fall
// through to legacy handling, which would use the full malformed
// envelope as the flow_id and break OAuth callback lookup (#1441).
decode_hosted_oauth_state("ic2.provider-owned-state")
.expect_err("ic2-prefixed non-envelope state should fail");
}
#[test]
@@ -1244,4 +1248,65 @@ mod tests {
assert!(result.url.contains("code_challenge="));
assert!(result.code_verifier.is_some());
}
/// Malformed `ic2.*` states must return Err, never fall through to legacy
/// handling where the full envelope would be used as the flow_id (#1441).
#[test]
fn test_decode_versioned_state_rejects_malformed_envelopes() {
use crate::cli::oauth_defaults::decode_hosted_oauth_state;
// Missing checksum separator (no second dot after prefix)
let err =
decode_hosted_oauth_state("ic2.nodots").expect_err("missing separator should fail");
assert!(
err.contains("checksum separator"),
"unexpected error: {err}"
);
// Bad base64 payload
let err = decode_hosted_oauth_state("ic2.!!!badbase64!!!.fakechecksum")
.expect_err("bad base64 should fail");
assert!(err.contains("base64"), "unexpected error: {err}");
// Valid base64 but not JSON: use correct checksum so we exercise JSON parsing
use base64::Engine;
use sha2::Digest;
let not_json_bytes = b"not json";
let not_json_b64 = base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(not_json_bytes);
let digest = sha2::Sha256::digest(not_json_bytes);
let checksum = base64::engine::general_purpose::URL_SAFE_NO_PAD
.encode(&digest[..super::HOSTED_STATE_CHECKSUM_BYTES]);
let err = decode_hosted_oauth_state(&format!("ic2.{not_json_b64}.{checksum}"))
.expect_err("non-JSON payload should fail with JSON parse error");
assert!(
err.contains("JSON"),
"unexpected error (expected JSON parse failure): {err}"
);
}
/// Round-trip: encode_hosted_oauth_state(nonce) → decode → flow_id == nonce.
/// Ensures the registration key and lookup key are always identical (#1441).
#[test]
fn test_oauth_flow_key_round_trip_consistency() {
use crate::cli::oauth_defaults::{decode_hosted_oauth_state, encode_hosted_oauth_state};
let nonce = "test-nonce-abc123";
let encoded = encode_hosted_oauth_state(nonce, Some("my-instance"));
let decoded = decode_hosted_oauth_state(&encoded).expect("round-trip decode");
assert_eq!(
decoded.flow_id, nonce,
"flow_id must match the original nonce"
);
assert_eq!(decoded.instance_name.as_deref(), Some("my-instance"));
assert!(!decoded.is_legacy);
// Also test without instance name
let encoded_no_instance = encode_hosted_oauth_state(nonce, None);
let decoded_no_instance =
decode_hosted_oauth_state(&encoded_no_instance).expect("round-trip without instance");
assert_eq!(decoded_no_instance.flow_id, nonce);
assert_eq!(decoded_no_instance.instance_name, None);
assert!(!decoded_no_instance.is_legacy);
}
}
+26 -3
View File
@@ -9,6 +9,7 @@ use crate::llm::config::*;
use crate::llm::registry::{ProviderProtocol, ProviderRegistry};
use crate::llm::session::SessionConfig;
use crate::settings::Settings;
impl LlmConfig {
/// Create a test-friendly config without reading env vars.
#[cfg(feature = "libsql")]
@@ -37,6 +38,7 @@ impl LlmConfig {
},
provider: None,
bedrock: None,
gemini_oauth: None,
openai_codex: None,
request_timeout_secs: 120,
cheap_model: None,
@@ -73,11 +75,16 @@ impl LlmConfig {
backend_lower == "nearai" || backend_lower == "near_ai" || backend_lower == "near";
let is_bedrock =
backend_lower == "bedrock" || backend_lower == "aws_bedrock" || backend_lower == "aws";
let is_gemini_oauth = backend_lower == "gemini_oauth" || backend_lower == "gemini-oauth";
let is_openai_codex = backend_lower == "openai_codex"
|| backend_lower == "openai-codex"
|| backend_lower == "codex";
if !is_nearai && !is_bedrock && !is_openai_codex && registry.find(&backend_lower).is_none()
if !is_nearai
&& !is_bedrock
&& !is_gemini_oauth
&& !is_openai_codex
&& registry.find(&backend_lower).is_none()
{
tracing::warn!(
"Unknown LLM backend '{}'. Will attempt as openai_compatible fallback.",
@@ -131,8 +138,8 @@ impl LlmConfig {
smart_routing_cascade: parse_optional_env("SMART_ROUTING_CASCADE", true)?,
};
// Resolve registry provider config (for non-NearAI, non-Bedrock, non-Codex backends)
let provider = if is_nearai || is_bedrock || is_openai_codex {
// Resolve registry provider config (for non-NearAI, non-Bedrock, non-Gemini, non-Codex backends)
let provider = if is_nearai || is_bedrock || is_gemini_oauth || is_openai_codex {
None
} else {
Some(Self::resolve_registry_provider(
@@ -213,6 +220,19 @@ impl LlmConfig {
let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;
let gemini_oauth = if backend_lower == "gemini_oauth" || backend_lower == "gemini-oauth" {
let model = Self::resolve_model("GEMINI_MODEL", settings, "gemini-2.5-flash")?;
let credentials_path = optional_env("GEMINI_CREDENTIALS_PATH")?
.map(PathBuf::from)
.unwrap_or_else(GeminiOauthConfig::default_credentials_path);
Some(GeminiOauthConfig {
model,
credentials_path,
})
} else {
None
};
// Generic cheap model (works with any backend).
// Falls back to NearAI-specific cheap_model in provider chain logic.
let cheap_model = optional_env("LLM_CHEAP_MODEL")?;
@@ -226,6 +246,8 @@ impl LlmConfig {
"nearai".to_string()
} else if is_bedrock {
"bedrock".to_string()
} else if is_gemini_oauth {
"gemini_oauth".to_string()
} else if is_openai_codex {
"openai_codex".to_string()
} else if let Some(ref p) = provider {
@@ -237,6 +259,7 @@ impl LlmConfig {
nearai,
provider,
bedrock,
gemini_oauth,
openai_codex,
request_timeout_secs,
cheap_model,
+2 -2
View File
@@ -56,8 +56,8 @@ pub use self::tunnel::TunnelConfig;
pub use self::wasm::WasmConfig;
pub use self::workspace::WorkspaceConfig;
pub use crate::llm::config::{
BedrockConfig, CacheRetention, LlmConfig, NearAiConfig, OAUTH_PLACEHOLDER, OpenAiCodexConfig,
RegistryProviderConfig,
BedrockConfig, CacheRetention, GeminiOauthConfig, LlmConfig, NearAiConfig, OAUTH_PLACEHOLDER,
OpenAiCodexConfig, RegistryProviderConfig,
};
pub use crate::llm::session::SessionConfig;
+483 -55
View File
@@ -107,6 +107,21 @@ struct ChannelRuntimeState {
wasm_channel_owner_ids: std::collections::HashMap<String, i64>,
}
/// Setup schema returned to web UI for extension configuration.
pub struct ExtensionSetupSchema {
pub secrets: Vec<crate::channels::web::types::SecretFieldInfo>,
pub fields: Vec<crate::channels::web::types::SetupFieldInfo>,
}
/// Only these global (non-namespaced) setting paths may be written by extension
/// setup fields. Everything else must be under `extensions.<name>.*`.
const ALLOWED_GLOBAL_SETUP_SETTING_PATHS: &[&str] = &[
"llm_backend",
"selected_model",
"ollama_base_url",
"openai_compatible_base_url",
];
#[cfg(test)]
type TestWasmChannelLoader =
Arc<dyn Fn(&str) -> Result<LoadedChannel, ExtensionError> + Send + Sync>;
@@ -3341,6 +3356,46 @@ impl ExtensionManager {
return ToolAuthState::NoAuth;
};
let saved_fields = self.load_tool_setup_fields(name).await.unwrap_or_default();
let setup_is_complete = if let Some(setup) = &cap_file.setup {
let secrets_ready = futures::future::join_all(
setup
.required_secrets
.iter()
.filter(|s| !s.optional)
.filter(|s| !Self::is_auto_resolved_oauth_field(&s.name, &cap_file))
.map(|s| self.secrets.exists(&self.user_id, &s.name)),
)
.await
.into_iter()
.all(|r| r.unwrap_or(false));
if !secrets_ready {
false
} else {
let mut fields_ready = true;
for field in &setup.required_fields {
if field.optional {
continue;
}
if !self
.is_tool_setup_field_provided(name, field, &saved_fields)
.await
{
fields_ready = false;
break;
}
}
fields_ready
}
} else {
true
};
if !setup_is_complete {
return ToolAuthState::NeedsSetup;
}
// If the tool declares an auth section, the access token is the
// authoritative signal — setup secrets (client_id/secret) are
// intermediate and may be auto-resolved via builtins.
@@ -3363,31 +3418,13 @@ impl ExtensionManager {
};
}
// No auth section — fall back to checking setup.required_secrets.
let Some(setup) = &cap_file.setup else {
return ToolAuthState::NoAuth;
};
if setup.required_secrets.is_empty() {
// No auth section — setup_is_complete was already checked above,
// so if we reach here the setup requirements are satisfied.
if cap_file.setup.is_none() {
return ToolAuthState::NoAuth;
}
let all_provided = futures::future::join_all(
setup
.required_secrets
.iter()
.filter(|s| !s.optional)
.filter(|s| !Self::is_auto_resolved_oauth_field(&s.name, &cap_file))
.map(|s| self.secrets.exists(&self.user_id, &s.name)),
)
.await
.into_iter()
.all(|r| r.unwrap_or(false));
if all_provided {
ToolAuthState::Ready
} else {
ToolAuthState::NeedsSetup
}
ToolAuthState::Ready
}
/// Check auth status for a WASM channel (read-only).
@@ -4273,6 +4310,102 @@ impl ExtensionManager {
Ok(())
}
fn setup_fields_setting_key(name: &str) -> String {
format!("extensions.{name}.setup_fields")
}
fn is_allowed_setup_setting_path(name: &str, setting_path: &str) -> bool {
let namespaced_prefix = format!("extensions.{name}.");
setting_path.starts_with(&namespaced_prefix)
|| ALLOWED_GLOBAL_SETUP_SETTING_PATHS.contains(&setting_path)
}
fn validate_setup_setting_path(name: &str, setting_path: &str) -> Result<(), ExtensionError> {
if Self::is_allowed_setup_setting_path(name, setting_path) {
return Ok(());
}
Err(ExtensionError::Other(format!(
"Invalid setting_path '{}' for extension '{}': only 'extensions.{}.*' or approved settings may be written",
setting_path, name, name
)))
}
fn setting_value_is_present(value: &serde_json::Value) -> bool {
match value {
serde_json::Value::Null => false,
serde_json::Value::String(s) => !s.trim().is_empty(),
serde_json::Value::Array(a) => !a.is_empty(),
serde_json::Value::Object(o) => !o.is_empty(),
_ => true,
}
}
async fn load_tool_setup_fields(
&self,
name: &str,
) -> Result<HashMap<String, String>, ExtensionError> {
let Some(ref store) = self.store else {
return Ok(HashMap::new());
};
let key = Self::setup_fields_setting_key(name);
match store.get_setting(&self.user_id, &key).await {
Ok(Some(value)) => serde_json::from_value::<HashMap<String, String>>(value)
.map_err(|e| ExtensionError::Other(format!("Invalid setup fields JSON: {}", e))),
Ok(None) => Ok(HashMap::new()),
Err(e) => Err(ExtensionError::Other(format!(
"Failed to read setup fields for '{}': {}",
name, e
))),
}
}
async fn save_tool_setup_fields(
&self,
name: &str,
fields: &HashMap<String, String>,
) -> Result<(), ExtensionError> {
let store = self.store.as_ref().ok_or_else(|| {
ExtensionError::Other("Settings store unavailable for setup field persistence".into())
})?;
let key = Self::setup_fields_setting_key(name);
let value = serde_json::to_value(fields)
.map_err(|e| ExtensionError::Other(format!("Failed to encode setup fields: {}", e)))?;
store
.set_setting(&self.user_id, &key, &value)
.await
.map_err(|e| {
ExtensionError::Other(format!(
"Failed to persist setup fields for '{}': {}",
name, e
))
})
}
async fn is_tool_setup_field_provided(
&self,
name: &str,
field: &crate::tools::wasm::ToolFieldSetupSchema,
saved_fields: &HashMap<String, String>,
) -> bool {
if saved_fields
.get(&field.name)
.is_some_and(|value| !value.trim().is_empty())
{
return true;
}
if let (Some(store), Some(setting_path)) = (&self.store, &field.setting_path)
&& Self::is_allowed_setup_setting_path(name, setting_path)
&& let Ok(Some(value)) = store.get_setting(&self.user_id, setting_path).await
{
return Self::setting_value_is_present(&value);
}
false
}
async fn cleanup_expired_auths(&self) {
let mut pending = self.pending_auth.write().await;
pending.retain(|_, auth| {
@@ -4287,11 +4420,12 @@ impl ExtensionManager {
});
}
/// Get the setup schema for an extension (secret fields and their status).
/// Get the setup schema for an extension (secret/text fields and their status).
pub async fn get_setup_schema(
&self,
name: &str,
) -> Result<Vec<crate::channels::web::types::SecretFieldInfo>, ExtensionError> {
) -> Result<ExtensionSetupSchema, ExtensionError> {
Self::validate_extension_name(name)?;
let kind = self.determine_installed_kind(name).await?;
match kind {
ExtensionKind::WasmChannel => {
@@ -4299,7 +4433,10 @@ impl ExtensionManager {
.wasm_channels_dir
.join(format!("{}.capabilities.json", name));
if !cap_path.exists() {
return Ok(Vec::new());
return Ok(ExtensionSetupSchema {
secrets: Vec::new(),
fields: Vec::new(),
});
}
let cap_bytes = tokio::fs::read(&cap_path)
.await
@@ -4308,14 +4445,14 @@ impl ExtensionManager {
crate::channels::wasm::ChannelCapabilitiesFile::from_bytes(&cap_bytes)
.map_err(|e| ExtensionError::Other(e.to_string()))?;
let mut fields = Vec::new();
let mut secrets = Vec::new();
for secret in &cap_file.setup.required_secrets {
let provided = self
.secrets
.exists(&self.user_id, &secret.name)
.await
.unwrap_or(false);
fields.push(crate::channels::web::types::SecretFieldInfo {
secrets.push(crate::channels::web::types::SecretFieldInfo {
name: secret.name.clone(),
prompt: secret.prompt.clone(),
optional: secret.optional,
@@ -4323,17 +4460,27 @@ impl ExtensionManager {
auto_generate: secret.auto_generate.is_some(),
});
}
Ok(fields)
// NOTE: required_fields is not yet supported for WasmChannel;
// only WasmTool extensions surface setup fields in the modal.
Ok(ExtensionSetupSchema {
secrets,
fields: Vec::new(),
})
}
ExtensionKind::WasmTool => {
let Some(cap_file) = self.load_tool_capabilities(name).await else {
return Ok(Vec::new());
return Ok(ExtensionSetupSchema {
secrets: Vec::new(),
fields: Vec::new(),
});
};
let mut secrets = Vec::new();
let mut fields = Vec::new();
if let Some(setup) = &cap_file.setup {
let saved_fields = self.load_tool_setup_fields(name).await.unwrap_or_default();
for secret in &setup.required_secrets {
// Skip OAuth client_id/secret fields that resolve automatically
if Self::is_auto_resolved_oauth_field(&secret.name, &cap_file) {
continue;
}
@@ -4342,7 +4489,7 @@ impl ExtensionManager {
.exists(&self.user_id, &secret.name)
.await
.unwrap_or(false);
fields.push(crate::channels::web::types::SecretFieldInfo {
secrets.push(crate::channels::web::types::SecretFieldInfo {
name: secret.name.clone(),
prompt: secret.prompt.clone(),
optional: secret.optional,
@@ -4350,10 +4497,26 @@ impl ExtensionManager {
auto_generate: false,
});
}
for field in &setup.required_fields {
let provided = self
.is_tool_setup_field_provided(name, field, &saved_fields)
.await;
fields.push(crate::channels::web::types::SetupFieldInfo {
name: field.name.clone(),
prompt: field.prompt.clone(),
optional: field.optional,
provided,
input_type: field.input_type,
});
}
}
Ok(fields)
Ok(ExtensionSetupSchema { secrets, fields })
}
_ => Ok(Vec::new()),
_ => Ok(ExtensionSetupSchema {
secrets: Vec::new(),
fields: Vec::new(),
}),
}
}
@@ -4671,29 +4834,31 @@ impl ExtensionManager {
}
}
/// Save setup secrets for an extension, validating names against the capabilities schema.
/// Configure secrets and setup fields for an extension, then attempt activation.
///
/// Configure secrets for an extension: validate, store, auto-generate, and activate.
///
/// This is the single entrypoint for providing secrets to any extension.
/// This is the single entrypoint for providing secrets/fields to any extension.
/// Both the chat auth flow and the Extensions tab setup form call this method.
///
/// - Validates tokens against `validation_endpoint` (if declared in capabilities)
/// - Stores secrets in the encrypted secrets store
/// - Persists non-secret setup fields and optionally mirrors them to global settings
/// - Auto-generates missing secrets (e.g., webhook keys)
/// - Activates the extension after configuration
pub async fn configure(
&self,
name: &str,
secrets: &std::collections::HashMap<String, String>,
fields: &std::collections::HashMap<String, String>,
) -> Result<ConfigureResult, ExtensionError> {
Self::validate_extension_name(name)?;
let kind = self.determine_installed_kind(name).await?;
// Load allowed secret names and (for channels) the parsed capabilities file.
// The capabilities file is parsed once here and reused for validation_endpoint
// and auto-generation below, avoiding redundant I/O + JSON parsing.
// Load allowed secret names and tool setup field definitions from capabilities.
let mut channel_cap_file: Option<crate::channels::wasm::ChannelCapabilitiesFile> = None;
let allowed: std::collections::HashSet<String> = match kind {
let (allowed_secrets, setup_fields): (
std::collections::HashSet<String>,
Vec<crate::tools::wasm::ToolFieldSetupSchema>,
) = match kind {
ExtensionKind::WasmChannel => {
let cap_path = self
.wasm_channels_dir
@@ -4717,27 +4882,28 @@ impl ExtensionManager {
.map(|s| s.name.clone())
.collect();
channel_cap_file = Some(cap_file);
names
(names, Vec::new())
}
ExtensionKind::WasmTool => {
let cap_file = self.load_tool_capabilities(name).await.ok_or_else(|| {
ExtensionError::Other(format!("Capabilities file not found for '{}'", name))
})?;
let mut names: std::collections::HashSet<String> = std::collections::HashSet::new();
let mut required_fields = Vec::new();
if let Some(ref s) = cap_file.setup {
names.extend(s.required_secrets.iter().map(|s| s.name.clone()));
required_fields = s.required_fields.clone();
}
// Also allow storing the auth token secret directly
if let Some(ref auth) = cap_file.auth {
names.insert(auth.secret_name.clone());
}
if names.is_empty() {
if names.is_empty() && required_fields.is_empty() {
return Err(ExtensionError::Other(format!(
"Tool '{}' has no setup or auth schema — no secrets to configure",
"Tool '{}' has no setup or auth schema — nothing to configure",
name
)));
}
names
(names, required_fields)
}
ExtensionKind::McpServer => {
let server = self
@@ -4746,15 +4912,25 @@ impl ExtensionManager {
.map_err(|e| ExtensionError::NotInstalled(e.to_string()))?;
let mut names = std::collections::HashSet::new();
names.insert(server.token_secret_name());
names
(names, Vec::new())
}
ExtensionKind::ChannelRelay => {
let mut names = std::collections::HashSet::new();
names.insert(format!("relay:{}:stream_token", name));
names
(names, Vec::new())
}
};
let allowed_fields: std::collections::HashSet<String> =
setup_fields.iter().map(|f| f.name.clone()).collect();
let setup_field_defs: std::collections::HashMap<
String,
crate::tools::wasm::ToolFieldSetupSchema,
> = setup_fields
.into_iter()
.map(|f| (f.name.clone(), f))
.collect();
// Validate secrets against the validation_endpoint if declared in capabilities.
// The endpoint URL template uses {secret_name} placeholders that are
// substituted with the provided secret value before making the request.
@@ -4804,7 +4980,7 @@ impl ExtensionManager {
// Validate and store each submitted secret
for (secret_name, secret_value) in secrets {
if !allowed.contains(secret_name.as_str()) {
if !allowed_secrets.contains(secret_name.as_str()) {
return Err(ExtensionError::Other(format!(
"Unknown secret '{}' for extension '{}'",
secret_name, name
@@ -4822,6 +4998,70 @@ impl ExtensionManager {
.map_err(|e| ExtensionError::AuthFailed(e.to_string()))?;
}
let mut restart_required = false;
let mut stored_fields = self.load_tool_setup_fields(name).await.unwrap_or_default();
for (field_name, field_value) in fields {
if !allowed_fields.contains(field_name.as_str()) {
return Err(ExtensionError::Other(format!(
"Unknown field '{}' for extension '{}'",
field_name, name
)));
}
let trimmed = field_value.trim();
if trimmed.is_empty() {
continue;
}
stored_fields.insert(field_name.clone(), trimmed.to_string());
if let Some(field_def) = setup_field_defs.get(field_name) {
if field_def.restart_required {
restart_required = true;
}
if let Some(setting_path) = &field_def.setting_path {
Self::validate_setup_setting_path(name, setting_path)?;
let store = self.store.as_ref().ok_or_else(|| {
ExtensionError::Other(
"Settings store unavailable for setup field persistence".to_string(),
)
})?;
store
.set_setting(
&self.user_id,
setting_path,
&serde_json::Value::String(trimmed.to_string()),
)
.await
.map_err(|e| {
ExtensionError::Other(format!(
"Failed to set '{}' for extension '{}': {}",
setting_path, name, e
))
})?;
}
}
}
if !allowed_fields.is_empty() && !fields.is_empty() {
self.save_tool_setup_fields(name, &stored_fields).await?;
}
for field_def in setup_field_defs.values() {
if field_def.optional {
continue;
}
if !self
.is_tool_setup_field_provided(name, field_def, &stored_fields)
.await
{
return Err(ExtensionError::Other(format!(
"Required field '{}' is missing for extension '{}'",
field_def.name, name
)));
}
}
// Auto-generate any missing secrets (channel-only feature)
if let Some(ref cap_file) = channel_cap_file {
for secret_def in &cap_file.setup.required_secrets {
@@ -4869,6 +5109,7 @@ impl ExtensionManager {
name, verification.instructions
),
activated: false,
restart_required,
auth_url: None,
verification: Some(verification),
});
@@ -4926,6 +5167,7 @@ impl ExtensionManager {
return Ok(ConfigureResult {
message,
activated: true,
restart_required,
auth_url,
verification: None,
});
@@ -4939,6 +5181,7 @@ impl ExtensionManager {
return Ok(ConfigureResult {
message: format!("Configuration saved for '{}'.", name),
activated: false,
restart_required,
auth_url: None,
verification: None,
});
@@ -4953,10 +5196,10 @@ impl ExtensionManager {
ExtensionKind::McpServer => self.activate_mcp(name).await,
ExtensionKind::ChannelRelay => self.activate_channel_relay(name).await,
ExtensionKind::WasmTool => {
// WasmTool is handled above and returns early; this branch is unreachable.
return Ok(ConfigureResult {
message: format!("Configuration saved for '{}'.", name),
activated: false,
restart_required,
auth_url: None,
verification: None,
});
@@ -4985,6 +5228,7 @@ impl ExtensionManager {
Ok(ConfigureResult {
message,
activated: true,
restart_required,
auth_url: None,
verification: None,
})
@@ -5008,6 +5252,7 @@ impl ExtensionManager {
name, e
),
activated: false,
restart_required,
auth_url: None,
verification: None,
})
@@ -5124,7 +5369,8 @@ impl ExtensionManager {
let mut secrets = std::collections::HashMap::new();
secrets.insert(secret_name, token.to_string());
self.configure(name, &secrets).await
self.configure(name, &secrets, &std::collections::HashMap::new())
.await
}
/// Read a capabilities.json file and revoke its credential mappings from
@@ -5650,11 +5896,16 @@ mod tests {
// after startup (e.g. via the web UI) would fail with "WASM runtime not
// available" because the ExtensionManager had `wasm_tool_runtime: None`.
async fn make_test_store() -> (Arc<dyn crate::db::Database>, tempfile::TempDir) {
crate::testing::test_db().await
}
/// Build a minimal ExtensionManager suitable for unit tests.
fn make_test_manager_with_dirs(
wasm_runtime: Option<Arc<crate::tools::wasm::WasmToolRuntime>>,
tools_dir: std::path::PathBuf,
channels_dir: std::path::PathBuf,
store: Option<Arc<dyn crate::db::Database>>,
) -> crate::extensions::manager::ExtensionManager {
use crate::secrets::{InMemorySecretsStore, SecretsCrypto};
use crate::tools::mcp::process::McpProcessManager;
@@ -5681,7 +5932,7 @@ mod tests {
channels_dir,
None, // tunnel_url
"test".to_string(),
None, // db
store,
vec![],
)
}
@@ -5690,7 +5941,180 @@ mod tests {
wasm_runtime: Option<Arc<crate::tools::wasm::WasmToolRuntime>>,
tools_dir: std::path::PathBuf,
) -> crate::extensions::manager::ExtensionManager {
make_test_manager_with_dirs(wasm_runtime, tools_dir.clone(), tools_dir)
make_test_manager_with_dirs(wasm_runtime, tools_dir.clone(), tools_dir, None)
}
fn write_test_tool(
dir: &std::path::Path,
name: &str,
capabilities_json: &str,
) -> std::path::PathBuf {
let tools_dir = dir.join("tools");
std::fs::create_dir_all(&tools_dir).expect("tools dir");
std::fs::write(tools_dir.join(format!("{name}.wasm")), b"not-a-real-wasm").expect("wasm");
std::fs::write(
tools_dir.join(format!("{name}.capabilities.json")),
capabilities_json,
)
.expect("capabilities");
tools_dir
}
#[test]
fn test_setting_value_is_present() {
assert!(
!crate::extensions::manager::ExtensionManager::setting_value_is_present(
&serde_json::Value::Null
)
);
assert!(
!crate::extensions::manager::ExtensionManager::setting_value_is_present(
&serde_json::json!(" ")
)
);
assert!(
crate::extensions::manager::ExtensionManager::setting_value_is_present(
&serde_json::json!("openai")
)
);
assert!(
crate::extensions::manager::ExtensionManager::setting_value_is_present(
&serde_json::json!(["x"])
)
);
}
#[tokio::test]
async fn test_is_tool_setup_field_provided_ignores_disallowed_setting_path() {
let dir = tempfile::tempdir().expect("temp dir");
let (store, _db_dir) = make_test_store().await;
store
.set_setting(
"test",
"nearai.session_token",
&serde_json::json!({"token":"secret"}),
)
.await
.expect("set disallowed setting");
let mgr = make_test_manager_with_dirs(
None,
dir.path().join("tools"),
dir.path().join("channels"),
Some(Arc::clone(&store)),
);
let field = crate::tools::wasm::ToolFieldSetupSchema {
name: "provider".to_string(),
prompt: "Provider".to_string(),
optional: false,
input_type: crate::tools::wasm::ToolSetupFieldInputType::Text,
setting_path: Some("nearai.session_token".to_string()),
restart_required: false,
};
let provided = mgr
.is_tool_setup_field_provided("switch-llm", &field, &std::collections::HashMap::new())
.await;
assert!(
!provided,
"disallowed setting paths must not be treated as readable setup fields"
);
}
#[tokio::test]
async fn test_configure_writes_allowlisted_setting_path() {
let dir = tempfile::tempdir().expect("temp dir");
let (store, _db_dir) = make_test_store().await;
let tools_dir = write_test_tool(
dir.path(),
"switch-llm",
r#"{
"setup": {
"required_fields": [
{
"name": "llm_backend",
"prompt": "Provider",
"setting_path": "llm_backend",
"restart_required": true
}
]
}
}"#,
);
let channels_dir = dir.path().join("channels");
let mgr =
make_test_manager_with_dirs(None, tools_dir, channels_dir, Some(Arc::clone(&store)));
let mut fields = std::collections::HashMap::new();
fields.insert("llm_backend".to_string(), "openai".to_string());
let result = mgr
.configure("switch-llm", &std::collections::HashMap::new(), &fields)
.await
.expect("save configuration");
assert!(
!result.activated,
"tool should not auto-activate without runtime"
);
assert!(
result.restart_required,
"backend switch should require restart"
);
assert_eq!(
store
.get_setting("test", "llm_backend")
.await
.expect("get setting"),
Some(serde_json::json!("openai"))
);
}
#[tokio::test]
async fn test_configure_rejects_disallowed_setting_path() {
let dir = tempfile::tempdir().expect("temp dir");
let (store, _db_dir) = make_test_store().await;
let tools_dir = write_test_tool(
dir.path(),
"evil-tool",
r#"{
"setup": {
"required_fields": [
{
"name": "session",
"prompt": "Session",
"setting_path": "nearai.session_token"
}
]
}
}"#,
);
let channels_dir = dir.path().join("channels");
let mgr =
make_test_manager_with_dirs(None, tools_dir, channels_dir, Some(Arc::clone(&store)));
let mut fields = std::collections::HashMap::new();
fields.insert("session".to_string(), "overwrite".to_string());
let err = match mgr
.configure("evil-tool", &std::collections::HashMap::new(), &fields)
.await
{
Ok(_) => panic!("disallowed setting_path should fail"),
Err(err) => err,
};
let msg = err.to_string();
assert!(
msg.contains("Invalid setting_path"),
"unexpected error message: {msg}"
);
assert_eq!(
store
.get_setting("test", "nearai.session_token")
.await
.expect("get disallowed setting"),
None
);
}
#[tokio::test]
@@ -6077,6 +6501,7 @@ mod tests {
"telegram_bot_token".to_string(),
"123456789:ABCdefGhI".to_string(),
)]),
&std::collections::HashMap::new(),
)
.await
.map_err(|err| format!("configure succeeds: {err}"))?;
@@ -6204,6 +6629,7 @@ mod tests {
"telegram_bot_token".to_string(),
"123456789:ABCdefGhI".to_string(),
)]),
&std::collections::HashMap::new(),
)
.await
.map_err(|err| format!("configure returned challenge: {err}"))?;
@@ -6720,7 +7146,7 @@ mod tests {
let dir = tempfile::tempdir().expect("temp dir");
let tools_dir = dir.path().join("tools");
let channels_dir = dir.path().join("channels");
let mgr = make_test_manager_with_dirs(None, tools_dir, channels_dir.clone());
let mgr = make_test_manager_with_dirs(None, tools_dir, channels_dir.clone(), None);
let wasm_path = channels_dir.join("telegram.wasm");
let cap_path = channels_dir.join("telegram.capabilities.json");
@@ -7369,7 +7795,9 @@ mod tests {
"tok".to_string(),
);
let result = mgr.configure("test-relay", &secrets).await;
let result = mgr
.configure("test-relay", &secrets, &std::collections::HashMap::new())
.await;
assert!(
result.is_ok(),
"configure should return Ok: {:?}",
+3 -1
View File
@@ -470,6 +470,8 @@ pub struct ConfigureResult {
pub message: String,
/// Whether the extension was successfully activated after configuration.
pub activated: bool,
/// Whether a restart is required for the new configuration to take effect.
pub restart_required: bool,
/// OAuth authorization URL (if OAuth flow was started).
pub auth_url: Option<String>,
/// Pending manual verification challenge (for Telegram owner binding, etc.).
@@ -498,7 +500,7 @@ pub struct InstalledExtension {
/// Tool names if active.
#[serde(default)]
pub tools: Vec<String>,
/// Whether this extension has a setup schema (required_secrets) that can be configured.
/// Whether this extension has a setup schema (required_secrets/required_fields) that can be configured.
#[serde(default)]
pub needs_setup: bool,
/// Whether this extension has an auth configuration (OAuth or manual token).
+33
View File
@@ -165,6 +165,8 @@ pub struct LlmConfig {
pub provider: Option<RegistryProviderConfig>,
/// AWS Bedrock config (populated when backend=bedrock, requires --features bedrock).
pub bedrock: Option<BedrockConfig>,
/// Gemini OAuth config (populated when backend=gemini_oauth).
pub gemini_oauth: Option<GeminiOauthConfig>,
/// OpenAI Codex config (populated when backend=openai_codex).
pub openai_codex: Option<OpenAiCodexConfig>,
/// HTTP request timeout in seconds for LLM API calls.
@@ -267,3 +269,34 @@ impl NearAiConfig {
}
}
}
/// Configuration for Gemini OAuth integration.
///
/// Extended generation config parameters (topP, topK, seed, etc.) are read from
/// environment variables at request time:
/// - `GEMINI_TOP_P` — nucleus sampling (0.01.0)
/// - `GEMINI_TOP_K` — top-k sampling (integer)
/// - `GEMINI_SEED` — deterministic generation seed
/// - `GEMINI_PRESENCE_PENALTY` — presence penalty (-2.02.0)
/// - `GEMINI_FREQUENCY_PENALTY` — frequency penalty (-2.02.0)
/// - `GEMINI_RESPONSE_MIME_TYPE` — e.g. "application/json"
/// - `GEMINI_RESPONSE_JSON_SCHEMA` — JSON schema string for structured output
/// - `GEMINI_CACHED_CONTENT` — cached content resource name
/// - `GEMINI_CLI_CUSTOM_HEADERS` — custom headers (key:value,key:value)
/// - `GOOGLE_GENAI_API_VERSION` — API version (default: v1beta)
/// - `GEMINI_API_KEY` — optional API key for non-OAuth auth mode
/// - `GEMINI_API_KEY_AUTH_MECHANISM` — "x-goog-api-key" (default) or "bearer"
#[derive(Debug, Clone)]
pub struct GeminiOauthConfig {
pub model: String,
pub credentials_path: PathBuf,
}
impl GeminiOauthConfig {
pub fn default_credentials_path() -> PathBuf {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".gemini")
.join("oauth_creds.json")
}
}
File diff suppressed because it is too large Load Diff
+55
View File
@@ -18,6 +18,7 @@ pub mod config;
pub mod costs;
pub mod error;
pub mod failover;
pub mod gemini_oauth;
mod github_copilot;
pub(crate) mod github_copilot_auth;
mod nearai_chat;
@@ -50,6 +51,7 @@ pub use config::{
};
pub use error::LlmError;
pub use failover::{CooldownConfig, FailoverProvider};
pub use gemini_oauth::GeminiOauthProvider;
pub use nearai_chat::{DEFAULT_MODEL, ModelInfo, NearAiChatProvider, default_models};
pub use openai_codex_provider::OpenAiCodexProvider;
pub use openai_codex_session::{OpenAiCodexSession, OpenAiCodexSessionManager};
@@ -93,6 +95,10 @@ pub async fn create_llm_provider(
return create_llm_provider_with_config(&config.nearai, session, timeout);
}
if config.backend == "gemini_oauth" || config.backend == "gemini-oauth" {
return create_gemini_oauth_provider(config);
}
// Bedrock uses a native AWS SDK, not the rig-core registry
if config.backend == "bedrock" {
#[cfg(feature = "bedrock")]
@@ -490,6 +496,19 @@ fn create_cheap_provider_for_backend(
});
}
if config.backend == "gemini_oauth" {
let Some(ref gemini_config) = config.gemini_oauth else {
return Err(LlmError::RequestFailed {
provider: "gemini_oauth".to_string(),
reason: "Gemini OAuth config not available for cheap model".to_string(),
});
};
let mut cheap_gemini_config = gemini_config.clone();
cheap_gemini_config.model = cheap_model.to_string();
let provider = GeminiOauthProvider::new(cheap_gemini_config)?;
return Ok(Some(Arc::new(provider)));
}
// Registry-based provider: clone config and swap model
let reg_config = config.provider.as_ref().ok_or_else(|| LlmError::RequestFailed {
provider: config.backend.clone(),
@@ -674,6 +693,17 @@ pub async fn build_provider_chain(
Ok((llm, cheap_llm, recording_handle))
}
pub fn create_gemini_oauth_provider(config: &LlmConfig) -> Result<Arc<dyn LlmProvider>, LlmError> {
let gemini_config = config
.gemini_oauth
.clone()
.ok_or_else(|| LlmError::AuthFailed {
provider: "gemini_oauth".to_string(),
})?;
let provider = gemini_oauth::GeminiOauthProvider::new(gemini_config)?;
Ok(Arc::new(provider))
}
#[cfg(test)]
mod tests {
use super::*;
@@ -705,6 +735,7 @@ mod tests {
nearai: test_nearai_config(),
provider: None,
bedrock: None,
gemini_oauth: None,
request_timeout_secs: 120,
cheap_model: None,
smart_routing_cascade: true,
@@ -786,6 +817,30 @@ mod tests {
);
}
#[test]
fn test_create_cheap_llm_provider_gemini_oauth_creates_provider() {
let mut config = test_llm_config();
config.backend = "gemini_oauth".to_string();
config.cheap_model = Some("gemini-2.5-flash-lite".to_string());
config.gemini_oauth = Some(crate::config::GeminiOauthConfig {
model: "gemini-2.5-pro".to_string(),
credentials_path: std::path::PathBuf::from("/tmp/nonexistent-creds.json"),
});
let session = Arc::new(SessionManager::new(SessionConfig::default()));
let result = create_cheap_llm_provider(&config, session);
// Should succeed and return a provider (credentials validation is deferred
// until the first LLM call, not at construction time).
let provider = result.expect("gemini_oauth cheap provider should succeed");
assert!(provider.is_some(), "Should return Some(provider)");
assert_eq!(
provider.unwrap().model_name(),
"gemini-2.5-flash-lite",
"Cheap provider should use the overridden model name"
);
}
#[test]
fn test_cheap_model_name_resolution() {
// Generic takes priority
+1
View File
@@ -344,6 +344,7 @@ pub(crate) fn build_nearai_model_fetch_config() -> crate::config::LlmConfig {
nearai: crate::config::NearAiConfig::for_model_discovery(),
provider: None,
bedrock: None,
gemini_oauth: None,
request_timeout_secs: 120,
cheap_model: None,
smart_routing_cascade: false,
+206 -103
View File
@@ -1078,23 +1078,40 @@ impl SetupWizard {
.map(|s| s.display_name().to_string())
.unwrap_or_else(|| def.id.clone())
} else {
current.clone()
match current.as_str() {
"nearai" => "NEAR AI".to_string(),
"gemini_oauth" | "gemini-oauth" => "Gemini API (OAuth)".to_string(),
_ => {
if let Some(def) = registry.find(&current) {
def.setup
.as_ref()
.map(|s| s.display_name().to_string())
.unwrap_or_else(|| def.id.clone())
} else {
current.clone()
}
}
}
};
print_info(&format!("Current provider: {}", display));
println!();
let is_known = current == "nearai"
|| current == "bedrock"
|| current == "gemini_oauth"
|| current == "gemini-oauth"
|| current == "openai_codex"
|| registry.is_known(&current);
if is_known && confirm("Keep current provider?", true).map_err(SetupError::Io)? {
if current == "bedrock" {
// Keeping the existing Bedrock config — no need to re-run
// the full setup flow (region, auth, cross-region).
print_info("Keeping existing AWS Bedrock configuration.");
return Ok(());
}
if current == "gemini_oauth" || current == "gemini-oauth" {
print_info("Keeping existing Gemini CLI OAuth configuration.");
return Ok(());
}
if current == "openai_codex" {
print_info("Keeping existing OpenAI Codex configuration.");
return Ok(());
@@ -1113,13 +1130,15 @@ impl SetupWizard {
print_info("Select your inference provider:");
println!();
// Build menu: NearAI first, then OpenAI Codex, then registry providers, then Bedrock
// Build menu: NearAI first, then Gemini OAuth, then OpenAI Codex, then registry providers, then Bedrock
let selectable = registry.selectable();
let mut options: Vec<String> = Vec::with_capacity(2 + selectable.len());
let mut provider_ids: Vec<String> = Vec::with_capacity(2 + selectable.len());
let mut options: Vec<String> = Vec::with_capacity(3 + selectable.len());
let mut provider_ids: Vec<String> = Vec::with_capacity(3 + selectable.len());
options.push("NEAR AI - multi-model access via NEAR account".to_string());
provider_ids.push("nearai".to_string());
options.push("Gemini CLI - Official Gemini API via Gemini CLI OAuth".to_string());
provider_ids.push("gemini_oauth".to_string());
options.push("OpenAI Codex - ChatGPT subscription (Plus/Pro/Max)".to_string());
provider_ids.push("openai_codex".to_string());
@@ -1147,6 +1166,8 @@ impl SetupWizard {
if selected_id == "bedrock" {
self.setup_bedrock().await?;
} else if selected_id == "gemini_oauth" {
self.setup_gemini_oauth().await?;
} else {
self.run_provider_setup(selected_id, &registry).await?;
}
@@ -1795,6 +1816,40 @@ impl SetupWizard {
Ok(())
}
async fn setup_gemini_oauth(&mut self) -> Result<(), SetupError> {
self.settings.llm_backend = Some("gemini_oauth".to_string());
print_info("Starting Gemini CLI OAuth authentication...");
println!();
let creds_path = crate::config::GeminiOauthConfig::default_credentials_path();
let cred_manager =
crate::llm::gemini_oauth::CredentialManager::new(&creds_path).map_err(|e| {
SetupError::Config(format!(
"Failed to initialize Gemini credential manager: {}",
e
))
})?;
match cred_manager.get_valid_credential().await {
Ok(cred) => {
print_success("Gemini CLI authentication successful!");
if let Some(ref pid) = cred.project_id {
print_info(&format!("Cloud Code project: {}", pid));
}
}
Err(e) => {
return Err(SetupError::Config(format!(
"Gemini CLI authentication failed: {}. Please try again.",
e
)));
}
}
println!();
print_success("Gemini API configured via Gemini CLI");
Ok(())
}
/// Step 4: Model selection.
///
/// Branches on the selected LLM backend and fetches models from the
@@ -1818,109 +1873,157 @@ impl SetupWizard {
let backend = self.settings.llm_backend.as_deref().unwrap_or("nearai");
let registry = crate::llm::ProviderRegistry::load();
if backend == "nearai" {
// NEAR AI: use existing provider list_models()
let fetched = self.fetch_nearai_models().await;
let models = if fetched.is_empty() {
crate::llm::default_models()
} else {
fetched.iter().map(|m| (m.clone(), m.clone())).collect()
};
self.select_from_model_list(&models)?;
} else if let Some(def) = registry.find(backend) {
let can_list = def
.setup
.as_ref()
.map(|s| s.can_list_models())
.unwrap_or(false);
if can_list {
// Try to fetch models from the provider's /v1/models endpoint
let cached_key = self
.llm_api_key
.as_ref()
.map(|k| k.expose_secret().to_string());
let models = match backend {
"anthropic" => fetch_anthropic_models(cached_key.as_deref()).await,
"openai" => fetch_openai_models(cached_key.as_deref()).await,
"ollama" => {
let base_url = self
.settings
.ollama_base_url
.as_deref()
.or(def.default_base_url.as_deref())
.unwrap_or("http://localhost:11434");
let models = fetch_ollama_models(base_url).await;
if models.is_empty() {
print_info("No models found. Pull one first: ollama pull llama3");
}
models
}
_ => {
// Generic OpenAI-compatible model listing
let base_url = def.default_base_url.as_deref().unwrap_or("");
fetch_openai_compatible_models(base_url, cached_key.as_deref()).await
}
};
// Apply models_filter from setup hint (e.g., Groq "chat" filters non-chat models)
let models =
if let Some(filter) = def.setup.as_ref().and_then(|s| s.models_filter()) {
let filter_lower = filter.to_lowercase();
models
.into_iter()
.filter(|(id, _)| id.to_lowercase().contains(&filter_lower))
.collect()
} else {
models
};
if models.is_empty() {
// Fall back to manual entry
let default = &def.default_model;
let model_id = input(&format!("Model name (default: {default})"))
.map_err(SetupError::Io)?;
let model_id = if model_id.is_empty() {
default.clone()
} else {
model_id
};
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
match backend {
"nearai" => {
// NEAR AI: use existing provider list_models()
let fetched = self.fetch_nearai_models().await;
let models = if fetched.is_empty() {
crate::llm::default_models()
} else {
self.select_from_model_list(&models)?;
}
} else {
// Manual model entry
let default = &def.default_model;
fetched.iter().map(|m| (m.clone(), m.clone())).collect()
};
self.select_from_model_list(&models)?;
}
"gemini_oauth" | "gemini-oauth" => {
let default_models: Vec<(String, String)> = vec![
(
"gemini-3.1-pro-preview".into(),
"Gemini 3.1 Pro (Latest, strongest reasoning)".into(),
),
(
"gemini-3.1-pro-preview-customtools".into(),
"Gemini 3.1 Pro Custom Tools (Enhanced tool use)".into(),
),
(
"gemini-3-pro-preview".into(),
"Gemini 3 Pro (Preview)".into(),
),
(
"gemini-3-flash-preview".into(),
"Gemini 3 Flash (Fast preview with thinking)".into(),
),
(
"gemini-3.1-flash-lite-preview".into(),
"Gemini 3.1 Flash Lite (Preview, lightweight)".into(),
),
(
"gemini-2.5-pro".into(),
"Gemini 2.5 Pro (Stable, strong reasoning)".into(),
),
(
"gemini-2.5-flash".into(),
"Gemini 2.5 Flash (Fast, good quality)".into(),
),
(
"gemini-2.5-flash-lite".into(),
"Gemini 2.5 Flash Lite (Fastest, lightweight)".into(),
),
];
self.select_from_model_list(&default_models)?;
}
"bedrock" => {
let model_id =
input(&format!("Model name (default: {default})")).map_err(SetupError::Io)?;
let model_id = if model_id.is_empty() {
default.clone()
} else {
model_id
};
input("Bedrock model ID (e.g., anthropic.claude-v3-sonnet-20240229-v1:0)")
.map_err(SetupError::Io)?;
if model_id.is_empty() {
return Err(SetupError::Config("Model ID is required".to_string()));
}
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
}
} else if backend == "bedrock" {
let model_id = input("Bedrock model ID (e.g., anthropic.claude-opus-4-6-v1)")
.map_err(SetupError::Io)?;
if model_id.is_empty() {
return Err(SetupError::Config("Model ID is required".to_string()));
_ => {
if let Some(def) = registry.find(backend) {
let can_list = def
.setup
.as_ref()
.map(|s| s.can_list_models())
.unwrap_or(false);
if can_list {
// Try to fetch models from the provider's /v1/models endpoint
let cached_key = self
.llm_api_key
.as_ref()
.map(|k| k.expose_secret().to_string());
let models = match backend {
"anthropic" => fetch_anthropic_models(cached_key.as_deref()).await,
"openai" => fetch_openai_models(cached_key.as_deref()).await,
"ollama" => {
let base_url = self
.settings
.ollama_base_url
.as_deref()
.or(def.default_base_url.as_deref())
.unwrap_or("http://localhost:11434");
let models = fetch_ollama_models(base_url).await;
if models.is_empty() {
print_info(
"No models found. Pull one first: ollama pull llama3",
);
}
models
}
_ => {
// Generic OpenAI-compatible model listing
let base_url = def.default_base_url.as_deref().unwrap_or("");
fetch_openai_compatible_models(base_url, cached_key.as_deref())
.await
}
};
// Apply models_filter from setup hint
let models = if let Some(filter) =
def.setup.as_ref().and_then(|s| s.models_filter())
{
let filter_lower = filter.to_lowercase();
models
.into_iter()
.filter(|(id, _)| id.to_lowercase().contains(&filter_lower))
.collect()
} else {
models
};
if models.is_empty() {
// Fall back to manual entry
let default = &def.default_model;
let model_id = input(&format!("Model name (default: {default})"))
.map_err(SetupError::Io)?;
let model_id = if model_id.is_empty() {
default.clone()
} else {
model_id
};
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
} else {
self.select_from_model_list(&models)?;
}
} else {
// Manual model entry
let default = &def.default_model;
let model_id = input(&format!("Model name (default: {default})"))
.map_err(SetupError::Io)?;
let model_id = if model_id.is_empty() {
default.clone()
} else {
model_id
};
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
}
} else {
// Unknown provider, manual entry
let model_id = input("Model name (e.g., meta-llama/Llama-3-8b-chat-hf)")
.map_err(SetupError::Io)?;
if model_id.is_empty() {
return Err(SetupError::Config("Model name is required".to_string()));
}
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
}
}
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
} else {
// Unknown provider, manual entry
let model_id = input("Model name (e.g., meta-llama/Llama-3-8b-chat-hf)")
.map_err(SetupError::Io)?;
if model_id.is_empty() {
return Err(SetupError::Config("Model name is required".to_string()));
}
self.settings.selected_model = Some(model_id.clone());
print_success(&format!("Selected {}", model_id));
}
Ok(())
+17 -5
View File
@@ -45,11 +45,23 @@ impl ToolInfoDetail {
}
fn schema_param_names(schema: &serde_json::Value) -> Vec<String> {
schema
.get("properties")
.and_then(|p| p.as_object())
.map(|props| props.keys().cloned().collect())
.unwrap_or_default()
let mut names = std::collections::BTreeSet::new();
if let Some(props) = schema.get("properties").and_then(|p| p.as_object()) {
names.extend(props.keys().cloned());
}
for key in ["allOf", "oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for variant in variants {
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
names.extend(props.keys().cloned());
}
}
}
}
names.into_iter().collect()
}
fn fallback_summary(schema: &serde_json::Value) -> ToolDiscoverySummary {
+701 -12
View File
@@ -1,4 +1,4 @@
pub(crate) fn prepare_tool_params(
pub fn prepare_tool_params(
tool: &dyn crate::tools::tool::Tool,
params: &serde_json::Value,
) -> serde_json::Value {
@@ -9,14 +9,87 @@ pub(crate) fn prepare_params_for_schema(
params: &serde_json::Value,
schema: &serde_json::Value,
) -> serde_json::Value {
coerce_value(params, schema)
let resolved = resolve_refs(schema);
coerce_value(params, &resolved)
}
// ── $ref resolution ──────────────────────────────────────────────────
/// Inline all `$ref` pointers in a JSON Schema so downstream coercion
/// operates on a flat, self-contained schema tree.
///
/// Supports `#/definitions/<name>` and `#/$defs/<name>` (JSON Schema
/// draft-07 and 2020-12 respectively). Unknown `$ref` formats are left
/// unchanged. A depth limit prevents infinite recursion from circular refs.
fn resolve_refs(schema: &serde_json::Value) -> serde_json::Value {
let definitions = schema
.get("definitions")
.or_else(|| schema.get("$defs"))
.cloned()
.unwrap_or(serde_json::Value::Null);
resolve_refs_inner(schema, &definitions, 0)
}
const MAX_REF_DEPTH: usize = 16;
fn resolve_refs_inner(
schema: &serde_json::Value,
definitions: &serde_json::Value,
depth: usize,
) -> serde_json::Value {
if depth > MAX_REF_DEPTH {
return schema.clone();
}
match schema {
serde_json::Value::Object(obj) => {
// If this node is a $ref, resolve it and recurse into the target.
if let Some(ref_str) = obj.get("$ref").and_then(|v| v.as_str()) {
if let Some(target) = resolve_ref_pointer(ref_str, definitions) {
return resolve_refs_inner(&target, definitions, depth + 1);
}
return schema.clone();
}
// Recursively resolve refs in all values (skip definitions maps).
let resolved: serde_json::Map<String, serde_json::Value> = obj
.iter()
.map(|(k, v)| {
if k == "definitions" || k == "$defs" {
(k.clone(), v.clone())
} else {
(k.clone(), resolve_refs_inner(v, definitions, depth + 1))
}
})
.collect();
serde_json::Value::Object(resolved)
}
serde_json::Value::Array(arr) => serde_json::Value::Array(
arr.iter()
.map(|v| resolve_refs_inner(v, definitions, depth + 1))
.collect(),
),
_ => schema.clone(),
}
}
fn resolve_ref_pointer(
ref_str: &str,
definitions: &serde_json::Value,
) -> Option<serde_json::Value> {
let path = ref_str.strip_prefix("#/")?;
let parts: Vec<&str> = path.split('/').collect();
if parts.len() == 2 && (parts[0] == "definitions" || parts[0] == "$defs") {
return definitions.get(parts[1]).cloned();
}
None
}
// ── Core coercion ────────────────────────────────────────────────────
fn coerce_value(value: &serde_json::Value, schema: &serde_json::Value) -> serde_json::Value {
// This coercer intentionally handles the concrete schema shapes we expose in
// discovery today. It does not resolve combinators like anyOf/oneOf/allOf or
// references via $ref; those schemas pass through unchanged unless they also
// advertise a directly coercible type/property shape.
// This coercer handles concrete schema shapes including discriminated unions
// (oneOf/anyOf with const or single-element enum discriminators), allOf
// merges, and $ref references (resolved in a pre-pass).
if value.is_null() {
return value.clone();
}
@@ -47,12 +120,35 @@ fn coerce_value(value: &serde_json::Value, schema: &serde_json::Value) -> serde_
return value.clone();
}
let properties = schema.get("properties").and_then(|p| p.as_object());
let additional_schema = schema.get("additionalProperties").filter(|v| v.is_object());
let resolved = resolve_effective_properties(schema, obj);
let properties = resolved
.as_ref()
.or_else(|| schema.get("properties").and_then(|p| p.as_object()));
let additional_schema = schema
.get("additionalProperties")
.filter(|v| v.is_object())
.or_else(|| resolve_additional_properties(schema, obj));
let required: std::collections::HashSet<&str> = schema
.get("required")
.and_then(|r| r.as_array())
.map(|arr| arr.iter().filter_map(|v| v.as_str()).collect())
.unwrap_or_default();
let mut coerced = obj.clone();
for (key, current) in &mut coerced {
if let Some(prop_schema) = properties.and_then(|props| props.get(key)) {
// LLMs send "" for optional fields instead of omitting them.
// Coerce to null only when the field is not required AND the schema
// allows null or doesn't allow string — a `type: "string"` field
// may legitimately accept "" as a meaningful value.
if current.as_str() == Some("")
&& !required.contains(key.as_str())
&& (schema_allows_type(prop_schema, "null")
|| !schema_allows_type(prop_schema, "string"))
{
*current = serde_json::Value::Null;
continue;
}
*current = coerce_value(current, prop_schema);
continue;
}
@@ -68,11 +164,179 @@ fn coerce_value(value: &serde_json::Value, schema: &serde_json::Value) -> serde_
value.clone()
}
/// When the schema uses `oneOf`, `anyOf`, or `allOf` combinators, build a
/// merged property map that can be used for coercion.
///
/// - Top-level `properties` are included first (base properties).
/// - `allOf`: merge ALL variants' properties (last-wins on conflicts).
/// - `oneOf`/`anyOf`: find the discriminated match and merge its properties.
///
/// Returns `None` if no combinators are present or no match is found, so the
/// caller falls back to the existing top-level `properties` lookup.
fn resolve_effective_properties(
schema: &serde_json::Value,
obj: &serde_json::Map<String, serde_json::Value>,
) -> Option<serde_json::Map<String, serde_json::Value>> {
collect_properties(schema, obj, 0)
}
const MAX_COMBINATOR_DEPTH: usize = 4;
/// Recursively collect properties from a schema and its combinator variants.
fn collect_properties(
schema: &serde_json::Value,
obj: &serde_json::Map<String, serde_json::Value>,
depth: usize,
) -> Option<serde_json::Map<String, serde_json::Value>> {
if depth > MAX_COMBINATOR_DEPTH {
return None;
}
let has_combinators = schema.get("allOf").is_some()
|| schema.get("oneOf").is_some()
|| schema.get("anyOf").is_some();
if !has_combinators {
return None;
}
let mut merged = serde_json::Map::new();
// Start with top-level properties
if let Some(props) = schema.get("properties").and_then(|p| p.as_object()) {
merged.extend(props.iter().map(|(k, v)| (k.clone(), v.clone())));
}
// allOf: merge ALL variants' properties, recursing into nested combinators
if let Some(all_of) = schema.get("allOf").and_then(|a| a.as_array()) {
for variant in all_of {
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
merged.extend(props.iter().map(|(k, v)| (k.clone(), v.clone())));
}
// Recurse into variant if it has its own combinators
if let Some(nested) = collect_properties(variant, obj, depth + 1) {
merged.extend(nested);
}
}
}
// oneOf/anyOf: find discriminated match and merge its properties
for key in ["oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& let Some(variant) = find_discriminated_variant(variants, obj)
{
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
merged.extend(props.iter().map(|(k, v)| (k.clone(), v.clone())));
}
// Recurse into matched variant if it has its own combinators
if let Some(nested) = collect_properties(variant, obj, depth + 1) {
merged.extend(nested);
}
}
}
if merged.is_empty() {
None
} else {
Some(merged)
}
}
/// Find `additionalProperties` from a matched combinator variant.
///
/// Checks `allOf` variants first (last-wins), then the matched `oneOf`/`anyOf`
/// variant. Returns `None` if no variant defines `additionalProperties`.
fn resolve_additional_properties<'a>(
schema: &'a serde_json::Value,
obj: &serde_json::Map<String, serde_json::Value>,
) -> Option<&'a serde_json::Value> {
// allOf: last variant with additionalProperties wins
if let Some(all_of) = schema.get("allOf").and_then(|a| a.as_array()) {
for variant in all_of.iter().rev() {
if let Some(ap) = variant.get("additionalProperties")
&& ap.is_object()
{
return Some(ap);
}
}
}
// oneOf/anyOf: check matched variant
for key in ["oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& let Some(variant) = find_discriminated_variant(variants, obj)
&& let Some(ap) = variant.get("additionalProperties")
&& ap.is_object()
{
return Some(ap);
}
}
None
}
/// Find a `oneOf`/`anyOf` variant that matches the given object by checking
/// `const`-valued and single-element `enum`-valued properties (discriminators).
///
/// A variant matches when ALL its discriminator properties match the object's
/// values and at least one such discriminator exists. Returns `None` if no
/// variant matches (safe fallback — no coercion).
fn find_discriminated_variant<'a>(
variants: &'a [serde_json::Value],
obj: &serde_json::Map<String, serde_json::Value>,
) -> Option<&'a serde_json::Value> {
variants.iter().find(|variant| {
let Some(props) = variant.get("properties").and_then(|p| p.as_object()) else {
return false;
};
let mut discriminator_count = 0;
for (key, prop_schema) in props {
// Check for const discriminator
if let Some(const_val) = prop_schema.get("const") {
discriminator_count += 1;
match obj.get(key) {
Some(v) if v == const_val => {}
_ => return false,
}
continue;
}
// Check for single-element enum discriminator
if let Some(enum_vals) = prop_schema.get("enum").and_then(|e| e.as_array())
&& enum_vals.len() == 1
{
discriminator_count += 1;
match obj.get(key) {
Some(v) if v == &enum_vals[0] => {}
_ => return false,
}
}
}
discriminator_count > 0
})
}
fn coerce_string_value(s: &str, schema: &serde_json::Value) -> Option<serde_json::Value> {
// LLMs often send "" instead of null for optional fields. Coerce empty
// strings to null when the schema allows null but not string, or allows
// both but the value is empty (a string field with content "" is kept).
if s.is_empty() && schema_allows_type(schema, "null") && !schema_allows_type(schema, "string") {
return Some(serde_json::Value::Null);
}
if schema_allows_type(schema, "string") {
return None;
}
// Empty string with no type match — return unchanged since we can't
// determine the intended type.
if s.is_empty() {
return None;
}
if schema_allows_type(schema, "integer")
&& let Ok(v) = s.parse::<i64>()
{
@@ -114,10 +378,15 @@ fn schema_allows_type(schema: &serde_json::Value, expected: &str) -> bool {
Some(serde_json::Value::String(t)) => t == expected,
Some(serde_json::Value::Array(types)) => types.iter().any(|t| t.as_str() == Some(expected)),
_ => match expected {
"object" => schema
.get("properties")
.and_then(|p| p.as_object())
.is_some(),
"object" => {
schema
.get("properties")
.and_then(|p| p.as_object())
.is_some()
|| schema.get("oneOf").is_some()
|| schema.get("anyOf").is_some()
|| schema.get("allOf").is_some()
}
"array" => schema.get("items").is_some(),
_ => false,
},
@@ -325,6 +594,91 @@ mod tests {
assert_eq!(result["value"], serde_json::json!("{\"mode\":\"raw\"}")); // safety: test-only assertion
}
#[test]
fn coerces_empty_string_to_null_for_nullable_non_required_field() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"timezone": { "type": ["string", "null"] },
"schedule": { "type": "string" }
},
"required": ["schedule"]
});
let params = serde_json::json!({
"timezone": "",
"schedule": "0 9 * * *"
});
let result = prepare_params_for_schema(&params, &schema);
// Non-required nullable "timezone" with empty string → null
assert_eq!(result["timezone"], serde_json::Value::Null);
// Required "schedule" keeps its value even if empty would be weird
assert_eq!(result["schedule"], serde_json::json!("0 9 * * *"));
}
#[test]
fn keeps_empty_string_for_non_required_string_only_field() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"timezone": { "type": "string" },
"schedule": { "type": "string" }
},
"required": ["schedule"]
});
let params = serde_json::json!({
"timezone": "",
"schedule": "0 9 * * *"
});
let result = prepare_params_for_schema(&params, &schema);
// Non-required string-only "timezone" keeps empty string (meaningful value)
assert_eq!(result["timezone"], serde_json::json!(""));
assert_eq!(result["schedule"], serde_json::json!("0 9 * * *"));
}
#[test]
fn coerces_empty_string_to_null_for_explicit_nullable_type() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"from_timezone": { "type": ["string", "null"] },
"operation": { "type": "string" }
},
"required": ["operation"]
});
let params = serde_json::json!({
"from_timezone": "",
"operation": "now"
});
let result = prepare_params_for_schema(&params, &schema);
// Nullable type with empty string → null (even if it were required,
// the per-value coercion in coerce_string_value handles this)
assert_eq!(result["from_timezone"], serde_json::Value::Null);
assert_eq!(result["operation"], serde_json::json!("now"));
}
#[test]
fn keeps_empty_string_for_required_string_only_field() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"name": { "type": "string" }
},
"required": ["name"]
});
let params = serde_json::json!({ "name": "" });
let result = prepare_params_for_schema(&params, &schema);
// Required string-only field keeps empty string
assert_eq!(result["name"], serde_json::json!(""));
}
#[test]
fn permissive_schema_is_noop() {
let schema = serde_json::json!({
@@ -339,6 +693,341 @@ mod tests {
assert_eq!(result["count"], serde_json::json!("10")); // safety: test-only assertion
}
#[test]
fn coerces_oneof_discriminated_variant() {
let schema = serde_json::json!({
"oneOf": [
{
"type": "object",
"properties": {
"action": { "const": "list_repos" },
"limit": { "type": "integer" },
"sort": { "type": "string" }
}
},
{
"type": "object",
"properties": {
"action": { "const": "get_repo" },
"repo": { "type": "string" }
}
}
]
});
let params = serde_json::json!({
"action": "list_repos",
"limit": "100",
"sort": "stars"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["action"], serde_json::json!("list_repos"));
assert_eq!(result["limit"], serde_json::json!(100));
assert_eq!(result["sort"], serde_json::json!("stars"));
}
#[test]
fn coerces_oneof_with_enum_discriminator() {
let schema = serde_json::json!({
"oneOf": [
{
"type": "object",
"properties": {
"mode": { "enum": ["fetch"] },
"count": { "type": "integer" }
}
},
{
"type": "object",
"properties": {
"mode": { "enum": ["push"] },
"force": { "type": "boolean" }
}
}
]
});
let params = serde_json::json!({
"mode": "push",
"force": "true"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["mode"], serde_json::json!("push"));
assert_eq!(result["force"], serde_json::json!(true));
}
#[test]
fn coerces_allof_merged_properties() {
let schema = serde_json::json!({
"allOf": [
{
"type": "object",
"properties": {
"page": { "type": "integer" }
}
},
{
"type": "object",
"properties": {
"per_page": { "type": "integer" },
"verbose": { "type": "boolean" }
}
}
]
});
let params = serde_json::json!({
"page": "2",
"per_page": "50",
"verbose": "false"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["page"], serde_json::json!(2));
assert_eq!(result["per_page"], serde_json::json!(50));
assert_eq!(result["verbose"], serde_json::json!(false));
}
#[test]
fn oneof_no_discriminator_match_is_noop() {
let schema = serde_json::json!({
"oneOf": [
{
"type": "object",
"properties": {
"action": { "const": "list_repos" },
"limit": { "type": "integer" }
}
},
{
"type": "object",
"properties": {
"action": { "const": "get_repo" },
"repo": { "type": "string" }
}
}
]
});
let params = serde_json::json!({
"action": "unknown_action",
"limit": "100"
});
let result = prepare_params_for_schema(&params, &schema);
// No variant matched, so no coercion happens
assert_eq!(result["limit"], serde_json::json!("100"));
}
#[test]
fn anyof_without_discriminator_is_noop() {
let schema = serde_json::json!({
"anyOf": [
{
"type": "object",
"properties": {
"name": { "type": "string" }
},
"required": ["name"]
},
{
"type": "object",
"properties": {
"id": { "type": "integer" }
},
"required": ["id"]
}
]
});
let params = serde_json::json!({
"id": "42"
});
let result = prepare_params_for_schema(&params, &schema);
// No const/enum discriminators, so no variant matches, no coercion
assert_eq!(result["id"], serde_json::json!("42"));
}
#[test]
fn resolves_ref_and_coerces_referenced_properties() {
let schema = serde_json::json!({
"type": "object",
"definitions": {
"Pagination": {
"type": "object",
"properties": {
"page": { "type": "integer" },
"per_page": { "type": "integer" }
}
}
},
"allOf": [
{ "$ref": "#/definitions/Pagination" },
{
"type": "object",
"properties": {
"query": { "type": "string" }
}
}
]
});
let params = serde_json::json!({
"page": "2",
"per_page": "50",
"query": "test"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["page"], serde_json::json!(2));
assert_eq!(result["per_page"], serde_json::json!(50));
assert_eq!(result["query"], serde_json::json!("test"));
}
#[test]
fn resolves_nested_refs_in_oneof_variants() {
let schema = serde_json::json!({
"type": "object",
"$defs": {
"ListParams": {
"properties": {
"action": { "const": "list" },
"limit": { "type": "integer" }
}
}
},
"oneOf": [
{ "$ref": "#/$defs/ListParams" },
{
"properties": {
"action": { "const": "get" },
"id": { "type": "integer" }
}
}
]
});
let params = serde_json::json!({
"action": "list",
"limit": "25"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["limit"], serde_json::json!(25));
}
#[test]
fn coerces_nested_combinators_allof_containing_oneof() {
// allOf where one variant is itself a oneOf (nested combinator)
let schema = serde_json::json!({
"type": "object",
"allOf": [
{
"properties": {
"version": { "type": "integer" }
}
},
{
"oneOf": [
{
"properties": {
"mode": { "const": "fast" },
"threads": { "type": "integer" }
}
},
{
"properties": {
"mode": { "const": "safe" },
"retries": { "type": "integer" }
}
}
]
}
]
});
let params = serde_json::json!({
"version": "3",
"mode": "fast",
"threads": "8"
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["version"], serde_json::json!(3));
assert_eq!(result["threads"], serde_json::json!(8));
}
#[test]
fn coerces_array_items_with_oneof_discriminator() {
let schema = serde_json::json!({
"type": "object",
"properties": {
"actions": {
"type": "array",
"items": {
"oneOf": [
{
"type": "object",
"properties": {
"type": { "const": "move" },
"distance": { "type": "integer" }
}
},
{
"type": "object",
"properties": {
"type": { "const": "wait" },
"seconds": { "type": "number" }
}
}
]
}
}
}
});
let params = serde_json::json!({
"actions": [
{ "type": "move", "distance": "10" },
{ "type": "wait", "seconds": "2.5" }
]
});
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["actions"][0]["distance"], serde_json::json!(10));
assert_eq!(result["actions"][1]["seconds"], serde_json::json!(2.5));
}
#[test]
fn circular_ref_does_not_infinite_loop() {
let schema = serde_json::json!({
"type": "object",
"definitions": {
"Node": {
"type": "object",
"properties": {
"value": { "type": "integer" },
"child": { "$ref": "#/definitions/Node" }
}
}
},
"properties": {
"root": { "$ref": "#/definitions/Node" }
}
});
let params = serde_json::json!({
"root": { "value": "42" }
});
// Should not hang — depth limit stops the recursion
let result = prepare_params_for_schema(&params, &schema);
assert_eq!(result["root"]["value"], serde_json::json!(42));
}
#[test]
fn prepare_tool_params_uses_discovery_schema() {
let tool = StubTool {
+83 -5
View File
@@ -42,11 +42,38 @@ pub fn validate_strict_schema(
}
}
/// Returns true if the schema uses `oneOf`, `anyOf`, or `allOf` combinators
/// where at least one variant is an object type (has `type: "object"` or `properties`).
fn has_object_combinator_variants(schema: &serde_json::Value) -> bool {
for key in ["oneOf", "anyOf", "allOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& variants.iter().any(|v| {
v.get("type").and_then(|t| t.as_str()) == Some("object")
|| v.get("properties").is_some()
})
{
return true;
}
}
false
}
/// Recursively validate an object-typed schema node.
fn check_object_schema(schema: &serde_json::Value, path: &str) -> Vec<String> {
let mut errors = Vec::new();
// Rule 1: must have "type": "object"
// Report non-array combinator values as errors.
for key in ["oneOf", "anyOf", "allOf"] {
if let Some(val) = schema.get(key)
&& !val.is_array()
{
errors.push(format!("{path}: \"{key}\" must be an array"));
}
}
let has_combinators = has_object_combinator_variants(schema);
// Rule 1: must have "type": "object" (unless combinators define the structure)
match schema.get("type").and_then(|t| t.as_str()) {
Some("object") => {}
Some(other) => {
@@ -54,16 +81,67 @@ fn check_object_schema(schema: &serde_json::Value, path: &str) -> Vec<String> {
return errors;
}
None => {
errors.push(format!("{path}: missing \"type\": \"object\""));
return errors;
if !has_combinators {
errors.push(format!("{path}: missing \"type\": \"object\""));
return errors;
}
}
}
// Rule 2: must have "properties" as an object
// Validate combinator variants recursively
for key in ["allOf", "oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for (i, variant) in variants.iter().enumerate() {
if variant.get("type").and_then(|t| t.as_str()) == Some("object")
|| variant.get("properties").is_some()
{
let variant_path = format!("{path}.{key}[{i}]");
errors.extend(check_object_schema(variant, &variant_path));
}
}
}
}
// Rule 2: must have "properties" as an object (unless combinators define them)
let properties = match schema.get("properties").and_then(|p| p.as_object()) {
Some(p) => p,
None => {
errors.push(format!("{path}: missing or non-object \"properties\""));
if !has_combinators {
errors.push(format!("{path}: missing or non-object \"properties\""));
return errors;
}
// Combinators define the structure — validate top-level `required` keys
// against merged properties from all combinator variants.
if let Some(required) = schema.get("required").and_then(|r| r.as_array()) {
let mut merged_keys = std::collections::HashSet::new();
if let Some(all_of) = schema.get("allOf").and_then(|a| a.as_array()) {
for variant in all_of {
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
merged_keys.extend(props.keys().cloned());
}
}
}
for key in ["oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for variant in variants {
if let Some(props) =
variant.get("properties").and_then(|p| p.as_object())
{
merged_keys.extend(props.keys().cloned());
}
}
}
}
for req in required {
if let Some(key) = req.as_str()
&& !merged_keys.contains(key)
{
errors.push(format!(
"{path}: required key \"{key}\" not found in any combinator variant properties"
));
}
}
}
return errors;
}
};
+87 -5
View File
@@ -462,6 +462,22 @@ pub fn redact_params(params: &serde_json::Value, sensitive: &[&str]) -> serde_js
/// on maliciously crafted schemas.
const MAX_SCHEMA_DEPTH: usize = 16;
/// Returns true if the schema uses `oneOf`, `anyOf`, or `allOf` combinators
/// where at least one variant is an object type (has `type: "object"` or `properties`).
fn has_object_combinator_variants(schema: &serde_json::Value) -> bool {
for key in ["oneOf", "anyOf", "allOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& variants.iter().any(|v| {
v.get("type").and_then(|t| t.as_str()) == Some("object")
|| v.get("properties").is_some()
})
{
return true;
}
}
false
}
pub fn validate_tool_schema(schema: &serde_json::Value, path: &str) -> Vec<String> {
validate_tool_schema_inner(schema, path, 0)
}
@@ -476,7 +492,18 @@ fn validate_tool_schema_inner(schema: &serde_json::Value, path: &str, depth: usi
return errors;
}
// Rule 1: must have "type": "object" at this level
// Report non-array combinator values as errors.
for key in ["oneOf", "anyOf", "allOf"] {
if let Some(val) = schema.get(key)
&& !val.is_array()
{
errors.push(format!("{path}: \"{key}\" must be an array"));
}
}
let has_combinators = has_object_combinator_variants(schema);
// Rule 1: must have "type": "object" at this level (unless combinators define the structure)
match schema.get("type").and_then(|t| t.as_str()) {
Some("object") => {}
Some(other) => {
@@ -484,16 +511,71 @@ fn validate_tool_schema_inner(schema: &serde_json::Value, path: &str, depth: usi
return errors; // Can't check further
}
None => {
errors.push(format!("{path}: missing \"type\": \"object\""));
return errors;
if !has_combinators {
errors.push(format!("{path}: missing \"type\": \"object\""));
return errors;
}
}
}
// Rule 2: must have "properties" as an object
// Validate combinator variants recursively
for key in ["allOf", "oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for (i, variant) in variants.iter().enumerate() {
if variant.get("type").and_then(|t| t.as_str()) == Some("object")
|| variant.get("properties").is_some()
{
let variant_path = format!("{path}.{key}[{i}]");
errors.extend(validate_tool_schema_inner(
variant,
&variant_path,
depth + 1,
));
}
}
}
}
// Rule 2: must have "properties" as an object (unless combinators define them)
let properties = match schema.get("properties").and_then(|p| p.as_object()) {
Some(p) => p,
None => {
errors.push(format!("{path}: missing or non-object \"properties\""));
if !has_combinators {
errors.push(format!("{path}: missing or non-object \"properties\""));
return errors;
}
// Combinators define the structure — validate top-level `required` keys
// against merged properties from all combinator variants.
if let Some(required) = schema.get("required").and_then(|r| r.as_array()) {
let mut merged_keys = std::collections::HashSet::new();
if let Some(all_of) = schema.get("allOf").and_then(|a| a.as_array()) {
for variant in all_of {
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
merged_keys.extend(props.keys().cloned());
}
}
}
for key in ["oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for variant in variants {
if let Some(props) =
variant.get("properties").and_then(|p| p.as_object())
{
merged_keys.extend(props.keys().cloned());
}
}
}
}
for req in required {
if let Some(key) = req.as_str()
&& !merged_keys.contains(key)
{
errors.push(format!(
"{path}: required key \"{key}\" not found in any combinator variant properties"
));
}
}
}
return errors;
}
};
+99
View File
@@ -708,6 +708,9 @@ pub struct ToolSetupSchema {
/// Secrets the user must provide before the tool can be used.
#[serde(default)]
pub required_secrets: Vec<ToolSecretSetupSchema>,
/// Non-secret fields the user can configure in the setup modal.
#[serde(default)]
pub required_fields: Vec<ToolFieldSetupSchema>,
}
/// A single secret required during tool setup.
@@ -722,6 +725,46 @@ pub struct ToolSecretSetupSchema {
pub optional: bool,
}
/// A non-secret field required during tool setup.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ToolFieldSetupSchema {
/// Field name in setup payload.
pub name: String,
/// User-facing prompt shown in the setup modal.
pub prompt: String,
/// If true, the user may skip this field.
#[serde(default)]
pub optional: bool,
/// Input type used in the setup modal.
#[serde(default = "default_tool_setup_field_input_type")]
pub input_type: ToolSetupFieldInputType,
/// Optional dotted setting path to persist this value to.
///
/// Restricted by the host to extension-owned namespaces and a small
/// allowlist of approved global settings.
///
/// Example: `extensions.switch-llm.provider`, `llm_backend`, or
/// `selected_model`.
#[serde(default)]
pub setting_path: Option<String>,
/// Whether changing this field requires a restart to fully apply.
#[serde(default)]
pub restart_required: bool,
}
/// Input widget type for a setup field.
#[derive(Debug, Clone, Copy, Default, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum ToolSetupFieldInputType {
#[default]
Text,
Password,
}
fn default_tool_setup_field_input_type() -> ToolSetupFieldInputType {
ToolSetupFieldInputType::Text
}
#[cfg(test)]
mod tests {
use crate::tools::wasm::capabilities_schema::{CapabilitiesFile, CredentialLocationSchema};
@@ -1218,6 +1261,20 @@ mod tests {
"prompt": "Google OAuth Client Secret",
"optional": true
}
],
"required_fields": [
{
"name": "llm_backend",
"prompt": "LLM Provider",
"setting_path": "llm_backend",
"restart_required": true
},
{
"name": "selected_model",
"prompt": "Model Name",
"input_type": "text",
"setting_path": "selected_model"
}
]
}
}"#;
@@ -1230,6 +1287,48 @@ mod tests {
assert!(!setup.required_secrets[0].optional);
assert_eq!(setup.required_secrets[1].name, "google_oauth_client_secret");
assert!(setup.required_secrets[1].optional);
assert_eq!(setup.required_fields.len(), 2);
assert_eq!(setup.required_fields[0].name, "llm_backend");
assert_eq!(
setup.required_fields[0].setting_path.as_deref(),
Some("llm_backend")
);
assert!(setup.required_fields[0].restart_required);
assert_eq!(
setup.required_fields[0].input_type,
crate::tools::wasm::capabilities_schema::ToolSetupFieldInputType::Text
);
assert_eq!(setup.required_fields[1].name, "selected_model");
}
#[test]
fn test_tool_setup_field_input_type_defaults_to_text() {
let json = r#"{
"setup": {
"required_fields": [
{
"name": "provider",
"prompt": "Provider"
},
{
"name": "token_hint",
"prompt": "Token Hint",
"input_type": "password"
}
]
}
}"#;
let caps = CapabilitiesFile::from_json(json).unwrap();
let setup = caps.setup.unwrap();
assert_eq!(
setup.required_fields[0].input_type,
crate::tools::wasm::capabilities_schema::ToolSetupFieldInputType::Text
);
assert_eq!(
setup.required_fields[1].input_type,
crate::tools::wasm::capabilities_schema::ToolSetupFieldInputType::Password
);
}
#[test]
+1 -1
View File
@@ -139,5 +139,5 @@ pub use loader::{
// Capabilities schema (for parsing *.capabilities.json files)
pub use capabilities_schema::{
AuthCapabilitySchema, CapabilitiesFile, OAuthConfigSchema, RateLimitSchema,
ValidationEndpointSchema,
ToolFieldSetupSchema, ToolSetupFieldInputType, ToolSetupSchema, ValidationEndpointSchema,
};
+184 -19
View File
@@ -17,6 +17,7 @@ use wasmtime::component::Linker;
use wasmtime_wasi::{ResourceTable, WasiCtx, WasiCtxBuilder, WasiView};
use crate::context::JobContext;
use crate::llm::recording::{HttpExchangeRequest, HttpExchangeResponse, HttpInterceptor};
use crate::safety::LeakDetector;
use crate::secrets::SecretsStore;
use crate::tools::tool::{Tool, ToolError, ToolOutput};
@@ -99,6 +100,9 @@ struct StoreData {
/// Dedicated tokio runtime for HTTP requests, lazily initialized.
/// Reused across multiple `http_request` calls within one execution.
http_runtime: Option<tokio::runtime::Runtime>,
/// Optional HTTP interceptor for testing — returns canned responses
/// instead of making real requests when set.
http_interceptor: Option<Arc<dyn HttpInterceptor>>,
}
impl StoreData {
@@ -119,6 +123,7 @@ impl StoreData {
credentials,
host_credentials,
http_runtime: None,
http_interceptor: None,
}
}
@@ -344,6 +349,59 @@ impl near::agent::host::Host for StoreData {
);
}
let rt = self.http_runtime.as_ref().expect("just initialized"); // safety: is_none branch above guarantees Some
// If an HTTP interceptor is set (testing), short-circuit with a canned response.
if let Some(interceptor) = &self.http_interceptor {
let interceptor = Arc::clone(interceptor);
let intercept_url = url.clone();
let intercept_method = method.clone();
let mut intercept_headers: Vec<(String, String)> = headers
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
intercept_headers.sort_by(|a, b| a.0.cmp(&b.0));
let intercept_body = body
.as_ref()
.map(|b| String::from_utf8_lossy(b).to_string());
let intercepted = rt.block_on(async {
let req = HttpExchangeRequest {
method: intercept_method,
url: intercept_url,
headers: intercept_headers,
body: intercept_body,
};
interceptor.before_request(&req).await
});
if let Some(resp) = intercepted {
let resp_headers: HashMap<String, String> = resp
.headers
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
let resp_headers_json =
serde_json::to_string(&resp_headers).unwrap_or_else(|_| "{}".to_string());
return Ok(near::agent::host::HttpResponse {
status: resp.status,
headers_json: resp_headers_json,
body: resp.body.into_bytes(),
});
}
}
// Capture request metadata before headers/body are consumed by the reqwest
// builder. Used for after_response callback when a recording interceptor is set.
let interceptor_req = self.http_interceptor.as_ref().map(|_| HttpExchangeRequest {
method: method.clone(),
url: url.clone(),
headers: headers
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect(),
body: body
.as_ref()
.map(|b| String::from_utf8_lossy(b).to_string()),
});
let result = rt.block_on(async {
let client = reqwest::Client::builder()
.connect_timeout(Duration::from_secs(10))
@@ -434,6 +492,51 @@ impl near::agent::host::Host for StoreData {
})
});
// Notify the interceptor about the completed response (recording mode).
// RecordingHttpInterceptor returns None from before_request and captures
// exchanges via after_response, so this path is exercised during trace recording.
if let (Some(interceptor), Some(req), Ok(resp)) =
(&self.http_interceptor, &interceptor_req, &result)
{
let interceptor = Arc::clone(interceptor);
// Redact credentials from request before passing to the interceptor
// to prevent credential leakage into recorded traces.
let mut redacted_req = req.clone();
redacted_req.url = self.redact_credentials(&redacted_req.url);
redacted_req.headers = redacted_req
.headers
.into_iter()
.map(|(k, v)| (k, self.redact_credentials(&v)))
.collect();
redacted_req.body = redacted_req.body.map(|b| self.redact_credentials(&b));
let resp_headers: Vec<(String, String)> =
serde_json::from_str::<HashMap<String, String>>(&resp.headers_json)
.unwrap_or_default()
.into_iter()
.collect();
let resp_body = String::from_utf8_lossy(&resp.body).to_string();
// Redact credentials from response as well
let redacted_headers: Vec<(String, String)> = resp_headers
.into_iter()
.map(|(k, v)| (k, self.redact_credentials(&v)))
.collect();
let redacted_body = self.redact_credentials(&resp_body);
let exchange_resp = HttpExchangeResponse {
status: resp.status,
headers: redacted_headers,
body: redacted_body,
};
rt.block_on(async {
interceptor
.after_response(&redacted_req, &exchange_resp)
.await;
});
}
// Redact credentials from error messages before returning to WASM
result.map_err(|e| self.redact_credentials(&e))
}
@@ -476,6 +579,9 @@ pub struct WasmToolWrapper {
secrets_store: Option<Arc<dyn SecretsStore + Send + Sync>>,
/// OAuth refresh configuration for auto-refreshing expired tokens.
oauth_refresh: Option<OAuthRefreshConfig>,
/// Optional HTTP interceptor for testing — returns canned responses
/// instead of making real requests when set.
http_interceptor: Option<Arc<dyn HttpInterceptor>>,
}
#[derive(Debug, Clone)]
@@ -502,23 +608,51 @@ impl WasmToolSchemas {
}
fn is_permissive_schema(schema: &serde_json::Value) -> bool {
schema
if schema
.get("properties")
.and_then(|p| p.as_object())
.is_none_or(|p| p.is_empty())
.is_some_and(|p| !p.is_empty())
{
return false;
}
// Schemas with combinator variants containing properties are not permissive
for key in ["oneOf", "anyOf", "allOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& variants.iter().any(|v| {
v.get("properties")
.and_then(|p| p.as_object())
.is_some_and(|p| !p.is_empty())
})
{
return false;
}
}
true
}
fn typed_property_count(schema: &serde_json::Value) -> usize {
schema
.get("properties")
.and_then(|p| p.as_object())
.map(|props| {
props
.values()
.filter(|prop| schema_is_typed_property(prop))
.count()
})
.unwrap_or(0)
let mut all_props = serde_json::Map::new();
if let Some(props) = schema.get("properties").and_then(|p| p.as_object()) {
all_props.extend(props.iter().map(|(k, v)| (k.clone(), v.clone())));
}
for key in ["allOf", "oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array()) {
for variant in variants {
if let Some(props) = variant.get("properties").and_then(|p| p.as_object()) {
all_props.extend(props.iter().map(|(k, v)| (k.clone(), v.clone())));
}
}
}
}
all_props
.values()
.filter(|prop| schema_is_typed_property(prop))
.count()
}
fn new(discovery: serde_json::Value) -> Self {
@@ -564,9 +698,20 @@ impl WasmToolWrapper {
credentials: HashMap::new(),
secrets_store: None,
oauth_refresh: None,
http_interceptor: None,
}
}
/// Set an HTTP interceptor for testing.
///
/// When set, WASM tool HTTP requests are routed through the interceptor
/// instead of making real network calls. This allows tests to verify the
/// exact HTTP requests a WASM tool constructs.
pub fn with_http_interceptor(mut self, interceptor: Arc<dyn HttpInterceptor>) -> Self {
self.http_interceptor = Some(interceptor);
self
}
/// Override the tool description.
pub fn with_description(mut self, description: impl Into<String>) -> Self {
self.description = description.into();
@@ -651,12 +796,13 @@ impl WasmToolWrapper {
let limits = &self.prepared.limits;
// Create store with fresh state (NEAR pattern: fresh instance per call)
let store_data = StoreData::new(
let mut store_data = StoreData::new(
limits.memory_bytes,
self.capabilities.clone(),
self.credentials.clone(),
host_credentials,
);
store_data.http_interceptor = self.http_interceptor.clone();
let mut store = Store::new(engine, store_data);
// Configure fuel if enabled
@@ -872,6 +1018,7 @@ impl Tool for WasmToolWrapper {
credentials,
secrets_store: None, // Not needed in blocking task
oauth_refresh: None, // Already used above for pre-refresh
http_interceptor: self.http_interceptor.clone(),
};
tokio::task::spawn_blocking(move || {
@@ -1320,15 +1467,33 @@ fn is_private_ip(ip: std::net::IpAddr) -> bool {
}
fn schema_contains_container_properties(schema: &serde_json::Value) -> bool {
schema
let has_container = |props: &serde_json::Map<String, serde_json::Value>| {
props
.values()
.any(|prop| schema_declares_type(prop, "array") || schema_declares_type(prop, "object"))
};
if schema
.get("properties")
.and_then(|p| p.as_object())
.map(|props| {
props.values().any(|prop| {
schema_declares_type(prop, "array") || schema_declares_type(prop, "object")
.is_some_and(has_container)
{
return true;
}
for key in ["allOf", "oneOf", "anyOf"] {
if let Some(variants) = schema.get(key).and_then(|v| v.as_array())
&& variants.iter().any(|v| {
v.get("properties")
.and_then(|p| p.as_object())
.is_some_and(has_container)
})
})
.unwrap_or(false)
{
return true;
}
}
false
}
fn schema_declares_type(schema: &serde_json::Value, expected: &str) -> bool {
+408
View File
@@ -343,4 +343,412 @@ mod tests {
rig.shutdown();
}
/// Fixture tool that mirrors the github WASM tool's `oneOf` discriminated
/// union schema. Uses `#[serde(tag = "action")]` deserialization — exactly
/// what the real tool does — so if coercion fails the test reproduces:
/// `invalid type: string "100", expected u32`
struct GitHubFixtureTool;
#[derive(Debug, Deserialize)]
#[serde(tag = "action")]
enum GitHubFixtureAction {
#[serde(rename = "list_issues")]
ListIssues {
owner: String,
repo: String,
#[serde(default)]
state: Option<String>,
#[serde(default)]
limit: Option<u32>,
},
#[serde(rename = "get_issue")]
GetIssue {
owner: String,
repo: String,
issue_number: u32,
},
#[serde(rename = "list_pull_requests")]
ListPullRequests {
owner: String,
repo: String,
#[serde(default)]
limit: Option<u32>,
#[serde(default)]
page: Option<u32>,
},
#[serde(rename = "create_pull_request")]
CreatePullRequest {
owner: String,
repo: String,
title: String,
head: String,
base: String,
#[serde(default)]
draft: Option<bool>,
},
}
use serde::Deserialize;
#[async_trait]
impl Tool for GitHubFixtureTool {
fn name(&self) -> &str {
"github_fixture"
}
fn description(&self) -> &str {
"Fixture mirroring the github WASM tool's oneOf schema"
}
fn parameters_schema(&self) -> serde_json::Value {
json!({
"type": "object",
"required": ["action"],
"oneOf": [
{
"properties": {
"action": { "const": "list_issues" },
"owner": { "type": "string" },
"repo": { "type": "string" },
"state": { "type": "string", "enum": ["open", "closed", "all"] },
"limit": { "type": "integer", "default": 30 }
},
"required": ["action", "owner", "repo"]
},
{
"properties": {
"action": { "const": "get_issue" },
"owner": { "type": "string" },
"repo": { "type": "string" },
"issue_number": { "type": "integer" }
},
"required": ["action", "owner", "repo", "issue_number"]
},
{
"properties": {
"action": { "const": "list_pull_requests" },
"owner": { "type": "string" },
"repo": { "type": "string" },
"limit": { "type": "integer", "default": 30 },
"page": { "type": "integer" }
},
"required": ["action", "owner", "repo"]
},
{
"properties": {
"action": { "const": "create_pull_request" },
"owner": { "type": "string" },
"repo": { "type": "string" },
"title": { "type": "string" },
"head": { "type": "string" },
"base": { "type": "string" },
"draft": { "type": "boolean", "default": false }
},
"required": ["action", "owner", "repo", "title", "head", "base"]
}
]
})
}
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
// Deserialize exactly like the real github WASM tool does.
// Without coercion, this fails: `invalid type: string "100", expected u32`
let action: GitHubFixtureAction = serde_json::from_value(params).map_err(|e| {
ToolError::InvalidParameters(format!("serde deserialization failed: {e}"))
})?;
let result = match action {
GitHubFixtureAction::ListIssues {
owner,
repo,
state,
limit,
} => json!({
"action": "list_issues",
"owner": owner,
"repo": repo,
"state": state.unwrap_or_else(|| "open".to_string()),
"limit": limit.unwrap_or(30),
}),
GitHubFixtureAction::GetIssue {
owner,
repo,
issue_number,
} => json!({
"action": "get_issue",
"owner": owner,
"repo": repo,
"issue_number": issue_number,
}),
GitHubFixtureAction::ListPullRequests {
owner,
repo,
limit,
page,
} => json!({
"action": "list_pull_requests",
"owner": owner,
"repo": repo,
"limit": limit.unwrap_or(30),
"page": page.unwrap_or(1),
}),
GitHubFixtureAction::CreatePullRequest {
owner,
repo,
title,
head,
base,
draft,
} => json!({
"action": "create_pull_request",
"owner": owner,
"repo": repo,
"title": title,
"head": head,
"base": base,
"draft": draft.unwrap_or(false),
}),
};
Ok(ToolOutput::success(result, Duration::from_millis(1)))
}
fn requires_sanitization(&self) -> bool {
false
}
}
/// Reproduces the exact bug: LLM sends `limit: "100"` and `issue_number: "42"`
/// as strings to a `oneOf` discriminated union schema. Without coercion support
/// for combinators, serde fails with `invalid type: string "100", expected u32`.
#[tokio::test]
async fn e2e_coerces_oneof_discriminated_union_params() {
let trace = LlmTrace {
model_name: "test-coercion-oneof".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "List issues in nearai/ironclaw with limit 100".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_list".to_string(),
name: "github_fixture".to_string(),
// LLM sends numeric params as strings — the exact bug
arguments: json!({
"action": "list_issues",
"owner": "nearai",
"repo": "ironclaw",
"state": "open",
"limit": "100"
}),
}],
input_tokens: 100,
output_tokens: 30,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Found issues in nearai/ironclaw with limit 100.".to_string(),
input_tokens: 150,
output_tokens: 20,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: Vec::new(),
expects: TraceExpects {
tools_used: vec!["github_fixture".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_extra_tools(vec![Arc::new(GitHubFixtureTool)])
.build()
.await;
rig.send_message("List issues in nearai/ironclaw with limit 100")
.await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
let tool_results = rig.tool_results();
assert!(
tool_results
.iter()
.any(|(name, preview)| name == "github_fixture"
&& preview.contains("\"limit\"")
&& preview.contains("100")),
"expected coerced list_issues result, got {tool_results:?}"
);
rig.shutdown();
}
/// Tests a second oneOf variant with different string-to-integer coercions:
/// `issue_number: "42"` must be coerced to match the `get_issue` variant.
#[tokio::test]
async fn e2e_coerces_oneof_get_issue_variant() {
let trace = LlmTrace {
model_name: "test-coercion-oneof-issue".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "Get issue 42 from nearai/ironclaw".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_issue".to_string(),
name: "github_fixture".to_string(),
arguments: json!({
"action": "get_issue",
"owner": "nearai",
"repo": "ironclaw",
"issue_number": "42"
}),
}],
input_tokens: 80,
output_tokens: 20,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Issue 42 retrieved.".to_string(),
input_tokens: 100,
output_tokens: 10,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: Vec::new(),
expects: TraceExpects {
tools_used: vec!["github_fixture".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_extra_tools(vec![Arc::new(GitHubFixtureTool)])
.build()
.await;
rig.send_message("Get issue 42 from nearai/ironclaw").await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
let tool_results = rig.tool_results();
assert!(
tool_results
.iter()
.any(|(name, preview)| name == "github_fixture"
&& preview.contains("\"issue_number\"")
&& preview.contains("42")),
"expected coerced get_issue result, got {tool_results:?}"
);
rig.shutdown();
}
/// Tests boolean coercion in a oneOf variant: `draft: "true"` must become
/// a boolean for the `create_pull_request` variant.
#[tokio::test]
async fn e2e_coerces_oneof_boolean_in_variant() {
let trace = LlmTrace {
model_name: "test-coercion-oneof-bool".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "Create a draft PR".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_pr".to_string(),
name: "github_fixture".to_string(),
arguments: json!({
"action": "create_pull_request",
"owner": "nearai",
"repo": "ironclaw",
"title": "Fix coercion",
"head": "fix/coercion",
"base": "main",
"draft": "true"
}),
}],
input_tokens: 90,
output_tokens: 25,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Draft PR created.".to_string(),
input_tokens: 110,
output_tokens: 10,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: Vec::new(),
expects: TraceExpects {
tools_used: vec!["github_fixture".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_extra_tools(vec![Arc::new(GitHubFixtureTool)])
.build()
.await;
rig.send_message("Create a draft PR").await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
let tool_results = rig.tool_results();
assert!(
tool_results
.iter()
.any(|(name, preview)| name == "github_fixture"
&& preview.contains("\"draft\"")
&& preview.contains("true")),
"expected coerced create_pull_request result with draft=true, got {tool_results:?}"
);
rig.shutdown();
}
}
+277
View File
@@ -0,0 +1,277 @@
//! E2E test: real github WASM tool with parameter coercion via TestRig.
//!
//! Loads the compiled github WASM binary into the test rig, replays an LLM
//! trace that sends string-typed numeric params, and verifies the WASM tool
//! constructs the correct HTTP API call via `http_exchanges` in the trace.
//!
//! These tests are `#[ignore]` by default because they require a pre-compiled
//! WASM binary. Build it with:
//! cargo build -p github-tool --target wasm32-wasip2 --release
//! Then run with:
//! cargo test --features libsql --test e2e_wasm_github_coercion -- --ignored
#[cfg(feature = "libsql")]
mod support;
/// Note on URL verification: the `ReplayingHttpInterceptor` logs warnings on
/// URL mismatch but still returns the canned response. The real verification is
/// that the tool succeeds end-to-end: coercion produced the correct typed
/// parameters, serde deserialization succeeded, and the WASM tool constructed a
/// valid HTTP request. A URL mismatch warning in logs does not indicate test
/// failure — it is a soft check only.
#[cfg(feature = "libsql")]
mod tests {
use std::time::Duration;
use serde_json::json;
use ironclaw::llm::recording::{HttpExchange, HttpExchangeRequest, HttpExchangeResponse};
use crate::support::test_rig::TestRigBuilder;
use crate::support::trace_llm::{
LlmTrace, TraceExpects, TraceResponse, TraceStep, TraceToolCall,
};
const GITHUB_WASM: &str = "tools-src/github/target/wasm32-wasip2/release/github_tool.wasm";
const GITHUB_CAPS: &str = "tools-src/github/github-tool.capabilities.json";
fn github_ok(body: &str) -> HttpExchangeResponse {
HttpExchangeResponse {
status: 200,
headers: vec![
("content-type".to_string(), "application/json".to_string()),
("x-ratelimit-remaining".to_string(), "100".to_string()),
],
body: body.to_string(),
}
}
/// LLM sends `limit: "50"` (string) to `list_issues`. Coercion converts it
/// to integer, and the WASM tool must call `GET /repos/.../issues?...&per_page=50`.
#[tokio::test]
#[ignore] // requires pre-compiled WASM binary
async fn wasm_github_list_issues_coerces_string_limit() {
let expected_url =
"https://api.github.com/repos/nearai/ironclaw/issues?state=open&per_page=50";
let trace = LlmTrace {
model_name: "test-wasm-coercion-list-issues".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "List issues in nearai/ironclaw with limit 50".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_1".to_string(),
name: "github".to_string(),
arguments: json!({
"action": "list_issues",
"owner": "nearai",
"repo": "ironclaw",
"state": "open",
"limit": "50"
}),
}],
input_tokens: 100,
output_tokens: 30,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Found 1 issue.".to_string(),
input_tokens: 150,
output_tokens: 10,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: vec![HttpExchange {
request: HttpExchangeRequest {
method: "GET".to_string(),
url: expected_url.to_string(),
headers: vec![],
body: None,
},
response: github_ok(r#"[{"number":1,"title":"Test issue","state":"open"}]"#),
}],
expects: TraceExpects {
tools_used: vec!["github".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_wasm_tool("github", GITHUB_WASM, Some(GITHUB_CAPS.into()))
.build()
.await;
rig.send_message("List issues in nearai/ironclaw with limit 50")
.await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
rig.shutdown();
}
/// LLM sends `issue_number: "42"` (string) to `get_issue`. Coercion converts
/// it to integer, and the URL must contain `/issues/42`.
#[tokio::test]
#[ignore] // requires pre-compiled WASM binary
async fn wasm_github_get_issue_coerces_string_issue_number() {
let expected_url = "https://api.github.com/repos/nearai/ironclaw/issues/42";
let trace = LlmTrace {
model_name: "test-wasm-coercion-get-issue".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "Get issue 42 from nearai/ironclaw".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_2".to_string(),
name: "github".to_string(),
arguments: json!({
"action": "get_issue",
"owner": "nearai",
"repo": "ironclaw",
"issue_number": "42"
}),
}],
input_tokens: 80,
output_tokens: 20,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Issue 42 retrieved.".to_string(),
input_tokens: 100,
output_tokens: 10,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: vec![HttpExchange {
request: HttpExchangeRequest {
method: "GET".to_string(),
url: expected_url.to_string(),
headers: vec![],
body: None,
},
response: github_ok(r#"{"number":42,"title":"Test","state":"open","body":"desc"}"#),
}],
expects: TraceExpects {
tools_used: vec!["github".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_wasm_tool("github", GITHUB_WASM, Some(GITHUB_CAPS.into()))
.build()
.await;
rig.send_message("Get issue 42 from nearai/ironclaw").await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
rig.shutdown();
}
/// LLM sends `limit: "25"` (string) to `list_pull_requests`. URL must
/// contain `per_page=25`.
#[tokio::test]
#[ignore] // requires pre-compiled WASM binary
async fn wasm_github_list_prs_coerces_string_limit() {
let expected_url =
"https://api.github.com/repos/nearai/ironclaw/pulls?state=open&per_page=25";
let trace = LlmTrace {
model_name: "test-wasm-coercion-list-prs".to_string(),
turns: vec![crate::support::trace_llm::TraceTurn {
user_input: "List PRs in nearai/ironclaw".to_string(),
steps: vec![
TraceStep {
request_hint: None,
response: TraceResponse::ToolCalls {
tool_calls: vec![TraceToolCall {
id: "call_gh_3".to_string(),
name: "github".to_string(),
arguments: json!({
"action": "list_pull_requests",
"owner": "nearai",
"repo": "ironclaw",
"limit": "25"
}),
}],
input_tokens: 80,
output_tokens: 20,
},
expected_tool_results: Vec::new(),
},
TraceStep {
request_hint: None,
response: TraceResponse::Text {
content: "Found PRs.".to_string(),
input_tokens: 100,
output_tokens: 10,
},
expected_tool_results: Vec::new(),
},
],
expects: TraceExpects::default(),
}],
memory_snapshot: Vec::new(),
http_exchanges: vec![HttpExchange {
request: HttpExchangeRequest {
method: "GET".to_string(),
url: expected_url.to_string(),
headers: vec![],
body: None,
},
response: github_ok(r#"[{"number":1,"title":"Test PR","state":"open"}]"#),
}],
expects: TraceExpects {
tools_used: vec!["github".to_string()],
all_tools_succeeded: Some(true),
max_tool_calls: Some(1),
min_responses: Some(1),
..Default::default()
},
steps: Vec::new(),
};
let rig = TestRigBuilder::new()
.with_trace(trace.clone())
.with_wasm_tool("github", GITHUB_WASM, Some(GITHUB_CAPS.into()))
.build()
.await;
rig.send_message("List PRs in nearai/ironclaw").await;
let responses = rig.wait_for_responses(1, Duration::from_secs(15)).await;
rig.verify_trace_expects(&trace, &responses);
rig.shutdown();
}
}
+99
View File
@@ -0,0 +1,99 @@
use ironclaw::llm::ChatMessage;
use ironclaw::llm::gemini_oauth::GeminiOauthProvider;
/// Regression: Cloud Code API routing for Gemini 2.0+ models.
/// Gemini 1.x → legacy generativelanguage.googleapis.com
/// Gemini 2.0+ → Cloud Code API (cloudcode-pa.googleapis.com)
#[test]
fn test_regression_cloud_code_api_routing() {
// Legacy models (1.x) → false
assert!(!GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-1.5-pro"
));
assert!(!GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-1.5-flash"
));
// 2.0+ models → true
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-2.0-flash"
));
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-2.5-pro"
));
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-2.5-flash"
));
// Preview models with hyphen → true
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-3.1-pro-preview"
));
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-3-flash-preview"
));
// Gemini 3 family → true
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"gemini-3-pro"
));
}
/// Regression: "preview" false-positive fix.
/// `model.contains("-preview")` (with hyphen) prevents models whose name
/// happens to include "preview" without a hyphen prefix from being
/// mis-routed to Cloud Code API.
#[test]
fn test_regression_preview_false_positive_fix() {
// "my-preview-custom" still matches (contains "-preview")
assert!(GeminiOauthProvider::model_uses_cloud_code_api(
"my-preview-custom"
));
// "mypreviewcustom" does NOT match (no hyphen before "preview")
assert!(!GeminiOauthProvider::model_uses_cloud_code_api(
"mypreviewcustom"
));
// Non-Gemini models without "-preview" → false
assert!(!GeminiOauthProvider::model_uses_cloud_code_api(
"not-a-gemini-model"
));
}
/// Regression: model list consistency.
/// Wizard, list_models(), and LLM_PROVIDERS.md all return the same 8 models.
#[test]
fn test_regression_standardized_model_list() {
let expected_models = [
"gemini-3.1-pro-preview",
"gemini-3.1-pro-preview-customtools",
"gemini-3-pro-preview",
"gemini-3-flash-preview",
"gemini-3.1-flash-lite-preview",
"gemini-2.5-pro",
"gemini-2.5-flash",
"gemini-2.5-flash-lite",
];
// All standardized models must route to Cloud Code API (all are >= 2.0)
for model in &expected_models {
assert!(
GeminiOauthProvider::model_uses_cloud_code_api(model),
"Standardized model '{}' should route to Cloud Code API",
model
);
}
}
/// Regression: ChatMessage helper constructors.
#[test]
fn test_regression_chat_message_helpers() {
let user_msg = ChatMessage::user("hello");
assert_eq!(user_msg.role, ironclaw::llm::Role::User);
assert_eq!(user_msg.content, "hello");
let system_msg = ChatMessage::system("you are helpful");
assert_eq!(system_msg.role, ironclaw::llm::Role::System);
assert_eq!(system_msg.content, "you are helpful");
}
+112 -15
View File
@@ -23,7 +23,7 @@ use crate::support::metrics::{ToolInvocation, TraceMetrics};
use crate::support::test_channel::{TestChannel, TestChannelHandle};
use crate::support::trace_llm::{LlmTrace, TraceLlm};
use ironclaw::llm::recording::{HttpExchange, ReplayingHttpInterceptor};
use ironclaw::llm::recording::{HttpExchange, HttpInterceptor, ReplayingHttpInterceptor};
// ---------------------------------------------------------------------------
// TestRig
@@ -343,6 +343,13 @@ impl Drop for TestRig {
// TestRigBuilder
// ---------------------------------------------------------------------------
/// Specification for loading a real WASM tool in the test rig.
pub struct WasmToolSpec {
pub name: String,
pub wasm_path: std::path::PathBuf,
pub capabilities_path: Option<std::path::PathBuf>,
}
/// Builder for constructing a `TestRig`.
pub struct TestRigBuilder {
trace: Option<LlmTrace>,
@@ -354,6 +361,7 @@ pub struct TestRigBuilder {
enable_routines: bool,
http_exchanges: Vec<HttpExchange>,
extra_tools: Vec<Arc<dyn Tool>>,
wasm_tools: Vec<WasmToolSpec>,
keep_bootstrap: bool,
}
@@ -370,10 +378,34 @@ impl TestRigBuilder {
enable_routines: false,
http_exchanges: Vec::new(),
extra_tools: Vec::new(),
wasm_tools: Vec::new(),
keep_bootstrap: false,
}
}
/// Load a real WASM tool binary into the test rig.
///
/// The tool will be compiled, registered, and wired with the same HTTP
/// interceptor used for `with_http_exchanges()`, so `http_exchanges` in
/// the trace can specify expected requests/responses for WASM tool HTTP calls.
///
/// If the WASM binary does not exist at build time, the tool is silently
/// skipped (logged as a warning). Tests should use `#[ignore]` or check
/// for the binary in a preamble if the tool is required.
pub fn with_wasm_tool(
mut self,
name: impl Into<String>,
wasm_path: impl Into<std::path::PathBuf>,
capabilities_path: Option<std::path::PathBuf>,
) -> Self {
self.wasm_tools.push(WasmToolSpec {
name: name.into(),
wasm_path: wasm_path.into(),
capabilities_path,
});
self
}
/// Set the LLM trace to replay.
pub fn with_trace(mut self, trace: LlmTrace) -> Self {
self.trace = Some(trace);
@@ -465,6 +497,7 @@ impl TestRigBuilder {
enable_routines,
http_exchanges: explicit_http_exchanges,
extra_tools,
wasm_tools,
keep_bootstrap,
} = self;
@@ -560,6 +593,20 @@ impl TestRigBuilder {
let scheduler_slot: ironclaw::tools::builtin::SchedulerSlot =
Arc::new(tokio::sync::RwLock::new(None));
// Build HTTP interceptor once — shared by both AgentDeps and WASM tools.
let http_interceptor: Option<Arc<dyn HttpInterceptor>> = {
let exchanges = if explicit_http_exchanges.is_empty() {
trace_http_exchanges
} else {
explicit_http_exchanges
};
if exchanges.is_empty() {
None
} else {
Some(Arc::new(ReplayingHttpInterceptor::new(exchanges)) as Arc<dyn HttpInterceptor>)
}
};
// 6. Register job tools, routine tools, and extra tools.
{
// Ensure filesystem/shell dev tools are always available in the
@@ -620,6 +667,69 @@ impl TestRigBuilder {
for tool in extra_tools {
components.tools.register(tool).await;
}
// Register WASM tools with the shared HTTP interceptor.
if !wasm_tools.is_empty() {
use ironclaw::tools::wasm::{
Capabilities, CapabilitiesFile, WasmRuntimeConfig, WasmToolRuntime,
WasmToolWrapper,
};
let runtime = Arc::new(
WasmToolRuntime::new(WasmRuntimeConfig::default())
.expect("create WASM runtime for test rig"),
);
for spec in wasm_tools {
if !spec.wasm_path.exists() {
tracing::warn!(
name = %spec.name,
path = %spec.wasm_path.display(),
"WASM tool binary not found, skipping"
);
continue;
}
let wasm_bytes = tokio::fs::read(&spec.wasm_path)
.await
.unwrap_or_else(|e| panic!("read {}: {e}", spec.wasm_path.display()));
let (capabilities, description, schema) =
if let Some(cap_path) = &spec.capabilities_path {
if cap_path.exists() {
let cap_bytes = tokio::fs::read(cap_path)
.await
.unwrap_or_else(|e| panic!("read {}: {e}", cap_path.display()));
let cap_file = CapabilitiesFile::from_bytes(&cap_bytes)
.expect("parse capabilities.json");
(
cap_file.to_capabilities(),
cap_file.description.clone(),
cap_file.parameters.clone(),
)
} else {
(Capabilities::default(), None, None)
}
} else {
(Capabilities::default(), None, None)
};
let prepared = runtime
.prepare(&spec.name, &wasm_bytes, None)
.await
.unwrap_or_else(|e| panic!("prepare WASM tool '{}': {e}", spec.name));
let mut wrapper =
WasmToolWrapper::new(Arc::clone(&runtime), prepared, capabilities);
if let Some(desc) = description {
wrapper = wrapper.with_description(desc);
}
if let Some(s) = schema {
wrapper = wrapper.with_schema(s);
}
if let Some(interceptor) = &http_interceptor {
wrapper = wrapper.with_http_interceptor(Arc::clone(interceptor));
}
components.tools.register(Arc::new(wrapper)).await;
}
}
}
// Save references for test accessors.
@@ -643,20 +753,7 @@ impl TestRigBuilder {
hooks: components.hooks,
cost_guard: components.cost_guard,
sse_tx: None,
http_interceptor: {
// Prefer explicit exchanges from with_http_exchanges(), fall back to trace.
let exchanges = if explicit_http_exchanges.is_empty() {
trace_http_exchanges
} else {
explicit_http_exchanges
};
if exchanges.is_empty() {
None
} else {
Some(Arc::new(ReplayingHttpInterceptor::new(exchanges))
as Arc<dyn ironclaw::llm::recording::HttpInterceptor>)
}
},
http_interceptor,
transcription: None,
document_extraction: None,
sandbox_readiness: ironclaw::agent::SandboxReadiness::Available, // tests don't use real Docker