From 5e7758598fb858dc48bc08cdb57da674a31b4339 Mon Sep 17 00:00:00 2001 From: Henry Park Date: Thu, 12 Mar 2026 21:32:19 -0700 Subject: [PATCH] chore: periodic sync main into staging (resolved conflicts) (#1098) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * chore: promote staging to main (2026-03-10 15:19 UTC) (#865) * fix: Channel HTTP: server doesn't start after config change (no hot-r… (#779) * fix: Channel HTTP: server doesn't start after config change (no hot-reload) * review fixes * review fixes * fix linter * fix code style * fix: prevent session lock contention blocking message processing (#783) * fix: prevent session lock contention blocking message processing ## Problem After container restart, POST /api/chat/send returns 202 ACCEPTED but messages don't appear in conversation_messages and agent never responds. Messages get stuck in "stale state" after restart. Root cause: Session lock was held for entire duration of chat_threads_handler and chat_history_handler, including during slow database queries. This blocked the agent loop from acquiring the session lock to process incoming messages, causing them to hang indefinitely. ## Solution 1. **Release session lock early in chat_threads_handler**: Only acquire lock when reading active_thread at response time, not during DB queries for thread list. DB operations no longer block message processing. 2. **Release session lock early in chat_history_handler**: Only acquire lock when accessing in-memory thread state, not during paginated DB queries or thread ownership checks. DB operations no longer block message processing. 3. **Add comprehensive logging**: Track message flow from receipt through session resolution, thread hydration, and state transitions. Helps diagnose future issues: - Message queued to agent loop (chat_send_handler) - Processing message from channel (handle_message) - Hydrating thread from DB (maybe_hydrate_thread) - Resolving session and thread (resolve_thread) - Checking thread state (process_user_input) - Persisting user message (persist_user_message) ## Impact - Message processing no longer blocks on session lock contention - API response times for thread list/history queries unaffected (DB queries still happen, but lock is not held) - Better diagnostics for future debugging ## Testing - All 2756 tests pass - Code compiles with zero clippy warnings - No changes to user-facing API or behavior, only lock timing Co-Authored-By: Claude Haiku 4.5 * security: redact PII from info-level logs Downgrade user_id and channel logging to debug level to prevent exposing Personally Identifiable Information (PII) in production logs. The user_id field can contain sensitive information such as phone numbers (e.g., for Signal messages). Logging PII in cleartext at the info level creates a security and privacy risk, as these logs may be stored in persistent storage, indexed by log management systems, or accessible to unauthorized personnel. Changes: - Info level: logs only message_id (UUID) for tracking - Debug level: logs user_id, channel, thread_id for troubleshooting This maintains debugging capability for developers while protecting user privacy in production logs. Co-Authored-By: Claude Haiku 4.5 --------- Co-authored-by: Claude Haiku 4.5 * chore: sync main into staging (#855) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809) Add declarative `unsupported_params` field to provider definitions in providers.json. Parameters listed are stripped from requests before sending, preventing 400 errors from providers that reject them (e.g. gpt-5 family and kimi-k2.5 rejecting custom temperature values). - Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig - Propagate from registry through config resolution - Generic strip helpers handle temperature, max_tokens, stop_sequences - Apply filtering in RigAdapter and AnthropicOAuthProvider - Mark openai and tinfoil providers as unsupporting temperature - Update openai default model to gpt-5-mini Co-authored-by: Claude Opus 4.6 --------- Co-authored-by: Claude Sonnet 4.6 Co-authored-by: Illia Polosukhin * fix: Chat input is hidden in mobile browser mode (#877) * fix: stop XML-escaping tool output content (#598) (#874) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809) Add declarative `unsupported_params` field to provider definitions in providers.json. Parameters listed are stripped from requests before sending, preventing 400 errors from providers that reject them (e.g. gpt-5 family and kimi-k2.5 rejecting custom temperature values). - Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig - Propagate from registry through config resolution - Generic strip helpers handle temperature, max_tokens, stop_sequences - Apply filtering in RigAdapter and AnthropicOAuthProvider - Mark openai and tinfoil providers as unsupporting temperature - Update openai default model to gpt-5-mini Co-authored-by: Claude Opus 4.6 * fix: stop XML-escaping tool output content in wrap_for_llm (#598) Remove content escaping that corrupted JSON in tool output. The structural boundary is preserved but content now passes through raw, fixing downstream parse failures. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 * fix(safety): allow empty string tool params (#848) * fix(safety): allow empty string tool params * fix(safety): preserve heuristic checks and add path context to tool validation This follow-up refactor addresses PR review feedback by restoring heuristic checks (whitespace ratio, character repetition) for tool parameter validation and improving error reporting. Changes: - Restored heuristic warnings in validate_non_empty_input so they apply to both user input and tool parameters (when non-empty). - Refactored check_strings to recursively build and pass JSON paths (e.g., "metadata.tags[1]"). - Updated validation errors to use the specific JSON path as the field name instead of the generic "input". - Added regression tests for whitespace/repetition warnings and JSON path reporting in tool parameters. This ensures the safety layer remains semantically neutral about empty strings (fixing the memory_tree path: "" issue) while maintaining rigorous protection and providing better developer ergonomics. * style: run cargo fmt * perf: optimize release and dist build profiles (#843) * perf: optimize release and dist build profiles Add [profile.release] with strip=true and panic="abort" for smaller, faster release binaries. Upgrade [profile.dist] from lto="thin" to lto="fat" with codegen-units=1 for maximum optimization in CI releases. Co-Authored-By: Claude Opus 4.6 * fix: remove panic=abort from release profile Reviewers (zmanian, Copilot, Gemini) correctly flagged that panic=abort in the release profile would kill the entire process on any tokio task panic, breaking fault isolation for the long-running server. Removed from release profile entirely. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat: add PR template with risk assessment (#837) * feat: add PR template with risk assessment and review tracks Add a pull request template that includes summary, change type, validation checklist, security/database impact sections, blast radius, and rollback plan. Update CONTRIBUTING.md with review track definitions (A/B/C) based on change risk level. Co-Authored-By: Claude Opus 4.6 * fix: expand CONTRIBUTING.md with setup, workflow, and guidelines Add getting started, development workflow, code style summary, database change guidance, and dependency management sections. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat: add fuzzing targets for untrusted input parsers (#835) * feat: add fuzzing targets for untrusted input parsers Add cargo-fuzz infrastructure with 5 fuzz targets exercising security-critical code paths: - fuzz_safety_sanitizer: Aho-Corasick + regex injection detection - fuzz_safety_validator: Input validation (length, encoding, patterns) - fuzz_leak_detector: Secret leak scanning (API keys, tokens) - fuzz_tool_params: Tool parameter JSON validation - fuzz_config_env: TOML/JSON config parsing Each target exercises real IronClaw business logic with invariant assertions. Includes corpus directories and setup documentation. Co-Authored-By: Claude Opus 4.6 * fix: improve fuzz targets to exercise real IronClaw code paths - fuzz_config_env: exercise SafetyLayer end-to-end (sanitize, validate, policy check) instead of generic TOML/JSON parsing - fuzz_tool_params: add validate_tool_schema coverage alongside validate_tool_params - Add "fuzz" to workspace exclude in root Cargo.toml - Update README descriptions to match actual target behavior [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: replace redundant detect() call with meaningful invariant assertion Replace the double sanitize()+detect() call with an assertion that critical severity warnings always trigger content modification. Co-Authored-By: Claude Opus 4.6 * fix: rewrite fuzz_config_env to exercise IronClaw safety code directly Replace SafetyLayer wrapper usage with direct Sanitizer, Validator, and LeakDetector instantiation and invocation. Adds meaningful consistency assertions (non-empty output, valid-means-no-errors, scan/clean agreement). Removes the config construction that was only exercising struct instantiation. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * fix(wasm): run leak scan before credential injection in tools wrapper (#791) * fix(wasm): run leak scan before credential injection in tools wrapper The tools WASM wrapper runs the LeakDetector on HTTP request headers AFTER inject_host_credentials() has already substituted real secrets (e.g., xoxb- Slack bot tokens). This causes the leak detector to flag the tool's own legitimate outbound API calls as secret exfiltration. Move the scan to run on raw_headers before any credential injection, matching the fix already applied to the channels wrapper in #421. Fixes the same class of bug as #421 (which only fixed channels/wasm/wrapper.rs). Co-Authored-By: Claude Opus 4.6 * perf: inline leak scan to avoid Vec allocation on every HTTP request Address review feedback: instead of cloning all header keys/values into a Vec to pass to scan_http_request(), iterate over raw_headers directly using scan_and_clean(). This also provides more specific error messages (URL vs header vs body). Co-Authored-By: Claude Opus 4.6 * style: fix cargo fmt formatting in leak scan loop Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * fix(setup): drain residual terminal events before secret input (#747) (#849) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809) Add declarative `unsupported_params` field to provider definitions in providers.json. Parameters listed are stripped from requests before sending, preventing 400 errors from providers that reject them (e.g. gpt-5 family and kimi-k2.5 rejecting custom temperature values). - Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig - Propagate from registry through config resolution - Generic strip helpers handle temperature, max_tokens, stop_sequences - Apply filtering in RigAdapter and AnthropicOAuthProvider - Mark openai and tinfoil providers as unsupporting temperature - Update openai default model to gpt-5-mini Co-authored-by: Claude Opus 4.6 * fix: skip the regression check [skip-regression-check] --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 Co-authored-by: Illia Polosukhin * feat(agent): add context size logging before LLM prompt (#810) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * feat(agent): add context size logging before LLM prompt --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 Co-authored-by: Illia Polosukhin * fix: preserve text before tool-call XML in forced-text responses (#852) * fix: preserve text before tool-call XML in forced-text responses (#789) Local models (Qwen3, DeepSeek, GLM) emit XML even when no tools are available (force_text mode). The existing strip_xml_tag() discards everything from an unclosed opening tag onward, producing an empty string that triggers the "I'm not sure how to respond" fallback. Add truncate_at_tool_tags() — a code-region-aware pre-processing step that truncates at the first tool-call XML tag BEFORE clean_response() runs, preserving all useful text before the tag. Protect all 7 clean_response() call sites. Case-insensitive matching handles models that emit or variants. Secondary fix: add has_native_thinking() model detection to skip / system prompt injection for models with built-in reasoning (Qwen3, QwQ, DeepSeek-R1, GLM-Z1, etc.), preventing thinking-only responses that clean to empty. Wire with_model_name(active_model_name()) at all 9 production sites that construct Reasoning, so the runtime model name (not static config) drives system prompt generation. 126 new/updated tests covering truncation edge cases, code-block awareness, Unicode, case-insensitivity, StubLlm integration for complete/plan/evaluate_success/respond_with_tools paths, model detection, and conditional system prompt generation. Closes #789 Co-Authored-By: Claude Opus 4.6 * fix: address Copilot review — unclosed-only truncation, ASCII case folding - truncate_at_tool_tags() now only truncates at UNCLOSED tool tags; properly closed tags (e.g. ...) are left intact for clean_response() to strip normally, preserving any text after them - Switch from to_lowercase() to to_ascii_lowercase() to prevent byte offset misalignment with non-ASCII characters whose lowercase form has different byte length (e.g. Kelvin sign U+212A) - Add closing_tag_for() helper to derive closing tags from open patterns - Fix doc comment: "fenced markdown code blocks or inline code spans" (not "indented", which find_code_regions() doesn't detect) - Add regression tests: closed vs unclosed for each tag variant, Unicode + case-insensitive offset safety, and mixed closed/unclosed Co-Authored-By: Claude Opus 4.6 * fix: minor review items — consistent ascii_lowercase, closing_tag_for tests - Switch has_native_thinking() from to_lowercase() to to_ascii_lowercase() for consistency with truncate_at_tool_tags() approach - Add unit tests for closing_tag_for(): standard tags, space-suffixed patterns, pipe-delimited tags, and exhaustive coverage of all TOOL_TAG_PATTERNS entries - Add test for mixed closed+unclosed tags of different types Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * Feat/docker shell edition (#804) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 * fix(mcp): strip top-level null params before forwarding to MCP servers (#795) * feat(llm): per-provider unsupported parameter filtering (#749, #728) (#809) Add declarative `unsupported_params` field to provider definitions in providers.json. Parameters listed are stripped from requests before sending, preventing 400 errors from providers that reject them (e.g. gpt-5 family and kimi-k2.5 rejecting custom temperature values). - Add `unsupported_params` to ProviderDefinition and RegistryProviderConfig - Propagate from registry through config resolution - Generic strip helpers handle temperature, max_tokens, stop_sequences - Apply filtering in RigAdapter and AnthropicOAuthProvider - Mark openai and tinfoil providers as unsupporting temperature - Update openai default model to gpt-5-mini Co-authored-by: Claude Opus 4.6 * fix(mcp): strip top-level null params before forwarding to MCP servers LLMs frequently emit `"field": null` for optional parameters in tool calls. Many MCP servers reject explicit nulls for fields that should simply be absent — e.g. Notion returns 400 for `"sort": null` in a search call, expecting the field to be omitted entirely. Strip top-level null keys from the params object before calling `call_tool()`. Only top-level keys are stripped; nested nulls are preserved since they may be semantically meaningful. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Illia Polosukhin Co-authored-by: Claude Opus 4.6 * Add event-triggered routines and workflow skill templates (#756) * Add event-triggered routines and workflow skill templates * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix: address PR review feedback for event_emit security and quality Security fixes: - Require approval (UnlessAutoApproved) for event_emit, matching routine_fire - Enable sanitization on event_emit payload (external JSON reaches LLM) - Remove user_id parameter from event_emit to prevent IDOR — always use ctx.user_id Correctness fixes: - Rename source → event_source in event_emit for consistency with routine_create - Use json_value_as_filter_string for filter parsing (handles numbers/booleans) - Case-insensitive matching for event source and event_type - Add debug logging for missing filter keys in payload - Fix skill_install_routine_webhook_sim test missing .with_skills() - Fix schema_validator test for event_emit payload properties Code quality: - Move EventEmitTool struct/impl after RoutineHistoryTool (fix split layout) - Deduplicate routine_to_info into RoutineInfo::from_routine in types.rs - Add test section headers in e2e_routine_heartbeat.rs - Clarify event_emit description to specify system_event routines only Co-Authored-By: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * fix: make routine_system_event_emit test create routine before emitting - Add routine_create step to trace fixture so event_emit has a matching routine to fire - Assert fired_routines > 0, not just key presence (Copilot review) - Add .with_auto_approve_tools(true) since event_emit now requires approval Co-Authored-By: Claude Opus 4.6 * fix: renumber test headers after system_event test insertion Test 4 was duplicated (routine_cooldown and heartbeat_findings). Renumber heartbeat_findings to Test 5 and heartbeat_empty_skip to Test 6. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: merge staging and add missing RoutineEngine args in test RoutineEngine::new on staging requires `tools` and `safety` params. Update system_event_trigger_matches_and_filters test to pass them. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address new Copilot review comments - Add .with_auto_approve_tools(true) to skill_install_routine_webhook_sim test so event_emit doesn't block on approval - Fix module-level doc comment for event_emit to specify system_event trigger [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: deduplicate json_value_as_string helper Remove private `json_value_as_string` from routine_engine.rs and use the identical public `json_value_as_filter_string` from routine.rs, eliminating divergence risk. (Copilot review) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 * fix: enable WASM credential injection in No-DB environments (#845) * fix(wasm): enable credential injection in no-DB environments via env var fallback When a secrets store is unavailable (e.g. no-DB mode), WASM channel credentials were silently not injected, causing channels to start without credentials. Fix by: - Changing `inject_channel_credentials_from_secrets` to accept `Option<&dyn SecretsStore>` — secrets store is tried first when present - Adding env var fallback (`inject_env_credentials`) for credentials not covered by the secrets store - Enforcing a channel-name prefix security check on env var names to prevent WASM channels from reading unrelated host credentials (e.g. `AWS_SECRET_ACCESS_KEY`) - Extracting pure `resolve_env_credentials` helper for testability - Adding case-insensitive prefix matching for secrets store lookup Co-Authored-By: Claude Sonnet 4.6 * fix(wasm): inject credentials at startup when no secrets store (setup.rs path) The startup path (setup_wasm_channels -> register_channel) was guarded by `if let Some(secrets) = secrets_store`, so in No-DB mode credentials were never injected and the channel started without them. Fix by: - Changing inject_channel_credentials to accept Option<&dyn SecretsStore> - Always calling it (removing the if-let guard) — env var fallback runs even when secrets_store is None - Adding channel-name prefix security check to the env var fallback path (e.g. TELEGRAM_ for channel "telegram"), consistent with manager.rs Co-Authored-By: Claude Sonnet 4.6 * fix(test): correct misleading comment on ICTEST1_UNRELATED_OTHER placeholder * fix(wasm): guard against empty channel name in credential injection An empty channel_name would produce prefix "_", allowing any env var starting with "_" to pass the security check and be injected. Add an early-return guard in resolve_env_credentials, inject_env_credentials, and inject_channel_credentials. Add a test to cover this path. Co-Authored-By: Claude Sonnet 4.6 --------- Co-authored-by: lizican123 Co-authored-by: Claude Sonnet 4.6 * fix: promote to main (#878) * fix: replace unsafe env::set_var with thread-safe inject_single_var in SIGHUP handler Fixes race condition where SIGHUP handler modifies global environment variables while other threads may be reading them via Config::from_env(). Changes: - Replace unsafe { std::env::set_var() } with ironclaw::config::inject_single_var() - Uses INJECTED_VARS mutex instead of unsafe global state modification - All reads via optional_env() check the thread-safe overlay first - Prevents data races between SIGHUP reload and concurrent config reads Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * fix: spawn webhook restart as background task to avoid blocking I/O across lock Prevents holding Mutex lock during async I/O operations (TcpListener::bind, task shutdown). The SIGHUP handler no longer blocks webhook processing during listener restart. Changes: - Read old_addr and drop lock immediately - Spawn restart_with_addr() as background task via tokio::spawn - Lock is only held during the actual restart operation, not the signal handler Benefits: - SIGHUP handler returns immediately without blocking - Webhook requests not delayed by listener restart I/O - Lock contention significantly reduced Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * fix: add graceful shutdown mechanism for SIGHUP handler background task Prevents unbounded loop without cancellation token. The SIGHUP handler now listens for a shutdown signal and exits cleanly during graceful termination. Changes: - Create broadcast channel for shutdown signaling - SIGHUP handler uses tokio::select! to wait for shutdown or SIGHUP - Send shutdown signal to all background tasks after agent.run() completes - Ensures clean task lifecycle and no orphaned background tasks Benefits: - Proper task cancellation during graceful shutdown - Follows Tokio best practices for background task management - No background tasks orphaned when runtime shuts down Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * refactor: replace stringly-typed parameter filtering with typed enum and single helper Fixes DRY violation where unsupported parameter filtering was duplicated across rig_adapter.rs and anthropic_oauth.rs using string contains checks. Changes: - Add UnsupportedParam typed enum in provider.rs (Temperature, MaxTokens, StopSequences) - Create strip_unsupported_completion_params() helper function - Create strip_unsupported_tool_params() helper function - Update rig_adapter.rs to use shared helpers - Update anthropic_oauth.rs to use shared helpers - Replace 60+ lines of duplicate stringly-typed logic Benefits: - Type safety: parameter names checked at compile time - Single source of truth: adding a new param updates one place - Reduced maintenance burden: no duplicate logic to keep in sync - Better code clarity: named enum variant is self-documenting Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * docs: clarify intentional parameter asymmetry between completion and tool requests Add documentation explaining why strip_unsupported_tool_params does not handle StopSequences: the field doesn't exist in ToolCompletionRequest. Changes: - Add clarifying comments to strip_unsupported_tool_params() - Explain why StopSequences is only in CompletionRequest - Note that ToolCompletionRequest only supports Temperature and MaxTokens - Inline comment confirms no action needed for StopSequences This addresses the appearance of incomplete implementation without changing logic, as the asymmetry is intentional and correct (ToolCompletionRequest lacks the field). Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * perf: isolate webhook_secret to reduce lock contention on hot path Move webhook_secret from shared HttpChannelState RwLock into its own Arc>. This eliminates contention between secret validation and other state operations. Changes: - Change webhook_secret field type from RwLock> to Arc>> - Update initialization in HttpChannel::new() - Update comments to explain isolation rationale Benefits: - Reduce lock contention on webhook request hot path (secret validation) - Rarely-changing field (SIGHUP only) isolated from frequent state accesses - Other state operations (tx, pending_responses) no longer wait behind secret reads - Minimal code change: only field declaration and initialization The Arc wrapper allows cloning the RwLock handle to separate concerns. With this change, every webhook request acquires its own isolated lock for secret validation, not the shared HttpChannelState lock. This scales better under high request volume. Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * fix: prevent partial state corruption on SIGHUP restart failure Ensure atomicity of configuration reload: if webhook listener restart fails, secret update is skipped to prevent inconsistent state. Changes: - Wait for restart_with_addr() to complete (don't spawn background task) - Track restart result with restart_failed flag - Only update secret if restart succeeded or wasn't needed - Ensure listener and secret stay synchronized Problem addressed: - Before: restart spawned as background task, secret updated immediately - If restart failed, secret was changed but listener still on old address - This left system in inconsistent state (partial corruption) Solution: - Make restart blocking (SIGHUP handler can wait, it's not on request hot path) - Atomically update secret only after successful restart - Flag prevents race between restart and secret update Benefits: - Configuration changes are atomic (both succeed or both fail together) - No partial state corruption on restart failure - Failed restarts don't silently leave inconsistent state - Secret and listener address stay in sync Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * refactor: generalize hot-secret-swapping with ChannelSecretUpdater trait Decouple SIGHUP handler from HTTP channel internals by introducing a trait for channels that support zero-downtime secret updates. Changes: - Add ChannelSecretUpdater trait in channels/channel.rs - Implement ChannelSecretUpdater for HttpChannelState - Export trait from channels module - Update SIGHUP handler to use trait-based secret updater collection - Replace explicit HTTP channel knowledge with generic updater loop Benefits: - SIGHUP handler no longer depends on HttpChannelState details - Tight coupling removed: main.rs doesn't need HTTP channel imports - Extensible: new channels can opt-in by implementing the trait - Scalable: multiple channels supported without main.rs changes - Maintainable: adding channels requires only trait implementation, not SIGHUP handler edits Pattern: - ChannelSecretUpdater trait defines the interface for all updaters - Channels that support hot-secret-swapping implement the trait - SIGHUP handler loops through all registered updaters generically Verification: - All 2,787 tests pass - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 * feat: validate parameter names at deserialization time, not just tests Add custom serde deserializer for unsupported_params that validates parameter names at runtime when loading providers.json (or user overrides). Changes: - Add unsupported_params_de module with custom deserializer - Only allows: "temperature", "max_tokens", "stop_sequences" - Invalid parameter names cause immediate deserialization error - Update ProviderDefinition to use custom deserializer - Enhanced test with explicit parameter name validation - Add new test that verifies invalid parameters are rejected Problem solved: - Before: Invalid param names (e.g., "temperrature") silently ignored - Now: Rejected at deserialization time with clear error message - Prevents runtime failures caused by typos in configuration Example error: unsupported parameter name 'temperrature': must be one of: temperature, max_tokens, stop_sequences Benefits: - Fail-fast: errors caught when loading config, not at runtime - Clear feedback: error message lists valid parameter names - Type safety: validators run during deserialization - Configuration errors detected immediately, not silently ignored Verification: - All 2,788 tests pass (including new validation test) - Zero clippy warnings - Code compiles successfully Co-Authored-By: Claude Haiku 4.5 --------- Co-authored-by: Claude Haiku 4.5 * merge: resolve conflicts for PR #800 and #822 into staging (#881) * fix(ci): secrets can't be used in step if conditions [skip-regression-check] (#787) GitHub Actions step-level `if:` doesn't have access to `secrets` context. Replace `if: secrets.X != ''` with `continue-on-error: true` and let the Set token step handle the fallback. Co-authored-by: Claude Sonnet 4.6 * fix(ci): clean up staging pipeline — remove hacks, skip redundant checks [skip-regression-check] (#794) - Remove continue-on-error from staging-ci.yml app token steps (secrets are configured) - Skip test.yml and code_style.yml on PRs targeting staging (staging-ci.yml already runs tests before promoting, promotion PR gets full CI on main) - Allow ironclaw-ci[bot] in Claude Code review for bot-created promotion PRs Co-authored-by: Claude Opus 4.6 * fix(ci): run fmt + clippy on staging PRs, skip Windows clippy [skip-regression-check] (#802) - Remove branches:[main] filter from code_style.yml so it runs on all PRs - Gate clippy-windows with `if: github.base_ref == 'main'` (skip on staging PRs) - Update rollup job to allow skipped clippy-windows - Simplify claude-review.yml to only trigger on labeled event (avoids duplicate runs) Co-authored-by: Claude Opus 4.6 * feat: persist user_id in save_job and expose job_id on routine runs (#709) * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 * feat: wire RoutineEngine into gateway for direct manual trigger firing Replace the message-channel hack in routines_trigger_handler with a direct call to RoutineEngine::fire_manual(), ensuring FullJob routines dispatch correctly when triggered from the web UI. Inject the engine into GatewayState from Agent::run after construction. Also persists user_id in save_job for both PG and libSQL backends, removes the source='sandbox' filter so all jobs are visible, and exposes job_id on RoutineRunInfo for the frontend job link. Co-Authored-By: Claude Opus 4.6 * fix: remove stale gateway_state argument from Agent::new test call sites The gateway_state parameter was removed from Agent::new during rebase (replaced by post-construction set_routine_engine_slot), but three test call sites still passed the extra None argument. Co-Authored-By: Claude Opus 4.6 * fix: address PR review — restore sandbox source filter, remove blank lines - Revert removal of `source = 'sandbox'` filter in all SandboxStore queries (8 sites across PG and libSQL). Sandbox-specific APIs should stay scoped to sandbox jobs; unified job listing for the Jobs tab should use a separate query path. - Remove extra blank lines in agent_loop.rs and worker.rs that caused formatting CI failure. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * fix: address review — regenerate Cargo.lock, add user_id regression test - Regenerate Cargo.lock from main's lockfile to eliminate dependency version downgrades (anyhow, syn, etc.) that were churn from rebase. - Add regression test verifying user_id round-trips through save_job and get_job in the libSQL backend. Co-Authored-By: Claude Opus 4.6 * style: remove trailing blank line in libsql jobs.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 * test: add Postgres-side regression test for user_id persistence in save_job Mirrors the existing libSQL test (test_save_job_persists_user_id) for the Postgres backend. Gated behind #[cfg(feature = "postgres")] + #[ignore] since it requires a running PostgreSQL instance (integration tier). Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * refactor: unify three agentic loops into single AgenticLoop engine (#654) Replace three independent copy-pasted agentic loops (dispatcher, worker, container runtime) with a single shared engine in `agentic_loop.rs` that all consumers customize via the `LoopDelegate` trait. Phase 1 — Shared engine (`src/agent/agentic_loop.rs`, 205 lines): - `run_agentic_loop()` owns the core LLM → tool exec → repeat cycle - `LoopDelegate` trait (Send + Sync, &dyn dispatch) with 6 hook points - Tool intent nudge logic consolidated (was duplicated in 3 files) - Iteration limit + force-text behavior preserved Phase 2 — Three delegate implementations: - `ChatDelegate` (dispatcher.rs): 3-phase approval flow, hooks, cost guard, context compaction, skill attenuation, interruption - `JobDelegate` (worker/job.rs): planning pre-loop phase, parallel JoinSet exec, mark_completed/stuck/failed, SSE streaming, self-repair - `ContainerDelegate` (worker/container.rs): sequential tool exec, HTTP-proxied LLM, container-safe tools, credential injection Phase 3 — File moves and cleanup: - Delete `src/agent/worker.rs` — job logic moved to `src/worker/job.rs` - Rename `src/worker/runtime.rs` → `src/worker/container.rs` - Re-export `Worker`/`WorkerDeps` from `crate::worker` in `agent/mod.rs` - Update `scheduler.rs` imports to new worker location Shared helpers (`src/tools/execute.rs`): - `execute_tool_with_safety()` replaces 4 copies of validate → timeout → execute → serialize - `process_tool_result()` replaces 3 copies of sanitize → wrap → ChatMessage (also used by thread_ops.rs approval resume paths) Net result: -2,408 lines, zero duplicated loop logic, single code path for tool intent nudge and completion detection. Closes #654 Co-Authored-By: Claude Opus 4.6 * fix: address review feedback from Copilot 1. scheduler.rs: Replace `unwrap_or` fallback with proper error propagation when parsing tool output JSON — surfaces bugs instead of silently changing the output type. 2. worker/job.rs: Drop MutexGuard before the cancellation `.await` in `check_signals()` to avoid holding a lock across an async I/O call (prevents `await_holding_lock` lint). 3. worker/job.rs: Restore consecutive rate-limit counter (MAX_CONSECUTIVE_RATE_LIMITS = 10) so sustained rate limiting marks the job stuck with "Persistent rate limiting" instead of silently burning through max_iterations. Co-Authored-By: Claude Opus 4.6 * fix: incorporate staging changes — token budget tracking + mark_failed Merge staging's changes into the refactored JobDelegate: - Add token budget tracking in call_llm (update_context/add_tokens) - mark_stuck → mark_failed for iteration cap and rate-limit exhaustion (aligns with staging's #788 fix) Co-Authored-By: Claude Opus 4.6 * fix: address zmanian's PR review — eliminate type erasure, clean up Address all 6 review points from zmanian on PR #800: 1. Replace LoopOutcome::Custom(Box) with typed LoopOutcome::NeedApproval(Box) — eliminates type erasure and downcast, resolves clippy large_enum_variant. 2. Remove dead max_tool_iterations field from ChatDelegate struct. 3. Add on_tool_intent_nudge() hook to LoopDelegate trait with implementations in Job and Container delegates for observability. 4. Fix SSE events in job worker to emit raw sanitized content instead of XML-wrapped tags. 5. Remove 4 duplicate completion tests from job.rs that were already covered by the shared util module. 6. Avoid logging full tool results — use result_size_bytes in debug logs (execute.rs, job.rs). Also updates path references in CLAUDE.md, COVERAGE_PLAN.md, and add-sse-event.md command. Co-Authored-By: Claude Opus 4.6 * feat(doctor): expand diagnostics from 7 to 16 health checks * test: add unit tests for agentic_loop and execute shared modules Add 16 tests covering the two new critical shared modules: agentic_loop.rs (10 tests): - Text response exits loop immediately - Tool call → text response continuation - LoopSignal::Stop exits before LLM call - LoopSignal::InjectMessage adds user message to context - Max iterations terminates with LoopOutcome::MaxIterations - Tool intent nudge fires twice then caps - before_llm_call early exit bypasses LLM - truncate_for_preview: short string, long string, multibyte safety execute.rs (6 tests): - execute_tool_with_safety success path - Missing tool returns ToolError::NotFound - Tool execution failure propagates - Per-tool timeout enforcement (50ms) - process_tool_result XML wrapping on success - process_tool_result error formatting All 2,777 unit tests pass, 0 clippy warnings. Co-Authored-By: Claude Opus 4.6 * style: cargo fmt Co-Authored-By: Claude Opus 4.6 * fix: address code review — 9 issues across agentic loop, job worker, container CRITICAL fixes: - Rate-limit exhaustion now returns Err(LlmError::RateLimited) instead of Ok(Text("")), stopping the loop immediately with no ghost iteration. Below-threshold retries still use Text("") with an explicit empty-string guard in handle_text_response to skip injection. - check_signals drains the entire message channel before returning, prioritizing Stop over UserMessage. Previously returned early on first UserMessage, silently dropping any queued Stop or additional messages. - check_signals now detects all non-progressing job states (Cancelled, Failed, Stuck, Completed, Submitted, Accepted) instead of only Cancelled and Failed. HIGH fixes: - Error path in process_tool_result_job applies truncate_for_preview to bound error strings in SSE/DB events (was unbounded). - Document Send+Sync lifetime constraint on LoopDelegate trait. - Test mock before_llm_call refactored from double-lock to single lock acquisition, eliminating deadlock risk on refactor. MEDIUM fixes: - CompletionReport includes actual iteration count via shared Arc> tracker (was hardcoded 0). - process_tool_result_job return type changed from Result to Result<()> — the bool was always false (dead API). - Deduplicate truncate in container.rs; now uses truncate_for_preview from agentic_loop. Verified: 0 clippy warnings, 2781 tests pass, cargo fmt clean. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Henry Park Co-authored-by: Claude Sonnet 4.6 Co-authored-by: Illia Polosukhin Co-authored-by: Umesh Kumar Singh Co-authored-by: reidliu41 * Revert "Feat/docker shell edition" + fix fmt/clippy (#886) * Revert "Feat/docker shell edition (#804)" This reverts commit c566faf28fb77c2fa4df92c2947fb48f1a25df9b. * style: fix formatting issues from revert Run cargo fmt to fix formatting across 7 files after the revert of the docker shell edition feature. Co-Authored-By: Claude Opus 4.6 --------- Co-authored-by: Claude Opus 4.6 * refactor: centralize test credential constants into testing::credentials (#829) * refactor: central… * feat(i18n): Add internationalization support with Chinese and English translations (#929) (#950) * feat(i18n): Add internationalization support with Chinese and English translations * fix(i18n): fix duplicate keys, broken placeholders, and dead overrides --------- Co-authored-by: jinxin <106428113+italic-jinxin@users.noreply.github.com> Co-authored-by: zwb1982 <133180666+zwb1982@users.noreply.github.com> * chore: release v0.18.0 (#885) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * chore: update WASM artifact SHA256 checksums [skip ci] (#954) Co-authored-by: github-actions[bot] * feat(embeddings): add EMBEDDING_BASE_URL for OpenAI-compatible embedding providers (#1082) * feat(embeddings): add EMBEDDING_BASE_URL for OpenAI-compatible embedding providers Allow configuring a custom base URL for OpenAI-compatible embedding endpoints (e.g., Azure OpenAI, local proxies, vLLM) via the EMBEDDING_BASE_URL environment variable. When unset, defaults to https://api.openai.com. Changes: - Extract hardcoded OpenAI URL to OPENAI_API_BASE_URL constant - Add base_url field to OpenAiEmbeddings with builder method with_base_url() - Auto-prepend https:// for schemeless URLs, strip trailing slashes - Add openai_base_url field to EmbeddingsConfig, parsed from EMBEDDING_BASE_URL - Wire base URL through create_provider() with debug logging - Add EMBEDDING_BASE_URL to clear_embedding_env() in tests - Add unit tests for URL validation and env var parsing * refactor: address Gemini review — in-place trailing slash strip, simplify config logic - Use while/pop() instead of trim_end_matches().to_string() for zero extra allocation when stripping trailing slashes in with_base_url() - Remove double openai_base_url check in create_provider() — create provider first, then branch on base_url for logging + configuration --------- Co-authored-by: SMKRV --------- Co-authored-by: ironclaw-ci[bot] <266877842+ironclaw-ci[bot]@users.noreply.github.com> Co-authored-by: Nick Pismenkov <50764773+nickpismenkov@users.noreply.github.com> Co-authored-by: Claude Haiku 4.5 Co-authored-by: Illia Polosukhin Co-authored-by: Xing Ji <41811005+micsama@users.noreply.github.com> Co-authored-by: Nick Stebbings <47646783+nick-stebbings@users.noreply.github.com> Co-authored-by: Reid <61492567+reidliu41@users.noreply.github.com> Co-authored-by: Umesh Kumar Singh Co-authored-by: 智方云cubecloud-io Co-authored-by: lizican <44971766+xiaocan66@users.noreply.github.com> Co-authored-by: lizican123 Co-authored-by: Zaki Manian Co-authored-by: reidliu41 Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: jinxin <106428113+italic-jinxin@users.noreply.github.com> Co-authored-by: zwb1982 <133180666+zwb1982@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] Co-authored-by: smkrv <17809065+smkrv@users.noreply.github.com> Co-authored-by: SMKRV --- registry/channels/discord.json | 2 +- registry/channels/slack.json | 2 +- registry/channels/telegram.json | 2 +- registry/channels/whatsapp.json | 2 +- registry/tools/github.json | 4 +- registry/tools/gmail.json | 2 +- registry/tools/google-calendar.json | 2 +- registry/tools/google-docs.json | 2 +- registry/tools/google-drive.json | 2 +- registry/tools/google-sheets.json | 2 +- registry/tools/google-slides.json | 2 +- registry/tools/slack.json | 4 +- registry/tools/telegram.json | 4 +- registry/tools/web-search.json | 2 +- src/config/embeddings.rs | 70 ++++++++++++++++++++++++++--- src/workspace/embeddings.rs | 68 +++++++++++++++++++++++++++- 16 files changed, 147 insertions(+), 25 deletions(-) diff --git a/registry/channels/discord.json b/registry/channels/discord.json index cf057245..6f5cd4e7 100644 --- a/registry/channels/discord.json +++ b/registry/channels/discord.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/discord-0.2.0-wasm32-wasip2.tar.gz", "sha256": "efa1b9019fa33e243f8db1e1fcc732731d45836336bdd26ca19b6fe227ca8b69" } }, diff --git a/registry/channels/slack.json b/registry/channels/slack.json index 64b28e3b..e6d36604 100644 --- a/registry/channels/slack.json +++ b/registry/channels/slack.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-0.2.1-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/slack-0.2.1-wasm32-wasip2.tar.gz", "sha256": "d4667e35126986509d862bc3a0088777305d8f41c75de83c1e223b42312ede48" } }, diff --git a/registry/channels/telegram.json b/registry/channels/telegram.json index 9a4d8918..36be1fc7 100644 --- a/registry/channels/telegram.json +++ b/registry/channels/telegram.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-0.2.3-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/telegram-0.2.3-wasm32-wasip2.tar.gz", "sha256": "b9a83d5a2d1285ce0ec116b354336a1f245f893291ccb01dffbcaccf89d72aed" } }, diff --git a/registry/channels/whatsapp.json b/registry/channels/whatsapp.json index d1017276..be3faf0d 100644 --- a/registry/channels/whatsapp.json +++ b/registry/channels/whatsapp.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/whatsapp-0.2.0-wasm32-wasip2.tar.gz", "sha256": "feb9194719d9bed796b070ab4dc30348dbfb5d3dec56f9f21e02d14137abab01" } }, diff --git a/registry/tools/github.json b/registry/tools/github.json index e2dd1168..e84f756d 100644 --- a/registry/tools/github.json +++ b/registry/tools/github.json @@ -2,7 +2,7 @@ "name": "github", "display_name": "GitHub", "kind": "tool", - "version": "0.2.1", + "version": "0.2.0", "wit_version": "0.3.0", "description": "GitHub integration for issues, PRs, repos, and code search", "keywords": [ @@ -19,7 +19,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/github-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/github-0.2.0-wasm32-wasip2.tar.gz", "sha256": "da9fac56b6f20197a415489bbaec9fefb085a5cf6324cab79ea48a47eb19c13b" } }, diff --git a/registry/tools/gmail.json b/registry/tools/gmail.json index dc9e6c40..08913ce6 100644 --- a/registry/tools/gmail.json +++ b/registry/tools/gmail.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/gmail-0.2.0-wasm32-wasip2.tar.gz", "sha256": "ee9574e02e92bc1d481f1310eb88afd99ee52bf6971074ab33bd76bf99b34b1d" } }, diff --git a/registry/tools/google-calendar.json b/registry/tools/google-calendar.json index 0b773f69..c43112d3 100644 --- a/registry/tools/google-calendar.json +++ b/registry/tools/google-calendar.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-calendar-0.2.0-wasm32-wasip2.tar.gz", "sha256": "2fa47150ea222e787c122182ad6f4dfa2ffaf5fe490d05e8de887a76445f8d2d" } }, diff --git a/registry/tools/google-docs.json b/registry/tools/google-docs.json index 66ddd407..9f1ab133 100644 --- a/registry/tools/google-docs.json +++ b/registry/tools/google-docs.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-docs-0.2.0-wasm32-wasip2.tar.gz", "sha256": "40e134a1c1564f832ca861c3396895d4e33ec67b99313fc1f97baf8d971423a9" } }, diff --git a/registry/tools/google-drive.json b/registry/tools/google-drive.json index 6ee52089..9766e555 100644 --- a/registry/tools/google-drive.json +++ b/registry/tools/google-drive.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-drive-0.2.0-wasm32-wasip2.tar.gz", "sha256": "002a341a1d58125563a7c69561b26fbc2629b04ea723cade744102bdc0fbb71f" } }, diff --git a/registry/tools/google-sheets.json b/registry/tools/google-sheets.json index 1cf5c808..b63265e1 100644 --- a/registry/tools/google-sheets.json +++ b/registry/tools/google-sheets.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-sheets-0.2.0-wasm32-wasip2.tar.gz", "sha256": "8aa2c9d52f033edea3a6c2311b0ec694ccb6d0a54ef07e94d72bf8be1ce8009a" } }, diff --git a/registry/tools/google-slides.json b/registry/tools/google-slides.json index 9c5684b8..54187531 100644 --- a/registry/tools/google-slides.json +++ b/registry/tools/google-slides.json @@ -17,7 +17,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/google-slides-0.2.0-wasm32-wasip2.tar.gz", "sha256": "e931a97d4fd0b0b938e464dc7c7f2be6ea6b4d1508f5ea3cd931d44db23f05f5" } }, diff --git a/registry/tools/slack.json b/registry/tools/slack.json index 194f1ffe..11bd7fff 100644 --- a/registry/tools/slack.json +++ b/registry/tools/slack.json @@ -17,8 +17,8 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-0.2.0-wasm32-wasip2.tar.gz", - "sha256": "8af3f884240de8413d272845fad2164a347d7d2a502a0d148aa38425b93f62ed" + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/slack-0.2.1-wasm32-wasip2.tar.gz", + "sha256": "d4667e35126986509d862bc3a0088777305d8f41c75de83c1e223b42312ede48" } }, "auth_summary": { diff --git a/registry/tools/telegram.json b/registry/tools/telegram.json index 0213126b..680d6fdb 100644 --- a/registry/tools/telegram.json +++ b/registry/tools/telegram.json @@ -18,8 +18,8 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-0.2.0-wasm32-wasip2.tar.gz", - "sha256": "2c66245913854be4294021fc6bb479e43f7d65830c5cec25cf6c60a71d1af468" + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/telegram-0.2.2-wasm32-wasip2.tar.gz", + "sha256": "b9a83d5a2d1285ce0ec116b354336a1f245f893291ccb01dffbcaccf89d72aed" } }, "auth_summary": { diff --git a/registry/tools/web-search.json b/registry/tools/web-search.json index 36cc6f6b..4da5744b 100644 --- a/registry/tools/web-search.json +++ b/registry/tools/web-search.json @@ -18,7 +18,7 @@ }, "artifacts": { "wasm32-wasip2": { - "url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-0.2.0-wasm32-wasip2.tar.gz", + "url": "https://github.com/nearai/ironclaw/releases/download/v0.18.0/web-search-0.2.0-wasm32-wasip2.tar.gz", "sha256": "56834573c54ea2a33cea1eb0f04bbdf59f1ef8d8702995cf431b0921302eeccc" } }, diff --git a/src/config/embeddings.rs b/src/config/embeddings.rs index c5a84c00..a1c3ecd7 100644 --- a/src/config/embeddings.rs +++ b/src/config/embeddings.rs @@ -23,6 +23,9 @@ pub struct EmbeddingsConfig { pub ollama_base_url: String, /// Embedding vector dimension. Inferred from the model name when not set explicitly. pub dimension: usize, + /// Custom base URL for OpenAI-compatible embedding providers. + /// When set, overrides the default `https://api.openai.com`. + pub openai_base_url: Option, } impl Default for EmbeddingsConfig { @@ -36,6 +39,7 @@ impl Default for EmbeddingsConfig { model, ollama_base_url: "http://localhost:11434".to_string(), dimension, + openai_base_url: None, } } } @@ -74,6 +78,8 @@ impl EmbeddingsConfig { let enabled = parse_bool_env("EMBEDDING_ENABLED", settings.embeddings.enabled)?; + let openai_base_url = optional_env("EMBEDDING_BASE_URL")?; + Ok(Self { enabled, provider, @@ -81,6 +87,7 @@ impl EmbeddingsConfig { model, ollama_base_url, dimension, + openai_base_url, }) } @@ -130,16 +137,27 @@ impl EmbeddingsConfig { } _ => { if let Some(api_key) = self.openai_api_key() { - tracing::debug!( - "Embeddings enabled via OpenAI (model: {}, dim: {})", - self.model, - self.dimension, - ); - Some(Arc::new(crate::workspace::OpenAiEmbeddings::with_model( + let mut provider = crate::workspace::OpenAiEmbeddings::with_model( api_key, &self.model, self.dimension, - ))) + ); + if let Some(ref base_url) = self.openai_base_url { + tracing::debug!( + "Embeddings enabled via OpenAI (model: {}, base_url: {}, dim: {})", + self.model, + base_url, + self.dimension, + ); + provider = provider.with_base_url(base_url); + } else { + tracing::debug!( + "Embeddings enabled via OpenAI (model: {}, dim: {})", + self.model, + self.dimension, + ); + } + Some(Arc::new(provider)) } else { tracing::warn!("Embeddings configured but OPENAI_API_KEY not set"); None @@ -164,6 +182,7 @@ mod tests { std::env::remove_var("EMBEDDING_PROVIDER"); std::env::remove_var("EMBEDDING_MODEL"); std::env::remove_var("OPENAI_API_KEY"); + std::env::remove_var("EMBEDDING_BASE_URL"); } } @@ -247,4 +266,41 @@ mod tests { std::env::remove_var("EMBEDDING_ENABLED"); } } + + #[test] + fn embedding_base_url_parsed_from_env() { + let _guard = ENV_MUTEX.lock().expect("env mutex poisoned"); + clear_embedding_env(); + + // SAFETY: Under ENV_MUTEX, no concurrent env access. + unsafe { + std::env::set_var("EMBEDDING_BASE_URL", "https://custom.example.com"); + } + + let settings = Settings::default(); + let config = EmbeddingsConfig::resolve(&settings).expect("resolve should succeed"); + assert_eq!( + config.openai_base_url.as_deref(), + Some("https://custom.example.com"), + "EMBEDDING_BASE_URL env var should be parsed into openai_base_url" + ); + + // SAFETY: Under ENV_MUTEX. + unsafe { + std::env::remove_var("EMBEDDING_BASE_URL"); + } + } + + #[test] + fn embedding_base_url_defaults_to_none() { + let _guard = ENV_MUTEX.lock().expect("env mutex poisoned"); + clear_embedding_env(); + + let settings = Settings::default(); + let config = EmbeddingsConfig::resolve(&settings).expect("resolve should succeed"); + assert!( + config.openai_base_url.is_none(), + "openai_base_url should be None when EMBEDDING_BASE_URL is not set" + ); + } } diff --git a/src/workspace/embeddings.rs b/src/workspace/embeddings.rs index 42340fcb..e40337eb 100644 --- a/src/workspace/embeddings.rs +++ b/src/workspace/embeddings.rs @@ -60,12 +60,18 @@ pub trait EmbeddingProvider: Send + Sync { } } +/// Default base URL for the OpenAI API. +const OPENAI_API_BASE_URL: &str = "https://api.openai.com"; + /// OpenAI embedding provider using text-embedding-ada-002 or text-embedding-3-small. +/// +/// Supports any OpenAI-compatible embedding endpoint via [`with_base_url`](Self::with_base_url). pub struct OpenAiEmbeddings { client: reqwest::Client, api_key: String, model: String, dimension: usize, + base_url: String, } impl OpenAiEmbeddings { @@ -78,6 +84,7 @@ impl OpenAiEmbeddings { api_key: api_key.into(), model: "text-embedding-3-small".to_string(), dimension: 1536, + base_url: OPENAI_API_BASE_URL.to_string(), } } @@ -88,6 +95,7 @@ impl OpenAiEmbeddings { api_key: api_key.into(), model: "text-embedding-ada-002".to_string(), dimension: 1536, + base_url: OPENAI_API_BASE_URL.to_string(), } } @@ -98,6 +106,7 @@ impl OpenAiEmbeddings { api_key: api_key.into(), model: "text-embedding-3-large".to_string(), dimension: 3072, + base_url: OPENAI_API_BASE_URL.to_string(), } } @@ -112,8 +121,35 @@ impl OpenAiEmbeddings { api_key: api_key.into(), model: model.into(), dimension, + base_url: OPENAI_API_BASE_URL.to_string(), } } + + /// Set a custom base URL for OpenAI-compatible embedding providers. + /// + /// The URL must use `http://` or `https://` scheme. If no scheme is present, + /// `https://` is prepended automatically. Trailing slashes are stripped. + pub fn with_base_url(mut self, base_url: &str) -> Self { + let url = base_url.trim(); + + // Auto-prepend https:// if no scheme is present. + let mut url = if !url.starts_with("http://") && !url.starts_with("https://") { + tracing::debug!( + "No scheme in embedding base URL '{}', prepending https://", + url + ); + format!("https://{url}") + } else { + url.to_string() + }; + + while url.ends_with('/') { + url.pop(); + } + + self.base_url = url; + self + } } #[derive(Debug, Serialize)] @@ -173,9 +209,11 @@ impl EmbeddingProvider for OpenAiEmbeddings { input: texts, }; + let url = format!("{}/v1/embeddings", self.base_url); + let response = self .client - .post("https://api.openai.com/v1/embeddings") + .post(&url) .header("Authorization", format!("Bearer {}", self.api_key)) .json(&request) .send() @@ -575,9 +613,37 @@ mod tests { let provider = OpenAiEmbeddings::new("test-key"); assert_eq!(provider.dimension(), 1536); assert_eq!(provider.model_name(), "text-embedding-3-small"); + assert_eq!(provider.base_url, OPENAI_API_BASE_URL); let provider = OpenAiEmbeddings::large("test-key"); assert_eq!(provider.dimension(), 3072); assert_eq!(provider.model_name(), "text-embedding-3-large"); + assert_eq!(provider.base_url, OPENAI_API_BASE_URL); + } + + #[test] + fn test_openai_with_base_url_valid() { + let provider = + OpenAiEmbeddings::new("test-key").with_base_url("https://custom.example.com"); + assert_eq!(provider.base_url, "https://custom.example.com"); + } + + #[test] + fn test_openai_with_base_url_strips_trailing_slashes() { + let provider = + OpenAiEmbeddings::new("test-key").with_base_url("https://custom.example.com///"); + assert_eq!(provider.base_url, "https://custom.example.com"); + } + + #[test] + fn test_openai_with_base_url_http_scheme() { + let provider = OpenAiEmbeddings::new("test-key").with_base_url("http://localhost:8080"); + assert_eq!(provider.base_url, "http://localhost:8080"); + } + + #[test] + fn test_openai_with_base_url_schemeless_prepends_https() { + let provider = OpenAiEmbeddings::new("test-key").with_base_url("custom.example.com/v1"); + assert_eq!(provider.base_url, "https://custom.example.com/v1"); } }