mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
30790439eebd4edf8a18e20b285bd08c1dced2d7
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
30790439ee |
perf: build system prompt once per turn, skip tools on force-text (#583)
* perf: build system prompt once per turn, skip tools on force-text, fix nudge role (#565) Three fixes to agentic loop prompt handling: 1. Build system prompt once per turn instead of every tool iteration. `build_system_prompt_with_tools` is now pub; callers pass the result via `ReasoningContext::system_prompt` to avoid rebuilding ~1,500 tokens per iteration. 2. Skip `## Available Tools` section when `force_text = true`. The dispatcher passes a no-tools prompt variant on the final iteration, saving ~460 tokens and removing misleading instructions. 3. Change nudge message from `Role::System` to `Role::User`. A second system message mid-conversation is unsupported by most providers. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: revert nudge role change to keep ChatMessage::system Copilot review correctly identified that using Role::User for the nudge breaks compact_messages_for_retry, which uses rposition for Role::User to find the last real user message. Role::Assistant would cause back-to-back assistant messages. Since no production issues were reported with the original system role, revert to ChatMessage::system. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review — omit tool guidance when tools empty, rename shadowed var - Conditionalize "Call tools…" guidelines and "## Tool Call Style" section in the system prompt so they are only included when tools are non-empty. Previously the force-text (no-tools) prompt still contained misleading tool-calling instructions. (Copilot review comment) - Rename `system_prompt` → `cached_prompt` in dispatcher to avoid shadowing the earlier workspace identity `system_prompt` variable. (Copilot review) - Add regression tests: `test_system_prompt_with_tools_contains_tool_guidance` and extended assertions in `test_system_prompt_without_tools_omits_tools_section`. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: [email protected] <[email protected]> |
||
|
|
45ec691f4c |
Improve test infrastructure: StubChannel, gateway helpers, security tests, search edge cases (#623)
* feat(testing): add StubChannel test double for Channel trait Adds StubChannel to src/testing.rs alongside StubLlm. Supports message injection via mpsc sender, response/status capture, and configurable health check toggling. Includes handle methods for use after ownership transfer to ChannelManager. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(testing): wire StubChannel into TestHarnessBuilder Add with_stub_channel() builder method that creates a StubChannel pre-registered in a ChannelManager. Tests can inject messages via the sender and verify routing through the manager. The channel field on TestHarness is Optional, defaulting to None for backward compat. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: gate external-service tests behind integration feature flag Replace silent try_connect() skip pattern with explicit feature gating. cargo test now runs only self-contained tests. cargo test --features integration runs tests requiring PostgreSQL. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(channels): add ChannelManager unit tests using StubChannel Cover add/start_all stream merging, respond routing, unknown channel errors, health_check_all with mixed health, empty-channels error path, and injection channel merging -- all via StubChannel test double. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: document test tier separation (unit/integration/live) Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add architecture boundary check script Grep-based checks for three architecture boundaries: - Direct database driver usage (tokio_postgres/libsql) outside src/db/ - .unwrap()/.expect() in production code (warning only) - Direct std::env::var reads outside config layer (warning only) The DB driver check is a hard violation; the other two are warnings for gradual cleanup. Run with: bash scripts/check-boundaries.sh Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(search): add RRF edge case tests for empty inputs, limits, and config modes Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(security): add regression tests for skill installer ZIP and SSRF protections Add 11 regression tests covering the security controls in skill_tools: ZIP extraction safety: - Valid SKILL.md extraction works correctly - Non-SKILL.md entries are ignored (returns error) - Path traversal entries (../../SKILL.md) do not match - Nested path entries (subdir/SKILL.md) do not match - Oversized entries (>1MB uncompressed) are rejected SSRF prevention: - Loopback addresses (127.0.0.1) are blocked - Private ranges (10.x, 172.16.x, 192.168.x) are blocked - Link-local addresses (169.254.x) are blocked - Public IPs (8.8.8.8, 1.1.1.1) are allowed - IPv4-mapped IPv6 unwrapping logic works correctly - Metadata endpoints and .internal/.local hostnames are blocked - Normal hostnames (github.com, clawhub.dev) are allowed Also documents a known gap: url::Url::host_str() returns bracketed IPv6 addresses that std::net::IpAddr cannot parse, so IPv4-mapped IPv6 URLs currently bypass IP-based checks in validate_fetch_url. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(testing): extract TestGatewayBuilder to eliminate gateway test duplication Both ws_gateway_integration.rs and openai_compat_integration.rs manually constructed GatewayState with 19+ fields. Extracted to a shared builder in src/channels/web/test_helpers.rs that provides sensible defaults and lets tests override only what they need. Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs: add implementation plans for testing batches 1 and 2 Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): close IPv6 SSRF bypass in validate_fetch_url validate_fetch_url used host_str() which returns bracketed IPv6 (e.g. "[::ffff:7f00:1]") that IpAddr::parse() cannot handle, silently skipping IP-based SSRF checks for all IPv6 URLs. Switch to url::Host enum matching to extract proper IpAddr values without string parsing. IPv4-mapped IPv6 addresses like ::ffff:127.0.0.1 are now correctly unwrapped and blocked. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(skills): add activation criteria limits enforcement tests Adds test_activation_criteria_enforce_limits to verify that enforce_limits() correctly trims excess patterns (>5), keywords (>20), and tags (>10), and filters out short keywords/tags (<3 chars). Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(wasm): add security regression tests for WASM tool loader Add 6 tests covering: tool name path separator rejection, empty name rejection, nonexistent file handling, invalid WASM bytes rejection, dotfile discovery behavior, and subdirectory non-recursion. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: address PR review feedback - Remove plan files from repo (ilblackdragon review) - Replace CLAUDE.md test tier rules with pointer to check-boundaries.sh - Add Check 4 to check-boundaries.sh: enforces integration tests are gated behind the 'integration' feature flag Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: add try_connect silent-skip pattern check to check-boundaries.sh Check 5 catches try_connect() and similar silent-skip patterns in integration tests. Tests should use feature gates to fail loudly when prerequisites are missing, not silently return. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): harden skill fetch SSRF checks * fix(scripts): use bash arrays in check-boundaries.sh tier violation check Refactor Check 4 in check-boundaries.sh to use bash arrays and printf instead of string concatenation with echo -e. This is more robust with special characters in filenames and avoids portability concerns with echo -e. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
5c2ba44f12 |
feat(llm): declarative provider registry (#618)
* feat(llm): declarative provider registry, replace hardcoded provider configs Replace the hardcoded LlmBackend enum and per-provider config structs with a declarative JSON registry. Adding a new OpenAI-compatible provider now requires zero Rust code changes -- just add an entry to providers.json. - Add providers.json with 14 providers (openai, anthropic, ollama, openai_compatible, tinfoil, openrouter, groq, nvidia, venice, together, fireworks, deepseek, cerebras, sambanova) - Add src/llm/registry.rs with ProviderProtocol, SetupHint, ProviderDefinition, and ProviderRegistry types - Rewrite src/config/llm.rs: remove LlmBackend enum and 5 per-provider config structs, replace with generic RegistryProviderConfig - Simplify src/llm/mod.rs: remove 5 create_*_provider functions, dispatch on ProviderProtocol (3 code paths for all providers) - Dynamic setup wizard: menu built from registry.selectable(), generic credential collection dispatched by SetupHint kind - Dynamic secret injection: inject_llm_keys_from_secrets() discovers secret-to-env mappings from registry instead of hardcoded list - Users can extend with ~/.ironclaw/providers.json (no recompile) - Subsumes open provider PRs: Groq #570, NVIDIA NIM #576, Venice.ai #451 (Gemini #476 excluded -- not OpenAI-compatible) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(llm): self-sufficient provider auth, onboard --provider-only, extract SessionConfig - NearAiChatProvider handles its own session auth lazily in resolve_bearer_token() instead of requiring main.rs to pre-check. Triggers OAuth/API-key login on first request when no token exists. - Add `ironclaw onboard --provider-only` to reconfigure just the LLM provider and model selection without re-running the full wizard. - Extract auth_base_url and session_path from NearAiConfig into LlmConfig::session (SessionConfig). Callers now use config.llm.session directly instead of reaching into nearai fields. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(llm): address PR review comments on provider registry - Use registry.selectable() instead of registry.all() for secret injection to avoid duplicates from user provider overrides. - Fix selectable() dedup bug: check setup hint on the final (overridden) definition, not the first occurrence. User overrides that add a setup hint are now included correctly. - Only store openai_compatible_base_url for providers that actually use LLM_BASE_URL, preventing base URL pollution for groq/nvidia/etc. - Normalize provider_id to canonical registry def.id instead of using the raw user-supplied alias string. - Add comment explaining why .completions_api() is used over the default Responses API path. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(docker): copy providers.json into build context The declarative provider registry uses `include_str!("../../providers.json")` at compile time, so the file must be present in the Docker builder stage. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(llm): address second-round PR review comments (#618) - Make --channels-only and --provider-only mutually exclusive via clap conflicts_with (Copilot: cli/mod.rs) - Add 5s timeout to fetch_openai_compatible_models(), matching the other three model-fetch helpers (Copilot: wizard.rs) - Apply models_filter from setup hints when listing models, so Groq's "chat" filter actually excludes non-chat models (Copilot: wizard.rs) - Normalize LlmConfig.backend to the canonical provider ID instead of the raw user-supplied alias string (Copilot: llm.rs) - Add models_filter() accessor to SetupHint with regression test Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(test): relax flaky parallel speedup timing threshold The test_parallel_speedup test asserted <500ms but CI runners can be slow enough to exceed that while still proving parallelism. Bumped to 800ms which still validates parallel execution (sequential would be ~600ms minimum) while tolerating CI jitter. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(llm): handle api_key_login path in resolve_bearer_token, warn on missing keys - resolve_bearer_token() now checks NEARAI_API_KEY env var after ensure_authenticated(), handling the case where the user entered an API key via the interactive login flow (which sets the env var but not a session token) - Add tracing::warn when creating an OpenAI-compatible provider without an API key, making 401 errors easier to diagnose - Add regression test for resolve_bearer_token auth paths Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in nearai_chat test [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(llm): correct bearer token priority, handle setup-less providers (#618) - resolve_bearer_token(): session token now takes priority over NEARAI_API_KEY env var, preventing unexpected auth mode switches. The env var fallback only triggers after ensure_authenticated() when no session token was stored (api_key_login path). - run_provider_setup(): providers with setup: None no longer error, allowing env-var-only providers to be kept during re-onboarding. - Split bearer token test into 3 focused tests: config api_key path, session token path, and session-beats-env-var precedence test. - Add test for wizard handling of providers without setup hints. Co-Authored-By: Claude Opus 4.6 <[email protected]> * test(llm): comprehensive tests for provider registry, config, and auth Add 13 new tests covering the critical paths in the provider system: Bearer token auth priority (nearai_chat.rs): - config api_key wins over session token and env var - session token wins over env var (prevents mid-run auth mode switches) - config api_key path works in isolation - session token path works in isolation Config resolution (config/llm.rs): - backend alias normalization (open_ai → openai) - unknown backend falls back to openai_compatible - nearai aliases (nearai, near_ai, near) all resolve correctly - base URL resolution priority (env > settings > registry default) Registry dedup (registry.rs): - user override adds setup hint → appears in selectable() - user override removes setup hint → excluded from selectable() - selectable() preserves insertion order during dedup - all built-in ApiKey providers have api_key_env set Wizard (wizard.rs): - setup: None providers don't error during re-onboarding Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
448383cfb0 |
refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably - Filter out `type: "reasoning"` output items from NEAR AI Responses API parsing so chain-of-thought never reaches the UI (nearai.rs) - Rewrite clean_response with regex-based tag stripping that is code-aware (preserves tags inside fenced blocks and inline backticks), supports 9+ tag names (think, thought, reasoning, reflection, etc.), handles <final> extraction, pipe-delimited tags, and case/whitespace tolerance (reasoning.rs) - Add Reasoning::complete() helper so all non-agentic LLM call sites (summarize, suggest, heartbeat, compaction) get automatic response cleaning; thread SafetyLayer through to those callers - Change persist_turn from fire-and-forget tokio::spawn to awaited async so both user and assistant messages are written before returning, preventing data loss on shutdown/restart - Pass input_count through seed_response_chain so response chaining delta calculation is accurate after thread hydration on restart - Make NearAiResponse.usage optional and preserve response_id in alt response path for chaining continuity - Persist session token to DB during onboarding wizard so runtime loads it without legacy-key fallback; suppress spurious warning on fresh installs - Fix dev tool double-registration when builder already registers them - Load dotenv/ironclaw env for doctor and status subcommands - Reduce startup log noise (demote info→debug for skills, remove redundant info lines) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Nudge to not loop over tools continuesly * refactor: remove Responses API, consolidate NEAR AI to Chat Completions only The Responses API provider (nearai.rs, 1278 lines) added significant complexity (response chaining state machine, delta message calculation, previous_response_id persistence) for marginal benefit. This consolidates to the Chat Completions API only, upgrading NearAiChatProvider with dual auth (session token + API key) and 401 retry for session token renewal. - Delete src/llm/nearai.rs (Responses API provider) - Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models - Remove response_id from CompletionResponse and ToolCompletionResponse - Remove seed_response_chain/get_response_chain_id from LlmProvider trait - Remove response chain persistence from agent (thread_ops, session) - Remove NearAiApiMode enum and NEARAI_API_MODE config - Clean up all wrapper providers (retry, circuit_breaker, failover, cache) - Update documentation (CLAUDE.md, .env.example) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: runtime log level control via gateway UI and URL parameter Add server-side log level switching using tracing_subscriber::reload::Layer so the EnvFilter can be swapped at runtime without restarting. Expose via GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs toolbar, and a ?log_level=debug URL parameter for one-click activation. Also applies cargo fmt to pre-existing files (llm/, tests/). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
fa64df05ff |
feat: wire memory hygiene into the heartbeat loop (#195)
* feat: wire memory hygiene into heartbeat loop (#166) * refactor: address PR review comments for hygiene wiring * style: fix fmt import ordering and clippy too_many_arguments warning * fix: update heartbeat integration test to pass HygieneConfig argument HeartbeatRunner::new() now requires a HygieneConfig as its second argument after the hygiene wiring refactor. Pass the default config in the integration test. Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
ffb1cc9be8 |
refactor: architecture improvements for contributor velocity (#198)
* refactor: split large files and consolidate test stubs for contributor velocity - Extract 7 Database sub-traits (ConversationStore, JobStore, SandboxStore, RoutineStore, ToolFailureStore, SettingsStore, WorkspaceStore) with Database as a supertrait combining them all - Split libsql_backend.rs (2769 lines) into src/db/libsql/ directory with one file per sub-trait implementation - Split config.rs (1753 lines) into src/config/ directory with 16 domain files - Consolidate 3 duplicate test LLM stubs into shared StubLlm in src/testing.rs - Split server.rs handlers into src/channels/web/handlers/ directory - Extract main.rs init phases into AppBuilder (src/app.rs) - Add developer setup script (scripts/dev-setup.sh) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: move heartbeat test from examples/ to tests/ Convert standalone example binary into a proper #[ignore] integration test, matching the convention of the other integration tests. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt formatting for CI Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review comments from Copilot - tunnel.rs: replace .ok().flatten() with ? to propagate env var errors - secrets.rs: remove misleading "process-wide cache" comment - database.rs: use uppercase "DATABASE_URL" in error key - testing.rs: gate harness tests with #[cfg(feature = "libsql")] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Illia Polosukhin <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> |