mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
424a0366a9
* feat(llm): add Anthropic prompt caching and cache token tracking - Inject cache_control via additional_params for Claude models in rig_adapter - Add cache_read_input_tokens and cache_creation_input_tokens to CompletionResponse and ToolCompletionResponse - Extract cached_input_tokens from rig-core unified Usage - Add is_anthropic_model() detection helper with provider prefix support - Log prompt cache hits at debug level (consistent with response_cache) - Add 7 unit tests for cache injection and model detection - Update all mock providers and test fixtures with new fields * feat(cost): apply 90% cache discount to prompt-cached tokens in CostGuard - Add cache_read_input_tokens to TokenUsage so cache counts flow from CompletionResponse through the reasoning layer to the dispatcher - Update CostGuard::record_llm_call() to accept cache_read_input_tokens: cached tokens are billed at 10% of the normal input rate - Thread cache_read_input_tokens from dispatcher into CostGuard - Add test_cache_discount_reduces_cost verifying exact savings match 90% of input cost for fully-cached requests - Update all existing test callers with zero-cache parameter * refactor(cache): scope cache_control to Anthropic backend and validate model support - Replace model-name-based is_anthropic_model() with explicit enable_prompt_cache flag on RigAdapter, set only for the direct Anthropic backend via with_prompt_cache(true) - Add supports_prompt_cache() to validate model names per Anthropic docs: only Claude 3+ models support caching; claude-2 and claude-instant are excluded to prevent 400 errors - Warn when caching is enabled but model does not support it - Replace is_anthropic_model tests with flag-based and model validation tests * fix(cache): validate model at construction and propagate cache metrics through proxy - Move supports_prompt_cache() check into with_prompt_cache() so unsupported models are detected once at construction, not per request - Add cache_read_input_tokens and cache_creation_input_tokens to ProxyCompletionResponse and ProxyToolCompletionResponse with serde(default) for backward compatibility - Pass cache metrics through orchestrator proxy instead of zeroing - Use claude-opus-4-6 in cache discount test to match Anthropic semantics * feat(llm): add configurable cache retention with write surcharge - Add CacheRetention enum (none/short/long) to AnthropicDirectConfig - Parse ANTHROPIC_CACHE_RETENTION env var (default: short) - Inject TTL-aware cache_control (short=5m ephemeral, long=1h) - Extract cache_creation_input_tokens from raw Anthropic response - Add cache_write_multiplier() to LlmProvider trait (1.25x short, 2.0x long) - Pipe dynamic write multiplier through dispatcher to CostGuard - Add TokenUsage.cache_creation_input_tokens field - Add tests for Long TTL injection, 5m and 1h write surcharges - Document ANTHROPIC_CACHE_RETENTION in .env.example * docs: fix stale cache_retention field comment * fix: resolve CI failures after upstream merge - Add missing cost_per_token arg to cache test callsites - Apply cargo fmt to long lines in tests and tracing macros * fix: address Copilot review feedback - Use saturating_add for cache token sum to prevent u32 overflow - Tighten supports_prompt_cache to explicitly match claude-3+/claude-4+ and named families (claude-sonnet/claude-opus/claude-haiku) * fix: adapt prompt caching to registry architecture and add missing cache fields - Resolve merge conflicts: adapt CacheRetention and cache injection to the declarative provider registry (RegistryProviderConfig replaces AnthropicDirectConfig) - Parse ANTHROPIC_CACHE_RETENTION env var in create_anthropic_from_registry() - Use Anthropic automatic caching via top-level cache_control in additional_params (rig-core #[serde(flatten)] places it at request root) - Add cache_read/creation_input_tokens fields to all mock LlmProviders added on main after PR #291 branched (response_cache, dispatcher, provider_chaos, trace_llm) - Suppress clippy::too_many_arguments on record_llm_call and build_rig_request - Add regression tests for cache injection (short/long/none) and cache_write_multiplier values Co-Authored-By: Canvinus <[email protected]> * fix: delegate cache_write_multiplier through provider wrappers and make cache_read_discount configurable The 6 decorator providers (Retry, CircuitBreaker, Failover, SmartRouting, CachedProvider, RecordingLlm) did not delegate cache_write_multiplier() to their inner provider, causing it to always return 1.0 instead of the actual 1.25x/2.0x from RigAdapter. This fix adds delegation for both cache_write_multiplier() and the new cache_read_discount() method. Also makes the cache read discount per-provider instead of hardcoding Anthropic's 90% discount (÷10). OpenAI uses 50% (÷2), so the discount is now returned by each provider via the LlmProvider trait. Addresses review feedback on PR #660. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: cargo fmt Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add CacheRetention FromStr/Display unit tests Tests cover primary values, aliases (off/disabled/5m/ephemeral/1h), case-insensitivity, invalid input error, and Display round-trip. Addresses Copilot review feedback on PR #660. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Andrey <[email protected]> Co-authored-by: Andrey Gruzdev <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]>
139 lines
5.2 KiB
Bash
139 lines
5.2 KiB
Bash
# Database Configuration
|
||
DATABASE_URL=postgres://localhost/ironclaw
|
||
DATABASE_POOL_SIZE=10
|
||
|
||
# LLM Provider
|
||
# LLM_BACKEND=nearai # default
|
||
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
|
||
|
||
# === NEAR AI (Chat Completions API) ===
|
||
# Two auth modes:
|
||
# 1. Session token (default): Uses browser OAuth (GitHub/Google) on first run.
|
||
# Session token stored in ~/.ironclaw/session.json automatically.
|
||
# Base URL defaults to https://private.near.ai
|
||
# 2. API key: Set NEARAI_API_KEY to use API key auth from cloud.near.ai.
|
||
# Base URL defaults to https://cloud-api.near.ai
|
||
NEARAI_MODEL=zai-org/GLM-5-FP8
|
||
NEARAI_BASE_URL=https://private.near.ai
|
||
NEARAI_AUTH_URL=https://private.near.ai
|
||
# NEARAI_SESSION_TOKEN=sess_... # hosting providers: set this
|
||
# NEARAI_SESSION_PATH=~/.ironclaw/session.json # optional, default shown
|
||
# NEARAI_API_KEY=... # API key from cloud.near.ai
|
||
|
||
# Local LLM Providers (Ollama, LM Studio, vLLM, LiteLLM)
|
||
|
||
# === Ollama ===
|
||
# OLLAMA_MODEL=llama3.2
|
||
# LLM_BACKEND=ollama
|
||
# OLLAMA_BASE_URL=http://localhost:11434 # default
|
||
|
||
# === OpenAI-compatible (LM Studio, vLLM, Anything-LLM) ===
|
||
# LLM_MODEL=llama-3.2-3b-instruct-q4_K_M
|
||
# LLM_BACKEND=openai_compatible
|
||
# LLM_BASE_URL=http://localhost:1234/v1
|
||
# LLM_API_KEY=sk-... # optional for local servers
|
||
# Custom HTTP headers for OpenAI-compatible providers
|
||
# Format: comma-separated key:value pairs
|
||
# LLM_EXTRA_HEADERS=HTTP-Referer:https://github.com/nearai/ironclaw,X-Title:ironclaw
|
||
|
||
# === OpenRouter (300+ models via OpenAI-compatible) ===
|
||
# LLM_MODEL=anthropic/claude-sonnet-4 # see openrouter.ai/models for IDs
|
||
# LLM_BACKEND=openai_compatible
|
||
# LLM_BASE_URL=https://openrouter.ai/api/v1
|
||
# LLM_API_KEY=sk-or-...
|
||
|
||
# LLM_EXTRA_HEADERS=HTTP-Referer:https://myapp.com,X-Title:MyApp
|
||
|
||
|
||
# === Together AI (via OpenAI-compatible) ===
|
||
# LLM_MODEL=meta-llama/Llama-3.3-70B-Instruct-Turbo
|
||
# LLM_BACKEND=openai_compatible
|
||
# LLM_BASE_URL=https://api.together.xyz/v1
|
||
# LLM_API_KEY=...
|
||
|
||
# === Fireworks AI (via OpenAI-compatible) ===
|
||
# LLM_MODEL=accounts/fireworks/models/llama4-maverick-instruct-basic
|
||
# LLM_BACKEND=openai_compatible
|
||
# LLM_BASE_URL=https://api.fireworks.ai/inference/v1
|
||
# LLM_API_KEY=fw_...
|
||
|
||
# === Anthropic Direct ===
|
||
# LLM_BACKEND=anthropic
|
||
# ANTHROPIC_MODEL=claude-sonnet-4-6
|
||
# ANTHROPIC_API_KEY=sk-ant-...
|
||
# ANTHROPIC_BASE_URL=https://api.anthropic.com # default
|
||
# Prompt cache retention — controls Anthropic server-side prompt caching:
|
||
# none = disabled (no cache_control injected)
|
||
# short = 5-minute TTL, 1.25× (125%) write surcharge (default)
|
||
# long = 1-hour TTL, 2.0× (200%) write surcharge
|
||
# ANTHROPIC_CACHE_RETENTION=short
|
||
|
||
# For full provider setup guide see docs/LLM_PROVIDERS.md
|
||
|
||
# Channel Configuration
|
||
# CLI is always enabled
|
||
|
||
# Slack Bot (optional)
|
||
SLACK_BOT_TOKEN=xoxb-...
|
||
SLACK_APP_TOKEN=xapp-...
|
||
SLACK_SIGNING_SECRET=...
|
||
|
||
# Telegram Bot (optional)
|
||
TELEGRAM_BOT_TOKEN=...
|
||
|
||
# HTTP Webhook Server (optional)
|
||
HTTP_HOST=0.0.0.0
|
||
HTTP_PORT=8080
|
||
HTTP_WEBHOOK_SECRET=your-webhook-secret
|
||
|
||
# Signal Channel (optional, requires signal-cli daemon --http)
|
||
# SIGNAL_HTTP_URL=http://127.0.0.1:8080
|
||
# SIGNAL_ACCOUNT=+1234567890
|
||
# SIGNAL_ALLOW_FROM=+1234567890,uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # comma-separated, * for all, empty = deny/require pairing
|
||
# SIGNAL_ALLOW_FROM_GROUPS= # comma-separated group IDs, * for all, empty = deny all groups
|
||
# SIGNAL_DM_POLICY=pairing # open | allowlist | pairing
|
||
# SIGNAL_GROUP_POLICY=allowlist # allowlist | open | disabled
|
||
# SIGNAL_GROUP_ALLOW_FROM= # comma-separated, empty = inherit from ALLOW_FROM
|
||
# SIGNAL_IGNORE_ATTACHMENTS=false
|
||
# SIGNAL_IGNORE_STORIES=true
|
||
|
||
# Agent Settings
|
||
AGENT_NAME=ironclaw
|
||
AGENT_MAX_PARALLEL_JOBS=5
|
||
AGENT_JOB_TIMEOUT_SECS=3600
|
||
AGENT_STUCK_THRESHOLD_SECS=300
|
||
# Enable planning phase before tool execution (default: true)
|
||
AGENT_USE_PLANNING=true
|
||
|
||
# Self-repair settings
|
||
SELF_REPAIR_CHECK_INTERVAL_SECS=60
|
||
SELF_REPAIR_MAX_ATTEMPTS=3
|
||
|
||
# Heartbeat settings (proactive periodic execution)
|
||
# When enabled, reads HEARTBEAT.md checklist and reports findings
|
||
HEARTBEAT_ENABLED=false
|
||
HEARTBEAT_INTERVAL_SECS=1800
|
||
HEARTBEAT_NOTIFY_CHANNEL=cli
|
||
HEARTBEAT_NOTIFY_USER=default
|
||
|
||
# Memory hygiene settings (automatic cleanup of stale workspace documents)
|
||
# Runs on each heartbeat tick; identity files (IDENTITY.md, SOUL.md) are never deleted
|
||
# MEMORY_HYGIENE_ENABLED=true
|
||
# MEMORY_HYGIENE_DAILY_RETENTION_DAYS=30 # delete daily/ docs older than this many days
|
||
# MEMORY_HYGIENE_CONVERSATION_RETENTION_DAYS=7 # delete conversations/ docs older than this many days
|
||
# MEMORY_HYGIENE_CADENCE_HOURS=12 # minimum hours between cleanup passes
|
||
|
||
# Safety settings
|
||
SAFETY_MAX_OUTPUT_LENGTH=100000
|
||
SAFETY_INJECTION_CHECK_ENABLED=true
|
||
|
||
# Restart Feature (Docker containers only)
|
||
# Set IRONCLAW_IN_DOCKER=true in the container entrypoint to enable the restart feature.
|
||
# Without this, the restart tool and /restart command will be disabled.
|
||
# IRONCLAW_IN_DOCKER=false
|
||
# IRONCLAW_RESTART_DELAY=5 # default wait before exit (seconds, range: 1-30)
|
||
# IRONCLAW_MAX_FAILURES=10 # max consecutive failures before container exits
|
||
|
||
# Logging
|
||
RUST_LOG=ironclaw=debug,tower_http=debug
|