mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-09-03 10:09:30 +00:00
feat: 10 infrastructure improvements from zeroclaw (#126)
* refactor: break up agent_loop.rs into four focused modules Split the monolithic 2835-line agent_loop.rs into: - agent_loop.rs (722L): Agent struct, event loop, message dispatch - dispatcher.rs (635L): Agentic tool loop, tool execution, auth detection - commands.rs (484L): System commands, job handlers, heartbeat, summarize - thread_ops.rs (1059L): Thread lifecycle, approval, undo/redo, persistence Each module gets its own impl Agent block. Agent fields changed to pub(super) so sibling modules in the agent package can access them. All 16 existing tests pass in their new locations. Inspired by ZeroClaw's agent module split (agent.rs, loop_.rs, dispatcher.rs, prompt.rs, memory_loader.rs). Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add cost caps and guardrails for autonomous agent spending Daily budget (MAX_COST_PER_DAY_CENTS) and hourly action rate (MAX_ACTIONS_PER_HOUR) limits prevent runaway agents from burning through API credits, especially in daemon/heartbeat modes. - CostGuard with pre-flight check and post-call recording - Sliding window for hourly rate, midnight-UTC daily reset - 80% threshold warning, atomic fast-path for exceeded budget - Wired into dispatcher loop (check before LLM call, record after) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add circuit breaker on LLM providers Wraps LlmProvider with a Closed/Open/HalfOpen state machine that trips after consecutive transient failures, preventing request storms against a degraded backend. Automatically probes for recovery. - CircuitBreakerProvider implements LlmProvider (drop-in wrapper) - Transient error classification (server, rate-limit, network, auth infra) - Client errors (wrong model, context overflow) don't trip the breaker - Configurable via CIRCUIT_BREAKER_THRESHOLD and CIRCUIT_BREAKER_RECOVERY_SECS - Composes with existing FailoverProvider (circuit breaker wraps failover) - 12 tests covering full state machine and error classification Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add tunnel abstraction for remote access Trait-based tunnel system with lifecycle management (start/stop/health) for exposing the agent to the internet through external tunnel binaries. Five providers: - Cloudflare Tunnel (cloudflared, Zero Trust token auth) - Tailscale (serve for tailnet, funnel for public) - ngrok (with optional custom domain) - Custom (arbitrary command with {host}/{port} placeholders) - None (local-only, no external exposure) Config via TUNNEL_PROVIDER + provider-specific env vars. Extends existing TunnelConfig with optional managed provider alongside the static TUNNEL_URL path. Factory, shared process management, and 37 tests covering all providers and edge cases. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add OS service management (launchd/systemd) Adds `ironclaw service {install,start,stop,status,uninstall}` for running the agent as a background daemon. macOS uses launchd plists under ~/Library/LaunchAgents, Linux uses systemd user units. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add observability trait system with noop, log, and multi backends Introduces an Observer trait for recording agent lifecycle events and metrics, with pluggable backends. The noop backend compiles to zero overhead, log backend uses tracing, and multi fans out to multiple observers. Configured via OBSERVABILITY_BACKEND env var. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add in-memory LLM response cache with TTL and LRU eviction CachedProvider wraps any LlmProvider and caches complete() responses keyed by SHA-256(model + messages). Tool-calling requests are never cached since they trigger side effects. Configurable via RESPONSE_CACHE_ENABLED, RESPONSE_CACHE_TTL_SECS, and RESPONSE_CACHE_MAX_ENTRIES env vars. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add memory hygiene with cadence-gated daily log cleanup Adds workspace::hygiene module that automatically deletes daily log documents older than a configurable retention period (default 30 days). Runs on a 12-hour cadence tracked via a local state file to avoid redundant passes. Best-effort design: failures are logged, never fatal. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add doctor diagnostics command for active health probing Probes external dependencies (Docker, cloudflared, ngrok, tailscale), validates NEAR AI session, checks database connectivity, and verifies workspace directory. Complements the passive `status` command. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add structured TOML config file support Adds ~/.ironclaw/config.toml as a configuration layer between env vars and database settings. Priority: env var > TOML file > DB > defaults. - `ironclaw config init` generates a commented config.toml from current settings - `ironclaw --config path/to/config.toml` loads a custom config file - Settings.merge_from() only overlays non-default values from the TOML file - `ironclaw config path` now shows TOML file status Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address codex review findings - apply_toml_overlay now returns Result and errors on explicit missing or invalid config paths (was log-only, violating the documented contract that explicit paths are fatal) - custom tunnel url_pattern is now used to filter extracted URLs, not just as a gate for scanning stdout - systemd ExecStart path is now quoted to handle spaces in paths Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback - Cache key now includes max_tokens, temperature, and stop_sequences so different request parameters produce distinct keys - to_cents() uses .trunc() + parse::<u64> instead of f64 intermediary, avoiding precision loss for large values - Tailscale public URL no longer includes local port (serve/funnel expose on standard HTTPS port 443) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: wire up tunnel lifecycle and fix audit findings Connect the tunnel module to the rest of the application so that setting TUNNEL_PROVIDER actually starts a managed tunnel at boot and stops it on shutdown. Previously create_tunnel() was never called outside tests. Changes: - Expand TunnelSettings with provider credential fields (settings.rs) - TunnelConfig::resolve() falls back to DB settings when env vars unset - Start tunnel at boot, stop on shutdown, show URL in boot screen - Setup wizard collects provider-specific credentials (ngrok, cloudflare, tailscale, custom, static URL) - Fix public_url() returning None under lock contention (SharedUrl) - Fix local_host parameter ignored by cloudflare/ngrok/tailscale - Fix tailscale silent fallback to "localhost" on bad JSON - Fix ngrok globally mutating config via add-authtoken (use env var) - Add 10s timeout to tailscale status --json Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review comments - Document split_whitespace limitation in CustomTunnel doc comment - Remove unnecessary quotes from systemd ExecStart directive Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback (round 3) - doctor: missing libSQL DB on fresh install is Pass, not Fail - service: quote ExecStart path for systemd space handling Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: correct cost guard doc comment (LLM calls, not LLM/tool) Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
436dda0f2f
commit
a158eee1b0
+179
-6
@@ -39,22 +39,32 @@ pub struct Config {
|
||||
pub routines: RoutineConfig,
|
||||
pub sandbox: SandboxModeConfig,
|
||||
pub claude_code: ClaudeCodeConfig,
|
||||
pub observability: crate::observability::ObservabilityConfig,
|
||||
}
|
||||
|
||||
impl Config {
|
||||
/// Load configuration from environment variables and the database.
|
||||
///
|
||||
/// Priority: env var > DB settings > default.
|
||||
/// Priority: env var > TOML config file > DB settings > default.
|
||||
/// This is the primary way to load config after DB is connected.
|
||||
pub async fn from_db(
|
||||
store: &dyn crate::db::Database,
|
||||
user_id: &str,
|
||||
) -> Result<Self, ConfigError> {
|
||||
Self::from_db_with_toml(store, user_id, None).await
|
||||
}
|
||||
|
||||
/// Load from DB with an optional TOML config file overlay.
|
||||
pub async fn from_db_with_toml(
|
||||
store: &dyn crate::db::Database,
|
||||
user_id: &str,
|
||||
toml_path: Option<&std::path::Path>,
|
||||
) -> Result<Self, ConfigError> {
|
||||
let _ = dotenvy::dotenv();
|
||||
crate::bootstrap::load_ironclaw_env();
|
||||
|
||||
// Load all settings from DB into a Settings struct
|
||||
let db_settings = match store.get_all_settings(user_id).await {
|
||||
let mut db_settings = match store.get_all_settings(user_id).await {
|
||||
Ok(map) => Settings::from_db_map(&map),
|
||||
Err(e) => {
|
||||
tracing::warn!("Failed to load settings from DB, using defaults: {}", e);
|
||||
@@ -62,6 +72,9 @@ impl Config {
|
||||
}
|
||||
};
|
||||
|
||||
// Overlay TOML config file (values win over DB settings)
|
||||
Self::apply_toml_overlay(&mut db_settings, toml_path)?;
|
||||
|
||||
Self::build(&db_settings).await
|
||||
}
|
||||
|
||||
@@ -74,12 +87,63 @@ impl Config {
|
||||
/// Loads both `./.env` (standard, higher priority) and `~/.ironclaw/.env`
|
||||
/// (lower priority) via dotenvy, which never overwrites existing vars.
|
||||
pub async fn from_env() -> Result<Self, ConfigError> {
|
||||
Self::from_env_with_toml(None).await
|
||||
}
|
||||
|
||||
/// Load from env with an optional TOML config file overlay.
|
||||
pub async fn from_env_with_toml(
|
||||
toml_path: Option<&std::path::Path>,
|
||||
) -> Result<Self, ConfigError> {
|
||||
let _ = dotenvy::dotenv();
|
||||
crate::bootstrap::load_ironclaw_env();
|
||||
let settings = Settings::load();
|
||||
let mut settings = Settings::load();
|
||||
|
||||
// Overlay TOML config file (values win over JSON settings)
|
||||
Self::apply_toml_overlay(&mut settings, toml_path)?;
|
||||
|
||||
Self::build(&settings).await
|
||||
}
|
||||
|
||||
/// Load and merge a TOML config file into settings.
|
||||
///
|
||||
/// If `explicit_path` is `Some`, loads from that path (errors are fatal).
|
||||
/// If `None`, tries the default path `~/.ironclaw/config.toml` (missing
|
||||
/// file is silently ignored).
|
||||
fn apply_toml_overlay(
|
||||
settings: &mut Settings,
|
||||
explicit_path: Option<&std::path::Path>,
|
||||
) -> Result<(), ConfigError> {
|
||||
let path = explicit_path
|
||||
.map(std::path::PathBuf::from)
|
||||
.unwrap_or_else(Settings::default_toml_path);
|
||||
|
||||
match Settings::load_toml(&path) {
|
||||
Ok(Some(toml_settings)) => {
|
||||
settings.merge_from(&toml_settings);
|
||||
tracing::debug!("Loaded TOML config from {}", path.display());
|
||||
}
|
||||
Ok(None) => {
|
||||
if explicit_path.is_some() {
|
||||
return Err(ConfigError::ParseError(format!(
|
||||
"Config file not found: {}",
|
||||
path.display()
|
||||
)));
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
if explicit_path.is_some() {
|
||||
return Err(ConfigError::ParseError(format!(
|
||||
"Failed to load config file {}: {}",
|
||||
path.display(),
|
||||
e
|
||||
)));
|
||||
}
|
||||
tracing::warn!("Failed to load default config file: {}", e);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Build config from settings (shared by from_env and from_db).
|
||||
async fn build(settings: &Settings) -> Result<Self, ConfigError> {
|
||||
Ok(Self {
|
||||
@@ -97,6 +161,9 @@ impl Config {
|
||||
routines: RoutineConfig::resolve()?,
|
||||
sandbox: SandboxModeConfig::resolve()?,
|
||||
claude_code: ClaudeCodeConfig::resolve()?,
|
||||
observability: crate::observability::ObservabilityConfig {
|
||||
backend: std::env::var("OBSERVABILITY_BACKEND").unwrap_or_else(|_| "none".into()),
|
||||
},
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -105,10 +172,21 @@ impl Config {
|
||||
///
|
||||
/// Used by channels and tools that need public webhook endpoints.
|
||||
/// The tunnel URL is shared across all channels (Telegram, Slack, etc.).
|
||||
///
|
||||
/// Two modes:
|
||||
/// - **Static URL** (`TUNNEL_URL`): set the public URL directly (manual tunnel)
|
||||
/// - **Managed provider** (`TUNNEL_PROVIDER`): lifecycle-managed tunnel process
|
||||
///
|
||||
/// When a managed provider is configured _and_ no static URL is set,
|
||||
/// the gateway starts the tunnel on boot and populates `public_url`.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct TunnelConfig {
|
||||
/// Public URL from tunnel provider (e.g., "https://abc123.ngrok.io").
|
||||
/// Set statically via `TUNNEL_URL` or populated at runtime by a managed tunnel.
|
||||
pub public_url: Option<String>,
|
||||
/// Provider configuration for lifecycle-managed tunnels.
|
||||
/// `None` when using a static URL or no tunnel at all.
|
||||
pub provider: Option<crate::tunnel::TunnelProviderConfig>,
|
||||
}
|
||||
|
||||
impl TunnelConfig {
|
||||
@@ -125,12 +203,65 @@ impl TunnelConfig {
|
||||
});
|
||||
}
|
||||
|
||||
Ok(Self { public_url })
|
||||
// Resolve managed tunnel provider config.
|
||||
// Priority: env var > settings > default (none).
|
||||
let provider_name = optional_env("TUNNEL_PROVIDER")?
|
||||
.or_else(|| settings.tunnel.provider.clone())
|
||||
.unwrap_or_default();
|
||||
|
||||
let provider = if provider_name.is_empty() || provider_name == "none" {
|
||||
None
|
||||
} else {
|
||||
Some(crate::tunnel::TunnelProviderConfig {
|
||||
provider: provider_name.clone(),
|
||||
cloudflare: optional_env("TUNNEL_CF_TOKEN")?
|
||||
.or_else(|| settings.tunnel.cf_token.clone())
|
||||
.map(|token| crate::tunnel::CloudflareTunnelConfig { token }),
|
||||
tailscale: Some(crate::tunnel::TailscaleTunnelConfig {
|
||||
funnel: optional_env("TUNNEL_TS_FUNNEL")
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|s| s == "true" || s == "1")
|
||||
.unwrap_or(settings.tunnel.ts_funnel),
|
||||
hostname: optional_env("TUNNEL_TS_HOSTNAME")
|
||||
.ok()
|
||||
.flatten()
|
||||
.or_else(|| settings.tunnel.ts_hostname.clone()),
|
||||
}),
|
||||
ngrok: optional_env("TUNNEL_NGROK_TOKEN")?
|
||||
.or_else(|| settings.tunnel.ngrok_token.clone())
|
||||
.map(|auth_token| crate::tunnel::NgrokTunnelConfig {
|
||||
auth_token,
|
||||
domain: optional_env("TUNNEL_NGROK_DOMAIN")
|
||||
.ok()
|
||||
.flatten()
|
||||
.or_else(|| settings.tunnel.ngrok_domain.clone()),
|
||||
}),
|
||||
custom: optional_env("TUNNEL_CUSTOM_COMMAND")?
|
||||
.or_else(|| settings.tunnel.custom_command.clone())
|
||||
.map(|start_command| crate::tunnel::CustomTunnelConfig {
|
||||
start_command,
|
||||
health_url: optional_env("TUNNEL_CUSTOM_HEALTH_URL")
|
||||
.ok()
|
||||
.flatten()
|
||||
.or_else(|| settings.tunnel.custom_health_url.clone()),
|
||||
url_pattern: optional_env("TUNNEL_CUSTOM_URL_PATTERN")
|
||||
.ok()
|
||||
.flatten()
|
||||
.or_else(|| settings.tunnel.custom_url_pattern.clone()),
|
||||
}),
|
||||
})
|
||||
};
|
||||
|
||||
Ok(Self {
|
||||
public_url,
|
||||
provider,
|
||||
})
|
||||
}
|
||||
|
||||
/// Check if a tunnel is configured.
|
||||
/// Check if a tunnel is configured (static URL or managed provider).
|
||||
pub fn is_enabled(&self) -> bool {
|
||||
self.public_url.is_some()
|
||||
self.public_url.is_some() || self.provider.is_some()
|
||||
}
|
||||
|
||||
/// Get the webhook URL for a given path.
|
||||
@@ -419,6 +550,19 @@ pub struct NearAiConfig {
|
||||
/// With the default of 3, the provider makes up to 4 total attempts
|
||||
/// (1 initial + 3 retries) before giving up.
|
||||
pub max_retries: u32,
|
||||
/// Consecutive transient failures before the circuit breaker opens.
|
||||
/// None = disabled (default). E.g. 5 means after 5 consecutive failures
|
||||
/// all requests are rejected until recovery timeout elapses.
|
||||
pub circuit_breaker_threshold: Option<u32>,
|
||||
/// How long (seconds) the circuit stays open before allowing a probe (default: 30).
|
||||
pub circuit_breaker_recovery_secs: u64,
|
||||
/// Enable in-memory response caching for `complete()` calls.
|
||||
/// Saves tokens on repeated prompts within a session. Default: false.
|
||||
pub response_cache_enabled: bool,
|
||||
/// TTL in seconds for cached responses (default: 3600 = 1 hour).
|
||||
pub response_cache_ttl_secs: u64,
|
||||
/// Max cached responses before LRU eviction (default: 1000).
|
||||
pub response_cache_max_entries: usize,
|
||||
/// Cooldown duration in seconds for the failover provider (default: 300).
|
||||
/// When a provider accumulates enough consecutive failures it is skipped
|
||||
/// for this many seconds.
|
||||
@@ -485,6 +629,17 @@ impl LlmConfig {
|
||||
api_key: nearai_api_key,
|
||||
fallback_model: optional_env("NEARAI_FALLBACK_MODEL")?,
|
||||
max_retries: parse_optional_env("NEARAI_MAX_RETRIES", 3)?,
|
||||
circuit_breaker_threshold: optional_env("CIRCUIT_BREAKER_THRESHOLD")?
|
||||
.map(|s| s.parse())
|
||||
.transpose()
|
||||
.map_err(|e| ConfigError::InvalidValue {
|
||||
key: "CIRCUIT_BREAKER_THRESHOLD".to_string(),
|
||||
message: format!("must be a positive integer: {e}"),
|
||||
})?,
|
||||
circuit_breaker_recovery_secs: parse_optional_env("CIRCUIT_BREAKER_RECOVERY_SECS", 30)?,
|
||||
response_cache_enabled: parse_optional_env("RESPONSE_CACHE_ENABLED", false)?,
|
||||
response_cache_ttl_secs: parse_optional_env("RESPONSE_CACHE_TTL_SECS", 3600)?,
|
||||
response_cache_max_entries: parse_optional_env("RESPONSE_CACHE_MAX_ENTRIES", 1000)?,
|
||||
failover_cooldown_secs: parse_optional_env("LLM_FAILOVER_COOLDOWN_SECS", 300)?,
|
||||
failover_cooldown_threshold: parse_optional_env("LLM_FAILOVER_THRESHOLD", 3)?,
|
||||
};
|
||||
@@ -755,6 +910,10 @@ pub struct AgentConfig {
|
||||
pub session_idle_timeout: Duration,
|
||||
/// Allow chat to use filesystem/shell tools directly (bypass sandbox).
|
||||
pub allow_local_tools: bool,
|
||||
/// Maximum daily LLM spend in cents (e.g. 10000 = $100). None = unlimited.
|
||||
pub max_cost_per_day_cents: Option<u64>,
|
||||
/// Maximum LLM/tool actions per hour. None = unlimited.
|
||||
pub max_actions_per_hour: Option<u64>,
|
||||
}
|
||||
|
||||
impl AgentConfig {
|
||||
@@ -833,6 +992,20 @@ impl AgentConfig {
|
||||
message: format!("must be 'true' or 'false': {e}"),
|
||||
})?
|
||||
.unwrap_or(false),
|
||||
max_cost_per_day_cents: optional_env("MAX_COST_PER_DAY_CENTS")?
|
||||
.map(|s| s.parse())
|
||||
.transpose()
|
||||
.map_err(|e| ConfigError::InvalidValue {
|
||||
key: "MAX_COST_PER_DAY_CENTS".to_string(),
|
||||
message: format!("must be a positive integer: {e}"),
|
||||
})?,
|
||||
max_actions_per_hour: optional_env("MAX_ACTIONS_PER_HOUR")?
|
||||
.map(|s| s.parse())
|
||||
.transpose()
|
||||
.map_err(|e| ConfigError::InvalidValue {
|
||||
key: "MAX_ACTIONS_PER_HOUR".to_string(),
|
||||
message: format!("must be a positive integer: {e}"),
|
||||
})?,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user