mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 15:40:18 +00:00
* refactor(setup): extract init logic from wizard into owning modules (#1210) * refactor(setup): extract init logic from wizard into owning modules Move database, LLM model discovery, and secrets initialization logic out of the setup wizard and into their owning modules, following the CLAUDE.md principle that module-specific initialization must live in the owning module as a public factory function. Database (src/db/mod.rs, src/config/database.rs): - Add DatabaseConfig::from_postgres_url() and from_libsql_path() - Add connect_without_migrations() for connectivity testing - Add validate_postgres() returning structured PgDiagnostic results LLM (src/llm/models.rs — new file): - Extract 8 model-fetching functions from wizard.rs (~380 lines) - fetch_anthropic_models, fetch_openai_models, fetch_ollama_models, fetch_openai_compatible_models, build_nearai_model_fetch_config, and OpenAI sorting/filtering helpers Secrets (src/secrets/mod.rs): - Add resolve_master_key() unifying env var + keychain resolution - Add crypto_from_hex() convenience wrapper Wizard restructuring (src/setup/wizard.rs): - Replace cfg-gated db_pool/db_backend fields with generic db: Option<Arc<dyn Database>> + db_handles: Option<DatabaseHandles> - Delete 6 backend-specific methods (reconnect_postgres/libsql, test_database_connection_postgres/libsql, run_migrations_postgres/ libsql, create_postgres/libsql_secrets_store) - Simplify persist_settings, try_load_existing_settings, persist_session_to_db, init_secrets_context to backend-agnostic implementations using the new module factories - Eliminate all references to deadpool_postgres, PoolConfig, LibSqlBackend, Store::from_pool, refinery::embed_migrations Net: -878 lines from wizard, +395 lines in owning modules, +378 new. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test(settings): add wizard re-run regression tests Add 10 tests covering settings preservation during wizard re-runs: - provider_only rerun preserves channels/embeddings/heartbeat - channels_only rerun preserves provider/model/embeddings - quick mode rerun preserves prior channels and heartbeat - full rerun same provider preserves model through merge - full rerun different provider clears model through merge - incremental persist doesn't clobber prior steps - switching DB backend allows fresh connection settings - merge preserves true booleans when overlay has default false - embeddings survive rerun that skips step 5 These cover the scenarios where re-running the wizard would previously risk resetting models, providers, or channel settings. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(setup): eliminate cfg(feature) gates from wizard methods Replace compile-time #[cfg(feature)] dispatch in the wizard with runtime dispatch via DatabaseBackend enum and cfg!() macro constants. - Merge step_database_postgres + step_database_libsql into step_database using runtime backend selection - Rewrite auto_setup_database without feature gates - Remove cfg(feature = "postgres") from mask_password_in_url (pure fn) - Remove cfg(feature = "postgres") from test_mask_password_in_url Only one internal #[cfg(feature = "postgres")] remains: guarding the call to db::validate_postgres() which is itself feature-gated. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(db): fold PG validation into connect_without_migrations Move PostgreSQL prerequisite validation (version >= 15, pgvector) from the wizard into connect_without_migrations() in the db module. The validation now returns DatabaseError directly with user-facing messages, eliminating the PgDiagnostic enum and the last #[cfg(feature)] gate from the wizard. The wizard's test_database_connection() is now a 5-line method that calls the db module factory and stores the result. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments [skip-regression-check] - Use .as_ref().map() to avoid partial move of db_config.libsql_path (gemini-code-assist) - Default to available backend when DATABASE_BACKEND is invalid, not unconditionally to Postgres which may not be compiled (Copilot) - Match DatabaseBackend::Postgres explicitly instead of _ => wildcard in connect_with_handles, connect_without_migrations, and create_secrets_store to avoid silently routing LibSql configs through the Postgres path when libsql feature is disabled (Copilot) - Upgrade Ollama connection failure log from info to warn with the base URL for better visibility in wizard UX (Copilot) - Clarify crypto_from_hex doc: SecretsCrypto validates key length, not hex encoding (Copilot) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address zmanian's PR review feedback [skip-regression-check] - Update src/setup/README.md to reflect Arc<dyn Database> flow - Remove stale "Test PostgreSQL connection" doc comment - Replace unwrap_or(0) in validate_postgres with descriptive error - Add NearAiConfig::for_model_discovery() constructor - Narrow pub to pub(crate) for internal model helpers Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address Copilot review comments (quick-mode postgres gate, empty env vars) [skip-regression-check] - Gate DATABASE_URL auto-detection on POSTGRES_AVAILABLE in quick mode so libsql-only builds don't attempt a postgres connection - Match empty-env-var filtering in key source detection to align with resolve_master_key() behavior - Filter empty strings to None in DatabaseConfig::from_libsql_path() for turso_url/turso_token Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> * fix: Telegram bot token validation fails intermittently (HTTP 404) (#1166) * fix: Telegram bot token validation fails intermittently (HTTP 404) * fix: code style * fix * fix * fix * review fix --------- Co-authored-by: Illia Polosukhin <[email protected]> Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: Nick Pismenkov <[email protected]>
168 lines
6.5 KiB
Rust
168 lines
6.5 KiB
Rust
//! Secrets management for secure credential storage and injection.
|
|
//!
|
|
//! This module provides:
|
|
//! - AES-256-GCM encrypted secret storage
|
|
//! - Per-secret key derivation (HKDF-SHA256)
|
|
//! - PostgreSQL persistence
|
|
//! - OS keychain integration for master key
|
|
//! - Access control for WASM tools
|
|
//!
|
|
//! # Security Model
|
|
//!
|
|
//! ```text
|
|
//! ┌─────────────────────────────────────────────────────────────────────────────┐
|
|
//! │ Secret Lifecycle │
|
|
//! │ │
|
|
//! │ User stores secret ──► Encrypt with AES-256-GCM ──► Store in PostgreSQL │
|
|
//! │ (per-secret key via HKDF) │
|
|
//! │ │
|
|
//! │ WASM requests HTTP ──► Host checks allowlist ──► Decrypt secret ──► │
|
|
//! │ & allowed_secrets (in memory only) │
|
|
//! │ │ │
|
|
//! │ ▼ │
|
|
//! │ Inject into request ──► Execute HTTP call │
|
|
//! │ (WASM never sees value) │
|
|
//! │ │ │
|
|
//! │ ▼ │
|
|
//! │ Leak detector scans ──► Return response to WASM │
|
|
//! │ response for secrets │
|
|
//! └─────────────────────────────────────────────────────────────────────────────┘
|
|
//! ```
|
|
//!
|
|
//! # Master Key Storage
|
|
//!
|
|
//! The master key for encrypting secrets can come from:
|
|
//! - **OS Keychain** (recommended for local installs): Auto-generated and stored securely
|
|
//! - **Environment variable** (for CI/Docker): Set `SECRETS_MASTER_KEY`
|
|
//!
|
|
//! # Example
|
|
//!
|
|
//! ```ignore
|
|
//! use ironclaw::secrets::{SecretsStore, PostgresSecretsStore, SecretsCrypto, CreateSecretParams};
|
|
//! use secrecy::SecretString;
|
|
//!
|
|
//! // Initialize crypto with master key from environment
|
|
//! let master_key = SecretString::from(std::env::var("SECRETS_MASTER_KEY")?);
|
|
//! let crypto = Arc::new(SecretsCrypto::new(master_key)?);
|
|
//!
|
|
//! // Create store
|
|
//! let store = PostgresSecretsStore::new(pool, crypto);
|
|
//!
|
|
//! // Store a secret
|
|
//! store.create("user_123", CreateSecretParams::new("openai_key", "sk-...")).await?;
|
|
//!
|
|
//! // Check if secret exists (WASM can call this)
|
|
//! let exists = store.exists("user_123", "openai_key").await?;
|
|
//!
|
|
//! // Decrypt for injection (host boundary only)
|
|
//! let decrypted = store.get_decrypted("user_123", "openai_key").await?;
|
|
//! ```
|
|
|
|
mod crypto;
|
|
pub mod keychain;
|
|
mod store;
|
|
mod types;
|
|
|
|
pub use crypto::SecretsCrypto;
|
|
#[cfg(feature = "libsql")]
|
|
pub use store::LibSqlSecretsStore;
|
|
#[cfg(feature = "postgres")]
|
|
pub use store::PostgresSecretsStore;
|
|
pub use store::SecretsStore;
|
|
pub use types::{
|
|
CreateSecretParams, CredentialLocation, CredentialMapping, DecryptedSecret, Secret,
|
|
SecretError, SecretRef,
|
|
};
|
|
|
|
pub use store::in_memory::InMemorySecretsStore;
|
|
|
|
/// Create a secrets store from a master key and database handles.
|
|
///
|
|
/// Returns `None` if no matching backend handle is available (e.g. when
|
|
/// running without a database). This is a normal condition in no-db mode,
|
|
/// not an error — callers should treat `None` as "secrets unavailable".
|
|
pub fn create_secrets_store(
|
|
crypto: std::sync::Arc<SecretsCrypto>,
|
|
handles: &crate::db::DatabaseHandles,
|
|
) -> Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> {
|
|
let store: Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> = None;
|
|
|
|
#[cfg(feature = "libsql")]
|
|
let store = store.or_else(|| {
|
|
handles.libsql_db.as_ref().map(|db| {
|
|
std::sync::Arc::new(LibSqlSecretsStore::new(
|
|
std::sync::Arc::clone(db),
|
|
std::sync::Arc::clone(&crypto),
|
|
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
|
|
})
|
|
});
|
|
|
|
#[cfg(feature = "postgres")]
|
|
let store = store.or_else(|| {
|
|
handles.pg_pool.as_ref().map(|pool| {
|
|
std::sync::Arc::new(PostgresSecretsStore::new(
|
|
pool.clone(),
|
|
std::sync::Arc::clone(&crypto),
|
|
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
|
|
})
|
|
});
|
|
|
|
store
|
|
}
|
|
|
|
/// Try to resolve an existing master key from env var or OS keychain.
|
|
///
|
|
/// Resolution order:
|
|
/// 1. `SECRETS_MASTER_KEY` environment variable (hex-encoded)
|
|
/// 2. OS keychain (macOS Keychain / Linux secret-service)
|
|
///
|
|
/// Returns `None` if no key is available (caller should generate one).
|
|
pub async fn resolve_master_key() -> Option<String> {
|
|
// 1. Check env var
|
|
if let Ok(env_key) = std::env::var("SECRETS_MASTER_KEY")
|
|
&& !env_key.is_empty()
|
|
{
|
|
return Some(env_key);
|
|
}
|
|
|
|
// 2. Try OS keychain
|
|
if let Ok(keychain_key_bytes) = keychain::get_master_key().await {
|
|
let key_hex: String = keychain_key_bytes
|
|
.iter()
|
|
.map(|b| format!("{:02x}", b))
|
|
.collect();
|
|
return Some(key_hex);
|
|
}
|
|
|
|
None
|
|
}
|
|
|
|
/// Create a `SecretsCrypto` from a master key string.
|
|
///
|
|
/// The key is typically hex-encoded (from `generate_master_key_hex` or
|
|
/// the `SECRETS_MASTER_KEY` env var), but `SecretsCrypto::new` validates
|
|
/// only key length, not encoding. Any sufficiently long string works.
|
|
pub fn crypto_from_hex(hex: &str) -> Result<std::sync::Arc<SecretsCrypto>, SecretError> {
|
|
let crypto = SecretsCrypto::new(secrecy::SecretString::from(hex.to_string()))?;
|
|
Ok(std::sync::Arc::new(crypto))
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn test_crypto_from_hex_valid() {
|
|
// 32 bytes = 64 hex chars
|
|
let hex = "0123456789abcdef".repeat(4); // 64 hex chars
|
|
let result = crypto_from_hex(&hex);
|
|
assert!(result.is_ok()); // safety: test assertion
|
|
}
|
|
|
|
#[test]
|
|
fn test_crypto_from_hex_invalid() {
|
|
let result = crypto_from_hex("too_short");
|
|
assert!(result.is_err()); // safety: test assertion
|
|
}
|
|
}
|