Files
optimclaw/src/secrets/mod.rs
T
fe53f6993f chore: promote staging to staging-promote/57c397bd-23120362128 (2026-03-16 05:35 UTC) (#1236)
* refactor(setup): extract init logic from wizard into owning modules (#1210)

* refactor(setup): extract init logic from wizard into owning modules

Move database, LLM model discovery, and secrets initialization logic
out of the setup wizard and into their owning modules, following the
CLAUDE.md principle that module-specific initialization must live in
the owning module as a public factory function.

Database (src/db/mod.rs, src/config/database.rs):
- Add DatabaseConfig::from_postgres_url() and from_libsql_path()
- Add connect_without_migrations() for connectivity testing
- Add validate_postgres() returning structured PgDiagnostic results

LLM (src/llm/models.rs — new file):
- Extract 8 model-fetching functions from wizard.rs (~380 lines)
- fetch_anthropic_models, fetch_openai_models, fetch_ollama_models,
  fetch_openai_compatible_models, build_nearai_model_fetch_config,
  and OpenAI sorting/filtering helpers

Secrets (src/secrets/mod.rs):
- Add resolve_master_key() unifying env var + keychain resolution
- Add crypto_from_hex() convenience wrapper

Wizard restructuring (src/setup/wizard.rs):
- Replace cfg-gated db_pool/db_backend fields with generic
  db: Option<Arc<dyn Database>> + db_handles: Option<DatabaseHandles>
- Delete 6 backend-specific methods (reconnect_postgres/libsql,
  test_database_connection_postgres/libsql, run_migrations_postgres/
  libsql, create_postgres/libsql_secrets_store)
- Simplify persist_settings, try_load_existing_settings,
  persist_session_to_db, init_secrets_context to backend-agnostic
  implementations using the new module factories
- Eliminate all references to deadpool_postgres, PoolConfig,
  LibSqlBackend, Store::from_pool, refinery::embed_migrations

Net: -878 lines from wizard, +395 lines in owning modules, +378 new.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* test(settings): add wizard re-run regression tests

Add 10 tests covering settings preservation during wizard re-runs:
- provider_only rerun preserves channels/embeddings/heartbeat
- channels_only rerun preserves provider/model/embeddings
- quick mode rerun preserves prior channels and heartbeat
- full rerun same provider preserves model through merge
- full rerun different provider clears model through merge
- incremental persist doesn't clobber prior steps
- switching DB backend allows fresh connection settings
- merge preserves true booleans when overlay has default false
- embeddings survive rerun that skips step 5

These cover the scenarios where re-running the wizard would
previously risk resetting models, providers, or channel settings.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(setup): eliminate cfg(feature) gates from wizard methods

Replace compile-time #[cfg(feature)] dispatch in the wizard with
runtime dispatch via DatabaseBackend enum and cfg!() macro constants.

- Merge step_database_postgres + step_database_libsql into step_database
  using runtime backend selection
- Rewrite auto_setup_database without feature gates
- Remove cfg(feature = "postgres") from mask_password_in_url (pure fn)
- Remove cfg(feature = "postgres") from test_mask_password_in_url

Only one internal #[cfg(feature = "postgres")] remains: guarding the
call to db::validate_postgres() which is itself feature-gated.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* refactor(db): fold PG validation into connect_without_migrations

Move PostgreSQL prerequisite validation (version >= 15, pgvector)
from the wizard into connect_without_migrations() in the db module.
The validation now returns DatabaseError directly with user-facing
messages, eliminating the PgDiagnostic enum and the last
#[cfg(feature)] gate from the wizard.

The wizard's test_database_connection() is now a 5-line method that
calls the db module factory and stores the result.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address PR review comments [skip-regression-check]

- Use .as_ref().map() to avoid partial move of db_config.libsql_path
  (gemini-code-assist)
- Default to available backend when DATABASE_BACKEND is invalid, not
  unconditionally to Postgres which may not be compiled (Copilot)
- Match DatabaseBackend::Postgres explicitly instead of _ => wildcard
  in connect_with_handles, connect_without_migrations, and
  create_secrets_store to avoid silently routing LibSql configs through
  the Postgres path when libsql feature is disabled (Copilot)
- Upgrade Ollama connection failure log from info to warn with the
  base URL for better visibility in wizard UX (Copilot)
- Clarify crypto_from_hex doc: SecretsCrypto validates key length,
  not hex encoding (Copilot)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address zmanian's PR review feedback [skip-regression-check]

- Update src/setup/README.md to reflect Arc<dyn Database> flow
- Remove stale "Test PostgreSQL connection" doc comment
- Replace unwrap_or(0) in validate_postgres with descriptive error
- Add NearAiConfig::for_model_discovery() constructor
- Narrow pub to pub(crate) for internal model helpers

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

* fix: address Copilot review comments (quick-mode postgres gate, empty env vars) [skip-regression-check]

- Gate DATABASE_URL auto-detection on POSTGRES_AVAILABLE in quick mode
  so libsql-only builds don't attempt a postgres connection
- Match empty-env-var filtering in key source detection to align with
  resolve_master_key() behavior
- Filter empty strings to None in DatabaseConfig::from_libsql_path()
  for turso_url/turso_token

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

* fix: Telegram bot token validation fails intermittently (HTTP 404) (#1166)

* fix: Telegram bot token validation fails intermittently (HTTP 404)

* fix: code style

* fix

* fix

* fix

* review fix

---------

Co-authored-by: Illia Polosukhin <[email protected]>
Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
Co-authored-by: Nick Pismenkov <[email protected]>
2026-03-16 08:09:34 +00:00

168 lines
6.5 KiB
Rust

//! Secrets management for secure credential storage and injection.
//!
//! This module provides:
//! - AES-256-GCM encrypted secret storage
//! - Per-secret key derivation (HKDF-SHA256)
//! - PostgreSQL persistence
//! - OS keychain integration for master key
//! - Access control for WASM tools
//!
//! # Security Model
//!
//! ```text
//! ┌─────────────────────────────────────────────────────────────────────────────┐
//! │ Secret Lifecycle │
//! │ │
//! │ User stores secret ──► Encrypt with AES-256-GCM ──► Store in PostgreSQL │
//! │ (per-secret key via HKDF) │
//! │ │
//! │ WASM requests HTTP ──► Host checks allowlist ──► Decrypt secret ──► │
//! │ & allowed_secrets (in memory only) │
//! │ │ │
//! │ ▼ │
//! │ Inject into request ──► Execute HTTP call │
//! │ (WASM never sees value) │
//! │ │ │
//! │ ▼ │
//! │ Leak detector scans ──► Return response to WASM │
//! │ response for secrets │
//! └─────────────────────────────────────────────────────────────────────────────┘
//! ```
//!
//! # Master Key Storage
//!
//! The master key for encrypting secrets can come from:
//! - **OS Keychain** (recommended for local installs): Auto-generated and stored securely
//! - **Environment variable** (for CI/Docker): Set `SECRETS_MASTER_KEY`
//!
//! # Example
//!
//! ```ignore
//! use ironclaw::secrets::{SecretsStore, PostgresSecretsStore, SecretsCrypto, CreateSecretParams};
//! use secrecy::SecretString;
//!
//! // Initialize crypto with master key from environment
//! let master_key = SecretString::from(std::env::var("SECRETS_MASTER_KEY")?);
//! let crypto = Arc::new(SecretsCrypto::new(master_key)?);
//!
//! // Create store
//! let store = PostgresSecretsStore::new(pool, crypto);
//!
//! // Store a secret
//! store.create("user_123", CreateSecretParams::new("openai_key", "sk-...")).await?;
//!
//! // Check if secret exists (WASM can call this)
//! let exists = store.exists("user_123", "openai_key").await?;
//!
//! // Decrypt for injection (host boundary only)
//! let decrypted = store.get_decrypted("user_123", "openai_key").await?;
//! ```
mod crypto;
pub mod keychain;
mod store;
mod types;
pub use crypto::SecretsCrypto;
#[cfg(feature = "libsql")]
pub use store::LibSqlSecretsStore;
#[cfg(feature = "postgres")]
pub use store::PostgresSecretsStore;
pub use store::SecretsStore;
pub use types::{
CreateSecretParams, CredentialLocation, CredentialMapping, DecryptedSecret, Secret,
SecretError, SecretRef,
};
pub use store::in_memory::InMemorySecretsStore;
/// Create a secrets store from a master key and database handles.
///
/// Returns `None` if no matching backend handle is available (e.g. when
/// running without a database). This is a normal condition in no-db mode,
/// not an error — callers should treat `None` as "secrets unavailable".
pub fn create_secrets_store(
crypto: std::sync::Arc<SecretsCrypto>,
handles: &crate::db::DatabaseHandles,
) -> Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> {
let store: Option<std::sync::Arc<dyn SecretsStore + Send + Sync>> = None;
#[cfg(feature = "libsql")]
let store = store.or_else(|| {
handles.libsql_db.as_ref().map(|db| {
std::sync::Arc::new(LibSqlSecretsStore::new(
std::sync::Arc::clone(db),
std::sync::Arc::clone(&crypto),
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
})
});
#[cfg(feature = "postgres")]
let store = store.or_else(|| {
handles.pg_pool.as_ref().map(|pool| {
std::sync::Arc::new(PostgresSecretsStore::new(
pool.clone(),
std::sync::Arc::clone(&crypto),
)) as std::sync::Arc<dyn SecretsStore + Send + Sync>
})
});
store
}
/// Try to resolve an existing master key from env var or OS keychain.
///
/// Resolution order:
/// 1. `SECRETS_MASTER_KEY` environment variable (hex-encoded)
/// 2. OS keychain (macOS Keychain / Linux secret-service)
///
/// Returns `None` if no key is available (caller should generate one).
pub async fn resolve_master_key() -> Option<String> {
// 1. Check env var
if let Ok(env_key) = std::env::var("SECRETS_MASTER_KEY")
&& !env_key.is_empty()
{
return Some(env_key);
}
// 2. Try OS keychain
if let Ok(keychain_key_bytes) = keychain::get_master_key().await {
let key_hex: String = keychain_key_bytes
.iter()
.map(|b| format!("{:02x}", b))
.collect();
return Some(key_hex);
}
None
}
/// Create a `SecretsCrypto` from a master key string.
///
/// The key is typically hex-encoded (from `generate_master_key_hex` or
/// the `SECRETS_MASTER_KEY` env var), but `SecretsCrypto::new` validates
/// only key length, not encoding. Any sufficiently long string works.
pub fn crypto_from_hex(hex: &str) -> Result<std::sync::Arc<SecretsCrypto>, SecretError> {
let crypto = SecretsCrypto::new(secrecy::SecretString::from(hex.to_string()))?;
Ok(std::sync::Arc::new(crypto))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_crypto_from_hex_valid() {
// 32 bytes = 64 hex chars
let hex = "0123456789abcdef".repeat(4); // 64 hex chars
let result = crypto_from_hex(&hex);
assert!(result.is_ok()); // safety: test assertion
}
#[test]
fn test_crypto_from_hex_invalid() {
let result = crypto_from_hex("too_short");
assert!(result.is_err()); // safety: test assertion
}
}