mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
* feat(ux): complete UX overhaul — design system, boot screen, onboarding, web polish Shared design system: CSS custom properties for spacing, typography, transitions, and color tokens used across web UI and boot screen. Boot screen: compact feature-tags line showing enabled subsystems (db, tools, routines, heartbeat, skills, sandbox, embeddings) at a glance. Downgrade startup info logs (libSQL, webhook, workspace seed) to debug level since the boot screen now covers this. Onboarding wizard: model picker with live API fetch, provider-aware auth flow, improved error recovery and progress display. Web UI: ARIA attributes, welcome card, streaming debounce, connection status banner, skeleton loaders, send cooldown. CLI: doctor command enhancements, status command cleanup, REPL banner consolidation, shared fmt module. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(ux): Apple-level design refinements — spring physics, glass morphism, chat polish Merge staging theme support (dark/light/system toggle) and layer UX polish on top: spring-physics motion, glass morphism depth, chat experience improvements, and responsive mobile refinements. Design system: - Restore and extend design token system (spacing, typography, timing, easing) with legacy aliases for theme compatibility - Add shadow tiers, accent glow, glass morphism, spring easing tokens - Tokens defined in both dark (:root) and light ([data-theme="light"]) Micro-interactions (Phase 2): - Spring-overshoot message entry animation (slideUp) - Spring-scale button press on all interactive buttons - Tab crossfade animation, tool card smooth accordion (max-height) - Modal scale(0.95) + blur(8px) entry, toast spring slide - Sidebar width crossfade, card hover lift Visual depth (Phase 3): - Tab bar glass morphism + surface highlight + sliding indicator - Active tab accent background pill - Assistant message accent left border, user message bubble tail - Floating input area (rounded + shadow + margin) Chat polish (Phase 4): - Smooth streaming cursor (cursorPulse), message hover timestamps - Time separators (Today/Yesterday/date) - Textarea smooth auto-expand, send button glow Settings & forms (Phase 5): - iOS-style toggle switches for boolean settings - Input focus glow, save feedback spring animation - Welcome card with gradient background + proper spacing - Sticky settings group headers with glass backdrop Accessibility & mobile (Phase 6): - Animated focus ring, prefers-reduced-motion global kill-switch - Touch target audit (44px min), mobile bottom-sheet modals - Mobile bottom tab bar, toast redesign (icon + border + countdown) - Thread hover translateX, badge in_progress pulse Bug fixes: - Gateway/TEE popover z-index (tab-bar z-index: 200, popovers 500) - Connection lost banner as fixed top bar instead of flex child - Sidebar collapse keeps toggle + new thread buttons visible - Downgrade noisy startup logs (db, webhook, vector) to debug - Remove green dot pulse animation on connected status - Deduplicate confirm-modal in HTML, add tab-indicator div Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(web): mobile layout improvements — sidebar toggle, settings drill-down, tab bar polish - Fix mobile sidebar toggle: use expanded-mobile class instead of collapsed, add backdrop overlay, auto-close on thread select, outside-click dismiss - Settings: replace cramped horizontal tabs with drill-down navigation (category list → detail view → back button) - Bottom tab bar: add glass morphism, hide theme toggle, flip tab indicator to top edge - Keep thread toggle button visible in collapsed 36px sidebar strip Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(repl): interactive approval selector and transient status lines - Replace ASCII-art approval box with clean horizontal rule card - Add inquire-based interactive selector for tool approvals (↑↓ + Enter) - Selector runs directly from send_status via spawn_blocking, with stdin_locked flag to prevent readline from competing for stdin - Transient thinking/tool-started lines: each replaces the previous, all erased before final output (no clutter left in scrollback) - Esc in selector sends denial so agent never gets stuck Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: widen TurnCost token fields to u64 and remove unused variable - Change input_tokens/output_tokens from u32 to u64 in StatusUpdate::TurnCost, SseEvent::TurnCost, and the thread_ops emit site to avoid truncation on large conversations - Remove unused _routine_engine_for_loop binding in agent_loop.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore: reduce startup log noise — demote info to debug Demote routine startup messages (builder, WASM tools, tunnel, WASM channels) from info to debug so the default log output stays clean. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): allow CDN scripts in CSP connect-src directive Add cdn.jsdelivr.net and cdnjs.cloudflare.com to connect-src so the browser can fetch marked.js and DOMPurify without CSP violations. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix cargo fmt in repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): gate turn_cost SSE handler on current thread Prevents cost badge from attaching to the wrong message when switching threads or receiving events from background threads. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * ci: retrigger CI * fix: add missing extension_manager to webhook EngineContext The webhook trigger path added in #736 was missing the extension_manager field introduced by #1453. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore: ignore RUSTSEC-2026-0049 rustls-webpki CRL advisory Low impact — requires compromised CA to exploit. Tracked for upstream rustls-webpki upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(routines): use fields.join for cron normalization Use split_whitespace fields instead of re-trimming the original string to avoid preserving extra internal whitespace in cron expressions. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(repl): Apple-style approval card — clean vertical flow - Drop verbose tool description (the command IS the decision surface) - Unified vertical pipe layout: ◆ header → │ params → │ selector - Selector options show keyboard shortcuts inline: Approve (y) - Compact help message, answered state uses └ to close the flow - No horizontal rules, no blank-line padding — just breathing room Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(repl): replace inquire with crossterm for approval selector Drop the inquire dependency (which pulled in crossterm 0.25, duplicating the existing 0.28). The 3-option approval selector is now built directly with crossterm raw mode — same UX, zero new dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore(deps): upgrade crossterm 0.28 → 0.29, eliminate duplication termimad (via crokey) uses crossterm 0.29. Upgrading our direct dependency from 0.28 to 0.29 collapses to a single crossterm version in the dependency tree. Also migrated termimad::crossterm:: references to the direct crossterm import. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review comments — box_top off-by-one, smart_truncate overflow, mobile theme toggle - Fix box_top() fill calculation: was off-by-one, producing boxes 1 char too wide (fmt.rs) - Fix smart_truncate(): account for "..." in the budget so output never exceeds max_chars (repl.rs) - Move theme toggle to settings sidebar on mobile instead of display:none, so mobile users can still switch themes (style.css, index.html, app.js) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: cargo fmt repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review — retry duplication, CSP connect-src, deny color - Remove failed message before retry to prevent duplicate user messages - Revert connect-src to 'self' — CDN hosts only need script-src - Use red for Deny confirmation in REPL approval selector Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>
711 lines
26 KiB
Rust
711 lines
26 KiB
Rust
//! Database abstraction layer.
|
|
//!
|
|
//! Provides a backend-agnostic `Database` trait that unifies all persistence
|
|
//! operations. Two implementations exist behind feature flags:
|
|
//!
|
|
//! - `postgres` (default): Uses `deadpool-postgres` + `tokio-postgres`
|
|
//! - `libsql`: Uses libSQL (Turso's SQLite fork) for embedded/edge deployment
|
|
//!
|
|
//! The existing `Store`, `Repository`, `SecretsStore`, and `WasmToolStore`
|
|
//! types become thin wrappers that delegate to `Arc<dyn Database>`.
|
|
|
|
#[cfg(feature = "postgres")]
|
|
pub mod postgres;
|
|
|
|
#[cfg(feature = "postgres")]
|
|
pub mod tls;
|
|
|
|
#[cfg(feature = "libsql")]
|
|
pub mod libsql;
|
|
|
|
#[cfg(feature = "libsql")]
|
|
pub mod libsql_migrations;
|
|
|
|
use std::collections::HashMap;
|
|
use std::sync::Arc;
|
|
|
|
use async_trait::async_trait;
|
|
use chrono::{DateTime, Utc};
|
|
use rust_decimal::Decimal;
|
|
use uuid::Uuid;
|
|
|
|
use crate::agent::BrokenTool;
|
|
use crate::agent::routine::{Routine, RoutineRun, RunStatus};
|
|
use crate::context::{ActionRecord, JobContext, JobState};
|
|
use crate::error::DatabaseError;
|
|
use crate::error::WorkspaceError;
|
|
use crate::history::{
|
|
AgentJobRecord, AgentJobSummary, ConversationMessage, ConversationSummary, JobEventRecord,
|
|
LlmCallRecord, SandboxJobRecord, SandboxJobSummary, SettingRow,
|
|
};
|
|
use crate::workspace::{MemoryChunk, MemoryDocument, WorkspaceEntry};
|
|
use crate::workspace::{SearchConfig, SearchResult};
|
|
|
|
/// Create a database backend from configuration, run migrations, and return it.
|
|
///
|
|
/// This is the shared helper for CLI commands and other call sites that need
|
|
/// a simple `Arc<dyn Database>` without retaining backend-specific handles
|
|
/// (e.g., `pg_pool` or `libsql_conn` for the secrets store). The main agent
|
|
/// startup in `main.rs` uses its own initialization block because it also
|
|
/// captures those backend-specific handles.
|
|
pub async fn connect_from_config(
|
|
config: &crate::config::DatabaseConfig,
|
|
) -> Result<Arc<dyn Database>, DatabaseError> {
|
|
let (db, _handles) = connect_with_handles(config).await?;
|
|
Ok(db)
|
|
}
|
|
|
|
/// Backend-specific handles retained after database connection.
|
|
///
|
|
/// These are needed by satellite stores (e.g., `SecretsStore`) that require
|
|
/// a backend-specific handle rather than the generic `Arc<dyn Database>`.
|
|
#[derive(Default)]
|
|
pub struct DatabaseHandles {
|
|
#[cfg(feature = "postgres")]
|
|
pub pg_pool: Option<deadpool_postgres::Pool>,
|
|
#[cfg(feature = "libsql")]
|
|
pub libsql_db: Option<Arc<::libsql::Database>>,
|
|
}
|
|
|
|
/// Connect to the database, run migrations, and return both the generic
|
|
/// `Database` trait object and the backend-specific handles.
|
|
pub async fn connect_with_handles(
|
|
config: &crate::config::DatabaseConfig,
|
|
) -> Result<(Arc<dyn Database>, DatabaseHandles), DatabaseError> {
|
|
let mut handles = DatabaseHandles::default();
|
|
|
|
match config.backend {
|
|
#[cfg(feature = "libsql")]
|
|
crate::config::DatabaseBackend::LibSql => {
|
|
use secrecy::ExposeSecret as _;
|
|
|
|
let default_path = crate::config::default_libsql_path();
|
|
let db_path = config.libsql_path.as_deref().unwrap_or(&default_path);
|
|
|
|
let backend = if let Some(ref url) = config.libsql_url {
|
|
let token = config.libsql_auth_token.as_ref().ok_or_else(|| {
|
|
DatabaseError::Pool(
|
|
"LIBSQL_AUTH_TOKEN required when LIBSQL_URL is set".to_string(),
|
|
)
|
|
})?;
|
|
libsql::LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
} else {
|
|
libsql::LibSqlBackend::new_local(db_path)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
};
|
|
backend.run_migrations().await?;
|
|
tracing::debug!("libSQL database connected and migrations applied");
|
|
|
|
handles.libsql_db = Some(backend.shared_db());
|
|
|
|
Ok((Arc::new(backend) as Arc<dyn Database>, handles))
|
|
}
|
|
#[cfg(feature = "postgres")]
|
|
crate::config::DatabaseBackend::Postgres => {
|
|
let pg = postgres::PgBackend::new(config)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?;
|
|
pg.run_migrations().await?;
|
|
tracing::info!("PostgreSQL database connected and migrations applied");
|
|
|
|
handles.pg_pool = Some(pg.pool());
|
|
|
|
Ok((Arc::new(pg) as Arc<dyn Database>, handles))
|
|
}
|
|
#[allow(unreachable_patterns)]
|
|
_ => Err(DatabaseError::Pool(format!(
|
|
"Database backend '{}' is not available. Rebuild with the appropriate feature flag.",
|
|
config.backend
|
|
))),
|
|
}
|
|
}
|
|
|
|
/// Create a secrets store from database and secrets configuration.
|
|
///
|
|
/// This is the shared factory for CLI commands and other call sites that need
|
|
/// a `SecretsStore` without going through the full `AppBuilder`. Mirrors the
|
|
/// pattern of [`connect_from_config`] but returns a secrets-specific store.
|
|
pub async fn create_secrets_store(
|
|
config: &crate::config::DatabaseConfig,
|
|
crypto: Arc<crate::secrets::SecretsCrypto>,
|
|
) -> Result<Arc<dyn crate::secrets::SecretsStore + Send + Sync>, DatabaseError> {
|
|
match config.backend {
|
|
#[cfg(feature = "libsql")]
|
|
crate::config::DatabaseBackend::LibSql => {
|
|
use secrecy::ExposeSecret as _;
|
|
|
|
let default_path = crate::config::default_libsql_path();
|
|
let db_path = config.libsql_path.as_deref().unwrap_or(&default_path);
|
|
|
|
let backend = if let Some(ref url) = config.libsql_url {
|
|
let token = config.libsql_auth_token.as_ref().ok_or_else(|| {
|
|
DatabaseError::Pool(
|
|
"LIBSQL_AUTH_TOKEN required when LIBSQL_URL is set".to_string(),
|
|
)
|
|
})?;
|
|
libsql::LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
} else {
|
|
libsql::LibSqlBackend::new_local(db_path)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
};
|
|
backend.run_migrations().await?;
|
|
|
|
Ok(Arc::new(crate::secrets::LibSqlSecretsStore::new(
|
|
backend.shared_db(),
|
|
crypto,
|
|
)))
|
|
}
|
|
#[cfg(feature = "postgres")]
|
|
crate::config::DatabaseBackend::Postgres => {
|
|
let pg = postgres::PgBackend::new(config)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?;
|
|
pg.run_migrations().await?;
|
|
|
|
Ok(Arc::new(crate::secrets::PostgresSecretsStore::new(
|
|
pg.pool(),
|
|
crypto,
|
|
)))
|
|
}
|
|
#[allow(unreachable_patterns)]
|
|
_ => Err(DatabaseError::Pool(format!(
|
|
"Database backend '{}' is not available for secrets. Rebuild with the appropriate feature flag.",
|
|
config.backend
|
|
))),
|
|
}
|
|
}
|
|
|
|
// ==================== Wizard / testing helpers ====================
|
|
|
|
/// Connect to the database WITHOUT running migrations, validating
|
|
/// prerequisites when applicable (PostgreSQL version, pgvector).
|
|
///
|
|
/// Returns both the `Database` trait object and backend-specific handles.
|
|
/// Used by the wizard to test connectivity before committing — call
|
|
/// [`Database::run_migrations`] on the returned trait object when ready.
|
|
pub async fn connect_without_migrations(
|
|
config: &crate::config::DatabaseConfig,
|
|
) -> Result<(Arc<dyn Database>, DatabaseHandles), DatabaseError> {
|
|
let mut handles = DatabaseHandles::default();
|
|
|
|
match config.backend {
|
|
#[cfg(feature = "libsql")]
|
|
crate::config::DatabaseBackend::LibSql => {
|
|
use secrecy::ExposeSecret as _;
|
|
|
|
let default_path = crate::config::default_libsql_path();
|
|
let db_path = config.libsql_path.as_deref().unwrap_or(&default_path);
|
|
|
|
let backend = if let Some(ref url) = config.libsql_url {
|
|
let token = config.libsql_auth_token.as_ref().ok_or_else(|| {
|
|
DatabaseError::Pool(
|
|
"LIBSQL_AUTH_TOKEN required when LIBSQL_URL is set".to_string(),
|
|
)
|
|
})?;
|
|
libsql::LibSqlBackend::new_remote_replica(db_path, url, token.expose_secret())
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
} else {
|
|
libsql::LibSqlBackend::new_local(db_path)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?
|
|
};
|
|
|
|
handles.libsql_db = Some(backend.shared_db());
|
|
|
|
Ok((Arc::new(backend) as Arc<dyn Database>, handles))
|
|
}
|
|
#[cfg(feature = "postgres")]
|
|
crate::config::DatabaseBackend::Postgres => {
|
|
let pg = postgres::PgBackend::new(config)
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(e.to_string()))?;
|
|
|
|
handles.pg_pool = Some(pg.pool());
|
|
|
|
// Validate PostgreSQL prerequisites (version, pgvector)
|
|
validate_postgres(&pg.pool()).await?;
|
|
|
|
Ok((Arc::new(pg) as Arc<dyn Database>, handles))
|
|
}
|
|
#[allow(unreachable_patterns)]
|
|
_ => Err(DatabaseError::Pool(format!(
|
|
"Database backend '{}' is not available. Rebuild with the appropriate feature flag.",
|
|
config.backend
|
|
))),
|
|
}
|
|
}
|
|
|
|
/// Validate PostgreSQL prerequisites (version >= 15, pgvector available).
|
|
///
|
|
/// Returns `Ok(())` if all prerequisites are met, or a `DatabaseError`
|
|
/// with a user-facing message describing the issue.
|
|
#[cfg(feature = "postgres")]
|
|
async fn validate_postgres(pool: &deadpool_postgres::Pool) -> Result<(), DatabaseError> {
|
|
let client = pool
|
|
.get()
|
|
.await
|
|
.map_err(|e| DatabaseError::Pool(format!("Failed to connect: {}", e)))?;
|
|
|
|
// Check PostgreSQL server version (need 15+ for pgvector).
|
|
let version_row = client
|
|
.query_one("SHOW server_version", &[])
|
|
.await
|
|
.map_err(|e| DatabaseError::Query(format!("Failed to query server version: {}", e)))?;
|
|
let version_str: &str = version_row.get(0);
|
|
let major_version = version_str
|
|
.split('.')
|
|
.next()
|
|
.and_then(|v| v.parse::<u32>().ok())
|
|
.ok_or_else(|| {
|
|
DatabaseError::Pool(format!(
|
|
"Could not parse PostgreSQL version from '{}'. \
|
|
Expected a numeric major version (e.g., '15.2').",
|
|
version_str
|
|
))
|
|
})?;
|
|
|
|
const MIN_PG_MAJOR_VERSION: u32 = 15;
|
|
|
|
if major_version < MIN_PG_MAJOR_VERSION {
|
|
return Err(DatabaseError::Pool(format!(
|
|
"PostgreSQL {} detected. IronClaw requires PostgreSQL {} or later \
|
|
for pgvector support.\n\
|
|
Upgrade: https://www.postgresql.org/download/",
|
|
version_str, MIN_PG_MAJOR_VERSION
|
|
)));
|
|
}
|
|
|
|
// Check if pgvector extension is available.
|
|
let pgvector_row = client
|
|
.query_opt(
|
|
"SELECT 1 FROM pg_available_extensions WHERE name = 'vector'",
|
|
&[],
|
|
)
|
|
.await
|
|
.map_err(|e| {
|
|
DatabaseError::Query(format!("Failed to check pgvector availability: {}", e))
|
|
})?;
|
|
|
|
if pgvector_row.is_none() {
|
|
return Err(DatabaseError::Pool(format!(
|
|
"pgvector extension not found on your PostgreSQL server.\n\n\
|
|
Install it:\n \
|
|
macOS: brew install pgvector\n \
|
|
Ubuntu: apt install postgresql-{0}-pgvector\n \
|
|
Docker: use the pgvector/pgvector:pg{0} image\n \
|
|
Source: https://github.com/pgvector/pgvector#installation\n\n\
|
|
Then restart PostgreSQL and re-run: ironclaw onboard",
|
|
major_version
|
|
)));
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
|
|
// ==================== Sub-traits ====================
|
|
//
|
|
// Each sub-trait groups related persistence methods. The `Database` supertrait
|
|
// combines them all, so existing `Arc<dyn Database>` consumers keep working.
|
|
// Leaf consumers can depend on a specific sub-trait instead.
|
|
|
|
#[async_trait]
|
|
pub trait ConversationStore: Send + Sync {
|
|
async fn create_conversation(
|
|
&self,
|
|
channel: &str,
|
|
user_id: &str,
|
|
thread_id: Option<&str>,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn touch_conversation(&self, id: Uuid) -> Result<(), DatabaseError>;
|
|
async fn add_conversation_message(
|
|
&self,
|
|
conversation_id: Uuid,
|
|
role: &str,
|
|
content: &str,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn ensure_conversation(
|
|
&self,
|
|
id: Uuid,
|
|
channel: &str,
|
|
user_id: &str,
|
|
thread_id: Option<&str>,
|
|
) -> Result<bool, DatabaseError>;
|
|
async fn list_conversations_with_preview(
|
|
&self,
|
|
user_id: &str,
|
|
channel: &str,
|
|
limit: i64,
|
|
) -> Result<Vec<ConversationSummary>, DatabaseError>;
|
|
async fn list_conversations_all_channels(
|
|
&self,
|
|
user_id: &str,
|
|
limit: i64,
|
|
) -> Result<Vec<ConversationSummary>, DatabaseError>;
|
|
async fn get_or_create_routine_conversation(
|
|
&self,
|
|
routine_id: Uuid,
|
|
routine_name: &str,
|
|
user_id: &str,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn get_or_create_heartbeat_conversation(
|
|
&self,
|
|
user_id: &str,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn get_or_create_assistant_conversation(
|
|
&self,
|
|
user_id: &str,
|
|
channel: &str,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn create_conversation_with_metadata(
|
|
&self,
|
|
channel: &str,
|
|
user_id: &str,
|
|
metadata: &serde_json::Value,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn list_conversation_messages_paginated(
|
|
&self,
|
|
conversation_id: Uuid,
|
|
before: Option<DateTime<Utc>>,
|
|
limit: i64,
|
|
) -> Result<(Vec<ConversationMessage>, bool), DatabaseError>;
|
|
async fn update_conversation_metadata_field(
|
|
&self,
|
|
id: Uuid,
|
|
key: &str,
|
|
value: &serde_json::Value,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn get_conversation_metadata(
|
|
&self,
|
|
id: Uuid,
|
|
) -> Result<Option<serde_json::Value>, DatabaseError>;
|
|
async fn list_conversation_messages(
|
|
&self,
|
|
conversation_id: Uuid,
|
|
) -> Result<Vec<ConversationMessage>, DatabaseError>;
|
|
async fn conversation_belongs_to_user(
|
|
&self,
|
|
conversation_id: Uuid,
|
|
user_id: &str,
|
|
) -> Result<bool, DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait JobStore: Send + Sync {
|
|
async fn save_job(&self, ctx: &JobContext) -> Result<(), DatabaseError>;
|
|
async fn get_job(&self, id: Uuid) -> Result<Option<JobContext>, DatabaseError>;
|
|
async fn update_job_status(
|
|
&self,
|
|
id: Uuid,
|
|
status: JobState,
|
|
failure_reason: Option<&str>,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn mark_job_stuck(&self, id: Uuid) -> Result<(), DatabaseError>;
|
|
async fn get_stuck_jobs(&self) -> Result<Vec<Uuid>, DatabaseError>;
|
|
async fn list_agent_jobs(&self) -> Result<Vec<AgentJobRecord>, DatabaseError>;
|
|
async fn agent_job_summary(&self) -> Result<AgentJobSummary, DatabaseError>;
|
|
/// Get the failure reason for a single agent job (O(1) lookup).
|
|
async fn get_agent_job_failure_reason(&self, id: Uuid)
|
|
-> Result<Option<String>, DatabaseError>;
|
|
async fn save_action(&self, job_id: Uuid, action: &ActionRecord) -> Result<(), DatabaseError>;
|
|
async fn get_job_actions(&self, job_id: Uuid) -> Result<Vec<ActionRecord>, DatabaseError>;
|
|
async fn record_llm_call(&self, record: &LlmCallRecord<'_>) -> Result<Uuid, DatabaseError>;
|
|
async fn save_estimation_snapshot(
|
|
&self,
|
|
job_id: Uuid,
|
|
category: &str,
|
|
tool_names: &[String],
|
|
estimated_cost: Decimal,
|
|
estimated_time_secs: i32,
|
|
estimated_value: Decimal,
|
|
) -> Result<Uuid, DatabaseError>;
|
|
async fn update_estimation_actuals(
|
|
&self,
|
|
id: Uuid,
|
|
actual_cost: Decimal,
|
|
actual_time_secs: i32,
|
|
actual_value: Option<Decimal>,
|
|
) -> Result<(), DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait SandboxStore: Send + Sync {
|
|
async fn save_sandbox_job(&self, job: &SandboxJobRecord) -> Result<(), DatabaseError>;
|
|
async fn get_sandbox_job(&self, id: Uuid) -> Result<Option<SandboxJobRecord>, DatabaseError>;
|
|
async fn list_sandbox_jobs(&self) -> Result<Vec<SandboxJobRecord>, DatabaseError>;
|
|
async fn update_sandbox_job_status(
|
|
&self,
|
|
id: Uuid,
|
|
status: &str,
|
|
success: Option<bool>,
|
|
message: Option<&str>,
|
|
started_at: Option<DateTime<Utc>>,
|
|
completed_at: Option<DateTime<Utc>>,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn cleanup_stale_sandbox_jobs(&self) -> Result<u64, DatabaseError>;
|
|
async fn sandbox_job_summary(&self) -> Result<SandboxJobSummary, DatabaseError>;
|
|
async fn list_sandbox_jobs_for_user(
|
|
&self,
|
|
user_id: &str,
|
|
) -> Result<Vec<SandboxJobRecord>, DatabaseError>;
|
|
async fn sandbox_job_summary_for_user(
|
|
&self,
|
|
user_id: &str,
|
|
) -> Result<SandboxJobSummary, DatabaseError>;
|
|
async fn sandbox_job_belongs_to_user(
|
|
&self,
|
|
job_id: Uuid,
|
|
user_id: &str,
|
|
) -> Result<bool, DatabaseError>;
|
|
async fn update_sandbox_job_mode(&self, id: Uuid, mode: &str) -> Result<(), DatabaseError>;
|
|
async fn get_sandbox_job_mode(&self, id: Uuid) -> Result<Option<String>, DatabaseError>;
|
|
async fn save_job_event(
|
|
&self,
|
|
job_id: Uuid,
|
|
event_type: &str,
|
|
data: &serde_json::Value,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn list_job_events(
|
|
&self,
|
|
job_id: Uuid,
|
|
limit: Option<i64>,
|
|
) -> Result<Vec<JobEventRecord>, DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait RoutineStore: Send + Sync {
|
|
async fn create_routine(&self, routine: &Routine) -> Result<(), DatabaseError>;
|
|
async fn get_routine(&self, id: Uuid) -> Result<Option<Routine>, DatabaseError>;
|
|
async fn get_routine_by_name(
|
|
&self,
|
|
user_id: &str,
|
|
name: &str,
|
|
) -> Result<Option<Routine>, DatabaseError>;
|
|
async fn list_routines(&self, user_id: &str) -> Result<Vec<Routine>, DatabaseError>;
|
|
async fn list_all_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
|
|
async fn list_event_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
|
|
async fn list_due_cron_routines(&self) -> Result<Vec<Routine>, DatabaseError>;
|
|
async fn update_routine(&self, routine: &Routine) -> Result<(), DatabaseError>;
|
|
async fn update_routine_runtime(
|
|
&self,
|
|
id: Uuid,
|
|
last_run_at: DateTime<Utc>,
|
|
next_fire_at: Option<DateTime<Utc>>,
|
|
run_count: u64,
|
|
consecutive_failures: u32,
|
|
state: &serde_json::Value,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn delete_routine(&self, id: Uuid) -> Result<bool, DatabaseError>;
|
|
async fn create_routine_run(&self, run: &RoutineRun) -> Result<(), DatabaseError>;
|
|
async fn complete_routine_run(
|
|
&self,
|
|
id: Uuid,
|
|
status: RunStatus,
|
|
result_summary: Option<&str>,
|
|
tokens_used: Option<i32>,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn list_routine_runs(
|
|
&self,
|
|
routine_id: Uuid,
|
|
limit: i64,
|
|
) -> Result<Vec<RoutineRun>, DatabaseError>;
|
|
async fn count_running_routine_runs(&self, routine_id: Uuid) -> Result<i64, DatabaseError>;
|
|
async fn count_running_routine_runs_batch(
|
|
&self,
|
|
routine_ids: &[Uuid],
|
|
) -> Result<HashMap<Uuid, i64>, DatabaseError>;
|
|
async fn link_routine_run_to_job(
|
|
&self,
|
|
run_id: Uuid,
|
|
job_id: Uuid,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn get_webhook_routine_by_path(
|
|
&self,
|
|
path: &str,
|
|
) -> Result<Option<Routine>, DatabaseError>;
|
|
|
|
/// List routine runs that were dispatched as full_job but have not yet
|
|
/// been finalized (status='running' with a linked job_id).
|
|
async fn list_dispatched_routine_runs(&self) -> Result<Vec<RoutineRun>, DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait ToolFailureStore: Send + Sync {
|
|
async fn record_tool_failure(
|
|
&self,
|
|
tool_name: &str,
|
|
error_message: &str,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn get_broken_tools(&self, threshold: i32) -> Result<Vec<BrokenTool>, DatabaseError>;
|
|
async fn mark_tool_repaired(&self, tool_name: &str) -> Result<(), DatabaseError>;
|
|
async fn increment_repair_attempts(&self, tool_name: &str) -> Result<(), DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait SettingsStore: Send + Sync {
|
|
async fn get_setting(
|
|
&self,
|
|
user_id: &str,
|
|
key: &str,
|
|
) -> Result<Option<serde_json::Value>, DatabaseError>;
|
|
async fn get_setting_full(
|
|
&self,
|
|
user_id: &str,
|
|
key: &str,
|
|
) -> Result<Option<SettingRow>, DatabaseError>;
|
|
async fn set_setting(
|
|
&self,
|
|
user_id: &str,
|
|
key: &str,
|
|
value: &serde_json::Value,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn delete_setting(&self, user_id: &str, key: &str) -> Result<bool, DatabaseError>;
|
|
async fn list_settings(&self, user_id: &str) -> Result<Vec<SettingRow>, DatabaseError>;
|
|
async fn get_all_settings(
|
|
&self,
|
|
user_id: &str,
|
|
) -> Result<HashMap<String, serde_json::Value>, DatabaseError>;
|
|
async fn set_all_settings(
|
|
&self,
|
|
user_id: &str,
|
|
settings: &HashMap<String, serde_json::Value>,
|
|
) -> Result<(), DatabaseError>;
|
|
async fn has_settings(&self, user_id: &str) -> Result<bool, DatabaseError>;
|
|
}
|
|
|
|
#[async_trait]
|
|
pub trait WorkspaceStore: Send + Sync {
|
|
async fn get_document_by_path(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
path: &str,
|
|
) -> Result<MemoryDocument, WorkspaceError>;
|
|
async fn get_document_by_id(&self, id: Uuid) -> Result<MemoryDocument, WorkspaceError>;
|
|
async fn get_or_create_document_by_path(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
path: &str,
|
|
) -> Result<MemoryDocument, WorkspaceError>;
|
|
async fn update_document(&self, id: Uuid, content: &str) -> Result<(), WorkspaceError>;
|
|
async fn delete_document_by_path(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
path: &str,
|
|
) -> Result<(), WorkspaceError>;
|
|
async fn list_directory(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
directory: &str,
|
|
) -> Result<Vec<WorkspaceEntry>, WorkspaceError>;
|
|
async fn list_all_paths(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
) -> Result<Vec<String>, WorkspaceError>;
|
|
async fn list_documents(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
) -> Result<Vec<MemoryDocument>, WorkspaceError>;
|
|
async fn delete_chunks(&self, document_id: Uuid) -> Result<(), WorkspaceError>;
|
|
async fn insert_chunk(
|
|
&self,
|
|
document_id: Uuid,
|
|
chunk_index: i32,
|
|
content: &str,
|
|
embedding: Option<&[f32]>,
|
|
) -> Result<Uuid, WorkspaceError>;
|
|
async fn update_chunk_embedding(
|
|
&self,
|
|
chunk_id: Uuid,
|
|
embedding: &[f32],
|
|
) -> Result<(), WorkspaceError>;
|
|
async fn get_chunks_without_embeddings(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
limit: usize,
|
|
) -> Result<Vec<MemoryChunk>, WorkspaceError>;
|
|
async fn hybrid_search(
|
|
&self,
|
|
user_id: &str,
|
|
agent_id: Option<Uuid>,
|
|
query: &str,
|
|
embedding: Option<&[f32]>,
|
|
config: &SearchConfig,
|
|
) -> Result<Vec<SearchResult>, WorkspaceError>;
|
|
}
|
|
|
|
/// Backend-agnostic database supertrait.
|
|
///
|
|
/// Combines all sub-traits into one. Existing `Arc<dyn Database>` consumers
|
|
/// continue to work; leaf consumers can depend on a specific sub-trait instead.
|
|
#[async_trait]
|
|
pub trait Database:
|
|
ConversationStore
|
|
+ JobStore
|
|
+ SandboxStore
|
|
+ RoutineStore
|
|
+ ToolFailureStore
|
|
+ SettingsStore
|
|
+ WorkspaceStore
|
|
+ Send
|
|
+ Sync
|
|
{
|
|
/// Run schema migrations for this backend.
|
|
async fn run_migrations(&self) -> Result<(), DatabaseError>;
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// Regression test: `create_secrets_store` selects the correct backend at
|
|
/// runtime based on `DatabaseConfig`, not at compile time. Previously the
|
|
/// CLI duplicated this logic with compile-time `#[cfg]` gates that always
|
|
/// chose postgres when both features were enabled (PR #209).
|
|
#[cfg(feature = "libsql")]
|
|
#[tokio::test]
|
|
async fn test_create_secrets_store_libsql_backend() {
|
|
use secrecy::SecretString;
|
|
|
|
let tmp = tempfile::tempdir().unwrap();
|
|
let db_path = tmp.path().join("test.db");
|
|
|
|
let config = crate::config::DatabaseConfig {
|
|
backend: crate::config::DatabaseBackend::LibSql,
|
|
libsql_path: Some(db_path),
|
|
libsql_url: None,
|
|
libsql_auth_token: None,
|
|
url: SecretString::from("unused://libsql".to_string()),
|
|
pool_size: 1,
|
|
ssl_mode: crate::config::SslMode::default(),
|
|
};
|
|
|
|
let master_key = SecretString::from("a]".repeat(16));
|
|
let crypto = Arc::new(crate::secrets::SecretsCrypto::new(master_key).unwrap());
|
|
|
|
let store = create_secrets_store(&config, crypto).await;
|
|
assert!(
|
|
store.is_ok(),
|
|
"create_secrets_store should succeed for libsql backend"
|
|
);
|
|
|
|
// Verify basic operation works
|
|
let store = store.unwrap();
|
|
let exists = store.exists("test_user", "nonexistent_secret").await;
|
|
assert!(exists.is_ok());
|
|
assert!(!exists.unwrap());
|
|
}
|
|
}
|