Files
optimclaw/src/cli/memory.rs
T
e843c18141 feat: add libSQL/Turso embedded database backend (#47)
* feat: add libSQL/Turso database backend with full feature parity

Introduce a Database trait abstraction (~60 async methods) enabling
compile-time backend selection between PostgreSQL and libSQL/Turso.
Convert all modules from concrete Store to Arc<dyn Database>, add
LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire
libsql stores throughout CLI and main entry points, and make the
setup wizard backend-agnostic.

Key changes:
- src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend
  with native SQLite-dialect SQL, and idempotent migration system
- src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods)
- src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods)
- src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring
- src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore>
- Feature-gate postgres-only tests and examples

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: enable onboarding wizard for libSQL builds

Refactor the setup wizard to work with both postgres and libsql feature
flags. Previously the wizard was gated behind #[cfg(feature = "postgres")]
only, so libsql-only builds would print an error on `ironclaw onboard`.

- Add libsql fields to Settings (database_backend, libsql_path, libsql_url)
- Split wizard database/migration/secrets methods into feature-gated variants
- Add step_database_libsql() with local path and Turso remote replica prompts
- Update setup/mod.rs and main.rs feature gates to any(postgres, libsql)
- Extend check_onboard_needed() to detect libsql database presence

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback for libSQL backend

- P0: Switch libsql_backend to connection-per-operation pattern to fix
  shared Connection concurrency issue across tokio tasks
- P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race
- P0: Document encryption-at-rest limitations and json_patch divergence
- P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated
  empty strings with NULL
- P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent
  RFC 3339 timestamps across all queries
- P2: Use explicit _rowid column in FTS5 triggers and joins for stability
  across VACUUM operations
- P2: Add tracing::warn when embedding provided but vector search disabled
  in hybrid_search
- Extract shared connect_from_config() helper to deduplicate DB connection
  logic across main.rs, cli/config.rs, and cli/mcp.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add missing JobContext fields and resolve fmt/clippy warnings

Add total_tokens_used and max_tokens fields to JobContext in
libsql_backend.rs, apply cargo fmt, and fix clippy warnings.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: review fixes for libSQL backend (shared connections, panics, indexes)

- Replace .expect() with proper error propagation in 3 call sites
- Share Arc<Database> between backend and stores instead of single Connection
- Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore
- Wrap store() INSERT + SELECT-back in a transaction
- Add ~22 missing indexes for parity with PostgreSQL schema
- Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration
- Fix super:: import to use crate:: style
- Gate mask_password_in_url behind #[cfg(feature = "postgres")]
- Rewrite secrets store init with or_else chain for runtime backend selection

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Resolve clippy lints (collapsible_if, too_many_arguments)

Collapse nested if blocks into let_chains to satisfy clippy's
collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments
on libsql_row_to_tool_at since refactoring the positional index
pattern would be a larger change.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Illia Polosukhin <[email protected]>
2026-02-14 02:05:05 +00:00

294 lines
8.0 KiB
Rust

//! Memory/workspace CLI commands.
//!
//! Exposes the workspace system for direct CLI use without starting the agent.
use std::io::Read;
use std::sync::Arc;
use clap::Subcommand;
use crate::workspace::{EmbeddingProvider, SearchConfig, Workspace};
/// Run a memory command using the Database trait (works with any backend).
pub async fn run_memory_command_with_db(
cmd: MemoryCommand,
db: std::sync::Arc<dyn crate::db::Database>,
embeddings: Option<Arc<dyn EmbeddingProvider>>,
) -> anyhow::Result<()> {
let mut workspace = Workspace::new_with_db("default", db);
if let Some(emb) = embeddings {
workspace = workspace.with_embeddings(emb);
}
match cmd {
MemoryCommand::Search { query, limit } => search(&workspace, &query, limit).await,
MemoryCommand::Read { path } => read(&workspace, &path).await,
MemoryCommand::Write {
path,
content,
append,
} => write(&workspace, &path, content, append).await,
MemoryCommand::Tree { path, depth } => tree(&workspace, &path, depth).await,
MemoryCommand::Status => status(&workspace).await,
}
}
#[derive(Subcommand, Debug, Clone)]
pub enum MemoryCommand {
/// Search workspace memory (hybrid full-text + semantic)
Search {
/// Search query
query: String,
/// Maximum number of results
#[arg(short, long, default_value = "5")]
limit: usize,
},
/// Read a file from the workspace
Read {
/// File path (e.g., "MEMORY.md", "daily/2024-01-15.md")
path: String,
},
/// Write content to a workspace file
Write {
/// File path (e.g., "notes/idea.md")
path: String,
/// Content to write (omit to read from stdin)
content: Option<String>,
/// Append instead of overwrite
#[arg(short, long)]
append: bool,
},
/// Show workspace directory tree
Tree {
/// Root path to start from
#[arg(default_value = "")]
path: String,
/// Maximum depth to traverse
#[arg(short, long, default_value = "3")]
depth: usize,
},
/// Show workspace status (document count, index health)
Status,
}
/// Run a memory command (PostgreSQL backend).
#[cfg(feature = "postgres")]
pub async fn run_memory_command(
cmd: MemoryCommand,
pool: deadpool_postgres::Pool,
embeddings: Option<Arc<dyn EmbeddingProvider>>,
) -> anyhow::Result<()> {
let mut workspace = Workspace::new("default", pool);
if let Some(emb) = embeddings {
workspace = workspace.with_embeddings(emb);
}
match cmd {
MemoryCommand::Search { query, limit } => search(&workspace, &query, limit).await,
MemoryCommand::Read { path } => read(&workspace, &path).await,
MemoryCommand::Write {
path,
content,
append,
} => write(&workspace, &path, content, append).await,
MemoryCommand::Tree { path, depth } => tree(&workspace, &path, depth).await,
MemoryCommand::Status => status(&workspace).await,
}
}
async fn search(workspace: &Workspace, query: &str, limit: usize) -> anyhow::Result<()> {
let config = SearchConfig::default().with_limit(limit.min(50));
let results = workspace.search_with_config(query, config).await?;
if results.is_empty() {
println!("No results found for: {}", query);
return Ok(());
}
println!("Found {} result(s) for \"{}\":\n", results.len(), query);
for (i, result) in results.iter().enumerate() {
let score_bar = score_indicator(result.score);
println!("{}. [{}] (score: {:.3})", i + 1, score_bar, result.score);
// Show a content preview (first 200 chars)
let preview = truncate_content(&result.content, 200);
for line in preview.lines() {
println!(" {}", line);
}
println!();
}
Ok(())
}
async fn read(workspace: &Workspace, path: &str) -> anyhow::Result<()> {
match workspace.read(path).await {
Ok(doc) => {
println!("{}", doc.content);
}
Err(crate::error::WorkspaceError::DocumentNotFound { .. }) => {
anyhow::bail!("File not found: {}", path);
}
Err(e) => return Err(e.into()),
}
Ok(())
}
async fn write(
workspace: &Workspace,
path: &str,
content: Option<String>,
append: bool,
) -> anyhow::Result<()> {
let content = match content {
Some(c) => c,
None => {
// Read from stdin
let mut buf = String::new();
std::io::stdin().read_to_string(&mut buf)?;
buf
}
};
if append {
workspace.append(path, &content).await?;
println!("Appended to {}", path);
} else {
workspace.write(path, &content).await?;
println!("Wrote to {}", path);
}
Ok(())
}
async fn tree(workspace: &Workspace, path: &str, max_depth: usize) -> anyhow::Result<()> {
let root = if path.is_empty() { "." } else { path };
println!("{}/", root);
print_tree(workspace, path, "", max_depth, 0).await?;
Ok(())
}
async fn print_tree(
workspace: &Workspace,
path: &str,
prefix: &str,
max_depth: usize,
current_depth: usize,
) -> anyhow::Result<()> {
if current_depth >= max_depth {
return Ok(());
}
let entries = workspace.list(path).await?;
let count = entries.len();
for (i, entry) in entries.iter().enumerate() {
let is_last = i == count - 1;
let connector = if is_last { "└── " } else { "├── " };
let child_prefix = if is_last { " " } else { "│ " };
if entry.is_directory {
println!("{}{}{}/", prefix, connector, entry.name());
Box::pin(print_tree(
workspace,
&entry.path,
&format!("{}{}", prefix, child_prefix),
max_depth,
current_depth + 1,
))
.await?;
} else {
println!("{}{}{}", prefix, connector, entry.name());
}
}
Ok(())
}
async fn status(workspace: &Workspace) -> anyhow::Result<()> {
let all_paths = workspace.list_all().await?;
let file_count = all_paths.len();
// Count directories by collecting unique parent paths
let mut dirs: std::collections::HashSet<String> = std::collections::HashSet::new();
for path in &all_paths {
if let Some(parent) = path.rsplit_once('/') {
dirs.insert(parent.0.to_string());
}
}
println!("Workspace Status");
println!(" User: {}", workspace.user_id());
println!(" Files: {}", file_count);
println!(" Directories: {}", dirs.len());
// Check key files
let key_files = [
"MEMORY.md",
"HEARTBEAT.md",
"IDENTITY.md",
"SOUL.md",
"AGENTS.md",
"USER.md",
];
println!("\n Identity files:");
for path in &key_files {
let exists = workspace.exists(path).await.unwrap_or(false);
let marker = if exists { "+" } else { "-" };
println!(" [{}] {}", marker, path);
}
Ok(())
}
fn truncate_content(s: &str, max_len: usize) -> String {
if s.len() <= max_len {
s.to_string()
} else {
format!("{}...", &s[..max_len])
}
}
fn score_indicator(score: f32) -> &'static str {
if score > 0.8_f32 {
"=====>"
} else if score > 0.5_f32 {
"====>"
} else if score > 0.3_f32 {
"===>"
} else if score > 0.1_f32 {
"==>"
} else {
"=>"
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_score_indicator() {
assert_eq!(score_indicator(0.9_f32), "=====>");
assert_eq!(score_indicator(0.6_f32), "====>");
assert_eq!(score_indicator(0.4_f32), "===>");
assert_eq!(score_indicator(0.2_f32), "==>");
assert_eq!(score_indicator(0.05_f32), "=>");
}
#[test]
fn test_truncate_content() {
assert_eq!(truncate_content("hello", 10), "hello");
assert_eq!(truncate_content("hello world", 5), "hello...");
}
}