mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
* feat: LRU embedding cache for workspace search (#165) Add CachedEmbeddingProvider that wraps any EmbeddingProvider with an in-memory LRU cache keyed by SHA-256(model_name + text). This avoids redundant HTTP calls when the same text is embedded multiple times (common during reindexing and repeated searches). - Cache uses HashMap + last_accessed tracking with manual LRU eviction (same pattern as llm::response_cache::CachedProvider) - Lock is never held during HTTP calls to prevent blocking - embed_batch() partitions into hits/misses and only fetches misses - Default 10,000 entries (~58 MB for 1536-dim vectors) - Configurable via EMBEDDING_CACHE_SIZE env var - Workspace.with_embeddings() auto-wraps; with_embeddings_uncached() available for tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review comments on embedding cache - Validate embed_batch return count matches expected miss count - Replace unwrap_or_default() with proper error propagation - Fix batch eviction: run final eviction pass after insert to enforce cap - Fix test: use different-length inputs to verify ordering correctness - Reject EMBEDDING_CACHE_SIZE=0 in config validation (minimum is 1) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: replace .expect() with proper error handling in embed_batch The all-cache-hits early-return path used .expect("all cache hits") which violates the project convention of no .unwrap()/.expect() in production code. Replaced with the same ok_or_else pattern used in the normal path. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: clarify memory sizing docs and use saturating_add for eviction - Update memory comments in embedding_cache.rs, config/embeddings.rs, and workspace/mod.rs to note the ~58 MB figure is payload-only (actual memory is higher due to HashMap/key/allocation overhead) - Use saturating_add(1) instead of + 1 for eviction threshold to prevent overflow if max_entries is usize::MAX Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address Copilot review on embedding cache - Avoid double-clone per miss in embed_batch: move embedding into results, clone only for the cache entry - Evict per-insert instead of after all inserts to keep peak memory bounded during large batches - Clamp max_entries to at least 1 in constructor to prevent unexpected eviction behavior when set to 0 via the public API Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: reduce embedding_cache module visibility to private Types are already re-exported via `pub use`, so the module itself doesn't need to be public. Reduces unnecessary API surface. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address serrrfirat review feedback on embedding cache - Add TODO comment for O(n) LRU eviction scalability - Add thundering herd note at lock release in embed() - Warn when cache max_entries exceeds 100k - Use with_embeddings_uncached() in integration test - Add tests: error_does_not_pollute_cache, embed_batch_empty_input - Update README with cache-aware with_embeddings() docs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: prevent u32 wrapping in FailThenSucceedMock failure counter fetch_sub(1) wraps to u32::MAX when called past zero, silently breaking the mock for 3+ calls. Switch to load-then-store to avoid the wrapping bug in both embed() and embed_batch(). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address Copilot and serrrfirat review findings on embedding cache - Switch tokio::sync::Mutex to std::sync::Mutex (lock never held across .await — cheaper synchronous lock) - Extract DEFAULT_EMBEDDING_CACHE_SIZE constant to avoid 10_000 duplication between EmbeddingCacheConfig and EmbeddingsConfig Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add all-misses batch test for embedding cache Adds embed_batch_all_misses test covering the case where every text in a batch is a cache miss — fulfilling the commitment from serrrfirat's review. Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: trigger CI re-check after rebase * fix: use raw [u8;32] cache keys and pre-allocate HashMap capacity Address Copilot review findings: - cache_key() now returns [u8; 32] instead of hex String, avoiding a 64-byte allocation per lookup - HashMap::with_capacity(max_entries) avoids incremental reallocation - Fix pre-existing staging compilation error in cli/routines.rs (missing max_tool_rounds/use_tools fields) [skip-regression-check] * fix: make cache accessors sync and update doc for [u8;32] keys Address Copilot review: - len(), is_empty(), clear() are now sync since they only take a std::sync::Mutex lock with no .await points - Update cache_size doc comment to reflect [u8;32] keys instead of String keys [skip-regression-check] * fix: remove clone_on_copy for [u8; 32] cache keys [skip-regression-check] * ci: add safety comments to test code for no-panics check The CI no-panics grep check cannot distinguish test code inside src/ files from production code. Add // safety: test annotations to .unwrap(), .expect(), and assert!() calls in #[cfg(test)] modules. * fix: correct cache doc and demote hit/miss logs to trace - Fix misleading "String keys" in memory comment (cache uses [u8; 32]) - Demote per-request hit/miss logs from debug to trace to reduce noise on hot paths (batch summary stays at trace too) * docs: add missing Arc import in workspace README example * perf: batch eviction in embed_batch to avoid O(n×m) cost Replace per-insert evict_lru call with a single evict_k_oldest pass that computes eviction count upfront and removes the k oldest entries in one O(n) scan. Avoids O(n×m) HashMap iterations while holding the mutex during batch inserts. * fix: cap batch cache inserts at max_entries and use O(n) selection - evict_k_oldest now uses select_nth_unstable_by_key for O(n) average partial selection instead of O(n log n) full sort - embed_batch caps cached entries at max_entries when misses exceed capacity, preventing the cache from growing unbounded - Added test: batch_exceeding_capacity_respects_max_entries * fix: flatten test assert for fmt compatibility Shorten assert message to fit single line so cargo fmt doesn't split the safety annotation onto a separate line. * fix: address review feedback and improve embedding cache (takeover #235) - Fix merge conflict: add missing allow_always field in PendingApproval - Thread EmbeddingCacheConfig through CLI memory commands so they respect EMBEDDING_CACHE_SIZE instead of silently using default (fixes #235 review) - Cap HashMap pre-allocation at min(max_entries, 1024) to avoid upfront memory waste at large cache sizes - Fix FailThenSucceedMock race: replace load+store with atomic fetch_update - Remove noisy '// safety: test' comments (40+ lines of diff noise) - Fix collapsed lines from comment removal - Simplify redundant Ok(...collect()?) to just collect() Co-Authored-By: ztsalexey <[email protected]> Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(embedding-cache): skip eviction on concurrent duplicate insert When the lock is released for the HTTP call, another caller may insert the same key. Re-check under lock and just update the existing entry without evicting, avoiding unnecessary cache churn under concurrency. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: ztsalexey <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: ztsalexey <[email protected]>
410 lines
11 KiB
Rust
410 lines
11 KiB
Rust
#![cfg(feature = "postgres")]
|
|
//! Integration tests for the workspace module.
|
|
//!
|
|
//! Requires a running PostgreSQL with pgvector extension.
|
|
//! Set DATABASE_URL=postgres://localhost/ironclaw_test
|
|
|
|
use std::sync::Arc;
|
|
|
|
use ironclaw::workspace::{MockEmbeddings, SearchConfig, Workspace, paths};
|
|
|
|
fn get_pool() -> deadpool_postgres::Pool {
|
|
let database_url = std::env::var("DATABASE_URL")
|
|
.unwrap_or_else(|_| "postgres://localhost/ironclaw_test".to_string());
|
|
|
|
let config: tokio_postgres::Config = database_url.parse().expect("Invalid DATABASE_URL");
|
|
|
|
let mgr = deadpool_postgres::Manager::new(config, tokio_postgres::NoTls);
|
|
deadpool_postgres::Pool::builder(mgr)
|
|
.max_size(4)
|
|
.build()
|
|
.expect("Failed to create pool")
|
|
}
|
|
|
|
/// Try to get a connection, returning None if Postgres is unreachable.
|
|
/// Tests call this to skip gracefully in CI where no database is available.
|
|
async fn try_connect(pool: &deadpool_postgres::Pool) -> Option<()> {
|
|
match pool.get().await {
|
|
Ok(_) => Some(()),
|
|
Err(e) => {
|
|
eprintln!("skipping: database unavailable ({e})");
|
|
None
|
|
}
|
|
}
|
|
}
|
|
|
|
async fn cleanup_user(pool: &deadpool_postgres::Pool, user_id: &str) {
|
|
let conn = pool.get().await.expect("Failed to get connection");
|
|
conn.execute(
|
|
"DELETE FROM memory_documents WHERE user_id = $1",
|
|
&[&user_id],
|
|
)
|
|
.await
|
|
.ok();
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_write_and_read() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_write_read";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write a file
|
|
let doc = workspace
|
|
.write("README.md", "# Hello World\n\nThis is a test.")
|
|
.await
|
|
.expect("Failed to write");
|
|
|
|
assert_eq!(doc.path, "README.md");
|
|
assert!(doc.content.contains("Hello World"));
|
|
|
|
// Read it back
|
|
let doc2 = workspace.read("README.md").await.expect("Failed to read");
|
|
assert_eq!(doc2.content, "# Hello World\n\nThis is a test.");
|
|
|
|
// Cleanup
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_append() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_append";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write initial content
|
|
workspace
|
|
.write("notes.md", "Line 1")
|
|
.await
|
|
.expect("Failed to write");
|
|
|
|
// Append more
|
|
workspace
|
|
.append("notes.md", "Line 2")
|
|
.await
|
|
.expect("Failed to append");
|
|
|
|
// Read and verify
|
|
let doc = workspace.read("notes.md").await.expect("Failed to read");
|
|
assert_eq!(doc.content, "Line 1\nLine 2");
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_nested_paths() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_nested";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write nested files
|
|
workspace
|
|
.write("projects/alpha/README.md", "# Alpha")
|
|
.await
|
|
.expect("Failed to write alpha");
|
|
workspace
|
|
.write("projects/alpha/notes.md", "Notes here")
|
|
.await
|
|
.expect("Failed to write notes");
|
|
workspace
|
|
.write("projects/beta/README.md", "# Beta")
|
|
.await
|
|
.expect("Failed to write beta");
|
|
|
|
// List root
|
|
let root = workspace.list("").await.expect("Failed to list root");
|
|
assert_eq!(root.len(), 1); // just "projects/"
|
|
assert!(root[0].is_directory);
|
|
assert_eq!(root[0].name(), "projects");
|
|
|
|
// List projects
|
|
let projects = workspace
|
|
.list("projects")
|
|
.await
|
|
.expect("Failed to list projects");
|
|
assert_eq!(projects.len(), 2); // alpha/, beta/
|
|
|
|
// List alpha
|
|
let alpha = workspace
|
|
.list("projects/alpha")
|
|
.await
|
|
.expect("Failed to list alpha");
|
|
assert_eq!(alpha.len(), 2); // README.md, notes.md
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_delete() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_delete";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write and verify exists
|
|
workspace
|
|
.write("temp.md", "temporary")
|
|
.await
|
|
.expect("Failed to write");
|
|
assert!(workspace.exists("temp.md").await.expect("exists failed"));
|
|
|
|
// Delete
|
|
workspace.delete("temp.md").await.expect("Failed to delete");
|
|
|
|
// Verify gone
|
|
assert!(!workspace.exists("temp.md").await.expect("exists failed"));
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_memory_operations() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_memory_ops";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Append to memory
|
|
workspace
|
|
.append_memory("User prefers dark mode")
|
|
.await
|
|
.expect("Failed to append memory");
|
|
workspace
|
|
.append_memory("User's timezone is PST")
|
|
.await
|
|
.expect("Failed to append memory");
|
|
|
|
// Read memory
|
|
let memory = workspace.memory().await.expect("Failed to get memory");
|
|
assert!(memory.content.contains("dark mode"));
|
|
assert!(memory.content.contains("PST"));
|
|
// Entries should be separated by double newline
|
|
assert!(memory.content.contains("\n\n"));
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_daily_log() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_daily_log";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Append to daily log (timestamped)
|
|
workspace
|
|
.append_daily_log("Started working on feature X")
|
|
.await
|
|
.expect("Failed to append daily log");
|
|
|
|
// Read today's log
|
|
let log = workspace
|
|
.today_log()
|
|
.await
|
|
.expect("Failed to get today log");
|
|
assert!(log.content.contains("feature X"));
|
|
// Should have timestamp prefix like [HH:MM:SS]
|
|
assert!(log.content.contains("["));
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_fts_search() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_fts_search";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write some documents
|
|
workspace
|
|
.write(
|
|
"docs/authentication.md",
|
|
"# Authentication\n\nThe system uses JWT tokens for authentication.",
|
|
)
|
|
.await
|
|
.expect("write failed");
|
|
workspace
|
|
.write(
|
|
"docs/database.md",
|
|
"# Database\n\nWe use PostgreSQL with pgvector for vector search.",
|
|
)
|
|
.await
|
|
.expect("write failed");
|
|
workspace
|
|
.write(
|
|
"docs/api.md",
|
|
"# API\n\nThe REST API uses JSON for request and response bodies.",
|
|
)
|
|
.await
|
|
.expect("write failed");
|
|
|
|
// Search for JWT (FTS only since no embeddings)
|
|
let results = workspace
|
|
.search_with_config("JWT authentication", SearchConfig::default().fts_only())
|
|
.await
|
|
.expect("search failed");
|
|
|
|
assert!(!results.is_empty(), "Should find results for JWT");
|
|
assert!(
|
|
results[0].content.contains("JWT"),
|
|
"Top result should contain JWT"
|
|
);
|
|
|
|
// Search for PostgreSQL
|
|
let results = workspace
|
|
.search_with_config("PostgreSQL database", SearchConfig::default().fts_only())
|
|
.await
|
|
.expect("search failed");
|
|
|
|
assert!(!results.is_empty(), "Should find results for PostgreSQL");
|
|
assert!(
|
|
results[0].content.contains("PostgreSQL"),
|
|
"Top result should contain PostgreSQL"
|
|
);
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_hybrid_search_with_mock_embeddings() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_hybrid_search";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
// Create workspace with mock embeddings (1536 dimensions to match OpenAI)
|
|
let embeddings = Arc::new(MockEmbeddings::new(1536));
|
|
let workspace = Workspace::new(user_id, pool.clone()).with_embeddings_uncached(embeddings);
|
|
|
|
// Write documents
|
|
workspace
|
|
.write(
|
|
"memory.md",
|
|
"The user prefers dark mode and vim keybindings.",
|
|
)
|
|
.await
|
|
.expect("write failed");
|
|
workspace
|
|
.write(
|
|
"prefs.md",
|
|
"Settings: theme=dark, editor=vim, font=monospace",
|
|
)
|
|
.await
|
|
.expect("write failed");
|
|
|
|
// Hybrid search
|
|
let results = workspace
|
|
.search("dark theme preference", 5)
|
|
.await
|
|
.expect("search failed");
|
|
|
|
assert!(!results.is_empty(), "Should find results");
|
|
// At least one result should be a hybrid match (found by both FTS and vector)
|
|
// or we should have results from either method
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_list_all() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_list_all";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write files at various depths
|
|
workspace.write("README.md", "root").await.unwrap();
|
|
workspace.write("docs/intro.md", "intro").await.unwrap();
|
|
workspace.write("docs/api/rest.md", "rest").await.unwrap();
|
|
workspace.write("src/main.md", "main").await.unwrap();
|
|
|
|
// List all
|
|
let all = workspace.list_all().await.expect("list_all failed");
|
|
assert_eq!(all.len(), 4);
|
|
assert!(all.contains(&"README.md".to_string()));
|
|
assert!(all.contains(&"docs/intro.md".to_string()));
|
|
assert!(all.contains(&"docs/api/rest.md".to_string()));
|
|
assert!(all.contains(&"src/main.md".to_string()));
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_workspace_system_prompt() {
|
|
let pool = get_pool();
|
|
if try_connect(&pool).await.is_none() {
|
|
return;
|
|
}
|
|
let user_id = "test_system_prompt";
|
|
cleanup_user(&pool, user_id).await;
|
|
|
|
let workspace = Workspace::new(user_id, pool.clone());
|
|
|
|
// Write identity files
|
|
workspace
|
|
.write(paths::AGENTS, "You are a helpful assistant.")
|
|
.await
|
|
.unwrap();
|
|
workspace
|
|
.write(paths::SOUL, "Be kind and thorough.")
|
|
.await
|
|
.unwrap();
|
|
workspace.write(paths::USER, "Name: Alice").await.unwrap();
|
|
|
|
// Get system prompt
|
|
let prompt = workspace
|
|
.system_prompt()
|
|
.await
|
|
.expect("system_prompt failed");
|
|
|
|
assert!(
|
|
prompt.contains("helpful assistant"),
|
|
"Should include AGENTS.md"
|
|
);
|
|
assert!(
|
|
prompt.contains("kind and thorough"),
|
|
"Should include SOUL.md"
|
|
);
|
|
assert!(prompt.contains("Alice"), "Should include USER.md");
|
|
|
|
cleanup_user(&pool, user_id).await;
|
|
}
|