Human reviewer (zmanian):
- H1: Accept configurable embedding dimension in LanceDbVectorStore::new()
instead of hardcoding 1536. Dimension is sourced from
EmbeddingProvider::dimension() at init time.
- H2: Skip double-write of embeddings to DB when external vector store
is active (pass None to insert_chunk for embedding column).
- H3: Update PR title from "refactor" to "feat" (net-new feature).
- H4: Document non-atomic update_embedding in struct doc comment.
Bot reviewer (Copilot):
- Cache LanceDB table handle via tokio::sync::OnceCell (avoid
open_table per operation).
- Cache Arc<Schema> in struct (avoid rebuilding per insert).
- Fix error variants: ChunkingFailed → EmbeddingFailed for LanceDB
store/delete operations.
- Propagate store_embedding errors in reindex_document instead of
warn-only (prevents silent data loss).
- Prefetch document metadata map in backfill_embeddings to avoid N+1
queries.
- Add lancedb feature + protoc to CI test matrix so LanceDB tests
actually run on Linux.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Instead of wrapping all ~80 Database trait methods in a 664-line decorator
(lancedb_wrapper.rs), introduce a 4-method VectorStore trait that any vector
backend can implement. Workspace composes FTS from the database with vector
search from the external store via RRF fusion.
- Add src/workspace/vector_store.rs with VectorStore trait
- Rewrite lancedb_store.rs to implement VectorStore (not wrap Database)
- Delete src/db/lancedb_wrapper.rs (664 lines removed)
- Remove get_chunk_by_id from Database trait and all backends
- Workspace gains with_vector_store() builder for optional composition
- Fix LanceDB tests: bypass_vector_index() for brute-force search
- Fix integration tests: use temp file DB (libSQL :memory: is per-connection)
- Merge duplicate mod tests in config.rs
Net: -724 lines. Adding a new vector backend requires 4 methods, not 80.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Introduced optional LanceDB vector store for semantic search, configurable via environment variables.
- Updated `.env.example` and `Cargo.toml` to include LanceDB settings.
- Enhanced `DatabaseConfig` to support vector backend selection and LanceDB path configuration.
- Implemented `VectorBackend` enum to manage vector store options.
- Added functionality to connect to LanceDB in the database connection logic.
- Updated relevant documentation to reflect new features and configuration options.
This change allows users to leverage LanceDB as an alternative to pgvector/libsql for improved search capabilities.
