mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-26 15:40:18 +00:00
b441ebec02bdedf650abbcc89c6321b477247504
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b441ebec02 |
feat: multi-tenant auth with per-user workspace isolation (#1118)
* feat: multi-tenant auth with per-user scoping Multi-user authentication and authorization for IronClaw gateway: - Token-based auth mapping tokens to user IDs via GATEWAY_USER_TOKENS - Per-user SSE broadcast scoping - Per-user rate limiting with poisoned lock recovery - Handler auth and ownership checks for jobs, settings, routines - Extension secrets scoped per-user - Chat handlers use authenticated identity - Reverse proxy deployment documentation - Comprehensive integration tests for auth, SSE, rate limiting, and job isolation * fix: scope memory tools per-user in multi-tenant mode Memory tools (search, write, read, tree) held a single workspace created at startup with GATEWAY_USER_ID. In multi-tenant mode, all users' tool calls searched the default user's scope. Add WorkspaceResolver trait that resolves workspaces per-request using JobContext.user_id. In single-user mode, returns the startup workspace. In multi-tenant mode (GATEWAY_USER_TOKENS configured), creates and caches per-user workspaces on demand. Includes regression tests for workspace resolution and user isolation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: comprehensive multi-tenant isolation audit Address all review findings from @serrrfirat plus 7 additional gaps found via full security audit: Reviewer findings (5): - WorkspacePool now applies search config, memory layers, embedding cache, identity read scopes, and global config scopes (was bare) - jobs_summary_handler uses per-user queries instead of global counters - jobs_prompt_handler restructured to not 404 agent jobs + ownership check - jobs_restart_handler agent branch now verifies user ownership - agent_job_summary_for_user added to Database trait + both backends Audit findings (7): - Delete dead handlers/memory.rs (stale copies with no auth) - Add AuthenticatedUser to logs_events, logs_level_get, logs_level_set - Add AuthenticatedUser to extensions_tools_handler, gateway_status_handler - Add auth + ownership checks to all 6 routines handlers - Add auth to all 4 skills handlers with audit logging on mutations - Scope extension setup SSE broadcast to user (broadcast_for_user) - Fix pre-existing test compilation errors in extensions/manager.rs 17 new multi-tenant isolation tests covering: - WorkspacePool config propagation and scope merging - Jobs handler per-user isolation (summary, restart, prompt, cancel) - Routines handler auth enforcement and cross-user rejection - Auth middleware enforcement on logs, skills, status endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: second-pass multi-tenant audit — scope SSE broadcasts, DB queries, dead handlers Second audit pass applying learned patterns across the codebase: - OAuth callback SSE broadcasts now use broadcast_for_user (lines 773, 912) - jobs_list_handler uses list_agent_jobs_for_user instead of fetching all users' jobs and filtering in Rust - list_agent_jobs_for_user added to Database trait + postgres + libsql - Dead handler files (extensions.rs, static_files.rs) hardened with AuthenticatedUser to prevent auth regression if migrated Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — token hashing, broadcast scoping, error handling Security fixes: - Hash tokens with SHA-256 at construction time so authentication compares fixed-size 32-byte digests, eliminating length-oracle timing leaks - Scope auth SSE broadcasts per-user in chat_auth_token_handler — AuthRequired/AuthCompleted events were leaking across tenants - Propagate DB errors in restart handlers instead of silently swallowing via `if let Ok(Some(...))` pattern Code quality: - Log SSE serialization failures instead of silently producing empty strings via unwrap_or_default() - Remove dead `pub type AuthState = MultiAuthState` alias - Replace `.unwrap()` with `Arc::clone(db)` in app.rs multi-tenant workspace setup (db is guaranteed Some in context, but unwrap violates project convention) - Fix telegram setup test to inject UserIdentity into request extensions (handler now requires AuthenticatedUser) - Add safety comments on test-only expect/unwrap calls for CI - Apply cargo fmt to fix pre-existing formatting Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — unify workspace pool, fix SSE regression, cache job owners - Unify WorkspacePool and PerUserWorkspaceResolver: WorkspacePool now implements WorkspaceResolver, eliminating duplicate per-user workspace construction logic. app.rs uses WorkspacePool directly. - Fix sse_tx: None scheduler regression: change scheduler/worker SSE broadcasting from broadcast::Sender<SseEvent> to Arc<SseManager>, restoring SSE event delivery for scheduled agent jobs. - Cache job owner in orchestrator: add job_owner_cache to OrchestratorState so job_event_handler avoids a DB round-trip on every event after the first per job. - Deduplicate ext_user_id computation in main.rs. - Remove unused _gateway_state variable. - Fix pre-existing test: first_token() returns None in multi-user mode by design; align test assertion. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in app.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: extract memory handlers back into handlers/memory.rs Move memory API handlers out of server.rs into their own module, consistent with how jobs, routines, and skills handlers are organized. The resolve_workspace() helper moves with them since it is only used by memory handlers. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: [email protected] <[email protected]> |
||
|
|
d9358b0fa9 |
feat(workspace): multi-scope workspace reads (#1117)
* feat(workspace): multi-scope workspace reads Adds the ability for a workspace to read from multiple user scopes while keeping writes isolated to the primary scope. Configuration via WORKSPACE_READ_SCOPES env var (comma-separated user IDs). Includes identity file isolation (read_primary), multi-scope search, list, and read operations, WorkspaceConfig refactor, and comprehensive integration tests. * fix: address review feedback for multi-scope workspace reads - fix(memory): deduplicate timezone parsing for daily_log target parse_timezone was called twice when target was "daily_log" without a layer — once in path resolution, again in the fallback. Now computed once and reused. - fix(config): add character validation for WORKSPACE_READ_SCOPES and layer scopes — both enforce [a-zA-Z0-9_-] to prevent path traversal or injection via scope strings used as user_id in SQL queries. - fix(config): use chars().take(32) instead of byte-index slicing for scope length error messages (UTF-8 safety). - fix(error): remove unused WorkspaceError::NotFound variant Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: downgrade search log to debug, add comments on list iteration - Downgrade hybrid_search_multi tracing::info! to debug! — fires on every multi-scope search with the default backend, too noisy for info - Add comments explaining why list/list_all iterate per-scope instead of using _multi trait methods (identity path filtering needs scope attribution that merged results lose) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: [email protected] <[email protected]> Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
212d661e20 |
feat(workspace): layered memory with sensitivity-based privacy redirect (#1112)
* feat(workspace): layered memory with sensitivity-based privacy redirect Introduce MemoryLayer type for named memory layers with sensitivity levels and write permissions. Layers map to synthetic user_id values in workspace tables, enabling shared/private memory isolation. - Add MemoryLayer, LayerSensitivity types with default_for_user() - Add layer-aware write methods (write_to_layer, append_to_layer) - Add PatternPrivacyClassifier to guard shared layer writes - Add optional 'layer' parameter to memory_write tool and HTTP API - Add 'redirected' and 'actual_layer' fields to write response - Add MEMORY_LAYERS env var (JSON) for layer configuration - Workspace user_id now derived from GATEWAY_USER_ID (was hardcoded "default") - 10 integration tests for layered memory operations Addresses prerequisite for Issue #59 (multi-tenancy). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add explicit default to memory_write layer schema Add "default": "private" to the layer parameter's JSON schema so LLM tool consumers can see the default without reading code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: extract resolve_layer_target to deduplicate layer writes Consolidate shared layer-lookup, writable check, and privacy classification logic from write_to_layer and append_to_layer into a single resolve_layer_target helper. Flagged on #349 review — the duplication originates in this PR. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address review feedback on layered memory PR - Fix email regex pipe bug in TLD character class (privacy.rs) - Add append support to web memory_write handler via `append` field - Validate MemoryLayer name/scope: reject empty, check duplicates - Remove hardcoded 'private' default from tool schema; omit layer fields from output when no layer specified - Document scope isolation risk for multi-tenant (Issue #59) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address adversarial review findings - CRITICAL: fix identity file protection bypass via trailing slash (normalize target path before protection checks) - HIGH: check private layer is writable before privacy redirect - HIGH: map LayerNotFound/ReadOnly to proper 4xx HTTP status codes - HIGH: honor `append` field in non-layer HTTP write path - MEDIUM: remove redundant DB fetch in append_to_layer (narrower TOCTOU window) - MEDIUM: remove dead memory_write_handler from handlers/memory.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: opt-in privacy classifier, force override, confidence scoring Address review feedback from @zmanian: - Privacy classifier is now opt-in via with_privacy_classifier() instead of always-on. Default hardcoded patterns (doctor, therapy, email, phone) had unacceptable false positive rates in household contexts. LLM chooses the correct layer via system prompt; regex can't improve on that. - Add ConfigurablePrivacyClassifier for operator-supplied patterns. - PatternPrivacyClassifier defaults narrowed to hard PII only (SSN, credit card, credentials). - Add force param to write_to_layer/append_to_layer to skip classifier. - PrivacyClassifier trait returns SensitivityResult { is_sensitive, confidence } instead of bool, ready for probabilistic classifiers. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: remove redundant heartbeat match arm in memory_write The heartbeat arm was identical to the catch-all — resolved_path already points to paths::HEARTBEAT when target is "heartbeat". Addresses review feedback from gemini-code-assist on #1112. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: return Result from PatternPrivacyClassifier::new() Replace .expect() with proper error propagation per project no-panics policy. Remove Default impl (unused in production). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: move memory_layers from GatewayConfig to WorkspaceConfig Resolve merge conflicts between HEAD (transcription, search, env helpers) and the workspace config branch. GatewayConfig no longer owns memory_layers; WorkspaceConfig::resolve() handles parsing, validation (name length >64, character set, empty scope, duplicates), and fallback defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test: strengthen privacy classifier and layer isolation coverage Add 8 privacy classifier edge case tests (format variants, keywords, longer documents, empty/partial inputs) and 5 layer write isolation integration tests (cross-scope invisibility, overwrite, empty path, sensitive-to-private no-redirect). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: tautological test assertion and add WorkspaceConfig validation tests Replace always-true `is_ok() || is_err()` in write_empty_path_to_layer with actual behavior assertion (write succeeds with normalized empty path). Add 8 unit tests for WorkspaceConfig::resolve() covering valid JSON parsing, invalid JSON, empty/long/invalid-char layer names, empty scopes, duplicates, and default fallback behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: cargo fmt after staging merge Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: [email protected] <[email protected]> |
||
|
|
806d402876 |
feat: chat onboarding and routine advisor (#927)
* feat: port NPA psychographic profiling system into IronClaw
Port the complete psychographic profiling system from NPA into IronClaw,
including enriched profile schema, conversational onboarding, profile
evolution, and three-tier prompt augmentation.
Personal onboarding moved from wizard Step 9 to first assistant
interaction per maintainer feedback — the First Contact system prompt
block now instructs the LLM to conduct a natural onboarding conversation
that builds the psychographic profile via memory_write.
Changes:
- Enrich profile.rs with 5 new structs, 9-dimension analysis framework,
custom deserializers for backward compatibility, and rendering methods
- Add conversational onboarding engine with one-step-removed questioning
technique, personality framework, and confidence-scored profile generation
- Add profile evolution with confidence gating, analysis metadata tracking,
and weekly update routine
- Replace thin interaction style injection with three-tier system gated on
confidence > 0.6 and profile recency
- Replace wizard Step 9 with First Contact system prompt block that drives
conversational onboarding during the user's first interaction
- Add autonomy progression to SOUL.md seed and personality framework to
AGENTS.md seed
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat: replace chat-based onboarding with bootstrap greeting and workspace seeds
Remove the interactive onboarding_chat.rs engine in favor of a simpler
bootstrap flow: fresh workspaces get a proactive LLM greeting that
naturally profiles the user. Identity files are now seeded from
src/workspace/seeds/ instead of being hardcoded. Also removes the
identity-file write protection (seeds are now managed), adds routine
advisor integration, and includes an e2e trace for bootstrap greeting.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat(safety): sanitize identity file writes via Sanitizer to prevent prompt injection
Identity files (SOUL.md, AGENTS.md, USER.md, IDENTITY.md) are injected into
every system prompt. Rather than hard-blocking writes (which broke onboarding),
scan content through the existing Sanitizer and reject writes with High/Critical
severity injection patterns. Medium/Low warnings are logged but allowed.
Also clarifies AGENTS.md identity file roles (USER.md = user info, IDENTITY.md =
agent identity) and adds IDENTITY.md setup as an explicit bootstrap step.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* docs: update profile_onboarding_completed comment to reflect current wiring
The field is now actively used by the agent loop to suppress BOOTSTRAP.md
injection — remove the stale "not yet wired" TODO.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(setup): use env_or_override for NEARAI_API_KEY in model fetch config
When the user authenticates via NEAR AI Cloud API key (option 4),
api_key_login() stores the key via set_runtime_env(). But
build_nearai_model_fetch_config() was using std::env::var() which
doesn't check the runtime overlay — so model listing fell back to
session-token auth and re-triggered the interactive NEAR AI
authentication menu.
Switch to env_or_override() which checks both real env vars and the
runtime overlay.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(agent): correct channel/user_id in bootstrap greeting persist call
persist_assistant_response was called with channel="default",
user_id="system" but the assistant thread was created via
get_or_create_assistant_conversation("default", "gateway") which owns
the conversation as user_id="default", channel="gateway". The mismatch
caused ensure_writable_conversation to reject the write with:
WARN Rejected write for unavailable thread id user=system channel=default
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(web): remove all inline event handlers for CSP compliance
The Content-Security-Policy header (added in
|
||
|
NigeandGitHub
|
c54f739354 |
fix: resolve bug_bash UX/logging issues (#1054 #1055 #1058) (#1072)
* fix(web,db): improve auth UX + reduce naive timestamp log noise * fix(clippy): keep memory test modules at end of file |
||
|
NigeandGitHub
|
7a9cbb3b50 |
fix(routines): run cron checks immediately on ticker startup (#1066)
* fix(routines): run cron check immediately at ticker startup * test/ci: add routine_engine test and fix style lint drift |
||
|
|
e1691a8d42 |
feat: configurable hybrid search fusion strategy (#234)
* feat: configurable hybrid search fusion strategy (#169) Add WeightedScore fusion as an alternative to the default RRF algorithm. Users can now tune search behavior via env vars (SEARCH_FUSION_STRATEGY, SEARCH_FTS_WEIGHT, SEARCH_VECTOR_WEIGHT, SEARCH_RRF_K) or by passing SearchConfig with the new fields. Default behavior (RRF, k=60) is unchanged. - Add FusionStrategy enum (Rrf/WeightedScore) to workspace::search - Add weighted_score_fusion() and fuse_results() dispatcher - Add config/search.rs with WorkspaceSearchConfig from env vars - Wire search defaults through Workspace struct - Update both postgres and libsql backends to use fuse_results() - Add 7 new tests (4 fusion + 3 config) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: swap default search weights to match issue #169 spec (0.7 vector / 0.3 FTS) The issue spec says "0.7/0.3 (vector/keyword) for weighted mode" but our defaults had fts_weight=0.7, vector_weight=0.3 (inverted). Also fixes the misleading docstring on weighted_score_fusion that claimed 1/rank normalizes to [0,1]. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: validate weight inputs and update stale doc comments - Reject NaN, infinite, and negative values for SEARCH_FTS_WEIGHT and SEARCH_VECTOR_WEIGHT with a clear ConfigError - Fix module-level docs that incorrectly claimed WeightedScore "normalizes per-method scores to [0,1]" - Update SearchResult.score doc from "Combined RRF score" to strategy-agnostic "Combined fusion score" Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: validate weight setters against NaN/inf/negative values with_fts_weight() and with_vector_weight() now silently ignore non-finite (NaN, ±inf) and negative values, matching the env var validation already in place for SEARCH_FTS_WEIGHT / SEARCH_VECTOR_WEIGHT. Values > 1.0 remain valid since weights are normalized internally. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use crate-wide ENV_MUTEX in search config tests Replace the module-local `ENV_MUTEX` in `search.rs` with a shared `crate::config::helpers::ENV_MUTEX` to prevent cross-module env races when `cargo test` runs tests in parallel. Addresses copilot review comment. Tracked in #245. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: per-strategy weight defaults to match issue #169 spec RRF mode now defaults to 0.5/0.5 (fts/vector) and WeightedScore defaults to 0.3/0.7, matching the acceptance criteria in #169. Previously both modes used 0.3/0.7 uniformly. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: reject both weights=0 in weighted fusion mode When both SEARCH_FTS_WEIGHT and SEARCH_VECTOR_WEIGHT are 0.0 under WeightedScore strategy, all scores would be 0.0, producing arbitrary ordering. RRF mode is unaffected since it ignores weights entirely. Addresses Copilot review comment. The other comment (rrf_k=0 division by zero) is a false positive — ranks are 1-based, so k=0 just gives inverse-rank scoring with no infinity. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: clarify weight doc comments and error key - SearchConfig field docs: clarify that Default always uses 0.5, per-strategy defaults only apply via WorkspaceSearchConfig::resolve() - WorkspaceSearchConfig field docs: same clarification - Error key for both-weights-zero now references both env vars Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: remove broken intra-doc links to pub(crate) resolve() WorkspaceSearchConfig::resolve is pub(crate), so linking to it from public field docs triggers rustdoc private_intra_doc_links warnings. Switch to plain-text references. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add document_path to weighted_score_fusion results The weighted_score_fusion function was missing the document_path field added in a recent main branch commit, causing a compile error after rebase. Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: trigger CI re-check after rebase * fix: resolve pre-existing staging fmt and clippy issues - Fix import ordering in cli/mod.rs (cargo fmt) - Fix line wrapping in tools/mcp/auth.rs (cargo fmt) - Move path_routing_tests before MemoryTreeTool to fix clippy::items_after_test_module [skip-regression-check] * fix: remove duplicate path_routing_tests module after rebase [skip-regression-check] --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
8ac24e775b |
style: fix formatting in cli/mod.rs and mcp/auth.rs (#1071)
* style: fix formatting in cli/mod.rs and mcp/auth.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(cli): add missing use_tools and max_tool_rounds fields to routines create The routines CLI create command was missing the new Lightweight fields added after the cron->routines rename merged. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(clippy): move path_routing_tests after production code in memory.rs Fixes items_after_test_module lint by moving the test module to the end of the file, after all production structs and impls. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
NigeandGitHub
|
d420abfa6a |
fix(memory): reject absolute filesystem paths with corrective routing (#934)
* ci(staging): use default branch instead of hardcoded main * fix(memory): route absolute paths to filesystem tools |
||
|
|
df3635d6be |
feat(timezone): add timezone-aware session context (#671)
* feat(timezone): add timezone-aware session context (#661) All timestamps were UTC-only, causing daily logs to split at UTC midnight, cron schedules to fire in UTC, and no quiet hours for heartbeat. This adds timezone as a per-session property flowing from the client. Key changes: - New `src/timezone.rs` module with resolution chain, parsing, and detection - `IncomingMessage` carries optional timezone from client - `JobContext.user_timezone` flows timezone to tools - `next_cron_fire()` accepts timezone for schedule evaluation - `Trigger::Cron` stores optional timezone (backward-compatible) - Workspace gains `_tz` variants for daily logs and system prompt - Heartbeat supports quiet hours (`HEARTBEAT_QUIET_START/END`) - Web frontend sends `Intl.DateTimeFormat().resolvedOptions().timeZone` - REPL auto-detects system timezone - `DEFAULT_TIMEZONE` env var / settings for server-wide default Storage stays UTC. Conversion happens at display boundaries. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(timezone): address review feedback on timezone-aware sessions - Validate quiet hours values (0-23) in HeartbeatConfig::resolve() - Fall back to settings values when env vars are unset for quiet hours - Validate IANA timezone strings in routine_create/update with parse_timezone - Add timezone field to routine_create tool schema - Allow standalone timezone update on cron routines without changing schedule - Return path from append_daily_log_tz to avoid TOCTOU race at midnight - Delegate append_daily_log to append_daily_log_tz(entry, UTC) to avoid drift - Preserve timezone through approval flow via PendingApproval.user_timezone - Improve test_today_in_tz to not depend on hardcoded year - Add 3 regression tests for quiet hours config validation Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in routine.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(timezone): address second round of review feedback - Remove .claude/scheduled_tasks.lock from repo and add to .gitignore - Store resolved timezone (not raw message.timezone) in PendingApproval - Carry forward user_timezone through chained approvals in thread_ops - Wire quiet_hours_start/end from config to HeartbeatRunner - Support X-Timezone header as fallback in chat_send_handler [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(timezone): include user's local time in time tool response The time tool's "now" operation now returns local_iso and timezone fields based on ctx.user_timezone, so the LLM can report time in the user's timezone instead of always UTC. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in time.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(timezone): address Copilot review round 3 — validation, deterministic tests, schema fixes - Validate DEFAULT_TIMEZONE and HEARTBEAT_TIMEZONE at config load time - Add timezone field to HeartbeatSettings and config::HeartbeatConfig - Wire heartbeat timezone from config through agent_loop to HeartbeatRunner - Add timezone to routine_update tool schema (was accepted but not advertised) - Error on schedule/timezone update for non-cron routines - Validate timezone in Trigger::from_db (coerce invalid to None with warning) - Validate timezone in approval path (thread_ops.rs) before overwriting - Time tool always includes timezone/local_iso fields (fallback to UTC) - Make quiet hours tests deterministic using current UTC hour - Add regression tests for config validation [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
d562dc8d90 |
fix(workspace): thread document path through search results (#503)
* fix(workspace): thread document path through search results Memory search results were showing chunk UUIDs instead of source file paths. Thread document_path through RankedResult, SearchResult, and the RRF fusion pipeline so handlers can display the actual file path. Fixes #481 Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: use into_iter to move values instead of cloning Address review feedback: consume results with into_iter() to move String fields directly instead of cloning them. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
5f841554d5 |
feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import (#477)
* feat(workspace): add TOOLS.md, BOOTSTRAP.md, and disk-to-DB import Add two new OpenClaw-compatible workspace markdown files: - TOOLS.md: Environment-specific tool notes (SSH hosts, device names, etc.) injected into the system prompt under "## Tool Notes". Seeded as comment-only (like HEARTBEAT.md) so it's effectively empty until the user adds real content. Not write-protected — the agent can update it as it learns the environment. - BOOTSTRAP.md: First-run onboarding ritual. Injected FIRST in the system prompt when present. Guides the agent through introducing itself, learning about the user, and updating workspace files. Only seeded on truly fresh workspaces (no existing identity files) to avoid triggering the ritual on existing deployments. Agent clears it via `memory_write(target="bootstrap")` when done. Add `Workspace::import_from_directory()` for disk-to-DB import: - Scans a directory for *.md files and imports any that don't already exist in the database (never overwrites user edits) - Controlled by WORKSPACE_IMPORT_DIR env var, runs after seed_if_empty() - Enables Docker images / deployment scripts to ship customized workspace templates that override generic seeds - Backwards compatible: no-op when env var is unset Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments - Use stable `path.extension() != Some(OsStr::new("md"))` instead of unstable `is_none_or` (nightly-only) - Use `tokio::join!` for concurrent DB reads in fresh-workspace check - Skip unreadable directory entries instead of failing the entire import - Skip unreadable files instead of failing the entire import Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
0a30c95ee1 |
Feat: add rate limiting for built-in tools (closes #171) (#276)
* feat: add rate limiting for built-in tools (closes #171) Extend the Tool trait with an optional rate_limit_config() method and wire a shared sliding-window RateLimiter into the tool execution path in worker.rs so that per-tool per-user limits are enforced at runtime. - Add ToolRateLimitConfig struct (requests_per_minute / requests_per_hour) and rate_limit_config() default method to the Tool trait - Extract shared RateLimiter from tools/wasm/ into tools/rate_limiter.rs; WASM rate_limiter.rs now re-exports from the shared module - Add RateLimited error variant to crate::error::ToolError - Register RateLimiter on ToolRegistry and check limits in execute_tool_inner - Apply conservative configs to high-impact tools: ShellTool 30 rpm / 300 rph HttpTool 30 rpm / 500 rph WriteFileTool 20 rpm / 200 rph ApplyPatchTool 20 rpm / 200 rph MemoryWriteTool 20 rpm / 200 rph CreateJobTool 5 rpm / 30 rph Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * refactor: address Gemini review comments on rate limiter - worker.rs: collapse nested if-let into a single `if let ... && let ...` (clippy::collapsible_if) - rate_limiter.rs: extract check_internal(record: bool) helper to DRY up check_and_record / check (were identical except for the increment step) - rate_limiter.rs: replace magic numbers 60 / 3600 with MINUTE_SECS / HOUR_SECS constants Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: firat.sertgoz <[email protected]> |
||
|
|
ca8d5c6b5e |
refactor: deduplicate tool code and remove dead stubs (#98)
* refactor: deduplicate tool parameter extraction and remove dead stub tools Delete 4 never-registered stub tools (marketplace, restaurant, ecommerce, taskrabbit) removing ~625 lines of dead code. Add require_str/require_param helpers to tool.rs and refactor ~30 call sites across 10 tool files from 4-6 line inline extractions to single-line calls. Consolidate worker HTTP client with get_json/post_json helpers, reducing boilerplate in 4 methods. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: return JSON from orchestrator /complete endpoint The report_complete handler returned bare StatusCode::OK (no body), which broke the post_json helper that expects a JSON response. Return {"status": "ok"} for consistency with other worker endpoints. Addresses review feedback on PR #98. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
e843c18141 |
feat: add libSQL/Turso embedded database backend (#47)
* feat: add libSQL/Turso database backend with full feature parity Introduce a Database trait abstraction (~60 async methods) enabling compile-time backend selection between PostgreSQL and libSQL/Turso. Convert all modules from concrete Store to Arc<dyn Database>, add LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire libsql stores throughout CLI and main entry points, and make the setup wizard backend-agnostic. Key changes: - src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend with native SQLite-dialect SQL, and idempotent migration system - src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods) - src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods) - src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring - src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore> - Feature-gate postgres-only tests and examples Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: enable onboarding wizard for libSQL builds Refactor the setup wizard to work with both postgres and libsql feature flags. Previously the wizard was gated behind #[cfg(feature = "postgres")] only, so libsql-only builds would print an error on `ironclaw onboard`. - Add libsql fields to Settings (database_backend, libsql_path, libsql_url) - Split wizard database/migration/secrets methods into feature-gated variants - Add step_database_libsql() with local path and Turso remote replica prompts - Update setup/mod.rs and main.rs feature gates to any(postgres, libsql) - Extend check_onboard_needed() to detect libsql database presence Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback for libSQL backend - P0: Switch libsql_backend to connection-per-operation pattern to fix shared Connection concurrency issue across tokio tasks - P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race - P0: Document encryption-at-rest limitations and json_patch divergence - P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated empty strings with NULL - P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent RFC 3339 timestamps across all queries - P2: Use explicit _rowid column in FTS5 triggers and joins for stability across VACUUM operations - P2: Add tracing::warn when embedding provided but vector search disabled in hybrid_search - Extract shared connect_from_config() helper to deduplicate DB connection logic across main.rs, cli/config.rs, and cli/mcp.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing JobContext fields and resolve fmt/clippy warnings Add total_tokens_used and max_tokens fields to JobContext in libsql_backend.rs, apply cargo fmt, and fix clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: review fixes for libSQL backend (shared connections, panics, indexes) - Replace .expect() with proper error propagation in 3 call sites - Share Arc<Database> between backend and stores instead of single Connection - Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore - Wrap store() INSERT + SELECT-back in a transaction - Add ~22 missing indexes for parity with PostgreSQL schema - Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration - Fix super:: import to use crate:: style - Gate mask_password_in_url behind #[cfg(feature = "postgres")] - Rewrite secrets store init with or_else chain for runtime backend selection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (collapsible_if, too_many_arguments) Collapse nested if blocks into let_chains to satisfy clippy's collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments on libsql_row_to_tool_at since refactoring the positional index pattern would be a larger change. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
33ef0a6ea5 |
fix: security hardening across all layers (#35)
* fix: comprehensive security hardening across all layers Critical: - Replace --dangerously-skip-permissions with explicit tool allowlist via settings.json (Claude Code bridge) - Constant-time token comparison (subtle crate) in web auth and orchestrator auth to prevent timing attacks High: - Revoke tokens and clean up handles on container creation failure - Drop SETUID/SETGID capabilities from containers (keep only CHOWN) - Disable redirect following in HTTP tool and WASM wrapper (SSRF) - Reject URL userinfo (@) in WASM allowlist parser (host confusion) - Fix binary body bypassing leak detection (from_utf8 -> from_utf8_lossy) - Protect identity files from LLM overwrites (prompt injection defense) - Prevent tool shadowing: built-in tools cannot be replaced dynamically - User-scoped job APIs: list/detail/cancel/restart/prompt/events/files - CORS restricted to localhost origins, WebSocket origin validation - Sandbox shell fail-closed: no silent fallback to unsandboxed execution - Scrub secrets from log broadcaster before SSE broadcast - XSS sanitization on rendered markdown in web UI - WASM epoch ticker thread so timeout deadlines actually fire Medium: - Cap state transition history at 200 entries - SSE/WebSocket connection limit (100 max) - Request body size limit (1MB) - Response body size limit enforcement in WASM HTTP - UTF-8 safe string truncation (routine engine, shell tool) - Fix PolicyAction::Sanitize to actually run the sanitizer - TOCTOU fix in scheduler and context manager (hold write lock) - Project file serving moved behind auth - Path traversal guard on project_id - Session file permissions set to 0600 on unix - AtomicUsize for routine running_count (panic-safe) - Completion detection hardened against false positives and tool injection - Tool output no longer drives job completion (only LLM response) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address security review findings across all layers - Fix path traversal sandbox bypass via lexical normalization (file.rs) - Fix SSRF via DNS rebinding with pre-request hostname resolution (http.rs) - Add token budget enforcement on LLM calls (reasoning.rs, state.rs) - Fix cross-user chat history leak with ownership verification (store.rs, server.rs) - Add sliding-window rate limiter on gateway chat endpoint (server.rs) - Harden extension install: HTTPS-only, 50MB cap, WASM magic validation (manager.rs) - Add destructive command blocklist that overrides shell auto-approval (shell.rs) - Add 5MB response body size cap to HTTP tool (http.rs) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: deduplicate shared helpers and remove dead code Extract floor_char_boundary and llm_signals_completion into src/util.rs, unifying diverging phrase lists from agent/worker.rs and worker/runtime.rs. Remove dead RespondResult::usage(), duplicate PROTECTED_IDENTITY_FILES constant, double LeakDetector scanning in WebLogLayer, and invalid 0.0.0.0 origin from WebSocket allow list. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review findings and CI test failures - Fix record_failed_approve: .truncate(true) wiped the attempts file before reading, so failed pairing attempts never accumulated and rate limiting never triggered. - Guard wizard WASM test: skip gracefully when channel build artifacts are absent (CI doesn't compile wasm32-wasip2 targets). - Fix DNS rebinding check: use port 0 instead of hardcoded 443, since the port is irrelevant for hostname resolution. - Remove hardcoded CORS port 3001: the dynamic addr.port() entries already cover the actual server port. - Require WebSocket Origin header: reject connections that omit it entirely, since browsers always send Origin for WS upgrades and a missing header indicates a non-browser client bypassing the check. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR review findings - store.rs: reintroduce file locking around read-modify-write in record_failed_approve (concurrent callers could clobber each other). - sse.rs: replace load+check+fetch_add with atomic fetch_update in both subscribe_raw() and subscribe() to prevent overshooting max_connections. - ws.rs: decrement WS tracker before early return when subscribe_raw() returns None (connection limit reached), fixing a counter leak. - server.rs: parse WS Origin host exactly instead of prefix matching, preventing bypass via crafted origins like http://localhost.evil.com. - workspace_integration.rs: skip tests gracefully when Postgres is unreachable instead of panicking (fixes 10 CI failures). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add Origin header to WS integration tests The Origin header requirement added in a3b0190 broke the WS gateway integration tests. Test clients now send Origin: http://127.0.0.1:{port} to match the server's localhost validation. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.6
|
7fcc2279cc |
Route HEARTBEAT writes to workspace DB and broadcast notifications
- Add dedicated "heartbeat" target in memory_write tool so the LLM routes HEARTBEAT.md writes to the database instead of the filesystem - Update tool description to clarify it's database-backed storage - Broadcast heartbeat notifications to all channels when no explicit notify target is configured, instead of silently logging them Co-Authored-By: Claude Opus 4.6 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.5
|
aec42aceda |
Fix Telegram Markdown formatting and clarify tool/memory distinctions
- Add escape_telegram_markdown() to handle underscores in tool names (e.g., build_software was breaking Telegram's Markdown parser) - Use Telegram-compatible *bold* syntax instead of **bold** - Clarify workspace memory vs filesystem tool descriptions to prevent LLM from using read_file on memory_tree paths - Update build_software to strongly prefer Rust WASM for agent tools - Rewrite WASM tool template to use Component Model with wit_bindgen instead of outdated extern "C" approach Co-Authored-By: Claude Opus 4.5 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.5
|
b316a7f4f5 |
Compact memory_tree output format
- Directories indicated by trailing `/` instead of is_directory field
- Files are plain strings, dirs with children are objects
- Remove redundant name/path (just show name in tree)
- Remove preview and updated_at (use memory_read if needed)
- Output is just the tree array, no wrapper object
Example: ["MEMORY.md", {"daily/": ["2024-01-15.md"]}, "projects/"]
Co-Authored-By: Claude Opus 4.5 <[email protected]>
|
||
|
Illia PolosukhinandClaude Opus 4.5
|
3f7624eefc |
Replace memory_list with memory_tree tool
The memory_tree tool provides a hierarchical view of the workspace with configurable depth (default 1). This is more useful for exploring nested directory structures. Co-Authored-By: Claude Opus 4.5 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.5
|
3718cfa767 |
Simplify workspace to path-based storage, remove legacy code
- Consolidate all migrations into V1__initial.sql - Replace DocType enum with flexible path-based file storage - Add list_workspace_files SQL function for directory listing - Update memory tools for path-based API (memory_read, memory_write, memory_search, memory_list) - Remove unused OpenAI/Anthropic providers (NEAR AI only) - Simplify config to remove multi-provider support - Update CLAUDE.md documentation Co-Authored-By: Claude Opus 4.5 <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.5
|
4e238e60ac |
Add workspace and memory system (OpenClaw-inspired)
Implements persistent memory for agents with hybrid search: - Database-backed workspace with PostgreSQL (not filesystem) - Memory documents: MEMORY.md, daily logs, identity files - Chunked content with FTS (tsvector) + vector (pgvector) indexes - Reciprocal Rank Fusion (RRF) for hybrid search combining BM25 and semantic - Memory tools: memory_search, memory_write, memory_read - Proactive heartbeat system for periodic execution (30 min default) - OpenAI embeddings provider (text-embedding-3-small) Key patterns from OpenClaw: - "Memory is files, not RAM" - explicit persistence required - Two-tier memory: daily logs (raw) + curated MEMORY.md - Session isolation via user_id/agent_id scoping Co-Authored-By: Claude Opus 4.5 <[email protected]> |