mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-25 14:53:34 +00:00
3d43917cd0244d58acdb9cb24a5beadac80fddc7
21
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
01678be61d |
fix(routines): normalize status display across web and CLI (#1469)
* fix(routines): normalize status display across web and CLI surfaces (#1319) - Use Display (lowercase) instead of Debug (PascalCase) for RunStatus serialization in web handler - Update JavaScript status class mapping to match lowercase values from the API - Enrich CLI `routines list` to show running/attention states by querying last run status [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(routines): address review -- batch last-run query, consistent status, simplify ternary (#1319) - Parallelize last-run lookups with join_all to avoid N+1 sequential queries - Normalize status in /api/routines/{id}/runs handler to match lowercase convention - Remove redundant 'running' check in app.js runStatusClass logic Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(db): replace N+1 last-run-status queries with batch method The CLI routines list was firing a separate list_routine_runs query per routine to determine each one's last run status. For large routine sets this overwhelms the connection pool. Add batch_get_last_run_status to the Database trait with implementations for both PostgreSQL (DISTINCT ON + ORDER BY) and libSQL (correlated subquery + in-memory filter). Update the CLI to call the batch method once instead of N times. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: cargo fmt https://claude.ai/code/session_01Va9wwvATNWFAx35GG7Zek7 --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
b441ebec02 |
feat: multi-tenant auth with per-user workspace isolation (#1118)
* feat: multi-tenant auth with per-user scoping Multi-user authentication and authorization for IronClaw gateway: - Token-based auth mapping tokens to user IDs via GATEWAY_USER_TOKENS - Per-user SSE broadcast scoping - Per-user rate limiting with poisoned lock recovery - Handler auth and ownership checks for jobs, settings, routines - Extension secrets scoped per-user - Chat handlers use authenticated identity - Reverse proxy deployment documentation - Comprehensive integration tests for auth, SSE, rate limiting, and job isolation * fix: scope memory tools per-user in multi-tenant mode Memory tools (search, write, read, tree) held a single workspace created at startup with GATEWAY_USER_ID. In multi-tenant mode, all users' tool calls searched the default user's scope. Add WorkspaceResolver trait that resolves workspaces per-request using JobContext.user_id. In single-user mode, returns the startup workspace. In multi-tenant mode (GATEWAY_USER_TOKENS configured), creates and caches per-user workspaces on demand. Includes regression tests for workspace resolution and user isolation. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: comprehensive multi-tenant isolation audit Address all review findings from @serrrfirat plus 7 additional gaps found via full security audit: Reviewer findings (5): - WorkspacePool now applies search config, memory layers, embedding cache, identity read scopes, and global config scopes (was bare) - jobs_summary_handler uses per-user queries instead of global counters - jobs_prompt_handler restructured to not 404 agent jobs + ownership check - jobs_restart_handler agent branch now verifies user ownership - agent_job_summary_for_user added to Database trait + both backends Audit findings (7): - Delete dead handlers/memory.rs (stale copies with no auth) - Add AuthenticatedUser to logs_events, logs_level_get, logs_level_set - Add AuthenticatedUser to extensions_tools_handler, gateway_status_handler - Add auth + ownership checks to all 6 routines handlers - Add auth to all 4 skills handlers with audit logging on mutations - Scope extension setup SSE broadcast to user (broadcast_for_user) - Fix pre-existing test compilation errors in extensions/manager.rs 17 new multi-tenant isolation tests covering: - WorkspacePool config propagation and scope merging - Jobs handler per-user isolation (summary, restart, prompt, cancel) - Routines handler auth enforcement and cross-user rejection - Auth middleware enforcement on logs, skills, status endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: second-pass multi-tenant audit — scope SSE broadcasts, DB queries, dead handlers Second audit pass applying learned patterns across the codebase: - OAuth callback SSE broadcasts now use broadcast_for_user (lines 773, 912) - jobs_list_handler uses list_agent_jobs_for_user instead of fetching all users' jobs and filtering in Rust - list_agent_jobs_for_user added to Database trait + postgres + libsql - Dead handler files (extensions.rs, static_files.rs) hardened with AuthenticatedUser to prevent auth regression if migrated Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — token hashing, broadcast scoping, error handling Security fixes: - Hash tokens with SHA-256 at construction time so authentication compares fixed-size 32-byte digests, eliminating length-oracle timing leaks - Scope auth SSE broadcasts per-user in chat_auth_token_handler — AuthRequired/AuthCompleted events were leaking across tenants - Propagate DB errors in restart handlers instead of silently swallowing via `if let Ok(Some(...))` pattern Code quality: - Log SSE serialization failures instead of silently producing empty strings via unwrap_or_default() - Remove dead `pub type AuthState = MultiAuthState` alias - Replace `.unwrap()` with `Arc::clone(db)` in app.rs multi-tenant workspace setup (db is guaranteed Some in context, but unwrap violates project convention) - Fix telegram setup test to inject UserIdentity into request extensions (handler now requires AuthenticatedUser) - Add safety comments on test-only expect/unwrap calls for CI - Apply cargo fmt to fix pre-existing formatting Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review findings — unify workspace pool, fix SSE regression, cache job owners - Unify WorkspacePool and PerUserWorkspaceResolver: WorkspacePool now implements WorkspaceResolver, eliminating duplicate per-user workspace construction logic. app.rs uses WorkspacePool directly. - Fix sse_tx: None scheduler regression: change scheduler/worker SSE broadcasting from broadcast::Sender<SseEvent> to Arc<SseManager>, restoring SSE event delivery for scheduled agent jobs. - Cache job owner in orchestrator: add job_owner_cache to OrchestratorState so job_event_handler avoids a DB round-trip on every event after the first per job. - Deduplicate ext_user_id computation in main.rs. - Remove unused _gateway_state variable. - Fix pre-existing test: first_token() returns None in multi-user mode by design; align test assertion. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in app.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor: extract memory handlers back into handlers/memory.rs Move memory API handlers out of server.rs into their own module, consistent with how jobs, routines, and skills handlers are organized. The resolve_workspace() helper moves with them since it is only used by memory handlers. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: [email protected] <[email protected]> |
||
|
|
d9358b0fa9 |
feat(workspace): multi-scope workspace reads (#1117)
* feat(workspace): multi-scope workspace reads Adds the ability for a workspace to read from multiple user scopes while keeping writes isolated to the primary scope. Configuration via WORKSPACE_READ_SCOPES env var (comma-separated user IDs). Includes identity file isolation (read_primary), multi-scope search, list, and read operations, WorkspaceConfig refactor, and comprehensive integration tests. * fix: address review feedback for multi-scope workspace reads - fix(memory): deduplicate timezone parsing for daily_log target parse_timezone was called twice when target was "daily_log" without a layer — once in path resolution, again in the fallback. Now computed once and reused. - fix(config): add character validation for WORKSPACE_READ_SCOPES and layer scopes — both enforce [a-zA-Z0-9_-] to prevent path traversal or injection via scope strings used as user_id in SQL queries. - fix(config): use chars().take(32) instead of byte-index slicing for scope length error messages (UTF-8 safety). - fix(error): remove unused WorkspaceError::NotFound variant Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: downgrade search log to debug, add comments on list iteration - Downgrade hybrid_search_multi tracing::info! to debug! — fires on every multi-scope search with the default backend, too noisy for info - Add comments explaining why list/list_all iterate per-scope instead of using _multi trait methods (identity path filtering needs scope attribution that merged results lose) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: [email protected] <[email protected]> Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
a09c023642 |
feat(ux): complete UX overhaul — design system, onboarding, web polish (#1277)
* feat(ux): complete UX overhaul — design system, boot screen, onboarding, web polish Shared design system: CSS custom properties for spacing, typography, transitions, and color tokens used across web UI and boot screen. Boot screen: compact feature-tags line showing enabled subsystems (db, tools, routines, heartbeat, skills, sandbox, embeddings) at a glance. Downgrade startup info logs (libSQL, webhook, workspace seed) to debug level since the boot screen now covers this. Onboarding wizard: model picker with live API fetch, provider-aware auth flow, improved error recovery and progress display. Web UI: ARIA attributes, welcome card, streaming debounce, connection status banner, skeleton loaders, send cooldown. CLI: doctor command enhancements, status command cleanup, REPL banner consolidation, shared fmt module. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(ux): Apple-level design refinements — spring physics, glass morphism, chat polish Merge staging theme support (dark/light/system toggle) and layer UX polish on top: spring-physics motion, glass morphism depth, chat experience improvements, and responsive mobile refinements. Design system: - Restore and extend design token system (spacing, typography, timing, easing) with legacy aliases for theme compatibility - Add shadow tiers, accent glow, glass morphism, spring easing tokens - Tokens defined in both dark (:root) and light ([data-theme="light"]) Micro-interactions (Phase 2): - Spring-overshoot message entry animation (slideUp) - Spring-scale button press on all interactive buttons - Tab crossfade animation, tool card smooth accordion (max-height) - Modal scale(0.95) + blur(8px) entry, toast spring slide - Sidebar width crossfade, card hover lift Visual depth (Phase 3): - Tab bar glass morphism + surface highlight + sliding indicator - Active tab accent background pill - Assistant message accent left border, user message bubble tail - Floating input area (rounded + shadow + margin) Chat polish (Phase 4): - Smooth streaming cursor (cursorPulse), message hover timestamps - Time separators (Today/Yesterday/date) - Textarea smooth auto-expand, send button glow Settings & forms (Phase 5): - iOS-style toggle switches for boolean settings - Input focus glow, save feedback spring animation - Welcome card with gradient background + proper spacing - Sticky settings group headers with glass backdrop Accessibility & mobile (Phase 6): - Animated focus ring, prefers-reduced-motion global kill-switch - Touch target audit (44px min), mobile bottom-sheet modals - Mobile bottom tab bar, toast redesign (icon + border + countdown) - Thread hover translateX, badge in_progress pulse Bug fixes: - Gateway/TEE popover z-index (tab-bar z-index: 200, popovers 500) - Connection lost banner as fixed top bar instead of flex child - Sidebar collapse keeps toggle + new thread buttons visible - Downgrade noisy startup logs (db, webhook, vector) to debug - Remove green dot pulse animation on connected status - Deduplicate confirm-modal in HTML, add tab-indicator div Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(web): mobile layout improvements — sidebar toggle, settings drill-down, tab bar polish - Fix mobile sidebar toggle: use expanded-mobile class instead of collapsed, add backdrop overlay, auto-close on thread select, outside-click dismiss - Settings: replace cramped horizontal tabs with drill-down navigation (category list → detail view → back button) - Bottom tab bar: add glass morphism, hide theme toggle, flip tab indicator to top edge - Keep thread toggle button visible in collapsed 36px sidebar strip Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(repl): interactive approval selector and transient status lines - Replace ASCII-art approval box with clean horizontal rule card - Add inquire-based interactive selector for tool approvals (↑↓ + Enter) - Selector runs directly from send_status via spawn_blocking, with stdin_locked flag to prevent readline from competing for stdin - Transient thinking/tool-started lines: each replaces the previous, all erased before final output (no clutter left in scrollback) - Esc in selector sends denial so agent never gets stuck Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: widen TurnCost token fields to u64 and remove unused variable - Change input_tokens/output_tokens from u32 to u64 in StatusUpdate::TurnCost, SseEvent::TurnCost, and the thread_ops emit site to avoid truncation on large conversations - Remove unused _routine_engine_for_loop binding in agent_loop.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore: reduce startup log noise — demote info to debug Demote routine startup messages (builder, WASM tools, tunnel, WASM channels) from info to debug so the default log output stays clean. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): allow CDN scripts in CSP connect-src directive Add cdn.jsdelivr.net and cdnjs.cloudflare.com to connect-src so the browser can fetch marked.js and DOMPurify without CSP violations. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix cargo fmt in repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(web): gate turn_cost SSE handler on current thread Prevents cost badge from attaching to the wrong message when switching threads or receiving events from background threads. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * ci: retrigger CI * fix: add missing extension_manager to webhook EngineContext The webhook trigger path added in #736 was missing the extension_manager field introduced by #1453. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore: ignore RUSTSEC-2026-0049 rustls-webpki CRL advisory Low impact — requires compromised CA to exploit. Tracked for upstream rustls-webpki upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix(routines): use fields.join for cron normalization Use split_whitespace fields instead of re-trimming the original string to avoid preserving extra internal whitespace in cron expressions. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * feat(repl): Apple-style approval card — clean vertical flow - Drop verbose tool description (the command IS the decision surface) - Unified vertical pipe layout: ◆ header → │ params → │ selector - Selector options show keyboard shortcuts inline: Approve (y) - Compact help message, answered state uses └ to close the flow - No horizontal rules, no blank-line padding — just breathing room Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(repl): replace inquire with crossterm for approval selector Drop the inquire dependency (which pulled in crossterm 0.25, duplicating the existing 0.28). The 3-option approval selector is now built directly with crossterm raw mode — same UX, zero new dependencies. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * chore(deps): upgrade crossterm 0.28 → 0.29, eliminate duplication termimad (via crokey) uses crossterm 0.29. Upgrading our direct dependency from 0.28 to 0.29 collapses to a single crossterm version in the dependency tree. Also migrated termimad::crossterm:: references to the direct crossterm import. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review comments — box_top off-by-one, smart_truncate overflow, mobile theme toggle - Fix box_top() fill calculation: was off-by-one, producing boxes 1 char too wide (fmt.rs) - Fix smart_truncate(): account for "..." in the budget so output never exceeds max_chars (repl.rs) - Move theme toggle to settings sidebar on mobile instead of display:none, so mobile users can still switch themes (style.css, index.html, app.js) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: cargo fmt repl.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address review — retry duplication, CSP connect-src, deny color - Remove failed message before retry to prevent duplicate user messages - Revert connect-src to 'self' — CDN hosts only need script-src - Use red for Deny confirmation in REPL approval selector Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
6d847c6009 |
feat(webhooks): add public webhook trigger endpoint for routines (#736)
* feat(webhooks): add public webhook trigger endpoint for routines
Add POST /api/webhooks/{path} endpoint that matches incoming webhooks
against routines with Trigger::Webhook, validates secrets using
constant-time comparison (subtle crate), and fires the matched routine
through the message pipeline.
Closes #651
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(webhooks): address PR review feedback - access control, targeted query, rate limiting
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(ci): add missing webhook_rate_limiter field and fix formatting
Add the webhook_rate_limiter field to the GatewayState initializer in
gateway_workflow_harness.rs and fix rustfmt formatting for the webhook
tuple in types.rs.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(security): require webhook secret, add rate limiting, improve tests
Extract validate_webhook_secret() from the handler so the security-critical
secret validation logic (mandatory secret, constant-time comparison) is
directly testable without mocking the database layer. Improves the error
message for misconfigured routines to guide users toward the fix.
Replaces the previous unit tests (which only tested Rust pattern matching
and status code constants) with tests that exercise the actual validation
function against all rejection paths: missing secret (403), non-webhook
trigger (403), wrong secret (401), empty secret (401), and different-length
secret (401).
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* Route webhook triggers through RoutineEngine instead of chat pipeline
Adds fire_webhook() to RoutineEngine and updates the webhook handler
to use it. This ensures webhook-triggered routines get proper run
tracking, guardrail enforcement (cooldown + max_concurrent),
notifications, and FullJob dispatch support.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: fix formatting in webhook handler
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: [email protected] <[email protected]>
|
||
|
|
455f543ba5 |
fix(routines): surface errors when sandbox unavailable for full_job routines (#769)
* feat(db): add list_dispatched_routine_runs to RoutineStore trait Add method to query routine runs with status='running' AND job_id IS NOT NULL, enabling the routine engine to sync completion status from background jobs. Implements for both PostgreSQL and libSQL backends. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(routines): sync dispatched full-job runs with background job status (#697) Full-job routines were immediately marked Ok on dispatch, so failures/completions were never reflected in the routine run record. Now dispatch returns Running status, and a periodic sync checks linked jobs to update the run when the job completes, fails, or is cancelled. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(routines): fail fast when sandbox unavailable at dispatch time (#697) Thread sandbox_available bool from Docker detection through AgentDeps to RoutineEngine. Full-job routines now fail immediately with a clear error message when sandbox is enabled but Docker is not available, instead of dispatching a job that silently fails. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat(startup): notify user when sandbox unavailable (#697) When sandbox is enabled but Docker is not installed or not running, send a user-visible warning through all channels at startup (with a 2s delay to let channels connect). Previously this was only logged via tracing::warn, invisible to TUI/web users. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix formatting in routine_engine.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(tests): set sandbox_available=true in test rig for full_job traces Test rig doesn't use real Docker — full_job routines execute via trace replay. Setting sandbox_available=true allows the routine_news_digest trace test to dispatch full_job routines as before. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(routines): address review feedback on sync_dispatched_runs (#697) - Sanitize last_reason from job transitions before using in notifications (truncate to 500 chars, strip control characters) - Treat Submitted as in-progress (can still transition to Failed), only Completed and Accepted are terminal success states - Add test for sanitize_summary Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(tests): add missing sandbox_available field to test constructors Staging added sandbox_available to AgentDeps and RoutineEngine::new. Add the missing field/argument in test files to fix CI compilation. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: sanitize job reason in notifications, fix state handling for Submitted/Accepted - Enhance sanitize_summary to strip HTML tags and collapse whitespace, preventing injection via untrusted container job reasons - Use char-boundary-safe truncation to avoid panics on multi-byte strings - Treat Submitted and Accepted as in-progress states (continue polling) rather than terminal success, since they can still transition to Failed - Increase channel-connect delay from 2s to 5s and add debug log for sandbox-unavailable warning delivery Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * Replace sandbox_available bool with SandboxReadiness enum Distinguishes DisabledByConfig from DockerUnavailable so full-job routine errors give actionable guidance instead of a generic message. Co-Authored-By: Claude Opus 4.6 <[email protected]> * ci: re-trigger CI with latest changes Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing owner_id arg to send_notification call Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: update e2e tests to use SandboxReadiness enum Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: [email protected] <[email protected]> |
||
|
|
14abd60917 |
fix: full_job routine runs stay running until linked job completion (#1374)
* fix: full_job routine runs stay running until linked job completion (#1317) Previously, execute_full_job() returned RunStatus::Ok immediately after dispatching the job, causing routine runs to be marked as completed before the linked worker job had actually finished. This meant failure notifications were never sent and max_concurrent guardrails stopped applying once the run was prematurely finalized. Changes: - execute_full_job() now returns RunStatus::Running instead of Ok - execute_routine() skips finalization for Running status (leaves run open) - New sync_dispatched_runs() polls on each cron tick, checks linked job state, and finalizes runs when jobs reach terminal states - New list_dispatched_routine_runs() DB method on both backends - Deferred notifications are sent when the run is actually finalized - consecutive_failures is preserved (not reset) while outcome is unknown Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review feedback (watcher predicate, running_count safety) - FullJobWatcher: use is_parallel_blocking() instead of is_active() so the watcher exits when a job reaches Completed (not terminal but finished executing). Fixes infinite-poll for routine jobs. - Remove running_count decrement from sync_dispatched_runs() — in normal flow execute_routine() handles it; sync only runs for crash recovery where the counter is already 0. - Update PR description to match actual FullJobWatcher behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: sync only at startup to prevent double-completion race - Move sync_dispatched_runs() out of cron loop into startup-only path. During normal operation FullJobWatcher handles finalization inline; running sync on every tick would race with the watcher. - Update complete_dispatched_run() to properly advance runtime fields (last_run_at, next_fire_at, run_count) for crash recovery — in that scenario execute_routine() never reached its runtime update. - Fix stale doc comment on complete_dispatched_run(). Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: use boot_time filter for safe periodic sync of orphaned runs - Add boot_time field to RoutineEngine, set to Utc::now() at creation. - sync_dispatched_runs() now filters runs by started_at < boot_time, so it only processes orphans from a previous process — never races with FullJobWatcher instances from the current process. - Move sync back into the cron loop (safe with boot_time filter) and run it BEFORE check_cron_triggers to avoid picking up freshly dispatched runs. - Fix doc comments to match actual behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
e81fb7e5cb |
refactor(setup): extract init logic from wizard into owning modules (#1210)
* refactor(setup): extract init logic from wizard into owning modules Move database, LLM model discovery, and secrets initialization logic out of the setup wizard and into their owning modules, following the CLAUDE.md principle that module-specific initialization must live in the owning module as a public factory function. Database (src/db/mod.rs, src/config/database.rs): - Add DatabaseConfig::from_postgres_url() and from_libsql_path() - Add connect_without_migrations() for connectivity testing - Add validate_postgres() returning structured PgDiagnostic results LLM (src/llm/models.rs — new file): - Extract 8 model-fetching functions from wizard.rs (~380 lines) - fetch_anthropic_models, fetch_openai_models, fetch_ollama_models, fetch_openai_compatible_models, build_nearai_model_fetch_config, and OpenAI sorting/filtering helpers Secrets (src/secrets/mod.rs): - Add resolve_master_key() unifying env var + keychain resolution - Add crypto_from_hex() convenience wrapper Wizard restructuring (src/setup/wizard.rs): - Replace cfg-gated db_pool/db_backend fields with generic db: Option<Arc<dyn Database>> + db_handles: Option<DatabaseHandles> - Delete 6 backend-specific methods (reconnect_postgres/libsql, test_database_connection_postgres/libsql, run_migrations_postgres/ libsql, create_postgres/libsql_secrets_store) - Simplify persist_settings, try_load_existing_settings, persist_session_to_db, init_secrets_context to backend-agnostic implementations using the new module factories - Eliminate all references to deadpool_postgres, PoolConfig, LibSqlBackend, Store::from_pool, refinery::embed_migrations Net: -878 lines from wizard, +395 lines in owning modules, +378 new. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * test(settings): add wizard re-run regression tests Add 10 tests covering settings preservation during wizard re-runs: - provider_only rerun preserves channels/embeddings/heartbeat - channels_only rerun preserves provider/model/embeddings - quick mode rerun preserves prior channels and heartbeat - full rerun same provider preserves model through merge - full rerun different provider clears model through merge - incremental persist doesn't clobber prior steps - switching DB backend allows fresh connection settings - merge preserves true booleans when overlay has default false - embeddings survive rerun that skips step 5 These cover the scenarios where re-running the wizard would previously risk resetting models, providers, or channel settings. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(setup): eliminate cfg(feature) gates from wizard methods Replace compile-time #[cfg(feature)] dispatch in the wizard with runtime dispatch via DatabaseBackend enum and cfg!() macro constants. - Merge step_database_postgres + step_database_libsql into step_database using runtime backend selection - Rewrite auto_setup_database without feature gates - Remove cfg(feature = "postgres") from mask_password_in_url (pure fn) - Remove cfg(feature = "postgres") from test_mask_password_in_url Only one internal #[cfg(feature = "postgres")] remains: guarding the call to db::validate_postgres() which is itself feature-gated. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * refactor(db): fold PG validation into connect_without_migrations Move PostgreSQL prerequisite validation (version >= 15, pgvector) from the wizard into connect_without_migrations() in the db module. The validation now returns DatabaseError directly with user-facing messages, eliminating the PgDiagnostic enum and the last #[cfg(feature)] gate from the wizard. The wizard's test_database_connection() is now a 5-line method that calls the db module factory and stores the result. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address PR review comments [skip-regression-check] - Use .as_ref().map() to avoid partial move of db_config.libsql_path (gemini-code-assist) - Default to available backend when DATABASE_BACKEND is invalid, not unconditionally to Postgres which may not be compiled (Copilot) - Match DatabaseBackend::Postgres explicitly instead of _ => wildcard in connect_with_handles, connect_without_migrations, and create_secrets_store to avoid silently routing LibSql configs through the Postgres path when libsql feature is disabled (Copilot) - Upgrade Ollama connection failure log from info to warn with the base URL for better visibility in wizard UX (Copilot) - Clarify crypto_from_hex doc: SecretsCrypto validates key length, not hex encoding (Copilot) Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address zmanian's PR review feedback [skip-regression-check] - Update src/setup/README.md to reflect Arc<dyn Database> flow - Remove stale "Test PostgreSQL connection" doc comment - Replace unwrap_or(0) in validate_postgres with descriptive error - Add NearAiConfig::for_model_discovery() constructor - Narrow pub to pub(crate) for internal model helpers Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * fix: address Copilot review comments (quick-mode postgres gate, empty env vars) [skip-regression-check] - Gate DATABASE_URL auto-detection on POSTGRES_AVAILABLE in quick mode so libsql-only builds don't attempt a postgres connection - Match empty-env-var filtering in key source detection to align with resolve_master_key() behavior - Filter empty strings to None in DatabaseConfig::from_libsql_path() for turso_url/turso_token Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
Nick PismenkovandGitHub
|
994a0b194f |
fix: N+1 query pattern in event trigger loop (routine_engine) (#1163)
* fix: N+1 query pattern in event trigger loop (routine_engine) * fix: linter |
||
|
pikaxingeandGitHub
|
2094d6e30d |
fix(agent): block thread_id-based context pollution across users (#760)
* fix(agent): prevent forged thread UUID context/write contamination * fix(agent): close thread_id race and reject forged UUID hydration * fix(ci): satisfy clippy and fmt checks after rebase |
||
|
|
94d101924e |
refactor: encapsulate leaked abstractions into owning modules (#778)
* refactor: encapsulate leaked abstractions from main.rs and app.rs into owning modules Move module-specific initialization logic out of main.rs (1222→665 lines, -46%) and app.rs (944→780 lines, -17%) into their respective owning modules as public factory functions. This enforces separation of concerns so that adding a new DB backend, MCP transport, or channel doesn't require editing main.rs/app.rs. Key changes: - Tracing init functions → src/tracing_fmt.rs - DB connection factory (connect_with_handles + DatabaseHandles) → src/db/mod.rs - Secrets store factory (create_secrets_store) → src/secrets/mod.rs - MCP transport dispatch factory (create_client_from_config) → src/tools/mcp/factory.rs - Orchestrator setup (setup_orchestrator + OrchestratorSetup) → src/orchestrator/mod.rs - WASM channel setup (setup_wasm_channels) → src/channels/wasm/setup.rs - Worker entry points (run_worker, run_claude_bridge) → src/worker/mod.rs - Shared CLI secrets init (init_secrets_store) → src/cli/mod.rs - Tunnel startup (start_managed_tunnel) → src/tunnel/mod.rs - Onboard check (check_onboard_needed) → src/setup/mod.rs - ExtensionManager unified MCP: uses create_client_from_config via McpProcessManager, enabling stdio/Unix transports for hot-activated MCP servers - Deduplicated ~130 lines of secrets store init across cli/mcp.rs and cli/tool.rs - CLAUDE.md updated with module-owned initialization guideline [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: address review feedback — deduplicate db factory, extract channel helper - connect_from_config() now delegates to connect_with_handles() to eliminate duplicated backend-matching logic (Copilot review feedback) - Extract register_channel() helper from setup_wasm_channels() loop body to improve readability (Gemini review feedback) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt line wrapping in setup_wasm_channels Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add integration test for module-owned initialization factories Exercises the full factory chain end-to-end to verify nothing was lost when initialization logic was moved from main.rs/app.rs into owning modules: - connect_with_handles returns Database + populated backend handles - connect_from_config delegates correctly (produces working Database) - secrets::create_secrets_store builds working store from DatabaseHandles - db::create_secrets_store standalone factory round-trips secrets - Both secrets factories produce compatible stores (cross-read works) - ExtensionManager constructs with McpProcessManager and is functional - DatabaseHandles default is empty All tests run without external services using libsql in-memory/tempfile. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: wire cli/mcp.rs and cli/tool.rs to shared init_secrets_store() Both files had inline implementations identical to cli::init_secrets_store(). Replace with delegation to complete the claimed deduplication. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt line wrapping in integration test Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(review): remove unused Config import and deduplicate Error Handling section - Remove `#[allow(unused_imports)]` and unused `use crate::config::Config` from cli/tool.rs (no longer needed after delegating to shared `cli::init_secrets_store()`) - Remove duplicate Error Handling subsection from CLAUDE.md Key Patterns (all four bullets already exist in Code Style section and review-discipline.md) Addresses Copilot review comments. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(review): address remaining Copilot review comments - secrets/mod.rs: clarify docstring that None is a normal no-db condition - app.rs: add comment explaining the empty_handles fallback path - orchestrator/mod.rs: combine duplicated sandbox condition into single block - setup/mod.rs: document env var reads and thread-safety caveat [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Henry Park <[email protected]> |
||
|
|
d8dcc34319 |
fix: CLI commands ignore runtime DATABASE_BACKEND when both features compiled (#740)
* fix: CLI commands ignore runtime DATABASE_BACKEND when both features compiled
`tool auth` and `mcp` CLI subcommands used compile-time `#[cfg]` gates
to select the database backend for secrets storage. When the binary is
compiled with both `postgres` and `libsql` features, the `#[cfg(feature
= "postgres")]` block always wins regardless of the runtime
`DATABASE_BACKEND` setting. This causes `tool auth` and `mcp auth` to
fail with a connection error for users running the libsql backend.
Switch both functions to `match config.database.backend { ... }` with
inner `#[cfg]` guards on each arm, matching the pattern already used in
`main.rs` and `app.rs`.
Also adds a top-level `auth` section to the Telegram channel
capabilities file so `ironclaw tool auth telegram` works for channels
(previously only tools had this section).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: extract create_secrets_store factory into src/db, bump telegram version
- Move duplicated DB backend selection logic from cli/tool.rs and
cli/mcp.rs into a shared db::create_secrets_store() factory, following
the existing db::connect_from_config() pattern.
- Bump telegram channel version 0.2.0 → 0.2.1 to fix CI Version Bump Check.
- Add regression test for create_secrets_store with libsql backend.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix: address review feedback — wizard.rs pattern, formatting, version bump
- Convert setup/wizard.rs secrets store creation from tri-branch #[cfg]
to runtime match on selected_backend (same pattern as CLI fix).
- Fix formatting (assert! line wrapping caught by CI).
- Bump telegram version to 0.2.2 (main already has 0.2.1).
- Merge latest main.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* chore: fix regression test doc comment formatting
Co-Authored-By: Claude Opus 4.6 <[email protected]>
[skip-regression-check]
* fix: address Copilot review — wizard default backend, error chain preservation
- Fix wizard.rs: default selected_backend to "libsql" in libsql-only
builds so create_libsql_secrets_store is not skipped.
- Preserve error chain: replace .map_err(|e| anyhow!("{}", e)) with ?
in cli/tool.rs and cli/mcp.rs since DatabaseError implements
std::error::Error.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: Tiny Tim <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: firat.sertgoz <[email protected]>
|
||
|
|
9f71bd0d44 |
feat: unified thread model for web gateway (#607)
* feat: unified thread model for web gateway Every piece of activity (user chat, routine run, heartbeat alert, external channel message) now lives in its own thread, properly isolated, with meaningful titles and visual distinction. Key changes: - Add `channel` field to ConversationSummary and ThreadInfo so the gateway can distinguish thread origins (gateway, telegram, routine, heartbeat). - Add `list_conversations_all_channels` to Database trait (both postgres and libsql) so chat_threads_handler shows cross-channel threads. - Routine runs get a persistent conversation per routine via `get_or_create_routine_conversation`; notifications carry thread_id. - Heartbeat gets a persistent conversation via `get_or_create_heartbeat_conversation`; HeartbeatRunner accepts an optional Database store and binds notifications to the thread. - Fix broadcast() in web gateway to propagate response.thread_id instead of hardcoding empty string. - Fix isCurrentThread(null) returning true (the core notification leak bug) — now returns false so events without a thread_id don't leak into the active thread. - Rewrite frontend thread sidebar: meaningful titles with channel-specific fallbacks, relative timestamps instead of turn counts, channel badges for non-gateway threads, unread notification dots, read-only indicator for external channel threads. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — TOCTOU races, stale comment, debounce, broadcast warning - Fix TOCTOU race in get_or_create_routine_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_routine unique index + SELECT-back. - Fix TOCTOU race in get_or_create_heartbeat_conversation (postgres): use INSERT ON CONFLICT on new uq_conv_heartbeat unique index + SELECT-back. - Fix TOCTOU race in get_or_create_routine_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Fix TOCTOU race in get_or_create_heartbeat_conversation (libsql): use BEGIN IMMEDIATE transaction to serialize concurrent writers. - Add V11 migration with partial unique indexes for postgres. - Add matching unique indexes to libsql schema. - Update stale comment on isCurrentThread (said "always shown" but logic now returns false for missing thread_id). - Debounce loadThreads() on off-thread SSE events to prevent request storms. - Log warning in broadcast() when thread_id is None (clients will drop it). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: sort in-memory thread fallback by updated_at descending The in-memory thread list fallback (when no DB is available) used HashMap::values() which has no guaranteed ordering. Sort by updated_at descending to match the SQL query ordering. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: retry libsql connect() on transient "unable to open database file" The cron ticker's background task occasionally fails with "unable to open database file" when creating a new SQLite connection concurrently with the main thread. Add retry with exponential backoff (50ms, 100ms, 200ms) to handle transient VFS/locking issues in libsql's local mode. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use ON CONFLICT with index expressions instead of named constraints PostgreSQL ON CONFLICT ON CONSTRAINT requires a named table constraint, but V11 migration creates unique indexes. Switch to the expression form (ON CONFLICT (columns) WHERE condition) which works with unique indexes. Also fix dead code in threadTitle() where thread.title was already checked on the previous line. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt chain collapse in heartbeat.rs [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: skip broadcast when thread_id is None instead of sending empty Clients drop SSE events with empty thread_id anyway, so avoid the unnecessary network traffic by returning early. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * test: add libsql routine/heartbeat conversation idempotency tests Add tests proving get_or_create_routine_conversation returns the same conversation ID across multiple invocations with the same routine_id. Add debug logging to routine engine to track conversation resolution. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: show "New chat" title for empty threads - threadTitle() returns "New chat" when turn_count is 0 - Assistant thread label updates dynamically from API data - Default HTML label changed from "Assistant" to "New chat" - New threads naturally sort to top via last_activity DESC [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: thread sorting, routine isolation, and UI polish - Fix libsql timestamp format mismatch causing broken thread sort order. SQLite defaults used `datetime('now')` (space-separated) while Rust code used RFC3339 (T-separated), breaking string-based ORDER BY. All INSERTs now use RFC3339, and queries use `datetime()` to normalize comparison. - Route manual routine triggers through RoutineEngine.fire_manual() instead of injecting as regular chat messages, so routines always run in their dedicated conversation thread. - Add RoutineEngineSlot to GatewayState for gateway<->engine communication. - Derive routine thread titles from conversation metadata (routine_name) instead of showing truncated UUID hashes. - Make chat_new_thread_handler persist to DB synchronously so loadThreads() sees newly created threads immediately. - Fix enableChatInput() no-op and wrong element ID in disableChatInputReadOnly(). - Fix handlers/chat.rs stale gateway-only query (use list_conversations_all_channels). - Sort in-memory threads by DateTime before converting to RFC3339 strings. - Trigger debouncedLoadThreads() on thinking/status SSE events for non-current threads so routine/heartbeat threads appear in sidebar promptly. - Remove "Threads" text from sidebar header. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: routine history display, orphaned tool_results, duplicate system messages Three independent fixes with regression tests: 1. Routine conversations now display in the web UI. build_turns_from_db_messages() handles standalone assistant messages (no preceding user message) by creating turns with empty user_input. Frontend skips empty user bubbles. 2. Worker select_tools and execute_plan paths now push an assistant_with_tool_calls message before tool execution, preventing sanitize_tool_messages from rewriting tool_results as orphaned user messages. 3. Reasoning::plan() and respond_with_tools() merge system messages from context into a single system prompt instead of creating [system, system, ...] sequences that strict LLM providers (Qwen) reject. Also: sidebar padding/spacing improvements, wider thread panel (240px). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #607 review — RwLock held across await, missing ownership check, heartbeat config - Clone Arc<RoutineEngine> out of RwLock before .await in trigger handler - Add user_id ownership check to fire_manual() with NotAuthorized error - Wire heartbeat notify_user/notify_channel from config to AgentHeartbeatConfig Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: gitignore trace_*.json files and remove stale traces Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: remove trace JSON files from repo Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: proper HTTP status codes for routine errors, read-only input guard, respond thread_id - Map RoutineError::NotFound → 404, NotAuthorized → 403, Disabled → 409 - Guard enableChatInput() against re-enabling on read-only threads - Skip respond() when thread_id is None (matches broadcast() behavior) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
f18fb5173b |
feat(web): fix jobs UI parity for non-sandbox mode (#491)
* feat(web): fix jobs UI parity for non-sandbox mode The web gateway Jobs UI was built primarily for sandbox (Docker) jobs. When running without sandbox (common for NEAR AI hosted envs), multiple features were broken. This change fixes all of them: - Agent jobs now broadcast live SSE events to the web UI (Activity tab) - Agent job restart via scheduler.dispatch_job (not chat message) - Follow-up prompts for agent jobs via WorkerMessage injection - Capability flags (can_restart, can_prompt, job_kind) in job detail API - Rate-limit retry with cap (10 consecutive) and Retry-After header parsing - Plan interruption on user message (breaks out of plan, re-evaluates) - Correct SSE status field in mark_completed/mark_failed/mark_stuck - SseManager preserved across rebuild_state calls Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> * style: fix formatting in db/mod.rs and nearai_chat.rs Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
3362081192 |
fix: add TLS support for PostgreSQL connections (#363) (#427)
All PostgreSQL connection sites hardcoded NoTls, preventing connections to managed providers that require TLS (AWS RDS, Neon, Supabase, etc.). - Add tokio-postgres-rustls with rustls + system root certificates - Add SslMode enum (disable/prefer/require) via DATABASE_SSLMODE env var - Replace NoTls at all 4 production call sites with TLS-aware pool creation - Add SslMode::from_env() helper for lightweight CLI tools - Log native cert loading errors and warn on empty root store Default mode is Prefer (attempts TLS, matching most managed providers). Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
6481448d50 |
feat(web): DB-backed Jobs tab + scheduler-dispatched local jobs (#436)
* feat(web): DB-backed Jobs tab, scheduler-dispatched local jobs, remove active-jobs-bar - Remove active-jobs-bar UI element (HTML, CSS, JS polling) - Move job handlers from server.rs to handlers/jobs.rs - Remove user_id scoping (single-user gateway) - Add list_agent_jobs() and agent_job_summary() to Database trait (both postgres and libsql backends) for non-sandbox job visibility - Wire SchedulerSlot into CreateJobTool so execute_local dispatches via scheduler (persists to DB + spawns worker) instead of creating phantom ContextManager-only jobs - Update /status and /list slash commands to read from DB for consistency with Jobs tab - Fix worker mark_completed: skip if already terminal or stuck - Add agent job cancel via DB update in both web handler and slash cmd - Add Stuck → Completed guard with tracing in worker completion path Co-Authored-By: Claude Sonnet 4.6 <[email protected]> * fix: address PR review comments - Log warning when get_context fails in worker completion path - Extract duplicated status-counting logic into AgentJobSummary::add_count() helper, used by both postgres and libsql backends Co-Authored-By: Claude Sonnet 4.6 <[email protected]> --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]> Co-authored-by: Nick Pismenkov <[email protected]> |
||
|
|
a65b282066 |
fix: web UI routines tab shows all routines regardless of creating channel (#391)
Routines created via Telegram (or any WASM channel) were invisible in the web UI because the routines list endpoint filtered by GATEWAY_USER_ID, which didn't match the Telegram user's ID stored on the routine. Add list_all_routines() to the RoutineStore trait (both libSQL and PostgreSQL backends) and use it in the web dashboard handlers so all routines are visible regardless of which channel created them. Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> |
||
|
|
04d3b005b1 |
feat: implement FullJob routine mode with scheduler dispatch (#288)
* feat: implement FullJob routine mode with scheduler dispatch FullJob routines previously fell back to lightweight mode (single LLM call, no tools) with a warning. This wires them to the existing Scheduler/Worker infrastructure so they dispatch real jobs with full tool access. Fire-and-forget model: the routine creates a job via ContextManager, schedules it, links the routine_run to the job_id, and completes immediately. The job runs independently with full tool access. - Add RoutineError::JobDispatchFailed variant - Add RoutineStore::link_routine_run_to_job (PostgreSQL + libSQL) - Add execute_full_job() in routine_engine with context_manager/scheduler - Wire context_manager + scheduler into RoutineEngine from agent_loop - Fix pre-existing clippy warnings in tests/html_to_markdown.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: persist job to DB before scheduling in execute_full_job The worker emits job_actions and llm_calls rows that reference agent_jobs via foreign key. Without persisting the job first, those inserts can fail. Match the pattern from commands.rs: fetch JobContext, save_job(), then schedule. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: consolidate job dispatch into Scheduler::dispatch_job and wire max_iterations Move the create + persist + schedule sequence into a single Scheduler::dispatch_job() method so callers (commands.rs, routine_engine.rs) don't duplicate the logic. FullJob routines now pass max_iterations via job metadata, and the worker reads it (defaulting to 50 if unset). Also removes the context_manager field from RoutineEngine since dispatch_job handles everything internally. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: clamp max_iterations to 500 and log category update failures Address PR review feedback: - worker.rs: clamp max_iterations from metadata to MAX_WORKER_ITERATIONS (500) to prevent unbounded LLM token usage from malicious/buggy configs - commands.rs: log warning on category update failure instead of silently discarding the error Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: persist worker events to DB and fix activity tab rendering In-process Worker (used by Scheduler::dispatch_job) now persists events via save_job_event at key execution points: plan creation, LLM responses, tool_use, tool_result, and job completion/failure/stuck. Event data shapes match the container worker format so the gateway activity tab renders them correctly. Frontend: tool_result errors now show a red X icon with danger styling instead of a silent empty output. The result event falls back to the error field when message is absent. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
ffb1cc9be8 |
refactor: architecture improvements for contributor velocity (#198)
* refactor: split large files and consolidate test stubs for contributor velocity - Extract 7 Database sub-traits (ConversationStore, JobStore, SandboxStore, RoutineStore, ToolFailureStore, SettingsStore, WorkspaceStore) with Database as a supertrait combining them all - Split libsql_backend.rs (2769 lines) into src/db/libsql/ directory with one file per sub-trait implementation - Split config.rs (1753 lines) into src/config/ directory with 16 domain files - Consolidate 3 duplicate test LLM stubs into shared StubLlm in src/testing.rs - Split server.rs handlers into src/channels/web/handlers/ directory - Extract main.rs init phases into AppBuilder (src/app.rs) - Add developer setup script (scripts/dev-setup.sh) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: move heartbeat test from examples/ to tests/ Convert standalone example binary into a proper #[ignore] integration test, matching the convention of the other integration tests. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt formatting for CI Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review comments from Copilot - tunnel.rs: replace .ok().flatten() with ? to propagate env var errors - secrets.rs: remove misleading "process-wide cache" comment - database.rs: use uppercase "DATABASE_URL" in error key - testing.rs: gate harness tests with #[cfg(feature = "libsql")] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Illia Polosukhin <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
cfb579a4bb |
feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows (#57)
* feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows Adds JobEventsTool and JobPromptTool so the main agent can read container event logs and send follow-up prompts to running Claude Code sessions. A background JobMonitor forwards container assistant messages into the agent loop via a new inject channel on ChannelManager. CreateJobTool now accepts a project_dir parameter for mounting existing cloned repos into containers, and spawns the monitor automatically for async jobs. Also: Dockerfile bumped to Rust 1.88 (rig-core needs let chains), GITHUB_TOKEN forwarded into containers for gh CLI auth, and truncate() fixed for multi-byte char boundary panics. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Add ownership checks to JobEventsTool and JobPromptTool via ContextManager to prevent users from accessing other users' jobs (IDOR) - Combine Dockerfile gh CLI install into single apt-get layer - Handle truncate() edge case when max falls inside first multi-byte char - Log actual count of registered job management tools - Document fire-and-forget job monitor lifecycle - Add tests for ownership rejection and schema validation Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Replace hardcoded GITHUB_TOKEN with on-demand credential delivery Containers now fetch credentials via authenticated GET /worker/{id}/credentials endpoint instead of receiving them baked into env vars at creation time. Secrets are decrypted from SecretsStore on demand, scoped per-job via CredentialGrant, and revoked automatically when the job completes. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address sandbox audit findings (CONNECT tunnel, readonly_rootfs, type consolidation) - Implement real CONNECT tunnel with bidirectional TCP piping via hyper upgrade - Fix readonly_rootfs to apply for both ReadOnly and WorkspaceWrite policies - Consolidate duplicate CredentialMapping/CredentialLocation into secrets::types - Share reqwest::Client across proxy requests instead of per-request allocation - Store Docker connection and reuse across executions - Remove .unwrap() from proxy response builders with safe fallbacks - Add output truncation to direct (non-container) execution (64KB limit) - Delete dead src/tools/sandbox.rs (ToolSandbox never used) - Fix connect_docker error message to list all attempted socket paths - Update proxy credential injection to handle all CredentialLocation variants - Use glob-based host_patterns matching for credential lookup in proxy policy Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Dockerfile: install curl+ca-certificates before fetching GitHub CLI GPG key - JobEventsTool/JobPromptTool: reject missing context (prevents IDOR bypass) - parse_credentials: validate env var names against denylist and pattern - resolve_project_dir: require explicit paths to exist before validation - Credential serving: lower log level from info to debug Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address orchestrator audit findings (constant-time auth, error handling, tests) - auth: constant-time token comparison via subtle::ConstantTimeEq - auth: replace hand-rolled hex_encode with std::fmt::Write fold - api: report_status now updates ContainerHandle (was a no-op) - api: log complete_job errors instead of silently discarding - job_manager: log Docker cleanup errors in stop_job/complete_job - job_manager: extract validate_bind_mount_path with proper error on missing home_dir and mandatory base dir creation before canonicalize - job_manager: cache Docker connection across operations - error: remove dead OrchestratorError::AuthFailed and ContainerTimeout - Add 13 new tests (prompt queue, credentials, events, status, paths) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use floor_char_boundary in sandbox manager truncate to prevent multi-byte panics String::truncate() panics when the index falls mid-way through a multi-byte UTF-8 character. Use the same floor_char_boundary utility already used in worker/runtime.rs and tools/builtin/shell.rs. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: default base_url to private.near.ai for Responses API mode Session tokens only authenticate against private.near.ai, not cloud-api.near.ai. The default base_url now matches the api_mode: - Responses (session token): https://private.near.ai - ChatCompletions (API key): https://cloud-api.near.ai This broke when the multi-provider merge introduced cloud-api.near.ai as the unconditional default. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use private.near.ai as default base URL for all API modes private.near.ai now supports both Responses and ChatCompletions endpoints, so there is no reason to route through cloud-api.near.ai. This also fixes session token auth which only works against private.near.ai. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden libSQL concurrency, fix Claude Code Docker auth and permissions Three fixes for the sandbox/Claude Code pipeline: 1. SQLite "database is locked": set WAL journal mode in migrations and PRAGMA busy_timeout=5000 on every connection across LibSqlBackend, LibSqlSecretsStore, and LibSqlWasmToolStore (~83 async call sites). 2. Claude Code container auth: extract OAuth token from macOS Keychain (or Linux ~/.claude/.credentials.json) at startup and inject via CLAUDE_CODE_OAUTH_TOKEN env var. Removes the broken bind-mount approach that failed on uid mismatch. 3. Claude Code tool permissions: wire CLAUDE_CODE_ALLOWED_TOOLS env var through to the worker binary (was hardcoded to empty vec), and expand defaults to include all standard tools (Read, Write, Edit, Glob, Grep, NotebookEdit, Bash, Task, WebFetch, WebSearch). Also adds --verbose flag to claude CLI (required with stream-json + -p), failover provider model switching, and nearai models endpoint fix. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: stream event parsing, job ID prefix resolution, session renewal in list_models Three fixes for the Docker/gateway pipeline: 1. Claude Code stream event parsing (claude_bridge.rs): Rewrite ClaudeStreamEvent to match actual NDJSON format where content blocks are nested under message.content[], not at the top level. Add handler for "user" events (tool_result blocks) and emit result text as a "message" event so reviews appear in gateway activity view. 2. Job ID prefix resolution (job.rs): Add resolve_job_id() that accepts short hex prefixes (like git short SHAs) in addition to full UUIDs. The LLM sees truncated IDs in job monitor messages like "[Job f2854dd8]" and can now use them directly with job_status/cancel/events/prompt tools. 3. Session renewal in list_models (nearai.rs): list_models() now retries with OAuth renewal on 401, matching send_request()'s existing behavior. Previously it returned SessionExpired immediately, causing the setup wizard to fall back to defaults instead of prompting re-authentication. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: /model command now lists available models Previously /model with no args only showed the current model name. Now it fetches and displays all available models from the provider, marking the active one, so users can see what's available before switching with /model <name>. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #57 review findings (set_var UB, tunnel timeout, restart creds) - Replace unsafe `std::env::set_var` in worker runtime and Claude bridge with `Command::envs()` injection via a new `extra_env` field on `JobContext`, avoiding undefined behavior in the multi-threaded tokio runtime. - Add 30-minute timeout to CONNECT tunnel `copy_bidirectional` in the sandbox proxy to prevent stuck connections from leaking spawned tasks. - Persist credential grants (as JSON in the description column) on `SandboxJobRecord` so `jobs_restart_handler` can restore them instead of passing `vec![]`, which caused restarted containers to lose access to their original secrets. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR #57 review comments - Normalize host_patterns to lowercase in proxy policy matching - Push LIMIT into SQL for list_job_events (Database trait + both backends) - Remove unused was_explicit binding in job tool - Return 500 instead of 200 in make_response fallback path - Update copy_auth_from_mount docstring for env-var default - Use entry.file_type() instead of is_dir() to avoid following symlinks Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address third round of PR #57 review comments - Restore glob patterns in default_claude_code_allowed_tools (Bash -> Bash(*)) - Add tracing::warn for credential grant serialize/deserialize failures - Wrap extra_env in Arc<HashMap> to avoid deep cloning per tool call - Document unsupported credential locations (AuthorizationBasic, UrlPath) - Document TOCTOU window in validate_bind_mount_path - Expand doc comments on JobEventsTool and JobPromptTool Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fourth round of PR #57 review comments - Document CONNECT tunnel task lifecycle (timeout is the cleanup mechanism) - Remove secret names from error-level credential logs to prevent leaking - Expand DANGEROUS_ENV_VARS denylist with language runtime hijack vectors Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fifth round of PR #57 review comments - Promote job monitor startup log to info level for observability - Require minimum 4-char prefix in resolve_job_id to limit enumeration - Cap credential grants at 20 per job to bound column storage - Clamp job events limit to 1..1000 to prevent memory abuse Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing closing brace for SkillsConfig impl block The merge resolution dropped the closing `}` for `impl SkillsConfig`, causing a compilation error in CI. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
e843c18141 |
feat: add libSQL/Turso embedded database backend (#47)
* feat: add libSQL/Turso database backend with full feature parity Introduce a Database trait abstraction (~60 async methods) enabling compile-time backend selection between PostgreSQL and libSQL/Turso. Convert all modules from concrete Store to Arc<dyn Database>, add LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire libsql stores throughout CLI and main entry points, and make the setup wizard backend-agnostic. Key changes: - src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend with native SQLite-dialect SQL, and idempotent migration system - src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods) - src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods) - src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring - src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore> - Feature-gate postgres-only tests and examples Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: enable onboarding wizard for libSQL builds Refactor the setup wizard to work with both postgres and libsql feature flags. Previously the wizard was gated behind #[cfg(feature = "postgres")] only, so libsql-only builds would print an error on `ironclaw onboard`. - Add libsql fields to Settings (database_backend, libsql_path, libsql_url) - Split wizard database/migration/secrets methods into feature-gated variants - Add step_database_libsql() with local path and Turso remote replica prompts - Update setup/mod.rs and main.rs feature gates to any(postgres, libsql) - Extend check_onboard_needed() to detect libsql database presence Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback for libSQL backend - P0: Switch libsql_backend to connection-per-operation pattern to fix shared Connection concurrency issue across tokio tasks - P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race - P0: Document encryption-at-rest limitations and json_patch divergence - P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated empty strings with NULL - P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent RFC 3339 timestamps across all queries - P2: Use explicit _rowid column in FTS5 triggers and joins for stability across VACUUM operations - P2: Add tracing::warn when embedding provided but vector search disabled in hybrid_search - Extract shared connect_from_config() helper to deduplicate DB connection logic across main.rs, cli/config.rs, and cli/mcp.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing JobContext fields and resolve fmt/clippy warnings Add total_tokens_used and max_tokens fields to JobContext in libsql_backend.rs, apply cargo fmt, and fix clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: review fixes for libSQL backend (shared connections, panics, indexes) - Replace .expect() with proper error propagation in 3 call sites - Share Arc<Database> between backend and stores instead of single Connection - Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore - Wrap store() INSERT + SELECT-back in a transaction - Add ~22 missing indexes for parity with PostgreSQL schema - Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration - Fix super:: import to use crate:: style - Gate mask_password_in_url behind #[cfg(feature = "postgres")] - Rewrite secrets store init with or_else chain for runtime backend selection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (collapsible_if, too_many_arguments) Collapse nested if blocks into let_chains to satisfy clippy's collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments on libsql_row_to_tool_at since refactoring the positional index pattern would be a larger change. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |