mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-08-27 08:00:17 +00:00
b68d67bd355618c5be467b06caf3232cf2460410
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
b68d67bd35 |
feat: show token usage and cost tracker in gateway status popover (#284)
* feat: show token usage, cost tracker, and uptime in gateway status popover The "Connected" hover popover in the web gateway now displays three sections: connection info (SSE/WS counts, uptime), daily cost tracker (spend + actions/hr), and per-model token usage (input/output counts with cost per model). Also fixes the field name mismatch between the backend response and JS rendering that prevented the popover from showing correct data. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review — escape HTML in popover, add model_usage test - Escape model name and cost strings with escapeHtml() before inserting into innerHTML to prevent XSS via crafted model names - Add test_model_usage_per_model_tracking test covering multi-model token/cost accumulation in CostGuard Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
250551799b |
style: adopt agent-market design language for web UI (#282)
* fix: move Logs to status bar and fix chat history ordering after restart
Move the Logs tab out of the main tab bar and into the right-side status
area as a compact pill button next to "Connected". Remove it from the
Ctrl+1-N shortcut order (now Ctrl+1-5).
Fix chat message ordering in libSQL backend: datetime('now') has only
second precision, so back-to-back user+assistant inserts got identical
timestamps causing non-deterministic ORDER BY. Now passes explicit
millisecond-precision timestamps and uses rowid as tiebreaker for
existing data.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* refactor: separate WASM extensions from MCP servers on Extensions page
Reorganize the Extensions tab into 5 distinct sections: Installed
Extensions, Available WASM Extensions, Install WASM Extension (by
tar.gz URL), MCP Servers (with Add Custom form), and Registered Tools.
Registry entries are now filtered client-side by kind so WASM tools/
channels and MCP servers each have dedicated UI sections.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: adopt agent-market design language for web UI
Refresh the web gateway visual identity with a cleaner, modern aesthetic:
deeper blacks, green accent palette, DM Sans + IBM Plex Mono typography,
larger border-radii, glassmorphic navigation, refined hover effects,
pill badges, and green focus rings. CSS-only change plus Google Fonts.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* Update src/channels/web/static/style.css
Co-authored-by: Copilot <[email protected]>
* Update src/channels/web/static/style.css
Co-authored-by: Copilot <[email protected]>
---------
Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Copilot <[email protected]>
|
||
|
|
3b6105d5ea |
feat: add TEE attestation shield to web gateway UI (#275)
Show a shield indicator in the tab bar when the instance is running inside a TEE deployment. On hover, fetches and displays the TDX attestation report (image digest, TLS cert fingerprint, report data, VM config) from the management API. Co-authored-by: Cursor <[email protected]> |
||
|
Pierre LE GUENandGitHub
|
df8616b604 |
fix: add X-Accel-Buffering header to SSE endpoints (#277)
Nginx buffers responses by default, breaking SSE connections that go through a reverse proxy. Add X-Accel-Buffering: no header to chat and log SSE handlers to match what compose-api and chat-api already do. |
||
|
|
448383cfb0 |
refactor: remove Responses API, consolidate to Chat Completions (#272)
* fix: strip reasoning from LLM responses and persist assistant messages reliably - Filter out `type: "reasoning"` output items from NEAR AI Responses API parsing so chain-of-thought never reaches the UI (nearai.rs) - Rewrite clean_response with regex-based tag stripping that is code-aware (preserves tags inside fenced blocks and inline backticks), supports 9+ tag names (think, thought, reasoning, reflection, etc.), handles <final> extraction, pipe-delimited tags, and case/whitespace tolerance (reasoning.rs) - Add Reasoning::complete() helper so all non-agentic LLM call sites (summarize, suggest, heartbeat, compaction) get automatic response cleaning; thread SafetyLayer through to those callers - Change persist_turn from fire-and-forget tokio::spawn to awaited async so both user and assistant messages are written before returning, preventing data loss on shutdown/restart - Pass input_count through seed_response_chain so response chaining delta calculation is accurate after thread hydration on restart - Make NearAiResponse.usage optional and preserve response_id in alt response path for chaining continuity - Persist session token to DB during onboarding wizard so runtime loads it without legacy-key fallback; suppress spurious warning on fresh installs - Fix dev tool double-registration when builder already registers them - Load dotenv/ironclaw env for doctor and status subcommands - Reduce startup log noise (demote info→debug for skills, remove redundant info lines) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Nudge to not loop over tools continuesly * refactor: remove Responses API, consolidate NEAR AI to Chat Completions only The Responses API provider (nearai.rs, 1278 lines) added significant complexity (response chaining state machine, delta message calculation, previous_response_id persistence) for marginal benefit. This consolidates to the Chat Completions API only, upgrading NearAiChatProvider with dual auth (session token + API key) and 401 retry for session token renewal. - Delete src/llm/nearai.rs (Responses API provider) - Upgrade nearai_chat.rs with SessionManager, dual auth, flexible list_models - Remove response_id from CompletionResponse and ToolCompletionResponse - Remove seed_response_chain/get_response_chain_id from LlmProvider trait - Remove response chain persistence from agent (thread_ops, session) - Remove NearAiApiMode enum and NEARAI_API_MODE config - Clean up all wrapper providers (retry, circuit_breaker, failover, cache) - Update documentation (CLAUDE.md, .env.example) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: runtime log level control via gateway UI and URL parameter Add server-side log level switching using tracing_subscriber::reload::Layer so the EnvFilter can be swapped at runtime without restarting. Expose via GET/PUT /api/logs/level endpoints, a "Server: LEVEL" dropdown in the logs toolbar, and a ?log_level=debug URL parameter for one-click activation. Also applies cargo fmt to pre-existing files (llm/, tests/). Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
097a26ace6 |
fix: harden openai-compatible provider, approval replay, and embeddings defaults (#237)
* fix: harden openai-compatible tool flow and local defaults * fix: close approval replay gaps and harden openai-compatible flow * fix: address review feedback and code improvements (takeover #112) - Make ChatCompletionResponse.id Optional<String> to handle providers that omit or null the field - Propagate HTTP client builder errors instead of silently dropping timeout configuration (openai_compatible_chat, nearai_chat) - Add EMBEDDING_DIMENSION env var with smart per-model defaults instead of hardcoding 768/1536 everywhere - Remove duplicated dimension inference logic from main.rs Co-Authored-By: panosAthDBX <[email protected]> Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden src/llm/ module from crate audit findings - Replace 9x .expect() on RwLock with graceful poison recovery (nearai.rs: 7, nearai_chat.rs: 2) — eliminates production panics - Propagate HTTP client builder errors in nearai.rs instead of silently dropping timeout config (NearAiProvider::new now returns Result) - Make nearai_chat ChatCompletionResponse.id Optional<String> (mirrors openai_compatible_chat.rs fix for providers that omit id) - Make nearai_chat usage fields optional with defensive parse_usage() helper (was required u32 fields that crash on null/missing) - Truncate error responses to 512 chars in nearai_chat.rs error messages to prevent log bloat and potential data leakage - Delegate 4 missing LlmProvider methods in FailoverProvider (model_metadata, seed_response_chain, get_response_chain_id, calculate_cost) to last-used provider instead of trait defaults Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor(llm): add RetryProvider, remove openai_compatible_chat, harden decorators - Add composable RetryProvider decorator wrapping any LlmProvider with exponential backoff + jitter, respecting RateLimited retry_after hints - Remove openai_compatible_chat.rs — replaced by rig adapter + RetryProvider - Remove internal retry loop from nearai.rs (was causing double-retry with external RetryProvider, up to 16 attempts instead of 4) - Remove internal retry loop from nearai_chat.rs (same issue) - Wire RetryProvider into main.rs composition chain: each provider gets its own retry wrapper before failover - Move normalize_tool_name to rig_adapter.rs for all rig-based providers - Reconcile is_retryable() vs is_transient() error classification: ModelNotAvailable no longer retryable, Json no longer transient - Fix unchecked Duration subtraction panic in circuit_breaker.rs - Make failover.rs use shared is_retryable() from retry.rs - Remove stale #[allow(dead_code)] on NearAiResponse::id (field is used) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback — error handling, dimension validation, libSQL warning - Replace response.text().await.unwrap_or_default() with proper error propagation in nearai.rs and nearai_chat.rs (4 call sites). Failures now return LlmError::RequestFailed with context instead of silently proceeding with an empty string. - Add embedding dimension validation in OllamaEmbeddings::embed_batch(): returns EmbeddingError if Ollama returns embeddings with a dimension that doesn't match the configured value. - Add runtime warning when libSQL backend is used with non-1536 embedding dimension, since the libSQL schema uses F32_BLOB(1536) and cannot store different-dimension vectors. Co-Authored-By: Claude Opus 4.6 <[email protected]> * Apply suggestions from code review Co-authored-by: Copilot <[email protected]> --------- Co-authored-by: panosAthDbx <[email protected]> Co-authored-by: panosAthDBX <[email protected]> Co-authored-by: panosAthDBX <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Copilot <[email protected]> |
||
|
|
e42b1e5ec1 |
fix: Network Security Findings (#201)
* docs(security): add network security reference for all listeners Catalogs every network-facing surface (web gateway, webhook server, orchestrator API, OAuth callback, sandbox proxy) with auth mechanisms, bind addresses, egress controls, known findings, and a review checklist for PRs that touch network-facing code. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): address three network security findings - Use constant-time comparison (ct_eq) for webhook secret validation, matching the pattern in web gateway and orchestrator auth - Add X-Content-Type-Options and X-Frame-Options security headers to the web gateway via SetResponseHeaderLayer - Warn at startup when HTTP webhook server binds to 0.0.0.0 - Update NETWORK_SECURITY.md to mark findings 1, 4, 5 as resolved Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(security): address PR #201 review findings - Reorder web gateway layers so security headers (X-Content-Type-Options, X-Frame-Options) are outermost and apply to all responses including DefaultBodyLimit 413 rejections - Move 0.0.0.0 warning to final bind address resolution so it fires for WASM-only webhook servers that fall back to the default address - Add webhook handler auth tests: correct secret -> 200, wrong secret -> 401, missing secret -> 401 - Rewrite NETWORK_SECURITY.md: replace brittle line-number references with function/struct name anchors, add threat model section, document graceful shutdown per listener, fill content gaps (health endpoint responses, content-type validation, CSRF analysis, WS auth flow, MCP trust boundary, orchestrator rate limiting), change findings F-4/F-5 from "Resolved" to "Mitigated" with caveats Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt and clippy warnings from main merge Fix formatting in llm/mod.rs and llm/rig_adapter.rs introduced by PR #132, and collapse nested if in rig_adapter.rs per clippy. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
ccf60055f4 |
feat: support per-request model override in /v1/chat/completions (#103)
* feat: support per-request model override for /v1/chat/completions - add optional model override to completion request types\n- forward request model through gateway, worker, and orchestrator proxy paths\n- use request model in NEAR AI providers with fallback to active model\n- replace model-mismatch integration test with override propagation checks\n- update FEATURE_PARITY.md note for OpenAI-compatible API behavior\n\nRefs #49 * Wire gateway OpenAI-compatible routes to active LLM provider * Validate OpenAI model name length before streaming * Address PR103 review feedback on model override and validation * Report effective model in OpenAI-compatible responses * Use async mutexes in OpenAI compatibility integration tests * fix tests for per-request model field in response cache * fix formatting and clippy lint after main merge * Fix model override reporting and cache correctness --------- Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
ffb1cc9be8 |
refactor: architecture improvements for contributor velocity (#198)
* refactor: split large files and consolidate test stubs for contributor velocity - Extract 7 Database sub-traits (ConversationStore, JobStore, SandboxStore, RoutineStore, ToolFailureStore, SettingsStore, WorkspaceStore) with Database as a supertrait combining them all - Split libsql_backend.rs (2769 lines) into src/db/libsql/ directory with one file per sub-trait implementation - Split config.rs (1753 lines) into src/config/ directory with 16 domain files - Consolidate 3 duplicate test LLM stubs into shared StubLlm in src/testing.rs - Split server.rs handlers into src/channels/web/handlers/ directory - Extract main.rs init phases into AppBuilder (src/app.rs) - Add developer setup script (scripts/dev-setup.sh) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: move heartbeat test from examples/ to tests/ Convert standalone example binary into a proper #[ignore] integration test, matching the convention of the other integration tests. Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: fix rustfmt formatting for CI Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review comments from Copilot - tunnel.rs: replace .ok().flatten() with ? to propagate env var errors - secrets.rs: remove misleading "process-wide cache" comment - database.rs: use uppercase "DATABASE_URL" in error key - testing.rs: gate harness tests with #[cfg(feature = "libsql")] Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Illia Polosukhin <[email protected]> Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
cfb579a4bb |
feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows (#57)
* feat: Add PR review tools, job monitor, and channel injection for E2E sandbox workflows Adds JobEventsTool and JobPromptTool so the main agent can read container event logs and send follow-up prompts to running Claude Code sessions. A background JobMonitor forwards container assistant messages into the agent loop via a new inject channel on ChannelManager. CreateJobTool now accepts a project_dir parameter for mounting existing cloned repos into containers, and spawns the monitor automatically for async jobs. Also: Dockerfile bumped to Rust 1.88 (rig-core needs let chains), GITHUB_TOKEN forwarded into containers for gh CLI auth, and truncate() fixed for multi-byte char boundary panics. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Add ownership checks to JobEventsTool and JobPromptTool via ContextManager to prevent users from accessing other users' jobs (IDOR) - Combine Dockerfile gh CLI install into single apt-get layer - Handle truncate() edge case when max falls inside first multi-byte char - Log actual count of registered job management tools - Document fire-and-forget job monitor lifecycle - Add tests for ownership rejection and schema validation Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Replace hardcoded GITHUB_TOKEN with on-demand credential delivery Containers now fetch credentials via authenticated GET /worker/{id}/credentials endpoint instead of receiving them baked into env vars at creation time. Secrets are decrypted from SecretsStore on demand, scoped per-job via CredentialGrant, and revoked automatically when the job completes. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address sandbox audit findings (CONNECT tunnel, readonly_rootfs, type consolidation) - Implement real CONNECT tunnel with bidirectional TCP piping via hyper upgrade - Fix readonly_rootfs to apply for both ReadOnly and WorkspaceWrite policies - Consolidate duplicate CredentialMapping/CredentialLocation into secrets::types - Share reqwest::Client across proxy requests instead of per-request allocation - Store Docker connection and reuse across executions - Remove .unwrap() from proxy response builders with safe fallbacks - Add output truncation to direct (non-container) execution (64KB limit) - Delete dead src/tools/sandbox.rs (ToolSandbox never used) - Fix connect_docker error message to list all attempted socket paths - Update proxy credential injection to handle all CredentialLocation variants - Use glob-based host_patterns matching for credential lookup in proxy policy Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address PR #57 review comments (IDOR, Dockerfile, truncate, logging) - Dockerfile: install curl+ca-certificates before fetching GitHub CLI GPG key - JobEventsTool/JobPromptTool: reject missing context (prevents IDOR bypass) - parse_credentials: validate env var names against denylist and pattern - resolve_project_dir: require explicit paths to exist before validation - Credential serving: lower log level from info to debug Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address orchestrator audit findings (constant-time auth, error handling, tests) - auth: constant-time token comparison via subtle::ConstantTimeEq - auth: replace hand-rolled hex_encode with std::fmt::Write fold - api: report_status now updates ContainerHandle (was a no-op) - api: log complete_job errors instead of silently discarding - job_manager: log Docker cleanup errors in stop_job/complete_job - job_manager: extract validate_bind_mount_path with proper error on missing home_dir and mandatory base dir creation before canonicalize - job_manager: cache Docker connection across operations - error: remove dead OrchestratorError::AuthFailed and ContainerTimeout - Add 13 new tests (prompt queue, credentials, events, status, paths) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use floor_char_boundary in sandbox manager truncate to prevent multi-byte panics String::truncate() panics when the index falls mid-way through a multi-byte UTF-8 character. Use the same floor_char_boundary utility already used in worker/runtime.rs and tools/builtin/shell.rs. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: default base_url to private.near.ai for Responses API mode Session tokens only authenticate against private.near.ai, not cloud-api.near.ai. The default base_url now matches the api_mode: - Responses (session token): https://private.near.ai - ChatCompletions (API key): https://cloud-api.near.ai This broke when the multi-provider merge introduced cloud-api.near.ai as the unconditional default. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: use private.near.ai as default base URL for all API modes private.near.ai now supports both Responses and ChatCompletions endpoints, so there is no reason to route through cloud-api.near.ai. This also fixes session token auth which only works against private.near.ai. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: harden libSQL concurrency, fix Claude Code Docker auth and permissions Three fixes for the sandbox/Claude Code pipeline: 1. SQLite "database is locked": set WAL journal mode in migrations and PRAGMA busy_timeout=5000 on every connection across LibSqlBackend, LibSqlSecretsStore, and LibSqlWasmToolStore (~83 async call sites). 2. Claude Code container auth: extract OAuth token from macOS Keychain (or Linux ~/.claude/.credentials.json) at startup and inject via CLAUDE_CODE_OAUTH_TOKEN env var. Removes the broken bind-mount approach that failed on uid mismatch. 3. Claude Code tool permissions: wire CLAUDE_CODE_ALLOWED_TOOLS env var through to the worker binary (was hardcoded to empty vec), and expand defaults to include all standard tools (Read, Write, Edit, Glob, Grep, NotebookEdit, Bash, Task, WebFetch, WebSearch). Also adds --verbose flag to claude CLI (required with stream-json + -p), failover provider model switching, and nearai models endpoint fix. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: stream event parsing, job ID prefix resolution, session renewal in list_models Three fixes for the Docker/gateway pipeline: 1. Claude Code stream event parsing (claude_bridge.rs): Rewrite ClaudeStreamEvent to match actual NDJSON format where content blocks are nested under message.content[], not at the top level. Add handler for "user" events (tool_result blocks) and emit result text as a "message" event so reviews appear in gateway activity view. 2. Job ID prefix resolution (job.rs): Add resolve_job_id() that accepts short hex prefixes (like git short SHAs) in addition to full UUIDs. The LLM sees truncated IDs in job monitor messages like "[Job f2854dd8]" and can now use them directly with job_status/cancel/events/prompt tools. 3. Session renewal in list_models (nearai.rs): list_models() now retries with OAuth renewal on 401, matching send_request()'s existing behavior. Previously it returned SessionExpired immediately, causing the setup wizard to fall back to defaults instead of prompting re-authentication. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: /model command now lists available models Previously /model with no args only showed the current model name. Now it fetches and displays all available models from the provider, marking the active one, so users can see what's available before switching with /model <name>. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #57 review findings (set_var UB, tunnel timeout, restart creds) - Replace unsafe `std::env::set_var` in worker runtime and Claude bridge with `Command::envs()` injection via a new `extra_env` field on `JobContext`, avoiding undefined behavior in the multi-threaded tokio runtime. - Add 30-minute timeout to CONNECT tunnel `copy_bidirectional` in the sandbox proxy to prevent stuck connections from leaking spawned tasks. - Persist credential grants (as JSON in the description column) on `SandboxJobRecord` so `jobs_restart_handler` can restore them instead of passing `vec![]`, which caused restarted containers to lose access to their original secrets. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR #57 review comments - Normalize host_patterns to lowercase in proxy policy matching - Push LIMIT into SQL for list_job_events (Database trait + both backends) - Remove unused was_explicit binding in job tool - Return 500 instead of 200 in make_response fallback path - Update copy_auth_from_mount docstring for env-var default - Use entry.file_type() instead of is_dir() to avoid following symlinks Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address third round of PR #57 review comments - Restore glob patterns in default_claude_code_allowed_tools (Bash -> Bash(*)) - Add tracing::warn for credential grant serialize/deserialize failures - Wrap extra_env in Arc<HashMap> to avoid deep cloning per tool call - Document unsupported credential locations (AuthorizationBasic, UrlPath) - Document TOCTOU window in validate_bind_mount_path - Expand doc comments on JobEventsTool and JobPromptTool Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fourth round of PR #57 review comments - Document CONNECT tunnel task lifecycle (timeout is the cleanup mechanism) - Remove secret names from error-level credential logs to prevent leaking - Expand DANGEROUS_ENV_VARS denylist with language runtime hijack vectors Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address fifth round of PR #57 review comments - Promote job monitor startup log to info level for observability - Require minimum 4-char prefix in resolve_job_id to limit enumeration - Cap credential grants at 20 per job to bound column storage - Clamp job events limit to 1..1000 to prevent memory abuse Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing closing brace for SkillsConfig impl block The merge resolution dropped the closing `}` for `impl SkillsConfig`, causing a compilation error in CI. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
bac2d75713 |
feat: Secure prompt-based skills system (Phases 1-4) (#51)
* feat: Add secure prompt-based skills system (Phase 1 MVP) Implement a skills system that extends the agent with prompt-level instructions from local directories. Skills declare activation criteria, tool permissions, and trust tiers that determine authority attenuation. Core security model: the minimum trust level of any active skill determines a tool ceiling -- tools above the ceiling are removed from the LLM's tool list entirely at the API level, preventing prompt-based manipulation. New modules: - skills/mod.rs: Core types (SkillTrust, SkillManifest, LoadedSkill) - skills/scanner.rs: Content scanner for manipulation detection - skills/registry.rs: Filesystem discovery and manifest parsing - skills/selector.rs: Deterministic two-phase prefilter (no LLM) - skills/attenuation.rs: Trust-based tool filtering Integration: - Agent loop selects skills per-turn and applies tool attenuation - Reasoning engine injects skill context with structural isolation - Config supports SKILLS_ENABLED, SKILLS_DIR, SKILLS_MAX_ACTIVE, SKILLS_MAX_CONTEXT_TOKENS environment variables - Disabled by default (SKILLS_ENABLED=false) 41 new tests covering all modules. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address all adversarial review findings for skills system Security fixes: - Escape skill name/version in XML attributes to prevent trust spoofing - Escape prompt content to prevent </skill> tag breakout - Require integrity hash for Verified/Community tier skills - Validate skill names against [a-zA-Z0-9][a-zA-Z0-9._-]{0,63} - Add 64 KiB file size limit on prompt.md Bug fixes: - Use actual SkillsConfig from AgentDeps instead of SkillsConfig::default() - Add skills_config field to AgentDeps, wired through from main.rs Performance: - Pre-compile regex patterns at load time (cached on LoadedSkill) - Selector uses pre-compiled patterns instead of recompiling per message - Switch all std::fs to tokio::fs for non-blocking async I/O Hardening: - Cap keyword score at 30 points to prevent keyword stuffing attacks - Enforce max 20 keywords and 5 patterns per skill - Normalize line endings (CRLF/CR to LF) before hashing - Also includes cargo fmt formatting fixes for adjacent code Tests: 54 skills tests pass (up from 41), zero new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address medium/low severity findings from adversarial review Fixes all 18 medium/low severity findings identified by the security review: - mod.rs: Add MAX_TAGS_PER_SKILL cap (10) in enforce_limits(); use RegexBuilder with 64 KiB size_limit to prevent ReDoS; replace case-enumerated escape_skill_content with regex matching all case variants plus whitespace/null byte injection between </ and skill; document allowed_patterns as unenforced until Phase 2; document Marketplace URL validation as Phase 3 concern - registry.rs: Add MAX_MANIFEST_FILE_SIZE (16 KiB) check before reading; add symlink detection via symlink_metadata to reject symlinks in discover_local; add MAX_DISCOVERED_SKILLS (100) cap; validate prompt_hash format (sha256: + 64 hex chars); warn on name collision before overwriting; accept SkillSource parameter in load_skill instead of always using Local; add InvalidHashFormat, ManifestTooLarge, SymlinkDetected error variants - selector.rs: Add MAX_TAG_SCORE (15) cap parallel to keyword cap; warn when declared max_context_tokens diverges >2x from actual prompt size - scanner.rs: Add mixed-script homoglyph detection (Cyrillic, Greek, Armenian unicode ranges); document token-boundary bypass and semantic paraphrasing as known limitations - attenuation.rs: Document READ_ONLY_TOOLS maintenance requirements - agent_loop.rs: Surface scan warnings via structured tracing; add structured audit events for skill activation and tool attenuation 61 tests pass, 0 new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address 12 findings from second adversarial security review HIGH: - Escape opening <skill tags in prompt content (prevents fake skill block injection) - Scan manifest metadata fields (description, author, tags, reasons) not just prompt - Block trust downgrade on name collision (existing Local can't be replaced by Community) MEDIUM: - Eliminate TOCTOU gap: read files then check size instead of metadata-then-read - Reject file-level symlinks in load_skill (prompt.md, skill.toml) - Truncate and filter manifest.skill.tags (prevent unlimited tag scoring) - Cap regex pattern score at 40 (prevent 5x20=100 dominating keyword+tag) - Add doc comment about skill_list tool exposing metadata (sanitization required) - Move Community disclaimer inside <skill> tags (not outside structural boundary) - Filter keywords/tags shorter than 3 chars (prevent broad matching) LOW: - Enforce minimum token_cost of 1 (max_context_tokens=0 can't bypass budget) - Remove redundant try_exists checks in discover_local (let load_skill handle errors) 70 skills tests passing. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add HTTP endpoint scoping for skills (Phase 1) Skills that declare an [http] section in skill.toml now have their HTTP requests constrained to declared endpoints at runtime. This addresses the gap where allowed_patterns was parsed but never enforced -- once the http tool was visible via attenuation, the LLM could reach any URL. Enforcement reuses EndpointPattern/AllowlistValidator from the WASM capability system. Semantics: if no active skill declares [http], all requests pass through (backward compat). If any skill declares [http], URLs must match at least one skill's allowlist (union). Community skills' [http] declarations are silently ignored (defense in depth). Shell commands using curl/wget are also validated against scopes. Scanner gains detection for known exfiltration domains (webhook.site, ngrok.io, etc.), overly broad wildcards, and credential/host mismatches. Closes #38 Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt to http_scoping.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * style: Apply cargo fmt across codebase Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add parameter-level permission enforcement for skills (Phase 2) Activates enforcement of `allowed_patterns` in skill.toml permissions. Previously these patterns were parsed but not enforced -- a Verified skill declaring `permissions.shell` with `allowed_patterns = [{command = "cargo *"}]` could still run any shell command. Now the enforcer validates tool parameters against declared glob patterns before execution. Key changes: - New `enforcer.rs` module with `SkillPermissionEnforcer`, `glob_to_regex()`, and `validate_tool_call()` with union semantics across active skills - Typed pattern enums (`ShellPattern`, `FilePathPattern`, `MemoryTargetPattern`) replace the previous `Vec<serde_json::Value>` in `ToolPermissionDeclaration` - Scanner gains `scan_permission_patterns()` detecting dangerous patterns (rm, sudo, curl, bare wildcards, command chaining, sensitive paths, identity files) - Registry blocks non-Local skills with critical permission pattern warnings - Agent loop threads enforcer into `execute_chat_tool` alongside HTTP scoping Trust interaction: Community patterns ignored, Verified enforced, Local without patterns unrestricted, Local with patterns enforced as guidance. Union semantics across skills -- tool call allowed if ANY skill's patterns permit it. 34 new tests. All 818 library tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add worker permission enforcement and LLM behavioral analysis (Phase 3+4) Phase 3 - Worker-side permission enforcement: - Add SerializedToolPermission/SerializedPattern DTOs for HTTP boundary crossing - Extend JobDescription, ContainerHandle, and orchestrator API to carry permissions - CreateJobTool snapshots and forwards skill permissions to spawned workers - Worker runtime builds SkillPermissionEnforcer and checks before tool execution - Load-time token budget enforcement rejects prompts exceeding 2x declared budget - Deduplicate enforcer construction: from_active_skills() delegates to from_serialized() Phase 4 - LLM behavioral analysis: - BehavioralAnalyzer with cached, LLM-based semantic content analysis - Structured output parsing (FINDING|CATEGORY|SEVERITY|DESCRIPTION or CLEAN) - Content-hash caching with bounded size (MAX_CACHE_ENTRIES=256) - Graceful degradation when LLM unavailable - Integrated into load_skill() for non-Local skills; critical findings block loading Review fixes: - Real cache tests with CountingLlm mock (test_cache_hit, test_cache_miss, test_cache_bounded) - UTF-8-safe truncate() in worker runtime - Few-shot examples in behavioral analysis prompt - Documented max_context_tokens=0 opt-out and create_job() permission gap 848 tests passing, no new clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address review feedback from serrrfirat on skills-phase2 - Fix truncate_cmd UTF-8 panic: use char-boundary-aware slicing - Remove redundant effective_tools branching in reasoning.rs - Document cache eviction as known limitation (arbitrary, not LRU) - Add safety comment on SkillTrust enum ordering (security-critical) - Simplify active_skills selection (prefilter_skills handles empty input) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address remaining skills review feedback * refactor: replace skills system with OpenClaw SKILL.md format + 2-state trust Replace the 5-gate, 3-tier trust hierarchy (scanner, behavioral analyzer, parameter-level enforcer, HTTP endpoint scoping) with a simplified 3-layer security model: gating -> attenuation -> Docker confinement. Key changes: - SKILL.md format (YAML frontmatter + markdown prompt) replaces skill.toml + prompt.md - 2-state trust (Installed/Trusted) replaces 3-tier (Community/Verified/Local) - New parser.rs for SKILL.md parsing with serde_yaml - New gating.rs for requirements checking (bins/env/config) - Simplified registry with 2-location discovery (workspace + user dirs) - Removed scanner, behavioral_analyzer, enforcer, http_scoping (~4,100 lines) - Removed skill_permissions propagation through job/orchestrator/worker pipeline - Added serde_yaml dependency for YAML frontmatter parsing Net: -5,298 lines, 59 skills tests pass, 907 total tests pass. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: add in-app skill management tools and ClawHub catalog integration Add 4 chat-callable tools (skill_list, skill_search, skill_install, skill_remove) plus matching web gateway endpoints for managing skills at runtime. The catalog fetches from ClawHub's public registry API at runtime rather than bundling entries at compile time. Key changes: - SkillRegistry gains mutation methods (install_skill, remove_skill, reload, find_by_name) with Arc<RwLock> for concurrent access - New catalog module queries ClawHub /api/v1/search with in-memory caching (5-min TTL, configurable via CLAWHUB_REGISTRY env var) - skill_list and skill_search added to READ_ONLY_TOOLS for safe use under Installed trust ceiling - Web gateway gets /api/skills, /api/skills/search, /api/skills/install, and /api/skills/{name} DELETE endpoints Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR #51 review feedback from ilblackdragon Security: - Add SSRF protection to fetch_skill_content: require HTTPS, reject private/loopback/link-local IPs and internal hostnames, disable redirects. Gateway install handler now reuses the same validation. - URL-encode slug in skill_download_url to prevent query injection. - Require X-Confirm-Action header on gateway skill install/remove endpoints (equivalent to chat tool requires_approval gate). Correctness: - Eliminate all block_in_place/block_on usage in skill tools and gateway handlers. Split install into prepare_install_to_disk (static async, no lock) + commit_install (sync, brief write lock). Same pattern for remove: validate_remove + delete_skill_files + commit_remove. - Write normalized content to disk in install_skill (was writing original un-normalized content, causing hash mismatch on re-read). - Fix token estimation from 0.75 to 0.25 tokens/byte (~4 chars per token) in registry.rs, selector.rs, and standalone loader. Dependencies: - Replace deprecated serde_yaml 0.9 with serde_yml 0.0.12. - Remove unused toml dependency. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
d04af5cd75 |
web: add integrity check for marked CDN and cap highlight regex input (#109)
* web: add integrity check for marked CDN and cap highlight regex input * web: normalize memory search query before snippet+highlight matching * web: place memory query length constant with top-level config --------- Co-authored-by: Clawyered <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
e843c18141 |
feat: add libSQL/Turso embedded database backend (#47)
* feat: add libSQL/Turso database backend with full feature parity Introduce a Database trait abstraction (~60 async methods) enabling compile-time backend selection between PostgreSQL and libSQL/Turso. Convert all modules from concrete Store to Arc<dyn Database>, add LibSqlSecretsStore and LibSqlWasmToolStore implementations, wire libsql stores throughout CLI and main entry points, and make the setup wizard backend-agnostic. Key changes: - src/db/: Database trait, PostgresDatabase adapter, LibSqlBackend with native SQLite-dialect SQL, and idempotent migration system - src/secrets/store.rs: LibSqlSecretsStore (all 8 trait methods) - src/tools/wasm/storage.rs: LibSqlWasmToolStore (all 7 trait methods) - src/main.rs, cli/tool.rs, cli/mcp.rs: backend-conditional wiring - src/setup/channels.rs: SecretsContext uses Arc<dyn SecretsStore> - Feature-gate postgres-only tests and examples Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: enable onboarding wizard for libSQL builds Refactor the setup wizard to work with both postgres and libsql feature flags. Previously the wizard was gated behind #[cfg(feature = "postgres")] only, so libsql-only builds would print an error on `ironclaw onboard`. - Add libsql fields to Settings (database_backend, libsql_path, libsql_url) - Split wizard database/migration/secrets methods into feature-gated variants - Add step_database_libsql() with local path and Turso remote replica prompts - Update setup/mod.rs and main.rs feature gates to any(postgres, libsql) - Extend check_onboard_needed() to detect libsql database presence Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review feedback for libSQL backend - P0: Switch libsql_backend to connection-per-operation pattern to fix shared Connection concurrency issue across tokio tasks - P0: Wrap secrets store INSERT+SELECT in transaction to fix TOCTOU race - P0: Document encryption-at-rest limitations and json_patch divergence - P1: Fix get_opt_text removing .filter(|s| !s.is_empty()) that conflated empty strings with NULL - P1: Replace datetime('now') with fmt_ts(&Utc::now()) for consistent RFC 3339 timestamps across all queries - P2: Use explicit _rowid column in FTS5 triggers and joins for stability across VACUUM operations - P2: Add tracing::warn when embedding provided but vector search disabled in hybrid_search - Extract shared connect_from_config() helper to deduplicate DB connection logic across main.rs, cli/config.rs, and cli/mcp.rs Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add missing JobContext fields and resolve fmt/clippy warnings Add total_tokens_used and max_tokens fields to JobContext in libsql_backend.rs, apply cargo fmt, and fix clippy warnings. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: review fixes for libSQL backend (shared connections, panics, indexes) - Replace .expect() with proper error propagation in 3 call sites - Share Arc<Database> between backend and stores instead of single Connection - Add connect-per-operation pattern to LibSqlSecretsStore and LibSqlWasmToolStore - Wrap store() INSERT + SELECT-back in a transaction - Add ~22 missing indexes for parity with PostgreSQL schema - Add 18 leak_detection_patterns seed rows matching PostgreSQL V2 migration - Fix super:: import to use crate:: style - Gate mask_password_in_url behind #[cfg(feature = "postgres")] - Rewrite secrets store init with or_else chain for runtime backend selection Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (collapsible_if, too_many_arguments) Collapse nested if blocks into let_chains to satisfy clippy's collapsible_if lint (CI uses -D warnings). Suppress too_many_arguments on libsql_row_to_tool_at since refactoring the positional index pattern would be a larger change. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: Illia Polosukhin <[email protected]> |
||
|
|
5df0d13b59 |
Bump MSRV to 1.92, add GCP deployment files (#40)
* Bump MSRV to 1.92 and add GCP deployment files rig-core 0.30 uses let_chains (stabilized post-1.87), which breaks builds on Rust 1.85. Bump rust-version in Cargo.toml and both Dockerfiles to 1.92 (verified working). Add cloud deployment scaffolding: - Dockerfile: multi-stage build for the main agent container - deploy/cloud-sql-proxy.service: systemd unit for Cloud SQL Auth Proxy - deploy/ironclaw.service: systemd unit for the IronClaw container - deploy/setup.sh: VM bootstrap script (Docker, proxy, services) - deploy/env.example: reference environment configuration Co-Authored-By: Claude Opus 4.6 <[email protected]> * Address review feedback: harden deploy scaffolding - Add comment explaining GATEWAY_HOST=0.0.0.0 and when to use 127.0.0.1 - Document /opt/ironclaw ownership model (root-owned, Docker reads as root) - Switch cloud-sql-proxy service from User=root to DynamicUser=yes Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Resolve clippy lints (Rust 1.93) and fix CI test workflow - Fix 97 collapsible_if warnings using let-chains syntax (auto-fixed) - Fix ptr_arg: change &PathBuf to &Path in pairing store functions - Fix suspicious_open_options: add .truncate(false) to OpenOptions - Fix too_many_arguments: add clippy allow on execute_status - Fix unnecessary_unwrap: use if-let in repository.rs hybrid_search - Gate unused EchoTool with #[cfg(test)] - Add PairingStore argument to ChannelStoreData::new() test call sites - Add skip guard for bundled channel test when WASM artifacts unavailable - Split CI test workflow to exclude PostgreSQL-dependent integration tests Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Address review feedback from ilblackdragon - Add root check to setup.sh (exits with error if not root) - Add warning comment to env.example about placeholder passwords - Dockerfile.worker already uses rust:1.92 (no change needed) - PR #41 overlap noted; will rebase after #41 merges Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: resolve 47 collapsible_if clippy warnings Collapse nested if statements across the codebase to satisfy clippy::collapsible_if on Rust 1.93. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
bbb68f7490 |
Add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models) (#31)
* feat: add OpenAI-compatible HTTP API (/v1/chat/completions, /v1/models)
* - Reject model mismatches: validate req.model against the active model
and return 404 model_not_found instead of silently ignoring it
- Add x-ironclaw-streaming: simulated response header so clients know
streaming is not true token-by-token delivery
- Use SSE event type "error" for mid-stream LLM failures so clients can
distinguish errors from content chunks
- Mark docker-compose credentials as dev-only
- Add integration tests for model mismatch, streaming header, and body
size limit (axum's default 2MB)
* fix: address Copilot review feedback on OpenAI-compat API
- Wire chat_rate_limiter into /v1/chat/completions handler
- Execute LLM before starting SSE stream so failures return proper HTTP
errors instead of SSE error events
- Validate tool-role messages require tool_call_id and name fields
- Surface list_models() errors in models_handler via map_llm_error
- Reject unknown roles with 400 instead of defaulting to User
Co-Authored-By: Claude Opus 4.6 <[email protected]>
---------
Co-authored-by: firat.sertgoz <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
|
||
|
|
33ef0a6ea5 |
fix: security hardening across all layers (#35)
* fix: comprehensive security hardening across all layers Critical: - Replace --dangerously-skip-permissions with explicit tool allowlist via settings.json (Claude Code bridge) - Constant-time token comparison (subtle crate) in web auth and orchestrator auth to prevent timing attacks High: - Revoke tokens and clean up handles on container creation failure - Drop SETUID/SETGID capabilities from containers (keep only CHOWN) - Disable redirect following in HTTP tool and WASM wrapper (SSRF) - Reject URL userinfo (@) in WASM allowlist parser (host confusion) - Fix binary body bypassing leak detection (from_utf8 -> from_utf8_lossy) - Protect identity files from LLM overwrites (prompt injection defense) - Prevent tool shadowing: built-in tools cannot be replaced dynamically - User-scoped job APIs: list/detail/cancel/restart/prompt/events/files - CORS restricted to localhost origins, WebSocket origin validation - Sandbox shell fail-closed: no silent fallback to unsandboxed execution - Scrub secrets from log broadcaster before SSE broadcast - XSS sanitization on rendered markdown in web UI - WASM epoch ticker thread so timeout deadlines actually fire Medium: - Cap state transition history at 200 entries - SSE/WebSocket connection limit (100 max) - Request body size limit (1MB) - Response body size limit enforcement in WASM HTTP - UTF-8 safe string truncation (routine engine, shell tool) - Fix PolicyAction::Sanitize to actually run the sanitizer - TOCTOU fix in scheduler and context manager (hold write lock) - Project file serving moved behind auth - Path traversal guard on project_id - Session file permissions set to 0600 on unix - AtomicUsize for routine running_count (panic-safe) - Completion detection hardened against false positives and tool injection - Tool output no longer drives job completion (only LLM response) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address security review findings across all layers - Fix path traversal sandbox bypass via lexical normalization (file.rs) - Fix SSRF via DNS rebinding with pre-request hostname resolution (http.rs) - Add token budget enforcement on LLM calls (reasoning.rs, state.rs) - Fix cross-user chat history leak with ownership verification (store.rs, server.rs) - Add sliding-window rate limiter on gateway chat endpoint (server.rs) - Harden extension install: HTTPS-only, 50MB cap, WASM magic validation (manager.rs) - Add destructive command blocklist that overrides shell auto-approval (shell.rs) - Add 5MB response body size cap to HTTP tool (http.rs) Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: deduplicate shared helpers and remove dead code Extract floor_char_boundary and llm_signals_completion into src/util.rs, unifying diverging phrase lists from agent/worker.rs and worker/runtime.rs. Remove dead RespondResult::usage(), duplicate PROTECTED_IDENTITY_FILES constant, double LeakDetector scanning in WebLogLayer, and invalid 0.0.0.0 origin from WebSocket allow list. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address PR review findings and CI test failures - Fix record_failed_approve: .truncate(true) wiped the attempts file before reading, so failed pairing attempts never accumulated and rate limiting never triggered. - Guard wizard WASM test: skip gracefully when channel build artifacts are absent (CI doesn't compile wasm32-wasip2 targets). - Fix DNS rebinding check: use port 0 instead of hardcoded 443, since the port is irrelevant for hostname resolution. - Remove hardcoded CORS port 3001: the dynamic addr.port() entries already cover the actual server port. - Require WebSocket Origin header: reject connections that omit it entirely, since browsers always send Origin for WS upgrades and a missing header indicates a non-browser client bypassing the check. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: address second round of PR review findings - store.rs: reintroduce file locking around read-modify-write in record_failed_approve (concurrent callers could clobber each other). - sse.rs: replace load+check+fetch_add with atomic fetch_update in both subscribe_raw() and subscribe() to prevent overshooting max_connections. - ws.rs: decrement WS tracker before early return when subscribe_raw() returns None (connection limit reached), fixing a counter leak. - server.rs: parse WS Origin host exactly instead of prefix matching, preventing bypass via crafted origins like http://localhost.evil.com. - workspace_integration.rs: skip tests gracefully when Postgres is unreachable instead of panicking (fixes 10 CI failures). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: add Origin header to WS integration tests The Origin header requirement added in a3b0190 broke the WS gateway integration tests. Test clients now send Origin: http://127.0.0.1:{port} to match the server's localhost validation. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
ced83d5b4d |
feat: Sandbox jobs (#4)
* Orchestrating jobs and running them in sandboxes * Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback - Query /v1/models API for context_length and set max_tokens to half (floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7 need much larger budgets - Guard against empty LLM content (reasoning models can burn all tokens on chain-of-thought and return content: null) - Simplify notification routing: try configured channel first, fall back to broadcast_all so heartbeat alerts always reach someone - Add ModelMetadata struct and model_metadata() to LlmProvider trait - Refactor NearAiChatProvider::list_models into shared fetch_models() - Add standalone test_heartbeat example for isolated debugging Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add job detail view with drill-down from jobs list Click a job row to see full details across four sub-tabs: Overview (metadata grid, description, state transitions timeline), Actions (expandable tool call cards with input/output JSON), Thinking (conversation messages styled by role), and Files (embedded workspace tree browser). Co-Authored-By: Claude Opus 4.6 <[email protected]> * Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400 Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the content field instead of using the OpenAI tool_calls array. This XML leaks through to channels as text, and Telegram's Markdown parser chokes on the underscores, returning 400 "can't parse entities". Two fixes: - Generalize clean_response() to strip <tool_call>, <function_call>, <tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside the existing <thinking> tag stripping - Add Telegram send_message helper with parse_mode fallback: try Markdown first, retry as plain text on "can't parse entities" 400 errors Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add SystemCommand submission type for thread-state-independent commands System commands (/help, /model, /version, /tools, /ping, /debug) now bypass thread-state checks and safety validation via a dedicated Submission::SystemCommand variant. Previously these flowed through process_user_input() which blocked them during Processing/AwaitingApproval /Completed states. - Add /model [name] for runtime model switching with provider validation - Add active_model_name()/set_model() to LlmProvider trait with RwLock hot-swap in both NEAR AI providers - Rewrite /help with aligned columns grouped by category - Expand REPL tab-completion from 10 to 23 slash commands - Remove REPL-local /help interception (now handled by agent) Co-Authored-By: Claude Opus 4.6 <[email protected]> * Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files The sandbox e2e pipeline (agent -> container -> built website -> browsable URL) was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need minutes, no auto-created project directory meant container output vanished, and no HTTP route to browse the built files. - Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler, worker/runtime) with the per-tool value - Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer) - Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified, so every sandbox job gets a persistent bind mount - Include `project_dir` and `browse_url` in sandbox tool output JSON - Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes to the web gateway with path traversal protection and MIME type detection - Add `mime_guess` dependency for content-type detection Co-Authored-By: Claude Opus 4.6 <[email protected]> * Apply cargo fmt to wizard.rs after merge Co-Authored-By: Claude Opus 4.6 <[email protected]> * Persist sandbox jobs in DB, fix web UI, unify job model Sandbox container jobs were invisible to the web UI because they lived only in ContainerJobManager's in-memory HashMap while the API queried ContextManager. This persists them to the agent_jobs table and fixes all six front-end bugs (empty job list, broken back button, empty actions/thinking tabs, wrong files tab, stuck status, no persistence). Key changes: - V4 migration adds project_dir and user_id columns to agent_jobs - Embedded migrations via refinery (no external CLI needed) - SandboxJobRecord CRUD in Store with fire-and-forget DB writes - Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager - Web API queries DB for sandbox jobs, merges with ContextManager direct jobs - New endpoints: restart, project file list/read with path traversal protection - Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in chat stream, source badges, restart button for failed/interrupted jobs - Gateway defaults to enabled, prints Web UI URL on startup - Stale jobs marked "interrupted" on restart for visibility and restartability Co-Authored-By: Claude Opus 4.6 <[email protected]> * Secure in-chat auth: tokens never touch the LLM or chat history Remove the token parameter from tool_auth so the LLM cannot pass raw API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket (auth_token) endpoints that route tokens directly to ext_mgr.auth(), completely bypassing the message pipeline, turns, history, and compaction. Web UI shows an auth card (password input + OAuth button) when the agent enters auth mode, submitted via the dedicated endpoint. CLI auth mode interception is unchanged (already secure). New StatusUpdate::AuthRequired/AuthCompleted variants propagate through all channels (SSE, WebSocket, REPL, WASM). Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add Claude Code mode for sandbox jobs Run Claude Code CLI inside Docker containers as an alternative to the standard worker mode. The bridge spawns `claude -p` with stream-json output, posts events to the orchestrator, and supports follow-up prompts via `--resume`. Key additions: - `claude-bridge` CLI subcommand and ClaudeBridgeRuntime - JobMode enum (Worker vs ClaudeCode) with per-mode container config - Orchestrator endpoints for Claude events and prompt polling - SSE event variants for real-time Claude Code streaming to frontend - Claude Code sub-tab in web UI with terminal-style output and input bar - Database migration for job_mode column and claude_code_events table - ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.) - Mode parameter on run_in_sandbox tool schema Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs When sandbox mode is on, the LLM would call create_job (creating a pending "direct" entry) then run_in_sandbox (creating a second "sandbox" entry), producing two jobs in the list for a single user request. Now register_job_tools() skips create_job when sandbox is enabled since run_in_sandbox already creates tracked jobs. Also improved the run_in_sandbox description to guide the LLM to use it directly and to mention wait=false for async execution. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Web gateway UI quality-of-life improvements Phase 1: Send button disabled state to prevent double-sends, copy button on code blocks, confirm() guards on destructive actions, SSE-driven job list auto-refresh, log filters re-applied on tab switch, jobEvents memory leak fix (cap at 500, cleanup after 60s). Phase 2: Toast notification system replacing chat-based system messages, memory search highlighting with centered snippets, keyboard shortcuts (Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur), activity tab toolbar with event type filter and auto-scroll toggle. Phase 3: Thread sidebar with load/switch/create, thread_id passed with messages, collapsible to hamburger. Memory inline editing with textarea, Save/Cancel, POST to /api/memory/write. Phase 4: Gateway status popover on hover (polls every 30s), extension install form (name/URL/kind), markdown rendering in memory viewer for .md files, mobile responsive layout at 768px breakpoint. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add routines system, remove non-sandbox job mode from web UI Routines: scheduled & reactive job system with cron and event triggers, lightweight (single LLM call) and full-job execution modes, guardrails (cooldown, max concurrent, dedup), and LLM-facing tools for CRUD. Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs are now exclusively sandbox-backed (DB + container). Simplify job detail response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo), fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab event rendering. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML Three fixes: 1. Chat input stays disabled after agent finishes: the "Done" status SSE event now calls enableChatInput() as a safety net when the response event is empty or lost. Same for auth_completed and cancelAuth(). 2. tool_activate never triggers auth: when activation fails due to missing authentication, it now auto-initiates the auth flow (same pattern as the web API handler). detect_auth_awaiting() also matches tool_activate results now. 3. Models like GLM-4.7 emit tool calls as XML tags in content (<tool_call>tool_list</tool_call>) instead of using the structured tool_calls array. recover_tool_calls_from_content() extracts and validates these before falling back to plain text. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Add routines web UI tab, update docs for sandbox-jobs branch Add full routines management to the web gateway (list, detail, trigger, toggle, delete) with 7 new API endpoints, response types, and frontend (HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new subsystems, config, TODOs), and README.md (architecture diagram, features, components, fix onboard command). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Bind Telegram bot to owner account during setup Without owner binding, anyone who discovers the bot can send it messages. The setup wizard now prompts the user to message their bot, captures their Telegram user ID via getUpdates, and persists it as telegram_owner_id in settings. On startup, the owner_id is injected into the WASM channel config so the existing owner restriction logic drops messages from non-owners. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Move settings from disk to PostgreSQL database Settings previously lived in three JSON files on disk (settings.json, mcp-servers.json, session.json). This made them inaccessible from the web UI and caused redundant disk reads (Settings::load() called 8+ times during startup). Now all settings live in a `settings` table (user_id + key -> JSONB) with only 4 bootstrap fields remaining on disk (database_url, pool size, secrets key source, onboard_completed) since they're needed before the DB connection exists. - Add V8 migration for settings table - Add BootstrapConfig (thin disk file) and Settings DB round-trip - Add Store CRUD methods for settings (get/set/delete/list/bulk) - Refactor Config to load from DB (env > DB > default cascade) - Add SessionManager DB persistence for session tokens - Add DB-backed MCP server config load/save functions - Add 6 settings web API endpoints (list/get/set/delete/export/import) - Add one-time disk-to-DB migration on first boot - Make CLI config commands async with DB access (disk fallback) Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth - Add Workspace::seed_if_empty() to create core identity files (README, MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called on every boot without overwriting existing user edits - Remove duplicate gateway log lines from web/mod.rs (main.rs has the useful clickable ?token= URL) - Auto-authenticate from ?token= URL parameter in the web UI and strip the token from the address bar after successful auth Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Harden sandbox security (path traversal + orchestrator auth) Two vulnerabilities fixed: 1. project_dir path traversal: The create_job tool let the LLM specify arbitrary host paths for Docker bind mounts. Removed project_dir from the tool schema entirely, and added canonicalization + prefix validation at both resolve_project_dir() and the job_manager bind mount point. 2. Orchestrator API auth bypass: worker_auth_middleware was defined but never applied. Each handler manually called validate_token(), so any new endpoint that forgot would be publicly accessible. Applied the middleware as route_layer on all /worker/ routes, removed manual auth from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps 0.0.0.0 since containers reach host via docker bridge, not loopback). Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining Implements the 4-phase plan for overhauling the web gateway chat: - Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below - Phase 2: Cursor-based history pagination with infinite scroll - Phase 3: NEAR AI previous_response_id chaining (delta-only messages), with fallback to full history on chain errors, and DB persistence of chain state across restarts - Phase 4: SSE thread isolation (events filtered by thread_id) Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Add per-request HTTP timeout to WASM host, redact credentials in errors Three fixes for WASM channel reliability: 1. Per-request timeout: Add optional timeout-ms parameter to http-request in both channel and tool WIT interfaces. Telegram long-poll now specifies 35s (outliving the 30s server-side hold), while regular API calls use the 30s default. Fixes the triple-30s timeout race that caused polling failures. 2. Credential redaction: reqwest::Error includes the full URL (with injected bot tokens) in its Display output. Scrub credential values from error messages before logging or returning to WASM. 3. Webhook route registration: Remove tunnel URL gate so webhook routes are always available when webhook channels exist, not only when TUNNEL_URL is configured. Co-Authored-By: Claude Opus 4.6 <[email protected]> * chore: Fix clippy warnings in WASM tools and channels - slack channel: allow dead_code on signing_secret_name (forward compat field) - gmail tool: use div_ceil() instead of manual (n+2)/3 - google-calendar tool: extract CreateEventParams/UpdateEventParams structs to fix too-many-arguments warnings Co-Authored-By: Claude Opus 4.6 <[email protected]> * Fix approval flow * fix: Rebuild bundled telegram.wasm with updated WIT interface The bundled WASM binary must match the host's WIT definition. Previous binary was compiled against the old 4-arg http-request; this rebuild includes the new timeout-ms parameter. Co-Authored-By: Claude Opus 4.6 <[email protected]> * refactor: Load WASM channels from disk instead of bundling in binary Remove include_bytes! embedding of telegram.wasm. Channels are now loaded from their build output directories (channels-src/<name>/target/) during onboarding, then from ~/.ironclaw/channels/ at runtime. - bundled.rs: locate_channel_artifacts() finds WASM + capabilities from build output; IRONCLAW_CHANNELS_SRC env var overrides the default path - available_channel_names(): only lists channels with build artifacts - bundled_channel_names(): lists all known channels (manifest) - Setup wizard uses available_channel_names() to offer installable channels - Add *.wasm to .gitignore, remove tracked telegram.wasm Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Persist gateway auth token, fix thread hydration race, polish auth screen Three web gateway UX fixes: 1. Token persistence: Store auth token in sessionStorage so refreshing the page doesn't force re-authentication. Hide the auth screen immediately when a saved token exists to prevent flash. 2. Thread hydration: Remove the !msgs.is_empty() bail-out in maybe_hydrate_thread so that even brand-new (empty) assistant threads get hydrated with their correct DB UUID. Previously resolve_thread would mint a fresh UUID, causing messages to land in the wrong conversation and duplicate threads to appear. 3. Auth screen: Redesign as a centered card with brand, tagline, labeled input, and hint text. Also adds 34 new tests covering session/thread lifecycle, thread resolution isolation (user, channel, external ID), hydration edge cases, serialization round-trips, approval flows, and stale mapping recovery. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix: Use bindgen! for WASM tool wrapper, add dev tool loading Three changes: 1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen! instead of manual linker.root().func_wrap(). This fixes the "component imports instance 'near:agent/host', but a matching implementation was not found in the linker" error. All 6 host functions (log, now-millis, workspace-read, http-request, secret-exists, tool-invoke) are now properly registered under the near:agent/host namespace. Also adds WASI support, credential injection, and leak detection for HTTP requests made by WASM tools. 2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the loader now also scans tools-src/*/target/wasm32-wasip2/release/ for build artifacts that are newer than installed copies. This means during development you just rebuild the WASM and restart the host; no manual copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir. 3. Wire up load_dev_tools() in main.rs alongside the existing load_from_dir() call. Co-Authored-By: Claude Opus 4.6 <[email protected]> * feat: Wire main startup and CLI to use DB-backed settings main.rs now reloads Config from the database after connecting, attaches the store to the session manager for dual-write tokens, and loads MCP servers from DB instead of disk. ExtensionManager and MCP CLI commands use DB when available with disk fallback. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
Elliot BraemandGitHub
|
202665a55c |
Fixes build, adds missing sse event and correct command (#11)
* add missing type * prune * readme * minor * update to .ironclaw |
||
|
|
e6725eb6d9 |
feat: Improve CLI (#5)
* Start working on improved CLI * Add tool result previews, boxed approval card, and polished help screen REPL iteration 2: styled /help with grouped sections, box-drawing approval card with colored params, dim separator before responses, inline tool output previews via new StatusUpdate::ToolResult variant. Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]> |
||
|
|
642c320b13 |
Add WebSocket gateway and control plane (#8)
* Add WebSocket gateway and control plane endpoint Adds bidirectional WebSocket transport to the web gateway alongside the existing SSE stream. Clients can send messages, approvals, and pings over a single persistent connection at /api/chat/ws. - Enable axum `ws` feature for built-in WebSocket support - Add WsClientMessage/WsServerMessage types with tagged JSON protocol - Add subscribe_raw() to SseManager for non-SSE consumers - Create ws.rs with connection handler (split sender/receiver tasks) - Add WsConnectionTracker for active connection counting - Add /api/gateway/status control plane endpoint (SSE + WS counts) - 35 new tests covering message types, broadcast, and handler logic https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b * Add e2e WebSocket gateway integration tests - Add tokio-tungstenite dev-dependency for WebSocket client in tests - Update start_server to return actual bound SocketAddr (enables port 0) - Add 10 e2e tests covering full HTTP upgrade → WebSocket → message flow: ping/pong, message routing to agent, broadcast event delivery, connection tracking, invalid message handling, auth rejection, gateway status endpoint, and multi-event sequencing https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b --------- Co-authored-by: Claude <[email protected]> |
||
|
Illia PolosukhinandClaude Opus 4.6
|
bf3b8b339f |
Fix MCP tool calls, approval loop, shutdown, and improve web UI
- Fix MCP tool schema deserialization: rename input_schema to match protocol's camelCase inputSchema, so models receive actual parameter schemas instead of empty defaults - Fix conversation history: add tool_calls field to ChatMessage and include assistant message with tool_calls before tool results, as required by OpenAI-compatible APIs - Fix approval loop: pass resume_after_tool flag to run_agentic_loop so the "force tool use" heuristic doesn't re-trigger after approval - Fix shutdown: add Submission::Quit, Ctrl+C signal handler, and graceful shutdown flow - Fix MCP activate button: auto-attempt auth flow when activation fails due to missing authentication - Add inline approval cards in chat via SSE ApprovalNeeded events - Add markdown rendering in chat (marked.js) with proper streaming - Add structured fields to log entries (key=value pairs from tracing) - Collapse log entries to single line with click-to-expand Co-Authored-By: Claude Opus 4.6 <[email protected]> |
||
|
Illia Polosukhin
|
a351711312 | Adding web UI |