Commit Graph
33 Commits
Author SHA1 Message Date
Illia PolosukhinandClaude Opus 4.6 2e62d71567 feat: Add NEAR key management with transaction signing and policy engine
Implements hybrid-custody NEAR key management where the agent holds scoped
function-call keys for routine operations while high-value operations require
explicit user approval through the existing channel approval flow.

Core infrastructure:
- Ed25519 key generation/import via ed25519-dalek (not near-crypto)
- AES-256-GCM encrypted storage via existing SecretsStore
- Hand-rolled borsh-serializable NEAR transaction types
- NEP-413 intent signing and MPC chain signature support
- Configurable policy engine with transaction analysis pipeline
- Daily spend tracking with automatic midnight UTC reset
- Encrypted backup/restore with Argon2id KDF
- CLI subcommands: generate, import, list, info, remove, export, policy, backup, restore
- NEAR ed25519 secret key leak detection (Critical/Block)
- WASM sign-payload host function (keys never enter WASM memory)
- KeyManager wired into AgentDeps for agent-wide access

Security invariants: private keys never reach the LLM or WASM boundary,
signing happens in host Rust code with Zeroize on drop, every transaction
is analyzed before signing, most-restrictive policy rule wins.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-09 16:24:47 -08:00
6831a54793 Onboarding: select bundled Telegram channel and auto-install (#3)
Co-authored-by: Firat Sertgoz <[email protected]>
2026-02-07 06:56:01 +00:00
Illia PolosukhinandClaude Opus 4.6 bf3b8b339f Fix MCP tool calls, approval loop, shutdown, and improve web UI
- Fix MCP tool schema deserialization: rename input_schema to match
  protocol's camelCase inputSchema, so models receive actual parameter
  schemas instead of empty defaults
- Fix conversation history: add tool_calls field to ChatMessage and
  include assistant message with tool_calls before tool results, as
  required by OpenAI-compatible APIs
- Fix approval loop: pass resume_after_tool flag to run_agentic_loop
  so the "force tool use" heuristic doesn't re-trigger after approval
- Fix shutdown: add Submission::Quit, Ctrl+C signal handler, and
  graceful shutdown flow
- Fix MCP activate button: auto-attempt auth flow when activation
  fails due to missing authentication
- Add inline approval cards in chat via SSE ApprovalNeeded events
- Add markdown rendering in chat (marked.js) with proper streaming
- Add structured fields to log entries (key=value pairs from tracing)
- Collapse log entries to single line with click-to-expand

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-06 18:04:44 -08:00
Illia PolosukhinandClaude Opus 4.6 2cdd04a359 Add auth mode, fix MCP token handling, and parallelize startup loading
Auth mode: when a tool requires an API key, the thread enters a special
mode where the next user message is routed directly to the credential
store, bypassing logs, turns, history, and compaction entirely. This
prevents tokens from leaking into debug output or persistent storage.

Fix MCP auth: auth_mcp now actually uses the token parameter (was
ignored as _token) and falls back to manual token entry when OAuth
and DCR are both unsupported.

Parallel loading: WASM tools, WASM channels, and MCP servers now load
concurrently at startup. Within each loader, individual items also
load in parallel (join_all for WASM, JoinSet for MCP servers).

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-06 13:35:17 -08:00
Illia Polosukhin cb987321a9 Merge remote-tracking branch 'origin/main' 2026-02-06 08:53:27 -08:00
Illia PolosukhinandClaude Opus 4.6 4f8fd4ad5f Reject workspace paths in write_file, force LLM to use memory_write
write_file now detects workspace files (HEARTBEAT.md, MEMORY.md,
SOUL.md, etc.) and daily/context/ prefixes, returning an error that
tells the LLM to use memory_write with the correct target instead.
This prevents the LLM from writing workspace data to the local
filesystem when it should go to the database.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-05 20:15:53 -08:00
Illia PolosukhinandClaude Opus 4.6 ae3c86a7ea Add in-chat extension discovery, auth, and activation system
Introduces a unified extension abstraction over MCP servers and WASM tools
with six agent-callable tools (tool_search, tool_install, tool_auth,
tool_activate, tool_list, tool_remove) so users can add capabilities
conversationally without CLI commands.

Includes built-in registry of 11 MCP servers, online discovery via URL
probing and GitHub search, OAuth 2.1 flows for MCP servers, and manual
token auth for WASM tools.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-05 20:11:29 -08:00
Illia PolosukhinandClaude Opus 4.6 7fcc2279cc Route HEARTBEAT writes to workspace DB and broadcast notifications
- Add dedicated "heartbeat" target in memory_write tool so the LLM
  routes HEARTBEAT.md writes to the database instead of the filesystem
- Update tool description to clarify it's database-backed storage
- Broadcast heartbeat notifications to all channels when no explicit
  notify target is configured, instead of silently logging them

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-02-05 20:02:33 -08:00
Illia Polosukhin 1c9f9db420 Merge remote-tracking branch 'origin/main' 2026-02-05 17:52:10 -08:00
Illia PolosukhinandClaude Opus 4.5 974bc8d407 Add hosted MCP server support with OAuth 2.1 and token refresh
Enables connecting to official MCP servers (like Notion) instead of
building custom WASM tools. Uses OAuth 2.1 with PKCE and supports
Dynamic Client Registration for zero-config authentication.

Key features:
- OAuth 2.1 flow with PKCE for secure browser-based auth
- Dynamic Client Registration (DCR) for servers without pre-configured clients
- Automatic token refresh on 401 responses
- Session management with Mcp-Session-Id headers
- SSE streaming response handling

New CLI commands:
- `mcp add <name> <url>` - Add an MCP server
- `mcp remove <name>` - Remove an MCP server
- `mcp list` - List configured servers
- `mcp auth <name>` - Authenticate with a server
- `mcp test <name>` - Test connection

Also removes the Notion WASM tool example since it's superseded by the
Notion MCP server which provides 13 official tools.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 17:43:26 -08:00
Illia Polosukhin 3e6dfb8409 Addressing vareity of security issues 2026-02-05 10:40:24 -08:00
Illia PolosukhinandClaude Opus 4.5 598dd43b1c Rebrand to IronClaw with security-first mission
Renamed project from "near-agent" to "ironclaw" throughout the codebase.
Updated documentation to emphasize the core philosophy:
- Your data stays yours (local, encrypted, no telemetry)
- Self-expanding capabilities (build tools on the fly)
- Defense in depth (WASM sandbox, prompt injection defense)
- Always on user's side

Key changes:
- Package name: near-agent -> ironclaw
- Config paths: ~/.near-agent/ -> ~/.ironclaw/
- Database name in docs: near_agent -> ironclaw
- CLI binary: near-agent -> ironclaw
- Log filters: RUST_LOG=near_agent -> RUST_LOG=ironclaw
- All user-facing strings (welcome messages, help text, etc.)

Preserved for compatibility:
- HKDF salt "near-agent-secrets-v1" (changing would break existing secrets)
- WIT interface names (near::agent::*)
- NEAR AI provider config (NEARAI_* env vars)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 01:10:40 -08:00
Illia PolosukhinandClaude Opus 4.5 2486065fa7 Fix build_software tool stuck in planning mode loop
The builder would get stuck when the LLM returned JSON specs or planning
text instead of tool calls. The loop would continue for all iterations
without making progress, eventually timing out.

Changes:
- Make initial prompt directive: "Use write_file NOW" instead of passive
  "Start by creating the project structure"
- Add consecutive_text_responses counter to detect stuck state
- Fail fast after 2 consecutive text-only responses with clear error
- Send strong nudge on first text response: "STOP. Call write_file..."
- Reset counter once tools have been executed (completion phase)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 00:59:44 -08:00
Illia PolosukhinandClaude Opus 4.5 aec42aceda Fix Telegram Markdown formatting and clarify tool/memory distinctions
- Add escape_telegram_markdown() to handle underscores in tool names
  (e.g., build_software was breaking Telegram's Markdown parser)
- Use Telegram-compatible *bold* syntax instead of **bold**
- Clarify workspace memory vs filesystem tool descriptions to prevent
  LLM from using read_file on memory_tree paths
- Update build_software to strongly prefer Rust WASM for agent tools
- Rewrite WASM tool template to use Component Model with wit_bindgen
  instead of outdated extern "C" approach

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 00:37:19 -08:00
Illia PolosukhinandClaude Opus 4.5 e6946172f7 Apply Telegram channel learnings to WhatsApp implementation
- Fix metadata flow: store sender_phone for response routing
- Add credential injection in headers (Bearer {WHATSAPP_ACCESS_TOKEN})
- Add secret_validated check for webhook defense in depth
- Add status message filtering to prevent loops
- Add proper WhatsApp API error response parsing
- Create whatsapp.capabilities.json with setup/secrets/rate limits
- Add docs/BUILDING_CHANNELS.md with patterns and examples

Also fix UTF-8 truncation bugs across codebase:
- wrapper.rs: content preview, response body, webhook body logging
- agent_loop.rs: params truncation for approval display
- shell.rs: truncate_for_error() helper

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:32:23 -08:00
Illia Polosukhin ce87ec1dbe Merge remote-tracking branch 'origin/main' 2026-02-04 22:11:37 -08:00
Illia PolosukhinandClaude Opus 4.5 7baf9e379d Replace hardcoded intent patterns with job tools
Remove the brittle natural language pattern matching from the router
and add job management tools to the normal tool registry instead.

- Add job tools: create_job, list_jobs, job_status, cancel_job
- Router now only handles explicit /commands
- Natural language goes through agentic loop with all tools
- LLM naturally picks appropriate tools based on user intent
- Share ContextManager between job tools and Agent

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:52 -08:00
Illia PolosukhinandClaude Opus 4.5 a39f5aa1a4 Add Docker execution sandbox for secure shell command isolation
Implements a general-purpose Docker sandbox (inspired by Codex) that provides:
- Container isolation for shell commands with ephemeral containers
- HTTP proxy for network access control with domain allowlist
- Credential injection by proxy (secrets never enter containers)
- Three security policies: ReadOnly, WorkspaceWrite, FullAccess
- Resource limits (memory, CPU, timeout enforcement)

Key components:
- SandboxManager: Main entry point coordinating proxy and containers
- NetworkProxy: HTTP proxy validating requests and injecting credentials
- ContainerRunner: Docker lifecycle management via bollard
- DomainAllowlist: Pattern matching for allowed network destinations

The ShellTool now routes commands through the sandbox when enabled,
with automatic fallback to direct execution if Docker is unavailable.

Configuration via SANDBOX_ENABLED, SANDBOX_POLICY, SANDBOX_TIMEOUT_SECS,
SANDBOX_MEMORY_LIMIT_MB, SANDBOX_IMAGE, SANDBOX_EXTRA_DOMAINS env vars.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:52 -08:00
Illia PolosukhinandClaude Opus 4.5 7955c9742e Add Telegram webhook support with credential injection
Enable instant message delivery for Telegram via webhooks instead of polling.

Key changes:
- Add tunnel URL configuration for local development (ngrok, cloudflare)
- Auto-register webhook with Telegram API on startup using setWebhook
- Implement webhook secret validation via X-Telegram-Bot-Api-Secret-Token header
- Add credential injection for bot token via URL placeholder substitution
- Fix metadata preservation in respond() to route replies correctly
- Fix serde flatten with Option<T> issue in capabilities schema parsing

The credential injection pattern replaces {TELEGRAM_BOT_TOKEN} placeholders
in URLs with the actual token from the secrets store, keeping credentials
out of WASM module memory until the HTTP request is made.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 21:19:42 -08:00
Illia Polosukhin 74d94d33b0 Implementing channels to be handled in wasm 2026-02-03 14:57:27 -08:00
Illia Polosukhin 2c26ba8431 Support non interactive mode and model selection 2026-02-03 14:47:26 -08:00
Illia Polosukhin 7210470544 Wiring more 2026-02-03 10:08:06 -08:00
Illia Polosukhin c9ebb117ab Adding builder capability 2026-02-03 08:36:55 -08:00
Illia Polosukhin 343782524f Load tools at launch 2026-02-03 00:48:22 -08:00
Illia PolosukhinandClaude Opus 4.5 4ae59ef52c Fix TuiChannel integration and enable in main.rs
- Store event_tx in Arc<Mutex<>> so respond() can send to TUI
- Fix run_event_loop to take owned receiver
- Switch main.rs from SimpleCliChannel to TuiChannel
- Add proper terminal cleanup on TUI error

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:47:41 -08:00
Illia PolosukhinandClaude Opus 4.5 09032d69cb Integrate Codex patterns: task scheduler, TUI, sessions, compaction
Add Codex-inspired patterns for improved agent architecture:

**Task Scheduler (Phase 1 & 4)**
- New Task enum with Job, ToolExec, Background variants
- TaskHandler trait for custom background tasks
- Scheduler.spawn_subtask() and spawn_batch() for parallel execution
- Worker executes multiple tools in parallel via futures::join_all

**Tool Approval System (Phase 6)**
- Tool.requires_approval() method (default false for sandboxed tools)
- HttpTool marked as requiring approval (external network)
- MCP protocol annotations: destructive_hint, side_effects_hint

**Ratatui TUI CLI (Phase 2)**
- Replace blocking stdin with event-driven TUI
- ChatComposer with history navigation and tab completion
- ApprovalOverlay modal with y/n/a keyboard shortcuts
- Raw mode with proper terminal cleanup

**Session/Turn Model (Phase 3)**
- Session, Thread, Turn structs for conversation tracking
- Submission enum for user input, approvals, undo, interrupt
- UndoManager with checkpoint-based undo/redo

**Context Compaction (Phase 5)**
- ContextMonitor with token estimation and threshold checks
- CompactionStrategy: Summarize, Truncate, MoveToWorkspace
- ContextCompactor writes summaries to workspace daily logs

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:41:57 -08:00
Illia PolosukhinandClaude Opus 4.5 32bfd24154 Add WASM sandbox secure API extension
Extends the WASM sandbox with HTTP API capabilities, secrets management,
tool aliasing, and leak detection. Key security principle: WASM never
sees credentials, injection happens at host boundary.

New modules:
- secrets: AES-256-GCM encrypted storage with HKDF key derivation
- leak_detector: Aho-Corasick + regex pattern matching for secret exfiltration
- capabilities: Extended capability system (HTTP, ToolInvoke, Secrets)
- allowlist: HTTP endpoint validation with glob patterns
- credential_injector: Host-boundary credential injection
- rate_limiter: Sliding window per-tool rate limiting
- storage: WASM binary storage with BLAKE3 integrity verification

Leak detection happens at two points:
1. Before HTTP request (prevents exfiltration via URL/headers/body)
2. After response (prevents exposure in outputs returned to WASM)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:22:52 -08:00
Illia Polosukhin aea3f47f8b Implementing WASM runtime 2026-02-02 22:47:02 -08:00
Illia PolosukhinandClaude Opus 4.5 b316a7f4f5 Compact memory_tree output format
- Directories indicated by trailing `/` instead of is_directory field
- Files are plain strings, dirs with children are objects
- Remove redundant name/path (just show name in tree)
- Remove preview and updated_at (use memory_read if needed)
- Output is just the tree array, no wrapper object

Example: ["MEMORY.md", {"daily/": ["2024-01-15.md"]}, "projects/"]

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:49:32 -08:00
Illia PolosukhinandClaude Opus 4.5 3f7624eefc Replace memory_list with memory_tree tool
The memory_tree tool provides a hierarchical view of the workspace
with configurable depth (default 1). This is more useful for
exploring nested directory structures.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:44:07 -08:00
Illia PolosukhinandClaude Opus 4.5 3718cfa767 Simplify workspace to path-based storage, remove legacy code
- Consolidate all migrations into V1__initial.sql
- Replace DocType enum with flexible path-based file storage
- Add list_workspace_files SQL function for directory listing
- Update memory tools for path-based API (memory_read, memory_write,
  memory_search, memory_list)
- Remove unused OpenAI/Anthropic providers (NEAR AI only)
- Simplify config to remove multi-provider support
- Update CLAUDE.md documentation

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:38:53 -08:00
Illia PolosukhinandClaude Opus 4.5 4e238e60ac Add workspace and memory system (OpenClaw-inspired)
Implements persistent memory for agents with hybrid search:

- Database-backed workspace with PostgreSQL (not filesystem)
- Memory documents: MEMORY.md, daily logs, identity files
- Chunked content with FTS (tsvector) + vector (pgvector) indexes
- Reciprocal Rank Fusion (RRF) for hybrid search combining BM25 and semantic
- Memory tools: memory_search, memory_write, memory_read
- Proactive heartbeat system for periodic execution (30 min default)
- OpenAI embeddings provider (text-embedding-3-small)

Key patterns from OpenClaw:
- "Memory is files, not RAM" - explicit persistence required
- Two-tier memory: daily logs (raw) + curated MEMORY.md
- Session isolation via user_id/agent_id scoping

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:18:47 -08:00
Illia Polosukhin 8c38566378 Initial implementation of the agent framework 2026-02-02 20:41:05 -08:00