Commit Graph
104 Commits
Author SHA1 Message Date
Illia PolosukhinandClaude Opus 4.5 aec42aceda Fix Telegram Markdown formatting and clarify tool/memory distinctions
- Add escape_telegram_markdown() to handle underscores in tool names
  (e.g., build_software was breaking Telegram's Markdown parser)
- Use Telegram-compatible *bold* syntax instead of **bold**
- Clarify workspace memory vs filesystem tool descriptions to prevent
  LLM from using read_file on memory_tree paths
- Update build_software to strongly prefer Rust WASM for agent tools
- Rewrite WASM tool template to use Component Model with wit_bindgen
  instead of outdated extern "C" approach

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 00:37:19 -08:00
Illia PolosukhinandClaude Opus 4.5 9edc666ee3 Simplify Telegram channel config with host-injected tunnel/webhook settings
The WASM channel no longer needs its own polling_enabled/tunnel_url settings.
Instead, the host injects tunnel_url and webhook_secret into the channel config
at runtime before start() is called.

Changes:
- Add update_config() method to WasmChannel for runtime config injection
- Simplify TelegramConfig to only have bot_username, respond_to_all_group_messages
- Host injects tunnel_url (from Settings) and webhook_secret (from secrets store)
- Channel checks if tunnel_url is present to determine webhook vs polling mode
- Add delete_webhook() for clean transition to polling mode when no tunnel

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-05 00:05:32 -08:00
Illia PolosukhinandClaude Opus 4.5 e6946172f7 Apply Telegram channel learnings to WhatsApp implementation
- Fix metadata flow: store sender_phone for response routing
- Add credential injection in headers (Bearer {WHATSAPP_ACCESS_TOKEN})
- Add secret_validated check for webhook defense in depth
- Add status message filtering to prevent loops
- Add proper WhatsApp API error response parsing
- Create whatsapp.capabilities.json with setup/secrets/rate limits
- Add docs/BUILDING_CHANNELS.md with patterns and examples

Also fix UTF-8 truncation bugs across codebase:
- wrapper.rs: content preview, response body, webhook body logging
- agent_loop.rs: params truncation for approval display
- shell.rs: truncate_for_error() helper

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:32:23 -08:00
Illia Polosukhin ce87ec1dbe Merge remote-tracking branch 'origin/main' 2026-02-04 22:11:37 -08:00
Illia PolosukhinandClaude Opus 4.5 c14911009f Add WhatsApp channel WASM module
Implements the sandboxed-channel WIT interface for WhatsApp Cloud API:
- Webhook verification (GET with hub.mode=subscribe)
- Incoming message handling (POST webhooks)
- Outgoing responses via Graph API
- Parses WhatsApp webhook payload format

Built and tested in Docker sandbox with wasm32-wasip2 target.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:53 -08:00
Illia Polosukhin 9d3f993b4e Docker file for sandbox 2026-02-04 22:09:52 -08:00
Illia PolosukhinandClaude Opus 4.5 7baf9e379d Replace hardcoded intent patterns with job tools
Remove the brittle natural language pattern matching from the router
and add job management tools to the normal tool registry instead.

- Add job tools: create_job, list_jobs, job_status, cancel_job
- Router now only handles explicit /commands
- Natural language goes through agentic loop with all tools
- LLM naturally picks appropriate tools based on user intent
- Share ContextManager between job tools and Agent

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:52 -08:00
Illia PolosukhinandClaude Opus 4.5 27ffc12f6c Fix router test to match intentional job creation patterns
The test expected "Can you create a website for me?" to route as CreateJob,
but the extract_intent logic intentionally requires explicit job creation
patterns (containing both "create" and "job") to avoid capturing general
conversation as job requests.

Updated test to verify:
- "create job: ..." routes to CreateJob
- Messages with both "create" and "job" route to CreateJob
- General requests without explicit "job" fall through to Chat

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:52 -08:00
Illia PolosukhinandClaude Opus 4.5 a39f5aa1a4 Add Docker execution sandbox for secure shell command isolation
Implements a general-purpose Docker sandbox (inspired by Codex) that provides:
- Container isolation for shell commands with ephemeral containers
- HTTP proxy for network access control with domain allowlist
- Credential injection by proxy (secrets never enter containers)
- Three security policies: ReadOnly, WorkspaceWrite, FullAccess
- Resource limits (memory, CPU, timeout enforcement)

Key components:
- SandboxManager: Main entry point coordinating proxy and containers
- NetworkProxy: HTTP proxy validating requests and injecting credentials
- ContainerRunner: Docker lifecycle management via bollard
- DomainAllowlist: Pattern matching for allowed network destinations

The ShellTool now routes commands through the sandbox when enabled,
with automatic fallback to direct execution if Docker is unavailable.

Configuration via SANDBOX_ENABLED, SANDBOX_POLICY, SANDBOX_TIMEOUT_SECS,
SANDBOX_MEMORY_LIMIT_MB, SANDBOX_IMAGE, SANDBOX_EXTRA_DOMAINS env vars.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 22:09:52 -08:00
Illia Polosukhin 0596a6c847 Webhook and polling integrations for channels 2026-02-04 22:02:13 -08:00
Illia PolosukhinandClaude Opus 4.5 7955c9742e Add Telegram webhook support with credential injection
Enable instant message delivery for Telegram via webhooks instead of polling.

Key changes:
- Add tunnel URL configuration for local development (ngrok, cloudflare)
- Auto-register webhook with Telegram API on startup using setWebhook
- Implement webhook secret validation via X-Telegram-Bot-Api-Secret-Token header
- Add credential injection for bot token via URL placeholder substitution
- Fix metadata preservation in respond() to route replies correctly
- Fix serde flatten with Option<T> issue in capabilities schema parsing

The credential injection pattern replaces {TELEGRAM_BOT_TOKEN} placeholders
in URLs with the actual token from the secrets store, keeping credentials
out of WASM module memory until the HTTP request is made.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 21:19:42 -08:00
Illia PolosukhinandClaude Opus 4.5 4ab20ff939 Move setup wizard credentials to database storage
Changes setup wizard to save channel secrets (Telegram bot token, HTTP
webhook secret) to PostgreSQL via SecretsStore instead of files.

This enables the WASM channel credential injector to find and inject
the secrets properly, since it reads from the database.

Requires:
- DATABASE_URL to be set
- SECRETS_MASTER_KEY (will generate and display if not set)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 09:49:11 -08:00
Illia PolosukhinandClaude Opus 4.5 c7f0e8014d Add interactive setup wizard for first-run configuration
Introduces `near-agent setup` command that guides users through:
- NEAR AI authentication (reuses existing OAuth flow)
- Model selection (fetches from API or shows defaults)
- Channel configuration (HTTP webhook, Telegram)

Features:
- First-run detection: auto-runs wizard if no session exists
- Respects existing settings: shows current model with keep/change option
- Saves channel secrets to ~/.near-agent/secrets/ with 0600 permissions
- Validates Telegram bot tokens via API before saving

Also fixes default NEARAI_BASE_URL to use cloud-api.near.ai (api.near.ai
returns 410 Gone).

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 09:42:41 -08:00
Illia PolosukhinandClaude Opus 4.5 1605939e2a Add Telegram Bot API channel as WASM module
Implements a loadable WASM channel for Telegram following the existing
Slack channel pattern:

- Webhook-based message receiving at /webhook/telegram
- Private chat and group chat support (with @mention filtering)
- Reply threading via reply_to_message_id
- User name extraction from Telegram user objects
- Bot token injection by host (never exposed to WASM)

Files:
- channels-src/telegram/src/lib.rs - Main implementation
- channels-src/telegram/Cargo.toml - Dependencies
- channels-src/telegram/telegram.capabilities.json - Permissions
- channels-src/telegram/build.sh - Build script

To use: copy telegram.wasm and telegram.capabilities.json to
~/.near-agent/channels/ and configure telegram_bot_token secret.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-04 09:16:03 -08:00
Illia PolosukhinandClaude Opus 4.5 be18812376 Add OpenClaw feature parity tracking matrix
Comprehensive comparison of IronClaw vs OpenClaw features to enable
coordinated development. Includes status indicators, priority levels,
owner fields for claiming work, and documented architectural deviations.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 22:06:48 -08:00
Illia PolosukhinandClaude Opus 4.5 7ef6362d83 Add Chat Completions API support and expand REPL debugging
- Add NearAiChatProvider using /v1/chat/completions endpoint with API key auth
- Add NEARAI_API_KEY and NEARAI_API_MODE config options
- Auto-detect API mode from presence of API key
- Keep existing Responses API (NearAiProvider) for session-based auth
- Fix response parsing to accept input_text/output_text/text content types
- Expand REPL with /help, /debug toggle, colored output
- Better tool status display (dots vs verbose based on debug mode)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 21:55:05 -08:00
Illia Polosukhin 74d94d33b0 Implementing channels to be handled in wasm 2026-02-03 14:57:27 -08:00
Illia Polosukhin 2c26ba8431 Support non interactive mode and model selection 2026-02-03 14:47:26 -08:00
Illia PolosukhinandClaude Opus 4.5 2cc9aed364 Implement tool approval, fix tool definition refresh, and wire embeddings
This commit addresses three critical issues from code review:

1. Tool approval enforcement: Tools declaring requires_approval() (shell,
   http, file write/patch, build_software) now gate execution. Adds
   PendingApproval struct, session-scoped auto-approved tools set, and
   approval flow with yes/no/always commands.

2. Tool definition refresh: Tool definitions now refresh each iteration
   in both chat and job loops, so newly built tools become visible
   immediately within the same session.

3. Worker tool call handling: Changed respond() to respond_with_tools()
   when select_tools returns empty, properly executing tool calls instead
   of formatting them as text.

Also includes prior work from the plan:
- Wire embeddings provider (OpenAI + NEAR AI) to workspace
- Load workspace system prompt (identity files) into LLM context
- Route heartbeat notifications through channel manager
- Enable auto-context compaction when threshold exceeded
- Refactor to config structs (AgentDeps, WorkerDeps, LlmCallRecord)
- Fix clippy warnings (saturating_sub, too_many_arguments)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 11:34:10 -08:00
Illia Polosukhin 8af48390a9 Tool use 2026-02-03 10:50:20 -08:00
Illia Polosukhin 7210470544 Wiring more 2026-02-03 10:08:06 -08:00
Illia PolosukhinandClaude Opus 4.5 235f6aae18 Add heartbeat integration, planning phase, and auto-repair
- Add HeartbeatConfig for proactive periodic execution with channel notifications
- Add use_planning option to Worker for ActionPlan generation before tool execution
- Implement tool failure tracking in database (V3 migration)
- Add auto-repair via Builder for broken WASM tools in self_repair.rs
- Record tool failures in Worker for self-repair tracking
- Update .env.example with new configuration options

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 09:32:01 -08:00
Illia Polosukhin 2df4a4f5f0 Login flow 2026-02-03 09:20:26 -08:00
Illia Polosukhin dedda9c51d Extend support for session management 2026-02-03 08:59:59 -08:00
Illia Polosukhin c9ebb117ab Adding builder capability 2026-02-03 08:36:55 -08:00
Illia Polosukhin 343782524f Load tools at launch 2026-02-03 00:48:22 -08:00
Illia PolosukhinandClaude Opus 4.5 575269546b Fix multiline message rendering in TUI
- Switch from List to Paragraph widget for message display
- Split message content by newlines and render each line separately
- Auto-scroll to show most recent messages
- Add empty line between messages for readability
- Wrap long lines properly

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:38:19 -08:00
Illia PolosukhinandClaude Opus 4.5 066df79595 Parse NEAR AI alternative response format with output field
The API returns a different JSON structure with output as a Value.
Added extract_text_from_output() to handle various output formats:
- String: return directly
- Array with text/content fields: extract and join
- Object with text/content/message fields: extract

Both complete() and complete_with_tools() now handle this format.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:29:36 -08:00
Illia PolosukhinandClaude Opus 4.5 635aeb66a7 Handle NEAR AI plain text responses
The API sometimes returns plain text instead of JSON structure.
Now we detect this and extract the text content directly.

- Detect "Unexpected response format" errors with raw text
- Parse as JSON string if quoted, otherwise use raw
- Works for both complete() and complete_with_tools()

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:26:11 -08:00
Illia PolosukhinandClaude Opus 4.5 2b2f0d56c6 Disable mouse capture to allow text selection in TUI
- Remove EnableMouseCapture/DisableMouseCapture from terminal setup
- Users can now select and copy text using normal terminal selection
- We weren't using mouse events anyway

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:24:18 -08:00
Illia PolosukhinandClaude Opus 4.5 cd426b7e9d Add verbose logging to debug empty NEAR AI responses
- Log request body before sending
- Log full response text before parsing
- Log parse errors with full response content
- This helps debug why output array is empty

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:23:33 -08:00
Illia PolosukhinandClaude Opus 4.5 1ae64981d8 Improve NEAR AI response parsing for varying response formats
- Add direct text field support on NearAiOutputItem
- Accept both "output_text" and "text" content types
- Join all content items instead of taking just the first
- Better debug logging showing item.text field

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:18:34 -08:00
Illia PolosukhinandClaude Opus 4.5 42d6e87186 Show status/thinking messages in chat window, debug empty responses
- Add ThinkingMessage variant to AppEvent for chat-visible status
- Add set_thinking() and clear_thinking() to AppState
- Thinking messages show as system messages with spinner indicator
- Auto-clear thinking messages when agent response arrives
- Tool started/completed now shown in chat window
- Add debug logging to NEAR AI provider to diagnose empty responses
- Log raw response output when text extraction returns empty

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:17:40 -08:00
Illia Polosukhin 3d709eb253 Improving slack tool example 2026-02-03 00:16:48 -08:00
Illia PolosukhinandClaude Opus 4.5 11d3e816e5 Add timeout and logging to NEAR AI provider
- Add 2 minute timeout to HTTP client for LLM calls
- Add debug logging for requests and error logging for failures
- Prevents indefinite hangs when NEAR AI is slow or unavailable

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:13:40 -08:00
Illia PolosukhinandClaude Opus 4.5 7f9f0cd21e Add status updates to show agent thinking/processing state
- Add StatusUpdate enum with Thinking, ToolStarted, ToolCompleted, StreamChunk, Status variants
- Add send_status method to Channel trait (default no-op)
- Implement send_status in TuiChannel to show status in UI
- Add send_status to ChannelManager for routing to specific channels
- Update handle_message to send "Processing..." status for Chat/CreateJob
- Update handle_chat to send "Generating response..." and show errors

Now when a user sends a message, they see feedback that the agent is working.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:12:48 -08:00
Illia PolosukhinandClaude Opus 4.5 d1cb748914 Add CLI subcommands for WASM tool management
Introduces a new CLI module with subcommands for managing WASM tools:
- `tool install`: Build and install tools from source or .wasm files
- `tool list`: List installed tools with optional verbose output
- `tool remove`: Remove installed tools
- `tool info`: Show detailed tool information including capabilities

The install command supports building from Cargo source directories using
cargo-component, or installing pre-compiled .wasm files directly. It
auto-detects capabilities JSON sidecar files and validates them before
installation.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-03 00:05:12 -08:00
Illia PolosukhinandClaude Opus 4.5 9232e623e8 Fix TUI shutdown: send /shutdown message and handle in agent loop
When the TUI quits (Ctrl+D twice), it now:
1. Sends a "/shutdown" message through the channel before closing
2. Explicitly drops msg_tx to ensure channel closure

The agent loop now:
1. Returns Option<String> from handle_message (None = shutdown)
2. Handles /quit, /exit, /shutdown commands by returning None
3. Breaks out of the main loop on shutdown signal
4. Lists /quit in help menu

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:59:28 -08:00
Illia PolosukhinandClaude Opus 4.5 b52122a275 Remove SimpleCliChannel, add Ctrl+D twice quit, redirect logs to TUI
- Remove SimpleCliChannel (only TuiChannel remains)
- Add ctrl_d_pending flag to AppState for two-press quit behavior
- Implement Ctrl+D twice to quit (first press shows hint, second quits)
- Add TuiLogWriter with MakeWriter impl for tracing integration
- Create TuiChannel event channel upfront in new() so log_writer() works
- Configure tracing to send logs to TUI status line
- Any key press clears the Ctrl+D pending state

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:54:01 -08:00
Illia PolosukhinandClaude Opus 4.5 4ae59ef52c Fix TuiChannel integration and enable in main.rs
- Store event_tx in Arc<Mutex<>> so respond() can send to TUI
- Fix run_event_loop to take owned receiver
- Switch main.rs from SimpleCliChannel to TuiChannel
- Add proper terminal cleanup on TUI error

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:47:41 -08:00
Illia PolosukhinandClaude Opus 4.5 09032d69cb Integrate Codex patterns: task scheduler, TUI, sessions, compaction
Add Codex-inspired patterns for improved agent architecture:

**Task Scheduler (Phase 1 & 4)**
- New Task enum with Job, ToolExec, Background variants
- TaskHandler trait for custom background tasks
- Scheduler.spawn_subtask() and spawn_batch() for parallel execution
- Worker executes multiple tools in parallel via futures::join_all

**Tool Approval System (Phase 6)**
- Tool.requires_approval() method (default false for sandboxed tools)
- HttpTool marked as requiring approval (external network)
- MCP protocol annotations: destructive_hint, side_effects_hint

**Ratatui TUI CLI (Phase 2)**
- Replace blocking stdin with event-driven TUI
- ChatComposer with history navigation and tab completion
- ApprovalOverlay modal with y/n/a keyboard shortcuts
- Raw mode with proper terminal cleanup

**Session/Turn Model (Phase 3)**
- Session, Thread, Turn structs for conversation tracking
- Submission enum for user input, approvals, undo, interrupt
- UndoManager with checkpoint-based undo/redo

**Context Compaction (Phase 5)**
- ContextMonitor with token estimation and threshold checks
- CompactionStrategy: Summarize, Truncate, MoveToWorkspace
- ContextCompactor writes summaries to workspace daily logs

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:41:57 -08:00
Illia Polosukhin d047c23b2d Adding LICENSE 2026-02-02 23:28:43 -08:00
Illia PolosukhinandClaude Opus 4.5 4b147038db Add README with IronClaw branding
Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:24:17 -08:00
Illia PolosukhinandClaude Opus 4.5 32bfd24154 Add WASM sandbox secure API extension
Extends the WASM sandbox with HTTP API capabilities, secrets management,
tool aliasing, and leak detection. Key security principle: WASM never
sees credentials, injection happens at host boundary.

New modules:
- secrets: AES-256-GCM encrypted storage with HKDF key derivation
- leak_detector: Aho-Corasick + regex pattern matching for secret exfiltration
- capabilities: Extended capability system (HTTP, ToolInvoke, Secrets)
- allowlist: HTTP endpoint validation with glob patterns
- credential_injector: Host-boundary credential injection
- rate_limiter: Sliding window per-tool rate limiting
- storage: WASM binary storage with BLAKE3 integrity verification

Leak detection happens at two points:
1. Before HTTP request (prevents exfiltration via URL/headers/body)
2. After response (prevents exposure in outputs returned to WASM)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 23:22:52 -08:00
Illia PolosukhinandClaude Opus 4.5 45bbfa026d Wire database Store into agent loop
Persist jobs and actions to PostgreSQL using fire-and-forget pattern:
- Scheduler passes store to Worker, persists cancellations
- Worker persists job status changes and tool execution actions
- Agent persists new jobs on creation
- All DB writes use tokio::spawn to avoid blocking execution

Store remains optional to preserve --no-db mode.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 22:53:12 -08:00
Illia Polosukhin aea3f47f8b Implementing WASM runtime 2026-02-02 22:47:02 -08:00
Illia PolosukhinandClaude Opus 4.5 383fb21c07 Add workspace integration tests
- Create tests/workspace_integration.rs with 10 database tests
- Export MockEmbeddings for use in integration tests
- Tests cover: read/write, append, nested paths, delete, memory ops,
  daily log, FTS search, hybrid search, list_all, system_prompt

Requires: DATABASE_URL=postgres://localhost/near_agent_test

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 22:01:09 -08:00
Illia PolosukhinandClaude Opus 4.5 b316a7f4f5 Compact memory_tree output format
- Directories indicated by trailing `/` instead of is_directory field
- Files are plain strings, dirs with children are objects
- Remove redundant name/path (just show name in tree)
- Remove preview and updated_at (use memory_read if needed)
- Output is just the tree array, no wrapper object

Example: ["MEMORY.md", {"daily/": ["2024-01-15.md"]}, "projects/"]

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:49:32 -08:00
Illia PolosukhinandClaude Opus 4.5 3f7624eefc Replace memory_list with memory_tree tool
The memory_tree tool provides a hierarchical view of the workspace
with configurable depth (default 1). This is more useful for
exploring nested directory structures.

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:44:07 -08:00
Illia PolosukhinandClaude Opus 4.5 3718cfa767 Simplify workspace to path-based storage, remove legacy code
- Consolidate all migrations into V1__initial.sql
- Replace DocType enum with flexible path-based file storage
- Add list_workspace_files SQL function for directory listing
- Update memory tools for path-based API (memory_read, memory_write,
  memory_search, memory_list)
- Remove unused OpenAI/Anthropic providers (NEAR AI only)
- Simplify config to remove multi-provider support
- Update CLAUDE.md documentation

Co-Authored-By: Claude Opus 4.5 <[email protected]>
2026-02-02 21:38:53 -08:00