Files
optimclaw/src/setup/wizard.rs
T
ced83d5b4d feat: Sandbox jobs (#4)
* Orchestrating jobs and running them in sandboxes

* Fix heartbeat: dynamic max_tokens, empty content guard, notification fallback

- Query /v1/models API for context_length and set max_tokens to half
  (floor 4096) instead of hardcoded 1024; reasoning models like GLM-4.7
  need much larger budgets
- Guard against empty LLM content (reasoning models can burn all tokens
  on chain-of-thought and return content: null)
- Simplify notification routing: try configured channel first, fall back
  to broadcast_all so heartbeat alerts always reach someone
- Add ModelMetadata struct and model_metadata() to LlmProvider trait
- Refactor NearAiChatProvider::list_models into shared fetch_models()
- Add standalone test_heartbeat example for isolated debugging

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add job detail view with drill-down from jobs list

Click a job row to see full details across four sub-tabs:
Overview (metadata grid, description, state transitions timeline),
Actions (expandable tool call cards with input/output JSON),
Thinking (conversation messages styled by role), and
Files (embedded workspace tree browser).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Strip model-internal XML tags from LLM responses, fix Telegram parse_mode 400

Some models (GLM-4.7, etc.) emit <tool_call>tool_list</tool_call> in the
content field instead of using the OpenAI tool_calls array. This XML leaks
through to channels as text, and Telegram's Markdown parser chokes on the
underscores, returning 400 "can't parse entities".

Two fixes:
- Generalize clean_response() to strip <tool_call>, <function_call>,
  <tool_calls>, and pipe-delimited variants (<|tool_call|>) alongside
  the existing <thinking> tag stripping
- Add Telegram send_message helper with parse_mode fallback: try Markdown
  first, retry as plain text on "can't parse entities" 400 errors

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add SystemCommand submission type for thread-state-independent commands

System commands (/help, /model, /version, /tools, /ping, /debug) now
bypass thread-state checks and safety validation via a dedicated
Submission::SystemCommand variant. Previously these flowed through
process_user_input() which blocked them during Processing/AwaitingApproval
/Completed states.

- Add /model [name] for runtime model switching with provider validation
- Add active_model_name()/set_model() to LlmProvider trait with RwLock
  hot-swap in both NEAR AI providers
- Rewrite /help with aligned columns grouped by category
- Expand REPL tab-completion from 10 to 23 slash commands
- Remove REPL-local /help interception (now handled by agent)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Add per-tool execution timeouts, auto-create sandbox project dirs, serve built files

The sandbox e2e pipeline (agent -> container -> built website -> browsable URL)
was broken by three gaps: hardcoded 60s timeouts killed sandbox jobs that need
minutes, no auto-created project directory meant container output vanished, and
no HTTP route to browse the built files.

- Add `execution_timeout()` to the `Tool` trait (default 60s), replace all four
  hardcoded `Duration::from_secs(60)` call sites (agent_loop, worker, scheduler,
  worker/runtime) with the per-tool value
- Override to 660s in `RunInSandboxTool` (10 min polling + 60s buffer)
- Auto-create `~/.ironclaw/projects/{uuid}/` when no `project_dir` is specified,
  so every sandbox job gets a persistent bind mount
- Include `project_dir` and `browse_url` in sandbox tool output JSON
- Add `/projects/{id}` and `/projects/{id}/{path}` static file serving routes
  to the web gateway with path traversal protection and MIME type detection
- Add `mime_guess` dependency for content-type detection

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Apply cargo fmt to wizard.rs after merge

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Persist sandbox jobs in DB, fix web UI, unify job model

Sandbox container jobs were invisible to the web UI because they lived
only in ContainerJobManager's in-memory HashMap while the API queried
ContextManager. This persists them to the agent_jobs table and fixes
all six front-end bugs (empty job list, broken back button, empty
actions/thinking tabs, wrong files tab, stuck status, no persistence).

Key changes:
- V4 migration adds project_dir and user_id columns to agent_jobs
- Embedded migrations via refinery (no external CLI needed)
- SandboxJobRecord CRUD in Store with fire-and-forget DB writes
- Unified job_id: sandbox tool generates UUID, passes to ContainerJobManager
- Web API queries DB for sandbox jobs, merges with ContextManager direct jobs
- New endpoints: restart, project file list/read with path traversal protection
- Front-end: rebuild DOM on back navigation, sandbox-aware tabs, job cards in
  chat stream, source badges, restart button for failed/interrupted jobs
- Gateway defaults to enabled, prints Web UI URL on startup
- Stale jobs marked "interrupted" on restart for visibility and restartability

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Secure in-chat auth: tokens never touch the LLM or chat history

Remove the token parameter from tool_auth so the LLM cannot pass raw
API keys. Add dedicated REST (POST /api/chat/auth-token) and WebSocket
(auth_token) endpoints that route tokens directly to ext_mgr.auth(),
completely bypassing the message pipeline, turns, history, and compaction.

Web UI shows an auth card (password input + OAuth button) when the agent
enters auth mode, submitted via the dedicated endpoint. CLI auth mode
interception is unchanged (already secure).

New StatusUpdate::AuthRequired/AuthCompleted variants propagate through
all channels (SSE, WebSocket, REPL, WASM).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add Claude Code mode for sandbox jobs

Run Claude Code CLI inside Docker containers as an alternative to the
standard worker mode. The bridge spawns `claude -p` with stream-json
output, posts events to the orchestrator, and supports follow-up
prompts via `--resume`.

Key additions:
- `claude-bridge` CLI subcommand and ClaudeBridgeRuntime
- JobMode enum (Worker vs ClaudeCode) with per-mode container config
- Orchestrator endpoints for Claude events and prompt polling
- SSE event variants for real-time Claude Code streaming to frontend
- Claude Code sub-tab in web UI with terminal-style output and input bar
- Database migration for job_mode column and claude_code_events table
- ClaudeCodeConfig with env var support (CLAUDE_CODE_ENABLED, etc.)
- Mode parameter on run_in_sandbox tool schema

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Skip create_job tool when sandbox is enabled to prevent duplicate jobs

When sandbox mode is on, the LLM would call create_job (creating a
pending "direct" entry) then run_in_sandbox (creating a second "sandbox"
entry), producing two jobs in the list for a single user request.

Now register_job_tools() skips create_job when sandbox is enabled since
run_in_sandbox already creates tracked jobs. Also improved the
run_in_sandbox description to guide the LLM to use it directly and to
mention wait=false for async execution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Web gateway UI quality-of-life improvements

Phase 1: Send button disabled state to prevent double-sends, copy button
on code blocks, confirm() guards on destructive actions, SSE-driven job
list auto-refresh, log filters re-applied on tab switch, jobEvents memory
leak fix (cap at 500, cleanup after 60s).

Phase 2: Toast notification system replacing chat-based system messages,
memory search highlighting with centered snippets, keyboard shortcuts
(Ctrl+1-5 tabs, Ctrl+K focus, Ctrl+N new thread, Escape close/blur),
activity tab toolbar with event type filter and auto-scroll toggle.

Phase 3: Thread sidebar with load/switch/create, thread_id passed with
messages, collapsible to hamburger. Memory inline editing with textarea,
Save/Cancel, POST to /api/memory/write.

Phase 4: Gateway status popover on hover (polls every 30s), extension
install form (name/URL/kind), markdown rendering in memory viewer for
.md files, mobile responsive layout at 768px breakpoint.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add routines system, remove non-sandbox job mode from web UI

Routines: scheduled & reactive job system with cron and event triggers,
lightweight (single LLM call) and full-job execution modes, guardrails
(cooldown, max concurrent, dedup), and LLM-facing tools for CRUD.

Web UI: remove ContextManager-backed "direct" job mode entirely. Jobs
are now exclusively sandbox-backed (DB + container). Simplify job detail
response, drop dead types (ActionInfo, MessageInfo, MessageToolCallInfo),
fix Browse Files CSS loading (trailing-slash redirect), fix Activity tab
event rendering.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Re-enable chat input on agent completion, auto-auth on tool_activate, recover tool calls from content XML

Three fixes:

1. Chat input stays disabled after agent finishes: the "Done" status
   SSE event now calls enableChatInput() as a safety net when the
   response event is empty or lost. Same for auth_completed and
   cancelAuth().

2. tool_activate never triggers auth: when activation fails due to
   missing authentication, it now auto-initiates the auth flow
   (same pattern as the web API handler). detect_auth_awaiting()
   also matches tool_activate results now.

3. Models like GLM-4.7 emit tool calls as XML tags in content
   (<tool_call>tool_list</tool_call>) instead of using the structured
   tool_calls array. recover_tool_calls_from_content() extracts and
   validates these before falling back to plain text.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Add routines web UI tab, update docs for sandbox-jobs branch

Add full routines management to the web gateway (list, detail, trigger,
toggle, delete) with 7 new API endpoints, response types, and frontend
(HTML, JS, CSS). Update FEATURE_PARITY.md (~23 rows), CLAUDE.md (new
subsystems, config, TODOs), and README.md (architecture diagram,
features, components, fix onboard command).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Bind Telegram bot to owner account during setup

Without owner binding, anyone who discovers the bot can send it messages.
The setup wizard now prompts the user to message their bot, captures their
Telegram user ID via getUpdates, and persists it as telegram_owner_id in
settings. On startup, the owner_id is injected into the WASM channel config
so the existing owner restriction logic drops messages from non-owners.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Move settings from disk to PostgreSQL database

Settings previously lived in three JSON files on disk (settings.json,
mcp-servers.json, session.json). This made them inaccessible from the
web UI and caused redundant disk reads (Settings::load() called 8+
times during startup).

Now all settings live in a `settings` table (user_id + key -> JSONB)
with only 4 bootstrap fields remaining on disk (database_url, pool
size, secrets key source, onboard_completed) since they're needed
before the DB connection exists.

- Add V8 migration for settings table
- Add BootstrapConfig (thin disk file) and Settings DB round-trip
- Add Store CRUD methods for settings (get/set/delete/list/bulk)
- Refactor Config to load from DB (env > DB > default cascade)
- Add SessionManager DB persistence for session tokens
- Add DB-backed MCP server config load/save functions
- Add 6 settings web API endpoints (list/get/set/delete/export/import)
- Add one-time disk-to-DB migration on first boot
- Make CLI config commands async with DB access (disk fallback)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Seed workspace on boot, fix gateway duplicate logs and URL auto-auth

- Add Workspace::seed_if_empty() to create core identity files (README,
  MEMORY, IDENTITY, SOUL, AGENTS, USER, HEARTBEAT) when missing, called
  on every boot without overwriting existing user edits
- Remove duplicate gateway log lines from web/mod.rs (main.rs has the
  useful clickable ?token= URL)
- Auto-authenticate from ?token= URL parameter in the web UI and strip
  the token from the address bar after successful auth

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Harden sandbox security (path traversal + orchestrator auth)

Two vulnerabilities fixed:

1. project_dir path traversal: The create_job tool let the LLM specify
   arbitrary host paths for Docker bind mounts. Removed project_dir from
   the tool schema entirely, and added canonicalization + prefix validation
   at both resolve_project_dir() and the job_manager bind mount point.

2. Orchestrator API auth bypass: worker_auth_middleware was defined but
   never applied. Each handler manually called validate_token(), so any
   new endpoint that forgot would be publicly accessible. Applied the
   middleware as route_layer on all /worker/ routes, removed manual auth
   from all 7 handlers. Bind to 127.0.0.1 on macOS/Windows (Linux keeps
   0.0.0.0 since containers reach host via docker bridge, not loopback).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Rework gateway chat with pinned assistant, pagination, and NEAR AI response chaining

Implements the 4-phase plan for overhauling the web gateway chat:

- Phase 1: Pinned "Assistant" thread at top of sidebar, regular threads below
- Phase 2: Cursor-based history pagination with infinite scroll
- Phase 3: NEAR AI previous_response_id chaining (delta-only messages),
  with fallback to full history on chain errors, and DB persistence of
  chain state across restarts
- Phase 4: SSE thread isolation (events filtered by thread_id)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Add per-request HTTP timeout to WASM host, redact credentials in errors

Three fixes for WASM channel reliability:

1. Per-request timeout: Add optional timeout-ms parameter to http-request
   in both channel and tool WIT interfaces. Telegram long-poll now specifies
   35s (outliving the 30s server-side hold), while regular API calls use
   the 30s default. Fixes the triple-30s timeout race that caused polling
   failures.

2. Credential redaction: reqwest::Error includes the full URL (with injected
   bot tokens) in its Display output. Scrub credential values from error
   messages before logging or returning to WASM.

3. Webhook route registration: Remove tunnel URL gate so webhook routes are
   always available when webhook channels exist, not only when TUNNEL_URL
   is configured.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: Fix clippy warnings in WASM tools and channels

- slack channel: allow dead_code on signing_secret_name (forward compat field)
- gmail tool: use div_ceil() instead of manual (n+2)/3
- google-calendar tool: extract CreateEventParams/UpdateEventParams structs
  to fix too-many-arguments warnings

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* Fix approval flow

* fix: Rebuild bundled telegram.wasm with updated WIT interface

The bundled WASM binary must match the host's WIT definition.
Previous binary was compiled against the old 4-arg http-request;
this rebuild includes the new timeout-ms parameter.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: Load WASM channels from disk instead of bundling in binary

Remove include_bytes! embedding of telegram.wasm. Channels are now
loaded from their build output directories (channels-src/<name>/target/)
during onboarding, then from ~/.ironclaw/channels/ at runtime.

- bundled.rs: locate_channel_artifacts() finds WASM + capabilities from
  build output; IRONCLAW_CHANNELS_SRC env var overrides the default path
- available_channel_names(): only lists channels with build artifacts
- bundled_channel_names(): lists all known channels (manifest)
- Setup wizard uses available_channel_names() to offer installable channels
- Add *.wasm to .gitignore, remove tracked telegram.wasm

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Persist gateway auth token, fix thread hydration race, polish auth screen

Three web gateway UX fixes:

1. Token persistence: Store auth token in sessionStorage so refreshing
   the page doesn't force re-authentication. Hide the auth screen
   immediately when a saved token exists to prevent flash.

2. Thread hydration: Remove the !msgs.is_empty() bail-out in
   maybe_hydrate_thread so that even brand-new (empty) assistant threads
   get hydrated with their correct DB UUID. Previously resolve_thread
   would mint a fresh UUID, causing messages to land in the wrong
   conversation and duplicate threads to appear.

3. Auth screen: Redesign as a centered card with brand, tagline, labeled
   input, and hint text.

Also adds 34 new tests covering session/thread lifecycle, thread
resolution isolation (user, channel, external ID), hydration edge cases,
serialization round-trips, approval flows, and stale mapping recovery.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Use bindgen! for WASM tool wrapper, add dev tool loading

Three changes:

1. Rewrite src/tools/wasm/wrapper.rs to use wasmtime::component::bindgen!
   instead of manual linker.root().func_wrap(). This fixes the
   "component imports instance 'near:agent/host', but a matching
   implementation was not found in the linker" error. All 6 host functions
   (log, now-millis, workspace-read, http-request, secret-exists,
   tool-invoke) are now properly registered under the near:agent/host
   namespace. Also adds WASI support, credential injection, and leak
   detection for HTTP requests made by WASM tools.

2. Add dev tool loading to src/tools/wasm/loader.rs. During startup, the
   loader now also scans tools-src/*/target/wasm32-wasip2/release/ for
   build artifacts that are newer than installed copies. This means during
   development you just rebuild the WASM and restart the host; no manual
   copy step needed. Set IRONCLAW_TOOLS_SRC to override the source dir.

3. Wire up load_dev_tools() in main.rs alongside the existing
   load_from_dir() call.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: Wire main startup and CLI to use DB-backed settings

main.rs now reloads Config from the database after connecting,
attaches the store to the session manager for dual-write tokens,
and loads MCP servers from DB instead of disk. ExtensionManager
and MCP CLI commands use DB when available with disk fallback.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-02-11 08:31:25 +00:00

1124 lines
38 KiB
Rust

//! Main setup wizard orchestration.
//!
//! The wizard guides users through:
//! 1. Database connection
//! 2. Security (secrets master key)
//! 3. NEAR AI authentication
//! 4. Model selection
//! 5. Embeddings
//! 6. Channel configuration
//! 7. Heartbeat (background tasks)
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use deadpool_postgres::{Config as PoolConfig, Runtime};
use secrecy::SecretString;
use tokio_postgres::NoTls;
use crate::channels::wasm::{
ChannelCapabilitiesFile, available_channel_names, install_bundled_channel,
};
use crate::llm::{SessionConfig, SessionManager};
use crate::secrets::SecretsCrypto;
use crate::settings::{KeySource, Settings};
use crate::setup::channels::{
SecretsContext, setup_http, setup_telegram, setup_tunnel, setup_wasm_channel,
};
use crate::setup::prompts::{
confirm, input, optional_input, print_error, print_header, print_info, print_step,
print_success, select_many, select_one,
};
/// Setup wizard error.
#[derive(Debug, thiserror::Error)]
pub enum SetupError {
#[error("I/O error: {0}")]
Io(#[from] std::io::Error),
#[error("Authentication error: {0}")]
Auth(String),
#[error("Database error: {0}")]
Database(String),
#[error("Configuration error: {0}")]
Config(String),
#[error("Channel setup error: {0}")]
Channel(String),
#[error("User cancelled")]
Cancelled,
}
/// Setup wizard configuration.
#[derive(Debug, Clone, Default)]
pub struct SetupConfig {
/// Skip authentication step (use existing session).
pub skip_auth: bool,
/// Only reconfigure channels.
pub channels_only: bool,
}
/// Interactive setup wizard for IronClaw.
pub struct SetupWizard {
config: SetupConfig,
settings: Settings,
session_manager: Option<Arc<SessionManager>>,
/// Database pool (created during setup).
db_pool: Option<deadpool_postgres::Pool>,
/// Secrets crypto (created during setup).
secrets_crypto: Option<Arc<SecretsCrypto>>,
}
impl SetupWizard {
/// Create a new setup wizard.
pub fn new() -> Self {
Self {
config: SetupConfig::default(),
settings: Settings::load(),
session_manager: None,
db_pool: None,
secrets_crypto: None,
}
}
/// Create a wizard with custom configuration.
pub fn with_config(config: SetupConfig) -> Self {
Self {
config,
settings: Settings::load(),
session_manager: None,
db_pool: None,
secrets_crypto: None,
}
}
/// Set the session manager (for reusing existing auth).
pub fn with_session(mut self, session: Arc<SessionManager>) -> Self {
self.session_manager = Some(session);
self
}
/// Run the setup wizard.
pub async fn run(&mut self) -> Result<(), SetupError> {
print_header("IronClaw Setup Wizard");
if self.config.channels_only {
// Channels-only mode: just step 6
print_step(1, 1, "Channel Configuration");
self.step_channels().await?;
} else {
let total_steps = 7;
// Step 1: Database
print_step(1, total_steps, "Database Connection");
self.step_database().await?;
// Step 2: Security
print_step(2, total_steps, "Security");
self.step_security().await?;
// Step 3: Authentication (unless skipped)
if !self.config.skip_auth {
print_step(3, total_steps, "NEAR AI Authentication");
self.step_authentication().await?;
} else {
print_info("Skipping authentication (using existing session)");
}
// Step 4: Model selection
print_step(4, total_steps, "Model Selection");
self.step_model_selection().await?;
// Step 5: Embeddings
print_step(5, total_steps, "Embeddings (Semantic Search)");
self.step_embeddings()?;
// Step 6: Channel configuration
print_step(6, total_steps, "Channel Configuration");
self.step_channels().await?;
// Step 7: Heartbeat
print_step(7, total_steps, "Background Tasks");
self.step_heartbeat()?;
}
// Save settings and print summary
self.save_and_summarize()?;
Ok(())
}
/// Step 1: Database connection.
async fn step_database(&mut self) -> Result<(), SetupError> {
// Check if we have an existing URL in env or settings
let existing_url = std::env::var("DATABASE_URL")
.ok()
.or_else(|| self.settings.database_url.clone());
if let Some(ref url) = existing_url {
// Mask the password for display
let display_url = mask_password_in_url(url);
print_info(&format!("Existing database URL: {}", display_url));
if confirm("Use this database?", true).map_err(SetupError::Io)? {
// Test the connection
if let Err(e) = self.test_database_connection(url).await {
print_error(&format!("Connection failed: {}", e));
print_info("Let's configure a new database URL.");
} else {
print_success("Database connection successful");
self.settings.database_url = Some(url.clone());
return Ok(());
}
}
}
// Prompt for new URL
println!();
print_info("Enter your PostgreSQL connection URL.");
print_info("Format: postgres://user:password@host:port/database");
println!();
loop {
let url = input("Database URL").map_err(SetupError::Io)?;
if url.is_empty() {
print_error("Database URL is required.");
continue;
}
// Test the connection
print_info("Testing connection...");
match self.test_database_connection(&url).await {
Ok(()) => {
print_success("Database connection successful");
// Ask if we should run migrations
if confirm("Run database migrations?", true).map_err(SetupError::Io)? {
self.run_migrations().await?;
}
self.settings.database_url = Some(url);
return Ok(());
}
Err(e) => {
print_error(&format!("Connection failed: {}", e));
if !confirm("Try again?", true).map_err(SetupError::Io)? {
return Err(SetupError::Database(
"Database connection failed".to_string(),
));
}
}
}
}
}
/// Test database connection and store the pool.
async fn test_database_connection(&mut self, url: &str) -> Result<(), SetupError> {
let mut cfg = PoolConfig::new();
cfg.url = Some(url.to_string());
cfg.pool = Some(deadpool_postgres::PoolConfig {
max_size: 5,
..Default::default()
});
let pool = cfg
.create_pool(Some(Runtime::Tokio1), NoTls)
.map_err(|e| SetupError::Database(format!("Failed to create pool: {}", e)))?;
// Test the connection
let _ = pool
.get()
.await
.map_err(|e| SetupError::Database(format!("Failed to connect: {}", e)))?;
self.db_pool = Some(pool);
Ok(())
}
/// Run database migrations.
async fn run_migrations(&self) -> Result<(), SetupError> {
if let Some(ref pool) = self.db_pool {
use refinery::embed_migrations;
embed_migrations!("migrations");
print_info("Running migrations...");
let mut client = pool
.get()
.await
.map_err(|e| SetupError::Database(format!("Pool error: {}", e)))?;
migrations::runner()
.run_async(&mut **client)
.await
.map_err(|e| SetupError::Database(format!("Migration failed: {}", e)))?;
print_success("Migrations applied");
}
Ok(())
}
/// Step 2: Security (secrets master key).
async fn step_security(&mut self) -> Result<(), SetupError> {
// Check current configuration
let env_key_exists = std::env::var("SECRETS_MASTER_KEY").is_ok();
let keychain_key_exists = crate::secrets::keychain::has_master_key();
if env_key_exists {
print_info("Secrets master key found in SECRETS_MASTER_KEY environment variable.");
self.settings.secrets_master_key_source = KeySource::Env;
print_success("Security configured (env var)");
return Ok(());
}
if keychain_key_exists {
print_info("Existing master key found in OS keychain.");
if confirm("Use existing keychain key?", true).map_err(SetupError::Io)? {
self.settings.secrets_master_key_source = KeySource::Keychain;
print_success("Security configured (keychain)");
return Ok(());
}
}
// Offer options
println!();
print_info("The secrets master key encrypts sensitive data like API tokens.");
print_info("Choose where to store it:");
println!();
let options = [
"OS Keychain (recommended for local installs)",
"Environment variable (for CI/Docker)",
"Skip (disable secrets features)",
];
let choice = select_one("Select storage method:", &options).map_err(SetupError::Io)?;
match choice {
0 => {
// Generate and store in keychain
print_info("Generating master key...");
let key = crate::secrets::keychain::generate_master_key();
crate::secrets::keychain::store_master_key(&key).map_err(|e| {
SetupError::Config(format!("Failed to store in keychain: {}", e))
})?;
// Also create crypto instance
let key_hex: String = key.iter().map(|b| format!("{:02x}", b)).collect();
self.secrets_crypto = Some(Arc::new(
SecretsCrypto::new(SecretString::from(key_hex))
.map_err(|e| SetupError::Config(e.to_string()))?,
));
self.settings.secrets_master_key_source = KeySource::Keychain;
print_success("Master key generated and stored in OS keychain");
}
1 => {
// Env var mode
print_info("Generate a key and add it to your environment:");
let key_hex = crate::secrets::keychain::generate_master_key_hex();
println!();
println!(" export SECRETS_MASTER_KEY={}", key_hex);
println!();
print_info("Add this to your shell profile or .env file.");
self.settings.secrets_master_key_source = KeySource::Env;
print_success("Configured for environment variable");
}
_ => {
self.settings.secrets_master_key_source = KeySource::None;
print_info("Secrets features disabled. Channel tokens must be set via env vars.");
}
}
Ok(())
}
/// Step 3: NEAR AI authentication.
async fn step_authentication(&mut self) -> Result<(), SetupError> {
// Check if we already have a session
if let Some(ref session) = self.session_manager {
if session.has_token().await {
print_info("Existing session found. Validating...");
match session.ensure_authenticated().await {
Ok(()) => {
print_success("Session valid");
return Ok(());
}
Err(e) => {
print_info(&format!("Session invalid: {}. Re-authenticating...", e));
}
}
}
}
// Create session manager if we don't have one
let session = if let Some(ref s) = self.session_manager {
Arc::clone(s)
} else {
let config = SessionConfig::default();
Arc::new(SessionManager::new(config))
};
// Trigger authentication flow
session
.ensure_authenticated()
.await
.map_err(|e| SetupError::Auth(e.to_string()))?;
self.session_manager = Some(session);
Ok(())
}
/// Step 4: Model selection.
async fn step_model_selection(&mut self) -> Result<(), SetupError> {
// Show current model if already configured
if let Some(ref current) = self.settings.selected_model {
print_info(&format!("Current model: {}", current));
println!();
let options = ["Keep current model", "Change model"];
let choice =
select_one("What would you like to do?", &options).map_err(SetupError::Io)?;
if choice == 0 {
print_success(&format!("Keeping {}", current));
return Ok(());
}
}
// Try to fetch available models
let models = if let Some(ref session) = self.session_manager {
self.fetch_available_models(session).await
} else {
vec![]
};
// Default models if we couldn't fetch
let default_models = [
(
"fireworks::accounts/fireworks/models/llama4-maverick-instruct-basic",
"Llama 4 Maverick (default, fast)",
),
(
"anthropic::claude-sonnet-4-20250514",
"Claude Sonnet 4 (best quality)",
),
("openai::gpt-4o", "GPT-4o"),
];
println!("Available models:");
println!();
let options: Vec<&str> = if models.is_empty() {
default_models.iter().map(|(_, desc)| *desc).collect()
} else {
models.iter().map(|m| m.as_str()).collect()
};
// Add custom option
let mut all_options = options.clone();
all_options.push("Custom model ID");
let choice = select_one("Select a model:", &all_options).map_err(SetupError::Io)?;
let selected_model = if choice == all_options.len() - 1 {
// Custom model
input("Enter model ID").map_err(SetupError::Io)?
} else if models.is_empty() {
default_models[choice].0.to_string()
} else {
models[choice].clone()
};
self.settings.selected_model = Some(selected_model.clone());
print_success(&format!("Selected {}", selected_model));
Ok(())
}
/// Fetch available models from the API.
async fn fetch_available_models(&self, session: &Arc<SessionManager>) -> Vec<String> {
use crate::config::LlmConfig;
use crate::llm::create_llm_provider;
let base_url = std::env::var("NEARAI_BASE_URL")
.unwrap_or_else(|_| "https://cloud-api.near.ai".to_string());
let auth_base_url = std::env::var("NEARAI_AUTH_URL")
.unwrap_or_else(|_| "https://private.near.ai".to_string());
let config = LlmConfig {
nearai: crate::config::NearAiConfig {
model: "dummy".to_string(),
base_url,
auth_base_url,
session_path: crate::llm::session::default_session_path(),
api_mode: crate::config::NearAiApiMode::Responses,
api_key: None,
},
};
match create_llm_provider(&config, Arc::clone(session)) {
Ok(provider) => match provider.list_models().await {
Ok(models) => models,
Err(e) => {
print_info(&format!("Could not fetch models: {}. Using defaults.", e));
vec![]
}
},
Err(e) => {
print_info(&format!(
"Could not initialize provider: {}. Using defaults.",
e
));
vec![]
}
}
}
/// Step 5: Embeddings configuration.
fn step_embeddings(&mut self) -> Result<(), SetupError> {
print_info("Embeddings enable semantic search in your workspace memory.");
println!();
if !confirm("Enable semantic search?", true).map_err(SetupError::Io)? {
self.settings.embeddings.enabled = false;
print_info("Embeddings disabled. Workspace will use keyword search only.");
return Ok(());
}
let options = [
"NEAR AI (uses same auth, no extra cost)",
"OpenAI (requires API key)",
];
let choice = select_one("Select embeddings provider:", &options).map_err(SetupError::Io)?;
match choice {
0 => {
self.settings.embeddings.enabled = true;
self.settings.embeddings.provider = "nearai".to_string();
self.settings.embeddings.model = "text-embedding-3-small".to_string();
print_success("Embeddings enabled via NEAR AI");
}
1 => {
// Check if API key is set
if std::env::var("OPENAI_API_KEY").is_err() {
print_info("OPENAI_API_KEY not set in environment.");
print_info("Add it to your .env file or environment to enable embeddings.");
}
self.settings.embeddings.enabled = true;
self.settings.embeddings.provider = "openai".to_string();
self.settings.embeddings.model = "text-embedding-3-small".to_string();
print_success("Embeddings configured for OpenAI");
}
_ => unreachable!(),
}
Ok(())
}
/// Initialize secrets context for channel setup.
async fn init_secrets_context(&mut self) -> Result<SecretsContext, SetupError> {
// Get database pool (should be set from step 1)
let pool = if let Some(ref p) = self.db_pool {
p.clone()
} else {
// Fall back to creating one from settings/env
let url = self
.settings
.database_url
.clone()
.or_else(|| std::env::var("DATABASE_URL").ok())
.ok_or_else(|| SetupError::Config("Database URL not configured".to_string()))?;
self.test_database_connection(&url).await?;
// Ensure secrets-related tables exist for channels-only onboarding flows.
self.run_migrations().await?;
self.db_pool.clone().unwrap()
};
// Get crypto (should be set from step 2, or load from keychain/env)
let crypto = if let Some(ref c) = self.secrets_crypto {
Arc::clone(c)
} else {
// Try to load master key from keychain or env
let key = if let Ok(env_key) = std::env::var("SECRETS_MASTER_KEY") {
env_key
} else if let Ok(keychain_key) = crate::secrets::keychain::get_master_key() {
keychain_key.iter().map(|b| format!("{:02x}", b)).collect()
} else {
return Err(SetupError::Config(
"Secrets not configured. Run full setup or set SECRETS_MASTER_KEY.".to_string(),
));
};
let crypto = SecretsCrypto::new(SecretString::from(key))
.map_err(|e| SetupError::Config(e.to_string()))?;
self.secrets_crypto = Some(Arc::new(crypto));
Arc::clone(self.secrets_crypto.as_ref().unwrap())
};
Ok(SecretsContext::new(pool, crypto, "default"))
}
/// Step 6: Channel configuration.
async fn step_channels(&mut self) -> Result<(), SetupError> {
// First, configure tunnel (shared across all channels that need webhooks)
match setup_tunnel() {
Ok(Some(url)) => {
self.settings.tunnel.public_url = Some(url);
}
Ok(None) => {
self.settings.tunnel.public_url = None;
}
Err(e) => {
print_info(&format!("Tunnel setup skipped: {}", e));
}
}
println!();
// Discover available WASM channels
let channels_dir = dirs::home_dir()
.unwrap_or_default()
.join(".ironclaw/channels");
let mut discovered_channels = discover_wasm_channels(&channels_dir).await;
let installed_names: HashSet<String> = discovered_channels
.iter()
.map(|(name, _)| name.clone())
.collect();
let wasm_channel_names = wasm_channel_option_names(&discovered_channels);
// Build options list dynamically
let mut options: Vec<(String, bool)> = vec![
("CLI/TUI (always enabled)".to_string(), true),
(
"HTTP webhook".to_string(),
self.settings.channels.http_enabled,
),
];
// Add available WASM channels (installed + bundled)
for name in &wasm_channel_names {
let is_enabled = self.settings.channels.wasm_channels.contains(name);
let display_name = format!("{} (WASM)", capitalize_first(name));
options.push((display_name, is_enabled));
}
let options_refs: Vec<(&str, bool)> =
options.iter().map(|(s, b)| (s.as_str(), *b)).collect();
let selected = select_many("Which channels do you want to enable?", &options_refs)
.map_err(SetupError::Io)?;
let selected_wasm_channels: Vec<String> = wasm_channel_names
.iter()
.enumerate()
.filter_map(|(idx, name)| {
if selected.contains(&(idx + 2)) {
Some(name.clone())
} else {
None
}
})
.collect();
if let Some(installed) = install_selected_bundled_channels(
&channels_dir,
&selected_wasm_channels,
&installed_names,
)
.await?
{
if !installed.is_empty() {
print_success(&format!("Installed channels: {}", installed.join(", ")));
discovered_channels = discover_wasm_channels(&channels_dir).await;
}
}
// Determine if we need secrets context
let needs_secrets = selected.contains(&1) || !selected_wasm_channels.is_empty();
let secrets = if needs_secrets {
match self.init_secrets_context().await {
Ok(ctx) => Some(ctx),
Err(e) => {
print_info(&format!("Secrets not available: {}", e));
print_info("Channel tokens must be set via environment variables.");
None
}
}
} else {
None
};
// HTTP is index 1
if selected.contains(&1) {
println!();
if let Some(ref ctx) = secrets {
let result = setup_http(ctx).await.map_err(SetupError::Channel)?;
self.settings.channels.http_enabled = result.enabled;
self.settings.channels.http_port = Some(result.port);
} else {
self.settings.channels.http_enabled = true;
self.settings.channels.http_port = Some(8080);
print_info("HTTP webhook enabled on port 8080 (set HTTP_WEBHOOK_SECRET in env)");
}
} else {
self.settings.channels.http_enabled = false;
}
let discovered_by_name: HashMap<String, ChannelCapabilitiesFile> =
discovered_channels.into_iter().collect();
// Process selected WASM channels
let mut enabled_wasm_channels = Vec::new();
for channel_name in selected_wasm_channels {
println!();
if let Some(ref ctx) = secrets {
let result = if let Some(cap_file) = discovered_by_name.get(&channel_name) {
if !cap_file.setup.required_secrets.is_empty() {
setup_wasm_channel(ctx, &channel_name, &cap_file.setup)
.await
.map_err(SetupError::Channel)?
} else if channel_name == "telegram" {
let telegram_result =
setup_telegram(ctx).await.map_err(SetupError::Channel)?;
if let Some(owner_id) = telegram_result.owner_id {
self.settings.channels.telegram_owner_id = Some(owner_id);
}
crate::setup::channels::WasmChannelSetupResult {
enabled: telegram_result.enabled,
channel_name: "telegram".to_string(),
}
} else {
print_info(&format!(
"No setup configuration found for {}",
channel_name
));
crate::setup::channels::WasmChannelSetupResult {
enabled: true,
channel_name: channel_name.clone(),
}
}
} else {
print_info(&format!(
"Channel '{}' is selected but not available on disk.",
channel_name
));
continue;
};
if result.enabled {
enabled_wasm_channels.push(result.channel_name);
}
} else {
// No secrets context, just enable the channel
print_info(&format!(
"{} enabled (configure tokens via environment)",
capitalize_first(&channel_name)
));
enabled_wasm_channels.push(channel_name.clone());
}
}
self.settings.channels.wasm_channels = enabled_wasm_channels;
Ok(())
}
/// Step 7: Heartbeat configuration.
fn step_heartbeat(&mut self) -> Result<(), SetupError> {
print_info("Heartbeat runs periodic background tasks (e.g., checking your calendar,");
print_info("monitoring for notifications, running scheduled workflows).");
println!();
if !confirm("Enable heartbeat?", false).map_err(SetupError::Io)? {
self.settings.heartbeat.enabled = false;
print_info("Heartbeat disabled.");
return Ok(());
}
self.settings.heartbeat.enabled = true;
// Interval
let interval_str = optional_input("Check interval in minutes", Some("default: 30"))
.map_err(SetupError::Io)?;
if let Some(s) = interval_str {
if let Ok(mins) = s.parse::<u64>() {
self.settings.heartbeat.interval_secs = mins * 60;
}
} else {
self.settings.heartbeat.interval_secs = 1800; // 30 minutes
}
// Notify channel
let notify_channel = optional_input("Notify channel on findings", Some("e.g., telegram"))
.map_err(SetupError::Io)?;
self.settings.heartbeat.notify_channel = notify_channel;
print_success(&format!(
"Heartbeat enabled (every {} minutes)",
self.settings.heartbeat.interval_secs / 60
));
Ok(())
}
/// Save settings and print summary.
fn save_and_summarize(&mut self) -> Result<(), SetupError> {
self.settings.onboard_completed = true;
self.settings.save().map_err(|e| {
SetupError::Io(std::io::Error::new(
std::io::ErrorKind::Other,
format!("Failed to save settings: {}", e),
))
})?;
println!();
print_success("Configuration saved to ~/.ironclaw/");
println!();
// Print summary
println!("Configuration Summary:");
println!("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
if self.settings.database_url.is_some() {
println!(" Database: configured");
}
match self.settings.secrets_master_key_source {
KeySource::Keychain => println!(" Security: OS keychain"),
KeySource::Env => println!(" Security: environment variable"),
KeySource::None => println!(" Security: disabled"),
}
if let Some(ref model) = self.settings.selected_model {
// Truncate long model names
let display = if model.len() > 40 {
format!("{}...", &model[..37])
} else {
model.clone()
};
println!(" Model: {}", display);
}
if self.settings.embeddings.enabled {
println!(
" Embeddings: {} ({})",
self.settings.embeddings.provider, self.settings.embeddings.model
);
} else {
println!(" Embeddings: disabled");
}
if let Some(ref tunnel_url) = self.settings.tunnel.public_url {
println!(" Tunnel: {}", tunnel_url);
}
println!(" Channels:");
println!(" - CLI/TUI: enabled");
if self.settings.channels.http_enabled {
let port = self.settings.channels.http_port.unwrap_or(8080);
println!(" - HTTP: enabled (port {})", port);
}
for channel_name in &self.settings.channels.wasm_channels {
let mode = if self.settings.tunnel.public_url.is_some() {
"webhook"
} else {
"polling"
};
println!(
" - {}: enabled ({})",
capitalize_first(channel_name),
mode
);
}
if self.settings.heartbeat.enabled {
println!(
" Heartbeat: every {} minutes",
self.settings.heartbeat.interval_secs / 60
);
}
println!();
println!("To start the agent, run:");
println!(" ironclaw");
println!();
println!("To change settings later:");
println!(" ironclaw config set <setting> <value>");
println!(" ironclaw onboard");
println!();
Ok(())
}
}
impl Default for SetupWizard {
fn default() -> Self {
Self::new()
}
}
/// Mask password in a database URL for display.
fn mask_password_in_url(url: &str) -> String {
// URL format: scheme://user:password@host/database
// Find "://" to locate start of credentials
let Some(scheme_end) = url.find("://") else {
return url.to_string();
};
let credentials_start = scheme_end + 3; // After "://"
// Find "@" to locate end of credentials
let Some(at_pos) = url[credentials_start..].find('@') else {
return url.to_string();
};
let at_abs = credentials_start + at_pos;
// Find ":" in the credentials section (separates user from password)
let credentials = &url[credentials_start..at_abs];
let Some(colon_pos) = credentials.find(':') else {
return url.to_string();
};
// Build masked URL: scheme://user:****@host/database
let scheme = &url[..credentials_start]; // "postgres://"
let username = &credentials[..colon_pos]; // "user"
let after_at = &url[at_abs..]; // "@localhost/db"
format!("{}{}:****{}", scheme, username, after_at)
}
/// Discover WASM channels in a directory.
///
/// Returns a list of (channel_name, capabilities_file) pairs.
async fn discover_wasm_channels(dir: &std::path::Path) -> Vec<(String, ChannelCapabilitiesFile)> {
let mut channels = Vec::new();
if !dir.is_dir() {
return channels;
}
let mut entries = match tokio::fs::read_dir(dir).await {
Ok(e) => e,
Err(_) => return channels,
};
while let Ok(Some(entry)) = entries.next_entry().await {
let path = entry.path();
// Look for .capabilities.json files
let extension = path.file_name().and_then(|n| n.to_str()).unwrap_or("");
if !extension.ends_with(".capabilities.json") {
continue;
}
// Extract channel name
let name = extension.trim_end_matches(".capabilities.json").to_string();
if name.is_empty() {
continue;
}
// Check if corresponding .wasm file exists
let wasm_path = dir.join(format!("{}.wasm", name));
if !wasm_path.exists() {
continue;
}
// Parse capabilities file
match tokio::fs::read(&path).await {
Ok(bytes) => match ChannelCapabilitiesFile::from_bytes(&bytes) {
Ok(cap_file) => {
channels.push((name, cap_file));
}
Err(e) => {
tracing::warn!(
path = %path.display(),
error = %e,
"Failed to parse channel capabilities file"
);
}
},
Err(e) => {
tracing::warn!(
path = %path.display(),
error = %e,
"Failed to read channel capabilities file"
);
}
}
}
// Sort by name for consistent ordering
channels.sort_by(|a, b| a.0.cmp(&b.0));
channels
}
/// Capitalize the first letter of a string.
fn capitalize_first(s: &str) -> String {
let mut chars = s.chars();
match chars.next() {
None => String::new(),
Some(first) => first.to_uppercase().chain(chars).collect(),
}
}
#[cfg(test)]
async fn install_missing_bundled_channels(
channels_dir: &std::path::Path,
already_installed: &HashSet<String>,
) -> Result<Vec<String>, SetupError> {
let mut installed = Vec::new();
for name in available_channel_names().iter().copied() {
if already_installed.contains(name) {
continue;
}
install_bundled_channel(name, channels_dir, false)
.await
.map_err(SetupError::Channel)?;
installed.push(name.to_string());
}
Ok(installed)
}
fn wasm_channel_option_names(discovered: &[(String, ChannelCapabilitiesFile)]) -> Vec<String> {
let mut names: Vec<String> = discovered.iter().map(|(name, _)| name.clone()).collect();
for bundled in available_channel_names().iter().copied() {
if !names.iter().any(|name| name == bundled) {
names.push(bundled.to_string());
}
}
names
}
async fn install_selected_bundled_channels(
channels_dir: &std::path::Path,
selected_channels: &[String],
already_installed: &HashSet<String>,
) -> Result<Option<Vec<String>>, SetupError> {
let bundled: HashSet<&str> = available_channel_names().iter().copied().collect();
let selected_missing: HashSet<String> = selected_channels
.iter()
.filter(|name| bundled.contains(name.as_str()) && !already_installed.contains(*name))
.cloned()
.collect();
if selected_missing.is_empty() {
return Ok(None);
}
let mut installed = Vec::new();
for name in selected_missing {
install_bundled_channel(&name, channels_dir, false)
.await
.map_err(SetupError::Channel)?;
installed.push(name);
}
installed.sort();
Ok(Some(installed))
}
#[cfg(test)]
mod tests {
use std::collections::HashSet;
use tempfile::tempdir;
use super::*;
#[test]
fn test_wizard_creation() {
let wizard = SetupWizard::new();
assert!(!wizard.config.skip_auth);
assert!(!wizard.config.channels_only);
}
#[test]
fn test_wizard_with_config() {
let config = SetupConfig {
skip_auth: true,
channels_only: false,
};
let wizard = SetupWizard::with_config(config);
assert!(wizard.config.skip_auth);
}
#[test]
fn test_mask_password_in_url() {
assert_eq!(
mask_password_in_url("postgres://user:secret@localhost/db"),
"postgres://user:****@localhost/db"
);
// URL without password
assert_eq!(
mask_password_in_url("postgres://localhost/db"),
"postgres://localhost/db"
);
}
#[test]
fn test_capitalize_first() {
assert_eq!(capitalize_first("telegram"), "Telegram");
assert_eq!(capitalize_first("CAPS"), "CAPS");
assert_eq!(capitalize_first(""), "");
}
#[tokio::test]
async fn test_install_missing_bundled_channels_installs_telegram() {
let dir = tempdir().unwrap();
let installed = HashSet::<String>::new();
install_missing_bundled_channels(dir.path(), &installed)
.await
.unwrap();
assert!(dir.path().join("telegram.wasm").exists());
assert!(dir.path().join("telegram.capabilities.json").exists());
}
#[test]
fn test_wasm_channel_option_names_includes_available_when_missing() {
let discovered = Vec::new();
let options = wasm_channel_option_names(&discovered);
let available = available_channel_names();
// All available (built) channels should appear
for name in &available {
assert!(
options.contains(&name.to_string()),
"expected '{}' in options",
name
);
}
}
#[test]
fn test_wasm_channel_option_names_dedupes_available() {
let discovered = vec![(String::from("telegram"), ChannelCapabilitiesFile::default())];
let options = wasm_channel_option_names(&discovered);
// telegram should appear exactly once despite being both discovered and available
assert_eq!(
options.iter().filter(|n| *n == "telegram").count(),
1,
"telegram should not be duplicated"
);
}
}