gitops/optimclaw
1
0
Fork 0
mirror of https://github.com/outbackdingo/optimclaw.git synced 2026-09-01 00:59:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: onboarding errors reset flow and remote server auth (#185, #186) (#248)

Browse Source 
* fix: incremental settings persistence and remote server auth (#185, #186)

Persist settings after each wizard step so failures don't lose prior
progress. Load existing settings on re-run to recover from partial
onboarding. Add manual token paste option for remote/headless servers
where browser OAuth is unreachable, and support IRONCLAW_OAUTH_CALLBACK_URL
for custom callback URLs. Color prompt output (green/red/blue prefixes).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: replace session token paste with API key entry, address PR review

Replace option 4 in NEAR AI auth menu from session token paste to NEAR
AI Cloud API key entry (cloud.near.ai). Also address all PR review
feedback: restrict .env file permissions to 0o600, mask API key input
with secret_input, fix libsql loaded flag in try_load_existing_settings,
add ENV_MUTEX to oauth_defaults tests, and add NEARAI_API_KEY to secrets
injection.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: deduplicate keys in upsert_bootstrap_var

When the .env file contains duplicate keys (e.g. from manual editing),
only write the replacement once and skip subsequent duplicates.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: NEARAI_SESSION_TOKEN env var takes precedence over file-based tokens

Hosting providers inject session tokens via env var and expect them to
be used directly. Previously the env var was only picked up when no
session file existed and was treated as a legacy migration. Now the env
var always wins, without persisting to disk.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* docs: distinguish NEAR AI Chat and NEAR AI Cloud providers

Split documentation into two clearly named modes:
- NEAR AI Chat: Responses API at private.near.ai, session token auth
- NEAR AI Cloud: Chat Completions API at cloud-api.near.ai, API key auth

Update default base URLs so each mode points to its correct endpoint.
Update .env.example, deploy/env.example, CLAUDE.md, setup spec, and
code comments across config/llm.rs, nearai.rs, nearai_chat.rs, mod.rs.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: wizard recovery ordering — load DB before persist, fresh choices win

Previously, persist_after_step() ran after Step 1 but before
try_load_existing_settings(), bulk-upserting defaults that clobbered
prior settings. Additionally, merge_from gave stale DB values
precedence over fresh Step 1 choices.

Fix: snapshot Step 1 settings, load DB, then re-apply the snapshot.
This ensures prior progress (steps 2-7) is recovered while fresh
Step 1 choices override stale DB values.

Add two tests verifying wizard recovery merge ordering.

Addresses PR review comments from Copilot on wizard.rs:150,
wizard.rs:1607, and wizard.rs:1626.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting in config/llm.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: collapse nested if per clippy collapsible_if lint

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use print_success for API key confirmation, fix menu spacing

- Use print_success() for colored output consistency in api_key_login
- Fix box-drawing alignment: options 1-2 had an extra trailing space

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
2026-02-20 08:02:22 +00:00
committed by GitHub
co-authored by Claude Opus 4.6
parent bfe393eb38
commit 5725a62c83
15 changed files with 764 additions and 199 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/setup/README.md
+86 -16
View File
@@ -167,7 +167,8 @@ env-var mode or skipped secrets.
| Provider | Auth Method | Secret Name | Env Var |
|----------|-------------|-------------|---------|
| NEAR AI | Browser OAuth | (session token) | `NEARAI_SESSION_TOKEN` |
| NEAR AI Chat | Browser OAuth or session token | - | `NEARAI_SESSION_TOKEN` |
| NEAR AI Cloud | API key | `llm_nearai_api_key` | `NEARAI_API_KEY` |
| Anthropic | API key | `anthropic_api_key` | `ANTHROPIC_API_KEY` |
| OpenAI | API key | `openai_api_key` | `OPENAI_API_KEY` |
| Ollama | None | - | - |
@@ -180,8 +181,18 @@ env-var mode or skipped secrets.
4. **Cache key in `self.llm_api_key`** for model fetching in Step 4
**NEAR AI** (`setup_nearai`):
- Calls `session_manager.ensure_authenticated()` which opens browser
- Session token saved to `~/.ironclaw/session.json`
- Calls `session_manager.ensure_authenticated()` which shows the auth menu:
- Options 1-2 (GitHub/Google): browser OAuth → **NEAR AI Chat** mode
(Responses API at `private.near.ai`, session token auth)
- Option 4: NEAR AI Cloud API key → **NEAR AI Cloud** mode
(Chat Completions API at `cloud-api.near.ai`, API key auth)
- **NEAR AI Chat** path: session token saved to `~/.ironclaw/session.json`.
Hosting providers can set `NEARAI_SESSION_TOKEN` env var directly (takes
precedence over file-based tokens).
- **NEAR AI Cloud** path: `NEARAI_API_KEY` saved to `~/.ironclaw/.env`
(bootstrap) and encrypted secrets store (`llm_nearai_api_key`).
`LlmConfig::resolve()` auto-selects `ChatCompletions` mode when the
API key is present.
**`self.llm_api_key` caching:** The wizard caches the API key as
`Option<SecretString>` so that Step 4 (model fetching) and Step 5
@@ -372,25 +383,60 @@ heartbeat.enabled = "true"
heartbeat.interval_secs = "300"
```
### Incremental Persistence
Settings are persisted **after every successful step**, not just at the end.
This prevents data loss if a later step fails (e.g., the user enters an
API key in step 3 but step 5 crashes — they won't need to re-enter it).
**`persist_after_step()`** is called after each step in `run()` and:
1. Writes bootstrap vars to `~/.ironclaw/.env` via `write_bootstrap_env()`
2. Writes all current settings to the database via `persist_settings()`
3. Silently ignores errors (e.g., if called before Step 1 establishes a DB)
**`try_load_existing_settings()`** is called after Step 1 establishes a
database connection. It loads any previously saved settings from the
database using `get_all_settings("default")` → `Settings::from_db_map()`
→ `merge_from()`. This recovers progress from prior partial wizard runs.
**Ordering after Step 1 is critical:**
```
step_database() → sets DB fields in self.settings
let step1 = self.settings.clone() → snapshot Step 1 choices
try_load_existing_settings() → merge DB values into self.settings
self.settings.merge_from(&step1) → re-apply Step 1 (fresh wins over stale)
persist_after_step() → save merged state
```
This ordering ensures:
- Prior progress (steps 2-7 from a previous partial run) is recovered
- Fresh Step 1 choices override stale DB values (not the reverse)
- The first DB persist doesn't clobber prior settings with defaults
### save_and_summarize()
Final step of the wizard:
```
1. Mark onboard_completed = true
2. Write ALL settings to database (try postgres pool, then libSQL backend)
3. Write bootstrap vars to ~/.ironclaw/.env:
- DATABASE_BACKEND (always)
- DATABASE_URL (if postgres)
- LIBSQL_PATH (if libsql)
- LIBSQL_URL (if turso sync)
- LLM_BACKEND (always, when set)
- LLM_BASE_URL (if openai_compatible)
- OLLAMA_BASE_URL (if ollama)
- ONBOARD_COMPLETED (always, "true")
2. Call persist_settings() for final write (idempotent — ensures
onboard_completed flag is saved)
3. Call write_bootstrap_env() for final .env write (idempotent)
4. Print configuration summary
```
Bootstrap vars written to `~/.ironclaw/.env`:
- `DATABASE_BACKEND` (always)
- `DATABASE_URL` (if postgres)
- `LIBSQL_PATH` (if libsql)
- `LIBSQL_URL` (if turso sync)
- `LLM_BACKEND` (always, when set)
- `LLM_BASE_URL` (if openai_compatible)
- `OLLAMA_BASE_URL` (if ollama)
- `NEARAI_API_KEY` (if API key auth path)
- `ONBOARD_COMPLETED` (always, "true")
**Invariant:** Both Layer 1 and Layer 2 must be written. If the database
write fails, the wizard returns an error and the `.env` file is not written.
@@ -498,9 +544,9 @@ anthropic_api_key → encrypted API key
| `confirm(label, default)` | `[Y/n]` or `[y/N]` prompt |
| `print_header(text)` | Bold section header with underline |
| `print_step(n, total, text)` | `[1/7] Step Name` |
| `print_success(text)` | Green checkmark prefix |
| `print_error(text)` | Red X prefix |
| `print_info(text)` | Blue info prefix |
| `print_success(text)` | Green `✓` prefix (ANSI color), message in default color |
| `print_error(text)` | Red `✗` prefix (ANSI color), message in default color |
| `print_info(text)` | Blue `ℹ` prefix (ANSI color), message in default color |
`select_many` uses `crossterm` raw mode for arrow key navigation.
Must properly restore terminal state on all exit paths.
@@ -523,6 +569,30 @@ Must properly restore terminal state on all exit paths.
- May need `gnome-keyring` daemon running
- Collection unlock may prompt for password
### Remote Server Authentication
On remote/VPS servers, the browser-based OAuth flow for NEAR AI may not
work because `http://127.0.0.1:9876` is unreachable from the user's
local browser.
**Solutions:**
1. **NEAR AI Cloud API key (option 4 in auth menu):** Get an API key
from `https://cloud.near.ai` and paste it into the terminal. No
local listener is needed. The key is saved to `~/.ironclaw/.env`
and the encrypted secrets store. Uses the OpenAI-compatible
ChatCompletions API mode.
2. **Custom callback URL:** Set `IRONCLAW_OAUTH_CALLBACK_URL` to a
publicly accessible URL (e.g., via SSH tunnel or reverse proxy) that
forwards to port 9876 on the server:
```bash
export IRONCLAW_OAUTH_CALLBACK_URL=https://myserver.example.com:9876
```
The `callback_url()` function in `oauth_defaults.rs` checks this env var
and falls back to `http://127.0.0.1:{OAUTH_CALLBACK_PORT}`.
### URL Passwords
- `#` is common in URL-encoded passwords (`%23` decoded)
src/setup/prompts.rs
+29 -6
View File
@@ -293,19 +293,31 @@ pub fn print_step(current: usize, total: usize, name: &str) {
println!();
}
/// Print a success message with checkmark.
/// Print a success message with green checkmark.
pub fn print_success(message: &str) {
println!("✓ {}", message);
let mut stdout = io::stdout();
let _ = execute!(stdout, SetForegroundColor(Color::Green));
print!("✓");
let _ = execute!(stdout, ResetColor);
println!(" {}", message);
}
/// Print an error message.
/// Print an error message with red X.
pub fn print_error(message: &str) {
eprintln!("✗ {}", message);
let mut stderr = io::stderr();
let _ = execute!(stderr, SetForegroundColor(Color::Red));
eprint!("✗");
let _ = execute!(stderr, ResetColor);
eprintln!(" {}", message);
}
/// Print an info message.
/// Print an info message with blue info icon.
pub fn print_info(message: &str) {
println!(" {}", message);
let mut stdout = io::stdout();
let _ = execute!(stdout, SetForegroundColor(Color::Blue));
print!("ℹ");
let _ = execute!(stdout, ResetColor);
println!(" {}", message);
}
/// Read a simple line of input with a prompt.
@@ -358,4 +370,15 @@ mod tests {
super::print_step(1, 3, "Test Step");
super::print_step(3, 3, "Final Step");
}
#[test]
fn test_print_functions_do_not_panic() {
super::print_success("operation completed");
super::print_error("something went wrong");
super::print_info("here is some information");
// Also test with empty strings
super::print_success("");
super::print_error("");
super::print_info("");
}
}
src/setup/wizard.rs
+235 -97
View File
@@ -125,6 +125,11 @@ impl SetupWizard {
}
/// Run the setup wizard.
///
/// Settings are persisted incrementally after each successful step so
/// that progress is not lost if a later step fails. On re-run, existing
/// settings are loaded from the database after Step 1 establishes a
/// connection, so users don't have to re-enter everything.
pub async fn run(&mut self) -> Result<(), SetupError> {
print_header("IronClaw Setup Wizard");
@@ -141,9 +146,22 @@ impl SetupWizard {
print_step(1, total_steps, "Database Connection");
self.step_database().await?;
// After establishing a DB connection, load any previously saved
// settings so we recover progress from prior partial runs.
// We must load BEFORE persisting, otherwise persist_after_step()
// would overwrite prior settings with defaults.
// Save Step 1 choices first so they aren't clobbered by stale
// DB values (merge_from only applies non-default fields).
let step1_settings = self.settings.clone();
self.try_load_existing_settings().await;
self.settings.merge_from(&step1_settings);
self.persist_after_step().await;
// Step 2: Security
print_step(2, total_steps, "Security");
self.step_security().await?;
self.persist_after_step().await;
// Step 3: Inference provider selection (unless skipped)
if !self.config.skip_auth {
@@ -152,18 +170,22 @@ impl SetupWizard {
} else {
print_info("Skipping inference provider setup (using existing config)");
}
self.persist_after_step().await;
// Step 4: Model selection
print_step(4, total_steps, "Model Selection");
self.step_model_selection().await?;
self.persist_after_step().await;
// Step 5: Embeddings
print_step(5, total_steps, "Embeddings (Semantic Search)");
self.step_embeddings()?;
self.persist_after_step().await;
// Step 6: Channel configuration
print_step(6, total_steps, "Channel Configuration");
self.step_channels().await?;
self.persist_after_step().await;
// Step 7: Extensions (tools)
print_step(7, total_steps, "Extensions");
@@ -172,6 +194,7 @@ impl SetupWizard {
// Step 8: Heartbeat
print_step(8, total_steps, "Background Tasks");
self.step_heartbeat()?;
self.persist_after_step().await;
}
// Save settings and print summary
@@ -802,6 +825,20 @@ impl SetupWizard {
.map_err(|e| SetupError::Auth(e.to_string()))?;
self.session_manager = Some(session);
// If the user chose the API key path, NEARAI_API_KEY is now set
// in the environment. Persist it to the encrypted secrets store
// so inject_llm_keys_from_secrets() can load it on future runs.
if let Ok(api_key) = std::env::var("NEARAI_API_KEY")
&& !api_key.is_empty()
&& let Ok(ctx) = self.init_secrets_context().await
{
let key = SecretString::from(api_key);
if let Err(e) = ctx.save_secret("llm_nearai_api_key", &key).await {
tracing::warn!("Failed to persist NEARAI_API_KEY to secrets: {}", e);
}
}
print_success("NEAR AI configured");
Ok(())
}
@@ -1719,110 +1756,211 @@ impl SetupWizard {
Ok(())
}
/// Persist current settings to the database.
///
/// Returns `Ok(true)` if settings were saved, `Ok(false)` if no database
/// connection is available yet (e.g., before Step 1 completes).
async fn persist_settings(&self) -> Result<bool, SetupError> {
let db_map = self.settings.to_db_map();
let saved = false;
#[cfg(feature = "postgres")]
let saved = if !saved {
if let Some(ref pool) = self.db_pool {
let store = crate::history::Store::from_pool(pool.clone());
store
.set_all_settings("default", &db_map)
.await
.map_err(|e| {
SetupError::Database(format!("Failed to save settings to database: {}", e))
})?;
true
} else {
false
}
} else {
saved
};
#[cfg(feature = "libsql")]
let saved = if !saved {
if let Some(ref backend) = self.db_backend {
use crate::db::SettingsStore as _;
backend
.set_all_settings("default", &db_map)
.await
.map_err(|e| {
SetupError::Database(format!("Failed to save settings to database: {}", e))
})?;
true
} else {
false
}
} else {
saved
};
Ok(saved)
}
/// Write bootstrap environment variables to `~/.ironclaw/.env`.
///
/// These are the chicken-and-egg settings needed before the database is
/// connected (DATABASE_BACKEND, DATABASE_URL, LLM_BACKEND, etc.).
fn write_bootstrap_env(&self) -> Result<(), SetupError> {
let mut env_vars: Vec<(&str, String)> = Vec::new();
if let Some(ref backend) = self.settings.database_backend {
env_vars.push(("DATABASE_BACKEND", backend.clone()));
}
if let Some(ref url) = self.settings.database_url {
env_vars.push(("DATABASE_URL", url.clone()));
}
if let Some(ref path) = self.settings.libsql_path {
env_vars.push(("LIBSQL_PATH", path.clone()));
}
if let Some(ref url) = self.settings.libsql_url {
env_vars.push(("LIBSQL_URL", url.clone()));
}
// LLM bootstrap vars: same chicken-and-egg problem as DATABASE_BACKEND.
// Config::from_env() needs the backend before the DB is connected.
if let Some(ref backend) = self.settings.llm_backend {
env_vars.push(("LLM_BACKEND", backend.clone()));
}
if let Some(ref url) = self.settings.openai_compatible_base_url {
env_vars.push(("LLM_BASE_URL", url.clone()));
}
if let Some(ref url) = self.settings.ollama_base_url {
env_vars.push(("OLLAMA_BASE_URL", url.clone()));
}
// Preserve NEARAI_API_KEY if present (set by API key auth flow)
if let Ok(api_key) = std::env::var("NEARAI_API_KEY")
&& !api_key.is_empty()
{
env_vars.push(("NEARAI_API_KEY", api_key));
}
// Always write ONBOARD_COMPLETED so that check_onboard_needed()
// (which runs before the DB is connected) knows to skip re-onboarding.
if self.settings.onboard_completed {
env_vars.push(("ONBOARD_COMPLETED", "true".to_string()));
}
if !env_vars.is_empty() {
let pairs: Vec<(&str, &str)> = env_vars.iter().map(|(k, v)| (*k, v.as_str())).collect();
crate::bootstrap::save_bootstrap_env(&pairs).map_err(|e| {
SetupError::Io(std::io::Error::other(format!(
"Failed to save bootstrap env to .env: {}",
e
)))
})?;
}
Ok(())
}
/// Persist settings to DB and bootstrap .env after each step.
///
/// Silently ignores errors (e.g., DB not connected yet before step 1
/// completes). This is best-effort incremental persistence.
async fn persist_after_step(&self) {
// Write bootstrap .env (always possible)
if let Err(e) = self.write_bootstrap_env() {
tracing::debug!("Could not write bootstrap env after step: {}", e);
}
// Persist to DB
match self.persist_settings().await {
Ok(true) => tracing::debug!("Settings persisted to database after step"),
Ok(false) => tracing::debug!("No DB connection yet, skipping settings persist"),
Err(e) => tracing::debug!("Could not persist settings after step: {}", e),
}
}
/// Load previously saved settings from the database after Step 1
/// establishes a connection.
///
/// This enables recovery from partial onboarding runs: if the user
/// completed steps 1-4 previously but step 5 failed, re-running
/// the wizard will pre-populate settings from the database.
///
/// **Callers must re-apply any wizard choices made before this call**
/// via `self.settings.merge_from(&step_settings)`, since `merge_from`
/// prefers the `other` argument's non-default values. Without this,
/// stale DB values would overwrite fresh user choices.
async fn try_load_existing_settings(&mut self) {
let loaded = false;
#[cfg(feature = "postgres")]
let loaded = if !loaded {
if let Some(ref pool) = self.db_pool {
let store = crate::history::Store::from_pool(pool.clone());
match store.get_all_settings("default").await {
Ok(db_map) if !db_map.is_empty() => {
let existing = Settings::from_db_map(&db_map);
self.settings.merge_from(&existing);
tracing::info!("Loaded {} existing settings from database", db_map.len());
true
}
Ok(_) => false,
Err(e) => {
tracing::debug!("Could not load existing settings: {}", e);
false
}
}
} else {
false
}
} else {
loaded
};
#[cfg(feature = "libsql")]
let loaded = if !loaded {
if let Some(ref backend) = self.db_backend {
use crate::db::SettingsStore as _;
match backend.get_all_settings("default").await {
Ok(db_map) if !db_map.is_empty() => {
let existing = Settings::from_db_map(&db_map);
self.settings.merge_from(&existing);
tracing::info!("Loaded {} existing settings from database", db_map.len());
true
}
Ok(_) => false,
Err(e) => {
tracing::debug!("Could not load existing settings: {}", e);
false
}
}
} else {
false
}
} else {
loaded
};
// Suppress unused variable warning when only one backend is compiled.
let _ = loaded;
}
/// Save settings to the database and `~/.ironclaw/.env`, then print summary.
async fn save_and_summarize(&mut self) -> Result<(), SetupError> {
self.settings.onboard_completed = true;
// Write all settings to the database (whichever backend is active).
{
let db_map = self.settings.to_db_map();
let saved = false;
// Final persist (idempotent — earlier incremental saves already wrote
// most settings, but this ensures onboard_completed is saved).
let saved = self.persist_settings().await?;
#[cfg(feature = "postgres")]
let saved = if !saved {
if let Some(ref pool) = self.db_pool {
let store = crate::history::Store::from_pool(pool.clone());
store
.set_all_settings("default", &db_map)
.await
.map_err(|e| {
SetupError::Database(format!(
"Failed to save settings to database: {}",
e
))
})?;
true
} else {
false
}
} else {
saved
};
#[cfg(feature = "libsql")]
let saved = if !saved {
if let Some(ref backend) = self.db_backend {
use crate::db::SettingsStore as _;
backend
.set_all_settings("default", &db_map)
.await
.map_err(|e| {
SetupError::Database(format!(
"Failed to save settings to database: {}",
e
))
})?;
true
} else {
false
}
} else {
saved
};
if !saved {
return Err(SetupError::Database(
"No database connection, cannot save settings".to_string(),
));
}
if !saved {
return Err(SetupError::Database(
"No database connection, cannot save settings".to_string(),
));
}
// Persist database bootstrap vars to ~/.ironclaw/.env.
// These are the chicken-and-egg settings: we need them to decide
// which database to connect to, so they can't live in the database.
{
let mut env_vars: Vec<(&str, String)> = Vec::new();
if let Some(ref backend) = self.settings.database_backend {
env_vars.push(("DATABASE_BACKEND", backend.clone()));
}
if let Some(ref url) = self.settings.database_url {
env_vars.push(("DATABASE_URL", url.clone()));
}
if let Some(ref path) = self.settings.libsql_path {
env_vars.push(("LIBSQL_PATH", path.clone()));
}
if let Some(ref url) = self.settings.libsql_url {
env_vars.push(("LIBSQL_URL", url.clone()));
}
// LLM bootstrap vars: same chicken-and-egg problem as DATABASE_BACKEND.
// Config::from_env() needs the backend before the DB is connected.
if let Some(ref backend) = self.settings.llm_backend {
env_vars.push(("LLM_BACKEND", backend.clone()));
}
if let Some(ref url) = self.settings.openai_compatible_base_url {
env_vars.push(("LLM_BASE_URL", url.clone()));
}
if let Some(ref url) = self.settings.ollama_base_url {
env_vars.push(("OLLAMA_BASE_URL", url.clone()));
}
// Always write ONBOARD_COMPLETED so that check_onboard_needed()
// (which runs before the DB is connected) knows to skip re-onboarding.
env_vars.push(("ONBOARD_COMPLETED", "true".to_string()));
if !env_vars.is_empty() {
let pairs: Vec<(&str, &str)> =
env_vars.iter().map(|(k, v)| (*k, v.as_str())).collect();
crate::bootstrap::save_bootstrap_env(&pairs).map_err(|e| {
SetupError::Io(std::io::Error::other(format!(
"Failed to save bootstrap env to .env: {}",
e
)))
})?;
}
}
// Write bootstrap env (also idempotent)
self.write_bootstrap_env()?;
println!();
print_success("Configuration saved to database");