mirror of
https://github.com/outbackdingo/optimclaw.git
synced 2026-09-02 09:39:37 +00:00
feat: chat onboarding and routine advisor (#927)
* feat: port NPA psychographic profiling system into IronClaw
Port the complete psychographic profiling system from NPA into IronClaw,
including enriched profile schema, conversational onboarding, profile
evolution, and three-tier prompt augmentation.
Personal onboarding moved from wizard Step 9 to first assistant
interaction per maintainer feedback — the First Contact system prompt
block now instructs the LLM to conduct a natural onboarding conversation
that builds the psychographic profile via memory_write.
Changes:
- Enrich profile.rs with 5 new structs, 9-dimension analysis framework,
custom deserializers for backward compatibility, and rendering methods
- Add conversational onboarding engine with one-step-removed questioning
technique, personality framework, and confidence-scored profile generation
- Add profile evolution with confidence gating, analysis metadata tracking,
and weekly update routine
- Replace thin interaction style injection with three-tier system gated on
confidence > 0.6 and profile recency
- Replace wizard Step 9 with First Contact system prompt block that drives
conversational onboarding during the user's first interaction
- Add autonomy progression to SOUL.md seed and personality framework to
AGENTS.md seed
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat: replace chat-based onboarding with bootstrap greeting and workspace seeds
Remove the interactive onboarding_chat.rs engine in favor of a simpler
bootstrap flow: fresh workspaces get a proactive LLM greeting that
naturally profiles the user. Identity files are now seeded from
src/workspace/seeds/ instead of being hardcoded. Also removes the
identity-file write protection (seeds are now managed), adds routine
advisor integration, and includes an e2e trace for bootstrap greeting.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* feat(safety): sanitize identity file writes via Sanitizer to prevent prompt injection
Identity files (SOUL.md, AGENTS.md, USER.md, IDENTITY.md) are injected into
every system prompt. Rather than hard-blocking writes (which broke onboarding),
scan content through the existing Sanitizer and reject writes with High/Critical
severity injection patterns. Medium/Low warnings are logged but allowed.
Also clarifies AGENTS.md identity file roles (USER.md = user info, IDENTITY.md =
agent identity) and adds IDENTITY.md setup as an explicit bootstrap step.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* docs: update profile_onboarding_completed comment to reflect current wiring
The field is now actively used by the agent loop to suppress BOOTSTRAP.md
injection — remove the stale "not yet wired" TODO.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(setup): use env_or_override for NEARAI_API_KEY in model fetch config
When the user authenticates via NEAR AI Cloud API key (option 4),
api_key_login() stores the key via set_runtime_env(). But
build_nearai_model_fetch_config() was using std::env::var() which
doesn't check the runtime overlay — so model listing fell back to
session-token auth and re-triggered the interactive NEAR AI
authentication menu.
Switch to env_or_override() which checks both real env vars and the
runtime overlay.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(agent): correct channel/user_id in bootstrap greeting persist call
persist_assistant_response was called with channel="default",
user_id="system" but the assistant thread was created via
get_or_create_assistant_conversation("default", "gateway") which owns
the conversation as user_id="default", channel="gateway". The mismatch
caused ensure_writable_conversation to reject the write with:
WARN Rejected write for unavailable thread id user=system channel=default
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(web): remove all inline event handlers for CSP compliance
The Content-Security-Policy header (added in f48fe95) blocks inline JS
via script-src 'self'. All onclick/onchange attributes in index.html
are replaced with getElementById().addEventListener() calls. Dynamic
inline handlers in app.js (jobs, routines, memory breadcrumb, code
blocks, TEE report) are replaced with data-action attributes and a
single delegated click handler on document.
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(agent): align bootstrap message user/channel and update fixture schema field
- Bootstrap IncomingMessage now uses ("default", "gateway") consistently
with persist and session registration calls
- Update bootstrap_greeting.json fixture: schema_version → version to
match current PROFILE_JSON_SCHEMA
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: cargo fmt
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(safety): address PR review — expand injection scanning and harden profile sync
- BOOTSTRAP.md: fix target "profile" → "context/profile.json" so the
write hits the correct path and triggers profile sync
- IDENTITY_FILES: add context/assistant-directives.md to the scanned
set since it is also injected into the system prompt
- sync_profile_documents(): scan derived USER.md and assistant-directives
content through Sanitizer before writing, rejecting High/Critical
injection patterns
- profile_evolution_prompt(): wrap recent_messages_summary in <user_data>
delimiters with untrusted-data instruction to mitigate indirect
prompt injection
- routine-advisor skill: update cron examples from 6-field to standard
5-field format for consistency with routine_create tool docs
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* style: cargo fmt
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 <[email protected]>
* fix(setup): detect env-provided LLM keys during quick-mode onboarding
Quick-mode wizard now checks LLM_BACKEND, NEARAI_API_KEY,
ANTHROPIC_API_KEY, and OPENAI_API_KEY env vars to pre-populate
the provider setting, so users aren't re-prompted for credentials
they already supplied. Also teaches setup_nearai() to recognize
NEARAI_API_KEY from env (previously only checked session tokens).
Includes web UI cleanup (remove duplicate event listeners) and
e2e test response count adjustment.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(test): update routine_create_list to expect 7-field normalized cron
The cron normalizer now always expands to 7-field format, so the
stored schedule is "0 0 9 * * * *" not "0 0 9 * * *".
[skip-regression-check]
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* feat(setup): skip LLM provider prompts when NEARAI_API_KEY is present
In quick mode, if NEARAI_API_KEY is set in the environment and the
backend was auto-detected as nearai, skip the interactive inference
provider and model selection steps. The API key is persisted to the
secrets store and a default model is set automatically.
Also simplify the static fallback model list for nearai to a single
default entry.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: unify default model, static bootstrap greeting, and web UI cleanup
- Add DEFAULT_MODEL const and default_models() fallback list in
llm/nearai_chat.rs; use from config, wizard, and .env.example so the
default model is defined in one place
- Restore multi-model fallback list in setup wizard (was reduced to 1)
- Move BOOTSTRAP_GREETING to module-level const (out of run() body)
- Replace LLM-based bootstrap with static greeting (persist to DB before
channels start, then broadcast — eliminates startup LLM call and race)
- Fix double env::var read for NEARAI_API_KEY in quick setup path
- Move thread sidebar buttons into threads-section-header (web UI)
- Remove orphaned .thread-sidebar-header CSS and fix double blank line
- Update bootstrap e2e test for static greeting (no LLM trace needed)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix(safety): move prompt injection scanning into Workspace write/append
Addresses PR #927 review comments (#1, #3) — identity file write
protection and unsanitized profile fields in system prompt.
Instead of scanning at the tool layer (memory.rs) or the sync layer
(sync_profile_documents), injection scanning now lives in
Workspace::write() and Workspace::append() for all files that are
injected into the system prompt. This ensures every code path that
writes to these files is protected, including future ones.
- Add SYSTEM_PROMPT_FILES const and reject_if_injected() in workspace
- Add WorkspaceError::InjectionRejected variant
- Add map_write_err() in memory.rs to convert InjectionRejected to
ToolError::NotAuthorized
- Remove redundant IDENTITY_FILES/Sanitizer from memory.rs
- Remove redundant sanitizer calls from sync_profile_documents()
- Move sanitization tests to workspace::tests
- Existing integration test (test_memory_write_rejects_injection)
continues to pass through the new path
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address Copilot review — merge marker order, orphan thread, stale fixture
- merge_profile_section: search for END marker after BEGIN position to
avoid matching a stray END earlier in the file
- Bootstrap phase 2: use get_or_create_session + Thread::with_id instead
of resolve_thread(None) to avoid creating an orphan thread
- setup_nearai: use env_or_override for NEARAI_API_KEY consistency with
runtime overlay
- Delete orphaned bootstrap_greeting.json fixture (no test references it)
- Add test_merge_end_marker_must_follow_begin regression test
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: fmt agent_loop.rs (CI stable rustfmt)
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: lazy-init sanitizer, check profile non-empty before skipping bootstrap
Address Copilot review:
- Use LazyLock<Sanitizer> to avoid rebuilding Aho-Corasick + regexes
on every workspace write
- has_profile check now requires non-empty content, not just file
existence, to prevent empty profile.json from suppressing onboarding
- Add seed_tests integration tests (libsql-backed) verifying:
- Empty profile.json does not suppress BOOTSTRAP.md seeding
- Non-empty profile.json correctly suppresses bootstrap for upgrades
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: duplicate language handler, empty LLM_BACKEND, test_rig style
Address Copilot review on PR #927:
- Remove duplicate language-option click listeners (delegated
data-action handler already covers them)
- Guard LLM_BACKEND env prefill against empty string to prevent
suppressing API-key-based auto-detection
- Use destructured local `keep_bootstrap` instead of `self.keep_bootstrap`
in test_rig for consistency after destructure
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: update stale BOOTSTRAP.md write-protection comment [skip-regression-check]
BOOTSTRAP.md is now in SYSTEM_PROMPT_FILES and gets injection scanning
on write. The old comment incorrectly stated it was not write-protected.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: replace debug_assert panics with graceful error returns [skip-regression-check]
debug_assert! in execute_tool_with_safety and JobContext::transition_to
panicked in test builds before the graceful error path could run.
Existing tests (test_cancel_job_completed, test_execute_empty_tool_name_returns_not_found)
already cover these paths — they were the ones failing.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address Copilot review — schema label, env var check, path normalization, profile validation
1. Label ANALYSIS_FRAMEWORK and PROFILE_JSON_SCHEMA sections separately
in bootstrap prompt so the LLM knows which blob is the target structure.
2. Wizard quick-mode backend auto-detection now rejects empty env vars
(std::env::var().is_ok_and(|v| !v.is_empty())) to avoid selecting the
wrong backend when e.g. NEARAI_API_KEY="" is set.
3. Normalize the target path before comparing with paths::PROFILE in
memory_write so non-canonical variants like "context//profile.json"
still trigger profile sync.
4. seed_if_empty now requires valid JSON parse of context/profile.json
before treating it as a populated profile. Corrupted content no longer
permanently suppresses bootstrap seeding.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* style: cargo fmt
* fix: address Copilot review — append scan, profile validation, env_or_override
1. Workspace::append() now scans the combined content (existing + new)
for prompt injection, not just the appended chunk. Prevents split-
injection evasion across multiple appends.
2. seed_if_empty() now deserializes into PsychographicProfile instead of
serde_json::Value for profile validation. Stray/legacy JSON that
doesn't match the expected schema no longer suppresses bootstrap.
3. Wizard quick-mode backend auto-detection now uses env_or_override()
to honor runtime overlays and injected secrets. LLM_BACKEND value
is trimmed before storage.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* test: add bootstrap_onboarding_clears_bootstrap E2E trace test
Exercises the full onboarding flow end-to-end:
1. Bootstrap greeting fires automatically on fresh workspace
2. User converses for 3 turns (name, tools, work style)
3. Agent writes psychographic profile to context/profile.json
4. Profile sync generates USER.md and assistant-directives.md
5. Agent writes IDENTITY.md (chosen persona)
6. Agent clears BOOTSTRAP.md via memory_write(target: "bootstrap")
Verifies:
- BOOTSTRAP.md is non-empty before onboarding, empty after
- bootstrap_completed flag is set
- Profile contains expected user data (name, profession, interests)
- USER.md contains profile-derived content (name, tone, profession)
- Assistant-directives.md references user and communication style
- IDENTITY.md contains agent's chosen persona name
- All memory_write calls succeed
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
* fix: address Copilot review — slash collapse, env_or_override, cron trim [skip-regression-check]
1. memory.rs path normalization now uses the same char-by-char loop as
Workspace::normalize_path() to fully collapse consecutive slashes
(e.g. "context///profile.json" → "context/profile.json").
2. Quick-mode NEARAI_API_KEY check (line 239) now uses env_or_override()
consistently with the backend auto-detection block above it.
3. normalize_cron_expression() trims input before field counting so the
passthrough branch (7+ fields) also strips whitespace.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
---------
Co-authored-by: Jay Zalowitz <[email protected]>
Co-authored-by: Claude Opus 4.6 <[email protected]>
This commit is contained in:
Illia Polosukhin
co-authored by
Jay Zalowitz
Claude Opus 4.6
Jay Zalowitz
Claude Opus 4.6
parent
3a523347b0
commit
806d402876
+97
-24
@@ -217,13 +217,52 @@ impl SetupWizard {
|
||||
self.auto_setup_security().await?;
|
||||
self.persist_after_step().await;
|
||||
|
||||
print_step(1, 2, "Inference Provider");
|
||||
self.step_inference_provider().await?;
|
||||
self.persist_after_step().await;
|
||||
// Pre-populate backend from env so step_inference_provider
|
||||
// can offer "Keep current provider?" instead of asking from scratch.
|
||||
if self.settings.llm_backend.is_none() {
|
||||
use crate::config::helpers::env_or_override;
|
||||
if let Some(b) = env_or_override("LLM_BACKEND")
|
||||
&& !b.trim().is_empty()
|
||||
{
|
||||
self.settings.llm_backend = Some(b.trim().to_string());
|
||||
} else if env_or_override("NEARAI_API_KEY").is_some() {
|
||||
self.settings.llm_backend = Some("nearai".to_string());
|
||||
} else if env_or_override("ANTHROPIC_API_KEY").is_some()
|
||||
|| env_or_override("ANTHROPIC_OAUTH_TOKEN").is_some()
|
||||
{
|
||||
self.settings.llm_backend = Some("anthropic".to_string());
|
||||
} else if env_or_override("OPENAI_API_KEY").is_some() {
|
||||
self.settings.llm_backend = Some("openai".to_string());
|
||||
}
|
||||
}
|
||||
|
||||
print_step(2, 2, "Model Selection");
|
||||
self.step_model_selection().await?;
|
||||
self.persist_after_step().await;
|
||||
if let Some(api_key) = crate::config::helpers::env_or_override("NEARAI_API_KEY")
|
||||
&& self.settings.llm_backend.as_deref() == Some("nearai")
|
||||
{
|
||||
// NEARAI_API_KEY is set and backend auto-detected — skip interactive prompts
|
||||
print_info("NEARAI_API_KEY found — using NEAR AI provider");
|
||||
if let Ok(ctx) = self.init_secrets_context().await {
|
||||
let key = SecretString::from(api_key.clone());
|
||||
if let Err(e) = ctx.save_secret("llm_nearai_api_key", &key).await {
|
||||
tracing::warn!("Failed to persist NEARAI_API_KEY to secrets: {}", e);
|
||||
}
|
||||
}
|
||||
self.llm_api_key = Some(SecretString::from(api_key));
|
||||
if self.settings.selected_model.is_none() {
|
||||
let default = crate::llm::DEFAULT_MODEL;
|
||||
self.settings.selected_model = Some(default.to_string());
|
||||
print_info(&format!("Using default model: {default}"));
|
||||
}
|
||||
self.persist_after_step().await;
|
||||
} else {
|
||||
print_step(1, 2, "Inference Provider");
|
||||
self.step_inference_provider().await?;
|
||||
self.persist_after_step().await;
|
||||
|
||||
print_step(2, 2, "Model Selection");
|
||||
self.step_model_selection().await?;
|
||||
self.persist_after_step().await;
|
||||
}
|
||||
} else {
|
||||
let total_steps = 9;
|
||||
|
||||
@@ -285,6 +324,10 @@ impl SetupWizard {
|
||||
print_step(9, total_steps, "Background Tasks");
|
||||
self.step_heartbeat()?;
|
||||
self.persist_after_step().await;
|
||||
|
||||
// Personal onboarding now happens conversationally during the
|
||||
// user's first interaction with the assistant (see bootstrap
|
||||
// block in workspace/mod.rs system_prompt_for_context).
|
||||
}
|
||||
|
||||
// Save settings and print summary
|
||||
@@ -1195,6 +1238,27 @@ impl SetupWizard {
|
||||
async fn setup_nearai(&mut self) -> Result<(), SetupError> {
|
||||
self.set_llm_backend_preserving_model("nearai");
|
||||
|
||||
// Check if NEARAI_API_KEY is already provided via environment or runtime overlay
|
||||
if let Some(existing) = crate::config::helpers::env_or_override("NEARAI_API_KEY")
|
||||
&& !existing.is_empty()
|
||||
{
|
||||
print_info(&format!(
|
||||
"NEARAI_API_KEY found: {}",
|
||||
mask_api_key(&existing)
|
||||
));
|
||||
if confirm("Use this key?", true).map_err(SetupError::Io)? {
|
||||
if let Ok(ctx) = self.init_secrets_context().await {
|
||||
let key = SecretString::from(existing.clone());
|
||||
if let Err(e) = ctx.save_secret("llm_nearai_api_key", &key).await {
|
||||
tracing::warn!("Failed to persist NEARAI_API_KEY to secrets: {}", e);
|
||||
}
|
||||
}
|
||||
self.llm_api_key = Some(SecretString::from(existing));
|
||||
print_success("NEAR AI configured (from env)");
|
||||
return Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
// Check if we already have a session
|
||||
if let Some(ref session) = self.session_manager
|
||||
&& session.has_token().await
|
||||
@@ -1623,25 +1687,8 @@ impl SetupWizard {
|
||||
if backend == "nearai" {
|
||||
// NEAR AI: use existing provider list_models()
|
||||
let fetched = self.fetch_nearai_models().await;
|
||||
let default_models: Vec<(String, String)> = vec![
|
||||
(
|
||||
"zai-org/GLM-latest".into(),
|
||||
"GLM Latest (default, fast)".into(),
|
||||
),
|
||||
(
|
||||
"anthropic::claude-sonnet-4-20250514".into(),
|
||||
"Claude Sonnet 4 (best quality)".into(),
|
||||
),
|
||||
(
|
||||
"openai::gpt-5.3-codex".into(),
|
||||
"GPT-5.3 Codex (flagship)".into(),
|
||||
),
|
||||
("openai::gpt-5.2".into(), "GPT-5.2".into()),
|
||||
("openai::gpt-4o".into(), "GPT-4o".into()),
|
||||
];
|
||||
|
||||
let models = if fetched.is_empty() {
|
||||
default_models
|
||||
crate::llm::default_models()
|
||||
} else {
|
||||
fetched.iter().map(|m| (m.clone(), m.clone())).collect()
|
||||
};
|
||||
@@ -3839,4 +3886,30 @@ mod tests {
|
||||
"config should have no api_key when env var is empty"
|
||||
);
|
||||
}
|
||||
|
||||
/// Regression: API key set via set_runtime_env (interactive api_key_login
|
||||
/// path) must be picked up by build_nearai_model_fetch_config so that
|
||||
/// model listing doesn't fall back to session-token auth and re-trigger
|
||||
/// the NEAR AI authentication menu.
|
||||
#[test]
|
||||
fn test_build_nearai_model_fetch_config_picks_up_runtime_env() {
|
||||
let _lock = ENV_MUTEX.lock().unwrap();
|
||||
// Ensure the real env var is unset so the only source is the overlay.
|
||||
let _guard = EnvGuard::clear("NEARAI_API_KEY");
|
||||
|
||||
crate::config::helpers::set_runtime_env("NEARAI_API_KEY", "test-key-from-overlay");
|
||||
let config = build_nearai_model_fetch_config();
|
||||
|
||||
// Clean up runtime overlay
|
||||
crate::config::helpers::set_runtime_env("NEARAI_API_KEY", "");
|
||||
|
||||
assert!(
|
||||
config.nearai.api_key.is_some(),
|
||||
"config must pick up NEARAI_API_KEY from runtime overlay"
|
||||
);
|
||||
assert_eq!(
|
||||
config.nearai.base_url, "https://cloud-api.near.ai",
|
||||
"API key auth must use cloud-api base URL"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user