Compare commits

..
Author SHA1 Message Date
[email protected]andClaude Opus 4.6 c5b1cdc2f9 fix: address PR review comments (round 2)
Human reviewer (zmanian):
- H1: Accept configurable embedding dimension in LanceDbVectorStore::new()
  instead of hardcoding 1536. Dimension is sourced from
  EmbeddingProvider::dimension() at init time.
- H2: Skip double-write of embeddings to DB when external vector store
  is active (pass None to insert_chunk for embedding column).
- H3: Update PR title from "refactor" to "feat" (net-new feature).
- H4: Document non-atomic update_embedding in struct doc comment.

Bot reviewer (Copilot):
- Cache LanceDB table handle via tokio::sync::OnceCell (avoid
  open_table per operation).
- Cache Arc<Schema> in struct (avoid rebuilding per insert).
- Fix error variants: ChunkingFailed → EmbeddingFailed for LanceDB
  store/delete operations.
- Propagate store_embedding errors in reindex_document instead of
  warn-only (prevents silent data loss).
- Prefetch document metadata map in backfill_embeddings to avoid N+1
  queries.
- Add lancedb feature + protoc to CI test matrix so LanceDB tests
  actually run on Linux.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-08 17:34:11 -07:00
[email protected] fe17196367 Merge remote-tracking branch 'origin/main' into feat/lancedb-backend 2026-03-08 14:10:48 -07:00
605a4ba46e fix(docker): bind postgres to localhost only (#686)
5432:5432 → 127.0.0.1:5432:5432 — the default docker-compose.yml
exposed postgres on all interfaces, making it reachable from the
local network in any docker compose deployment.

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Zaki Manian <[email protected]>
2026-03-08 20:55:57 +00:00
fe91ba2ab4 fix(repl): skip /quit on EOF when stdin is not a TTY (#724)
When running as a launchd/systemd daemon, stdin is /dev/null.
rustyline reads EOF immediately and the REPL thread was sending
a /quit message, causing the agent to shut down right after
startup — making service mode non-functional on both macOS and Linux.

Fix: check std::io::stdin().is_terminal() before sending /quit on
EOF. In daemon mode (no TTY) the REPL thread exits silently, leaving
other channels (gateway, telegram, …) running as expected.

Fixes #723

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Zaki Manian <[email protected]>
2026-03-08 20:40:56 +00:00
da2569bb77 fix(web): prevent Enter key from sending message during IME composition (#715)
Co-authored-by: Zaki Manian <[email protected]>
2026-03-08 20:40:31 +00:00
732b3ecfeb test(agent): wire TestRig job tools through the scheduler (#716)
Align TestRig with the production agent wiring so create_job exercises the real scheduler path instead of silently falling back to an unscheduled context-only job. Tighten the e2e assertion to lock in the in-progress scheduler behavior for future refactors.

Made-with: Cursor

Co-authored-by: Zaki Manian <[email protected]>
2026-03-08 20:40:03 +00:00
461d7712e8 fix(config): init_secrets no longer overwrites entire config (#726)
* fix(config): init_secrets no longer overwrites entire config

init_secrets() was calling Config::from_db_with_toml() to re-resolve
config after injecting credentials. This rebuilt the entire config from
env/DB/defaults, nuking all other config fields (agent, safety, tools,
etc.) even though only LlmConfig depends on injected credentials.

This caused 5 CI test failures: the test rig's carefully chosen config
values (max_tool_iterations, allow_local_tools, etc.) were silently
overwritten with production defaults after secret injection.

Fix: add Config::re_resolve_llm() that re-resolves only the LLM config
after credential injection, leaving all other config fields untouched.
Also fix TraceLlm::complete() to skip ToolCalls steps when called in
force_text mode (iteration limit).

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(test): update test to match TraceLlm::complete() skip-tool-calls behavior [skip-regression-check]

TraceLlm::complete() now skips ToolCalls steps (force_text mode) instead
of erroring. Update the test to verify it skips past a ToolCalls step and
returns the subsequent Text step.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Zaki <[email protected]>
2026-03-08 13:32:42 -07:00
ReidandGitHub 1c5117eded feat: add PID-based gateway lock to prevent multiple instances (#717) 2026-03-08 13:17:46 -07:00
ReidandGitHub 33b02eabb7 fix(cli): status command ignores config.toml and settings.json (#354) (#734) 2026-03-08 13:17:43 -07:00
ReidandGitHub 068ad2d4b7 Fix single-message mode to exit after one turn when background channels are enabled (#719) 2026-03-08 12:54:18 -07:00
[email protected]andClaude Opus 4.6 e84448d5ea ci: install protoc for lancedb feature builds
LanceDB depends on lance-encoding which requires protoc for protobuf
compilation. Add arduino/setup-protoc to all CI jobs that build with
--all-features.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-08 01:23:11 -08:00
[email protected]andClaude Opus 4.6 3f92b9cb24 Merge origin/main into feat/lancedb-backend
Resolve merge conflicts from main's config refactoring (config.rs split
into config/ directory), app builder pattern (src/app.rs), module renames
(libsql_backend → libsql), and new RankedResult.document_path field.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-08 01:19:25 -08:00
56b7218897 fix(setup): preserve model name when re-running onboarding with same provider (#600) (#694)
Each provider setup function unconditionally cleared selected_model,
so re-running the wizard with "Keep current provider? Yes" would lose
the model name, forcing the user to re-select it every time.

Now only clears selected_model when the backend actually changes
(old model may be invalid for the new provider). When keeping the
same provider, the model is preserved and Step 4 shows the
"Keep current model" prompt.

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:32:02 +00:00
200aed16cd feat: configurable LLM request timeout via LLM_REQUEST_TIMEOUT_SECS (#615) (#630)
Add LLM_REQUEST_TIMEOUT_SECS env var (default: 120) to configure the
HTTP request timeout for LLM API calls. Primarily useful for local
models (Ollama, vLLM, LM Studio) that need more time for prompt
evaluation on consumer hardware.

The timeout is applied to the NearAI provider's HTTP client. Other
providers (Anthropic, OpenAI) use rig-core's default client.

- Add request_timeout_secs field to LlmConfig
- Thread timeout through create_llm_provider -> NearAiChatProvider
- Add NearAiChatProvider::new_with_timeout constructor
- Add .env.example documentation
- 2 regression tests for default and custom timeout values

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:30:52 +00:00
4c0275bcdc fix(setup): initialize secrets crypto for env-var security option (#666) (#706)
The "Environment variable" option in the setup wizard's security step
generated a master key but never initialized `secrets_crypto`, causing
subsequent API key saves to fail silently. Fix by:

1. Creating SecretsCrypto from the generated key (matching keychain path)
2. Storing the key hex in settings for write_bootstrap_env to persist
3. Auto-writing SECRETS_MASTER_KEY to ~/.ironclaw/.env
4. Using inject_single_var for thread-safe env overlay
5. Fixing misleading message (shell profiles don't work, only .env)

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:30:02 +00:00
272d31797e chore: remove dead code (#648) (#703)
* chore: remove dead code (LlmEvaluator, chunk_by_paragraphs, bundled channel installer, Reasoning::safety)

Delete unused code flagged in #648:
- evaluation/success.rs: delete LlmEvaluator struct/impl, remove #[allow(dead_code)] from RuleBasedEvaluator methods
- workspace/chunker.rs: delete chunk_by_paragraphs() and its tests (zero production callers)
- extensions/manager.rs: delete install_bundled_channel_from_artifacts() (hot-activation never shipped)
- llm/reasoning.rs: remove unused safety field from Reasoning struct; cascade removal through ContextCompactor, HeartbeatRunner, LlmSoftwareBuilder, and all callers

Closes #648

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: move RuleBasedEvaluator into test module to fix dead_code warning

RuleBasedEvaluator has no production callers -- it was only used in
tests of itself. Moving it into #[cfg(test)] eliminates the clippy
dead_code error that broke CI.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:26:04 +00:00
edff54b0b1 fix: persist /model selection across restarts (#707)
* fix: persist /model selection across restarts

The /model command called set_model() on the LLM provider but never
saved the choice to settings, so the model reverted on restart. Now
persists to both the DB settings store and config.toml.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address CI clippy lint and use spawn_blocking for TOML I/O

- Use struct init syntax instead of field reassignment in test (clippy)
- Wrap sync filesystem operations in spawn_blocking to avoid blocking
  the tokio executor

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt formatting

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address review feedback — handle JoinError, remove exists() guard

- Log warning if spawn_blocking task panics/is cancelled (JoinError)
- Remove toml_path.exists() guard; load_toml already returns Ok(None)
  for missing files, so permission errors are no longer silently skipped

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:10:46 +00:00
[email protected]andClaude Opus 4.6 1e0494e72d refactor: replace LanceDB Database decorator with VectorStore composition
Instead of wrapping all ~80 Database trait methods in a 664-line decorator
(lancedb_wrapper.rs), introduce a 4-method VectorStore trait that any vector
backend can implement. Workspace composes FTS from the database with vector
search from the external store via RRF fusion.

- Add src/workspace/vector_store.rs with VectorStore trait
- Rewrite lancedb_store.rs to implement VectorStore (not wrap Database)
- Delete src/db/lancedb_wrapper.rs (664 lines removed)
- Remove get_chunk_by_id from Database trait and all backends
- Workspace gains with_vector_store() builder for optional composition
- Fix LanceDB tests: bypass_vector_index() for brute-force search
- Fix integration tests: use temp file DB (libSQL :memory: is per-connection)
- Merge duplicate mod tests in config.rs

Net: -724 lines. Adding a new vector backend requires 4 methods, not 80.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-03-08 00:06:04 -08:00
4d61d3eedf fix(routines): resolve message tool channel/target from per-job metadata (#708)
* fix(routines): resolve message tool channel/target from per-job metadata

When a routine's notify.channel is None, the message tool had no way to
resolve channel/target for full-job workers, causing "No target specified"
errors. The previous approach mutated shared global state via
set_message_tool_context(), which also raced with concurrent jobs.

Now the routine's notify config (channel + user) is carried in the job's
metadata JSON, and MessageTool::execute falls back to ctx.metadata when
neither explicit params nor conversation defaults are available. This
eliminates both the None-channel bug and the concurrent-job race.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: apply cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(message): broadcast to all channels when notify.channel is None

Address review feedback:
- Fix stale "see above" comment → "populated below"
- When notify.channel is None, use broadcast_all instead of erroring
  with "No channel specified". This matches NotifyConfig semantics
  where channel=None means "broadcast to all channels"
- Channel resolution is now Option<String>: param → default → metadata → None
- When None, MessageTool uses ChannelManager::broadcast_all(target, response)
  and reports which channels succeeded/failed
- Add regression test for broadcast-all behavior

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use failed channels in error message, remove redundant comment

Address review feedback:
- Use `failed` vec in error message instead of re-querying channel_names
- Remove redundant orphaned comment block in routine_engine.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:04:19 +00:00
df3635d6be feat(timezone): add timezone-aware session context (#671)
* feat(timezone): add timezone-aware session context (#661)

All timestamps were UTC-only, causing daily logs to split at UTC midnight,
cron schedules to fire in UTC, and no quiet hours for heartbeat. This adds
timezone as a per-session property flowing from the client.

Key changes:
- New `src/timezone.rs` module with resolution chain, parsing, and detection
- `IncomingMessage` carries optional timezone from client
- `JobContext.user_timezone` flows timezone to tools
- `next_cron_fire()` accepts timezone for schedule evaluation
- `Trigger::Cron` stores optional timezone (backward-compatible)
- Workspace gains `_tz` variants for daily logs and system prompt
- Heartbeat supports quiet hours (`HEARTBEAT_QUIET_START/END`)
- Web frontend sends `Intl.DateTimeFormat().resolvedOptions().timeZone`
- REPL auto-detects system timezone
- `DEFAULT_TIMEZONE` env var / settings for server-wide default

Storage stays UTC. Conversion happens at display boundaries.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(timezone): address review feedback on timezone-aware sessions

- Validate quiet hours values (0-23) in HeartbeatConfig::resolve()
- Fall back to settings values when env vars are unset for quiet hours
- Validate IANA timezone strings in routine_create/update with parse_timezone
- Add timezone field to routine_create tool schema
- Allow standalone timezone update on cron routines without changing schedule
- Return path from append_daily_log_tz to avoid TOCTOU race at midnight
- Delegate append_daily_log to append_daily_log_tz(entry, UTC) to avoid drift
- Preserve timezone through approval flow via PendingApproval.user_timezone
- Improve test_today_in_tz to not depend on hardcoded year
- Add 3 regression tests for quiet hours config validation

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in routine.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(timezone): address second round of review feedback

- Remove .claude/scheduled_tasks.lock from repo and add to .gitignore
- Store resolved timezone (not raw message.timezone) in PendingApproval
- Carry forward user_timezone through chained approvals in thread_ops
- Wire quiet_hours_start/end from config to HeartbeatRunner
- Support X-Timezone header as fallback in chat_send_handler

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(timezone): include user's local time in time tool response

The time tool's "now" operation now returns local_iso and timezone
fields based on ctx.user_timezone, so the LLM can report time in
the user's timezone instead of always UTC.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting in time.rs

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix(timezone): address Copilot review round 3 — validation, deterministic tests, schema fixes

- Validate DEFAULT_TIMEZONE and HEARTBEAT_TIMEZONE at config load time
- Add timezone field to HeartbeatSettings and config::HeartbeatConfig
- Wire heartbeat timezone from config through agent_loop to HeartbeatRunner
- Add timezone to routine_update tool schema (was accepted but not advertised)
- Error on schedule/timezone update for non-cron routines
- Validate timezone in Trigger::from_db (coerce invalid to None with warning)
- Validate timezone in approval path (thread_ops.rs) before overwriting
- Time tool always includes timezone/local_iso fields (fallback to UTC)
- Make quiet hours tests deterministic using current UTC hour
- Add regression tests for config validation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-08 08:01:56 +00:00
ReidandGitHub a20e19ab16 fix: sanitize HTML error bodies from MCP servers to prevent web UI white screen (#263) (#656)
* fix: sanitize HTML error bodies from MCP servers to prevent web UI white screen (#263)

* style: fix cargo fmt formatting in sanitize_error_body tests
2026-03-08 02:53:02 +00:00
3b57d5bec9 chore: add reviewer-feedback guardrails (CLAUDE.md, pre-commit hook, skill) (#665)
* chore: add reviewer-feedback guardrails (CLAUDE.md, pre-commit hook, skill)

Analysis of ~50 PRs from the past week identified 10 recurring themes
in Copilot and Gemini code review comments. This change addresses them
at development time through three layers:

1. CLAUDE.md additions (7 new rules):
   - Transaction safety for multi-step DB operations
   - UTF-8 string safety (no byte-index slicing)
   - Case-insensitive comparisons for paths/media types
   - Decorator/wrapper trait method delegation
   - Sensitive data redaction in logs/SSE
   - tempfile crate for test temporary files
   - Trust boundaries for worker container data

2. Pre-commit hook (scripts/pre-commit-safety.sh):
   Mechanical checks for unsafe byte slicing, case-sensitive
   extension comparisons, hardcoded /tmp paths, unredacted
   tool parameter logging, and non-transactional DB operations.
   Installed via dev-setup.sh alongside existing commit-msg hook.

3. Review checklist skill (skills/review-checklist/SKILL.md):
   Activates on "review"/"merge" keywords. Covers the judgment-based
   items that can't be linted: transaction safety, SSRF validation,
   approval checks, decorator delegation, test quality, and doc accuracy.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review feedback on pre-commit-safety.sh

- Cache diff output in variable to avoid ~10 redundant git diff calls (Gemini)
- Add early exit when no .rs files are changed (Gemini)
- Fix header comment: list all 5 checks, not just 4 (Copilot)
- Fix check 2 comment: only mentions file extensions, not media types (Copilot)
- Add resolve_base_ref() with fallback candidates instead of hardcoded
  origin/main for standalone mode (Copilot)
- TX check: use -W (function context) to reduce false positives, honor
  // safety: suppression, print triggering lines (Copilot)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-07 21:20:37 +00:00
11c5e25422 feat(setup): Anthropic OAuth onboarding with setup-token support (#384)
* feat(setup): add Anthropic OAuth and Codex OAuth onboarding flows

Add OAuth token authentication as an alternative to API keys during
onboarding for both Anthropic (via `claude login`) and OpenAI/Codex
(via `~/.codex/auth.json`).

Key changes:
- New `AnthropicOAuthProvider` using `Authorization: Bearer` header
  (rig-core hardcodes `x-api-key` which rejects OAuth tokens)
- Wizard auth method selector: "Direct API Key" vs "OAuth Token"
  for both Anthropic and OpenAI providers
- Codex token extraction from `$CODEX_HOME/auth.json` / `~/.codex/auth.json`
- Claude Code sandbox sub-step in Docker setup (checks for credentials)
- Secret injection mappings for `ANTHROPIC_OAUTH_TOKEN` and `CODEX_OAUTH_TOKEN`
- `CODEX_OAUTH_TOKEN` falls back to `OPENAI_API_KEY` (same Bearer auth)

Supersedes #143 which had a broken auth flow (OAuth token sent as
x-api-key → 401). Credit to @bigguybobby for the original approach.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist OAuth tokens in bootstrap .env and re-extract at startup

OAuth tokens stored only in the secrets DB were invisible to
Config::from_env() which runs before the DB connects (chicken-and-egg).

Two fixes:
1. write_bootstrap_env() now persists ANTHROPIC_OAUTH_TOKEN and
   CODEX_OAUTH_TOKEN to ~/.ironclaw/.env (same pattern as NEARAI_API_KEY)
2. main.rs re-extracts a fresh token from the OS credential store
   (macOS Keychain / ~/.claude/.credentials.json) before config resolution,
   handling token expiry (8-12h) gracefully

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: persist all LLM credentials in bootstrap .env, not just NEAR AI

All providers had the same chicken-and-egg issue: API keys stored in the
secrets DB were invisible to Config::from_env() which runs before DB
connects. Only NEARAI_API_KEY was written to bootstrap .env.

Now write_bootstrap_env() persists all credential env vars:
NEARAI_API_KEY, ANTHROPIC_API_KEY, ANTHROPIC_OAUTH_TOKEN, OPENAI_API_KEY,
CODEX_OAUTH_TOKEN, LLM_API_KEY, TINFOIL_API_KEY.

Also: setup_api_key_provider() now sets the env var during the wizard
session so write_bootstrap_env() can pick it up.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address security review findings for OAuth onboarding

- Extract "oauth-placeholder" to named OAUTH_PLACEHOLDER constant shared
  across config and wizard to prevent silent drift
- Document plaintext credential tradeoff in write_bootstrap_env (API keys
  stored with 0o600 permissions, recommend full-disk encryption)
- Add blocking "Press Enter" wait in Anthropic OAuth retry flow so user
  has time to run `claude login` in another terminal
- Add escape hatch from manual OAuth paste back to API key flow (empty
  input switches to setup_api_key_provider)
- Fix Retry-After header: parse u64 seconds into Duration before passing
  to LlmError::RateLimited
- Make config::llm module pub(crate) for constant visibility
- Use .bearer_auth() instead of manual format!("Bearer {}")
- Remove response body from debug log (may contain PII)
- Update Anthropic API version to 2024-10-22

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* security: remove plaintext credentials from bootstrap .env

Credentials (API keys, OAuth tokens) were being written in plaintext to
~/.ironclaw/.env to work around a chicken-and-egg problem: Config::from_env()
runs before the encrypted secrets DB is connected.

Instead of storing secrets on disk, LlmConfig::resolve() now defers
gracefully when credentials are missing — it returns None for the provider
config instead of hard-erroring with MissingRequired. After the DB connects,
AppBuilder::build_all() loads secrets from encrypted storage via
inject_llm_keys_from_secrets() and re-resolves the config.

For Anthropic OAuth tokens (which expire in 8-12h), the secret injection
step also tries the OS credential store (macOS Keychain / Linux
credentials.json) for a fresh token, overriding the potentially stale
copy in the DB.

Changes:
- LlmConfig::resolve(): OpenAI, Anthropic, OpenAI-compatible, and Tinfoil
  all return None instead of MissingRequired when credentials are absent
- write_bootstrap_env(): no longer writes any credential env vars
- inject_llm_keys_from_secrets(): refreshes Anthropic OAuth from OS
  credential store before overlay is finalized
- main.rs: removed OAuth re-extraction hack (no longer needed)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: load OS credential store tokens even without secrets DB

The OAuth token extraction from macOS Keychain / Linux credentials files
was only running inside inject_llm_keys_from_secrets(), which requires
the encrypted secrets DB. When no master key is configured, init_secrets()
returned early — skipping both DB secret loading AND OS credential store
extraction, leaving the Anthropic OAuth token unavailable.

Split into two paths:
- inject_llm_keys_from_secrets(): loads from encrypted DB + OS stores
- inject_os_credentials(): loads from OS stores only (no DB needed)

init_secrets() now calls inject_os_credentials() and re-resolves config
even in the no-master-key early-return path, so `claude login` tokens
are always available regardless of secrets DB state.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: add anthropic-beta header required for OAuth authentication

Anthropic's api.anthropic.com requires the `anthropic-beta: oauth-2025-04-20`
header to accept OAuth Bearer tokens. Without it, the API returns 401
"OAuth authentication is currently not supported."

Also reverts API version to 2023-06-01 since the OAuth beta flag does
not support the 2024-10-22 version (returns 400 "not a valid version").

This was the same bug that caused PR #143's 401 errors — the beta header
was missing entirely.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: Anthropic and OpenAI model resolution respects selected_model

The Anthropic and OpenAI config resolution ignored settings.selected_model
entirely, only checking the provider-specific env var (ANTHROPIC_MODEL,
OPENAI_MODEL) and falling back to a hardcoded default. This meant the
model chosen during onboarding wizard was silently overridden.

Now follows the same pattern as NearAI and OpenAI-compatible:
env var > settings.selected_model > hardcoded default.

Also deduplicated the Anthropic config construction (two identical
branches for API key vs OAuth now share model/base_url resolution).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add provider resolution tests for all LLM backends

Covers deferred resolution (no credentials → None instead of error),
credential presence, model selection fallback chain, and OAuth token
routing for Anthropic, OpenAI, Tinfoil, Ollama, and NearAI.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: handle nested tokens.access_token format in Codex auth.json

Codex CLI stores OAuth tokens in a nested format under
tokens.access_token (ChatGPT OAuth flow), not at the top level.
Also adds ENV_MUTEX to Codex token tests for thread safety.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: remove Codex OAuth onboarding (incompatible with OpenAI API)

Codex CLI OAuth tokens use a different endpoint
(chatgpt.com/backend-api/codex) and the Responses API wire format,
not api.openai.com with Chat Completions. The tokens lack the
model.request scope needed for the platform API, so they can't be
used as drop-in OPENAI_API_KEY replacements.

Removes: extract_codex_oauth_token(), wizard Codex OAuth flow,
CODEX_OAUTH_TOKEN env var support, and related tests.

OpenAI onboarding now uses direct API key only.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix formatting for CI (cargo fmt)

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address Gemini review feedback

- Use ? operator for ANTHROPIC_MODEL/BASE_URL env resolution instead of
  .ok().flatten() to propagate ConfigErrors consistently
- Skip Tool messages without tool_call_id with a warning instead of
  using unwrap_or_default() which would send empty string to Anthropic
- Extract credential check into closure to reduce duplication in
  Claude Code sandbox setup

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor(review): address PR review feedback for OAuth onboarding

- Gate ANTHROPIC_OAUTH_TOKEN resolution to Anthropic provider only
  (was needlessly checked for all registry providers)
- Add 3 regression tests for OAuth config resolution:
  - oauth_token sets placeholder api_key
  - real api_key takes priority over oauth
  - non-Anthropic providers don't pick up oauth_token
- Validate OAuth token prefix (sk-ant-oat) in wizard to catch
  accidentally pasted API keys
- Improve error body read handling in AnthropicOAuthProvider
  (was silently swallowing read errors with unwrap_or_default)
- Remove extra blank line in write_bootstrap_env
- Remove stale blank line in RegistryProviderConfig doc comment

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #384 review comments

Blocker:
- Replace OnceLock<HashMap> with LazyLock<Mutex<HashMap>> for INJECTED_VARS
  so both inject_os_credentials() and inject_llm_keys_from_secrets() merge
  data instead of the second caller silently dropping its entries.

High:
- Add 401 retry with OS credential store re-extraction in
  AnthropicOAuthProvider, recovering from expired OAuth tokens (~8-12h)
  without manual intervention.
- Fix comment in app.rs: ~/.codex/auth.json → ~/.claude/.credentials.json.

Medium:
- Remove unsafe { std::env::set_var } from wizard; use thread-safe
  inject_single_var() overlay instead (safe on multi-threaded Tokio).
- Add post-init validation in AppBuilder: fail early with clear error when
  LLM_BACKEND is set but no credentials were resolved after secret injection.
- Add sk-ant-oat prefix validation in parse_oauth_access_token().
- Only route to AnthropicOAuthProvider when api_key is missing or equals
  OAUTH_PLACEHOLDER (API key takes priority over OAuth token).
- Teach fetch_anthropic_models() to use Bearer auth when only OAuth token
  is available (model listing no longer fails for OAuth-only users).

Low:
- Use optional_env() in wizard credential checks to read from injected
  overlay, not just raw env vars.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: cargo fmt

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: [email protected] <[email protected]>
2026-03-07 20:59:17 +00:00
ArtemandGitHub 12ba79ffc3 feat(llm): add Google Gemini, AWS Bedrock, io.net, Mistral, Yandex, and Cloudflare WS AI providers (#676)
* feat(llm): add Google Gemini and AWS Bedrock providers

* feat(llm): add io.net, Mistral, Yandex, and Cloudflare WS AI providers
2026-03-07 20:49:26 +00:00
github-actions[bot]GitHubgithub-actions[bot] <github-actions[bot]@users.noreply.github.com>
d3cf637d4a chore: update WASM artifact SHA256 checksums [skip ci] (#631)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-07 20:06:00 +00:00
b6cf2a6b73 fix: prevent Instant duration overflow on Windows (#657) (#664)
* fix: use checked_sub to prevent Instant duration overflow on Windows (#657)

On Windows, Instant starts from system boot time. Subtracting a duration
longer than uptime (e.g., 1 hour on a freshly booted system) panics with
"overflow when subtracting duration from instant", crashing the tokio
worker thread.

Replace `Instant::now() - Duration` with `Instant::now().checked_sub()`
in cost_guard.rs (production), server.rs and session.rs (tests).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use expect() instead of unwrap_or() in test code

Address PR review: unwrap_or(Instant::now()) silently breaks test
semantics when checked_sub returns None. Using expect() ensures tests
fail explicitly with a clear message about insufficient system uptime.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-07 20:00:40 +00:00
9851f2a6ae docs: add explanatory comments to coverage workflow (#610)
Add comprehensive documentation at the top of the coverage workflow file
to help developers understand:
- What the coverage workflow does
- How to view coverage reports (Codecov links)
- What coverage files are generated
- Configuration options and requirements

This improves developer experience by making the CI/CD pipeline more
transparent and easier to understand for contributors.

Co-authored-by: enihsago <[email protected]>
2026-03-07 19:56:07 +00:00
Eric ElizesandGitHub 8dc4ca5a98 fix: enable libsql remote + tls features for Turso cloud sync (#587)
The onboard wizard offers Turso cloud sync, but the libsql dependency
is compiled without the `remote` and `tls` features, causing a panic
at runtime when LIBSQL_URL is set:

  "The `tls` feature is disabled, you must provide your own http connector"

This adds the missing features to the libsql dependency.
2026-03-07 19:55:11 +00:00
9f71bd0d44 feat: unified thread model for web gateway (#607)
* feat: unified thread model for web gateway

Every piece of activity (user chat, routine run, heartbeat alert, external
channel message) now lives in its own thread, properly isolated, with
meaningful titles and visual distinction.

Key changes:
- Add `channel` field to ConversationSummary and ThreadInfo so the gateway
  can distinguish thread origins (gateway, telegram, routine, heartbeat).
- Add `list_conversations_all_channels` to Database trait (both postgres
  and libsql) so chat_threads_handler shows cross-channel threads.
- Routine runs get a persistent conversation per routine via
  `get_or_create_routine_conversation`; notifications carry thread_id.
- Heartbeat gets a persistent conversation via
  `get_or_create_heartbeat_conversation`; HeartbeatRunner accepts an
  optional Database store and binds notifications to the thread.
- Fix broadcast() in web gateway to propagate response.thread_id instead
  of hardcoding empty string.
- Fix isCurrentThread(null) returning true (the core notification leak
  bug) — now returns false so events without a thread_id don't leak into
  the active thread.
- Rewrite frontend thread sidebar: meaningful titles with channel-specific
  fallbacks, relative timestamps instead of turn counts, channel badges
  for non-gateway threads, unread notification dots, read-only indicator
  for external channel threads.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR review — TOCTOU races, stale comment, debounce, broadcast warning

- Fix TOCTOU race in get_or_create_routine_conversation (postgres):
  use INSERT ON CONFLICT on new uq_conv_routine unique index + SELECT-back.
- Fix TOCTOU race in get_or_create_heartbeat_conversation (postgres):
  use INSERT ON CONFLICT on new uq_conv_heartbeat unique index + SELECT-back.
- Fix TOCTOU race in get_or_create_routine_conversation (libsql):
  use BEGIN IMMEDIATE transaction to serialize concurrent writers.
- Fix TOCTOU race in get_or_create_heartbeat_conversation (libsql):
  use BEGIN IMMEDIATE transaction to serialize concurrent writers.
- Add V11 migration with partial unique indexes for postgres.
- Add matching unique indexes to libsql schema.
- Update stale comment on isCurrentThread (said "always shown" but logic
  now returns false for missing thread_id).
- Debounce loadThreads() on off-thread SSE events to prevent request storms.
- Log warning in broadcast() when thread_id is None (clients will drop it).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: sort in-memory thread fallback by updated_at descending

The in-memory thread list fallback (when no DB is available) used
HashMap::values() which has no guaranteed ordering. Sort by
updated_at descending to match the SQL query ordering.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: retry libsql connect() on transient "unable to open database file"

The cron ticker's background task occasionally fails with "unable to
open database file" when creating a new SQLite connection concurrently
with the main thread. Add retry with exponential backoff (50ms, 100ms,
200ms) to handle transient VFS/locking issues in libsql's local mode.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: use ON CONFLICT with index expressions instead of named constraints

PostgreSQL ON CONFLICT ON CONSTRAINT requires a named table constraint,
but V11 migration creates unique indexes. Switch to the expression form
(ON CONFLICT (columns) WHERE condition) which works with unique indexes.

Also fix dead code in threadTitle() where thread.title was already
checked on the previous line.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* style: fix rustfmt chain collapse in heartbeat.rs

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: skip broadcast when thread_id is None instead of sending empty

Clients drop SSE events with empty thread_id anyway, so avoid the
unnecessary network traffic by returning early.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add libsql routine/heartbeat conversation idempotency tests

Add tests proving get_or_create_routine_conversation returns the same
conversation ID across multiple invocations with the same routine_id.
Add debug logging to routine engine to track conversation resolution.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* feat: show "New chat" title for empty threads

- threadTitle() returns "New chat" when turn_count is 0
- Assistant thread label updates dynamically from API data
- Default HTML label changed from "Assistant" to "New chat"
- New threads naturally sort to top via last_activity DESC

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: thread sorting, routine isolation, and UI polish

- Fix libsql timestamp format mismatch causing broken thread sort order.
  SQLite defaults used `datetime('now')` (space-separated) while Rust code
  used RFC3339 (T-separated), breaking string-based ORDER BY. All INSERTs
  now use RFC3339, and queries use `datetime()` to normalize comparison.
- Route manual routine triggers through RoutineEngine.fire_manual() instead
  of injecting as regular chat messages, so routines always run in their
  dedicated conversation thread.
- Add RoutineEngineSlot to GatewayState for gateway<->engine communication.
- Derive routine thread titles from conversation metadata (routine_name)
  instead of showing truncated UUID hashes.
- Make chat_new_thread_handler persist to DB synchronously so loadThreads()
  sees newly created threads immediately.
- Fix enableChatInput() no-op and wrong element ID in disableChatInputReadOnly().
- Fix handlers/chat.rs stale gateway-only query (use list_conversations_all_channels).
- Sort in-memory threads by DateTime before converting to RFC3339 strings.
- Trigger debouncedLoadThreads() on thinking/status SSE events for non-current
  threads so routine/heartbeat threads appear in sidebar promptly.
- Remove "Threads" text from sidebar header.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: routine history display, orphaned tool_results, duplicate system messages

Three independent fixes with regression tests:

1. Routine conversations now display in the web UI. build_turns_from_db_messages()
   handles standalone assistant messages (no preceding user message) by creating
   turns with empty user_input. Frontend skips empty user bubbles.

2. Worker select_tools and execute_plan paths now push an
   assistant_with_tool_calls message before tool execution, preventing
   sanitize_tool_messages from rewriting tool_results as orphaned user messages.

3. Reasoning::plan() and respond_with_tools() merge system messages from
   context into a single system prompt instead of creating [system, system, ...]
   sequences that strict LLM providers (Qwen) reject.

Also: sidebar padding/spacing improvements, wider thread panel (240px).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR #607 review — RwLock held across await, missing ownership check, heartbeat config

- Clone Arc<RoutineEngine> out of RwLock before .await in trigger handler
- Add user_id ownership check to fire_manual() with NotAuthorized error
- Wire heartbeat notify_user/notify_channel from config to AgentHeartbeatConfig

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: gitignore trace_*.json files and remove stale traces

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* chore: remove trace JSON files from repo

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: proper HTTP status codes for routine errors, read-only input guard, respond thread_id

- Map RoutineError::NotFound → 404, NotAuthorized → 403, Disabled → 409
- Guard enableChatInput() against re-enabling on read-only threads
- Skip respond() when thread_id is None (matches broadcast() behavior)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <[email protected]>

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
2026-03-07 19:53:43 +00:00
c9dbcb2627 Update .env.example
Co-authored-by: Copilot <[email protected]>
2026-02-22 19:52:40 +04:00
Ilgın KanatandGitHub 98f44754e4 Merge branch 'main' into feat/lancedb-backend 2026-02-18 17:03:53 +04:00
e239ea850f Update src/workspace/lancedb_store.rs
Co-authored-by: Copilot <[email protected]>
2026-02-18 16:58:38 +04:00
ILGIN KANAT 6489c433f6 feat: add SQL-style escaping for LanceDB predicate values
- Introduced `escape_predicate_value` function to safely escape strings for use in LanceDB predicate expressions, preventing SQL injection.
- Updated delete operations in the LanceDB vector store to utilize the new escaping function for `chunk_id` and `document_id`.
- Enhanced filtering logic to apply escaping for `user_id` and `agent_id` in query conditions.

This change improves security by ensuring that user inputs are properly sanitized before being used in database queries.
2026-02-18 16:57:46 +04:00
ILGIN KANAT 327e009622 feat: add LanceDB support for workspace semantic search
- Introduced optional LanceDB vector store for semantic search, configurable via environment variables.
- Updated `.env.example` and `Cargo.toml` to include LanceDB settings.
- Enhanced `DatabaseConfig` to support vector backend selection and LanceDB path configuration.
- Implemented `VectorBackend` enum to manage vector store options.
- Added functionality to connect to LanceDB in the database connection logic.
- Updated relevant documentation to reflect new features and configuration options.

This change allows users to leverage LanceDB as an alternative to pgvector/libsql for improved search capabilities.
2026-02-18 11:06:09 +04:00
111 changed files with 9329 additions and 1079 deletions
+19
View File
@@ -2,9 +2,28 @@
DATABASE_URL=postgres://localhost/ironclaw
DATABASE_POOL_SIZE=10
# Vector store for workspace memory (optional)
# When set to "lancedb", uses LanceDB for semantic search instead of pgvector/libsql
# VECTOR_BACKEND=builtin # default: use database's built-in index (pgvector or libsql_vector_idx); "pgvector" is also accepted as an alias for "builtin"
# VECTOR_BACKEND=lancedb
# LANCEDB_PATH=~/.ironclaw/lancedb # path for LanceDB when VECTOR_BACKEND=lancedb
# LLM Provider
# LLM_BACKEND=nearai # default
# Possible values: nearai, ollama, openai_compatible, openai, anthropic, tinfoil
# LLM_REQUEST_TIMEOUT_SECS=120 # Increase for local LLMs (Ollama, vLLM, LM Studio)
# === Anthropic Direct ===
# Two auth modes:
# 1. API key: Set ANTHROPIC_API_KEY (from console.anthropic.com/settings/keys)
# 2. OAuth token: Set ANTHROPIC_OAUTH_TOKEN (from `claude login`)
# OAuth tokens use Authorization: Bearer instead of x-api-key header.
# ANTHROPIC_API_KEY=sk-ant-...
# ANTHROPIC_OAUTH_TOKEN=sk-ant-oat01-... # from `claude login` credentials
# ANTHROPIC_MODEL=claude-sonnet-4-20250514
# === OpenAI Direct ===
# OPENAI_API_KEY=sk-...
# === NEAR AI (Chat Completions API) ===
# Two auth modes:
+10
View File
@@ -36,6 +36,11 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install protoc (for lancedb)
if: matrix.name == 'all-features'
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: Swatinem/rust-cache@v2
with:
key: clippy-${{ matrix.name }}
@@ -62,6 +67,11 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- name: Install protoc (for lancedb)
if: matrix.name == 'all-features'
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: Swatinem/rust-cache@v2
with:
key: clippy-windows-${{ matrix.name }}
+28
View File
@@ -1,3 +1,31 @@
# Code Coverage Workflow
#
# This workflow runs test coverage analysis and uploads reports to Codecov.
# Coverage reports help identify untested code paths and maintain code quality.
#
# What it does:
# - Runs unit and integration tests with coverage instrumentation
# - Runs E2E tests with coverage instrumentation
# - Uploads coverage reports to Codecov (https://codecov.io/gh/nearai/ironclaw)
#
# Viewing coverage reports:
# - PRs automatically get coverage comments showing changes in coverage
# - Visit https://codecov.io/gh/nearai/ironclaw for detailed coverage reports
# - Coverage reports are generated for three configurations:
# 1. all-features: Full feature set
# 2. default: Default features
# 3. libsql-only: Minimal libSQL-only configuration
# - E2E coverage tracks end-to-end test coverage separately
#
# Coverage files:
# - Unit/integration: lcov.info (uploaded to Codecov with "unit" flag)
# - E2E: e2e-coverage.info (uploaded to Codecov with "e2e" flag)
#
# Requirements:
# - Uses cargo-llvm-cov for coverage instrumentation
# - Requires PostgreSQL for integration tests (pgvector/pgvector:pg16)
# - E2E tests require Python 3.12 and Playwright
name: Code Coverage
on:
push:
+15 -1
View File
@@ -14,7 +14,7 @@ jobs:
matrix:
include:
- name: all-features
flags: "--features postgres,libsql,html-to-markdown"
flags: "--features postgres,libsql,lancedb,html-to-markdown"
- name: default
flags: ""
- name: libsql-only
@@ -26,6 +26,11 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-wasip2
- name: Install protoc (for lancedb)
if: matrix.name == 'all-features'
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.name }}
@@ -66,6 +71,11 @@ jobs:
uses: actions/checkout@v6
- name: Install Rust
uses: dtolnay/rust-toolchain@stable
- name: Install protoc (for lancedb)
if: matrix.name == 'all-features'
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: Swatinem/rust-cache@v2
with:
key: windows-${{ matrix.name }}
@@ -82,6 +92,10 @@ jobs:
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-wasip2
- name: Install protoc (for lancedb)
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: Swatinem/rust-cache@v2
with:
key: wasm-extensions
+3 -1
View File
@@ -4,8 +4,9 @@
.env.*
!.env.example
# Claude Code worktrees
# Claude Code worktrees and lock files
.claude/worktrees/
.claude/scheduled_tasks.lock
# Sidecar tool data
.sidecar/
@@ -22,3 +23,4 @@ bench-results/
# WASM build artifacts (loaded from disk, not bundled)
*.wasm
trace_*.json
+15
View File
@@ -387,12 +387,27 @@ Dead code behind the wrong `#[cfg]` gate will only show up when building with a
**Zero clippy warnings policy:** Fix ALL clippy warnings before committing, including pre-existing ones in files you didn't change. Never leave warnings behind — treat `cargo clippy` output as a zero-tolerance gate.
**Transaction safety:** Multi-step database operations (INSERT+INSERT, UPDATE+DELETE, read-then-write) MUST be wrapped in a transaction. Never assume sequential calls are atomic. Before committing DB code, ask: "If this crashes between step N and N+1, is the database consistent?" If not, wrap in a transaction. This applies to both postgres and libsql backends.
**UTF-8 string safety:** Never use byte-index slicing (`&s[..n]`) on user-supplied or external strings — it panics on multi-byte characters. Use `is_char_boundary()` to walk backwards from the desired length, or iterate with `char_indices()`. Grep for `[..` in changed files to catch violations.
**Case-insensitive comparisons:** When comparing user-supplied strings (file paths, media types, extension names), always normalize to lowercase first with `.to_ascii_lowercase()`. On case-insensitive filesystems (macOS, Windows), path comparisons must be case-insensitive. File extension checks (`.png`, `.jpg`) and media type checks (`image/jpeg`) are common offenders.
**Decorator/wrapper trait delegation:** When adding a new method to `LlmProvider` (or any trait with decorator wrappers), you MUST update ALL wrapper types to delegate to their inner provider. Grep for `impl LlmProvider for` to find all implementations. Add a test that exercises the method through the full provider chain (`build_provider_chain()`), not just the base impl.
**Sensitive data in logs & events:** Tool parameters and outputs MUST be redacted before logging or broadcasting via SSE/WebSocket. Use `redact_params()` before any `tracing::info!`, `JobEvent`, or SSE emission that includes tool call data. Never log raw parameters from tool calls.
**Test temporary files:** Use the `tempfile` crate for test directories/files. Never hardcode `/tmp/...` paths — they collide in parallel test runs and break on non-Unix platforms.
**Trust boundaries in multi-process architecture:** Data from worker containers is untrusted. The orchestrator MUST validate: tool domain (never execute `Container`-domain tools on the host), nesting depth (server-side tracking, not client-supplied), and parameter sensitivity (redact before logging/broadcasting).
**Mechanical verification before committing:** Run these checks on changed files before committing:
- `cargo clippy --all --benches --tests --examples --all-features` -- zero warnings
- `grep -rnE '\.unwrap\(|\.expect\(' <files>` -- no panics in production
- `grep -rn 'super::' <files>` -- use `crate::` imports
- If you fixed a pattern bug, `grep` for other instances of that pattern across `src/`
- Fix commits must include regression tests (enforced by `commit-msg` hook; bypass with `[skip-regression-check]`)
- Run `scripts/pre-commit-safety.sh` to catch UTF-8, case-sensitivity, hardcoded /tmp, and logging issues
## Configuration
Generated
+3044 -36
View File
File diff suppressed because it is too large Load Diff
+9 -1
View File
@@ -56,7 +56,7 @@ rustls = { version = "0.23", optional = true, default-features = false }
rustls-native-certs = { version = "0.8", optional = true }
# Database - libSQL/Turso (optional embedded database)
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication"] }
libsql = { version = "0.6", optional = true, default-features = false, features = ["core", "replication", "remote", "tls"] }
# Error handling
thiserror = "2"
@@ -73,6 +73,8 @@ toml = "0.8"
# Core types
uuid = { version = "1", features = ["v4", "v5", "serde"] }
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = "0.10"
iana-time-zone = "0.1"
rust_decimal = { version = "1", features = ["serde", "serde-with-str", "maths"] }
rust_decimal_macros = "1"
@@ -123,6 +125,11 @@ open = "5"
# The postgres feature provides ToSql/FromSql for postgres-types (shared by tokio-postgres)
pgvector = { version = "0.4", features = ["postgres"], optional = true }
# LanceDB vector store (optional alternative to pgvector/libsql for workspace search)
lancedb = { version = "0.26", optional = true }
arrow-array = { version = "57", optional = true }
arrow-schema = { version = "57", optional = true }
# WASM sandbox for untrusted tool execution
wasmtime = { version = "28", features = ["component-model"] }
wasmtime-wasi = "28" # WASI support for component model
@@ -187,6 +194,7 @@ insta = "1.46.3"
[features]
default = ["postgres", "libsql", "html-to-markdown"]
lancedb = ["dep:lancedb", "dep:arrow-array", "dep:arrow-schema"]
postgres = [
"dep:deadpool-postgres",
"dep:tokio-postgres",
+9 -5
View File
@@ -39,7 +39,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Network modes (loopback/LAN/remote) | ✅ | 🚧 | HTTP only |
| OpenAI-compatible HTTP API | ✅ | ✅ | /v1/chat/completions, per-request `model` override |
| Canvas hosting | ✅ | ❌ | Agent-driven UI |
| Gateway lock (PID-based) | ✅ | | |
| Gateway lock (PID-based) | ✅ | | `fs4` flock-based, acquired in `main.rs` before agent startup |
| launchd/systemd integration | ✅ | ❌ | |
| Bonjour/mDNS discovery | ✅ | ❌ | |
| Tailscale integration | ✅ | ❌ | |
@@ -215,9 +215,13 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| NEAR AI | ✅ | ✅ | - | Primary provider |
| Anthropic (Claude) | ✅ | 🚧 | - | Via NEAR AI proxy; Opus 4.5, Sonnet 4, Sonnet 4.6 |
| OpenAI | ✅ | 🚧 | - | Via NEAR AI proxy |
| AWS Bedrock | ✅ | | P3 | |
| Google Gemini | ✅ | | P3 | |
| NVIDIA API | ✅ | | P3 | New provider |
| AWS Bedrock | ✅ | | P3 | Via `openai_compatible` adapter (e.g. LiteLLM) |
| Google Gemini | ✅ | | P3 | Via `gemini` adapter |
| io.net | ✅ | | P3 | Via `ionet` adapter |
| Mistral | ✅ | ✅ | P3 | Via `mistral` adapter |
| Yandex AI Studio | ✅ | ✅ | P3 | Via `yandex` adapter |
| Cloudflare Workers AI | ✅ | ✅ | P3 | Via `cloudflare` adapter |
| NVIDIA API | ✅ | ✅ | P3 | Via `nvidia` adapter and `providers.json` |
| OpenRouter | ✅ | ✅ | - | Via OpenAI-compatible provider (RigAdapter) |
| Tinfoil | ❌ | ✅ | - | Private inference provider (IronClaw-only) |
| OpenAI-compatible | ❌ | ✅ | - | Generic OpenAI-compatible endpoint (RigAdapter) |
@@ -336,7 +340,7 @@ This document tracks feature parity between IronClaw (Rust implementation) and O
| Gemini embeddings | ✅ | ❌ | |
| Local embeddings | ✅ | ❌ | |
| SQLite-vec backend | ✅ | ❌ | IronClaw uses PostgreSQL |
| LanceDB backend | ✅ | | Configurable auto-capture max length |
| LanceDB backend | ✅ | | VectorStore trait + LanceDbVectorStore (configured via VECTOR_BACKEND=lancedb) |
| QMD backend | ✅ | ❌ | |
| Atomic reindexing | ✅ | ✅ | |
| Embeddings batching | ✅ | ✅ | `embed_batch` on EmbeddingProvider trait |
+1 -1
View File
@@ -3,7 +3,7 @@ services:
postgres:
image: pgvector/pgvector:pg16
ports:
- "5432:5432"
- "127.0.0.1:5432:5432"
environment:
POSTGRES_DB: ironclaw
POSTGRES_USER: ironclaw
+6
View File
@@ -11,6 +11,12 @@ configurations.
| NEAR AI | `nearai` | OAuth (browser) | Default; multi-model |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` | Claude models |
| OpenAI | `openai` | `OPENAI_API_KEY` | GPT models |
| Google Gemini | `gemini` | `GEMINI_API_KEY` | Gemini models |
| AWS Bedrock | `bedrock` | `BEDROCK_ACCESS_KEY` | Requires OpenAI proxy (e.g. LiteLLM) |
| io.net | `ionet` | `IONET_API_KEY` | Intelligence API |
| Mistral | `mistral` | `MISTRAL_API_KEY` | Mistral models |
| Yandex AI Studio | `yandex` | `YANDEX_API_KEY` | YandexGPT models |
| Cloudflare Workers AI | `cloudflare` | `CLOUDFLARE_API_KEY` | Access to Workers AI |
| Ollama | `ollama` | No | Local inference |
| OpenRouter | `openai_compatible` | `LLM_API_KEY` | 300+ models |
| Together AI | `openai_compatible` | `LLM_API_KEY` | Fast inference |
@@ -0,0 +1,13 @@
-- Partial unique indexes to prevent duplicate singleton conversations.
-- These guard against TOCTOU races in get_or_create_routine_conversation
-- and get_or_create_heartbeat_conversation.
-- One routine conversation per user per routine_id.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_routine
ON conversations (user_id, (metadata->>'routine_id'))
WHERE metadata->>'routine_id' IS NOT NULL;
-- One heartbeat conversation per user.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_heartbeat
ON conversations (user_id)
WHERE metadata->>'thread_type' = 'heartbeat';
+161 -11
View File
@@ -1,7 +1,9 @@
[
{
"id": "openai",
"aliases": ["open_ai"],
"aliases": [
"open_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "OPENAI_API_KEY",
"api_key_required": true,
@@ -19,7 +21,9 @@
},
{
"id": "anthropic",
"aliases": ["claude"],
"aliases": [
"claude"
],
"protocol": "anthropic",
"api_key_env": "ANTHROPIC_API_KEY",
"api_key_required": true,
@@ -52,7 +56,10 @@
},
{
"id": "openai_compatible",
"aliases": ["openai-compatible", "compatible"],
"aliases": [
"openai-compatible",
"compatible"
],
"protocol": "open_ai_completions",
"base_url_env": "LLM_BASE_URL",
"base_url_required": true,
@@ -89,7 +96,9 @@
},
{
"id": "openrouter",
"aliases": ["open_router"],
"aliases": [
"open_router"
],
"protocol": "open_ai_completions",
"default_base_url": "https://openrouter.ai/api/v1",
"api_key_env": "OPENROUTER_API_KEY",
@@ -126,7 +135,10 @@
},
{
"id": "nvidia",
"aliases": ["nvidia_nim", "nim"],
"aliases": [
"nvidia_nim",
"nim"
],
"protocol": "open_ai_completions",
"default_base_url": "https://integrate.api.nvidia.com/v1",
"api_key_env": "NVIDIA_API_KEY",
@@ -144,7 +156,10 @@
},
{
"id": "venice",
"aliases": ["venice_ai", "veniceai"],
"aliases": [
"venice_ai",
"veniceai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.venice.ai/api/v1",
"api_key_env": "VENICE_API_KEY",
@@ -162,7 +177,10 @@
},
{
"id": "together",
"aliases": ["together_ai", "togetherai"],
"aliases": [
"together_ai",
"togetherai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.together.xyz/v1",
"api_key_env": "TOGETHER_API_KEY",
@@ -180,7 +198,9 @@
},
{
"id": "fireworks",
"aliases": ["fireworks_ai"],
"aliases": [
"fireworks_ai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.fireworks.ai/inference/v1",
"api_key_env": "FIREWORKS_API_KEY",
@@ -198,7 +218,9 @@
},
{
"id": "deepseek",
"aliases": ["deep_seek"],
"aliases": [
"deep_seek"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.deepseek.com/v1",
"api_key_env": "DEEPSEEK_API_KEY",
@@ -234,7 +256,9 @@
},
{
"id": "sambanova",
"aliases": ["samba_nova"],
"aliases": [
"samba_nova"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.sambanova.ai/v1",
"api_key_env": "SAMBANOVA_API_KEY",
@@ -249,5 +273,131 @@
"display_name": "SambaNova",
"can_list_models": false
}
},
{
"id": "gemini",
"aliases": [
"google_gemini",
"google"
],
"protocol": "open_ai_completions",
"default_base_url": "https://generativelanguage.googleapis.com/v1beta/openai",
"api_key_env": "GEMINI_API_KEY",
"api_key_required": true,
"model_env": "GEMINI_MODEL",
"default_model": "gemini-2.5-flash",
"description": "Google Gemini (via OpenAI-compatible endpoint)",
"setup": {
"kind": "api_key",
"secret_name": "llm_gemini_api_key",
"key_url": "https://aistudio.google.com/app/apikey",
"display_name": "Google Gemini",
"can_list_models": true
}
},
{
"id": "bedrock",
"aliases": [
"aws_bedrock",
"aws"
],
"protocol": "open_ai_completions",
"api_key_env": "BEDROCK_ACCESS_KEY",
"api_key_required": false,
"base_url_env": "BEDROCK_BASE_URL",
"model_env": "BEDROCK_MODEL",
"default_model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
"description": "AWS Bedrock (requires LiteLLM or OpenAI-compatible proxy)",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_bedrock_api_key",
"display_name": "AWS Bedrock",
"can_list_models": false
}
},
{
"id": "ionet",
"aliases": [
"io_net",
"io.net"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.intelligence.io.solutions/api/v1",
"api_key_env": "IONET_API_KEY",
"api_key_required": true,
"model_env": "IONET_MODEL",
"default_model": "deepseek-coder-v2-instruct",
"description": "io.net Intelligence API",
"setup": {
"kind": "api_key",
"secret_name": "llm_ionet_api_key",
"key_url": "https://cloud.io.net/intelligence",
"display_name": "io.net",
"can_list_models": true
}
},
{
"id": "mistral",
"aliases": [
"mistral_ai",
"mistralai"
],
"protocol": "open_ai_completions",
"default_base_url": "https://api.mistral.ai/v1",
"api_key_env": "MISTRAL_API_KEY",
"api_key_required": true,
"model_env": "MISTRAL_MODEL",
"default_model": "mistral-large-latest",
"description": "Mistral AI API",
"setup": {
"kind": "api_key",
"secret_name": "llm_mistral_api_key",
"key_url": "https://console.mistral.ai/api-keys",
"display_name": "Mistral",
"can_list_models": true
}
},
{
"id": "yandex",
"aliases": [
"yandex_ai_studio",
"yandexgpt",
"yandex_gpt"
],
"protocol": "open_ai_completions",
"default_base_url": "https://ai.api.cloud.yandex.net/v1",
"api_key_env": "YANDEX_API_KEY",
"api_key_required": true,
"model_env": "YANDEX_MODEL",
"extra_headers_env": "YANDEX_EXTRA_HEADERS",
"default_model": "yandexgpt-lite",
"description": "Yandex AI Studio (YandexGPT)",
"setup": {
"kind": "api_key",
"secret_name": "llm_yandex_api_key",
"key_url": "https://aistudio.yandex.ru/platform/folders/",
"display_name": "Yandex AI Studio",
"can_list_models": true
}
},
{
"id": "cloudflare",
"aliases": [
"cloudflare_ai",
"cf_ai"
],
"protocol": "open_ai_completions",
"api_key_env": "CLOUDFLARE_API_KEY",
"api_key_required": true,
"base_url_env": "CLOUDFLARE_BASE_URL",
"model_env": "CLOUDFLARE_MODEL",
"default_model": "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
"description": "Cloudflare Workers AI",
"setup": {
"kind": "open_ai_compatible",
"secret_name": "llm_cloudflare_api_key",
"display_name": "Cloudflare Workers AI",
"can_list_models": false
}
}
]
]
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/discord-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "030707431717bca3411a48f311c6ab5f92a45c747de26cafe4f6e3e23a8b3b2d"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "6ed36077b67ac70a041f06f760f93ba79b33269885413c3c3f2c8c87ee60807e"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "98c86895a9c4b0a1e19fe8a47f1ccbfe7e972e112b05e584bc897130dc32283a"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/whatsapp-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "bd35cad18d87292ea8d2f52db9b514ed9f814a414de910f59073d475c26c4c14"
}
},
"auth_summary": {
+1 -1
View File
@@ -20,7 +20,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/github-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "6fcd32719a4ff15641a4b50fff8984686550f0c491dce60518f4126857d0c544"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/gmail-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "023da7000b17568bf0e64b2e5013c8a042b2f323c85f1632339231c73d500e39"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-calendar-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "fc42277b65881d6e9bcc5403dc54c7f5b3ddeaaaf04617fce2c5da05d76325f0"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-docs-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "385c04abd1e6b8011ccc330e1f4bd7ce58577e488959b51594aa04eb26cbe7cc"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-drive-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "1b107d575a5d52cc8c76d9a681802190f4373fb485f7f54f445533f097fa37c0"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-sheets-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "c4f6b1e8c5126ac2c8a4b98e4283a3afa32223d2488fc3c3a609758c0c9beb90"
}
},
"auth_summary": {
+1 -1
View File
@@ -18,7 +18,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/google-slides-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "7110b8565340c888e51f99e9c013bf4de8f8a7f7b33bace00eb8fc47831ff20b"
}
},
"auth_summary": {
+1 -1
View File
@@ -18,7 +18,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/slack-tool-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "6ed36077b67ac70a041f06f760f93ba79b33269885413c3c3f2c8c87ee60807e"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/telegram-mtproto-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "98c86895a9c4b0a1e19fe8a47f1ccbfe7e972e112b05e584bc897130dc32283a"
}
},
"auth_summary": {
+1 -1
View File
@@ -19,7 +19,7 @@
"artifacts": {
"wasm32-wasip2": {
"url": "https://github.com/nearai/ironclaw/releases/latest/download/web-search-wasm32-wasip2.tar.gz",
"sha256": null
"sha256": "66cb2b9b00652385e9f30f17c74902b9222c17c53e9d3bd1ef42f5cab705bcf6"
}
},
"auth_summary": {
+4 -2
View File
@@ -51,9 +51,11 @@ echo "[6/6] Installing git hooks..."
HOOKS_DIR=$(git rev-parse --git-path hooks 2>/dev/null) || true
if [ -n "$HOOKS_DIR" ]; then
mkdir -p "$HOOKS_DIR"
SCRIPT_ABS="$(cd "$(dirname "$0")" && pwd)/commit-msg-regression.sh"
ln -sf "$SCRIPT_ABS" "$HOOKS_DIR/commit-msg"
SCRIPTS_ABS="$(cd "$(dirname "$0")" && pwd)"
ln -sf "$SCRIPTS_ABS/commit-msg-regression.sh" "$HOOKS_DIR/commit-msg"
echo " commit-msg hook installed (regression test enforcement)"
ln -sf "$SCRIPTS_ABS/pre-commit-safety.sh" "$HOOKS_DIR/pre-commit"
echo " pre-commit hook installed (UTF-8, case-sensitivity, /tmp, redaction checks)"
else
echo " Skipped: not a git repository"
fi
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env bash
# Pre-commit safety checks for common issues caught by AI code reviewers.
#
# Can be run standalone: bash scripts/pre-commit-safety.sh
# Or installed as a git pre-commit hook via dev-setup.sh.
#
# Checks staged .rs files for:
# 1. Unsafe UTF-8 byte slicing (panics on multi-byte chars)
# 2. Case-sensitive file extension comparisons
# 3. Hardcoded /tmp paths in tests (flaky in parallel runs)
# 4. Tool parameters logged without redaction (secret leaks)
# 5. Multi-step DB operations without transaction wrapping
#
# Suppress individual lines with an inline "// safety: <reason>" comment.
set -euo pipefail
# Determine a suitable base ref for standalone diffs.
resolve_base_ref() {
local candidates=(
"@{upstream}"
"origin/HEAD"
"origin/main"
"origin/master"
"main"
"master"
)
for ref in "${candidates[@]}"; do
if git rev-parse --verify --quiet "$ref" >/dev/null 2>&1; then
echo "$ref"
return 0
fi
done
echo "pre-commit-safety: could not determine a base Git ref for diff (tried: ${candidates[*]})." >&2
echo "pre-commit-safety: ensure your repository has an upstream or a local main/master branch." >&2
exit 1
}
# Support both pre-commit hook (staged files) and standalone (all changed vs base)
if git diff --cached --quiet 2>/dev/null; then
# No staged changes -- compare working tree against a resolved base ref
BASE_REF="$(resolve_base_ref)"
DIFF_OUTPUT=$(git diff "$BASE_REF" -- '*.rs' 2>/dev/null || true)
else
DIFF_OUTPUT=$(git diff --cached -U0 -- '*.rs' 2>/dev/null || true)
fi
# Early exit if there are no relevant .rs changes
if [ -z "$DIFF_OUTPUT" ]; then
exit 0
fi
WARNINGS=0
warn() {
if [ "$WARNINGS" -eq 0 ]; then
echo ""
echo "=== Pre-commit Safety Checks ==="
echo ""
fi
WARNINGS=$((WARNINGS + 1))
echo " [$1] $2"
}
# 1. Unsafe UTF-8 byte slicing: &s[..N] or &s[..some_var] on strings
# Safe patterns: is_char_boundary, char_indices, // safety:
if echo "$DIFF_OUTPUT" | grep -nE '^\+' | grep -E '\[\.\..*\]' | grep -vE 'is_char_boundary|char_indices|// safety:|as_bytes|Vec<|&\[u8\]|\[u8\]|bytes\(\)|&bytes' | head -3 | grep -q .; then
warn "UTF8" "Possible unsafe byte-index string slicing. Use is_char_boundary() or char_indices()."
echo "$DIFF_OUTPUT" | grep -nE '^\+' | grep -E '\[\.\..*\]' | grep -vE 'is_char_boundary|char_indices|// safety:|as_bytes|Vec<|&\[u8\]|\[u8\]|bytes\(\)|&bytes' | head -3 | sed 's/^/ /'
fi
# 2. Case-sensitive file extension checks
# Match: .ends_with(".png") without prior to_lowercase
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*ends_with\("\.([pP][nN][gG]|[jJ][pP][eE]?[gG]|[gG][iI][fF]|[wW][eE][bB][pP]|[mM][dD])"\)' | grep -vE 'to_lowercase|to_ascii_lowercase|// safety:' | head -3 | grep -q .; then
warn "CASE" "Case-sensitive file extension comparison. Normalize to lowercase first."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*ends_with\("\.([pP][nN][gG]|[jJ][pP][eE]?[gG]|[gG][iI][fF]|[wW][eE][bB][pP]|[mM][dD])"\)' | grep -vE 'to_lowercase|to_ascii_lowercase|// safety:' | head -3 | sed 's/^/ /'
fi
# 3. Hardcoded /tmp paths in test files
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*"/tmp/' | grep -vE 'tempfile|tempdir|// safety:' | head -3 | grep -q .; then
warn "TMPDIR" "Hardcoded /tmp path. Use tempfile::tempdir() for parallel-safe tests."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*"/tmp/' | grep -vE 'tempfile|tempdir|// safety:' | head -3 | sed 's/^/ /'
fi
# 4. Logging tool parameters without redaction
if echo "$DIFF_OUTPUT" | grep -nE '^\+.*tracing::(info|debug|warn|error).*param' | grep -vE 'redact|// safety:' | head -3 | grep -q .; then
warn "REDACT" "Logging tool parameters without redaction. Use redact_params() first."
echo "$DIFF_OUTPUT" | grep -nE '^\+.*tracing::(info|debug|warn|error).*param' | grep -vE 'redact|// safety:' | head -3 | sed 's/^/ /'
fi
# 5. Multi-step DB operations without transaction
# Uses -W (function context) to reduce false positives from existing transactions.
# Suppressible with "// safety:" in the hunk.
DIFF_W_OUTPUT=$(git diff --cached -W -- '*.rs' 2>/dev/null || git diff "$(resolve_base_ref)" -W -- '*.rs' 2>/dev/null || true)
if [ -n "$DIFF_W_OUTPUT" ]; then
HUNK_COUNT=$(echo "$DIFF_W_OUTPUT" | awk '
/^@@/ {
if (count >= 2 && !has_tx && !has_safety) found++
count=0; has_tx=0; has_safety=0
}
/^\+.*\.(execute|query)\(/ { count++ }
/^\+.*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/ .*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/\/\/ safety:/ { has_safety=1 }
END {
if (count >= 2 && !has_tx && !has_safety) found++
print found+0
}
')
if [ "$HUNK_COUNT" -gt 0 ]; then
warn "TX" "Multiple DB operations in same function without transaction. Wrap in a transaction for atomicity."
echo "$DIFF_W_OUTPUT" | awk '
/^@@/ {
if (count >= 2 && !has_tx && !has_safety) { print buf }
buf=""; count=0; has_tx=0; has_safety=0
}
/^\+.*\.(execute|query)\(/ { count++ }
/^\+.*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/ .*(transaction|\.tx\.|\.begin\()/ { has_tx=1 }
/\/\/ safety:/ { has_safety=1 }
{ buf = buf "\n" $0 }
END {
if (count >= 2 && !has_tx && !has_safety) { print buf }
}
' | grep -E '^\+.*\.(execute|query)\(' | head -4 | sed 's/^/ /'
fi
fi
if [ "$WARNINGS" -gt 0 ]; then
echo ""
echo "Found $WARNINGS potential issue(s). Fix them or add '// safety: <reason>' to suppress."
echo ""
exit 1
fi
+54
View File
@@ -0,0 +1,54 @@
---
name: review-checklist
version: 0.1.0
description: Pre-merge review checklist based on recurring AI reviewer feedback patterns
activation:
patterns:
- "review.*checklist"
- "ready to merge"
- "pre-merge check"
- "check.*before.*merge"
keywords:
- review
- checklist
- merge
- pre-merge
max_context_tokens: 1500
---
# Pre-Merge Review Checklist
Before merging, verify these items. They represent the most common issues caught by automated code reviewers (Copilot, Gemini) on IronClaw PRs.
## Database Operations
- [ ] Multi-step DB operations are wrapped in transactions (INSERT+INSERT, UPDATE+DELETE, read-modify-write)
- [ ] Both postgres AND libsql backends updated for any new Database trait methods
- [ ] Migrations are atomic (SQL execution + version recording in same transaction)
## Security & Data Safety
- [ ] Tool parameters are redacted via `redact_params()` before logging or SSE/WebSocket broadcast
- [ ] URL validation resolves DNS before checking for private/loopback IPs (anti-SSRF via DNS rebinding)
- [ ] Destructive tools have `requires_approval()` returning `Always` or `UnlessAutoApproved`
- [ ] Data from worker containers is treated as untrusted (tool domain checks, server-side nesting depth)
- [ ] No secrets or credentials in error messages, logs, or SSE events
## String Safety
- [ ] No byte-index slicing (`&s[..n]`) on external/user strings -- use `is_char_boundary()` or `char_indices()`
- [ ] File extension and media type comparisons are case-insensitive (`.to_ascii_lowercase()` before matching)
- [ ] Path comparisons are case-insensitive where needed (macOS/Windows filesystems)
## Trait Wrappers & Decorator Chain
- [ ] New `LlmProvider` trait methods are delegated in ALL wrapper types (grep `impl LlmProvider for`)
- [ ] New trait methods are tested through the full decorator/provider chain, not just the base impl
- [ ] Default trait method implementations are intentional -- wrappers that silently return defaults are bugs
## Tests
- [ ] Temporary files/dirs use `tempfile` crate, no hardcoded `/tmp/` paths
- [ ] Tests don't mutate global statics without synchronization (use per-test state or `serial_test`)
- [ ] Tests don't make real network requests (use mocks, stubs, or RFC 5737 TEST-NET IPs like 192.0.2.1)
- [ ] Test names and comments match actual test behavior and assertions
## Comments & Documentation
- [ ] Code comments match actual behavior (especially route paths, tool names, function semantics)
- [ ] Spec/README files updated if module behavior changed
- [ ] Error messages are clear and non-redundant (don't nest tool name inside tool error that already contains it)
+30 -2
View File
@@ -96,6 +96,9 @@ pub struct Agent {
pub(super) heartbeat_config: Option<HeartbeatConfig>,
pub(super) hygiene_config: Option<crate::config::HygieneConfig>,
pub(super) routine_config: Option<RoutineConfig>,
/// Optional slot to expose the routine engine to the gateway for manual triggering.
pub(super) routine_engine_slot:
Option<Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>>,
}
impl Agent {
@@ -148,9 +151,18 @@ impl Agent {
heartbeat_config,
hygiene_config,
routine_config,
routine_engine_slot: None,
}
}
/// Set the routine engine slot for exposing the engine to the gateway.
pub fn set_routine_engine_slot(
&mut self,
slot: Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>,
) {
self.routine_engine_slot = Some(slot);
}
// Convenience accessors
/// Get the scheduler (for external wiring, e.g. CreateJobTool).
@@ -342,8 +354,19 @@ impl Agent {
let heartbeat_handle = if let Some(ref hb_config) = self.heartbeat_config {
if hb_config.enabled {
if let Some(workspace) = self.workspace() {
let config = AgentHeartbeatConfig::default()
let mut config = AgentHeartbeatConfig::default()
.with_interval(std::time::Duration::from_secs(hb_config.interval_secs));
config.quiet_hours_start = hb_config.quiet_hours_start;
config.quiet_hours_end = hb_config.quiet_hours_end;
config.timezone = hb_config
.timezone
.clone()
.or_else(|| Some(self.config.default_timezone.clone()));
if let (Some(user), Some(channel)) =
(&hb_config.notify_user, &hb_config.notify_channel)
{
config = config.with_notify(user, channel);
}
// Set up notification channel
let (notify_tx, mut notify_rx) =
@@ -394,8 +417,8 @@ impl Agent {
hygiene,
workspace.clone(),
self.cheap_llm().clone(),
self.safety().clone(),
Some(notify_tx),
self.store().map(Arc::clone),
))
} else {
tracing::warn!("Heartbeat enabled but no workspace available");
@@ -486,6 +509,11 @@ impl Agent {
// SAFETY: self is consumed by run(), we can smuggle the engine in
// via a local to use in the message loop below.
// Expose engine to gateway for manual triggering
if let Some(ref slot) = self.routine_engine_slot {
*slot.write().await = Some(Arc::clone(&engine));
}
tracing::info!(
"Routines enabled: cron ticker every {}s, max {} concurrent",
rt_config.cron_check_interval_secs,
+48 -7
View File
@@ -345,7 +345,6 @@ impl Agent {
crate::workspace::hygiene::HygieneConfig::default(),
workspace.clone(),
self.llm().clone(),
self.safety().clone(),
);
match runner.check_heartbeat().await {
@@ -406,7 +405,7 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
let reasoning = Reasoning::new(self.llm().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Thread Summary:\n\n{}",
@@ -454,7 +453,7 @@ impl Agent {
.with_max_tokens(512)
.with_temperature(0.5);
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
let reasoning = Reasoning::new(self.llm().clone());
match reasoning.complete(request).await {
Ok((text, _usage)) => Ok(SubmissionResult::response(format!(
"Suggested Next Steps:\n\n{}",
@@ -663,10 +662,14 @@ impl Agent {
}
match self.llm().set_model(requested) {
Ok(()) => Ok(SubmissionResult::response(format!(
"Switched model to: {}",
requested
))),
Ok(()) => {
// Persist the model choice so it survives restarts.
self.persist_selected_model(requested).await;
Ok(SubmissionResult::response(format!(
"Switched model to: {}",
requested
)))
}
Err(e) => Ok(SubmissionResult::error(format!(
"Failed to switch model: {}",
e
@@ -822,4 +825,42 @@ impl Agent {
_ => Ok(None),
}
}
/// Persist the selected model to the settings store (DB and/or TOML config).
///
/// Best-effort: logs warnings on failure but does not propagate errors,
/// since the in-memory model switch already succeeded.
async fn persist_selected_model(&self, model: &str) {
// 1. Persist to DB if available.
if let Some(store) = self.store() {
let value = serde_json::Value::String(model.to_string());
if let Err(e) = store.set_setting("default", "selected_model", &value).await {
tracing::warn!("Failed to persist model to DB: {}", e);
}
}
// 2. Update TOML config file if it exists (sync I/O in spawn_blocking).
let model_owned = model.to_string();
if let Err(e) = tokio::task::spawn_blocking(move || {
let toml_path = crate::settings::Settings::default_toml_path();
match crate::settings::Settings::load_toml(&toml_path) {
Ok(Some(mut settings)) => {
settings.selected_model = Some(model_owned);
if let Err(e) = settings.save_toml(&toml_path) {
tracing::warn!("Failed to persist model to config.toml: {}", e);
}
}
Ok(None) => {
// No config file on disk; nothing to update.
}
Err(e) => {
tracing::warn!("Failed to load config.toml for model persistence: {}", e);
}
}
})
.await
{
tracing::warn!("Model TOML persistence task failed: {}", e);
}
}
}
+4 -12
View File
@@ -13,7 +13,6 @@ use crate::agent::context_monitor::{CompactionStrategy, ContextBreakdown};
use crate::agent::session::Thread;
use crate::error::Error;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
/// Result of a compaction operation.
@@ -34,13 +33,12 @@ pub struct CompactionResult {
/// Compacts conversation context to stay within limits.
pub struct ContextCompactor {
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
}
impl ContextCompactor {
/// Create a new context compactor.
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
Self { llm, safety }
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
}
/// Compact a thread's context using the given strategy.
@@ -233,7 +231,7 @@ Be brief but capture all important details. Use bullet points."#,
.with_max_tokens(1024)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let reasoning = Reasoning::new(self.llm.clone());
let (text, _) = reasoning.complete(request).await?;
Ok(text)
}
@@ -346,17 +344,11 @@ mod tests {
// === QA Plan - Compaction strategy tests ===
use crate::agent::context_monitor::CompactionStrategy;
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
/// Helper: build a `ContextCompactor` with the given `StubLlm`.
fn make_compactor(llm: Arc<StubLlm>) -> ContextCompactor {
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
ContextCompactor::new(llm, safety)
ContextCompactor::new(llm)
}
/// Helper: build a thread with `n` completed turns.
+42 -7
View File
@@ -131,10 +131,12 @@ impl CostGuard {
// Check hourly rate
if let Some(limit) = self.config.max_actions_per_hour {
let mut window = self.action_window.lock().await;
let cutoff = Instant::now() - std::time::Duration::from_secs(3600);
// Drain expired entries
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
// checked_sub avoids panic when system uptime < 1 hour (Windows)
if let Some(cutoff) = Instant::now().checked_sub(std::time::Duration::from_secs(3600)) {
// Drain expired entries
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
}
let count = window.len() as u64;
if count >= limit {
@@ -260,9 +262,11 @@ impl CostGuard {
/// Number of actions in the current hourly window.
pub async fn actions_this_hour(&self) -> u64 {
let mut window = self.action_window.lock().await;
let cutoff = Instant::now() - std::time::Duration::from_secs(3600);
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
// checked_sub avoids panic when system uptime < 1 hour (Windows)
if let Some(cutoff) = Instant::now().checked_sub(std::time::Duration::from_secs(3600)) {
while window.front().is_some_and(|t| *t < cutoff) {
window.pop_front();
}
}
window.len() as u64
}
@@ -621,4 +625,35 @@ mod tests {
"surcharge should be 100% of input cost for 1h cache writes"
);
}
/// Regression test for #657: Instant::now() - Duration panics on Windows
/// when system uptime is less than the subtracted duration.
#[tokio::test]
async fn test_checked_sub_no_panic_on_fresh_guard() {
// A fresh CostGuard with rate limits should not panic even if
// checked_sub returns None (simulating short uptime).
let guard = CostGuard::new(CostGuardConfig {
max_cost_per_day_cents: None,
max_actions_per_hour: Some(100),
});
// These must not panic regardless of system uptime
assert!(guard.check_allowed().await.is_ok());
assert_eq!(guard.actions_this_hour().await, 0);
// Record some actions and verify again
guard
.record_llm_call("gpt-4o", 10, 10, 0, 0, Decimal::ONE, Decimal::ONE, None)
.await;
assert!(guard.check_allowed().await.is_ok());
assert_eq!(guard.actions_this_hour().await, 1);
}
/// Verify that checked_sub itself behaves as expected for the pattern we use.
#[test]
fn test_instant_checked_sub_returns_none_for_overflow() {
// Duration::MAX will always exceed uptime, so checked_sub must return None
let result = Instant::now().checked_sub(std::time::Duration::MAX);
assert!(result.is_none());
}
}
+20 -12
View File
@@ -50,8 +50,18 @@ impl Agent {
// Load workspace system prompt (identity files: AGENTS.md, SOUL.md, etc.)
// In group chats, MEMORY.md is excluded to prevent leaking personal context.
// Resolve the user's timezone
let user_tz = crate::timezone::resolve_timezone(
message.timezone.as_deref(),
None, // user setting lookup can be added later
&self.config.default_timezone,
);
let system_prompt = if let Some(ws) = self.workspace() {
match ws.system_prompt_for_context(is_group_chat).await {
match ws
.system_prompt_for_context_tz(is_group_chat, user_tz)
.await
{
Ok(prompt) if !prompt.is_empty() => Some(prompt),
Ok(_) => None,
Err(e) => {
@@ -103,7 +113,7 @@ impl Agent {
None
};
let mut reasoning = Reasoning::new(self.llm().clone(), self.safety().clone())
let mut reasoning = Reasoning::new(self.llm().clone())
.with_channel(message.channel.clone())
.with_model_name(self.llm().active_model_name())
.with_group_chat(is_group_chat);
@@ -130,6 +140,7 @@ impl Agent {
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
job_ctx.user_timezone = user_tz.name().to_string();
// Build system prompts once for this turn. Two variants: with tools
// (normal iterations) and without (force_text final iteration).
@@ -785,6 +796,7 @@ impl Agent {
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: tool_calls[approval_idx + 1..].to_vec(),
user_timezone: Some(user_tz.name().to_string()),
};
return Ok(AgenticLoopResult::NeedApproval { pending });
@@ -1146,6 +1158,7 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations: 50,
auto_approve_tools: false,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
@@ -1248,6 +1261,7 @@ mod tests {
arguments: serde_json::json!({"message": "done"}),
},
],
user_timezone: None,
};
let json = serde_json::to_string(&pending).expect("serialize");
@@ -1595,12 +1609,8 @@ mod tests {
use crate::testing::StubLlm;
let stub = Arc::new(StubLlm::failing_non_transient("ctx-bomb"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(stub.clone(), safety);
let reasoning = Reasoning::new(stub.clone());
// Build a fat context with lots of history.
let messages = vec![
@@ -1710,11 +1720,7 @@ mod tests {
use crate::llm::{Reasoning, ReasoningContext, RespondResult, ToolDefinition};
let provider = Arc::new(AlwaysToolCallProvider);
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
let reasoning = Reasoning::new(provider, safety);
let reasoning = Reasoning::new(provider);
let tool_def = ToolDefinition {
name: "echo".to_string(),
@@ -1900,6 +1906,7 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
@@ -2015,6 +2022,7 @@ mod tests {
max_actions_per_hour: None,
max_tool_iterations: max_iter,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
},
deps,
Arc::new(ChannelManager::new()),
+168 -8
View File
@@ -29,8 +29,8 @@ use std::time::Duration;
use tokio::sync::mpsc;
use crate::channels::OutgoingResponse;
use crate::db::Database;
use crate::llm::{ChatMessage, CompletionRequest, LlmProvider, Reasoning};
use crate::safety::SafetyLayer;
use crate::workspace::Workspace;
use crate::workspace::hygiene::HygieneConfig;
@@ -47,6 +47,12 @@ pub struct HeartbeatConfig {
pub notify_user_id: Option<String>,
/// Channel to notify on heartbeat findings.
pub notify_channel: Option<String>,
/// Hour (0-23) when quiet hours start.
pub quiet_hours_start: Option<u32>,
/// Hour (0-23) when quiet hours end.
pub quiet_hours_end: Option<u32>,
/// Timezone for quiet hours evaluation (IANA name).
pub timezone: Option<String>,
}
impl Default for HeartbeatConfig {
@@ -57,6 +63,9 @@ impl Default for HeartbeatConfig {
max_failures: 3,
notify_user_id: None,
notify_channel: None,
quiet_hours_start: None,
quiet_hours_end: None,
timezone: None,
}
}
}
@@ -74,6 +83,26 @@ impl HeartbeatConfig {
self
}
/// Check whether the current time falls within configured quiet hours.
pub fn is_quiet_hours(&self) -> bool {
use chrono::Timelike;
let (Some(start), Some(end)) = (self.quiet_hours_start, self.quiet_hours_end) else {
return false;
};
let tz = self
.timezone
.as_deref()
.and_then(crate::timezone::parse_timezone)
.unwrap_or(chrono_tz::UTC);
let now_hour = crate::timezone::now_in_tz(tz).hour();
if start <= end {
now_hour >= start && now_hour < end
} else {
// Wraps midnight, e.g. 22..06
now_hour >= start || now_hour < end
}
}
/// Set the notification target.
pub fn with_notify(mut self, user_id: impl Into<String>, channel: impl Into<String>) -> Self {
self.notify_user_id = Some(user_id.into());
@@ -101,8 +130,8 @@ pub struct HeartbeatRunner {
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
store: Option<Arc<dyn Database>>,
consecutive_failures: u32,
}
@@ -113,15 +142,14 @@ impl HeartbeatRunner {
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
) -> Self {
Self {
config,
hygiene_config,
workspace,
llm,
safety,
response_tx: None,
store: None,
consecutive_failures: 0,
}
}
@@ -132,6 +160,12 @@ impl HeartbeatRunner {
self
}
/// Set the database store for persistent heartbeat conversations.
pub fn with_store(mut self, store: Arc<dyn Database>) -> Self {
self.store = Some(store);
self
}
/// Run the heartbeat loop.
///
/// This runs forever, checking periodically based on the configured interval.
@@ -153,6 +187,12 @@ impl HeartbeatRunner {
loop {
interval.tick().await;
// Skip during quiet hours
if self.config.is_quiet_hours() {
tracing::debug!("Heartbeat skipped: quiet hours");
continue;
}
// Run memory hygiene in the background so it never delays the
// heartbeat checklist. Failures are logged inside run_if_due.
let hygiene_workspace = Arc::clone(&self.workspace);
@@ -263,7 +303,7 @@ impl HeartbeatRunner {
.with_max_tokens(max_tokens)
.with_temperature(0.3);
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let reasoning = Reasoning::new(self.llm.clone());
let (content, _usage) = match reasoning.complete(request).await {
Ok(r) => r,
Err(e) => return HeartbeatResult::Failed(format!("LLM call failed: {}", e)),
@@ -292,9 +332,32 @@ impl HeartbeatRunner {
return;
};
let user_id = self.config.notify_user_id.as_deref().unwrap_or("default");
// Persist to heartbeat conversation and get thread_id
let thread_id = if let Some(ref store) = self.store {
match store.get_or_create_heartbeat_conversation(user_id).await {
Ok(conv_id) => {
if let Err(e) = store
.add_conversation_message(conv_id, "assistant", message)
.await
{
tracing::error!("Failed to persist heartbeat message: {}", e);
}
Some(conv_id.to_string())
}
Err(e) => {
tracing::error!("Failed to get heartbeat conversation: {}", e);
None
}
}
} else {
None
};
let response = OutgoingResponse {
content: format!("🔔 *Heartbeat Alert*\n\n{}", message),
thread_id: None,
thread_id,
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "heartbeat",
@@ -354,13 +417,16 @@ pub fn spawn_heartbeat(
hygiene_config: HygieneConfig,
workspace: Arc<Workspace>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
response_tx: Option<mpsc::Sender<OutgoingResponse>>,
store: Option<Arc<dyn Database>>,
) -> tokio::task::JoinHandle<()> {
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm, safety);
let mut runner = HeartbeatRunner::new(config, hygiene_config, workspace, llm);
if let Some(tx) = response_tx {
runner = runner.with_response_channel(tx);
}
if let Some(s) = store {
runner = runner.with_store(s);
}
tokio::spawn(async move {
runner.run().await;
@@ -495,4 +561,98 @@ mod tests {
let content = "<!-- comment -->\nActual task here";
assert!(!is_effectively_empty(content));
}
// ==================== quiet hours ====================
#[test]
fn test_quiet_hours_inside() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
let start = hour;
let end = (hour + 1) % 24;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// Current UTC hour is inside [start, end) by construction
assert!(config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_outside() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
let start = (hour + 1) % 24;
let end = (hour + 2) % 24;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// Current UTC hour is outside [start, end) by construction
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_wraparound_excludes_now() {
use chrono::{Timelike, Utc};
let now_utc = Utc::now();
let hour = now_utc.hour();
// Window covers all hours except the current one
let start = (hour + 1) % 24;
let end = hour;
let config = HeartbeatConfig {
quiet_hours_start: Some(start),
quiet_hours_end: Some(end),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_none_configured() {
let config = HeartbeatConfig::default();
assert!(!config.is_quiet_hours());
}
#[test]
fn test_quiet_hours_same_start_end() {
let config = HeartbeatConfig {
quiet_hours_start: Some(10),
quiet_hours_end: Some(10),
timezone: Some("UTC".to_string()),
..HeartbeatConfig::default()
};
// start == end means zero-width window, should be false
assert!(!config.is_quiet_hours());
}
#[test]
fn test_spawn_heartbeat_accepts_store_param() {
// Regression: spawn_heartbeat must accept an optional Database store
// for persisting heartbeat notifications to a dedicated conversation.
// Compile-time check: the 7th parameter is `Option<Arc<dyn Database>>`.
#[allow(clippy::type_complexity)]
let _fn_ptr: fn(
HeartbeatConfig,
HygieneConfig,
Arc<crate::workspace::Workspace>,
Arc<dyn crate::llm::LlmProvider>,
Option<tokio::sync::mpsc::Sender<crate::channels::OutgoingResponse>>,
Option<Arc<dyn crate::db::Database>>,
) -> tokio::task::JoinHandle<()> = spawn_heartbeat;
let _ = _fn_ptr;
}
}
+87 -9
View File
@@ -57,7 +57,11 @@ pub struct Routine {
#[serde(tag = "type", rename_all = "snake_case")]
pub enum Trigger {
/// Fire on a cron schedule (e.g. "0 9 * * MON-FRI" or "every 2h").
Cron { schedule: String },
Cron {
schedule: String,
#[serde(default)]
timezone: Option<String>,
},
/// Fire when a channel message matches a pattern.
Event {
/// Optional channel filter (e.g. "telegram", "slack").
@@ -99,7 +103,21 @@ impl Trigger {
field: "schedule".into(),
})?
.to_string();
Ok(Trigger::Cron { schedule })
let timezone = config
.get("timezone")
.and_then(|v| v.as_str())
.and_then(|tz| {
if crate::timezone::parse_timezone(tz).is_some() {
Some(tz.to_string())
} else {
tracing::warn!(
"Ignoring invalid timezone '{}' from DB for cron trigger",
tz
);
None
}
});
Ok(Trigger::Cron { schedule, timezone })
}
"event" => {
let pattern = config
@@ -137,7 +155,10 @@ impl Trigger {
/// Serialize trigger-specific config to JSON for DB storage.
pub fn to_config_json(&self) -> serde_json::Value {
match self {
Trigger::Cron { schedule } => serde_json::json!({ "schedule": schedule }),
Trigger::Cron { schedule, timezone } => serde_json::json!({
"schedule": schedule,
"timezone": timezone,
}),
Trigger::Event { channel, pattern } => serde_json::json!({
"pattern": pattern,
"channel": channel,
@@ -415,12 +436,25 @@ pub fn content_hash(content: &str) -> u64 {
}
/// Parse a cron expression and compute the next fire time from now.
pub fn next_cron_fire(schedule: &str) -> Result<Option<DateTime<Utc>>, RoutineError> {
///
/// When `timezone` is provided and valid, the schedule is evaluated in that
/// timezone and the result is converted back to UTC. Otherwise UTC is used.
pub fn next_cron_fire(
schedule: &str,
timezone: Option<&str>,
) -> Result<Option<DateTime<Utc>>, RoutineError> {
let cron_schedule =
cron::Schedule::from_str(schedule).map_err(|e| RoutineError::InvalidCron {
reason: e.to_string(),
})?;
Ok(cron_schedule.upcoming(Utc).next())
if let Some(tz) = timezone.and_then(crate::timezone::parse_timezone) {
Ok(cron_schedule
.upcoming(tz)
.next()
.map(|dt| dt.with_timezone(&Utc)))
} else {
Ok(cron_schedule.upcoming(Utc).next())
}
}
#[cfg(test)]
@@ -433,10 +467,11 @@ mod tests {
fn test_trigger_roundtrip() {
let trigger = Trigger::Cron {
schedule: "0 9 * * MON-FRI".to_string(),
timezone: None,
};
let json = trigger.to_config_json();
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { schedule } if schedule == "0 9 * * MON-FRI"));
assert!(matches!(parsed, Trigger::Cron { schedule, .. } if schedule == "0 9 * * MON-FRI"));
}
#[test]
@@ -509,16 +544,58 @@ mod tests {
#[test]
fn test_next_cron_fire_valid() {
// Every minute should always have a next fire
let next = next_cron_fire("* * * * * *").expect("valid cron");
let next = next_cron_fire("* * * * * *", None).expect("valid cron");
assert!(next.is_some());
}
#[test]
fn test_next_cron_fire_invalid() {
let result = next_cron_fire("not a cron");
let result = next_cron_fire("not a cron", None);
assert!(result.is_err());
}
#[test]
fn test_trigger_cron_timezone_roundtrip() {
let trigger = Trigger::Cron {
schedule: "0 9 * * MON-FRI".to_string(),
timezone: Some("America/New_York".to_string()),
};
let json = trigger.to_config_json();
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { schedule, timezone }
if schedule == "0 9 * * MON-FRI"
&& timezone.as_deref() == Some("America/New_York")));
}
#[test]
fn test_trigger_cron_no_timezone_backward_compat() {
let json = serde_json::json!({"schedule": "0 9 * * *"});
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(matches!(parsed, Trigger::Cron { timezone, .. } if timezone.is_none()));
}
#[test]
fn test_trigger_cron_invalid_timezone_coerced_to_none() {
let json = serde_json::json!({"schedule": "0 9 * * *", "timezone": "Fake/Zone"});
let parsed = Trigger::from_db("cron", json).expect("parse cron");
assert!(
matches!(parsed, Trigger::Cron { timezone, .. } if timezone.is_none()),
"invalid timezone should be coerced to None"
);
}
#[test]
fn test_next_cron_fire_with_timezone() {
let next_utc = next_cron_fire("0 0 9 * * * *", None)
.expect("valid cron")
.expect("has next");
let next_est = next_cron_fire("0 0 9 * * * *", Some("America/New_York"))
.expect("valid cron")
.expect("has next");
// EST is UTC-5 (or EDT UTC-4), so the UTC result should differ
assert_ne!(next_utc, next_est, "timezone should shift the fire time");
}
#[test]
fn test_guardrails_default() {
let g = RoutineGuardrails::default();
@@ -531,7 +608,8 @@ mod tests {
fn test_trigger_type_tag() {
assert_eq!(
Trigger::Cron {
schedule: String::new()
schedule: String::new(),
timezone: None,
}
.type_tag(),
"cron"
+60 -15
View File
@@ -170,7 +170,7 @@ impl RoutineEngine {
continue;
}
let detail = if let Trigger::Cron { ref schedule } = routine.trigger {
let detail = if let Trigger::Cron { ref schedule, .. } = routine.trigger {
Some(schedule.clone())
} else {
None
@@ -184,7 +184,11 @@ impl RoutineEngine {
///
/// Bypasses cooldown checks (those only apply to cron/event triggers).
/// Still enforces enabled check and concurrent run limit.
pub async fn fire_manual(&self, routine_id: Uuid) -> Result<Uuid, RoutineError> {
pub async fn fire_manual(
&self,
routine_id: Uuid,
user_id: Option<&str>,
) -> Result<Uuid, RoutineError> {
let routine = self
.store
.get_routine(routine_id)
@@ -194,6 +198,13 @@ impl RoutineEngine {
})?
.ok_or(RoutineError::NotFound { id: routine_id })?;
// Enforce ownership when a user_id is provided (gateway calls).
if let Some(uid) = user_id
&& routine.user_id != uid
{
return Err(RoutineError::NotAuthorized { id: routine_id });
}
if !routine.enabled {
return Err(RoutineError::Disabled {
name: routine.name.clone(),
@@ -369,8 +380,12 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
// Update routine runtime state
let now = Utc::now();
let next_fire = if let Trigger::Cron { ref schedule } = routine.trigger {
next_cron_fire(schedule).unwrap_or(None)
let next_fire = if let Trigger::Cron {
ref schedule,
ref timezone,
} = routine.trigger
{
next_cron_fire(schedule, timezone.as_deref()).unwrap_or(None)
} else {
None
};
@@ -396,6 +411,39 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
tracing::error!(routine = %routine.name, "Failed to update runtime state: {}", e);
}
// Persist routine result to its dedicated conversation thread
let thread_id = match ctx
.store
.get_or_create_routine_conversation(routine.id, &routine.name, &routine.user_id)
.await
{
Ok(conv_id) => {
tracing::debug!(
routine = %routine.name,
routine_id = %routine.id,
conversation_id = %conv_id,
"Resolved routine conversation thread"
);
// Record the run result as a conversation message
let msg = match (&summary, status) {
(Some(s), _) => format!("[{}] {}: {}", run.trigger_type, status, s),
(None, _) => format!("[{}] {}", run.trigger_type, status),
};
if let Err(e) = ctx
.store
.add_conversation_message(conv_id, "assistant", &msg)
.await
{
tracing::error!(routine = %routine.name, "Failed to persist routine message: {}", e);
}
Some(conv_id.to_string())
}
Err(e) => {
tracing::error!(routine = %routine.name, "Failed to get routine conversation: {}", e);
None
}
};
// Send notifications based on config
send_notification(
&ctx.notify_tx,
@@ -403,6 +451,7 @@ async fn execute_routine(ctx: EngineContext, routine: Routine, run: RoutineRun)
&routine.name,
status,
summary.as_deref(),
thread_id.as_deref(),
)
.await;
}
@@ -443,18 +492,13 @@ async fn execute_full_job(
reason: "scheduler not available".to_string(),
})?;
// Set the message tool's default channel/target from the routine's notify config
// so the LLM can send results without triggering cross-channel approval.
// TODO: This mutates shared global state and can race with concurrent jobs.
// Move notify config into JobContext metadata and apply per-job instead.
let mut metadata = serde_json::json!({ "max_iterations": max_iterations });
// Carry the routine's notify config in job metadata so the message tool
// can resolve channel/target per-job without global state mutation.
if let Some(channel) = &routine.notify.channel {
scheduler
.tools()
.set_message_tool_context(Some(channel.clone()), Some(routine.notify.user.clone()))
.await;
metadata["notify_channel"] = serde_json::json!(channel);
}
let metadata = serde_json::json!({ "max_iterations": max_iterations });
metadata["notify_user"] = serde_json::json!(&routine.notify.user);
// Build approval context: UnlessAutoApproved tools are auto-approved for routines;
// Always tools require explicit listing in tool_permissions.
@@ -611,6 +655,7 @@ async fn send_notification(
routine_name: &str,
status: RunStatus,
summary: Option<&str>,
thread_id: Option<&str>,
) {
let should_notify = match status {
RunStatus::Ok => notify.on_success,
@@ -637,7 +682,7 @@ async fn send_notification(
let response = OutgoingResponse {
content: message,
thread_id: None,
thread_id: thread_id.map(String::from),
attachments: Vec::new(),
metadata: serde_json::json!({
"source": "routine",
+6
View File
@@ -164,6 +164,10 @@ pub struct PendingApproval {
/// executed yet when approval was requested.
#[serde(default)]
pub deferred_tool_calls: Vec<ToolCall>,
/// User timezone at the time the approval was requested, so it persists
/// through the approval flow even if the approval message lacks timezone.
#[serde(default)]
pub user_timezone: Option<String>,
}
/// A conversation thread within a session.
@@ -976,6 +980,7 @@ mod tests {
tool_call_id: "call_123".to_string(),
context_messages: vec![ChatMessage::user("do it")],
deferred_tool_calls: vec![],
user_timezone: None,
};
thread.await_approval(approval);
@@ -1001,6 +1006,7 @@ mod tests {
tool_call_id: "call_456".to_string(),
context_messages: vec![],
deferred_tool_calls: vec![],
user_timezone: None,
};
thread.await_approval(approval);
+14 -2
View File
@@ -230,7 +230,7 @@ impl Agent {
)
.await;
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
let compactor = ContextCompactor::new(self.llm().clone());
if let Err(e) = compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -627,7 +627,7 @@ impl Agent {
crate::agent::context_monitor::CompactionStrategy::Summarize { keep_recent: 5 },
);
let compactor = ContextCompactor::new(self.llm().clone(), self.safety().clone());
let compactor = ContextCompactor::new(self.llm().clone());
match compactor
.compact(thread, strategy, self.workspace().map(|w| w.as_ref()))
.await
@@ -746,6 +746,16 @@ impl Agent {
let mut job_ctx =
JobContext::with_user(&message.user_id, "chat", "Interactive chat session");
job_ctx.http_interceptor = self.deps.http_interceptor.clone();
// Prefer a valid timezone from the approval message, fall back to the
// resolved timezone stored when the approval was originally requested.
let tz_candidate = message
.timezone
.as_deref()
.filter(|tz| crate::timezone::parse_timezone(tz).is_some())
.or(pending.user_timezone.as_deref());
if let Some(tz) = tz_candidate {
job_ctx.user_timezone = tz.to_string();
}
let _ = self
.channels
@@ -1111,6 +1121,8 @@ impl Agent {
tool_call_id: tc.id.clone(),
context_messages: context_messages.clone(),
deferred_tool_calls: deferred_tool_calls[approval_idx + 1..].to_vec(),
// Carry forward the resolved timezone from the original pending approval
user_timezone: pending.user_timezone.clone(),
};
let request_id = new_pending.request_id;
+65 -34
View File
@@ -15,7 +15,8 @@ use crate::db::Database;
use crate::error::Error;
use crate::hooks::HookRegistry;
use crate::llm::{
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolSelection,
ActionPlan, ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolCall,
ToolSelection,
};
use crate::safety::SafetyLayer;
use crate::tools::rate_limiter::RateLimitResult;
@@ -211,7 +212,7 @@ impl Worker {
let job_ctx = self.context_manager().get_context(self.job_id).await?;
// Create reasoning engine
let reasoning = Reasoning::new(self.llm().clone(), self.safety().clone());
let reasoning = Reasoning::new(self.llm().clone());
// Build initial reasoning context (tool definitions refreshed each iteration in execution_loop)
let mut reason_ctx = ReasoningContext::new().with_job(&job_ctx.description);
@@ -576,37 +577,54 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
}
}
}
} else if selections.len() == 1 {
consecutive_tool_intent_nudges = 0;
// Single tool: execute directly
let selection = &selections[0];
tracing::debug!(
"Job {} selecting tool: {} - {}",
self.job_id,
selection.tool_name,
selection.reasoning
);
let result = self
.execute_tool(&selection.tool_name, &selection.parameters)
.await;
self.process_tool_result(reason_ctx, selection, result)
.await?;
} else {
// Multiple tools: execute in parallel
tracing::debug!(
"Job {} executing {} tools in parallel",
self.job_id,
selections.len()
);
consecutive_tool_intent_nudges = 0;
let results = self.execute_tools_parallel(&selections).await;
// Record the assistant tool_calls message so that tool_result
// messages have a matching parent (prevents orphaned rewrites).
let tool_calls: Vec<ToolCall> = selections
.iter()
.map(|s| ToolCall {
id: s.tool_call_id.clone(),
name: s.tool_name.clone(),
arguments: s.parameters.clone(),
})
.collect();
reason_ctx
.messages
.push(ChatMessage::assistant_with_tool_calls(None, tool_calls));
// Process all results
for (selection, result) in selections.iter().zip(results) {
self.process_tool_result(reason_ctx, selection, result.result)
if selections.len() == 1 {
// Single tool: execute directly
let selection = &selections[0];
tracing::debug!(
"Job {} selecting tool: {} - {}",
self.job_id,
selection.tool_name,
selection.reasoning
);
let result = self
.execute_tool(&selection.tool_name, &selection.parameters)
.await;
self.process_tool_result(reason_ctx, selection, result)
.await?;
} else {
// Multiple tools: execute in parallel
tracing::debug!(
"Job {} executing {} tools in parallel",
self.job_id,
selections.len()
);
let results = self.execute_tools_parallel(&selections).await;
// Process all results
for (selection, result) in selections.iter().zip(results) {
self.process_tool_result(reason_ctx, selection, result.result)
.await?;
}
}
}
@@ -1087,11 +1105,6 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
action.reasoning
);
// Execute the planned tool
let result = self
.execute_tool(&action.tool_name, &action.parameters)
.await;
// Create a synthetic ToolSelection for process_tool_result.
// Plan actions don't originate from an LLM tool_call response so
// there is no real tool_call_id; generate a unique one.
@@ -1103,6 +1116,24 @@ Report when the job is complete or if you encounter issues you cannot resolve."#
tool_call_id: format!("plan_{}_{}", self.job_id, i),
};
// Record the assistant tool_calls message so that the tool_result
// has a matching parent (prevents orphaned rewrites).
reason_ctx
.messages
.push(ChatMessage::assistant_with_tool_calls(
None,
vec![ToolCall {
id: selection.tool_call_id.clone(),
name: selection.tool_name.clone(),
arguments: selection.parameters.clone(),
}],
));
// Execute the planned tool
let result = self
.execute_tool(&action.tool_name, &action.parameters)
.await;
// Process the result
let completed = self
.process_tool_result(reason_ctx, &selection, result)
+83 -17
View File
@@ -244,11 +244,31 @@ impl AppBuilder {
let master_key = match self.config.secrets.master_key() {
Some(k) => k,
None => {
// No secrets DB available, but we can still load tokens from
// OS credential stores (e.g., Anthropic OAuth via Claude Code's
// macOS Keychain / Linux ~/.claude/.credentials.json).
crate::config::inject_os_credentials();
// Consume unused handles
#[cfg(feature = "libsql")]
{
self.libsql_db.take();
}
// Re-resolve only the LLM config with OS credentials.
let store: Option<&(dyn crate::db::SettingsStore + Sync)> =
self.db.as_ref().map(|db| db.as_ref() as _);
let toml_path = self.toml_path.as_deref();
if let Err(e) = self
.config
.re_resolve_llm(store, "default", toml_path)
.await
{
tracing::warn!(
"Failed to re-resolve LLM config after OS credential injection: {e}"
);
}
return Ok(());
}
};
@@ -291,18 +311,16 @@ impl AppBuilder {
// Inject LLM API keys from encrypted storage
crate::config::inject_llm_keys_from_secrets(secrets.as_ref(), "default").await;
// Re-resolve config with newly available keys
if let Some(ref db) = self.db {
let toml_path = self.toml_path.as_deref();
match Config::from_db_with_toml(db.as_ref(), "default", toml_path).await {
Ok(refreshed) => {
self.config = refreshed;
tracing::debug!("LlmConfig re-resolved after secret injection");
}
Err(e) => {
tracing::warn!("Failed to re-resolve config after secret injection: {}", e);
}
}
// Re-resolve only the LLM config with newly available keys.
let store: Option<&(dyn crate::db::SettingsStore + Sync)> =
self.db.as_ref().map(|db| db.as_ref() as _);
let toml_path = self.toml_path.as_deref();
if let Err(e) = self
.config
.re_resolve_llm(store, "default", toml_path)
.await
{
tracing::warn!("Failed to re-resolve LLM config after secret injection: {e}");
}
}
@@ -368,12 +386,53 @@ impl AppBuilder {
.embeddings
.create_provider(&self.config.llm.nearai.base_url, self.session.clone());
// Create optional external vector store for workspace semantic search
let vector_store: Option<Arc<dyn crate::workspace::VectorStore>> = {
#[cfg(feature = "lancedb")]
{
if self.config.database.vector_backend == crate::config::VectorBackend::LanceDb {
let path = self
.config
.database
.lancedb_path
.clone()
.unwrap_or_else(crate::config::default_lancedb_path);
let dim = embeddings.as_ref().map(|p| p.dimension());
match crate::workspace::LanceDbVectorStore::new(path, dim).await {
Ok(store) => {
tracing::info!("LanceDB vector store connected for workspace search");
Some(Arc::new(store) as Arc<dyn crate::workspace::VectorStore>)
}
Err(e) => {
tracing::warn!("Failed to initialize LanceDB: {}", e);
None
}
}
} else {
None
}
}
#[cfg(not(feature = "lancedb"))]
{
if self.config.database.vector_backend == crate::config::VectorBackend::LanceDb {
tracing::warn!(
"VECTOR_BACKEND=lancedb but 'lancedb' feature not enabled; \
falling back to built-in vector search"
);
}
None
}
};
// Register memory tools if database is available
let workspace = if let Some(ref db) = self.db {
let mut ws = Workspace::new_with_db("default", db.clone());
if let Some(ref emb) = embeddings {
ws = ws.with_embeddings(emb.clone());
}
if let Some(ref vs) = vector_store {
ws = ws.with_vector_store(vs.clone());
}
let ws = Arc::new(ws);
tools.register_memory_tools(Arc::clone(&ws));
Some(ws)
@@ -386,11 +445,7 @@ impl AppBuilder {
&& (self.config.agent.allow_local_tools || !self.config.sandbox.enabled)
{
tools
.register_builder_tool(
llm.clone(),
safety.clone(),
Some(self.config.builder.to_builder_config()),
)
.register_builder_tool(llm.clone(), Some(self.config.builder.to_builder_config()))
.await;
tracing::info!("Builder mode enabled");
}
@@ -665,6 +720,17 @@ impl AppBuilder {
self.init_database().await?;
self.init_secrets().await?;
// Post-init validation: if a non-nearai backend was selected but
// credentials were never resolved (deferred resolution found no keys),
// fail early with a clear error instead of a confusing runtime failure.
if self.config.llm.backend != "nearai" && self.config.llm.provider.is_none() {
let backend = &self.config.llm.backend;
anyhow::bail!(
"LLM_BACKEND={backend} is configured but no credentials were found. \
Set the appropriate API key environment variable or run the setup wizard."
);
}
let (llm, cheap_llm, recording_handle) = if let Some(llm) = self.llm_override.take() {
(llm, None, None)
} else {
+251
View File
@@ -414,10 +414,103 @@ pub enum MigrationError {
Io(String),
}
// ── PID Lock ──────────────────────────────────────────────────────────────
/// Path to the PID lock file: `~/.ironclaw/ironclaw.pid`.
pub fn pid_lock_path() -> PathBuf {
ironclaw_base_dir().join("ironclaw.pid")
}
/// A PID-based lock that prevents multiple IronClaw instances from running
/// simultaneously.
///
/// Uses `fs4::try_lock_exclusive()` for atomic locking (no TOCTOU race),
/// then writes the current PID into the locked file for diagnostics.
/// The OS-level lock is held for the lifetime of this struct and
/// automatically released on drop (along with the PID file cleanup).
#[derive(Debug)]
pub struct PidLock {
path: PathBuf,
/// Held open to maintain the OS-level exclusive lock.
_file: std::fs::File,
}
/// Errors from PID lock acquisition.
#[derive(Debug, thiserror::Error)]
pub enum PidLockError {
#[error("Another IronClaw instance is already running (PID {pid})")]
AlreadyRunning { pid: u32 },
#[error("Failed to acquire PID lock: {0}")]
Io(#[from] std::io::Error),
}
impl PidLock {
/// Try to acquire the PID lock.
///
/// Uses an exclusive file lock (`flock`/`LockFileEx`) so that two
/// concurrent processes cannot both acquire the lock — no TOCTOU race.
/// If the lock file exists but the holding process is gone (stale),
/// the lock is reclaimed automatically by the OS.
pub fn acquire() -> Result<Self, PidLockError> {
Self::acquire_at(pid_lock_path())
}
/// Acquire at a specific path (for testing).
fn acquire_at(path: PathBuf) -> Result<Self, PidLockError> {
use fs4::FileExt;
use std::fs::OpenOptions;
use std::io::Write;
// Ensure parent directory exists
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
// Open (or create) the lock file
let mut file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(false)
.open(&path)?;
// Try non-blocking exclusive lock — if another process holds it,
// this fails immediately instead of blocking.
if let Err(e) = file.try_lock_exclusive() {
if e.kind() == std::io::ErrorKind::WouldBlock {
// Lock held by another process — read its PID for the error message
let pid = std::fs::read_to_string(&path)
.ok()
.and_then(|s| s.trim().parse::<u32>().ok())
.unwrap_or(0);
return Err(PidLockError::AlreadyRunning { pid });
}
// Other errors (permissions, unsupported filesystem, etc.)
return Err(PidLockError::Io(e));
}
// We hold the exclusive lock — write our PID
file.set_len(0)?; // truncate
write!(file, "{}", std::process::id())?;
Ok(PidLock { path, _file: file })
}
}
impl Drop for PidLock {
fn drop(&mut self) {
// Remove the PID file; the OS-level lock is released when _file is dropped.
let _ = std::fs::remove_file(&self.path);
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::process::Command;
use std::sync::Mutex;
use std::thread;
use std::time::{Duration, Instant};
use tempfile::tempdir;
static ENV_MUTEX: Mutex<()> = Mutex::new(());
@@ -986,4 +1079,162 @@ INJECTED="pwned"#;
unsafe { std::env::remove_var("IRONCLAW_BASE_DIR") };
}
}
// ── PID Lock tests ───────────────────────────────────────────────
#[test]
fn test_pid_lock_acquire_and_drop() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Acquire lock
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
assert!(pid_path.exists());
// PID file should contain our PID
let contents = std::fs::read_to_string(&pid_path).unwrap();
assert_eq!(contents.trim().parse::<u32>().unwrap(), std::process::id());
// Drop should remove the file
drop(lock);
assert!(!pid_path.exists());
}
#[test]
fn test_pid_lock_rejects_second_acquire() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// First lock succeeds
let _lock1 = PidLock::acquire_at(pid_path.clone()).unwrap();
// Second acquire on same file must fail (exclusive flock held)
let result = PidLock::acquire_at(pid_path.clone());
assert!(result.is_err());
match result.unwrap_err() {
PidLockError::AlreadyRunning { pid } => {
assert_eq!(pid, std::process::id());
}
other => panic!("expected AlreadyRunning, got: {}", other),
}
}
#[test]
fn test_pid_lock_reclaims_after_drop() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Acquire and release
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
drop(lock);
// Should succeed — OS lock was released on drop
let lock2 = PidLock::acquire_at(pid_path).unwrap();
drop(lock2);
}
#[test]
fn test_pid_lock_reclaims_stale_file_without_flock() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Write a stale PID file manually (no flock held)
std::fs::write(&pid_path, "4294967294").unwrap();
// Should succeed because no OS lock is held on the file
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
let contents = std::fs::read_to_string(&pid_path).unwrap();
assert_eq!(contents.trim().parse::<u32>().unwrap(), std::process::id());
drop(lock);
}
#[test]
fn test_pid_lock_handles_corrupt_pid_file() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
// Write garbage (no flock held)
std::fs::write(&pid_path, "not-a-number").unwrap();
// Should succeed — no OS lock held, file is reclaimed
let lock = PidLock::acquire_at(pid_path).unwrap();
drop(lock);
}
#[test]
fn test_pid_lock_creates_parent_dirs() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("nested").join("deep").join("ironclaw.pid");
let lock = PidLock::acquire_at(pid_path.clone()).unwrap();
assert!(pid_path.exists());
drop(lock);
}
#[test]
fn test_pid_lock_child_helper_holds_lock() {
if std::env::var("IRONCLAW_PID_LOCK_CHILD").ok().as_deref() != Some("1") {
return;
}
let pid_path = PathBuf::from(
std::env::var("IRONCLAW_PID_LOCK_PATH").expect("IRONCLAW_PID_LOCK_PATH missing"),
);
let hold_ms = std::env::var("IRONCLAW_PID_LOCK_HOLD_MS")
.ok()
.and_then(|s| s.parse::<u64>().ok())
.unwrap_or(3000);
let _lock = PidLock::acquire_at(pid_path).expect("child failed to acquire pid lock");
thread::sleep(Duration::from_millis(hold_ms));
}
#[test]
fn test_pid_lock_rejects_lock_held_by_other_process() {
let dir = tempdir().unwrap();
let pid_path = dir.path().join("ironclaw.pid");
let current_exe = std::env::current_exe().unwrap();
let mut child = Command::new(current_exe)
.args([
"--exact",
"bootstrap::tests::test_pid_lock_child_helper_holds_lock",
"--nocapture",
"--test-threads=1",
])
.env("IRONCLAW_PID_LOCK_CHILD", "1")
.env("IRONCLAW_PID_LOCK_PATH", pid_path.display().to_string())
.env("IRONCLAW_PID_LOCK_HOLD_MS", "3000")
.spawn()
.unwrap();
let started = Instant::now();
while started.elapsed() < Duration::from_secs(2) {
if pid_path.exists() {
break;
}
if let Some(status) = child.try_wait().unwrap() {
panic!("child exited before acquiring lock: {}", status);
}
thread::sleep(Duration::from_millis(20));
}
assert!(
pid_path.exists(),
"child did not create lock file in time: {}",
pid_path.display()
);
let result = PidLock::acquire_at(pid_path.clone());
match result.unwrap_err() {
PidLockError::AlreadyRunning { .. } => {}
other => panic!("expected AlreadyRunning, got: {}", other),
}
let status = child.wait().unwrap();
assert!(status.success(), "child process failed: {}", status);
// After the child exits, lock should be released and reacquirable.
let lock = PidLock::acquire_at(pid_path).unwrap();
drop(lock);
}
}
+15
View File
@@ -79,6 +79,8 @@ pub struct IncomingMessage {
pub received_at: DateTime<Utc>,
/// Channel-specific metadata.
pub metadata: serde_json::Value,
/// IANA timezone string from the client (e.g. "America/New_York").
pub timezone: Option<String>,
/// File or media attachments on this message.
pub attachments: Vec<IncomingAttachment>,
}
@@ -99,6 +101,7 @@ impl IncomingMessage {
thread_id: None,
received_at: Utc::now(),
metadata: serde_json::Value::Null,
timezone: None,
attachments: Vec::new(),
}
}
@@ -121,6 +124,12 @@ impl IncomingMessage {
self
}
/// Set the client timezone.
pub fn with_timezone(mut self, tz: impl Into<String>) -> Self {
self.timezone = Some(tz.into());
self
}
/// Set attachments.
pub fn with_attachments(mut self, attachments: Vec<IncomingAttachment>) -> Self {
self.attachments = attachments;
@@ -454,4 +463,10 @@ mod tests {
panic!("expected ToolCompleted variant");
}
}
#[test]
fn test_incoming_message_with_timezone() {
let msg = IncomingMessage::new("test", "user1", "hello").with_timezone("America/New_York");
assert_eq!(msg.timezone.as_deref(), Some("America/New_York"));
}
}
+50 -9
View File
@@ -18,7 +18,7 @@
//! - `Esc` - Interrupt current operation
use std::borrow::Cow;
use std::io::{self, Write};
use std::io::{self, IsTerminal, Write};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
@@ -297,10 +297,15 @@ impl Channel for ReplChannel {
let esc_interrupt_triggered_for_thread = Arc::new(AtomicBool::new(false));
std::thread::spawn(move || {
let sys_tz = crate::timezone::detect_system_timezone().name().to_string();
// Single message mode: send it and return
if let Some(msg) = single_message {
let incoming = IncomingMessage::new("repl", "default", &msg);
let incoming = IncomingMessage::new("repl", "default", &msg).with_timezone(&sys_tz);
let _ = tx.blocking_send(incoming);
// Ensure the agent exits after handling exactly one turn in -m mode,
// even when other channels (gateway/http) are enabled.
let _ = tx.blocking_send(IncomingMessage::new("repl", "default", "/quit"));
return;
}
@@ -361,7 +366,8 @@ impl Channel for ReplChannel {
"/quit" | "/exit" => {
// Forward shutdown command so the agent loop exits even
// when other channels (e.g. web gateway) are still active.
let msg = IncomingMessage::new("repl", "default", "/quit");
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let _ = tx.blocking_send(msg);
break;
}
@@ -382,7 +388,8 @@ impl Channel for ReplChannel {
_ => {}
}
let msg = IncomingMessage::new("repl", "default", line);
let msg =
IncomingMessage::new("repl", "default", line).with_timezone(&sys_tz);
if tx.blocking_send(msg).is_err() {
break;
}
@@ -390,21 +397,29 @@ impl Channel for ReplChannel {
Err(ReadlineError::Interrupted) => {
if esc_interrupt_triggered_for_thread.swap(false, Ordering::Relaxed) {
// Esc: interrupt current operation and keep REPL open.
let msg = IncomingMessage::new("repl", "default", "/interrupt");
let msg = IncomingMessage::new("repl", "default", "/interrupt")
.with_timezone(&sys_tz);
if tx.blocking_send(msg).is_err() {
break;
}
} else {
// Ctrl+C (VINTR): request graceful shutdown.
let msg = IncomingMessage::new("repl", "default", "/quit");
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let _ = tx.blocking_send(msg);
break;
}
}
Err(ReadlineError::Eof) => {
// Ctrl+D: send /quit so the agent loop runs graceful shutdown
let msg = IncomingMessage::new("repl", "default", "/quit");
let _ = tx.blocking_send(msg);
// Ctrl+D in interactive mode: graceful shutdown.
// In daemon mode (stdin = /dev/null, no TTY), EOF arrives
// immediately — just drop the REPL thread silently so other
// channels (gateway, telegram, …) keep running.
if std::io::stdin().is_terminal() {
let msg = IncomingMessage::new("repl", "default", "/quit")
.with_timezone(&sys_tz);
let _ = tx.blocking_send(msg);
}
break;
}
Err(e) => {
@@ -614,3 +629,29 @@ impl Channel for ReplChannel {
Ok(())
}
}
#[cfg(test)]
mod tests {
use futures::StreamExt;
use super::*;
#[tokio::test]
async fn single_message_mode_sends_message_then_quit() {
let repl = ReplChannel::with_message("hi".to_string());
let mut stream = repl.start().await.expect("repl start should succeed");
let first = stream.next().await.expect("first message missing");
assert_eq!(first.channel, "repl");
assert_eq!(first.content, "hi");
let second = stream.next().await.expect("quit message missing");
assert_eq!(second.channel, "repl");
assert_eq!(second.content, "/quit");
assert!(
stream.next().await.is_none(),
"stream should end after /quit"
);
}
}
+38 -33
View File
@@ -426,7 +426,7 @@ pub async fn chat_threads_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if let Ok(summaries) = store
.list_conversations_with_preview(&state.user_id, "gateway", 50)
.list_conversations_all_channels(&state.user_id, 50)
.await
{
let mut assistant_thread = None;
@@ -441,6 +441,7 @@ pub async fn chat_threads_handler(
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
thread_type: s.thread_type.clone(),
channel: Some(s.channel.clone()),
};
if s.id == assistant_id {
@@ -460,6 +461,7 @@ pub async fn chat_threads_handler(
updated_at: chrono::Utc::now().to_rfc3339(),
title: None,
thread_type: Some("assistant".to_string()),
channel: Some("gateway".to_string()),
});
}
@@ -472,9 +474,10 @@ pub async fn chat_threads_handler(
}
// Fallback: in-memory only (no assistant thread without DB)
let threads: Vec<ThreadInfo> = sess
.threads
.values()
let mut sorted_threads: Vec<_> = sess.threads.values().collect();
sorted_threads.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
let threads: Vec<ThreadInfo> = sorted_threads
.into_iter()
.map(|t| ThreadInfo {
id: t.id,
state: format!("{:?}", t.state),
@@ -483,6 +486,7 @@ pub async fn chat_threads_handler(
updated_at: t.updated_at.to_rfc3339(),
title: None,
thread_type: None,
channel: Some("gateway".to_string()),
})
.collect();
@@ -502,38 +506,39 @@ pub async fn chat_new_thread_handler(
))?;
let session = session_manager.get_or_create_session(&state.user_id).await;
let mut sess = session.lock().await;
let thread = sess.create_thread();
let thread_id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
let (thread_id, info) = {
let mut sess = session.lock().await;
let thread = sess.create_thread();
let id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
channel: Some("gateway".to_string()),
};
(id, info)
};
// Persist the empty conversation row with thread_type metadata
// Persist the empty conversation row with thread_type metadata synchronously
// so that the subsequent loadThreads() call from the frontend sees it.
if let Some(ref store) = state.store {
let store = Arc::clone(store);
let user_id = state.user_id.clone();
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
});
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &state.user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
}
Ok(Json(info))
+24 -43
View File
@@ -10,9 +10,9 @@ use axum::{
use serde::Deserialize;
use uuid::Uuid;
use crate::channels::IncomingMessage;
use crate::channels::web::server::GatewayState;
use crate::channels::web::types::*;
use crate::error::RoutineError;
pub async fn routines_list_handler(
State(state): State<Arc<GatewayState>>,
@@ -133,56 +133,27 @@ pub async fn routines_trigger_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
// Clone the Arc out of the lock to avoid holding the RwLock across .await.
let engine = {
let guard = state.routine_engine.read().await;
guard.as_ref().cloned().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Routine engine not available".to_string(),
))?
};
let routine_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid routine ID".to_string()))?;
let routine = store
.get_routine(routine_id)
let run_id = engine
.fire_manual(routine_id, Some(&state.user_id))
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
if routine.user_id != state.user_id {
return Err((StatusCode::FORBIDDEN, "Access denied".to_string()));
}
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
crate::agent::routine::RoutineAction::FullJob {
title, description, ..
} => format!("{}: {}", title, description),
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let thread_id = format!(
"routine-{}-{}",
routine_id,
chrono::Utc::now().timestamp_millis()
);
let msg = IncomingMessage::new("gateway", &state.user_id, content).with_thread(thread_id);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Channel not started".to_string(),
))?;
tx.send(msg).await.map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
"Channel closed".to_string(),
)
})?;
.map_err(|e| (routine_error_status(&e), e.to_string()))?;
Ok(Json(serde_json::json!({
"status": "triggered",
"routine_id": routine_id,
"run_id": run_id,
})))
}
@@ -293,7 +264,7 @@ pub async fn routines_runs_handler(
/// Convert a Routine to the trimmed RoutineInfo for list display.
fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
let (trigger_type, trigger_summary) = match &r.trigger {
crate::agent::routine::Trigger::Cron { schedule } => {
crate::agent::routine::Trigger::Cron { schedule, .. } => {
("cron".to_string(), format!("cron: {}", schedule))
}
crate::agent::routine::Trigger::Event {
@@ -337,3 +308,13 @@ fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
status: status.to_string(),
}
}
/// Map `RoutineError` variants to appropriate HTTP status codes.
fn routine_error_status(err: &RoutineError) -> StatusCode {
match err {
RoutineError::NotFound { .. } => StatusCode::NOT_FOUND,
RoutineError::NotAuthorized { .. } => StatusCode::FORBIDDEN,
RoutineError::Disabled { .. } | RoutineError::MaxConcurrent { .. } => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
}
}
+21 -2
View File
@@ -99,6 +99,7 @@ impl GatewayChannel {
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
});
@@ -134,6 +135,7 @@ impl GatewayChannel {
chat_rate_limiter: server::RateLimiter::new(30, 60),
registry_entries: self.state.registry_entries.clone(),
cost_guard: self.state.cost_guard.clone(),
routine_engine: Arc::clone(&self.state.routine_engine),
startup_time: self.state.startup_time,
};
mutate(&mut new_state);
@@ -281,7 +283,15 @@ impl Channel for GatewayChannel {
msg: &IncomingMessage,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let thread_id = msg.thread_id.clone().unwrap_or_default();
let thread_id = match &msg.thread_id {
Some(tid) => tid.clone(),
None => {
tracing::warn!(
"Gateway respond with no thread_id — skipping (clients would drop it)"
);
return Ok(());
}
};
self.state.sse.broadcast(SseEvent::Response {
content: response.content,
@@ -387,9 +397,18 @@ impl Channel for GatewayChannel {
_user_id: &str,
response: OutgoingResponse,
) -> Result<(), ChannelError> {
let thread_id = match response.thread_id {
Some(tid) => tid,
None => {
tracing::warn!(
"Gateway broadcast with no thread_id — skipping (clients would drop it)"
);
return Ok(());
}
};
self.state.sse.broadcast(SseEvent::Response {
content: response.content,
thread_id: String::new(),
thread_id,
});
Ok(())
}
+81 -68
View File
@@ -57,6 +57,10 @@ pub type PromptQueue = Arc<
>,
>;
/// Slot for the routine engine, filled at runtime after the agent starts.
pub type RoutineEngineSlot =
Arc<tokio::sync::RwLock<Option<Arc<crate::agent::routine_engine::RoutineEngine>>>>;
/// Simple sliding-window rate limiter.
///
/// Tracks the number of requests in the current window. Resets when the window expires.
@@ -165,6 +169,8 @@ pub struct GatewayState {
pub registry_entries: Vec<crate::extensions::RegistryEntry>,
/// Cost guard for token/cost tracking.
pub cost_guard: Option<Arc<crate::agent::cost_guard::CostGuard>>,
/// Routine engine slot for manual routine triggering (filled at runtime).
pub routine_engine: RoutineEngineSlot,
/// Server startup time for uptime calculation.
pub startup_time: std::time::Instant,
}
@@ -604,6 +610,7 @@ async fn oauth_callback_handler(
async fn chat_send_handler(
State(state): State<Arc<GatewayState>>,
headers: axum::http::HeaderMap,
Json(req): Json<SendMessageRequest>,
) -> Result<(StatusCode, Json<SendMessageResponse>), (StatusCode, String)> {
tracing::debug!(
@@ -620,6 +627,14 @@ async fn chat_send_handler(
}
let mut msg = IncomingMessage::new("gateway", &state.user_id, &req.content);
// Prefer timezone from JSON body, fall back to X-Timezone header
let tz = req
.timezone
.as_deref()
.or_else(|| headers.get("X-Timezone").and_then(|v| v.to_str().ok()));
if let Some(tz) = tz {
msg = msg.with_timezone(tz);
}
if let Some(ref thread_id) = req.thread_id {
msg = msg.with_thread(thread_id);
@@ -1037,7 +1052,7 @@ async fn chat_threads_handler(
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?;
if let Ok(summaries) = store
.list_conversations_with_preview(&state.user_id, "gateway", 50)
.list_conversations_all_channels(&state.user_id, 50)
.await
{
let mut assistant_thread = None;
@@ -1052,6 +1067,7 @@ async fn chat_threads_handler(
updated_at: s.last_activity.to_rfc3339(),
title: s.title.clone(),
thread_type: s.thread_type.clone(),
channel: Some(s.channel.clone()),
};
if s.id == assistant_id {
@@ -1071,6 +1087,7 @@ async fn chat_threads_handler(
updated_at: chrono::Utc::now().to_rfc3339(),
title: None,
thread_type: Some("assistant".to_string()),
channel: Some("gateway".to_string()),
});
}
@@ -1083,9 +1100,10 @@ async fn chat_threads_handler(
}
// Fallback: in-memory only (no assistant thread without DB)
let threads: Vec<ThreadInfo> = sess
.threads
.values()
let mut sorted_threads: Vec<_> = sess.threads.values().collect();
sorted_threads.sort_by(|a, b| b.updated_at.cmp(&a.updated_at));
let threads: Vec<ThreadInfo> = sorted_threads
.into_iter()
.map(|t| ThreadInfo {
id: t.id,
state: format!("{:?}", t.state),
@@ -1094,6 +1112,7 @@ async fn chat_threads_handler(
updated_at: t.updated_at.to_rfc3339(),
title: None,
thread_type: None,
channel: Some("gateway".to_string()),
})
.collect();
@@ -1113,38 +1132,39 @@ async fn chat_new_thread_handler(
))?;
let session = session_manager.get_or_create_session(&state.user_id).await;
let mut sess = session.lock().await;
let thread = sess.create_thread();
let thread_id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
let (thread_id, info) = {
let mut sess = session.lock().await;
let thread = sess.create_thread();
let id = thread.id;
let info = ThreadInfo {
id: thread.id,
state: format!("{:?}", thread.state),
turn_count: thread.turns.len(),
created_at: thread.created_at.to_rfc3339(),
updated_at: thread.updated_at.to_rfc3339(),
title: None,
thread_type: Some("thread".to_string()),
channel: Some("gateway".to_string()),
};
(id, info)
};
// Persist the empty conversation row with thread_type metadata
// Persist the empty conversation row with thread_type metadata synchronously
// so that the subsequent loadThreads() call from the frontend sees it.
if let Some(ref store) = state.store {
let store = Arc::clone(store);
let user_id = state.user_id.clone();
tokio::spawn(async move {
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
});
if let Err(e) = store
.ensure_conversation(thread_id, "gateway", &state.user_id, None)
.await
{
tracing::warn!("Failed to persist new thread: {}", e);
}
let metadata_val = serde_json::json!("thread");
if let Err(e) = store
.update_conversation_metadata_field(thread_id, "thread_type", &metadata_val)
.await
{
tracing::warn!("Failed to set thread_type metadata: {}", e);
}
}
Ok(Json(info))
@@ -1965,47 +1985,35 @@ async fn routines_trigger_handler(
State(state): State<Arc<GatewayState>>,
Path(id): Path<String>,
) -> Result<Json<serde_json::Value>, (StatusCode, String)> {
let store = state.store.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Database not available".to_string(),
))?;
let engine = {
let guard = state.routine_engine.read().await;
guard.as_ref().cloned().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Routine engine not available".to_string(),
))?
};
let routine_id = Uuid::parse_str(&id)
.map_err(|_| (StatusCode::BAD_REQUEST, "Invalid routine ID".to_string()))?;
let routine = store
.get_routine(routine_id)
let run_id = engine
.fire_manual(routine_id, Some(&state.user_id))
.await
.map_err(|e| (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()))?
.ok_or((StatusCode::NOT_FOUND, "Routine not found".to_string()))?;
// Send the routine prompt through the message pipeline as a manual trigger.
let prompt = match &routine.action {
crate::agent::routine::RoutineAction::Lightweight { prompt, .. } => prompt.clone(),
crate::agent::routine::RoutineAction::FullJob {
title, description, ..
} => format!("{}: {}", title, description),
};
let content = format!("[routine:{}] {}", routine.name, prompt);
let msg = IncomingMessage::new("gateway", &state.user_id, content);
let tx_guard = state.msg_tx.read().await;
let tx = tx_guard.as_ref().ok_or((
StatusCode::SERVICE_UNAVAILABLE,
"Channel not started".to_string(),
))?;
tx.send(msg).await.map_err(|_| {
(
StatusCode::INTERNAL_SERVER_ERROR,
"Channel closed".to_string(),
)
})?;
.map_err(|e| {
let status = match &e {
crate::error::RoutineError::NotFound { .. } => StatusCode::NOT_FOUND,
crate::error::RoutineError::NotAuthorized { .. } => StatusCode::FORBIDDEN,
crate::error::RoutineError::Disabled { .. }
| crate::error::RoutineError::MaxConcurrent { .. } => StatusCode::CONFLICT,
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
(status, e.to_string())
})?;
Ok(Json(serde_json::json!({
"status": "triggered",
"routine_id": routine_id,
"run_id": run_id,
})))
}
@@ -2116,7 +2124,7 @@ async fn routines_runs_handler(
/// Convert a Routine to the trimmed RoutineInfo for list display.
fn routine_to_info(r: &crate::agent::routine::Routine) -> RoutineInfo {
let (trigger_type, trigger_summary) = match &r.trigger {
crate::agent::routine::Trigger::Cron { schedule } => {
crate::agent::routine::Trigger::Cron { schedule, .. } => {
("cron".to_string(), format!("cron: {}", schedule))
}
crate::agent::routine::Trigger::Event {
@@ -2463,6 +2471,7 @@ mod tests {
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: vec![],
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
})
}
@@ -2620,7 +2629,9 @@ mod tests {
secrets,
sse_sender: None,
gateway_token: None,
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
};
ext_mgr
@@ -2727,7 +2738,9 @@ mod tests {
sse_sender: None,
gateway_token: None,
// Expired — handler will reject after lookup (no network I/O)
created_at: std::time::Instant::now() - std::time::Duration::from_secs(600),
created_at: std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(600))
.expect("System uptime is too low to run expired flow test"),
};
ext_mgr
+130 -19
View File
@@ -5,6 +5,7 @@ let eventSource = null;
let logEventSource = null;
let currentTab = 'chat';
let currentThreadId = null;
let currentThreadIsReadOnly = false;
let assistantThreadId = null;
let hasMore = false;
let oldestTimestamp = null;
@@ -13,6 +14,8 @@ let sseHasConnectedBefore = false;
let jobEvents = new Map(); // job_id -> Array of events
let jobListRefreshTimer = null;
let pairingPollInterval = null;
let unreadThreads = new Map(); // thread_id -> unread count
let _loadThreadsTimer = null;
const JOB_EVENTS_CAP = 500;
const MEMORY_SEARCH_QUERY_MAX_LENGTH = 100;
@@ -178,6 +181,7 @@ function confirmRestart() {
body: {
content: '/restart',
thread_id: currentThreadId,
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
},
})
.then((response) => {
@@ -273,7 +277,13 @@ function connectSSE() {
eventSource.addEventListener('response', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) {
unreadThreads.set(data.thread_id, (unreadThreads.get(data.thread_id) || 0) + 1);
debouncedLoadThreads();
}
return;
}
finalizeActivityGroup();
addMessage('assistant', data.content);
enableChatInput();
@@ -288,7 +298,10 @@ function connectSSE() {
eventSource.addEventListener('thinking', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) debouncedLoadThreads();
return;
}
showActivityThinking(data.message);
});
@@ -324,7 +337,10 @@ function connectSSE() {
eventSource.addEventListener('status', (e) => {
const data = JSON.parse(e.data);
if (!isCurrentThread(data.thread_id)) return;
if (!isCurrentThread(data.thread_id)) {
if (data.thread_id) debouncedLoadThreads();
return;
}
// "Done" and "Awaiting approval" are terminal signals from the agent:
// the agentic loop finished, so re-enable input as a safety net in case
// the response SSE event is empty or lost.
@@ -414,9 +430,9 @@ function connectSSE() {
}
// Check if an SSE event belongs to the currently viewed thread.
// Events without a thread_id (legacy) are always shown.
// Events without a thread_id are dropped (prevents notification leaking).
function isCurrentThread(threadId) {
if (!threadId) return true;
if (!threadId) return false;
if (!currentThreadId) return true;
return threadId === currentThreadId;
}
@@ -439,14 +455,21 @@ function sendMessage() {
apiFetch('/api/chat/send', {
method: 'POST',
body: { content, thread_id: currentThreadId || undefined },
body: { content, thread_id: currentThreadId || undefined, timezone: Intl.DateTimeFormat().resolvedOptions().timeZone },
}).catch((err) => {
addMessage('system', 'Failed to send: ' + err.message);
});
}
function enableChatInput() {
// no-op: input and send button are always enabled
if (currentThreadIsReadOnly) return;
const input = document.getElementById('chat-input');
const btn = document.getElementById('send-btn');
if (input) {
input.disabled = false;
input.placeholder = 'Message or / for commands...';
}
if (btn) btn.disabled = false;
}
// --- Slash Autocomplete ---
@@ -541,6 +564,13 @@ function sendApprovalAction(requestId, action) {
function renderMarkdown(text) {
if (typeof marked !== 'undefined') {
// Escape raw HTML error pages instead of rendering them as markup.
// Only triggers when the text *starts with* a doctype or <html> tag
// (after optional whitespace), so normal messages that mention HTML
// tags in prose or code fences are not affected. See #263.
if (/^\s*<!doctype\s/i.test(text) || /^\s*<html[\s>]/i.test(text)) {
return escapeHtml(text);
}
let html = marked.parse(text);
// Sanitize HTML output to prevent XSS from tool output or LLM responses.
html = sanitizeRenderedHtml(html);
@@ -1134,7 +1164,9 @@ function loadHistory(before) {
// Fresh load: clear and render
container.innerHTML = '';
for (const turn of data.turns) {
addMessage('user', turn.user_input);
if (turn.user_input) {
addMessage('user', turn.user_input);
}
if (turn.tool_calls && turn.tool_calls.length > 0) {
addToolCallsSummary(turn.tool_calls);
}
@@ -1156,8 +1188,10 @@ function loadHistory(before) {
const savedHeight = container.scrollHeight;
const fragment = document.createDocumentFragment();
for (const turn of data.turns) {
const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv);
if (turn.user_input) {
const userDiv = createMessageElement('user', turn.user_input);
fragment.appendChild(userDiv);
}
if (turn.tool_calls && turn.tool_calls.length > 0) {
fragment.appendChild(createToolCallsSummaryElement(turn.tool_calls));
}
@@ -1256,6 +1290,37 @@ function removeScrollSpinner() {
// --- Threads ---
function threadTitle(thread) {
if (thread.title) return thread.title;
const ch = thread.channel || 'gateway';
if (thread.thread_type === 'heartbeat') return 'Heartbeat Alerts';
if (thread.thread_type === 'routine') return 'Routine';
if (ch !== 'gateway') return ch.charAt(0).toUpperCase() + ch.slice(1);
if (thread.turn_count === 0) return 'New chat';
return thread.id.substring(0, 8);
}
function relativeTime(isoStr) {
if (!isoStr) return '';
const diff = Date.now() - new Date(isoStr).getTime();
const mins = Math.floor(diff / 60000);
if (mins < 1) return 'now';
if (mins < 60) return mins + 'm ago';
const hrs = Math.floor(mins / 60);
if (hrs < 24) return hrs + 'h ago';
const days = Math.floor(hrs / 24);
return days + 'd ago';
}
function isReadOnlyChannel(channel) {
return channel && channel !== 'gateway' && channel !== 'routine' && channel !== 'heartbeat';
}
function debouncedLoadThreads() {
if (_loadThreadsTimer) clearTimeout(_loadThreadsTimer);
_loadThreadsTimer = setTimeout(() => { _loadThreadsTimer = null; loadThreads(); }, 500);
}
function loadThreads() {
apiFetch('/api/chat/threads').then((data) => {
// Pinned assistant thread
@@ -1264,9 +1329,13 @@ function loadThreads() {
const el = document.getElementById('assistant-thread');
const isActive = currentThreadId === assistantThreadId;
el.className = 'assistant-item' + (isActive ? ' active' : '');
const labelEl = document.getElementById('assistant-label');
if (labelEl) {
const at = data.assistant_thread;
labelEl.textContent = 'Assistant';
}
const meta = document.getElementById('assistant-meta');
const count = data.assistant_thread.turn_count || 0;
meta.textContent = count > 0 ? count + ' turns' : '';
meta.textContent = relativeTime(data.assistant_thread.updated_at);
}
// Regular threads
@@ -1275,16 +1344,38 @@ function loadThreads() {
const threads = data.threads || [];
for (const thread of threads) {
const item = document.createElement('div');
item.className = 'thread-item' + (thread.id === currentThreadId ? ' active' : '');
const isActive = thread.id === currentThreadId;
item.className = 'thread-item' + (isActive ? ' active' : '');
// Channel badge for non-gateway threads
const ch = thread.channel || 'gateway';
if (ch !== 'gateway') {
const badge = document.createElement('span');
badge.className = 'thread-badge thread-badge-' + ch;
badge.textContent = ch;
item.appendChild(badge);
}
const label = document.createElement('span');
label.className = 'thread-label';
label.textContent = thread.title || thread.id.substring(0, 8);
label.title = thread.title ? thread.title + ' (' + thread.id + ')' : thread.id;
label.textContent = threadTitle(thread);
label.title = (thread.title || '') + ' (' + thread.id + ')';
item.appendChild(label);
const meta = document.createElement('span');
meta.className = 'thread-meta';
meta.textContent = (thread.turn_count || 0) + ' turns';
meta.textContent = relativeTime(thread.updated_at);
item.appendChild(meta);
// Unread dot
const unread = unreadThreads.get(thread.id) || 0;
if (unread > 0 && !isActive) {
const dot = document.createElement('span');
dot.className = 'thread-unread';
dot.textContent = unread > 9 ? '9+' : String(unread);
item.appendChild(dot);
}
item.addEventListener('click', () => switchThread(thread.id));
list.appendChild(item);
}
@@ -1294,17 +1385,36 @@ function loadThreads() {
switchToAssistant();
}
// Enable chat input once a thread is available
// Enable/disable chat input based on channel type
if (currentThreadId) {
enableChatInput();
const currentThread = threads.find(t => t.id === currentThreadId);
const ch = currentThread ? currentThread.channel : 'gateway';
currentThreadIsReadOnly = isReadOnlyChannel(ch);
if (currentThreadIsReadOnly) {
disableChatInputReadOnly();
} else {
enableChatInput();
}
}
}).catch(() => {});
}
function disableChatInputReadOnly() {
const input = document.getElementById('chat-input');
const btn = document.getElementById('send-btn');
if (input) {
input.disabled = true;
input.placeholder = 'Read-only thread (external channel)';
}
if (btn) btn.disabled = true;
}
function switchToAssistant() {
if (!assistantThreadId) return;
finalizeActivityGroup();
currentThreadId = assistantThreadId;
currentThreadIsReadOnly = false;
unreadThreads.delete(assistantThreadId);
hasMore = false;
oldestTimestamp = null;
loadHistory();
@@ -1314,6 +1424,7 @@ function switchToAssistant() {
function switchThread(threadId) {
finalizeActivityGroup();
currentThreadId = threadId;
unreadThreads.delete(threadId);
hasMore = false;
oldestTimestamp = null;
loadHistory();
@@ -1370,7 +1481,7 @@ chatInput.addEventListener('keydown', (e) => {
}
}
if (e.key === 'Enter' && !e.shiftKey) {
if (e.key === 'Enter' && !e.shiftKey && !e.isComposing) {
e.preventDefault();
hideSlashAutocomplete();
sendMessage();
+2 -2
View File
@@ -113,12 +113,12 @@
<div class="tab-panel active" id="tab-chat">
<div class="thread-sidebar" id="thread-sidebar">
<div class="thread-sidebar-header">
<span>Threads</span>
<button class="thread-new-btn" onclick="createNewThread()" title="New thread (Ctrl/Cmd+N)">+</button>
<div class="spacer"></div>
<button class="thread-toggle-btn" id="thread-toggle-btn" onclick="toggleThreadSidebar()" title="Toggle sidebar">&laquo;</button>
</div>
<div class="assistant-item" id="assistant-thread" onclick="switchToAssistant()">
<span class="assistant-label">Assistant</span>
<span class="assistant-label" id="assistant-label">Assistant</span>
<span class="assistant-meta" id="assistant-meta"></span>
</div>
<div class="threads-section-header">
+51 -12
View File
@@ -3074,7 +3074,7 @@ mark {
}
.thread-sidebar {
width: 200px;
width: 240px;
background: var(--bg-secondary);
border-right: 1px solid var(--border);
display: flex;
@@ -3082,6 +3082,8 @@ mark {
flex-shrink: 0;
transition: width 0.2s ease;
overflow: hidden;
padding: 6px;
gap: 2px;
}
.thread-sidebar.collapsed {
@@ -3099,8 +3101,7 @@ mark {
.thread-sidebar-header {
display: flex;
align-items: center;
padding: 10px 12px;
border-bottom: 1px solid var(--border);
padding: 10px 10px;
font-size: 13px;
font-weight: 600;
gap: 8px;
@@ -3134,21 +3135,22 @@ mark {
display: flex;
align-items: center;
justify-content: space-between;
padding: 10px 12px;
padding: 12px 14px;
cursor: pointer;
font-size: 13px;
font-weight: 600;
color: var(--text);
border-bottom: 1px solid var(--border);
background: var(--bg-secondary);
background: var(--bg-tertiary);
border-radius: var(--radius);
margin-bottom: 2px;
}
.assistant-item:hover {
background: var(--bg-tertiary);
background: rgba(255, 255, 255, 0.06);
}
.assistant-item.active {
background: rgba(52, 211, 153, 0.08);
background: rgba(52, 211, 153, 0.1);
color: var(--accent);
border-left: 2px solid var(--accent);
}
@@ -3166,7 +3168,7 @@ mark {
}
.threads-section-header {
padding: 8px 12px 4px;
padding: 10px 10px 4px;
font-size: 11px;
font-weight: 500;
text-transform: uppercase;
@@ -3196,11 +3198,11 @@ mark {
display: flex;
align-items: center;
justify-content: space-between;
padding: 8px 12px;
padding: 10px 14px;
cursor: pointer;
font-size: 13px;
color: var(--text-secondary);
border-bottom: 1px solid rgba(255, 255, 255, 0.03);
border-radius: var(--radius);
}
.thread-item:hover {
@@ -3222,6 +3224,43 @@ mark {
.thread-meta {
font-size: 11px;
color: var(--text-secondary);
flex-shrink: 0;
}
.thread-badge {
display: inline-block;
font-size: 9px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.5px;
padding: 1px 5px;
border-radius: 3px;
background: rgba(255, 255, 255, 0.08);
color: var(--text-secondary);
margin-right: 6px;
flex-shrink: 0;
}
.thread-badge-routine { background: rgba(52, 211, 153, 0.15); color: var(--accent); }
.thread-badge-heartbeat { background: rgba(245, 166, 35, 0.15); color: var(--warning); }
.thread-badge-telegram { background: rgba(0, 136, 204, 0.15); color: #0088cc; }
.thread-badge-signal { background: rgba(59, 118, 240, 0.15); color: #3b76f0; }
.thread-badge-slack { background: rgba(74, 21, 75, 0.15); color: #e01e5a; }
.thread-unread {
display: inline-flex;
align-items: center;
justify-content: center;
min-width: 16px;
height: 16px;
font-size: 10px;
font-weight: 700;
background: var(--accent);
color: var(--bg);
border-radius: 8px;
padding: 0 4px;
margin-left: auto;
flex-shrink: 0;
}
/* --- Memory editing --- */
@@ -3620,7 +3659,7 @@ mark {
left: 0;
top: 0;
bottom: 0;
width: 200px;
width: 240px;
z-index: 50;
}
+1
View File
@@ -84,6 +84,7 @@ impl TestGatewayBuilder {
chat_rate_limiter: RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
})
}
+46 -2
View File
@@ -9,6 +9,7 @@ use uuid::Uuid;
pub struct SendMessageRequest {
pub content: String,
pub thread_id: Option<String>,
pub timezone: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -28,6 +29,8 @@ pub struct ThreadInfo {
pub title: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub thread_type: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub channel: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -611,6 +614,7 @@ pub enum WsClientMessage {
Message {
content: String,
thread_id: Option<String>,
timezone: Option<String>,
},
/// Approve or deny a pending tool execution.
#[serde(rename = "approval")]
@@ -796,7 +800,9 @@ mod tests {
let json = r#"{"type":"message","content":"hello","thread_id":"t1"}"#;
let msg: WsClientMessage = serde_json::from_str(json).unwrap();
match msg {
WsClientMessage::Message { content, thread_id } => {
WsClientMessage::Message {
content, thread_id, ..
} => {
assert_eq!(content, "hello");
assert_eq!(thread_id.as_deref(), Some("t1"));
}
@@ -809,7 +815,9 @@ mod tests {
let json = r#"{"type":"message","content":"hi"}"#;
let msg: WsClientMessage = serde_json::from_str(json).unwrap();
match msg {
WsClientMessage::Message { content, thread_id } => {
WsClientMessage::Message {
content, thread_id, ..
} => {
assert_eq!(content, "hi");
assert!(thread_id.is_none());
}
@@ -1063,4 +1071,40 @@ mod tests {
let req: AuthCancelRequest = serde_json::from_str(json).unwrap();
assert_eq!(req.extension_name, "telegram");
}
// ---- ThreadInfo channel field tests ----
#[test]
fn test_thread_info_channel_serialized() {
let info = ThreadInfo {
id: Uuid::nil(),
state: "Idle".to_string(),
turn_count: 0,
created_at: "2026-01-01T00:00:00Z".to_string(),
updated_at: "2026-01-01T00:00:00Z".to_string(),
title: None,
thread_type: None,
channel: Some("telegram".to_string()),
};
let json = serde_json::to_string(&info).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["channel"], "telegram");
}
#[test]
fn test_thread_info_channel_omitted_when_none() {
let info = ThreadInfo {
id: Uuid::nil(),
state: "Idle".to_string(),
turn_count: 0,
created_at: "2026-01-01T00:00:00Z".to_string(),
updated_at: "2026-01-01T00:00:00Z".to_string(),
title: None,
thread_type: None,
channel: None,
};
let json = serde_json::to_string(&info).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert!(parsed.get("channel").is_none());
}
}
+36
View File
@@ -83,6 +83,19 @@ pub fn build_turns_from_db_messages(
turns.push(turn);
turn_number += 1;
} else if msg.role == "assistant" {
// Standalone assistant message (e.g. routine output, heartbeat)
// with no preceding user message — render as a turn with empty input.
turns.push(TurnInfo {
turn_number,
user_input: String::new(),
response: Some(msg.content.clone()),
state: "Completed".to_string(),
started_at: msg.created_at.to_rfc3339(),
completed_at: Some(msg.created_at.to_rfc3339()),
tool_calls: Vec::new(),
});
turn_number += 1;
}
}
@@ -220,6 +233,29 @@ mod tests {
assert_eq!(turns[0].response.as_deref(), Some("Done"));
}
#[test]
fn test_build_turns_standalone_assistant_messages() {
// Routine conversations only have assistant messages (no user messages).
let messages = vec![
make_msg("assistant", "Routine executed: all checks passed", 0),
make_msg("assistant", "Routine executed: found 2 issues", 5000),
];
let turns = build_turns_from_db_messages(&messages);
assert_eq!(turns.len(), 2);
// Standalone assistant messages should have empty user_input
assert_eq!(turns[0].user_input, "");
assert_eq!(
turns[0].response.as_deref(),
Some("Routine executed: all checks passed")
);
assert_eq!(turns[0].state, "Completed");
assert_eq!(turns[1].user_input, "");
assert_eq!(
turns[1].response.as_deref(),
Some("Routine executed: found 2 issues")
);
}
#[test]
fn test_build_turns_backward_compatible() {
let messages = vec![
+11 -1
View File
@@ -156,8 +156,15 @@ async fn handle_client_message(
direct_tx: &mpsc::Sender<WsServerMessage>,
) {
match msg {
WsClientMessage::Message { content, thread_id } => {
WsClientMessage::Message {
content,
thread_id,
timezone,
} => {
let mut incoming = IncomingMessage::new("gateway", user_id, &content);
if let Some(ref tz) = timezone {
incoming = incoming.with_timezone(tz);
}
if let Some(ref tid) = thread_id {
incoming = incoming.with_thread(tid);
}
@@ -349,6 +356,7 @@ mod tests {
WsClientMessage::Message {
content: "hello agent".to_string(),
thread_id: Some("t1".to_string()),
timezone: None,
},
&state,
"user1",
@@ -373,6 +381,7 @@ mod tests {
WsClientMessage::Message {
content: "hello".to_string(),
thread_id: None,
timezone: None,
},
&state,
"user1",
@@ -493,6 +502,7 @@ mod tests {
chat_rate_limiter: crate::channels::web::server::RateLimiter::new(30, 60),
registry_entries: Vec::new(),
cost_guard: None,
routine_engine: Arc::new(tokio::sync::RwLock::new(None)),
startup_time: std::time::Instant::now(),
}
}
+123 -1
View File
@@ -8,9 +8,35 @@ use std::path::PathBuf;
use crate::bootstrap::ironclaw_base_dir;
use crate::settings::Settings;
/// Load settings from JSON and TOML config files, matching the runtime
/// priority: TOML overlay > settings.json > defaults.
///
/// This mirrors the loading chain in `Config::from_env_with_toml()` but
/// without resolving the full `Config` (which requires async + secrets).
fn load_settings() -> Settings {
load_settings_from(&Settings::default_path(), &Settings::default_toml_path())
}
/// Inner implementation with injectable paths (testable).
fn load_settings_from(json_path: &std::path::Path, toml_path: &std::path::Path) -> Settings {
let mut settings = Settings::load_from(json_path);
match Settings::load_toml(toml_path) {
Ok(Some(toml_settings)) => {
settings.merge_from(&toml_settings);
}
Ok(None) => {} // File not found — fine for default path
Err(e) => {
eprintln!("Warning: failed to parse {}: {}", toml_path.display(), e);
}
}
settings
}
/// Run the status command, printing system health info.
pub async fn run_status_command() -> anyhow::Result<()> {
let settings = Settings::default();
let settings = load_settings();
println!("IronClaw Status");
println!("===============\n");
@@ -209,3 +235,99 @@ fn default_tools_dir() -> PathBuf {
fn default_channels_dir() -> PathBuf {
ironclaw_base_dir().join("channels")
}
#[cfg(test)]
mod tests {
use super::load_settings_from;
/// Regression test for #354: load_settings_from must read config.toml.
#[test]
fn reads_toml_heartbeat_enabled() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
// No JSON file — only TOML
std::fs::write(
&toml_path,
"[heartbeat]\nenabled = true\ninterval_secs = 600",
)
.expect("write toml");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 600);
}
/// Without any config files, defaults are returned.
#[test]
fn defaults_without_config_files() {
let dir = tempfile::tempdir().expect("tempdir");
let settings = load_settings_from(
&dir.path().join("nonexistent.json"),
&dir.path().join("nonexistent.toml"),
);
assert!(!settings.heartbeat.enabled);
}
/// settings.json is respected.
#[test]
fn reads_json_heartbeat_enabled() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("nonexistent.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":true,"interval_secs":900}}"#,
)
.expect("write json");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 900);
}
/// TOML overlay wins over JSON settings.
#[test]
fn toml_overlay_wins_over_json() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":false,"interval_secs":100}}"#,
)
.expect("write json");
std::fs::write(
&toml_path,
"[heartbeat]\nenabled = true\ninterval_secs = 200",
)
.expect("write toml");
let settings = load_settings_from(&json_path, &toml_path);
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 200);
}
/// Invalid TOML is warned but doesn't crash; falls back to JSON/defaults.
#[test]
fn invalid_toml_falls_back_gracefully() {
let dir = tempfile::tempdir().expect("tempdir");
let json_path = dir.path().join("settings.json");
let toml_path = dir.path().join("config.toml");
std::fs::write(
&json_path,
r#"{"heartbeat":{"enabled":true,"interval_secs":500}}"#,
)
.expect("write json");
std::fs::write(&toml_path, "this is not valid toml [[[").expect("write bad toml");
let settings = load_settings_from(&json_path, &toml_path);
// Should fall back to JSON values, not crash
assert!(settings.heartbeat.enabled);
assert_eq!(settings.heartbeat.interval_secs, 500);
}
}
+37
View File
@@ -27,6 +27,8 @@ pub struct AgentConfig {
pub max_tool_iterations: usize,
/// When true, skip tool approval checks entirely. For benchmarks/CI.
pub auto_approve_tools: bool,
/// Default timezone for new sessions (IANA name, e.g. "America/New_York").
pub default_timezone: String,
}
impl AgentConfig {
@@ -47,6 +49,7 @@ impl AgentConfig {
max_actions_per_hour: None,
max_tool_iterations: 10,
auto_approve_tools: true,
default_timezone: "UTC".to_string(),
}
}
@@ -89,6 +92,40 @@ impl AgentConfig {
"AGENT_AUTO_APPROVE_TOOLS",
settings.agent.auto_approve_tools,
)?,
default_timezone: {
let tz: String = parse_optional_env(
"DEFAULT_TIMEZONE",
settings.agent.default_timezone.clone(),
)?;
if crate::timezone::parse_timezone(&tz).is_none() {
return Err(ConfigError::InvalidValue {
key: "DEFAULT_TIMEZONE".into(),
message: format!("invalid IANA timezone: '{tz}'"),
});
}
tz
},
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_default_timezone_rejects_invalid() {
let mut settings = Settings::default();
settings.agent.default_timezone = "Fake/Zone".to_string();
let result = AgentConfig::resolve(&settings);
assert!(result.is_err(), "invalid IANA timezone should be rejected");
}
#[test]
fn test_default_timezone_accepts_valid() {
let settings = Settings::default(); // default is "UTC"
let config = AgentConfig::resolve(&settings).expect("resolve");
assert_eq!(config.default_timezone, "UTC");
}
}
+92
View File
@@ -82,6 +82,31 @@ impl std::str::FromStr for SslMode {
}
}
/// Which vector store backend to use for workspace semantic search.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum VectorBackend {
/// Use the database's built-in vector support (pgvector or libsql_vector_idx).
#[default]
Builtin,
/// Use LanceDB as an external vector store (requires `lancedb` feature).
LanceDb,
}
impl std::str::FromStr for VectorBackend {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.to_lowercase().as_str() {
"" | "builtin" | "pgvector" | "libsql" => Ok(Self::Builtin),
"lancedb" | "lance" => Ok(Self::LanceDb),
_ => Err(format!(
"invalid vector backend '{}', expected 'builtin' or 'lancedb'",
s
)),
}
}
}
/// Database configuration.
#[derive(Debug, Clone)]
pub struct DatabaseConfig {
@@ -101,6 +126,12 @@ pub struct DatabaseConfig {
pub libsql_url: Option<String>,
/// Turso auth token (required when libsql_url is set).
pub libsql_auth_token: Option<SecretString>,
// -- Vector store fields --
/// Which vector store to use for workspace semantic search (default: Builtin).
pub vector_backend: VectorBackend,
/// Path to LanceDB directory (default: ~/.ironclaw/lancedb when vector_backend is LanceDb).
pub lancedb_path: Option<PathBuf>,
}
impl DatabaseConfig {
@@ -159,6 +190,25 @@ impl DatabaseConfig {
});
}
let vector_backend: VectorBackend = if let Some(s) = optional_env("VECTOR_BACKEND")? {
s.parse().map_err(|e| ConfigError::InvalidValue {
key: "VECTOR_BACKEND".to_string(),
message: e,
})?
} else {
VectorBackend::default()
};
let lancedb_path = optional_env("LANCEDB_PATH")?
.map(PathBuf::from)
.or_else(|| {
if vector_backend == VectorBackend::LanceDb {
Some(default_lancedb_path())
} else {
None
}
});
Ok(Self {
backend,
url: SecretString::from(url),
@@ -167,6 +217,8 @@ impl DatabaseConfig {
libsql_path,
libsql_url,
libsql_auth_token,
vector_backend,
lancedb_path,
})
}
@@ -195,6 +247,11 @@ pub fn default_libsql_path() -> PathBuf {
ironclaw_base_dir().join("ironclaw.db")
}
/// Default LanceDB directory (~/.ironclaw/lancedb).
pub fn default_lancedb_path() -> PathBuf {
ironclaw_base_dir().join("lancedb")
}
#[cfg(test)]
mod tests {
use super::*;
@@ -224,4 +281,39 @@ mod tests {
fn ssl_mode_parse_invalid() {
assert!("invalid".parse::<SslMode>().is_err());
}
#[test]
fn vector_backend_parse() {
assert_eq!(
"builtin".parse::<VectorBackend>().unwrap(),
VectorBackend::Builtin
);
assert_eq!(
"pgvector".parse::<VectorBackend>().unwrap(),
VectorBackend::Builtin
);
assert_eq!(
"libsql".parse::<VectorBackend>().unwrap(),
VectorBackend::Builtin
);
assert_eq!("".parse::<VectorBackend>().unwrap(), VectorBackend::Builtin);
assert_eq!(
"lancedb".parse::<VectorBackend>().unwrap(),
VectorBackend::LanceDb
);
assert_eq!(
"lance".parse::<VectorBackend>().unwrap(),
VectorBackend::LanceDb
);
assert!("invalid".parse::<VectorBackend>().is_err());
}
#[test]
fn default_lancedb_path_under_ironclaw() {
let path = super::default_lancedb_path();
assert!(path.to_string_lossy().contains("ironclaw"));
assert!(path.to_string_lossy().ends_with("lancedb"));
}
}
+102 -1
View File
@@ -1,4 +1,4 @@
use crate::config::helpers::{optional_env, parse_bool_env, parse_optional_env};
use crate::config::helpers::{optional_env, parse_bool_env, parse_option_env, parse_optional_env};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -13,6 +13,12 @@ pub struct HeartbeatConfig {
pub notify_channel: Option<String>,
/// User ID to notify on heartbeat findings.
pub notify_user: Option<String>,
/// Hour (0-23) when quiet hours start.
pub quiet_hours_start: Option<u32>,
/// Hour (0-23) when quiet hours end.
pub quiet_hours_end: Option<u32>,
/// Timezone for quiet hours evaluation (IANA name).
pub timezone: Option<String>,
}
impl Default for HeartbeatConfig {
@@ -22,6 +28,9 @@ impl Default for HeartbeatConfig {
interval_secs: 1800, // 30 minutes
notify_channel: None,
notify_user: None,
quiet_hours_start: None,
quiet_hours_end: None,
timezone: None,
}
}
}
@@ -38,6 +47,98 @@ impl HeartbeatConfig {
.or_else(|| settings.heartbeat.notify_channel.clone()),
notify_user: optional_env("HEARTBEAT_NOTIFY_USER")?
.or_else(|| settings.heartbeat.notify_user.clone()),
quiet_hours_start: parse_option_env::<u32>("HEARTBEAT_QUIET_START")?
.or(settings.heartbeat.quiet_hours_start)
.map(|h| {
if h > 23 {
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_QUIET_START".into(),
message: "must be 0-23".into(),
});
}
Ok(h)
})
.transpose()?,
quiet_hours_end: parse_option_env::<u32>("HEARTBEAT_QUIET_END")?
.or(settings.heartbeat.quiet_hours_end)
.map(|h| {
if h > 23 {
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_QUIET_END".into(),
message: "must be 0-23".into(),
});
}
Ok(h)
})
.transpose()?,
timezone: {
let tz = optional_env("HEARTBEAT_TIMEZONE")?
.or_else(|| settings.heartbeat.timezone.clone());
if let Some(ref tz_str) = tz
&& crate::timezone::parse_timezone(tz_str).is_none()
{
return Err(ConfigError::InvalidValue {
key: "HEARTBEAT_TIMEZONE".into(),
message: format!("invalid IANA timezone: '{tz_str}'"),
});
}
tz
},
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_quiet_hours_settings_fallback() {
// When env vars are not set, settings values should be used
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(22);
settings.heartbeat.quiet_hours_end = Some(6);
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.quiet_hours_start, Some(22));
assert_eq!(config.quiet_hours_end, Some(6));
}
#[test]
fn test_quiet_hours_rejects_invalid_hour() {
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(24);
let result = HeartbeatConfig::resolve(&settings);
assert!(result.is_err());
}
#[test]
fn test_quiet_hours_accepts_boundary_values() {
let mut settings = Settings::default();
settings.heartbeat.quiet_hours_start = Some(0);
settings.heartbeat.quiet_hours_end = Some(23);
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.quiet_hours_start, Some(0));
assert_eq!(config.quiet_hours_end, Some(23));
}
#[test]
fn test_heartbeat_timezone_rejects_invalid() {
let mut settings = Settings::default();
settings.heartbeat.timezone = Some("Fake/Zone".to_string());
let result = HeartbeatConfig::resolve(&settings);
assert!(result.is_err(), "invalid IANA timezone should be rejected");
}
#[test]
fn test_heartbeat_timezone_accepts_valid() {
let mut settings = Settings::default();
settings.heartbeat.timezone = Some("America/New_York".to_string());
let config = HeartbeatConfig::resolve(&settings).expect("resolve");
assert_eq!(config.timezone.as_deref(), Some("America/New_York"));
}
}
+7 -2
View File
@@ -25,8 +25,13 @@ pub(crate) fn optional_env(key: &str) -> Result<Option<String>, ConfigError> {
}
// Fall back to thread-safe overlay (secrets injected from DB)
if let Some(val) = INJECTED_VARS.get().and_then(|map| map.get(key)) {
return Ok(Some(val.clone()));
if let Some(val) = INJECTED_VARS
.lock()
.unwrap_or_else(|p| p.into_inner())
.get(key)
.cloned()
{
return Ok(Some(val));
}
Ok(None)
+175 -6
View File
@@ -9,6 +9,13 @@ use crate::llm::registry::{ProviderProtocol, ProviderRegistry};
use crate::llm::session::SessionConfig;
use crate::settings::Settings;
/// Sentinel value used as `api_key` when only an OAuth token is present.
///
/// When we only have an OAuth token the provider factory in `llm/mod.rs`
/// checks for this value and routes to `AnthropicOAuthProvider`, so this
/// placeholder is never sent over the wire.
pub const OAUTH_PLACEHOLDER: &str = "oauth-placeholder";
/// Prompt cache retention policy for Anthropic.
///
/// Controls Anthropic's automatic prompt caching via a top-level
@@ -66,6 +73,7 @@ pub struct RegistryProviderConfig {
/// Provider identifier (e.g., "groq", "openai", "tinfoil").
pub provider_id: String,
/// API key (optional for some providers like Ollama).
/// For Anthropic OAuth, this is set to `OAUTH_PLACEHOLDER`.
pub api_key: Option<SecretString>,
/// Base URL for the API endpoint.
pub base_url: String,
@@ -73,6 +81,9 @@ pub struct RegistryProviderConfig {
pub model: String,
/// Extra HTTP headers injected into every request.
pub extra_headers: Vec<(String, String)>,
/// OAuth token for providers that support Bearer auth (e.g. Anthropic via `claude login`).
/// When set, the provider factory routes to the OAuth-specific provider implementation.
pub oauth_token: Option<SecretString>,
}
/// LLM provider configuration.
@@ -92,6 +103,10 @@ pub struct LlmConfig {
/// Resolved provider config for registry-based providers.
/// `None` when backend is "nearai".
pub provider: Option<RegistryProviderConfig>,
/// HTTP request timeout in seconds for LLM API calls.
/// Default: 120. Increase for local LLMs (Ollama, vLLM, LM Studio) that
/// need more time for prompt evaluation on consumer hardware.
pub request_timeout_secs: u64,
}
/// NEAR AI configuration.
@@ -154,6 +169,7 @@ impl LlmConfig {
smart_routing_cascade: false,
},
provider: None,
request_timeout_secs: 120,
}
}
@@ -243,6 +259,8 @@ impl LlmConfig {
)?)
};
let request_timeout_secs = parse_optional_env("LLM_REQUEST_TIMEOUT_SECS", 120)?;
Ok(Self {
backend: if is_nearai {
"nearai".to_string()
@@ -254,6 +272,7 @@ impl LlmConfig {
session,
nearai,
provider,
request_timeout_secs,
})
}
@@ -366,6 +385,22 @@ impl LlmConfig {
Vec::new()
};
// Resolve OAuth token (Anthropic-specific: `claude login` flow).
// Only check for OAuth token when the provider is actually Anthropic.
let oauth_token = if canonical_id == "anthropic" {
optional_env("ANTHROPIC_OAUTH_TOKEN")?.map(SecretString::from)
} else {
None
};
let api_key = if api_key.is_none() && oauth_token.is_some() {
// OAuth token present but no API key: use a placeholder so the
// config block is populated. The provider factory will route to
// the OAuth provider instead of rig-core's x-api-key client.
Some(SecretString::from(OAUTH_PLACEHOLDER.to_string()))
} else {
api_key
};
Ok(RegistryProviderConfig {
protocol,
provider_id: canonical_id.to_string(),
@@ -373,6 +408,7 @@ impl LlmConfig {
base_url,
model,
extra_headers,
oauth_token,
})
}
}
@@ -677,8 +713,6 @@ mod tests {
#[test]
fn backend_alias_normalized_to_canonical_id() {
// When the user sets LLM_BACKEND to an alias (e.g., "open_ai"),
// LlmConfig.backend should resolve to the canonical ID ("openai").
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
// SAFETY: Under ENV_MUTEX.
@@ -705,8 +739,6 @@ mod tests {
#[test]
fn unknown_backend_falls_back_to_openai_compatible() {
// An unrecognized LLM_BACKEND should fall back to the openai_compatible
// provider definition instead of erroring.
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
// SAFETY: Under ENV_MUTEX.
@@ -717,7 +749,6 @@ mod tests {
let settings = Settings::default();
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
// Falls back to openai_compatible since "some_custom_provider" is unknown
assert_eq!(cfg.backend, "openai_compatible");
let provider = cfg.provider.expect("should have provider config");
assert_eq!(provider.provider_id, "openai_compatible");
@@ -759,7 +790,6 @@ mod tests {
#[test]
fn base_url_resolution_priority() {
// Env var > settings > registry default
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_openai_compatible_env();
@@ -800,6 +830,119 @@ mod tests {
}
}
// ── OAuth resolution tests ──────────────────────────────────────
/// Clear all Anthropic-related env vars.
fn clear_anthropic_env() {
// SAFETY: Only called under ENV_MUTEX in tests.
unsafe {
std::env::remove_var("LLM_BACKEND");
std::env::remove_var("ANTHROPIC_API_KEY");
std::env::remove_var("ANTHROPIC_OAUTH_TOKEN");
std::env::remove_var("ANTHROPIC_MODEL");
std::env::remove_var("ANTHROPIC_BASE_URL");
}
}
#[test]
fn anthropic_oauth_token_sets_placeholder_api_key() {
use secrecy::ExposeSecret;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("anthropic".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert_eq!(
provider
.api_key
.as_ref()
.map(|k| k.expose_secret().to_string()),
Some(OAUTH_PLACEHOLDER.to_string()),
"api_key should be the OAuth placeholder when only OAuth token is set"
);
assert!(
provider.oauth_token.is_some(),
"oauth_token should be populated"
);
assert_eq!(
provider.oauth_token.as_ref().unwrap().expose_secret(),
"sk-ant-oat01-test-token"
);
clear_anthropic_env();
}
#[test]
fn anthropic_api_key_takes_priority_over_oauth() {
use secrecy::ExposeSecret;
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_API_KEY", "sk-ant-real-key");
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("anthropic".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert_eq!(
provider
.api_key
.as_ref()
.map(|k| k.expose_secret().to_string()),
Some("sk-ant-real-key".to_string()),
"real API key should take priority over OAuth placeholder"
);
assert!(
provider.oauth_token.is_some(),
"oauth_token should still be populated"
);
clear_anthropic_env();
}
#[test]
fn non_anthropic_provider_has_no_oauth_token() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
clear_anthropic_env();
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("ANTHROPIC_OAUTH_TOKEN", "sk-ant-oat01-test-token");
}
let settings = Settings {
llm_backend: Some("openai".to_string()),
..Default::default()
};
let cfg = LlmConfig::resolve(&settings).expect("resolve should succeed");
let provider = cfg.provider.expect("provider config should be present");
assert!(
provider.oauth_token.is_none(),
"non-Anthropic providers should not pick up ANTHROPIC_OAUTH_TOKEN"
);
clear_anthropic_env();
}
// ── Cache retention tests ───────────────────────────────────────
#[test]
fn cache_retention_from_str_primary_values() {
assert_eq!(
@@ -881,4 +1024,30 @@ mod tests {
assert_eq!(parsed, variant, "round-trip failed for {s}");
}
}
#[test]
fn test_request_timeout_defaults_to_120() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::remove_var("LLM_REQUEST_TIMEOUT_SECS");
}
let config = LlmConfig::resolve(&Settings::default()).expect("resolve");
assert_eq!(config.request_timeout_secs, 120);
}
#[test]
fn test_request_timeout_configurable() {
let _guard = ENV_MUTEX.lock().expect("env mutex poisoned");
// SAFETY: Under ENV_MUTEX.
unsafe {
std::env::set_var("LLM_REQUEST_TIMEOUT_SECS", "300");
}
let config = LlmConfig::resolve(&Settings::default()).expect("resolve");
assert_eq!(config.request_timeout_secs, 300);
// SAFETY: Cleanup
unsafe {
std::env::remove_var("LLM_REQUEST_TIMEOUT_SECS");
}
}
}
+105 -6
View File
@@ -13,7 +13,7 @@ mod embeddings;
mod heartbeat;
pub(crate) mod helpers;
mod hygiene;
mod llm;
pub(crate) mod llm;
mod routines;
mod safety;
mod sandbox;
@@ -24,7 +24,7 @@ mod tunnel;
mod wasm;
use std::collections::HashMap;
use std::sync::OnceLock;
use std::sync::{LazyLock, Mutex};
use crate::error::ConfigError;
use crate::settings::Settings;
@@ -33,7 +33,10 @@ use crate::settings::Settings;
pub use self::agent::AgentConfig;
pub use self::builder::BuilderModeConfig;
pub use self::channels::{ChannelsConfig, CliConfig, GatewayConfig, HttpConfig, SignalConfig};
pub use self::database::{DatabaseBackend, DatabaseConfig, SslMode, default_libsql_path};
pub use self::database::{
DatabaseBackend, DatabaseConfig, SslMode, VectorBackend, default_lancedb_path,
default_libsql_path,
};
pub use self::embeddings::EmbeddingsConfig;
pub use self::heartbeat::HeartbeatConfig;
pub use self::hygiene::HygieneConfig;
@@ -53,7 +56,12 @@ pub use crate::llm::session::SessionConfig;
/// Used by `inject_llm_keys_from_secrets()` to make API keys available to
/// `optional_env()` without unsafe `set_var` calls. `optional_env()` checks
/// real env vars first, then falls back to this overlay.
static INJECTED_VARS: OnceLock<HashMap<String, String>> = OnceLock::new();
///
/// Uses `Mutex<HashMap>` instead of `OnceLock` so that both
/// `inject_os_credentials()` and `inject_llm_keys_from_secrets()` can merge
/// their data. Whichever runs first initialises the map; the second merges in.
static INJECTED_VARS: LazyLock<Mutex<HashMap<String, String>>> =
LazyLock::new(|| Mutex::new(HashMap::new()));
/// Main configuration for the agent.
#[derive(Debug, Clone)]
@@ -102,6 +110,8 @@ impl Config {
libsql_path: Some(libsql_path),
libsql_url: None,
libsql_auth_token: None,
vector_backend: VectorBackend::default(),
lancedb_path: None,
},
llm: LlmConfig::for_testing(),
embeddings: EmbeddingsConfig::default(),
@@ -252,6 +262,32 @@ impl Config {
Ok(())
}
/// Re-resolve only the LLM config after credential injection.
///
/// Called by `AppBuilder::init_secrets()` after injecting API keys into
/// the env overlay. Only rebuilds `self.llm` — all other config fields
/// are unaffected, preserving values from the initial config load (or
/// from `Config::for_testing()` in test mode).
pub async fn re_resolve_llm(
&mut self,
store: Option<&(dyn crate::db::SettingsStore + Sync)>,
user_id: &str,
toml_path: Option<&std::path::Path>,
) -> Result<(), ConfigError> {
let settings = if let Some(store) = store {
let mut s = match store.get_all_settings(user_id).await {
Ok(map) => Settings::from_db_map(&map),
Err(_) => Settings::default(),
};
Self::apply_toml_overlay(&mut s, toml_path)?;
s
} else {
Settings::default()
};
self.llm = LlmConfig::resolve(&settings)?;
Ok(())
}
/// Build config from settings (shared by from_env and from_db).
async fn build(settings: &Settings) -> Result<Self, ConfigError> {
Ok(Self {
@@ -285,6 +321,9 @@ impl Config {
/// env-var-first resolution in `LlmConfig::resolve()`. Keys in the overlay
/// are read by `optional_env()` before falling back to `std::env::var()`,
/// so explicit env vars always win.
///
/// Also loads tokens from OS credential stores (macOS Keychain, Linux
/// credentials files) which don't require the secrets DB.
pub async fn inject_llm_keys_from_secrets(
secrets: &dyn crate::secrets::SecretsStore,
user_id: &str,
@@ -292,7 +331,10 @@ pub async fn inject_llm_keys_from_secrets(
// Static mappings for well-known providers.
// The registry's setup hints define secret_name -> env_var mappings,
// so new providers added to providers.json get injection automatically.
let mut mappings: Vec<(&str, &str)> = vec![("llm_nearai_api_key", "NEARAI_API_KEY")];
let mut mappings: Vec<(&str, &str)> = vec![
("llm_nearai_api_key", "NEARAI_API_KEY"),
("llm_anthropic_oauth_token", "ANTHROPIC_OAUTH_TOKEN"),
];
// Dynamically discover secret->env mappings from the provider registry.
// Uses selectable() which deduplicates user overrides correctly.
@@ -331,5 +373,62 @@ pub async fn inject_llm_keys_from_secrets(
}
}
let _ = INJECTED_VARS.set(injected);
inject_os_credential_store_tokens(&mut injected);
merge_injected_vars(injected);
}
/// Load tokens from OS credential stores (no DB required).
///
/// Called unconditionally during startup — even when the encrypted secrets DB
/// is unavailable (no master key, no DB connection). This ensures OAuth tokens
/// from `claude login` (macOS Keychain / Linux credentials.json)
/// are available for config resolution.
pub fn inject_os_credentials() {
let mut injected = HashMap::new();
inject_os_credential_store_tokens(&mut injected);
merge_injected_vars(injected);
}
/// Merge new entries into the global injected-vars overlay.
///
/// New keys are inserted; existing keys are overwritten (later callers win,
/// e.g. fresh OS credential store tokens override stale DB copies).
fn merge_injected_vars(new_entries: HashMap<String, String>) {
if new_entries.is_empty() {
return;
}
match INJECTED_VARS.lock() {
Ok(mut map) => map.extend(new_entries),
Err(poisoned) => poisoned.into_inner().extend(new_entries),
}
}
/// Inject a single key-value pair into the overlay.
///
/// Used by the setup wizard to make credentials available to `optional_env()`
/// without calling `unsafe { std::env::set_var }`.
pub fn inject_single_var(key: &str, value: &str) {
match INJECTED_VARS.lock() {
Ok(mut map) => {
map.insert(key.to_string(), value.to_string());
}
Err(poisoned) => {
poisoned
.into_inner()
.insert(key.to_string(), value.to_string());
}
}
}
/// Shared helper: extract tokens from OS credential stores into the overlay map.
fn inject_os_credential_store_tokens(injected: &mut HashMap<String, String>) {
// Try the OS credential store for a fresh Anthropic OAuth token.
// Tokens from `claude login` expire in 8-12h, so the DB copy may be stale.
// A fresh extraction from macOS Keychain / Linux credentials.json wins
// over the (possibly expired) copy stored in the encrypted secrets DB.
if let Some(fresh) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
injected.insert("ANTHROPIC_OAUTH_TOKEN".to_string(), fresh);
tracing::debug!("Refreshed ANTHROPIC_OAUTH_TOKEN from OS credential store");
}
}
+16 -5
View File
@@ -233,9 +233,14 @@ impl ClaudeCodeConfig {
/// Expected shape: `{"claudeAiOauth": {"accessToken": "sk-ant-oat01-..."}}`
fn parse_oauth_access_token(json: &str) -> Option<String> {
let creds: serde_json::Value = serde_json::from_str(json).ok()?;
creds["claudeAiOauth"]["accessToken"]
.as_str()
.map(String::from)
let token = creds["claudeAiOauth"]["accessToken"].as_str()?;
// Validate that the token looks like a real OAuth token before using it.
// Claude CLI tokens start with "sk-ant-oat".
if !token.starts_with("sk-ant-oat") {
tracing::debug!("Ignoring credential store token with unexpected prefix");
return None;
}
Some(token.to_string())
}
#[cfg(test)]
@@ -401,14 +406,14 @@ mod tests {
fn parse_oauth_token_nested_extra_fields() {
let json = r#"{
"claudeAiOauth": {
"accessToken": "sk-ant-real-token",
"accessToken": "sk-ant-oat01-real-token",
"refreshToken": "rt-abc",
"expiresAt": 1700000000
}
}"#;
assert_eq!(
parse_oauth_access_token(json),
Some("sk-ant-real-token".to_string())
Some("sk-ant-oat01-real-token".to_string())
);
}
@@ -418,6 +423,12 @@ mod tests {
assert_eq!(parse_oauth_access_token(json), None);
}
#[test]
fn parse_oauth_token_rejects_invalid_prefix() {
let json = r#"{"claudeAiOauth": {"accessToken": "not-an-oauth-token"}}"#;
assert_eq!(parse_oauth_access_token(json), None);
}
// ── default_claude_code_allowed_tools ───────────────────────────
#[test]
+9
View File
@@ -164,6 +164,8 @@ pub struct JobContext {
/// previous results by ID via `$tool_call_id` parameter syntax.
#[serde(skip)]
pub tool_output_stash: Arc<tokio::sync::RwLock<HashMap<String, String>>>,
/// User's preferred timezone (IANA name, e.g. "America/New_York"). Defaults to "UTC".
pub user_timezone: String,
}
impl JobContext {
@@ -203,9 +205,16 @@ impl JobContext {
http_interceptor: None,
metadata: serde_json::Value::Null,
tool_output_stash: Arc::new(tokio::sync::RwLock::new(HashMap::new())),
user_timezone: "UTC".to_string(),
}
}
/// Set the user timezone on this context.
pub fn with_timezone(mut self, tz: impl Into<String>) -> Self {
self.user_timezone = tz.into();
self
}
/// Transition to a new state.
pub fn transition_to(
&mut self,
+346 -11
View File
@@ -20,9 +20,10 @@ impl ConversationStore for LibSqlBackend {
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
conn.execute(
"INSERT INTO conversations (id, channel, user_id, thread_id) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, opt_text(thread_id)],
"INSERT INTO conversations (id, channel, user_id, thread_id, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, opt_text(thread_id), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -71,8 +72,8 @@ impl ConversationStore for LibSqlBackend {
let now = fmt_ts(&Utc::now());
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, thread_id)
VALUES (?1, ?2, ?3, ?4)
INSERT INTO conversations (id, channel, user_id, thread_id, started_at, last_activity)
VALUES (?1, ?2, ?3, ?4, ?5, ?5)
ON CONFLICT (id) DO UPDATE SET last_activity = ?5
"#,
params![id.to_string(), channel, user_id, opt_text(thread_id), now],
@@ -97,6 +98,7 @@ impl ConversationStore for LibSqlBackend {
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
@@ -106,7 +108,7 @@ impl ConversationStore for LibSqlBackend {
) AS title
FROM conversations c
WHERE c.user_id = ?1 AND c.channel = ?2
ORDER BY c.last_activity DESC
ORDER BY datetime(c.last_activity) DESC
LIMIT ?3
"#,
params![user_id, channel, limit],
@@ -125,6 +127,13 @@ impl ConversationStore for LibSqlBackend {
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title = get_opt_text(&row, 6);
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
results.push(ConversationSummary {
id: row
.get::<String>(0)
@@ -133,14 +142,213 @@ impl ConversationStore for LibSqlBackend {
.unwrap_or_default(),
started_at: get_ts(&row, 1),
last_activity: get_ts(&row, 2),
message_count: get_i64(&row, 4),
title: get_opt_text(&row, 5),
message_count: get_i64(&row, 5),
title,
thread_type,
channel: get_text(&row, 4),
});
}
Ok(results)
}
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
let conn = self.connect().await?;
let mut rows = conn
.query(
r#"
SELECT
c.id,
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT substr(m2.content, 1, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
ORDER BY m2.created_at ASC, m2.rowid ASC
LIMIT 1
) AS title
FROM conversations c
WHERE c.user_id = ?1
ORDER BY datetime(c.last_activity) DESC
LIMIT ?2
"#,
params![user_id, limit],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let mut results = Vec::new();
while let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let metadata = get_json(&row, 3);
let thread_type = metadata
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title = get_opt_text(&row, 6);
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
results.push(ConversationSummary {
id: row
.get::<String>(0)
.unwrap_or_default()
.parse()
.unwrap_or_default(),
started_at: get_ts(&row, 1),
last_activity: get_ts(&row, 2),
message_count: get_i64(&row, 5),
title,
thread_type,
channel: get_text(&row, 4),
});
}
Ok(results)
}
/// Uses BEGIN IMMEDIATE to serialize concurrent writers and prevent
/// duplicate routine conversations (TOCTOU race).
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let rid = routine_id.to_string();
conn.execute("BEGIN IMMEDIATE", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let result: Result<Uuid, DatabaseError> = async {
let mut rows = conn
.query(
r#"
SELECT id FROM conversations
WHERE user_id = ?1 AND json_extract(metadata, '$.routine_id') = ?2
LIMIT 1
"#,
params![user_id, rid],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str: String = row.get(0).unwrap_or_default();
return id_str
.parse()
.map_err(|_| DatabaseError::Serialization("Invalid UUID".to_string()));
}
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({
"thread_type": "routine",
"routine_id": routine_id.to_string(),
"routine_name": routine_name,
});
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), "routine", user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(id)
}
.await;
match &result {
Ok(_) => {
conn.execute("COMMIT", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
}
Err(_) => {
let _ = conn.execute("ROLLBACK", params![]).await;
}
}
result
}
/// Uses BEGIN IMMEDIATE to serialize concurrent writers and prevent
/// duplicate heartbeat conversations (TOCTOU race).
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
conn.execute("BEGIN IMMEDIATE", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
let result: Result<Uuid, DatabaseError> = async {
let mut rows = conn
.query(
r#"
SELECT id FROM conversations
WHERE user_id = ?1 AND json_extract(metadata, '$.thread_type') = 'heartbeat'
LIMIT 1
"#,
params![user_id],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
if let Some(row) = rows
.next()
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?
{
let id_str: String = row.get(0).unwrap_or_default();
return id_str
.parse()
.map_err(|_| DatabaseError::Serialization("Invalid UUID".to_string()));
}
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({ "thread_type": "heartbeat" });
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), "heartbeat", user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
Ok(id)
}
.await;
match &result {
Ok(_) => {
conn.execute("COMMIT", params![])
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
}
Err(_) => {
let _ = conn.execute("ROLLBACK", params![]).await;
}
}
result
}
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
@@ -174,10 +382,11 @@ impl ConversationStore for LibSqlBackend {
// Create new
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
let metadata = serde_json::json!({"thread_type": "assistant", "title": "Assistant"});
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, metadata.to_string()],
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -192,9 +401,10 @@ impl ConversationStore for LibSqlBackend {
) -> Result<Uuid, DatabaseError> {
let conn = self.connect().await?;
let id = Uuid::new_v4();
let now = fmt_ts(&Utc::now());
conn.execute(
"INSERT INTO conversations (id, channel, user_id, metadata) VALUES (?1, ?2, ?3, ?4)",
params![id.to_string(), channel, user_id, metadata.to_string()],
"INSERT INTO conversations (id, channel, user_id, metadata, started_at, last_activity) VALUES (?1, ?2, ?3, ?4, ?5, ?5)",
params![id.to_string(), channel, user_id, metadata.to_string(), now],
)
.await
.map_err(|e| DatabaseError::Query(e.to_string()))?;
@@ -353,3 +563,128 @@ impl ConversationStore for LibSqlBackend {
Ok(found.is_some())
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::db::Database;
#[tokio::test]
async fn test_get_or_create_routine_conversation_is_idempotent() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_routine_conv.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let routine_id = Uuid::new_v4();
let user_id = "test_user";
// First call — creates the conversation
let id1 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
// Second call — should return the SAME conversation
let id2 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id2, "Expected same conversation ID on repeated calls");
// Third call — still the same
let id3 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id3);
// Different routine_id should get a different conversation
let other_routine_id = Uuid::new_v4();
let id4 = backend
.get_or_create_routine_conversation(other_routine_id, "other-routine", user_id)
.await
.unwrap();
assert_ne!(
id1, id4,
"Different routines should get different conversations"
);
}
#[tokio::test]
async fn test_routine_conversation_persists_across_messages() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_routine_persist.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let routine_id = Uuid::new_v4();
let user_id = "test_user";
// First invocation: create conversation and add a message
let id1 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
backend
.add_conversation_message(id1, "assistant", "[cron] Completed: all good")
.await
.unwrap();
// Second invocation: should find existing conversation
let id2 = backend
.get_or_create_routine_conversation(routine_id, "my-routine", user_id)
.await
.unwrap();
assert_eq!(id1, id2, "Second invocation should reuse same conversation");
backend
.add_conversation_message(id2, "assistant", "[cron] Completed: still good")
.await
.unwrap();
// Verify only one routine conversation exists (not two)
let convs = backend
.list_conversations_all_channels(user_id, 50)
.await
.unwrap();
let routine_convs: Vec<_> = convs.iter().filter(|c| c.channel == "routine").collect();
assert_eq!(
routine_convs.len(),
1,
"Should have exactly 1 routine conversation, found {}",
routine_convs.len()
);
}
#[tokio::test]
async fn test_get_or_create_heartbeat_conversation_is_idempotent() {
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_heartbeat_conv.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
let user_id = "test_user";
let id1 = backend
.get_or_create_heartbeat_conversation(user_id)
.await
.unwrap();
let id2 = backend
.get_or_create_heartbeat_conversation(user_id)
.await
.unwrap();
assert_eq!(
id1, id2,
"Expected same heartbeat conversation on repeated calls"
);
}
}
+3
View File
@@ -121,6 +121,9 @@ impl JobStore for LibSqlBackend {
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
// TODO(#661): persist user_timezone in agent_jobs table so
// background/routine jobs retain the session's timezone context.
user_timezone: "UTC".to_string(),
}))
}
None => Ok(None),
+59 -8
View File
@@ -118,15 +118,37 @@ impl LibSqlBackend {
/// Sets `PRAGMA busy_timeout = 5000` on every connection so concurrent
/// writers wait up to 5 seconds instead of failing instantly with
/// "database is locked".
///
/// Retries up to 3 times with exponential backoff to handle transient
/// "unable to open database file" errors from concurrent connection
/// creation (e.g. cron ticker vs main thread).
pub async fn connect(&self) -> Result<Connection, DatabaseError> {
let conn = self
.db
.connect()
.map_err(|e| DatabaseError::Pool(format!("Failed to create connection: {}", e)))?;
conn.query("PRAGMA busy_timeout = 5000", ())
.await
.map_err(|e| DatabaseError::Pool(format!("Failed to set busy_timeout: {}", e)))?;
Ok(conn)
let mut last_err = None;
for attempt in 0..3u32 {
match self.db.connect() {
Ok(conn) => {
conn.query("PRAGMA busy_timeout = 5000", ())
.await
.map_err(|e| {
DatabaseError::Pool(format!("Failed to set busy_timeout: {}", e))
})?;
return Ok(conn);
}
Err(e) => {
last_err = Some(e);
if attempt < 2 {
tokio::time::sleep(std::time::Duration::from_millis(
50 * 2u64.pow(attempt),
))
.await;
}
}
}
}
Err(DatabaseError::Pool(format!(
"Failed to create connection after 3 attempts: {}",
last_err.map(|e| e.to_string()).unwrap_or_default()
)))
}
}
@@ -459,4 +481,33 @@ mod tests {
let count: i64 = row.get(0).unwrap();
assert_eq!(count, 20);
}
#[tokio::test]
async fn test_connect_retry_succeeds_on_valid_db() {
// Verify connect() works with retry logic on a file-backed DB
// (exercises the retry path even though transient failures are hard
// to reproduce deterministically).
let dir = tempfile::tempdir().unwrap();
let db_path = dir.path().join("test_retry.db");
let backend = LibSqlBackend::new_local(&db_path).await.unwrap();
backend.run_migrations().await.unwrap();
// Multiple concurrent connect() calls should all succeed
let mut handles = Vec::new();
for _ in 0..10 {
let b = LibSqlBackend {
db: backend.shared_db(),
};
handles.push(tokio::spawn(async move { b.connect().await }));
}
for handle in handles {
let result = handle.await.unwrap();
assert!(
result.is_ok(),
"concurrent connect failed: {:?}",
result.err()
);
}
}
}
+9
View File
@@ -45,6 +45,15 @@ CREATE INDEX IF NOT EXISTS idx_conversations_channel ON conversations(channel);
CREATE INDEX IF NOT EXISTS idx_conversations_user ON conversations(user_id);
CREATE INDEX IF NOT EXISTS idx_conversations_last_activity ON conversations(last_activity);
-- Partial unique indexes to prevent duplicate singleton conversations.
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_routine
ON conversations (user_id, json_extract(metadata, '$.routine_id'))
WHERE json_extract(metadata, '$.routine_id') IS NOT NULL;
CREATE UNIQUE INDEX IF NOT EXISTS uq_conv_heartbeat
ON conversations (user_id)
WHERE json_extract(metadata, '$.thread_type') = 'heartbeat';
CREATE TABLE IF NOT EXISTS conversation_messages (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL REFERENCES conversations(id) ON DELETE CASCADE,
+15
View File
@@ -125,6 +125,21 @@ pub trait ConversationStore: Send + Sync {
channel: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError>;
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError>;
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError>;
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError>;
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
+30
View File
@@ -116,6 +116,36 @@ impl ConversationStore for PgBackend {
.await
}
async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
self.store
.list_conversations_all_channels(user_id, limit)
.await
}
async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
self.store
.get_or_create_routine_conversation(routine_id, routine_name, user_id)
.await
}
async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
self.store
.get_or_create_heartbeat_conversation(user_id)
.await
}
async fn get_or_create_assistant_conversation(
&self,
user_id: &str,
+3
View File
@@ -401,6 +401,9 @@ pub enum RoutineError {
#[error("Routine not found: {id}")]
NotFound { id: Uuid },
#[error("Not authorized to trigger routine {id}")]
NotAuthorized { id: Uuid },
#[error("Routine {name} at max concurrent runs")]
MaxConcurrent { name: String },
+123 -227
View File
@@ -1,13 +1,10 @@
//! Success evaluation for jobs.
use std::sync::Arc;
use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use crate::context::{ActionRecord, JobContext};
use crate::error::EvaluationError;
use crate::llm::LlmProvider;
/// Result of evaluating job success.
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -64,233 +61,132 @@ pub trait SuccessEvaluator: Send + Sync {
) -> Result<EvaluationResult, EvaluationError>;
}
/// Rule-based success evaluator.
pub struct RuleBasedEvaluator {
/// Minimum success rate for actions.
min_action_success_rate: f64,
/// Maximum allowed failures.
max_failures: u32,
}
impl RuleBasedEvaluator {
/// Create a new rule-based evaluator.
pub fn new() -> Self {
Self {
min_action_success_rate: 0.8,
max_failures: 3,
}
}
/// Set minimum action success rate.
#[allow(dead_code)] // Public API for configuring evaluation threshold
pub fn with_min_success_rate(mut self, rate: f64) -> Self {
self.min_action_success_rate = rate;
self
}
/// Set maximum failures.
#[allow(dead_code)] // Public API for configuring failure tolerance
pub fn with_max_failures(mut self, max: u32) -> Self {
self.max_failures = max;
self
}
}
impl Default for RuleBasedEvaluator {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl SuccessEvaluator for RuleBasedEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
_output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
let mut issues = Vec::new();
// Check if there were any actions
if actions.is_empty() {
return Ok(EvaluationResult::failure(
"No actions were taken",
vec!["No actions recorded".to_string()],
));
}
// Calculate action success rate
let successful = actions.iter().filter(|a| a.success).count();
let total = actions.len();
let success_rate = successful as f64 / total as f64;
if success_rate < self.min_action_success_rate {
issues.push(format!(
"Action success rate {:.1}% below threshold {:.1}%",
success_rate * 100.0,
self.min_action_success_rate * 100.0
));
}
// Count failures
let failures = actions.iter().filter(|a| !a.success).count() as u32;
if failures > self.max_failures {
issues.push(format!(
"Too many failures: {} (max {})",
failures, self.max_failures
));
}
// Check for critical errors
for action in actions.iter().filter(|a| !a.success) {
if let Some(ref error) = action.error
&& (error.to_lowercase().contains("critical")
|| error.to_lowercase().contains("fatal"))
{
issues.push(format!("Critical error in {}: {}", action.tool_name, error));
}
}
// Check job state
if job.state != crate::context::JobState::Completed
&& job.state != crate::context::JobState::Submitted
{
issues.push(format!("Job not in completed state: {:?}", job.state));
}
// Calculate quality score
let quality_score = if issues.is_empty() {
let base_score = (success_rate * 80.0) as u32;
let completion_bonus = if job.state == crate::context::JobState::Completed {
20
} else {
0
};
(base_score + completion_bonus).min(100)
} else {
((success_rate * 50.0) as u32).min(50)
};
if issues.is_empty() {
Ok(EvaluationResult::success(
format!(
"Job completed successfully with {}/{} actions succeeding ({:.1}%)",
successful,
total,
success_rate * 100.0
),
quality_score,
))
} else {
Ok(EvaluationResult {
success: false,
confidence: 0.85,
reasoning: format!("Job had {} issues", issues.len()),
issues,
suggestions: vec![
"Review failed actions for common patterns".to_string(),
"Consider adjusting retry logic".to_string(),
],
quality_score,
})
}
}
}
/// LLM-based success evaluator for more nuanced evaluation.
pub struct LlmEvaluator {
llm: Arc<dyn LlmProvider>,
}
impl LlmEvaluator {
/// Create a new LLM-based evaluator.
#[allow(dead_code)] // Public API for LLM-based evaluation
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self { llm }
}
}
#[async_trait]
impl SuccessEvaluator for LlmEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
// Build evaluation prompt
let actions_summary: Vec<String> = actions
.iter()
.map(|a| {
format!(
"- {}: {} ({})",
a.tool_name,
if a.success { "success" } else { "failed" },
a.error.as_deref().unwrap_or("ok")
)
})
.collect();
let prompt = format!(
r#"Evaluate if this job was completed successfully.
Job: {}
Description: {}
State: {:?}
Actions taken:
{}
{}
Respond in JSON format:
{{
"success": true/false,
"confidence": 0.0-1.0,
"reasoning": "...",
"issues": ["..."],
"suggestions": ["..."],
"quality_score": 0-100
}}"#,
job.title,
job.description,
job.state,
actions_summary.join("\n"),
output
.map(|o| format!("Output:\n{}", o))
.unwrap_or_default()
);
let request =
crate::llm::CompletionRequest::new(vec![crate::llm::ChatMessage::user(prompt)])
.with_max_tokens(1024)
.with_temperature(0.1);
let response = self
.llm
.complete(request)
.await
.map_err(|e| EvaluationError::Failed {
job_id: job.job_id,
reason: e.to_string(),
})?;
// Parse the response
let result: EvaluationResult =
serde_json::from_str(&response.content).map_err(|e| EvaluationError::Failed {
job_id: job.job_id,
reason: format!("Failed to parse LLM evaluation: {}", e),
})?;
Ok(result)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::context::JobContext;
use crate::context::{ActionRecord, JobContext};
use crate::error::EvaluationError;
/// Rule-based success evaluator (test-only; no production callers).
struct RuleBasedEvaluator {
min_action_success_rate: f64,
max_failures: u32,
}
impl RuleBasedEvaluator {
fn new() -> Self {
Self {
min_action_success_rate: 0.8,
max_failures: 3,
}
}
fn with_min_success_rate(mut self, rate: f64) -> Self {
self.min_action_success_rate = rate;
self
}
fn with_max_failures(mut self, max: u32) -> Self {
self.max_failures = max;
self
}
}
impl Default for RuleBasedEvaluator {
fn default() -> Self {
Self::new()
}
}
#[async_trait::async_trait]
impl SuccessEvaluator for RuleBasedEvaluator {
async fn evaluate(
&self,
job: &JobContext,
actions: &[ActionRecord],
_output: Option<&str>,
) -> Result<EvaluationResult, EvaluationError> {
let mut issues = Vec::new();
if actions.is_empty() {
return Ok(EvaluationResult::failure(
"No actions were taken",
vec!["No actions recorded".to_string()],
));
}
let successful = actions.iter().filter(|a| a.success).count();
let total = actions.len();
let success_rate = successful as f64 / total as f64;
if success_rate < self.min_action_success_rate {
issues.push(format!(
"Action success rate {:.1}% below threshold {:.1}%",
success_rate * 100.0,
self.min_action_success_rate * 100.0
));
}
let failures = actions.iter().filter(|a| !a.success).count() as u32;
if failures > self.max_failures {
issues.push(format!(
"Too many failures: {} (max {})",
failures, self.max_failures
));
}
for action in actions.iter().filter(|a| !a.success) {
if let Some(ref error) = action.error
&& (error.to_lowercase().contains("critical")
|| error.to_lowercase().contains("fatal"))
{
issues.push(format!("Critical error in {}: {}", action.tool_name, error));
}
}
if job.state != crate::context::JobState::Completed
&& job.state != crate::context::JobState::Submitted
{
issues.push(format!("Job not in completed state: {:?}", job.state));
}
let quality_score = if issues.is_empty() {
let base_score = (success_rate * 80.0) as u32;
let completion_bonus = if job.state == crate::context::JobState::Completed {
20
} else {
0
};
(base_score + completion_bonus).min(100)
} else {
((success_rate * 50.0) as u32).min(50)
};
if issues.is_empty() {
Ok(EvaluationResult::success(
format!(
"Job completed successfully with {}/{} actions succeeding ({:.1}%)",
successful,
total,
success_rate * 100.0
),
quality_score,
))
} else {
Ok(EvaluationResult {
success: false,
confidence: 0.85,
reasoning: format!("Job had {} issues", issues.len()),
issues,
suggestions: vec![
"Review failed actions for common patterns".to_string(),
"Consider adjusting retry logic".to_string(),
],
quality_score,
})
}
}
}
#[tokio::test]
async fn test_rule_based_evaluator_success() {
-32
View File
@@ -1405,38 +1405,6 @@ impl ExtensionManager {
Ok(())
}
#[allow(dead_code)] // Used by upcoming hot-activation flow
async fn install_bundled_channel_from_artifacts(
&self,
name: &str,
) -> Result<InstallResult, ExtensionError> {
// Check if already installed
let channel_wasm = self.wasm_channels_dir.join(format!("{}.wasm", name));
if channel_wasm.exists() {
return Err(ExtensionError::AlreadyInstalled(name.to_string()));
}
crate::channels::wasm::install_bundled_channel(name, &self.wasm_channels_dir, false)
.await
.map_err(ExtensionError::InstallFailed)?;
tracing::info!(
"Installed bundled channel '{}' to {}",
name,
self.wasm_channels_dir.display()
);
Ok(InstallResult {
name: name.to_string(),
kind: ExtensionKind::WasmChannel,
message: format!(
"Channel '{}' installed. \
Run tool_auth('{}') to configure authentication, then activate.",
name, name,
),
})
}
/// Install a WASM extension from local build artifacts (WasmBuildable source).
///
/// Resolves the build directory (relative to `CARGO_MANIFEST_DIR` or absolute),
+207 -1
View File
@@ -241,6 +241,9 @@ impl Store {
tool_output_stash: std::sync::Arc::new(tokio::sync::RwLock::new(
std::collections::HashMap::new(),
)),
// TODO(#661): persist user_timezone in agent_jobs table so
// background/routine jobs retain the session's timezone context.
user_timezone: "UTC".to_string(),
}))
}
None => Ok(None),
@@ -1377,6 +1380,8 @@ pub struct ConversationSummary {
pub last_activity: DateTime<Utc>,
/// Thread type extracted from metadata (e.g. "assistant", "thread").
pub thread_type: Option<String>,
/// Channel that owns this conversation (e.g. "gateway", "telegram", "routine").
pub channel: String,
}
/// A single message in a conversation.
@@ -1429,6 +1434,7 @@ impl Store {
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT LEFT(m2.content, 100)
FROM conversation_messages m2
@@ -1453,18 +1459,181 @@ impl Store {
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
let sql_title: Option<String> = r.get("title");
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
ConversationSummary {
id: r.get("id"),
title: r.get("title"),
title,
message_count: r.get("message_count"),
started_at: r.get("started_at"),
last_activity: r.get("last_activity"),
thread_type,
channel: r.get("channel"),
}
})
.collect())
}
/// List conversations across all channels with a title derived from the first user message.
pub async fn list_conversations_all_channels(
&self,
user_id: &str,
limit: i64,
) -> Result<Vec<ConversationSummary>, DatabaseError> {
let conn = self.conn().await?;
let rows = conn
.query(
r#"
SELECT
c.id,
c.started_at,
c.last_activity,
c.metadata,
c.channel,
(SELECT COUNT(*) FROM conversation_messages m WHERE m.conversation_id = c.id AND m.role = 'user') AS message_count,
(SELECT LEFT(m2.content, 100)
FROM conversation_messages m2
WHERE m2.conversation_id = c.id AND m2.role = 'user'
ORDER BY m2.created_at ASC
LIMIT 1
) AS title
FROM conversations c
WHERE c.user_id = $1
ORDER BY c.last_activity DESC
LIMIT $2
"#,
&[&user_id, &limit],
)
.await?;
Ok(rows
.iter()
.map(|r| {
let metadata: serde_json::Value = r.get("metadata");
let thread_type = metadata
.get("thread_type")
.and_then(|v| v.as_str())
.map(String::from);
// For routine/heartbeat threads, derive title from metadata
// since they may have no user messages.
let sql_title: Option<String> = r.get("title");
let title = sql_title.or_else(|| {
metadata
.get("routine_name")
.and_then(|v| v.as_str())
.map(String::from)
});
ConversationSummary {
id: r.get("id"),
title,
message_count: r.get("message_count"),
started_at: r.get("started_at"),
last_activity: r.get("last_activity"),
thread_type,
channel: r.get("channel"),
}
})
.collect())
}
/// Get or create a persistent conversation for a routine.
///
/// Looks for a conversation where `metadata->>'routine_id' = routine_id`.
/// Creates one if it doesn't exist. Uses INSERT ON CONFLICT to avoid
/// TOCTOU races under concurrent routine executions.
pub async fn get_or_create_routine_conversation(
&self,
routine_id: Uuid,
routine_name: &str,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.conn().await?;
let rid = routine_id.to_string();
// Attempt insert first; the partial unique index
// uq_conv_routine(user_id, (metadata->>'routine_id')) prevents duplicates.
let new_id = Uuid::new_v4();
let metadata = serde_json::json!({
"thread_type": "routine",
"routine_id": routine_id.to_string(),
"routine_name": routine_name,
});
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, metadata)
VALUES ($1, 'routine', $2, $3)
ON CONFLICT (user_id, (metadata->>'routine_id'))
WHERE metadata->>'routine_id' IS NOT NULL
DO NOTHING
"#,
&[&new_id, &user_id, &metadata],
)
.await?;
// Select back — always returns the winner.
let row = conn
.query_one(
r#"
SELECT id FROM conversations
WHERE user_id = $1 AND metadata->>'routine_id' = $2
LIMIT 1
"#,
&[&user_id, &rid],
)
.await?;
Ok(row.get("id"))
}
/// Get or create the singleton heartbeat conversation for a user.
///
/// Looks for a conversation where `metadata->>'thread_type' = 'heartbeat'`.
/// Creates one if it doesn't exist. Uses INSERT ON CONFLICT to avoid
/// TOCTOU races under concurrent heartbeat sends.
pub async fn get_or_create_heartbeat_conversation(
&self,
user_id: &str,
) -> Result<Uuid, DatabaseError> {
let conn = self.conn().await?;
// Attempt insert; the partial unique index
// uq_conv_heartbeat(user_id) prevents duplicates.
let new_id = Uuid::new_v4();
let metadata = serde_json::json!({
"thread_type": "heartbeat",
});
conn.execute(
r#"
INSERT INTO conversations (id, channel, user_id, metadata)
VALUES ($1, 'heartbeat', $2, $3)
ON CONFLICT (user_id)
WHERE metadata->>'thread_type' = 'heartbeat'
DO NOTHING
"#,
&[&new_id, &user_id, &metadata],
)
.await?;
// Select back — always returns the winner.
let row = conn
.query_one(
r#"
SELECT id FROM conversations
WHERE user_id = $1 AND metadata->>'thread_type' = 'heartbeat'
LIMIT 1
"#,
&[&user_id],
)
.await?;
Ok(row.get("id"))
}
/// Get or create the singleton "assistant" conversation for a user+channel.
///
/// Looks for a conversation where `metadata->>'thread_type' = 'assistant'`.
@@ -1928,3 +2097,40 @@ impl Store {
Ok(count > 0)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_conversation_summary_has_channel_field() {
// Regression: ConversationSummary must include a `channel` field
// so the gateway can distinguish thread origins.
let summary = ConversationSummary {
id: Uuid::nil(),
title: Some("Hello".to_string()),
message_count: 1,
started_at: Utc::now(),
last_activity: Utc::now(),
thread_type: Some("thread".to_string()),
channel: "telegram".to_string(),
};
assert_eq!(summary.channel, "telegram");
}
#[test]
fn test_conversation_summary_channel_various_values() {
for ch in ["gateway", "routine", "heartbeat", "telegram", "signal"] {
let summary = ConversationSummary {
id: Uuid::nil(),
title: None,
message_count: 0,
started_at: Utc::now(),
last_activity: Utc::now(),
thread_type: None,
channel: ch.to_string(),
};
assert_eq!(summary.channel, ch);
}
}
}
+1
View File
@@ -66,6 +66,7 @@ pub mod service;
pub mod settings;
pub mod setup;
pub mod skills;
pub mod timezone;
pub mod tools;
pub mod tracing_fmt;
pub mod transcription;
+641
View File
@@ -0,0 +1,641 @@
//! Anthropic OAuth provider (direct HTTP, `Authorization: Bearer`).
//!
//! This provider exists because the `rig-core` Anthropic client hardcodes the
//! `x-api-key` header, which is rejected by Anthropic's OAuth tokens from
//! `claude login`. OAuth tokens require `Authorization: Bearer <token>` instead.
//!
//! Pattern follows `nearai_chat.rs`: direct HTTP calls via `reqwest::Client`.
use async_trait::async_trait;
use reqwest::Client;
use rust_decimal::Decimal;
use secrecy::{ExposeSecret, SecretString};
use serde::{Deserialize, Serialize};
use crate::config::RegistryProviderConfig;
use crate::error::LlmError;
use crate::llm::costs;
use crate::llm::provider::{
ChatMessage, CompletionRequest, CompletionResponse, FinishReason, LlmProvider, Role, ToolCall,
ToolCompletionRequest, ToolCompletionResponse,
};
const ANTHROPIC_API_URL: &str = "https://api.anthropic.com/v1/messages";
/// OAuth beta requires 2023-06-01; the 2024-10-22 version is not valid with the beta flag.
const ANTHROPIC_API_VERSION: &str = "2023-06-01";
/// Required beta flag to enable OAuth Bearer auth on api.anthropic.com.
/// Without this header, the API returns 401 "OAuth authentication is currently not supported."
const ANTHROPIC_OAUTH_BETA: &str = "oauth-2025-04-20";
const DEFAULT_MAX_TOKENS: u32 = 8192;
/// Anthropic provider using OAuth Bearer authentication.
pub struct AnthropicOAuthProvider {
client: Client,
token: SecretString,
model: String,
base_url: Option<String>,
active_model: std::sync::RwLock<String>,
}
impl AnthropicOAuthProvider {
pub fn new(config: &RegistryProviderConfig) -> Result<Self, LlmError> {
let token = config
.oauth_token
.clone()
.ok_or_else(|| LlmError::AuthFailed {
provider: "anthropic_oauth".to_string(),
})?;
let client = Client::builder()
.timeout(std::time::Duration::from_secs(120))
.build()
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to build HTTP client: {}", e),
})?;
let active_model = std::sync::RwLock::new(config.model.clone());
let base_url = if config.base_url.is_empty() {
None
} else {
Some(config.base_url.clone())
};
Ok(Self {
client,
token,
model: config.model.clone(),
base_url,
active_model,
})
}
fn api_url(&self) -> String {
if let Some(ref base) = self.base_url {
let base = base.trim_end_matches('/');
format!("{}/v1/messages", base)
} else {
ANTHROPIC_API_URL.to_string()
}
}
async fn send_request<R: for<'de> Deserialize<'de>>(
&self,
body: &AnthropicRequest,
) -> Result<R, LlmError> {
let url = self.api_url();
tracing::debug!("Sending request to Anthropic OAuth: {}", url);
let response = self
.client
.post(&url)
.bearer_auth(self.token.expose_secret())
.header("anthropic-version", ANTHROPIC_API_VERSION)
.header("anthropic-beta", ANTHROPIC_OAUTH_BETA)
.header("Content-Type", "application/json")
.json(body)
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: e.to_string(),
})?;
let status = response.status();
if !status.is_success() {
// Parse Retry-After header before consuming the body.
let retry_after = response
.headers()
.get("retry-after")
.and_then(|v| v.to_str().ok())
.and_then(|v| v.parse::<u64>().ok())
.map(std::time::Duration::from_secs);
let response_text = response
.text()
.await
.unwrap_or_else(|e| format!("(failed to read error body: {e})"));
if status.as_u16() == 401 {
// OAuth tokens from `claude login` expire in ~8-12h. Attempt
// to re-extract a fresh token from the OS credential store
// (macOS Keychain / Linux credentials file) before giving up.
if let Some(fresh) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
let fresh_token = SecretString::from(fresh);
// Retry once with the refreshed token
let retry = self
.client
.post(&url)
.bearer_auth(fresh_token.expose_secret())
.header("anthropic-version", ANTHROPIC_API_VERSION)
.header("anthropic-beta", ANTHROPIC_OAUTH_BETA)
.header("Content-Type", "application/json")
.json(body)
.send()
.await
.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: e.to_string(),
})?;
if retry.status().is_success() {
let text = retry.text().await.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to read response body: {}", e),
})?;
return serde_json::from_str(&text).map_err(|e| {
let truncated = crate::agent::truncate_for_preview(&text, 512);
LlmError::InvalidResponse {
provider: "anthropic_oauth".to_string(),
reason: format!("JSON parse error: {}. Raw: {}", e, truncated),
}
});
}
tracing::warn!(
"Anthropic OAuth 401 retry with refreshed token also failed ({})",
retry.status()
);
}
return Err(LlmError::AuthFailed {
provider: "anthropic_oauth".to_string(),
});
}
if status.as_u16() == 429 {
return Err(LlmError::RateLimited {
provider: "anthropic_oauth".to_string(),
retry_after,
});
}
let truncated = crate::agent::truncate_for_preview(&response_text, 512);
return Err(LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("HTTP {}: {}", status, truncated),
});
}
let response_text = response.text().await.map_err(|e| LlmError::RequestFailed {
provider: "anthropic_oauth".to_string(),
reason: format!("Failed to read response body: {}", e),
})?;
tracing::debug!(
"Anthropic OAuth response: status={}, bytes={}",
status,
response_text.len()
);
serde_json::from_str(&response_text).map_err(|e| {
let truncated = crate::agent::truncate_for_preview(&response_text, 512);
LlmError::InvalidResponse {
provider: "anthropic_oauth".to_string(),
reason: format!("JSON parse error: {}. Raw: {}", e, truncated),
}
})
}
}
#[async_trait]
impl LlmProvider for AnthropicOAuthProvider {
async fn complete(&self, req: CompletionRequest) -> Result<CompletionResponse, LlmError> {
let model = req.model.unwrap_or_else(|| self.active_model_name());
let (system, messages) = convert_messages(req.messages);
let request = AnthropicRequest {
model,
messages,
system,
max_tokens: req.max_tokens.unwrap_or(DEFAULT_MAX_TOKENS),
temperature: req.temperature,
tools: None,
tool_choice: None,
};
let response: AnthropicResponse = self.send_request(&request).await?;
let (content, _tool_calls) = extract_response_content(&response);
let finish_reason = match response.stop_reason.as_deref() {
Some("end_turn") | Some("stop") => FinishReason::Stop,
Some("max_tokens") => FinishReason::Length,
Some("tool_use") => FinishReason::ToolUse,
_ => FinishReason::Unknown,
};
Ok(CompletionResponse {
content: content.unwrap_or_default(),
finish_reason,
input_tokens: response.usage.input_tokens,
output_tokens: response.usage.output_tokens,
cache_creation_input_tokens: response.usage.cache_creation_input_tokens,
cache_read_input_tokens: response.usage.cache_read_input_tokens,
})
}
async fn complete_with_tools(
&self,
req: ToolCompletionRequest,
) -> Result<ToolCompletionResponse, LlmError> {
let model = req.model.unwrap_or_else(|| self.active_model_name());
let (system, messages) = convert_messages(req.messages);
let tools: Vec<AnthropicTool> = req
.tools
.into_iter()
.map(|t| AnthropicTool {
name: t.name,
description: t.description,
input_schema: t.parameters,
})
.collect();
// Map tool_choice from OpenAI format to Anthropic format
let tool_choice = req.tool_choice.map(|tc| match tc.as_str() {
"auto" => AnthropicToolChoice {
choice_type: "auto".to_string(),
name: None,
},
"required" => AnthropicToolChoice {
choice_type: "any".to_string(),
name: None,
},
"none" => AnthropicToolChoice {
choice_type: "none".to_string(),
name: None,
},
specific => AnthropicToolChoice {
choice_type: "tool".to_string(),
name: Some(specific.to_string()),
},
});
let request = AnthropicRequest {
model,
messages,
system,
max_tokens: req.max_tokens.unwrap_or(DEFAULT_MAX_TOKENS),
temperature: req.temperature,
tools: if tools.is_empty() { None } else { Some(tools) },
tool_choice,
};
let response: AnthropicResponse = self.send_request(&request).await?;
let (content, tool_calls) = extract_response_content(&response);
let finish_reason = match response.stop_reason.as_deref() {
Some("end_turn") | Some("stop") => FinishReason::Stop,
Some("max_tokens") => FinishReason::Length,
Some("tool_use") => FinishReason::ToolUse,
_ => {
if !tool_calls.is_empty() {
FinishReason::ToolUse
} else {
FinishReason::Unknown
}
}
};
Ok(ToolCompletionResponse {
content,
tool_calls,
finish_reason,
input_tokens: response.usage.input_tokens,
output_tokens: response.usage.output_tokens,
cache_creation_input_tokens: response.usage.cache_creation_input_tokens,
cache_read_input_tokens: response.usage.cache_read_input_tokens,
})
}
fn model_name(&self) -> &str {
&self.model
}
fn cost_per_token(&self) -> (Decimal, Decimal) {
let model = self.active_model_name();
costs::model_cost(&model).unwrap_or_else(costs::default_cost)
}
fn active_model_name(&self) -> String {
match self.active_model.read() {
Ok(guard) => guard.clone(),
Err(poisoned) => poisoned.into_inner().clone(),
}
}
fn set_model(&self, model: &str) -> Result<(), LlmError> {
match self.active_model.write() {
Ok(mut guard) => {
*guard = model.to_string();
}
Err(poisoned) => {
*poisoned.into_inner() = model.to_string();
}
}
Ok(())
}
}
// --- Anthropic Messages API types ---
#[derive(Debug, Serialize)]
struct AnthropicRequest {
model: String,
messages: Vec<AnthropicMessage>,
#[serde(skip_serializing_if = "Option::is_none")]
system: Option<String>,
max_tokens: u32,
#[serde(skip_serializing_if = "Option::is_none")]
temperature: Option<f32>,
#[serde(skip_serializing_if = "Option::is_none")]
tools: Option<Vec<AnthropicTool>>,
#[serde(skip_serializing_if = "Option::is_none")]
tool_choice: Option<AnthropicToolChoice>,
}
#[derive(Debug, Serialize)]
struct AnthropicMessage {
role: String,
content: AnthropicContent,
}
/// Anthropic content can be a simple string or a list of content blocks.
#[derive(Debug, Serialize)]
#[serde(untagged)]
enum AnthropicContent {
Text(String),
Blocks(Vec<AnthropicContentBlock>),
}
#[derive(Debug, Serialize)]
#[serde(tag = "type")]
enum AnthropicContentBlock {
#[serde(rename = "text")]
Text { text: String },
#[serde(rename = "tool_use")]
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
#[serde(rename = "tool_result")]
ToolResult {
tool_use_id: String,
content: String,
},
}
#[derive(Debug, Serialize)]
struct AnthropicTool {
name: String,
description: String,
input_schema: serde_json::Value,
}
#[derive(Debug, Serialize)]
struct AnthropicToolChoice {
#[serde(rename = "type")]
choice_type: String,
#[serde(skip_serializing_if = "Option::is_none")]
name: Option<String>,
}
#[derive(Debug, Deserialize)]
struct AnthropicResponse {
content: Vec<AnthropicResponseBlock>,
#[serde(default)]
stop_reason: Option<String>,
usage: AnthropicUsage,
}
#[derive(Debug, Deserialize)]
#[serde(tag = "type")]
enum AnthropicResponseBlock {
#[serde(rename = "text")]
Text { text: String },
#[serde(rename = "tool_use")]
ToolUse {
id: String,
name: String,
input: serde_json::Value,
},
}
#[derive(Debug, Deserialize)]
struct AnthropicUsage {
#[serde(default)]
input_tokens: u32,
#[serde(default)]
output_tokens: u32,
#[serde(default)]
cache_creation_input_tokens: u32,
#[serde(default)]
cache_read_input_tokens: u32,
}
/// Convert ChatMessage list to Anthropic format.
///
/// Extracts system messages to the top-level `system` parameter (Anthropic
/// doesn't allow system messages in the `messages` array). Tool-call/tool-result
/// pairs are converted to content blocks.
fn convert_messages(messages: Vec<ChatMessage>) -> (Option<String>, Vec<AnthropicMessage>) {
let mut system_parts: Vec<String> = Vec::new();
let mut anthropic_msgs: Vec<AnthropicMessage> = Vec::new();
for msg in messages {
match msg.role {
Role::System => {
if !msg.content.is_empty() {
system_parts.push(msg.content);
}
}
Role::User => {
anthropic_msgs.push(AnthropicMessage {
role: "user".to_string(),
content: AnthropicContent::Text(msg.content),
});
}
Role::Assistant => {
if let Some(tool_calls) = msg.tool_calls {
// Assistant message with tool calls → content blocks
let mut blocks: Vec<AnthropicContentBlock> = Vec::new();
if !msg.content.is_empty() {
blocks.push(AnthropicContentBlock::Text { text: msg.content });
}
for tc in tool_calls {
blocks.push(AnthropicContentBlock::ToolUse {
id: tc.id,
name: tc.name,
input: tc.arguments,
});
}
anthropic_msgs.push(AnthropicMessage {
role: "assistant".to_string(),
content: AnthropicContent::Blocks(blocks),
});
} else {
anthropic_msgs.push(AnthropicMessage {
role: "assistant".to_string(),
content: AnthropicContent::Text(msg.content),
});
}
}
Role::Tool => {
let Some(tool_call_id) = msg.tool_call_id else {
tracing::warn!("Skipping Tool message without tool_call_id");
continue;
};
// Tool results go into a user message with tool_result blocks
let block = AnthropicContentBlock::ToolResult {
tool_use_id: tool_call_id,
content: msg.content,
};
// If the last message is already a user message with blocks,
// append to it (Anthropic requires consecutive tool results
// in one user message).
if let Some(last) = anthropic_msgs.last_mut()
&& last.role == "user"
&& let AnthropicContent::Blocks(ref mut blocks) = last.content
{
blocks.push(block);
continue;
}
anthropic_msgs.push(AnthropicMessage {
role: "user".to_string(),
content: AnthropicContent::Blocks(vec![block]),
});
}
}
}
let system = if system_parts.is_empty() {
None
} else {
Some(system_parts.join("\n\n"))
};
(system, anthropic_msgs)
}
/// Extract text content and tool calls from an Anthropic response.
fn extract_response_content(response: &AnthropicResponse) -> (Option<String>, Vec<ToolCall>) {
let mut text_parts: Vec<String> = Vec::new();
let mut tool_calls: Vec<ToolCall> = Vec::new();
for block in &response.content {
match block {
AnthropicResponseBlock::Text { text } => {
text_parts.push(text.clone());
}
AnthropicResponseBlock::ToolUse { id, name, input } => {
tool_calls.push(ToolCall {
id: id.clone(),
name: name.clone(),
arguments: input.clone(),
});
}
}
}
let content = if text_parts.is_empty() {
None
} else {
Some(text_parts.join(""))
};
(content, tool_calls)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_convert_messages_extracts_system() {
let messages = vec![
ChatMessage::system("You are helpful."),
ChatMessage::user("Hello"),
];
let (system, msgs) = convert_messages(messages);
assert_eq!(system, Some("You are helpful.".to_string()));
assert_eq!(msgs.len(), 1);
assert_eq!(msgs[0].role, "user");
}
#[test]
fn test_convert_messages_multiple_systems() {
let messages = vec![
ChatMessage::system("System 1"),
ChatMessage::system("System 2"),
ChatMessage::user("Hello"),
];
let (system, msgs) = convert_messages(messages);
assert_eq!(system, Some("System 1\n\nSystem 2".to_string()));
assert_eq!(msgs.len(), 1);
}
#[test]
fn test_convert_messages_tool_calls() {
let tool_calls = vec![ToolCall {
id: "call_1".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "test"}),
}];
let messages = vec![
ChatMessage::user("Search for test"),
ChatMessage::assistant_with_tool_calls(Some("Let me search.".to_string()), tool_calls),
ChatMessage::tool_result("call_1", "search", "found it"),
];
let (system, msgs) = convert_messages(messages);
assert!(system.is_none());
assert_eq!(msgs.len(), 3);
assert_eq!(msgs[0].role, "user");
assert_eq!(msgs[1].role, "assistant");
// Tool result should be a user message
assert_eq!(msgs[2].role, "user");
}
#[test]
fn test_extract_response_text_only() {
let response = AnthropicResponse {
content: vec![AnthropicResponseBlock::Text {
text: "Hello!".to_string(),
}],
stop_reason: Some("end_turn".to_string()),
usage: AnthropicUsage {
input_tokens: 10,
output_tokens: 5,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 0,
},
};
let (content, tool_calls) = extract_response_content(&response);
assert_eq!(content, Some("Hello!".to_string()));
assert!(tool_calls.is_empty());
}
#[test]
fn test_extract_response_with_tool_use() {
let response = AnthropicResponse {
content: vec![
AnthropicResponseBlock::Text {
text: "Let me search.".to_string(),
},
AnthropicResponseBlock::ToolUse {
id: "call_1".to_string(),
name: "search".to_string(),
input: serde_json::json!({"q": "test"}),
},
],
stop_reason: Some("tool_use".to_string()),
usage: AnthropicUsage {
input_tokens: 20,
output_tokens: 15,
cache_creation_input_tokens: 0,
cache_read_input_tokens: 0,
},
};
let (content, tool_calls) = extract_response_content(&response);
assert_eq!(content, Some("Let me search.".to_string()));
assert_eq!(tool_calls.len(), 1);
assert_eq!(tool_calls[0].name, "search");
}
}
+40 -4
View File
@@ -7,6 +7,7 @@
//! - **Ollama**: Local model inference
//! - **OpenAI-compatible**: Any endpoint that speaks the OpenAI API
mod anthropic_oauth;
pub mod circuit_breaker;
pub mod costs;
pub mod failover;
@@ -57,8 +58,10 @@ pub fn create_llm_provider(
config: &LlmConfig,
session: Arc<SessionManager>,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
let timeout = config.request_timeout_secs;
if config.backend == "nearai" || config.backend == "near_ai" || config.backend == "near" {
return create_llm_provider_with_config(&config.nearai, session);
return create_llm_provider_with_config(&config.nearai, session, timeout);
}
let reg_config = config
@@ -78,6 +81,7 @@ pub fn create_llm_provider(
pub fn create_llm_provider_with_config(
config: &NearAiConfig,
session: Arc<SessionManager>,
request_timeout_secs: u64,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
let auth_mode = if config.api_key.is_some() {
"API key"
@@ -88,9 +92,14 @@ pub fn create_llm_provider_with_config(
model = %config.model,
base_url = %config.base_url,
auth = auth_mode,
timeout_secs = request_timeout_secs,
"Using NEAR AI (Chat Completions API)"
);
Ok(Arc::new(NearAiChatProvider::new(config.clone(), session)?))
Ok(Arc::new(NearAiChatProvider::new_with_timeout(
config.clone(),
session,
request_timeout_secs,
)?))
}
/// Create a provider from a registry-resolved config.
@@ -178,6 +187,24 @@ fn create_openai_compat_from_registry(
fn create_anthropic_from_registry(
config: &RegistryProviderConfig,
) -> Result<Arc<dyn LlmProvider>, LlmError> {
// Route to OAuth provider when an OAuth token is present and no real API
// key was provided. When both are set, the API key takes priority (standard
// x-api-key auth via rig-core).
let api_key_is_placeholder = config
.api_key
.as_ref()
.is_some_and(|k| k.expose_secret() == crate::config::llm::OAUTH_PLACEHOLDER);
if config.oauth_token.is_some() && (config.api_key.is_none() || api_key_is_placeholder) {
tracing::info!(
provider = %config.provider_id,
model = %config.model,
base_url = if config.base_url.is_empty() { "default" } else { &config.base_url },
"Using Anthropic OAuth API"
);
let provider = anthropic_oauth::AnthropicOAuthProvider::new(config)?;
return Ok(Arc::new(provider));
}
use crate::config::CacheRetention;
use crate::config::helpers::optional_env;
use rig::providers::anthropic;
@@ -346,7 +373,11 @@ pub fn build_provider_chain(
let llm: Arc<dyn LlmProvider> = if let Some(ref cheap_model) = config.nearai.cheap_model {
let mut cheap_config = config.nearai.clone();
cheap_config.model = cheap_model.clone();
let cheap = create_llm_provider_with_config(&cheap_config, session.clone())?;
let cheap = create_llm_provider_with_config(
&cheap_config,
session.clone(),
config.request_timeout_secs,
)?;
let cheap: Arc<dyn LlmProvider> = if retry_config.max_retries > 0 {
Arc::new(RetryProvider::new(cheap, retry_config.clone()))
} else {
@@ -378,7 +409,11 @@ pub fn build_provider_chain(
}
let mut fallback_config = config.nearai.clone();
fallback_config.model = fallback_model.clone();
let fallback = create_llm_provider_with_config(&fallback_config, session.clone())?;
let fallback = create_llm_provider_with_config(
&fallback_config,
session.clone(),
config.request_timeout_secs,
)?;
tracing::info!(
primary = %llm.model_name(),
fallback = %fallback.model_name(),
@@ -484,6 +519,7 @@ mod tests {
session: SessionConfig::default(),
nearai: test_nearai_config(),
provider: None,
request_timeout_secs: 120,
}
}
+15 -4
View File
@@ -58,17 +58,28 @@ impl NearAiChatProvider {
/// By default this enables tool-message flattening for compatibility with
/// providers that reject `role: "tool"` messages.
pub fn new(config: NearAiConfig, session: Arc<SessionManager>) -> Result<Self, LlmError> {
Self::new_with_flatten(config, session, true)
Self::new_with_options(config, session, true, 120)
}
/// Create a chat completions provider with configurable tool-message flattening.
pub fn new_with_flatten(
/// Create a new provider with a custom request timeout.
pub fn new_with_timeout(
config: NearAiConfig,
session: Arc<SessionManager>,
request_timeout_secs: u64,
) -> Result<Self, LlmError> {
Self::new_with_options(config, session, true, request_timeout_secs)
}
/// Create a chat completions provider with configurable tool-message flattening
/// and request timeout.
pub fn new_with_options(
config: NearAiConfig,
session: Arc<SessionManager>,
flatten_tool_messages: bool,
request_timeout_secs: u64,
) -> Result<Self, LlmError> {
let client = Client::builder()
.timeout(std::time::Duration::from_secs(120))
.timeout(std::time::Duration::from_secs(request_timeout_secs))
.build()
.map_err(|e| LlmError::RequestFailed {
provider: "nearai_chat".to_string(),
+62
View File
@@ -522,4 +522,66 @@ mod tests {
assert_eq!(messages[3].role, Role::User); // call_2 orphaned
assert_eq!(messages[4].role, Role::User); // call_3 orphaned
}
/// Regression: worker's select_tools/execute_plan now emit
/// assistant_with_tool_calls before tool_result messages.
/// Verify sanitize_tool_messages preserves all tool_results when
/// each has a matching assistant tool_call.
#[test]
fn test_sanitize_preserves_tool_results_with_matching_assistant() {
let tc1 = ToolCall {
id: "call_sel_1".to_string(),
name: "search".to_string(),
arguments: serde_json::json!({"q": "test"}),
};
let tc2 = ToolCall {
id: "call_sel_2".to_string(),
name: "http".to_string(),
arguments: serde_json::json!({"url": "https://example.com"}),
};
let mut messages = vec![
ChatMessage::system("You are a helpful assistant."),
ChatMessage::assistant_with_tool_calls(None, vec![tc1, tc2]),
ChatMessage::tool_result("call_sel_1", "search", "found 3 results"),
ChatMessage::tool_result("call_sel_2", "http", "200 OK"),
];
sanitize_tool_messages(&mut messages);
// All tool_results must keep Role::Tool -- none should be rewritten.
assert_eq!(messages[2].role, Role::Tool);
assert_eq!(messages[2].tool_call_id, Some("call_sel_1".to_string()));
assert_eq!(messages[2].content, "found 3 results");
assert_eq!(messages[3].role, Role::Tool);
assert_eq!(messages[3].tool_call_id, Some("call_sel_2".to_string()));
assert_eq!(messages[3].content, "200 OK");
}
/// Regression: the OLD buggy worker code pushed tool_result messages
/// without a preceding assistant_with_tool_calls, causing
/// sanitize_tool_messages to rewrite them as orphaned user messages.
/// This test reproduces that buggy sequence and confirms the rewrite.
#[test]
fn test_sanitize_rewrites_orphaned_tool_results() {
let mut messages = vec![
ChatMessage::system("You are a helpful assistant."),
// No assistant_with_tool_calls -- mimics the old bug.
ChatMessage::tool_result("call_bug_1", "search", "found 3 results"),
ChatMessage::tool_result("call_bug_2", "http", "200 OK"),
];
sanitize_tool_messages(&mut messages);
// Both tool_results must be rewritten to Role::User.
assert_eq!(messages[1].role, Role::User);
assert!(messages[1].content.contains("[Tool `search` returned:"));
assert!(messages[1].content.contains("found 3 results"));
assert!(messages[1].tool_call_id.is_none());
assert!(messages[1].name.is_none());
assert_eq!(messages[2].role, Role::User);
assert!(messages[2].content.contains("[Tool `http` returned:"));
assert!(messages[2].content.contains("200 OK"));
assert!(messages[2].tool_call_id.is_none());
assert!(messages[2].name.is_none());
}
}
+88 -15
View File
@@ -8,9 +8,9 @@ use serde::{Deserialize, Serialize};
use crate::error::LlmError;
use crate::llm::{
ChatMessage, CompletionRequest, LlmProvider, ToolCall, ToolCompletionRequest, ToolDefinition,
ChatMessage, CompletionRequest, LlmProvider, Role, ToolCall, ToolCompletionRequest,
ToolDefinition,
};
use crate::safety::SafetyLayer;
/// Token the agent returns when it has nothing to say (e.g. in group chats).
/// The dispatcher should check for this and suppress the message.
@@ -342,8 +342,6 @@ pub struct RespondOutput {
/// Reasoning engine for the agent.
pub struct Reasoning {
llm: Arc<dyn LlmProvider>,
#[allow(dead_code)] // Will be used for sanitizing tool outputs
safety: Arc<SafetyLayer>,
/// Optional workspace for loading identity/system prompts.
workspace_system_prompt: Option<String>,
/// Optional skill context block to inject into system prompt.
@@ -361,10 +359,9 @@ pub struct Reasoning {
impl Reasoning {
/// Create a new reasoning engine.
pub fn new(llm: Arc<dyn LlmProvider>, safety: Arc<SafetyLayer>) -> Self {
pub fn new(llm: Arc<dyn LlmProvider>) -> Self {
Self {
llm,
safety,
workspace_system_prompt: None,
skill_context: None,
channel: None,
@@ -460,8 +457,15 @@ impl Reasoning {
pub async fn plan(&self, context: &ReasoningContext) -> Result<ActionPlan, LlmError> {
let system_prompt = self.build_planning_prompt(context);
let system_prompt = merge_system_messages(system_prompt, &context.messages);
let mut messages = vec![ChatMessage::system(system_prompt)];
messages.extend(context.messages.clone());
messages.extend(
context
.messages
.iter()
.filter(|m| m.role != Role::System)
.cloned(),
);
if let Some(ref job) = context.job_description {
messages.push(ChatMessage::user(format!(
@@ -612,8 +616,15 @@ Respond in JSON format:
None => self.build_system_prompt_with_tools(&context.available_tools),
};
let system_prompt = merge_system_messages(system_prompt, &context.messages);
let mut messages = vec![ChatMessage::system(system_prompt)];
messages.extend(context.messages.clone());
messages.extend(
context
.messages
.iter()
.filter(|m| m.role != Role::System)
.cloned(),
);
let effective_tools = if context.force_text {
Vec::new()
@@ -1026,6 +1037,22 @@ pub struct SuccessEvaluation {
pub suggestions: Vec<String>,
}
/// Merge the reasoning method's system prompt with any system messages already
/// present in the conversation context. Strict LLM providers (e.g. Qwen)
/// reject conversations with system messages that are not at the very
/// beginning, so we concatenate all system content into a single prompt.
fn merge_system_messages(primary: String, context_messages: &[ChatMessage]) -> String {
let extra: Vec<&str> = context_messages
.iter()
.filter(|m| m.role == Role::System)
.map(|m| m.content.as_str())
.collect();
if extra.is_empty() {
return primary;
}
format!("{}\n\n---\n\n{}", primary, extra.join("\n\n"))
}
/// Extract JSON from text that might contain other content.
fn extract_json(text: &str) -> Option<&str> {
// Find the first { and last } to extract JSON
@@ -2086,15 +2113,9 @@ That's my plan."#;
// ---- System prompt building tests (issue #565) ----
fn make_test_reasoning() -> Reasoning {
use crate::config::SafetyConfig;
use crate::safety::SafetyLayer;
use crate::testing::StubLlm;
let llm = Arc::new(StubLlm::new("test"));
let safety = Arc::new(SafetyLayer::new(&SafetyConfig {
max_output_length: 100_000,
injection_check_enabled: false,
}));
Reasoning::new(llm, safety)
Reasoning::new(llm)
}
#[test]
@@ -2198,6 +2219,58 @@ That's my plan."#;
assert!(cleaned.contains("Here are the results."));
}
// ---- merge_system_messages: duplicate system message regression (Bug #597) ----
#[test]
fn test_merge_system_messages_no_system_in_context() {
let messages = vec![
ChatMessage::user("Hello"),
ChatMessage::assistant("Hi there"),
];
let result = merge_system_messages("primary prompt".into(), &messages);
assert_eq!(result, "primary prompt");
}
#[test]
fn test_merge_system_messages_merges_worker_system() {
let messages = vec![
ChatMessage::system("You are an autonomous agent working on a job.\n\nJob: Test Job"),
ChatMessage::user("Do the thing"),
];
let result = merge_system_messages("planning prompt".into(), &messages);
assert!(
result.contains("planning prompt"),
"must contain the primary prompt"
);
assert!(
result.contains("autonomous agent"),
"must contain worker system text"
);
assert!(
result.contains("Test Job"),
"must contain job description from worker system message"
);
}
#[test]
fn test_merge_system_messages_multiple_system() {
let messages = vec![
ChatMessage::system("First system instruction"),
ChatMessage::system("Second system instruction"),
ChatMessage::user("Hello"),
];
let result = merge_system_messages("primary".into(), &messages);
assert!(result.contains("primary"), "must contain primary prompt");
assert!(
result.contains("First system instruction"),
"must contain first system message"
);
assert!(
result.contains("Second system instruction"),
"must contain second system message"
);
}
#[test]
fn test_system_prompt_without_tools_omits_tools_section() {
let reasoning = make_test_reasoning();
+2
View File
@@ -450,6 +450,8 @@ mod tests {
if def.protocol == ProviderProtocol::OpenAiCompletions
&& def.id != "openai"
&& def.id != "openai_compatible"
&& def.id != "bedrock"
&& def.id != "cloudflare"
{
assert!(
def.default_base_url.is_some(),
+2 -1
View File
@@ -200,9 +200,10 @@ impl SessionManager {
let status = response.status();
let body = response.text().await.unwrap_or_default();
let preview = crate::agent::truncate_for_preview(&body, 200);
Err(LlmError::SessionRenewalFailed {
provider: "nearai".to_string(),
reason: format!("Validation failed: HTTP {}: {}", status, body),
reason: format!("Validation failed: HTTP {status}: {preview}"),
})
}
+36 -9
View File
@@ -145,6 +145,24 @@ async fn async_main() -> anyhow::Result<()> {
}
}
// ── PID lock (prevent multiple instances) ────────────────────────
let _pid_lock = match ironclaw::bootstrap::PidLock::acquire() {
Ok(lock) => Some(lock),
Err(ironclaw::bootstrap::PidLockError::AlreadyRunning { pid }) => {
anyhow::bail!(
"Another IronClaw instance is already running (PID {}). \
If this is incorrect, remove the stale PID file: {}",
pid,
ironclaw::bootstrap::pid_lock_path().display()
);
}
Err(e) => {
eprintln!("Warning: Could not acquire PID lock: {}", e);
eprintln!("Continuing without PID lock protection.");
None
}
};
// ── Agent startup ──────────────────────────────────────────────────
// Enhanced first-run detection
@@ -158,18 +176,20 @@ async fn async_main() -> anyhow::Result<()> {
wizard.run().await?;
}
// Load initial config from env + disk + optional TOML (before DB is available)
// Load initial config from env + disk + optional TOML (before DB is available).
// Credentials may be missing at this point — that's fine. LlmConfig::resolve()
// defers gracefully, and AppBuilder::build_all() re-resolves after loading
// secrets from the encrypted DB.
let toml_path = cli.config.as_deref();
let config = match Config::from_env_with_toml(toml_path).await {
Ok(c) => c,
Err(ironclaw::error::ConfigError::MissingRequired { key, hint }) => {
eprintln!("Configuration error: Missing required setting '{}'", key);
eprintln!(" {}", hint);
eprintln!();
eprintln!(
"Run 'ironclaw onboard' to configure, or set the required environment variables."
anyhow::bail!(
"Configuration error: Missing required setting '{}'. {}. \
Run 'ironclaw onboard' to configure, or set the required environment variables.",
key,
hint
);
std::process::exit(1);
}
Err(e) => return Err(e.into()),
};
@@ -475,6 +495,7 @@ async fn async_main() -> anyhow::Result<()> {
let mut sse_sender: Option<
tokio::sync::broadcast::Sender<ironclaw::channels::web::types::SseEvent>,
> = None;
let mut routine_engine_slot: Option<ironclaw::channels::web::server::RoutineEngineSlot> = None;
if let Some(ref gw_config) = config.channels.gateway {
let mut gw =
GatewayChannel::new(gw_config.clone()).with_llm_provider(Arc::clone(&components.llm));
@@ -528,10 +549,11 @@ async fn async_main() -> anyhow::Result<()> {
tracing::info!("Web UI: http://{}:{}/", gw_config.host, gw_config.port);
// Capture SSE sender before moving gw into channels.
// Capture SSE sender and routine engine slot before moving gw into channels.
// IMPORTANT: This must come after all `with_*` calls since `rebuild_state`
// creates a new SseManager, which would orphan this sender.
sse_sender = Some(gw.state().sse.sender());
routine_engine_slot = Some(Arc::clone(&gw.state().routine_engine));
channel_names.push("gateway".to_string());
channels.add(Box::new(gw)).await;
@@ -678,7 +700,7 @@ async fn async_main() -> anyhow::Result<()> {
)),
};
let agent = Agent::new(
let mut agent = Agent::new(
config.agent.clone(),
deps,
channels,
@@ -692,6 +714,11 @@ async fn async_main() -> anyhow::Result<()> {
// Fill the scheduler slot now that Agent (and its Scheduler) exist.
*scheduler_slot.write().await = Some(agent.scheduler());
// Give the agent the routine engine slot so it can expose the engine to the gateway.
if let Some(slot) = routine_engine_slot {
agent.set_routine_engine_slot(slot);
}
agent.run().await?;
// ── Shutdown ────────────────────────────────────────────────────────
+58
View File
@@ -42,6 +42,10 @@ pub struct Settings {
#[serde(default)]
pub secrets_master_key_source: KeySource,
/// Generated master key hex (env var mode only, written to .env by wizard).
#[serde(default, skip_serializing)]
pub secrets_master_key_hex: Option<String>,
// === Step 3: Inference Provider ===
/// LLM backend: "nearai", "anthropic", "openai", "ollama", "openai_compatible".
#[serde(default)]
@@ -291,6 +295,18 @@ pub struct HeartbeatSettings {
/// User ID to notify on heartbeat findings.
#[serde(default)]
pub notify_user: Option<String>,
/// Hour (0-23) when quiet hours start (heartbeat skipped).
#[serde(default)]
pub quiet_hours_start: Option<u32>,
/// Hour (0-23) when quiet hours end (heartbeat resumes).
#[serde(default)]
pub quiet_hours_end: Option<u32>,
/// Timezone for quiet hours evaluation (IANA name, e.g. "America/New_York").
#[serde(default)]
pub timezone: Option<String>,
}
fn default_heartbeat_interval() -> u64 {
@@ -304,6 +320,9 @@ impl Default for HeartbeatSettings {
interval_secs: default_heartbeat_interval(),
notify_channel: None,
notify_user: None,
quiet_hours_start: None,
quiet_hours_end: None,
timezone: None,
}
}
}
@@ -351,6 +370,10 @@ pub struct AgentSettings {
/// When true, skip tool approval checks entirely. For benchmarks/CI.
#[serde(default)]
pub auto_approve_tools: bool,
/// Default timezone for new sessions (IANA name, e.g. "America/New_York").
#[serde(default = "default_timezone")]
pub default_timezone: String,
}
fn default_agent_name() -> String {
@@ -385,6 +408,10 @@ fn default_max_tool_iterations() -> usize {
50
}
fn default_timezone() -> String {
"UTC".to_string()
}
fn default_true() -> bool {
true
}
@@ -402,6 +429,7 @@ impl Default for AgentSettings {
session_idle_timeout_secs: default_session_idle_timeout(),
max_tool_iterations: default_max_tool_iterations(),
auto_approve_tools: false,
default_timezone: default_timezone(),
}
}
}
@@ -498,6 +526,10 @@ pub struct SandboxSettings {
/// Additional domains to allow through the network proxy.
#[serde(default)]
pub extra_allowed_domains: Vec<String>,
/// Whether Claude Code sandbox mode is enabled.
#[serde(default)]
pub claude_code_enabled: bool,
}
fn default_sandbox_policy() -> String {
@@ -531,6 +563,7 @@ impl Default for SandboxSettings {
image: default_sandbox_image(),
auto_pull_image: true,
extra_allowed_domains: Vec::new(),
claude_code_enabled: false,
}
}
}
@@ -1169,6 +1202,31 @@ mod tests {
assert_eq!(loaded.heartbeat.interval_secs, 900);
}
/// Regression test: /model command must persist selected_model to TOML config.
/// Prior to the fix, `set_model()` only changed the in-memory provider and the
/// choice was lost on restart.
#[test]
fn toml_selected_model_update_persists() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("config.toml");
// Start with a config that has a different model.
let settings = Settings {
selected_model: Some("old-model".to_string()),
..Default::default()
};
settings.save_toml(&path).unwrap();
// Simulate what persist_selected_model does: load, update, save.
let mut loaded = Settings::load_toml(&path).unwrap().unwrap();
loaded.selected_model = Some("new-model".to_string());
loaded.save_toml(&path).unwrap();
// Verify the change survived a reload.
let reloaded = Settings::load_toml(&path).unwrap().unwrap();
assert_eq!(reloaded.selected_model, Some("new-model".to_string()));
}
#[test]
fn toml_missing_file_returns_none() {
let result = Settings::load_toml(std::path::Path::new("/tmp/nonexistent_config.toml"));
+2 -6
View File
@@ -152,12 +152,8 @@ This is OS-level behavior we cannot prevent. To minimize pain:
rather than triggering system dialogs.
**Invariant:** After Step 2, `self.secrets_crypto` is `Some` if the user
chose Keychain or env-var mode (both generate a key and initialize crypto
immediately). It is `None` only if the user skipped secrets.
When env-var mode is chosen, the generated key is also stored in
`self.secrets_master_key_hex` so that `write_bootstrap_env()` can persist
it to `~/.ironclaw/.env` automatically.
chose Keychain or generated a new key. It may be `None` if the user chose
env-var mode or skipped secrets.
---
+307 -60
View File
@@ -22,6 +22,7 @@ use crate::bootstrap::ironclaw_base_dir;
use crate::channels::wasm::{
ChannelCapabilitiesFile, available_channel_names, install_bundled_channel,
};
use crate::config::llm::OAUTH_PLACEHOLDER;
use crate::llm::{SessionConfig, SessionManager};
use crate::secrets::{SecretsCrypto, SecretsStore};
use crate::settings::{KeySource, Settings};
@@ -90,8 +91,6 @@ pub struct SetupWizard {
db_backend: Option<crate::db::libsql::LibSqlBackend>,
/// Secrets crypto (created during setup).
secrets_crypto: Option<Arc<SecretsCrypto>>,
/// Generated master key hex (stored for writing to .env in env-var mode).
secrets_master_key_hex: Option<String>,
/// Cached API key from provider setup (used by model fetcher without env mutation).
llm_api_key: Option<SecretString>,
}
@@ -108,7 +107,6 @@ impl SetupWizard {
#[cfg(feature = "libsql")]
db_backend: None,
secrets_crypto: None,
secrets_master_key_hex: None,
llm_api_key: None,
}
}
@@ -124,7 +122,6 @@ impl SetupWizard {
#[cfg(feature = "libsql")]
db_backend: None,
secrets_crypto: None,
secrets_master_key_hex: None,
llm_api_key: None,
}
}
@@ -772,25 +769,31 @@ impl SetupWizard {
print_success("Master key generated and stored in OS keychain");
}
1 => {
// Env var mode: generate key, initialize crypto, and persist to .env
print_info("Generating master key...");
// Env var mode generate key, init crypto, and persist to .env
let key_hex = crate::secrets::keychain::generate_master_key_hex();
// Initialize crypto so subsequent steps (API key storage) work
// Initialize crypto so subsequent wizard steps (channel setup,
// API key storage) can encrypt secrets immediately.
self.secrets_crypto = Some(Arc::new(
SecretsCrypto::new(SecretString::from(key_hex.clone()))
.map_err(|e| SetupError::Config(e.to_string()))?,
));
// Store for write_bootstrap_env to persist to ~/.ironclaw/.env
self.secrets_master_key_hex = Some(key_hex.clone());
// Make visible to optional_env() for any subsequent config resolution.
crate::config::inject_single_var("SECRETS_MASTER_KEY", &key_hex);
// Store hex for write_bootstrap_env to persist to ~/.ironclaw/.env.
self.settings.secrets_master_key_hex = Some(key_hex.clone());
println!();
print_info(&format!("Generated master key: {}", mask_api_key(&key_hex)));
print_info("This key will be saved to ~/.ironclaw/.env automatically.");
print_info("Master key generated and will be saved to ~/.ironclaw/.env");
println!();
println!(" SECRETS_MASTER_KEY={}", key_hex);
println!();
print_info("You can also copy this to another .env file or CI secrets.");
self.settings.secrets_master_key_source = KeySource::Env;
print_success("Master key generated and configured for environment variable");
print_success("Configured for environment variable");
}
_ => {
self.settings.secrets_master_key_source = KeySource::None;
@@ -899,6 +902,11 @@ impl SetupWizard {
return Ok(());
};
// Anthropic has a custom flow: API key or OAuth token from `claude login`.
if provider_id == "anthropic" {
return self.setup_anthropic().await;
}
match setup {
crate::llm::registry::SetupHint::ApiKey {
secret_name,
@@ -1004,6 +1012,113 @@ impl SetupWizard {
Ok(())
}
/// Anthropic provider setup: API key or OAuth token from `claude login`.
async fn setup_anthropic(&mut self) -> Result<(), SetupError> {
let options = &["Direct API Key", "OAuth Token (from `claude login`)"];
let choice = select_one("How do you want to authenticate with Anthropic?", options)
.map_err(SetupError::Io)?;
if choice == 0 {
// Standard API key flow
self.setup_api_key_provider(
"anthropic",
"ANTHROPIC_API_KEY",
"llm_anthropic_api_key",
"Anthropic API key",
"https://console.anthropic.com/settings/keys",
None,
)
.await
} else {
// OAuth token flow
self.setup_anthropic_oauth().await
}
}
/// Anthropic OAuth setup: extract token from `claude login` credentials.
async fn setup_anthropic_oauth(&mut self) -> Result<(), SetupError> {
// Clear model only when switching providers (old model may be invalid)
if self.settings.llm_backend.as_deref() != Some("anthropic") {
self.settings.selected_model = None;
}
self.settings.llm_backend = Some("anthropic".to_string());
// Try to extract existing OAuth token from Claude Code credentials
if let Some(token) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
print_info(&format!("Found OAuth token: {}", mask_api_key(&token)));
if confirm("Use this token?", true).map_err(SetupError::Io)? {
return self.save_anthropic_oauth_token(&token).await;
}
} else {
print_info("No OAuth token found from `claude login`.");
print_info("Run `claude login` in a terminal to authenticate, then retry.");
println!();
if confirm("Retry after running `claude login`?", true).map_err(SetupError::Io)? {
// Block until the user has run `claude login` in another terminal
input("Press Enter after running `claude login` in another terminal...")
.map_err(SetupError::Io)?;
if let Some(token) = crate::config::ClaudeCodeConfig::extract_oauth_token() {
print_info(&format!("Found OAuth token: {}", mask_api_key(&token)));
return self.save_anthropic_oauth_token(&token).await;
}
print_error("Still no OAuth token found.");
}
}
// Fallback: let user paste the token manually, or switch to API key
print_info("You can paste your OAuth token directly (starts with sk-ant-oat01-).");
print_info("Or press Enter with no input to switch to the API key flow.");
let token = secret_input("Anthropic OAuth token").map_err(SetupError::Io)?;
let token_str = token.expose_secret();
if token_str.is_empty() {
print_info("Switching to API key flow...");
return self
.setup_api_key_provider(
"anthropic",
"ANTHROPIC_API_KEY",
"llm_anthropic_api_key",
"Anthropic API key",
"https://console.anthropic.com/settings/keys",
None,
)
.await;
}
self.save_anthropic_oauth_token(token_str).await
}
/// Save an Anthropic OAuth token to secrets and set env for immediate use.
async fn save_anthropic_oauth_token(&mut self, token: &str) -> Result<(), SetupError> {
// Validate token format to catch accidentally pasted API keys
if !token.starts_with("sk-ant-oat") {
print_error("Token doesn't look like an OAuth token (expected prefix: sk-ant-oat).");
print_info("If you have an API key instead, use the 'Direct API Key' option.");
return Err(SetupError::Config("Invalid OAuth token format".to_string()));
}
// Store in secrets if available
if let Ok(ctx) = self.init_secrets_context().await {
let key = SecretString::from(token.to_string());
ctx.save_secret("llm_anthropic_oauth_token", &key)
.await
.map_err(|e| SetupError::Config(format!("Failed to save OAuth token: {e}")))?;
print_success("OAuth token encrypted and saved");
} else {
print_info("Secrets not available. Set ANTHROPIC_OAUTH_TOKEN in your environment.");
}
// Make the token visible to `optional_env()` for subsequent config
// resolution (model selection step). Uses the thread-safe overlay
// instead of `std::env::set_var` to avoid UB on multi-threaded runtimes.
crate::config::inject_single_var("ANTHROPIC_OAUTH_TOKEN", token);
// Cache for model fetching
self.llm_api_key = Some(SecretString::from(token.to_string()));
print_success("Anthropic OAuth configured");
Ok(())
}
/// Shared setup flow for API-key-based providers.
async fn setup_api_key_provider(
&mut self,
@@ -1020,10 +1135,11 @@ impl SetupWizard {
other => other,
});
self.settings.llm_backend = Some(backend.to_string());
if self.settings.selected_model.is_some() {
// Clear model only when switching providers (old model may be invalid)
if self.settings.llm_backend.as_deref() != Some(backend) {
self.settings.selected_model = None;
}
self.settings.llm_backend = Some(backend.to_string());
// Check env var first
if let Ok(existing) = std::env::var(env_var) {
@@ -1065,6 +1181,11 @@ impl SetupWizard {
));
}
// Make key visible to `optional_env()` for subsequent config resolution.
// Uses the thread-safe overlay instead of `std::env::set_var` to avoid
// UB on multi-threaded runtimes.
crate::config::inject_single_var(env_var, key_str);
// Cache key in memory for model fetching later in the wizard
self.llm_api_key = Some(SecretString::from(key_str.to_string()));
@@ -1077,10 +1198,11 @@ impl SetupWizard {
&mut self,
def: &crate::llm::ProviderDefinition,
) -> Result<(), SetupError> {
self.settings.llm_backend = Some(def.id.clone());
if self.settings.selected_model.is_some() {
// Clear model only when switching providers (old model may be invalid)
if self.settings.llm_backend.as_deref() != Some(&def.id) {
self.settings.selected_model = None;
}
self.settings.llm_backend = Some(def.id.clone());
let default_url = self
.settings
@@ -1115,10 +1237,11 @@ impl SetupWizard {
secret_name: &str,
display_name: &str,
) -> Result<(), SetupError> {
self.settings.llm_backend = Some(backend_id.to_string());
if self.settings.selected_model.is_some() {
// Clear model only when switching providers (old model may be invalid)
if self.settings.llm_backend.as_deref() != Some(backend_id) {
self.settings.selected_model = None;
}
self.settings.llm_backend = Some(backend_id.to_string());
let existing_url = self
.settings
@@ -1372,6 +1495,7 @@ impl SetupWizard {
smart_routing_cascade: true,
},
provider: None,
request_timeout_secs: 120,
};
match create_llm_provider(&config, session) {
@@ -2001,6 +2125,67 @@ impl SetupWizard {
}
}
// Claude Code sandbox sub-step (only if Docker sandbox is enabled)
if self.settings.sandbox.enabled {
self.step_claude_code_sandbox().await?;
}
Ok(())
}
/// Claude Code sandbox sub-step: enable Claude CLI inside Docker containers.
async fn step_claude_code_sandbox(&mut self) -> Result<(), SetupError> {
println!();
print_info("Claude Code mode lets the agent delegate complex tasks to Claude CLI");
print_info("running inside sandboxed Docker containers.");
println!();
if !confirm("Enable Claude Code sandbox mode?", false).map_err(SetupError::Io)? {
self.settings.sandbox.claude_code_enabled = false;
return Ok(());
}
// Check for Anthropic credentials (API key or OAuth token).
// Uses `optional_env()` which reads both real env vars and the
// injected overlay (secrets DB, wizard-set values).
let has_credentials = || {
let has_api_key = crate::config::helpers::optional_env("ANTHROPIC_API_KEY")
.ok()
.flatten()
.is_some_and(|v| !v.is_empty() && v != OAUTH_PLACEHOLDER);
let has_oauth = crate::config::ClaudeCodeConfig::extract_oauth_token().is_some()
|| crate::config::helpers::optional_env("ANTHROPIC_OAUTH_TOKEN")
.ok()
.flatten()
.is_some_and(|v| !v.is_empty());
has_api_key || has_oauth
};
if has_credentials() {
self.settings.sandbox.claude_code_enabled = true;
print_success("Claude Code sandbox enabled");
} else {
print_error("No Anthropic credentials found.");
print_info(
"Claude Code needs ANTHROPIC_API_KEY or an OAuth token from `claude login`.",
);
println!();
if confirm("Retry after setting up credentials?", false).map_err(SetupError::Io)? {
if has_credentials() {
self.settings.sandbox.claude_code_enabled = true;
print_success("Claude Code sandbox enabled");
} else {
self.settings.sandbox.claude_code_enabled = false;
print_info("No credentials found. Claude Code disabled for now.");
print_info("Set ANTHROPIC_API_KEY or run `claude login` and enable later.");
}
} else {
self.settings.sandbox.claude_code_enabled = false;
print_info("Claude Code disabled. Enable with CLAUDE_CODE_ENABLED=true later.");
}
}
Ok(())
}
@@ -2094,6 +2279,12 @@ impl SetupWizard {
///
/// These are the chicken-and-egg settings needed before the database is
/// connected (DATABASE_BACKEND, DATABASE_URL, LLM_BACKEND, etc.).
///
/// **Credentials are NOT written here.** API keys and OAuth tokens live
/// only in the encrypted secrets DB. `LlmConfig::resolve()` defers
/// gracefully when credentials are missing during early startup, and the
/// re-resolution in `AppBuilder::build_all()` fills them in after
/// `inject_llm_keys_from_secrets()` loads from encrypted storage.
fn write_bootstrap_env(&self) -> Result<(), SetupError> {
let registry = crate::llm::ProviderRegistry::load();
let mut env_vars: Vec<(String, String)> = Vec::new();
@@ -2146,13 +2337,6 @@ impl SetupWizard {
env_vars.push((base_url_env.clone(), base_url.clone()));
}
// Persist SECRETS_MASTER_KEY when env-var mode was chosen in step 2
if self.settings.secrets_master_key_source == KeySource::Env
&& let Some(ref key_hex) = self.secrets_master_key_hex
{
env_vars.push(("SECRETS_MASTER_KEY".to_string(), key_hex.clone()));
}
// Preserve NEARAI_API_KEY if present (set by API key auth flow)
if let Ok(api_key) = std::env::var("NEARAI_API_KEY")
&& !api_key.is_empty()
@@ -2160,12 +2344,23 @@ impl SetupWizard {
env_vars.push(("NEARAI_API_KEY".to_string(), api_key));
}
// Secrets master key (env var mode): write to .env so it's available
// on next startup before the DB is connected.
if let Some(ref key_hex) = self.settings.secrets_master_key_hex {
env_vars.push(("SECRETS_MASTER_KEY".to_string(), key_hex.clone()));
}
// Always write ONBOARD_COMPLETED so that check_onboard_needed()
// (which runs before the DB is connected) knows to skip re-onboarding.
if self.settings.onboard_completed {
env_vars.push(("ONBOARD_COMPLETED".to_string(), "true".to_string()));
}
// Claude Code sandbox mode
if self.settings.sandbox.claude_code_enabled {
env_vars.push(("CLAUDE_CODE_ENABLED".to_string(), "true".to_string()));
}
// Signal channel env vars (chicken-and-egg: config resolves before DB).
if let Some(ref url) = self.settings.channels.signal_http_url {
env_vars.push(("SIGNAL_HTTP_URL".to_string(), url.clone()));
@@ -2533,22 +2728,39 @@ async fn fetch_anthropic_models(cached_key: Option<&str>) -> Vec<(String, String
let api_key = cached_key
.map(String::from)
.or_else(|| std::env::var("ANTHROPIC_API_KEY").ok())
.filter(|k| !k.is_empty());
.filter(|k| !k.is_empty() && k != crate::config::llm::OAUTH_PLACEHOLDER);
let api_key = match api_key {
Some(k) => k,
None => return static_defaults,
// Fall back to OAuth token if no API key
let oauth_token = if api_key.is_none() {
crate::config::helpers::optional_env("ANTHROPIC_OAUTH_TOKEN")
.ok()
.flatten()
.filter(|t| !t.is_empty())
} else {
None
};
let (key_or_token, is_oauth) = match (api_key, oauth_token) {
(Some(k), _) => (k, false),
(None, Some(t)) => (t, true),
(None, None) => return static_defaults,
};
let client = reqwest::Client::new();
let resp = match client
let mut request = client
.get("https://api.anthropic.com/v1/models")
.header("x-api-key", &api_key)
.header("anthropic-version", "2023-06-01")
.timeout(std::time::Duration::from_secs(5))
.send()
.await
{
.timeout(std::time::Duration::from_secs(5));
if is_oauth {
request = request
.bearer_auth(&key_or_token)
.header("anthropic-beta", "oauth-2025-04-20");
} else {
request = request.header("x-api-key", &key_or_token);
}
let resp = match request.send().await {
Ok(r) if r.status().is_success() => r,
_ => return static_defaults,
};
@@ -3313,36 +3525,45 @@ mod tests {
}
}
/// Regression test for #666: env var mode in step_security must initialize
/// secrets_crypto (for immediate API key storage) and secrets_master_key_hex
/// (for persisting to ~/.ironclaw/.env via write_bootstrap_env).
/// Regression test for #600: re-running provider setup for the same backend
/// must NOT clear selected_model. Only switching to a different backend should.
#[test]
fn test_env_var_mode_initializes_crypto_and_stores_key() {
fn test_same_provider_preserves_selected_model() {
let mut wizard = SetupWizard::new();
assert!(wizard.secrets_crypto.is_none());
assert!(wizard.secrets_master_key_hex.is_none());
wizard.settings.llm_backend = Some("ollama".to_string());
wizard.settings.selected_model = Some("llama3".to_string());
// Simulate the env-var branch of step_security
let key_hex = crate::secrets::keychain::generate_master_key_hex();
// Simulate re-entering the same provider -- model should survive
// (This is the check that each setup_* function now performs)
if wizard.settings.llm_backend.as_deref() != Some("ollama") {
wizard.settings.selected_model = None;
}
wizard.settings.llm_backend = Some("ollama".to_string());
// Verify it's a valid 64-char hex string (32 bytes = AES-256)
assert_eq!(key_hex.len(), 64);
assert!(key_hex.chars().all(|c| c.is_ascii_hexdigit()));
let crypto = SecretsCrypto::new(SecretString::from(key_hex.clone()))
.expect("SecretsCrypto::new should succeed with generated hex key");
wizard.secrets_crypto = Some(Arc::new(crypto));
wizard.secrets_master_key_hex = Some(key_hex.clone());
wizard.settings.secrets_master_key_source = KeySource::Env;
// Verify crypto is usable for immediate secret encryption
assert!(wizard.secrets_crypto.is_some());
// Verify the hex key is stored for write_bootstrap_env to persist
assert_eq!(
wizard.secrets_master_key_hex.as_deref(),
Some(key_hex.as_str())
wizard.settings.selected_model.as_deref(),
Some("llama3"),
"model should be preserved when re-selecting the same provider"
);
}
/// Regression test for #600: switching to a different provider must clear
/// selected_model since the old model may not be valid for the new backend.
#[test]
fn test_different_provider_clears_selected_model() {
let mut wizard = SetupWizard::new();
wizard.settings.llm_backend = Some("ollama".to_string());
wizard.settings.selected_model = Some("llama3".to_string());
// Simulate switching to a different provider -- model should be cleared
if wizard.settings.llm_backend.as_deref() != Some("openai") {
wizard.settings.selected_model = None;
}
wizard.settings.llm_backend = Some("openai".to_string());
assert!(
wizard.settings.selected_model.is_none(),
"model should be cleared when switching providers"
);
}
@@ -3383,4 +3604,30 @@ mod tests {
"backend should be set even without setup hint"
);
}
/// Regression test for #666: env-var security option must initialize
/// secrets_crypto so subsequent steps can encrypt API keys.
#[test]
fn test_env_var_security_initializes_crypto() {
use crate::secrets::SecretsCrypto;
use secrecy::SecretString;
// Simulate what option 1 in step_security() does after the fix:
let key_hex = crate::secrets::keychain::generate_master_key_hex();
// The fix: create SecretsCrypto from the generated key.
// Before the fix, this was skipped, leaving secrets_crypto = None.
let crypto = SecretsCrypto::new(SecretString::from(key_hex.clone()));
assert!(
crypto.is_ok(),
"generated key hex must produce valid SecretsCrypto"
);
// Verify the key is stored for bootstrap env persistence.
let settings = Settings {
secrets_master_key_hex: Some(key_hex),
..Settings::default()
};
assert!(settings.secrets_master_key_hex.is_some());
}
}
+1
View File
@@ -1009,6 +1009,7 @@ mod tests {
enabled: true,
trigger: Trigger::Cron {
schedule: "0 * * * *".to_string(),
timezone: None,
},
action: RoutineAction::Lightweight {
prompt: "Check status".to_string(),
+110
View File
@@ -0,0 +1,110 @@
//! Timezone resolution and utilities.
use chrono::{DateTime, NaiveDate, Utc};
use chrono_tz::Tz;
/// Resolve the effective timezone from a priority chain.
///
/// Priority: client_tz > user_setting > config_default > UTC
pub fn resolve_timezone(
client_tz: Option<&str>,
user_setting: Option<&str>,
config_default: &str,
) -> Tz {
// Try each in priority order, skipping invalid values
for candidate in [client_tz, user_setting, Some(config_default)] {
if let Some(tz) = candidate.and_then(parse_timezone) {
return tz;
}
}
Tz::UTC
}
/// Parse a timezone string (IANA name) into a `Tz`.
pub fn parse_timezone(s: &str) -> Option<Tz> {
s.parse::<Tz>().ok()
}
/// Get today's date in the given timezone.
pub fn today_in_tz(tz: Tz) -> NaiveDate {
Utc::now().with_timezone(&tz).date_naive()
}
/// Get the current time in the given timezone.
pub fn now_in_tz(tz: Tz) -> DateTime<Tz> {
Utc::now().with_timezone(&tz)
}
/// Detect the system's timezone, falling back to UTC.
pub fn detect_system_timezone() -> Tz {
iana_time_zone::get_timezone()
.ok()
.and_then(|s| parse_timezone(&s))
.unwrap_or(Tz::UTC)
}
#[cfg(test)]
mod tests {
use chrono::Datelike;
use super::*;
#[test]
fn test_resolve_client_wins() {
let tz = resolve_timezone(Some("America/New_York"), Some("Europe/London"), "UTC");
assert_eq!(tz, chrono_tz::America::New_York);
}
#[test]
fn test_resolve_user_setting_fallback() {
let tz = resolve_timezone(None, Some("Europe/London"), "UTC");
assert_eq!(tz, chrono_tz::Europe::London);
}
#[test]
fn test_resolve_config_fallback() {
let tz = resolve_timezone(None, None, "Asia/Tokyo");
assert_eq!(tz, chrono_tz::Asia::Tokyo);
}
#[test]
fn test_resolve_all_none_utc() {
let tz = resolve_timezone(None, None, "UTC");
assert_eq!(tz, Tz::UTC);
}
#[test]
fn test_resolve_invalid_client_skipped() {
let tz = resolve_timezone(Some("Fake/Zone"), Some("Europe/London"), "UTC");
assert_eq!(tz, chrono_tz::Europe::London);
}
#[test]
fn test_parse_valid() {
assert_eq!(
parse_timezone("America/Chicago"),
Some(chrono_tz::America::Chicago)
);
}
#[test]
fn test_parse_invalid() {
assert_eq!(parse_timezone("Fake/Zone"), None);
}
#[test]
fn test_detect_system_tz() {
// Should always return a valid Tz (at minimum UTC)
let tz = detect_system_timezone();
let _ = now_in_tz(tz); // Should not panic
}
#[test]
fn test_today_in_tz_returns_valid_date() {
let date = today_in_tz(Tz::UTC);
// Verify it returns a valid date (year, month, day are all positive)
assert!(date.year() > 0);
assert!((1..=12).contains(&date.month()));
assert!((1..=31).contains(&date.day()));
}
}
+4 -16
View File
@@ -43,7 +43,6 @@ use crate::error::ToolError as AgentToolError;
use crate::llm::{
ChatMessage, LlmProvider, Reasoning, ReasoningContext, RespondResult, ToolDefinition,
};
use crate::safety::SafetyLayer;
use crate::tools::ToolRegistry;
use crate::tools::tool::{ApprovalRequirement, Tool, ToolError, ToolOutput};
@@ -251,29 +250,18 @@ pub trait SoftwareBuilder: Send + Sync {
pub struct LlmSoftwareBuilder {
config: BuilderConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
tools: Arc<ToolRegistry>,
}
impl LlmSoftwareBuilder {
/// Create a new LLM-based software builder.
pub fn new(
config: BuilderConfig,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
tools: Arc<ToolRegistry>,
) -> Self {
pub fn new(config: BuilderConfig, llm: Arc<dyn LlmProvider>, tools: Arc<ToolRegistry>) -> Self {
// Ensure build directory exists
if let Err(e) = std::fs::create_dir_all(&config.build_dir) {
tracing::warn!("Failed to create build directory: {}", e);
}
Self {
config,
llm,
safety,
tools,
}
Self { config, llm, tools }
}
/// Get the build tools available for the build loop.
@@ -521,7 +509,7 @@ Create alongside the .wasm file to grant capabilities:
let mut iteration = 0;
// Create reasoning engine
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let reasoning = Reasoning::new(self.llm.clone());
// Build initial context
let tool_defs = self.get_build_tools().await;
@@ -822,7 +810,7 @@ Create alongside the .wasm file to grant capabilities:
impl SoftwareBuilder for LlmSoftwareBuilder {
async fn analyze(&self, description: &str) -> Result<BuildRequirement, AgentToolError> {
// Use LLM to parse the description
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let reasoning = Reasoning::new(self.llm.clone());
let prompt = format!(
r#"Analyze this software requirement and extract structured information.
+5 -4
View File
@@ -172,7 +172,7 @@ impl Tool for MemoryWriteTool {
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
@@ -239,11 +239,12 @@ impl Tool for MemoryWriteTool {
paths::MEMORY.to_string()
}
"daily_log" => {
let tz = crate::timezone::parse_timezone(&ctx.user_timezone)
.unwrap_or(chrono_tz::Tz::UTC);
self.workspace
.append_daily_log(content)
.append_daily_log_tz(content, tz)
.await
.map_err(|e| ToolError::ExecutionFailed(format!("Write failed: {}", e)))?;
format!("daily/{}.md", chrono::Utc::now().format("%Y-%m-%d"))
.map_err(|e| ToolError::ExecutionFailed(format!("Write failed: {}", e)))?
}
"heartbeat" => {
if append {
+184 -50
View File
@@ -105,42 +105,47 @@ impl Tool for MessageTool {
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
let content = require_str(&params, "content")?;
// Get channel: use param or fall back to default
let channel = if let Some(c) = params.get("channel").and_then(|v| v.as_str()) {
c.to_string()
} else {
self.default_channel
// Get channel: use param → conversation default → job metadata → None (broadcast all)
let channel: Option<String> =
if let Some(c) = params.get("channel").and_then(|v| v.as_str()) {
Some(c.to_string())
} else if let Some(c) = self
.default_channel
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
.ok_or_else(|| {
ToolError::ExecutionFailed(
"No channel specified and no active conversation. Provide channel parameter."
.to_string(),
)
})?
};
{
Some(c)
} else {
ctx.metadata
.get("notify_channel")
.and_then(|v| v.as_str())
.map(|c| c.to_string())
};
// Get target: use param or fall back to default
// Get target: use param → conversation default → job metadata
let target = if let Some(t) = params.get("target").and_then(|v| v.as_str()) {
t.to_string()
} else if let Some(t) = self
.default_target
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
{
t
} else if let Some(t) = ctx.metadata.get("notify_user").and_then(|v| v.as_str()) {
t.to_string()
} else {
self.default_target
.read()
.unwrap_or_else(|e| e.into_inner())
.clone()
.ok_or_else(|| {
ToolError::ExecutionFailed(
"No target specified and no active conversation. Provide target parameter."
.to_string(),
)
})?
return Err(ToolError::ExecutionFailed(
"No target specified and no active conversation. Provide target parameter."
.to_string(),
));
};
let attachments: Vec<String> = match params.get("attachments") {
@@ -181,36 +186,79 @@ impl Tool for MessageTool {
response = response.with_attachments(attachments);
}
match self
.channel_manager
.broadcast(&channel, &target, response)
.await
{
Ok(()) => {
tracing::info!(
message_sent = true,
channel = %channel,
target = %target,
attachments = attachment_count,
"Message sent via message tool"
);
let msg = format!("Sent message to {}:{}", channel, target);
Ok(ToolOutput::text(msg, start.elapsed()))
if let Some(ref channel) = channel {
// Send to a specific channel
match self
.channel_manager
.broadcast(channel, &target, response)
.await
{
Ok(()) => {
tracing::info!(
message_sent = true,
channel = %channel,
target = %target,
attachments = attachment_count,
"Message sent via message tool"
);
let msg = format!("Sent message to {}:{}", channel, target);
Ok(ToolOutput::text(msg, start.elapsed()))
}
Err(e) => {
let available = self.channel_manager.channel_names().await.join(", ");
let err_msg = if available.is_empty() {
format!(
"Failed to send to {}:{}: {}. No channels connected.",
channel, target, e
)
} else {
format!(
"Failed to send to {}:{}. Available channels: {}. Error: {}",
channel, target, available, e
)
};
Err(ToolError::ExecutionFailed(err_msg))
}
}
Err(e) => {
let available = self.channel_manager.channel_names().await.join(", ");
let err_msg = if available.is_empty() {
format!(
"Failed to send to {}:{}: {}. No channels connected.",
channel, target, e
)
} else {
// No channel specified — broadcast to all channels (routine with notify.channel = None)
let results = self.channel_manager.broadcast_all(&target, response).await;
let mut succeeded = Vec::new();
let mut failed: Vec<&str> = Vec::new();
for (ch, result) in &results {
match result {
Ok(()) => succeeded.push(ch.as_str()),
Err(e) => {
tracing::warn!(
channel = %ch,
target = %target,
"broadcast_all: channel failed: {}", e
);
failed.push(ch.as_str());
}
}
}
if succeeded.is_empty() {
let err_msg = if failed.is_empty() {
"No channels connected.".to_string()
} else {
format!(
"Failed to send to {}:{}. Available channels: {}. Error: {}",
channel, target, available, e
)
format!("All channels failed: {}", failed.join(", "))
};
Err(ToolError::ExecutionFailed(err_msg))
} else {
tracing::info!(
message_sent = true,
channels = ?succeeded,
target = %target,
attachments = attachment_count,
"Message broadcast via message tool"
);
let msg = format!(
"Broadcast message to {} (target: {})",
succeeded.join(", "),
target
);
Ok(ToolOutput::text(msg, start.elapsed()))
}
}
}
@@ -576,4 +624,90 @@ mod tests {
ApprovalRequirement::Never,
);
}
#[tokio::test]
async fn message_tool_falls_back_to_job_metadata() {
// Regression: when no conversation context is set (e.g. routine full-job),
// the message tool should fall back to notify_channel/notify_user from
// JobContext metadata instead of returning "No target specified".
let tool = MessageTool::new(Arc::new(ChannelManager::new()));
let mut ctx = crate::context::JobContext::new("routine-job", "price alert");
ctx.metadata = serde_json::json!({
"notify_channel": "telegram",
"notify_user": "123456789",
});
// No set_context called — simulates a routine full-job worker
let result = tool
.execute(serde_json::json!({"content": "NEAR price is $5"}), &ctx)
.await;
// Should fail at channel broadcast (no real channel), NOT at
// "No target specified and no active conversation"
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
!err.contains("No target specified"),
"Should not get 'No target specified' when metadata has notify_user, got: {}",
err
);
assert!(
!err.contains("No channel specified"),
"Should not get 'No channel specified' when metadata has notify_channel, got: {}",
err
);
}
#[tokio::test]
async fn message_tool_no_metadata_still_errors() {
// When neither conversation context nor metadata is set, should still
// return a clear error (target resolution fails).
let tool = MessageTool::new(Arc::new(ChannelManager::new()));
let ctx = crate::context::JobContext::new("orphan-job", "no notify config");
let result = tool
.execute(serde_json::json!({"content": "hello"}), &ctx)
.await;
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
err.contains("No target specified"),
"Expected 'No target specified' error, got: {}",
err
);
}
#[tokio::test]
async fn message_tool_broadcasts_all_when_no_channel() {
// Regression: when notify.channel is None but notify_user is set,
// the message tool should attempt broadcast_all instead of erroring
// with "No channel specified".
let tool = MessageTool::new(Arc::new(ChannelManager::new()));
let mut ctx = crate::context::JobContext::new("routine-job", "price alert");
ctx.metadata = serde_json::json!({
"notify_user": "123456789",
});
let result = tool
.execute(serde_json::json!({"content": "NEAR price is $5"}), &ctx)
.await;
// Should fail because no channels are registered (empty ChannelManager),
// NOT because "No channel specified".
assert!(result.is_err());
let err = result.unwrap_err().to_string();
assert!(
!err.contains("No channel specified"),
"Should not get 'No channel specified' when broadcasting, got: {}",
err
);
assert!(
err.contains("No channels connected") || err.contains("All channels failed"),
"Expected channel delivery error, got: {}",
err
);
}
}
+75 -13
View File
@@ -107,6 +107,10 @@ impl Tool for RoutineCreateTool {
"notify_user": {
"type": "string",
"description": "User/target to notify (e.g. username, chat ID). Defaults to 'default'."
},
"timezone": {
"type": "string",
"description": "IANA timezone for cron schedule evaluation (e.g. 'America/New_York'). Defaults to UTC."
}
},
"required": ["name", "trigger_type", "prompt"]
@@ -143,12 +147,26 @@ impl Tool for RoutineCreateTool {
"cron trigger requires 'schedule'".to_string(),
)
})?;
let timezone = params
.get("timezone")
.and_then(|v| v.as_str())
.map(|tz| {
crate::timezone::parse_timezone(tz)
.map(|_| tz.to_string())
.ok_or_else(|| {
ToolError::InvalidParameters(format!(
"invalid IANA timezone: '{tz}'"
))
})
})
.transpose()?;
// Validate cron expression
next_cron_fire(schedule).map_err(|e| {
next_cron_fire(schedule, timezone.as_deref()).map_err(|e| {
ToolError::InvalidParameters(format!("invalid cron schedule: {e}"))
})?;
Trigger::Cron {
schedule: schedule.to_string(),
timezone,
}
}
"event" => {
@@ -228,8 +246,12 @@ impl Tool for RoutineCreateTool {
.unwrap_or(300);
// Compute next fire time for cron
let next_fire = if let Trigger::Cron { ref schedule } = trigger {
next_cron_fire(schedule).unwrap_or(None)
let next_fire = if let Trigger::Cron {
ref schedule,
ref timezone,
} = trigger
{
next_cron_fire(schedule, timezone.as_deref()).unwrap_or(None)
} else {
None
};
@@ -412,6 +434,10 @@ impl Tool for RoutineUpdateTool {
"type": "string",
"description": "New cron schedule (for cron triggers)"
},
"timezone": {
"type": "string",
"description": "IANA timezone for cron schedule (e.g. 'America/New_York'). Only valid for cron triggers."
},
"description": {
"type": "string",
"description": "New description"
@@ -453,15 +479,47 @@ impl Tool for RoutineUpdateTool {
}
}
if let Some(schedule) = params.get("schedule").and_then(|v| v.as_str()) {
// Validate
next_cron_fire(schedule)
.map_err(|e| ToolError::InvalidParameters(format!("invalid cron schedule: {e}")))?;
// Validate timezone param if provided
let new_timezone = params
.get("timezone")
.and_then(|v| v.as_str())
.map(|tz| {
crate::timezone::parse_timezone(tz)
.map(|_| tz.to_string())
.ok_or_else(|| {
ToolError::InvalidParameters(format!("invalid IANA timezone: '{tz}'"))
})
})
.transpose()?;
routine.trigger = Trigger::Cron {
schedule: schedule.to_string(),
let new_schedule = params.get("schedule").and_then(|v| v.as_str());
if new_schedule.is_some() || new_timezone.is_some() {
// Extract existing cron fields (cloned to avoid borrow conflict)
let existing_cron = match &routine.trigger {
Trigger::Cron { schedule, timezone } => Some((schedule.clone(), timezone.clone())),
_ => None,
};
routine.next_fire_at = next_cron_fire(schedule).unwrap_or(None);
if let Some((old_schedule, old_tz)) = existing_cron {
let effective_schedule = new_schedule.unwrap_or(&old_schedule);
let effective_tz = new_timezone.or(old_tz);
// Validate
next_cron_fire(effective_schedule, effective_tz.as_deref()).map_err(|e| {
ToolError::InvalidParameters(format!("invalid cron schedule: {e}"))
})?;
routine.trigger = Trigger::Cron {
schedule: effective_schedule.to_string(),
timezone: effective_tz.clone(),
};
routine.next_fire_at =
next_cron_fire(effective_schedule, effective_tz.as_deref()).unwrap_or(None);
} else {
return Err(ToolError::InvalidParameters(
"Cannot update schedule or timezone on a non-cron routine.".to_string(),
));
}
}
self.store
@@ -620,9 +678,13 @@ impl Tool for RoutineFireTool {
.map_err(|e| ToolError::ExecutionFailed(format!("DB error: {e}")))?
.ok_or_else(|| ToolError::ExecutionFailed(format!("routine '{}' not found", name)))?;
let run_id = self.engine.fire_manual(routine.id).await.map_err(|e| {
ToolError::ExecutionFailed(format!("failed to fire routine '{}': {e}", name))
})?;
let run_id = self
.engine
.fire_manual(routine.id, None)
.await
.map_err(|e| {
ToolError::ExecutionFailed(format!("failed to fire routine '{}': {e}", name))
})?;
let result = serde_json::json!({
"name": name,
+46 -2
View File
@@ -48,7 +48,7 @@ impl Tool for TimeTool {
async fn execute(
&self,
params: serde_json::Value,
_ctx: &JobContext,
ctx: &JobContext,
) -> Result<ToolOutput, ToolError> {
let start = std::time::Instant::now();
@@ -57,10 +57,15 @@ impl Tool for TimeTool {
let result = match operation {
"now" => {
let now = Utc::now();
let tz =
crate::timezone::parse_timezone(&ctx.user_timezone).unwrap_or(chrono_tz::UTC);
let local = now.with_timezone(&tz);
serde_json::json!({
"iso": now.to_rfc3339(),
"unix": now.timestamp(),
"unix_millis": now.timestamp_millis()
"unix_millis": now.timestamp_millis(),
"local_iso": local.to_rfc3339(),
"timezone": tz.name()
})
}
"parse" => {
@@ -112,3 +117,42 @@ impl Tool for TimeTool {
false // Internal tool, no external data
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_now_includes_local_time_when_timezone_set() {
let tool = TimeTool;
let mut ctx = JobContext::with_user("test", "chat", "test");
ctx.user_timezone = "America/New_York".to_string();
let output = tool
.execute(serde_json::json!({"operation": "now"}), &ctx)
.await
.expect("execute");
assert!(
output.result.get("local_iso").is_some(),
"should have local_iso"
);
assert_eq!(
output.result["timezone"].as_str(),
Some("America/New_York"),
"should report timezone"
);
}
#[tokio::test]
async fn test_now_includes_utc_timezone_by_default() {
let tool = TimeTool;
let ctx = JobContext::with_user("test", "chat", "test");
// Default user_timezone is "UTC" which is a valid IANA timezone
let output = tool
.execute(serde_json::json!({"operation": "now"}), &ctx)
.await
.expect("execute");
assert!(output.result.get("iso").is_some(), "should have iso");
assert_eq!(output.result["timezone"].as_str(), Some("UTC"));
}
}
+123 -2
View File
@@ -261,9 +261,9 @@ impl McpClient {
if !response.status().is_success() {
let status = response.status();
let body = response.text().await.unwrap_or_default();
let preview = sanitize_error_body(&body);
return Err(ToolError::ExternalService(format!(
"MCP server returned status: {} - {}",
status, body
"MCP server returned status: {status} - {preview}",
)));
}
@@ -548,6 +548,58 @@ impl Tool for McpToolWrapper {
}
}
/// Sanitize an HTTP error response body for safe display.
///
/// Detects full HTML error pages (containing `<html` or `<!DOCTYPE`) and
/// strips all tags, collapsing whitespace. Non-HTML bodies are left
/// intact. In both cases the result is truncated to 200 *characters*
/// (char-boundary safe) so that large payloads don't bloat error messages.
///
/// See #263 — raw HTML error pages were propagating through the error
/// chain into the web UI, causing a white screen.
fn sanitize_error_body(body: &str) -> String {
const MAX_CHARS: usize = 200;
// Only strip tags when the body looks like a full HTML document.
// Plain text that happens to contain `<` / `>` (e.g. log lines,
// comparison expressions) is left untouched.
let lower = body.to_ascii_lowercase();
let is_html_document = lower.contains("<html") || lower.contains("<!doctype");
let text = if is_html_document {
let stripped = body
.chars()
.fold((String::new(), false), |(mut out, in_tag), c| {
if c == '<' {
(out, true)
} else if c == '>' {
(out, false)
} else if !in_tag {
out.push(c);
(out, false)
} else {
(out, true)
}
})
.0;
stripped.split_whitespace().collect::<Vec<_>>().join(" ")
} else {
body.to_string()
};
// Truncate at a char boundary (safe for multi-byte UTF-8).
if text.chars().count() > MAX_CHARS {
let byte_offset = text
.char_indices()
.nth(MAX_CHARS)
.map(|(i, _)| i)
.unwrap_or(text.len());
format!("{}... ({} bytes total)", &text[..byte_offset], body.len())
} else {
text
}
}
#[cfg(test)]
mod tests {
use super::*;
@@ -740,4 +792,73 @@ mod tests {
};
assert!(!tool.requires_approval());
}
// Regression tests for #263: HTML error bodies must not propagate raw
// markup through the error chain into the web UI.
#[test]
fn test_sanitize_error_body_strips_html_tags() {
let html =
r#"<!DOCTYPE html><html><body><h1>422 Error</h1><p>Invalid token</p></body></html>"#;
let result = sanitize_error_body(html);
assert!(!result.contains('<'), "HTML tags must be stripped");
assert!(!result.contains('>'), "HTML tags must be stripped");
assert!(result.contains("422 Error"));
assert!(result.contains("Invalid token"));
}
#[test]
fn test_sanitize_error_body_truncates_large_html_page() {
let html = format!(
"<html><body><p>{}</p></body></html>",
"error detail ".repeat(50)
);
let result = sanitize_error_body(&html);
assert!(result.contains("..."));
assert!(result.contains("bytes total)"));
assert!(!result.contains('<'));
}
#[test]
fn test_sanitize_error_body_passes_short_plain_text() {
assert_eq!(sanitize_error_body("Not Found"), "Not Found");
}
#[test]
fn test_sanitize_error_body_truncates_long_plain_text() {
let long = "x".repeat(300);
let result = sanitize_error_body(&long);
assert!(result.contains("..."));
assert!(result.contains("300 bytes total)"));
}
#[test]
fn test_sanitize_error_body_multibyte_no_panic() {
// 300 CJK characters = 900 bytes; truncation must land on a
// char boundary, not in the middle of a multi-byte sequence.
let cjk = "错误".repeat(150);
let result = sanitize_error_body(&cjk);
assert!(result.contains("..."));
// Must be valid UTF-8 (would have panicked otherwise).
assert!(result.is_char_boundary(result.len()));
}
#[test]
fn test_sanitize_error_body_strips_uppercase_html() {
let html = "<HTML><BODY><H1>500 Internal Server Error</H1></BODY></HTML>";
let result = sanitize_error_body(html);
assert!(
!result.contains('<'),
"uppercase HTML tags must be stripped"
);
assert!(result.contains("500 Internal Server Error"));
}
#[test]
fn test_sanitize_error_body_preserves_angle_brackets_in_non_html() {
// Text with < and > that is NOT an HTML document should be
// left untouched (e.g. log lines, comparison expressions).
let text = "value < 10 and value > 0";
assert_eq!(sanitize_error_body(text), text);
}
}
+3 -1
View File
@@ -204,7 +204,9 @@ mod tests {
assert!(!session.is_stale(1800));
// Manually set last_activity to the past to simulate staleness
session.last_activity = std::time::Instant::now() - std::time::Duration::from_secs(10);
session.last_activity = std::time::Instant::now()
.checked_sub(std::time::Duration::from_secs(10))
.expect("System uptime is too low to run staleness test");
assert!(session.is_stale(5));
assert!(!session.is_stale(15));
}
+1 -4
View File
@@ -10,7 +10,6 @@ use crate::db::Database;
use crate::extensions::ExtensionManager;
use crate::llm::{LlmProvider, ToolDefinition};
use crate::orchestrator::job_manager::ContainerJobManager;
use crate::safety::SafetyLayer;
use crate::secrets::SecretsStore;
use crate::skills::catalog::SkillCatalog;
use crate::skills::registry::SkillRegistry;
@@ -485,17 +484,15 @@ impl ToolRegistry {
pub async fn register_builder_tool(
self: &Arc<Self>,
llm: Arc<dyn LlmProvider>,
safety: Arc<SafetyLayer>,
config: Option<BuilderConfig>,
) {
// First register dev tools needed by the builder
self.register_dev_tools();
// Create the builder (arg order: config, llm, safety, tools)
// Create the builder (arg order: config, llm, tools)
let builder = Arc::new(LlmSoftwareBuilder::new(
config.unwrap_or_default(),
llm,
safety,
Arc::clone(self),
));
+1 -1
View File
@@ -133,7 +133,7 @@ impl WorkerRuntime {
.await?;
// Create reasoning engine
let reasoning = Reasoning::new(self.llm.clone(), self.safety.clone());
let reasoning = Reasoning::new(self.llm.clone());
// Build initial context
let mut reason_ctx = ReasoningContext::new().with_job(&job.description);
-116
View File
@@ -113,79 +113,6 @@ pub fn chunk_document(content: &str, config: ChunkConfig) -> Vec<String> {
chunks
}
/// Split content by paragraphs first, then chunk.
///
/// This is better for preserving semantic boundaries.
#[allow(dead_code)] // Alternative chunking strategy for paragraph-aware indexing
pub fn chunk_by_paragraphs(content: &str, config: ChunkConfig) -> Vec<String> {
if content.is_empty() {
return Vec::new();
}
// Split by double newlines (paragraphs)
let paragraphs: Vec<&str> = content
.split("\n\n")
.map(|p| p.trim())
.filter(|p| !p.is_empty())
.collect();
if paragraphs.is_empty() {
return chunk_document(content, config);
}
let mut chunks = Vec::new();
let mut current_chunk = String::new();
let mut current_word_count = 0;
for paragraph in paragraphs {
let para_words = paragraph.split_whitespace().count();
// If this paragraph alone exceeds chunk size, chunk it separately
if para_words > config.chunk_size {
// Flush current chunk first
if !current_chunk.is_empty() {
chunks.push(current_chunk.trim().to_string());
current_chunk = String::new();
current_word_count = 0;
}
// Chunk the large paragraph
let para_chunks = chunk_document(paragraph, config.clone());
chunks.extend(para_chunks);
continue;
}
// Check if adding this paragraph would exceed chunk size
if current_word_count + para_words > config.chunk_size {
// Flush current chunk
if !current_chunk.is_empty() {
chunks.push(current_chunk.trim().to_string());
}
current_chunk = paragraph.to_string();
current_word_count = para_words;
} else {
// Add paragraph to current chunk
if !current_chunk.is_empty() {
current_chunk.push_str("\n\n");
}
current_chunk.push_str(paragraph);
current_word_count += para_words;
}
}
// Flush remaining content
if !current_chunk.is_empty() {
// If too small, merge with previous chunk if possible
if current_word_count < config.min_chunk_size && !chunks.is_empty() {
let last = chunks.pop().unwrap();
chunks.push(format!("{}\n\n{}", last, current_chunk.trim()));
} else {
chunks.push(current_chunk.trim().to_string());
}
}
chunks
}
#[cfg(test)]
mod tests {
use super::*;
@@ -253,49 +180,6 @@ mod tests {
assert_eq!(config.step_size(), 85);
}
#[test]
fn test_paragraph_chunking() {
let config = ChunkConfig::default().with_chunk_size(20);
let content = "First paragraph with some words.\n\nSecond paragraph with different content.\n\nThird paragraph here.";
let chunks = chunk_by_paragraphs(content, config);
// Should preserve paragraph boundaries
assert!(!chunks.is_empty());
for chunk in &chunks {
// No chunk should start or end with \n\n
assert!(!chunk.starts_with("\n"));
assert!(!chunk.ends_with("\n"));
}
}
#[test]
fn test_large_paragraph_handling() {
let config = ChunkConfig {
chunk_size: 10,
overlap_percent: 0.15,
min_chunk_size: 3, // Low threshold for test
};
// Create a paragraph with 30 words
let large_para = (1..=30)
.map(|i| format!("word{}", i))
.collect::<Vec<_>>()
.join(" ");
let content = format!("Short intro.\n\n{}\n\nShort outro.", large_para);
let chunks = chunk_by_paragraphs(&content, config);
// Should have multiple chunks due to large paragraph
// 30 words + 2 intro + 2 outro = 34 words, chunk_size=10
// Expect at least 3 chunks
assert!(
chunks.len() >= 3,
"Expected at least 3 chunks for 34 words with chunk_size=10, got {}",
chunks.len()
);
}
#[test]
fn test_min_chunk_size_merging() {
let config = ChunkConfig {
+644
View File
@@ -0,0 +1,644 @@
//! LanceDB-backed vector store for workspace memory chunks.
//!
//! Provides an alternative to pgvector/libsql for semantic search when the
//! `lancedb` feature is enabled. Documents and metadata stay in the main
//! database; this store holds chunk embeddings for vector similarity search.
//!
//! Configuration:
//! LANCEDB_PATH=~/.ironclaw/lancedb # Default
//! VECTOR_BACKEND=lancedb # Use LanceDB for vector search
/// Default embedding dimension (text-embedding-3-small).
/// Override by passing the actual provider dimension to `LanceDbVectorStore::new()`.
pub const DEFAULT_EMBEDDING_DIM: i32 = 1536;
#[cfg(feature = "lancedb")]
mod impl_lancedb {
use std::sync::Arc;
use arrow_array::types::Float32Type;
use arrow_array::{Array, FixedSizeListArray, RecordBatch, RecordBatchIterator, StringArray};
use arrow_schema::{DataType, Field, Schema};
use async_trait::async_trait;
use futures::StreamExt;
use lancedb::query::{ExecutableQuery, QueryBase};
use uuid::Uuid;
use super::DEFAULT_EMBEDDING_DIM;
use crate::error::WorkspaceError;
use crate::workspace::search::RankedResult;
use crate::workspace::vector_store::VectorStore;
const TABLE_NAME: &str = "memory_chunks";
/// Escapes a string for safe use in LanceDB predicate expressions.
/// Uses SQL-style escaping: single quotes are doubled to prevent injection.
fn escape_predicate_value(s: &str) -> String {
s.replace('\'', "''")
}
/// LanceDB-backed vector store.
///
/// The `update_embedding` method uses delete-then-insert (not atomic).
/// LanceDB does not support transactions, so a crash between the two
/// operations can lose the embedding for that chunk. This is acceptable
/// for personal workspace sizes where data can be reindexed.
pub struct LanceDbVectorStore {
db: Arc<lancedb::Connection>,
table_name: String,
embedding_dim: i32,
schema: Arc<Schema>,
table: tokio::sync::OnceCell<lancedb::Table>,
}
impl LanceDbVectorStore {
/// Create a new LanceDB store at the given path.
///
/// `embedding_dim` should match `EmbeddingProvider::dimension()`.
/// Pass `None` to use the default (1536, text-embedding-3-small).
pub async fn new(
path: impl AsRef<std::path::Path>,
embedding_dim: Option<usize>,
) -> Result<Self, WorkspaceError> {
let path_str = path
.as_ref()
.to_str()
.ok_or_else(|| WorkspaceError::SearchFailed {
reason: "Invalid LanceDB path".to_string(),
})?;
let db = lancedb::connect(path_str).execute().await.map_err(|e| {
WorkspaceError::SearchFailed {
reason: format!("Failed to connect to LanceDB: {}", e),
}
})?;
let dim = embedding_dim.unwrap_or(DEFAULT_EMBEDDING_DIM as usize) as i32;
let schema = Arc::new(Self::build_schema(dim));
let store = Self {
db: Arc::new(db),
table_name: TABLE_NAME.to_string(),
embedding_dim: dim,
schema,
table: tokio::sync::OnceCell::new(),
};
store.ensure_table().await?;
Ok(store)
}
async fn ensure_table(&self) -> Result<(), WorkspaceError> {
let tables = self.db.table_names().execute().await.map_err(|e| {
WorkspaceError::SearchFailed {
reason: format!("Failed to list tables: {}", e),
}
})?;
if tables.iter().any(|t| t == &self.table_name) {
return Ok(());
}
self.db
.create_empty_table(&self.table_name, self.schema.clone())
.execute()
.await
.map_err(|e| WorkspaceError::SearchFailed {
reason: format!("Failed to create table: {}", e),
})?;
// Index creation is deferred — brute-force search via
// bypass_vector_index() works without a pre-built index and is
// sufficient for personal workspace sizes.
Ok(())
}
/// Get or open the cached table handle.
async fn table(&self) -> Result<&lancedb::Table, WorkspaceError> {
self.table
.get_or_try_init(|| async {
self.db
.open_table(&self.table_name)
.execute()
.await
.map_err(|e| WorkspaceError::SearchFailed {
reason: format!("Failed to open table: {}", e),
})
})
.await
}
fn build_schema(embedding_dim: i32) -> Schema {
Schema::new(vec![
Field::new("chunk_id", DataType::Utf8, false),
Field::new("document_id", DataType::Utf8, false),
Field::new("document_path", DataType::Utf8, false),
Field::new("user_id", DataType::Utf8, false),
Field::new("agent_id", DataType::Utf8, true),
Field::new("content", DataType::Utf8, false),
Field::new(
"vector",
DataType::FixedSizeList(
Arc::new(Field::new("item", DataType::Float32, true)),
embedding_dim,
),
false,
),
])
}
}
#[async_trait]
impl VectorStore for LanceDbVectorStore {
async fn store_embedding(
&self,
chunk_id: Uuid,
document_id: Uuid,
document_path: &str,
user_id: &str,
agent_id: Option<Uuid>,
content: &str,
embedding: &[f32],
) -> Result<(), WorkspaceError> {
if embedding.len() != self.embedding_dim as usize {
return Err(WorkspaceError::EmbeddingFailed {
reason: format!(
"Embedding dimension {} does not match expected {}",
embedding.len(),
self.embedding_dim
),
});
}
let table = self.table().await?;
let chunk_ids = StringArray::from(vec![chunk_id.to_string()]);
let document_ids = StringArray::from(vec![document_id.to_string()]);
let document_paths = StringArray::from(vec![document_path]);
let user_ids = StringArray::from(vec![user_id]);
let agent_ids = StringArray::from(vec![agent_id.map(|a| a.to_string())]);
let contents = StringArray::from(vec![content]);
let vec_values: Vec<Option<f32>> = embedding.iter().map(|&x| Some(x)).collect();
let vectors = FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(
vec![Some(vec_values)],
self.embedding_dim,
);
let batch = RecordBatch::try_new(
self.schema.clone(),
vec![
Arc::new(chunk_ids),
Arc::new(document_ids),
Arc::new(document_paths),
Arc::new(user_ids),
Arc::new(agent_ids),
Arc::new(contents),
Arc::new(vectors),
],
)
.map_err(|e| WorkspaceError::EmbeddingFailed {
reason: format!("Failed to create record batch: {}", e),
})?;
let batches =
RecordBatchIterator::new(vec![Ok(batch)].into_iter(), self.schema.clone());
table
.add(Box::new(batches) as Box<dyn arrow_array::RecordBatchReader + Send>)
.execute()
.await
.map_err(|e| WorkspaceError::EmbeddingFailed {
reason: format!("Failed to store embedding: {}", e),
})?;
Ok(())
}
async fn update_embedding(
&self,
chunk_id: Uuid,
document_id: Uuid,
document_path: &str,
user_id: &str,
agent_id: Option<Uuid>,
content: &str,
embedding: &[f32],
) -> Result<(), WorkspaceError> {
let table = self.table().await?;
table
.delete(&format!(
"chunk_id = '{}'",
escape_predicate_value(&chunk_id.to_string())
))
.await
.map_err(|e| WorkspaceError::EmbeddingFailed {
reason: format!("Failed to delete chunk for update: {}", e),
})?;
self.store_embedding(
chunk_id,
document_id,
document_path,
user_id,
agent_id,
content,
embedding,
)
.await
}
async fn delete_embeddings(&self, document_id: Uuid) -> Result<(), WorkspaceError> {
let table = self.table().await?;
table
.delete(&format!(
"document_id = '{}'",
escape_predicate_value(&document_id.to_string())
))
.await
.map_err(|e| WorkspaceError::EmbeddingFailed {
reason: format!("Failed to delete embeddings: {}", e),
})?;
Ok(())
}
async fn vector_search(
&self,
user_id: &str,
agent_id: Option<Uuid>,
embedding: &[f32],
limit: usize,
) -> Result<Vec<RankedResult>, WorkspaceError> {
let table = self.table().await?;
let filter = if let Some(aid) = agent_id {
format!(
"user_id = '{}' AND agent_id = '{}'",
escape_predicate_value(user_id),
escape_predicate_value(&aid.to_string())
)
} else {
format!(
"user_id = '{}' AND agent_id IS NULL",
escape_predicate_value(user_id)
)
};
let query = table
.query()
.nearest_to(embedding)
.map_err(|e| WorkspaceError::SearchFailed {
reason: format!("Invalid query vector: {}", e),
})?
.only_if(&filter)
.bypass_vector_index()
.limit(limit);
let mut stream = ExecutableQuery::execute(&query).await.map_err(|e| {
WorkspaceError::SearchFailed {
reason: format!("Vector search failed: {}", e),
}
})?;
let mut results = Vec::new();
let mut rank: u32 = 1;
while let Some(batch_result) = stream.next().await {
let batch = batch_result.map_err(|e| WorkspaceError::SearchFailed {
reason: format!("Stream error: {}", e),
})?;
let chunk_id_col = batch.column_by_name("chunk_id").ok_or_else(|| {
WorkspaceError::SearchFailed {
reason: "chunk_id column missing".to_string(),
}
})?;
let document_id_col = batch.column_by_name("document_id").ok_or_else(|| {
WorkspaceError::SearchFailed {
reason: "document_id column missing".to_string(),
}
})?;
let document_path_col = batch.column_by_name("document_path").ok_or_else(|| {
WorkspaceError::SearchFailed {
reason: "document_path column missing".to_string(),
}
})?;
let content_col = batch.column_by_name("content").ok_or_else(|| {
WorkspaceError::SearchFailed {
reason: "content column missing".to_string(),
}
})?;
let chunk_ids = chunk_id_col
.as_any()
.downcast_ref::<StringArray>()
.ok_or_else(|| WorkspaceError::SearchFailed {
reason: "chunk_id wrong type".to_string(),
})?;
let document_ids = document_id_col
.as_any()
.downcast_ref::<StringArray>()
.ok_or_else(|| WorkspaceError::SearchFailed {
reason: "document_id wrong type".to_string(),
})?;
let document_paths = document_path_col
.as_any()
.downcast_ref::<StringArray>()
.ok_or_else(|| WorkspaceError::SearchFailed {
reason: "document_path wrong type".to_string(),
})?;
let contents = content_col
.as_any()
.downcast_ref::<StringArray>()
.ok_or_else(|| WorkspaceError::SearchFailed {
reason: "content wrong type".to_string(),
})?;
for i in 0..batch.num_rows() {
let raw_chunk_id = chunk_ids.value(i);
let chunk_id =
raw_chunk_id
.parse::<Uuid>()
.map_err(|e| WorkspaceError::SearchFailed {
reason: format!("Invalid chunk_id UUID '{}': {}", raw_chunk_id, e),
})?;
let raw_document_id = document_ids.value(i);
let document_id = raw_document_id.parse::<Uuid>().map_err(|e| {
WorkspaceError::SearchFailed {
reason: format!(
"Invalid document_id UUID '{}': {}",
raw_document_id, e
),
}
})?;
let document_path = document_paths.value(i).to_string();
let content = contents.value(i).to_string();
results.push(RankedResult {
chunk_id,
document_id,
document_path,
content,
rank,
});
rank += 1;
}
}
Ok(results)
}
}
}
#[cfg(feature = "lancedb")]
pub use impl_lancedb::LanceDbVectorStore;
#[cfg(all(test, feature = "lancedb"))]
mod tests {
use tempfile::TempDir;
use uuid::Uuid;
use super::{DEFAULT_EMBEDDING_DIM, LanceDbVectorStore};
use crate::workspace::vector_store::VectorStore;
fn make_embedding(seed: f32) -> Vec<f32> {
(0..DEFAULT_EMBEDDING_DIM as usize)
.map(|i| (seed * (i as f32 + 1.0)).sin())
.collect()
}
#[tokio::test]
async fn test_insert_and_vector_search() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let chunk_id = Uuid::new_v4();
let document_id = Uuid::new_v4();
let user_id = "user1";
let content = "Rust is a systems programming language";
let embedding = make_embedding(1.0);
store
.store_embedding(
chunk_id,
document_id,
"test.md",
user_id,
None,
content,
&embedding,
)
.await
.unwrap();
let results = store
.vector_search(user_id, None, &embedding, 5)
.await
.unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].chunk_id, chunk_id);
assert_eq!(results[0].document_id, document_id);
assert_eq!(results[0].content, content);
assert_eq!(results[0].rank, 1);
}
#[tokio::test]
async fn test_insert_multiple_and_search_returns_ordered() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let doc_id = Uuid::new_v4();
let user_id = "user1";
for (i, seed) in [1.0, 2.0, 3.0].iter().enumerate() {
store
.store_embedding(
Uuid::new_v4(),
doc_id,
"test.md",
user_id,
None,
&format!("content {}", i),
&make_embedding(*seed),
)
.await
.unwrap();
}
let query_emb = make_embedding(2.0);
let results = store
.vector_search(user_id, None, &query_emb, 5)
.await
.unwrap();
assert_eq!(results.len(), 3);
let contents: Vec<_> = results.iter().map(|r| r.content.as_str()).collect();
assert!(contents.contains(&"content 0"));
assert!(contents.contains(&"content 1"));
assert!(contents.contains(&"content 2"));
}
#[tokio::test]
async fn test_delete_chunks() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let doc_id = Uuid::new_v4();
let user_id = "user1";
store
.store_embedding(
Uuid::new_v4(),
doc_id,
"test.md",
user_id,
None,
"content",
&make_embedding(1.0),
)
.await
.unwrap();
let results = store
.vector_search(user_id, None, &make_embedding(1.0), 5)
.await
.unwrap();
assert_eq!(results.len(), 1);
store.delete_embeddings(doc_id).await.unwrap();
let results_after = store
.vector_search(user_id, None, &make_embedding(1.0), 5)
.await
.unwrap();
assert!(results_after.is_empty());
}
#[tokio::test]
async fn test_update_chunk_embedding() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let chunk_id = Uuid::new_v4();
let doc_id = Uuid::new_v4();
let user_id = "user1";
let content = "original content";
store
.store_embedding(
chunk_id,
doc_id,
"test.md",
user_id,
None,
content,
&make_embedding(1.0),
)
.await
.unwrap();
let new_embedding = make_embedding(5.0);
store
.update_embedding(
chunk_id,
doc_id,
"test.md",
user_id,
None,
content,
&new_embedding,
)
.await
.unwrap();
let results = store
.vector_search(user_id, None, &new_embedding, 5)
.await
.unwrap();
assert_eq!(results.len(), 1);
assert_eq!(results[0].chunk_id, chunk_id);
}
#[tokio::test]
async fn test_vector_search_filters_by_user_and_agent() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let doc_id = Uuid::new_v4();
let embedding = make_embedding(1.0);
store
.store_embedding(
Uuid::new_v4(),
doc_id,
"test.md",
"user1",
None,
"user1 content",
&embedding,
)
.await
.unwrap();
store
.store_embedding(
Uuid::new_v4(),
doc_id,
"test.md",
"user2",
None,
"user2 content",
&embedding,
)
.await
.unwrap();
let results_user1 = store
.vector_search("user1", None, &embedding, 5)
.await
.unwrap();
assert_eq!(results_user1.len(), 1);
assert_eq!(results_user1[0].content, "user1 content");
let results_user2 = store
.vector_search("user2", None, &embedding, 5)
.await
.unwrap();
assert_eq!(results_user2.len(), 1);
assert_eq!(results_user2[0].content, "user2 content");
let results_wrong_user = store
.vector_search("user3", None, &embedding, 5)
.await
.unwrap();
assert!(results_wrong_user.is_empty());
}
#[tokio::test]
async fn test_insert_rejects_wrong_embedding_dim() {
let dir = TempDir::new().unwrap();
let store = LanceDbVectorStore::new(dir.path(), None).await.unwrap();
let wrong_dim: Vec<f32> = vec![1.0; 100];
let err = store
.store_embedding(
Uuid::new_v4(),
Uuid::new_v4(),
"test.md",
"user1",
None,
"content",
&wrong_dim,
)
.await
.unwrap_err();
assert!(matches!(
err,
crate::error::WorkspaceError::EmbeddingFailed { .. }
));
}
}

Some files were not shown because too many files have changed in this diff Show More